Docker

---
name: docker
description: "Expert coverage of Docker Engine across all supported versions: dockerd/containerd/runc architecture, BuildKit builds, Dockerfile optimization, Compose v2, networking, storage drivers, security hardening, and Docker Scout. Use for \"Docker\", \"Dockerfile\", \"docker-compose\", \"Docker Compose\", \"BuildKit\", \"docker build\", \"docker run\", \"dockerd\", \"Docker Desktop\", \"Docker Scout\"."
license: MIT
---

# Docker Engine

This skill covers Docker Engine across all supported versions (25.0 through 29.x), including:

- Docker daemon (`dockerd`) and its delegation to containerd and runc
- BuildKit build engine, multi-stage builds, cache optimization, multi-platform builds
- Dockerfile best practices, layer optimization, security hardening
- Docker Compose v2 (Go rewrite, CLI plugin)
- Networking (bridge, host, overlay, macvlan, ipvlan, nftables)
- Storage drivers and volume management
- Security (rootless mode, user namespaces, seccomp, AppArmor, capabilities)
- Docker Scout vulnerability scanning and SBOM generation
- Registry interaction (Docker Hub, private registries, Harbor)

When a question is version-specific, read the matching file in the Version-specific guidance section below. When the version is unknown, provide guidance based on the latest stable release (29.x).

## How to Approach Tasks

1. **Classify** the request:
   - **Build optimization** -- Load `references/best-practices.md` for Dockerfile patterns, multi-stage builds, cache strategies
   - **Troubleshooting** -- Load `references/diagnostics.md` for docker logs, inspect, stats, events, system df
   - **Architecture** -- Load `references/architecture.md` for daemon/containerd/runc flow, networking, storage
   - **Compose** -- Apply Compose v2 patterns, profiles, watch mode, depends_on conditions
   - **Security** -- Apply rootless, seccomp, capabilities, image scanning guidance

2. **Identify version** -- Determine Docker Engine version. Key boundaries: v25 (BuildKit default), v28 (nftables experimental), v29 (containerd image store default). If unclear, ask.

3. **Load context** -- Read the relevant reference file below.

4. **Analyze** -- Apply Docker-specific reasoning, not generic container advice.

5. **Recommend** -- Provide actionable guidance with CLI examples, Dockerfile snippets, or compose.yaml patterns.

6. **Verify** -- Suggest validation steps (`docker inspect`, `docker stats`, `docker scout cves`).

## Core Architecture

```
Docker CLI --> Docker Daemon (dockerd) --> containerd --> containerd-shim-runc-v2 --> runc --> Linux Kernel
```

### dockerd (Docker Daemon)

The daemon listens on a Unix socket (`/var/run/docker.sock`), TCP socket, or named pipe (Windows). It manages:
- Image builds (delegated to BuildKit)
- Container lifecycle (create, start, stop, remove)
- Volume and network management
- Plugin system (storage, network, authorization plugins)

The daemon delegates all container execution to containerd. As of Docker Engine v29, the daemon no longer manages its own image store -- that responsibility moved to containerd's content store.

### containerd Integration

Docker Engine bundles containerd as its execution backend:
- Image pull, push, and storage (content-addressable store)
- Container execution and supervision via shims
- containerd namespaces isolate Docker ("moby" namespace) from other clients
- Docker Engine v29 ships containerd 2.2.2 with config version 3

### runc and Shims

runc is the OCI reference runtime that creates containers:
- Reads OCI runtime spec (`config.json`) generated by containerd
- Sets up Linux namespaces, cgroups, seccomp, capabilities
- `containerd-shim-runc-v2` manages runc processes, keeping containers alive if containerd restarts

## Dockerfile Best Practices

### Multi-Stage Builds

The primary mechanism for minimal production images:

```dockerfile
# Build stage
FROM golang:1.23-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /app/server .

# Production stage -- scratch or distroless
FROM gcr.io/distroless/static-debian12:nonroot
WORKDIR /app
COPY --from=builder /app/server .
EXPOSE 8080
ENTRYPOINT ["/app/server"]
```

### Layer Caching Strategy

1. Put rarely-changing instructions first (OS packages, dependency install)
2. Copy dependency manifests before source code (`package.json` before `src/`)
3. Use BuildKit cache mounts for package managers:

```dockerfile
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y --no-install-recommends build-essential

RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    go build -o /app/server .
```

### Security Hardening

```dockerfile
# Non-root user
RUN adduser -u 10001 -D appuser
USER 10001

# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD wget -qO- http://localhost:8080/health || exit 1
```

- Use specific image tags, never `:latest` in production
- Prefer `COPY` over `ADD` (predictable, no auto-extraction)
- Use `ENTRYPOINT` for the command, `CMD` for default arguments
- Combine `RUN` with `&&` to minimize layers
- Always include `.dockerignore` to exclude `.git`, `node_modules`, `.env`, secrets

## BuildKit

BuildKit is the default build engine since Docker 23.0. It provides concurrent DAG-based builds with content-addressable caching.

### Multi-Platform Builds

```bash
docker buildx create --name mybuilder --use --bootstrap
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag registry.example.com/myapp:v1.0 \
  --push .
```

### Cache Export/Import

```bash
# Registry cache
docker buildx build \
  --cache-to type=registry,ref=registry.example.com/myapp:cache,mode=max \
  --cache-from type=registry,ref=registry.example.com/myapp:cache \
  --push -t registry.example.com/myapp:latest .

# GitHub Actions cache
docker buildx build \
  --cache-to type=gha,mode=max \
  --cache-from type=gha \
  --push -t myapp:latest .
```

Cache backends: `registry`, `local`, `gha`, `s3`, `azblob`, `inline`. `mode=max` caches all intermediate layers; `mode=min` (default) caches only final layers.

### Build Secrets

```bash
docker buildx build --secret id=npmrc,src=$HOME/.npmrc .
```
```dockerfile
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm install
```

Secrets are never baked into image layers.

### Docker Bake (HCL)

Declarative build orchestration for multi-service projects:

```hcl
group "default" {
  targets = ["api", "worker"]
}
target "api" {
  context    = "./api"
  platforms  = ["linux/amd64", "linux/arm64"]
  tags       = ["registry.example.com/api:latest"]
  cache-from = ["type=registry,ref=registry.example.com/api:cache"]
  cache-to   = ["type=registry,ref=registry.example.com/api:cache,mode=max"]
}
```

## Docker Compose v2

Compose v2 is a Go rewrite integrated as a Docker CLI plugin (`docker compose`). Legacy Python `docker-compose` (v1) is EOL. Compose Specification v5.0.0 (Dec 2025) removed the internal builder in favor of Docker Bake.

### Key Features

- **depends_on conditions**: `condition: service_healthy` waits for health check
- **Profiles**: Conditional service activation (`--profile debug`)
- **Watch mode**: File sync, rebuild, or sync+restart on file changes (`docker compose watch`)
- **Secrets**: Mount files as secrets, never in environment variables
- **Deploy resources**: CPU/memory limits via `deploy.resources.limits`

### Watch Mode (Compose v2.22.0+)

```yaml
services:
  app:
    build: .
    develop:
      watch:
        - action: sync
          path: ./src
          target: /app/src
        - action: rebuild
          path: package.json
        - action: sync+restart
          path: ./config
          target: /app/config
```

## Networking

### Network Drivers

| Driver | Scope | Use Case |
|---|---|---|
| bridge | local | Default; containers on same host communicate via DNS |
| host | local | No network isolation, use host stack directly |
| overlay | swarm | Multi-host (VXLAN encapsulation, UDP 4789) |
| macvlan | local | Container gets own MAC/IP on physical LAN |
| ipvlan | local | Like macvlan but shares MAC; L2 or L3 modes |
| none | local | No networking |

### Custom Bridge (recommended over default docker0)

```bash
docker network create \
  --driver bridge \
  --subnet 172.20.0.0/16 \
  --gateway 172.20.0.1 \
  myapp-net
```

Custom bridges provide automatic DNS resolution between containers by name.

### nftables (Docker Engine v29)

Experimental support for generating nftables rules directly instead of routing through iptables-nft translation. Requires `"experimental": true` in daemon.json.

## Storage

### Volume Types

- **Named volumes**: Managed by Docker, persist across container restarts (`docker volume create mydata`)
- **Bind mounts**: Map host path into container (`-v /host/path:/container/path`)
- **tmpfs**: In-memory, not persisted (`--tmpfs /tmp:rw,size=100m`)

### Storage Drivers (Docker Engine v29)

| Driver | Status | Notes |
|---|---|---|
| overlay2 | Default | Requires ftype=1 on XFS |
| fuse-overlayfs | Supported | Required for rootless on older kernels |
| btrfs | Supported | Native snapshotting |
| zfs | Supported | Enterprise features |
| devicemapper | Removed (v29) | Migrate to overlay2 |
| aufs | Removed (v29) | Migrate to overlay2 |

With the containerd image store (default in v29), storage is managed by containerd's overlayfs snapshotter.

## Security

### Rootless Mode

Run Docker daemon as non-root, eliminating container-escape-to-root risk:

```bash
dockerd-rootless-setuptool.sh install
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/docker.sock
```

Limitations: no macvlan/ipvlan, no AppArmor by default, ports < 1024 require `net.ipv4.ip_unprivileged_port_start=0`.

### Runtime Security Controls

```bash
# Drop all capabilities, add only needed
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE myimage

# Read-only filesystem
docker run --read-only --tmpfs /tmp --tmpfs /var/run myimage

# Custom seccomp profile
docker run --security-opt seccomp=/path/to/profile.json myimage

# User namespace remapping (daemon.json: "userns-remap": "default")
```

### Docker Scout

```bash
docker scout cves myimage:latest          # Scan for CVEs
docker scout compare myimage:v1 myimage:v2  # Compare versions
docker scout sbom myimage:latest          # Generate SBOM
docker scout recommendations myimage      # Upgrade suggestions
```

## Docker Desktop vs Docker Engine

| Feature | Docker Desktop | Docker Engine (Linux) |
|---|---|---|
| Platform | macOS, Windows, Linux | Linux only |
| License | Paid for large orgs (>250 employees / >$10M revenue) | Apache 2.0 (free) |
| VM isolation | HyperKit/WSL2/Virtualization.framework | Native (no VM) |
| GUI | Yes (dashboard, extensions) | No |
| Kubernetes | Built-in (optional) | No |

## Common Pitfalls

1. **Using `:latest` in production** -- Not a version, it is a moving target. Pin to specific tags or digests.
2. **COPY . . at the top of Dockerfile** -- Invalidates cache on every source change. Copy dependency manifests first.
3. **Running as root** -- Default for Docker. Always add a `USER` instruction or use `--user` flag.
4. **No .dockerignore** -- Build context includes `.git`, `node_modules`, secrets. Always create a `.dockerignore`.
5. **ADD instead of COPY** -- `ADD` auto-extracts archives and supports URLs, creating unexpected behavior. Use `COPY` unless extraction is intended.
6. **No health checks** -- Without `HEALTHCHECK`, Docker cannot distinguish a healthy container from a hung one.
7. **Storing secrets in images** -- Use BuildKit `--secret` mounts or runtime secrets. Never `ENV SECRET=...` or `COPY .env`.
8. **Not cleaning apt cache** -- `apt-get update && apt-get install` without `rm -rf /var/lib/apt/lists/*` wastes layer space. Use BuildKit cache mounts instead.

## Version-specific guidance

| Version | Reference | What's version-specific |
|---|---|---|
| 29.x | `references/versions/29.md` | containerd image store default, nftables, API minimum 1.44 |

## Reference Files

Read these for implementation depth:

- `references/architecture.md` -- Daemon/containerd/runc internals, BuildKit, networking drivers, storage drivers. Read for "how does X work" questions.
- `references/diagnostics.md` -- docker logs, inspect, stats, events, system df, troubleshooting workflows. Read when troubleshooting.
- `references/best-practices.md` -- Dockerfile patterns, multi-stage builds, security hardening, Compose patterns, image optimization. Read for design questions.

## Diagnostic Scripts

Ready-made docker CLI bundles (read-only, nothing pruned) in `scripts/`.

- `scripts/01-disk-usage.sh` -- Disk breakdown with reclaimable-space preview (dangling images/volumes)
- `scripts/02-container-health.sh` -- Restarting/unhealthy/exited sweep plus one-shot stats