Optimizing Docker Build Cache

Docker build cache dramatically speeds up image builds by reusing layers from previous builds. However, poor Dockerfile structure can invalidate the cache unnecessarily, causing full rebuilds that waste time and CI resources. This guide covers cache optimization techniques for fast, efficient Docker builds.

How Build Cache Works

Docker checks each instruction in order. If the instruction and its inputs have not changed since the last build, Docker reuses the cached layer. Once a cache miss occurs, all subsequent layers are rebuilt.

Rule 1: Order Instructions by Change Frequency

# BAD: Copying source code before installing dependencies
FROM node:20
WORKDIR /app
COPY . .                    # Source code changes → cache bust
RUN npm ci                  # Reinstalls ALL dependencies every time

# GOOD: Install dependencies first, then copy source
FROM node:20
WORKDIR /app
COPY package.json package-lock.json ./    # Only changes when deps change
RUN npm ci                                 # Cached if package*.json unchanged
COPY . .                                   # Only source code rebuild

Rule 2: Use Multi-Stage Builds

# Multi-stage: build dependencies are not in the final image
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/index.js"]
# Final image: small, no build tools, no source code

Rule 3: Use BuildKit Cache Mounts

# syntax=docker/dockerfile:1

# Cache package manager downloads across builds
FROM python:3.12
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt
COPY . .

# Go modules cache
FROM golang:1.22
WORKDIR /app
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
    go build -o /app/server .

# npm cache
FROM node:20
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .

Rule 4: Use .dockerignore

# .dockerignore
.git
.gitignore
node_modules
dist
*.md
.env*
.vscode
.idea
__pycache__
*.pyc
.pytest_cache
coverage
test
tests
Dockerfile
docker-compose*.yml
.dockerignore

Rule 5: Combine RUN Commands

# BAD: Each RUN creates a layer and can bust cache independently
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN apt-get clean

# GOOD: Single layer, cleanup in same layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

CI/CD Cache Strategies

# GitHub Actions with BuildKit cache
- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

# Registry-based cache (works with any CI)
docker buildx build \
    --cache-from type=registry,ref=registry.example.com/myapp:cache \
    --cache-to type=registry,ref=registry.example.com/myapp:cache,mode=max \
    -t registry.example.com/myapp:latest \
    --push .

# Local cache directory
docker buildx build \
    --cache-from type=local,src=/tmp/docker-cache \
    --cache-to type=local,dest=/tmp/docker-cache,mode=max \
    -t myapp:latest .

Measuring Cache Effectiveness

# Build with timing
DOCKER_BUILDKIT=1 docker build --progress=plain -t myapp . 2>&1 | grep -E "^#[0-9]+ (DONE|CACHED)"

# Count cached vs built layers
DOCKER_BUILDKIT=1 docker build -t myapp . 2>&1 | grep -c CACHED
DOCKER_BUILDKIT=1 docker build -t myapp . 2>&1 | grep -c DONE

Best Practices Summary

• Put rarely-changing instructions (base image, system packages) at the top
• Copy dependency files before source code for separate caching
• Use BuildKit cache mounts for package manager caches
• Use multi-stage builds to separate build and runtime
• Maintain a comprehensive .dockerignore file
• Use registry-based or GitHub Actions cache for CI builds
• Combine RUN commands to minimize layers and include cleanup in the same layer
• Measure cache hit rates and optimize the most frequently invalidated layers