Container Images: Building, Optimizing, and Distributing

If you have worked with Docker at all, you have encountered images. They are the foundation everything else builds on. But what is actually going on when you build an image, and why does Docker make such a big deal about layers?

This post goes deep on container images: how they work under the hood, how to write Dockerfiles that build fast and produce small images, and how to get your images safely to where they need to go.

How Container Images Work

A container image is not a single binary blob. It is a stack of read-only layers, each representing a discrete set of filesystem changes from the layer below it.

When Docker builds an image from a Dockerfile, each instruction that modifies the filesystem creates a new layer. The FROM instruction pulls in a base image. The RUN instruction executes a command and commits the result. The COPY instruction adds files from your context.

The Union Filesystem

Docker uses a union filesystem (OverlayFS on most Linux systems) to combine these layers into a single coherent view. When a container starts, Docker adds a writable layer on top. Writes go to this top layer, reads check the top layer first and fall through to lower layers.

+------------------+
|  Writable Layer |  (container layer)
+------------------+
|  Layer 3         |  (COPY app /app)
+------------------+
|  Layer 2         |  (RUN npm ci)
+------------------+
|  Layer 1         |  (COPY package*.json ./)
+------------------+
|  Base Image      |  (FROM node:20-alpine)
+------------------+

The writeable layer only stores what actually changes. If a file exists in layer 2 and layer 3 modifies it, layer 3 stores only the modified blocks, not a full copy. This is copy-on-write, and it is why containers start so fast.

Image Metadata

Beyond the filesystem layers, an image includes metadata:

• Config: The image configuration blob, containing the default command, environment variables, exposed ports, and working directory.
• Manifest: The manifest lists the layers and their sizes, plus the config blob reference. This is what the container runtime downloads.
• Architecture/OS: The platform the image is built for. You cannot run an linux/arm64 image on a linux/amd64 host without QEMU emulation.

Writing Efficient Dockerfiles

The way you write your Dockerfile has a massive impact on build time and image size. Here are the patterns that matter most.

Order Instructions to Maximize Cache Reuse

Docker caches each layer. When you rebuild, Docker checks if it has already built a layer with identical instruction and content. If so, it skips rebuilding and reuses the cached layer.

The catch: if any layer changes, all subsequent layers get invalidated.

This means you should order instructions from least to most frequently changing:

# Start with dependencies that change rarely
FROM node:20-alpine

WORKDIR /app

# Copy only package files first (this cache survives most code changes)
COPY package*.json ./

# Install dependencies (cached until package.json changes)
RUN npm ci --only=production

# Copy application code last (changes most frequently)
COPY . .

EXPOSE 3000

CMD ["node", "server.js"]

If you copy everything before running npm ci, every code change blows away the dependency cache and forces a full reinstall.

Minimize Layer Count

Each RUN, COPY, and ADD instruction creates a layer. While layers are deduplicated at runtime, they still add overhead during build, pull, and push operations.

Combine related commands using shell chaining:

# Anti-pattern: separate RUN instructions
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get clean

# Better: single RUN with cleanup
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

The apt-get clean and rm -rf /var/lib/apt/lists/* remove the package manager cache within the same layer. If you put cleanup in a separate RUN instruction, it becomes a separate layer that does not help image size.

Use Specific Base Image Versions

Avoid latest. Pin to specific versions or tags:

# Anti-pattern: moving target
FROM node:latest

# Better: pinned version
FROM node:20.11.1-alpine3.19

The latest tag changes without warning. Your reproducible build is not reproducible if the base image shifts out from under you.

Do Not Copy Unnecessary Files

Your build context includes everything in the directory where you run docker build. Large contexts mean slow uploads to the Docker daemon, especially for remote Docker hosts.

Use a .dockerignore file to exclude files:

.git
.node_modules
dist
*.log
.env
.env.*
coverage
README.md

Be explicit about what you need:

# Anti-pattern: copying everything
COPY . .

# Better: explicit paths
COPY package*.json ./
COPY src ./src
COPY public ./public

Multi-Stage Builds for Production

Multi-stage builds solve a specific problem: your build process needs tools and dependencies that your runtime does not. Compiler toolchains, test frameworks, and development libraries bloat the final image and expand the attack surface.

Multi-stage builds use multiple FROM statements. Each FROM starts a fresh stage. You copy artifacts from one stage to another, leaving behind everything the runtime does not need.

When to Use Multi-Stage Builds / When Not To

Use multi-stage builds when:

• Your application requires compilation or build steps (compiled languages, TypeScript, bundlers)
• You need to keep production images minimal and free of build artifacts
• Security and attack surface reduction are priorities
• You deploy frequently and image size affects deployment speed
• You need to run as non-root but build as root

Skip multi-stage builds when:

• Your application is a simple interpreted script
• You use a pre-built image with all dependencies already included
• Build time is not a concern and image size does not matter

Two-Stage Build Flow

flow TB
    subgraph Builder["Builder Stage (e.g., golang:1.22-alpine)"]
        B1[Copy Source Code]
        B2[Install Dependencies]
        B3[Compile Binary]
        B4[Build Output Ready]
        B1 --> B2 --> B3 --> B4
    end

    subgraph Runtime["Runtime Stage (e.g., alpine:3.19)"]
        R1[Minimal Base Image]
        R2[Copy Binary from Builder]
        R3[Set Non-Root User]
        R4[Configure Entrypoint]
        R1 --> R2 --> R3 --> R4
    end

    Builder --> |COPY --from=builder| Runtime
    Runtime --> FinalImage["Final Image<br/>(~15MB for Go)"]

Go Application Example

# Build stage
FROM golang:1.22-alpine AS builder

WORKDIR /app

# Install build dependencies
RUN apk add --no-cache git

# Copy source and build
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o myapp .

# Runtime stage
FROM alpine:3.19

WORKDIR /app

# Install runtime dependencies only
RUN apk add --no-cache ca-certificates && \
    addgroup -S appgroup && \
    adduser -S appuser -u 1001 -G appgroup

# Copy binary from builder stage
COPY --from=builder /app/myapp .

# Switch to non-root user
USER appuser

ENTRYPOINT ["./myapp"]

The final image contains only the binary, CA certificates for HTTPS, and the Alpine base. No Go compiler, no git, no source code.

Node.js Application Example

# Build stage
FROM node:20-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine AS production

WORKDIR /app

# Copy only production dependencies
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force

# Copy built artifacts
COPY --from=builder /app/dist ./dist

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001 -G nodejs

USER nodejs

EXPOSE 3000

CMD ["node", "dist/server.js"]

Python Application Example

# Build stage
FROM python:3.12-slim AS builder

WORKDIR /app

# Install build dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    gcc \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# Runtime stage
FROM python:3.12-slim

WORKDIR /app

# Copy installed packages from builder
COPY --from=builder /install /usr/local

# Copy application code
COPY . .

# Create non-root user
RUN useradd --create-home appuser
USER appuser

CMD ["python", "server.py"]

Image Registries and Distribution

An image registry is a server that stores and serves Docker images. The public Docker Hub is the default registry, but most organizations run private registries for their own images.

Understanding Registry URLs

Image names follow the pattern: registry.example.com/namespace/repository:tag

• docker.io/library/nginx: Official nginx image on Docker Hub (library is implicit for official images)
• ghcr.io/myorg/myapp:1.0.0: My container image on GitHub Container Registry
• gcr.io/project-id/myapp:latest: My app on Google Container Registry

If you do not specify a registry, Docker assumes Docker Hub. If you do not specify a tag, Docker assumes latest.

Pushing Images to a Registry

# Tag your image for the registry
docker tag myapp:1.0.0 registry.example.com/myorg/myapp:1.0.0

# Login to the registry (if private)
docker login registry.example.com

# Push
docker push registry.example.com/myorg/myapp:1.0.0

Pulling Images Efficiently

# Pull specific version
docker pull nginx:1.25.3-alpine

# Pull and run in one command
docker run -d --name nginx nginx:1.25.3-alpine

When you pull an image, Docker downloads only the layers your host does not already have. If another team member already pulled nginx:1.25.3-alpine, subsequent pulls are essentially instant because all layers are cached locally.

Running a Private Registry

For local development or enterprise use, you can run your own registry:

docker run -d -p 5000:5000 \
    --name registry \
    -v registry_data:/var/lib/registry \
    registry:2

Now you can push and pull from localhost:5000:

docker tag myapp:1.0.0 localhost:5000/myapp:1.0.0
docker push localhost:5000/myapp:1.0.0

For production private registries, consider managed services: AWS ECR, Google Artifact Registry, Azure Container Registry, or self-hosted solutions like Harbor.

Security Scanning and Image Signing

Images can contain vulnerabilities. The packages you install, the base image itself, anything copied into the image is a potential attack vector.

Scanning Images with Trivy

Trivy is an open-source vulnerability scanner for containers:

# Scan an image
trivy image nginx:1.25.3-alpine

# Scan and fail on HIGH or CRITICAL vulnerabilities
trivy image --exit-code 1 --severity HIGH,CRITICAL myapp:1.0.0

In CI/CD pipelines, you integrate Trivy to block deployments with known vulnerabilities:

# GitHub Actions example
- name: Scan image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: myapp:1.0.0
    format: sarif
    output: trivy-results.sarif
    severity: HIGH,CRITICAL

- name: Upload scan results
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: trivy-results.sarif

Image Signing with Cosign

Image signing verifies that the image you are pulling is actually the one that was built by your CI/CD pipeline, not an imposter.

Cosign signs images using keys stored in your infrastructure:

# Generate a key pair
cosign generate-key-pair

# Sign an image
cosign sign --key cosign.key registry.example.com/myorg/myapp:1.0.0

# Verify before deployment
cosign verify --key cosign.pub registry.example.com/myorg/myapp:1.0.0

The signature attestations get stored alongside the image in the registry. Your deployment pipeline verifies the signature before pulling the image, ensuring tamper-proof images.

Production Failure Scenarios

Container image pipelines fail in ways that are not always obvious. Here are the most common issues and how to diagnose them.

Layer Cache Corruption

Sometimes cached layers become corrupted. Docker detects this through content-addressable storage, but occasionally corruption slips through.

Symptoms: intermittent failures that appear random, checksum mismatches in build logs, “Function not found” errors.

Diagnosis:

# Verify layer integrity
docker inspect <image> | jq '.[0].RootFS.Layers'

# Pull with checksum verification
docker pull --checksum true <image>

Mitigation: Run docker build --no-cache periodically, and use BuildKit with build-time content hashing.

Registry Authentication Failures

Private registries require authentication. Tokens expire, credentials rotate, and network policies can block access.

Symptoms: “401 Unauthorized” errors on pull, “Image not found” when the image exists.

Diagnosis:

# Check registry connectivity
docker login <registry-url>

# Verify credentials stored
cat ~/.docker/config.json

Mitigation: Use robot accounts with least privilege, set up credential helpers (docker-credential-*), and implement token refresh logic in CI/CD.

Image Digest Mismatch

A digest mismatch means the image content changed between build and pull. This can happen through tampering, network corruption, or race conditions in multi-architecture builds.

Symptoms: “digest mismatch” errors, deployments succeed but application behaves differently.

Mitigation: Pin images by digest in production (image@sha256:...), use image signing (Cosign), and implement SBOM generation to track image contents.

Layer Caching Strategies

Efficient cache use is the difference between builds that take 2 minutes and builds that take 20.

Dependency Caching

The key insight: dependencies change less frequently than application code. Put them early in the Dockerfile, after the base image.

FROM node:20-alpine

WORKDIR /app

# Dependencies change weekly at most
COPY package*.json ./
RUN npm ci

# Code changes multiple times per day
COPY src ./src
COPY public ./public

CMD ["node", "src/server.js"]

Now when only src/ changes, Docker reuses the npm ci layer.

Build Arguments and Cache

Build arguments (ARG) do not persist in the image, but they still affect caching because they change the instruction hash.

ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}

RUN echo "Building for ${NODE_ENV}"

If you change NODE_ENV between builds, the cache breaks at that layer. Sometimes this is what you want, sometimes not.

Cache Invalidation on COPY

When you copy files, Docker hashes the files, not just the instruction. If the file contents are identical, the layer is cached.

This means copying identical files does not invalidate cache:

COPY package*.json ./
RUN npm ci
COPY . .  # Only invalidates if files differ from last build

But if package.json changes, both it and the subsequent RUN layer invalidate.

Optimizing Image Size

Small images pull faster, deploy faster, and have smaller attack surfaces.

Choose the Right Base Image

Image	Size	Use Case
ubuntu:24.04	~77MB	When you need a full OS
debian:bookworm-slim	~30MB	Smaller Debian, still compatible
alpine:3.19	~7MB	Minimal, musl libc
distroless/nodejs	~15MB	Minimal with Node, no shell
scratch	0MB	No OS, binary only

Alpine uses musl libc instead of glibc. Most Node.js, Python, and Go programs work fine on Alpine. Some software (notably anything with native modules that compile against glibc) does not.

Trade-off Table: Build Strategies

Aspect	Monolithic	Multi-Stage	Builder Pattern
Image Size	Large (includes build tools)	Small (runtime only)	Small
Build Speed	Fast (no copy overhead)	Moderate	Fast
Complexity	Simple	Moderate	High
Cache Efficiency	Poor (any change rebuilds all)	Good	Excellent
Security Surface	Large	Minimal	Minimal
Reproducibility	Variable	High	High
Best For	Debugging, interpreted languages	Compiled languages, production	Complex builds, monorepos

Remove Documentation and Man Pages

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    curl \
    && rm -rf /var/lib/apt/lists/* \
    /usr/share/doc/* \
    /usr/share/man/* \
    /var/log/* \
    /tmp/*

Use .dockerignore Aggressively

.git
.gitignore
*.md
docs
tests
.dockerignore
Dockerfile
docker-compose.yml
.env*

Every file you exclude is not uploaded to the Docker daemon, not stored in a layer, not pulled by users.

Anti-Patterns

These patterns cause problems in production. Avoid them.

Copying Entire Context

# Anti-pattern: copying everything
COPY . .

This includes build artifacts, temporary files, and potentially secrets. Always use explicit paths:

COPY package*.json ./
COPY src ./src
COPY public ./public

Not Cleaning Up in the Same Layer

# Anti-pattern: cleanup in separate layer
RUN apt-get update && apt-get install -y curl
RUN apt-get clean

The cleanup does not reduce image size because it happens in a different layer. Combine everything:

RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Using :latest Tag

# Anti-pattern: moving target
FROM node:latest

The latest tag changes without warning. Your reproducible build is not reproducible if the base image shifts out from under you. Always pin versions:

FROM node:20.11.1-alpine3.19

Not Using .dockerignore

Without .dockerignore, your build context includes everything:

.git
.node_modules
dist
*.log
.env
.env.*
coverage

This slows down builds and can leak secrets or large files into images.

Conclusion

Container images are built from layers, and understanding how those layers work lets you build faster and smaller images. Multi-stage builds separate what you build from what you run. Image registries distribute your builds, and security scanning plus signing keeps them safe.

The fundamentals covered here build directly on what you learned in the Docker Fundamentals guide. Next, explore how containers communicate through Docker Networking or how they persist data with Docker Volumes.

For production workloads, the next step is usually Kubernetes, but the image optimization principles apply anywhere containers run.

Recap Checklist

Use this checklist when writing or reviewing Dockerfiles:

• Order instructions from least to most frequently changing
• Copy dependency files before application code
• Combine related RUN commands to minimize layers
• Clean up in the same layer that creates artifacts
• Use specific base image versions, not latest
• Use .dockerignore to exclude unnecessary files
• Use multi-stage builds for compiled languages
• Copy only runtime dependencies to final stage
• Run as non-root user in production images
• Pin images by digest in production, not just tag
• Scan images for vulnerabilities before deployment
• Sign images with Cosign for tamper-proof distribution