Build and Store Container Images

Lesson: Build and Store Container Images

Introduction: The Foundation of Modern Software Delivery

In the modern landscape of software development, the way we package and distribute applications has undergone a fundamental shift. We have moved away from manual server configurations and fragile deployment scripts toward the predictable world of containerization. At the heart of this shift lies the container image—a lightweight, standalone, executable package that includes everything needed to run a piece of software: code, runtime, libraries, settings, and system tools.

Building and storing these images effectively is no longer just a task for systems administrators; it is a critical skill for every developer. If your images are bloated, insecure, or poorly structured, your entire development lifecycle suffers. Slow build times, unpredictable runtime behavior, and security vulnerabilities often trace their roots back to how an image was constructed. By mastering the art of building and storing container images, you ensure that your applications move consistently from your local machine to testing, staging, and finally, into production environments without the dreaded "it works on my machine" syndrome.

This lesson explores the technical mechanics of creating container images, the best practices for optimizing them, and the strategies for managing them in container registries. We will look past the basic "hello world" examples to understand how layers work, how to minimize the attack surface, and how to automate the lifecycle of your artifacts.

The Anatomy of a Container Image

To build effective images, you must first understand that a container image is not a single file, but a stack of read-only layers. Each instruction in a Dockerfile—the blueprint for your image—creates a new layer. When you run a container, the runtime adds a thin, writable layer on top of these read-only layers.

Understanding Layers and Caching

When you build an image, the container engine (such as Docker or Podman) checks each step in your Dockerfile. If a step hasn't changed since the last build, the engine reuses the existing layer from the local cache. This mechanism is the primary reason why build times can be fast or agonizingly slow.

If you place a command that changes frequently (like COPY . . which copies your source code) at the top of your Dockerfile, every subsequent layer must be rebuilt because the cache is invalidated. By ordering your instructions from least frequently changed to most frequently changed, you optimize the build process significantly.

Callout: Layers and Filesystems Think of an image layer like a Git commit. Each layer represents a set of changes to the filesystem. When you combine these layers, they form the final view of the filesystem the application sees. Because layers are shared between images, if you have ten different images based on the same Debian base, the host only stores that base layer once on disk. This deduplication is a key reason why containers are so efficient.

Writing an Efficient Dockerfile

A Dockerfile is essentially a recipe. To write a good one, you need to balance readability, security, and performance. Let's look at a practical example of a web application built using Node.js.

A Basic Example: The Naive Approach

# Start with a full Node image
FROM node:18

# Copy everything
COPY . .

# Install dependencies
RUN npm install

# Run the app
CMD ["node", "index.js"]

While this works, it is inefficient. Every time you change a single line of code, the npm install command runs again. Since installing dependencies can take minutes, this is a major productivity killer.

The Optimized Approach

# Use a smaller base image
FROM node:18-alpine

# Set the working directory
WORKDIR /app

# Copy only the package files first
COPY package*.json ./

# Install dependencies - this layer is cached unless package.json changes
RUN npm install

# Copy the rest of the source code
COPY . .

# Run the app
CMD ["node", "index.js"]

By separating the dependency installation from the source code copy, we ensure that npm install only runs when the package.json or package-lock.json files are modified. This simple change can reduce build times from minutes to seconds for most development cycles.

Tip: Use Alpine Images Alpine Linux is a security-oriented, lightweight Linux distribution. Using node:18-alpine instead of node:18 (which is based on Debian) can reduce your image size from hundreds of megabytes to just a few dozen. Smaller images download faster, consume less disk space, and have fewer vulnerabilities to exploit.

Advanced Build Techniques

As your applications grow, you will encounter scenarios where you need to compile code or handle complex build environments. Using multi-stage builds is the industry standard for keeping production images clean and secure.

Multi-Stage Builds

In a multi-stage build, you use one image to build your application and a separate, smaller image to run it. The build environment (which contains compilers, build tools, and source code) is discarded, leaving only the compiled binary or static files in the final image.

# Stage 1: Build
FROM golang:1.20 AS builder
WORKDIR /app
COPY . .
RUN go build -o my-app

# Stage 2: Production
FROM alpine:latest
WORKDIR /root/
# Copy only the binary from the builder stage
COPY --from=builder /app/my-app .
CMD ["./my-app"]

This approach provides two massive benefits:

1. Size: The final image does not include the Go compiler or the source code, making it significantly smaller.
2. Security: By removing build tools and source code, you reduce the "attack surface." If an attacker manages to get a shell inside your container, they won't find compilers or source code to help them escalate their access.

Best Practices for Image Construction

Building images is as much about discipline as it is about syntax. Follow these guidelines to ensure your images are production-ready:

1. Pin Your Base Images

Avoid using the latest tag. When you use FROM node:latest, your build might behave differently today than it does tomorrow because the underlying image has been updated. Always use specific version tags or, for maximum security, use the image digest (e.g., node:18.16.0@sha256:abcdef...).

2. Minimize Layers

While modern container runtimes handle many layers well, it is still good practice to combine related commands into a single RUN instruction using the && operator. This prevents the creation of intermediate layers that don't add value.

# Good
RUN apt-get update && apt-get install -y \
    curl \
    git \
    && rm -rf /var/lib/apt/lists/*

3. Run as a Non-Root User

By default, containers run as the root user. This is a significant security risk. If your application is compromised, the attacker starts with root privileges on the container. Always create a dedicated user and switch to it.

RUN addgroup -S appgroup && adduser -S appuser -G appgroup
USER appuser

4. Use .dockerignore

Just as you use .gitignore to prevent sensitive files from entering your Git repository, you should use .dockerignore to prevent files like .git, node_modules, or local environment variables from being copied into your image. This keeps images small and secure.

Storing Images: Container Registries

Once an image is built, it needs to be stored in a central location so that your servers, CI/CD pipelines, and teammates can access it. This location is called a Container Registry.

Types of Registries

• Public Registries: Services like Docker Hub or GitHub Container Registry (GHCR) allow you to share images with the world. These are great for open-source projects but require caution if you are working with proprietary code.
• Private Registries: Most cloud providers offer private registries (Amazon ECR, Google Artifact Registry, Azure Container Registry). These integrate tightly with their respective cloud platforms and offer fine-grained access control.
• Self-Hosted Registries: Tools like Harbor or the standard Docker Registry can be hosted within your own infrastructure if you require total control over your data and network.

Steps to Push an Image

To store an image, you must first tag it with the registry's address, then log in, and finally push it.

1. Tag the image: docker tag my-app:latest myregistry.com/my-app:v1.0.0
2. Log in to the registry: docker login myregistry.com
3. Push the image: docker push myregistry.com/my-app:v1.0.0

Warning: Secrets Management Never hardcode credentials, API keys, or database passwords inside your Dockerfile or your source code. Even if you don't push the code to a public repository, these secrets become part of the image layers and are permanent, even if you delete the file in a later layer. Use environment variables or secret management services (like HashiCorp Vault or AWS Secrets Manager) to inject these values at runtime.

Comparison of Registry Options

Common Pitfalls and How to Avoid Them

Pitfall 1: Bloated Images

The Problem: Including unnecessary tools like vim, gcc, or git in your production image. The Fix: Use multi-stage builds as described earlier. If you need to debug a running container, use ephemeral debugging tools or "sidecar" containers rather than stuffing everything into the main application container.

Pitfall 2: The "Latest" Tag Trap

The Problem: Relying on the latest tag in production environments leads to non-deterministic deployments. The Fix: Use semantic versioning (e.g., v1.2.3) for your images. This ensures that when you deploy an image, you know exactly what code version is running.

Pitfall 3: Ignoring Security Scanning

The Problem: Using outdated libraries inside the image that have known vulnerabilities (CVEs). The Fix: Most modern registries (ECR, GHCR, etc.) offer built-in vulnerability scanning. Enable this feature and treat a "Critical" vulnerability alert as a deployment blocker.

Pitfall 4: Long Build Times

The Problem: Re-installing dependencies on every code change. The Fix: Leverage the layer cache by ordering your Dockerfile instructions correctly. Copy the dependency manifest files first, run the install, and only then copy the rest of the source code.

Practical Workflow: The Full Lifecycle

Let’s walk through a typical developer workflow to solidify these concepts.

1. Development: You write code locally and test it using a docker-compose.yml file to spin up your app and a database.
2. Versioning: You commit your code to Git and tag the release as v1.0.0.
3. CI Build: Your CI/CD server triggers a build. It checks out the code, runs tests, and builds the image using a multi-stage Dockerfile.
4. Scanning: The CI pipeline pushes the image to a private registry. The registry automatically scans the image for vulnerabilities.
5. Deployment: If the scan passes, the pipeline triggers a deployment to your production environment (e.g., Kubernetes), pulling the v1.0.0 image from the private registry.

This workflow ensures that the exact same binary you tested in your environment is the one that eventually runs in production. There is no recompilation, no dependency mismatch, and no environment configuration drift.

Key Takeaways

• Images are Layered: Understand that each instruction in your Dockerfile creates a layer. Ordering matters for caching and build performance.
• Multi-Stage Builds are Essential: Always separate your build environment from your production runtime to keep images small and secure.
• Security is Paramount: Never run as root, never include secrets in your images, and always enable vulnerability scanning in your registry.
• Versioning is Non-negotiable: Avoid the latest tag in production. Use semantic versioning or image digests to ensure your deployments are predictable and repeatable.
• Keep it Lean: Use minimal base images like Alpine or Distroless to reduce the download time and the attack surface of your application.
• Automate Everything: Use CI/CD pipelines to build, test, scan, and push your images. Manual image building is prone to human error and inconsistency.

By adhering to these principles, you move from simply "using" containers to mastering the lifecycle of containerized applications. This foundation allows you to focus on writing great code, confident that the packaging and delivery mechanism will support your work reliably and securely.

Common Questions (FAQ)

Q: Should I use Distroless images? A: Distroless images contain only your application and its runtime dependencies. They contain no package managers, shells, or any other programs you would expect to find in a standard Linux distribution. They are excellent for security, but they can make debugging inside the container difficult. If you are confident in your application, they are a great choice.

Q: How often should I update my base image? A: You should have a policy for updating base images. At a minimum, you should rebuild your images whenever a base image receives a security patch. Many teams automate this by setting up their CI system to rebuild all images once a week or whenever a new version of the base image is released.

Q: Is it okay to store images on my local machine? A: It is fine for development, but never for production. Production environments must pull images from a centralized, secure, and highly available registry. Relying on a local image leads to single points of failure and makes scaling your infrastructure impossible.

Q: Does the order of instructions in a Dockerfile really affect performance that much? A: Yes, drastically. If you change a single character in your source code, the COPY . . instruction will trigger a cache miss. If that instruction is at the top, every subsequent command—including potentially slow npm install or pip install commands—will be re-executed. Moving the COPY to the bottom ensures that only the code compilation steps are re-run.