Docker in depth: images, layers, volumes, networking, and Compose

Docker is a powerful tool for packaging applications with all their dependencies into lightweight, isolated processes called containers. Containers share the host kernel instead of virtualizing hardware, making them more efficient than full virtual machines (VMs). This document covers Docker images, layers, volumes, networking, and how to use Compose to manage multi-container applications.

Overview

Docker packages an application together with everything it needs to run into a CONTAINER — a lightweight, isolated process that behaves identically on your laptop, server, or cloud VM. Containers share the host kernel instead of virtualizing hardware, making them more efficient than full VMs (see the VM-vs-container doc for that distinction).

Images vs. Containers

An IMAGE is an immutable template (the recipe plus ingredients, frozen). A CONTAINER is a running instance of an image (a meal cooked from the recipe). You can start many containers from one image.

Layers: The Core Efficiency Mechanism

An image is built as a stack of layers, one per instruction in the build file. Layers are content-addressed, cached, and shared between images — if ten images use the same Ubuntu base layer, it's stored once. Containers add a thin writable copy-on-write layer on top; changes live there and vanish when the container is removed (which is why data needs volumes).

The Dockerfile: Build Recipes

The Dockerfile contains instructions to build an image:

FROM: Base image.
RUN: Execute a command, creating a new layer.
COPY/ADD: Bring files in.
WORKDIR: Set the directory.
ENV: Environment variables.
ARG: Build-time variables.
EXPOSE: Document a port.
CMD (default command, overridable) vs. ENTRYPOINT (fixed executable).
Multi-stage builds: A heavy "build" stage compiles the app, then a slim final stage copies only the built artifact.

Best Practices

Small base images (Alpine or Distroless for minimal attack surface).
Use a .dockerignore file.
Pin versions (never trust "latest" in production).
Run as a non-root user.
Combine RUN commands to reduce layers.

Registries: Where Images Live

Registries store Docker images:

Docker Hub: Default, but has pull rate limits for anonymous users.
GitHub Container Registry: ghcr.io.
Self-hosted options like Harbor or the plain registry image.

Images are named registry/repo:tag. The "latest" tag is a trap — it's just a default tag, not "the newest." Pinning a real version tag makes deployments reproducible.

Volumes and Data Persistence

Containers are ephemeral — delete the container and its writable layer is gone. Anything that must survive goes in a VOLUME:

Named volumes: Docker manages storage (best for databases and app data).
Bind mounts: Map a host directory into the container (best for config files and development).
tmpfs mounts live in RAM.

This site's blog data survives container rebuilds because it uses a named volume to hold posts, so rebuilding the blog container never loses content. Backups target the volumes, not the containers.

Networking

Docker supports user-defined bridge networks:

Containers on a user-defined bridge network reach each other by container name via Docker's built-in DNS.
Published ports: Punch a container port through to the host; backends without published ports are unreachable from outside the Docker network, which is free isolation.
host and none network modes exist for special cases.

Docker Compose: Managing Multi-container Applications

Docker Compose allows you to declare a whole multi-container application in a single YAML file:

Services, their images, ports, volumes, networks, environment, restart policies, and depends_on ordering.
docker compose up -d brings the entire stack up.

This is the real-world way self-hosters run everything; this site is about a dozen Compose services (web, blog, API proxy, help desk, whiteboard, tunnel) in one file. Compose is the sweet spot for a single host — you get declarative, version-controllable infrastructure without the complexity of orchestration.

Running Containers in Production

docker ps: What's running.
docker logs: Output.
docker exec: Get a shell inside.
docker stats: Resource use.
Restart policies: Auto-recover crashed containers.
Healthchecks: Let Docker know when a container is actually ready, not just started.
Resource limits: Memory and CPU caps to stop one container from starving the host.
Logging drivers control where output goes.

Security

Key security practices:

Don't run containers as root — a root process that escapes the container is root on the host; use a non-root user.
The Docker socket (/var/run/docker.sock) is root on the host — mounting it into a container hands that container full control of the machine; treat it like a password.
Drop unneeded Linux capabilities, use read-only filesystems where possible.
Prefer official/verified images, scan for known vulnerabilities, and keep them updated (though auto-updaters like Watchtower trade convenience for risk).
Secrets don't belong in images or environment variables baked into the image — use Docker secrets or a mounted file.

Rootless and Alternatives

Rootless Docker runs the daemon as a normal user, shrinking the blast radius.

Podman: Daemonless, rootless-by-default alternative that's drop-in compatible with most Docker commands and generates Kubernetes manifests. Popular in Red Hat world and for security-conscious setups.

When to Use Docker

For a single host running a handful to a few dozen services, Docker + Compose is the right tool and orchestration is overkill. You reach for Kubernetes when you outgrow one machine: many nodes, self-healing across hosts, auto-scaling, zero-downtime rolling deploys, and service discovery at scale.

This document covers the basics of how to use Docker effectively in a homelab or solo app setup.

flowchart LR
  DF[Dockerfile] --> IMG[Image - stacked cached layers]
  IMG --> REG[Registry]
  IMG --> C[Container - running instance]
  VOL[Named volume] -. persists data .-> C