Docker Compose:
A Comprehensive Guide

Table of Contents

Last updated: April 13, 2025

1. Introduction to Docker Compose

Docker Compose is a tool that helps define and manage multi-container Docker applications. With Compose, you use a YAML file to configure your application's services, networks, and volumes, then create and start all services from that configuration with a single command. Essentially, it's orchestration for local development and testing environments.

While Docker allows you to run individual containers, real-world applications often consist of multiple interconnected components—a web server, a database, a caching service, etc. Docker Compose lets you define these components and their relationships in a declarative way, making it easy to recreate the entire environment consistently.

Use cases for Docker Compose include:

  • Development environments: When you need to run an application with all its dependencies locally.
  • Automated testing: Creating and tearing down isolated testing environments.
  • Single-host deployments: For simple production deployments on a single host.
  • CI/CD workflows: As part of continuous integration or deployment pipelines.
  • Demos and proof-of-concepts: Easily shareable application setups.

For more complex orchestration across multiple hosts in production environments, tools like Kubernetes or Docker Swarm are typically used instead.

2. Getting Started

2.1 Installation

Docker Compose installation depends on your operating system and Docker setup:

Docker Desktop

If you're using Docker Desktop for Windows, Mac, or Linux, Docker Compose is already included.

Linux with Docker Engine

On Linux systems with Docker Engine installed:

# Option 1: Using the compose plugin (recommended)
sudo apt-get update
sudo apt-get install docker-compose-plugin

# Option 2: Using the standalone docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/v2.20.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

To verify your installation:

# If using the plugin
docker compose version

# If using standalone docker-compose
docker-compose --version

2.2 Basic Concepts

Docker Compose revolves around a few key concepts:

Services

A service is a container and its configuration. It represents a component of your application—like a web server, database, or cache. Each service runs in its own container.

docker-compose.yml

This YAML file defines all the services, networks, and volumes for your application. It's the blueprint for your multi-container application.

Project

A Compose project is a collection of services defined in a docker-compose.yml file, usually representing one application. By default, the project name is the name of the directory containing your compose file.

Basic Workflow

The typical Docker Compose workflow:

  1. Create a Dockerfile for each service that needs a custom image
  2. Define services in docker-compose.yml
  3. Run docker compose up to start the application
  4. Run docker compose down when finished to stop and remove resources

3. The docker-compose.yml File

3.1 Basic Structure

A docker-compose.yml file is structured with several top-level keys:

version: '3.8'  # Compose file format version

services:     # Container definitions
  service1:
    # service1 configuration
  service2:
    # service2 configuration

networks:     # Network definitions (optional)
  network1:
    # network1 configuration

volumes:      # Volume definitions (optional)
  volume1:
    # volume1 configuration

configs:      # Configs definitions (optional)
  config1:
    # config1 configuration

secrets:      # Secrets definitions (optional)
  secret1:
    # secret1 configuration

The version key specifies which Compose file version you're using. For most modern Docker installations, version '3' or above is appropriate.

3.2 Services Configuration

The services section defines your containers and their configuration. Here's a detailed example:

services:
  web:
    image: nginx:alpine         # Use a pre-built image
    build:                      # Or build from a Dockerfile
      context: ./web            # Build context directory
      dockerfile: Dockerfile    # Path to Dockerfile
    container_name: my_web_app  # Custom container name
    ports:
      - "8080:80"               # Port mapping (host:container)
    volumes:
      - ./web:/usr/share/nginx/html  # Mount a host directory
    depends_on:                 # Dependency on other services
      - api
    restart: unless-stopped     # Restart policy
    environment:                # Environment variables
      - NODE_ENV=production
    env_file: .env              # Load environment from file
    networks:                   # Assign to networks
      - frontend
    deploy:                     # Resource constraints
      resources:
        limits:
          cpus: '0.5'
          memory: 500M
    healthcheck:                # Container health check
      test: ["CMD", "curl", "-f", "http://localhost"]
      interval: 1m
      timeout: 10s
      retries: 3

  api:
    build: ./api
    expose:                     # Expose ports without publishing
      - "3000"
    # Other configuration...

Common Service Configuration Options

  • image: Pre-built Docker image to use
  • build: Build a custom image from a Dockerfile
  • ports: Map container ports to host ports
  • expose: Expose ports without publishing to host
  • volumes: Mount volumes (host paths or named volumes)
  • environment: Set environment variables
  • env_file: Load environment variables from a file
  • networks: Connect to specific networks
  • depends_on: Express dependency on other services
  • restart: Restart policy (no, always, on-failure, unless-stopped)
  • deploy: Configuration related to deployment
  • healthcheck: Configure health checking
  • command: Override default command
  • entrypoint: Override default entrypoint

3.3 Networks

The networks section defines the networks that your services can connect to:

networks:
  frontend:
    driver: bridge              # Network driver to use
    
  backend:
    driver: bridge
    ipam:                       # IP Address Management
      config:
        - subnet: 172.20.0.0/16 # Custom subnet
          gateway: 172.20.0.1   # Custom gateway
    
  outside:
    external: true              # Use pre-existing network

By default, Docker Compose creates a single network for your app. However, you can create multiple isolated networks for different service groups. For instance, you might have a frontend network for web and proxy services, and a separate backend network for database and API services.

3.4 Volumes

The volumes section defines named volumes that your services can mount:

volumes:
  db-data:                      # Simple volume definition
    
  redis-data:
    driver: local               # Volume driver to use
    driver_opts:                # Driver-specific options
      type: 'none'
      o: 'bind'
      device: '/data/redis'
    
  config-vol:
    external: true              # Use pre-existing volume

Named volumes are useful for persisting data across container restarts and even between different compose projects. They provide a way to manage persistent data separately from your services definitions.

3.5 Environment Variables

Environment variables can be defined in several ways:

Inline in the Compose File

services:
  web:
    environment:
      - NODE_ENV=production     # Individual variables
      - DEBUG=0
      - API_URL=http://api:3000

Using an Environment File

services:
  web:
    env_file: 
      - ./common.env            # Load from environment file
      - ./web-specific.env      # Can specify multiple files

Using Variable Substitution

You can use variables in your Compose file:

services:
  web:
    image: "webapp:${TAG:-latest}"      # Use variable with default value
    ports:
      - "${WEB_PORT:-8080}:80"          # Variable substitution in ports

These variables can come from:

  • A .env file in the same directory as your docker-compose.yml
  • Environment variables set in your shell
  • Command line with -e flag: docker compose -e VAR=value up

4. Essential Docker Compose Commands

Docker Compose provides commands to manage the entire lifecycle of your multi-container application.

4.1 Lifecycle Commands

Starting Services

# Create and start all services
docker compose up

# Start in detached mode (background)
docker compose up -d

# Start specific services only
docker compose up -d service1 service2

# Force recreation of containers
docker compose up -d --force-recreate

# Build or rebuild images before starting
docker compose up -d --build

Stopping Services

# Stop services but keep containers
docker compose stop

# Stop specific services
docker compose stop service1 service2

# Stop and remove containers, networks
docker compose down

# Remove volumes as well
docker compose down -v

# Remove images too
docker compose down --rmi all

Building Images

# Build or rebuild all service images
docker compose build

# Build specific services
docker compose build service1 service2

# Build with no cache
docker compose build --no-cache

4.2 Monitoring Commands

Checking Status

# List running containers
docker compose ps

# List all containers (including stopped)
docker compose ps -a

Viewing Logs

# View logs from all services
docker compose logs

# Follow log output
docker compose logs -f

# Show logs for specific services
docker compose logs -f service1 service2

# Show last 100 lines
docker compose logs --tail=100

Running Commands

# Run a command in a service
docker compose exec service1 command

# Example: open shell in web service
docker compose exec web bash

# Run a one-time command in a new container
docker compose run service1 command

# Example: run tests
docker compose run web npm test

4.3 Scaling Services

Docker Compose allows you to run multiple instances of a service:

# Start 3 instances of the web service
docker compose up -d --scale web=3

# Scale multiple services
docker compose up -d --scale web=3 --scale worker=2

To enable scaling, ensure:

  • You don't set container_name (as it must be unique)
  • You don't publish ports to fixed host ports (use port ranges or let Docker assign)

5. Real-World Examples

5.1 Web Application with Database

A common pattern for web applications with database:

version: '3.8'

services:
  web:
    build: ./web
    ports:
      - "8080:80"
    depends_on:
      - db
    environment:
      - DATABASE_URL=postgresql://postgres:password@db:5432/app
    restart: unless-stopped
    volumes:
      - ./web/src:/app/src
      - /app/node_modules
    networks:
      - app-network

  db:
    image: postgres:13
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=postgres
      - POSTGRES_DB=app
    ports:
      - "5432:5432"  # Expose to host for development tools
    restart: unless-stopped
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

volumes:
  postgres-data:

5.2 Microservices Architecture

A more complex microservices setup with multiple services:

version: '3.8'

services:
  reverse-proxy:
    image: traefik:v2.6
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
    ports:
      - "80:80"
      - "8080:8080"  # Traefik dashboard
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - frontend

  auth-service:
    build: ./auth-service
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.auth.rule=PathPrefix(`/auth`)"
    environment:
      - REDIS_URL=redis://cache:6379
    depends_on:
      - cache
    networks:
      - frontend
      - backend

  user-service:
    build: ./user-service
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.user.rule=PathPrefix(`/users`)"
    environment:
      - MONGODB_URL=mongodb://mongodb:27017/users
    depends_on:
      - mongodb
    networks:
      - frontend
      - backend

  product-service:
    build: ./product-service
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.product.rule=PathPrefix(`/products`)"
    environment:
      - POSTGRES_URL=postgresql://postgres:password@postgres:5432/products
    depends_on:
      - postgres
    networks:
      - frontend
      - backend

  mongodb:
    image: mongo:5
    volumes:
      - mongodb-data:/data/db
    networks:
      - backend

  postgres:
    image: postgres:13
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=products
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - backend

  cache:
    image: redis:alpine
    networks:
      - backend

networks:
  frontend:
  backend:

volumes:
  mongodb-data:
  postgres-data:

5.3 Development Environment

A development environment with hot reloading:

version: '3.8'

services:
  frontend:
    build: 
      context: ./frontend
      target: development  # Using multi-stage builds
    ports:
      - "3000:3000"
    volumes:
      - ./frontend:/app
      - /app/node_modules
    command: npm start
    environment:
      - REACT_APP_API_URL=http://localhost:4000/api
    depends_on:
      - api

  api:
    build:
      context: ./api
      target: development
    ports:
      - "4000:4000"
    volumes:
      - ./api:/app
      - /app/node_modules
    command: npm run dev
    environment:
      - NODE_ENV=development
      - DB_HOST=db
      - DB_USER=postgres
      - DB_PASSWORD=postgres
      - DB_NAME=devdb
    depends_on:
      - db

  db:
    image: postgres:13-alpine
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=devdb
    ports:
      - "5432:5432"
    volumes:
      - postgres-dev-data:/var/lib/postgresql/data

volumes:
  postgres-dev-data:

6. Best Practices

6.1 File Organization

Organizing your Docker Compose files effectively:

  • Environment-specific files: Use override files for different environments 
    # Base configuration
docker-compose.yml

# Production overrides
docker-compose.prod.yml

# Development overrides
docker-compose.dev.yml

# Run with overrides
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
  • Project structure: A typical project might look like: 
    project/
├── docker-compose.yml
├── docker-compose.override.yml     # Local development overrides
├── docker-compose.prod.yml         # Production overrides
├── .env                            # Environment variables
├── service1/
│   ├── Dockerfile
│   ├── src/
│   └── ...
└── service2/
    ├── Dockerfile
    ├── src/
    └── ...
  • Extract common configurations: Use YAML anchors and aliases for DRY (Don't Repeat Yourself) configuration: 
    x-common-config: &common-config
  restart: unless-stopped
  logging:
    driver: "json-file"
    options:
      max-size: "10m"
      max-file: "3"

services:
  service1:
    <<: *common-config
    # service1 specific config
    
  service2:
    <<: *common-config
    # service2 specific config

6.2 Security Considerations

Important security practices:

  • Don't hardcode secrets: Use environment variables or Docker secrets 
    services:
  web:
    environment:
      - DB_PASSWORD=${DB_PASSWORD}  # From .env or shell environment
  • Use non-root users: Run containers as non-root users 
    services:
  app:
    build: ./app
    user: "1000:1000"  # Specify user:group IDs
  • Limit container capabilities: Reduce privileges 
    services:
  app:
    cap_drop:
      - ALL       # Drop all capabilities
    cap_add:
      - NET_BIND_SERVICE  # Add only what's needed
  • Use read-only file systems when possible: 
    services:
  app:
    read_only: true
    tmpfs:
      - /tmp      # Add writable tmpfs mounts where needed
      - /var/run
  • Network isolation: Separate services into different networks
  • Control exposed ports: Only expose what's necessary

6.3 Performance Optimization

Improving performance:

  • Build context optimization: Use .dockerignore to exclude unnecessary files
  • Multi-stage builds: Create smaller production images
  • Resource limits: Set resource constraints 
    services:
  app:
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 500M
        reservations:
          memory: 200M
  • Volume mount performance: For development on macOS/Windows, consider using volume delegations: 
    volumes:
  - ./src:/app:delegated  # Improves performance on macOS/Windows
  • Health checks: Add health checks to ensure services are ready 
    services:
  db:
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

7. Advanced Techniques

7.1 Multiple Compose Files

Compose allows combining multiple files for different environments or concerns:

# Base configuration
docker compose -f docker-compose.yml \
               -f docker-compose.prod.yml \
               up -d

Later files override values in earlier files. Common patterns:

  • docker-compose.yml - Base configuration
  • docker-compose.override.yml - Development overrides (loaded automatically)
  • docker-compose.prod.yml - Production settings
  • docker-compose.test.yml - Testing configuration

7.2 Profiles

Profiles allow you to selectively enable services for different use cases:

services:
  app:
    image: myapp:latest
    # Always started

  db:
    image: postgres:13
    # Always started
  
  pgadmin:
    image: dpage/pgadmin4
    profiles:
      - debug     # Only started when debug profile is enabled
  
  test-runner:
    image: myapp-tests
    profiles:
      - test      # Only started when test profile is enabled

To enable specific profiles:

# Enable the debug profile
docker compose --profile debug up

# Enable multiple profiles
docker compose --profile debug --profile test up

7.3 Extending Services

You can extend service definitions using the extends option:

# common-services.yml
services:
  app-base:
    build: ./app
    environment:
      - LOG_LEVEL=info
    restart: unless-stopped

# docker-compose.yml
services:
  web:
    extends:
      file: common-services.yml
      service: app-base
    ports:
      - "8080:80"
    
  api:
    extends:
      file: common-services.yml
      service: app-base
    command: ["./run-api.sh"]
    ports:
      - "8081:80"

8. Docker Compose Alternatives

While Docker Compose is excellent for local development and simple deployments, larger or more complex deployments might require alternative solutions:

Container Orchestration Systems

Tool Description Use Cases
Kubernetes Industry-standard container orchestration platform Complex, large-scale production deployments
Docker Swarm Docker's native clustering and orchestration tool Simpler production deployments, Docker-native workflow
Amazon ECS AWS container orchestration service AWS-based containerized applications
Nomad HashiCorp's workload scheduler Mixed workloads (containers, VMs, etc.)

Compose-Compatible Tools

  • Podman Compose: Docker Compose compatibility for Podman.
  • Kompose: Convert Docker Compose files to Kubernetes resources.
  • Compose on Kubernetes: Deploy Compose files directly to Kubernetes.

9. Additional Resources

To deepen your understanding of Docker Compose:

Official Documentation

Learning Resources

Related Pages on Our Site