This document provides comprehensive details about the Docker implementation in the IndexAgent project and its companion airflow-hub repository.
IndexAgent follows a Docker-first approach, with all components running in containers to ensure consistent deployment, isolation, and scalability. The architecture consists of multiple containers that work together to provide code search, indexing, and automated maintenance capabilities.
The IndexAgent stack consists of three primary containers:
-
zoekt-indexserver
- Image:
sourcegraph/zoekt-indexserver:latest - Purpose: Provides high-performance code indexing and search capabilities
- Ports: 6070 (Zoekt API and web UI)
- Volumes: Mounts
$HOME/reposto access source code repositories
- Image:
-
sourcebot
- Image:
ghcr.io/sourcebot-dev/sourcebot:latest - Purpose: Handles source code discovery, repository syncing, and metadata extraction
- Ports: 3000 (Sourcebot UI)
- Volumes: Mounts
$HOME/reposto access source code repositories
- Image:
-
indexagent
- Image:
indexagent:latest(built from local Dockerfile) - Purpose: Runs maintenance scripts and Claude Code CLI operations
- Ports: 8080 (IndexAgent API)
- Volumes:
$HOME/repos: Access to source code repositories../reports: For storing analysis reports./docs/auto: For generated documentation
- Environment Variables:
CONFIG_PATH: Path to configuration fileCLAUDE_MODEL: Claude model version (defaults toclaude-3-opus-2025-05-02)
- Secrets:
claude_api_key: For authenticating with the Claude API
- Image:
The Docker Compose configuration is defined in config/docker-compose.yml and orchestrates the deployment of all containers. Key features include:
- Volume mounting for code repositories
- Port mapping for service access
- Secret management for API keys
- Container restart policies
- Environment variable configuration
The IndexAgent container is built from a custom Dockerfile that:
- Uses
python:3.11-slimas the base image - Sets up the Python environment and working directory
- Copies source code, tests, and scripts into the container
- Installs Python dependencies (pytest, pytest-cov)
- Installs Node.js and npm for Claude CLI and markdownlint
- Installs the Claude Code CLI and markdownlint
- Configures the container to remain running for service availability
The airflow-hub repository uses Docker to run Apache Airflow 3.0, which orchestrates maintenance tasks by calling the IndexAgent services. The integration works as follows:
- Airflow DAGs in airflow-hub use SSHOperator to execute scripts in the IndexAgent container
- The scripts access mounted repositories, perform maintenance tasks, and push changes back to source repositories
- Metrics and reports generated by IndexAgent are available to Airflow for monitoring and alerting
IndexAgent uses Docker secrets for managing sensitive information:
- In production environments, secrets are stored using Docker's secrets management
- For local development, a
.envfile can be used - The Claude API key is stored as a secret and mounted into the indexagent container
The containers communicate over a Docker network:
- Zoekt and Sourcebot expose web UIs and APIs on mapped ports
- The IndexAgent container can access these services via internal Docker networking
- External access is controlled through port mapping in the docker-compose.yml
- Isolation: Each component runs in its own container
- Reproducibility: Container definitions ensure consistent environments
- Volume Mounting: Efficient sharing of code repositories
- Secret Management: Secure handling of API keys
- Port Mapping: Controlled access to services
To start the entire stack:
# From the IndexAgent repository root
docker compose -f config/docker-compose.yml up -d
# Or using the Makefile
make upTo stop the stack:
# From the IndexAgent repository root
docker compose -f config/docker-compose.yml down
# Or using the Makefile
make downThe Docker setup can be customized by:
- Modifying the
docker-compose.ymlfile to change port mappings, volume mounts, or add new services - Updating the Dockerfile to install additional dependencies or change the base image
- Adjusting environment variables in the docker-compose.yml or through a .env file
Common issues and solutions:
- Container fails to start: Check logs with
docker logs <container_name> - Volume mounting issues: Ensure the host directories exist and have appropriate permissions
- API key problems: Verify that secrets are correctly configured
- Network connectivity: Use
docker network inspectto troubleshoot network issues