Skip to content

Self-Hosting with Docker

This guide provides a complete walkthrough for deploying the Corgi Recommender Service in a production environment using Docker Compose. The deployment includes all necessary services: PostgreSQL database, Redis cache, and the Corgi API.

Prerequisites

Before beginning deployment, ensure you have:

  • Docker Engine 20.10 or later
  • Docker Compose v2.0 or later
  • Git for cloning the repository
  • 4GB RAM minimum (8GB recommended for production)
  • 10GB disk space for Docker images and data
  • Domain name (optional, for HTTPS)
  • SSL certificates (optional, for HTTPS)

Deployment Architecture

graph LR
    subgraph External
        CLIENT[Client Applications]
        MASTODON[Mastodon Instances]
    end

    subgraph Docker["Docker Network"]
        subgraph Services
            API[Corgi API<br/>Port 5002]
            POSTGRES[(PostgreSQL<br/>Port 5432)]
            REDIS[(Redis<br/>Port 6379)]
        end
    end

    CLIENT -->|HTTPS| API
    MASTODON -->|OAuth| API
    API --> POSTGRES
    API --> REDIS

    %% Styling
    classDef service fill:#2d3748,stroke:#4299e1,stroke-width:2px,color:#e2e8f0
    classDef storage fill:#2b6cb0,stroke:#63b3ed,stroke-width:2px,color:#e2e8f0
    classDef external fill:#4a5568,stroke:#cbd5e0,stroke-width:2px,color:#e2e8f0

    class API service
    class POSTGRES,REDIS storage
    class CLIENT,MASTODON external

Step 1: Repository Setup

Clone the repository and navigate to the project directory:

git clone https://github.com/your-org/corgi-recommender-service.git
cd corgi-recommender-service

Step 2: Environment Configuration

Create Production Environment File

Copy the example environment file and create your production configuration:

cp env.example .env

Configure Critical Security Settings

Security Critical

The following environment variables MUST be changed from their defaults for production deployment.

Edit .env and update these critical settings:

# SECURITY CRITICAL - Generate strong passwords
POSTGRES_PASSWORD=your-very-strong-database-password-here
API_SECRET_KEY=your-random-secret-key-minimum-32-chars
JWT_SECRET_KEY=another-random-secret-key-minimum-32-chars
USER_HASH_SALT=random-salt-for-user-hashing

# Set to production mode
CORGI_ENV=production
FLASK_ENV=production
DEBUG=false

# Configure CORS for your domains
CORS_ORIGINS=https://yourdomain.com,https://app.yourdomain.com

# Remove or update the Slack webhook
SLACK_WEBHOOK_URL=your-actual-slack-webhook-or-remove-this-line

Generate Secure Keys

Use these commands to generate secure random keys:

# Generate API_SECRET_KEY
python3 -c "import secrets; print(secrets.token_urlsafe(32))"

# Generate JWT_SECRET_KEY  
python3 -c "import secrets; print(secrets.token_urlsafe(32))"

# Generate USER_HASH_SALT
python3 -c "import secrets; print(secrets.token_hex(16))"

Complete Production Configuration

Here's a complete example of a production-ready .env file:

# Database Configuration
POSTGRES_USER=corgi_prod
POSTGRES_PASSWORD=vErY$tr0ng!P@ssw0rd#2024
POSTGRES_DB=corgi_production
POSTGRES_HOST=postgres
POSTGRES_PORT=5432

# Redis Configuration  
REDIS_HOST=redis
REDIS_PORT=6379
# Optional: Add Redis password for additional security
# REDIS_PASSWORD=redis-password-here

# API Configuration
CORGI_API_HOST_PORT=5002
CORGI_ENV=production
FLASK_ENV=production
DEBUG=false

# Security Keys (use generated values)
API_SECRET_KEY=Kb5P-qX9mJ3nR8vT2wY6zL4fG7hA0sD1
JWT_SECRET_KEY=Nm8R-tY3wX5vB2zK9pL6jH4fA7sG0dQ1
USER_HASH_SALT=a7b9c2d4e6f8a1b3c5d7e9f0a2b4c6d8

# CORS Configuration (update with your domains)
CORS_ORIGINS=https://api.example.com,https://app.example.com

# SSL Configuration (if using HTTPS)
USE_HTTPS=true
SSL_CERT_PATH=/app/certs/cert.pem
SSL_KEY_PATH=/app/certs/key.pem

# Rate Limiting
RATE_LIMITING_ENABLED=true
RATE_LIMIT_DEFAULT=2000 per hour
RATE_LIMIT_ANONYMOUS=1000 per hour

# Monitoring (optional)
SENTRY_DSN=your-sentry-dsn-if-using-sentry
LOG_LEVEL=INFO

Step 3: SSL Certificate Setup (Optional)

If you're using HTTPS, place your SSL certificates in the certs/ directory:

mkdir -p certs
cp /path/to/your/cert.pem certs/
cp /path/to/your/key.pem certs/
chmod 600 certs/key.pem

Step 4: Deploy the Stack

Build and Start Services

Use the standalone profile for API-only deployment:

docker-compose --profile standalone up -d --build

This command will: - Build the Corgi API image with production optimizations - Start PostgreSQL with persistent storage - Start Redis with AOF persistence - Configure health checks for all services - Create an isolated Docker network

Monitor Startup Progress

Watch the logs to ensure services start correctly:

docker-compose logs -f

Look for these success indicators: - postgres | database system is ready to accept connections - redis | Ready to accept connections - corgi-api | Listening at: http://0.0.0.0:5000

Step 5: Database Initialization

Run Database Migrations

After services are running, initialize the database schema:

docker-compose exec corgi-api python -m flask db upgrade

Verify Database Schema

docker-compose exec postgres psql -U corgi_prod -d corgi_production -c "\dt"

You should see tables like: - users - posts - interactions - user_embeddings - post_embeddings

Step 6: Verification

Health Check

Verify the API is running correctly:

curl http://localhost:5002/health

Expected response: 

{
  "status": "healthy",
  "version": "1.0.0",
  "services": {
    "database": "connected",
    "redis": "connected"
  }
}

API Endpoint Test

Test the recommendations endpoint:

curl -X GET http://localhost:5002/api/v1/recommendations \
  -H "Authorization: Bearer YOUR_MASTODON_TOKEN" \
  -H "X-Mastodon-Instance: mastodon.social"

For production, use a reverse proxy like Nginx:

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

    location / {
        proxy_pass http://localhost:5002;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Timeouts for long-running requests
        proxy_read_timeout 120s;
        proxy_connect_timeout 120s;
    }
}

Maintenance Procedures

Viewing Logs

View logs for specific services:

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f corgi-api

# Last 100 lines
docker-compose logs --tail=100 corgi-api

Database Backup

Create automated backups of your PostgreSQL database:

# Manual backup
docker-compose exec postgres pg_dump -U corgi_prod corgi_production > backup_$(date +%Y%m%d_%H%M%S).sql

# Automated daily backups (add to crontab)
0 2 * * * cd /path/to/corgi && docker-compose exec -T postgres pg_dump -U corgi_prod corgi_production > /backups/corgi_$(date +\%Y\%m\%d).sql

Updating the Application

To update to a new version:

# Pull latest changes
git pull origin main

# Rebuild and restart services
docker-compose --profile standalone down
docker-compose --profile standalone up -d --build

# Run any new migrations
docker-compose exec corgi-api python -m flask db upgrade

Monitoring Resources

Monitor Docker resource usage:

# Container stats
docker stats

# Detailed service status
docker-compose ps

# Check service health
docker-compose exec corgi-api curl http://localhost:5000/health

Troubleshooting

Service Won't Start

Check logs for specific error messages:

docker-compose logs corgi-api | grep ERROR

Common issues: - Database connection failed: Check POSTGRES_PASSWORD matches in .env - Port already in use: Change CORGI_API_HOST_PORT in .env - Permission denied: Ensure proper file permissions on SSL certificates

Database Connection Issues

Test database connectivity:

docker-compose exec corgi-api python -c "
from db.connection import get_db_connection
conn = get_db_connection()
print('Database connected successfully!')
"

Redis Connection Issues

Test Redis connectivity:

docker-compose exec redis redis-cli ping

High Memory Usage

Adjust worker processes in the Dockerfile:

# Reduce workers if memory constrained
CMD ["gunicorn", "--workers", "2", ...]

Security Checklist

Before going live, ensure:

  • [ ] All default passwords changed
  • [ ] API_SECRET_KEY and JWT_SECRET_KEY are unique and strong
  • [ ] USER_HASH_SALT is set
  • [ ] DEBUG is set to false
  • [ ] CORGI_ENV is set to production
  • [ ] CORS origins are properly configured
  • [ ] SSL certificates are installed (if using HTTPS)
  • [ ] Database backups are configured
  • [ ] Monitoring is in place
  • [ ] Rate limiting is enabled
  • [ ] Firewall rules restrict database/Redis ports

Next Steps

After successful deployment:

  1. Set up monitoring: See Monitoring and Observability
  2. Configure backups: Implement automated backup procedures
  3. Set up alerts: Configure Slack or email notifications
  4. Load testing: Verify performance under expected load
  5. Security audit: Run security scanning tools

Your Corgi Recommender Service is now running in production! Monitor the logs during the first few days to ensure stable operation.