Skip to content

Worker Modules

Workers are the services that power everything beyond raw email delivery: the API gateway, tracking, webhooks, analytics, rate limiting, storage accounting, AI search, client protocols (JMAP, ActiveSync, CalDAV), auto-configuration, migration, and monitoring. Each worker runs as its own Docker container with a single responsibility. Every worker is a FastAPI application, except ActiveSync (Z-Push on PHP/Apache).

Architecture Overview

flowchart TB
    subgraph Core["Core Workers"]
        API["API Gateway\n:8083→8080"]
        Tracking["Email Tracking\n:8086"]
        Webhooks["Webhooks\n:8081"]
        Analytics["Analytics\n:8087→8085"]
        RateLimit["Rate Limiter\n:8082"]
        QueueMgr["Queue Manager\n:8090"]
        Monitor["Monitoring\n:8085"]
    end

    subgraph Protocols["Protocol Workers"]
        JMAP["JMAP\n:8098"]
        ActiveSync["ActiveSync\n:8084→80"]
        CalDAV["CalDAV\n:8101"]
        Autoconfig["Autoconfig\n:8100"]
        OAuth["OAuth\n:8097→8091"]
    end

    subgraph DataProt["Data & Protection"]
        StorageUsage["Storage Usage\n:8092"]
        Archiver["Archiver\n:8089→8083"]
        Encryption["Encryption\n:8093→8084"]
        URLProt["URL Protection\n:8096→8090"]
        DeliveryOpt["Delivery Optimizer\n:8094→8088"]
        Migration["Migration\n:8099"]
    end

    subgraph Intelligence["Intelligence"]
        RAG["RAG / AI Search\n:8091→8090"]
        Dashboard["Dashboard\n:8088"]
    end

    MySQL[(MySQL)] --- API
    Redis[(Redis)] --- RateLimit
    Qdrant[(Qdrant)] --- RAG

    API --> Analytics
    API --> Tracking
    API --> RateLimit
    API --> QueueMgr
    API --> Monitor

Worker Directory

Every service below is built and enabled in docker-compose.yml. Ports are host → container for the dev compose; in production (docker-compose.prod.yml) internal services are republished on 127.0.0.1 and reached externally only through Traefik.

Worker Port (dev) Container What It Does
API Gateway 8083→8080 api Central REST API; 32 route modules; fronts most other workers. replicas: 2 in prod
Webhooks 8081 webhooks Email event ingest + signed webhook notifications. replicas: 2 in prod
Rate Limiter 8082 rate_limiter Redis-backed rate limiting; consulted by Postfix at DATA phase
ActiveSync 8084→80 activesync Z-Push 2.7.4 (Exchange ActiveSync) over Dovecot IMAP
Monitoring 8085 monitoring Health checks, container restarts via docker-proxy, alerting
Email Tracking 8086 tracking Open/click tracking, suppression list. replicas: 2 in prod
Analytics 8087→8085 analytics Delivery/engagement analytics over mail_logs; scheduled reports
Dashboard 8088 dashboard Server-rendered stats dashboard
Archiver 8089→8083 archiver age-encrypted archival of every message to S3; retention, legal hold
Queue Manager 8090 queue_manager Postfix queue inspection and control via the shared spool
RAG / AI Search 8091→8090 rag Semantic email search with Qdrant vector embeddings
Storage Usage 8092 storage_usage Quota accounting via Dovecot IMAP QUOTA; alerts
Encryption 8093→8084 encryption PGP/S-MIME key management, WKD, message crypto
Delivery Optimizer 8094→8088 delivery_optimizer ISP throttling, IP warming, bounce/FBL processing, reputation
URL Protection 8096→8090 url_protection Safe Links: rewrite + time-of-click verification
OAuth 8097→8091 oauth OAuth 2.0 / XOAUTH2 authorization server
JMAP 8098 jmap JMAP Core + Mail (RFC 8620/8621) over Dovecot
Migration 8099 migration IMAP-to-IMAP mailbox import/export with delta sync
Autoconfig 8100 autoconfig Client auto-setup (Autoconfig/Autodiscover/MTA-STS), publicly routed for all domains
DAV 8105 dav CalDAV/CardDAV server (sabre/dav) for calendar and contacts

Backups are handled by a host-side script and systemd timers, not a worker container -- see that page.

Shared Patterns

Health Endpoints

Every worker exposes:

Text Only
GET /health    → 200 OK if running
GET /metrics   → Prometheus metrics

Shared Modules

Workers import from shared/ at the project root:

Module Purpose
shared.logging_config Structured logging with service-specific loggers
shared.metrics Prometheus metrics collection and exposition
shared.webhook_dispatcher Signed webhook event dispatch over HTTP (reads WEBHOOK_URL, singular)

Services built with the build: ./worker/<name> shorthand don't get shared/ into their image -- the compose file bind-mounts ./shared:/app/shared for them at runtime.

Kafka is provisioned but unused

docker-compose.yml runs Kafka + Zookeeper, and shared/kafka_client.py defines producer/consumer helpers and canonical topic names (mailyte.inbound, mailyte.outbound.*, ...). As of 2026-08-30 no worker or mailer service imports it -- inter-service communication is HTTP plus the webhook dispatcher. Don't document (or debug) message flows through Kafka; there are none.

Database Access

Workers that need MySQL use these environment variables:

Text Only
DB_HOST=mysql
DB_PORT=3306
DB_NAME=mailserver
DB_USER=mailuser
DB_PASSWORD=<your-password>

Workers that need Redis use:

Text Only
REDIS_HOST=redis
REDIS_PORT=6379

The RAG worker connects to Qdrant for vector search:

Text Only
QDRANT_HOST=qdrant
QDRANT_PORT=6333