Webhooks Worker¶

The webhooks worker is the event-ingest and notification service for mail-flow events. It is a FastAPI service that receives raw email events over HTTP (inbound/outbound SMTP, IMAP, POP3), extracts metadata, and dispatches signed webhook notifications to external URLs. It also runs the webhook delivery-log cleanup service.

Event delivery across the platform is built on shared/webhook_dispatcher.py, which every service imports directly -- there is no Redis pub/sub fanout. This worker is one producer among several (tracking, storage_usage, log_ingestor, monitoring, and others all call dispatch_event() themselves); what is unique to this worker is the email-event HTTP ingest and the cleanup APIs.

What It Does¶

• Receives raw email events on /webhook/email/{inbound,outbound,imap,pop3} (multipart or JSON with the raw .eml), parses metadata and attachments
• Dispatches email.inbound / email.outbound / IMAP / POP3 events through shared.webhook_dispatcher with HMAC signing
• Maintains a legacy in-process queue (10,000 max) drained by worker threads, whose notification_sender fans out to per-event-type URLs configured in the webhook_urls MySQL table plus WEBHOOK_URLS / EMAIL_WEBHOOK_URLS / RATE_LIMIT_WEBHOOK_URLS env vars
• Cleans up old delivery logs on a schedule (WEBHOOK_CLEANUP_* settings)
• Prometheus metrics at /metrics

How It Works¶

flowchart LR
    subgraph Sources["Event Producers (all call shared.webhook_dispatcher directly)"]
        TK["Tracking Worker"]
        SU["Storage Usage"]
        LI["Log Ingestor"]
        MON["Monitoring"]
    end

    subgraph Webhooks["Webhooks Worker :8081"]
        API["HTTP ingest\n/webhook/email/*"]
        Queue["Legacy queue (10K)\n+ worker threads"]
        Sender["notification_sender\n(per-event URLs from MySQL)"]
        Cleanup["Cleanup service"]
    end

    API --> Queue --> Sender
    API -->|"dispatch_event()"| Dispatcher["shared.webhook_dispatcher"]
    TK --> Dispatcher
    SU --> Dispatcher
    LI --> Dispatcher
    MON --> Dispatcher
    Dispatcher -->|"HMAC-signed POST"| Target["WEBHOOK_URL target"]
    Sender -->|"HMAC-signed POST"| EP["Configured endpoints"]

Event Types¶

The canonical catalog is the Events class in shared/webhook_dispatcher.py -- roughly 80 event names across these families:

Payload Signing¶

shared.webhook_dispatcher signs every delivery twice:

1. Header: X-Webhook-Signature: sha256=<hex> -- HMAC-SHA256 of the full JSON body, keyed with WEBHOOK_SECRET. Accompanied by X-Webhook-Id, X-Webhook-Event, X-Webhook-Source, and X-Webhook-Timestamp.
2. Inline block for replay protection: payload["signature"] = {"timestamp": <unix>, "token": "<hex nonce>", "signature": "<hmac>"} where signature = hmac(secret, str(timestamp) + token).

Failed deliveries are recorded -- the dispatcher writes webhook_delivery_logs and, on exhausted retries, webhook_dead_letters.

API Endpoints¶

Copied from the route decorators in worker/webhooks/app.py:

POST /webhook/email/inbound    -- Ingest a raw inbound email event
POST /webhook/email/outbound   -- Ingest a raw outbound email event
POST /webhook/email/imap       -- Ingest an IMAP activity event
POST /webhook/email/pop3       -- Ingest a POP3 activity event
GET  /webhook/status           -- Queue size, worker stats
POST /webhook/test             -- Fire a test event through the dispatcher
GET  /cleanup/stats            -- Delivery-log cleanup statistics
POST /cleanup/now              -- Run cleanup immediately
GET  /cleanup/config           -- Current cleanup configuration
PUT  /cleanup/config           -- Update cleanup configuration
GET  /health                   -- Health check
GET  /metrics                  -- Prometheus metrics

worker/webhooks/health_monitor.py contains an additional Flask-style app (/webhook-health/*) that is not started by app.py -- legacy code, not part of the running service.

Configuration¶

Database Tables¶

Docker Configuration¶

webhooks:
  build: ./worker/webhooks
  container_name: webhooks
  ports:
    - "8081:8081"
  extra_hosts:
    - "host.docker.internal:host-gateway"
  depends_on:
    - mysql
    - migrate
    - redis

In production (docker-compose.prod.yml) the service runs with replicas: 2 -- container_name and the host port mapping are reset there, since fixed names and host ports cannot be shared between replicas.

Gotchas¶