Webhooks Worker¶

The webhooks worker is the event notification system for Mailyte. Whenever something interesting happens (email sent, delivered, bounced, opened, clicked, rate limit hit, cert renewed, etc.), this service dispatches the event to all registered webhook endpoints. Think of it as a pub/sub fanout that bridges the internal mail system to the outside world.

What It Does¶

• Receives events from all services via Redis pub/sub and a direct HTTP API
• Dispatches events to registered webhook URLs with HMAC signing
• Retry with exponential backoff for failed deliveries
• Queues events in-memory (up to 10,000) to absorb traffic spikes
• Processes events with a pool of worker threads (configurable)
• Tracks delivery metrics and exposes Prometheus endpoints

How It Works¶

flowchart LR
    subgraph Sources["Event Sources"]
        PF["Postfix\n(webhook_sender.py)"]
        TK["Tracking Worker"]
        RL["Rate Limiter"]
        CM["Cert Manager"]
        MON["Monitoring"]
    end

    subgraph Webhooks["Webhooks Worker :8081"]
        RedisSub["Redis\nSubscriber"]
        API["HTTP API"]
        Queue["Event Queue\n(10K max)"]
        Workers["Worker Threads\n(5 default)"]
        Sender["Notification\nSender"]
    end

    PF -->|"Redis pub/sub"| RedisSub
    TK -->|"Redis pub/sub"| RedisSub
    RL -->|"Redis pub/sub"| RedisSub
    CM -->|"HTTP"| API
    MON -->|"HTTP"| API

    RedisSub --> Queue
    API --> Queue
    Queue --> Workers
    Workers --> Sender

    Sender -->|"HMAC-signed HTTP POST"| EP1["Endpoint A"]
    Sender -->|"HMAC-signed HTTP POST"| EP2["Endpoint B"]
    Sender -->|"HMAC-signed HTTP POST"| EP3["Endpoint N"]

    MySQL[(MySQL)] -.->|"endpoint config"| Sender

Event Types¶

Webhook Payload Format¶

Every webhook delivery includes:

{
  "event": "email.delivered",
  "timestamp": "2025-01-15T10:30:00Z",
  "data": {
    "message_id": "abc123@mail.example.com",
    "from": "sender@example.com",
    "to": "recipient@example.com",
    "subject": "Hello World",
    "domain": "example.com",
    "organization_id": "org-uuid"
  }
}

HMAC Signing¶

Every payload is signed with HMAC-SHA256. The signature is in the X-Webhook-Signature header:

X-Webhook-Signature: sha256=a1b2c3d4e5f6...
X-Webhook-Timestamp: 1700000000
X-Webhook-Source: mailyte-enterprise

To verify on the receiving end:

import hmac, hashlib

expected = hmac.new(webhook_secret.encode(), payload_json.encode(), hashlib.sha256).hexdigest()

assert request.headers["X-Webhook-Signature"] == f"sha256={expected}"

Retry Logic¶

Failed deliveries are retried with exponential backoff:

After 5 failed attempts, the event is dropped and a warning is logged. The endpoint's failure count is tracked; after enough consecutive failures, the endpoint can be auto-disabled.

API Endpoints¶

POST /webhook/event     -- Submit an event for dispatch
GET  /webhook/status    -- Queue size, worker stats, delivery metrics
GET  /health            -- Health check
GET  /metrics           -- Prometheus metrics

Configuration¶

Database Tables¶

Docker Configuration¶

webhooks:
  build: ./worker/webhooks
  container_name: webhooks
  ports:
    - "8081:8081"
  depends_on:
    - mysql
    - redis

Gotchas¶