★ Enterprise Edition — not included in the Community Edition. Compare editions

Prometheus Metrics¶

Every worker service exposes a GET /metrics endpoint in Prometheus exposition format, generated by the shared builder in shared/metrics.py.

Metric names are prefixed with the service name

shared/metrics.py prepends the emitting service's name to every metric: the API's request counter is api_http_requests_total, the webhooks service's is webhooks_http_requests_total, and so on. There is no mailyte_ prefix anywhere. Queries written against unprefixed names silently match nothing — which for an alert means it never fires.

Common Metrics (every service)¶

For each service <svc> (see the prefix table below):

Metric	Type	Labels	Description
`<svc>_info`	Gauge	`version`, `service`	Constant `1`; carries version info
`<svc>_uptime_seconds`	Counter	—	Seconds since process start
`<svc>_cpu_usage_percent`	Gauge	—	Process-host CPU usage (psutil)
`<svc>_memory_usage_percent`	Gauge	—	Host memory usage percent
`<svc>_memory_usage_bytes`	Gauge	—	Host memory used, bytes
`<svc>_http_requests_total`	Counter	`method`, `path`, `status_code`	All HTTP requests (middleware)
`<svc>_http_errors_total`	Counter	`method`, `path`, `status_code`	Requests with status ≥ 400
`<svc>_http_request_duration_seconds`	Summary	`method`, `path`	Emitted as `_count`, `_sum`, and `quantile="0.5" / "0.95" / "0.99"` series (last 1000 observations)

Durations are summaries with pre-computed quantiles, not histograms — there are no _bucket series, so histogram_quantile() does not apply. Use the quantile label directly.

Service Prefixes and Scrape Targets¶

Service	Metric prefix	In-network target (container port)	Host port
api	`api_`	`api:8080`	8083
webhooks	`webhooks_`	`webhooks:8081`	8081
rate_limiter	`rate_limiter_`	`rate_limiter:8082`	8082
archiver	`archiver_`	`archiver:8083`	8089
encryption	`encryption_`	`encryption:8084`	8093
monitoring	`monitoring_`	`monitoring:8085`	8085
analytics	`analytics_`	`analytics:8085` (container)	8087
tracking	`tracking_`	`tracking:8086`	8086
dashboard	`dashboard_`	`dashboard:8088`	8088
delivery_optimizer	`delivery_optimizer_`	`delivery_optimizer:8088`	8094
queue_manager	`queue_manager_`	`queue_manager:8090`	8090
rag	`rag_`	`rag:8090` (container)	8091
url_protection	`url_protection_`	`url_protection:8090`	8096
oauth	`oauth_`	`oauth:8091`	8097
storage_usage	`storage_` (registers as `storage`)	`storage_usage:8092`	8092
jmap	`jmap_`	`jmap:8098`	8098
migration	`migration_`	`migration:8099`	8099
autoconfig	`autoconfig_`	`autoconfig:8100`	8100
caldav	`caldav_`	`caldav:8101`	8101

Two scrape-target bugs worth knowing

The shipped monitoring/prometheus/prometheus.yml targets analytics:8087 and rag:8091 — those are host ports; the containers actually listen on 8085 and 8090, so those two jobs scrape nothing until the config is corrected.

Backup & DR Metrics (monitoring service)¶

Published by worker/monitoring/services/backup_monitor.py from the backup_history table. All gauges, labels host and backup_type (full / incremental):

Metric	Description
`monitoring_backup_age_seconds`	Seconds since the newest completed backup; sentinel `315360000` (10 years) when a host has never backed up
`monitoring_backup_last_status`	`1` if the latest run completed, else `0`
`monitoring_backup_last_size_bytes`	Size of the latest completed backup
`monitoring_backup_unencrypted_runs`	Completed runs written without age encryption

Expected hosts come from BACKUP_EXPECTED_HOSTS. The alert rules in monitoring/prometheus/rules/backup_alerts.yml are written against exactly these series.

Archiver Metrics¶

Metric	Type	Labels	Description
`archiver_archive_messages_total`	Counter	`storage`	Messages archived, by storage backend (`s3` / `spool`)
`archiver_archive_store_failures_total`	Counter	—	`/archive` requests that neither S3 nor the spool accepted
`archiver_archive_spool_depth`	Gauge	—	Archived messages sitting in the local spool, not yet in S3

Monitoring Aggregation Endpoint¶

The monitoring service additionally serves GET /metrics/prometheus, a hand-built aggregation that inlines every scraped service's /metrics body and adds (without # TYPE lines):

monitoring_service_up, monitoring_service_services_monitored, monitoring_system_cpu_usage, monitoring_system_memory_usage, monitoring_system_disk_usage, monitoring_mail_active_domains, monitoring_mail_total_users, monitoring_mail_emails_today, and <service>_service_up 0 for each unreachable service.

Prefer scraping each service directly (the shipped Prometheus config does).

Infrastructure Metrics (via Exporters)¶

Only two exporters ship in docker-compose.yml:

MySQL Exporter (`mysql-exporter:9104`, `prom/mysqld-exporter`)¶

Metric	Description
`mysql_global_status_threads_connected`	Active connections
`mysql_global_variables_max_connections`	Max connections
`mysql_global_status_slow_queries`	Slow query count
`mysql_global_status_innodb_buffer_pool_read_requests`	Buffer pool reads
`mysql_global_status_innodb_buffer_pool_reads`	Buffer pool misses

The replica-status collector is disabled (--no-collect.slave_status) — there is no replica in this stack.

Redis Exporter (`redis-exporter:9121`, `oliver006/redis_exporter`)¶

Metric	Description
`redis_memory_used_bytes`	Redis memory usage
`redis_memory_max_bytes`	Redis max memory
`redis_connected_clients`	Connected clients
`redis_keyspace_hits_total` / `redis_keyspace_misses_total`	Cache hits / misses
`redis_commands_processed_total`	Total commands

Declared but not deployed

The Prometheus config also declares postfix (postfix-exporter:9154) and dovecot (dovecot-exporter:9166) jobs, but no such exporter containers exist in any compose file — both targets show as DOWN. There is also no node-exporter, so node_* series (referenced by two disk-space alert rules) don't exist. Postfix visibility comes from the log ingestor writing mail_logs instead. Rspamd's controller (rspamd:11334) does serve /metrics natively.

Useful PromQL Queries¶

PromQL

# API request rate
rate(api_http_requests_total[5m])

# API error ratio
rate(api_http_errors_total[5m]) / rate(api_http_requests_total[5m])

# API p95 latency (summary quantile — NOT histogram_quantile)
api_http_request_duration_seconds{quantile="0.95"}

# Hours since the last full backup, per host
monitoring_backup_age_seconds{backup_type="full"} / 3600

# Archive spool draining?
archiver_archive_spool_depth

# MySQL connections used %
mysql_global_status_threads_connected / mysql_global_variables_max_connections

# Redis hit rate
rate(redis_keyspace_hits_total[5m])
  / (rate(redis_keyspace_hits_total[5m]) + rate(redis_keyspace_misses_total[5m]))