Release Process¶

This documents how we version, build, and deploy Mailyte releases.

Branches and Pipelines¶

• develop is the working branch — CI (.github/workflows/ci.yml) runs on every push
• main is the release branch — PRs into main run CI, and production deploys must be launched from main (the Deploy workflow enforces this)
• Deploys are driven by .github/workflows/deploy.yml, a manually-triggered (workflow_dispatch) pipeline

Versioning¶

We use Semantic Versioning: MAJOR.MINOR.PATCH, tracked in pyproject.toml ([project] version).

Deployed images are additionally tagged per run by the Deploy workflow as {environment}-{short-sha}-{timestamp} — that tag, not the semver, is what identifies exactly what's running and what you roll back to.

Release Checklist¶

• All CI gates green on the release commit (lint/type baselines, tests, coverage ratchet, OpenAPI contract, Alembic validation)
• CHANGELOG.md updated
• Version bumped in pyproject.toml
• Database migrations tested up and down (CI's validate-migrations job proves fresh-install, idempotency, and upgrade-with-data)
• Docker images build (CI's build matrix covers every service)
• Documentation updated for new features
• Security scan results reviewed (Bandit / pip-audit job)

Step 1: Prepare the Release¶

Update the Changelog¶

Add a section for the new version at the top of CHANGELOG.md:

## [1.2.0] - 2026-08-30

### Added
- Mailbox export API endpoint (#123)

### Fixed
- Webhook delivery retry logic not respecting backoff (#125)

### Changed
- Upgraded Postfix base image to 3.8

Bump the Version¶

# pyproject.toml
[project]
version = "1.2.0"

Commit and Merge to main¶

git add CHANGELOG.md pyproject.toml
git commit -m "chore: prepare release 1.2.0"

Open a PR from develop into main and merge once CI is green.

Step 2: Tag the Release¶

VERSION="1.2.0"
git checkout main && git pull
git tag -a "v${VERSION}" -m "Release ${VERSION}"
git push origin "v${VERSION}"

gh release create "v${VERSION}" --title "v${VERSION}" --generate-notes

Step 3: Run the Deploy Workflow¶

Deploys are one workflow run — building, pushing, and rolling out are not manual steps:

gh workflow run deploy.yml \
  -f environment=production \
  -f services=all          # or a comma-separated subset, e.g. "api,tracking"

Optional inputs: skip_backup=true (skip the pre-deploy backup), dry_run=true (build images only, deploy nothing).

What the workflow does, in order:

1. Preflight — validates compose config, generates the image tag, and refuses a production deploy that isn't launched from main
2. CI verification — re-runs the full CI workflow
3. Pre-deployment backup (production only) — SSHes to the server and runs scripts/backup.sh --full
4. Build & push — builds each requested service image and pushes it to Amazon ECR as mailyte-{service}:{tag} and :latest
5. Deploy — SSHes to the target, pulls the new images, and rolls services one at a time with docker compose up -d --no-deps, then waits and reports any unhealthy containers
6. Verify & notify — polls the API /health endpoint and posts the result to Slack

Database Migrations During Deploys¶

Schema is owned by the migrate compose service: it runs alembic upgrade head (via scripts/run_migrations.py) once per docker compose up, before dependent services start — a failed migration stops the deploy rather than letting a half-migrated API come up.

Verify Manually¶

# On the server (or via the published ports)
docker compose ps
curl -s http://localhost:8083/health

# Spot-check worker health endpoints (host-mapped ports)
for port in 8081 8082 8083 8085 8086 8087 8088; do
  status=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:$port/health)
  echo "Port $port: $status"
done

Rollback¶

Every deploy's images stay in ECR under their immutable {environment}-{sha}-{timestamp} tag. To roll back:

1. Find the previous run's tag (in the Deploy workflow logs or ECR)
2. On the server: pull that tag for the affected services, re-tag it as what compose expects, and docker compose up -d --no-deps <service> — the same mechanics as step 5 above
3. If a migration must be undone:

python manage.py migrate:rollback            # last migration
python manage.py migrate:rollback --steps=2  # or N

Only roll back migrations that are genuinely reversible — CI proves downgrade() works with seeded data, but data written by the new code into new columns is still lost by a downgrade.

Hotfix Process¶

For urgent fixes in production:

1. Branch from the release tag: git checkout -b hotfix/1.2.1 v1.2.0
2. Fix the issue and write a test
3. Bump the patch version and update the changelog
4. PR into main, tag, and run the Deploy workflow with only the affected services
5. Merge the fix back to develop so it isn't lost