Skip to content

Deploy

Purpose

Step-by-step manual deploy of API and admin from a fresh merge to main. The Dify AI stack is deployed separately via docker-compose.dify.yml and is rarely touched. Pass 1 of HUPH does not have CI/CD — deploys are performed by an engineer on the production host.

Prerequisites

  • SSH access to the production host
  • sudo access (for nginx + systemd commands)
  • Clean local working tree on main
  • docker compose available on the host
  • You know which services your PR touched (API? admin? both?)

Before you deploy

  1. Confirm tests pass on main locally:
Bash
npm run test -w apps/api
cd apps/admin && npx tsc --noEmit && npm run build
  1. Confirm merge is on main:
Bash
git checkout main && git pull origin main
git log --oneline -5

  1. Alert the channel if this deploy changes behavior visible to operators (new feature, migration, env var addition).

  2. Take a snapshot of current state for rollback:

Bash
docker ps --format '{{.Names}}\t{{.Status}}' > /tmp/pre-deploy-state.txt
git rev-parse HEAD > /tmp/pre-deploy-sha.txt

Deploy API

Bash
cd /opt/huph
git pull origin main
docker compose build huph-api
docker compose up -d huph-api

Verify:

Bash
docker compose ps huph-api   # should show "Up (healthy)"
curl -s http://localhost:3101/health   # should return {"status":"ok"}
docker compose logs huph-api --tail 30 | grep -i error   # should be empty

Note: apps/api is built into the image at docker build time — docker restart huph-api is INSUFFICIENT after a code change. Always run docker compose build huph-api && up -d.

Deploy admin

The admin runs via systemd (not docker) in production, on port 47293.

Bash
cd /opt/huph
git pull origin main   # if not already done
cd apps/admin
npm install   # if package.json changed
npm run build
sudo systemctl restart huph-admin

Verify:

Bash
sudo systemctl status huph-admin --no-pager | head -20
curl -s http://localhost:47293/ | grep -q "<title>" && echo "OK"

Admin is served via nginx at admin.huph.val.id → 127.0.0.1:47293.

Deploy docs (docs.huph.val.id)

Documentation is a static site built with MkDocs Material. Build locally, nginx serves /opt/huph/site/ directly.

Bash
cd /opt/huph
source docs-env/bin/activate

# Build with strict validation
mkdocs build --strict
# Expected: "Documentation built in X.XX seconds"

# (Optional) inspect the new output
ls site/index.html site/id/index.html site/guide/operators/getting-started/

Nginx does not need to be restarted because the site is served from disk directly — the next request picks up the new files. However, if you have any Cloudflare caching or want to force Cloudflare to re-pull static assets, purge the docs.huph.val.id zone.

First-time deploy requires bootstrap of the Let's Encrypt cert via certbot. The repo vhost file references fullchain.pem / privkey.pem at /etc/letsencrypt/live/docs.huph.val.id/ — those don't exist until certbot issues them.

Bootstrap sequence:

Bash
# 1. Write an HTTP-only stub so certbot webroot challenge can reach nginx
sudo tee /etc/nginx/sites-available/docs.huph.val.id > /dev/null <<'EOF'
server {
    listen 80;
    listen [::]:80;
    server_name docs.huph.val.id;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 503;
    }
}
EOF
sudo ln -sfn /etc/nginx/sites-available/docs.huph.val.id /etc/nginx/sites-enabled/docs.huph.val.id
sudo nginx -t && sudo nginx -s reload

# 2. Request the Let's Encrypt cert
sudo certbot certonly --webroot -w /var/www/certbot -d docs.huph.val.id

# 3. Replace the stub with the full HTTP+HTTPS vhost from the repo
sudo cp /opt/huph/docker/nginx/docs.huph.val.id.conf /etc/nginx/sites-available/docs.huph.val.id
sudo nginx -t && sudo nginx -s reload

# 4. Smoke test
curl -sI https://docs.huph.val.id/ | head -3

Subsequent deploys (after the cert is live) only need steps 3–4. Cert renewals are handled automatically by the certbot.timer systemd unit (or crontab, depending on setup) because the vhost keeps the .well-known/acme-challenge/ location block in the HTTP server.

Nginx reload (if configs changed)

Bash
sudo nginx -t                    # test config first
sudo nginx -s reload              # graceful reload

If you changed docker/nginx/*.conf, copy to system nginx path first (follow the existing deploy.sh pattern or manual copy).

Database migrations

If the PR includes a new migration script under scripts/:

Bash
cd /opt/huph
docker exec -i huph-postgres psql -U huph -d huph < scripts/migrate-<name>.sql

Verify:

Bash
docker exec huph-postgres psql -U huph -d huph -c "\d <table_you_changed>"

Migrations are additive only (no down migrations currently tracked). Plan schema changes carefully — if you break a column, you break forward.

Post-deploy smoke tests

Run these after deploying any service:

Bash
# Health checks
curl -s http://localhost:3101/health
curl -sI http://localhost:47293/ | head -1
curl -s http://localhost:5001/health   # Dify API (only if Dify stack was touched)

# API v1 surface
curl -s http://localhost:3101/api/v1/health/realtime | jq

# Lead capture smoke (if API changed)
# See: docs/dev/operations/lead-capture-smoke.en.md

# Realtime smoke (if realtime changed)
# See: docs/dev/operations/realtime-smoke.en.md

Open the admin dashboard in a browser and:

  1. Log in
  2. Open Conversations — should load without errors
  3. Open Leads v2 — should load with 30s polling
  4. Check the realtime indicator in the header — should show "Connected"

If anything fails, rollback immediately (see rollback.en.md).

Zero-downtime caveats

  • API container restart causes a brief gap (~2–5 s) where requests fail. Acceptable for small deploys; not acceptable during peak WA traffic. Consider time-of-day.
  • Dify container restart causes ~30–60 s where chat dispatch returns 502. The apps/api ragClient retries with backoff but user-facing latency spikes. Avoid touching docker-compose.dify.yml during peak traffic.
  • Admin systemctl restart is roughly seamless (Next.js restarts in <3 s, nginx buffers briefly).
  • No blue-green deploy currently — live restart is the only option.

What to do after a successful deploy

  1. Update the internal deploy log (Slack/wiki/whatever the team uses)
  2. Monitor Phoenix + Langfuse for anomaly spikes for ~30 minutes
  3. Watch the API container logs for unexpected errors:
Bash
docker-compose logs -f huph-api --tail 100
  1. Delete the pre-deploy snapshot files if stable:
Bash
rm /tmp/pre-deploy-state.txt /tmp/pre-deploy-sha.txt

Gotchas

  1. .env changes don't propagate without container restart. The container reads env at boot. After changing .env, do docker-compose up -d --no-deps <service> to recreate with new env.
  2. Systemd admin service name. The unit file is huph-admin.service. If you see Admin or admin in a tab, verify first — wrong name means the restart silently does nothing.
  3. npm run build for admin is required before systemctl restart. Without it, you restart the old build.
  4. Dify stack is not deployed via this flow. Dify lives in a separate docker-compose.dify.yml. Its deploy is manual and rare. Don't accidentally docker-compose down from the main compose file — that only covers the main HUPH stack, not Dify.
  5. deploy.sh is an older helper script. It still works but was written before the current docker-compose workflow. Prefer the manual commands above.

See also