Skip to content

TLS certificates

How HTTPS is delivered for every public hostname. Where private keys live, how renewal happens, and where it can break.

Cross-references: domains.md for CAA and per-domain detail; proxies.md for which proxy issues which cert.

Strategy at a glance

Both Caddy instances issue TLS certs automatically via Let's Encrypt (Caddy default, ACME). Cert state lives on the host filesystem of E1 / O1 — same as the Caddyfile, with the same off-host risk.

Domain Cert pattern DNS-01 / HTTP-01 Issuing host
*.448.global Wildcard cert DNS-01 (requires DNS API token) [CONFIRM] Caddy on O1
*.projecteidos.com Wildcard [CONFIRM] (Vishnu: "more or less yes") DNS-01 if wildcard [CONFIRM] Caddy on E1
*.eidos-global.com Per-host (single SAN) [CONFIRM] HTTP-01 likely Caddy on E1
*.tneconnect.app Per-host [CONFIRM] HTTP-01 likely Caddy on E1

If *.projecteidos.com is wildcarded, then a DNS API token at GoDaddy lives somewhere with permission to write _acme-challenge.*.projecteidos.com records. That token is Tier-0.

Renewal monitoring

Item Status
Caddy auto-renewal (every ~60 days, before 90-day expiry) yes — relied on
External cert-expiry alerting none
Beszel cert-expiry alerts not configured
Caddy-side renewal-failure alerts [INFO NEEDED] (Caddy logs are on host only)
DNS API token rotation cadence [INFO NEEDED]
ACME account email [INFO NEEDED] (whatever Caddy was configured with — must be a real inbox someone watches)

Risk: the Caddyfile-not-in-Git incident also implies the Caddy ACME state can be lost (/data/caddy/...). After a rebuild, Caddy may need to re-issue every cert — Let's Encrypt rate-limits at 50 certs per registered domain per week. With ~30 hostnames split across 4 domains, a clean re-issue is feasible but tight.

DNS API tokens for DNS-01 / wildcard renewal

Wildcard certs require Caddy to write TXT _acme-challenge records into the domain. Caddy needs a credential for the DNS provider (GoDaddy in our case).

Domain DNS provider Wildcard? API token Vault path Last rotated
448.global GoDaddy [CONFIRM] yes [INFO NEEDED] [INFO NEEDED]
projecteidos.com GoDaddy likely yes [INFO NEEDED] [INFO NEEDED]
eidos-global.com GoDaddy likely no (per-host) n/a if per-host n/a
tneconnect.app GoDaddy likely no (per-host) n/a if per-host n/a

Treat any DNS API token as Tier-0: with it, an attacker can issue a valid cert for any host on the domain.

CAA records

CAA records on each domain limit which CAs can issue certs and reduce the blast radius of any future CA compromise or token leak.

Domain CAA configured? Recommended
projecteidos.com [INFO NEEDED] (please verify in GoDaddy DNS) letsencrypt.org
eidos-global.com [INFO NEEDED] letsencrypt.org
tneconnect.app [INFO NEEDED] letsencrypt.org
448.global [INFO NEEDED] letsencrypt.org

Cipher / TLS-version policy

Item Value
Minimum TLS version Caddy default = TLS 1.2 (auto-includes TLS 1.3) [CONFIRM]
HSTS enabled [INFO NEEDED] (Caddy header Strict-Transport-Security)
HSTS preload submitted [INFO NEEDED] (tneconnect.app already preloaded as part of .app TLD)
OCSP stapling Caddy enables by default [CONFIRM]
TLS-related Caddy directives in source control no — same risk as the Caddyfile itself

Recent incident context

The recent outage where "all internal apps and client apps broke" was triggered by OCI scheduled VPS maintenance — Oracle paused/rebuilt the VPS as part of routine ops. Because the Caddyfile and Caddy cert-state were on the host filesystem (and not in Git or persisted via durable volumes managed correctly), they were lost. See KI-001.

Risks

  • No cert-expiry alerting. If Caddy auto-renewal silently fails (network blip during DNS-01, expired DNS API token, ACME rate limit), the team learns about it from users.
  • Wildcard cert + leaked DNS API token = attacker can issue valid certs for the entire domain.
  • No CAA records = any CA accepting an attack-controlled validation can issue.
  • No off-host backup of Caddy's data dir = lose the host, lose the certs (and account), trigger rate-limit risk on re-issue.
  • .app TLD HSTS preload on tneconnect.app means a cert lapse is catastrophic — browsers refuse to connect with no fallback prompt.

Phase-2 actions

  1. Add cert-expiry alerts (Beszel custom check or external uptime tool checking <host>:443 cert NotAfter).
  2. Move Caddyfile + Caddy data dir into source control / durable volume / pre-rebuild script (overlaps with KI-001).
  3. Add CAA records on all 4 domains.
  4. Get the GoDaddy DNS API tokens into Vault with documented rotation.
  5. Confirm HSTS + headers on all hostnames.
  6. Document the "Caddy rebuild after host loss" runbook including ACME state and rate-limit considerations.