Production Readiness Review

System: Multi-tenant AI agent platform — Go API, Next.js frontend, PostgreSQL Reviewed: 29 July 2026 Method: REVIEW_RUBRIC.md Time: ~15 hours (re-run after rubric revision)

Worked example. This review was run against our own codebase, which is why the report can be published in full. A client engagement follows the same structure, but its evidence, failure scenarios, and priorities come from that system, under NDA.

Since-publication addendum: findings 1–5 are fixed, and Stripe billing has been removed entirely along with the subscription product it served — so the billing_webhook.go praised under "Done well" no longer exists.

One correction of fact. This report cites EU AI Act high-risk obligations as applying from 2 August 2026. Under the enacted AI Act, Annex III high-risk obligations apply from 2 August 2026. A deferral to 2 December 2027 was politically agreed in May 2026 and is expected to be adopted, but is not yet in force. Readers should check the current position with their own advisers; Article 50 transparency and Article 4 AI literacy were not deferred. The CRA reporting date of 11 September 2026 stands, but applies to manufacturers of products with digital elements rather than to systems because they affect people.

The report is otherwise left as written, because a review is a record of a system at a date and editing it afterwards would defeat the point.

Executive summary

The system is well engineered in the places most systems are weak. Tenant isolation, credential encryption, durable workflow recovery, retention and erasure, and JWT verification are all better than typical for a product at this stage, and the Stripe webhook is implemented correctly — including the parts people usually get wrong.

The problems are concentrated at the public edge. Two unauthenticated Slack endpoints can be used to read another organisation's data and execute agents on their behalf. Everything else in this report is secondary to that.

Fix these three first:

1. /api/slack/commands performs no signature verification at all. Anyone who knows a Slack team ID can run agents against that organisation and read the output in the HTTP response. (Critical — half a day)
2. /api/slack/events verifies signatures only when the headers are present. Omitting them skips verification entirely. (Critical — one hour, same fix)
3. The container deployment cannot start. Nothing reads DATABASE_URL, so the documented docker compose up path crash-loops. (High — half a day)

Also note finding 5: the Slack OAuth callback is broken and insecure at the same time. It always fails, so the integration cannot be connected — but the state parameter is never validated, so repairing the parse without also fixing the design would introduce a live OAuth CSRF. Fix both together.

Total remediation for everything below: roughly 5–7 days.

Findings

1. /api/slack/commands has no signature verification — Critical

Evidence: backend/internal/api/handlers/slack.go:204. The handler parses the form and reads team_id directly from the request body. There is no verification of Slack's signing secret anywhere in the function. The route is registered at backend/internal/api/server.go:122 outside the authenticated group.

team_id is used to look up the installation, and orgID is taken from that record, so the request body alone selects which tenant the command runs against.

Failure scenario: An attacker posts a form to /api/slack/commands with team_id set to a known Slack workspace ID and text=run business_operator. handleRun executes the agent for that organisation and returns its output in the HTTP response body. dashboard and briefing subcommands return that organisation's metrics and operating brief the same way. No credentials are required. Slack team IDs are not secrets — they appear in Slack URLs and are routinely shared.

Currently mitigated by there being no live Slack installations, so there is nothing to target today. The vulnerability becomes live the moment the first workspace connects.

Fix: Verify the Slack signature before reading any form field, using the same slack.VerifySignature the events endpoint already calls. Reject when the headers are absent.

2. Slack signature verification fails open in three places — Critical

Evidence: backend/internal/api/handlers/slack.go:152

if signature != "" && timestamp != "" {
    if !slack.VerifySignature(...) { /* reject */ }
}

Verification runs only when both headers are present. Omitting them skips the check and execution continues to processEvent.

Third, and worst, in the verification function itself — backend/internal/slack/types.go:163:

if signingSecret == "" {
    return true
}

An unset signing secret verifies everything. A deployment that forgets SLACK_SIGNING_SECRET — which nothing warns about at startup — accepts every Slack request as authentic, including on the /commands endpoint that executes agents.

Failure scenario: An attacker posts a crafted event payload with no X-Slack-Signature header. Verification is skipped, the payload is unmarshalled, and go h.processEvent(event.Event, event.TeamID) runs against the team ID in the attacker's own body.

Fix: Fail closed on an empty secret and on absent headers, and call the verifier unconditionally at both endpoints.

3. Container deployment cannot start — High

Evidence: backend/internal/config/config.go:65-84 reads seven environment variables: three Stripe, three HubSpot, and the public app URL. grep -rn "DATABASE_URL\|CLERK_SECRET_KEY" backend --include="*.go" returns nothing outside tests.

docker-compose.yml sets AETHER_ENV=production and passes DATABASE_URL, but nothing reads it, so cfg.Database.URL stays empty and startup reaches log.Fatal("production requires a PostgreSQL connection") in cmd/aether/main.go.

Failure scenario: A customer follows the documented quick start. The API container crash-loops. Clerk is unset for the same reason, so authentication would fail even if the database connected.

Production is unaffected because it deploys a binary onto a host whose config.yaml holds real values — which is itself the problem: the system can only be deployed by hand-editing a file on the host.

Note on how this was missed: CI verifies the images build. Building is not running. A pipeline that starts the stack and asserts /api/ready would have caught it.

Fix: One precedence rule — environment over file over default — applied to every setting, plus *_FILE variants for mounted secrets.

4. No rate limiting on public endpoints; unbounded goroutines — High

Evidence: backend/internal/api/server.go applies RateLimitMiddleware to the authenticated /api group only. The six public routes — Slack install, OAuth callback, events, commands, Stripe webhook, HubSpot callback — have none. slack.go:173 spawns go h.processEvent(...) per request with no bound.

Failure scenario: Sustained unauthenticated POSTs to /api/slack/events spawn one goroutine each, with a 30-second timeout apiece, until memory is exhausted. Combined with finding 2, no valid signature is needed.

Fix: Rate-limit public routes and bound event processing with a worker pool.

5. Slack OAuth callback always fails, and never validates state — High

Two defects in one handler. Either alone would be worth reporting; together they are a trap, because fixing the visible one exposes the hidden one.

Correction. An earlier draft of this report stated that this handler returns 400 on every request. That was wrong: the error branch assigns a fallback rather than rejecting. The corrected analysis is below. The parse behaviour was verified; the surrounding branch was not, and it should have been. Recorded here rather than quietly amended.

Defect A — every installation binds to the wrong organisation.

backend/internal/api/handlers/slack.go:85 parsed the state with:

var orgID string
if _, err := fmt.Sscanf(state, "%s|", &orgID); err != nil {
    orgID = "default"
}

%s in Sscanf consumes every non-whitespace character, so it swallows the | separator and the remainder, leaving the format's literal | with nothing to match. Verified directly:

input:  "default|550e8400-e29b-41d4-a716-446655440000"
result: n=1  err="unexpected EOF"

The error is non-nil on every well-formed input, so the fallback always fires and orgID is always "default".

Failure scenario: Two customers connect Slack. Both installations are written against organisation "default". Steward notifications for one customer are delivered to the other's Slack workspace. Multi-tenancy is not enforced for this integration at all.

Defect B — the state is never validated.

The state is generated in Install (line 59) as orgID|uuid and never persisted. The callback simply reads an organisation identifier out of a string the caller supplied. There is no single-use token, no expiry, and no lookup.

Compare the HubSpot flow in the same codebase, which does this correctly: CreateHubSpotOAuthState persists the state with a ten-minute expiry and ConsumeHubSpotOAuthState redeems it exactly once inside a transaction.

Failure scenario: an attacker authorises the Slack app against their own workspace and calls the callback with their own code and any state. Nothing is checked, so the installation completes and their workspace is bound into the shared "default" organisation — receiving its notifications and, combined with findings 1 and 2, able to issue commands against it.

Also: /api/slack/install is registered as a public route (server.go:119) but reads c.GetString("orgId"), which is only populated by the authenticated middleware. It is therefore always empty, and every legitimate install falls back to organisation "default".

Fix: persist the state with an expiry and consume it once, exactly as the HubSpot flow already does; parse with strings.Cut rather than Sscanf; and move /api/slack/install inside the authenticated group so the initiating organisation is real.

6. Two reachable Go dependency vulnerabilities — Medium

Evidence: govulncheck ./...

• GO-2026-5970 — golang.org/x/[email protected], fixed in v0.39.0. Reachable via db.Connect → pgxpool.NewWithConfig.
• GO-2026-5676 — HTTP/3 QPACK memory exhaustion in github.com/quic-go/[email protected], fixed in v0.59.1. Reachable via api.Server.Start.

Both are flagged as reachable from your code, not merely present in the module graph.

Fix: Upgrade both. Add govulncheck to CI so this is caught on every push — this also feeds the CRA vulnerability-handling obligation that begins 11 September 2026.

7. Four JavaScript dependency vulnerabilities — Medium

Evidence: bun audit

• postcss <8.5.10 — one moderate (XSS via unescaped </style>) and two high (arbitrary file read and path traversal via attacker-controlled sourceMappingURL in CSS comments). Reached through next and @tailwindcss/postcss.
• sharp <0.35.0 — high, inheriting four libvips CVEs. Reached through next.

PostCSS runs at build time, so exposure depends on whether untrusted CSS ever enters the build — low here, but it is a supply-chain path. sharp is used by Next.js image optimisation at runtime, which is the more direct concern.

Fix: bun update for the compatible range, and add bun audit to CI alongside govulncheck.

8. /api/metrics is publicly readable — Medium

Evidence: server.go:81, registered outside the authenticated group. Exposes total requests, total errors, and in-flight count.

Failure scenario: An unauthenticated observer polls the endpoint to infer traffic volume, error rates, and deployment timing. Low direct impact, but it is free reconnaissance and metrics endpoints are conventionally restricted.

Fix: Bind to an internal interface, or require a token.

9. No encryption key rotation path — Medium

Evidence: AETHER_ENCRYPTION_KEY is used to seal integration credentials (internal/secretbox). There is no re-encryption routine.

Failure scenario: The key is suspected compromised. There is no supported way to rotate it without invalidating every stored credential and requiring every customer to reconnect every integration. For a deployment intended to run for years, and for any security questionnaire, this will be asked about.

Fix: Support a key list — new key for writes, old keys retained for reads — plus a re-encryption command.

10. Documentation contradicts the code — Low

Three verified examples:

• apps/web/AGENTS.md:21 states the build is a static export. It is output: 'standalone' with dynamic rendering.
• SPEC.md:12 lists backend/mcp_servers.yaml as configuration. The file does not exist.
• apps/web/AGENTS.md:15 documents navigation links to /features and /pricing. Neither route exists.

Why it matters: Documentation that is wrong in checkable ways is a reliable indicator that other, less checkable claims have also drifted. It also actively misleads anyone onboarding.

11. Production host hardcoded; deploys run as root — Low

Evidence: .github/workflows/ci.yml:117,129 default DEPLOY_HOST to a literal IP address and DEPLOY_USER to root.

The workflow now supports overriding both, and the migration to a non-root account is documented, but the default remains root. Worth completing before customer data is present.

12. Published terms describe a different business — Low

apps/web/src/app/terms/page.tsx sets out SaaS subscription terms: monthly billing in advance, automatic renewal, free-trial conversion. The operator is moving to fixed-scope engagements, which need a master services agreement with statements of work. The privacy policy likewise describes processing customer CRM data through a platform.

Out of scope for engineering, in scope for anything that gets read during procurement.

Done well — do not regress these

Stated because knowing what is load-bearing is as useful as knowing what is broken.

• Stripe webhook verification (billing_webhook.go:82-111) — HMAC-SHA256, timestamp tolerance enforced in both directions, and hmac.Equal for constant-time comparison. Correct, including the parts commonly missed.
• JWT verification (auth/clerk.go) — algorithm pinned to RS256 so algorithm-confusion attacks fail, issuer and expiry validated, JWKS cached under a double-checked lock with a single refresh-and-retry. Fails closed when unconfigured.
• HubSpot OAuth — state is single-use and expiring, consumed transactionally.
• Durable workflow recovery — state persisted around every external boundary; an action interrupted mid-flight becomes needs_review rather than being blindly retried. This is the failure most systems get wrong.
• Tenant isolation — every customer query scoped by organisation, and the erasure path has a test that fails the build if a new table holding organisation data escapes it.
• Least privilege — the HubSpot integration requests only crm.objects.deals.read and crm.objects.tasks.write.
• API contract integrity — every path the frontend calls now resolves to a registered route.

What this review did not cover

• No load or performance testing. Findings about concurrency are from code inspection, not measurement.
• No live penetration testing. No exploit was executed against a running instance; the two Critical findings were confirmed by reading the code and the route registrations.
• Frontend reviewed shallowly — the API contract, auth boundary, build configuration, and dependency audit. No XSS or CSP review of the application code itself.
• Not reviewed in depth: RAG store, security gate, Steward and Guardian meta-agents, briefing and explorer handlers.
• No infrastructure review — the host, network, TLS termination, and database configuration were out of scope.

Suggested next steps

Findings 1–5 are worth doing regardless of what happens to the product.

Note on method

This report is the second pass. After the first, the rubric was revised to make dependency scanning step one and public-surface enumeration its own named step. Re-running found three defects the first pass missed: the JavaScript vulnerabilities, and both halves of finding 5. The checklist earning its keep immediately is the argument for having one.