Engineering deep diveHow a brand gets measured across AI engines
For engineers: the services, how one scan fans out across AI engines, and how results settle correctly when some calls fail.
Stack
- Backend
- NestJSTypeScriptgRPC
- Jobs
- BullMQRedis
- Data
- PostgreSQL 17
- AI engines
- OpenAIAnthropicPerplexityGeminiGoogle AI ModeAI Overviews
- Frontend
- Next.jsReactTanStack QueryTailwind
- Delivery
- RailwayStripeVitestPlaywright
Five domains behind two gateways
- AuthAccounts, sessions and login.
- BillingStripe subscriptions, credits and usage metering.
- BrandBrands, competitors and the prompts each brand is tracked on.
- ScanRuns prompts across AI engines and records mentions and citations.
- ContentTurns results into share of voice, insights and suggested actions.
- Web and Admin gatewaysEach has its own OpenAPI contract and calls the domains over gRPC.
One scan, from prompt to dashboard
Each prompt and engine pair is its own job, so one slow or failing engine never blocks the rest.
- TriggerA five-minute cadence tick, a manual start or a free scan starts a run.
- Fan-outThe run creates one job for every active prompt times every enabled engine, and records the total.
- WorkerUp to 10 jobs run at once. Each one asks the engine and stores the full answer.
- RetryTemporary errors retry up to 3 times with backoff. Bad API keys fail at once, with no wasted retries.
- ParseBrand and competitor mentions are matched as whole words, and cited URLs are de-duplicated.
- SettleEach finished job adds to a counter in one atomic update. The last one closes the run.
- InsightShare of voice and citation trends are recomputed and shown on the dashboard.
Run states
Running→Completed/Partial/Failed/Cancelled
Cancel and natural finish race on the same conditional update, so only one of them wins.
Key decisions
One job per prompt and engine
- Problem
- AI engines are slow, rate-limited and fail in different ways. One big job per scan meant one bad engine ruined the whole scan.
- Choice
- Fan out into small BullMQ jobs and settle the run with atomic counters: all passed is completed, some failed is partial, all failed is failed.
- Why
- Customers still get results when one engine is down, and the run status always tells the truth.
Domain services with frozen contracts
- Problem
- AI writes a lot of the code. Without hard boundaries, changes leak across features and break callers silently.
- Choice
- Five gRPC domain services behind Web and Admin gateways. Every endpoint has a frozen OpenAPI slice, and wire-shape tests check the gRPC messages.
- Why
- A change that breaks a contract fails in CI, not in front of a customer.
Transactions for every credit and charge
- Problem
- Credits are reserved, used and refunded by many workers at once. A crash halfway could charge twice or lose usage.
- Choice
- PostgreSQL with explicit SQL, unique constraints, transactions and versioned schema migrations.
- Why
- No double charges and no half-written records, and the database itself enforces the rules.
Settling a run, simplified
run-settlement.tssimplified sketch
async onJobFinished(runId: string, ok: boolean) {
// one atomic update per job: no read-modify-write race between workers
const run = await sql`
UPDATE runs SET completed_jobs = completed_jobs + ${ok ? 1 : 0},
failed_jobs = failed_jobs + ${ok ? 0 : 1}
WHERE id = ${runId} RETURNING *`;
if (run.completed_jobs + run.failed_jobs < run.total_jobs) return;
const status = run.failed_jobs === 0 ? 'completed'
: run.completed_jobs === 0 ? 'failed' : 'partial';
// CAS: only close a run that is still running (cancel may have won)
const won = await sql`
UPDATE runs SET status = ${status} WHERE id = ${runId} AND status = 'running'`;
if (won.count === 1) await insights.recompute(run.brand_id);
}
Scale
5domain services over gRPC
6AI answer engines tracked
10jobs in parallel per worker
5minute scheduling tick
3tries per job before it fails
Reliability and operations
- Gates on every changeTypecheck, lint, convention checks, tests, build and contract checks run before anything reaches main.
- Real-stack testsIntegration tests run against real PostgreSQL and Redis. Mocks never count as proof.
- Browser acceptancePlaywright runs real user flows in a browser before a feature is accepted.
- Release readbackEvery deploy exposes its commit and build, so what is live is always known.