Architecture
code
Client (Continue / app) --Bearer client-key--> /api/chat (Vercel function, Web Request/Response)
1. authenticate SHA-256(key) lookup in CLIENTS_JSON (src/ledger.ts)
2. gate budget (402) + sliding-window RPM (429) (src/ledger.ts)
3. sanitize req strip unsupported fields (min_p) (src/sanitize.ts)
4. forward upstream /chat/completions with upstream key, stream_options.include_usage
5. repair stream TransformStream: drop empty/reasoning-only chunks, strip mid-stream usage
6. bill on flush: tokens × PRICES_JSON → ledgerPrinciples: pure functions for logic, Web-standard APIs (portable to Vercel Edge/Node, Cloudflare), no runtime dependencies, secrets only in env, clients stored as hashes.
Known gaps before charging real money
- Persistence: the ledger is in-memory (per serverless instance). Replace
Ledgerwith Vercel KV/Upstash Redis (atomic INCRBYFLOAT + monthly key). Interface is already isolated. - Pre-flight cost estimate: budget is checked before, billed after; a single huge request can overshoot. Add max_tokens cap per client.
- Dashboard:
public/index.htmluses mock data; wire to a/api/usageread endpoint. - Admin: key issuing is manual (hash a key, paste in
CLIENTS_JSON). Add a small admin UI later (reuse patterns from oldweb-platformNext.js admin). - Streaming client abort: billing on abort relies on upstream usage; add fallback token estimation.
- Verify upstream behavior on
stream_optionsper provider before shipping.