Cost, latency, and tradeoffs
What the assistant costs to run, how fast it was observed to be, and the choices made to keep both small.
Text checked against the code and documentation on .
Cost
- Inference runs on Cloudflare Workers AI. Cloudflare documented a shared daily free allocation of 10,000 Neurons on 2026-10-09. Provider allowances change, and hosted inference stays metered and finite.
- Manual requests used about 2,700 to 4,100 tokens each, across about 3 model steps. Whether the default daily budget of 200 requests fits inside the free allocation has not been measured. Do not read the free allocation as a promise of free operation.
- The app caps cost in code: 5 requests per visitor per hour, one request at a time, a daily request budget, 3 steps, and 1,024 output tokens. When the budget is spent the route fails closed and the panel offers static links.
- No vector database, no stored chat history, and no feature-specific database. These are cost choices as much as design choices.
Latency
- Manual requests took about 2.6 to 6.1 seconds.
- One request stalled until the 25 second timeout. The server then sent its own fixed reply, and Agent Studio showed the stopped model step instead of a success. The cause is provider or timeout behavior and is not explained.
- Reasoning is switched off for the model. It used output tokens, added latency, and spent the free allocation, and lookups over a small corpus do not need it.
- A request makes about 3 model steps: choose a tool, optionally ask for a card, then write the answer. Each step is a provider call, so latency grows with steps. Retrying a quota error does not help, so the server does not retry; the visitor can press Retry.
Accessibility
- The portfolio works without the assistant. The assistant's instructions ask for a short text answer with every card, so the card is never the only carrier of the answer. Code does not enforce that.
- Messages are plain text. The activity summary is announced once, when the answer ends, not on every streamed token. The global reduced-motion rule applies, and the loading spinner stops animating under it.
- The core keyboard and screen-reader flows of the assistant were checked by the site owner when the cards shipped. The Agent Studio screen-reader and reduced-motion checks are still open.
- No WCAG audit has been run, so the site makes no conformance claim.
Operations
- Telemetry is one JSON line per event, written to the host's logs, with no prompt or answer text. Retention and access follow the hosting plan and have not been confirmed; no telemetry beyond these logs is enabled.
- The rate-limit guard is in memory and per instance. Moving it to a shared store is a launch task.
- Policy: every escaped factual or safety defect gets a frozen evaluation case, so the fix is checked on every later change.
Tradeoffs made on purpose
| Choice | Gain | Cost |
|---|---|---|
| Server-enforced grounding instead of trusting the prompt | General-knowledge answers cannot reach a visitor | A weak model that skips tools gets a generic reply instead of a partial answer |
| Plain text only, no Markdown | No model-written markup or links | Answers look plain |
| Deterministic search, no embeddings | Cheap, inspectable, and repeatable | Misses paraphrases that share no words with a record |
| Stateless chat | Nothing to store, retain, or delete | No history across reloads |
| Scripted evaluation fixtures | Free, fast, and repeatable on every change | They test the server, not the model, so they cannot show model quality |