ARIA

Prompt instructions alone are not a control. If the FinOps API is down and the model still produces “$1.2M”, no amount of “please don’t hallucinate” in the system prompt saved you. So the guard exists at three independent levels:

• Prompt — a Data Integrity section of eight absolute rules that override every other instruction: every number traces to a tool result, empty results are stated as empty, uncertainty is disclosed, no confidence bluffing.
• Code — after the final response, the agent scans for currency patterns. If the answer contains dollar figures but every tool errored (or no tool ran at all), the response is discarded and replaced with the specific tool errors.
• Tool — each failure returns a structured payload whose instruction field reads: “STOP. Tell the user this data source is unavailable. Do NOT fabricate numbers.” The model sees the guardrail inside its own context.

The detector deliberately exempts greetings and off-topic refusals so a canned response mentioning a dollar amount doesn’t trip the wire. The trade-off: a legitimate answer could in principle be blocked. I’d take that trade every time — a false “I can’t reach the data” costs a retry, a false number costs the project.

Early on, spend questions returned raw records and the model summed them. It was right most of the time — which is the worst possible failure mode, because “mostly right” is indistinguishable from “right” until someone checks.

Every financial tool now returns a summary object computed server-side: actual total, budget total, forecast, delta, COGS and non-COGS splits. The system prompt instructs the model to read summary.actual_total and never to add up rows itself. Aggregation happens in Python, where it is deterministic and testable; the model’s job is narrowed to selecting the right tool and explaining the result.

The same principle drives the grading and variance tools — scorecard weights, budget-adherence bands, and coefficient-of-variation stability scores are all calculated in code, not reasoned about in prose.

The first version answered everything with a wall of analysis. Ask what one subscription spent and you got a monthly breakdown, a trend commentary, and three caveats. Executives stopped reading at line two.

The critical rule: the tier is set by the user’s words, not by how much data came back. A tool returning 40 rows for a one-entity question means the agent aggregates to a single total and stays at Tier 1. Volume of available data is not a reason to spend the reader’s attention.

ARIA calls Claude through Vertex AI rather than the direct API. That single choice removed a whole category of operational risk:

• No API key exists — the pod authenticates as a GCP service account through Workload Identity. There is no secret to rotate, leak, or accidentally commit.
• Least privilege — a custom IAM role granting only aiplatform.endpoints.predict. The agent can call the model and do nothing else in the project.
• Data residency and governance — inference runs inside the organization’s own GCP project and region, which is what made this approvable for corporate financial data.
• Local development — the same code path works with application default credentials, so there is no “dev uses keys, prod uses identity” divergence.

Trade-off accepted: Vertex model identifiers lag the direct API slightly, and some newer features arrive later. For a corporate deployment handling financial data, credential elimination was worth more than being first to a model release.

Two model features doing opposite things to the bill, deliberately paired:

• Extended thinking (10,000 token budget) makes the model reason through tool results before writing. It measurably reduced the “grabbed the wrong field from the JSON” class of error. Enabling it forces temperature to 1 and requires the output ceiling to cover both thinking and the answer — a detail that silently truncates responses if you miss it. The budget is configurable, so thinking can be switched off entirely without a code change.
• Ephemeral prompt caching on the ~280-line system prompt. It is identical on every request, so paying full input price for it each time is pure waste. Cache reads are tracked per request in the audit log alongside token counts, so the savings are measurable rather than assumed.

Net effect: better reasoning on the part that varies, near-free on the part that doesn’t.

• Concurrent execution — when the model requests four tools in one round, all four run at once. This is only safe because every tool is read-only and independent; there is no ordering constraint to violate.
• Bounded rounds — a hard ceiling of 10 tool rounds. A confused agent burns budget, not the afternoon.
• Server-sent events — tool calls stream to the UI as they happen, so a 20-second answer shows “Fetching: get_budget_variance” instead of a spinner. Perceived latency collapses when users can see the work.
• 25-second heartbeat — the service mesh proxy closes idle connections. A background task emits a keepalive event so long multi-round answers survive the trip. This was found in production, not in design.
• Connection pooling with retries — upstream calls use a shared async client with keepalive connections, two retries on timeout or connection error, and no retry on HTTP errors (a 404 will still be a 404).

ARIA runs as a single pod behind an SSO proxy on an internal network. That context justified two deliberately unfashionable choices:

• SQLite over PostgreSQL — WAL mode gives concurrent reads during writes and a 30-second busy timeout absorbs contention. The honest cost is that it pins the app to one replica; if this ever needs horizontal scaling, the database is the migration. For the actual usage pattern, running a database server would have been infrastructure for its own sake.
• No login — the platform already enforces Okta SSO at the edge, so a second auth layer would add friction without adding security. Conversation IDs live in browser local storage, so each person sees their own history; user identity is read from proxy headers for the audit log only.

Both are reversible decisions documented as such — which is the point. Choosing the smaller thing is only defensible when you know exactly what would force you to change it.

ARIA answers cloud cost questions. Ask it to write code, explain the weather, or opine on security posture and it declines. That constraint is load-bearing: a tool that will attempt anything is a tool nobody can trust on the thing it’s actually for.

• Clearly off-topic questions get a rotating refusal drawn from a pool of eleven — movie quotes and Victorian-era demurrals, cycled per conversation so the same user never sees a repeat until they’ve seen them all.
• Adjacent questions — a real FinOps topic ARIA has no data for — get a redirect explaining what it can answer, not a joke.
• The refusal path is handled server-side: the model emits a marker, the API swaps in the canned text and increments the rotation counter.

The humor was not decoration. A blunt “I cannot answer that” reads as broken; a wink reads as designed, and it made people comfortable testing the edges — which is exactly the behavior you want during rollout.