AI costs need product accountability, not blanket caps. Give teams visible budgets, showback, alerts, and a shared optimization playbook. Attribute spend to workflow and successful outcome so a high-quality verifier is not cut while wasteful retries remain invisible.
Set a portfolio budget and per-workflow envelopes. Alert on abnormal changes, but preserve room for approved experiments. Protect safety budgets: a required human gate or evidence check is not waste merely because it adds tokens.
The operating goal is predictable unit economics and fast diagnosis, not the lowest possible provider invoice.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Optimize in this order: remove unnecessary model calls; use deterministic code for rules; reduce retrieved context; cache stable, authorized answers; batch offline work; route easy tasks to cheaper models; cap outputs; then consider fine-tuning or provider changes. Measure every change on quality, safety, latency, and unit economics.
Typical fixes: deduplicate document embeddings by checksum; retrieve six clauses instead of whole leases; use classifier confidence to avoid expensive synthesis; cancel parallel calls after a human gate; replace repeated policy prose with cached context. Never cut authorization, citations, audit traces, or emergency escalation to save money.
Keep an experiment log with hypothesis, baseline, eval result, rollout, and rollback. Cost work without quality gates becomes slow degradation. The best optimization is often deleting an agent step that never improved an outcome.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Alert on rate of change, not only monthly totals: cost per minute, cost per successful task, tokens per request, retry loops, cache miss spikes, unexpected model-route changes, and budget burn rate. Add thresholds by workflow and severity. A runaway tenant-triage retry should page quickly; a slow monthly drift can create a ticket.
Include trace links, top callers, model versions, error classes, and comparison with baseline in the alert. Automatically contain only proven failure modes, such as disabling an experimental route after a hard budget cap. Human review remains necessary when an alert might reflect legitimate volume.
Test alerts with controlled simulations. If operators cannot identify the likely cause in minutes, enrich the telemetry. Alert fatigue is a cost-control failure.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Assign cost centers by product and workflow: Belle Realty ingestion, lease RAG, tenant triage, Deal Leverage diligence, shared platform, and experiments. Record model, embedding, OCR, storage, reranker, cache, and human-review costs. Showback reports usage to owners even before formal chargeback.
Define monthly envelopes plus per-request ceilings for expensive paths. Separate production, staging, and experimentation. Budget alerts should identify the route and version causing change, not merely say “AI spend rose.” Review cost per successful workflow alongside raw spend; adoption can legitimately raise total cost while reducing unit cost.
Avoid incentives that encourage teams to hide usage or disable safety checks. Give owners an optimization checklist and escalation path for justified overages. Budgets are a planning instrument, not a substitute for observability.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.