Different models are good at different jobs, but adding models is not the same as adding reliability. Orchestration should make a workflow cheaper, safer, or measurably more accurate. If it only makes the architecture more impressive, delete it.
A practical Belle Realty stack has a fast structured classifier, an embedding/reranking path, a stronger evidence synthesizer, and deterministic policy services. Route by task and risk. Use a council only for bounded, high-value ambiguity where independent viewpoints can be judged against evidence. Run calls in parallel when they do not depend on one another; keep dependent evidence steps serial.
Every route needs an observable reason, fallback, timeout, budget, and eval. Model names are configuration, not product logic. Pin versions where possible and route through a gateway so you can change providers without rewriting agents.
The next documents cover router design, councils, and concurrency decisions. The default remains one capable model with a good retrieval and policy layer. Add another only when the eval tells you what it fixes.
Before adding complexity, run this design through a small representative eval and inspect the trace with the operator who will own failures. Make the boundary, escalation, and rollback visible in the product. That discipline will expose more useful work than another round of prompt cleverness.
Model Council: Independent Views, One Accountable Decision
A model council asks several independent models or prompts to analyze the same bounded question, then has a judge compare their outputs against evidence. Use it sparingly for high-value ambiguity: conflicting lease addenda, a complicated diligence exception, or a draft where legal/policy risk requires multiple structured perspectives. Do not use it for tenant message routing or routine extraction.
Make each member produce a compact structured claim set with citations, uncertainty, and recommended disposition. Independence matters: do not feed one member’s answer to another before the first vote. The judge receives the original evidence and blind member outputs, then selects, merges only supported claims, or escalates. A human owns the final decision for material risk.
Councils fail when they become majority-vote theater. Three models repeating the same hallucination are not evidence. Weight source support above agreement. If no member cites an authoritative document, the correct outcome may be abstention. Limit the council to two or three members and a single judge; otherwise cost and latency rise faster than quality.
Evaluate against a single-model baseline on the narrow slice it targets. Track consensus rate, judge agreement with humans, improved accuracy, unsupported-claim rate, cost, and latency. Keep the council behind a feature flag. The best council outcome is often a clearly documented “needs review,” not a forced answer.
Run calls in parallel only when they are independent and their combined outputs can be validated together. Examples: classify a tenant message while retrieving open tickets; retrieve lexical and vector candidates simultaneously; extract financial, legal, and insurance checklist sections from separate documents. Parallelism lowers wall-clock latency but raises burst cost and can complicate cancellation.
Use serial calls when later work depends on verified evidence from earlier work. A lease answer should identify the authorized active lease before retrieving clauses. A write proposal should pass policy validation before any draft is generated. A verifier should inspect the actual extractor output, not a competing guess. Serial steps preserve causality and reduce wasted requests.
Design a join contract for parallel work: each branch has a deadline, schema, confidence, and failure state. The joiner knows how to proceed if a noncritical branch times out and when to fail closed. Cancel remaining work after an emergency rule fires or a human gate is required. Do not wait for decorative enrichments while a tenant needs an escalation.
Measure p50/p95 latency, duplicate work, cancellation success, cost, and quality. Parallel calls are not free just because they are faster. Prefer a simple dependency graph you can trace over a swarm of agents that all read the same context.
A model router chooses an execution path from measurable inputs: task type, language, requested output, document quality, risk tier, estimated context size, prior confidence, latency target, and budget remaining. It returns a route ID and reason. Do not ask a general model to select its own unlimited tools and pricing tier.
Start deterministic. A known JSON classifier can use the low-cost route; scanned leases or conflicting addenda use the high-accuracy extraction route; emergencies skip drafting and go to policy plus human escalation. Add model-based routing only for ambiguity after you label examples. Calibrate thresholds on the eval set, then monitor actual route outcomes.
Implement fallbacks narrowly. If a primary model times out, retry once or use an approved fallback with the same schema. If structured output fails, return a review task rather than parsing free text creatively. Enforce maximum context, output, cost, and latency per route. A router that silently makes three expensive calls is a billing bug.
Evaluate routing accuracy and system outcomes together: quality by route, escalation rate, cost per successful task, tail latency, and safety failures. Review cases that route cheaply but later require human correction. Routing is a policy decision, so version it, feature-flag it, and allow fast rollback.