Scaling RAG means preserving access boundaries and retrieval quality while documents, tenants, versions, and workloads grow. It does not mean adding more vectors. Multi-tenant isolation, refresh orchestration, and retrieval SLOs are the operating foundation.
Belle Realty’s primary rule is scoped retrieval: authorization and organization filters happen before search, and every result preserves document version and citation. Deal rooms need equivalent deal-level isolation. Shared knowledge requires explicit publication and ownership—not accidental visibility through embeddings.
The supporting documents cover tenancy, refreshes, and service objectives. Treat relevance and groundedness as reliability metrics, not UX decoration.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Store tenant and organization identity as mandatory metadata on every source, chunk, embedding, cache entry, and trace. Enforce it in the serving path before lexical/vector search, reranking, and context assembly. Do not depend on a prompt that says “only use this tenant.” The database/vector layer must reject unauthorized content.
Use a shared index with hard metadata filters only if the store can prove filter enforcement and you test it. For higher isolation needs, use per-organization namespaces or collections. Access revocation must remove serving eligibility immediately, even when asynchronous cleanup follows. Preserve source version, document status, and visibility role.
Test with adversarial queries, guessed IDs, broad semantic searches, cached answers, and reranker inputs. Add canary chunks that should never cross a boundary. Log denial decisions without leaking existence. Multi-tenant RAG is an authorization system that happens to use retrieval.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Run document changes through a versioned event pipeline: detect change, validate ownership, normalize, derive chunks, embed, validate counts, publish atomically, then retire old artifacts. A manifest tracks each source checksum and derived version. This prevents half-indexed leases from becoming answerable.
Prioritize revocations and current signed documents. Coalesce rapid edits, skip unchanged content, and queue expensive backfills separately. Use a new index version for corpus-wide embedding or chunking changes, evaluate it, then switch traffic by flag. Keep rollback possible and trace which index answered a request.
Monitor queue age, failed stages, freshness lag, active-versus-source version mismatch, and retrieval regression. Refresh work is a deploy pipeline, not a background mystery.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.
Set SLOs that describe the retrieval system users depend on: active-document freshness, authorized-result correctness, gold-chunk recall@k, citation precision, answer groundedness, p95 retrieval latency, and index availability. Define an error budget and owner for each.
For example, legal lease answers may require 100% citation support, no cross-tenant results, and a strict freshness target after an approved document upload. A less critical marketing knowledge base can accept slower refresh. Track score slices by document type, scan quality, property, query intent, and language.
When an SLO fails, diagnose source data, metadata filters, chunking, lexical/vector retrieval, reranking, or answer policy in that order. Do not solve low recall by sending more irrelevant chunks to the model. Publish a simple status view for operators and tie material breaches to incident response.
Operating standard
Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.