The Curriculum / Reader / Agent Governance
LEVEL 2 · INTERMEDIATE · COMPANY TRACK

Agent Governance

This page compiles 5 files from the repository, verbatim, in reading order. The living version: this folder on GitHub.

level-2-intermediate/company/02-agent-governance/README.md

Agent Governance

Governance is how you decide which agents may exist, what they may do, and how they are stopped. It is not a committee that reviews every prompt change. Apply more control as impact rises: a document lookup assistant needs evidence and access controls; an agent proposing changes to financial or personnel systems needs formal approval, audit, and a kill switch.

Every agent has an owner, user, purpose, risk tier, data classification, tool list, eval gates, cost budget, launch status, and retirement plan. Record these in a registry. If nobody owns an agent, it should not be running.

The three supporting documents define a lightweight review board, a lifecycle, and an incident-ready kill switch. The principle is fast experimentation inside firm boundaries.

Related: the individual analogue

The org-wide approval matrix in this module is the multi-team version of the same three-zone framework Groundwork teaches individuals in Layer 3 (Governance): Auto-Execute / Draft & Wait / Never Automate. If a person on your team hasn't run Groundwork, they'll benefit from doing so — their personal approval rules feed straight into how they'll behave under your org governance. See GROUNDWORK-AND-FLUENCY.md and approval-flows.md in this module.

Operating standard

Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team's regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more customer or internal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.

level-2-intermediate/company/02-agent-governance/agent-lifecycle.md

Agent Lifecycle

  1. Brief: define user, decision, non-goals, data, tools, risk, and success metric. 2. Prototype: use synthetic or sandboxed data; no external side effects. 3. Offline evaluation: establish correctness, safety, retrieval, latency, and cost baselines. 4. Review: approve data/tool scope and rollout plan. 5. Shadow: run against live inputs without acting; compare with staff outcomes. 6. Limited launch: feature flag, narrow users, required human review. 7. Operate: monitor, sample outputs, refresh evals, track drift and costs. 8. Retire: revoke tools, archive traces under retention policy, and remove stale indexes.

Define exit criteria at every stage. A shadow agent moves forward only if it meets slice-level metrics, not an average score. An operational agent re-enters review after a material incident, model migration, policy change, or expansion of permissions.

Lifecycle state belongs in the agent registry and visible UI. Users should know whether they are seeing an experimental draft, a human-approved recommendation, or an automated action.

Operating standard

Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.

level-2-intermediate/company/02-agent-governance/agent-review-board.md

Agent Review Board

Use a small review board for medium- and high-risk agents: product owner, technical owner, security/privacy reviewer, and operations representative. Review the agent brief, intended users, data scope, tools, write proposals, human gates, eval evidence, cost ceiling, and rollback plan. Low-risk internal drafts can use an asynchronous checklist; do not turn every experiment into a meeting.

Classify risk by impact and reversibility. Any agent touching tenant records, money, legal terms, fair-housing matters, external communication, or cross-team knowledge needs a documented decision. Require a demo against adversarial cases, not only happy paths. Approval expires when core data scope, side effects, model family, or tool permissions materially change.

The board should reject vague products: “an assistant for operations” has no testable boundary. It should also provide an approved pattern library so teams can move faster next time. Governance earns trust by making safe deployment quicker than inventing controls from scratch.

Operating standard

Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.

level-2-intermediate/company/02-agent-governance/approval-flows.md

Approval flows: the three-tier framework, operationalized for teams

Read this if: You are about to let an agent send email, edit a database, spend money, or make any change that a customer, employee, or regulator will see. You need a rule set your team can actually apply under pressure.

You do not need to have read anything else in this program to use this doc. It stands alone.

The one idea

Every agent action falls into exactly one of three categories. The categories are not descriptive — they are prescriptive. The category decides whether a human is in the loop.

Category Rule Human in loop? Typical latency
1. Auto-Execute Low-risk, reversible, internal-only, cheap to correct No Seconds
2. Draft and Wait Anything that reaches another person or leaves your systems Yes, before send Minutes to hours
3. Never Automate Money, legal commitments, personnel decisions, sensitive comms Yes, and the human takes the action, not the agent As long as it takes

If you are unsure which category, the answer is Category 2. Demoting an agent from Category 1 after a bad send costs more than promoting it from Category 2 after it earns trust.

The four escalation rules

The categories cover the known cases. These four rules cover everything else:

  1. If the agent is uncertain about classification, default to Category 2.
  2. If the agent encounters information marked private or restricted, stop and ask.
  3. If the task involves money, legal, or personnel, Category 3. No exceptions.
  4. If the output will be seen outside the organization, Category 2 at minimum.

Print these four rules on the wall next to whoever runs the AI program. They do more work than any policy document you could write.

Where the rules live in code

Your agents should treat these categories as first-class configuration, not hidden behavior. In the reference companion repos (belle-mcp-server, lease-abstractor, support-triage-agent, diligence-agent), every write-capable tool has an auto_send: false flag that defaults off. That is Category 2 by construction. To move a tool to Category 1, an owner has to change the flag deliberately, sign off, and add a monitoring gate.

Recommended pattern for any agent you build:

# agent-manifest.yaml
name: support-triage
owner: cx-team@example.com
category: 2  # Draft and Wait
tools:
  - name: draft_reply
    category: 2
    auto_send: false
  - name: tag_conversation
    category: 1  # Auto-Execute — internal, reversible
    auto_send: true
  - name: issue_refund
    category: 3  # Never Automate
    auto_send: false
    human_action_required: true

The manifest is the contract. If it says Category 3, no code path in the agent may take that action, even with approval. Approval means a human uses their own credentials to act.

The seed rule list (start here)

Copy this into your first agent registry. It's short on purpose.

Task Category
File incoming documents into folders 1
Tag support tickets by topic 1
Update internal status logs 1
Generate a meeting summary 1
Draft an email response to a customer 2
Draft a marketing post 2
Reply to a Slack message on someone's behalf 2
Update a CRM record with a customer-facing note 2
Send a payment 3
Sign a contract 3
Terminate an employee's access 3
Post publicly on the company account 3

Extend as your team encounters new cases. Every added row goes to the review board (see agent-review-board.md) before it ships.

Who owns each category

Promotion path

An agent tool can be promoted from Category 2 to Category 1 only after:

  1. Volume: at least 200 draft-and-approved outputs across at least 30 days
  2. Approval rate: ≥ 95% of drafts approved without material edit
  3. Zero incidents: no false-positive send, no data leak, no policy violation
  4. Reviewer sign-off: the human who has been reviewing signs off in writing
  5. Owner sign-off: the process owner signs off in writing
  6. Rollback path: flipping the auto_send flag back to false requires no code change

Anything short of all six leaves the tool in Category 2. Ambiguity does not promote.

Common failure modes

What this doc replaces

If you already have a policy document, keep it, but put this three-tier table on page 1.

Related

level-2-intermediate/company/02-agent-governance/kill-switch-playbook.md

Kill-Switch Playbook

Provide three controls: global AI disable, per-agent disable, and per-tool/write disable. They must be server-side, fast, auditable, and tested. A disabled agent returns a clear fallback experience; it must not quietly call a different unreviewed model.

Trigger the switch for suspected data exposure, unsafe external action, emergency-routing regression, unauthorized tool activity, runaway spend, provider compromise, or material policy violation. First contain: disable writes and external sends, preserve traces, notify the agent owner and operations lead, and assess affected organizations. Do not erase logs during an incident.

Then investigate scope, revoke credentials if needed, communicate factually, fix the root cause, add regression cases, and require re-approval before re-enable. Run a quarterly drill. A kill switch that has never been exercised is only a diagram.

Operating standard

Make this practice operational, not aspirational. Assign one directly responsible owner and name the decision they can make without another meeting. Put the key measure, threshold, and review cadence in the owning team’s regular operating rhythm. A change to model, prompt, data scope, retrieval index, tool permission, or policy should be recorded with its expected impact and a rollback path. Preserve enough trace information to explain an individual bad outcome without exposing more tenant or deal data than necessary. Review a small sample of real runs with the people doing the work; dashboards reveal trends, but operators reveal missing context. When the rule is violated, capture the incident, contain impact, add an eval or control, and update this document if the standard itself was unclear. The point is repeatable judgment under real workload, not a one-time compliance exercise.

← Platform Engineering for AI RAG at Scale →