Wide layout

Table of contents

Governing AI Agents

Pillars of AI governance.

Whats wrong with Agents today?

  • Built as rapid POCs where governance is an afterthought

  • Prioritizing raw accuracy and stability while completely ignoring security controls

  • Granting overly broad, elevated permissions for convenience during early development

  • Operating without audit trails, making agent decisions and tool executions untraceable

Impact:
3x longer production deployment cycles to remediate governance and security debt

What is Agent Governance?

Data governance is the framework of polices, processes, and
standards that ensure data is accurate, secure and properly
managed throughout its lifecycle

mermaid
flowchart TB
    classDef teal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#000000;

    subgraph prepareData ["Prepare data"]
        direction TB
        vdb["VectorDB"]
        vm["View/Masks"]
        tbl["Tables"]
    end

    subgraph buildAgents ["Build agents"]
        direction TB
        llm["LLM Selection"]
        prompt["Prompt Eng."]
        tools["Build/Bind Tools"]
    end

    subgraph evalAgents ["Evaluate agents"]
        direction TB
        judges["LLM judges"]
        peer["Peer labeling"]
        trace["Tracing"]
    end

    subgraph deployAgents ["Deploy agents"]
        direction TB
        serving["Agent serving"]
        mlops["MLOps/LLMOps"]
        lineage["Lineage"]
    end

    subgraph governAgents ["Govern agents (Unity Catalog + MLflow)"]
        direction LR
        lm["Lifecycle Management"]:::teal
        rm["Risk Management"]:::teal
        sec["Security"]:::teal
        obs["Observability"]:::teal
    end

    prepareData --> buildAgents --> evalAgents --> deployAgents
    deployAgents --> governAgents --> prepareData
Pillar
Best Practice
Key Techniques & Tools

Lifecycle Management

Separation of Duties

Version Control, CI/CD Pipelines, Environment Management, Deployment Orchestration, Change Management

Risk Management

Defense in Depth

Data Quality Monitoring, PII Detection, Guardrails, Compliance, Model Validation

Security

Least Privilege Access

Authentication, Secrets Management, Access Control, Data Protection, Network Security

Observability

Audit Everything

Tracing, Logging, Monitoring, Lineage, Alerting

Pillar
Governance Checklist

Lifecycle Management

Can you safely promote changes through environments with proper review and rollback capabilities?

Risk Management

Do you have multiple layers of protection to catch issues before they impact production?

Security

Are all your data sources accessible only to authorized agents and users?

Observability

Can you trace every tool your agent used, when it ran, what data it accessed, and what it returned?

Unity Catalog

mermaid
flowchart TD
    classDef dashed fill:#f8fafc,stroke:#94a3b8,stroke-width:1.5px,stroke-dasharray: 4 4,color:#475569;

    metastore["Metastore"]

    %% Storage & Credentials (Metastore-level)
    serviceCred["Service<br/>credential"]:::dashed
    storageCred["Storage<br/>credential"]:::dashed
    extLoc["External<br/>location"]:::dashed
    extMeta["External<br/>metadata"]:::dashed

    %% Core 3-Level Namespace
    catalog["Catalog"]
    schema[("Schema")]

    table["Table"]
    view["View"]
    volume["Volume\n*unstructured storage"]
    function["Function<br/>including models"]

    %% Sharing & Collaboration
    share["Share"]:::dashed
    recipient["Recipient"]:::dashed
    provider["Provider"]:::dashed
    connection["Connection"]:::dashed
    cleanRoom["Clean Room"]:::dashed

    metastore -.-> serviceCred & storageCred & extLoc & extMeta
    metastore --> catalog
    metastore -.-> share & recipient & provider & connection & cleanRoom

    catalog --> schema
    schema --> table & view & volume & function

Identity

mermaid
flowchart TD
    classDef acct fill:#ffffff,stroke:#1e293b,stroke-width:2px,color:#0f172a;
    classDef item fill:#ffffff,stroke:#64748b,stroke-width:1.5px,color:#0f172a;
    classDef grpBlue fill:#e0e7ff,stroke:#4338ca,stroke-width:1.5px,color:#1e1b4b;
    classDef grpRed fill:#fee2e2,stroke:#dc2626,stroke-width:1.5px,color:#7f1d1d;
    classDef meta fill:#f0fdf4,stroke:#16a34a,stroke-width:1.5px,color:#14532d;

    subgraph account ["Account"]
        direction LR

        subgraph acctAdmin ["Account Management"]
            direction TB
            billing["Billing and Usage"]:::item
            acctAdmins["Account Administrators"]:::item
        end

        subgraph acctIdentities ["Users, Service Principals, Groups"]
            direction TB
            subgraph groupsRow [" "]
                direction LR
                g1["Group 1"]:::grpBlue
                g2["Group 2"]:::grpBlue
                g3["Group 3"]:::grpRed
            end
        end
    end

    subgraph deployments [" "]
        direction LR

        subgraph metastoreBox ["Metastore"]
            metastore["Data & AI Assets<br/>(Catalogs, Schemas, Tables)"]:::meta
        end

        subgraph ws1 ["Workspace"]
            ws1_g2["Group 2"]:::grpBlue
        end

        subgraph ws2 ["Workspace"]
            direction LR
            ws2_g1["Group 1"]:::grpBlue
            ws2_g3["Group 3"]:::grpRed
        end
    end

    account --> metastoreBox
    account --> ws1
    account --> ws2

    g2 -.-> ws1_g2
    g1 -.-> ws2_g1
    g3 -.-> ws2_g3

    metastoreBox -.- ws1
    metastoreBox -.- ws2

Databricks Identities

There are three types of Databricks identity:

  • Users: User identities recognized by Databricks and represented by email addresses.

  • Service principals: Identities for use with jobs, automated tools, and systems such as scripts, apps, and CI/CD platforms.

  • Groups: Groups simplify identity management, making it easier to assign access to workspaces, data, and other securable objects.

Administrative Roles

The following administrative roles manage Databricks identities:

  • Account Admins: Add users, service principals, and groups to the account, and assign admin roles.

  • Workspace Admins: Add users and service principals to the account; grant access to workspaces.

  • Group Managers: Manage group membership and assign the group manager role.

  • Service Principal Managers: Manage roles on service principals.

Agent Authentication Modes

mermaid
flowchart LR
    classDef user fill:#e0e7ff,stroke:#4338ca,stroke-width:1.5px,color:#1e1b4b;
    classDef agent fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f;
    classDef sp fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a;
    classDef res fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a;

    subgraph autoMode ["Automatic Passthrough (System Auth)"]
        direction TB
        caller1["Caller / User"]:::user --> agent1["AI Agent"]:::agent
        agent1 -->|"Agent's SP<br/>(Same identity)"| sp1["Agent Service Principal"]:::sp
        sp1 --> res1[("Declared Resources<br/>& Tools")]:::res
    end

    subgraph oboMode ["On-Behalf-of-User (OBO)"]
        direction TB
        caller2["Caller / User"]:::user -->|"Caller token & scopes"| agent2["AI Agent"]:::agent
        agent2 -->|"Caller's identity<br/>(User-level access)"| res2[("Sensitive User Data")]:::res
    end

    subgraph manualMode ["Manual Authentication"]
        direction TB
        secAdmin["Security Admin"]:::user -->|"Pre-provisions"| sp3["Central SP"]:::sp
        caller3["Caller / User"]:::user --> agent3["AI Agent"]:::agent
        agent3 -->|"Pre-configured credentials"| sp3
        sp3 --> res3[("Enterprise Resources")]:::res
    end

Automatic Authentication Passthrough

System Authentication:

  • Agent service principal gets automatic access to declared resources

  • SAME identity regardless of who calls it

  • Managed credentials automatically

On-Behalf-of-User Authentication

Delegated Authentication:

  • Agent uses end user's credentials (CALLER's identity)

  • Good for user-specific sensitive data access

  • Requires specifying API scopes

Manual Authentication

Centralized Provisioning:

  • Pre-created service principals with specific permissions

  • What some orgs prefer for CENTRALIZED identity management

  • No on-the-fly service principal creation

MLflow

MLflow is the observability and lifecycle layer of agent governance — the complement to Unity Catalog's data & access layer. Together they form the Govern agents ring in the pipeline above.

mermaid
flowchart LR
    classDef mlflow fill:#fff7ed,stroke:#c2410c,stroke-width:1.5px,color:#7c2d12;
    classDef artifact fill:#f8fafc,stroke:#475569,stroke-width:1.5px,color:#0f172a;
    classDef action fill:#e0e7ff,stroke:#4338ca,stroke-width:1.5px,color:#1e1b4b;
    classDef output fill:#f0fdf4,stroke:#16a34a,stroke-width:1.5px,color:#14532d;

    subgraph experiment ["Experiment Tracking"]
        direction TB
        params["Prompts, Temperatures,<br/>Model Names"]:::artifact
        metrics["Metrics & Output Quality"]:::artifact
        code["Code Versions & Datasets"]:::artifact
    end

    subgraph tracing ["Tracing"]
        direction TB
        chain["Agent Execution Tree"]:::artifact
        tools["Tool Calls & Retrieval Steps"]:::artifact
        latency["Token Counts & Latency"]:::artifact
    end

    subgraph registry ["Model Registry"]
        direction TB
        staging["Staging"]:::action
        production["Production"]:::action
        archived["Archived"]:::action
        staging --> production --> archived
    end

    subgraph evaluation ["Evaluation"]
        direction TB
        judge["LLM-as-a-Judge"]:::artifact
        criteria["Relevance, Groundedness,<br/>Toxicity, Hallucination"]:::artifact
    end

    agent["AI Agent"]:::mlflow

    agent --> experiment
    agent --> tracing
    agent --> evaluation
    evaluation -->|"winner promoted"| registry
    experiment -->|"best run registered"| registry
    registry -->|"deploy to"| prod[("Production")]:::output
    prod -->|"feedback traces"| tracing

Experiment Tracking

MLflow tracks every experiment run so you can compare and reproduce results:

  • What it logs: prompts, model names, temperatures, code versions, datasets, and output metrics.

  • Autologging works out of the box with LangChain, LlamaIndex, OpenAI, Anthropic, and HuggingFace — no boilerplate required.

Tracing

Multi-step agents are black boxes without tracing. MLflow Tracing gives granular visibility into every execution:

  • Records the full agent execution tree — chain-of-thought, tool calls, RAG retrievals.

  • Captures token counts, latency, and errors at each step.

  • OpenTelemetry compatible — stream traces into any production observability pipeline.

  • Operates asynchronously so production performance is never impacted.

Model Registry

Governance for agents means controlling what reaches production:

Stage
Description

Staging

Candidate version under review and testing

Production

Live version serving user traffic

Archived

Retired version kept for audit and rollback

Beyond models, the registry manages prompt templates — version-control your system prompts, compare which variant scored best in evaluation, and roll back instantly on regressions.

Evaluation

MLflow replaces manual QA with automated, repeatable evaluation:

  • LLM-as-a-Judge scores outputs on relevance, groundedness, toxicity, and hallucination rate.

  • Evaluation results are tied to the experiment run, so every registry promotion has a traceable score attached.

MLflow + Unity Catalog

Responsibility
Unity Catalog
MLflow

Data access

Tables, volumes, schemas, secrets

—

Model storage

Registers models as UC assets

Logs artifacts and promotes to UC

Lineage

Data lineage across pipelines

Experiment → run → model lineage

Observability

Audit logs of data access

Traces of agent tool calls and outputs

Lifecycle

Environment promotion via CI/CD

Staging → Production → Archived stages