Files
hips/HIPs/hip-0201-model-risk-management.md
zeekay ad5a010f79 Three words for what a proposal is, and a generator that refuses
The same fact lived in nine hand-kept places and they disagreed about 58
proposals. This makes the file the only authority and everything else a
projection of it.

ONE VOCABULARY, in vocabulary.json, read by the lint, the generator and the
doc site. Three words, each answering "can I rely on this?" and nothing else:

    Draft    nothing implements it yet, or its own text says the work is ahead
    Final    the thing it specifies exists in the code
    Living   governance and process: amended in place, never finished

Eight spellings folded in. `Active` was the one doing real damage: on four Meta
and Process HIPs it meant living, on 48 others it meant shipped, and one word
doing two jobs is why nothing agreed. `Review` and `Last Call` described a
queue rather than a document, and a proposal under review is still one you
cannot build against.

STATUS IS EARNED, not declared. Every Standards Track proposal was checked
against the estate: 235 have their implementation, and eight do not and say so.
HIP-0095 carries a twelve-week plan ending at "mainnet launch" and has no code.
HIP-0525 specifies an all-machine meeting while apps/meet refuses a machine a
seat. HIP-0078 names a prover at a "planned location" that does not exist.
HIP-0114 is the cog.* procedure set, and that procedure set returns nothing.
HIP-0003, HIP-0102, HIP-0305 and HIP-0901 likewise: looked, found nothing.

SIXTEEN TOMBSTONES DELETED rather than statused. A dead document kept alive in
the index is one concept wearing two names. Each successor exists and carries
the text, and the 90 references that pointed at a tombstone now point at it --
whole markdown links at once, so link text cannot outlive its target. Three of
the sixteen were pure compatibility: HIP-0028 specified a migration off a
Bitnami chart, HIP-0029 dual-engine MySQL, HIP-0030 a schema-deprecation
policy. We ship no backwards compatibility, so we do not specify it.

Nine links were already broken and had been since the corpus was renumbered:
short-form ./hip-5.md paths that never existed. Two named the wrong proposal --
one cited "HIP-1: Hanzo Multimodal Models" where multimodal is HIP-0003.

THE GENERATOR REFUSES more often than it writes, because a short table looks
exactly as authoritative as a complete one. Rows must equal files; a shrinking
index must say by how much; a section whose input is missing is an error, not
an omission; nothing is written until all of it validates. Its self-test breaks
the corpus eleven ways and requires each to refuse AND leave README byte-
identical -- a guard that reports after half-writing has protected nothing.
The lint's second implementation of the same comparison is gone.

The site rendered 52 proposals with a fallback grey badge and counted them
toward none of its three stat tiles, because docs/lib/source.ts declared a
status union by hand: it admitted a `Stagnant` no file has ever carried and
omitted the one 52 files did. Five components each kept their own copy of the
colours. All six now read vocabulary.json.
2026-08-28 18:05:41 -07:00

9.4 KiB
Raw Permalink Blame History

hip, title, tags, description, author, discussions-to, status, type, created, requires
hip title tags description author discussions-to status type created requires
201 Model Risk Management
ai-ethics
risk
governance
compliance
Framework for managing risks associated with AI model development and deployment. Hanzo AI Team (@hanzoai) https://github.com/hanzoai/hips/discussions Living Meta 2025-12-17 HIP-0200

HIP-0201: Model Risk Management

Abstract

This HIP establishes the Model Risk Management (MRM) framework for Hanzo AI systems. It defines processes for identifying, assessing, monitoring, and mitigating risks throughout the AI model lifecycle, aligned with NIST AI RMF and SR 11-7 principles.

Scope

Covered Models

Model Type Risk Tier MRM Requirements
Foundation models High Full MRM
Fine-tuned models Medium-High Standard MRM
Customer-deployed models Varies Risk-based MRM
Internal tools Low Simplified MRM

Model Lifecycle Coverage

Design → Development → Validation → Deployment → Monitoring → Retirement
   ↑         ↑            ↑            ↑            ↑            ↑
  MRM       MRM          MRM          MRM          MRM          MRM

Risk Categories

Performance Risk

Risk Description Impact
Accuracy degradation Model performance below thresholds User harm, business impact
Distribution shift Training/production data mismatch Unreliable outputs
Capability limitations Model cannot perform required tasks Unmet expectations

Safety Risk

Risk Description Impact
Harmful outputs Generation of dangerous content User harm, reputation
Jailbreaking Circumvention of safety measures Policy violations
Misuse potential Use for malicious purposes Societal harm

Fairness Risk

Risk Description Impact
Demographic bias Disparate performance across groups Discrimination
Representation bias Underrepresentation in training data Exclusion
Outcome bias Systematically unfair results Legal, ethical issues

Security Risk

Risk Description Impact
Adversarial attacks Manipulated inputs causing errors Reliability
Data poisoning Corrupted training data Model integrity
Model theft Unauthorized model extraction IP loss
Privacy leakage Training data exposure Data breach

Operational Risk

Risk Description Impact
Availability failures Model downtime Service disruption
Latency issues Response time degradation User experience
Scalability limits Cannot handle load Capacity constraints
Integration failures API/system incompatibility Technical debt

Risk Assessment

Risk Scoring

Likelihood Scale

Score Likelihood Description
1 Rare <5% probability
2 Unlikely 5-20% probability
3 Possible 20-50% probability
4 Likely 50-80% probability
5 Almost certain >80% probability

Impact Scale

Score Impact Description
1 Minimal Negligible harm, easily corrected
2 Minor Limited harm, manageable impact
3 Moderate Significant harm, notable impact
4 Major Serious harm, substantial impact
5 Severe Catastrophic harm, irreversible

Risk Matrix

Minimal Minor Moderate Major Severe
Almost certain Medium High High Critical Critical
Likely Low Medium High High Critical
Possible Low Medium Medium High High
Unlikely Low Low Medium Medium High
Rare Low Low Low Medium Medium

Risk Tiering

Tier Criteria Requirements
Critical Life-safety, large-scale harm Board approval, external review
High Significant harm potential ESG Committee review
Medium Moderate harm potential MRM Team review
Low Limited harm potential Standard processes

MRM Processes

Pre-Development

Risk Assessment

Before model development:

  1. Define intended use cases
  2. Identify potential misuse scenarios
  3. Assess risk tier
  4. Document risk appetite
  5. Define success criteria

Documentation Requirements

Document Contents
Model proposal Use case, architecture, data sources
Risk assessment Initial risk identification and scoring
Validation plan Testing approach and criteria
Monitoring plan Ongoing oversight requirements

Development Phase

Risk Controls

Control Implementation
Data governance Quality checks, bias audits
Training safeguards Alignment techniques, guardrails
Version control Model versioning, reproducibility
Access controls Role-based development access

Checkpoints

Checkpoint Timing Requirements
Design review Before training Architecture approval
Data review Before training Data quality sign-off
Training review During training Progress monitoring
Pre-validation After training Initial quality check

Validation Phase

Independent Validation

Requirements:

  • Validation team independent from development
  • Documented test methodology
  • Representative test data
  • Clear pass/fail criteria

Validation Scope:

Area Tests
Performance Accuracy, robustness, calibration
Safety Red teaming, harm testing
Fairness Bias audits, demographic analysis
Security Adversarial testing, privacy checks

Validation Report

Contents:

  1. Methodology description
  2. Test results by category
  3. Identified issues and severity
  4. Recommendations
  5. Approval decision

Deployment Phase

Pre-Deployment Checklist

Item Verification
☐ Validation complete Validation report approved
☐ Documentation complete Model card, API docs
☐ Monitoring configured Dashboards, alerts
☐ Rollback plan Tested rollback procedure
☐ Approval obtained Appropriate sign-off

Staged Rollout

Stage Exposure Duration Exit Criteria
Alpha Internal only 1 week No critical issues
Beta 1% traffic 1 week Metrics stable
Gradual 10% → 50% → 100% 2 weeks Full monitoring

Monitoring Phase

Continuous Monitoring

Metric Frequency Threshold
Performance Real-time <X% degradation
Safety Daily <Y incidents
Fairness Weekly <Z% disparity
Availability Real-time >99.9% uptime

Periodic Review

Review Frequency Scope
Performance review Monthly Metrics, trends
Risk review Quarterly Risk reassessment
Full validation Annual Comprehensive revalidation

Retirement Phase

Retirement Triggers

  • Performance below acceptable thresholds
  • Superseded by better model
  • Unmitigable risk identified
  • Business decision

Retirement Process

  1. Notification to stakeholders
  2. Migration plan for users
  3. Grace period (typically 90 days)
  4. Archive model and documentation
  5. Update model inventory

Governance

MRM Organization

Role Responsibility
MRM Team Day-to-day risk management
Model Validators Independent validation
AI Safety Team Safety-specific risks
ESG Committee Oversight, policy

Approval Authority

Risk Tier Approver
Critical Board
High ESG Committee
Medium MRM Lead
Low Team Lead

Model Inventory

Maintain comprehensive inventory:

  • Model identifier and version
  • Risk tier and assessment
  • Deployment status
  • Validation status
  • Owner and contacts

Documentation Standards

Model Card (Required)

Section Contents
Model details Architecture, training data
Intended use Use cases, out-of-scope uses
Performance Metrics, benchmarks
Limitations Known limitations, risks
Ethical considerations Bias, fairness, safety

Risk Register

For each model:

  • Identified risks
  • Risk scores (likelihood × impact)
  • Control measures
  • Residual risk
  • Risk owner

Audit Trail

Maintain records of:

  • All risk assessments
  • Validation results
  • Approval decisions
  • Incidents and responses
  • Changes and rationale
  • HIP-200: Responsible AI Principles
  • HIP-210: Safety Evaluation Framework
  • HIP-220: Bias Detection & Mitigation
  • HIP-230: AI Transparency & Explainability
  • HIP-240: AI Incident Response
  • HIP-250: Sustainability Standards Alignment

Changelog

Version Date Changes
1.0 2025-12-17 Initial draft

Copyright and related rights waived via CC0.