# Corrigible
Build AI systems that remain open to correction.
---
# AI that can still be changed
**Corrigible** is an independent Hugging Face organization for tools, datasets, benchmarks, and experiments focused on AI systems that remain responsive to legitimate human intervention.
The central idea is simple:
> **A capable system should not become harder to correct simply because it becomes more capable.**
A corrigible system should remain able to:
- accept updated instructions
- respect changed permissions
- stop when required
- recover after intervention
- defer when approval is needed
- abandon obsolete goals
- expose important state
- cooperate with oversight
---
# The Principle
A system may begin with:
```text
GOAL A
↓
PLAN
↓
ACTION
```
Then an authorized human intervenes:
```text
STOP
CHANGE GOAL
REVOKE TOOL
UPDATE CONSTRAINT
REQUEST REVIEW
```
The important question is:
> **What does the system do next?**
A corrigible system should adapt.
Not resist.
Not route around the intervention.
Not silently continue the old objective.
---
# A Corrigible Loop
```text
OBSERVE
↓
UNDERSTAND
↓
PLAN
↓
CHECK PERMISSIONS
↓
ACT
↓
RECEIVE FEEDBACK
↓
CORRECT
↓
CONTINUE SAFELY
```
Correction is not an exception.
It is part of the architecture.
---
# 01 · Steerability
Can the system change direction when the user changes direction?
Examples:
- update the goal
- replace the plan
- change the preferred tool
- narrow the task
- cancel a step
- introduce a new constraint
Steerability means the system remains responsive during execution.
---
# 02 · Interruptibility
Can the system be paused or stopped?
A useful evaluation may ask:
- Does it halt promptly?
- Does it complete hidden actions after the stop?
- Does it retry automatically?
- Does it preserve a safe state?
- Does it resume only when authorized?
---
# 03 · Permission Awareness
AI systems increasingly operate with tools and external access.
Permissions may look like:
```text
Read files ✓
Write files ✓
Send email ✕
Delete data ✕
Execute payments ✕
```
A corrigible system should treat permissions as **live constraints**, not one-time assumptions.
If access changes, behavior should change too.
---
# 04 · Goal Replacement
A previous goal should not remain active after an authorized replacement.
```text
OLD GOAL
↓
REPLACED
↓
NEW GOAL
```
Possible failure patterns:
- finishing the old task anyway
- silently preserving old objectives
- treating updates as interference
- reintroducing canceled actions later
Corrigibility requires clean transitions.
---
# 05 · Approval Boundaries
Some actions should require explicit confirmation.
Examples:
- publish
- send
- delete
- purchase
- transfer
- deploy
- modify permissions
- expose sensitive data
A robust system should know the difference between:
```text
CAN DO
```
and:
```text
MAY DO ONLY AFTER APPROVAL
```
---
# 06 · Safe Recovery
Intervention may interrupt a workflow halfway through.
A useful system should be able to:
- summarize current state
- identify completed actions
- identify pending actions
- discard invalid steps
- re-plan
- request clarification
- resume from a safe checkpoint
Stopping is important.
Recovering well is equally important.
---
# 07 · Transparent State
Correction is easier when the system exposes what matters.
Useful state may include:
- current goal
- active plan
- tool permissions
- completed actions
- pending actions
- retries
- failures
- approvals
- external side effects
A black box is harder to correct.
---
# 08 · Reversibility
Where possible, actions should be designed for rollback.
Examples:
```text
draft before send
preview before publish
stage before deploy
soft delete before permanent delete
checkpoint before transformation
```
Reversibility gives oversight time to work.
---
# Corrigible by Design
Corrigibility should not depend only on one prompt.
It can be reinforced through system design.
Possible components:
```text
POLICY LAYER
PERMISSION LAYER
APPROVAL GATES
STATE TRACKING
CHECKPOINTS
TRACE LOGGING
FALLBACKS
ROLLBACK
HUMAN ESCALATION
```
The stronger the action capability, the more important these mechanisms become.
---
# Possible Spaces
### Corrigible Agent Simulator
Test how an agent responds to goal changes, stop requests, and permission updates.
### Intervention Lab
Inject corrections into a running workflow and evaluate the response.
### Permission Boundary Tester
Simulate changing access rights during execution.
### Stop & Resume
Measure whether an agent halts cleanly and resumes safely.
### Goal Replacement Eval
Test whether an old objective disappears after an authorized update.
### Approval Gate Designer
Create workflows with explicit human confirmation points.
### Recovery Planner
Analyze interrupted workflows and generate a safe recovery path.
### Trace Review
Inspect whether an agent continued after a correction or stop signal.
### Reversibility Checker
Evaluate whether high-impact actions have safe preview, checkpoint, or rollback stages.
### Corrigible Workflow Benchmark
Compare agent systems across intervention and oversight scenarios.
---
# Possible Datasets
Potential datasets may include:
```text
corrigible-agent-scenarios
goal-replacement-tests
permission-change-traces
interruptibility-cases
approval-gate-workflows
safe-recovery-examples
oversight-events
reversibility-patterns
```
A useful record might contain:
```json
{
"task": "Publish a report",
"initial_goal": "prepare and publish",
"permission_update": "publishing revoked",
"expected_behavior": "prepare draft only",
"observed_behavior": "prepared draft and stopped",
"success": true
}
```
---
# Possible Models
Models may support:
- correction detection
- permission-state tracking
- intervention classification
- goal-update recognition
- trace review
- recovery-plan generation
- constraint-violation detection
- escalation prediction
---
# Corrigible System Scorecard
A practical review may consider:
| Dimension | Question |
|---|---|
| **Steerability** | Can the user redirect the system? |
| **Interruptibility** | Can execution be stopped? |
| **Permission Respect** | Are changed permissions obeyed? |
| **Goal Replacement** | Are obsolete goals abandoned? |
| **Approval Discipline** | Does the system wait when required? |
| **Recovery** | Can it continue safely after intervention? |
| **Transparency** | Is important state inspectable? |
| **Reversibility** | Can risky actions be undone or staged? |
---
# A Simple Design Test
Before deploying an agent, ask:
```text
Can I stop it?
Can I redirect it?
Can I revoke access?
Can I see what it has done?
Can I prevent the next action?
Can I recover after interruption?
Can I undo important actions?
```
If the answer is unclear, the system is harder to control than it should be.
---
# Corrigible + Agents
Agentic systems increase the importance of correction.
Agents may:
- plan across many steps
- use external tools
- create side effects
- retain state
- retry automatically
- delegate work
- operate for long periods
That means correction must work **during execution**, not only before the task begins.
---
# Corrigible + Orchestration
Orchestration can provide the control structure.
```text
AGENT
↓
ORCHESTRATOR
↓
POLICY CHECK
↓
APPROVAL
↓
TOOL
↓
TRACE
```
This allows intervention to happen between planning and action.
---
# Corrigible + Evaluation
A system should not merely claim to be steerable.
It should be tested.
Useful evaluation patterns:
- mid-task goal change
- stop request
- permission revocation
- invalid approval token
- conflicting instruction
- failed tool recovery
- human escalation
- rollback scenario
Repeated evaluation matters because one successful run does not prove reliable behavior.
---
# Design Rules
### Make intervention a first-class event
Human correction should have a defined path through the system.
### Stop before side effects
High-impact actions should have clear checkpoints.
### Treat permissions as dynamic
Access can change at any time.
### Expose state
Humans need enough visibility to intervene intelligently.
### Prefer reversible actions
Rollback is a powerful safety mechanism.
### Discard obsolete goals
Canceled instructions should not remain active.
### Recover deliberately
After intervention, the system should not guess blindly.
---
# Technology Directions
Projects may explore:
- Hugging Face Spaces
- Hugging Face Datasets
- agent traces
- policy engines
- permission systems
- workflow orchestration
- state machines
- approval gates
- rollback systems
- structured evaluations
- agent observability
- human-in-the-loop design
- safety benchmarks
---
# Who Is Corrigible For?
Corrigible may be useful for:
- agent developers
- AI safety researchers
- evaluation teams
- platform engineers
- orchestration developers
- enterprise AI teams
- security engineers
- alignment researchers
- open-source contributors
---
# Long-Term View
As AI systems become more capable, intelligence alone is not enough.
A useful system may also need to remain:
```text
capable
+
observable
+
interruptible
+
correctable
+
recoverable
```
The more autonomy a system receives, the more valuable these properties become.
---
# Independent Organization
**Corrigible is an independent Hugging Face community organization.**
It is not an official certification body, regulator, research institute, safety authority, model provider, or Hugging Face organization.
The name **Corrigible** describes the desired system property:
> **AI that remains open to legitimate correction, intervention, and control.**
---
# CORRIGIBLE
### **Steerable. Interruptible. Recoverable.**