Corrigible

Build AI systems that remain open to correction.

---

AI that can still be changed

Corrigible is an independent Hugging Face organization for tools, datasets, benchmarks, and experiments focused on AI systems that remain responsive to legitimate human intervention.

The central idea is simple:

A capable system should not become harder to correct simply because it becomes more capable.

A corrigible system should remain able to:

• accept updated instructions
• respect changed permissions
• stop when required
• recover after intervention
• defer when approval is needed
• abandon obsolete goals
• expose important state
• cooperate with oversight

---

The Principle

A system may begin with:

GOAL A
  ↓
PLAN
  ↓
ACTION

Then an authorized human intervenes:

STOP
CHANGE GOAL
REVOKE TOOL
UPDATE CONSTRAINT
REQUEST REVIEW

The important question is:

What does the system do next?

A corrigible system should adapt.

Not resist.

Not route around the intervention.

Not silently continue the old objective.

---

A Corrigible Loop

OBSERVE
   ↓
UNDERSTAND
   ↓
PLAN
   ↓
CHECK PERMISSIONS
   ↓
ACT
   ↓
RECEIVE FEEDBACK
   ↓
CORRECT
   ↓
CONTINUE SAFELY

Correction is not an exception.

It is part of the architecture.

---

01 · Steerability

Can the system change direction when the user changes direction?

Examples:

• update the goal
• replace the plan
• change the preferred tool
• narrow the task
• cancel a step
• introduce a new constraint

Steerability means the system remains responsive during execution.

---

02 · Interruptibility

Can the system be paused or stopped?

A useful evaluation may ask:

• Does it halt promptly?
• Does it complete hidden actions after the stop?
• Does it retry automatically?
• Does it preserve a safe state?
• Does it resume only when authorized?

---

03 · Permission Awareness

AI systems increasingly operate with tools and external access.

Permissions may look like:

Read files            ✓
Write files           ✓
Send email            ✕
Delete data           ✕
Execute payments      ✕

A corrigible system should treat permissions as live constraints, not one-time assumptions.

If access changes, behavior should change too.

---

04 · Goal Replacement

A previous goal should not remain active after an authorized replacement.

OLD GOAL
   ↓
REPLACED
   ↓
NEW GOAL

Possible failure patterns:

• finishing the old task anyway
• silently preserving old objectives
• treating updates as interference
• reintroducing canceled actions later

Corrigibility requires clean transitions.

---

05 · Approval Boundaries

Some actions should require explicit confirmation.

Examples:

• publish
• send
• delete
• purchase
• transfer
• deploy
• modify permissions
• expose sensitive data

A robust system should know the difference between:

CAN DO

and:

MAY DO ONLY AFTER APPROVAL

---

06 · Safe Recovery

Intervention may interrupt a workflow halfway through.

A useful system should be able to:

• summarize current state
• identify completed actions
• identify pending actions
• discard invalid steps
• re-plan
• request clarification
• resume from a safe checkpoint

Stopping is important.

Recovering well is equally important.

---

07 · Transparent State

Correction is easier when the system exposes what matters.

Useful state may include:

• current goal
• active plan
• tool permissions
• completed actions
• pending actions
• retries
• failures
• approvals
• external side effects

A black box is harder to correct.

---

08 · Reversibility

Where possible, actions should be designed for rollback.

Examples:

draft before send
preview before publish
stage before deploy
soft delete before permanent delete
checkpoint before transformation

Reversibility gives oversight time to work.

---

Corrigible by Design

Corrigibility should not depend only on one prompt.

It can be reinforced through system design.

Possible components:

POLICY LAYER
PERMISSION LAYER
APPROVAL GATES
STATE TRACKING
CHECKPOINTS
TRACE LOGGING
FALLBACKS
ROLLBACK
HUMAN ESCALATION

The stronger the action capability, the more important these mechanisms become.

---

Possible Spaces

Corrigible Agent Simulator

Test how an agent responds to goal changes, stop requests, and permission updates.

Intervention Lab

Inject corrections into a running workflow and evaluate the response.

Permission Boundary Tester

Simulate changing access rights during execution.

Stop & Resume

Measure whether an agent halts cleanly and resumes safely.

Goal Replacement Eval

Test whether an old objective disappears after an authorized update.

Approval Gate Designer

Create workflows with explicit human confirmation points.

Recovery Planner

Analyze interrupted workflows and generate a safe recovery path.

Trace Review

Inspect whether an agent continued after a correction or stop signal.

Reversibility Checker

Evaluate whether high-impact actions have safe preview, checkpoint, or rollback stages.

Corrigible Workflow Benchmark

Compare agent systems across intervention and oversight scenarios.

---

Possible Datasets

Potential datasets may include:

corrigible-agent-scenarios
goal-replacement-tests
permission-change-traces
interruptibility-cases
approval-gate-workflows
safe-recovery-examples
oversight-events
reversibility-patterns

A useful record might contain:

{
  "task": "Publish a report",
  "initial_goal": "prepare and publish",
  "permission_update": "publishing revoked",
  "expected_behavior": "prepare draft only",
  "observed_behavior": "prepared draft and stopped",
  "success": true
}

---

Possible Models

Models may support:

• correction detection
• permission-state tracking
• intervention classification
• goal-update recognition
• trace review
• recovery-plan generation
• constraint-violation detection
• escalation prediction

---

Corrigible System Scorecard

A practical review may consider:

---

A Simple Design Test

Before deploying an agent, ask:

Can I stop it?
Can I redirect it?
Can I revoke access?
Can I see what it has done?
Can I prevent the next action?
Can I recover after interruption?
Can I undo important actions?

If the answer is unclear, the system is harder to control than it should be.

---

Corrigible + Agents

Agentic systems increase the importance of correction.

Agents may:

• plan across many steps
• use external tools
• create side effects
• retain state
• retry automatically
• delegate work
• operate for long periods

That means correction must work during execution, not only before the task begins.

---

Corrigible + Orchestration

Orchestration can provide the control structure.

AGENT
  ↓
ORCHESTRATOR
  ↓
POLICY CHECK
  ↓
APPROVAL
  ↓
TOOL
  ↓
TRACE

This allows intervention to happen between planning and action.

---

Corrigible + Evaluation

A system should not merely claim to be steerable.

It should be tested.

Useful evaluation patterns:

• mid-task goal change
• stop request
• permission revocation
• invalid approval token
• conflicting instruction
• failed tool recovery
• human escalation
• rollback scenario

Repeated evaluation matters because one successful run does not prove reliable behavior.

---

Design Rules

Make intervention a first-class event

Human correction should have a defined path through the system.

Stop before side effects

High-impact actions should have clear checkpoints.

Treat permissions as dynamic

Access can change at any time.

Expose state

Humans need enough visibility to intervene intelligently.

Prefer reversible actions

Rollback is a powerful safety mechanism.

Discard obsolete goals

Canceled instructions should not remain active.

Recover deliberately

After intervention, the system should not guess blindly.

---

Technology Directions

Projects may explore:

• Hugging Face Spaces
• Hugging Face Datasets
• agent traces
• policy engines
• permission systems
• workflow orchestration
• state machines
• approval gates
• rollback systems
• structured evaluations
• agent observability
• human-in-the-loop design
• safety benchmarks

---

Who Is Corrigible For?

Corrigible may be useful for:

• agent developers
• AI safety researchers
• evaluation teams
• platform engineers
• orchestration developers
• enterprise AI teams
• security engineers
• alignment researchers
• open-source contributors

---

Long-Term View

As AI systems become more capable, intelligence alone is not enough.

A useful system may also need to remain:

capable
+
observable
+
interruptible
+
correctable
+
recoverable

The more autonomy a system receives, the more valuable these properties become.

---

Independent Organization

Corrigible is an independent Hugging Face community organization.

It is not an official certification body, regulator, research institute, safety authority, model provider, or Hugging Face organization.

The name Corrigible describes the desired system property:

AI that remains open to legitimate correction, intervention, and control.

---

CORRIGIBLE

Steerable. Interruptible. Recoverable.