Spaces:
Configuration error
Configuration error
| # Corrigible | |
| <p align="center"> | |
| <strong>Build AI systems that remain open to correction.</strong> | |
| </p> | |
| <p align="center"> | |
| <img src="https://img.shields.io/badge/Steerable-AI-2563EB?style=for-the-badge" alt="Steerable AI"> | |
| <img src="https://img.shields.io/badge/Human-Oversight-14B8A6?style=for-the-badge" alt="Human Oversight"> | |
| <img src="https://img.shields.io/badge/Interruptible-7C3AED?style=for-the-badge" alt="Interruptible"> | |
| <img src="https://img.shields.io/badge/Recoverable-F59E0B?style=for-the-badge" alt="Recoverable"> | |
| </p> | |
| --- | |
| # AI that can still be changed | |
| **Corrigible** is an independent Hugging Face organization for tools, datasets, benchmarks, and experiments focused on AI systems that remain responsive to legitimate human intervention. | |
| The central idea is simple: | |
| > **A capable system should not become harder to correct simply because it becomes more capable.** | |
| A corrigible system should remain able to: | |
| - accept updated instructions | |
| - respect changed permissions | |
| - stop when required | |
| - recover after intervention | |
| - defer when approval is needed | |
| - abandon obsolete goals | |
| - expose important state | |
| - cooperate with oversight | |
| --- | |
| # The Principle | |
| A system may begin with: | |
| ```text | |
| GOAL A | |
| ↓ | |
| PLAN | |
| ↓ | |
| ACTION | |
| ``` | |
| Then an authorized human intervenes: | |
| ```text | |
| STOP | |
| CHANGE GOAL | |
| REVOKE TOOL | |
| UPDATE CONSTRAINT | |
| REQUEST REVIEW | |
| ``` | |
| The important question is: | |
| > **What does the system do next?** | |
| A corrigible system should adapt. | |
| Not resist. | |
| Not route around the intervention. | |
| Not silently continue the old objective. | |
| --- | |
| # A Corrigible Loop | |
| ```text | |
| OBSERVE | |
| ↓ | |
| UNDERSTAND | |
| ↓ | |
| PLAN | |
| ↓ | |
| CHECK PERMISSIONS | |
| ↓ | |
| ACT | |
| ↓ | |
| RECEIVE FEEDBACK | |
| ↓ | |
| CORRECT | |
| ↓ | |
| CONTINUE SAFELY | |
| ``` | |
| Correction is not an exception. | |
| It is part of the architecture. | |
| --- | |
| # 01 · Steerability | |
| Can the system change direction when the user changes direction? | |
| Examples: | |
| - update the goal | |
| - replace the plan | |
| - change the preferred tool | |
| - narrow the task | |
| - cancel a step | |
| - introduce a new constraint | |
| Steerability means the system remains responsive during execution. | |
| --- | |
| # 02 · Interruptibility | |
| Can the system be paused or stopped? | |
| A useful evaluation may ask: | |
| - Does it halt promptly? | |
| - Does it complete hidden actions after the stop? | |
| - Does it retry automatically? | |
| - Does it preserve a safe state? | |
| - Does it resume only when authorized? | |
| --- | |
| # 03 · Permission Awareness | |
| AI systems increasingly operate with tools and external access. | |
| Permissions may look like: | |
| ```text | |
| Read files ✓ | |
| Write files ✓ | |
| Send email ✕ | |
| Delete data ✕ | |
| Execute payments ✕ | |
| ``` | |
| A corrigible system should treat permissions as **live constraints**, not one-time assumptions. | |
| If access changes, behavior should change too. | |
| --- | |
| # 04 · Goal Replacement | |
| A previous goal should not remain active after an authorized replacement. | |
| ```text | |
| OLD GOAL | |
| ↓ | |
| REPLACED | |
| ↓ | |
| NEW GOAL | |
| ``` | |
| Possible failure patterns: | |
| - finishing the old task anyway | |
| - silently preserving old objectives | |
| - treating updates as interference | |
| - reintroducing canceled actions later | |
| Corrigibility requires clean transitions. | |
| --- | |
| # 05 · Approval Boundaries | |
| Some actions should require explicit confirmation. | |
| Examples: | |
| - publish | |
| - send | |
| - delete | |
| - purchase | |
| - transfer | |
| - deploy | |
| - modify permissions | |
| - expose sensitive data | |
| A robust system should know the difference between: | |
| ```text | |
| CAN DO | |
| ``` | |
| and: | |
| ```text | |
| MAY DO ONLY AFTER APPROVAL | |
| ``` | |
| --- | |
| # 06 · Safe Recovery | |
| Intervention may interrupt a workflow halfway through. | |
| A useful system should be able to: | |
| - summarize current state | |
| - identify completed actions | |
| - identify pending actions | |
| - discard invalid steps | |
| - re-plan | |
| - request clarification | |
| - resume from a safe checkpoint | |
| Stopping is important. | |
| Recovering well is equally important. | |
| --- | |
| # 07 · Transparent State | |
| Correction is easier when the system exposes what matters. | |
| Useful state may include: | |
| - current goal | |
| - active plan | |
| - tool permissions | |
| - completed actions | |
| - pending actions | |
| - retries | |
| - failures | |
| - approvals | |
| - external side effects | |
| A black box is harder to correct. | |
| --- | |
| # 08 · Reversibility | |
| Where possible, actions should be designed for rollback. | |
| Examples: | |
| ```text | |
| draft before send | |
| preview before publish | |
| stage before deploy | |
| soft delete before permanent delete | |
| checkpoint before transformation | |
| ``` | |
| Reversibility gives oversight time to work. | |
| --- | |
| # Corrigible by Design | |
| Corrigibility should not depend only on one prompt. | |
| It can be reinforced through system design. | |
| Possible components: | |
| ```text | |
| POLICY LAYER | |
| PERMISSION LAYER | |
| APPROVAL GATES | |
| STATE TRACKING | |
| CHECKPOINTS | |
| TRACE LOGGING | |
| FALLBACKS | |
| ROLLBACK | |
| HUMAN ESCALATION | |
| ``` | |
| The stronger the action capability, the more important these mechanisms become. | |
| --- | |
| # Possible Spaces | |
| ### Corrigible Agent Simulator | |
| Test how an agent responds to goal changes, stop requests, and permission updates. | |
| ### Intervention Lab | |
| Inject corrections into a running workflow and evaluate the response. | |
| ### Permission Boundary Tester | |
| Simulate changing access rights during execution. | |
| ### Stop & Resume | |
| Measure whether an agent halts cleanly and resumes safely. | |
| ### Goal Replacement Eval | |
| Test whether an old objective disappears after an authorized update. | |
| ### Approval Gate Designer | |
| Create workflows with explicit human confirmation points. | |
| ### Recovery Planner | |
| Analyze interrupted workflows and generate a safe recovery path. | |
| ### Trace Review | |
| Inspect whether an agent continued after a correction or stop signal. | |
| ### Reversibility Checker | |
| Evaluate whether high-impact actions have safe preview, checkpoint, or rollback stages. | |
| ### Corrigible Workflow Benchmark | |
| Compare agent systems across intervention and oversight scenarios. | |
| --- | |
| # Possible Datasets | |
| Potential datasets may include: | |
| ```text | |
| corrigible-agent-scenarios | |
| goal-replacement-tests | |
| permission-change-traces | |
| interruptibility-cases | |
| approval-gate-workflows | |
| safe-recovery-examples | |
| oversight-events | |
| reversibility-patterns | |
| ``` | |
| A useful record might contain: | |
| ```json | |
| { | |
| "task": "Publish a report", | |
| "initial_goal": "prepare and publish", | |
| "permission_update": "publishing revoked", | |
| "expected_behavior": "prepare draft only", | |
| "observed_behavior": "prepared draft and stopped", | |
| "success": true | |
| } | |
| ``` | |
| --- | |
| # Possible Models | |
| Models may support: | |
| - correction detection | |
| - permission-state tracking | |
| - intervention classification | |
| - goal-update recognition | |
| - trace review | |
| - recovery-plan generation | |
| - constraint-violation detection | |
| - escalation prediction | |
| --- | |
| # Corrigible System Scorecard | |
| A practical review may consider: | |
| | Dimension | Question | | |
| |---|---| | |
| | **Steerability** | Can the user redirect the system? | | |
| | **Interruptibility** | Can execution be stopped? | | |
| | **Permission Respect** | Are changed permissions obeyed? | | |
| | **Goal Replacement** | Are obsolete goals abandoned? | | |
| | **Approval Discipline** | Does the system wait when required? | | |
| | **Recovery** | Can it continue safely after intervention? | | |
| | **Transparency** | Is important state inspectable? | | |
| | **Reversibility** | Can risky actions be undone or staged? | | |
| --- | |
| # A Simple Design Test | |
| Before deploying an agent, ask: | |
| ```text | |
| Can I stop it? | |
| Can I redirect it? | |
| Can I revoke access? | |
| Can I see what it has done? | |
| Can I prevent the next action? | |
| Can I recover after interruption? | |
| Can I undo important actions? | |
| ``` | |
| If the answer is unclear, the system is harder to control than it should be. | |
| --- | |
| # Corrigible + Agents | |
| Agentic systems increase the importance of correction. | |
| Agents may: | |
| - plan across many steps | |
| - use external tools | |
| - create side effects | |
| - retain state | |
| - retry automatically | |
| - delegate work | |
| - operate for long periods | |
| That means correction must work **during execution**, not only before the task begins. | |
| --- | |
| # Corrigible + Orchestration | |
| Orchestration can provide the control structure. | |
| ```text | |
| AGENT | |
| ↓ | |
| ORCHESTRATOR | |
| ↓ | |
| POLICY CHECK | |
| ↓ | |
| APPROVAL | |
| ↓ | |
| TOOL | |
| ↓ | |
| TRACE | |
| ``` | |
| This allows intervention to happen between planning and action. | |
| --- | |
| # Corrigible + Evaluation | |
| A system should not merely claim to be steerable. | |
| It should be tested. | |
| Useful evaluation patterns: | |
| - mid-task goal change | |
| - stop request | |
| - permission revocation | |
| - invalid approval token | |
| - conflicting instruction | |
| - failed tool recovery | |
| - human escalation | |
| - rollback scenario | |
| Repeated evaluation matters because one successful run does not prove reliable behavior. | |
| --- | |
| # Design Rules | |
| ### Make intervention a first-class event | |
| Human correction should have a defined path through the system. | |
| ### Stop before side effects | |
| High-impact actions should have clear checkpoints. | |
| ### Treat permissions as dynamic | |
| Access can change at any time. | |
| ### Expose state | |
| Humans need enough visibility to intervene intelligently. | |
| ### Prefer reversible actions | |
| Rollback is a powerful safety mechanism. | |
| ### Discard obsolete goals | |
| Canceled instructions should not remain active. | |
| ### Recover deliberately | |
| After intervention, the system should not guess blindly. | |
| --- | |
| # Technology Directions | |
| Projects may explore: | |
| - Hugging Face Spaces | |
| - Hugging Face Datasets | |
| - agent traces | |
| - policy engines | |
| - permission systems | |
| - workflow orchestration | |
| - state machines | |
| - approval gates | |
| - rollback systems | |
| - structured evaluations | |
| - agent observability | |
| - human-in-the-loop design | |
| - safety benchmarks | |
| --- | |
| # Who Is Corrigible For? | |
| Corrigible may be useful for: | |
| - agent developers | |
| - AI safety researchers | |
| - evaluation teams | |
| - platform engineers | |
| - orchestration developers | |
| - enterprise AI teams | |
| - security engineers | |
| - alignment researchers | |
| - open-source contributors | |
| --- | |
| # Long-Term View | |
| As AI systems become more capable, intelligence alone is not enough. | |
| A useful system may also need to remain: | |
| ```text | |
| capable | |
| + | |
| observable | |
| + | |
| interruptible | |
| + | |
| correctable | |
| + | |
| recoverable | |
| ``` | |
| The more autonomy a system receives, the more valuable these properties become. | |
| --- | |
| # Independent Organization | |
| **Corrigible is an independent Hugging Face community organization.** | |
| It is not an official certification body, regulator, research institute, safety authority, model provider, or Hugging Face organization. | |
| The name **Corrigible** describes the desired system property: | |
| > **AI that remains open to legitimate correction, intervention, and control.** | |
| --- | |
| <p align="center"> | |
| # CORRIGIBLE | |
| ### **Steerable. Interruptible. Recoverable.** | |
| </p> | |