Add Runtime Operators graph research card

README.md

@@ -0,0 +1,239 @@
+---
+library_name: neosyntropy
+tags:
+  - neosyntropy
+  - state-machine
+  - software-engineering
+  - agentic-workflow
+  - tool-use
+  - structured-output
+  - reasoning
+  - lora
+  - swe-bench
+---
+
+# NeoSyntropy Runtime Operators
+
+## Overfit the graph, not the benchmark.
+
+Language models are good at proposing actions. Production software must decide
+which actions are legal, what evidence is required, and when a task is complete.
+
+NeoSyntropy Runtime Operators is an experimental model family for testing one
+idea: small, specialized models can become reliable software-engineering workers
+when they are trained for narrow state-machine operators and executed inside an
+application-owned graph.
+
+This repository is currently a research card and evaluation specification. It
+does **not** contain trained weights or make benchmark-performance claims yet.
+
+## The model family
+
+The graph composes several model roles instead of asking one unconstrained model
+to own the entire workflow.
+
+| Model | Responsibility |
+| --- | --- |
+| [Runtime Structure](https://huggingface.co/Neosyntropy/runtime-structure) | Convert observations into application-owned schemas |
+| [Runtime Route](https://huggingface.co/Neosyntropy/runtime-route) | Propose a legal next node from declared candidates |
+| [Runtime Deterministic Reasoning](https://huggingface.co/Neosyntropy/runtime-deterministic-reasoning) | Perform a bounded reasoning step and declare tool calls |
+| [Runtime Stochastic Reasoning](https://huggingface.co/Neosyntropy/runtime-stochastic-reasoning) | Generate alternative plans or repairs inside an allowed search space |
+| [Runtime Guard](https://huggingface.co/Neosyntropy/runtime-guard) | Validate claims against rules, tests, and supplied evidence |
+| [Runtime Score](https://huggingface.co/Neosyntropy/runtime-score) | Produce rubric-grounded measurements for evaluation and selection |
+
+The planned unified operator adapter conditions these roles with explicit tokens
+such as `<OPERATOR:UNDERSTAND>`, `<OPERATOR:PROPOSE>`, and
+`<OPERATOR:REPAIR>`. NeoSyntropy remains responsible for execution, transition
+legality, state commits, and side effects.
+
+## The operator graph
+
+```text
+                            ┌───────────────┐
+                            │   RETRIEVE    │◄──────────────┐
+                            └───────┬───────┘               │
+                                    │ evidence              │ need information
+                                    ▼                       │
+┌────────────┐  requirements  ┌─────────────┐  candidates  ┌┴───────────┐
+│ UNDERSTAND │───────────────►│  DECOMPOSE  │─────────────►│  PROPOSE   │
+└────────────┘                └─────────────┘              └─────┬──────┘
+                                                                │ selected plan
+                                                                ▼
+┌────────────┐  complete   ┌─────────────┐  evidence      ┌─────────────┐
+│  SUCCESS   │◄────────────│   VERIFY    │◄───────────────│   OBSERVE   │
+└────────────┘             └──────┬──────┘                └──────▲──────┘
+                                  │ failed test                   │ result
+                                  ▼                               │
+                           ┌─────────────┐  corrected plan  ┌─────┴───────┐
+                           │   REPAIR    │─────────────────►│   EXECUTE   │
+                           └─────────────┘                  └─────────────┘
+```
+
+The same graph expressed as NeoSyntropy code:
+
+```python
+from neosyntropy import END, FSM, edge_deterministic
+
+def signal_is(expected):
+    return lambda state: state.get("signal") == expected
+
+def verified_complete(state):
+    verification = state.get("verification", {})
+    return state.get("signal") == "COMPLETE" and verification.get("passed") is True
+
+graph = FSM(
+    entry=understand,
+    nodes=[
+        understand,
+        decompose,
+        retrieve,
+        propose,
+        execute,
+        observe,
+        verify,
+        repair,
+        compress_memory,
+        success,
+    ],
+    edges=[
+        edge_deterministic("Understand", "Decompose", guard=signal_is("CONTINUE")),
+        edge_deterministic("Understand", "Retrieve", guard=signal_is("NEED_INFORMATION")),
+        edge_deterministic("Decompose", "Propose", guard=signal_is("CONTINUE")),
+        edge_deterministic("Decompose", "Retrieve", guard=signal_is("NEED_INFORMATION")),
+        edge_deterministic("Retrieve", "Propose", guard=signal_is("CONTINUE")),
+        edge_deterministic("Propose", "Execute", guard=signal_is("CONTINUE")),
+        edge_deterministic("Execute", "Observe", guard=signal_is("EXECUTED")),
+        edge_deterministic("Observe", "Verify", guard=signal_is("CONTINUE")),
+        edge_deterministic("Observe", "Compress", guard=signal_is("MEMORY_PRESSURE")),
+        edge_deterministic("Verify", "Success", guard=verified_complete),
+        edge_deterministic("Verify", "Repair", guard=signal_is("FAILED_TEST")),
+        edge_deterministic("Verify", "Retrieve", guard=signal_is("NEED_INFORMATION")),
+        edge_deterministic("Verify", "Decompose", guard=signal_is("WRONG_PLAN")),
+        edge_deterministic("Repair", "Execute", guard=signal_is("CONTINUE")),
+        edge_deterministic("Repair", "Retrieve", guard=signal_is("NEED_INFORMATION")),
+        edge_deterministic("Compress", "Decompose", guard=signal_is("CONTINUE")),
+        edge_deterministic("Success", END),
+    ],
+)
+```
+
+`Edge` guards are authoritative. A model may propose a route, but it cannot
+commit an undeclared transition or bypass a failed verification gate.
+
+## One shared state
+
+Operators do not own isolated hidden memories. They read a projection of one
+auditable workflow state and return a validated state patch.
+
+```json
+{
+  "goal": "Fix the reported repository issue",
+  "requirements": [],
+  "task_tree": [],
+  "current_plan": "",
+  "retrieved": [],
+  "observations": [],
+  "errors": [],
+  "evidence": [],
+  "history": [],
+  "scratch": {},
+  "signal": "CONTINUE"
+}
+```
+
+Each operator receives only the fields it needs:
+
+| Operator | Reads | Writes |
+| --- | --- | --- |
+| `UNDERSTAND` | goal, initial context | requirements, unknowns, signal |
+| `DECOMPOSE` | goal, requirements | task tree, signal |
+| `RETRIEVE` | goal, errors, repository context | retrieved evidence, signal |
+| `PROPOSE` | requirements, task tree, evidence | candidate plan, signal |
+| `EXECUTE` | selected plan | trusted execution result |
+| `OBSERVE` | execution result | observations, errors, signal |
+| `VERIFY` | requirements, observations, tests | evidence, verification result, signal |
+| `REPAIR` | current plan, errors, evidence | corrected plan, signal |
+| `COMPRESS` | accumulated state | compact state patch, signal |
+| `SUCCESS` | verified evidence | terminal outcome |
+
+`EXECUTE`, deterministic observation parsing, transition checks, and final
+success gates should remain trusted runtime operations. Learned models propose;
+the graph validates and commits.
+
+## What “overfit the graph” means
+
+The phrase describes deliberate specialization to a stable execution protocol:
+
+- fixed operator vocabulary;
+- explicit input projections and output schemas;
+- declared tools and legal transitions;
+- repair loops driven by real test evidence;
+- consistent prompts across teacher generation, training, and inference.
+
+It does **not** mean training on benchmark test patches, hidden tests, or expected
+answers. Benchmark instances and repositories used for final evaluation must be
+kept out of training and teacher-label generation. The hypothesis is that a model
+can learn the reusable procedure while still generalizing to unseen issues.
+
+## SWE evaluation plan
+
+The first experiment will compare the same base model and tool environment under
+four scaffolds:
+
+1. **Direct agent** — one unconstrained model call loop.
+2. **Prompted operators** — operator prompts without fine-tuning.
+3. **Trained operators** — the operator adapter without graph enforcement.
+4. **NeoSyntropy graph** — trained operators with schemas, guards, state, and
+   verified transitions.
+
+Evaluation will begin with the official
+[SWE-bench](https://www.swebench.com/) harness for reproducibility, while newer
+or contamination-resistant suites should be used for primary generalization
+claims. Static benchmark scores will always be reported with the exact harness,
+model, scaffold, context limit, tool budget, retry budget, and task exclusions.
+
+Primary metrics:
+
+- resolved instances (`pass@1`);
+- legal-transition rate;
+- schema-valid output rate;
+- tool-call success rate;
+- verification precision and false-success rate;
+- repair-loop recovery rate;
+- model tokens, wall-clock time, tool calls, and cost per resolved task;
+- average state transitions and repeated-transition rate;
+- outcome consistency across repeated seeds.
+
+The most important comparison is not only whether a task was solved, but whether
+the graph can reach the same or better result with smaller models, fewer wasted
+actions, and no unverified success transition.
+
+## Data-generation protocol
+
+Training traces are generated from complete graph executions rather than isolated
+first-step prompts:
+
+1. Sample an unseen repository task and construct the initial state.
+2. Use a stronger teacher to label the current operator only.
+3. Validate the output schema and reject invented tools or evidence.
+4. Execute approved actions in the benchmark environment.
+5. Record observations and test results as new evidence.
+6. Route through the declared graph and continue until success or budget expiry.
+7. Store each operator transition with its state projection, output, signal,
+   provenance, and final task outcome.
+8. Freeze repository-level train, validation, and test splits before fine-tuning.
+
+This produces training examples for the decision that was actually available at
+each state, including failed attempts and evidence-grounded repairs.
+
+## Status
+
+- Graph contract: specified.
+- Role-model cards: published separately.
+- Unified operator dataset: planned.
+- Unified operator adapter: not trained yet.
+- SWE baseline runs: not run yet.
+- Benchmark claims: none.
+
+Results, weights, datasets, and exact run manifests will be published only after
+reproducible evaluation.