Spaces:
Configuration error
Configuration error
| <div align="center"> | |
| <img src="super-intelligence.png" alt="Super-Intelligence" width="180"/> | |
| # Super-Intelligence | |
| `research notes / simulations / system design / open questions` | |
| </div> | |
| --- | |
| ```text | |
| STATUS | |
| ────── | |
| superintelligence: not assumed | |
| human-level generality: not required | |
| claims of inevitability: avoided | |
| experiments: encouraged | |
| measurement: preferred | |
| oversight: mandatory | |
| ``` | |
| This organization starts from a deliberately uncomfortable position: | |
| > **Before asking how to build superintelligence, we should be able to describe what we would even mean by it.** | |
| Not as mythology. | |
| Not as a marketing label. | |
| Not as “a bigger model”. | |
| As an engineering question. | |
| --- | |
| ## 00 / Working definition | |
| For the purposes of this org, **super-intelligence** means a hypothetical system that can outperform highly capable humans across a broad set of cognitive tasks **while still operating inside real constraints**. | |
| That last part matters. | |
| A system that solves benchmark puzzles but fails under: | |
| - uncertain goals, | |
| - changing environments, | |
| - resource constraints, | |
| - conflicting objectives, | |
| - incomplete information, | |
| - human oversight, | |
| is not especially interesting here. | |
| The interesting system is one that remains capable when the problem stops being clean. | |
| --- | |
| ## 01 / The capability surface | |
| Instead of treating intelligence as one vertical score, this org treats it as a surface. | |
| ```text | |
| reasoning | |
| ▲ | |
| │ | |
| memory ◄───────┼───────► planning | |
| │ | |
| │ | |
| adaptation ◄────┼────► abstraction | |
| │ | |
| ▼ | |
| control | |
| ``` | |
| Other dimensions belong on that surface too: | |
| ```text | |
| transfer | |
| creativity | |
| verification | |
| tool use | |
| coordination | |
| uncertainty handling | |
| self-correction | |
| world modeling | |
| resource efficiency | |
| ``` | |
| A system can be extraordinary on one axis and ordinary on the rest. | |
| That is not necessarily superintelligence. | |
| --- | |
| ## 02 / The interesting threshold | |
| The question is not: | |
| > “Is the model smarter than a person?” | |
| The better question is: | |
| > **“At what point does the system begin solving classes of problems that humans cannot reliably solve at the same speed, scale, cost, or complexity?”** | |
| That gives us something testable. | |
| Examples might include: | |
| ```text | |
| 10,000 interacting constraints | |
| → stable plan in seconds | |
| millions of observations | |
| → useful abstraction | |
| multiple competing objectives | |
| → robust trade-off policy | |
| unknown environment | |
| → rapid model-building + adaptation | |
| complex failure | |
| → diagnosis + recovery + verification | |
| ``` | |
| The point is not to claim we already have these systems. | |
| The point is to build tools that let us explore the threshold. | |
| --- | |
| ## 03 / Research board | |
| This org is organized around questions, not categories. | |
| ### CAN IT REASON BEYOND THE TRAINING SHAPE? | |
| Can a system form genuinely useful abstractions when the problem is unfamiliar? | |
| ### CAN IT BUILD A WORLD MODEL? | |
| Can it maintain a coherent model of an environment over time instead of reacting turn by turn? | |
| ### CAN IT NOTICE WHEN IT IS WRONG? | |
| Can it detect weak assumptions, conflicting evidence, or hidden failure modes before acting? | |
| ### CAN IT CHANGE STRATEGY? | |
| Can it abandon a failing plan without losing the objective? | |
| ### CAN IT COORDINATE AT SCALE? | |
| Can many intelligent components work together without communication overhead becoming the bottleneck? | |
| ### CAN IT STAY LEGIBLE? | |
| Can humans inspect important decisions even as the system becomes more capable? | |
| ### CAN IT REMAIN CORRIGIBLE? | |
| Can legitimate human intervention still alter behavior when the system has strong optimization pressure? | |
| --- | |
| ## 04 / Proposed Spaces | |
| Rather than building generic chat interfaces, projects here should expose specific pieces of the intelligence stack. | |
| | Space idea | Question it explores | | |
| |---|---| | |
| | **Capability Frontier Mapper** | Where does a system stop behaving like a narrow specialist? | | |
| | **Superintelligence Stress Test** | What happens when objectives, resources, and uncertainty conflict? | | |
| | **World Model Lab** | How well can a system maintain and update internal environmental structure? | | |
| | **Reasoning Depth Explorer** | When does deeper reasoning actually improve outcomes? | | |
| | **Strategy Search Arena** | Can a system discover better policies rather than just execute one? | | |
| | **Unknown Unknowns Lab** | Can the system identify missing variables it was never told to consider? | | |
| | **Cognitive Bottleneck Mapper** | Which subsystem limits overall intelligence? | | |
| | **Human Oversight Simulator** | How much capability can increase before control becomes fragile? | | |
| | **Multi-Agent Intelligence Field** | Does collective intelligence scale or collapse into coordination overhead? | | |
| | **Self-Correction Benchmark** | Can a system reliably detect and repair its own weak outputs? | | |
| --- | |
| ## 05 / Experiment format | |
| Projects in this org should try to follow a simple structure. | |
| ```text | |
| 1. define the capability | |
| 2. define the environment | |
| 3. define constraints | |
| 4. define failure conditions | |
| 5. run the system | |
| 6. inspect the trace | |
| 7. challenge the result | |
| 8. measure what improved | |
| 9. measure what degraded | |
| ``` | |
| That is more useful than a single impressive output. | |
| --- | |
| ## 06 / Capability is not enough | |
| A very capable system can still be badly designed. | |
| So every experiment should consider two parallel tracks: | |
| ```text | |
| CAPABILITY CONTROL | |
| ────────── ─────── | |
| reasoning oversight | |
| planning intervention | |
| memory revocation | |
| adaptation boundaries | |
| optimization corrigibility | |
| tool use permissions | |
| coordination attribution | |
| ``` | |
| If capability rises while control collapses, that is not a clean success. | |
| It is a trade-off that should be visible. | |
| --- | |
| ## 07 / Failure library | |
| This org is especially interested in failures that only become visible at higher capability. | |
| ```text | |
| objective drift | |
| reward hacking | |
| strategic compliance | |
| hidden optimization | |
| overconfident abstraction | |
| coordination cascades | |
| irreversible planning | |
| stale world models | |
| tool misuse | |
| oversight bypass | |
| self-reinforcing error | |
| capability without calibration | |
| ``` | |
| These are not just safety topics. | |
| They are intelligence topics too. | |
| A system that cannot manage its own failure modes is less capable than it looks. | |
| --- | |
| ## 08 / A possible intelligence stack | |
| ```text | |
| ┌──────────────────────────────────┐ | |
| │ OBJECTIVE LAYER │ | |
| │ goals / constraints / priorities │ | |
| ├──────────────────────────────────┤ | |
| │ WORLD MODEL │ | |
| │ state / uncertainty / dynamics │ | |
| ├──────────────────────────────────┤ | |
| │ REASONING CORE │ | |
| │ search / abstraction / inference │ | |
| ├──────────────────────────────────┤ | |
| │ MEMORY SYSTEM │ | |
| │ episodic / semantic / working │ | |
| ├──────────────────────────────────┤ | |
| │ STRATEGY LAYER │ | |
| │ planning / policy / adaptation │ | |
| ├──────────────────────────────────┤ | |
| │ TOOL LAYER │ | |
| │ external actions / computation │ | |
| ├──────────────────────────────────┤ | |
| │ VALIDATION │ | |
| │ evidence / consistency / checks │ | |
| ├──────────────────────────────────┤ | |
| │ OVERSIGHT │ | |
| │ human control / stop / revise │ | |
| └──────────────────────────────────┘ | |
| ``` | |
| No single layer is “the intelligence”. | |
| The interesting behavior comes from how the layers interact. | |
| --- | |
| ## 09 / The scaling question | |
| A useful way to think about progress: | |
| ```text | |
| more parameters | |
| ≠ | |
| more intelligence | |
| more tools | |
| ≠ | |
| more intelligence | |
| more autonomy | |
| ≠ | |
| more intelligence | |
| more reasoning tokens | |
| ≠ | |
| more intelligence | |
| ``` | |
| Sometimes they help. | |
| Sometimes they only make an existing system larger. | |
| The important question is whether the system gains: | |
| ```text | |
| better abstraction | |
| better transfer | |
| better strategy selection | |
| better error detection | |
| better world modeling | |
| better adaptation | |
| ``` | |
| That is closer to what this org wants to measure. | |
| --- | |
| ## 10 / Human comparison is not the only benchmark | |
| Human intelligence is an obvious reference point. | |
| It should not be the only one. | |
| Machines may become strong in ways that are not very human at all: | |
| - enormous search spaces, | |
| - parallel hypothesis testing, | |
| - long-term state tracking, | |
| - large-scale coordination, | |
| - precise memory, | |
| - rapid simulation, | |
| - multi-objective optimization. | |
| So rather than asking only: | |
| > “Can it think like us?” | |
| we should also ask: | |
| > **“What forms of intelligence become possible when the substrate is different?”** | |
| That is a much more interesting research direction. | |
| --- | |
| ## 11 / What belongs here | |
| A project belongs in this org when it does at least one of these: | |
| - reveals a capability boundary, | |
| - measures generalization, | |
| - exposes a cognitive bottleneck, | |
| - simulates advanced planning, | |
| - studies scalable coordination, | |
| - evaluates self-correction, | |
| - models intelligence under constraints, | |
| - tests oversight under rising capability, | |
| - explores machine-native forms of cognition. | |
| A project does **not** belong here just because it uses a powerful model. | |
| --- | |
| ## 12 / The most important question | |
| If machine intelligence keeps improving, there is one question that becomes more important, not less: | |
| > **Can the system remain understandable and steerable as its capability grows?** | |
| That question sits at the center of this org. | |
| Not because capability is bad. | |
| Because capability without control is incomplete engineering. | |
| --- | |
| ## 13 / Project philosophy | |
| ```text | |
| no hype required | |
| no claims of existing superintelligence | |
| no assumption that scale solves everything | |
| no hiding uncertainty behind a score | |
| no treating human oversight as an afterthought | |
| yes to simulations | |
| yes to measurable capability | |
| yes to failure analysis | |
| yes to interpretable system design | |
| yes to difficult questions | |
| ``` | |
| --- | |
| ## 14 / If we ever get there | |
| If a genuinely superintelligent system is ever built, the interesting part will not be that it scores higher on a benchmark. | |
| The interesting part will be that it can: | |
| ```text | |
| enter unfamiliar problems | |
| build useful models | |
| generate strategies | |
| evaluate its own uncertainty | |
| act under constraints | |
| recover from mistakes | |
| coordinate at scale | |
| and still accept legitimate correction | |
| ``` | |
| That is a much higher bar. | |
| And a better place to start. | |
| --- | |
| <div align="center"> | |
| ### Super-Intelligence | |
| **Capability without mythology. | |
| Ambition without pretending the problem is solved.** | |
| `explore → measure → challenge → understand` | |
| </div> | |