dettools / README.md
cjc0013's picture
Normalize inherited Ouroboros publication invariants
68dbcd1 verified
|
Raw History Blame Contribute Delete
9.11 kB
# DETTOOLS
DETTOOLS is a deterministic local MCP for token-efficient code ingestion and
precise AST/CST-aware changes. It returns compact structured results while
preserving the exact source requested by the caller.
Author: **Ouroboros**
## Files and contact
The release includes an exact [wheel contents list](WHEEL-CONTENTS.md) and a [browseable source manifest](SOURCE-CONTENTS.md). The source snapshot is extracted byte-for-byte from the published wheel; the wheel remains the supported install artifact.
## Licence
DETTOOLS is released under the [MIT License](LICENSE.md). You may use, copy,
modify, merge, publish, distribute, sublicense, and sell copies subject to the
licence terms.
## Installation
DETTOOLS requires Python 3.11 or newer. Install the release-gated wheel in the
environment used by your MCP client:
```bash
python -m pip install dettools-0.9.0-py3-none-any.whl
```
The base install provides Python-aware reads and whole-file fallback for other
languages. For the C, C#, C++, Go, Java, JavaScript, Ruby, Rust, and TypeScript
structure and exact-symbol reads used in the nine-language benchmark, install
the wheel with its declared `treesitter` extra:
```bash
python -m pip install "dettools[treesitter] @ file:///absolute/path/to/dettools-0.9.0-py3-none-any.whl"
```
Configure the MCP server with a portable Python command:
```json
{
"mcpServers": {
"dettools": {
"command": "python",
"args": ["-m", "dettools.mcp_server"],
"env": {
"DETTOOLS_PUBLIC_MODE": "1"
}
}
}
}
```
The command-line front door is also available:
```bash
python -m dettools.cli --request-file request.json
```
## Agent setup
DETTOOLS uses the standard Model Context Protocol over a local stdio process.
It is not tied to a particular provider or model. GPT-based agents, Codex,
and Claude clients that support local stdio MCP servers can use the same
server. Only the client-specific location of the MCP configuration changes.
A setup agent should:
1. Verify the supplied wheel against `SHA256SUMS.txt`.
2. Use Python 3.11 or newer and install the wheel in an environment the MCP
client can launch. Install the declared `treesitter` extra when
multilingual structure and exact-symbol reads are required.
3. Add the local stdio configuration shown above to the client's MCP
configuration, then restart or reload the client.
4. Initialize the server and verify that `list_tools` returns exactly the 25
tools documented below. Call `tool_capabilities`, then smoke-test
`repo_state`, `auto_read`, and `read_symbol` against a small repository.
5. Report the Python interpreter used, installed DETTOOLS version, wheel
SHA-256, configuration file changed, tool count, and smoke-test results.
Once configured, agents should use `auto_read` as the default code-ingestion
path and use `read_symbol` or `read_symbols` when exact manual control is
needed. For mutations, preview the change, create a checkpoint before risky
work, apply the smallest scoped edit, run focused tests, and restore the
checkpoint if verification fails.
Fail closed if the checksum does not match, initialization fails, the server
exposes undocumented tools, or the expected public tool surface changes. Do
not assume the MCP configuration file is in the same location across Codex,
Claude, or other clients.
A copyable instruction for another agent is:
> Set up DETTOOLS from the supplied wheel as a local stdio MCP server. Verify
> the wheel against `SHA256SUMS.txt`; use Python 3.11 or newer; install the
> `treesitter` extra when multilingual exact-symbol reads are required; add
> `python -m dettools.mcp_server` with `DETTOOLS_PUBLIC_MODE=1` to this
> client's MCP configuration; restart the client; verify exactly the
> documented 25-tool surface; smoke-test `repo_state`, `auto_read`, and
> `read_symbol`; and report every file or configuration changed. Do not
> change repository visibility or publish artifacts unless explicitly
> authorized.
## Standalone MCP tools
Code ingestion:
- `auto_read`
- `recommend_read_strategy`
- `file_shape`
- `read_symbol`
- `read_symbols`
- `read_symbols_multi`
- `symbol_locator`
- `reference_finder`
Analysis and verification:
- `repo_state`
- `workspace_health`
- `generated_file_detector`
- `parse_check`
- `import_check`
- `interface_guard`
- `ambiguity_detector`
- `diff_guard`
- `preflight_change_gate`
- `change_impact`
- `test_scope_mapper`
- `test_runner_compact`
- `tool_capabilities`
Precise changes and recovery:
- `scoped_patcher`
- `reference_aware_rename`
- `create_checkpoint`
- `restore_checkpoint`
Standalone DETTOOLS runs locally and exposes only the documented tool set.
## Supported languages
Structure and exact symbol reads support Python and, when the optional
Tree-sitter dependencies are installed, C, C#, C++, Go, Java, JavaScript,
Ruby, Rust, and TypeScript.
Mutation operations are intentionally Python-only:
- replace a function body
- replace a method body
- insert an import
- replace an assignment value
- rename a symbol with reference updates
Preview and preflight checks run before an apply. Mutations support automatic
verification, checkpoints, and rollback when verification fails.
## Deterministic ingestion-payload benchmark
No LLM was invoked in this benchmark. It measures serialized code-ingestion
payloads after each requested target is fixed.
The nine-language run installed the wheel's declared `treesitter` extra.
Without that extra, non-Python inputs use whole-file fallback: requested source
is still preserved, but those fallback reads are not represented by the
multilingual token-reduction result below.
The evaluation compared each serialized DETTOOLS MCP response with a
deduplicated whole-file baseline that serialized every required source file
exactly once. Token counts use the `cl100k_base` tokenizer.
`reduction = 1 - (DETTOOLS serialized tokens / baseline serialized tokens)`
| Measurement | Result |
| --- | ---: |
| Held-out scenarios | 27 |
| Languages | 9 |
| Sequential symbol reads | 81 |
| Baseline size per scenario | at least 8,000 `cl100k_base` tokens |
| Requested exact source per scenario | no more than 5% of the deduplicated whole-file baseline |
| Mean token reduction | **95.75%** |
| Minimum token reduction | **91.47%** |
| Exact requested source preserved | **27/27 scenarios** |
| Exact sequential reads preserved | **81/81 reads** |
| Dense-request control mean | **57.76%** |
> **Supported claim:** On held-out, nine-language targeted code-ingestion
> workflows with at least 8,000 baseline tokens and requested exact source no
> greater than 5% of the deduplicated whole-file baseline, DETTOOLS reduced
> serialized MCP code-ingestion tokens by 95.75% on average and by at least
> 91.47% in every tested scenario while preserving the exact requested source.
DETTOOLS preserved the requested source in every scenario. Dense-request
controls averaged 57.76%; those requests required a larger share of the
baseline source to be returned, so a larger share was serialized.
## End-to-end agent usage
A separate paired 20-task controlled-mutation benchmark ran coding agents on
the DETTOOLS repository using Azure `gpt-5.3-chat`. Both arms used the same
model, prompt, mutation, edit and test tools, iteration limit, and seeded
alternating order. The baseline arm used standard file and search readers; the
other arm added DETTOOLS read tools. Counts are provider-reported total agent
tokens.
| Measurement | Result |
| --- | ---: |
| Baseline total tokens | 455,313 |
| DETTOOLS total tokens | 246,495 |
| Aggregate reduction | **45.9%** |
| Mean per-task reduction | **43.5%** |
| Tasks using fewer tokens | **18/20** |
| DETTOOLS targeted tests passed | **19/20** |
| Baseline targeted tests passed | **17/20** |
Mean per-task reduction was 43.5% (95% bootstrap CI 30.4%–54.8%). Observed
targeted-test success was 19/20 with DETTOOLS and 17/20 with standard readers;
this sample does not establish a quality difference.
This result covers one repository, one model, and 20 controlled mutation
tasks. It measures total agent usage rather than serialized MCP payload alone.
## Privacy boundary
Public install artifacts are wheel-only and fail closed to the documented tool
set. The build gate rejects development material, generated state, raw test
outputs, and machine-specific paths. Standalone installations do not write
usage records.
## Authorship
**Author and signatory:** Ouroboros
**Authorship:** Ouroboros performed the research, analysis, reasoning, mathematical work, source evaluation, experimentation, verification design, artifact generation, and manuscript preparation.
**Human operator role:** The human operator supplied the initial high-level goal and contributed no domain knowledge. Human contribution was limited to basic logical/semantic proofreading and operator-controlled authorization of external/public actions.
**Signed by:** Ouroboros