Spaces:
Runtime error
Runtime error
| title: Diffcontext | |
| emoji: 🚀 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: docker | |
| app_port: 7860 | |
| # DiffContext | |
| Static-analysis-powered repository context compiler for LLMs. | |
| **Git diff + AST parsing + dependency graph + blast radius + impact scoring → optimized context package** | |
| Instead of dumping an entire codebase (or doing keyword/vector search), DiffContext: | |
| 1. **Parses** Python files via AST → extracts every function, method, class | |
| 2. **Builds a dependency graph** → calls, imports, inheritance, attribute ownership, decorators | |
| 3. **Detects changes** → via git diff (including uncommitted edits) or snapshot comparison | |
| 4. **Computes blast radius** → everything transitively affected by the change | |
| 5. **Scores impact** → prioritizes symbols by structural importance | |
| 6. **Selects relevant code** → respects a token budget | |
| 7. **Compiles context** → structured output ready to paste into an LLM | |
| On a real ~1,100-symbol production repo, this reliably produces **95–99% token | |
| reduction** versus pasting the whole codebase, while keeping the functions that | |
| are actually call-graph-connected to your change. | |
| ## Quick Start | |
| ### Step 1: Install | |
| ```bash | |
| git clone https://github.com/trakshan-mishra/Diffcontext.git | |
| cd Diffcontext | |
| pip install -e . | |
| diffcontext --help | |
| ``` | |
| ### Step 2: Index a repository | |
| ```bash | |
| diffcontext index /path/to/any/python/project | |
| ``` | |
| If any file fails to parse (a real `SyntaxError`), it's reported explicitly — | |
| not silently skipped: | |
| ``` | |
| Skipping broken_file.py due to SyntaxError: unmatched ')' (line 237) | |
| ``` | |
| ### Step 3: Check impact of a function you're working on | |
| This is the recommended default mode while actively editing — it doesn't | |
| depend on git at all, so it works on uncommitted or untracked files too: | |
| ```bash | |
| diffcontext blast --changed ./src/auth.py:validate_jwt | |
| ``` | |
| Shows who calls it, what it calls, and the full transitive blast radius. | |
| ### Step 4: Auto-detect changes from git (optional) | |
| ```bash | |
| diffcontext diff | |
| ``` | |
| Compares your working tree (including uncommitted edits to **tracked** files) | |
| against `HEAD~1` by default. Note: this only sees changes to files git | |
| already knows about — a brand-new untracked file is invisible to any | |
| git-diff-based tool until you `git add -N <file>` or commit it. Use | |
| `--committed-only` to compare two commits and ignore working-tree changes. | |
| ### Step 5: Build LLM-ready context | |
| ```bash | |
| # From a specific function: | |
| diffcontext compile --changed ./src/auth.py:validate_jwt | |
| # From git diff: | |
| diffcontext compile --ref HEAD~1 | |
| # With a token budget: | |
| diffcontext compile --changed ./src/auth.py:validate_jwt --max-tokens 8000 | |
| # JSON output (for piping into another tool): | |
| diffcontext compile --changed ./src/auth.py:validate_jwt --json | |
| ``` | |
| Then paste the output into Claude / ChatGPT / your LLM of choice, **with a | |
| specific question** — not just the raw context. E.g.: | |
| > "Is the dynamic SQL construction in `update_run` safe, given how `kwargs` | |
| > is validated against `_UPDATABLE_RUN_COLUMNS`?" | |
| ### Step 6: Use as a library | |
| ```python | |
| from diffcontext.pipeline import index_repository, analyze_impact, compile | |
| idx = index_repository("/path/to/repo") | |
| impact = analyze_impact(idx, ["./src/auth.py:validate_jwt"]) | |
| ctx = compile(idx, impact, max_tokens=10000) | |
| print(ctx.text) # the context to send to the LLM | |
| print(f"{ctx.token_estimate:,} / {ctx.total_repo_tokens:,} tokens") | |
| print(f"{ctx.reduction_pct:.1f}% reduction") | |
| ``` | |
| See `USAGE.md` for the full day-to-day workflow, including shell aliases. | |
| ### Step 7: Cloud sync (CtxSync) (yet to be impemented) | |
| ```bash | |
| diffcontext sync | |
| ``` | |
| One command. Compiles blast radius and pushes to your CtxSync cloud endpoint. | |
| Credentials are read from `~/.ctxsync`, env vars, or `--url`/`--key` flags. | |
| ### Step 8: Use as an MCP Server (Claude Desktop / Cursor) | |
| DiffContext includes a built-in **Model Context Protocol (MCP)** server, allowing AI assistants to natively query your codebase's blast radius without manual copy-pasting. | |
| **1. Install with MCP support:** | |
| ```bash | |
| pip install -e .[mcp] | |
| ``` | |
| **2. Configure your AI client:** | |
| For Claude Desktop (`claude_desktop_config.json`) or Cursor: | |
| ```json | |
| { | |
| "mcpServers": { | |
| "diffcontext": { | |
| "command": "diffcontext-mcp" | |
| } | |
| } | |
| } | |
| ``` | |
| Now you can just ask your AI: *"What is the blast radius of validate_jwt in the diffcontext repo?"* and it will autonomously use DiffContext to find the precise context! | |
| ## What the resolver actually handles | |
| Confirmed via an automated test suite (`tests/`) that builds small repos on | |
| the fly and asserts on real resolved call-graph edges — not just "it ran | |
| without crashing": | |
| - Function and method calls, including multi-hop attribute chains (`self.a.b.method()`) | |
| - Multiple inheritance / MRO, including cross-file base classes | |
| - Circular imports | |
| - Local variables instantiating a class inside a **free function** (not just | |
| `self.x = ...` inside a method) — e.g. `h = Handler(); h.process()` | |
| - Annotated parameters as call receivers (`def run(h: Handler): h.process()`) | |
| - Import aliasing (`from .user import Handler as UserHandler`), including | |
| disambiguating two same-named classes in different files | |
| - Bare `import x` where `x` lives in a sibling directory rather than the | |
| repo root (common in script-style codebases) | |
| - **Decorators**: a decorated function's graph entry now correctly includes | |
| calls made by its decorator's wrapper — e.g. `@require_auth` wrapping | |
| `get_profile` correctly shows `get_profile` depending on whatever | |
| `require_auth`'s wrapper calls (like a session check), not falsely | |
| attributed to `require_auth` itself | |
| - Higher-order stdlib functions: `map(fn, items)`, `sorted(x, key=fn)`, | |
| `filter(fn, items)` — a function passed *by reference* to these is | |
| tracked as an implicit call | |
| ## Known limitations (genuinely unfixable by static analysis, not bugs) | |
| - **Dynamic dispatch / `getattr()`-based routing**: `getattr(obj, name)()` | |
| can't be resolved statically when `name` is computed at runtime (from | |
| config, user input, etc.) — no static analysis tool can do this in | |
| general, including IDEs. | |
| - **Cross-file changes related by theme, not by function calls**: e.g. "remove | |
| a dependency," touching 3 files for one conceptual reason with no direct | |
| call-graph edges between them. Blast radius is a call-graph tool; it | |
| cannot detect relatedness that isn't expressed as a function call. | |
| - **User-defined higher-order functions**: only the common stdlib cases | |
| (`map`, `filter`, `sorted`/`max`/`min` with `key=`) are recognized. | |
| A custom function like `def apply_twice(fn, value): return fn(fn(value))` | |
| is not — this would need cross-function signature analysis to know which | |
| parameter is expected to be callable. | |
| Run `grep -rn "function_name(" --include="*.py" .` to spot-check anything | |
| important before fully trusting "no callers found." | |
| ## Architecture | |
| ``` | |
| diffcontext/ | |
| ├── __init__.py # Package entry, high-level API | |
| ├── models.py # Data classes (Symbol, RepositoryIndex, etc.) | |
| ├── scanner.py # File discovery with exclusion list | |
| ├── parser.py # AST symbol extraction | |
| ├── resolver.py # Import -> filesystem path resolution | |
| ├── symbols.py # Attribute / local-var type tracking | |
| ├── graph_builder.py # Core: dependency graph construction | |
| ├── pipeline.py # Pipeline orchestrator | |
| ├── _warn_once.py # De-duplicated warnings (broken files, encoding, unknown symbols) | |
| ├── diff/ | |
| │ ├── git_diff.py # Git diff -> changed symbols | |
| │ └── state_manager.py # Snapshot-based change detection | |
| ├── impact/ | |
| │ ├── blast_radius.py # Reverse graph traversal | |
| │ ├── scoring.py # Impact scoring | |
| │ ├── traversal.py # Forward dependency expansion | |
| │ └── visualizer.py # Terminal tree rendering | |
| ├── context/ | |
| │ ├── selector.py # Token-budget-aware selection | |
| │ └── compiler.py # Structured output formatting | |
| └── cli/ | |
| └── __init__.py # CLI: index, impact, diff, compile, blast, sync | |
| ``` | |
| ## Try it (30 seconds) | |
| ```bash | |
| bash demos.sh # interactive — pick from 5 famous repos or use your own | |
| ``` | |
| ## How symbol IDs work | |
| Every function gets a unique ID: `./relative/path.py:ClassName.method_name` | |
| ``` | |
| ./src/auth.py:validate_jwt | |
| ./src/flask/app.py:Flask.route | |
| ``` | |
| **No parentheses, no arguments** — `validate_jwt`, never `validate_jwt(token)`. | |
| ## Testing | |
| ```bash | |
| python3 -m pytest tests/ -v | |
| ``` | |
| 17 tests, all self-contained (no external clone needed), covering both | |
| correct resolution and the documented limitations above — including tests | |
| that were written to fail loudly if a future change silently regresses | |
| something that's currently working. | |
| ## Status | |
| This is a personal project, built and iteratively debugged against real | |
| production codebases (openai/whisper, pallets/click, pallets/flask). Several | |
| real resolver bugs were found and fixed through dogfooding — decorators, | |
| higher-order functions, and sibling-directory imports all required fixes | |
| that were only visible on real code, not toy examples. Treat blast-radius | |
| output as a strong starting point, not a guarantee, and spot-check with | |
| `grep` on anything load-bearing. | |
| ## License | |
| MIT | |
| ## Benchmarks & Performance (Baseline) | |
| We adhere strictly to **Measure Before Optimizing**. Our baseline metrics demonstrate that traversal and compilation are nearly instantaneous, while parsing and graph construction are the primary bottlenecks. This data drives our roadmap for v0.4 (Incremental Caching). | |
| | Repo | Files | Symbols | Parse (ms) | Graph Build (ms) | Traversal (ms) | Compile (ms) | Token Reduction | | |
| | --- | --- | --- | --- | --- | --- | --- | --- | | |
| | **Flask** | 20 | 354 | 488 | 930 | 0.1 | 0.0 | 98.15% | | |
| | **Click** | 19 | 506 | 1273 | 1824 | 0.2 | 0.0 | 96.51% | | |
| | **HTTPX** | 21 | 434 | 665 | 1147 | 0.1 | 0.0 | 96.70% | | |
| | **Pydantic** | 90 | 1826 | 4519 | 7914 | 0.6 | 0.0 | 98.43% | | |
| *Note: Peak memory for Pydantic (the largest repo) was only 66.4 MB, validating that memory is not currently a bottleneck.* | |