Muhammad Faraz Malik commited on
Commit Β·
96b8c55
1
Parent(s): 9925aa8
SPEC: pluggable sandbox backends and Streamlit deployment plan
Browse files
SPEC.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
-
# Autonomous Bug-Fixing Agent
|
| 2 |
|
| 3 |
-
Status: **DRAFT v0.1
|
| 4 |
|
| 5 |
## 1. Goal & Flow
|
| 6 |
|
|
@@ -39,7 +39,7 @@ The graph state is a `TypedDict` (LangGraph requirement for reducers). Structure
|
|
| 39 |
| Model | Fields |
|
| 40 |
|---|---|
|
| 41 |
| `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
|
| 42 |
-
| `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path
|
| 43 |
| `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
|
| 44 |
| `FilePatch` | `path: str`, `unified_diff: str` |
|
| 45 |
| `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
|
|
@@ -121,7 +121,7 @@ def route_after_approval(state) -> Literal["commit_and_pr", "analyze_failure", "
|
|
| 121 |
```
|
| 122 |
|
| 123 |
Invariants (each gets a test):
|
| 124 |
-
1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about
|
| 125 |
2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
|
| 126 |
3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.
|
| 127 |
|
|
@@ -156,7 +156,7 @@ class SandboxError(Exception): ... # daemon unavailable, image mis
|
|
| 156 |
class DockerSandbox:
|
| 157 |
def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
|
| 158 |
def start(self, host_repo_path: Path) -> None # create container, mount working copy
|
| 159 |
-
def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -
|
| 160 |
def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
|
| 161 |
def write_file(self, path: str, content: str) -> None
|
| 162 |
def read_file(self, path: str) -> str
|
|
@@ -169,7 +169,7 @@ class DockerSandbox:
|
|
| 169 |
Design decisions:
|
| 170 |
- **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
|
| 171 |
- **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
|
| 172 |
-
- **Timeouts:** enforced inside the container with `timeout <n> bash -
|
| 173 |
- **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
|
| 174 |
- **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.
|
| 175 |
|
|
@@ -187,30 +187,30 @@ Design decisions:
|
|
| 187 |
|
| 188 |
```
|
| 189 |
.
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
```
|
| 215 |
|
| 216 |
## 8. Phases
|
|
@@ -219,6 +219,7 @@ Design decisions:
|
|
| 219 |
2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
|
| 220 |
3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
|
| 221 |
4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
|
|
|
|
| 222 |
|
| 223 |
## 9. Open Questions for Review
|
| 224 |
|
|
@@ -228,3 +229,27 @@ Design decisions:
|
|
| 228 |
4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
|
| 229 |
5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.
|
| 230 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
ο»Ώ# Autonomous Bug-Fixing Agent Γ’β¬β Architecture Spec
|
| 2 |
|
| 3 |
+
Status: **DRAFT v0.1 Γ’β¬β for review.** No implementation exists yet; every file under `src/` is a stub.
|
| 4 |
|
| 5 |
## 1. Goal & Flow
|
| 6 |
|
|
|
|
| 39 |
| Model | Fields |
|
| 40 |
|---|---|
|
| 41 |
| `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
|
| 42 |
+
| `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path Γ’β β content snippet), `test_command: str`, `language: str` |
|
| 43 |
| `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
|
| 44 |
| `FilePatch` | `path: str`, `unified_diff: str` |
|
| 45 |
| `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
|
|
|
|
| 121 |
```
|
| 122 |
|
| 123 |
Invariants (each gets a test):
|
| 124 |
+
1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about 4Γβ nodes Γβ iterations) as a hard backstop.
|
| 125 |
2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
|
| 126 |
3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.
|
| 127 |
|
|
|
|
| 156 |
class DockerSandbox:
|
| 157 |
def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
|
| 158 |
def start(self, host_repo_path: Path) -> None # create container, mount working copy
|
| 159 |
+
def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -c
|
| 160 |
def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
|
| 161 |
def write_file(self, path: str, content: str) -> None
|
| 162 |
def read_file(self, path: str) -> str
|
|
|
|
| 169 |
Design decisions:
|
| 170 |
- **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
|
| 171 |
- **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
|
| 172 |
+
- **Timeouts:** enforced inside the container with `timeout <n> bash -c ...`. Exit code 124 maps to `timed_out=True`. This avoids orphaned processes, which a host-side timeout on `exec_run` would leave behind.
|
| 173 |
- **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
|
| 174 |
- **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.
|
| 175 |
|
|
|
|
| 187 |
|
| 188 |
```
|
| 189 |
.
|
| 190 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ SPEC.md
|
| 191 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ pyproject.toml
|
| 192 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ .env.example
|
| 193 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ .gitignore
|
| 194 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ src/
|
| 195 |
+
Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ agent/
|
| 196 |
+
Γ’ββ Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ state.py # models + AgentState (phase 1)
|
| 197 |
+
Γ’ββ Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ graph.py # build_graph, routers, checkpointer (phase 3)
|
| 198 |
+
Γ’ββ Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ nodes.py # node functions + pruner (phase 3)
|
| 199 |
+
Γ’ββ Γ’ββ Γ’ββΓ’ββ¬Γ’ββ¬ prompts.py # system / task prompts (phase 3)
|
| 200 |
+
Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ sandbox/
|
| 201 |
+
Γ’ββ Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ docker_runner.py # DockerSandbox (phase 1)
|
| 202 |
+
Γ’ββ Γ’ββ Γ’ββΓ’ββ¬Γ’ββ¬ Dockerfile # python:3.12-slim + pytest, non-root user
|
| 203 |
+
Γ’ββ Γ’ββΓ’ββ¬Γ’ββ¬ tools/
|
| 204 |
+
Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ file_tools.py # read/list/grep/apply_patch, path-jail (phase 2)
|
| 205 |
+
Γ’ββ Γ’ββΓ’ββ¬Γ’ββ¬ git_tools.py # branch/commit/push/PR (phase 2)
|
| 206 |
+
Γ’ββΓ’ββ¬Γ’ββ¬ tests/
|
| 207 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ conftest.py # `docker` marker, skip if daemon down
|
| 208 |
+
Γ’βΕΓ’ββ¬Γ’ββ¬ unit/
|
| 209 |
+
Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ test_state.py (phase 1)
|
| 210 |
+
Γ’ββ Γ’βΕΓ’ββ¬Γ’ββ¬ test_docker_runner.py (phase 1; integration-marked)
|
| 211 |
+
Γ’ββ Γ’ββΓ’ββ¬Γ’ββ¬ test_routing.py (phase 3)
|
| 212 |
+
Γ’ββΓ’ββ¬Γ’ββ¬ fixtures/
|
| 213 |
+
Γ’ββΓ’ββ¬Γ’ββ¬ buggy_repo/ # tiny repo with a known bug + expected repro test
|
| 214 |
```
|
| 215 |
|
| 216 |
## 8. Phases
|
|
|
|
| 219 |
2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
|
| 220 |
3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
|
| 221 |
4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
|
| 222 |
+
5. **Phase 5:** Streamlit UI and deployment (section 11). Requires the sandbox backend abstraction (section 10), which is built right after Phase 2.
|
| 223 |
|
| 224 |
## 9. Open Questions for Review
|
| 225 |
|
|
|
|
| 229 |
4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
|
| 230 |
5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.
|
| 231 |
|
| 232 |
+
|
| 233 |
+
## 10. Sandbox Backends (added for Streamlit deployment)
|
| 234 |
+
|
| 235 |
+
Streamlit Community Cloud has no Docker daemon, so the sandbox becomes pluggable.
|
| 236 |
+
|
| 237 |
+
- `src/sandbox/base.py`: a `Sandbox` `Protocol` with `start(path)`, `run`, `run_tests`, `write_file`, `read_file`, `stop`, and context-manager support. `SandboxResult` and `SandboxError` move here and are re-exported from `docker_runner.py`, so nothing else changes.
|
| 238 |
+
- `DockerSandbox` (existing): full isolation (section 5). The default for local use and any host with Docker.
|
| 239 |
+
- `LocalSandbox` (`src/sandbox/local_runner.py`): runs commands as subprocesses in the working-copy directory, with the same timeout and tail-truncated output contract.
|
| 240 |
+
- **Weaker isolation, and the UI says so.** It does not block the network, jail the filesystem, or cap memory.
|
| 241 |
+
- It scrubs the environment: `GROQ_API_KEY`, `GITHUB_TOKEN`, `DATABASE_URL`, and anything matching `*KEY*`, `*TOKEN*`, `*SECRET*`, so LLM-generated code cannot read secrets (Streamlit secrets are exposed as environment variables).
|
| 242 |
+
- The directory jail is enforced in `write_file`/`read_file` only. Restrict it to demo-sized repos cloned into a temp directory.
|
| 243 |
+
- `create_sandbox(settings)` picks the backend from `SANDBOX_BACKEND=docker|local|auto`. `auto` uses Docker if the daemon is reachable, otherwise `LocalSandbox` with a logged warning. Nodes depend only on the `Sandbox` protocol.
|
| 244 |
+
- Side benefit: agent nodes and the graph can be tested end to end without Docker.
|
| 245 |
+
|
| 246 |
+
## 11. Streamlit App (Phase 5)
|
| 247 |
+
|
| 248 |
+
- **Entry point:** `src/ui/streamlit_app.py`; extra dependency group `ui = ["streamlit"]`.
|
| 249 |
+
- **Inputs:** repo (GitHub URL, cloned to a temp dir with a size cap) and an issue URL or pasted text. The Groq key comes from `st.secrets` or a session-only password field. It is never logged or written to disk.
|
| 250 |
+
- **Run loop:** `graph.stream(..., thread_id=st.session_state.thread_id)`. Each node shows as a step in `st.status`, with the repro test, diffs and sandbox output in expanders.
|
| 251 |
+
- **HITL:** when the graph returns an interrupt, the UI renders the diff and test results with **Approve** and **Reject + feedback** buttons. The click calls `graph.invoke(Command(resume=...), config)`. `commit_and_pr` still runs only after this.
|
| 252 |
+
- **Commit/PR on the cloud:** the default is "download patch / commit message". Pushing a branch and opening a PR is opt-in and uses a user-supplied GitHub token held in session state only.
|
| 253 |
+
- **Persistence:** `MemorySaver` is per process, so state is lost on app restart or sleep. Use `PostgresSaver` (`CHECKPOINTER=postgres`) if runs must survive restarts.
|
| 254 |
+
- **Abuse and cost:** the 8-iteration cap, per-session run limit and token-aware pruning bound Groq usage. A public app should either ask users for their own key or be restricted. A shared key invites quota exhaustion.
|
| 255 |
+
- **Deploy:** the repo needs `requirements.txt` (or `pyproject.toml`) with `streamlit`, secrets set in the Community Cloud UI (`.streamlit/secrets.toml` is gitignored locally), and `SANDBOX_BACKEND=local` there. Free-tier apps have limited RAM and sleep when idle.
|