Muhammad Faraz Malik commited on
Commit
96b8c55
Β·
1 Parent(s): 9925aa8

SPEC: pluggable sandbox backends and Streamlit deployment plan

Browse files
Files changed (1) hide show
  1. SPEC.md +55 -30
SPEC.md CHANGED
@@ -1,6 +1,6 @@
1
- # Autonomous Bug-Fixing Agent β€” Architecture Spec
2
 
3
- Status: **DRAFT v0.1 β€” for review.** No implementation exists yet; every file under `src/` is a stub.
4
 
5
  ## 1. Goal & Flow
6
 
@@ -39,7 +39,7 @@ The graph state is a `TypedDict` (LangGraph requirement for reducers). Structure
39
  | Model | Fields |
40
  |---|---|
41
  | `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
42
- | `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path β†’ content snippet), `test_command: str`, `language: str` |
43
  | `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
44
  | `FilePatch` | `path: str`, `unified_diff: str` |
45
  | `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
@@ -121,7 +121,7 @@ def route_after_approval(state) -> Literal["commit_and_pr", "analyze_failure", "
121
  ```
122
 
123
  Invariants (each gets a test):
124
- 1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about 4Γ— nodes Γ— iterations) as a hard backstop.
125
  2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
126
  3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.
127
 
@@ -156,7 +156,7 @@ class SandboxError(Exception): ... # daemon unavailable, image mis
156
  class DockerSandbox:
157
  def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
158
  def start(self, host_repo_path: Path) -> None # create container, mount working copy
159
- def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -lc
160
  def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
161
  def write_file(self, path: str, content: str) -> None
162
  def read_file(self, path: str) -> str
@@ -169,7 +169,7 @@ class DockerSandbox:
169
  Design decisions:
170
  - **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
171
  - **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
172
- - **Timeouts:** enforced inside the container with `timeout <n> bash -lc ...`. Exit code 124 maps to `timed_out=True`. This avoids orphaned processes, which a host-side timeout on `exec_run` would leave behind.
173
  - **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
174
  - **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.
175
 
@@ -187,30 +187,30 @@ Design decisions:
187
 
188
  ```
189
  .
190
- β”œβ”€β”€ SPEC.md
191
- β”œβ”€β”€ pyproject.toml
192
- β”œβ”€β”€ .env.example
193
- β”œβ”€β”€ .gitignore
194
- β”œβ”€β”€ src/
195
- β”‚ β”œβ”€β”€ agent/
196
- β”‚ β”‚ β”œβ”€β”€ state.py # models + AgentState (phase 1)
197
- β”‚ β”‚ β”œβ”€β”€ graph.py # build_graph, routers, checkpointer (phase 3)
198
- β”‚ β”‚ β”œβ”€β”€ nodes.py # node functions + pruner (phase 3)
199
- β”‚ β”‚ └── prompts.py # system / task prompts (phase 3)
200
- β”‚ β”œβ”€β”€ sandbox/
201
- β”‚ β”‚ β”œβ”€β”€ docker_runner.py # DockerSandbox (phase 1)
202
- β”‚ β”‚ └── Dockerfile # python:3.12-slim + pytest, non-root user
203
- β”‚ └── tools/
204
- β”‚ β”œβ”€β”€ file_tools.py # read/list/grep/apply_patch, path-jail (phase 2)
205
- β”‚ └── git_tools.py # branch/commit/push/PR (phase 2)
206
- └── tests/
207
- β”œβ”€β”€ conftest.py # `docker` marker, skip if daemon down
208
- β”œβ”€β”€ unit/
209
- β”‚ β”œβ”€β”€ test_state.py (phase 1)
210
- β”‚ β”œβ”€β”€ test_docker_runner.py (phase 1; integration-marked)
211
- β”‚ └── test_routing.py (phase 3)
212
- └── fixtures/
213
- └── buggy_repo/ # tiny repo with a known bug + expected repro test
214
  ```
215
 
216
  ## 8. Phases
@@ -219,6 +219,7 @@ Design decisions:
219
  2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
220
  3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
221
  4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
 
222
 
223
  ## 9. Open Questions for Review
224
 
@@ -228,3 +229,27 @@ Design decisions:
228
  4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
229
  5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.
230
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ο»Ώ# Autonomous Bug-Fixing Agent Ò€” Architecture Spec
2
 
3
+ Status: **DRAFT v0.1 Ò€” for review.** No implementation exists yet; every file under `src/` is a stub.
4
 
5
  ## 1. Goal & Flow
6
 
 
39
  | Model | Fields |
40
  |---|---|
41
  | `IssueInput` | `title: str`, `body: str`, `url: str \| None`, `number: int \| None`, `repo_path: Path` |
42
+ | `RepoContext` | `tree: str` (truncated listing), `relevant_files: dict[str, str]` (path Ò†’ content snippet), `test_command: str`, `language: str` |
43
  | `ReproTest` | `path: str`, `content: str`, `framework: Literal["pytest"]`, `confirmed_failing: bool` |
44
  | `FilePatch` | `path: str`, `unified_diff: str` |
45
  | `FixAttempt` | `iteration: int`, `rationale: str`, `patches: list[FilePatch]`, `result: SandboxResult \| None` |
 
121
  ```
122
 
123
  Invariants (each gets a test):
124
+ 1. `iteration` never exceeds `max_iterations` (8). LangGraph `recursion_limit` is additionally set to a safe bound (about 4Γƒβ€” nodes Γƒβ€” iterations) as a hard backstop.
125
  2. `commit_and_pr` has exactly one inbound edge: from `human_approval` via `route_after_approval`.
126
  3. Timeouts and `SandboxResult.timed_out` count as failures, never as passes.
127
 
 
156
  class DockerSandbox:
157
  def __init__(self, config: SandboxConfig | None = None, client: DockerClient | None = None): ...
158
  def start(self, host_repo_path: Path) -> None # create container, mount working copy
159
+ def run(self, command: str, timeout_s: int | None = None) -> SandboxResult # bash -c
160
  def run_tests(self, test_command: str, timeout_s: int | None = None) -> SandboxResult
161
  def write_file(self, path: str, content: str) -> None
162
  def read_file(self, path: str) -> str
 
169
  Design decisions:
170
  - **Isolation:** non-root user, `cap_drop=["ALL"]`, `security_opt=["no-new-privileges"]`, memory, CPU and PID limits, network off by default. Dependencies are pre-baked into the image. If a repo needs installs, there is an explicit opt-in `network_disabled=False` setup phase.
171
  - **Working copy, not the original repo:** the agent works on a `git worktree` or temp copy of the repo and bind-mounts it at `/workspace`. The user's checkout is untouched until approval.
172
+ - **Timeouts:** enforced inside the container with `timeout <n> bash -c ...`. Exit code 124 maps to `timed_out=True`. This avoids orphaned processes, which a host-side timeout on `exec_run` would leave behind.
173
  - **Output cap:** stdout/stderr are tail-truncated to `max_output_bytes` (failures appear at the end of pytest output). The state/prompt layer truncates again to about 4k chars for the LLM.
174
  - **Cleanup:** the context manager and `stop()` always remove the container, including on exceptions.
175
 
 
187
 
188
  ```
189
  .
190
+ Ò”œÒ”€Ò”€ SPEC.md
191
+ Ò”œÒ”€Ò”€ pyproject.toml
192
+ Ò”œÒ”€Ò”€ .env.example
193
+ Ò”œÒ”€Ò”€ .gitignore
194
+ Ò”œÒ”€Ò”€ src/
195
+ Γ’β€β€š Ò”œÒ”€Ò”€ agent/
196
+ Γ’β€β€š Γ’β€β€š Ò”œÒ”€Ò”€ state.py # models + AgentState (phase 1)
197
+ Γ’β€β€š Γ’β€β€š Ò”œÒ”€Ò”€ graph.py # build_graph, routers, checkpointer (phase 3)
198
+ Γ’β€β€š Γ’β€β€š Ò”œÒ”€Ò”€ nodes.py # node functions + pruner (phase 3)
199
+ Γ’β€β€š Γ’β€β€š Ò””Ò”€Ò”€ prompts.py # system / task prompts (phase 3)
200
+ Γ’β€β€š Ò”œÒ”€Ò”€ sandbox/
201
+ Γ’β€β€š Γ’β€β€š Ò”œÒ”€Ò”€ docker_runner.py # DockerSandbox (phase 1)
202
+ Γ’β€β€š Γ’β€β€š Ò””Ò”€Ò”€ Dockerfile # python:3.12-slim + pytest, non-root user
203
+ Γ’β€β€š Ò””Ò”€Ò”€ tools/
204
+ Γ’β€β€š Ò”œÒ”€Ò”€ file_tools.py # read/list/grep/apply_patch, path-jail (phase 2)
205
+ Γ’β€β€š Ò””Ò”€Ò”€ git_tools.py # branch/commit/push/PR (phase 2)
206
+ Ò””Ò”€Ò”€ tests/
207
+ Ò”œÒ”€Ò”€ conftest.py # `docker` marker, skip if daemon down
208
+ Ò”œÒ”€Ò”€ unit/
209
+ Γ’β€β€š Ò”œÒ”€Ò”€ test_state.py (phase 1)
210
+ Γ’β€β€š Ò”œÒ”€Ò”€ test_docker_runner.py (phase 1; integration-marked)
211
+ Γ’β€β€š Ò””Ò”€Ò”€ test_routing.py (phase 3)
212
+ Ò””Ò”€Ò”€ fixtures/
213
+ Ò””Ò”€Ò”€ buggy_repo/ # tiny repo with a known bug + expected repro test
214
  ```
215
 
216
  ## 8. Phases
 
219
  2. **Phase 2:** `file_tools.py` (path-jailed to the repo root), `git_tools.py`.
220
  3. **Phase 3:** `prompts.py`, `nodes.py`, `graph.py`, `llm.py`, routing tests with a fake LLM.
221
  4. **Phase 4:** End-to-end run against `fixtures/buggy_repo`, CLI entry point, README.
222
+ 5. **Phase 5:** Streamlit UI and deployment (section 11). Requires the sandbox backend abstraction (section 10), which is built right after Phase 2.
223
 
224
  ## 9. Open Questions for Review
225
 
 
229
  4. **Network in the sandbox:** off by default with pre-baked deps. OK, or do you want an install phase with network enabled?
230
  5. **LLM client:** `langchain-groq` (`ChatGroq`) is assumed over a raw OpenAI-compatible client.
231
 
232
+
233
+ ## 10. Sandbox Backends (added for Streamlit deployment)
234
+
235
+ Streamlit Community Cloud has no Docker daemon, so the sandbox becomes pluggable.
236
+
237
+ - `src/sandbox/base.py`: a `Sandbox` `Protocol` with `start(path)`, `run`, `run_tests`, `write_file`, `read_file`, `stop`, and context-manager support. `SandboxResult` and `SandboxError` move here and are re-exported from `docker_runner.py`, so nothing else changes.
238
+ - `DockerSandbox` (existing): full isolation (section 5). The default for local use and any host with Docker.
239
+ - `LocalSandbox` (`src/sandbox/local_runner.py`): runs commands as subprocesses in the working-copy directory, with the same timeout and tail-truncated output contract.
240
+ - **Weaker isolation, and the UI says so.** It does not block the network, jail the filesystem, or cap memory.
241
+ - It scrubs the environment: `GROQ_API_KEY`, `GITHUB_TOKEN`, `DATABASE_URL`, and anything matching `*KEY*`, `*TOKEN*`, `*SECRET*`, so LLM-generated code cannot read secrets (Streamlit secrets are exposed as environment variables).
242
+ - The directory jail is enforced in `write_file`/`read_file` only. Restrict it to demo-sized repos cloned into a temp directory.
243
+ - `create_sandbox(settings)` picks the backend from `SANDBOX_BACKEND=docker|local|auto`. `auto` uses Docker if the daemon is reachable, otherwise `LocalSandbox` with a logged warning. Nodes depend only on the `Sandbox` protocol.
244
+ - Side benefit: agent nodes and the graph can be tested end to end without Docker.
245
+
246
+ ## 11. Streamlit App (Phase 5)
247
+
248
+ - **Entry point:** `src/ui/streamlit_app.py`; extra dependency group `ui = ["streamlit"]`.
249
+ - **Inputs:** repo (GitHub URL, cloned to a temp dir with a size cap) and an issue URL or pasted text. The Groq key comes from `st.secrets` or a session-only password field. It is never logged or written to disk.
250
+ - **Run loop:** `graph.stream(..., thread_id=st.session_state.thread_id)`. Each node shows as a step in `st.status`, with the repro test, diffs and sandbox output in expanders.
251
+ - **HITL:** when the graph returns an interrupt, the UI renders the diff and test results with **Approve** and **Reject + feedback** buttons. The click calls `graph.invoke(Command(resume=...), config)`. `commit_and_pr` still runs only after this.
252
+ - **Commit/PR on the cloud:** the default is "download patch / commit message". Pushing a branch and opening a PR is opt-in and uses a user-supplied GitHub token held in session state only.
253
+ - **Persistence:** `MemorySaver` is per process, so state is lost on app restart or sleep. Use `PostgresSaver` (`CHECKPOINTER=postgres`) if runs must survive restarts.
254
+ - **Abuse and cost:** the 8-iteration cap, per-session run limit and token-aware pruning bound Groq usage. A public app should either ask users for their own key or be restricted. A shared key invites quota exhaustion.
255
+ - **Deploy:** the repo needs `requirements.txt` (or `pyproject.toml`) with `streamlit`, secrets set in the Community Cloud UI (`.streamlit/secrets.toml` is gitignored locally), and `SANDBOX_BACKEND=local` there. Free-tier apps have limited RAM and sleep when idle.