| Overview of Protocol defined in [protocol.rs](../protocol/src/protocol.rs) and [agent.rs](../core/src/agent.rs). |
|
|
| The goal of this document is to define terminology used in the system and explain the expected behavior of the system. |
|
|
| NOTE: The code might not completely match this spec. There are a few minor changes that need to be made after this spec has been reviewed, which will not alter the existing TUI's functionality. |
|
|
| ## Entities |
|
|
| These are entities exit on the codex backend. The intent of this section is to establish vocabulary and construct a shared mental model for the `Codex` core system. |
|
|
| 0. `Model` |
| - In our case, this is the Responses REST API |
| 1. `Codex` |
| - The core engine of codex |
| - Runs locally, either in a background thread or separate process |
| - Communicated to via a queue pair – SQ (Submission Queue) / EQ (Event Queue) |
| - Takes user input, makes requests to the `Model`, executes commands and applies patches. |
| 2. `Session` |
| - The `Codex`'s current configuration and state |
| - `Codex` starts with no `Session`, and it is initialized by `Op::ConfigureSession`, which should be the first message sent by the UI. |
| - The current `Session` can be reconfigured with additional `Op::ConfigureSession` calls. |
| - Any running execution is aborted when the session is reconfigured. |
| 3. `Task` |
| - A `Task` is `Codex` executing work in response to user input. |
| - `Session` has at most one `Task` running at a time. |
| - Receiving user turn input starts a `Task` |
| - Consists of a series of `Turn`s |
| - The `Task` executes to until: |
| - The `Model` completes the task and there is no output to feed into an additional `Turn` |
| - Additional user-turn input aborts the current task and starts a new one |
| - UI interrupts with `Op::Interrupt` |
| - Fatal errors are encountered, eg. `Model` connection exceeding retry limits |
| - Blocked by user approval (executing a command or patch) |
| 4. `Turn` |
| - One cycle of iteration in a `Task`, consists of: |
| - A request to the `Model` - (initially) prompt + (optional) `last_response_id`, or (in loop) previous turn output |
| - The `Model` streams responses back in an SSE, which are collected until "completed" message and the SSE terminates |
| - `Codex` then executes command(s), applies patch(es), and outputs message(s) returned by the `Model` |
| - Pauses to request approval when necessary |
| - The output of one `Turn` is the input to the next `Turn` |
| - A `Turn` yielding no output terminates the `Task` |
|
|
| The term "UI" is used to refer to the application driving `Codex`. This may be the CLI / TUI chat-like interface that users operate, or it may be a GUI interface like a VSCode extension. The UI is external to `Codex`, as `Codex` is intended to be operated by arbitrary UI implementations. |
|
|
| When a `Turn` completes, the `response_id` from the `Model`'s final `response.completed` message is stored in the `Session` state to resume the thread given the next user turn. The `response_id` is also returned in the `EventMsg::TurnComplete` to the UI, which can be used to fork the thread from an earlier point by providing it in a future user turn. |
|
|
| Since only 1 `Task` can be run at a time, for parallel tasks it is recommended that a single `Codex` be run for each thread of work. |
|
|
| ## Interface |
|
|
| - `Codex` |
| - Communicates with UI via a `SQ` (Submission Queue) and `EQ` (Event Queue). |
| - `Submission` |
| - These are messages sent on the `SQ` (UI -> `Codex`) |
| - Has an string ID provided by the UI, referred to as `sub_id` |
| - `Op` refers to the enum of all possible `Submission` payloads |
| - In the current codebase these are primarily in-process Rust types rather than a stable serde wire contract |
| - This enum is `non_exhaustive`; variants can be added at future dates |
| - `Event` |
| - These are messages sent on the `EQ` (`Codex` -> UI) |
| - Each `Event` has a non-unique ID, matching the `sub_id` from the user-turn op that started the current task. |
| - `EventMsg` refers to the enum of all possible `Event` payloads |
| - This enum is `non_exhaustive`; variants can be added at future dates |
| - It should be expected that new `EventMsg` variants will be added over time to expose more detailed information about the model's actions. |
|
|
| For complete documentation of the `Op` and `EventMsg` variants, refer to [protocol.rs](../protocol/src/protocol.rs). Some example payload types: |
|
|
| - `Op` |
| - `Op::UserTurn` – Any input from the user to kick off a `Turn`, including full per-turn context such as cwd, model, sandbox, approval policy, and optional `approvals_reviewer` |
| - `Op::Interrupt` – Interrupts a running turn |
| - `Op::ExecApproval` – Approve or deny code execution |
| - `Op::UserInputAnswer` – Provide answers for a `request_user_input` tool call |
|
|
| - `EventMsg` |
| - `EventMsg::AgentMessage` – Messages from the `Model` |
| - `EventMsg::AgentMessageContentDelta` – Streaming assistant text |
| - `EventMsg::PlanDelta` – Streaming proposed plan text when the model emits a `<proposed_plan>` block in plan mode |
| - `EventMsg::ExecApprovalRequest` – Request approval from user to execute a command |
| - `EventMsg::RequestUserInput` – Request user input for a tool call (questions can include options plus `isOther` to add a free-form choice) |
| - `EventMsg::TurnStarted` – Turn start metadata including `model_context_window` and `collaboration_mode_kind` |
| - `EventMsg::TurnComplete` – A turn completed successfully |
| - `EventMsg::Error` – A turn stopped with an error |
| - `EventMsg::Warning` – A non-fatal warning that the client should surface to the user |
| - `EventMsg::TurnComplete` – Contains a `response_id` bookmark for last `response_id` executed by the turn. This can be used to continue the turn at a later point in time, perhaps with additional user input. |
|
|
| ### UserInput items |
|
|
| `Op::UserTurn` content items can include: |
|
|
| - `text` – Plain text plus optional UI text elements. |
| - `image` / `local_image` – Image inputs. |
| - `skill` – Explicit skill selection (`name`, `path` to `SKILL.md`). |
| - `mention` – Explicit app/connector selection (`name`, `path` in `app://{connector_id}` form). |
|
|
| Note: For v1 wire compatibility, `EventMsg::TurnStarted` and `EventMsg::TurnComplete` serialize as `task_started` / `task_complete`. The deserializer accepts both `task_*` and `turn_*` tags. |
|
|
| The `response_id` returned from each turn matches the OpenAI `response_id` stored in the API's `/responses` endpoint. It can be stored and used in future `Sessions` to resume threads of work. |
|
|
| ## Transport |
|
|
| Can operate over any transport that supports bi-directional streaming. - cross-thread channels - IPC channels - stdin/stdout - TCP - HTTP2 - gRPC |
|
|
| Events still serialize cleanly to newline-delimited JSON for non-framed transports, such as stdin/stdout and TCP. Submission payloads should be treated as implementation details unless a specific transport owns an explicit adapter. |
|
|
| ## Example Flows |
|
|
| Sequence diagram examples of common interactions. In each diagram, some unimportant events may be eliminated for simplicity. |
|
|
| ### Basic UI Flow |
|
|
| A single user input, followed by a 2-turn task |
|
|
| ```mermaid |
| sequenceDiagram |
| box UI |
| participant user as User |
| end |
| box Daemon |
| participant codex as Codex |
| participant session as Session |
| participant task as Task |
| end |
| box Rest API |
| participant agent as Model |
| end |
| user->>codex: Op::ConfigureSession |
| codex-->>session: create session |
| codex->>user: Event::SessionConfigured |
| user->>session: Op::UserTurn |
| session-->>+task: start task |
| task->>user: Event::TurnStarted |
| task->>agent: prompt |
| agent->>task: response (exec) |
| task->>-user: Event::ExecApprovalRequest |
| user->>+task: Op::ExecApproval::Allow |
| task->>user: Event::ExecStart |
| task->>task: exec |
| task->>user: Event::ExecStop |
| task->>user: Event::TurnComplete |
| task->>agent: stdout |
| agent->>task: response (patch) |
| task->>task: apply patch (auto-approved) |
| task->>agent: success |
| agent->>task: response<br/>(msg + completed) |
| task->>user: Event::AgentMessage |
| task->>user: Event::TurnComplete |
| task->>-user: Event::TurnComplete |
| ``` |
|
|
| ### Task Interrupt |
|
|
| Interrupting a task and continuing with additional user input. |
|
|
| ```mermaid |
| sequenceDiagram |
| box UI |
| participant user as User |
| end |
| box Daemon |
| participant session as Session |
| participant task1 as Task1 |
| participant task2 as Task2 |
| end |
| box Rest API |
| participant agent as Model |
| end |
| user->>session: Op::UserTurn |
| session-->>+task1: start task |
| task1->>user: Event::TurnStarted |
| task1->>agent: prompt |
| agent->>task1: response (exec) |
| task1->>task1: exec (auto-approved) |
| task1->>user: Event::TurnComplete |
| task1->>agent: stdout |
| task1->>agent: response (exec) |
| task1->>task1: exec (auto-approved) |
| user->>task1: Op::Interrupt |
| task1->>-user: Event::Error("interrupted") |
| user->>session: Op::UserTurn w/ response bookmark |
| session-->>+task2: start task |
| task2->>user: Event::TurnStarted |
| task2->>agent: prompt + Task1 last_response_id |
| agent->>task2: response (exec) |
| task2->>task2: exec (auto-approve) |
| task2->>user: Event::TurnComplete |
| task2->>agent: stdout |
| agent->>task2: msg + completed |
| task2->>user: Event::AgentMessage |
| task2->>user: Event::TurnComplete |
| task2->>-user: Event::TurnComplete |
| ``` |
|
|
