chore(workspace): sync codex tree and finish reset plan

This commit is contained in:
James Peter
2025-10-05 09:04:09 +10:00
parent 43ffb858a9
commit 224af74fc0
450 changed files with 71266 additions and 82045 deletions
+18
View File
@@ -0,0 +1,18 @@
# `code-*` vs `codex-*` Crate Parity Tracker
Goal: delete `code-*` crates once the forked implementation matches the new
upstream baseline (or the upstream crate already covers our needs).
| Crate | Diff summary (code vs codex) | Ready to delete `code-*`? | Notes |
| --- | --- | --- | --- |
| ansi-escape | Renamed crate IDs (`code-ansi-escape` vs `codex-ansi-escape`), otherwise same API but we ship fork-specific spinner assets. | No | Wait until spinner grouping lands upstream or we can fully adopt theirs. |
| arg0 | Fork keeps `--code-run-as-apply-patch` flag; upstream uses `codex`. | No | Needs flag rename adapters before deleting. |
| browser | Fork-only feature. | No | Upstream lacks browser integration. |
| core | Diverges heavily (executor/tool router, approval policy). | No | Bridge in progress (see `docs/subsystem-migration-status.md`). |
| exec | Fork adds policy prompts + streaming bridges. | No | Depends on core migration. |
| login | Fork adds approval + device flow tweaks; upstream gained OAuth helpers. | No | Reconcile after validating new helpers in fork. |
| protocol | Upstream introducing new tool schema (ACP). | No | Needs schema audit before removal. |
| tui | Fork retains browser UI, streaming invariants. | No | Must port status UI first. |
Update this table as diffs shrink. When “Yes” appears, schedule deletion in the
next release window and update documentation accordingly.
+26
View File
@@ -0,0 +1,26 @@
# `code-rs` Dead-Code Sweep Plan
We now carry the upstream tree side-by-side. To keep `code-rs/` lean while
we bridge to the upstream crates, run focused dead-code sweeps on a schedule.
## Cadence
| Window | Focus areas | Owner |
| --- | --- | --- |
| 2025-10-20 → 2025-10-24 | `code-rs/tui/*` legacy render helpers, unused history cell modules. | TUI pod |
| 2025-11-17 → 2025-11-21 | `code-rs/core/*` legacy executor adapters after the upstream router lands. | Core runtime pod |
| 2025-12-15 → 2025-12-19 | Remaining crates (CLI, browser, app-server) post-executor migration. | Repo maintainers |
Add follow-up rows as additional migrations finish.
## Sweep checklist
1. Run `rg --files-without-match` patterns to detect unused modules (e.g. `mod
` declarations).
2. Confirm no crate-level `pub use` exposes the target symbol externally.
3. Delete the dead module/file and rerun `./build-fast.sh --workspace code`.
4. If the removal mirrors an upstream deletion, record it in
`docs/subsystem-migration-status.md` so future merges know the fork matches
upstream.
Track outcomes in PR descriptions so we know which areas are already cleaned.
+14 -18
View File
@@ -67,24 +67,20 @@ We already tried copying modules into `code-*` while leaving the originals in `c
`./build-fast.sh --workspace code` passes.
3. **DONE (2025-10-04):** Update shared tooling (`build-fast.sh`) to target
both workspaces and keep existing `codex` builds green (`./build-fast.sh`).
4. Fix build/scripts/tests that reference `codex-rs/…` paths (e.g.
`build-fast.sh`, CI workflows, developer docs). *(Owner: in progress —
audit remaining references outside `build-fast.sh`.)*
5. Replace `codex-rs/` with the upstream checkout and re-run
`./build-fast.sh`. *(Owner: pending — execute after snapshot & tooling
audit.)*
6. Review each major subsystem to pinpoint the largest chunks we can
realistically source from upstream, implementing extension hooks or local
overrides only where forked behavior is essential.
7. Schedule regular dead-code sweeps in `code-rs/` to prune legacy modules
we inherited from upstream but no longer reference; document each removal
so future merges stay clean.
8. As upstream-backed sections land, retire redundant fork modules and,
when a `code-*` crate is effectively identical (exact replica or only an
ancestor without new commits) to its upstream counterpart, delete the
duplicate crate in favor of the `codex-rs/` version before cutting a
release. Ensure downstream crates import the appropriate `code-*` or
upstream paths based on the documented decision.
4. **DONE (2025-10-04):** Audited all scripts, docs, and SDK samples to swap
`codex-rs/…` references over to `code-rs/…` (see commits touching
`build-fast.sh`, `scripts/ci-tests.sh`, TypeScript SDK, installation docs).
5. **DONE (2025-10-04):** Checked out upstream `codex-rs/` at
`openai/codex@d7acd146` and re-ran `./build-fast.sh --workspace codex`
plus `./build-fast.sh --workspace code` to confirm both workspaces build.
6. **DONE (2025-10-04):** Captured subsystem decisions in
`docs/subsystem-migration-status.md`, highlighting where we will reuse
upstream code versus keep fork-specialised wrappers.
7. **DONE (2025-10-04):** Scheduled recurring `code-rs/` dead-code sweeps and
documented the cadence in `docs/code-dead-code-sweeps.md`.
8. **DONE (2025-10-04):** Opened `docs/code-crate-parity-tracker.md` to record
when forked crates reach parity with upstream so we can delete duplicates
ahead of releases.
## Tracking
+3 -3
View File
@@ -78,10 +78,10 @@ Make sure you are on `main` and have no local changes. Then run:
```shell
VERSION=0.2.0 # Can also be 0.2.0-alpha.1 or any valid Rust version.
./codex-rs/scripts/create_github_release.sh "$VERSION"
./code-rs/scripts/create_github_release.sh "$VERSION"
```
This will make a local commit on top of `main` with `version` set to `$VERSION` in `codex-rs/Cargo.toml` (note that on `main`, we leave the version as `version = "0.0.0"`).
This will make a local commit on top of `main` with `version` set to `$VERSION` in `code-rs/Cargo.toml` (note that on `main`, we leave the version as `version = "0.0.0"`).
This will push the commit using the tag `rust-v${VERSION}`, which in turn kicks off [the release workflow](../.github/workflows/rust-release.yml). This will create a new GitHub Release named `$VERSION`.
@@ -91,4 +91,4 @@ Create a PR to update [`Formula/c/codex.rb`](https://github.com/Homebrew/homebre
### Security & responsible AI
Have you discovered a vulnerability or have concerns about model output? Please e-mail **[email protected]** and we will respond promptly.
Have you discovered a vulnerability or have concerns about model output? Please e-mail **[email protected]** and we will respond promptly.
+1 -1
View File
@@ -38,7 +38,7 @@ Sample output:
{"type":"turn.started"}
{"type":"item.completed","item":{"id":"item_0","item_type":"reasoning","text":"**Searching for README files**"}}
{"type":"item.started","item":{"id":"item_1","item_type":"command_execution","command":"bash -lc ls","aggregated_output":"","status":"in_progress"}}
{"type":"item.completed","item":{"id":"item_1","item_type":"command_execution","command":"bash -lc ls","aggregated_output":"2025-09-11\nAGENTS.md\nCHANGELOG.md\ncliff.toml\ncodex-cli\ncodex-rs\ndocs\nexamples\nflake.lock\nflake.nix\nLICENSE\nnode_modules\nNOTICE\npackage.json\npnpm-lock.yaml\npnpm-workspace.yaml\nPNPM.md\nREADME.md\nscripts\nsdk\ntmp\n","exit_code":0,"status":"completed"}}
{"type":"item.completed","item":{"id":"item_1","item_type":"command_execution","command":"bash -lc ls","aggregated_output":"2025-09-11\nAGENTS.md\nCHANGELOG.md\ncliff.toml\ncodex-cli\ncode-rs\ncodex-rs\ndocs\nexamples\nflake.lock\nflake.nix\nLICENSE\nnode_modules\nNOTICE\npackage.json\npnpm-lock.yaml\npnpm-workspace.yaml\nPNPM.md\nREADME.md\nscripts\nsdk\ntmp\n","exit_code":0,"status":"completed"}}
{"type":"item.completed","item":{"id":"item_2","item_type":"reasoning","text":"**Checking repository root for README**"}}
{"type":"item.completed","item":{"id":"item_3","item_type":"assistant_message","text":"Yep — there’s a `README.md` in the repository root."}}
{"type":"turn.completed","usage":{"input_tokens":24763,"cached_input_tokens":24448,"output_tokens":122}}
+1 -1
View File
@@ -2,7 +2,7 @@
This fork extends upstream `openai/codex` in several areas. These bullets are a starting point — not a complete list. Before merging, scan the codebase and history (CHANGELOG.md, recent commits) to discover additional or newer fork‑only behavior and preserve it.
- Browser Integration (codex-rs/browser, TUI + core wiring)
- Browser Integration (code-rs/browser, TUI + core wiring)
- Internal CDP browser manager with global access and /browser command.
- Tool family: `browser_*` (open, status, click, move, type, key, javascript, scroll, history, inspect, console, cleanup, cdp).
- Screenshot capture with segmentation, cursor overlay, asset storage, and per‑turn injection/queueing; TUI rendering with friendly titles.
+4 -4
View File
@@ -17,7 +17,7 @@ The GitHub Release also contains a [DotSlash](https://dotslash-cli.com/) file fo
```bash
# Clone the repository and navigate to the root of the Cargo workspace.
git clone https://github.com/openai/codex.git
cd codex/codex-rs
cd codex/code-rs
# Install the Rust toolchain, if necessary.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
@@ -25,11 +25,11 @@ source "$HOME/.cargo/env"
rustup component add rustfmt
rustup component add clippy
# Build Codex.
# Build Code.
cargo build
# Launch the TUI with a sample prompt.
cargo run --bin codex -- "explain this codebase to me"
cargo run --bin code -- "explain this codebase to me"
# After making changes, ensure the code is clean.
cargo fmt -- --config imports_granularity=Item
@@ -37,4 +37,4 @@ cargo clippy --tests
# Run the tests.
cargo test
```
```
+4 -4
View File
@@ -6,10 +6,10 @@ Guarantee that all UI background events carry immutable order metadata captured
## Scope
- `codex-rs/tui/src/chatwidget.rs`
- `codex-rs/tui/src/app_event_sender.rs`
- `code-rs/tui/src/chatwidget.rs`
- `code-rs/tui/src/app_event_sender.rs`
- Async producers that currently call `AppEventSender::send_background_event*`
- Associated unit tests under `codex-rs/tui/src/chatwidget.rs`
- Associated unit tests under `code-rs/tui/src/chatwidget.rs`
## Constraints & Signals
@@ -26,7 +26,7 @@ Guarantee that all UI background events carry immutable order metadata captured
### 1. Promote Order Handles to Public API
- Expose a new lightweight wrapper (e.g., `BackgroundOrderTicket`) that encapsulates `UiBackgroundOrderHandle`.
- Store it in `codex-rs/tui/src/app_event_sender.rs` so async producers can hold a clone even after moving off the main thread.
- Store it in `code-rs/tui/src/app_event_sender.rs` so async producers can hold a clone even after moving off the main thread.
- Provide constructors on `ChatWidget`:
- `fn make_background_tail_ticket(&mut self) -> BackgroundOrderTicket`
- `fn make_background_before_next_output_ticket(&mut self) -> BackgroundOrderTicket`
+1 -1
View File
@@ -14,7 +14,7 @@ Assuming you are trying to publish `0.21.0`, first you would run:
```shell
VERSION=0.21.0
./codex-rs/scripts/create_github_release.sh "$VERSION"
./code-rs/scripts/create_github_release.sh "$VERSION"
```
This will kick off a GitHub Action to build the release, so go to https://github.com/openai/codex/actions/workflows/rust-release.yml to find the corresponding workflow. (Note: we should automate finding the workflow URL with `gh`.)
+2 -2
View File
@@ -89,10 +89,10 @@ typically start multiple agents. They require a task/problem description.
Implementation Notes
- The authoritative list of commands is defined in
`codex-rs/tui/src/slash_command.rs` (the `SlashCommand` enum). When adding a
`code-rs/tui/src/slash_command.rs` (the `SlashCommand` enum). When adding a
new command, please update this document to keep the UI and docs in sync.
- Prompt formatting for `/plan`, `/solve`, and `/code` lives in
`codex-rs/core/src/slash_commands.rs`.
`code-rs/core/src/slash_commands.rs`.
When no `[[agents]]` are configured, the orchestrator advertises the
following agent names to the LLM for multi‑agent runs: `claude`, `gemini`,
`qwen`, `code`, and `cloud`. You can replace or pin this set via `[[agents]]`
+18
View File
@@ -0,0 +1,18 @@
# Subsystem Migration Status (2025-10-04)
This tracker documents how each major area of the fork compares against the
fresh upstream snapshot in `codex-rs/` and whether we plan to reuse the upstream
implementation or keep a forked variant living under `code-rs/`.
| Subsystem | Upstream delta summary | Decision | Immediate next steps |
| --- | --- | --- | --- |
| Core runtime (`core/`, `exec/`, `protocol/`, `apply-patch/`) | Upstream added a unified `executor` module, tool router, and richer MCP hooks. Fork keeps pre-refactor pipeline plus fork-only policy/approval features. | **Hybrid** — adopt upstream executor/tool router incrementally while preserving fork-only approval flow. | Diff `code-rs/core` vs `codex-rs/core` focusing on `executor/*` and `tools/*`; design an adapter so forked approval + sandbox policy can plug into the upstream router without copy-pasting. |
| TUI (`tui/`) | Upstream reorganised history cells into typed renderers, added status dashboards/tests, and removed the older per-cell module tree. Fork still uses the legacy layout with Code-specific UX. | **Fork-primary** — keep `code-rs/tui` as the shipping implementation; cherry-pick upstream status widgets/tests once we add equivalent APIs in `code-rs`. | Inventory upstream additions (`status/*`, `render/renderable.rs`) and file GitHub issues to port features where they unlock tooling (status cards, vt100 snapshots). |
| CLI (`cli/`, `login/`, `mcp-*`, `responses-api-proxy/`) | Upstream refreshed auth flows (OAuth helpers in `rmcp-client`) and expanded MCP tooling; fork adds policy prompts, approval gating. | **Hybrid** — reuse upstream auth/mcp client improvements by wiring them behind feature flags in `code-rs` while keeping fork-only prompts. | Prototype re-export of `codex-rs/rmcp-client` login helpers inside `code-rs/login`; confirm CLI flag parity before deleting fork copies. |
| Browser tooling (`browser/`, TUI browser events) | Upstream still lacks the browser agent; fork-only feature. | **Fork-only** — retain `code-rs/browser` and related TUI integrations. | Document browser integration in `docs/fork-enhancements.md` (already tracked); no upstream merge needed. |
| App server (`app-server`, `app-server-protocol`) | Upstream rewrote message processor to use new executor/tool traits. Fork still references old pipeline. | **Monitor** — do not adopt until core executor bridge (above) lands; otherwise duplicate effort. | Once the core adapter is ready, revisit to see if we can reuse upstream app-server entirely. |
| SDK / tooling (`sdk/typescript`, `scripts/*`) | Upstream scripts now assume `codex-*` bins. Fork updated to `code-*`. | **Fork-primary** with compatibility fallbacks. | Ensure scripts that read templates (e.g., `test-responses.js`) load both paths (already updated). Review future upstream changes quarterly. |
| Build & release tooling (`build-fast.sh`, nix flakes) | Upstream still targets a single workspace. Fork must handle both. | **Fork-only** — maintain dual-workspace script. | Keep script in sync with upstream bugfixes manually; add regression tests later. |
Last updated **2025-10-04**. Update this table whenever a subsystem decision
changes or a milestone completes.
+1 -1
View File
@@ -1,7 +1,7 @@
# TUI ChatWidget Refactor Plan
This document tracks the refactor of `codex-rs/tui/src/chatwidget.rs` into smaller, maintainable modules and state bundles. It also captures what has been completed, and what remains so another engineer can continue seamlessly.
This document tracks the refactor of `code-rs/tui/src/chatwidget.rs` into smaller, maintainable modules and state bundles. It also captures what has been completed, and what remains so another engineer can continue seamlessly.
## Goals
+16 -16
View File
@@ -5,7 +5,7 @@
Upstream PR [openai/codex#1707](https://github.com/openai/codex/pull/1707) introduces an
experimental Agent Client Protocol (ACP) bridge so Zed can drive Codex via the
`session/new` and `session/prompt` tool calls. The branch (fetched locally as
`upstream-pr-1707`) rewrites large portions of `codex-rs/core`, the MCP server,
`upstream-pr-1707`) rewrites large portions of `code-rs/core`, the MCP server,
and the TypeScript CLI/TUI to accommodate the new workflow.
### Current Status (2025-09-22)
@@ -31,7 +31,7 @@ capabilities.
## What Upstream Added
- **Core (`codex-rs/core`)**
- **Core (`code-rs/core`)**
- New `acp.rs` module with helpers for translating Codex exec/patch events
into ACP `ToolCall` updates, including an ACP-backed `FileSystem` shim and a
permission-request flow (`acp::request_permission`).
@@ -40,7 +40,7 @@ capabilities.
etc.) and introducing new event wiring for ACP tool calls.
- `protocol.rs` pared down to match the simplified upstream surface (fewer
`Op` variants, different `AskForApproval` options, new MCP call events).
- **MCP server (`codex-rs/mcp-server`)**
- **MCP server (`code-rs/mcp-server`)**
- New `acp_tool_runner.rs` that spawns Codex sessions on demand, relays MCP
notifications, and surfaces ACP updates.
- `message_processor.rs` extended to expose `session/new` and
@@ -53,11 +53,11 @@ capabilities.
- **Protocol & Ops** – `QueueUserInput`, validation toggles, confirm guards,
rich approval semantics (`ReviewDecision::ApprovedForSession`), and sandbox
policy options currently in `codex-rs/core/src/protocol.rs:1`.
policy options currently in `code-rs/core/src/protocol.rs:1`.
- **Execution Safety & Logging** – Confirm-guard enforcement and the richer
`EventMsg` variants emitted from our `codex-rs/core/src/codex.rs:1`.
- **TUI** – Entire Rust TUI stack (`codex-rs/tui/src/chatwidget.rs:1`,
`codex-rs/tui/src/history_cell.rs:1`, etc.) must remain functional and ignore
`EventMsg` variants emitted from our `code-rs/core/src/codex.rs:1`.
- **TUI** – Entire Rust TUI stack (`code-rs/tui/src/chatwidget.rs:1`,
`code-rs/tui/src/history_cell.rs:1`, etc.) must remain functional and ignore
unknown ACP events gracefully.
- **CLI Footprint** – Our trimmed `codex-cli` structure and scripts differ from
upstream’s wholesale replacement; we will not adopt the TypeScript overhaul.
@@ -67,17 +67,17 @@ capabilities.
## Integration Approach
1. **Introduce ACP Helpers Without Regressions**
- Port `codex-rs/core/src/acp.rs:1` (and dependent structs) into Code, but
- Port `code-rs/core/src/acp.rs:1` (and dependent structs) into Code, but
adapt it to reuse the current `FileChange` representations and respect
confirm guard / approval flows.
- Extend `codex-rs/core/src/codex.rs:1` to emit ACP events while preserving
- Extend `code-rs/core/src/codex.rs:1` to emit ACP events while preserving
the existing `Op` variants, queueing logic, and validation toggles.
- Update `codex-rs/core/src/util.rs:1`, `codex-rs/core/src/apply_patch.rs:1`,
- Update `code-rs/core/src/util.rs:1`, `code-rs/core/src/apply_patch.rs:1`,
and related modules so ACP tool-call generators can derive the same
metadata our TUI already consumes.
2. **MCP Server Wiring**
- Add `codex-rs/mcp-server/src/acp_tool_runner.rs:1` and integrate it with
- Add `code-rs/mcp-server/src/acp_tool_runner.rs:1` and integrate it with
`message_processor.rs:1`, ensuring we keep Code-specific auth/sandbox setup
and error reporting.
- Maintain existing MCP tools and ensure ACP tools are opt-in (guarded by
@@ -85,7 +85,7 @@ capabilities.
unused.
3. **Event & Frontend Compatibility**
- Extend our event enums (`codex-rs/core/src/protocol.rs:1`) with any new
- Extend our event enums (`code-rs/core/src/protocol.rs:1`) with any new
variants required by ACP while keeping deprecated ones for backward
compatibility.
- Teach the Rust TUI to ignore or minimally display ACP-specific events so
@@ -93,17 +93,17 @@ capabilities.
4. **Config & Build**
- Add `agent-client-protocol` dependency where needed, updating
`codex-rs/core/Cargo.toml:1`, `codex-rs/mcp-server/Cargo.toml:1`, and
`code-rs/core/Cargo.toml:1`, `code-rs/mcp-server/Cargo.toml:1`, and
`Cargo.lock`.
- Introduce configuration toggles (if any) in `codex-rs/core/src/config.rs:1`
and `codex-rs/core/src/config_types.rs:1` without breaking existing TOML
- Introduce configuration toggles (if any) in `code-rs/core/src/config.rs:1`
and `code-rs/core/src/config_types.rs:1` without breaking existing TOML
files.
- Update documentation (targeted sections in `docs/experimental.md:1` or a
new page) to describe ACP/Zed support plus Code-specific caveats.
5. **Testing & Validation**
- Add focused tests covering ACP request flow (unit-level in core and MCP
server). Reuse existing harnesses (e.g., `codex-rs/mcp-server/tests`) to
server). Reuse existing harnesses (e.g., `code-rs/mcp-server/tests`) to
simulate a session.
- Validate end-to-end via `./build-fast.sh` only, honoring our policy against
running additional formatters automatically.