Commit Graph
154 Commits
Author SHA1 Message Date
James Peter 180bee67f7 Merge upstream/main: adopt tool parallelism and exec formatting 2025-10-06 11:19:57 +10:00
James Peter 81b678adef refactor(upstream): re-export crates and prune docs 2025-10-06 10:21:27 +10:00
Fouad MatinandGitHub 77a8b7fdeb add codex sandbox {linux|macos} (#4782)
## Summary
- add a `codex sandbox` subcommand with macOS and Linux targets while
keeping the legacy `codex debug` aliases
- update documentation to highlight the new sandbox entrypoints and
point existing references to the new command
- clarify the core README about the linux sandbox helper alias

## Testing
- just fmt
- just fix -p codex-cli
- cargo test -p codex-cli


------
https://chatgpt.com/codex/tasks/task_i_68e2e00ca1e8832d8bff53aa0b50b49e
2025-10-05 15:51:57 -07:00
James Peter 3c12646994 chore(tui): drop vt100-tests flag and sync docs 2025-10-06 08:40:54 +10:00
James PeterandClaude 105def2be2 docs: add comprehensive session summary for 2025-10-06
Created detailed summary document capturing all work completed during
the Phase 1 cleanup session.

Session Achievements:
- Completed Phase 1-3: Baseline & Tooling, Crate Adoption, Fork Cleanup
- Removed 35,000 lines of legacy test code (106 files)
- Adopted upstream crates (mcp-client, responses-api-proxy)
- Audited feature flags and module usage (all active, no dead code)
- Documented test scaffolding patterns (ChatWidgetHarness)
- Created 5 new migration/maintenance documentation files

Metrics Summary:
- Test files: 205 → 99 (-52%)
- Test code lines: 46,300 → 11,300 (-76%)
- Orphaned modules: 6 → 0 (100% removed)
- Build time: 1m 30s (no regression)
- Documentation: 5 new comprehensive guides

Commits Created (4):
1. Legacy test retirement and upstream crate adoption
2. Feature flag and module audit completion
3. Test scaffolding patterns documentation
4. Roadmap update with completion status

Next Steps Identified:
- Checkout upstream to codex-rs/ for test porting
- Begin Phase 2: Port 6 critical core runtime tests
- Begin Phase 3: Port 3 TUI rendering tests
- Expand smoke test coverage using documented patterns

Status: Phase 1 Complete ✅ | Phase 2-4 Ready 🔄 | Upstream Checkout Needed 📋

Session captured all deliverables, metrics, lessons learned, and blockers
for future reference and continuity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
2025-10-06 05:25:36 +10:00
James PeterandClaude 5f67759fbd chore: complete legacy test retirement and upstream crate adoption
This commit completes Phase 1 of the codebase cleanup plan documented in
ROADMAP.md and related migration documents.

Major changes:
- Removed legacy test suites from code-rs/ crates (~35,000 lines)
- Adopted upstream codex-mcp-client and codex-responses-api-proxy
- Deleted TUI compat layer, agent task planning docs, and orphaned modules
- Updated documentation to reflect current state and future phases

Test Coverage:
- Deleted tests from: app-server, apply-patch, chatgpt, cli, core, exec,
  login, mcp-server
- Documented 9 remaining coverage gaps requiring new tests (see
  docs/migration/legacy-tests-retirement.md)
- Preserved smoke test infrastructure in tui/tests/ui_smoke.rs

Crate Adoptions:
- mcp-client: Now re-exports codex-mcp-client with fork-specific buffer config
- responses-api-proxy: Now re-exports codex-responses-api-proxy with minimal
  wrapper
- mcp-types: Updated to use upstream types where possible

TUI Cleanup:
- Removed compat.rs ratatui adapter layer
- Deleted obsolete modules: app_backtrack, custom_terminal, scroll_view,
  text_block, transcript_app
- Archived agent_tasks/ planning docs to docs/archive/

Documentation Updates:
- ROADMAP.md: Marked Phases 1-2 complete, updated next actions
- TEST_SUITE_RESET.md: Documented Phase 1 completion
- DEAD_CODE_INVENTORY.md: Updated with deletion outcomes
- Added docs/migration/legacy-tests-retirement.md with 6-week plan
- Added docs/maintenance/prompt-architecture.md

Next Steps (Phase 2-3):
- Port critical test coverage (9 test files identified)
- Expand smoke test scenarios
- Continue upstream tracking per monthly cadence

See docs/migration/ for detailed migration status and roadmap.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
2025-10-06 05:15:48 +10:00
James Peter 3dd86b52d8 docs: reset roadmap and add maintenance guides 2025-10-05 20:48:06 +10:00
James Peter c2a9e0953d Revert "refactor(tui): route ratatui imports through compat"
This reverts commit 959ab00b6e.
2025-10-05 19:48:52 +10:00
James Peter 959ab00b6e refactor(tui): route ratatui imports through compat 2025-10-05 19:27:12 +10:00
James Peter 71046c5886 docs(tui): expand legacy plan and add smoke test scaffold 2025-10-05 18:34:07 +10:00
James Peter 14df1f4b64 feat(tui): add compat adapter and smoke documentation 2025-10-05 17:33:29 +10:00
Code Snapshot 29d532c2c5 code snapshot 2025-10-05 16:22:02 +10:00
James Peter 2bd41abdab fix(core/git_worktree): restore cache mirror toggle 2025-10-05 12:02:23 +10:00
James Peter ab6a22d09b fix(core/git_worktree): gate cache copying behind env 2025-10-05 11:32:59 +10:00
James Peter f95cca2b26 fix(core/git-worktree): gate target cache mirroring 2025-10-05 10:38:55 +10:00
James Peter 63c09bab80 chore(workspace): sync codex tree and finish reset plan 2025-10-05 09:04:09 +10:00
James Peter 30498218a8 docs(plan): record snapshot and pruning strategy 2025-10-05 04:57:59 +10:00
James Peter c8111bbb03 feat(code-rs): add forked workspace and dual build support 2025-10-05 04:27:49 +10:00
James Peter 31e8be5065 docs(plan): reset strategy for code/core refactor 2025-10-04 22:34:23 +10:00
Michael BolinandGitHub c32e9cfe86 chore: subject docs/*.md to Prettier checks (#4645)
Apparently we were not running our `pnpm run prettier` check in CI, so
many files that were covered by the existing Prettier check were not
well-formatted.

This updates CI and formats the files.
2025-10-03 11:35:48 -07:00
Gabriel PealandGitHub 1d17ca1fa3 [MCP] Add support for MCP Oauth credentials (#4517)
This PR adds oauth login support to streamable http servers when
`experimental_use_rmcp_client` is enabled.

This PR is large but represents the minimal amount of work required for
this to work. To keep this PR smaller, login can only be done with
`codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
or `codex mcp list` yet. Fingers crossed that this is the last large MCP
PR and that subsequent PRs can be smaller.

Under the hood, credentials are stored using platform credential
managers using the [keyring crate](https://crates.io/crates/keyring).
When the keyring isn't available, it falls back to storing credentials
in `CODEX_HOME/.credentials.json` which is consistent with how other
coding agents handle authentication.

I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
to test the dbus store on linux but did verify that the fallback works.

One quirk is that if you have credentials, during development, every
build will have its own ad-hoc binary so the keyring won't recognize the
reader as being the same as the write so it may ask for the user's
password. I may add an override to disable this or allow
users/enterprises to opt-out of the keyring storage if it causes issues.

<img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
/>
<img width="745" height="486" alt="image"
src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
/>
2025-10-03 13:43:12 -04:00
Michael BolinandGitHub 042d4d55d9 feat: codex exec writes only the final message to stdout (#4644)
This updates `codex exec` so that, by default, most of the agent's
activity is written to stderr so that only the final agent message is
written to stdout. This makes it easier to pipe `codex exec` into
another tool without extra filtering.

I introduced `#![deny(clippy::print_stdout)]` to help enforce this
change and renamed the `ts_println!()` macro to `ts_msg()` because (1)
it no longer calls `println!()` and (2), `ts_eprintln!()` seemed too
long of a name.

While here, this also adds `-o` as an alias for `--output-last-message`.

Fixes https://github.com/openai/codex/issues/1670
2025-10-03 16:22:12 +00:00
James Peter f3588b2c16 Merge upstream/main: adopt CLI stdin login and TUI fixes 2025-10-03 23:11:53 +10:00
Michael BolinandGitHub 69ac5153d4 fix: replace --api-key with --with-api-key in codex login (#4646)
Previously, users could supply their API key directly via:

```shell
codex login --api-key KEY
```

but this has the drawback that `KEY` is more likely to end up in shell
history, can be read from `/proc`, etc.

This PR removes support for `--api-key` and replaces it with
`--with-api-key`, which reads the key from stdin, so either of these are
better options:

```
printenv OPENAI_API_KEY | codex login --with-api-key
codex login --with-api-key < my_key.txt
```

Other CLIs, such as `gh auth login --with-token`, follow the same
practice.
2025-10-03 06:17:31 +00:00
James Peter d0a0f011d5 feat(protocol): add MCP shim and sync tool stack 2025-10-03 14:36:00 +10:00
James Peter 8793632001 merge(upstream): add automation prompts and exec docs 2025-10-03 14:26:48 +10:00
James Peter ae55326010 merge code-branch-20251002-131120 via /merge 2025-10-03 09:47:25 +10:00
James Peter 3003703db9 fix(tui/order): enforce ticketed background events 2025-10-03 08:32:36 +10:00
pakrym-oaiandGitHub b93cc0f431 Add a separate exec doc (#4583)
More/better docs.
2025-10-02 13:33:08 -07:00
James Peter 6556ac9267 feat(core/agents): add 'cloud' agent support across run_agent, /plan, /solve; docs and examples; basic tests
- core/agent_tool: recognize "cloud" model and append prompt positionally; honor per‑mode args
- core/codex(agent_run): include "cloud" in command resolution + PATH check
- core/slash_commands: default /plan,/solve models now include "cloud"; test asserts prompt includes it
- core/agent_defaults: add cloud case (no default flags); unit test ensures empty defaults
- docs: note default agent set includes "cloud"; add config example

Build passes via ./build-fast.sh. Full test suite currently fails on unrelated preexisting test drift (protocol/test-support imports, enum shapes).
2025-10-02 23:53:36 +10:00
James Peter 4e35c09619 feat(tui/review): add persistent auto resolve workflow 2025-10-02 22:10:37 +10:00
James Peter d069b4e627 refactor(history): centralize running tool bookkeeping 2025-10-02 14:00:44 +10:00
James Peter 6a785281fa feat(tui/history): consolidate history state and spinner cadence 2025-10-02 09:46:04 +10:00
easong-openaiandGitHub 400a5a90bf Fall back to configured instruction files if AGENTS.md isn't available (#4544)
Allow users to configure an agents.md alternative to consume, but warn
the user it may degrade model performance.

Fixes #4376
2025-10-01 18:19:59 +00:00
James Peter 73fff0317e feat(tui/cloud): add /cloud command for cloud task management 2025-10-01 18:58:27 +10:00
James Peter 9a92530031 chore(merge): prepare workspace for vitual cherry-pick 2025-10-01 04:11:40 +10:00
Michael BolinandGitHub d9dbf48828 fix: separate codex mcp into codex mcp-server and codex app-server (#4471)
This is a very large PR with some non-backwards-compatible changes.

Historically, `codex mcp` (or `codex mcp serve`) started a JSON-RPC-ish
server that had two overlapping responsibilities:

- Running an MCP server, providing some basic tool calls.
- Running the app server used to power experiences such as the VS Code
extension.

This PR aims to separate these into distinct concepts:

- `codex mcp-server` for the MCP server
- `codex app-server` for the "application server"

Note `codex mcp` still exists because it already has its own subcommands
for MCP management (`list`, `add`, etc.)

The MCP logic continues to live in `codex-rs/mcp-server` whereas the
refactored app server logic is in the new `codex-rs/app-server` folder.
Note that most of the existing integration tests in
`codex-rs/mcp-server/tests/suite` were actually for the app server, so
all the tests have been moved with the exception of
`codex-rs/mcp-server/tests/suite/mod.rs`.

Because this is already a large diff, I tried not to change more than I
had to, so `codex-rs/app-server/tests/common/mcp_process.rs` still uses
the name `McpProcess` for now, but I will do some mechanical renamings
to things like `AppServer` in subsequent PRs.

While `mcp-server` and `app-server` share some overlapping functionality
(like reading streams of JSONL and dispatching based on message types)
and some differences (completely different message types), I ended up
doing a bit of copypasta between the two crates, as both have somewhat
similar `message_processor.rs` and `outgoing_message.rs` files for now,
though I expect them to diverge more in the near future.

One material change is that of the initialize handshake for `codex
app-server`, as we no longer use the MCP types for that handshake.
Instead, we update `codex-rs/protocol/src/mcp_protocol.rs` to add an
`Initialize` variant to `ClientRequest`, which takes the `ClientInfo`
object we need to update the `USER_AGENT_SUFFIX` in
`codex-rs/app-server/src/message_processor.rs`.

One other material change is in
`codex-rs/app-server/src/codex_message_processor.rs` where I eliminated
a use of the `send_event_as_notification()` method I am generally trying
to deprecate (because it blindly maps an `EventMsg` into a
`JSONNotification`) in favor of `send_server_notification()`, which
takes a `ServerNotification`, as that is intended to be a custom enum of
all notification types supported by the app server. So to make this
update, I had to introduce a new variant of `ServerNotification`,
`SessionConfigured`, which is a non-backwards compatible change with the
old `codex mcp`, and clients will have to be updated after the next
release that contains this PR. Note that
`codex-rs/app-server/tests/suite/list_resume.rs` also had to be update
to reflect this change.

I introduced `codex-rs/utils/json-to-toml/src/lib.rs` as a small utility
crate to avoid some of the copying between `mcp-server` and
`app-server`.
2025-09-30 07:06:18 +00:00
Gabriel PealandGitHub 2e95e5602d Update MCP docs to reference experimental RMCP client (#4422) 2025-09-30 02:44:16 -04:00
dedrisian-oaiandGitHub 80ccec6530 Custom prompt args (numeric) (#4470)
[Cherry picked from /pull/3565]

Adds $1, $2, $3, $ARGUMENTS param parsing for custom prompts.
2025-09-29 16:14:37 -07:00
04c1782e52 OpenTelemetry events (#2103)
### Title

## otel

Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
that
describe each run: outbound API requests, streamed responses, user
input,
tool-approval decisions, and the result of every tool invocation. Export
is
**disabled by default** so local runs remain self-contained. Opt in by
adding an
`[otel]` table and choosing an exporter.

```toml
[otel]
environment = "staging"   # defaults to "dev"
exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
```

Codex tags every exported event with `service.name = "codex-cli"`, the
CLI
version, and an `env` attribute so downstream collectors can distinguish
dev/staging/prod traffic. Only telemetry produced inside the
`codex_otel`
crate—the events listed below—is forwarded to the exporter.

### Event catalog

Every event shares a common set of metadata fields: `event.timestamp`,
`conversation.id`, `app.version`, `auth_mode` (when available),
`user.account_id` (when available), `terminal.type`, `model`, and
`slug`.

With OTEL enabled Codex emits the following event types (in addition to
the
metadata above):

- `codex.api_request`
  - `cf_ray` (optional)
  - `attempt`
  - `duration_ms`
  - `http.response.status_code` (optional)
  - `error.message` (failures)
- `codex.sse_event`
  - `event.kind`
  - `duration_ms`
  - `error.message` (failures)
  - `input_token_count` (completion only)
  - `output_token_count` (completion only)
  - `cached_token_count` (completion only, optional)
  - `reasoning_token_count` (completion only, optional)
  - `tool_token_count` (completion only)
- `codex.user_prompt`
  - `prompt_length`
  - `prompt` (redacted unless `log_user_prompt = true`)
- `codex.tool_decision`
  - `tool_name`
  - `call_id`
- `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
  - `source` (`config` or `user`)
- `codex.tool_result`
  - `tool_name`
  - `call_id`
  - `arguments`
  - `duration_ms` (execution time for the tool)
  - `success` (`"true"` or `"false"`)
  - `output`

### Choosing an exporter

Set `otel.exporter` to control where events go:

- `none` – leaves instrumentation active but skips exporting. This is
the
  default.
- `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
Specify the
  endpoint, protocol, and headers your collector expects:

  ```toml
  [otel]
  exporter = { otlp-http = {
    endpoint = "https://otel.example.com/v1/logs",
    protocol = "binary",
    headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
  }}
  ```

- `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
and any
  metadata headers:

  ```toml
  [otel]
  exporter = { otlp-grpc = {
    endpoint = "https://otel.example.com:4317",
    headers = { "x-otlp-meta" = "abc123" }
  }}
  ```

If the exporter is `none` nothing is written anywhere; otherwise you
must run or point to your
own collector. All exporters run on a background batch worker that is
flushed on
shutdown.

If you build Codex from source the OTEL crate is still behind an `otel`
feature
flag; the official prebuilt binaries ship with the feature enabled. When
the
feature is disabled the telemetry hooks become no-ops so the CLI
continues to
function without the extra dependencies.

---------

Co-authored-by: Anton Panasenko <[email protected]>
2025-09-29 11:30:55 -07:00
James Peter 740be39c47 merge code-branch-20250929-011511 via /merge 2025-09-29 12:11:43 +10:00
James Peter fd7d3a7183 feat(auto-drive): add observer thread and telemetry 2025-09-29 12:09:06 +10:00
James Peter 19af324421 merge code-branch-20250928-230546 via /merge 2025-09-29 10:08:08 +10:00
James Peter 4b1638c37c merge code-branch-20250927-071430 via /merge 2025-09-27 18:56:48 +10:00
James Peter df497da54e Merge branch 'issue-263' into main 2025-09-27 16:32:44 +10:00
James Peter 91146a6218 merge local main into code-branch-20250926-220945 before merge 2025-09-27 09:43:56 +10:00
James Peter 0e39d1fccb fix(mcp-server/acp): expose model selection to Zed 2025-09-27 09:01:03 +10:00
James Peter 6a8aac2d39 feat(tui/notifications): add OSC toggle and slash command 2025-09-27 06:37:31 +10:00
James Peter ba7ce5459d refactor: remove pro mode infrastructure 2025-09-27 05:55:53 +10:00
just-every-code 3484b7fd3c Auto: address issue #263 - Using sub-agents to collect context for coding agents. 2025-09-26 12:19:10 +00:00