Commit Graph
153 Commits
Author SHA1 Message Date
James Peter 6aa5c5ce78 fix(exec/human_output): split stdout/stderr and add 'ERROR' divider on failed exec\n\nAlways compute truncated stdout and stderr.\nOn success, show dimmed stdout.\nOn failure, show dimmed stdout (if any), a visible 'ERROR' separator, then stderr without styling.\nThis keeps output readable when ANSI colors are stripped and clarifies which stream content came from. 2025-09-10 08:06:58 +10:00
just-every-code c6e4412afb merge(upstream): merge upstream/main into upstream-merge (by-bucket)
- Keep ours for TUI/CLI/workflows/docs; purge reintroduced rust-ci.yml
- Prefer theirs in protocol/deps; add serde_bytes + ICU/sys-locale/serde_with
- Keep our core codex/shell to preserve API; fix exec AuthMode import
- Make verify.sh CI-safe (no sccache; local CARGO dirs); check api_surface only

Build: ok; Verify: ok
2025-09-09 01:56:14 +00:00
jif-oaiandGitHub 3bd32af1b2 feat: Run cargo shear during CI (#3338)
Run cargo shear as part of the CI to ensure no unused dependencies
2025-09-09 01:05:08 +00:00
just-every-code 65fca1265c chore(exec): restore our exec from origin/upstream-merge to eliminate merge artifacts 2025-09-09 00:36:16 +00:00
just-every-code 8dd5a2e740 merge(upstream): merge upstream/main into upstream-merge (by-bucket)\n\n- Prefer ours: TUI, CLI, workflows, docs\n- Prefer theirs: core/common/protocol/exec/file-search\n- Purged disallowed assets; resolved conflicts per policy 2025-09-09 00:21:14 +00:00
Justin LebarandGitHub 1f5f4195ae Format large numbers in a more readable way. (#2046)
- In the bottom line of the TUI, print the number of tokens to 3 sigfigs
  with an SI suffix, e.g. "1.23K".
- Elsewhere where we print a number, I figure it's worthwhile to print
  the exact number, because e.g. it's a summary of your session. Here we print
  the numbers comma-separated.
2025-09-08 21:48:48 +00:00
Gabriel PealandGitHub 59fd77716d Use ConversationId instead of raw Uuids (#3282)
We're trying to migrate from `session_id: Uuid` to `conversation_id:
ConversationId`. Not only does this give us more type safety but it
unifies our terminology across Codex and with the implementation of
session resuming, a conversation (which can span multiple sessions) is
more appropriate.

I started this impl on https://github.com/openai/codex/pull/3219 as part
of getting resume working in the extension but it's big enough that it
should be broken out.
2025-09-07 23:22:25 -04:00
pakrym-oaiandGitHub 5f86738c8d Move token usage/context information to session level (#3221)
Move context information into the main loop so it can be used to
interrupt the loop or start auto-compaction.
2025-09-06 15:19:23 +00:00
pakrym-oaiandGitHub 73f9cf8c85 Never store requests (#3212)
When item ids are sent to Responses API it will load them from the
database ignoring the provided values. This adds extra latency.

Not having the mode to store requests also allows us to simplify the
code.

## Breaking change

The `disable_response_storage` configuration option is removed.
2025-09-05 10:41:47 -07:00
James Peter 1759c50dae feat(tui/layout): align composer/history wrap; add sanitize markers
Introduce layout_consts and replace hardcoded paddings to keep composer and history wrapping identical.

Extend sanitize::Options with debug_markers (default false) and plumb to history cell render paths.

Use codex-version crate for version reporting in TUI and wire crate deps.

Cleanup GitHub CI: remove unused workflows/assets; update issue-triage.
2025-09-05 23:07:36 +10:00
Ahmed IbrahimandGitHub 56434372da Dividing UserMsgs into categories to send it back to the tui (#3127)
This PR does the following:

- divides user msgs into 3 categories: plain, user instructions, and
environment context
- Centralizes adding user instructions and environment context to a
degree
- Improve the integration testing

Building on top of #3123

Specifically this
[comment](https://github.com/openai/codex/pull/3123#discussion_r2319885089).
We need to send the user message while ignoring the User Instructions
and Environment Context we attach.
2025-09-04 05:34:50 +00:00
Ahmed IbrahimandGitHub b7922b300c Replay EventMsgs from Response Items when resuming a session with history. (#3123)
### Overview

This PR introduces the following changes:
	1.	Adds a unified mechanism to convert ResponseItem into EventMsg.
2. Ensures that when a session is initialized with initial history, a
vector of EventMsg is sent along with the session configuration. This
allows clients to re-render the UI accordingly.
	3. 	Added integration testing

### Caveats

This implementation does not send every EventMsg that was previously
dispatched to clients. The excluded events fall into two categories:
	•	“Arguably” rolled-out events
Examples include tool calls and apply-patch calls. While these events
are conceptually rolled out, we currently only roll out ResponseItems.
These events are already being handled elsewhere and transformed into
EventMsg before being sent.
	•	Non-rolled-out events
Certain events such as TurnDiff, Error, and TokenCount are not rolled
out at all.

### Future Directions

At present, resuming a session involves maintaining two states:
	•	UI State
Clients can replay most of the important UI from the provided EventMsg
history.
	•	Model State
The model receives the complete session history to reconstruct its
internal state.

This design provides a solid foundation. If, in the future, more precise
UI reconstruction is needed, we have two potential paths:
1. Introduce a third data structure that allows us to derive both
ResponseItems and EventMsgs.
2. Clearly divide responsibilities: the core system ensures the
integrity of the model state, while clients are responsible for
reconstructing the UI.
2025-09-04 04:47:00 +00:00
James Peter 9c650fbad8 fix(core): complete upstream integration
- Update all AuthManager::shared and CodexAuth::from_codex_home call sites with originator
- Use rollout::policy helper to filter persisted items
- Fix shell tool description formatting and UA helper reference
- Remove unused import in chatgpt_client

Build: ./build-fast.sh clean with zero warnings
2025-09-04 07:21:34 +10:00
pakrym-oaiandGitHub 523e1e9f19 Add a common way to create HTTP client (#3110)
Ensure User-Agent and originator are always sent.
2025-09-03 10:11:02 -07:00
James Peter 5830abba54 merge(upstream): integrate upstream/main via merge branch; keep TUI fork; fix build\n\n- Create merge branch, merge origin/main and upstream/main with -X ours\n- Resolve conflicts: keep our TUI; drop upstream vt100 tests and snapshots\n- Decouple core from codex-login to break cycle; re-export auth in core\n- apply-patch: robust heredoc parsing via tree-sitter; minor fixes\n- protocol: align FileChange::Delete variants across core/wire code paths\n- TUI: adjust Git repo detection; simplify parsed command labels\n- mcp-server/exec: update imports and Delete handling\n- message_history: use fs2 locks; add dead_code allows\n\nValidated with ./build-fast.sh 2025-09-03 20:14:55 +10:00
James Peter 08609e46c7 merge: integrate upstream/main (prefer ours); keep TUI fork, drop legacy vt100 tests, retain local common.rs 2025-09-03 19:49:35 +10:00
James Peter e3a13174f4 chore(codex-rs): clean up tree; add CLI order_replay
- Remove legacy TUI test suite and large fixtures
- Add cli/src/bin/order_replay.rs for output-order analysis
- Update TUI streaming and widgets; minor core protocol/client tweaks
- Update Cargo manifests/lockfile

Build verified with ./build-fast.sh
2025-09-03 19:47:03 +10:00
pakrym-oaiandGitHub e5e9f27c79 Move CodexAuth and AuthManager to the core crate (#3074)
Fix a long standing layering issue.
2025-09-02 18:36:19 -07:00
Jeremy RoseandGitHub 434f743951 rework message styling (#2877)
https://github.com/user-attachments/assets/cf07f62b-1895-44bb-b9c3-7a12032eb371
2025-09-02 17:29:58 +00:00
James Peter c84b285080 merge(upstream-2025-08-29): integrate upstream main with -X ours; align web_search events; fix TUI popup APIs; clean warnings; build passes 2025-08-29 12:55:03 +10:00
Ahmed IbrahimandGitHub 148de719bf Following up on #2371 post commit feedback (#2852)
- Introduce websearch end to complement the begin 
- Moves the logic of adding the sebsearch tool to
create_tools_json_for_responses_api
- Making it the client responsibility to toggle the tool on or off 
- Other misc in #2371 post commit feedback
- Show the query:

<img width="1392" height="151" alt="image"
src="https://github.com/user-attachments/assets/8457f1a6-f851-44cf-bcca-0d4fe460ce89"
/>
2025-08-28 19:24:38 -07:00
dedrisian-oaiandGitHub 9e254eda06 Custom /prompts (#2696)
Adds custom `/prompts` to `~/.codex/prompts/<command>.md`.

<img width="239" height="107" alt="Screenshot 2025-08-25 at 6 22 42 PM"
src="https://github.com/user-attachments/assets/fe6ebbaa-1bf6-49d3-95f9-fdc53b752679"
/>

---

Details:

1. Adds `Op::ListCustomPrompts` to core.
2. Returns `ListCustomPromptsResponse` with list of `CustomPrompt`
(name, content).
3. TUI calls the operation on load, and populates the custom prompts
(excluding prompts that collide with builtins).
4. Selecting the custom prompt automatically sends the prompt to the
agent.
2025-08-29 02:16:39 +00:00
James Peter f52f1a172a Merge upstream/main into fork (prefer ours), integrate pager overlay; keep our transcript_app.rs for now 2025-08-29 11:50:50 +10:00
Michael BolinandGitHub 9cd9008862 chore: require uninlined_format_args from clippy (#2845)
- added `uninlined_format_args` to `[workspace.lints.clippy]` in the
`Cargo.toml` for the workspace
- ran `cargo clippy --tests --fix`
- ran `just fmt`
2025-08-28 11:25:23 -07:00
dedrisian-oaiandGitHub 22224063c4 Add "View Image" tool (#2723)
Adds a "View Image" tool so Codex can find and see images by itself:

<img width="1772" height="420" alt="Screenshot 2025-08-26 at 10 40
04 AM"
src="https://github.com/user-attachments/assets/7a459c7b-0b86-4125-82d9-05fbb35ade03"
/>
2025-08-27 17:41:23 -07:00
Ahmed IbrahimandGitHub 3ef5a8a5d8 send context window with task started (#2752)
- Send context window with task started
- Accounting for changing the model per turn
2025-08-27 00:04:21 -07:00
James Peter 2670f8a1d7 Merge upstream/main into fork (prefer ours); resolve TUI conflicts by adopting upstream status_indicator_widget and clipboard_paste implementations to keep features and compatibility. 2025-08-26 16:52:04 +10:00
DylanandGitHub db6b26c55d [exec] Clean up apply-patch tests (#2648)
## Summary
These tests were getting a bit unwieldy, and they're starting to become
load-bearing. Let's clean them up, and get them working solidly so we
can easily expand this harness with new tests.

## Test Plan
- [x] Tests continue to pass
2025-08-25 15:08:01 -07:00
James Peter 1a5e27361e Merge upstream/main into merge/upstream-20250825 preferring ours (-X ours) 2025-08-25 20:50:54 +10:00
James Peter 089e3aaf9d Diff wrapping: add one extra space to continuation hang indent\n\n- Continuation lines now reserve sign column + trailing space before content\n- Aligns wrapped rows exactly under the first-line content 2025-08-25 11:01:04 +10:00
James Peter 9c95a987a6 Prefer CODE_VERSION with fallback to CARGO_PKG_VERSION across codex-rs\n\n- Replace direct env!("CARGO_PKG_VERSION") with option_env!("CODE_VERSION").unwrap_or(env!("CARGO_PKG_VERSION"))\n- Update clap version field, TUI upgrade banners, user agent, MCP client init, and OpenAI header\n- Verified via ./build-fast.sh 2025-08-25 08:39:26 +10:00
Jeremy RoseandGitHub c677d5a858 test: faster test execution in codex-core (#2633)
this dramatically improves time to run `cargo test -p codex-core` (~25x
speedup).

before:
```
cargo test -p codex-core  35.96s user 68.63s system 19% cpu 8:49.80 total
```

after:
```
cargo test -p codex-core  5.51s user 8.16s system 63% cpu 21.407 total
```

both tests measured "hot", i.e. on a 2nd run with no filesystem changes,
to exclude compile times.

approach inspired by [Delete Cargo Integration
Tests](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
we move all test cases in tests/ into a single suite in order to have a
single binary, as there is significant overhead for each test binary
executed, and because test execution is only parallelized with a single
binary.
2025-08-24 11:10:53 -07:00
James Peter 864a8a3660 chore: commit local changes 2025-08-24 21:11:26 +10:00
James Peter 364cf6b54e Merge upstream/main: prefer our TUI, integrate web_search and apply_patch config; resolve conflicts and wire flags 2025-08-24 19:24:15 +10:00
95bf38298a Add web search tool (#2371)
Adds web_search tool, enabling the model to use Responses API web_search
tool.
- Disabled by default, enabled by --search flag
- When --search is passed, exposes web_search_request function tool to
the model, which triggers user approval. When approved, the model can
use the web_search tool for the remainder of the turn
<img width="1033" height="294" alt="image"
src="https://github.com/user-attachments/assets/62ac6563-b946-465c-ba5d-9325af28b28f"
/>

---------

Co-authored-by: Hanzo Dev <[email protected]>
2025-08-23 22:58:56 -07:00
James Peter 5f00bd05f9 merge: integrate upstream/main, fix protocol mismatches and compile errors (no TUI regressions) 2025-08-24 06:01:43 +10:00
James Peter a653242021 Merge remote-tracking branch 'upstream/main' into merge/upstream-main-2025-08-24 2025-08-24 05:56:25 +10:00
3d1dba2e57 send-aggregated output (#2364)
We want to send an aggregated output of stderr and stdout so we don't
have to aggregate it stderr+stdout as we lose order sometimes.

---------

Co-authored-by: Gabriel Peal <[email protected]>
2025-08-23 16:54:31 +00:00
James Peter 05a6095d09 fix(tui): clear full history region (incl. padding) each frame to eliminate rare margin artifacts and ensure background is fully painted 2025-08-24 00:57:37 +10:00
James Peter b60a88d2a1 Merge upstream/main; align protocol/core; revert TUI to fork and adapt; fix build 2025-08-23 18:13:52 +10:00
Ahmed IbrahimandGitHub fd2c3a4cc2 fork conversation from a previous message (#2575)
This can be the underlying logic in order to start a conversation from a
previous message. will need some love in the UI.

Base for building this: #2588
2025-08-22 17:06:09 -07:00
Jeremy RoseandGitHub 51c5af0c0d tui: coalesce command output; show unabridged commands in transcript (#2590)
https://github.com/user-attachments/assets/effec7c7-732a-4b61-a2ae-3cb297b6b19b
2025-08-22 16:32:31 -07:00
DylanandGitHub f83f221826 [apply_patch] freeform apply_patch tool (#2576)
## Summary
GPT-5 introduced the concept of [custom
tools](https://platform.openai.com/docs/guides/function-calling#custom-tools),
which allow the model to send a raw string result back, simplifying
json-escape issues. We are migrating gpt-5 to use this by default.

However, gpt-oss models do not support custom tools, only normal
functions. So we keep both tool definitions, and provide whichever one
the model family supports.

## Testing
- [x] Tested locally with various models
- [x] Unit tests pass
2025-08-22 13:42:34 -07:00
Eric TrautandGitHub 436bfc2bc9 Add AuthManager and enhance GetAuthStatus command (#2577)
This PR adds a central `AuthManager` struct that manages the auth
information used across conversations and the MCP server. Prior to this,
each conversation and the MCP server got their own private snapshots of
the auth information, and changes to one (such as a logout or token
refresh) were not seen by others.

This is especially problematic when multiple instances of the CLI are
run. For example, consider the case where you start CLI 1 and log in to
ChatGPT account X and then start CLI 2 and log out and then log in to
ChatGPT account Y. The conversation in CLI 1 is still using account X,
but if you create a new conversation, it will suddenly (and
unexpectedly) switch to account Y.

With the `AuthManager`, auth information is read from disk at the time
the `ConversationManager` is constructed, and it is cached in memory.
All new conversations use this same auth information, as do any token
refreshes.

The `AuthManager` is also used by the MCP server's GetAuthStatus
command, which now returns the auth method currently used by the MCP
server.

This PR also includes an enhancement to the GetAuthStatus command. It
now accepts two new (optional) input parameters: `include_token` and
`refresh_token`. Callers can use this to request the in-use auth token
and can optionally request to refresh the token.

The PR also adds tests for the login and auth APIs that I recently added
to the MCP server.
2025-08-22 13:10:11 -07:00
James Peter 63365c92cc Merge upstream/main with -X ours; adapt to protocol and TUI changes
- core: replace StreamError with Error events; wrap EnvironmentContext args; fix let chains
- exec: drop StreamError handling (use Error)
- mcp-server: stub unsupported auth methods; remove StreamError match
- tui: wire transcript overlay types; add missing imports; initialize new fields
- composer: add Command variant to InputResult
- chatwidget: remove legacy dispatch_command; handle Command result
- lib: expose transcript_app module
- transcript_app: update to current TUI API (draw, size)
2025-08-22 23:38:20 +10:00
James Peter 9aad892f29 Merge remote-tracking branch 'upstream/main' into merge-upstream-20250822-232701 2025-08-22 23:27:10 +10:00
easong-openaiandGitHub a835451e86 Parse and expose stream errors (#2540) 2025-08-21 01:15:24 -07:00
James Peter d5f85fa3b5 fix(browser-console): improve console inspect output; update core exec events and TUI handling; minor cleanups 2025-08-20 18:27:44 +10:00
Michael BolinandGitHub 3624515ce5 chore: upgrade to Rust 1.89 (#2465)
Codex created this PR from the following prompt:

> upgrade this entire repo to Rust 1.89. Note that this requires
updating codex-rs/rust-toolchain.toml as well as the workflows in
.github/. Make sure that things are "clippy clean" as this change will
likely uncover new Clippy errors. `just fmt` and `cargo clippy --tests`
are sufficient to check for correctness

Note this modifies a lot of lines because it folds nested `if`
statements using `&&`.

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2465).
* #2467
* __->__ #2465
2025-08-19 13:22:02 -07:00
DylanandGitHub f485cbc1a5 [tui] Support /mcp command (#2430)
## Summary
Adds a `/mcp` command to list active tools. We can extend this command
to allow configuration of MCP tools, but for now a simple list command
will help debug if your config.toml and your tools are working as
expected.
2025-08-19 09:00:31 -07:00