Commit Graph
154 Commits
Author SHA1 Message Date
James Peter fec6aa0f0c fix(proxy+exec): harden proxy TLS forwarding (set servername, reset Host, normalize hop-by-hop headers) and make exec exit non-zero on agent Error events so workflows fail fast when streaming fails 2025-09-12 18:48:18 +10:00
James Peter dff216ec51 fix(exec/human_output): split stdout/stderr and add 'ERROR' divider on failed exec\n\nAlways compute truncated stdout and stderr.\nOn success, show dimmed stdout.\nOn failure, show dimmed stdout (if any), a visible 'ERROR' separator, then stderr without styling.\nThis keeps output readable when ANSI colors are stripped and clarifies which stream content came from. 2025-09-10 08:06:58 +10:00
just-every-code 8f2081ee70 merge(upstream): merge upstream/main into upstream-merge (by-bucket)
- Keep ours for TUI/CLI/workflows/docs; purge reintroduced rust-ci.yml
- Prefer theirs in protocol/deps; add serde_bytes + ICU/sys-locale/serde_with
- Keep our core codex/shell to preserve API; fix exec AuthMode import
- Make verify.sh CI-safe (no sccache; local CARGO dirs); check api_surface only

Build: ok; Verify: ok
2025-09-09 01:56:14 +00:00
jif-oaiandGitHub a9c68ea270 feat: Run cargo shear during CI (#3338)
Run cargo shear as part of the CI to ensure no unused dependencies
2025-09-09 01:05:08 +00:00
just-every-code 45eb379f81 chore(exec): restore our exec from origin/upstream-merge to eliminate merge artifacts 2025-09-09 00:36:16 +00:00
just-every-code 8bcf8bba6d merge(upstream): merge upstream/main into upstream-merge (by-bucket)\n\n- Prefer ours: TUI, CLI, workflows, docs\n- Prefer theirs: core/common/protocol/exec/file-search\n- Purged disallowed assets; resolved conflicts per policy 2025-09-09 00:21:14 +00:00
Justin LebarandGitHub 18330c2362 Format large numbers in a more readable way. (#2046)
- In the bottom line of the TUI, print the number of tokens to 3 sigfigs
  with an SI suffix, e.g. "1.23K".
- Elsewhere where we print a number, I figure it's worthwhile to print
  the exact number, because e.g. it's a summary of your session. Here we print
  the numbers comma-separated.
2025-09-08 21:48:48 +00:00
Gabriel PealandGitHub c8fab51372 Use ConversationId instead of raw Uuids (#3282)
We're trying to migrate from `session_id: Uuid` to `conversation_id:
ConversationId`. Not only does this give us more type safety but it
unifies our terminology across Codex and with the implementation of
session resuming, a conversation (which can span multiple sessions) is
more appropriate.

I started this impl on https://github.com/openai/codex/pull/3219 as part
of getting resume working in the extension but it's big enough that it
should be broken out.
2025-09-07 23:22:25 -04:00
pakrym-oaiandGitHub 0269096229 Move token usage/context information to session level (#3221)
Move context information into the main loop so it can be used to
interrupt the loop or start auto-compaction.
2025-09-06 15:19:23 +00:00
pakrym-oaiandGitHub 5775174ec2 Never store requests (#3212)
When item ids are sent to Responses API it will load them from the
database ignoring the provided values. This adds extra latency.

Not having the mode to store requests also allows us to simplify the
code.

## Breaking change

The `disable_response_storage` configuration option is removed.
2025-09-05 10:41:47 -07:00
James Peter 9e3e0d8604 feat(tui/layout): align composer/history wrap; add sanitize markers
Introduce layout_consts and replace hardcoded paddings to keep composer and history wrapping identical.

Extend sanitize::Options with debug_markers (default false) and plumb to history cell render paths.

Use codex-version crate for version reporting in TUI and wire crate deps.

Cleanup GitHub CI: remove unused workflows/assets; update issue-triage.
2025-09-05 23:07:36 +10:00
Ahmed IbrahimandGitHub 2b96f9f569 Dividing UserMsgs into categories to send it back to the tui (#3127)
This PR does the following:

- divides user msgs into 3 categories: plain, user instructions, and
environment context
- Centralizes adding user instructions and environment context to a
degree
- Improve the integration testing

Building on top of #3123

Specifically this
[comment](https://github.com/openai/codex/pull/3123#discussion_r2319885089).
We need to send the user message while ignoring the User Instructions
and Environment Context we attach.
2025-09-04 05:34:50 +00:00
Ahmed IbrahimandGitHub f2036572b6 Replay EventMsgs from Response Items when resuming a session with history. (#3123)
### Overview

This PR introduces the following changes:
	1.	Adds a unified mechanism to convert ResponseItem into EventMsg.
2. Ensures that when a session is initialized with initial history, a
vector of EventMsg is sent along with the session configuration. This
allows clients to re-render the UI accordingly.
	3. 	Added integration testing

### Caveats

This implementation does not send every EventMsg that was previously
dispatched to clients. The excluded events fall into two categories:
	•	“Arguably” rolled-out events
Examples include tool calls and apply-patch calls. While these events
are conceptually rolled out, we currently only roll out ResponseItems.
These events are already being handled elsewhere and transformed into
EventMsg before being sent.
	•	Non-rolled-out events
Certain events such as TurnDiff, Error, and TokenCount are not rolled
out at all.

### Future Directions

At present, resuming a session involves maintaining two states:
	•	UI State
Clients can replay most of the important UI from the provided EventMsg
history.
	•	Model State
The model receives the complete session history to reconstruct its
internal state.

This design provides a solid foundation. If, in the future, more precise
UI reconstruction is needed, we have two potential paths:
1. Introduce a third data structure that allows us to derive both
ResponseItems and EventMsgs.
2. Clearly divide responsibilities: the core system ensures the
integrity of the model state, while clients are responsible for
reconstructing the UI.
2025-09-04 04:47:00 +00:00
James Peter 94a9aa5131 fix(core): complete upstream integration
- Update all AuthManager::shared and CodexAuth::from_codex_home call sites with originator
- Use rollout::policy helper to filter persisted items
- Fix shell tool description formatting and UA helper reference
- Remove unused import in chatgpt_client

Build: ./build-fast.sh clean with zero warnings
2025-09-04 07:21:34 +10:00
pakrym-oaiandGitHub c636f821ae Add a common way to create HTTP client (#3110)
Ensure User-Agent and originator are always sent.
2025-09-03 10:11:02 -07:00
James Peter ed50f15337 merge(upstream): integrate upstream/main via merge branch; keep TUI fork; fix build\n\n- Create merge branch, merge origin/main and upstream/main with -X ours\n- Resolve conflicts: keep our TUI; drop upstream vt100 tests and snapshots\n- Decouple core from codex-login to break cycle; re-export auth in core\n- apply-patch: robust heredoc parsing via tree-sitter; minor fixes\n- protocol: align FileChange::Delete variants across core/wire code paths\n- TUI: adjust Git repo detection; simplify parsed command labels\n- mcp-server/exec: update imports and Delete handling\n- message_history: use fs2 locks; add dead_code allows\n\nValidated with ./build-fast.sh 2025-09-03 20:14:55 +10:00
James Peter 0ff794ed6b merge: integrate upstream/main (prefer ours); keep TUI fork, drop legacy vt100 tests, retain local common.rs 2025-09-03 19:49:35 +10:00
James Peter 06daa92b31 chore(codex-rs): clean up tree; add CLI order_replay
- Remove legacy TUI test suite and large fixtures
- Add cli/src/bin/order_replay.rs for output-order analysis
- Update TUI streaming and widgets; minor core protocol/client tweaks
- Update Cargo manifests/lockfile

Build verified with ./build-fast.sh
2025-09-03 19:47:03 +10:00
pakrym-oaiandGitHub 03e2796ca4 Move CodexAuth and AuthManager to the core crate (#3074)
Fix a long standing layering issue.
2025-09-02 18:36:19 -07:00
Jeremy RoseandGitHub e442ecedab rework message styling (#2877)
https://github.com/user-attachments/assets/cf07f62b-1895-44bb-b9c3-7a12032eb371
2025-09-02 17:29:58 +00:00
James Peter 4d9874fb0a merge(upstream-2025-08-29): integrate upstream main with -X ours; align web_search events; fix TUI popup APIs; clean warnings; build passes 2025-08-29 12:55:03 +10:00
Ahmed IbrahimandGitHub 9dbe7284d2 Following up on #2371 post commit feedback (#2852)
- Introduce websearch end to complement the begin 
- Moves the logic of adding the sebsearch tool to
create_tools_json_for_responses_api
- Making it the client responsibility to toggle the tool on or off 
- Other misc in #2371 post commit feedback
- Show the query:

<img width="1392" height="151" alt="image"
src="https://github.com/user-attachments/assets/8457f1a6-f851-44cf-bcca-0d4fe460ce89"
/>
2025-08-28 19:24:38 -07:00
dedrisian-oaiandGitHub b8e8454b3f Custom /prompts (#2696)
Adds custom `/prompts` to `~/.codex/prompts/<command>.md`.

<img width="239" height="107" alt="Screenshot 2025-08-25 at 6 22 42 PM"
src="https://github.com/user-attachments/assets/fe6ebbaa-1bf6-49d3-95f9-fdc53b752679"
/>

---

Details:

1. Adds `Op::ListCustomPrompts` to core.
2. Returns `ListCustomPromptsResponse` with list of `CustomPrompt`
(name, content).
3. TUI calls the operation on load, and populates the custom prompts
(excluding prompts that collide with builtins).
4. Selecting the custom prompt automatically sends the prompt to the
agent.
2025-08-29 02:16:39 +00:00
James Peter b1b01d09a6 Merge upstream/main into fork (prefer ours), integrate pager overlay; keep our transcript_app.rs for now 2025-08-29 11:50:50 +10:00
Michael BolinandGitHub 74d2741729 chore: require uninlined_format_args from clippy (#2845)
- added `uninlined_format_args` to `[workspace.lints.clippy]` in the
`Cargo.toml` for the workspace
- ran `cargo clippy --tests --fix`
- ran `just fmt`
2025-08-28 11:25:23 -07:00
dedrisian-oaiandGitHub 4e9ad23864 Add "View Image" tool (#2723)
Adds a "View Image" tool so Codex can find and see images by itself:

<img width="1772" height="420" alt="Screenshot 2025-08-26 at 10 40
04 AM"
src="https://github.com/user-attachments/assets/7a459c7b-0b86-4125-82d9-05fbb35ade03"
/>
2025-08-27 17:41:23 -07:00
Ahmed IbrahimandGitHub d0e06f74e2 send context window with task started (#2752)
- Send context window with task started
- Accounting for changing the model per turn
2025-08-27 00:04:21 -07:00
James Peter 3989fa28f1 Merge upstream/main into fork (prefer ours); resolve TUI conflicts by adopting upstream status_indicator_widget and clipboard_paste implementations to keep features and compatibility. 2025-08-26 16:52:04 +10:00
DylanandGitHub 7f7d1e30f3 [exec] Clean up apply-patch tests (#2648)
## Summary
These tests were getting a bit unwieldy, and they're starting to become
load-bearing. Let's clean them up, and get them working solidly so we
can easily expand this harness with new tests.

## Test Plan
- [x] Tests continue to pass
2025-08-25 15:08:01 -07:00
James Peter 31ac20b0e7 Merge upstream/main into merge/upstream-20250825 preferring ours (-X ours) 2025-08-25 20:50:54 +10:00
James Peter bee040acdc Diff wrapping: add one extra space to continuation hang indent\n\n- Continuation lines now reserve sign column + trailing space before content\n- Aligns wrapped rows exactly under the first-line content 2025-08-25 11:01:04 +10:00
James Peter af3a8bcfac Prefer CODE_VERSION with fallback to CARGO_PKG_VERSION across codex-rs\n\n- Replace direct env!("CARGO_PKG_VERSION") with option_env!("CODE_VERSION").unwrap_or(env!("CARGO_PKG_VERSION"))\n- Update clap version field, TUI upgrade banners, user agent, MCP client init, and OpenAI header\n- Verified via ./build-fast.sh 2025-08-25 08:39:26 +10:00
Jeremy RoseandGitHub 32bbbbad61 test: faster test execution in codex-core (#2633)
this dramatically improves time to run `cargo test -p codex-core` (~25x
speedup).

before:
```
cargo test -p codex-core  35.96s user 68.63s system 19% cpu 8:49.80 total
```

after:
```
cargo test -p codex-core  5.51s user 8.16s system 63% cpu 21.407 total
```

both tests measured "hot", i.e. on a 2nd run with no filesystem changes,
to exclude compile times.

approach inspired by [Delete Cargo Integration
Tests](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html),
we move all test cases in tests/ into a single suite in order to have a
single binary, as there is significant overhead for each test binary
executed, and because test execution is only parallelized with a single
binary.
2025-08-24 11:10:53 -07:00
James Peter a6b48eba23 chore: commit local changes 2025-08-24 21:11:26 +10:00
James Peter 0994b780b3 Merge upstream/main: prefer our TUI, integrate web_search and apply_patch config; resolve conflicts and wire flags 2025-08-24 19:24:15 +10:00
363636f5eb Add web search tool (#2371)
Adds web_search tool, enabling the model to use Responses API web_search
tool.
- Disabled by default, enabled by --search flag
- When --search is passed, exposes web_search_request function tool to
the model, which triggers user approval. When approved, the model can
use the web_search tool for the remainder of the turn
<img width="1033" height="294" alt="image"
src="https://github.com/user-attachments/assets/62ac6563-b946-465c-ba5d-9325af28b28f"
/>

---------

Co-authored-by: easong-openai <[email protected]>
2025-08-23 22:58:56 -07:00
James Peter afbfea47d2 merge: integrate upstream/main, fix protocol mismatches and compile errors (no TUI regressions) 2025-08-24 06:01:43 +10:00
James Peter ccb70079b6 Merge remote-tracking branch 'upstream/main' into merge/upstream-main-2025-08-24 2025-08-24 05:56:25 +10:00
957d44918d send-aggregated output (#2364)
We want to send an aggregated output of stderr and stdout so we don't
have to aggregate it stderr+stdout as we lose order sometimes.

---------

Co-authored-by: Gabriel Peal <[email protected]>
2025-08-23 16:54:31 +00:00
James Peter ffd1120234 fix(tui): clear full history region (incl. padding) each frame to eliminate rare margin artifacts and ensure background is fully painted 2025-08-24 00:57:37 +10:00
James Peter 52d29c51b0 Merge upstream/main; align protocol/core; revert TUI to fork and adapt; fix build 2025-08-23 18:13:52 +10:00
Ahmed IbrahimandGitHub 311ad0ce26 fork conversation from a previous message (#2575)
This can be the underlying logic in order to start a conversation from a
previous message. will need some love in the UI.

Base for building this: #2588
2025-08-22 17:06:09 -07:00
Jeremy RoseandGitHub d994019f3f tui: coalesce command output; show unabridged commands in transcript (#2590)
https://github.com/user-attachments/assets/effec7c7-732a-4b61-a2ae-3cb297b6b19b
2025-08-22 16:32:31 -07:00
DylanandGitHub 236c4f76a6 [apply_patch] freeform apply_patch tool (#2576)
## Summary
GPT-5 introduced the concept of [custom
tools](https://platform.openai.com/docs/guides/function-calling#custom-tools),
which allow the model to send a raw string result back, simplifying
json-escape issues. We are migrating gpt-5 to use this by default.

However, gpt-oss models do not support custom tools, only normal
functions. So we keep both tool definitions, and provide whichever one
the model family supports.

## Testing
- [x] Tested locally with various models
- [x] Unit tests pass
2025-08-22 13:42:34 -07:00
Eric TrautandGitHub dc42ec0eb4 Add AuthManager and enhance GetAuthStatus command (#2577)
This PR adds a central `AuthManager` struct that manages the auth
information used across conversations and the MCP server. Prior to this,
each conversation and the MCP server got their own private snapshots of
the auth information, and changes to one (such as a logout or token
refresh) were not seen by others.

This is especially problematic when multiple instances of the CLI are
run. For example, consider the case where you start CLI 1 and log in to
ChatGPT account X and then start CLI 2 and log out and then log in to
ChatGPT account Y. The conversation in CLI 1 is still using account X,
but if you create a new conversation, it will suddenly (and
unexpectedly) switch to account Y.

With the `AuthManager`, auth information is read from disk at the time
the `ConversationManager` is constructed, and it is cached in memory.
All new conversations use this same auth information, as do any token
refreshes.

The `AuthManager` is also used by the MCP server's GetAuthStatus
command, which now returns the auth method currently used by the MCP
server.

This PR also includes an enhancement to the GetAuthStatus command. It
now accepts two new (optional) input parameters: `include_token` and
`refresh_token`. Callers can use this to request the in-use auth token
and can optionally request to refresh the token.

The PR also adds tests for the login and auth APIs that I recently added
to the MCP server.
2025-08-22 13:10:11 -07:00
James Peter 0f1dfd8280 Merge upstream/main with -X ours; adapt to protocol and TUI changes
- core: replace StreamError with Error events; wrap EnvironmentContext args; fix let chains
- exec: drop StreamError handling (use Error)
- mcp-server: stub unsupported auth methods; remove StreamError match
- tui: wire transcript overlay types; add missing imports; initialize new fields
- composer: add Command variant to InputResult
- chatwidget: remove legacy dispatch_command; handle Command result
- lib: expose transcript_app module
- transcript_app: update to current TUI API (draw, size)
2025-08-22 23:38:20 +10:00
James Peter b1833fb4eb Merge remote-tracking branch 'upstream/main' into merge-upstream-20250822-232701 2025-08-22 23:27:10 +10:00
easong-openaiandGitHub 8ad56be06e Parse and expose stream errors (#2540) 2025-08-21 01:15:24 -07:00
James Peter a76618fa45 fix(browser-console): improve console inspect output; update core exec events and TUI handling; minor cleanups 2025-08-20 18:27:44 +10:00
Michael BolinandGitHub 50c48e88f5 chore: upgrade to Rust 1.89 (#2465)
Codex created this PR from the following prompt:

> upgrade this entire repo to Rust 1.89. Note that this requires
updating codex-rs/rust-toolchain.toml as well as the workflows in
.github/. Make sure that things are "clippy clean" as this change will
likely uncover new Clippy errors. `just fmt` and `cargo clippy --tests`
are sufficient to check for correctness

Note this modifies a lot of lines because it folds nested `if`
statements using `&&`.

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2465).
* #2467
* __->__ #2465
2025-08-19 13:22:02 -07:00