feat(tui/history): drive exec and assistant streams via domain events

This commit is contained in:
James Peter
2025-09-27 12:55:59 +10:00
parent 384a1d02d3
commit 38abccc50d
10 changed files with 367 additions and 90 deletions
+3 -1
View File
@@ -99,7 +99,9 @@ Status legend: ✅ complete (semantic deterministic state ready), ⏳ still need
- **Needed:** emit `PatchRecord` entries in `HistoryState`, render from the recorded metadata, and retire the cached per-width line buffer.
### ⏳ Exec Commands – `ExecCell`
- **Status:** [~] `ExecRecord` lives in `HistoryState`, but rendering still depends on the legacy `ExecCell` caches.
- **Status:** [~] `ExecRecord` lives in `HistoryState`, but rendering still depends on the legacy `ExecCell` caches. Streaming stdout/stderr updates now emit
`HistoryDomainEvent::UpdateExecStream` so `HistoryState` remains the source of
truth for in-flight commands.
- **Current:** command metadata, layout caches, and wait state remain coupled to the UI struct instead of the serialized record.
- **Decision:** keep those caches energized until the Step 6 renderer cache is available; otherwise exec redraws regressed by >2× during testing.
- **Needed:** read/write `ExecRecord` via a dedicated `history_cell/exec.rs`, and move per-width layout caches into the renderer layer once the shared cache exists.
@@ -30,6 +30,22 @@
- Documentation update in `events_audit.md` marking these categories as
migrated.
## Status (2025-09-27)
- Domain-event constructors landed and `chatwidget.rs` now hydrates inserted
plain/loading/wait/background cells from `HistoryRecord` responses.
- `events_audit.md` updated to reflect the new flow.
- Outstanding: add automated coverage for the hydration helpers so regressions
are caught when more cell types migrate.
## Next Steps for Agent
- Extend `chatwidget` tests (or add focused unit coverage) that drive
`HistoryDomainEvent` inserts for plain/loading/wait/background cells and
assert the resulting `HistoryState` records.
- Confirm there are no remaining `history_record_from_cell` call sites for
these variants; remove or document any unavoidable leftovers.
- Update this file with findings and link any follow-up TODOs uncovered during
testing.
## References
- `codex-rs/tui/HISTORY_CELLS_PLAN.md` – Step 3, Event Pipeline Consolidation
Plan (Phase C – wave 1).
@@ -30,6 +30,22 @@
- Documentation update (HISTORY_CELLS_PLAN.md Step 6) noting completion and
instructions for removing cell-local caches.
## Status (2025-09-27)
- Prerequisites outstanding: exec/assistant/diff domain-event migrations still
mutate cells directly, so this cache must remain on hold until those flows
emit `HistoryDomainEvent`s.
- Current work-in-progress skeletons: none.
## Next Steps for Agent (Design Spike)
- Draft a lightweight design doc or code comments outlining cache key shapes,
invalidation triggers, and integration points with `HistoryRenderState`.
- Explore adding feature-gated scaffolding (e.g., new module with TODOs) without
altering existing rendering behavior.
- Identify instrumentation needs and note open dependencies on ongoing
migrations.
- Update this file with findings and explicitly list blockers before attempting
implementation.
## References
- `codex-rs/tui/HISTORY_CELLS_PLAN.md` – Step 6 renderer cache design sketch.
- `codex-rs/tui/src/chatwidget/history_render.rs` – existing memoization hooks.
@@ -32,6 +32,25 @@
- Notes added to `events_audit.md` marking the executive/assistant entries as
completed or updated with any residual technical debt.
## Status (2025-09-27)
- Infrastructure: `HistoryDomainEvent` enums + hydration helpers now exist and
wave 1 (plain/loading/wait/background) paths are migrated.
- Exec/streaming handlers still mutate `history_cells` and downcast cells in
place; caches must remain until the Step 6 renderer cache lands.
- No automated coverage yet exercises the domain events for these flows.
- ✅ Exec streaming deltas now emit `HistoryDomainEvent::UpdateExecStream`, and
the resulting `HistoryRecord::Exec` is used to hydrate the running `ExecCell`.
## Next Steps for Agent
- Sketch domain-event variants for exec lifecycle and assistant streaming while
preserving the interim caches (do not remove them until the shared renderer
cache is live).
- Convert one representative exec path (e.g., command begin → output chunk →
completion) and one assistant stream path to use the new domain events.
- Add focused regression tests via `run_script` covering the converted paths.
- Document remaining mutation spots in `events_audit.md` and update this file
with progress notes or follow-up TODOs.
## References
- `codex-rs/tui/HISTORY_CELLS_PLAN.md` – Step 2 bridge & Step 3 Phase C wave 2.
- `codex-rs/tui/src/chatwidget/events_audit.md` – exec/stream mutation list.
+90 -15
View File
@@ -4542,6 +4542,24 @@ impl ChatWidget<'_> {
return true;
}
}
HistoryRecord::Exec(state) => {
if let Some(exec) = cell
.as_any_mut()
.downcast_mut::<crate::history_cell::ExecCell>()
{
exec.sync_from_record(state);
return true;
}
}
HistoryRecord::AssistantStream(state) => {
if let Some(stream) = cell
.as_any_mut()
.downcast_mut::<crate::history_cell::StreamingContentCell>()
{
stream.update_from_state(state.clone(), &self.config);
return true;
}
}
HistoryRecord::RateLimits(state) => {
if let Some(rate_limits) = cell
.as_any_mut()
@@ -6962,17 +6980,50 @@ impl ChatWidget<'_> {
let call_id = ExecCallId(ev.call_id.clone());
if let Some(running) = self.exec.running_commands.get_mut(&call_id) {
let chunk = String::from_utf8_lossy(&ev.chunk).to_string();
match ev.stream {
ExecOutputStream::Stdout => running.stdout.push_str(&chunk),
ExecOutputStream::Stderr => running.stderr.push_str(&chunk),
}
let (stdout_chunk, stderr_chunk) = match ev.stream {
ExecOutputStream::Stdout => {
let offset = running.stdout.len();
running.stdout.push_str(&chunk);
(
Some(crate::history::state::ExecStreamChunk {
offset,
content: chunk.clone(),
}),
None,
)
}
ExecOutputStream::Stderr => {
let offset = running.stderr.len();
running.stderr.push_str(&chunk);
(
None,
Some(crate::history::state::ExecStreamChunk {
offset,
content: chunk.clone(),
}),
)
}
};
if let Some(idx) = running.history_index {
if idx < self.history_cells.len() {
if let Some(exec) = self.history_cells[idx]
.as_any_mut()
.downcast_mut::<history_cell::ExecCell>()
{
exec.update_stream_preview(&running.stdout, &running.stderr);
let mutation = self.history_state.apply_domain_event(
HistoryDomainEvent::UpdateExecStream {
index: idx,
stdout_chunk,
stderr_chunk,
},
);
if let HistoryMutation::Replaced {
record: HistoryRecord::Exec(exec_record),
..
} = mutation
{
if idx < self.history_cells.len() {
if let Some(exec) = self.history_cells[idx]
.as_any_mut()
.downcast_mut::<history_cell::ExecCell>()
{
exec.sync_from_record(&exec_record);
}
}
}
}
@@ -11648,9 +11699,11 @@ impl ChatWidget<'_> {
id,
Self::debug_fmt_order_key(key)
);
let new_idx = self.history_insert_with_key_global(
Box::new(history_cell::new_streaming_content(state, &self.config)),
let new_idx = self.history_insert_with_key_global_tagged(
Box::new(history_cell::new_streaming_content(state.clone(), &self.config)),
key,
"stream-begin",
Some(HistoryDomainRecord::AssistantStream(state)),
);
tracing::debug!(
"history.new StreamingContentCell at idx={} id={:?}",
@@ -11778,9 +11831,31 @@ impl ChatWidget<'_> {
received_at: SystemTime::now(),
})
};
self
.history_state
.upsert_assistant_stream_state(stream_id, preview, delta, None);
let mutation = self.history_state.apply_domain_event(
HistoryDomainEvent::UpsertAssistantStream {
stream_id: stream_id.to_string(),
preview_markdown: preview,
delta,
metadata: None,
},
);
if let HistoryMutation::Inserted { index, record, .. }
| HistoryMutation::Replaced { index, record, .. } = mutation
{
if let HistoryRecord::AssistantStream(state) = record {
if index < self.history_cells.len() {
if let Some(stream_cell) = self.history_cells[index]
.as_any_mut()
.downcast_mut::<history_cell::StreamingContentCell>()
{
stream_cell.update_from_state(state.clone(), &self.config);
}
} else {
self.refresh_streaming_cell_for_stream_id(stream_id, state);
}
}
}
}
fn finalize_answer_stream_state(
+6 -3
View File
@@ -30,9 +30,12 @@ Line numbers reference `codex-rs/tui/src/chatwidget.rs` as of 2025-09-27.
- final message hydration (`chatwidget.rs:11782-11860`)
- review-flow variants (`chatwidget.rs:11660-11747`)
These still rely on cell-owned caches and in-place mutation.
- **Exec lifecycle** – exec merge/update logic holds an active `ExecCell` in
`active_exec_cell` and downcasts cells when updates arrive
(`chatwidget.rs:7159-7184`, `7233-7242`).
- **Exec lifecycle** –
- ✅ Streaming deltas now flow through `HistoryDomainEvent::UpdateExecStream`
(`chatwidget.rs:6996-7034`), and the returned record hydrates the cached
`ExecCell` state.
- 🔲 End/update: merge logic still downcasts and mutates cells directly
(`chatwidget.rs:7159-7184`, `7233-7242`).
- **Background/system notices** – `push_system_cell` (`chatwidget.rs:1254`) now
uses `HistoryDomainRecord::BackgroundEvent` for inserts/replacements, so these
paths no longer reconstruct records from existing cells (migrated 2025-09-27).
+28 -3
View File
@@ -3,7 +3,9 @@
use super::ChatWidget;
use crate::app_event::AppEvent;
use crate::height_manager::HeightEvent;
use crate::history::state::ExecAction;
use crate::history::state::{
ExecAction, ExecRecord, ExecStatus, HistoryDomainRecord,
};
use crate::history_cell::CommandOutput;
use crate::history_cell::{self, HistoryCell};
use codex_core::parse_command::ParsedCommand;
@@ -38,6 +40,23 @@ fn find_trailing_explore_agg(chat: &ChatWidget<'_>) -> Option<usize> {
None
}
fn exec_record_from_begin(ev: &ExecCommandBeginEvent) -> ExecRecord {
let action = history_cell::action_enum_from_parsed(&ev.parsed_cmd);
ExecRecord {
id: crate::history::state::HistoryId::ZERO,
command: ev.command.clone(),
parsed: ev.parsed_cmd.clone(),
action,
status: ExecStatus::Running,
stdout_chunks: Vec::new(),
stderr_chunks: Vec::new(),
exit_code: None,
wait_notes: Vec::new(),
started_at: std::time::SystemTime::now(),
completed_at: None,
}
}
pub(super) fn finalize_exec_cell_at(
chat: &mut ChatWidget<'_>,
idx: usize,
@@ -554,9 +573,15 @@ pub(super) fn handle_exec_begin_now(
}
}
let cell = history_cell::new_active_exec_command(ev.command.clone(), parsed_command.clone());
let exec_record = exec_record_from_begin(&ev);
let key = ChatWidget::order_key_from_order_meta(order);
let idx = chat.history_insert_with_key_global(Box::new(cell), key);
let cell = history_cell::ExecCell::from_record(exec_record.clone());
let idx = chat.history_insert_with_key_global_tagged(
Box::new(cell),
key,
"exec-begin",
Some(HistoryDomainRecord::Exec(exec_record)),
);
chat.exec.running_commands.insert(
super::ExecCallId(ev.call_id.clone()),
super::RunningCommand {
+45 -1
View File
@@ -3,7 +3,7 @@
use super::*;
use crate::app_event::{AppEvent, BackgroundPlacement};
use crate::app_event_sender::AppEventSender;
use crate::history::state::{HistoryRecord, HistoryState};
use crate::history::state::{ExecStatus, HistoryRecord, HistoryState};
use crate::slash_command::SlashCommand;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
@@ -23,6 +23,8 @@ use codex_core::protocol::EventMsg;
use codex_core::protocol::ExecApprovalRequestEvent;
use codex_core::protocol::ExecCommandBeginEvent;
use codex_core::protocol::ExecCommandEndEvent;
use codex_core::protocol::ExecCommandOutputDeltaEvent;
use codex_core::protocol::ExecOutputStream;
use codex_core::protocol::FileChange;
use codex_core::protocol::RateLimitSnapshotEvent;
use codex_core::protocol::PatchApplyBeginEvent;
@@ -1007,6 +1009,48 @@ fn exec_history_cell_shows_working_then_failed() {
);
}
#[test]
fn exec_output_delta_updates_history_state() {
let (mut chat, _rx, _op_rx) = make_chatwidget_manual();
chat.handle_codex_event(Event {
id: "call-stream".into(),
msg: EventMsg::ExecCommandBegin(ExecCommandBeginEvent {
call_id: "call-stream".into(),
command: vec!["bash".into(), "-lc".into(), "echo streaming".into()],
cwd: std::env::current_dir().unwrap_or_else(|_| PathBuf::from(".")),
parsed_cmd: vec![codex_core::parse_command::ParsedCommand::Unknown {
cmd: "echo streaming".into(),
}],
}),
});
chat.handle_codex_event(Event {
id: "call-stream".into(),
msg: EventMsg::ExecCommandOutputDelta(ExecCommandOutputDeltaEvent {
call_id: "call-stream".into(),
stream: ExecOutputStream::Stdout,
chunk: b"hello".to_vec(),
}),
});
let exec_record = chat
.history_state()
.records
.iter()
.find_map(|record| match record {
HistoryRecord::Exec(rec) => Some(rec.clone()),
_ => None,
})
.expect("exec record present");
assert_eq!(exec_record.status, ExecStatus::Running);
assert_eq!(exec_record.stdout_chunks.len(), 1);
assert_eq!(exec_record.stdout_chunks[0].offset, 0);
assert_eq!(exec_record.stdout_chunks[0].content, "hello");
assert!(exec_record.stderr_chunks.is_empty());
}
#[tokio::test(flavor = "current_thread")]
async fn binary_size_transcript_matches_ideal_fixture() {
let (mut chat, rx, _op_rx) = make_chatwidget_manual();
+118 -38
View File
@@ -38,6 +38,17 @@ pub enum HistoryDomainEvent {
index: usize,
record: HistoryDomainRecord,
},
UpdateExecStream {
index: usize,
stdout_chunk: Option<ExecStreamChunk>,
stderr_chunk: Option<ExecStreamChunk>,
},
UpsertAssistantStream {
stream_id: String,
preview_markdown: String,
delta: Option<AssistantStreamDelta>,
metadata: Option<MessageMetadata>,
},
}
#[derive(Clone, Debug, PartialEq)]
@@ -47,6 +58,8 @@ pub enum HistoryDomainRecord {
Loading(LoadingState),
BackgroundEvent(BackgroundEventRecord),
RateLimits(RateLimitsRecord),
Exec(ExecRecord),
AssistantStream(AssistantStreamState),
}
impl From<PlainMessageState> for HistoryDomainRecord {
@@ -79,6 +92,18 @@ impl From<RateLimitsRecord> for HistoryDomainRecord {
}
}
impl From<ExecRecord> for HistoryDomainRecord {
fn from(state: ExecRecord) -> Self {
HistoryDomainRecord::Exec(state)
}
}
impl From<AssistantStreamState> for HistoryDomainRecord {
fn from(state: AssistantStreamState) -> Self {
HistoryDomainRecord::AssistantStream(state)
}
}
impl HistoryDomainRecord {
fn into_history_record(self) -> HistoryRecord {
match self {
@@ -102,6 +127,14 @@ impl HistoryDomainRecord {
state.id = HistoryId::ZERO;
HistoryRecord::RateLimits(state)
}
HistoryDomainRecord::Exec(mut state) => {
state.id = HistoryId::ZERO;
HistoryRecord::Exec(state)
}
HistoryDomainRecord::AssistantStream(mut state) => {
state.id = HistoryId::ZERO;
HistoryRecord::AssistantStream(state)
}
}
}
}
@@ -590,47 +623,17 @@ impl HistoryState {
delta: Option<AssistantStreamDelta>,
metadata: Option<&MessageMetadata>,
) -> HistoryId {
let now = SystemTime::now();
if let Some(idx) = self
.records
.iter()
.position(|record| matches!(record,
HistoryRecord::AssistantStream(state) if state.stream_id == stream_id))
{
if let Some(HistoryRecord::AssistantStream(state)) = self.records.get_mut(idx) {
if let Some(delta) = delta {
state.deltas.push(delta);
}
state.preview_markdown = preview_markdown;
if let Some(meta) = metadata {
state.citations = meta.citations.clone();
state.metadata = Some(meta.clone());
}
state.in_progress = true;
state.last_updated_at = now;
return state.id;
}
}
let mut deltas = Vec::new();
if let Some(delta) = delta {
deltas.push(delta);
}
let citations = metadata
.map(|meta| meta.citations.clone())
.unwrap_or_default();
let metadata = metadata.cloned();
let state = AssistantStreamState {
id: HistoryId(0),
let event = HistoryDomainEvent::UpsertAssistantStream {
stream_id: stream_id.to_string(),
preview_markdown,
deltas,
citations,
metadata,
in_progress: true,
last_updated_at: now,
delta,
metadata: metadata.cloned(),
};
self.push(HistoryRecord::AssistantStream(state))
match self.apply_domain_event(event) {
HistoryMutation::Inserted { id, .. }
| HistoryMutation::Replaced { id, .. } => id,
_ => HistoryId::ZERO,
}
}
pub fn finalize_assistant_stream_state(
@@ -766,6 +769,83 @@ impl HistoryState {
let record = record.into_history_record();
self.apply_event(HistoryEvent::Replace { index, record })
}
HistoryDomainEvent::UpdateExecStream {
index,
stdout_chunk,
stderr_chunk,
} => {
if let Some(HistoryRecord::Exec(existing)) = self.records.get(index).cloned() {
let mut updated = existing;
if let Some(chunk) = stdout_chunk {
updated.stdout_chunks.push(chunk);
}
if let Some(chunk) = stderr_chunk {
updated.stderr_chunks.push(chunk);
}
self.apply_event(HistoryEvent::Replace {
index,
record: HistoryRecord::Exec(updated),
})
} else {
HistoryMutation::Noop
}
}
HistoryDomainEvent::UpsertAssistantStream {
stream_id,
preview_markdown,
delta,
metadata,
} => {
let now = SystemTime::now();
if let Some(idx) = self.records.iter().position(|record| {
matches!(record,
HistoryRecord::AssistantStream(state) if state.stream_id == stream_id)
}) {
if let Some(HistoryRecord::AssistantStream(existing)) =
self.records.get(idx).cloned()
{
let mut updated = existing;
if let Some(delta) = delta {
updated.deltas.push(delta);
}
updated.preview_markdown = preview_markdown;
if let Some(meta) = metadata.clone() {
updated.citations = meta.citations.clone();
updated.metadata = Some(meta);
}
updated.in_progress = true;
updated.last_updated_at = now;
return self.apply_event(HistoryEvent::Replace {
index: idx,
record: HistoryRecord::AssistantStream(updated),
});
}
}
let mut deltas = Vec::new();
if let Some(delta) = delta {
deltas.push(delta);
}
let citations = metadata
.as_ref()
.map(|meta| meta.citations.clone())
.unwrap_or_default();
let assistant_state = AssistantStreamState {
id: HistoryId::ZERO,
stream_id,
preview_markdown,
deltas,
citations,
metadata,
in_progress: true,
last_updated_at: now,
};
let record = HistoryRecord::AssistantStream(assistant_state);
self.apply_event(HistoryEvent::Insert {
index: self.records.len(),
record,
})
}
}
}
}
+26 -29
View File
@@ -790,38 +790,35 @@ impl ExecCell {
(pre, out, status)
}
pub(crate) fn update_stream_preview(&mut self, stdout: &str, stderr: &str) {
if stdout.is_empty() && stderr.is_empty() {
if self.stream_preview.is_none() {
return;
pub(crate) fn sync_from_record(&mut self, record: &ExecRecord) {
self.record = record.clone();
self.command = record.command.clone();
self.parsed = record.parsed.clone();
self.has_bold_command = command_has_bold_token(&self.command);
if matches!(record.status, ExecStatus::Running) {
let stdout = chunks_to_string(&record.stdout_chunks);
let stderr = chunks_to_string(&record.stderr_chunks);
if stdout.is_empty() && stderr.is_empty() {
self.stream_preview = None;
} else {
self.stream_preview = Some(CommandOutput {
exit_code: STREAMING_EXIT_CODE,
stdout,
stderr,
});
}
self.stream_preview = None;
self.record.stdout_chunks.clear();
self.record.stderr_chunks.clear();
} else {
self.stream_preview = Some(CommandOutput {
exit_code: STREAMING_EXIT_CODE,
stdout: stdout.to_string(),
stderr: stderr.to_string(),
});
self.record.stdout_chunks = if stdout.is_empty() {
Vec::new()
} else {
vec![ExecStreamChunk {
offset: 0,
content: stdout.to_string(),
}]
};
self.record.stderr_chunks = if stderr.is_empty() {
Vec::new()
} else {
vec![ExecStreamChunk {
offset: 0,
content: stderr.to_string(),
}]
};
self.stream_preview = None;
}
self.invalidate_render_caches();
// Invalidate cached layouts so the refreshed state re-renders.
self.cached_display_lines.borrow_mut().take();
self.cached_pre_lines.borrow_mut().take();
self.cached_out_lines.borrow_mut().take();
self.cached_layout.borrow_mut().take();
self.cached_command_lines.borrow_mut().take();
self.cached_wait_extras.borrow_mut().take();
}
fn exec_render_parts_generic(