mirror of
https://github.com/hanzoai/dev.git
synced 2026-10-17 15:34:54 +00:00
chore: complete legacy test retirement and upstream crate adoption
This commit completes Phase 1 of the codebase cleanup plan documented in ROADMAP.md and related migration documents. Major changes: - Removed legacy test suites from code-rs/ crates (~35,000 lines) - Adopted upstream codex-mcp-client and codex-responses-api-proxy - Deleted TUI compat layer, agent task planning docs, and orphaned modules - Updated documentation to reflect current state and future phases Test Coverage: - Deleted tests from: app-server, apply-patch, chatgpt, cli, core, exec, login, mcp-server - Documented 9 remaining coverage gaps requiring new tests (see docs/migration/legacy-tests-retirement.md) - Preserved smoke test infrastructure in tui/tests/ui_smoke.rs Crate Adoptions: - mcp-client: Now re-exports codex-mcp-client with fork-specific buffer config - responses-api-proxy: Now re-exports codex-responses-api-proxy with minimal wrapper - mcp-types: Updated to use upstream types where possible TUI Cleanup: - Removed compat.rs ratatui adapter layer - Deleted obsolete modules: app_backtrack, custom_terminal, scroll_view, text_block, transcript_app - Archived agent_tasks/ planning docs to docs/archive/ Documentation Updates: - ROADMAP.md: Marked Phases 1-2 complete, updated next actions - TEST_SUITE_RESET.md: Documented Phase 1 completion - DEAD_CODE_INVENTORY.md: Updated with deletion outcomes - Added docs/migration/legacy-tests-retirement.md with 6-week plan - Added docs/maintenance/prompt-architecture.md Next Steps (Phase 2-3): - Port critical test coverage (9 test files identified) - Expand smoke test scenarios - Continue upstream tracking per monthly cadence See docs/migration/ for detailed migration status and roadmap. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
James Peter
@@ -0,0 +1,52 @@
|
||||
# Plain/Background/Loading/Wait Migration
|
||||
|
||||
## Summary
|
||||
- Replace all plain/background/loading/wait cell insertions and replacements in
|
||||
`chatwidget.rs` with `HistoryDomainEvent` calls so `HistoryState` becomes the
|
||||
source of truth for these stable cell types.
|
||||
- Remove any remaining uses of `history_record_from_cell` for these families
|
||||
and ensure IDs are assigned via the new domain-event helpers.
|
||||
|
||||
## Prerequisites
|
||||
- `HistoryDomainEvent` infrastructure landed and exported from
|
||||
`history/state.rs`.
|
||||
- Background system notices already routed through
|
||||
`history_replace_with_record` (completed 2025-09-27).
|
||||
|
||||
## Scope
|
||||
- `history_insert_plain_cell_with_key`, `history_push_plain_cell`, wait tool
|
||||
completions, loading spinners, and generic system helpers (`push_system_cell`).
|
||||
- Update `assign_history_id` branches that become redundant once the domain
|
||||
events populate IDs.
|
||||
- Extend `events_audit.md` with status notes as paths are migrated.
|
||||
|
||||
## Deliverables
|
||||
- New domain-event constructors for `PlainMessageState`, `WaitStatusState`,
|
||||
`LoadingState`, and `BackgroundEventRecord`.
|
||||
- `chatwidget.rs` updated to call `history_state.apply_domain_event(...)`
|
||||
(exact API name per infrastructure work).
|
||||
- Unit or integration coverage that emits events and confirms `HistoryState`
|
||||
stores the expected records without referencing `history_cells` directly.
|
||||
- Documentation update in `events_audit.md` marking these categories as
|
||||
migrated.
|
||||
|
||||
## Status (2025-09-27)
|
||||
- Domain-event constructors landed and `chatwidget.rs` now hydrates inserted
|
||||
plain/loading/wait/background cells from `HistoryRecord` responses.
|
||||
- `events_audit.md` updated to reflect the new flow.
|
||||
- Outstanding: add automated coverage for the hydration helpers so regressions
|
||||
are caught when more cell types migrate.
|
||||
|
||||
## Next Steps for Agent
|
||||
- Extend `chatwidget` tests (or add focused unit coverage) that drive
|
||||
`HistoryDomainEvent` inserts for plain/loading/wait/background cells and
|
||||
assert the resulting `HistoryState` records.
|
||||
- Confirm there are no remaining `history_record_from_cell` call sites for
|
||||
these variants; remove or document any unavoidable leftovers.
|
||||
- Update this file with findings and link any follow-up TODOs uncovered during
|
||||
testing.
|
||||
|
||||
## References
|
||||
- `code-rs/tui/HISTORY_CELLS_PLAN.md` – Step 3, Event Pipeline Consolidation
|
||||
Plan (Phase C – wave 1).
|
||||
- `code-rs/tui/src/chatwidget/events_audit.md` – mutation inventory.
|
||||
@@ -0,0 +1,51 @@
|
||||
# History Renderer Cache Implementation
|
||||
|
||||
## Summary
|
||||
- Implement the shared renderer cache described in Step 6 so per-cell layout
|
||||
caches (exec/assistant/diff) can be retired once all migrations are complete.
|
||||
- Provide APIs for cache lookup, invalidation, and telemetry that integrate with
|
||||
the `HistoryDomainEvent` pipeline.
|
||||
|
||||
## Prerequisites
|
||||
- Domain-event migrations for waves 1 & 2 complete (cells reconstruct from
|
||||
`HistoryState` without mutating `history_cells`).
|
||||
- `HistoryRenderState` ready to accept cache invalidation signals.
|
||||
|
||||
## Scope
|
||||
- Introduce `RenderedCell` structs, cache key types, and LRU storage inside
|
||||
`history_render.rs` (or a new module) based on the design sketch in
|
||||
`HISTORY_CELLS_PLAN.md`.
|
||||
- Expose invalidation hooks that respond to History mutations (insert/replace/
|
||||
remove) and viewport/theme changes.
|
||||
- Wire ChatWidget rendering paths to consult the shared cache before invoking
|
||||
cell-specific rendering logic.
|
||||
- Define instrumentation (counters or tracing) to measure hit/miss rates.
|
||||
- Plan removal of legacy per-cell caches once cache hit rates are validated.
|
||||
|
||||
## Deliverables
|
||||
- New caching module with unit tests covering key eviction, width/theme
|
||||
invalidation, and reasoning-visibility variants.
|
||||
- ChatWidget rendering updated to consume cached buffers for all migrated cell
|
||||
types.
|
||||
- Documentation update (HISTORY_CELLS_PLAN.md Step 6) noting completion and
|
||||
instructions for removing cell-local caches.
|
||||
|
||||
## Status (2025-09-27)
|
||||
- Prerequisites outstanding: exec/assistant/diff domain-event migrations still
|
||||
mutate cells directly, so this cache must remain on hold until those flows
|
||||
emit `HistoryDomainEvent`s.
|
||||
- Current work-in-progress skeletons: none.
|
||||
|
||||
## Next Steps for Agent (Design Spike)
|
||||
- Draft a lightweight design doc or code comments outlining cache key shapes,
|
||||
invalidation triggers, and integration points with `HistoryRenderState`.
|
||||
- Explore adding feature-gated scaffolding (e.g., new module with TODOs) without
|
||||
altering existing rendering behavior.
|
||||
- Identify instrumentation needs and note open dependencies on ongoing
|
||||
migrations.
|
||||
- Update this file with findings and explicitly list blockers before attempting
|
||||
implementation.
|
||||
|
||||
## References
|
||||
- `code-rs/tui/HISTORY_CELLS_PLAN.md` – Step 6 renderer cache design sketch.
|
||||
- `code-rs/tui/src/chatwidget/history_render.rs` – existing memoization hooks.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Streaming / Exec / Assistant Migration
|
||||
|
||||
## Summary
|
||||
- Move streaming deltas, finalized assistant messages, exec lifecycle updates,
|
||||
and merged exec summaries onto the `HistoryDomainEvent` pipeline.
|
||||
- Eliminate cell-side state mutations and redundant cached line buffers where
|
||||
practical, while preserving the interim cache bridge called out in Step 2.
|
||||
|
||||
## Prerequisites
|
||||
- Infrastructure: `HistoryDomainEvent` enums + helpers merged.
|
||||
- Wave 1 (plain/background/loading/wait) migrated so supporting utilities are
|
||||
battle-tested.
|
||||
|
||||
## Scope
|
||||
- Streaming handlers: `ensure_answer_stream_state`, tail updates, review-flow
|
||||
paths, and final message insertions (`chatwidget.rs:11380-11890`).
|
||||
- Exec lifecycle: active exec state, merged exec summaries, wait notes, and
|
||||
tool completion merges (`chatwidget.rs:6515-7330`).
|
||||
- Ensure `history_cell::ExecCell`, `StreamingContentCell`, and
|
||||
`AssistantMarkdownCell` rebuild purely from `HistoryState` data after events.
|
||||
- Update `assign_history_id` only where still required; prefer storing IDs via
|
||||
the domain events.
|
||||
|
||||
## Deliverables
|
||||
- Domain-event variants covering exec begin/end/stream chunks, assistant stream
|
||||
deltas, assistant final messages, and merged exec summaries.
|
||||
- ChatWidget handlers updated to emit domain events instead of mutating
|
||||
`history_cells` directly.
|
||||
- Regression tests (prefer snapshot-based tests around
|
||||
`chatwidget::tests::run_script`) validating that streaming/exec flows produce
|
||||
equivalent UI output after the migration.
|
||||
- Notes added to `events_audit.md` marking the executive/assistant entries as
|
||||
completed or updated with any residual technical debt.
|
||||
|
||||
## Status (2025-09-27)
|
||||
- Infrastructure: `HistoryDomainEvent` enums + hydration helpers now exist and
|
||||
wave 1 (plain/loading/wait/background) paths are migrated.
|
||||
- Exec/streaming handlers still mutate `history_cells` and downcast cells in
|
||||
place; caches must remain until the Step 6 renderer cache lands.
|
||||
- No automated coverage yet exercises the domain events for these flows.
|
||||
- ✅ Exec streaming deltas now emit `HistoryDomainEvent::UpdateExecStream`, and
|
||||
the resulting `HistoryRecord::Exec` is used to hydrate the running `ExecCell`.
|
||||
|
||||
## Next Steps for Agent
|
||||
- Sketch domain-event variants for exec lifecycle and assistant streaming while
|
||||
preserving the interim caches (do not remove them until the shared renderer
|
||||
cache is live).
|
||||
- Convert one representative exec path (e.g., command begin → output chunk →
|
||||
completion) and one assistant stream path to use the new domain events.
|
||||
- Add focused regression tests via `run_script` covering the converted paths.
|
||||
- Document remaining mutation spots in `events_audit.md` and update this file
|
||||
with progress notes or follow-up TODOs.
|
||||
|
||||
## References
|
||||
- `code-rs/tui/HISTORY_CELLS_PLAN.md` – Step 2 bridge & Step 3 Phase C wave 2.
|
||||
- `code-rs/tui/src/chatwidget/events_audit.md` – exec/stream mutation list.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Prompt Architecture Overview
|
||||
|
||||
Updated: 2025-10-05
|
||||
|
||||
This document captures the current prompt surface area for `code-rs`, notes
|
||||
where each prompt is consumed, and flags any fork-specific guidance that must
|
||||
be preserved when we sync with upstream Codex.
|
||||
|
||||
## Prompt Inventory
|
||||
|
||||
| Category | File | Runtime Usage | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Base instructions | `code-rs/core/prompt.md` | Loaded by default for non–GPT‑5 models | Mirrors upstream, kept in sync during merges. |
|
||||
| Base instructions | `code-rs/core/gpt_5_code_prompt.md` | Selected when GPT‑5/Codex family is active | Upstream content; differences tracked in `UPSTREAM_PARITY_CHECKLIST.md`. |
|
||||
| Fork overlay | `code-rs/core/prompt_coder.md` | Appended after base prompt in fork builds | Holds browser/tooling UX guidance and approve/sandbox policies. |
|
||||
| Review flows | `code-rs/core/review_prompt.md` | Used by review executor | Shared with upstream; lightweight customisations allowed. |
|
||||
| Compact summaries | `code-rs/core/templates/compact/prompt.md` | Generated during conversation compaction | Template strings only; keep deterministic for snapshots. |
|
||||
| Onboarding | `code-rs/tui/prompt_for_init_command.md` | `/init` command and AGENTS.md bootstrap | Fork-only instructions for building agent manifests. |
|
||||
| Operational playbooks | `prompts/MERGE.md` | Release & upstream merge checklist | Human-facing; live in repo root for quick reference. |
|
||||
| Operational playbooks | `prompts/TRIAGE.md` | Issue triage workflow | Human-facing; fork-specific wording. |
|
||||
|
||||
## Layering Strategy
|
||||
|
||||
1. **Base prompt** is selected by model family (`prompt.md` or
|
||||
`gpt_5_code_prompt.md`).
|
||||
2. **Fork overlay** (`prompt_coder.md`) is appended to inject tool UX, sandbox
|
||||
policy, and parallel-agent expectations. All edits here must remain fork
|
||||
scoped.
|
||||
3. **Scenario prompts** (review, compact, onboarding) are loaded as-needed.
|
||||
|
||||
This layering lets upstream change base instructions without colliding with our
|
||||
fork guidance. During merges, diff base prompts first, then verify overlay
|
||||
still reads correctly.
|
||||
|
||||
## Maintenance Guidelines
|
||||
|
||||
- When upstream updates a base prompt, record the delta in
|
||||
`UPSTREAM_PARITY_CHECKLIST.md` and regenerate local snapshots if wording
|
||||
changes.
|
||||
- Fork-specific copy belongs in `prompt_coder.md` or the operational playbooks;
|
||||
avoid patching upstream files unless we intend to upstream the change.
|
||||
- Avoid introducing new prompt files without documenting them here; stale
|
||||
prompts should move to `docs/archive/` so the active surface stays small.
|
||||
- Tests that rely on prompt text should treat strings as data fixtures to avoid
|
||||
tying unit tests to copy changes.
|
||||
|
||||
## Open Follow-ups
|
||||
|
||||
- Evaluate whether onboarding instructions in
|
||||
`code-rs/tui/prompt_for_init_command.md` can be slimmed down now that the
|
||||
`/init` command defaults to the fork UX.
|
||||
- For new agents or tooling flows, extend `prompt_coder.md` rather than forking
|
||||
additional prompt variants unless we need per-feature toggles.
|
||||
@@ -117,9 +117,11 @@ Validates that the fork still builds and maintains critical functionality after
|
||||
- Version handling intact
|
||||
- Branding consistency
|
||||
|
||||
## Typical Workflow
|
||||
## Monthly Cadence
|
||||
|
||||
### Weekly Upstream Check
|
||||
We follow a two-phase cadence. Use the first Monday for lightweight scans; reserve the second Monday for merge planning if changes warrant action.
|
||||
|
||||
### First Monday — Quick Diff
|
||||
|
||||
```bash
|
||||
# Fetch latest upstream
|
||||
@@ -128,12 +130,31 @@ git fetch upstream
|
||||
# Check for new commits
|
||||
git log HEAD..upstream/main --oneline
|
||||
|
||||
# If changes exist, start tracking
|
||||
# Generate structural diff
|
||||
./scripts/upstream-merge/diff-crates.sh --all
|
||||
./scripts/upstream-merge/highlight-critical-changes.sh --all
|
||||
|
||||
# Review high-level summary
|
||||
cat .github/auto/upstream-diffs/SUMMARY.md
|
||||
```
|
||||
|
||||
### Planning a Merge
|
||||
Goal: understand upstream churn. If nothing critical changed, stop here.
|
||||
|
||||
### Second Monday — Merge Planning (Conditional)
|
||||
|
||||
```bash
|
||||
# Highlight critical changes
|
||||
./scripts/upstream-merge/highlight-critical-changes.sh --all
|
||||
|
||||
# Review critical summary
|
||||
cat .github/auto/upstream-diffs/critical-changes/CRITICAL-SUMMARY.md
|
||||
|
||||
# Initialize merge log when planning an integration
|
||||
./scripts/upstream-merge/log-merge.sh init upstream/main
|
||||
```
|
||||
|
||||
Goal: categorize changes (adopt/adapt/preserve) and document decisions.
|
||||
|
||||
## Typical Workflow
|
||||
|
||||
```bash
|
||||
# 1. Initialize merge log
|
||||
|
||||
@@ -1,149 +1,18 @@
|
||||
# TUI Adapter Migration Status
|
||||
# TUI Adapter Status (Archived)
|
||||
|
||||
## Overview
|
||||
We briefly introduced `code-rs/tui/src/compat.rs` as a thin layer that
|
||||
re-exported ratatui types (
|
||||
`Buffer`, `Line`, `Rect`, `Style`, `Widget`, …) to ease mass refactors. The
|
||||
experiment proved unnecessary once the module system stabilized, so the adapter
|
||||
was removed on **2025-10-06**.
|
||||
|
||||
This document tracks the migration of `code-rs/tui` modules from direct `ratatui` imports to the internal `compat` adapter module. The adapter provides a stable internal API that insulates the codebase from upstream breaking changes.
|
||||
All call sites now import directly from ratatui, e.g.:
|
||||
|
||||
## Adapter Module
|
||||
|
||||
**Location:** `code-rs/tui/src/compat.rs`
|
||||
|
||||
The compat module re-exports commonly-used ratatui types:
|
||||
|
||||
- **Core rendering**: `Buffer`, `Rect`, `Layout`, `Constraint`, `Alignment`, `Margin`
|
||||
- **Styling**: `Color`, `Style`, `Modifier`, `Stylize`
|
||||
- **Text**: `Line`, `Span`, `Text` (Text only available under `#[cfg(test)]`)
|
||||
- **Widgets**: `Widget`, `WidgetRef`, `StatefulWidgetRef`, `Paragraph`, `Block`, `Borders`, `Clear`, `Table`, `Row`, `Cell`, `Wrap`
|
||||
|
||||
**Note:** Additional types will be added to the compat module as migration progresses. Unused re-exports are avoided to maintain zero-warning builds.
|
||||
|
||||
## Migration Pattern
|
||||
|
||||
### Before
|
||||
```rust
|
||||
use ratatui::buffer::Buffer;
|
||||
use ratatui::layout::Rect;
|
||||
use ratatui::style::Style;
|
||||
use ratatui::text::Line;
|
||||
use ratatui::widgets::Widget;
|
||||
```
|
||||
|
||||
### After
|
||||
```rust
|
||||
use crate::compat::{Buffer, Line, Rect, Style, Widget};
|
||||
```
|
||||
|
||||
## Migrated Modules (Representative Sample)
|
||||
|
||||
The following modules have been updated to use the compat adapter:
|
||||
|
||||
1. `src/compat.rs` - **New adapter module**
|
||||
2. `src/lib.rs` - Added `mod compat;` declaration
|
||||
3. `src/colors.rs` - Color utilities
|
||||
4. `src/height_manager.rs` - Layout management
|
||||
5. `src/render/line_utils.rs` - Line manipulation helpers
|
||||
6. `src/markdown_render_tests.rs` - Test module
|
||||
7. `src/onboarding/onboarding_screen.rs` - Onboarding coordinator
|
||||
8. `src/onboarding/trust_directory.rs` - Trust directory widget
|
||||
9. `src/bottom_pane/selection_popup_common.rs` - Selection UI helpers
|
||||
10. `src/bottom_pane/chat_composer.rs` - Chat input composer
|
||||
11. `src/bottom_pane/mcp_settings_view.rs` - MCP settings UI
|
||||
|
||||
These represent approximately **13%** of the 84 files with direct ratatui imports, covering:
|
||||
- Core utilities (colors, height_manager, render/line_utils)
|
||||
- Onboarding flow widgets
|
||||
- Bottom pane UI components
|
||||
- Test modules
|
||||
|
||||
## Remaining Work
|
||||
|
||||
### Coverage Analysis
|
||||
|
||||
**Total files with ratatui imports:** 84
|
||||
**Files migrated:** 11 (including compat.rs)
|
||||
**Files remaining:** 73
|
||||
|
||||
### Most Impactful Remaining Modules
|
||||
|
||||
Based on import frequency, the following modules should be prioritized:
|
||||
|
||||
1. `src/chatwidget.rs` (30+ imports) - Main chat widget
|
||||
2. `src/chatwidget/*.rs` submodules - Chat widget components
|
||||
3. `src/history_cell/*.rs` modules - History cell renderers
|
||||
4. `src/bottom_pane/*.rs` (remaining ~15 files) - UI panels
|
||||
5. `src/markdown_render.rs` - Markdown rendering
|
||||
6. `src/diff_render.rs` - Diff visualization
|
||||
7. `src/app.rs` - Main app coordinator
|
||||
|
||||
### Special Cases
|
||||
|
||||
The following patterns require careful handling:
|
||||
|
||||
**Renamed imports (keep as-is for now):**
|
||||
```rust
|
||||
use ratatui::text::Line as RtLine;
|
||||
use ratatui::text::Span as RtSpan;
|
||||
use ratatui::text::Text as RtText;
|
||||
```
|
||||
|
||||
**Prelude wildcards (migrate case-by-case):**
|
||||
```rust
|
||||
use ratatui::prelude::*; // Evaluate if compat::prelude or specific imports needed
|
||||
```
|
||||
|
||||
**Scrollbar symbols:**
|
||||
```rust
|
||||
use ratatui::symbols::scrollbar as scrollbar_symbols; // Already in compat
|
||||
```
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Phase 1: Foundation (✅ Complete)
|
||||
- Create `compat.rs` adapter module
|
||||
- Update representative sample across module categories
|
||||
- Validate pattern with surgical refactor
|
||||
|
||||
### Phase 2: Systematic Migration (Recommended Next Steps)
|
||||
1. Update all `bottom_pane/*.rs` modules (consistency in UI layer)
|
||||
2. Update all `chatwidget/*.rs` modules (largest subsystem)
|
||||
3. Update all `history_cell/*.rs` modules (rendering pipeline)
|
||||
4. Update `markdown_render.rs` and related markdown modules
|
||||
5. Update `app.rs` and top-level coordination modules
|
||||
|
||||
### Phase 3: Cleanup
|
||||
- Search for any remaining `use ratatui::` imports
|
||||
- Update renamed imports (RtLine, RtSpan, RtText) if beneficial
|
||||
- Consider prelude migration strategy
|
||||
|
||||
## Build Validation
|
||||
|
||||
After each batch of updates, run:
|
||||
|
||||
```bash
|
||||
cargo check --lib # Fast check for library code only
|
||||
./build-fast.sh --workspace code # Full build including binaries
|
||||
```
|
||||
|
||||
Expect: **Zero warnings**, clean build.
|
||||
|
||||
### Phase 1 Validation Results
|
||||
|
||||
```bash
|
||||
$ cargo check --lib
|
||||
Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.13s
|
||||
```
|
||||
|
||||
✅ **Zero warnings**, clean compilation confirmed for library code.
|
||||
|
||||
## Notes
|
||||
|
||||
- Keep changes surgical: modify only import statements, no logic changes
|
||||
- Maintain alphabetical ordering in multi-line import blocks
|
||||
- Prefer explicit imports over wildcards (except where prelude is clearly needed)
|
||||
- The adapter pattern allows future migration to a different TUI library with minimal code churn
|
||||
|
||||
## Last Updated
|
||||
|
||||
**Date:** 2025-10-05
|
||||
**Branch:** code-claude-audit-code-rs-tui-modules
|
||||
**Status:** Phase 1 complete, Phase 2 ready to begin
|
||||
This document is kept only as historical context; no further action is required.
|
||||
|
||||
Reference in New Issue
Block a user