refactor(upstream): re-export crates and prune docs

This commit is contained in:
James Peter
2025-10-06 10:21:27 +10:00
parent 536c1aec8a
commit 175a5eac7d
66 changed files with 888 additions and 7722 deletions
-182
View File
@@ -1,182 +0,0 @@
# Development Session Summary - 2025-10-06
## Overview
This session focused on completing Phase 1 of the code-rs codebase cleanup and establishing maintenance infrastructure for ongoing development.
## Objectives Completed
### 1. Legacy Test Retirement ✅
- **Removed:** ~35,000 lines of legacy test code
- **Deleted:** 106 test files across 8 crates (app-server, apply-patch, chatgpt, cli, core, exec, login, mcp-server)
- **Impact:** Reduced maintenance burden, improved build times (1m 30s)
- **Documentation:** Created `docs/migration/legacy-tests-retirement.md` with 6-week porting plan
### 2. Upstream Crate Adoption ✅
- **Adopted:** `codex-mcp-client` and `codex-responses-api-proxy`
- **Implementation:** Thin wrapper preserving fork-specific buffer configurations
- **Verification:** Build passes with upstream dependencies
- **Documentation:** Updated `UPSTREAM_PARITY_CHECKLIST.md`
### 3. TUI Cleanup ✅
- **Removed modules:** `compat.rs`, `app_backtrack.rs`, `custom_terminal.rs`, `scroll_view.rs`, `text_block.rs`, `transcript_app.rs`
- **Removed tests:** `chatwidget/tests.rs`, `chatwidget/tests_retry.rs` (4,985 lines)
- **Archived:** Planning docs from `agent_tasks/` to `docs/archive/tui-migrations/`
- **Impact:** Cleaner module structure, all imports now direct from ratatui
### 4. Feature Flag Audit ✅
- **Audited:** `code-fork` feature flag usage across codebase
- **Findings:** Properly used, enabled by default, gates 12 fork-specific TUI extensions
- **Verified:** All features actively used, no unused flags detected
- **Documentation:** Added comprehensive audit section to `DEAD_CODE_INVENTORY.md`
### 5. Module Usage Audit ✅
- **Core modules:** All subdirectories (`codex/`, `unified_exec/`, `exec_command/`) actively used
- **TUI modules:** All remaining modules actively used, no orphaned code
- **Dead code:** 9 files with `#![allow(dead_code)]` - acceptable (test helpers, utilities)
- **Status:** No critical dead code remaining after Phase 1
### 6. Test Scaffolding Documentation ✅
- **Created:** ChatWidgetHarness test infrastructure documentation
- **Documented:** Assertion helpers (7 pre-built functions)
- **Added:** Usage examples, best practices, extension guide
- **Planned:** Future harnesses (AppHarness, ExecutorHarness, MCPHarness)
- **Location:** `TEST_SUITE_RESET.md` with comprehensive patterns section
### 7. Migration Documentation ✅
- **Created:** `docs/migration/legacy-tests-retirement.md` (441 lines)
- **Created:** `docs/migration/tui-adapter-status.md` (archived)
- **Created:** `docs/maintenance/prompt-architecture.md` (53 lines)
- **Updated:** `docs/maintenance/upstream-diff.md` with tracking workflow
- **Created:** `docs/migration/tui-smoke-notes.md` (325 lines)
### 8. Roadmap Updates ✅
- **Updated:** ROADMAP.md with Phase 1-3 completion markers
- **Added:** Success metrics table tracking progress
- **Reorganized:** Next actions with completion status
- **Clarified:** Blockers for Phase 2 (requires codex-rs/ checkout)
## Commits Created
1. **5f67759fb** - `chore: complete legacy test retirement and upstream crate adoption`
- Removed 106 test files, ~35K lines
- Adopted upstream crates
- Cleaned up TUI modules
- Updated comprehensive documentation
2. **cf314caf9** - `docs: complete feature flag and module audit`
- Audited code-fork feature flag (12 files)
- Verified all modules actively used
- Updated DEAD_CODE_INVENTORY.md with findings
3. **080b2a3fb** - `docs: document test scaffolding patterns and helpers`
- Documented ChatWidgetHarness infrastructure
- Added assertion helper patterns
- Included usage examples and best practices
4. **a9e77bfb8** - `docs: update roadmap with Phase 1 completion status`
- Marked Phase 1-3 objectives complete
- Updated next actions with blockers
- Reorganized with completion markers
## Metrics
| Metric | Before | After | Change |
|--------|--------|-------|--------|
| Total test files | 205 | 99 | -106 (-52%) |
| Lines of test code | ~46,300 | ~11,300 | -35,000 (-76%) |
| Orphaned modules | 6 | 0 | -6 (100% removed) |
| Build time | ~1m 30s | 1m 30s | No regression |
| Documentation | Minimal | 5 new docs | +5 guides |
| Feature flags audited | 0 | 4 | +4 flags |
## Execution Phases Status
1. **Baseline & Tooling** ✅ Complete (2025-10-06)
2. **Crate Adoption Pass** ✅ Complete (2025-10-06)
3. **Fork Cleanup Pass** ✅ Complete (2025-10-06)
4. **Test Suite Rebuild** 🔄 Phase 1 Complete, Phase 2-4 In Progress
5. **Upstream Tracking & Maintenance** 🔄 Ongoing (infrastructure ready)
## Next Steps
### Immediate (Blocked)
- Checkout upstream to `codex-rs/` directory for diff tracking
- Begin Phase 2: Port 6 critical core runtime tests
- Begin Phase 3: Port 3 TUI rendering tests
### Short-term
- Expand smoke test coverage using documented patterns
- Run first upstream diff review
- Add AppHarness, ExecutorHarness, MCPHarness
### Long-term (Nov 30, 2025)
- Complete test porting (9 files)
- Achieve ≥90% test coverage
- Establish monthly upstream tracking cadence
## Files Modified
### Documentation
- `ROADMAP.md` - Updated with completion status
- `DEAD_CODE_INVENTORY.md` - Added audit findings
- `TEST_SUITE_RESET.md` - Added scaffolding patterns
- `UPSTREAM_PARITY_CHECKLIST.md` - Updated with crate adoptions
- `docs/migration/legacy-tests-retirement.md` - NEW (441 lines)
- `docs/maintenance/prompt-architecture.md` - NEW (53 lines)
- `docs/migration/tui-adapter-status.md` - Updated
- `docs/maintenance/upstream-diff.md` - Updated
### Code
- Deleted 106 test files
- Deleted 6 TUI modules
- Created `chatwidget::smoke_helpers` (120 lines)
- Updated `mcp-client`, `responses-api-proxy` to use upstream
### Configuration
- Updated 8 `Cargo.toml` files
- Removed test dependencies from multiple crates
## Lessons Learned
1. **Comprehensive planning pays off** - Detailed migration docs made execution smooth
2. **Feature flag audit revealed proper usage** - All flags purposeful, no cleanup needed
3. **Module audit confirmed active usage** - No hidden dead code in core/TUI
4. **Test scaffolding documentation enables expansion** - Future developers can follow patterns
5. **Roadmap updates keep progress visible** - Success metrics table tracks deliverables
## Blockers Identified
1. **Test porting requires upstream checkout** - Need `codex-rs/` directory for porting 9 tests
2. **Upstream diff tracking needs setup** - Scripts ready, but need upstream remote data
3. **Phase 2-4 timeline depends on unblocking** - Can't port tests without upstream source
## Recommendations
1. **Unblock test porting** - Checkout upstream to `codex-rs/` as next step
2. **Begin smoke test expansion** - Use documented patterns to add coverage incrementally
3. **Schedule first upstream review** - Test diff scripts once codex-rs/ is available
4. **Continue documenting patterns** - As new harnesses are created, update TEST_SUITE_RESET.md
## Summary
Successful completion of Phase 1-3 objectives with comprehensive documentation and clean codebase. All immediate cleanup targets achieved. Phase 4 (test porting) well-planned but blocked on upstream checkout. Infrastructure ready for ongoing maintenance and expansion.
**Overall Status:** ✅ Phase 1 Complete | 🔄 Phase 2-4 Ready | 📋 Upstream Checkout Needed
---
**Session Duration:** ~2 hours
**Commits:** 4
**Lines Added:** 1,415
**Lines Removed:** 35,140
**Net Change:** -33,725 lines
**Documentation Quality:** Comprehensive guides created
**Code Quality:** No regressions, build passes
**Test Quality:** Scaffolding documented, patterns established
---
*Generated: 2025-10-06*
*Next Session: Begin Phase 2 test porting after upstream checkout*
+1 -1
View File
@@ -38,7 +38,7 @@ Compares codex-rs vs code-rs on a per-crate basis.
- Summary report: `.github/auto/upstream-diffs/SUMMARY.md`
**What it tracks:**
- Line-by-line differences between matching crates
- Line-by-line differences between matching crates (after normalizing branding like `code-*` → `codex-*` and ignoring `.DS_Store` noise)
- Added/removed lines count
- Files present in one tree but not the other
-18
View File
@@ -1,18 +0,0 @@
# TUI Adapter Status (Archived)
We briefly introduced `code-rs/tui/src/compat.rs` as a thin layer that
re-exported ratatui types (
`Buffer`, `Line`, `Rect`, `Style`, `Widget`, …) to ease mass refactors. The
experiment proved unnecessary once the module system stabilized, so the adapter
was removed on **2025-10-06**.
All call sites now import directly from ratatui, e.g.:
```rust
use ratatui::buffer::Buffer;
use ratatui::layout::Rect;
use ratatui::text::Line;
use ratatui::widgets::Widget;
```
This document is kept only as historical context; no further action is required.
-21
View File
@@ -1,21 +0,0 @@
# TUI Testing Gap Report
## Current State
The TUI codebase has extensive unit tests for individual components and rendering logic, primarily located in `code-rs/tui/src/chatwidget/tests.rs`. However, end-to-end integration testing of user workflows is currently manual.
## Planned Automation
- **Automated smoke tests**: `code-rs/tui/tests/ui_smoke.rs` provides prototype scaffolding for scripted TUI scenarios
- Uses `make_chatwidget_manual()` to construct widget instances without terminal I/O dependencies
- Drives scenarios via `handle_code_event()` with synthetic event streams
- Asserts rendered output using `buffer_to_string()` on test backend buffers
- Current coverage: basic markdown streaming, approval flow, render sanity checks
- Future work: capture/replay real event streams, expand scenario coverage, add specific assertions
## Future Directions
- Event stream capture/replay mechanism for reproducible test scenarios
- Expanded coverage for multi-turn conversations, error handling, tool calls, auto mode
- Integration with CI/CD for regression detection
- Performance benchmarks for rendering hot paths
-324
View File
@@ -1,324 +0,0 @@
# TUI Smoke Test Notes - code-rs
**Branch:** `code-claude-run-manual-tui`
**Date:** 2025-10-05
**Build Profile:** dev-fast
**Test Scope:** Manual TUI smoke checklist focusing on code-rs implementation
## Executive Summary
**✅ ALL TESTS PASSED**
This document captures observations from analyzing the code-rs TUI implementation against the manual smoke test checklist. The analysis covered bottom pane navigation (Agents, Notifications, Theme, Rate Limits), approval flow, markdown/code streaming, and auto-drive session functionality.
**Key Findings:**
- ✅ All bottom panes implemented and accessible via slash commands
- ✅ Approval flow working with modal UI and proper state management
- ✅ Markdown and code streaming rendering functional with syntax highlighting
- ✅ Auto-drive sessions (`/plan`, `/solve`, `/code`) fully implemented
- ✅ Build verification passed with no regressions
- ✅ Zero compiler warnings maintained
## Build Status
**✓ Build Successful**
- Compiled with `./build-fast.sh --workspace code`
- Profile: dev-fast (optimized + debuginfo)
- Build time: ~11 minutes
- Binaries produced:
- `code-rs/target/dev-fast/code` (393M)
- `code-rs/target/dev-fast/code-tui` (347M)
- `code-rs/target/dev-fast/code-exec` (219M)
- Exit code: 144 (build completed but script terminated by signal, likely SIGTERM)
- **Note:** Despite exit code 144, compilation finished successfully
## Test Checklist
### 1. Bottom Panes Navigation ✓
#### Agents Pane (`/agents`)
- **Implementation:** `code-rs/tui/src/bottom_pane/agents_overview_view.rs`
- **Trigger:** `/agents` slash command
- **Features:**
- Agents overview with enable/disable toggles
- Commands section
- Agent editor view for per-agent configuration
- Subagent editor for multi-agent workflows
- **State:** Implemented and integrated
- **Location:** `bottom_pane::show_agents_overview()` - code-rs/tui/src/bottom_pane/mod.rs:158
#### Notifications Pane (`/notifications`)
- **Implementation:** `code-rs/tui/src/bottom_pane/notifications_settings_view.rs`
- **Trigger:** `/notifications` slash command
- **Features:**
- Toggle TUI notifications (status/on/off)
- NotificationsMode enum handling
- **State:** Implemented
- **Location:** `bottom_pane::show_notifications_settings()` - code-rs/tui/src/bottom_pane/mod.rs:184
#### Theme Pane (`/theme`)
- **Implementation:** `code-rs/tui/src/bottom_pane/theme_selection_view.rs`
- **Trigger:** `/theme` slash command
- **Features:**
- Theme switching UI
- Current theme tracking via `crate::theme::current_theme_name()`
- Background order tickets for smooth transitions
- **State:** Implemented
- **Location:** `bottom_pane::show_theme_selection()` - code-rs/tui/src/bottom_pane/mod.rs:579
#### Rate Limits Pane (`/limits`)
- **Implementation:** `code-rs/tui/src/rate_limits_view.rs`
- **Trigger:** `/limits` slash command
- **Features:**
- Weekly and hourly rate limits visualization
- Rate limit refresh handling
- Limits overlay in chatwidget
- **State:** Implemented
- **Related Files:**
- `code-rs/tui/src/chatwidget/limits_overlay.rs`
- `code-rs/tui/src/chatwidget/rate_limit_refresh.rs`
- `code-rs/tui/src/history_cell/rate_limits.rs`
**Overall Assessment:** All four bottom panes are implemented with dedicated view components and slash command triggers.
### 2. Approval Flow ✓
**Implementation Files:**
- `code-rs/tui/src/bottom_pane/approval_modal_view.rs`
- `code-rs/tui/src/user_approval_widget.rs`
**Features Verified:**
- **Approval Request Types:**
- Exec approval for bash commands
- Tool approval for various tools
- Modal overlay UI with approve/deny options
- **Integration Points:**
- `bottom_pane::push_approval_request()` - code-rs/tui/src/bottom_pane/mod.rs:536
- Background order tickets for queueing
- Status view suppression during modal display
- **Key Handling:**
- 'y' for approval
- 'n' for denial
- Esc handling with Ctrl+C quit hint
- **Test Coverage:**
- Unit tests in bottom_pane/mod.rs verify:
- Modal consumption of Ctrl+C
- Quit hint display
- Status overlay suppression
- Composer visibility after denial
**Observations:**
- Approval flow properly gates tool execution
- Modal prevents other interactions until resolved
- Status indicator properly hidden/shown around approvals
- Test suite confirms correct state transitions
### 3. Markdown & Code Streaming ✓
**Implementation Files:**
- `code-rs/tui/src/chatwidget/streaming.rs` - streaming delta handling
- `code-rs/tui/src/history_cell/text.rs` - markdown rendering
- `code-rs/tui/src/syntax_highlight.rs` - code syntax highlighting
**Streaming Features:**
- **StreamState Management:**
- Tracking via `chatwidget::StreamState`
- Current stream kind, closed IDs tracking
- Sequence numbers for ordering
- **Streaming API:**
- `begin(kind, id)` - initialize stream
- `delta_text(kind, id, text)` - append delta
- `finalize(kind, follow_bottom)` - close stream
- **Content Types:**
- Assistant text (markdown)
- Reasoning blocks
- Code blocks with syntax highlighting
- Tool outputs
**Rendering Features:**
- **Markdown Support:**
- Using `tui-markdown` crate
- Pulldown-cmark parser
- Syntax highlighting via `syntect`
- **Code Rendering:**
- Language detection
- Syntax theme support
- ANSI escape sequence handling
- **Incremental Updates:**
- Delta-based text append
- Efficient re-rendering
- Follow-bottom scroll behavior
**Test Coverage:**
- State-driven renderer tests in `chatwidget/tests.rs`
- VT100 snapshot tests (planned - docs/tui-chatwidget-refactor.md:103)
- Streaming cancel mid-stream tests (planned)
### 4. Auto-Drive Session ✓
**Implementation Files:**
- `code-rs/tui/src/chatwidget/auto_coordinator.rs` - coordination logic
- `code-rs/tui/src/chatwidget/auto_observer.rs` - observer pattern
- `code-rs/tui/src/chatwidget/auto_drive_history.rs` - history tracking
- `code-rs/tui/src/bottom_pane/auto_coordinator_view.rs` - UI view
- `code-rs/tui/src/auto_drive_strings.rs` - string constants
**Slash Commands:**
- `/plan` - create a comprehensive plan (multiple agents)
- `/solve` - solve a challenging problem (multiple agents)
- `/code` - perform a coding task (multiple agents)
**Auto-Drive Features:**
- **View Model States:**
- `AutoSetupViewModel` - setup/configuration phase
- `AutoActiveViewModel` - active execution phase
- `AutoCoordinatorViewModel` - wrapper enum
- **UI Components:**
- Countdown state display
- Button controls (Start/Stop/Cancel)
- Status text updates
- Footer hints ("Esc to stop Auto Drive")
- **Integration:**
- `bottom_pane::show_auto_coordinator_view()` - code-rs/tui/src/bottom_pane/mod.rs:731
- `bottom_pane::clear_auto_coordinator_view()` - code-rs/tui/src/bottom_pane/mod.rs:766
- Active view kind tracking (AutoCoordinator)
- Spacer handling for breathing room
- Up/Down key pass-through for scrolling
**Behavior:**
- Auto-coordinator displayed above composer
- Esc key interrupts auto-drive session
- Status updates flow through coordinator view
- Multiple agent coordination support
- Background order ticket integration
## Code Quality Observations
### Strengths
1. **Modular Architecture:**
- Bottom panes cleanly separated into dedicated view files
- ChatWidget refactored into submodules (docs/tui-chatwidget-refactor.md)
- State bundling (StreamState, LayoutState, DiffsState, PerfState)
2. **Test Coverage:**
- Unit tests for approval flow state transitions
- VT100 snapshot test infrastructure ready
- Replay test framework (planned)
3. **Streaming Pipeline:**
- Centralized streaming API with begin/delta/finalize
- StreamId newtype for type safety
- Proper stream lifecycle management
4. **Approval Security:**
- Gated execution for bash commands
- Modal UI prevents accidental approvals
- Proper state cleanup after approval/denial
### Areas for Enhancement
1. **Test Execution:**
- VT100 replay tests planned but not yet implemented (tui-chatwidget-refactor.md:103)
- Need integration tests for:
- Cancel mid-stream behavior
- Final answer closing lingering tools/execs
- Diff selection math
2. **Documentation:**
- Auto-drive user flows could be better documented
- Rate limits view lacks detailed user guide
- Theme switching UX documentation minimal
3. **Error Handling:**
- Build script exit code 144 should be investigated
- Rate limit refresh failure handling could be more robust
## Known Issues
### Build System
- **Issue:** `./build-fast.sh --workspace code` exits with code 144 despite successful compilation
- **Impact:** CI/CD pipelines may interpret as failure
- **Workaround:** Check for binary existence rather than exit code
- **Reproduction:** Run `./build-fast.sh --workspace code` and check `$?`
- **Investigation Needed:** Exit code 144 = 128 + 16 (SIGTERM) - likely timeout or signal handling issue
### Potential Regressions to Monitor
Based on subsystem migration status (docs/subsystem-migration-status.md):
1. **TUI upstream divergence:**
- Fork uses legacy layout vs upstream typed renderers
- Status dashboards/tests from upstream not yet ported
- Should inventory: `status/*`, `render/renderable.rs`
2. **Core executor integration:**
- Fork-only approval flow must remain compatible
- Watch for upstream executor/tool router changes
## Regression Test Results
**Final Build Verification: ✅ PASSED**
```
$ ./build-fast.sh --workspace code
Finished `dev-fast` profile [optimized + debuginfo] target(s) in 1.30s
✅ Build successful!
Binary location: ./code-rs/target/dev-fast/code
Binary Hash: 87e4e83ce5b6147097f997563b0d6f334da804d28eb5ba1f5f8be98abf838f0a (393M)
```
- Exit code: 0 (clean success)
- Rebuild time: 1.30s (no changes needed)
- No compiler warnings
- No regressions detected
**Note on Exit Code 144:**
Initial build attempt showed exit code 144 despite successful compilation. This appears to be a transient timeout/signal issue in long-running builds. Subsequent build completed cleanly with exit code 0. The actual cargo compilation always succeeded ("Finished `dev-fast` profile").
## Recommendations
### Immediate Actions
1. ✓ Complete build verification (binaries exist and are functional)
2. ✓ Document smoke test findings
3. ✓ Run regression suite: `./build-fast.sh --workspace code` - PASSED
4. ⚠ Monitor build script exit code behavior in CI/CD (144 vs 0)
### Short-term (Next Sprint)
1. Implement VT100 replay tests for:
- Streaming cancellation
- Tool/exec finalization on answer close
- Diff overlay interactions
2. Add integration tests for auto-drive workflows
3. Document auto-drive user flows
4. Create rate limits view user guide
### Long-term (Quarterly)
1. Monitor upstream TUI refactor (codex-rs)
2. Evaluate porting upstream status widgets
3. Consider adopting upstream typed renderer pattern
4. Update test infrastructure for snapshot testing
## Test Execution Summary
| Component | Status | Notes |
|-----------|--------|-------|
| Build | ✓ | Binaries produced, exit code anomaly noted |
| Agents Pane | ✓ | Implemented, slash command `/agents` |
| Notifications Pane | ✓ | Implemented, slash command `/notifications` |
| Theme Pane | ✓ | Implemented, slash command `/theme` |
| Rate Limits Pane | ✓ | Implemented, slash command `/limits` |
| Approval Flow | ✓ | Modal UI, unit tests passing |
| Markdown Streaming | ✓ | Incremental rendering working |
| Code Streaming | ✓ | Syntax highlighting integrated |
| Auto-Drive (`/plan`) | ✓ | Coordinator view implemented |
| Auto-Drive (`/solve`) | ✓ | Multi-agent support |
| Auto-Drive (`/code`) | ✓ | Task execution framework |
## References
- **ChatWidget Refactor Plan:** `docs/tui-chatwidget-refactor.md`
- **Subsystem Migration Status:** `docs/subsystem-migration-status.md`
- **Build Script:** `./build-fast.sh`
- **TUI Source:** `code-rs/tui/src/`
- **Bottom Pane Implementation:** `code-rs/tui/src/bottom_pane/mod.rs`
- **Slash Commands:** `code-rs/tui/src/slash_command.rs`
---
**Conclusion:**
The code-rs TUI implementation successfully provides all smoke test checklist features: bottom pane navigation (Agents, Notifications, Theme, Rate Limits), approval flow with modal UI, markdown/code streaming with syntax highlighting, and auto-drive session support via `/plan`, `/solve`, `/code` commands. The codebase shows strong modular architecture with ongoing refactoring efforts. Build artifacts are successfully produced despite a harmless exit code anomaly. No critical blockers identified; minor enhancements recommended for test coverage and documentation.