mirror of
https://github.com/hanzoai/next.js.git
synced 2026-09-24 17:00:51 +00:00
feat(trace-server): add query CLI and MCP API to turbopack-trace-server (#92030)
### What? Adds a [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) HTTP server and a companion CLI to the Turbopack trace server (`next internal trace`), so AI agents and developers can explore trace data from a build or dev session incrementally through a structured tool API. Two new user-facing features: 1. **`--mcp-port` flag on `next internal trace`** — starts an MCP-over-HTTP endpoint (at `/mcp`) alongside the existing WebSocket trace viewer. 2. **`next internal query-trace`** — a standalone CLI command that queries a running trace server from the terminal, using the same MCP endpoint under the hood. ### Why? Trace files from Turbopack builds can be very large. An agent querying raw trace data would be overwhelmed by tens of thousands of spans. The MCP API exposes the data incrementally: - **Pagination** (20 spans per page) prevents context overflow. - **Aggregation** groups repeated spans by name (same logic as the WebSocket viewer's graph), so agents see one entry per span type with count/total/avg statistics rather than thousands of identical entries. - **Drill-down** lets agents pick a span ID from one response and fetch its children in the next call. - **Search** filters spans by name/args using `SpanRef::search` (full subtree search index, same as the WebSocket viewer). - **Sort** orders by corrected (wall-clock) duration to surface the slowest spans first. - **Output format** — `outputType: "json"` for machine-readable structured data, `"markdown"` (default) for human-readable rendering. ### How? #### Rust (`turbopack/crates/turbopack-trace-server/src/lib.rs`) - `start_turbopack_trace_server_non_blocking(path, port) -> Arc<StoreContainer>` — starts the reader and WebSocket server on background threads and returns the store handle immediately (previously the function blocked forever). - `QueryOptions` — struct with `parent`, `aggregated`, `sort`, `search`, `page` fields. - `query_spans(store, opts) -> QueryResult` — queries up to 20 spans per page. In aggregated mode it uses the existing `SpanGraphRef` graph logic (the same grouping the WebSocket viewer uses). Waits up to 10s for the store to finish initial data loading on the first call. - Helper functions extracted: `paginate()`, `format_span_name()`, `build_span_id()` to avoid duplication between aggregated and raw code paths. #### Span ID format IDs encode both the span type and the navigation path: | Span kind | Leaf format | Example full path | |---|---|---| | Raw span | `<index>` (decimal) | `a1-a5-20` | | Aggregated span | `a<first-span-index>` | `a1-a5-a34` | Segments are joined by `-` as the caller drills deeper. `resolve_span_by_id` only needs the last segment to look up the underlying store index. #### NAPI bridge (`crates/next-napi-bindings/src/turbo_trace_server.rs`) - `TraceServerHandle` — opaque NAPI class wrapping `Arc<StoreContainer>`. - `startTurbopackTraceServerHandle(path, port)` — calls the non-blocking Rust function. - `queryTraceSpans(handle, options)` — calls `query_spans` and returns a plain JS object. - Doc comments on `SpanInfo` fields clarify that `cpu_duration`/`corrected_duration` hold the first example span's values for aggregated groups (while `total_*` fields hold group totals). #### TypeScript (`packages/next/src/...`) - `generated-native.d.ts` / `types.ts` / `index.ts` — type declarations, native wrappers, and WASM stubs. - `next.ts` — adds `--mcp-port <port>` to `next internal trace` (defaults to `5748`); registers `next internal query-trace` subcommand with `--json` flag. - `turbo-trace-server.ts` — rewrites the CLI handler to: 1. Start the WebSocket trace viewer server non-blocking (was blocking). 2. Start an HTTP server at `/mcp` running MCP `StreamableHTTPServerTransport` (stateless — one transport per request, `sessionIdGenerator: undefined`). 3. `renderSpanMarkdown()` — extracted helper that renders a single span as markdown. 4. Error handling: `server.on('error')` for `EADDRINUSE`, `try/catch` around `loadBindings()` and `startTurbopackTraceServerHandle()`. - `query-trace.ts` — new CLI command that POSTs JSON-RPC to the MCP endpoint and prints the response. On connection failure, shows instructions to start `next internal trace` first. Supports `--json` flag for structured output. #### `next internal query-trace` CLI ``` Usage: next internal query-trace [options] Options: --port <port> MCP port of the running trace server. Defaults to 5748. --parent <parent> Span ID to enumerate children of. Omit for root level. --no-aggregated Disable aggregation of spans by name (aggregated by default). --sort Sort results by corrected duration descending (default: false). --search <search> Substring filter on span name/category. --json Output as JSON instead of markdown. --page <page> Page number (1-based, default 1). -h, --help Displays this message. ``` #### Startup output When `next internal trace` starts with `--mcp-port`: ``` Turbopack trace server started. View trace at https://trace.nextjs.org?port=5747 Query this trace from the command line: next internal query-trace --help Alternatively, connect an MCP client to http://127.0.0.1:5748/mcp ``` #### Markdown output format (per span) ```markdown ### `<name>` (ID: `<id>`) - CPU Duration: … - Corrected Duration: … - Start (relative to parent): … - End (relative to parent): … **Attributes:** - `key`: value ``` For aggregated spans with count > 1, totals and averages are shown first, then one example span's raw data. ### Tests **E2E test (`test/e2e/turbopack-trace-server-query/turbopack-trace-server.test.ts`)** Uses `nextTestSetup` with `env: { NEXT_TURBOPACK_TRACING: '1' }` to produce a real trace file. Spawns `next internal trace <file> --mcp-port <port>`, waits for the MCP server to be ready, then runs both MCP HTTP and CLI queries: - Root listing (markdown format) - Aggregation mode - Pagination - Drill-down by span ID (using JSON output to extract IDs) - Search with a real span name match and a non-matching term - Sort by duration - JSON output format (`outputType: 'json'`) - CLI: `--sort`, `--search`, `--no-aggregated`, `--parent`, `--json` - Error path: connection failure message with instructions Turbopack-only (skips via `if (!isTurbopack) return` at describe level). The error-path test runs regardless of bundler. ### Usage ```bash # Build with tracing NEXT_TURBOPACK_TRACING=1 next build # Start the trace viewer + MCP server next internal trace .next-profiles/trace-turbopack --mcp-port 5748 # Query from CLI next internal query-trace --sort next internal query-trace --parent a1 --sort next internal query-trace --search "turbo_tasks::function" --page 2 next internal query-trace --no-aggregated next internal query-trace --json # Or connect an MCP client to http://127.0.0.1:5748/mcp ``` <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <[email protected]> Co-authored-by: Claude <[email protected]>
This commit is contained in:
Tobias Koppers
co-authored by
Tobias Koppers
Claude
Tobias Koppers
Claude
parent
26b87589ee
commit
5ff5ae217b
@@ -2,6 +2,7 @@
|
||||
dist
|
||||
!.github/actions/*/dist
|
||||
.next
|
||||
.next-profiles
|
||||
target
|
||||
packages/next/wasm/@next
|
||||
tarballs/
|
||||
|
||||
@@ -1,9 +1,128 @@
|
||||
use std::path::PathBuf;
|
||||
use std::{path::PathBuf, sync::Arc};
|
||||
|
||||
use napi_derive::napi;
|
||||
use turbopack_trace_server::{
|
||||
QueryOptions, query_spans, start_turbopack_trace_server, store_container::StoreContainer,
|
||||
};
|
||||
|
||||
/// An opaque handle to a running trace server instance.
|
||||
/// Holds a reference to the shared store so that `query_trace_spans` can
|
||||
/// query it without blocking Node.js with the WebSocket server loop.
|
||||
#[napi]
|
||||
pub fn start_turbopack_trace_server(path: String, port: Option<u16>) {
|
||||
let path_buf = PathBuf::from(path);
|
||||
turbopack_trace_server::start_turbopack_trace_server(path_buf, port);
|
||||
pub struct TraceServerHandle {
|
||||
store: Arc<StoreContainer>,
|
||||
}
|
||||
|
||||
/// Options for `query_trace_spans`.
|
||||
#[napi(object)]
|
||||
pub struct TraceQueryOptions {
|
||||
/// Optional parent span ID (as returned by a previous query).
|
||||
/// Omit or set to `null`/`undefined` for root-level spans.
|
||||
pub parent: Option<String>,
|
||||
/// When `true` (default), aggregate child spans with the same name.
|
||||
pub aggregated: Option<bool>,
|
||||
/// When `true`, sort results by corrected duration descending. Default `false`.
|
||||
pub sort: Option<bool>,
|
||||
/// Optional substring search query applied to span name/category.
|
||||
pub search: Option<String>,
|
||||
/// 1-based page number. Default `1`.
|
||||
pub page: Option<u32>,
|
||||
}
|
||||
|
||||
/// Information about a single span or aggregated span group.
|
||||
#[napi(object)]
|
||||
pub struct TraceSpanInfo {
|
||||
/// Span ID. Pass this as `parent` in a follow-up call to get children.
|
||||
pub id: String,
|
||||
/// Display name of the span.
|
||||
pub name: String,
|
||||
/// Raw CPU total time in internal ticks (100 ticks = 1 µs).
|
||||
pub cpu_duration: i64,
|
||||
/// Concurrency-corrected total time in internal ticks (100 ticks = 1 µs).
|
||||
pub corrected_duration: i64,
|
||||
/// Start time relative to parent start, in internal ticks.
|
||||
pub start_relative_to_parent: i64,
|
||||
/// End time relative to parent start, in internal ticks.
|
||||
pub end_relative_to_parent: i64,
|
||||
/// Key-value attributes attached to the span.
|
||||
pub args: Vec<Vec<String>>,
|
||||
/// True if this entry represents an aggregated group of spans.
|
||||
pub is_aggregated: bool,
|
||||
/// Number of spans in this aggregated group (only set when `is_aggregated`).
|
||||
pub count: Option<i64>,
|
||||
/// Sum of CPU duration across all spans in the group.
|
||||
pub total_cpu_duration: Option<i64>,
|
||||
/// Average CPU duration across spans in the group.
|
||||
pub avg_cpu_duration: Option<i64>,
|
||||
/// Sum of corrected duration across all spans in the group.
|
||||
pub total_corrected_duration: Option<i64>,
|
||||
/// Average corrected duration across spans in the group.
|
||||
pub avg_corrected_duration: Option<i64>,
|
||||
/// Raw span ID for aggregated groups (the index of the first span).
|
||||
pub first_span_id: Option<String>,
|
||||
}
|
||||
|
||||
/// The result of a `query_trace_spans` call.
|
||||
#[napi(object)]
|
||||
pub struct TraceQueryResult {
|
||||
pub spans: Vec<TraceSpanInfo>,
|
||||
/// Current page (1-based).
|
||||
pub page: u32,
|
||||
/// Total number of pages available.
|
||||
pub total_pages: u32,
|
||||
/// Total number of matching spans across all pages.
|
||||
pub total_count: u32,
|
||||
}
|
||||
|
||||
/// Starts the turbopack trace server on a background thread and returns a
|
||||
/// handle immediately (non-blocking). The WebSocket server will be available
|
||||
/// at `ws://127.0.0.1:<port>` (default port 5747).
|
||||
#[napi]
|
||||
pub fn start_turbopack_trace_server_handle(path: String, port: Option<u16>) -> TraceServerHandle {
|
||||
let store = start_turbopack_trace_server(PathBuf::from(path), port);
|
||||
TraceServerHandle { store }
|
||||
}
|
||||
|
||||
/// Query spans from the trace store held by a `TraceServerHandle`.
|
||||
#[napi]
|
||||
pub fn query_trace_spans(
|
||||
handle: &TraceServerHandle,
|
||||
options: TraceQueryOptions,
|
||||
) -> TraceQueryResult {
|
||||
let result = query_spans(
|
||||
&handle.store,
|
||||
QueryOptions {
|
||||
parent: options.parent,
|
||||
aggregated: options.aggregated.unwrap_or(true),
|
||||
sort: options.sort.unwrap_or(false),
|
||||
search: options.search,
|
||||
page: options.page.unwrap_or(1) as usize,
|
||||
},
|
||||
);
|
||||
|
||||
TraceQueryResult {
|
||||
spans: result
|
||||
.spans
|
||||
.into_iter()
|
||||
.map(|s| TraceSpanInfo {
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
cpu_duration: s.cpu_duration as i64,
|
||||
corrected_duration: s.corrected_duration as i64,
|
||||
start_relative_to_parent: s.start_relative_to_parent,
|
||||
end_relative_to_parent: s.end_relative_to_parent,
|
||||
args: s.args.into_iter().map(|(k, v)| vec![k, v]).collect(),
|
||||
is_aggregated: s.is_aggregated,
|
||||
count: s.count.map(|c| c as i64),
|
||||
total_cpu_duration: s.total_cpu_duration.map(|v| v as i64),
|
||||
avg_cpu_duration: s.avg_cpu_duration.map(|v| v as i64),
|
||||
total_corrected_duration: s.total_corrected_duration.map(|v| v as i64),
|
||||
avg_corrected_duration: s.avg_corrected_duration.map(|v| v as i64),
|
||||
first_span_id: s.first_span_id,
|
||||
})
|
||||
.collect(),
|
||||
page: result.page as u32,
|
||||
total_pages: result.total_pages as u32,
|
||||
total_count: result.total_count as u32,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -601,11 +601,66 @@ internal
|
||||
parseValidPositiveInteger
|
||||
)
|
||||
)
|
||||
.action((file: string, options: { port: number | undefined }) => {
|
||||
return import('../cli/internal/turbo-trace-server.js').then((mod) =>
|
||||
mod.startTurboTraceServerCli(file, options.port)
|
||||
.addOption(
|
||||
new Option(
|
||||
'--mcp-port <mcpPort>',
|
||||
'Port for the MCP (Model Context Protocol) server. Defaults to 5748.'
|
||||
).argParser(parseValidPositiveInteger)
|
||||
)
|
||||
.action(
|
||||
(
|
||||
file: string,
|
||||
options: { port: number | undefined; mcpPort: number | undefined }
|
||||
) => {
|
||||
return import('../cli/internal/turbo-trace-server.js').then((mod) =>
|
||||
mod.startTurboTraceServerCli(file, options.port, options.mcpPort)
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
internal
|
||||
.command('query-trace')
|
||||
.description(
|
||||
'Query a running turbopack trace server (started with `next internal trace --mcp-port <port>`).'
|
||||
)
|
||||
.addOption(
|
||||
new Option(
|
||||
'--port <port>',
|
||||
'MCP port of the running trace server. Defaults to 5748.'
|
||||
).argParser(parseValidPositiveInteger)
|
||||
)
|
||||
.addOption(
|
||||
new Option(
|
||||
'--parent <parent>',
|
||||
'Span ID to enumerate children of. Omit for root level.'
|
||||
)
|
||||
})
|
||||
)
|
||||
.addOption(
|
||||
new Option(
|
||||
'--no-aggregated',
|
||||
'Disable aggregation of spans by name (aggregated by default).'
|
||||
)
|
||||
)
|
||||
.addOption(
|
||||
new Option(
|
||||
'--sort',
|
||||
'Sort results by corrected duration descending (default: false).'
|
||||
)
|
||||
)
|
||||
.addOption(
|
||||
new Option('--search <search>', 'Substring filter on span name/category.')
|
||||
)
|
||||
.addOption(new Option('--json', 'Output as JSON instead of markdown.'))
|
||||
.addOption(
|
||||
new Option('--page <page>', 'Page number (1-based, default 1).').argParser(
|
||||
parseValidPositiveInteger
|
||||
)
|
||||
)
|
||||
.action((options) =>
|
||||
import('../cli/internal/query-trace.js').then((mod) =>
|
||||
mod.queryTraceCli(options)
|
||||
)
|
||||
)
|
||||
|
||||
internal
|
||||
.command('post-build')
|
||||
|
||||
+75
-2
@@ -615,10 +615,77 @@ export declare function transformSync(
|
||||
isModule: boolean,
|
||||
options: Buffer
|
||||
): object
|
||||
export declare function startTurbopackTraceServer(
|
||||
/** Options for `query_trace_spans`. */
|
||||
export interface TraceQueryOptions {
|
||||
/**
|
||||
* Optional parent span ID (as returned by a previous query).
|
||||
* Omit or set to `null`/`undefined` for root-level spans.
|
||||
*/
|
||||
parent?: string
|
||||
/** When `true` (default), aggregate child spans with the same name. */
|
||||
aggregated?: boolean
|
||||
/** When `true`, sort results by corrected duration descending. Default `false`. */
|
||||
sort?: boolean
|
||||
/** Optional substring search query applied to span name/category. */
|
||||
search?: string
|
||||
/** 1-based page number. Default `1`. */
|
||||
page?: number
|
||||
}
|
||||
/** Information about a single span or aggregated span group. */
|
||||
export interface TraceSpanInfo {
|
||||
/** Span ID. Pass this as `parent` in a follow-up call to get children. */
|
||||
id: string
|
||||
/** Display name of the span. */
|
||||
name: string
|
||||
/** Raw CPU total time in internal ticks (100 ticks = 1 µs). */
|
||||
cpuDuration: number
|
||||
/** Concurrency-corrected total time in internal ticks (100 ticks = 1 µs). */
|
||||
correctedDuration: number
|
||||
/** Start time relative to parent start, in internal ticks. */
|
||||
startRelativeToParent: number
|
||||
/** End time relative to parent start, in internal ticks. */
|
||||
endRelativeToParent: number
|
||||
/** Key-value attributes attached to the span. */
|
||||
args: Array<Array<string>>
|
||||
/** True if this entry represents an aggregated group of spans. */
|
||||
isAggregated: boolean
|
||||
/** Number of spans in this aggregated group (only set when `is_aggregated`). */
|
||||
count?: number
|
||||
/** Sum of CPU duration across all spans in the group. */
|
||||
totalCpuDuration?: number
|
||||
/** Average CPU duration across spans in the group. */
|
||||
avgCpuDuration?: number
|
||||
/** Sum of corrected duration across all spans in the group. */
|
||||
totalCorrectedDuration?: number
|
||||
/** Average corrected duration across spans in the group. */
|
||||
avgCorrectedDuration?: number
|
||||
/** Raw span ID for aggregated groups (the index of the first span). */
|
||||
firstSpanId?: string
|
||||
}
|
||||
/** The result of a `query_trace_spans` call. */
|
||||
export interface TraceQueryResult {
|
||||
spans: Array<TraceSpanInfo>
|
||||
/** Current page (1-based). */
|
||||
page: number
|
||||
/** Total number of pages available. */
|
||||
totalPages: number
|
||||
/** Total number of matching spans across all pages. */
|
||||
totalCount: number
|
||||
}
|
||||
/**
|
||||
* Starts the turbopack trace server on a background thread and returns a
|
||||
* handle immediately (non-blocking). The WebSocket server will be available
|
||||
* at `ws://127.0.0.1:<port>` (default port 5747).
|
||||
*/
|
||||
export declare function startTurbopackTraceServerHandle(
|
||||
path: string,
|
||||
port?: number | undefined | null
|
||||
): void
|
||||
): TraceServerHandle
|
||||
/** Query spans from the trace store held by a `TraceServerHandle`. */
|
||||
export declare function queryTraceSpans(
|
||||
handle: TraceServerHandle,
|
||||
options: TraceQueryOptions
|
||||
): TraceQueryResult
|
||||
export interface NextBuildContext {
|
||||
/** The root directory of the workspace. */
|
||||
root?: string
|
||||
@@ -667,3 +734,9 @@ export declare function initCustomTraceSubscriber(
|
||||
export declare function teardownTraceSubscriber(
|
||||
guardExternal: ExternalObject<RefCell>
|
||||
): void
|
||||
/**
|
||||
* An opaque handle to a running trace server instance.
|
||||
* Holds a reference to the shared store so that `query_trace_spans` can
|
||||
* query it without blocking Node.js with the WebSocket server loop.
|
||||
*/
|
||||
export declare class TraceServerHandle {}
|
||||
|
||||
@@ -1395,10 +1395,16 @@ async function loadWasm(importPath = '') {
|
||||
`Use the --webpack flag instead.`
|
||||
)
|
||||
},
|
||||
startTurbopackTraceServer(
|
||||
startTurbopackTraceServerHandle(
|
||||
_traceFilePath: string,
|
||||
_port: number | undefined
|
||||
): void {
|
||||
) {
|
||||
throw new Error(
|
||||
`Turbopack trace server is not supported on this platform (${PlatformName}/${ArchName}) because native bindings are not available. ` +
|
||||
`Only WebAssembly (WASM) bindings were loaded, and Turbopack requires native bindings.`
|
||||
)
|
||||
},
|
||||
queryTraceSpans(_handle: any, _options: any) {
|
||||
throw new Error(
|
||||
`Turbopack trace server is not supported on this platform (${PlatformName}/${ArchName}) because native bindings are not available. ` +
|
||||
`Only WebAssembly (WASM) bindings were loaded, and Turbopack requires native bindings.`
|
||||
@@ -1644,15 +1650,15 @@ function loadNative(importPath?: string): Binding {
|
||||
customBindingsPath ?? bindingsPath!,
|
||||
false
|
||||
),
|
||||
startTurbopackTraceServer(traceFilePath, port) {
|
||||
Log.warn(
|
||||
`Turbopack trace server started. View trace at https://trace.nextjs.org${port != null ? `?port=${port}` : ''}`
|
||||
)
|
||||
;(customBindings ?? bindings).startTurbopackTraceServer(
|
||||
startTurbopackTraceServerHandle(traceFilePath, port) {
|
||||
return (customBindings ?? bindings).startTurbopackTraceServerHandle(
|
||||
traceFilePath,
|
||||
port
|
||||
)
|
||||
},
|
||||
queryTraceSpans(handle, options) {
|
||||
return (customBindings ?? bindings).queryTraceSpans(handle, options)
|
||||
},
|
||||
databaseCompact(dbPath: string) {
|
||||
return (customBindings ?? bindings).turbopackDatabaseCompact(dbPath)
|
||||
},
|
||||
|
||||
@@ -9,8 +9,13 @@ import type {
|
||||
NapiPartialProjectOptions,
|
||||
NapiCodeFrameOptions,
|
||||
NapiCodeFrameLocation,
|
||||
TraceServerHandle,
|
||||
TraceQueryOptions,
|
||||
TraceQueryResult,
|
||||
} from './generated-native'
|
||||
|
||||
export type { TraceServerHandle, TraceQueryOptions, TraceQueryResult }
|
||||
|
||||
export type { NapiTurboEngineOptions as TurboEngineOptions }
|
||||
|
||||
export type Lockfile = { __napiType: 'Lockfile' }
|
||||
@@ -27,10 +32,14 @@ export interface Binding {
|
||||
turboEngineOptions?: NapiTurboEngineOptions,
|
||||
callbacks?: TurbopackProjectCallbacks
|
||||
): Promise<Project>
|
||||
startTurbopackTraceServer(
|
||||
startTurbopackTraceServerHandle(
|
||||
traceFilePath: string,
|
||||
port: number | undefined
|
||||
): void
|
||||
): TraceServerHandle
|
||||
queryTraceSpans(
|
||||
handle: TraceServerHandle,
|
||||
options: TraceQueryOptions
|
||||
): TraceQueryResult
|
||||
databaseCompact(path: string): Promise<void>
|
||||
|
||||
nextBuild?: any
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
/**
|
||||
* CLI client for querying a running turbopack trace server via its MCP endpoint.
|
||||
* Sends a JSON-RPC `tools/call` request and prints the response to stdout.
|
||||
* Use --json for machine-readable JSON output (default: markdown).
|
||||
*
|
||||
* Usage: next internal query-trace [options]
|
||||
*/
|
||||
|
||||
const DEFAULT_MCP_PORT = 5748 // Keep in sync with turbo-trace-server.ts
|
||||
|
||||
interface QueryTraceOptions {
|
||||
port: number | undefined
|
||||
parent: string | undefined
|
||||
aggregated: boolean | undefined
|
||||
sort: boolean | undefined
|
||||
search: string | undefined
|
||||
page: number | undefined
|
||||
json: boolean | undefined
|
||||
}
|
||||
|
||||
export async function queryTraceCli(options: QueryTraceOptions): Promise<void> {
|
||||
const port = options.port ?? DEFAULT_MCP_PORT
|
||||
|
||||
// Build arguments — only include values explicitly set by the user.
|
||||
const args: Record<string, unknown> = {}
|
||||
if (options.parent !== undefined) args.parent = options.parent
|
||||
if (options.aggregated !== undefined) args.aggregated = options.aggregated
|
||||
if (options.sort !== undefined) args.sort = options.sort
|
||||
if (options.search !== undefined) args.search = options.search
|
||||
if (options.page !== undefined) args.page = options.page
|
||||
if (options.json) args.outputType = 'json'
|
||||
|
||||
const requestBody = JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
method: 'tools/call',
|
||||
params: {
|
||||
name: 'query_spans',
|
||||
arguments: args,
|
||||
},
|
||||
id: 1,
|
||||
})
|
||||
|
||||
let res: Response
|
||||
try {
|
||||
res = await fetch(`http://127.0.0.1:${port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json, text/event-stream',
|
||||
},
|
||||
body: requestBody,
|
||||
})
|
||||
} catch {
|
||||
console.error(
|
||||
[
|
||||
`Error: Could not connect to trace server on port ${port}.`,
|
||||
'',
|
||||
'Make sure the trace server is running in the background:',
|
||||
'',
|
||||
` next internal trace <file> --mcp-port ${port}`,
|
||||
'',
|
||||
'Then run your query in another terminal. For all available options, see:',
|
||||
'',
|
||||
' next internal query-trace --help',
|
||||
].join('\n')
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
console.error(`Error: MCP server responded with HTTP ${res.status}`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const body = await res.text()
|
||||
|
||||
// Parse SSE stream: each event looks like:
|
||||
// event: message
|
||||
// data: <JSON-RPC response JSON>
|
||||
//
|
||||
// We scan for "data: " lines, parse the JSON, and extract the text content.
|
||||
for (const line of body.split('\n')) {
|
||||
if (!line.startsWith('data: ')) continue
|
||||
let msg: {
|
||||
result?: { content?: Array<{ type: string; text?: string }> }
|
||||
error?: unknown
|
||||
}
|
||||
try {
|
||||
msg = JSON.parse(line.slice('data: '.length))
|
||||
} catch {
|
||||
continue
|
||||
}
|
||||
if (msg.error) {
|
||||
console.error(`Error: MCP tool error: ${JSON.stringify(msg.error)}`)
|
||||
process.exit(1)
|
||||
}
|
||||
const text = msg.result?.content?.find((c) => c.type === 'text')?.text
|
||||
if (text !== undefined) {
|
||||
process.stdout.write(text)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
console.error(`Error: No text content in MCP response:\n${body}`)
|
||||
process.exit(1)
|
||||
}
|
||||
@@ -1,9 +1,265 @@
|
||||
import http from 'node:http'
|
||||
import z from 'next/dist/compiled/zod'
|
||||
import { loadBindings } from '../../build/swc'
|
||||
import type { TraceSpanInfo } from '../../build/swc/generated-native'
|
||||
|
||||
const { McpServer } =
|
||||
require('next/dist/compiled/@modelcontextprotocol/sdk/server/mcp') as typeof import('next/dist/compiled/@modelcontextprotocol/sdk/server/mcp')
|
||||
const { StreamableHTTPServerTransport } =
|
||||
require('next/dist/compiled/@modelcontextprotocol/sdk/server/streamableHttp') as typeof import('next/dist/compiled/@modelcontextprotocol/sdk/server/streamableHttp')
|
||||
|
||||
const DEFAULT_WS_PORT = 5747
|
||||
const DEFAULT_MCP_PORT = 5748 // Keep in sync with query-trace.ts
|
||||
|
||||
/** 100 internal ticks = 1 µs */
|
||||
const TICKS_PER_US = 100
|
||||
const TICKS_PER_MS = TICKS_PER_US * 1000
|
||||
const TICKS_PER_S = TICKS_PER_MS * 1000
|
||||
|
||||
function formatDuration(ticks: number): string {
|
||||
if (ticks < TICKS_PER_MS) {
|
||||
const us = ticks / TICKS_PER_US
|
||||
return `${us.toFixed(0)}µs`
|
||||
}
|
||||
if (ticks < TICKS_PER_S) {
|
||||
const ms = ticks / TICKS_PER_MS
|
||||
return `${ms.toFixed(2)}ms`
|
||||
}
|
||||
const s = ticks / TICKS_PER_S
|
||||
return `${s.toFixed(3)}s`
|
||||
}
|
||||
|
||||
function formatRelative(ticks: number): string {
|
||||
// ticks may be negative if the child starts before the parent reference point
|
||||
const prefix = ticks < 0 ? '-' : ''
|
||||
return prefix + formatDuration(Math.abs(ticks))
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a single span (or aggregated span group) as a markdown section.
|
||||
*/
|
||||
function renderSpanMarkdown(span: TraceSpanInfo): string {
|
||||
let md = `### \`${span.name}\` (ID: \`${span.id}\`)\n`
|
||||
|
||||
if (span.isAggregated && span.count !== undefined && span.count > 1) {
|
||||
md += `- **Count:** ${span.count} spans\n`
|
||||
if (span.totalCpuDuration !== undefined) {
|
||||
md += `- **Total CPU Duration:** ${formatDuration(span.totalCpuDuration)}\n`
|
||||
}
|
||||
if (span.avgCpuDuration !== undefined) {
|
||||
md += `- **Avg CPU Duration:** ${formatDuration(span.avgCpuDuration)}\n`
|
||||
}
|
||||
if (span.totalCorrectedDuration !== undefined) {
|
||||
md += `- **Total Corrected Duration:** ${formatDuration(span.totalCorrectedDuration)}\n`
|
||||
}
|
||||
if (span.avgCorrectedDuration !== undefined) {
|
||||
md += `- **Avg Corrected Duration:** ${formatDuration(span.avgCorrectedDuration)}\n`
|
||||
}
|
||||
md += `- **Start (relative to parent):** ${formatRelative(span.startRelativeToParent)}\n`
|
||||
md += `- **End (relative to parent):** ${formatRelative(span.endRelativeToParent)}\n`
|
||||
const exampleId = span.firstSpanId ?? span.id
|
||||
md += `\n#### First span as example (ID: \`${exampleId}\`)\n`
|
||||
md += `- **CPU Duration:** ${formatDuration(span.cpuDuration)}\n`
|
||||
md += `- **Corrected Duration:** ${formatDuration(span.correctedDuration)}\n`
|
||||
} else {
|
||||
md += `- **CPU Duration:** ${formatDuration(span.cpuDuration)}\n`
|
||||
md += `- **Corrected Duration:** ${formatDuration(span.correctedDuration)}\n`
|
||||
md += `- **Start (relative to parent):** ${formatRelative(span.startRelativeToParent)}\n`
|
||||
md += `- **End (relative to parent):** ${formatRelative(span.endRelativeToParent)}\n`
|
||||
}
|
||||
|
||||
if (span.args && span.args.length > 0) {
|
||||
md += `\n**Attributes:**\n`
|
||||
for (const [k, v] of span.args) {
|
||||
md += `- \`${k}\`: ${v}\n`
|
||||
}
|
||||
}
|
||||
|
||||
md += '\n---\n\n'
|
||||
return md
|
||||
}
|
||||
|
||||
export async function startTurboTraceServerCli(
|
||||
file: string,
|
||||
port: number | undefined
|
||||
port: number | undefined,
|
||||
mcpPort: number | undefined
|
||||
) {
|
||||
let bindings = await loadBindings()
|
||||
bindings.turbo.startTurbopackTraceServer(file, port)
|
||||
const wsPort = port ?? DEFAULT_WS_PORT
|
||||
const httpPort = mcpPort ?? DEFAULT_MCP_PORT
|
||||
|
||||
let bindings
|
||||
try {
|
||||
bindings = await loadBindings()
|
||||
} catch {
|
||||
console.error(
|
||||
'Error: Could not load native bindings. The trace server requires native (non-WASM) bindings.'
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
let handle
|
||||
try {
|
||||
// Start the WebSocket trace server on a background thread (non-blocking).
|
||||
handle = bindings.turbo.startTurbopackTraceServerHandle(file, wsPort)
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`Error: Could not start trace server for "${file}": ${err instanceof Error ? err.message : err}`
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Turbopack trace server started. View trace at https://trace.nextjs.org?port=${wsPort}`
|
||||
)
|
||||
|
||||
// Create the MCP server.
|
||||
const mcpServer = new McpServer({
|
||||
name: 'Next.js Trace Server MCP',
|
||||
version: '0.1.0',
|
||||
})
|
||||
|
||||
mcpServer.registerTool(
|
||||
'query_spans',
|
||||
{
|
||||
description:
|
||||
'Query spans from a turbopack trace file. Returns spans with timing, CPU usage, and attribute details. Set `outputType` to "json" for machine-readable output or "markdown" (default) for human-readable output. Use the `parent` parameter (with an ID from a previous result) to drill into children. Results are paginated to 20 spans per page.',
|
||||
inputSchema: {
|
||||
parent: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Span ID to enumerate children of. Omit for root-level spans. Use the `id` field from a previous result.'
|
||||
),
|
||||
aggregated: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
'When true (default), aggregate spans with the same name into a single entry. Set to false to see individual raw spans.'
|
||||
),
|
||||
sort: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
'When true, sort results by corrected duration descending. Default false.'
|
||||
),
|
||||
search: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Substring search query applied to span name and category.'
|
||||
),
|
||||
page: z.number().optional().describe('1-based page number. Default 1.'),
|
||||
outputType: z
|
||||
.enum(['markdown', 'json'])
|
||||
.optional()
|
||||
.describe(
|
||||
'Output format. "markdown" (default) returns human-readable markdown. "json" returns structured JSON with all span fields.'
|
||||
),
|
||||
},
|
||||
},
|
||||
(args) => {
|
||||
const result = bindings.turbo.queryTraceSpans(handle, {
|
||||
parent: args.parent,
|
||||
aggregated: args.aggregated ?? true,
|
||||
sort: args.sort ?? false,
|
||||
search: args.search,
|
||||
page: args.page ?? 1,
|
||||
})
|
||||
|
||||
const { spans, page, totalPages, totalCount } = result
|
||||
|
||||
if (args.outputType === 'json') {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({ spans, page, totalPages, totalCount }),
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
const parentLabel = args.parent
|
||||
? `children of ID \`${args.parent}\``
|
||||
: 'root level'
|
||||
let md = `## Spans at ${parentLabel} — Page ${page} of ${totalPages} (${totalCount} total)\n\n`
|
||||
|
||||
if (spans.length === 0) {
|
||||
md += '_No spans found._\n'
|
||||
}
|
||||
|
||||
for (const span of spans) {
|
||||
md += renderSpanMarkdown(span)
|
||||
}
|
||||
|
||||
if (page < totalPages) {
|
||||
md += `Use \`page=${page + 1}\` to see more results.\n`
|
||||
}
|
||||
|
||||
return { content: [{ type: 'text', text: md }] }
|
||||
}
|
||||
)
|
||||
|
||||
// Start the HTTP server for MCP (served at /mcp).
|
||||
const server = http.createServer(async (req, res) => {
|
||||
if (req.url !== '/mcp') {
|
||||
res.writeHead(404, { 'Content-Type': 'text/plain' })
|
||||
res.end('Not found. MCP endpoint is at /mcp\n')
|
||||
return
|
||||
}
|
||||
const transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: undefined,
|
||||
})
|
||||
try {
|
||||
res.on('close', () => transport.close())
|
||||
await mcpServer.connect(transport)
|
||||
let body = ''
|
||||
req.setEncoding('utf8')
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
req.on('data', (chunk: string) => {
|
||||
body += chunk
|
||||
})
|
||||
req.on('end', resolve)
|
||||
req.on('error', reject)
|
||||
})
|
||||
await transport.handleRequest(
|
||||
req,
|
||||
res,
|
||||
body ? JSON.parse(body) : undefined
|
||||
)
|
||||
} catch (err) {
|
||||
if (!res.headersSent) {
|
||||
res.writeHead(500, {
|
||||
'Content-Type': 'application/json; charset=utf-8',
|
||||
})
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
error: { code: -32000, message: 'Internal server error' },
|
||||
id: null,
|
||||
})
|
||||
)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
server.on('error', (err: NodeJS.ErrnoException) => {
|
||||
if (err.code === 'EADDRINUSE') {
|
||||
console.error(
|
||||
`Error: MCP port ${httpPort} is already in use. Use --mcp-port to specify a different port.`
|
||||
)
|
||||
} else {
|
||||
console.error(`Error starting MCP server: ${err.message}`)
|
||||
}
|
||||
process.exit(1)
|
||||
})
|
||||
|
||||
server.listen(httpPort, '127.0.0.1', () => {
|
||||
console.log(
|
||||
`Query this trace from the command line: next internal query-trace --help`
|
||||
)
|
||||
console.log(
|
||||
`Alternatively, connect an MCP client to http://127.0.0.1:${httpPort}/mcp`
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
export default function RootLayout({
|
||||
children,
|
||||
}: {
|
||||
children: React.ReactNode
|
||||
}) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<body>{children}</body>
|
||||
</html>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
export default function Page() {
|
||||
return <p>Hello from turbopack-trace-server-query test</p>
|
||||
}
|
||||
@@ -0,0 +1,404 @@
|
||||
/**
|
||||
* E2E test for the turbopack trace server MCP API and CLI.
|
||||
*
|
||||
* Flow:
|
||||
* 1. Start the Next.js app with `NEXT_TURBOPACK_TRACING=1`.
|
||||
* - In dev mode: fetches `/` to trigger compilation so trace data is written.
|
||||
* - In start mode: the build already produced the trace file.
|
||||
* 2. Wait for the trace file to appear.
|
||||
* 3. Spawn `next internal trace <file> --mcp-port <port>` in the background.
|
||||
* 4. Wait for the MCP HTTP server to be ready.
|
||||
* 5. Run MCP HTTP and CLI queries and verify the response structure.
|
||||
*/
|
||||
import { nextTestSetup, isNextDeploy } from 'e2e-utils'
|
||||
import { existsSync } from 'fs'
|
||||
import path from 'path'
|
||||
import spawn from 'cross-spawn'
|
||||
import type { ChildProcess } from 'child_process'
|
||||
import treeKill from 'tree-kill'
|
||||
import { findPort, retry } from 'next-test-utils'
|
||||
|
||||
// ─── helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
const nextBin = path.join(
|
||||
path.dirname(require.resolve('next/package')),
|
||||
'dist/bin/next'
|
||||
)
|
||||
|
||||
/**
|
||||
* POST a JSON-RPC `tools/call` to the MCP server and return the text content.
|
||||
* The Streamable-HTTP transport responds with Server-Sent Events; we find the
|
||||
* first `data:` line that contains a `result.content[].text` field.
|
||||
*/
|
||||
async function callMcpTool(
|
||||
port: number,
|
||||
toolName: string,
|
||||
args: Record<string, unknown>
|
||||
): Promise<string> {
|
||||
const res = await fetch(`http://127.0.0.1:${port}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json, text/event-stream',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
method: 'tools/call',
|
||||
params: { name: toolName, arguments: args },
|
||||
id: 1,
|
||||
}),
|
||||
})
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`)
|
||||
const body = await res.text()
|
||||
for (const line of body.split('\n')) {
|
||||
if (!line.startsWith('data: ')) continue
|
||||
const msg = JSON.parse(line.slice('data: '.length)) as {
|
||||
result?: { content?: Array<{ type: string; text?: string }> }
|
||||
error?: unknown
|
||||
}
|
||||
if (msg.error) throw new Error(`MCP error: ${JSON.stringify(msg.error)}`)
|
||||
const text = msg.result?.content?.find((c) => c.type === 'text')?.text
|
||||
if (text !== undefined) return text
|
||||
}
|
||||
throw new Error(`No text content in MCP response:\n${body}`)
|
||||
}
|
||||
|
||||
type SpanData = {
|
||||
id: string
|
||||
name: string
|
||||
[key: string]: unknown
|
||||
}
|
||||
|
||||
type QuerySpansResult = {
|
||||
spans: SpanData[]
|
||||
page: number
|
||||
totalPages: number
|
||||
totalCount: number
|
||||
}
|
||||
|
||||
/** Call query_spans with outputType:'json' and return the parsed result. */
|
||||
async function querySpansJson(
|
||||
port: number,
|
||||
args: Record<string, unknown> = {}
|
||||
): Promise<QuerySpansResult> {
|
||||
const text = await callMcpTool(port, 'query_spans', {
|
||||
...args,
|
||||
outputType: 'json',
|
||||
})
|
||||
return JSON.parse(text)
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `next internal query-trace` with the given extra arguments.
|
||||
* Returns captured stdout, stderr, and exit code.
|
||||
*/
|
||||
function runQueryTraceCli(
|
||||
extraArgs: string[]
|
||||
): Promise<{ stdout: string; stderr: string; exitCode: number }> {
|
||||
return new Promise((resolve) => {
|
||||
const proc = spawn(
|
||||
'node',
|
||||
['--no-deprecation', nextBin, 'internal', 'query-trace', ...extraArgs],
|
||||
{ stdio: 'pipe' }
|
||||
)
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
proc.stdout?.on('data', (chunk: Buffer) => (stdout += chunk.toString()))
|
||||
proc.stderr?.on('data', (chunk: Buffer) => (stderr += chunk.toString()))
|
||||
proc.on('close', (code) => resolve({ stdout, stderr, exitCode: code ?? 1 }))
|
||||
})
|
||||
}
|
||||
|
||||
// ─── test suite ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('turbopack-trace-server', () => {
|
||||
if (isNextDeploy) {
|
||||
it('skipped for deploy mode', () => {})
|
||||
return
|
||||
}
|
||||
|
||||
const { next, isTurbopack, isNextDev, skipped } = nextTestSetup({
|
||||
files: __dirname,
|
||||
env: { NEXT_TURBOPACK_TRACING: '1' },
|
||||
skipDeployment: true,
|
||||
})
|
||||
|
||||
if (skipped) return
|
||||
|
||||
// Error-path test: does not need turbopack or a running trace server.
|
||||
it('CLI: should show an error when the trace server is not running', async () => {
|
||||
const unusedPort = await findPort()
|
||||
const { stderr, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(unusedPort),
|
||||
])
|
||||
expect(exitCode).toBe(1)
|
||||
expect(stderr).toContain(
|
||||
`Could not connect to trace server on port ${unusedPort}`
|
||||
)
|
||||
expect(stderr).toContain('next internal trace <file>')
|
||||
expect(stderr).toContain('next internal query-trace --help')
|
||||
})
|
||||
|
||||
// Skip turbopack-specific tests for non-turbopack builds.
|
||||
if (!isTurbopack) return
|
||||
|
||||
let traceServerProcess: ChildProcess | undefined
|
||||
let mcpPort: number
|
||||
|
||||
beforeAll(async () => {
|
||||
// In dev mode, trigger compilation so the trace file has real span data.
|
||||
if (isNextDev) {
|
||||
const res = await next.fetch('/')
|
||||
if (res.status !== 200) {
|
||||
throw new Error(`Dev server returned ${res.status} for /`)
|
||||
}
|
||||
}
|
||||
|
||||
// Wait for the trace file to appear.
|
||||
// Since Next.js 16.2.1-canary.25, trace is written to .next-profiles/ instead of distDir.
|
||||
// Fall back to distDir for compatibility with older locally-installed binaries.
|
||||
const traceFileNewPath = path.join(
|
||||
next.testDir,
|
||||
'.next-profiles',
|
||||
'trace-turbopack'
|
||||
)
|
||||
const traceFileOldPath = path.join(
|
||||
next.testDir,
|
||||
next.distDir,
|
||||
'trace-turbopack'
|
||||
)
|
||||
let traceFile: string = traceFileNewPath
|
||||
await retry(
|
||||
async () => {
|
||||
if (existsSync(traceFileNewPath)) {
|
||||
traceFile = traceFileNewPath
|
||||
} else if (existsSync(traceFileOldPath)) {
|
||||
traceFile = traceFileOldPath
|
||||
} else {
|
||||
throw new Error(
|
||||
`Trace file not found yet: tried ${traceFileNewPath} and ${traceFileOldPath}`
|
||||
)
|
||||
}
|
||||
},
|
||||
15_000,
|
||||
500
|
||||
)
|
||||
|
||||
// Allocate a port and start the trace server with an MCP endpoint.
|
||||
mcpPort = await findPort()
|
||||
traceServerProcess = spawn(
|
||||
'node',
|
||||
[
|
||||
'--no-deprecation',
|
||||
nextBin,
|
||||
'internal',
|
||||
'trace',
|
||||
traceFile,
|
||||
'--mcp-port',
|
||||
String(mcpPort),
|
||||
],
|
||||
{ stdio: 'inherit' }
|
||||
)
|
||||
|
||||
// Wait for the MCP HTTP server to be ready.
|
||||
await retry(
|
||||
async () => {
|
||||
const res = await fetch(`http://127.0.0.1:${mcpPort}/mcp`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json, text/event-stream',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
method: 'tools/list',
|
||||
id: 0,
|
||||
}),
|
||||
})
|
||||
if (res.status >= 500) {
|
||||
throw new Error(`MCP server not ready (HTTP ${res.status})`)
|
||||
}
|
||||
},
|
||||
30_000,
|
||||
500
|
||||
)
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
if (traceServerProcess?.pid) {
|
||||
await new Promise<void>((resolve) => {
|
||||
treeKill(traceServerProcess!.pid!, 'SIGKILL', () => resolve())
|
||||
})
|
||||
traceServerProcess = undefined
|
||||
}
|
||||
})
|
||||
|
||||
// ─── MCP HTTP API tests ──────────────────────────────────────────────────
|
||||
|
||||
it('should return root-level spans in markdown format', async () => {
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', {})
|
||||
expect(md).toContain('## Spans at root level')
|
||||
expect(md).toMatch(/ID: `[a-z0-9-]+`/)
|
||||
expect(md).toMatch(/CPU Duration|Corrected Duration/)
|
||||
})
|
||||
|
||||
it('should support aggregated mode (default) grouping spans by name', async () => {
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', { aggregated: true })
|
||||
expect(md).toContain('## Spans at root level')
|
||||
expect(md).toMatch(/###/)
|
||||
})
|
||||
|
||||
it('should support pagination', async () => {
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', { page: 1 })
|
||||
expect(md).toMatch(/Page \d+ of \d+/)
|
||||
expect(md).toMatch(/\d+ total/)
|
||||
})
|
||||
|
||||
it('should drill into children of a span using its ID', async () => {
|
||||
const { spans } = await querySpansJson(mcpPort, { sort: true })
|
||||
const spanId = spans[0].id
|
||||
|
||||
const childMd = await callMcpTool(mcpPort, 'query_spans', {
|
||||
parent: spanId,
|
||||
})
|
||||
expect(childMd).toContain(`children of ID \`${spanId}\``)
|
||||
expect(childMd).toMatch(/Page \d+ of \d+/)
|
||||
})
|
||||
|
||||
it('should return no results for an impossible search term', async () => {
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', {
|
||||
search: 'zzz_unlikely_span_name_zzz',
|
||||
})
|
||||
expect(md).toMatch(/\b0 total/)
|
||||
})
|
||||
|
||||
it('should return results when searching for a real span name', async () => {
|
||||
// Get a real span name from the root level via JSON output.
|
||||
const { spans } = await querySpansJson(mcpPort, { sort: true })
|
||||
const searchTerm = spans[0].name.slice(0, 20)
|
||||
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', {
|
||||
search: searchTerm,
|
||||
})
|
||||
// Should find at least the span we took the name from.
|
||||
expect(md).not.toMatch(/\b0 total/)
|
||||
expect(md).toMatch(/###/)
|
||||
})
|
||||
|
||||
it('should support sort by duration', async () => {
|
||||
const md = await callMcpTool(mcpPort, 'query_spans', { sort: true })
|
||||
expect(md).toContain('## Spans at root level')
|
||||
expect(md).toMatch(/###/)
|
||||
})
|
||||
|
||||
it('should return JSON when outputType is json', async () => {
|
||||
const data = await querySpansJson(mcpPort)
|
||||
expect(Array.isArray(data.spans)).toBe(true)
|
||||
expect(data.spans.length).toBeGreaterThan(0)
|
||||
expect(data).toHaveProperty('page')
|
||||
expect(data).toHaveProperty('totalPages')
|
||||
expect(data).toHaveProperty('totalCount')
|
||||
// Each span should have the expected fields.
|
||||
const span = data.spans[0]
|
||||
expect(span).toHaveProperty('id')
|
||||
expect(span).toHaveProperty('name')
|
||||
expect(span).toHaveProperty('cpuDuration')
|
||||
expect(span).toHaveProperty('correctedDuration')
|
||||
expect(span).toHaveProperty('isAggregated')
|
||||
})
|
||||
|
||||
// ─── CLI tests ───────────────────────────────────────────────────────────
|
||||
|
||||
it('CLI: should return root-level spans', async () => {
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).toContain('## Spans at root level')
|
||||
expect(stdout).toMatch(/ID: `[a-z0-9-]+`/)
|
||||
expect(stdout).toMatch(/CPU Duration|Corrected Duration/)
|
||||
})
|
||||
|
||||
it('CLI: should support --sort flag', async () => {
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--sort',
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).toContain('## Spans at root level')
|
||||
expect(stdout).toMatch(/###/)
|
||||
})
|
||||
|
||||
it('CLI: should support --search flag with a real match', async () => {
|
||||
// Get a real span name to search for via JSON output.
|
||||
const { spans } = await querySpansJson(mcpPort, { sort: true })
|
||||
const searchTerm = spans[0].name.slice(0, 20)
|
||||
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--search',
|
||||
searchTerm,
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).not.toMatch(/\b0 total/)
|
||||
expect(stdout).toMatch(/###/)
|
||||
})
|
||||
|
||||
it('CLI: should support --search flag with no match', async () => {
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--search',
|
||||
'zzz_unlikely_span_name_zzz',
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).toMatch(/\b0 total/)
|
||||
})
|
||||
|
||||
it('CLI: should support --no-aggregated flag', async () => {
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--no-aggregated',
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).toContain('## Spans at root level')
|
||||
// Raw span IDs are plain numbers (no "a" prefix).
|
||||
expect(stdout).toMatch(/ID: `\d+`/)
|
||||
})
|
||||
|
||||
it('CLI: should support --parent to drill into children', async () => {
|
||||
const { spans } = await querySpansJson(mcpPort, { sort: true })
|
||||
const spanId = spans[0].id
|
||||
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--parent',
|
||||
spanId,
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
expect(stdout).toContain(`children of ID \`${spanId}\``)
|
||||
expect(stdout).toMatch(/Page \d+ of \d+/)
|
||||
})
|
||||
|
||||
it('CLI: should support --json flag', async () => {
|
||||
const { stdout, exitCode } = await runQueryTraceCli([
|
||||
'--port',
|
||||
String(mcpPort),
|
||||
'--json',
|
||||
])
|
||||
expect(exitCode).toBe(0)
|
||||
const data = JSON.parse(stdout)
|
||||
expect(data).toHaveProperty('spans')
|
||||
expect(data).toHaveProperty('page')
|
||||
expect(data).toHaveProperty('totalPages')
|
||||
expect(data).toHaveProperty('totalCount')
|
||||
expect(Array.isArray(data.spans)).toBe(true)
|
||||
expect(data.spans.length).toBeGreaterThan(0)
|
||||
})
|
||||
})
|
||||
@@ -1,11 +1,20 @@
|
||||
#![feature(box_patterns)]
|
||||
#![feature(bufreader_peek)]
|
||||
|
||||
use std::{hash::BuildHasherDefault, path::PathBuf, sync::Arc};
|
||||
use std::{
|
||||
hash::BuildHasherDefault,
|
||||
path::PathBuf,
|
||||
sync::Arc,
|
||||
thread,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
use rustc_hash::FxHasher;
|
||||
|
||||
use self::{reader::TraceReader, server::serve, store_container::StoreContainer};
|
||||
use self::{
|
||||
reader::TraceReader, server::serve, span_graph_ref::SpanGraphEventRef, span_ref::SpanRef,
|
||||
store_container::StoreContainer,
|
||||
};
|
||||
|
||||
mod bottom_up;
|
||||
mod reader;
|
||||
@@ -16,7 +25,7 @@ mod span_bottom_up_ref;
|
||||
mod span_graph_ref;
|
||||
mod span_ref;
|
||||
mod store;
|
||||
mod store_container;
|
||||
pub mod store_container;
|
||||
mod string_tuple_ref;
|
||||
mod timestamp;
|
||||
mod u64_empty_string;
|
||||
@@ -29,11 +38,344 @@ mod viewer;
|
||||
)]
|
||||
type FxIndexMap<K, V> = indexmap::IndexMap<K, V, BuildHasherDefault<FxHasher>>;
|
||||
|
||||
pub fn start_turbopack_trace_server(path: PathBuf, port: Option<u16>) {
|
||||
/// Starts the trace server on a background thread and returns the store
|
||||
/// immediately. The WebSocket server runs non-blocking.
|
||||
pub fn start_turbopack_trace_server(path: PathBuf, port: Option<u16>) -> Arc<StoreContainer> {
|
||||
let store = Arc::new(StoreContainer::new());
|
||||
let reader = TraceReader::spawn(store.clone(), path);
|
||||
|
||||
serve(store, port.unwrap_or(5747));
|
||||
let store_for_reader = store.clone();
|
||||
let store_for_server = store.clone();
|
||||
|
||||
reader.join().unwrap();
|
||||
TraceReader::spawn(store_for_reader, path);
|
||||
|
||||
thread::spawn(move || {
|
||||
serve(store_for_server, port.unwrap_or(5747));
|
||||
});
|
||||
|
||||
store
|
||||
}
|
||||
|
||||
const PAGE_SIZE: usize = 20;
|
||||
|
||||
/// Options for querying spans from the trace store.
|
||||
pub struct QueryOptions {
|
||||
/// Optional parent span ID (as produced by `SpanInfo::id`).
|
||||
/// `None` means root level.
|
||||
pub parent: Option<String>,
|
||||
/// When true, aggregate child spans with the same name.
|
||||
pub aggregated: bool,
|
||||
/// When true, sort results by corrected duration descending.
|
||||
pub sort: bool,
|
||||
/// Optional substring search query.
|
||||
pub search: Option<String>,
|
||||
/// 1-based page number.
|
||||
pub page: usize,
|
||||
}
|
||||
|
||||
/// Information about a single span (or aggregated group of spans).
|
||||
pub struct SpanInfo {
|
||||
/// Span ID string.
|
||||
///
|
||||
/// The format encodes both the type and the navigation path:
|
||||
/// - A **raw span** leaf is its decimal index: `"123"`.
|
||||
/// - An **aggregated span** leaf is `"a"` + the first-span index: `"a123"`.
|
||||
/// - When the span is a child of another span, the parent's ID is prepended with a dash
|
||||
/// separator, e.g. `"a5-a34"` or `"1-a5-a34-20"`.
|
||||
///
|
||||
/// Pass the full ID as the `parent` option of the next `query_spans` call
|
||||
/// to enumerate the children of that span.
|
||||
pub id: String,
|
||||
/// Display name: `"category title"` or just `"title"`.
|
||||
pub name: String,
|
||||
/// Raw CPU total time in internal ticks (100 ticks = 1 µs).
|
||||
/// For aggregated spans, this is the **first (example) span's** value, not the group total.
|
||||
/// See `total_cpu_duration` for the group total.
|
||||
pub cpu_duration: u64,
|
||||
/// Concurrency-corrected total time in internal ticks.
|
||||
/// For aggregated spans, this is the **first (example) span's** value, not the group total.
|
||||
/// See `total_corrected_duration` for the group total.
|
||||
pub corrected_duration: u64,
|
||||
/// Start of span relative to parent start, in internal ticks.
|
||||
pub start_relative_to_parent: i64,
|
||||
/// End of span relative to parent start, in internal ticks.
|
||||
pub end_relative_to_parent: i64,
|
||||
/// Key-value attributes from the span.
|
||||
pub args: Vec<(String, String)>,
|
||||
/// True if this entry represents an aggregated group of spans.
|
||||
pub is_aggregated: bool,
|
||||
/// Number of spans in the group (only set for aggregated spans).
|
||||
pub count: Option<u64>,
|
||||
/// Sum of cpu_duration across all spans in the group.
|
||||
pub total_cpu_duration: Option<u64>,
|
||||
/// Average cpu_duration across all spans in the group.
|
||||
pub avg_cpu_duration: Option<u64>,
|
||||
/// Sum of corrected_duration across all spans in the group.
|
||||
pub total_corrected_duration: Option<u64>,
|
||||
/// Average corrected_duration across all spans in the group.
|
||||
pub avg_corrected_duration: Option<u64>,
|
||||
/// Raw span ID for aggregated groups (the index of the first span).
|
||||
pub first_span_id: Option<String>,
|
||||
}
|
||||
|
||||
/// Result of a `query_spans` call.
|
||||
pub struct QueryResult {
|
||||
pub spans: Vec<SpanInfo>,
|
||||
pub page: usize,
|
||||
pub total_pages: usize,
|
||||
pub total_count: usize,
|
||||
}
|
||||
|
||||
/// Paginate a vec of items. Returns `(page_items, clamped_page, total_pages, total_count)`.
|
||||
fn paginate<T>(items: Vec<T>, page: usize) -> (Vec<T>, usize, usize, usize) {
|
||||
let total_count = items.len();
|
||||
let total_pages = total_count.div_ceil(PAGE_SIZE).max(1);
|
||||
let page = page.clamp(1, total_pages);
|
||||
let start = (page - 1) * PAGE_SIZE;
|
||||
let page_items = items.into_iter().skip(start).take(PAGE_SIZE).collect();
|
||||
(page_items, page, total_pages, total_count)
|
||||
}
|
||||
|
||||
fn format_span_name(cat: &str, title: &str) -> String {
|
||||
if cat.is_empty() {
|
||||
title.to_string()
|
||||
} else {
|
||||
format!("{cat} {title}")
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a span ID by appending a leaf segment to the optional parent path.
|
||||
fn build_span_id(parent: Option<&str>, leaf: &str) -> String {
|
||||
match parent {
|
||||
Some(p) => format!("{p}-{leaf}"),
|
||||
None => leaf.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Query spans from the store.
|
||||
///
|
||||
/// Waits up to 10 seconds for at least some data to be loaded before
|
||||
/// returning, so callers don't need to poll separately.
|
||||
pub fn query_spans(store: &Arc<StoreContainer>, options: QueryOptions) -> QueryResult {
|
||||
// Wait briefly for initial data if the store is empty.
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
loop {
|
||||
{
|
||||
let guard = store.read();
|
||||
// root span always exists (index 0); real spans start at index 1
|
||||
if guard.spans.len() > 1 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if Instant::now() >= deadline {
|
||||
break;
|
||||
}
|
||||
thread::sleep(Duration::from_millis(50));
|
||||
}
|
||||
|
||||
let store_guard = store.read();
|
||||
let store_ref = &*store_guard;
|
||||
|
||||
// Resolve the parent span.
|
||||
let parent_span: Option<SpanRef<'_>> = if let Some(ref parent_id) = options.parent {
|
||||
resolve_span_by_id(store_ref, parent_id)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let parent_start = parent_span.as_ref().map(|s| *s.start()).unwrap_or_default();
|
||||
|
||||
if options.aggregated {
|
||||
// Collect aggregated children (SpanGraphRef) from either the resolved
|
||||
// parent span or the root.
|
||||
let graph_children: Vec<_> = if let Some(ref parent) = parent_span {
|
||||
// The parent might be an aggregated node: look up which graph node
|
||||
// the parent ID refers to, then iterate its children's graph events.
|
||||
// For simplicity, resolve via the parent span's graph().
|
||||
parent
|
||||
.graph()
|
||||
.filter_map(|event| match event {
|
||||
SpanGraphEventRef::Child { graph } => Some(graph),
|
||||
SpanGraphEventRef::SelfTime { .. } => None,
|
||||
})
|
||||
.collect()
|
||||
} else {
|
||||
// Root level: use root span's graph.
|
||||
store_ref
|
||||
.root_span()
|
||||
.graph()
|
||||
.filter_map(|event| match event {
|
||||
SpanGraphEventRef::Child { graph } => Some(graph),
|
||||
SpanGraphEventRef::SelfTime { .. } => None,
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
|
||||
// Apply search filter.
|
||||
let mut filtered: Vec<_> = if let Some(ref query) = options.search {
|
||||
graph_children
|
||||
.into_iter()
|
||||
.filter(|g| {
|
||||
let (cat, title) = g.nice_name();
|
||||
cat.contains(query.as_str()) || title.contains(query.as_str())
|
||||
})
|
||||
.collect()
|
||||
} else {
|
||||
graph_children
|
||||
};
|
||||
|
||||
// Sort if requested.
|
||||
if options.sort {
|
||||
filtered.sort_by(|a, b| {
|
||||
b.corrected_total_time()
|
||||
.cmp(&a.corrected_total_time())
|
||||
.then_with(|| b.total_time().cmp(&a.total_time()))
|
||||
});
|
||||
}
|
||||
|
||||
let (page_items, page, total_pages, total_count) = paginate(filtered, options.page);
|
||||
|
||||
let spans = page_items
|
||||
.into_iter()
|
||||
.map(|graph| {
|
||||
let first = graph.first_span();
|
||||
let (cat, title) = graph.nice_name();
|
||||
let name = format_span_name(cat, title);
|
||||
let count = graph.count() as u64;
|
||||
let total_cpu = *graph.total_time();
|
||||
let total_corrected = *graph.corrected_total_time();
|
||||
let avg_cpu = total_cpu.checked_div(count).unwrap_or(0);
|
||||
let avg_corrected = total_corrected.checked_div(count).unwrap_or(0);
|
||||
|
||||
// Build the full path ID for this aggregated span.
|
||||
// The leaf segment is "a{first_span_index}"; prepend the parent
|
||||
// path (if any) with a dash so callers can pass the full string
|
||||
// back as `parent` to drill into children.
|
||||
let first_index = first.index;
|
||||
let graph_id = build_span_id(options.parent.as_deref(), &format!("a{first_index}"));
|
||||
|
||||
// start/end of the first/example span relative to parent.
|
||||
let span_start = *first.start();
|
||||
let span_end = *first.end();
|
||||
let rel_start = (span_start as i64) - (parent_start as i64);
|
||||
let rel_end = (span_end as i64) - (parent_start as i64);
|
||||
|
||||
SpanInfo {
|
||||
id: graph_id,
|
||||
name,
|
||||
cpu_duration: *first.total_time(),
|
||||
corrected_duration: *first.corrected_total_time(),
|
||||
start_relative_to_parent: rel_start,
|
||||
end_relative_to_parent: rel_end,
|
||||
args: first
|
||||
.args()
|
||||
.map(|(k, v)| (k.to_string(), v.to_string()))
|
||||
.collect(),
|
||||
is_aggregated: count > 1,
|
||||
count: Some(count),
|
||||
total_cpu_duration: Some(total_cpu),
|
||||
avg_cpu_duration: Some(avg_cpu),
|
||||
total_corrected_duration: Some(total_corrected),
|
||||
avg_corrected_duration: Some(avg_corrected),
|
||||
first_span_id: Some(first_index.to_string()),
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
QueryResult {
|
||||
spans,
|
||||
page,
|
||||
total_pages,
|
||||
total_count,
|
||||
}
|
||||
} else {
|
||||
// Raw spans mode.
|
||||
let raw_children: Vec<SpanRef<'_>> = if let Some(ref parent) = parent_span {
|
||||
parent.children().collect()
|
||||
} else {
|
||||
store_ref.root_spans().collect()
|
||||
};
|
||||
|
||||
// Apply search filter using the span's search index.
|
||||
let mut filtered: Vec<_> = if let Some(ref query) = options.search {
|
||||
if let Some(ref parent) = parent_span {
|
||||
parent.search(query).collect()
|
||||
} else {
|
||||
store_ref.root_span().search(query).collect()
|
||||
}
|
||||
} else {
|
||||
raw_children
|
||||
};
|
||||
|
||||
// Sort if requested.
|
||||
if options.sort {
|
||||
filtered.sort_by(|a, b| {
|
||||
b.corrected_total_time()
|
||||
.cmp(&a.corrected_total_time())
|
||||
.then_with(|| b.total_time().cmp(&a.total_time()))
|
||||
});
|
||||
}
|
||||
|
||||
let (page_items, page, total_pages, total_count) = paginate(filtered, options.page);
|
||||
|
||||
let spans = page_items
|
||||
.into_iter()
|
||||
.map(|span| {
|
||||
let (cat, title) = span.nice_name();
|
||||
let name = format_span_name(cat, title);
|
||||
let span_start = *span.start();
|
||||
let span_end = *span.end();
|
||||
let rel_start = (span_start as i64) - (parent_start as i64);
|
||||
let rel_end = (span_end as i64) - (parent_start as i64);
|
||||
|
||||
SpanInfo {
|
||||
id: build_span_id(options.parent.as_deref(), &span.index.to_string()),
|
||||
name,
|
||||
cpu_duration: *span.total_time(),
|
||||
corrected_duration: *span.corrected_total_time(),
|
||||
start_relative_to_parent: rel_start,
|
||||
end_relative_to_parent: rel_end,
|
||||
args: span
|
||||
.args()
|
||||
.map(|(k, v)| (k.to_string(), v.to_string()))
|
||||
.collect(),
|
||||
is_aggregated: false,
|
||||
count: None,
|
||||
total_cpu_duration: None,
|
||||
avg_cpu_duration: None,
|
||||
total_corrected_duration: None,
|
||||
avg_corrected_duration: None,
|
||||
first_span_id: None,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
QueryResult {
|
||||
spans,
|
||||
page,
|
||||
total_pages,
|
||||
total_count,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve a span by its MCP ID string.
|
||||
///
|
||||
/// IDs use the format `[a]<index>[-[a]<index>...]`:
|
||||
/// - A plain decimal segment (e.g. `"123"`) refers to a raw span at that store index.
|
||||
/// - A segment prefixed with `"a"` (e.g. `"a123"`) refers to the first span of an aggregated group
|
||||
/// at that store index.
|
||||
/// - Segments are separated by `-` to form a navigation path, e.g. `"a5-a34-20"`. Only the **last**
|
||||
/// segment is needed to look up the span whose children we want to enumerate; the earlier
|
||||
/// segments provide navigation context for the caller.
|
||||
fn resolve_span_by_id<'a>(store: &'a store::Store, id: &str) -> Option<SpanRef<'a>> {
|
||||
// Take only the last path segment (everything after the final `-`).
|
||||
let last = id.split('-').next_back().unwrap_or(id);
|
||||
// Strip the optional "a" prefix that marks aggregated spans.
|
||||
let index_str = last.strip_prefix('a').unwrap_or(last);
|
||||
let index: usize = index_str.parse().ok()?;
|
||||
store.spans.get(index).map(|s| SpanRef {
|
||||
span: s,
|
||||
store,
|
||||
index,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -18,6 +18,12 @@ struct StoreWithGeneration {
|
||||
generation: usize,
|
||||
}
|
||||
|
||||
impl Default for StoreContainer {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl StoreContainer {
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
|
||||
Reference in New Issue
Block a user