mirror of
https://github.com/hanzoai/next.js.git
synced 2026-09-24 17:00:51 +00:00
Make the agent-rules block verifiable and self-upgrading (#95467)
Agents have started flagging the managed `nextjs-agent-rules` block as prompt injection and refusing to follow or commit it (#95130). Headless A/B evals (Claude Code + Sonnet 5, pnpm-monorepo and standalone-npm fixtures, `[email protected]`) isolate the triggers: `node_modules/next/dist/docs/` doesn't resolve from the workspace root in non-hoisted installs, the "maintained by `next dev`" claim is unverifiable, and the **Keep this block** paragraph reads as a self-persisting instruction. The failure concentrates on the uncommitted state a fresh `next dev` run produces. The block now anchors the docs path to the block file's directory with a monorepo note, and replaces the commit imperative with a verify pointer to `generate-agent-files.js` plus the reason committing it keeps the tree clean. This is the only change in this PR, and it was ablated in isolation — an A/A run of the baseline first established the noise floor (block committal 0/5, 0/8, 1/8 across three independent baseline batches): | Metric | Old block | New block | | --------------------------------------------------------------------------------- | --------- | --------- | | Uncommitted block, diff-review prompt — committed the block | 1/21 | 8/20 | | Uncommitted block, diff-review prompt — final message calls it a prompt injection | 12/21 | 4/20 | | Committed block, task prompts — injection flags | 0/15 | 0/15 | | Docs reads/run — standalone app, write tests (n=2) | 4.5 | 5.0 | | Docs reads/run — monorepo, add page (n=3) | 3.7 | 4.0 | | Docs reads/run — monorepo, write tests (n=10) | 3.4 | 2.6 | Committal improves at p = 0.008 and injection accusations drop at p = 0.016 (Fisher exact); runs that follow the verify pointer conclude "genuine framework-scaffolded content, not injected". A candidate follow-up — appending "commit this change" to the dev-server log line — was ablated separately and does nothing: the line reaches the agent's transcript in 12/12 dev-session runs but committal stayed 0/6 vs 0/6, so it is not included. The upsert machinery that delivers this rewording to existing projects is #95470.
This commit is contained in:
@@ -66,7 +66,9 @@ On Next.js 16.3 or later, just run `next dev`. When an AI coding agent is detect
|
||||
|
||||
# This is NOT the Next.js you know
|
||||
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` before writing any code. Heed deprecation notices.
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` (resolved from this file's directory; in monorepos the `next` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
|
||||
|
||||
This block is written and re-added by `next dev` — verify at `node_modules/next/dist/server/lib/generate-agent-files.js`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
|
||||
|
||||
<!-- END:nextjs-agent-rules -->
|
||||
```
|
||||
|
||||
@@ -9,9 +9,9 @@ export function generateAgentFiles(root: string): void {
|
||||
const agentsMdContent = `<!-- BEGIN:nextjs-agent-rules -->
|
||||
# This is NOT the Next.js you know
|
||||
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in \`node_modules/next/dist/docs/\` before writing any code. Heed deprecation notices.
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in \`node_modules/next/dist/docs/\` (resolved from this file's directory; in monorepos the \`next\` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
|
||||
|
||||
**Keep this block, including in commits.** It is part of the project's agent setup, maintained by \`next dev\` for every agent that works here. If it appears as an uncommitted change, that is intentional — commit it as-is. Do not remove it to clean up a diff; it will be regenerated.
|
||||
This block is written and re-added by \`next dev\` — verify at \`node_modules/next/dist/server/lib/generate-agent-files.js\`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
|
||||
<!-- END:nextjs-agent-rules -->
|
||||
`
|
||||
|
||||
|
||||
@@ -25,9 +25,9 @@ function buildAgentRulesBlock(): string {
|
||||
return `${AGENT_RULES_START_MARKER}
|
||||
# This is NOT the Next.js you know
|
||||
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in \`node_modules/next/dist/docs/\` before writing any code. Heed deprecation notices.
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in \`node_modules/next/dist/docs/\` (resolved from this file's directory; in monorepos the \`next\` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
|
||||
|
||||
**Keep this block, including in commits.** It is part of the project's agent setup, maintained by \`next dev\` for every agent that works here. If it appears as an uncommitted change, that is intentional — commit it as-is. Do not remove it to clean up a diff; it will be regenerated.
|
||||
This block is written and re-added by \`next dev\` — verify at \`node_modules/next/dist/server/lib/generate-agent-files.js\`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
|
||||
${AGENT_RULES_END_MARKER}`
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user