Files
Andrew ImmandGitHub ca414736ba telemetry: add agentName to anonymous metadata (#95586)
Add agentName to anonymous telemetry metadata

### What

Adds an agentName: string | null field to the anonymous telemetry
metadata block that accompanies every payload, alongside existing fields
like ciName, nextVersion, and isCI. The value is the detected AI coding
agent driving the process (e.g. claude, cursor, codex), or null when
none is detected.

### Why

We currently have no visibility into how much Next.js development
happens under AI coding agents. This field lets us understand that split
in the same anonymous way we already track other CLI usage.

### How

Agent detection via @vercel/detect-agent
- Vendored @vercel/detect-agent (from the Vercel CLI) into
`next/dist/compiled/@vercel/detect-agent` (new taskfile ncc task,
externals entry, and type declaration), matching how I *understand*
other third-party deps are bundled.
- The vendored detection method is async (it does a filesystem probe for
Devin in addition to env-var checks) and potentially slower than we'd
want for logging, so it's wrapped in a memoized helper:
`telemetry/agent-name.ts`. `getAgentName()` caches the promise from
`determineAgent()`: the first call runs detection once, concurrent
callers share that in-flight promise, and every later call resolves
against the already-settled result. The agent shouldn't change over the
process lifetime, so this is safe. Detection failures resolve to null so
telemetry never throws on this path.

Wiring into the payload
- `getAnonymousMeta()` is now async and includes `agentName: await
getAgentName()`. `storage.ts` awaits it on the send path (the record
method was thankfully already async).
- The NEXT_TELEMETRY_DEBUG=1 output now includes the meta block
alongside each event, so the full payload — including agentName — is
observable locally without sending anything. We didn't output this
before, so it was harder to debug these fields.

Consolidating agent detection
- There was a second, synchronous, hand-vendored copy of agent detection
(telemetry/detect-agent.ts) used only by the AGENTS.md/CLAUDE.md
auto-generation hook. It's been removed and that hook
(ensureAgentRulesForDev) now uses the same vendored `getAgentName()`.
Now we have a single source of truth and can easily get upstream changes
from the vercel CLI package. `ensureAgentRulesForDev` became async, its
only caller in start-server.ts was already in an async scope, so this is
easy.

### Testing

- NEXT_TELEMETRY_DEBUG=1 next build shows agentName in the payload meta
block next to ciName/nextVersion.
- Verified the memoized helper: agent present -> name; cleared env ->
null; repeated calls return the same promise instantly.
- agent-rules-auto-generate dev tests pass (agent-detected generates
files; cleared-env does not)
- next-server-nft trace snapshot updated: adds @vercel/detect-agent +
agent-name.js, removes detect-agent.js.
- ts, build, existing telemetry unit test, lint
2026-07-14 16:56:01 -07:00

200 lines
6.4 KiB
TypeScript

import { nextTestSetup } from 'e2e-utils'
import fs from 'fs'
import os from 'os'
import path from 'path'
import { writeAgentFiles } from 'next/dist/server/lib/generate-agent-files'
const AGENT_RULES_MARKER = '<!-- BEGIN:nextjs-agent-rules -->'
/**
* The canonical block as the version under test generates it,
* obtained by running the real generator into a temp dir — the test
* never hardcodes the wording.
*/
function currentAgentRulesBlock(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-rules-'))
writeAgentFiles(dir)
return fs.readFileSync(path.join(dir, 'AGENTS.md'), 'utf-8').trimEnd()
}
describe('agent-rules auto-generate on next dev (agent detected)', () => {
const { next } = nextTestSetup({
files: __dirname,
env: { CLAUDECODE: '1' },
})
it('creates AGENTS.md and CLAUDE.md at the project root when neither exists', async () => {
// A request is required to synchronize the test with the auto-gen
// hook — `✓ Ready in X` is logged before the config load that runs
// the hook, so `next.start()` resolves too early. `next.fetch` blocks
// on the request handler, which only becomes ready after the hook.
await next.fetch('/')
const agentsContent = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(agentsContent).toContain(AGENT_RULES_MARKER)
expect(agentsContent).toContain('node_modules/next/dist/docs/')
const claudeContent = fs.readFileSync(
path.join(next.testDir, 'CLAUDE.md'),
'utf-8'
)
expect(claudeContent).toBe('@AGENTS.md\n')
})
})
describe('agent-rules auto-generate on next dev (no agent)', () => {
const { next } = nextTestSetup({
files: __dirname,
// Explicitly clear every env var the agent detector inspects so the
// test doesn't inherit one from the host shell (e.g. running it
// inside Claude Code would otherwise trigger generation).
env: {
AI_AGENT: '',
CURSOR_TRACE_ID: '',
CURSOR_AGENT: '',
GEMINI_CLI: '',
CODEX_SANDBOX: '',
CODEX_CI: '',
CODEX_THREAD_ID: '',
ANTIGRAVITY_AGENT: '',
AUGMENT_AGENT: '',
OPENCODE_CLIENT: '',
CLAUDECODE: '',
CLAUDE_CODE: '',
REPL_ID: '',
COPILOT_MODEL: '',
COPILOT_ALLOW_ALL: '',
COPILOT_GITHUB_TOKEN: '',
},
})
it('does not create AGENTS.md or CLAUDE.md when no agent is detected', async () => {
await next.fetch('/')
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
expect(fs.existsSync(path.join(next.testDir, 'CLAUDE.md'))).toBe(false)
})
})
describe('agent-rules auto-generate on next dev (agentRules: false)', () => {
const { next } = nextTestSetup({
files: __dirname,
env: { CLAUDECODE: '1' },
nextConfig: {
agentRules: false,
},
})
it('does not generate files when agentRules is disabled in next.config', async () => {
await next.fetch('/')
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
expect(fs.existsSync(path.join(next.testDir, 'CLAUDE.md'))).toBe(false)
})
})
describe('agent-rules auto-generate on next dev (AGENTS.md has an outdated managed block)', () => {
const { next } = nextTestSetup({
files: __dirname,
env: { CLAUDECODE: '1' },
skipStart: true,
})
// A block written by a different Next.js version — the content
// between the markers should be upgraded in place while everything
// around it is preserved. Uses synthetic body text so this test does
// not depend on any particular shipped wording.
const OUTDATED_BLOCK = `${AGENT_RULES_MARKER}
# Stale heading from an older Next.js
Stale body from an older Next.js.
<!-- END:nextjs-agent-rules -->`
beforeAll(async () => {
await next.patchFile(
'AGENTS.md',
`# Team rules\n\nUse tabs, not spaces.\n\n${OUTDATED_BLOCK}\n`
)
await next.start()
})
it('refreshes the block in place and preserves surrounding content', async () => {
await next.fetch('/')
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain('Use tabs, not spaces.')
expect(content).not.toContain('Stale body from an older Next.js.')
expect(content).toContain(currentAgentRulesBlock())
// Exactly one managed block — upgraded, not duplicated.
expect(content.split(AGENT_RULES_MARKER).length - 1).toBe(1)
expect(fs.existsSync(path.join(next.testDir, 'CLAUDE.md'))).toBe(false)
})
it('is idempotent across dev server restarts', async () => {
await next.fetch('/')
const before = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
await next.stop()
await next.start()
await next.fetch('/')
const after = fs.readFileSync(path.join(next.testDir, 'AGENTS.md'), 'utf-8')
expect(after).toBe(before)
})
})
describe('agent-rules auto-generate on next dev (AGENTS.md exists without marker)', () => {
const { next } = nextTestSetup({
files: __dirname,
env: { CLAUDECODE: '1' },
skipStart: true,
})
beforeAll(async () => {
// User-authored AGENTS.md without the managed marker — auto-gen
// should upsert the block while preserving existing content.
await next.patchFile('AGENTS.md', '# Team rules\n\nUse tabs, not spaces.\n')
await next.start()
})
it('upserts the managed block and preserves existing content', async () => {
await next.fetch('/')
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain('Use tabs, not spaces.')
expect(content).toContain(AGENT_RULES_MARKER)
// CLAUDE.md must stay alone when AGENTS.md already exists.
expect(fs.existsSync(path.join(next.testDir, 'CLAUDE.md'))).toBe(false)
})
})
describe('agent-rules auto-generate on next dev (CLAUDE.md exists, no AGENTS.md)', () => {
const { next } = nextTestSetup({
files: __dirname,
env: { CLAUDECODE: '1' },
skipStart: true,
})
beforeAll(async () => {
await next.patchFile('CLAUDE.md', '# My rules\n\nBe concise.\n')
await next.start()
})
it('upserts into CLAUDE.md and does not create AGENTS.md', async () => {
await next.fetch('/')
const claudeContent = fs.readFileSync(
path.join(next.testDir, 'CLAUDE.md'),
'utf-8'
)
expect(claudeContent).toContain('Be concise.')
expect(claudeContent).toContain(AGENT_RULES_MARKER)
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
})
})