Compare commits

...
417 Commits
Author SHA1 Message Date
Nimar 1bd069f24f chore: release v3.167.4 2026-04-10 19:33:39 +02:00
52dcb23953 chore(deps): override path-to-regexp to bump to non-vulnerable version (#12931)
'path-to-regexp' v0.1.13 was released 5 days ago to fix CVE-2026-4867
https://github.com/pillarjs/path-to-regexp/commit/7ccf02cee33402f06ed2125085992ee9cd3a7c45

This PR manually override `path-to-regexp` dependency coming from `dd-trace` to patch the CVE

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-04-10 19:33:05 +02:00
NimarandGitHub 2abaa0438e chore(ci): fix docker image upload (#13113)
* chore(ci): fix docker image upload

* simp
2026-04-10 17:31:11 +00:00
1e6d0a70a0 fix(worker): advance experiment backfill cursor when no items to process (#13107)
The backfill cursor was only advanced inside the chunk-processing loop,
which is never reached when the query returns zero dataset run items.
This caused last_run_delay_seconds to grow indefinitely in environments
with no recent experiment activity.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 15:41:29 +00:00
Nimar e91239e3ac chore: release v3.167.3 2026-04-10 17:35:44 +02:00
NimarandGitHub 418a2bf308 chore(ci): use blacksmith arm runners for docker image build (#13103)
* chore(ci): use blacksmith arm runners for docker image build

* stable rerun

* fix login
2026-04-10 15:34:13 +00:00
marliessophieandGitHub d08ce5bb71 style(experiment-compare): Add structured experiment color styles and visual accents for grid rows/columns (#13061)
* feat(web): improve experiment detail compare run styling

* fix(web): refine experiment compare accent markers

* fix(web): replace grid accent lines with marker bars

* style: improve

* refactor: streamline rendering logic in ExperimentItemsTable and enhance loading state presentation
2026-04-10 14:55:00 +00:00
baab82adae chore(ai): add skill to upgrade dependencies easily (#13094)
* chore(ai): add skill to upgrade dependencies easily

* fix(evals): prevent llm-as-a-judge queue stalls (#13037)

* clneaup

* bump transitive deps if possible

* up skill

* fix

* add timeout

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-04-10 14:54:45 +00:00
Valery MeleshkinandGitHub 932bd18df8 fix(codex): run clickhouse server as clickhouse user (#13101) 2026-04-10 16:01:38 +02:00
Nimar 4a13377e35 chore: release v3.167.2 2026-04-10 15:40:12 +02:00
NimarandGitHub 30af822ac9 chore(deps): bump defu (#13100) 2026-04-10 13:32:54 +00:00
NimarandGitHub c2c0b661e7 chore(deps): bump hono to 4.12.12 (#13099) 2026-04-10 13:22:12 +00:00
Hassieb PakzadandGitHub 2e94ebfe4b fix(evals): prevent llm-as-a-judge queue stalls (#13037) 2026-04-10 14:11:59 +02:00
NimarandGitHub b8544b3423 chore(deps): bump next to 16.2.3 (#13092) 2026-04-10 10:20:59 +00:00
NimarandGitHub 24cc309fb8 chore(deps): bump lodash 4.18.1 (#13090) 2026-04-10 09:44:35 +00:00
NimarandGitHub 1ca70d7033 chore(deps): bump langchain 1.1.39 and related (#13089) 2026-04-10 09:33:32 +00:00
NimarandGitHub ba980c302e chore(deps): bump slack and thus axios 1.15.0 (#13088) 2026-04-10 09:26:48 +00:00
Hassieb PakzadandGitHub ea197e4287 fix(llm-execution-tracing): imperatively set internal tracing environment on events (#13085) 2026-04-10 11:28:48 +02:00
NimarandGitHub 0b20e4d366 chore(deps): build go migrate with clickhouse only (#13082)
* chore(deps): build go migrate with clickhouse only

* add comment
2026-04-10 09:21:55 +00:00
NimarandGitHub 31a1a34616 chore(deps): bump node mocks to 1.17.2 (#13087) 2026-04-10 09:16:59 +00:00
Valery MeleshkinandGitHub 3c3d4bf129 chore: add scripts to provision and run local cloud dependencies (Postgres, Redis, ClickHouse, MinIO) and setup/maintenance helpers (#13054) 2026-04-10 11:19:24 +02:00
07cae52cc7 fix: validate Azure blob storage container names (#13080)
* fix: validate Azure blob storage container names

Azure requires container names to be 3-63 chars, lowercase alphanumeric
and hyphens only. Add Zod superRefine validation to the form schema,
tRPC router, and public API schema so invalid names like "Feedback N8N Bot"
are rejected at submission time with a clear error message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback for Azure container name validation

- Add empty-string guard in validateAzureContainerName to avoid double
  error when bucketName is blank
- Add .min(1) to public API bucketName schema to match tRPC form schema
- Add Fern docs note describing Azure container naming constraints
- Add server test for invalid Azure container name rejection
- Add client test for empty-string guard behavior

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:11:04 +00:00
Ben BachemandGitHub a81edec0be fix(scores-table): Unused omittedFilter prop (#13079) 2026-04-10 09:07:16 +00:00
Ben BachemandGitHub 497179934d fix(web): Table padding issues (#13060)
* fix(web): Table padding issues

* Increase cell padding in `SelectDashboardDialog` and `SelectWidgetDialog`

* Fix memoization comparison for cellPadding in DataTable

* Set cellPadding="comfortable" for `MembersTable` in org settings
2026-04-10 09:07:08 +00:00
NimarandGitHub ad9dfc41a2 chore(deps): bump vitest to 4.1.4 (#13086) 2026-04-10 09:03:48 +00:00
Tobias Wochinger 9cc69f4c67 chore: release v3.167.1 2026-04-10 10:29:22 +02:00
557f284cd1 fix(web): allow all unicode letters for signups (#12999)
* fix(web): allow unicode letters in signup name validation

* refactor(web): share name schema between signup and display name

* fix(web): enforce 100-char limit in shared name schema

* fix(web): allow hyphens, apostrophes, and periods in name validation

The nameSchema regex was too strict, rejecting common name characters
like O'Brien, Smith-Jones, and Dr. Smith. Also align the backend
updateDisplayName schema with the shared nameSchema for consistency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): normalize smart quotes and require letter in name validation

Normalize curly/smart apostrophes (U+2018, U+2019, U+02BC) from mobile
autocorrect to straight apostrophe before validation. Require at least
one letter to reject degenerate punctuation-only names like "---".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): require base letter not combining mark in name validation

The "must contain at least one letter" refine accepted standalone
combining marks (\p{M}) without an actual letter (\p{L}), allowing
inputs like "\u0301\u0301" to pass as valid names.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): require base letter not combining mark in name validation

Add NFC normalization before validation so decomposed characters merge
into precomposed form, and add a negative lookahead (?!\p{M}) to reject
names that still start with a combining mark after normalization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): revert display name form to permissive schema and simplify nameSchema

Revert settings and userAccount display name validation back to
StringNoHTML.min(1).max(100) — the signup-oriented nameSchema is too
restrictive for existing display names containing underscores, ampersands, etc.

Simplify nameSchema: merge transforms, combine regex constraints into a single
refine that requires names start with a letter, and remove U+02BC from
smart-quote normalization (it's a linguistic letter, not a typographic quote).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:35:41 +00:00
NimarandGitHub 6702c7b50f chore(deps): bump lodash to 4.18.1 in worker (#13063)
* chore(deps): update package wait to 5days

* remove superfluous

* chore(deps): bump lodash to 4.18.1 in worker
2026-04-09 16:57:03 +00:00
25d99aa371 fix: make Slack integration more robust (#13004)
* fix: make Slack integration more robust

* refactor: deduplicate scopes

* fix(slack): make SlackChannel isPrivate and isMember optional

These fields are only known for channels from the fetched list, not for
manually-typed channel names. Making them optional avoids placeholder
booleans and fixes a type error when constructing partial channel objects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: gracefully handle missing scopes

* fix(slack): cap rate-limit retry, resolve manual channel IDs, add empty state

- Cap retryAfter to 60s max to avoid gateway timeouts on large Slack values
- Add onSuccess handler in SlackActionForm to resolve #channel names to real IDs
- Show empty state message in ChannelSelector when bot has no accessible channels

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(slack): resolve manual channel IDs, virtualize list, fix audit log

- Use resolved Slack channel ID in audit log instead of #-prefixed input
- Replace VirtualizedList with cmdk Command + @tanstack/react-virtual
  for keyboard navigation and DOM-efficient rendering of ~5k channels
- Move "Use typed name" fallback to separate CommandGroup so it stays
  visible when the virtualized group has zero height
- Import SlackChannel type from @langfuse/shared instead of redeclaring
- Add getChannelInfo mock and #-prefixed channelId test
- Use .concat() instead of spread for channel pagination (repo convention)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: switch to SDK retry policies

* fix(slack): strip duplicate # prefix and use functional setState

Strip leading # from channelId fallback in test message block to avoid
displaying ##general for manually-typed channel names. Use functional
setSelectedChannel form in slack.tsx to match SlackActionForm.tsx pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(slack): guard CommandEmpty on filteredChannels length

Prevent flash of "No channels available." on popover open by explicitly
guarding CommandEmpty rendering on filteredChannels.length === 0 instead
of relying on cmdk's internal item count, which is 0 on the first
render before the virtualizer scroll container mounts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: review comments

* chore: another round of review feedback

* chore: more review comments

* fix(slack): improve channel selector search

* fix comment

* fix(slack): refine channel selector search

* fix(slack): sync manifest scopes

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 16:46:36 +00:00
NimarandGitHub e4d5f914cc chore(deps): bump next to 16.2.2 (#13068)
* chore(deps): bump next to 16.2.2

* bump big
2026-04-09 18:26:23 +02:00
Hassieb PakzadandGitHub dcb5dbf528 fix(llm-connections): validate new LLM base URLs (#13073) 2026-04-09 17:25:59 +02:00
Valery MeleshkinandGitHub d003a9c3f4 fix: limt media deletion batch size to avoid pg bind limits (#13072) 2026-04-09 16:29:25 +02:00
Valery MeleshkinandGitHub d2d56f0337 fix: allow Bearer auth on POST scores API (#13064)
fix: allow Bearer auth on POST scores API.

Addresses https://github.com/langfuse/langfuse/issues/12947
2026-04-09 13:45:29 +00:00
NimarandGitHub 6c0cf07a5a chore(deps): update package wait to 5days (#13062)
* chore(deps): update package wait to 5days

* remove superfluous
2026-04-09 12:39:11 +00:00
Hassieb Pakzad dd632fea9e chore: release v3.167.0 2026-04-09 14:26:32 +02:00
Hassieb PakzadandGitHub 7527bb0d84 fix(web): require secret key for LLM test base URL changes (#13055) 2026-04-09 14:25:31 +02:00
NimarandGitHub 8cc4a5537f fix(cicd): re-add nextauth etc to docker (#12865)
* fix(cicd): re-add nextauth etc to docker

* clarify

* fix prisma version

* one more comment
2026-04-09 12:09:01 +00:00
marliessophieandGitHub 0a6d3f108a chore(experiments): Link dataset cell in Experiments table to dataset page and show display name (#13059)
fix(experiments): render dataset badge label without table-id component
2026-04-09 12:05:11 +00:00
Ben BachemandGitHub 1bf83313e3 fix(trace-table): Only disable URL persistence for ScoresTable in peek mode (#12963)
fix(web): Only disable URL persistence for `ScoresTable` in peek mode
2026-04-09 11:52:40 +00:00
marliessophieandGitHub 9c3a715d77 fix(annotation): Wait for session to load before rendering annotation queue items (#13058)
fix(web): simplify annotation queue loading state guard
2026-04-09 11:32:58 +00:00
marliessophieandGitHub 0cf2a33473 fix(v4-add-to-dataset): Allow non-string JSON prefill values for new dataset items (#13053)
* fix(datasets): normalize add-to-dataset prefill values

* fix(datasets): preserve parsed null prefill values
2026-04-09 11:11:09 +00:00
Valery MeleshkinandGitHub 51554eb066 chore(dx): add blob storage docs review checks to AGENTS.md (#13056) 2026-04-09 12:24:47 +02:00
marliessophieandGitHub cbc21bb9cc feat(annotation-queues): integrate session handling and beta feature flag in AnnotationQueueItemPage and update router for observation fetching (#13050) 2026-04-09 09:44:50 +00:00
Valery MeleshkinandGitHub 8e30694214 Revert "chore: optional docker setup in Codex setup/maintenance scripts" (#13049)
Revert "chore: optional docker setup in Codex setup/maintenance scripts (#13035)"

This reverts commit 0d20d9de2b.
2026-04-09 10:38:13 +02:00
Valery MeleshkinandGitHub 0d20d9de2b chore: optional docker setup in Codex setup/maintenance scripts (#13035)
* fix(codex): install golang-migrate in docker setup

* fix(codex): include local bin path in maintenance

* fix(codex): load local bin path in setup shell

* fix(codex): verify migrate checksum and safe extract

* fix(codex): improve migrate install error guidance
2026-04-08 17:03:11 +00:00
Tobias Wochinger eeeba25439 chore: release v3.166.0 2026-04-08 19:03:32 +02:00
a59630d656 chore(dx): add more steps for pre-commit (#12901)
* chore(dx): add more stop for pre-commit

* chore: add type checking as well

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add auto-fixed files to diff

* chore: change to not modifying / remove typecheck

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 16:04:41 +00:00
Valery MeleshkinandGitHub 10e7dea9c9 fix: getTraceById metadata (#13042) 2026-04-08 17:35:07 +02:00
marliessophieandGitHub 9bf326db4f Revert "fix(dataset-items): update current dataset item versions in upsert function" (#13043)
Revert "fix(dataset-items): update current dataset item versions in upsert fu…"

This reverts commit c393c64a40.
2026-04-08 17:26:31 +02:00
13190c3ec4 perf(trace-ui): Exclude and tool columns from trace observations query when IO is not requested (#12948)
* put prompt_* and tool_* behind 'includeIO'

* ordering

* put prompt outside of io

* also omit for events

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-04-08 15:05:50 +00:00
Ben BachemandGitHub 7a4ce9ee5d fix(web): Inconsistent search results between editor and controller (#13038) 2026-04-08 15:05:37 +00:00
NimarandGitHub bc02989ccf fix(ui): remove right screen side handle on mobile (#13036) 2026-04-08 17:02:55 +02:00
NimarandGitHub f0dac0299c chore(deps): bump turbo to 2.9.5 (#13032) 2026-04-08 14:09:50 +00:00
Valery MeleshkinandGitHub 19997064c1 feat(api): add fields parameter to GetTraceById endpoint (#13015) 2026-04-08 13:58:34 +00:00
marliessophieandGitHub c393c64a40 fix(dataset-items): update current dataset item versions in upsert function (#13034) 2026-04-08 13:42:47 +00:00
d04e027107 fix(prompt-automations): prompt creations no longer trigger webhooks for unfiltered event actions (#13000)
* fix(prompt): webhook triggers honor eventAction filters

* fix(automations): added validation of event actions

* test(automations): add deleted event action test case to promptVersionProcessor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Revert "fix(automations): added validation of event actions"

This reverts commit 210344f49b45fd80e79fecf6841dd37dbe822a28.

* fix(test): update setupTriggerAndAction to match all event actions

The helper used eventActions: ["updated"] which broke the prompt
creation test after eventActions filtering was enforced. Using []
matches all actions, covering both created and updated test cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(ci): retrigger stuck license cla check

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:21:03 +00:00
marliessophieandGitHub 53869c7200 fix(dataset-items): handle version conflict errors in dataset item upsert (#13031) 2026-04-08 12:49:20 +00:00
6d964894fb fix(api): return archived item when getting dataset item by ID (#13028)
fix(api): return archived dataset items from GET endpoint

Previously GET /api/public/dataset-items/{id} returned 404 for archived
items. Now it returns them with their status, matching user expectations
for direct ID lookups.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 12:24:44 +00:00
81653b2bd2 ci: add cla-assistant workflow to retrigger stuck CLA checks (#13021)
Workaround for a known cla-assistant bug where the CLA check gets stuck
after a contributor signs. Comment `/check-cla` on any PR to manually
retrigger. See: https://github.com/cla-assistant/cla-assistant/issues/528

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 12:02:08 +00:00
marliessophieandGitHub 02d6486f10 chore(dataset-items): patch API on version collision (#13029)
* chore(dataset-items): patch API to throw 4xx instead on version collision

* chore: push
2026-04-08 11:46:38 +00:00
07ee4ed961 fix(web): Improve search highlighting in CodeMirrorEditor (#12961)
* fix(web): Improve search highlighting in CodeMirrorEditor

* Add color variables

* Always call `syncEditorsToQuery` if the active changed

* make selector more spefific

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-04-08 11:24:22 +00:00
Hassieb Pakzad 00f770b053 chore: release v3.165.0 2026-04-08 13:25:58 +02:00
Hassieb PakzadandGitHub e12386f9d4 fix(llm-connections): enforce write permissions on LLM connection test endpoints (#13027)
* fix(llm-connections): require llmApiKeys:update permissions for testUpdate endpoint

* fix(llm-connections): preserve forbidden errors in testUpdate
2026-04-08 13:25:13 +02:00
Ben BachemandGitHub b1ac930462 fix(web): Add hint about secrets when copying .env in api key settings (#13001)
* fix(web): Add hint about secrets when copying .env in api key settings

* Integrate claude review feedback

* Address PR feedback
2026-04-08 08:34:07 +00:00
marliessophieandGitHub 8e9521522f fix: Add peek-mode local preview navigation for variable mapping previews (#12945)
* fix(evals): reset preview selection when peek target changes

* chore: push
2026-04-07 20:09:14 +00:00
NimarandGitHub 87ed9fe7e1 fix(data-table): don't auto refetch by default (#13018) 2026-04-07 18:53:07 +00:00
NimarandGitHub 1c40823f0c chore(deps): bump prisma to 6.19.3 for effect (#13017)
* chore(deps): bump prisma to 6.19.3 for effect

* remove override

* remove override
2026-04-07 20:42:35 +02:00
NimarandGitHub 730946f91e chore(deps): bump mcp to 1.29 (#13016) 2026-04-07 18:16:25 +00:00
fbec38c79e ci(sdk): replace poetry with uv in SDK API spec generation workflow (#13012)
* ci(sdk): replace poetry with uv in SDK API spec generation workflow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: add frozen flag

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 15:49:08 +00:00
6603be6711 feat(scores): make TEXT scores available via public API (#12937)
* feat(scores):  make `TEXT` scores available via public API

* fix(scores): address review feedback for TEXT score public API

- Remove TEXT example from v1 docs to match CORRECTION pattern
- Split PostScoresBody into v1 (excludes TEXT) and v2 (includes TEXT)
- Fix v2 schema to keep value required for non-TEXT types using
  per-branch extend instead of weakening the foundation schema
- Migrate merge() to extend() across validation schemas

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scores): address second round of review feedback

- Use local TextData without length constraints in v2 response schema
- Add CreateScoreDataTypeV1 enum in Fern to exclude TEXT from v1 POST
- Use function overloads in convertScoreToPublicApi for proper typing
- Fix misleading comment about v2 POST endpoint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(scores): support TEXT scores in v1 API

TEXT scores are now fully supported across both v1 and v2 APIs for
create, list, and get-by-id. Only CORRECTION remains v2-only. Uses
LISTABLE_SCORE_TYPES instead of AGGREGATABLE_SCORE_TYPES for v1 filtering.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scores): include TEXT in CreateScoreValue docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(scores): update error message and Fern inline docs to mention TEXT scores

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shared): use literal tuple for LISTABLE_SCORE_TYPES to narrow type

Array.filter() without a type predicate infers ScoreDataTypeType[],
so ListableScoreDataType incorrectly included CORRECTION at the type
level. Define as a literal tuple with `as const` to match the pattern
used by AGGREGATABLE_SCORE_TYPES.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): spread readonly LISTABLE_SCORE_TYPES to satisfy mutable array type

The `as const` readonly tuple was incompatible with the mutable array
parameter expected by `useSidebarFilterState`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shared): use TEXT_SCORE_MAX_LENGTH global constant in API and ingestion schemas

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): route rollback errors to correct form field for TEXT scores

The rollback error handlers unconditionally set errors on the `.value`
field, but TEXT scores render their `<FormMessage>` on `.stringValue`.
This caused server errors to be silently dropped for TEXT annotations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(web): fix type errors in rollback error field routing for TEXT scores

Use if/else branches instead of ternary to preserve template literal
types for react-hook-form's setError and clearErrors field paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 14:33:02 +00:00
859c58e96c feat(scores): implement internal changes for TEXT (free form) scores (#12902)
* feat: introduce free form scores

* chore: implement review comments

* chore: rename to `Text` score

* chore: review feedback

* chore: fix exposing scores in traces/observation view, export as well as prompts

* style: polishing for long values

* chore: fix CI issues

* fix(shared,worker): propagate TEXT score data type in export streams and session aggregation

- Extend ClickHouse tuples to include data_type as third element in
  buildScoresAggregationCTE, observation-stream, and trace-stream
- Read actual data_type from tuple instead of hardcoding CATEGORICAL
  in event-stream, observation-stream, and trace-stream
- Include TEXT scores in eventsSessionScoresAggregation filter

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: review

* fix(shared): preserve TEXT data type when inflating ingested scores

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(shared): extract TEXT score max length into global constant

Replace hardcoded max length (500) for TEXT scores with a shared
TEXT_SCORE_MAX_LENGTH constant for reuse across ingestion and public API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shared): address PR review comments for TEXT score type

- Rename misleading test names to match actual assertions (TEXT is included)
- Use ListableScoreDataType return type for getScoresGroupedByNameSourceType
- Replace hardcoded maxLength={500} with TEXT_SCORE_MAX_LENGTH constant

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(shared): widen score types to include TEXT in prompt scores and ScoreSimplified

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: last review comment

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 13:43:14 +00:00
marliessophieandGitHub 316f02f9dc fix(annotation): read data from events table if v4 beta is enabled (#13008)
* fix(annotation): read data from events table if v4 beta is enabled

* fix: read sessions from events table
2026-04-07 12:15:01 +00:00
Valery MeleshkinandGitHub 221d4e4489 feat(otel): warn on oversized request bodies exceeding 16MB (#13007) 2026-04-07 13:47:43 +02:00
Valery MeleshkinandGitHub 4a16f6cf33 fix(worker): clamp Decimal64(12) cost values before ClickHouse insertion (#13005) 2026-04-07 11:32:19 +00:00
Hassieb Pakzad 97789970ad chore: release v3.164.0 2026-04-07 11:36:54 +02:00
Hassieb PakzadandGitHub 863de18f68 fix(traces): include trace-scoped score columns (#12978) 2026-04-07 11:35:53 +02:00
7928adfe32 perf(worker): advance experiment backfill cursor per chunk (#12973)
perf(worker): advance experiment backfill cursor per chunk and add query timeouts

Previously the backfill cursor only advanced after ALL chunks succeeded.
If any chunk failed, the entire window was retried from scratch—causing
chunk 1 to re-run (with duplicate writes) and the same failing chunk to
block progress indefinitely.

Now the cursor advances after each successful chunk (items ordered ASC),
so on retry only the remaining chunks are processed. Also adds explicit
60s query timeouts to getRelevantObservations and getRelevantTraces, and
reduces the default chunk size from 200 to 100 for smaller blast radius.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 21:41:10 +00:00
6cfb4b6eac perf(worker): skip redundant IngestionService enrichment in experiment backfill (#12972)
* perf(worker): skip redundant IngestionService enrichment in experiment backfill

Spans processed by the experiment backfill already have model match,
usage details, cost details, and pricing tier data from ClickHouse.
Running them through IngestionService.createEventRecord() redundantly
re-does model matching (Redis + Postgres), tokenization, and cost
calculation. Convert EnrichedSpan directly to EventRecordInsertType
and write to ClickHouse, removing the IngestionService dependency.

Refs: LFE-9149

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore drop unused await

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 21:03:18 +00:00
17fc3bef71 feat(worker): add env var to exclude project IDs from experiment backfill (#12970)
Adds LANGFUSE_EXPERIMENT_BACKFILL_EXCLUDE_PROJECT_IDS env var (comma-separated)
to filter out specific projects after fetching eligible dataset run items,
preventing them from being processed in the experiment dual-write backfill.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 18:22:01 +00:00
Mark SalpeterandGitHub 93e4a5184c chore(clickhouse): add langfuse user-agent header to all ClickHouse requests (#12962) 2026-04-02 18:19:09 +02:00
Valery MeleshkinandGitHub 03483e7ccc chore: add explicit redis socket timeout and keepalive (#12964) 2026-04-02 17:38:30 +02:00
Valery MeleshkinandGitHub 68cc2b24f1 fix(otel): add defensive logging for bad cost/usage details and oversized spans (#12941)
* fix(otel): add defensive logging for bad cost/usage details and
oversized spans

* chore: last ditch logging when entire batch fails

* chore: tests for malformed _details handling
2026-04-02 13:03:45 +00:00
1793101288 fix(datasets): archived items public API (#12101)
* fix: exclude archived dataset items from public API responses

- Add status: ACTIVE filter to GET /dataset-items endpoint
- Add status: ACTIVE filter to GET /dataset-items/{id} endpoint
- Add comprehensive tests for archived item filtering

Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com>

* chore: adjust logs

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com>
Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2026-04-02 12:18:24 +00:00
fd6692186a perf(worker): optimize experiment backfill query and add delay metrics (#12960)
* perf(worker): optimize experiment backfill query and add delay metrics

Replace the broad events_core table scan with a CTE-driven approach that
first identifies candidate DRIs in the time window, then uses that small
set to drive the anti-join. This avoids scanning the full events_core
history.

Add two gauges to track backfill cursor delay so drift is detected early
rather than accumulating silently over months.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf(worker): add upper time bound to observation and trace queries

Adds a maxTime upper bound (chunkEnd + 7 days) to the getRelevantObservations
and getRelevantTraces queries so ClickHouse scans a bounded time range instead
of everything from minTime to now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 09:30:13 +00:00
Max DeichmannandGitHub 427305bb80 fix(worker): add llm-as-judge concurrency env (#12956) 2026-04-02 04:43:25 +02:00
marliessophieandGitHub 837800c014 chore(experiments): add searchable baseline selector (#12928)
* chore(experiments): add searchable baseline selector

* style: minor edits

* style: font size
2026-04-01 22:38:58 +00:00
e42ba4d28d fix(worker): cap experiment backfill to 8-hour windows (#12952)
* fix(worker): disable experiment backfill in event propagation queue

Temporarily disables the experiment backfill step in the event propagation
processor to address performance issues.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(worker): cap experiment backfill to 8-hour windows and re-enable

The backfill was disabled because a stale Redis timestamp caused unbounded
query windows (e.g. 5+ weeks). Each execution now processes at most 8 hours
of data, letting the scheduler catch up incrementally across runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 22:03:16 +02:00
295875d464 fix(worker): add ClickHouse request timeout for experiment backfill query (#12951)
Adds a 2-minute request timeout to the getDatasetRunItemsSinceLastRun
ClickHouse query to prevent long-running queries from hanging indefinitely.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:42:48 +00:00
Valery MeleshkinandGitHub ccd97fb8d2 perf: route get filter queries to readonly replicas (#12949) 2026-04-01 16:47:29 +00:00
7939cabc30 feat(tables-ui): support full text search targeting input/ouput directly (#11999)
* feat(tables-ui): support full text search targeting input or output

* feat(tests): add search functionality tests for generations, traces, and dataset items by input and output

* fix(search): use import type for TracingSearchType

Fix ESLint warning by using type-only import for TracingSearchType
since it's only used as a type annotation, not a runtime value.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(ui): add visual indicator to Full Text submenu when selected

- Add dot indicator next to Full Text submenu trigger when any child option is selected
- Matches existing pattern used for IDs/Names radio item
- Indicator appears for all full-text search modes (content, input, output)

* chore: build

* chore: build

* chore: build

* fix(prompts): enhance search functionality to include tag matching

* fix(tests): correct dataset items search test to use proper API signature

- Update test to use filterState instead of filter parameter
- Change offset to page parameter
- Use createDatasetItemFilterState helper for proper filter construction
- Fixes TypeError: Cannot read properties of undefined (reading 'map')

* fix(tests): use unique dataset name to avoid constraint conflicts

- Change hardcoded dataset name to v4() for uniqueness
- Prevents Unique constraint failed error when running full test suite

* test:generations

* docs: wording

* make faster

* fix: after rebase

* fix: imports

* fix: update searchType defaults in dataset items and prompt router

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-04-01 15:33:02 +00:00
Ben BachemandGitHub a94ebdf1c2 fix(web): Improve v4 promo banner styling on small displays (#12942)
* fix(web): Improve v4 promo banner styling on small displays
2026-04-01 15:45:51 +02:00
Hassieb PakzadandGitHub a408c63746 chore: bump langfuse-langchain (#12934) 2026-04-01 10:35:45 +02:00
Max DeichmannandGitHub 2121029663 revert(redis): add command and socket timeouts (#12930)
Revert "feat(redis): add command and socket timeouts for fail-fast behavior during outages (#12574)"

This reverts commit 1463ddc1aa.
2026-04-01 03:09:49 +02:00
marliessophieandGitHub 97a22a6ef2 style(experiments): render baseline as badge in header (#12927)
* style(experiments): render baseline as badge in header

* style: rm rounding

* style: tailwind classes
2026-03-31 20:47:08 +00:00
620f40c8cf fix(ui): shorten environment filter empty state help text (#12923)
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-31 19:22:05 +02:00
Valery MeleshkinandGitHub 942db0fc4e feat(blob-export): add more fields to v3 and v4 blob storage exports (#12840)
* feat(blob-export): add more fields to v3 and v4 blob storage exports

* feat(blob-export): add model pricing enrichment and missing fields to
v3/v4 exports
2026-03-31 17:04:21 +00:00
marliessophieandGitHub 0946411905 fix(filters): update sidebar filter state to support session persistence option (#12924) 2026-03-31 16:49:16 +00:00
Nimar a59d3236ae chore: release v3.163.0 2026-03-31 18:28:53 +02:00
NimarandGitHub 0d0b404a20 chore(score-analytics): refactor tests for speed (#12919)
* chore(score-analytics): refactor tests for speed

* less bins
2026-03-31 13:54:25 +00:00
1bfc25352d ci(GitHub actions): pin to SHA and configure dependabot for GH actions (#12916)
* ci: pin all GitHub Actions to commit SHAs

Pin all third-party GitHub Actions to their full commit SHAs for
supply-chain security, with the version tag preserved as a comment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ci: add dependabot config for grouped weekly GitHub Actions updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:39:33 +00:00
NimarandGitHub 347bb964e0 chore(deps): delay upgrades for min. 8 days (#12912)
* chore(deps): delay upgrades for min. 8 days

* exclude packages

* add
2026-03-31 12:42:31 +00:00
1463ddc1aa feat(redis): add command and socket timeouts for fail-fast behavior during outages (#12574)
During a Redis outage, commands hang indefinitely because ioredis has no
commandTimeout, no socketTimeout, and enableOfflineQueue defaults to true.
This causes cascading latency across the application.

- Add REDIS_COMMAND_TIMEOUT (2s default) for singleton and rate limiter
- Add REDIS_REQUEST_SOCKET_TIMEOUT_MS (5s default) for request/response connections
- Add REDIS_BLOCKING_SOCKET_TIMEOUT_MS (30s default) for all connections including
  BullMQ workers (safe because BZPOPMIN returns every ~5s drain delay)
- Centralize enableOfflineQueue: false in defaultRedisOptions, removing duplication
  from 34 queue files
- Fix cluster mode to forward enableOfflineQueue to top-level ClusterOptions
  (previously silently ignored in redisOptions)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 11:56:11 +00:00
cf29cbb43b feat(billing): add universal $4K default spend alert for all cloud plans (#12914)
feat(billing): add universal $4K default spend alert for all plans

Adds a $4000 spend alert on top of the existing plan-specific default
alerts on new subscriptions, as requested in LFE-8154 to help catch
unintentional high-spend loads across all plan tiers.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 11:39:40 +00:00
blacksmith-sh[bot]GitHubblacksmith-sh[bot] <157653362+blacksmith-sh[bot]@users.noreply.github.com>Nimar
231429548b chore(actions): Migrate workflows to Blacksmith runners (#12895)
* Migrate workflows to Blacksmith

* wait for up

* make redis IPs play well with blacksmith

---------

Co-authored-by: blacksmith-sh[bot] <157653362+blacksmith-sh[bot]@users.noreply.github.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-31 10:15:04 +00:00
8db45b440a fix(dashboard): fix race condition in InlineFilterBuilder preventing filter addition (#12715)
Fixes a race condition where adding a new filter row in the dashboard
widget form would immediately be overwritten before the user could
interact with it. The useEffect had wipFilterState in its dependency
array, causing it to re-run on every filter interaction; combined with
onChange being called inside _setWipFilterState's updater, this could
produce a stale hasWipFilters=false snapshot that reset the WIP state.

Applies the same prevFilterStateRef pattern already used in
PopoverFilterBuilder: bail out early when filterState reference hasn't
changed, and read current WIP state via functional updater to avoid
the race.

Closes #12569

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-31 09:52:04 +00:00
marliessophieandGitHub fc6ded3002 feat: polish experiments beta experience (#12883)
* feat: polish experiments beta experience

* chore: improve wording

* fix: rename runs tab label to "experiments"

* refactor: remove experiments beta toggle from navigation and update ExperimentsBetaSwitch with tooltip

* chore: default filter experiments

* chore: rename `runs` to `experiments` everywhere

* feat: enhance experiment navigation and layout with new analytics page and tab functionality

* chore: url navigation

* fix: redirect

* chore: push

* chore: disable url persistence
2026-03-31 09:03:08 +00:00
Jannik MaierhöferandGitHub 9a6d4a6284 feat(ui): switch from plain follow up email to pylon follow up email (#12911)
feat(ui): switch from plain follow up email to pylon follow up email after from submit
2026-03-31 08:53:37 +00:00
Jannik MaierhöferandGitHub 14f53d16fd feat(ui): write new support issues to pylon and plain (#12858) 2026-03-31 08:22:02 +00:00
NimarandGitHub 5ada0b4962 chore(deps): bump ajv to 8.18 (#12900) 2026-03-30 16:35:59 +00:00
afe51cd07b feat(exports): decode unicode escapes in batch export pipeline (#12882)
* feat(exports): decode unicode escapes in batch export pipeline

Apply decodeUnicodeEscapesOnly() to the batch export stringify functions
so that \uXXXX sequences (produced by Python SDK's json.dumps with
ensure_ascii=True) are decoded to their original characters in exported
CSV/JSON/JSONL files.

This follows the same approach already used in the Web UI (PR #9686)
where decodeUnicodeEscapesOnly() was added to IOTableCell for display.

Closes #10972

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback - move unicode.ts to shared, use greedy mode

- Move decodeUnicodeEscapesOnly to shared package and re-export from web
- Use greedy=true to match Web UI behavior (IOTableCell.tsx)
- Export from @langfuse/shared main index for Jest compatibility

* refactor: add early return for perf, fix test import path

- Add indexOf('\\') early return in decodeUnicodeEscapesOnly for fast
  path when no backslashes present
- Export stringify/stringifyForCsv from transforms barrel
- Use @langfuse/shared/src/server import in tests instead of relative path

* fix: add lone-surrogate guard in greedy mode

In greedy mode, when tryDecodeSurrogatePair fails due to double-escaped
backslashes, lone surrogates were emitted via String.fromCharCode(),
producing WTF-16 strings that corrupt to U+FFFD on UTF-8 write.

Fix: add greedy-aware surrogate pair decoding that skips extra backslashes,
and preserve lone surrogates as literal \uXXXX text (same as non-greedy mode).

Added tests for lone high/low surrogates in greedy mode.

* test: add backslash-between-surrogates edge case test in greedy mode

* minimize test cases

* preserve

* skip

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-30 16:20:19 +00:00
NimarandGitHub a140f624be chore(deps): remove ai sdk (#12894) 2026-03-30 14:33:12 +00:00
f449e66a36 fix(traces): fix v4 search query & filters (#12660)
* fix(traces): simplify v4 search query

* chore(traces): rename name -> traceName for filterOptions

* chores(traces): rename tags to traceTags for filterOptions

* chore(traces): simplify

* fix: revert change

* fix: add aliases to filter-builder

* fix: check for undefined

* fix: check aliases for saved orderBys

* chore: add explanation comment

* fix: evaluator custom select keys

* chore: simplify backwards compatible handling of legacy columns

* fix: keep rename isolated to filter layer, not table layer

* fix: trace tag propagation warning

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-30 16:01:22 +02:00
Valery MeleshkinandGitHub 07e1a8715c feat: LANGFUSE_TRACE_DELETE_SKIP_PROJECT_IDS -> LANGFUSE_DELETE_SKIP_PROJECT_IDS now also applies to score deletions (#12891)
feat: LANGFUSE_TRACE_DELETE_SKIP_PROJECT_IDS ->
LANGFUSE_DELETE_SKIP_PROJECT_IDS now also applies to score deletions
2026-03-30 13:57:46 +00:00
NimarandGitHub 08ea5fdaf3 chore(deps): bump many minors of FE deps (#12885)
* chore(deps): bump mcp to 1.28

* chore(deps): bump many minors of FE deps

* fix icons

* bump sensibly

* mcp later

* remove remoxicon , x-tree-view

* fix items
2026-03-30 12:45:22 +00:00
NimarandGitHub c1759cbc1a chore(deps): bump mcp to 1.28 (#12884)
* chore(deps): bump mcp to 1.28

* bump sensibly

* cleanup
2026-03-30 12:02:26 +00:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
6c11c651d2 feat(public-api): apply trace-equivalent rate limits to score deletions (#12886)
* Initial plan

* feat(api): apply trace-style rate limits to score deletion endpoint

Agent-Logs-Url: https://github.com/langfuse/langfuse/sessions/3e413637-8eb6-46a4-8bf8-40cb00b14484

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-03-30 11:44:09 +00:00
46c1d9239e fix(prompts): return correct duplicated prompt (#12861)
* fix: return correct duplicated prompt

* fix: invalidate cache to match createPrompt and duplicateFolder

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-30 11:37:42 +02:00
227648148b chore: bump effect to non-vulnerable version (#12864)
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-30 09:13:15 +00:00
db11cd63d2 chore(experiments): add single experiment paginated list view (#12359)
* chore: seed experiments data

* fixup: VERIFY type change

* feat: build experiments page on events table

* chore: support experiment filter cols

* chore: support table presets for experiments

* chore: adjust feature flag to include v4 check

* feat: support latency and cost columns

* chore: refactor to extract CTEs

* chore: disable sorting

* feat: support scores filters and columns on experiments table

* chore: enhance experiments table with pre-aggregation filters and new metadata fields

* chore: lint

* chore: revert changes to data table controls

* chore: push

* docs: show latency in s

* chore: push

* chore: lint

* chore: lint

* test: trying out commit-stash

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: experiment item queries

* feat: add experiment items repository queries and table mappings

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add tRPC endpoints for experiment items (phase 3)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add ExperimentItemsTable component with sidebar filters (Phase 4 & 6)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fixup: simplify queries

* refactor: update aggregation functions in event query builder for experiments

* refactor: change clickhouse table name from events_core to events_proto in experiment and scores column mappings

* fix: remove automatic experiment filters from event query builder and apply them in query fragments

* refactor: build subquery with query builder

* revert: changes to dev-tables script

* refactor: integrate EventsQueryBuilder for event subquery in getScoresForExperimentItems

* chore: add eventsExperimentTraceIds function for lightweight experiment-to-trace mapping in scores queries

* refactor: update event query handling in scores retrieval and restore experiment score columns in the UI

* chore: remove updated_at

* perf: move to start-time filter for performance

* chore: adjust typing

* chore: adjust typing

* chore: use start_time rather than created_at

* chore: types

* chore: simplify filters

* tests: replace created_at with start_time

* Revert "test: trying out commit-stash"

This reverts commit 2ec5cffbd845127d9eb767ccb2076c15cc9cb2ca.

* refactor: update experiment item properties and improve table structure

* tests: fix latency assumptions given new start_time filter

* chore: rename experiment event fields for consistency

* chore: link to correct dataset item version if possible

* chore: support metadata

* tests: experiment items

* chore: rm items metadata filter due to events table filter issues

* fix: pass project_id

* chore: remove `whereRaw` usages

* chore: move `applyFilters` up to `BaseEventsQueryBuilder`

* chore: export buildExperimentFilterState and integrate it into getScoresForExperimentItems

* chore: split trace/observation level scores for experiments; adjust CTE to use query builder

* chore: properly use events-query builder for experiment queries

* chore: collect trace-level metrics for experimetns

* fix: imports

* chore: lint

* chore: build

* chore: rebase build

* chore: push

* chore: simplify

* chore: aggregation filtering

* chore: fix count

* chore: rm outdated tests

* fix: drop prefix

* chore: refactor overview panel to general component

* fixup: add comparison dropdown and url state management

* chore: format

* fix: after rebase

* chore: explicitly include p_id

* fixup: support experiment compare view and queries

* feat: enhance experiment item count query with HAVING clause support

* fixup: score filters

* chore: support compare view

* feat: clean up grid view

* feat: add cost and latency data

* tests: add test suite

* chore: push

* chore: readjust after rebase

* chore: refactor

* chore: push

* chore: push

* test: item visibility

* chore: comments

* chore: rebase

* fix: join

* feat: add peek-view to experiment item view (#12797)

* feat(experiments): allow clearing baseline  (#12803)

* feat(experiment-compare): clear baseline functionality

* feat(experiment): enhance baseline controls and table components with no results messaging

* chore: address feedback

* chore: adjust cost

* chore: fix

* chore: add v4 compatible seed data for experiments (#12827)

* chore: add v4 compatible seed data for experiments

* chore: clean diff

* chore: revert changes

* chore: remove orderBy from tests

* chore: remove unused param

* chore: gate

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-30 08:47:07 +00:00
marliessophieandGitHub bc7f1c785c chore(evals-ux): Preserve edit mode in URL and add quick 'Edit template' link from evaluator form (#12853)
* refactor(evals): avoid hardcoded edit mode query string

* style: move button

* chore: lint
2026-03-30 08:33:49 +00:00
Marc KlingenandGitHub 1e7c7f9125 chore: Rename Codex Guidelines to Agent Guidelines (#12869) 2026-03-27 18:25:34 +01:00
marliessophieandGitHub 24b22d5c49 chore(experiments): add centralized experiments access logic, beta toggle UI, and integrate into navigation and experiments page (#12859)
* refactor(web): unify experiments access hook and beta switch component

* feat(web): add beta page placeholders on dataset views

* chore: show new UI in dataset-run page routes

* chore: propagate to all pages
2026-03-27 14:55:40 +00:00
Hassieb PakzadandGitHub f75aa9fdee chore: log entity change event payload only in debug (#12860)
* chore: log entity change event payload only in debug

* push
2026-03-27 14:48:09 +00:00
NimarandGitHub 26a5ecc52f fix(lint): workers lint correctly with pnpmv10 (#12862) 2026-03-27 15:59:16 +01:00
NimarandGitHub 2fa84468cd fix(sessions): fix position in trace to default to 1st (#12856)
* fix(sessions): fix position in trace to default to 1st

* add test

* add presets

* root

* refactor position in trace a bit

* test

* new test

* fix type

* type
2026-03-27 14:42:07 +00:00
marliessophieandGitHub 162ef5124b chore(filters): add displayLabel support for categorical facets and sidebar filters (#12828)
* chore(filters): add `displayLabel` support for categorical facets and sidebar filters

* chore: search labels and values
2026-03-27 13:14:12 +00:00
Hassieb PakzadandGitHub bea079756f feat(evals): add boolean scores for LLM-as-a-judge (#12836) 2026-03-27 14:07:40 +01:00
NimarandGitHub dbc390590c chore(dx): upgrade to pnpm v10 (#12844)
* chore(dx): upgrade to pnpm v10

* move overrides up

* use corepack
2026-03-27 09:11:21 +00:00
marliessophieandGitHub 998712e282 fix(web): left-align wrapped sidebar filter labels (#12831) 2026-03-27 08:45:58 +00:00
c066fedcac feat(prompts): add duplicate folder action and tests (#12484)
* feat(prompts): add duplicate folder action and tests

Adds folder-level prompt duplication in prompts UI and tRPC, including nested path handling and single/all-version copy modes. This enables teams to clone prompt hierarchies while preserving webhook trigger behavior for copied prompts

* rewrite prompt references

* add text to clarify behaviour when copying only latest + refrences

* escape

* add test

* text

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-26 21:48:40 +00:00
NimarandGitHub 89a58980aa chore(deps): bump release-it and undici (#12843)
* chore(deps): upgrade worker tests to vitest4

* add

* less

* upgrade less

* fix fixtures

* dont isolate

* chore(deps): bump release-it and undici
2026-03-26 21:06:28 +00:00
NimarandGitHub 96e3c67e52 chore(deps): upgrade worker tests to vitest4 (#12841)
* chore(deps): upgrade worker tests to vitest4

* add

* less

* upgrade less

* fix fixtures

* dont isolate
2026-03-26 20:46:19 +00:00
NimarandGitHub 793fdeddf3 chore(deps): bump sentry to 10.46.0 (#12839) 2026-03-26 17:24:47 +00:00
Tobias WochingerandGitHub 16f0352dec style(web): consistent button cursor behavior (#12824)
* fix(web): use pointer cursor for enabled buttons

* fix(web): mark disabled onboarding buttons as aria-disabled

* fix(web): tighten pointer cursor and keyboard semantics
2026-03-26 17:10:40 +00:00
NimarandGitHub c761d807c0 chore(deps): upgrade to nextjs 16.2.1 (#12835) 2026-03-26 15:56:37 +00:00
NimarandGitHub 38ba952ef9 perf(dashboards): cache in frontend (#12299)
* perf(dashboards): cache in frontend

* clarify

* fix sse re-fetch

* fix
2026-03-26 15:35:36 +00:00
a9cbf0f1e1 feat(ui): support form to write to Pylon (#12378)
* feat(ui): migrate support form

* push

* push

* code clean up

* metadata, first response

* fix

* only write emails with langfuse CH to Pylon

* error toast if sending message fails

* add langfuse plan to issue and account

---------

Co-authored-by: Marc Klingen <2834609+marcklingen@users.noreply.github.com>
2026-03-26 14:55:54 +00:00
Hassieb PakzadandGitHub 23ab911b95 chore: remove datadog mcp (#12833) 2026-03-26 16:00:55 +01:00
10640a0eb2 fix(playground): make ctrl+enter shortcut work on mac (#12826)
* fix(playground): prioritize cmd+enter run-all shortcut

* fix(playground): use cmd+enter only on mac

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-26 14:23:29 +00:00
4595109551 feat: allow LLM-as-a-judge to filter by tool names and tool call count (#12799)
* feat: allow LLM-as-a-judge to filter by tool names and tool call count

* chore: remove mention of events table

* style: simplify

* chore: fix direct instantations of `ObservationForEval`

* chore: address review comments

* chore: review feedback

* chore: add calledToolNames to columnsWithCustomSelect

Allow users to type custom tool names in the eval filter dropdown,
consistent with how tags and name filters work.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 14:14:31 +00:00
a75e6f0a2e chore: bring back click on sidebar to collapse (#12821)
* Revert "style(web): remove resize cursor for non resizable sidebar (#12761)"

This reverts commit 5150609b38.

* style(web): replace resize cursor with pointer on SidebarRail

The rail is a toggle button, not a resize handle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: review comments

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 13:37:23 +00:00
NimarandGitHub bd68c5ee48 fix(annotation-queue): directly display inlined images (#12816) 2026-03-26 14:22:46 +01:00
Marc KlingenandGitHub bb7fe64b4e docs: update telemetry docs in localized READMEs (#12820)
Update localized telemetry docs for OSS wording
2026-03-26 12:27:24 +00:00
Marc KlingenandGitHub f5ae0deb9e docs: improve telemetry section in readme (#12819) 2026-03-26 12:33:40 +01:00
Hassieb PakzadandGitHub f73452b797 chore: improve AGENTS.md (#12818) 2026-03-26 12:01:02 +01:00
Hassieb PakzadandGitHub 1dc9ce65cf chore(agent-setup): centralize shared config and skills under .agents (#12795) 2026-03-26 11:31:21 +01:00
cf4028d17f chore(dx): add vitest config for IDE test discovery (#12800)
ci(worker): add vitest config for IDE test discovery

Add vitest.config.ts and vitest.workspace.ts so the Vitest VS Code
extension can discover worker tests. Load ../.env via dotenv to match
the CLI setup and inline @langfuse/shared for correct module resolution.

Fix vi.mock hoisting errors by wrapping mock variables in vi.hoisted()
in 4 test files where top-level variables were referenced inside
vi.mock factories.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 09:50:14 +00:00
Max DeichmannandGitHub c401fe0e63 fix(widgets): add retry action for failed widgets (#12807)
* fix(widgets): add retry action for failed widgets

* chore(widgets): remove manual error trigger
2026-03-25 22:28:02 +00:00
Max DeichmannandGitHub c561a7da5d fix(widgets): simplify fullscreen loading states and codex bootstrap db generation (#12801)
* Simplify widget loading states and fix Codex setup generation

* Keep legacy widgets spinner-only outside streamed progress

* chore(widgets): remove pr screenshots
2026-03-25 20:59:07 +00:00
Max DeichmannandGitHub 6f663bc449 fix(web): simplify chart error overlay (#12806) 2026-03-25 20:21:50 +00:00
marliessophieandGitHub 48a6221487 chore(dataset-run-items-api): propagate createdAt timestamp for dataset runs (#12804) 2026-03-25 19:42:22 +00:00
NimarandGitHub 7e15aa30ab chore(deps): upgrade to recharts v3.8.0 (#12770)
* chore(deps): upgrade to recharts v3.8.0

* improve

* no tooltip animationes

* never escape

* cleanup

* tiucks and color
2026-03-25 17:40:59 +00:00
02c2932bc7 chore(ci): only add build artifacts to deploy container (#12755)
* fix: CVE vulns in docker images

* chore(ci): only add build artifacts to deploy container

* fix comment

---------

Co-authored-by: Thorsten Spieker <6549175+coffee4tw@users.noreply.github.com>
2026-03-25 17:03:08 +00:00
NimarandGitHub ad996c06f0 fix(trace-table): remove position in trace filter gracefully (#12793) 2026-03-25 16:21:38 +01:00
marliessophieandGitHub 2f235d8315 fix(peek): decode timestamp param properly (#12796) 2026-03-25 15:06:35 +00:00
marliessophieandGitHub 82f6a9cf71 feat(web): add run experiment dialog on experiments page (#12790)
* feat(web): add run experiment dialog on experiments page

* fix(web): refresh experiments table after creating run

* fix(web): show sdk run options on experiments dialog
2026-03-25 13:26:32 +00:00
a09701cc87 perf(prisma): add index on job_executions.job_configuration_id (#12760)
* perf(prisma): add index on job_executions.job_configuration_id

Speeds up cascade deletes when removing an LLM-as-a-judge evaluator
by indexing the foreign-key lookup on jobConfigurationId.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(db): add IF NOT EXISTS to concurrent index migration

Makes the migration idempotent so retries after partial failures
(e.g., index created but Prisma completion record not written) don't
block deployments with "already exists" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:36:40 +01:00
Valery MeleshkinandGitHub fae8ea0925 feat: add dataset_run_id to scores blob storage export (#12792) 2026-03-25 12:48:46 +00:00
marliessophieandGitHub 5ebe890fb8 fix(score-configs-ui): update category value append logic (#12784)
* fix(score-configs-ui): update category value append logic

* chore: push
2026-03-25 12:35:39 +00:00
f89342393e feat(shared): add session_id to scores blob storage export (#12789)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 11:40:42 +00:00
Valery MeleshkinandGitHub bdc6b7853b fix(api): return 400 instead of 500 for invalid filter column names (#12787) 2026-03-25 11:55:15 +01:00
Max DeichmannandGitHub 3e911e9aaf fix: improve widget loading states for small dashboards (#12769)
* fix: improve widget loading states for small dashboards

* Remove Codex artifacts and ignore generated previews

* Add tight chart loading state and indeterminate query progress

* fix(web): remove fake widget form query progress

* chore(web): format loading state components
2026-03-24 19:14:00 +00:00
5150609b38 style(web): remove resize cursor for non resizable sidebar (#12761)
* refactor(web): make SidebarRail a non-interactive div

The sidebar rail showed resize cursors and a hover accent bar despite
not supporting drag-to-resize. Replace the button with a plain div since
the SidebarTrigger in the page header already handles toggling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(web): remove orphaned after:left-full class from SidebarRail

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(web): remove unused SidebarRail component

The rail only existed as a click-to-toggle hit target. Now that it is
no longer interactive, the invisible div serves no purpose.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 17:32:33 +00:00
NimarandGitHub 4532a30ffa chore(agent-dx): add clickhouse skills (#12765) 2026-03-24 17:08:05 +00:00
NimarandGitHub 2ffcc7b17f chore(deps): bump types/lodash to 4.17.24 (#12767) 2026-03-24 16:53:24 +00:00
NimarandGitHub 73bf1f1b78 chore(agent-dx): add turborepo skills (#12766) 2026-03-24 17:32:23 +01:00
NimarandGitHub 91b627cd75 fix(trace-ui): don't truncate root obs on dual write in display (#12764) 2026-03-24 16:22:57 +00:00
f674b38934 feat(dashboards): SSE query progress streaming frontend (#12445)
* feat(dashboards): SSE query progress streaming frontend

* fix(dashboards): prevent stale SSE spinner on date range changes

* fix(dashboards): enable SSE dashboard widgets

* fix(dashboards): gate SSE streaming behind v4 beta

* style(dashboards): format loading state classes

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2026-03-24 15:55:52 +00:00
steffen911 37598017a6 chore: release v3.162.0 2026-03-24 16:27:13 +01:00
a6c38c6ff7 feat: add gzip compression for blob storage exports (#12762)
* feat: add gzip compression option for blob storage integration exports

Add opt-in gzip compression for blob storage integration exports.
When enabled, exported files use .csv.gz/.json.gz/.jsonl.gz extensions
with application/gzip content type. New integrations default to
compressed; existing integrations are backfilled as uncompressed.

Closes LFE-8944

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add compressed: false to existing tests and fix form defaults

Existing worker tests download files as plaintext, so they need
compressed: false since the DB default is now true for new rows.
Also add compressed to the UI form defaultValues and reset call.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:07:59 +01:00
Tobias WochingerandGitHub d4d67431a2 chore: make setting a default model more explanatory (#12758)
* chore: drop default model edit button

Button is redundant as it's duplicating the default behavior of the page

* improvement: make warning for default model more self explanatory

- include link to docs
- explain why default model is needed
2026-03-24 13:38:10 +00:00
marliessophieandGitHub 2eb1819d5d chore(scores): add rawKey option to useScoreColumns for direct score access (#12753) 2026-03-24 13:34:48 +00:00
fb2529c7ad feat(dashboards): add SSE streaming endpoint for ClickHouse query progress (#12429)
* chore: extract shared helpers and decompose executeQuery

- Extract sendClickhouseQuery, setSpanQueryAttributes, recordSummaryOnSpan,
  and ClickhouseQueryOpts type from duplicated inline code in queryClickhouse
  and queryClickhouseStream
- Prevent double ClickHouseResourceError wrapping in queryClickhouseStream

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(dashboards): add SSE streaming endpoint for ClickHouse query progress

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 11:58:56 +00:00
d26b247535 fix(build): fix cve vulnerabilities in web and worker docker images (#12732)
fix: CVE vulns in docker images

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-24 10:21:12 +00:00
Hassieb Pakzad 25da74e436 chore: add claude comment action 2026-03-24 11:43:30 +01:00
Hassieb Pakzad d671be3ab7 chore: add claude comment action 2026-03-24 11:38:26 +01:00
Hassieb PakzadandGitHub 95af771847 chore: add claude comment action (#12754) 2026-03-24 11:21:34 +01:00
Tobias WochingerandGitHub 5475c0d9f2 docs: switch to recommended installation method (#12731) 2026-03-24 10:03:58 +00:00
Valery MeleshkinandGitHub 6f64419615 feat(query): add env var to enable single-level query optimization for v1 (#12752)
feat(query): add env var to enable single-level query optimization for
v1
2026-03-24 09:54:17 +00:00
f8db6403c5 chore: extract shared helpers and decompose executeQuery (#12428)
- Extract sendClickhouseQuery, setSpanQueryAttributes, recordSummaryOnSpan,
  and ClickhouseQueryOpts type from duplicated inline code in queryClickhouse
  and queryClickhouseStream
- Prevent double ClickHouseResourceError wrapping in queryClickhouseStream

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 09:41:30 +00:00
marliessophieandGitHub b628d8f826 feat(experiments): paginated experiments list (#12204)
* chore: seed experiments data

* fixup: VERIFY type change

* feat: build experiments page on events table

* chore: support experiment filter cols

* chore: support table presets for experiments

* chore: adjust feature flag to include v4 check

* feat: support latency and cost columns

* chore: refactor to extract CTEs

* chore: disable sorting

* feat: support scores filters and columns on experiments table

* chore: enhance experiments table with pre-aggregation filters and new metadata fields

* chore: lint

* chore: revert changes to data table controls

* chore: push

* docs: show latency in s

* chore: push

* chore: lint

* chore: lint

* refactor: update aggregation functions in event query builder for experiments

* refactor: change clickhouse table name from events_core to events_proto in experiment and scores column mappings

* fix: remove automatic experiment filters from event query builder and apply them in query fragments

* refactor: build subquery with query builder

* revert: changes to dev-tables script

* refactor: integrate EventsQueryBuilder for event subquery in getScoresForExperimentItems

* chore: add eventsExperimentTraceIds function for lightweight experiment-to-trace mapping in scores queries

* refactor: update event query handling in scores retrieval and restore experiment score columns in the UI

* chore: remove updated_at

* perf: move to start-time filter for performance

* chore: adjust typing

* chore: adjust typing

* tests: replace created_at with start_time

* tests: fix latency assumptions given new start_time filter

* fix: pass project_id

* chore: remove `whereRaw` usages

* chore: move `applyFilters` up to `BaseEventsQueryBuilder`

* chore: export buildExperimentFilterState and integrate it into getScoresForExperimentItems

* chore: split trace/observation level scores for experiments; adjust CTE to use query builder

* chore: properly use events-query builder for experiment queries

* chore: collect trace-level metrics for experimetns

* fix: imports

* chore: lint

* chore: build

* chore: rebase build

* chore: push

* chore: simplify

* chore: aggregation filtering

* chore: fix count

* chore: rm outdated tests

* fix: drop prefix

* chore: format

* fix: after rebase

* chore: explicitly include p_id

* chore(event-query-builder): consolidate JOIN methods for improved readability and maintainability

* chore(queries): update groupBy method typing

* test: add empty
2026-03-24 09:33:47 +00:00
marliessophieandGitHub 534fa3b306 chore(routes): add label "Beta" to evaluation route (#12735) 2026-03-23 17:32:03 +00:00
marliessophieandGitHub 5c6e01ff32 fix(evals): update mapping initialization logic in InnerEvaluatorForm to check for vars before setting mapping (#12734) 2026-03-23 17:10:09 +00:00
Steffen SchmitzandGitHub 3dfa5226a5 chore: remove trailing /index on span names after next upgrade (#12730) 2026-03-23 14:08:07 +00:00
Valery MeleshkinandGitHub 8968960581 chore: remove dead traces CTE join from observations query (#12727) 2026-03-23 13:28:18 +01:00
Hassieb PakzadandGitHub fa9310d7ba chore: upgrade to zod v4 (#12726) 2026-03-23 12:01:11 +01:00
Max DeichmannandGitHub 0c23b6399c chore: Add Playwright MCP setup for frontend agent review (#12719)
Add Playwright MCP setup for frontend agent review
2026-03-22 17:21:19 +00:00
Max DeichmannandGitHub 393dab7164 chore: Replace Claude hook setup with agent instructions (#12470) 2026-03-22 16:56:24 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
c2c39e89d1 chore(deps-dev): bump @types/express-serve-static-core from 5.1.0 to 5.1.1 in the express group (#11910)
chore(deps-dev): bump @types/express-serve-static-core

Bumps the express group with 1 update: [@types/express-serve-static-core](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/express-serve-static-core).


Updates `@types/express-serve-static-core` from 5.1.0 to 5.1.1
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/express-serve-static-core)

---
updated-dependencies:
- dependency-name: "@types/express-serve-static-core"
  dependency-version: 5.1.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: express
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 12:18:16 +00:00
Max DeichmannandGitHub 50a0b54009 feat(web): enable error level filtering in custom dashboard widgets (#12481)
* feat(web): add error level filter option for observation widgets

* fix
2026-03-22 10:31:00 +00:00
Max DeichmannandGitHub 4b4a0a4451 fix(codex): bootstrap env files for new git worktrees (#12717)
* fix: bootstrap worktree env files from the primary checkout

* fix: create Codex env files from examples only
2026-03-22 09:59:51 +00:00
Max Deichmann dc676736c9 chore: release v3.161.0 2026-03-22 10:30:17 +01:00
Steffen SchmitzandGitHub b366d3d05f build: change base image of codespaces (#12696)
* build: change base image of codespaces

* chore: install clickhouse
2026-03-21 20:58:45 +00:00
6711e374cf chore: upgrade release-it to 19.2.4 to resolve undici 6.21.3 (#12714)
chore: upgrade release-it to 19.2.4 to resolve undici 6.21.3 vulnerability

Upgrades release-it from ^19.0.4 to ^19.2.4, which ships with undici@6.23.0
instead of 6.21.3, removing the vulnerable transitive dependency.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 14:56:45 +00:00
4bf0f9510b fix(deps): upgrade @slack/web-api and @google-cloud/storage to resolve Dependabot security alerts (#12713)
fix(deps): upgrade @slack/web-api and @google-cloud/storage to resolve security vulnerabilities

- @slack/web-api ^7.10.0 → ^7.15.0: v7.15.0 requires axios@^1.13.5, fixing HIGH CVE for axios DoS via __proto__ key (Dependabot #163)
- @google-cloud/storage ^7.18.0 → ^7.19.0: v7.19.0 moved to fast-xml-parser@^5.3.4, fixing MEDIUM CVE for entity expansion bypass (Dependabot #225)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 14:34:32 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ec87cdc659 chore(deps): bump undici from 6.21.3 to 7.24.5 (#12712)
Bumps [undici](https://github.com/nodejs/undici) from 6.21.3 to 7.24.5.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.21.3...v7.24.5)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 7.24.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-21 14:14:06 +00:00
57a51f6507 chore(deps): upgrade prettier to 3.8.1, bump typescript-eslint (#12711)
chore(deps): upgrade prettier to 3.8.1, bump typescript-eslint, clean up devDependencies

- Prettier 3.6.2 → 3.8.1 across all packages
- typescript-eslint 8.50.1 → 8.57.1 in packages/config-eslint
- Remove unused @eslint/compat and @eslint/eslintrc from packages/config-eslint
- Remove redundant devDependencies from worker, shared, ee, and web
  (eslint-config-standard, eslint-config-prettier, eslint-plugin-prettier,
  @typescript-eslint/parser, @typescript-eslint/eslint-plugin — all already
  provided transitively via @repo/eslint-config)
- Apply Prettier 3.8 formatting fixes across ~20 files

Note: ESLint 10 upgrade was blocked by eslint-plugin-react incompatibility
(used by eslint-config-next). Will revisit once the React ESLint ecosystem
catches up.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 14:10:48 +00:00
9033020296 chore: bump undici and @modelcontextprotocol/sdk (#12708)
chore: bump undici and @modelcontextprotocol/sdk to fix security vulnerabilities

Bump undici ^7.18.0 → ^7.24.0 and @modelcontextprotocol/sdk 1.26.0 → 1.27.1
to resolve 9 Dependabot alerts (CVEs in undici, express-rate-limit,
@hono/node-server, hono, and flatted).

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:58:30 +00:00
0442d30f79 fix: upgrade vulnerable dependencies (dompurify, fast-xml-parser) (#12706)
* fix: upgrade vulnerable dependencies (dompurify, fast-xml-parser)

- dompurify: 3.2.4 → 3.3.3 (fixes CVE-2025-15599 XSS)
- @google-cloud/storage: 7.18.0 → 7.19.0 (moves to fast-xml-parser ^5.3.4)
- @azure/storage-blob: 12.26.0 → 12.31.0 (moves to fast-xml-parser ^5 via @azure/core-xml 1.5.0)
- @types/nodemailer: 7.0.4 → 7.0.11 (drops @aws-sdk/client-sesv2 dep with old fast-xml-parser)

All fast-xml-parser versions now ≥5.5.6 (fixes CVE-2026-26278)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: revert @azure/storage-blob upgrade to fix Azurite compatibility

@azure/storage-blob 12.31.0 sends x-ms-version 2026-02-06 which is
not yet supported by the Azurite emulator used in CI, causing OTEL
ingestion tests to fail with 500 errors.

Reverting to ^12.26.0 (resolves to 12.26.0 in lockfile). The
fast-xml-parser vulnerability is already resolved since pnpm resolves
@azure/core-xml to 1.5.0 which uses fast-xml-parser ^5.0.7.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 18:19:08 +00:00
22b24b4fdc chore: upgrade @langchain/aws dependencies (#12704)
* chore: upgrade @langchain/aws to ^1.3.3 to resolve fast-xml-parser vulnerability

The previous @langchain/aws@1.2.x pulled in @aws-sdk/client-bedrock-agent-runtime@3.825.0
which depended on fast-xml-parser@4.4.1 (vulnerable). The new version uses @aws-sdk/*@^3.1006.0
which depends on fast-xml-parser v5.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve TS2349 union type error in withStructuredOutput call

After upgrading @aws-sdk packages, the ChatBedrockConverse type's
withStructuredOutput signature diverged enough from the other chat model
types that TypeScript could no longer call it on the union. Cast to
ChatOpenAI (which has a compatible signature) to fix the build.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: upgrade @langchain/core to 1.1.34 for missing exports

@langchain/aws@1.3.3 requires @langchain/core exports for
'./utils/standard_schema' and './language_models/structured_output'
that were not available in @langchain/core@1.1.18.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 18:19:46 +01:00
907b6582cc fix(playground): properly handle non-streaming on RunAll playground windows (#12636)
* fix: properly handle non-streaming on RunAll playground windows

* chore: address PR feedback

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-20 16:20:00 +00:00
NimarandGitHub 86b3078496 fix(trace-ui): directly render tagged images without click (#12705) 2026-03-20 16:09:40 +00:00
335d3d6445 fix(traces): set latency on root to fix timeline (#12697)
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-20 16:58:04 +01:00
252156d439 chore: upgrade @aws-sdk/* dependencies to ^3.1013.0 (#12702)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 15:26:47 +00:00
80dc225cbb refactor: replace Kysely with Prisma for all database queries (#12692)
* refactor: replace Kysely with Prisma for all database queries

Remove Kysely as a dependency and migrate all query builder usage to
Prisma ORM, simplifying the database layer to a single query interface.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused DatasetItem import after Kysely removal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct Prisma model names and snake_case column lookups

- Use prisma.datasetRuns (plural) matching the DatasetRuns model name
- Add snakeToCamel fallback in parseDatabaseRowToString for column IDs
  like expected_output that map to Prisma's camelCase expectedOutput

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add tests for allDatasetsMetrics tRPC procedure

Cover the $queryRaw SQL that replaced the Kysely compile-to-SQL
pattern in the dataset router, verifying correct JOIN, COUNT, and
GROUP BY behavior for datasets with/without runs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): use camelCase field names for Prisma results in filtering tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add versioned dataset item and jsonSelector variable extraction tests

Adds coverage for two previously untested code paths:
1. Versioned dataset items with datasetItemValidFrom - tests exact version match vs latest (validTo=null) fallback
2. jsonSelector via JSONPath for dataset items and traces - tests nested field extraction from JSON columns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: select only the needed column when fetching dataset items for eval

Instead of fetching the entire dataset item row (which can be large due
to input, expectedOutput, and metadata JSON fields), only select the
specific column referenced by the variable mapping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): update evalService.test.ts for outputDefinition rename and remove kyselyPrisma

- Rename 7 remaining `outputSchema` references to `outputDefinition` (field
  renamed in #12540 but tests were incompletely migrated)
- Replace `kyselyPrisma.$kysely` call with `prisma.llmApiKeys.create()`

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 14:56:20 +00:00
hassiebbotandGitHub a7ccf50bb0 feat(evals): support categorical llm-as-a-judge outputs (#12540) 2026-03-20 14:41:05 +01:00
c137127799 feat(billing): auto-create default spend alerts on new subscriptions (#12694)
* feat(billing): auto-create default spend alerts on new subscriptions

When a new paid subscription is created via Stripe webhook, automatically
create default spend alerts to prevent billing surprises. Thresholds are
plan-specific: Core $200, Pro/Team $1,000, Enterprise $2,000. Skips
creation if the org already has alerts (idempotent). Wrapped in try/catch
so failures don't break the main subscription flow.

Ref: LFE-8154

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: patch review

* chore: tests

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:15:31 +00:00
Steffen SchmitzandGitHub 9bd55a690d chore: add clickhouse-js size restriction on insert tests (#12670)
* fix: ignore invalid UInt16 values on prompt_version

* chore: modify tests

* chore: add clickhouse-js size restriction on insert tests

* chore: teardown clickhouse in tests

* chore: switch maybeIt block
2026-03-20 09:30:51 +00:00
NimarandGitHub 0690b07c4d fix(trace-ui): render tagged images inline (#12681)
* fix(trace-ui): render tagged images inline

* add media util
2026-03-20 08:39:34 +00:00
NimarandGitHub 45e7900049 chore(deps): bump turbo to 2.8.20 (#12690) 2026-03-20 08:25:41 +00:00
NimarandGitHub 298daa4adc chore(dx): upgrade to nextjs 16.2 (#12682)
* chore(dx): upgrade to nextjs 16.2

* bump sentry

* bump react types

* add nextjs docs to agent files

* add docs ignore

* fix lint

* moar cache
2026-03-20 07:53:03 +00:00
Valery MeleshkinandGitHub 214b72e8e8 fix(events): qualify search columns to avoid ambiguous identifier in ClickHouse (#12685)
fix(events): qualify search columns to avoid ambiguous identifier in
ClickHouse
2026-03-19 22:17:31 +00:00
Hassieb PakzadandGitHub 22386700c8 fix(model-prices): add input_text key for gemini 3.1 (#12679)
* fix(model-prices): add input_text key for gemini 3.1

* push
2026-03-19 18:10:22 +00:00
8940434fd8 feat(otel): support Genkit spans in OTel pipeline (#12199)
feat: support Genkit spans in OTel pipeline

Co-authored-by: Nimar <l.nimar.b@gmail.com>
Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-03-19 17:07:40 +01:00
Steffen SchmitzandGitHub 686541313a fix: ignore invalid UInt16 values on prompt_version (#12663)
* fix: ignore invalid UInt16 values on prompt_version

* chore: modify tests

* chore: teardown clickhouse in tests

* chore: teardown clickhouse in tests
2026-03-19 14:52:38 +00:00
Steffen SchmitzandGitHub 19a6997f18 perf: drop unused job-execution table indexes (#12662) 2026-03-19 14:12:03 +00:00
NimarandGitHub 3b20b1b0e9 fix(trace-ui): properly render attached images (#12672)
* fix(trace-ui): properly render attached images

* fix seeder

* fix click on media
2026-03-19 13:42:50 +00:00
57235c8108 chore(dx): remove cross-env as dep (#12673)
* build: fix break points during testing

* chore: also fix others

* chore(dx): remove cross-env as dep

---------

Co-authored-by: Tobias Wochinger <tobias.wochinger@deepset.ai>
Co-authored-by: Tobias Wochinger <mail@tobias-wochinger.de>
2026-03-19 10:46:15 +00:00
2477aee3fa fix(ChatMLAdapter): prevent gemini adapter taking precedence over pydantic/agent-framework (#12676)
* [ChatMLAdapter] Add missing test file

* [ChatMLAdapter] Make the obs download script sort-stable

This allow to re-run the download script without changing the previously downloaded traces/obs ordering, reducing commit noise

* [ChatMLAdapter] Improve "update" mode for the chatml integration test

It will fully create the missing chatml expectation file if needed, simplifying adding new traces

* [ChatMLAdapter] Grand-father the buggy pydantic+gemini trace #11307

* [ChatMLAdapter] Make the obs skip download for preexisting files

* [ChatMLAdapter] Grand-father the buggy csharp agent+gemini trace #12550

* [ChatMLAdapter] Fix gemini taking precedence over pydantic/agent-framework

* [ChatMLAdapter] Remove langchain-deepagent trace for the moment

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-19 12:35:57 +01:00
f4c73b30a7 chore(dx): allow vscode debugger break point setting (#12477)
* build: fix break points during testing

* chore: also fix others

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-19 11:23:01 +01:00
NimarandGitHub 2e5b28be25 fix(session): dont truncate IO on v4 (#12671) 2026-03-19 10:31:08 +01:00
a1ab294456 fix(otel): map gen_ai tool definitions into input payload (#12624)
fix(otel): map tool definitions into input payload

Map OTel tool definition attributes into input.tools when gen_ai.input.messages is present so backend tool extraction can persist definitions consistently.

Adds a regression test to prevent future ingestion regressions for gen_ai.tool.definitions mapping.

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-19 09:14:33 +00:00
fa838a7bf3 chore(ChatMLAdapter): Add test cases against seed spans (#12552)
* pnpm format

* [ChatMLAdapter] Add test cases against seed spans

Changing the current adaption logic is brittle as several adapters relies on each others (cf the various "exclusions" cases).
Having stronger E2E test for the adapter would make future changes safer.

* [ChatMLAdapter] Clarify test name

* [ChatMLAdapter] Add test case for koog seed

* [ChatMLAdapter] Create dedicated trace folder for tests

Download example traces from the doc + keeps a few traces from the seed file because they are not public.
Old traces (from 2024 for example) are not kept as they corresponing to the v2 SDK

* [ChatMLAdapter] Add exclusions for non-passing observation to make tests pass

This is the starting point.

* [ChatMLAdapter] Create adaption e2e test

Asserting the actual observations --> chatML conversion result this like a better way to improve the conversion logic without being constrained by the current implementation details

* [ChatMLAdapter] Fixing pydantic tool mapping

Example of how the E2E test allows to more finely understand the impact of chaning the mapping logic

* run formatter

* [ChatMLAdapter] Disable spellchecking for traces/chatml test files

* fix spelling

* remove langchain deep

* rename fixture

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-19 07:54:56 +00:00
19145198e9 fix(billing): hide invalid payment method error for invoice/wire-transfer customers (#12659)
fix(billing): skip payment method check for invoice/wire-transfer customers

For customers paying via invoice/wire transfer (collection_method === "send_invoice"),
the billing page incorrectly showed a "You do not have a valid payment method" error.
This happened because listPaymentMethods() only returns card-type methods, so invoice
customers always had zero results. Now we check the subscription's collection_method
first and only require a payment method for auto-charge subscriptions.

Closes LFE-8872

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 07:18:34 +00:00
Steffen SchmitzandGitHub 892079d8a1 fix: fallback on invalid environments instead of rejecting (#12664) 2026-03-19 07:18:19 +00:00
Steffen SchmitzandGitHub e6b3c3c01b chore: move received metrics query log to debug (#12665) 2026-03-18 20:52:48 +00:00
marliessophieandGitHub b3adfc3fc2 chore(dataset-run-items): POST /dataset-run-items API to accept createdAt param (#12637)
* chore(dataset-run-items): support `createdAt` param in request body

* tests: add
2026-03-18 19:43:37 +00:00
24377648e1 feat(models): Added support for gpt-5.4-mini and gpt-5.4-nano (#12649)
Added support for gpt-5.4-mini and gpt-5.4-nano

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-18 14:06:23 +00:00
Steffen SchmitzandGitHub 387e09f622 fix: increase cloud usage query timeouts and improve tracing (#12655) 2026-03-18 13:07:53 +00:00
a3123356a0 fix(posthog): normalize hostname URL to prevent whitespace in DB (#12652)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:24:13 +00:00
53dc982208 test(api-auth): assert API keys never contain colons (#12653)
* fix(api-auth): parse Basic auth header on first colon only

Replaces split(":") with indexOf + slice so that API secrets
containing colons are preserved rather than silently truncated.
Also adds an early rejection guard when no colon is present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(api-auth): add header parsing tests for colon-in-secret fix

Covers two cases:
- A decoded Basic auth value with no colon is rejected early with
  "Invalid authorization header"
- A decoded value whose password contains colons is parsed correctly
  (failure comes from DB lookup, not from credential extraction)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(api-auth): add green test for standard key parsing

Ensures that the colon-split change does not regress authentication
for normal API keys (no colons in the secret).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(api-auth): revert indexOf split; assert key format never contains colon

Reverts the indexOf-based Basic-Auth split back to the original split(':'),
which already rejects malformed headers via the existing !username || !password
guard.

Replaces the three speculative parsing tests with a single, targeted assertion
on the real generated fixture keys: publicKey and secretKey must never contain
a colon.  Because keys are generated as `pk-lf-<uuid>` / `sk-lf-<uuid>` (UUIDs
are hex + hyphens only), this is already structurally guaranteed — the test
makes that invariant explicit so any future key-format change that would break
Basic-Auth parsing is caught immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(api-auth): generate 30 key pairs and assert none contain colons

Export generateKeySet so the test can call it directly in a loop
without needing a DB round-trip. Generates 30 key pairs and asserts
neither pk nor sk contains a colon.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 10:45:38 +00:00
1c8331ffc0 feat: insert directly into events_full table (#12081)
* feat(clickhouse): add events_green tables for lightweight queries

Add events_green and events_green_input_output tables to split the events
table into a lightweight version (without input/output) for fast queries
and a separate table for full content retrieval. Includes materialized
views to auto-populate from the events table and backfill queries.

See LFE-5394 for ongoing discussion.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: prep new events_full and events_core

* chore: remove backfill queries

* chore: limit backfill events historic to Jan/Dec period

* chore: make compatible with new events layout

* chore: move to use dual event table. WIP commit

* chore: tune settings for initial run

* fix: trace io correct handling. other clenaup

* fix: fix more FROM events occurances

* fix: explicit events_proto in filter column definitions

* fix: more test fixes

* fix: comments and more events references

* chore: some more comment fixes

* fix: update newly added null handling for parentObservationId

* fix: update newly added null handling for parentObservationId

* chore: drop old events table

* chore: adjust boundaries

* chore: typing

* chore: patch tests

* chore: cleanup

* chore: patch schema

* chore: remove unused metadata column

* chore: cleanup

* chore; revert

* chore: tests

* chore: patch tests

* chore: patch tests

* chore: tests

* chore: patch

* chore: patch

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Valery Meleshkin <valeriy@langfuse.com>
2026-03-18 08:54:35 +00:00
Valery MeleshkinandGitHub 2ba9027dfa fix(blob-export): reset lastSyncAt when export mode changes (#12640) 2026-03-17 16:42:29 +00:00
Valery Meleshkin f78b1cdc9a chore: release v3.160.0 2026-03-17 16:57:26 +01:00
013986d5f1 fix(otel): reduce Prisma OTEL span noise via ignoreSpanTypes (#12638)
Filter out intermediate Prisma spans (serialize, engine query, connection,
response serialization) to keep only the top-level client operation and
db_query spans, reducing per-call span count from 5-6 to 1-2.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 15:18:40 +00:00
747f0bf59c fix(events): deduplicate redundant metadata keys consistently (#12630)
* test(events): add tests for duplicate metadata key resolution

Adds tests asserting that when metadata_names contains duplicate keys,
the first value should be used consistently for both reads and filters.
Currently, mapFromArrays (read path) returns the last value while
indexOf (filter path) returns the first — exposing the inconsistency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(events): use first-value-wins for duplicate metadata keys in
mapFromArrays

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Valery Meleshkin <valeriy@langfuse.com>
2026-03-17 14:10:26 +00:00
58f5c45fa9 feat(models): add gemini-live-2.5-flash-native-audio model pricing (#12607)
* feat: add gemini-live-2.5-flash-native-audio model pricing

Add pricing configuration for gemini-live-2.5-flash-native-audio model
(Vertex AI only) with support for text, audio, image, and video tokens.

* refactor: simplify gemini-live-2.5-flash-native-audio pricing keys

Remove redundant pricing keys (input, output, etc.) from the model entry.
Keep only modality-specific keys for clarity.

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-17 14:04:30 +00:00
3d9621a376 fix(otel): fix event metadata scope missing (#12633)
* fix(otel): fix event metadata scope missing

* tests: fix

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2026-03-17 13:29:30 +00:00
marliessophieandGitHub eccb4bcd2a fix(metadata): write attributes to metadata in direct write + S3 /evals write for non-langfuse-SDK-spans (#12631) 2026-03-17 12:57:27 +00:00
0dda88c607 chore(eslint): add next-check to ignore (#12634)
Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-03-17 13:47:07 +01:00
Max DeichmannandGitHub c495f8f170 chore: disable ai feat (#12632)
push
2026-03-17 12:10:53 +00:00
Valery MeleshkinandGitHub aa6594ea6f feat(worker): send email to project admins on blob storage export failure (#12617)
feat(worker): send email to project admins on blob storage export
failure
2026-03-17 11:35:08 +00:00
marliessophieandGitHub 06f41509b0 fix(evals): never apply default view if table controls are hidden (#12625) 2026-03-16 22:33:59 +00:00
Valery MeleshkinandGitHub aefd13944b fix(traces): split IO columns into separate CTE to reduce memory usage (#12619) 2026-03-16 17:16:19 +00:00
e7e1662f86 feat(otel): pydantic gen ai.system instructions mapping (#12442)
* feat(otel): pydantic gen_ai.system_instructions mapping

* pretty

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-16 16:24:45 +00:00
Max DeichmannandGitHub 87d43d63d4 chore: Shard additional worker queues (#12583)
* Shard additional worker queues

* Enforce Conventional Commit titles

* revert: remove package override changes

* test: stabilize sharded eval redis consumer test

* fix(web): clean up admin queue endpoints

* refactor(worker): remove llm judge processor wrapper

* fix(queues): address sharding review feedback

* fix(web): validate replay event types

* fix(worker): handle queue shutdown connection errors

* refactor(web): align replay queue job types

* fix(worker): avoid private bullmq connection access

* fix: align replay event types and restore worker teardown

* Delete web/src/__tests__/server/admin-ingestion-replay.servertest.ts
2026-03-16 14:24:50 +00:00
marliessophieandGitHub 4d971579d9 fix(evals): show alert if variable mapping drifts between template and config (#12616) 2026-03-16 13:43:42 +00:00
Hassieb PakzadandGitHub 8980409118 fix(eval-templates): apply next base path to URL (#12596) 2026-03-16 14:15:37 +01:00
steffen911 e43aedbb1d chore: release v3.159.0 2026-03-16 11:47:14 +01:00
Jannik MaierhöferandGitHub e382b6271c docs: add sdk upgrade note to v4 beta popup (#12611) 2026-03-16 10:12:43 +00:00
5732087d58 fix(api): return 404 instead of 501 for v2 APIs on self-hosted (#12610)
fix(api): return 404 instead of 501 for v2 APIs on self-hosted instances

Self-hosted users alerting on 5xx patterns get false positives from
the beta-only v2/observations and v2/metrics endpoints returning 501.
Switch to LangfuseNotFoundError (404) since these endpoints are not
available outside Langfuse Cloud.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 09:16:42 +00:00
a967d54303 feat(prompts): add "Select all N / Clear" links and search+create input for custom labels (#12496)
* feat(prompts): add 'Select all N / Clear' links and search+create input for custom labels

- Replace individual label toggling with a 'Select all N' text link showing
  the count of unselected custom labels, and a 'Clear' link to deselect all.
  Both links are disabled when the action would be a no-op.
- Move label input to the top of the Custom labels section. The input doubles
  as a live search filter and a create trigger: when the typed value has no
  exact match in existing labels, an inline 'Create a new label: {input}'
  option appears at the bottom of the filtered list.
- Remove the now-unused AddLabelForm component (toggle + separate form).
- The production label section is unchanged to preserve its destructive-action
  confirmation UX.

Closes https://github.com/orgs/langfuse/discussions/12468

* formatting

* fix

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-13 16:26:35 +00:00
Valery MeleshkinandGitHub bc9f5937ac feat(web): show sync status badge and error alert in blob storage settings (#12575)
feat(web): show sync status badge and error alert in blob storage
settings
2026-03-13 17:03:20 +01:00
NimarandGitHub 4f591c9113 fix(invoice-table): fix type (#12594) 2026-03-13 14:40:47 +00:00
NimarandGitHub fa3afb4bd8 fix(playground): dont cut off selected tools/schemas (#12593) 2026-03-13 14:33:47 +00:00
9e9d9488eb feat: replay ingestion events v2 (#12319)
* docs: add replay ingestion events v2 README

Documents the new S3 ingestion event replay flow that replaces direct
Redis/ClickHouse/PostgreSQL access with an admin API endpoint, reducing
on-call requirements to just a CSV, host URL, and admin API key.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: document initial athena setup

* chore: add actual replay script

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 14:05:48 +00:00
5098cec4c0 feat(evals): add docs link for backfilling on observation-level LLM-a… (#12567)
feat(evals): add docs link for backfilling on observation-level LLM-as-a-judge

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 13:14:44 +00:00
NimarandGitHub 6736d121fa chore(deps): upgrade to tailwind v4 (#12590)
* chore(deps): upgrade to tailwind v4

* move to modern tailwind config
2026-03-13 14:34:53 +01:00
NimarandGitHub cb8ce33f29 chore(deps): bump turbo to 2.8.16 (#12586) 2026-03-13 10:34:18 +00:00
NimarandGitHub 102d514fe7 chore(deps): bump use-query-params to 2.2.2 (#12585) 2026-03-13 10:31:17 +00:00
bd0eda471a fix(prompt-management): allow unicode prompt variables (#12173)
fix: allow unicode prompt variables

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-13 10:21:56 +00:00
d16a60c4c9 fix: avoid FINAL on traces table in traces.metrics endpoint (#12546)
Pass orderBy to getTracesTableMetrics so ClickHouse uses LIMIT 1 BY
instead of FINAL for deduplication, matching traces.all behavior.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
2026-03-13 10:36:16 +01:00
Nimar 703c4411e2 chore: release v3.158.0 2026-03-13 09:58:16 +01:00
NimarandGitHub d5aa2a057c feat(playground/prompts): enable fulltext search across message windows (#12578)
* feat(playground): enable fulltext search across message windows

* up libs

* fix dep

* fix lock

* fix dependencies

* refactor search to be usable in prompts too

* fix controller
2026-03-13 08:06:43 +00:00
6a82c24134 feat: add intro dialog for Fast (Preview) toggle (#12543)
* feat: add intro dialog for Fast (Preview) toggle

Show an informational dialog the first time a user enables the Fast
(Preview) v4 beta toggle, explaining performance improvements and
key changes to the UI.

* feat: show intro dialog from promo banner, swap image to jpg

Move intro dialog state into useV4Beta hook so both the sidebar toggle
and the promo banner trigger the dialog on first enable. Replace png
with jpg image.

* chore: compress intro dialog image from 508KB to 72KB

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-12 20:46:30 +00:00
NimarandGitHub 0780d06bcc fix(dashboards): nicer formatting eg thousands separator (#12565)
* fix(dashboards): nicer formatting eg thousands separator

* formatt
2026-03-12 17:37:33 +00:00
Hassieb PakzadandGitHub 11f2d1b564 docs: run fern generate (#12573) 2026-03-12 18:01:50 +01:00
Valery MeleshkinandGitHub e84ca4576a feat: add blob storage integration status endpoint with error tracking (#12570) 2026-03-12 16:09:54 +00:00
eb9e6f644f feat: add unresolved fetches to the v2 prompts API (#12559)
* feat(prompts): support unresolved fetches on v2 api

* refactor(mcp): share prompt read tool logic

* push

* push

* push

* push

* push

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-03-12 16:07:27 +00:00
NimarandGitHub da0f13106f fix(home): rename span to observation latencies and fix filter (#12563) 2026-03-12 15:05:44 +00:00
NimarandGitHub 92997d4f78 fix(otel): fix metadata flattening (#12553)
* fix(otel): fix metadata flattening

* fix worker

* up test

* simplify test

* fix tests

* fix index

* fix test

* fix
2026-03-12 15:04:42 +00:00
Steffen SchmitzandGitHub a0abb26101 chore: update region switch (#12566)
* chore: update region switch

* chore: make jp dedicated region

* chore: make jp dedicated region
2026-03-12 15:47:45 +01:00
Hassieb PakzadandGitHub fbd24fc8f2 fix(ui-evals): update link in callout to SDK upgrade path (#12564) 2026-03-12 15:31:09 +01:00
Steffen SchmitzandGitHub c6dc19336b build: add new deployment region (#12558)
* build: add new deployment region

* chore: add region switch info

* chore: extend email list
2026-03-12 13:48:42 +00:00
5d98c46690 feat(web): allow members to edit llm tools (#12557)
Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-03-12 13:17:31 +00:00
Hassieb PakzadandGitHub 2793b51442 fix(evals-ui): lazy load execution counts (#12556)
* fix(evals-ui): lazy load execution counts

* push
2026-03-12 12:44:31 +00:00
hassiebbotandGitHub fceab185e2 fix(web): lazy load evaluator job execution counts (#12549)
* fix(web): lazy load evaluator job execution counts

* refactor(evals): share evaluator execution count contract
2026-03-12 11:36:16 +01:00
Valery MeleshkinandGitHub cbffc9d953 feat: media and blob batch cleaner for project deletion (#12535) 2026-03-11 17:21:32 +00:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>sumerman
adbeb69e9d fix: normalize leaked orderBy time aliases across table routes and return 400 for invalid order columns (#12533)
* Initial plan

* fix: normalize leaked orderBy time aliases and return 400 for invalid order columns

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

* test: refine orderBy normalization/error assertions after review

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

* test: use shared server orderBy export and strict InvalidRequestError assertion

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>
2026-03-11 16:27:30 +00:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>sumermanValery Meleshkin
f80b212778 fix: prevent BlobStorageIntegrationProcessing requeue deadlocks on failed retries (#12525)
* Initial plan

* fix(worker): prevent blob storage failed-job dedupe deadlock

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

* test(worker): remove mocked blob storage schedule unit test

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>
Co-authored-by: Valery Meleshkin <valeriy@langfuse.com>
2026-03-11 16:27:21 +00:00
Valery MeleshkinandGitHub 5764f98a28 chore: extract shared media deletion utilities from duplicated worker code. (#12530)
projectDelete now marks media as deleted immediately. Making it cheaper on
retries.
2026-03-11 14:34:31 +00:00
Hassieb PakzadandGitHub 00ea2f4176 fix(evals): use org owners if no project admin or owners (#12532) 2026-03-11 15:51:06 +01:00
c6931e08a4 feat(evals): add config blocking (#12452)
* feat(evals): add config blocking

* fix(evals): address review feedback on config blocking

* fix(evals): use prisma block enums and orm helpers

* push

* fix(evals): address follow-up review feedback

* push

* refactor: clean up eval blocking follow-ups

* refactor: remove redundant llm provider fallback

* refactor: rename evaluator blocking terms

* feat: notify admins when evaluators are blocked

* psuh

* fix: avoid duplicate evaluator block notifications

* refactor: centralize evaluator block side effects

* refactor: simplify evaluator block finalization

* push

* push

* push

* push

* push

* push

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2026-03-11 14:57:52 +01:00
hassiebbotandGitHub 77cd1c1373 fix(web): correct evaluator target filtering (#12521) 2026-03-11 13:38:05 +01:00
NimarandGitHub 80258df544 chore(v4): promote faster dashboards on home (#12526)
* chore(v4): promote faster dashboards on home

* up

* up

* track
2026-03-11 12:02:23 +00:00
Max DeichmannandGitHub c409ced370 chore: Track v4 beta flag on PostHog person and super property (#12522) 2026-03-11 11:52:44 +01:00
Valery MeleshkinandGitHub f94be319cf fix(export): batch export fails when categorical score filter is applied (#12511)
Regression from #12376 which changed score_categories to tuple encoding
in batch export CTEs, breaking hasAny filters that expect Array(String).
2026-03-10 16:57:53 +00:00
NimarandGitHub 6c3d2692fa chore(v4): make beta toggle public (#12501) 2026-03-10 17:47:14 +01:00
Valery MeleshkinandGitHub c5b403a7e5 fix: validate filter type compatibility to prevent 500 errors (#12509)
fix: validate filter type compatibility to prevent 500 errors.

Incompatible combinations are rejected with InvalidRequestError.
2026-03-10 16:37:11 +00:00
Hassieb PakzadandGitHub 09c911a66d feat(ui): change v4 beta to preview (#12508) 2026-03-10 17:05:37 +01:00
NimarandGitHub 33af584917 chore(v4): rename frontend toggle to preview (#12503) 2026-03-10 15:59:00 +01:00
NimarandGitHub cb6a28a6da fix(prompts): parse chat prompts references correctly (#12421)
* fix(prompts): parse chat prompts references correctly

* simplify

* fix build

* simp

* fix references

* fix
2026-03-10 13:36:49 +00:00
Steffen SchmitzandGitHub c65ff4db31 chore: track v4BetaEnabled flag on posthog person (#12498)
analytics: track v4BetaEnabled flag on posthog person
2026-03-10 13:10:17 +00:00
NimarandGitHub 6367e80bf7 fix(events-table): can filter for generations related to a prompt (#12500) 2026-03-10 12:24:27 +00:00
Valery MeleshkinandGitHub 017e628c62 fix: improve nullIf handling in v4 queries (#12472) 2026-03-10 11:16:46 +00:00
659246d78f fix(otel): don't include both attributes.metadata and new toplevel metadata in metatadata bloc (#12476)
* dont show both attributes.metadata and new toplevel metadata

* fix: use startsWith and add ai.telemetry.metadata to metadata dedup filter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
2026-03-10 09:47:39 +00:00
Hassieb Pakzad 57f4b97af2 chore: release v3.157.0 2026-03-10 10:41:50 +01:00
NimarandGitHub 1183d09a15 chore(events-table): enable public private setting of events (#12475)
* chore(events-table): enable public private setting of events

* opptimist

* simplifyt

* simplifyt

* simplifyt

* simplifyt

* simplifyt
2026-03-10 08:55:47 +00:00
Max DeichmannandGitHub 6ef56d9c2d chore: Add type filter to default-expanded events table (#12479)
Add auto-expand type filter
2026-03-09 21:18:34 +00:00
Max DeichmannandGitHub 24edc9926e chore: Document Codex setup tooling in AGENTS guides (#12478)
Fix missing Jest client tests
2026-03-09 21:08:57 +00:00
Max DeichmannandGitHub 22ee870d29 chore: document Codex environment setup and add bootstrap scripts (#12471)
Document Codex cloud setup
2026-03-09 21:38:35 +01:00
881d920707 fix(dashboards): load filterOptions based on events table if v4 enabled (#12467)
* fix(dashboards): load filterOptions based on isBetaEnabled

* fix time filter

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-09 19:41:05 +00:00
marliessophieandGitHub 59b65bf550 chore(evals): increase parsing depth for variable extraction (#12474) 2026-03-09 18:16:02 +00:00
NimarandGitHub ba3484533d feat(saved-views): slide from left (#12469)
* feat(saved-views): slide from left

* add preview

* refactor

* fix badges

* cleanup
2026-03-09 18:13:56 +00:00
Valery MeleshkinandGitHub 3fedbb7dbf fix: refine annotation queue nullable vs optional spec (#12464) 2026-03-09 12:40:38 +00:00
NimarandGitHub 6326bcc774 feat(filters): default hide llm as a judge envs (#12465) 2026-03-09 12:33:39 +00:00
Max DeichmannandGitHub c0b44f8c26 chore: allow users to filter from trace detail view (#12461)
* Fix sidebar badge URL encoding

* Update observation options menu

* Add observation filters to dropdown

* Update events filter dropdown
2026-03-09 09:46:00 +00:00
NimarandGitHub 294b9c2282 feat(filters): default hide llm as a judge envs (#12451) 2026-03-07 08:33:00 +00:00
NimarandGitHub b7dfad96f3 chore(events-table): show trace level scores in obs level score table (#12267)
chore(events-table): show trace level scores in obs level score talbe
2026-03-06 18:10:43 +00:00
NimarandGitHub e4bad8986e chore(events-table): drop invalid filter columns frontend when switch… (#12409)
chore(events-table): drop invalid filter columns frontend when switching back from v4
2026-03-06 18:09:24 +00:00
NimarandGitHub 16e6affe82 feat(models): add gpt-5.4 (#12420)
* feat(models): add gpt-5.4

* fix
2026-03-06 17:23:37 +00:00
NimarandGitHub db51d86184 fix(ci): docker build connect (#12437)
* fix(ci): docker build connect

* wget

* ip

* bind host

* cleaneup

* min diff
2026-03-06 17:02:25 +00:00
Valery MeleshkinandGitHub 464ee1224e chore: show logs when docker build fails (#12443)
* chore: show logs when docker build fails

* chore: bump healthcheck timeout
2026-03-06 15:58:07 +00:00
Valery MeleshkinandGitHub e6afc239ef fix(export): double quoting of JSON string in some CSV exports. Reduce memeory usage. (#12441) 2026-03-06 16:49:46 +01:00
Hassieb PakzadandGitHub 2eaf041003 chore(public-api): move legacy endpoints to separate namespace (#12435)
* chore(public-api): move legacy endpoints to separate namespace

* push

* push
2026-03-06 15:44:19 +01:00
Achilleas Athanasiou FragkoulisandGitHub c9e310c5d2 fix(redis): REDIS_KEY_PREFIX handling for BullMQ compatibility (#11898) 2026-03-06 08:51:29 +01:00
NimarandGitHub d1efa37dc1 fix(events-table): rename total cost to cost (#12418) 2026-03-05 20:11:56 +01:00
Hassieb Pakzad 44591ce857 chore: release v3.156.0 2026-03-05 18:13:27 +01:00
NimarandGitHub dc53f0b217 fix(events-table): show trace scores on top level node in tree (#12412) 2026-03-05 15:52:16 +00:00
Max DeichmannandGitHub 8fd1da4f22 fix: Add client test for sidebar notification badge URL (#12411)
Fix sidebar badge URL encoding
2026-03-05 14:57:13 +00:00
4a710b4122 fix(llm-models): update pattern for newer Claude Sonnet 4.6 model plus global (#12367)
* fix: update match patterns for newer Claude models

* fix: update match pattern for claude-opus-4-6 to include versioning

* fix line end

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-05 14:26:24 +00:00
NimarandGitHub e8054f28be chore(events-table): show filters + columns for trace level scores (#12385) 2026-03-05 15:06:52 +01:00
NimarandGitHub daccb54303 chore: fix turbo and bump to 2.8.13 (#12384) 2026-03-05 14:46:53 +01:00
180c1fef81 fix(ui): preserve query params in ResizableImage custom loader (#12386)
The customLoader for Next.js Image always prepends a `?` when appending
width and quality parameters. For S3/MinIO presigned URLs that already
contain query parameters, this creates a malformed URL with two `?`
characters, causing signature verification to fail and images to not
render inline.

Use `&` as separator when the URL already contains query parameters.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-05 11:00:10 +00:00
217540c899 feat(clickhouse): add analytics_events_core view (LFE-8734) (#12399)
feat(clickhouse): add analytics_events_core view for project-level analytics (LFE-8734)

Adds a ClickHouse VIEW on events_core with per-project, per-hour aggregations
including type/source/scope/SDK counts via sumMap, unique counts via uniqIf
and uniqArray, and has_* boolean flags for feature detection.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:22:56 +00:00
marliessophieandGitHub 3f6a8fc5ee fix(evals): parsing of IO for observation-level evals with json-path (#12394)
* fix(evals): parsing of IO for observation-level evals with json-path

* fix(evaluation): handle JSON parsing errors in observation variable extraction

* chore: skip json parse on primitives

* chore: add debug statement
2026-03-05 09:58:34 +00:00
Jannik MaierhöferandGitHub f3b4d22ee1 feat(costs): add gemini input_text price (#12375)
* feat(costs): add gemini input_text price

* update updatedAt

* add info to skill
2026-03-05 09:56:24 +00:00
Hassieb PakzadandGitHub 3227faaf55 fix(ingestion): python beta OTEL spans should go direct event write path (#12395) 2026-03-05 10:47:05 +01:00
Valery MeleshkinandGitHub bd0e26f410 fix(storage): add buffered stream uploader with per-part retry for resilient S3 uploads (#12360)
* fix(storage): add buffered stream uploader with per-part retry for resilient S3 uploads

* fix(storage): add concurrent part uploads to buffered stream uploader

* chore: put new codepath behind an env var

* chore: make first error handling foolproof

* chore: addressing PR feedback
2026-03-04 19:36:23 +00:00
NimarandGitHub 84f0eef583 feat(filters): default hide langfuse environments (#12346) 2026-03-04 20:07:38 +01:00
d53d376f7a feat(model-prices): add gemini-3.1-flash-lite-preview pricing (#12369)
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-03-04 19:01:42 +00:00
Hassieb PakzadandGitHub e26a288dec fix(model-prices): parse token creation counts by duration (#12382) 2026-03-04 18:00:08 +00:00
Valery MeleshkinandGitHub 87997f6d96 fix(dashboard): prevent very hight cardinality dimensions from being used in v2 observations widgets (#12377)
fix(dashboard): prevent very hight cardinality dimensions from being
used in v2 observations widgets
2026-03-04 14:50:06 +00:00
Hassieb Pakzad e5bd4bfbcd chore(prompts): change log wording 2026-03-04 14:51:17 +01:00
Hassieb PakzadandGitHub 5d2427a4db fix(prompt-management): cache concurrency safety (#12363) 2026-03-04 14:24:10 +01:00
Valery MeleshkinandGitHub d88446fbd0 fix(export): categorical scores with colons in name exported as null (#12376)
Batch export streams encoded categorical scores as concat(name, ':', string_value)
in ClickHouse and decoded with split(":") in TypeScript. When a score name contains
colons (e.g. "Name: Subname"), the split incorrectly parses the name/value
pair, causing the value to be dropped and exported as null.
2026-03-04 11:53:51 +00:00
120ea1d452 feat(analytics): track PostHog event for v4 Beta sidebar toggle (#12362)
Capture sidebar:v4_beta_toggled event with { enabled } property when users click the v4 Beta toggle, to understand adoption patterns.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 09:27:52 +00:00
Valery MeleshkinandGitHub 70122f242a feat(dashboard): validate high-cardinality dimensions require top-N shape on v2 path (#12345)
feat(query): validate high-cardinality dimensions require top-N shape on v2 path
2026-03-03 18:16:56 +00:00
af875bae8c fix(mixpanel): sanitize bad distinct_id values and handle partial import errors (#12358)
Mixpanel's /import?strict=1 API rejects events whose distinct_id matches
a blocklist of "bad IDs" (e.g. "undefined", "null", "0"). This caused
400 errors that threw even when 999/1000 records imported successfully.

- Add MIXPANEL_BAD_DISTINCT_IDS blocklist and isBadDistinctId helper to
  transformers; fall back to $insert_id for blocked values
- Parse 400 response JSON in sendBatch; log warning on partial success
  instead of throwing
- Add tests covering bad distinct_id fallback for all four transformers

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 13:07:35 +00:00
marliessophieandGitHub fc75ea35f2 chore(data-table): format total count and pages in select all banner (#12353) 2026-03-03 09:49:51 +00:00
marliessophieandGitHub f42db498dd fix(api): ensure proper error handling for silent HTTP codes in TRPCClientError (#12352) 2026-03-03 09:47:11 +00:00
marliessophieandGitHub 14b7ca32d4 chore(remote-experiment): increase timeout to 20s and improve error handling (#12351)
* chore(remote-experiment): increase timeout to 30 sec

* chore(remote-experiment): reduce timeout to 20 sec and improve error handling
2026-03-03 09:19:49 +00:00
d6da3111a2 fix(models): prevent focus loss when typing unit name in price editor (#12344)
Use array index as React key instead of the unit name so React doesn't
unmount/remount the input on every keystroke. Fixes LFE-8629.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 19:08:08 +01:00
NimarandGitHub 2be7672022 fix(sessions): show data in json beta viewer too (#12343) 2026-03-02 16:32:19 +00:00
Hassieb PakzadandGitHub 80c90950cf fix(eval): extract AI SDK names equivalent to legacy pipeline (#12342) 2026-03-02 15:24:37 +00:00
724ff495fa perf(dashboards): skip rootEventCondition subquery for wide time windows (#12318)
* perf(dashboards): skip rootEventCondition subquery for wide time windows

For large time windows (>7 days by default), the rootEventCondition
subquery has diminishing returns and causes significant performance
overhead. This makes the filter conditional on the query time window
size, controlled by LANGFUSE_ROOT_EVENT_CONDITION_MIN_HOURS env var
(default: 168h / 7 days). Set to 0 to always apply the filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add test case

* chore: adjust test case

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:52:31 +00:00
NimarandGitHub bf12501418 fix(score-analytics): correct mapping of boolean values (#12339)
* fix(score-analytics): correct mapping of boolean values

* fix bool mapping

* fix test
2026-03-02 13:20:34 +00:00
aa2f7568a4 perf(query): switch to INNER JOIN and add useFinal flag to tableRelations (#12340)
LEFT JOIN was unnecessary since joined relations always have timestamp
filters in the global WHERE clause that reject NULLs. INNER JOIN lets
ClickHouse optimize join strategy from the start. Also adds a per-relation
useFinal flag (defaults to true) so already-deduplicated tables like
events_core can skip the FINAL modifier.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 13:17:41 +00:00
6df18fec4c fix(auth): support Authentik authorization URL override (#12300)
* support authentik authorization url

* add comment

* require issuer

* prettier

---------

Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
2026-03-02 13:17:37 +00:00
NimarandGitHub 1405200de8 fix(events-table): read from events_core again for position in trace (#12329)
* fix(events-table): read from events_core again for position in trace

* Update events.ts
2026-02-28 11:17:17 +00:00
Nimar 258a41a1fe revert(events-table): remove level in trace filter 2026-02-28 09:31:35 +01:00
Max DeichmannandGitHub 03b736ef4a chore: add delay (#12321)
* fix

* push

* fix(auth): equalize bcrypt work on invalid credential paths

* chore: update eval execution settings
2026-02-27 18:02:49 +00:00
marliessophieandGitHub 667c3bf475 docs(evals): position observation-level evals as recommended approach (#12293)
* docs(evals): position observation-level evals as recommended approach

* chore: push

* chore: lint

* fix: targeting observations

* chore: lint
2026-02-27 16:17:54 +00:00
marliessophieandGitHub c46834e5b8 fix(evals): ensure inline filter state remounts when target changes (#12255) 2026-02-27 16:11:47 +00:00
Valery MeleshkinandGitHub 5968eb8a72 fix(dashboard): adding uniqueUserIds/uniqueSessionIds to the V4 dashboards (#12317)
fix(dashboard): adding uniqueUserIds/uniqueSessionIds to the V4
dashboards
2026-02-27 14:35:28 +00:00
Jannik MaierhöferandGitHub 1d0fafd713 feat(ui): edit wording on billing page (#12314) 2026-02-27 13:01:44 +00:00
Valery MeleshkinandGitHub f156ecc726 perf: add bloom filter index on provided_model_name and cache settings to events tables (#12313) 2026-02-27 14:06:21 +01:00
Valery MeleshkinandGitHub 91d7a79ea0 fix(dashboard): add filterSql for correct Trace Name filtering on events_traces view (#12298)
* fix(dashboard): add filterSql for correct Trace Name filtering on events_traces view

The events_traces view reconstructs trace names via aggregation
(argMaxIf), but filters on "Trace Name" were hitting the endsWith("Name")
fallback and generating `events_core.name IN (..)` — matching observation
names instead of trace names.

Introduces filterSql on view dimensions to support two-phase filtering:
- WHERE pruning: OR'd filters across raw columns for pre-aggregation row reduction
- HAVING: exact match on the aggregated expression after GROUP BY

* chore: switch to theoretically slightly less correct having-less approach. we don't expect traceName to diverge across trace

* chore: cleanup
2026-02-27 14:05:54 +01:00
Hassieb PakzadandGitHub 780da40d92 chore: add release:cloud script (#12312) 2026-02-27 13:09:33 +01:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>sumermanValery Meleshkin
caad13496d fix: default view in new widget form when v4 beta is enabled (#12297)
* Initial plan

* fix: default view in new widget form when v4 beta is enabled

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

* fix: improve useEffect dependency comment per review feedback

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

* fix: address review feedback on widget default view useEffect and return type

Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sumerman <222471+sumerman@users.noreply.github.com>
Co-authored-by: Valery Meleshkin <valeriy@langfuse.com>
2026-02-27 11:32:01 +00:00
Valery MeleshkinandGitHub 359fdc2784 fix: make traces view slightly more useful on projects with rootless traces (#12307) 2026-02-27 10:59:43 +00:00
Thorsten SpiekerandGitHub 08e54f336f chore: add http.response.status_code to data dog span (#12091) 2026-02-27 12:00:01 +01:00
a93f65a148 feat(api): type ObservationsV2Response data field instead of map<string, unknown> (#12287)
Define ObservationV2 type in Fern with core fields required and
field-group fields optional, so SDK users get autocomplete and type
safety instead of Record<string, unknown>.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 21:10:38 +00:00
NimarandGitHub 81ef8a65a2 fix(dashboards): resizable turbopack (#12294) 2026-02-26 20:26:03 +00:00
NimarandGitHub a67f460994 perf(dashboards): use query scheduler to reduce concurrency (#12291)
* perf(dashboards): use query scheduler to reduce concurrency

* rename env filter hash key

* cleanup

* fix lint
2026-02-26 20:10:29 +00:00
Max DeichmannandGitHub 69a017a839 fix: block outgoing (#12296)
fix(webhooks): block AWS metadata IPv6 endpoint
2026-02-26 19:08:48 +00:00
Valery MeleshkinandGitHub 7b86d14447 fix(dashboard): add fallback when is empty in v2 queries (#12285)
* fix(dashboard): add  fallback when  is empty in v2 queries

* chore: fix seeder
2026-02-26 15:13:39 +00:00
NimarandGitHub 18bcf18433 perf(env-filter): make envs cached again (#12286)
* fix(env-filter): make envs cached again

* simplify

* sim
2026-02-26 14:57:46 +00:00
NimarandGitHub 031421f960 chore(events-table): show scores in table (#12264)
chore(events-table): show scores
2026-02-26 14:23:42 +00:00
Hassieb PakzadandGitHub 90974e772c feat(llm-connections): allow extraHeaders for anthropic adapter (#12284) 2026-02-26 14:13:46 +00:00
Hassieb PakzadandGitHub c0e915f22c fix(ui-v4-banner): adjust padding (#12283) 2026-02-26 14:43:05 +01:00
f84ba94f7f feat(sso): support tokenEndpointAuthMethod in multi-tenant SSO configs (#12270)
Allow overriding the OAuth token endpoint auth method (e.g.
client_secret_post) per SSO config stored in the database, matching the
capability already available for static env-var-based providers.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:32:21 +00:00
Steffen SchmitzandGitHub 6431f1e2e1 chore: upgrade fernapi version (#12256) 2026-02-26 12:31:47 +00:00
9dbd137ea4 fix(docker): add named volume for Redis (#12258)
fix(docker): add named volume for Redis to prevent anonymous volume clutter

The redis:7 image declares VOLUME /data in its Dockerfile, causing Docker
to create anonymous volumes when no explicit mapping is provided. This adds
a named volume consistent with how Postgres, ClickHouse, and MinIO are
already configured.

Closes #12187

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:31:40 +00:00
8aa21b7e84 fix(auth): gate project membership creation behind rbac-project-roles entitlement (#12262)
`createProjectMembershipsOnSignup` was unconditionally creating
`ProjectMembership` records for all plans on signup. Since
`resolveProjectRole` uses explicit project memberships over the org
role, this caused the init user (org-level OWNER) to be downgraded
to VIEWER at the project level — blocking API key management and
other owner actions.

Two fixes:

1. `createProjectMembershipsOnSignup`: Skip creating project
   memberships when `rbac-project-roles` entitlement is absent.
   Without it, users inherit their org role for all projects.

2. `initialize.ts`: For EE plans where project memberships ARE
   created, correct the init user's project membership to OWNER
   after the org membership is established (fixing the timing
   issue where `createUserEmailPassword` runs
   `createProjectMembershipsOnSignup` before the org role is set).

Closes #11871

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:31:33 +00:00
2a8b063832 feat: parse livekit attributes on otel spans (#10771)
* feat: parse mlflow attributes on otel spans

* add observation types

* ingestion

* trace tree - hide DEBUG but not children

* format

* fix

* add on_enter and start_agent_activity as debug spans

* consolidate tests

* dont map if error

* only map livekit traces

* speed up

* consolidate tests

* fix issue of reverse ordering when root span is hidden

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-26 09:20:36 +00:00
Hassieb PakzadandGitHub 25b51393d3 fix(api-scores): return executionTraceId (#12254) 2026-02-26 09:58:38 +01:00
Valery MeleshkinandGitHub ba94dd7235 fix(evals): add missing observation columns to checkTraceExistsAndGetTimestamp CTE (#12269)
The observations_agg CTE only included level-related columns, causing
ClickHouse errors when eval automations filtered on latency, cost, or
token columns (e.g. "Identifier 'o.latency_milliseconds' cannot be
resolved"). Add latency_milliseconds, usage_details, and cost_details
aggregations to the CTE.
2026-02-26 08:41:42 +00:00
2a2ddf08d5 feat(dashboard): add defenition version tracking to dashboard widgets (#12239)
* feat(dashboard): add defenition version tracking to dashboard widgets

* feat(dashboard): auto-detect min_version for widget v2 requirements

* feat(dashboard): validate measure-aggregation compatibility for widget
definitions

* fix naming

* fix state

---------

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-25 22:03:49 +00:00
711a1ae5d8 feat: include triggering user info in webhook and GitHub dispatch payloads (#12074)
* feat(webhooks): add optional user field to webhook and entity change schemas

Add user info (id, name, email) as optional field to
PromptWebhookOutboundSchema, WebhookOutboundEnvelopeSchema, and
EntityChangeEventSchema so triggering user context flows through the
event pipeline to outbound webhook/GitHub dispatch payloads.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(webhooks): thread triggering user info from tRPC call sites through event sourcing

Pass ctx.session.user (id, name, email) from all prompt mutation
call sites (create, duplicate, delete, deleteVersion, setLabels,
setTags) into promptChangeEventSourcing, which forwards it into the
entity change queue payload.

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(webhooks): include user info in outbound webhook and GitHub dispatch payloads

Thread user from entity change event through prompt version processor
to webhook queue, then include in final HTTP payload for both webhook
and GitHub dispatch actions. User field is optional and omitted when
not available (e.g. API-key-triggered changes).

Co-Authored-By: Claude <noreply@anthropic.com>

* test: add user info tests for webhook and GitHub dispatch payloads

Adds three tests verifying user info is correctly included in webhook
payloads when provided, omitted when absent, and included in GitHub
dispatch payloads.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(webhooks): remove user id from outbound payloads

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2026-02-25 20:43:04 +00:00
Hassieb PakzadandGitHub 61c2def792 feat(ui-v4): add top banner (#12266) 2026-02-25 18:26:11 +01:00
Max DeichmannandGitHub 9f896dd31f fix(docker): pin turbo version in build images (#12265) 2026-02-25 17:11:27 +00:00
NimarandGitHub ce3fd8e34c chore(dx): update agents.md file (#12243) 2026-02-25 16:06:07 +01:00
Max DeichmannandGitHub afc527933a chore: introduce secondary eval execution queue (#12252) 2026-02-25 14:50:28 +01:00
d3481ddab7 feat: add ClickHouse Cloud auth provider (#12115)
* feat: add "Sign in with ClickHouse Cloud" auth provider (Cloud only)

Adds a dedicated clickhouse-cloud auth provider using Auth0Provider under
the hood with a custom provider ID, giving it its own callback URL
(/api/auth/callback/clickhouse-cloud). Gated behind NEXT_PUBLIC_LANGFUSE_CLOUD_REGION
so it only appears on Langfuse Cloud.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: lint

* chore: use clickhouse icon

* chore: overwrite audience

* chore: change audience to langfuse

* chore: skip audience

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 13:10:11 +00:00
9308c05573 feat(query): enable ClickHouse query condition cache for analytics queries (#12251)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-25 12:37:07 +00:00
b8b9e10121 fix(filters): decode empty arrayOptions value as empty array in URL roundtrip (#12229)
fix(web): decode empty arrayOptions value as empty array in URL round-trip

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-24 20:52:26 +00:00
9db7f55e07 feat(trace-table): show full trace data on hover at small row height (#12237)
feat: show full trace data even when row height is small

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-24 18:33:08 +00:00
NimarandGitHub c55ab46d2b chore: upgrade react-resizable-panels to v4 (#12238)
* chore: upgrade react-resizable-panels

* fix sticky layouts

* update
2026-02-24 18:24:45 +00:00
Hassieb PakzadandGitHub 784ab09e43 chore: upgrade fern for SDK majors (#11267) 2026-02-24 17:12:06 +01:00
NimarandGitHub 34c7a8a005 feat(playground/evals): handle thinking parts (#12233)
* test1

* update

* up

* test

* simplifty

* Update llmConnections.test.ts

* reasoning with tool calls

* simplify

* docs
2026-02-24 15:44:04 +00:00
marliessophieandGitHub 55b1c32cd0 chore(dialog-ui): prevent dialog propagation on enter space; add breadcrumb to dashboards detail page (#12236)
* fix(ui): prevent Enter/Space key events from propagating in dialog component

* fix(ui): add stopPropagationOnEnterSpace prop to dialog component

* chore(dashboard): add breadcrumb navigation to dashboard detail page
2026-02-24 14:00:21 +00:00
Valery MeleshkinandGitHub f3d2b133ba fix(dashboard): pass metricsVersion to cost/usage-by-type chart queries (#12235)
The queryCostByType and queryUsageByType calls in ModelUsageChart were
not passing the version prop, so they always defaulted to v1 and hit
the legacy `observations FINAL` path instead of the faster v2
`events_core` path.
2026-02-24 12:57:13 +00:00
Hassieb PakzadandGitHub eca346c7f2 chore(evals): add adapter to eval.call-llm span (#12234)
* chore(evals): add adapter to eval.call-llm span

* push
2026-02-24 12:03:10 +00:00
Hassieb PakzadandGitHub a42e6f9e46 chore(evals): add adapter to eval.call-llm span (#12232)
* chore(evals): add adapter to eval.call-llm span

* push
2026-02-24 10:40:58 +00:00
Steffen SchmitzandGitHub 4463615770 chore: parallelize dual write inserts (#12225) 2026-02-23 21:14:05 +00:00
Max DeichmannandGitHub 2548665e0a fix(web): prevent chain (#12222)
* fix(web): prevent prototype-chain RBAC bypass

* refactor(web): simplify RBAC own-property guards

* refactor(web): centralize safe RBAC role guard

* chore: increase admin dedupe window
2026-02-23 20:43:34 +00:00
53cb3349ac feat(ui): map i/o to pydantic root span (#12068)
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-23 20:14:28 +00:00
Steffen SchmitzandGitHub 347b21e8e3 perf: reduce scan size for event prop by converting OR to GREATEST (#12221) 2026-02-23 19:13:15 +01:00
Hassieb PakzadandGitHub 95d495c433 fix(model-prices): claude version identifier to optional (#12219) 2026-02-23 18:39:22 +01:00
Marlies Mayerhofer 1cdd28d393 chore: release v3.155.1 2026-02-23 18:08:27 +01:00
Max DeichmannandGitHub bbd2091944 chore: increase admin dedupe window (#12218) 2026-02-23 18:00:30 +01:00
Steffen SchmitzandGitHub 1bf610ec82 chore: separate flag behaviour for event table propagation (#12216) 2026-02-23 17:56:26 +01:00
Valery MeleshkinandGitHub eef7bb0a81 fix(prisma): make pending_deletions index migration schema-agnostic (#12209)
Remove hardcoded "public" schema prefix and add IF EXISTS/IF NOT EXISTS
guards so the migration works with custom Postgres schemas. Add cleanup.sql
entry to force re-application on existing deployments with stale checksum.

Fixes #11946
2026-02-23 16:07:22 +00:00
Hassieb PakzadandGitHub 84e00d3451 fix(llm-connection-google): allow passing thinking config for google adapters via provider options (#12211) 2026-02-23 17:28:43 +01:00
marliessophieandGitHub dc355e4c5d fix(evals): correctly destructure v4 beta hook (#12214) 2026-02-23 17:03:21 +01:00
a5d52864d3 chore: send webhooks for admin actions (#12207)
* chore: send webhooks for admin access

* test: improve admin access webhook test robustness and coverage

Move env restoration and fake timer cleanup into afterEach for proper
test isolation. Add tests for dedupe with different keys, fetch
rejection, and non-ok response handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fixes

* fix: fixes

* fix: fixes

* fix: fixes

* fix: enable e2e tests again

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 14:42:09 +00:00
4e6dc3d0e5 feat(evals): show evaluation prompt on hover (#12208)
* feat: show evaluation prompt on hover

* Remove unused 'Info' import from evaluator-selector

---------

Co-authored-by: Max Deichmann <maxdeichmann@icloud.com>
Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-23 14:35:41 +00:00
Max DeichmannandGitHub 5a6390a2ce fix: enable e2e tests again (#12212) 2026-02-23 14:17:49 +00:00
Max DeichmannandGitHub b005f4110b fix: add default timestamps for otel events (#12200)
* fix: add default timestamps for otel events

* fix: fixes
2026-02-23 13:33:33 +00:00
Hassieb Pakzad 6c2c243f7f Revert "feat(otel): support mapping of custom trace_id for litellm (#11553)"
This reverts commit f79a5cc52f.
2026-02-23 13:44:01 +01:00
0bb69f3cab fix(filter-sidebar): use positive matching for arrayOptions checkbox filter (#12206)
fix(web): use positive matching for arrayOptions checkbox filter

Co-authored-by: Nimar <l.nimar.b@gmail.com>
2026-02-23 12:03:27 +00:00
df9f1953a0 chore(events-table): query builder for scores (#12137)
* chore(events-table): use aggregate filter builder for scores

* fix build

* fix

* add clarification comments

* performance with having clause

* refactor(scores): replace eventsTracesAggregation with flat events query for v4 scores

Replace the heavy GROUP BY aggregation builder (eventsTracesAggregation) with a lightweight flat EventsQueryBuilder (eventsTraceMetadata) that selects one row per trace via LIMIT 1 BY.

* clean up

* add warning

* only show trace cols as filter where relevant

---------

Co-authored-by: Valery Meleshkin <valeriy@langfuse.com>
2026-02-23 11:39:08 +00:00
marliessophieandGitHub a27f8dce0e feat(experiments): add experiments pages with routing and admin flag checks (#12064)
* feat(experiments): add experiments pages with routing and admin flag checks

* chore: only allow experiments fir cloud admins

* chore: lint

* chore(experiments): remove unused projectId variable from experiments and experiment detail pages
2026-02-23 09:29:10 +00:00
Max DeichmannandGitHub c7e32af3a7 chore: another attempt to fix e2e tests (#12195) 2026-02-22 16:36:23 +01:00
Max DeichmannandGitHub 7af02d8274 chore: remove database pruning from worker tests (#12196)
* chore: remove database pruning from worker tests

* chore: remove database pruning from worker tests

* chore: fix
2026-02-22 15:09:56 +00:00
Max DeichmannandGitHub 77dce36e40 refactor(web): unify server test layout and CI (#12182)
* refactor(web): unify server test layout and CI

* refactor(web): remove pruneDatabase CI check and dead helper

* test(web): stabilize queryBuilder and model definitions assertions

* chore: move tests to async

* chore: move tests to async

* chore: move tests to async

* chore: move tests to async

* test(web): make model definitions assertions pagination-safe

* test(web): gate media e2e checks for azure blob mode

* test(web): fix azure media test gating condition

* test(web): make api-auth redis hooks cluster-compatible

* test(web): avoid redis quit errors in cluster hooks

* fix(web): use injected redis client for api key invalidation

* test(web): harden api-auth redis cluster test client lifecycle

* chore: move tests to async

* chore: move tests to async

* chore: move tests to async
2026-02-22 14:24:54 +00:00
1550 changed files with 101455 additions and 36162 deletions
+178
View File
@@ -0,0 +1,178 @@
# Agent Guidelines for Langfuse
This is the canonical root agent guide for the repo. The root `AGENTS.md`
should remain only as a discovery symlink so tools that require that filename
continue to work while `.agents/` stays the source of truth.
Langfuse is an open source LLM engineering platform for developing, monitoring,
evaluating, and debugging AI applications.
## Maintenance Contract
- `AGENTS.md` is a living document.
- Keep this file concise and router-like. Push narrow or conditional workflows
into package-local `AGENTS.md` files or shared skills under `skills/`.
- Update this file in the same PR when monorepo-level architecture, workflows,
dependency boundaries, mandatory verification commands, or release/security
processes materially change.
- Update this file and the relevant shared skills when user feedback introduces
a durable repo-level default for future agents. Do not edit this file for
one-off task preferences.
- For package-local material changes, update the nearest package `AGENTS.md` in
the same PR.
## Start Here By Task
- Repo-wide agent setup, `.agents/**`, provider shims, or MCP/bootstrap config:
[`README.md`](README.md),
[`skills/agent-setup-maintenance/SKILL.md`](skills/agent-setup-maintenance/SKILL.md)
- Backend/API work in `web/src/server/**`, `web/src/pages/api/public/**`,
`worker/src/**`, or `packages/shared/src/**`:
[`skills/backend-dev-guidelines/SKILL.md`](skills/backend-dev-guidelines/SKILL.md)
- Model pricing work in `worker/src/constants/default-model-prices.json`,
`packages/shared/src/server/llm/types.ts`, or related pricing files:
[`skills/add-model-price/SKILL.md`](skills/add-model-price/SKILL.md)
- Code review tasks:
[`skills/code-review/SKILL.md`](skills/code-review/SKILL.md)
- Changelog drafting for completed feature branches:
[`skills/changelog-writing/SKILL.md`](skills/changelog-writing/SKILL.md)
- ClickHouse schema/query review:
[`skills/clickhouse-best-practices/SKILL.md`](skills/clickhouse-best-practices/SKILL.md)
- Monorepo/Turbo task graph changes:
[`skills/turborepo/SKILL.md`](skills/turborepo/SKILL.md)
- pnpm dependency upgrades, package-version bumps, or `minimumReleaseAgeExclude`
decisions in `pnpm-workspace.yaml`:
[`skills/pnpm-upgrade-package/SKILL.md`](skills/pnpm-upgrade-package/SKILL.md)
- User-visible frontend changes, Playwright review, or browser signoff:
[`skills/frontend-browser-review/SKILL.md`](skills/frontend-browser-review/SKILL.md)
- Web UI and frontend entry points:
`../web/AGENTS.md`
- Worker queues and processors:
`../worker/AGENTS.md`
- Shared contracts, exports, schema, and migrations:
`../packages/shared/AGENTS.md`
- EE-only work:
`../ee/AGENTS.md`
Read the minimal set required for the task. More-specific package guides and
shared skills take precedence over this root file for their scoped areas.
## Project Structure
```text
langfuse/
├─ web/ # Next.js app (UI + tRPC + public REST)
├─ worker/ # Queue consumers and background processing
├─ packages/shared/ # Shared domain, DB, queue contracts, repositories
├─ ee/ # Enterprise package consumed by web
├─ generated/ # Generated API clients (do not hand-edit)
├─ fern/ # API definition sources
└─ scripts/ # Repo scripts
```
- Dependency direction:
- `web` -> `@langfuse/shared`, `@langfuse/ee`
- `worker` -> `@langfuse/shared`
- `@langfuse/ee` -> `@langfuse/shared`
- `@langfuse/shared` -> no imports from `web`, `worker`, or `ee`
- Queue payload schemas and queue-name contracts are owned by
`packages/shared/src/server/queues.ts`.
- High-signal shared entry points:
- Domain models: `packages/shared/src/domain/{observations,traces,scores}.ts`
- Postgres schema: `packages/shared/prisma/schema.prisma`
- ClickHouse migrations:
`packages/shared/clickhouse/migrations/{clustered,unclustered}/*.sql`
- Architecture handbook:
[langfuse.com/handbook/product-engineering/architecture](https://langfuse.com/handbook/product-engineering/architecture)
with source markdown in
`../langfuse-docs/content/handbook/product-engineering/architecture.mdx`
## Core Commands
- Install deps: `pnpm install`
- Dev all packages: `pnpm run dev`
- Dev web only: `pnpm run dev:web`
- Dev worker only: `pnpm run dev:worker`
- Lint all: `pnpm run lint`
- Typecheck all: `pnpm run typecheck` / `pnpm tc`
- Build check: `pnpm run build:check`
- Full build: `pnpm run build`
- Full reset/bootstrap (destructive): `pnpm run dx`
- Codex environment bootstrap: `bash scripts/codex/setup.sh`
- Codex environment maintenance: `bash scripts/codex/maintenance.sh`
- Install Playwright Chromium for agent browser review: `pnpm run playwright:install`
Minimum verification matrix:
| Change scope | Minimum verification |
| --- | --- |
| `web/**` only | `pnpm --filter web run lint` + targeted web tests |
| `worker/**` only | `pnpm --filter worker run lint` + targeted worker tests |
| `packages/shared/**` (non-schema) | `pnpm --filter @langfuse/shared run lint` + one targeted web check + one targeted worker check |
| `packages/shared/prisma/**` or `packages/shared/clickhouse/**` | `pnpm --filter @langfuse/shared run lint` + `pnpm run db:generate` + targeted web/worker regressions |
| Public API contract (`web/src/pages/api/public/**`, `web/src/features/public-api/types/**`, `fern/apis/**`) | web lint + targeted server API tests + Fern update/regeneration; never hand-edit `generated/**` |
| Cross-package refactor (`web` + `worker` + `shared`) | `pnpm run lint` + `pnpm run typecheck` + targeted tests per impacted package |
## Repo Rules
- Keep changes scoped; avoid unrelated refactors.
- Prefer package-local implementation details in package `AGENTS.md` files.
- Do not hand-edit generated/build artifacts:
- `generated/*`
- `web/.next/*`
- `web/.next-check/*`
- `*/dist/*`
- `packages/shared/prisma/generated/*`
- Public API contract changes must update Fern sources in `fern/apis/**` and
regenerated outputs; never hand-edit `generated/**`.
- Keep tests independent and parallel-safe.
- For bug fixes, write the failing test first, confirm it fails, then fix the
bug.
- For user-visible frontend changes in `web/**`, review the affected flow in a
real browser with the Playwright MCP server before signoff. Use
`skills/frontend-browser-review/SKILL.md` and `../web/AGENTS.md` for the
browser-review loop.
- Never commit secrets or credentials. Keep `.env*.example` files in sync with
required env vars.
## Shared Agent Setup
- `.agents/AGENTS.md` is the canonical root guide.
- Root `AGENTS.md` is a symlink to `.agents/AGENTS.md`.
- Root `CLAUDE.md` is a compatibility symlink to `AGENTS.md`.
- Shared agent/tool config lives in `config.json` and shared skills live in
`skills/`.
- Project-scoped provider discovery files are generated local artifacts. Edit
the canonical files under `.agents/` instead of editing generated tool
directories by hand.
- If you change `.agents/config.json`, `skills/**`, or the shim-generation
workflow, run:
- `pnpm run agents:sync`
- `pnpm run agents:check`
- Do not commit generated provider config or shim outputs under `.claude/`,
`.cursor/`, `.codex/`, `.vscode/`, or `.mcp.json`.
- Durable cross-tool guidance belongs in root/package `AGENTS.md` files or
`skills/**`, not only in tool-specific config directories.
## Commit, PR, and Release Rules
- Commit messages and PR titles must follow Conventional Commits:
`type(scope): description` or `type: description`.
- PR titles are validated by `.github/workflows/validate-pr-title.yml`.
- In PR descriptions, list impacted packages and executed verification commands.
- Release workflow is managed at root with `pnpm run release`.
- Promote `main` to `production` via
`.github/workflows/promote-main-to-production.yml` or
`pnpm run release:cloud`.
- Do not change release/versioning flow without updating this file and impacted
package guides.
## Git and Tooling Notes
- Use `gh search issues` for GitHub issue search.
- Do not use destructive git commands such as `reset --hard` unless explicitly
requested.
- Do not revert unrelated working-tree changes.
- Keep commits focused and atomic.
- Remaining `.cursor/rules/*.mdc` files should stay thin wrappers around shared
docs or skills rather than owning durable repo guidance directly.
+181
View File
@@ -0,0 +1,181 @@
# Shared Agent Setup
This directory is the neutral, repo-owned source of truth for agent behavior in
Langfuse.
Use `.agents/` for configuration and guidance that should apply across tools.
Do not put durable shared guidance only in `.claude/`, `.codex/`, `.cursor/`,
or `.vscode/`.
## Layout
- `AGENTS.md`: canonical shared root instructions
- `config.json`: shared bootstrap and MCP configuration used to generate
tool-specific shims
- `skills/`: shared, tool-neutral implementation guidance for recurring
workflows
## `config.json`
`.agents/config.json` contains four kinds of data:
- `shared`: defaults used across tools
- `mcpServers`: project MCP servers and how to connect to them
- `claude`: Claude-specific generated settings inputs
- `codex`: Codex-specific generated settings inputs
- `cursor`: Cursor-specific generated settings inputs
Current shape:
```json
{
"shared": {
"setupScript": "bash scripts/codex/setup.sh",
"devCommand": "pnpm run dev",
"devTerminalDescription": "Main development terminal running the development server"
},
"mcpServers": {
"playwright": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "@playwright/mcp@latest"]
},
"datadog": {
"transport": "http",
"url": "https://mcp.datadoghq.com/api/unstable/mcp-server/mcp"
}
},
"claude": {
"settings": {}
},
"codex": {
"environment": {
"version": 1,
"name": "langfuse"
}
},
"cursor": {
"environment": {
"agentCanUpdateSnapshot": false
}
}
}
```
## How Shims Are Generated
`scripts/agents/sync-agent-shims.mjs` reads `.agents/config.json` and writes the
tool discovery files that those products require.
Generated local artifacts:
- `.claude/settings.json`
- `.claude/skills/*`
- `.cursor/environment.json`
- `.cursor/mcp.json`
- `.vscode/mcp.json`
- `.mcp.json`
- `.codex/config.toml`
- `.codex/environments/environment.toml`
The repo root discovery files remain committed as symlinks:
- `AGENTS.md` -> `.agents/AGENTS.md`
- `CLAUDE.md` -> `AGENTS.md`
This keeps provider discovery stable while `.agents/` remains the source of
truth.
## When To Edit `config.json`
Edit `.agents/config.json` when you need to:
- add, remove, or update a shared MCP server
- change the shared setup/bootstrap command
- change the default dev command or terminal label used by generated shims
- adjust generated Claude, Cursor, or Codex settings that are intentionally
modeled in the shared config
Do not edit generated shim files by hand. Edit the canonical files in
`.agents/` instead.
## How To Extend `config.json`
### Add an MCP server
Add a new entry under `mcpServers`.
For `stdio` servers:
```json
{
"mcpServers": {
"example": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "some-package"]
}
}
}
```
For HTTP servers:
```json
{
"mcpServers": {
"example": {
"transport": "http",
"url": "https://example.com/mcp"
}
}
}
```
Optional fields:
- `env` for `stdio` servers
- `headers` for HTTP servers
### Change bootstrap or default dev command
Update values in `shared`:
- `setupScript`
- `devCommand`
- `devTerminalDescription`
### Add tool-specific generated inputs
Only add tool-specific fields when they are required to generate a discovery
file for a supported tool. Keep the shared config minimal and neutral.
## Workflow
After editing `.agents/config.json`:
1. Run `pnpm run agents:sync`
2. Run `pnpm run agents:check`
3. Verify you did not stage any generated files under `.claude/skills/` or the
generated MCP/runtime config paths
4. Update `AGENTS.md` or `CONTRIBUTING.md` if the shared workflow materially
changed
`pnpm install` also runs the sync/check flow via `postinstall`.
## Adding Shared Skills
Shared skills live under `.agents/skills/`.
Use them for durable, reusable guidance such as:
- backend implementation patterns
- provider-specific maintenance workflows
- repeated repo-specific review checklists
Do not use skills for one-off task notes or tool runtime configuration.
`pnpm run agents:sync` projects the shared skills into `.claude/skills/` so
Claude can discover the same repo-owned skills.
For the skill authoring workflow, see [skills/README.md](skills/README.md).
+59
View File
@@ -0,0 +1,59 @@
{
"shared": {
"setupScript": "bash scripts/codex/setup.sh",
"devCommand": "pnpm run dev",
"devTerminalDescription": "Main development terminal running the development server"
},
"mcpServers": {
"playwright": {
"transport": "stdio",
"command": "npx",
"args": [
"-y",
"@playwright/mcp@latest",
"--isolated",
"--save-trace",
"--output-dir",
".playwright-mcp",
"--test-id-attribute",
"data-testid"
]
},
"langfuse-docs": {
"transport": "http",
"url": "https://langfuse.com/api/mcp"
},
"linear": {
"transport": "http",
"url": "https://mcp.linear.app/mcp"
}
},
"claude": {
"settings": {
"permissions": {
"allow": [
"Bash(find:*)",
"Bash(rg:*)",
"Bash(grep:*)",
"Bash(ls:*)",
"Bash(cat:*)",
"Bash(head:*)",
"Bash(tail:*)"
],
"deny": []
},
"enableAllProjectMcpServers": true
}
},
"codex": {
"environment": {
"version": 1,
"name": "langfuse"
}
},
"cursor": {
"environment": {
"agentCanUpdateSnapshot": false
}
}
}
+120
View File
@@ -0,0 +1,120 @@
# Shared Skills
Shared repo skills for any coding agent working in Langfuse.
Use these from `AGENTS.md`. Claude Code reaches the same shared instructions via
the root `CLAUDE.md` compatibility symlink. Shared skills should stay focused on
reusable implementation guidance rather than runtime automation.
For the shared agent config and generated shim model, start with
[`../README.md`](../README.md).
Claude discovers these shared skills through symlinks under `.claude/skills/`.
Those discovery links are created and verified by `pnpm run agents:sync` and
`pnpm run agents:check`.
Shared skills should use progressive disclosure:
- `SKILL.md` is the short entrypoint with trigger guidance and navigation.
- `AGENTS.md` is optional and should stay concise when it exists.
- `references/` holds focused prose references that agents should open only
when the task needs them.
- `scripts/` holds deterministic helpers for repetitive or fragile steps.
## Available Skills
### agent-setup-maintenance
Use for:
- `.agents/config.json`, `.agents/AGENTS.md`, or `.agents/README.md`
- shared skill additions or shared skill routing changes
- generated shim behavior in `scripts/agents/sync-agent-shims.mjs`
- install-time agent sync behavior and provider discovery paths
Open: [agent-setup-maintenance/SKILL.md](agent-setup-maintenance/SKILL.md)
### frontend-browser-review
Use for:
- user-visible changes in `web/**`
- Playwright MCP browser review before signoff
- checking visible regressions in layout, styling, navigation, or responsive behavior
Open: [frontend-browser-review/SKILL.md](frontend-browser-review/SKILL.md)
### backend-dev-guidelines
Use for:
- tRPC routers and procedures
- public API endpoints
- worker queue processors
- Prisma and ClickHouse backed services
- backend auth, validation, observability, and tests
Open: [backend-dev-guidelines/SKILL.md](backend-dev-guidelines/SKILL.md)
### add-model-price
Use for:
- `worker/src/constants/default-model-prices.json`
- `packages/shared/src/server/llm/types.ts`
- pricing tiers, tokenizer IDs, and model `matchPattern` changes
Open: [add-model-price/SKILL.md](add-model-price/SKILL.md)
### code-review
Use for:
- PR or branch review
- correctness, regression, and risk-focused review tasks
- applying the repo-specific review policy in
`code-review/references/review-checklist.md`
Open: [code-review/SKILL.md](code-review/SKILL.md)
### changelog-writing
Use for:
- changelog entries for completed features
- drafting user-facing release notes
- checking related docs links for changelog posts
Open: [changelog-writing/SKILL.md](changelog-writing/SKILL.md)
### pnpm-upgrade-package
Use for:
- pnpm dependency bumps that need a specific target version
- interactive upgrades where the package name or version may be missing
- checking whether `pnpm-workspace.yaml` `minimumReleaseAgeExclude` must change
- comparing registry latest with the latest version installable under the
current release-age gate
Open: [pnpm-upgrade-package/SKILL.md](pnpm-upgrade-package/SKILL.md)
## Adding a New Shared Skill
1. Codex may create or refine shared skills under `.agents/skills/` when a
repo-specific workflow becomes repeated enough to justify durable guidance.
2. Create a concise `.agents/skills/<skill-name>/SKILL.md`.
3. Add `.agents/skills/<skill-name>/AGENTS.md` only when the skill benefits
from a short router or checklist on top of `SKILL.md`.
4. Prefer `references/` for detailed prose and `scripts/` for deterministic
execution helpers.
5. Keep the skill tightly scoped to one domain or workflow.
6. Link the skill from `AGENTS.md` if it is relevant across the repo.
7. Run `pnpm run agents:sync` and `pnpm run agents:check` so Claude's projected
`.claude/skills/` view stays in sync.
8. Update `AGENTS.md` or package-local `AGENTS.md` if the new skill changes the
default reusable workflow for future agents.
9. Run the relevant verification for the package or workflow the skill affects.
## Skill Design Rules
- Keep the skill tool-neutral.
- Use `SKILL.md` as the short entrypoint, not the full knowledge dump.
- Prefer `references/` for deeper docs and `scripts/` for deterministic helpers.
- Avoid copying large sections of repo docs into the skill when a stable link is
enough.
- If the skill is web- or package-specific, link the nearest package
`AGENTS.md` or package docs instead of restating them.
+544
View File
@@ -0,0 +1,544 @@
# Add Model Price
Guide for adding or updating model pricing entries in Langfuse. Use this when
editing `worker/src/constants/default-model-prices.json`,
`packages/shared/src/server/llm/types.ts`, model `matchPattern` values,
tokenizer IDs, or pricing tiers.
## Purpose
This guide keeps model pricing changes consistent across providers and runtime
surfaces so Langfuse can calculate token costs accurately.
## How to Use This Skill
1. Read [references/schema-and-tiers.md](references/schema-and-tiers.md) for
the JSON shape and pricing-tier rules.
2. Read
[references/provider-sources-and-price-keys.md](references/provider-sources-and-price-keys.md)
for official pricing URLs, per-token conversion, and provider-specific usage
keys.
3. Read [references/match-patterns.md](references/match-patterns.md) when you
need to add or expand regex coverage.
4. Read
[references/workflow-and-validation.md](references/workflow-and-validation.md)
for the end-to-end edit workflow, validation rules, and common mistakes.
## Deterministic Helpers
- Validate the pricing file:
`node .agents/skills/add-model-price/scripts/validate-pricing-file.mjs`
- Test a regex directly:
`node .agents/skills/add-model-price/scripts/test-match-pattern.mjs --pattern '(?i)^(openai/)?(gpt-4o)$' --accept gpt-4o openai/gpt-4o --reject gpt-4o-mini`
- Test the regex for an existing model entry:
`node .agents/skills/add-model-price/scripts/test-match-pattern.mjs --model gpt-4o --accept gpt-4o openai/gpt-4o --reject gpt-4o-mini`
## Quick Start Checklist
### Adding a New Model
- [ ] Gather official pricing from the provider documentation
- [ ] Generate a lowercase UUID for the model entry
- [ ] Create a `matchPattern` that covers supported provider formats
- [ ] Add at least one default pricing tier
- [ ] Insert the pricing entry into
`worker/src/constants/default-model-prices.json`
- [ ] Update `packages/shared/src/server/llm/types.ts` if the model should be
selectable in playground or evaluation flows
- [ ] Validate the JSON after editing
### Updating an Existing Model
- [ ] Update the relevant prices, keys, tiers, or regexes
- [ ] Refresh `updatedAt` to today's ISO-8601 timestamp
- [ ] Validate the JSON after editing
## Target Files
- Pricing data:
`worker/src/constants/default-model-prices.json`
- Shared model types:
`packages/shared/src/server/llm/types.ts`
- Validation logic:
`packages/shared/src/features/model-pricing/validation.ts`
- Matching logic:
`packages/shared/src/server/pricing-tiers/matcher.ts`
- Tests:
`worker/src/__tests__/pricing-tier-matcher.test.ts`
## Data Structure
### Complete Model Entry Schema
```json
{
"id": "uuid-generated-with-uuidgen",
"modelName": "model-name-identifier",
"matchPattern": "(?i)^regex-pattern$",
"createdAt": "ISO-8601-timestamp",
"updatedAt": "ISO-8601-timestamp",
"tokenizerConfig": null,
"tokenizerId": "claude|openai|null",
"pricingTiers": [
{
"id": "model-uuid_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {
"input": 0.000005,
"output": 0.000025
}
}
]
}
```
### Required Fields
| Field | Type | Description |
| --- | --- | --- |
| `id` | string | Unique lowercase UUID |
| `modelName` | string | Primary model identifier |
| `matchPattern` | string | Regex for matching model names |
| `createdAt` | string | ISO-8601 timestamp set on creation |
| `updatedAt` | string | ISO-8601 timestamp refreshed whenever the entry changes |
| `pricingTiers` | array | At least one pricing tier |
### Optional Fields
| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `tokenizerId` | string | `null` | `"claude"`, `"openai"`, or `null` |
| `tokenizerConfig` | object | `null` | Custom tokenizer settings |
## Pricing Tier Structure
### Default Tier
Every model must have exactly one default tier:
```json
{
"id": "{model-id}_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {}
}
```
Rules for the default tier:
- `isDefault` must be `true`
- `priority` must be `0`
- `conditions` must be `[]`
### Additional Tiers
Use extra tiers for context-window or usage-based pricing:
```json
{
"id": "uuid-for-tier",
"name": "Large Context (>200K)",
"isDefault": false,
"priority": 1,
"conditions": [
{
"usageDetailPattern": "(input|prompt|cached)",
"operator": "gt",
"value": 200000,
"caseSensitive": false
}
],
"prices": {}
}
```
Supported condition operators: `gt`, `gte`, `lt`, `lte`, `eq`, `neq`
## Official Pricing Sources
Always fetch pricing from the provider's official docs before editing. Do not
infer or estimate missing values.
| Provider | Source |
| --- | --- |
| Anthropic Claude | `https://platform.claude.com/docs/en/about-claude/pricing` |
| OpenAI | `https://openai.com/api/pricing/` |
| Google Gemini | `https://ai.google.dev/pricing` |
| AWS Bedrock | `https://aws.amazon.com/bedrock/pricing/` |
| Azure OpenAI | `https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/` |
Gather:
1. Base input token price per million tokens
2. Output token price per million tokens
3. Cache write price when supported
4. Cache read price when supported
5. Any long-context pricing tiers
6. All model ID formats that Langfuse should match
## Price Conversion
Values in `default-model-prices.json` are per token, not per million tokens.
| Provider Price | JSON Value |
| --- | --- |
| `$5 / MTok` | `5e-6` |
| `$25 / MTok` | `25e-6` |
| `$0.50 / MTok` | `0.5e-6` |
| `$6.25 / MTok` | `6.25e-6` |
Formula:
```text
price_per_token = price_per_mtok / 1_000_000
```
## Common Price Keys by Provider
### Anthropic Claude Models
```json
{
"input": "<base_input_price>",
"input_tokens": "<base_input_price>",
"output": "<output_price>",
"output_tokens": "<output_price>",
"cache_creation_input_tokens": "<cache_write_price>",
"input_cache_creation": "<cache_write_price>",
"cache_read_input_tokens": "<cache_read_price>",
"input_cache_read": "<cache_read_price>"
}
```
### OpenAI Models
```json
{
"input": "<input_price>",
"input_cached_tokens": "<cached_input_price>",
"input_cache_read": "<cached_input_price>",
"output": "<output_price>"
}
```
### Google Gemini Models
```json
{
"input": "<input_price>",
"input_modality_1": "<input_price>",
"prompt_token_count": "<input_price>",
"promptTokenCount": "<input_price>",
"input_cached_tokens": "<cached_price>",
"cached_content_token_count": "<cached_price>",
"output": "<output_price>",
"output_modality_1": "<output_price>",
"candidates_token_count": "<output_price>",
"candidatesTokenCount": "<output_price>"
}
```
## Match Pattern Examples
### Anthropic Claude: API + Bedrock + Vertex
```regex
(?i)^(anthropic\/)?(claude-opus-4-6|(eu\\.|us\\.|apac\\.)?anthropic\\.claude-opus-4-6-v1(:0)?|claude-opus-4-6)$
```
Matches:
- `claude-opus-4-6`
- `anthropic/claude-opus-4-6`
- `anthropic.claude-opus-4-6-v1:0`
- `us.anthropic.claude-opus-4-6-v1:0`
- `claude-opus-4-6`
### With Version Date
```regex
(?i)^(anthropic\/)?(claude-opus-4-5-20251101|(eu\\.|us\\.|apac\\.)?anthropic\\.claude-opus-4-5-20251101-v1:0|claude-opus-4-5@20251101)$
```
### OpenAI
```regex
(?i)^(openai\/)?(gpt-4o)$
```
### Google Gemini
```regex
(?i)^(google\/)?(gemini-2.5-pro)$
```
### Pattern Components
| Component | Purpose | Example |
| --- | --- | --- |
| `(?i)` | Case-insensitive match | `gpt-4o` and `GPT-4O` |
| `^...$` | Full-string match | Avoids partial matches |
| `(provider\/)?` | Optional provider prefix | `openai/gpt-4o` |
| `(eu\\.|us\\.|apac\\.)?` | Optional AWS region prefix | `us.anthropic.model` |
| `(:0)?` | Optional version suffix | Bedrock model versions |
| `@date` | Vertex AI version format | `claude-3-5-sonnet@20240620` |
## Step-by-Step Workflow
### 1. Fetch Official Pricing
Open the official provider pricing page and capture the model's input, output,
cache write, and cache read prices.
### 2. Generate a Lowercase UUID
```bash
uuidgen
```
Convert the output to lowercase before using it.
### 3. Create the JSON Entry
Example for a model with $5 input, $25 output, $6.25 cache write, and
$0.50 cache read:
```json
{
"id": "13458bc0-1c20-44c2-8753-172f54b67647",
"modelName": "claude-opus-4-6",
"matchPattern": "(?i)^(anthropic\/)?(claude-opus-4-6|(eu\\.|us\\.|apac\\.)?anthropic\\.claude-opus-4-6-v1(:0)?|claude-opus-4-6)$",
"createdAt": "2026-03-09T00:00:00.000Z",
"updatedAt": "2026-03-09T00:00:00.000Z",
"tokenizerConfig": null,
"tokenizerId": "claude",
"pricingTiers": [
{
"id": "13458bc0-1c20-44c2-8753-172f54b67647_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {
"input": 5e-6,
"input_tokens": 5e-6,
"output": 25e-6,
"output_tokens": 25e-6,
"cache_creation_input_tokens": 6.25e-6,
"input_cache_creation": 6.25e-6,
"cache_read_input_tokens": 0.5e-6,
"input_cache_read": 0.5e-6
}
}
]
}
```
### 4. Insert the Entry
Add the entry to the JSON array in
`worker/src/constants/default-model-prices.json`. Keep related models grouped
together.
### 5. Update Shared Model Types When Needed
If the model should be available in the playground or LLM-as-judge flows, add
it to the correct array in `packages/shared/src/server/llm/types.ts`.
Model arrays include:
- `anthropicModels`
- `openAIModels`
- `vertexAIModels`
- `googleAIStudioModels`
Do not add a new model as the first entry in one of these arrays. The first
entry is used as a default model in some test or evaluation paths and newer
models may not be available to all users yet.
### 6. Validate the Change
```bash
jq . worker/src/constants/default-model-prices.json > /dev/null
```
You can also inspect a specific entry:
```bash
jq '.[] | select(.modelName == "claude-opus-4-6")' worker/src/constants/default-model-prices.json
```
## Multi-Tier Example
For models with long-context pricing:
```json
{
"id": "uuid-here",
"modelName": "model-name",
"matchPattern": "...",
"pricingTiers": [
{
"id": "uuid-here_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {
"input": 5e-6,
"output": 25e-6
}
},
{
"id": "uuid-for-large-context-tier",
"name": "Large Context (>200K)",
"isDefault": false,
"priority": 1,
"conditions": [
{
"usageDetailPattern": "(input|prompt|cached)",
"operator": "gt",
"value": 200000,
"caseSensitive": false
}
],
"prices": {
"input": 10e-6,
"output": 37.5e-6
}
}
]
}
```
## Validation Rules
1. Exactly one default tier must have `isDefault: true`
2. The default tier must have `priority: 0`
3. The default tier must have `conditions: []`
4. Non-default tiers must have `priority > 0`
5. Non-default tiers must have at least one condition
6. Priorities must be unique within a model
7. Tier names must be unique within a model
8. Each tier must contain at least one price
9. All tiers must expose the same usage-type keys
10. Regex patterns must be valid and safe
## Common Mistakes
### Guessing Instead of Using Official Pricing
Wrong:
```json
{
"cache_creation_input_tokens": "input_price * 1.25"
}
```
Correct:
```json
{
"cache_creation_input_tokens": 6.25e-6
}
```
### Using MTok Values Directly
Wrong:
```json
{
"input": 5
}
```
Correct:
```json
{
"input": 5e-6
}
```
### Missing the Default Tier Suffix
Wrong:
```json
{
"id": "some-uuid"
}
```
Correct:
```json
{
"id": "model-uuid_tier_default"
}
```
### Invalid Regex Escaping
Wrong:
```json
{
"matchPattern": "anthropic.claude"
}
```
Correct:
```json
{
"matchPattern": "anthropic\\.claude"
}
```
### Forgetting to Update `updatedAt`
Wrong:
```json
{
"updatedAt": "2025-12-12T15:00:06.513Z"
}
```
Correct:
```json
{
"updatedAt": "2026-03-09T00:00:00.000Z"
}
```
## Testing Model Matching
After adding a model, verify that the regex matches the intended provider
variants:
```javascript
const pattern = new RegExp(matchPattern);
console.log(pattern.test("claude-opus-4-6")); // true
console.log(pattern.test("anthropic/claude-opus-4-6")); // true
console.log(pattern.test("anthropic.claude-opus-4-6-v1:0")); // true
console.log(pattern.test("us.anthropic.claude-opus-4-6-v1:0")); // true
```
## Existing Model Templates
Use nearby entries as templates:
- `claude-opus-4-5-20251101` for Anthropic multi-provider patterns
- `gpt-4o` for a simple OpenAI pattern
- `gemini-2.5-pro` for a multi-tier Gemini entry
+40
View File
@@ -0,0 +1,40 @@
---
name: add-model-price
description: Use when editing worker/src/constants/default-model-prices.json, packages/shared/src/server/llm/types.ts, pricing tiers, tokenizer IDs, or matchPattern regexes for OpenAI, Anthropic, Bedrock, Vertex, Azure, or Gemini model pricing.
---
# Add Model Price
Use this skill for model pricing changes in `worker/` and shared LLM type
updates in `packages/shared/`.
## When to Apply
- Editing `worker/src/constants/default-model-prices.json`
- Editing `packages/shared/src/server/llm/types.ts`
- Adding a new priced model
- Updating provider prices, cache pricing, or tier conditions
- Expanding regex coverage for Bedrock, Vertex, Azure, or provider-prefixed
model names
## How to Read This Skill
- Start with [AGENTS.md](AGENTS.md) for the high-level workflow and helper
scripts.
- Then open only the specific reference file that matches the task.
## Reference Map
| Topic | Read this when | File |
| --- | --- | --- |
| Schema and tier rules | You need the entry shape or pricing-tier invariants | [references/schema-and-tiers.md](references/schema-and-tiers.md) |
| Provider sources and price keys | You need official pricing URLs, per-token conversion, or provider-specific usage keys | [references/provider-sources-and-price-keys.md](references/provider-sources-and-price-keys.md) |
| Match patterns | You are editing `matchPattern` regexes or provider coverage | [references/match-patterns.md](references/match-patterns.md) |
| Workflow and validation | You are applying the end-to-end edit process or checking common mistakes | [references/workflow-and-validation.md](references/workflow-and-validation.md) |
## Deterministic Helpers
- Pricing file validator:
`node .agents/skills/add-model-price/scripts/validate-pricing-file.mjs`
- Match-pattern tester:
`node .agents/skills/add-model-price/scripts/test-match-pattern.mjs --model <modelName> --accept <sample...> --reject <sample...>`
@@ -0,0 +1,51 @@
# Match Patterns
## Anthropic Claude: API + Bedrock + Vertex
```regex
(?i)^(anthropic\/)?(claude-opus-4-6|(eu\\.|us\\.|apac\\.)?anthropic\\.claude-opus-4-6-v1(:0)?|claude-opus-4-6)$
```
Matches:
- `claude-opus-4-6`
- `anthropic/claude-opus-4-6`
- `anthropic.claude-opus-4-6-v1:0`
- `us.anthropic.claude-opus-4-6-v1:0`
## With Version Date
```regex
(?i)^(anthropic\/)?(claude-opus-4-5-20251101|(eu\\.|us\\.|apac\\.)?anthropic\\.claude-opus-4-5-20251101-v1:0|claude-opus-4-5@20251101)$
```
## OpenAI
```regex
(?i)^(openai\/)?(gpt-4o)$
```
## Google Gemini
```regex
(?i)^(google\/)?(gemini-2.5-pro)$
```
## Pattern Components
| Component | Purpose | Example |
| --- | --- | --- |
| `(?i)` | Case-insensitive match | `gpt-4o` and `GPT-4O` |
| `^...$` | Full-string match | Avoids partial matches |
| `(provider\/)?` | Optional provider prefix | `openai/gpt-4o` |
| `(eu\\.|us\\.|apac\\.)?` | Optional AWS region prefix | `us.anthropic.model` |
| `(:0)?` | Optional version suffix | Bedrock model versions |
| `@date` | Vertex AI version format | `claude-3-5-sonnet@20240620` |
## Testing Patterns
Use the bundled helper script:
```bash
node .agents/skills/add-model-price/scripts/test-match-pattern.mjs --model gpt-4o --accept gpt-4o openai/gpt-4o --reject gpt-4o-mini
```
@@ -0,0 +1,84 @@
# Provider Sources and Price Keys
## Official Pricing Sources
Always fetch pricing from the provider's official docs before editing.
| Provider | Source |
| --- | --- |
| Anthropic Claude | `https://platform.claude.com/docs/en/about-claude/pricing` |
| OpenAI | `https://openai.com/api/pricing/` |
| Google Gemini | `https://ai.google.dev/pricing` |
| AWS Bedrock | `https://aws.amazon.com/bedrock/pricing/` |
| Azure OpenAI | `https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/` |
Capture:
1. Base input token price per million tokens
2. Output token price per million tokens
3. Cache write price when supported
4. Cache read price when supported
5. Any long-context or conditional pricing
6. All model ID variants that Langfuse should match
## Price Conversion
Values in `default-model-prices.json` are per token, not per million tokens.
| Provider Price | JSON Value |
| --- | --- |
| `$5 / MTok` | `5e-6` |
| `$25 / MTok` | `25e-6` |
| `$0.50 / MTok` | `0.5e-6` |
| `$6.25 / MTok` | `6.25e-6` |
Formula:
```text
price_per_token = price_per_mtok / 1_000_000
```
## Common Price Keys by Provider
### Anthropic Claude
```json
{
"input": "<base_input_price>",
"input_tokens": "<base_input_price>",
"output": "<output_price>",
"output_tokens": "<output_price>",
"cache_creation_input_tokens": "<cache_write_price>",
"input_cache_creation": "<cache_write_price>",
"cache_read_input_tokens": "<cache_read_price>",
"input_cache_read": "<cache_read_price>"
}
```
### OpenAI
```json
{
"input": "<input_price>",
"input_cached_tokens": "<cached_input_price>",
"input_cache_read": "<cached_input_price>",
"output": "<output_price>"
}
```
### Google Gemini
```json
{
"input": "<input_price>",
"input_modality_1": "<input_price>",
"prompt_token_count": "<input_price>",
"promptTokenCount": "<input_price>",
"input_cached_tokens": "<cached_price>",
"cached_content_token_count": "<cached_price>",
"output": "<output_price>",
"output_modality_1": "<output_price>",
"candidates_token_count": "<output_price>",
"candidatesTokenCount": "<output_price>"
}
```
@@ -0,0 +1,96 @@
# Schema and Tiers
## Target Files
- Pricing data: `worker/src/constants/default-model-prices.json`
- Shared model types: `packages/shared/src/server/llm/types.ts`
## Complete Model Entry Schema
```json
{
"id": "uuid-generated-with-uuidgen",
"modelName": "model-name-identifier",
"matchPattern": "(?i)^regex-pattern$",
"createdAt": "ISO-8601-timestamp",
"updatedAt": "ISO-8601-timestamp",
"tokenizerConfig": null,
"tokenizerId": "claude|openai|null",
"pricingTiers": [
{
"id": "model-uuid_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {
"input": 0.000005,
"output": 0.000025
}
}
]
}
```
## Required Fields
| Field | Type | Description |
| --- | --- | --- |
| `id` | string | Unique lowercase ID used by the pricing file |
| `modelName` | string | Primary model identifier |
| `matchPattern` | string | Regex used to match provider model names |
| `createdAt` | string | ISO-8601 timestamp set on creation |
| `updatedAt` | string | ISO-8601 timestamp refreshed whenever the entry changes |
| `pricingTiers` | array | At least one pricing tier |
## Optional Fields
| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `tokenizerId` | string | `null` | Usually `"claude"`, `"openai"`, or `null` |
| `tokenizerConfig` | object | `null` | Custom tokenizer settings |
## Default Tier
Every model must have exactly one default tier:
```json
{
"id": "{model-id}_tier_default",
"name": "Standard",
"isDefault": true,
"priority": 0,
"conditions": [],
"prices": {}
}
```
Rules:
- `isDefault` must be `true`
- `priority` must be `0`
- `conditions` must be `[]`
## Additional Tiers
Use extra tiers for context-window or usage-based pricing:
```json
{
"id": "uuid-for-tier",
"name": "Large Context (>200K)",
"isDefault": false,
"priority": 1,
"conditions": [
{
"usageDetailPattern": "(input|prompt|cached)",
"operator": "gt",
"value": 200000,
"caseSensitive": false
}
],
"prices": {}
}
```
Supported operators: `gt`, `gte`, `lt`, `lte`, `eq`, `neq`
@@ -0,0 +1,63 @@
# Workflow and Validation
## Step-by-Step Workflow
### 1. Fetch Official Pricing
Open the provider's official pricing page and collect input, output, cache
write, and cache read prices.
### 2. Generate a Lowercase ID
```bash
uuidgen
```
Convert the output to lowercase before using it.
### 3. Create or Update the Entry
Use nearby models in `worker/src/constants/default-model-prices.json` as the
template, then:
- add the new entry near related models
- refresh `updatedAt` when editing an existing entry
- update `packages/shared/src/server/llm/types.ts` when the model should be
selectable in product flows
### 4. Validate the Result
Run the bundled validator:
```bash
node .agents/skills/add-model-price/scripts/validate-pricing-file.mjs
```
## Validation Rules
1. Exactly one default tier must have `isDefault: true`
2. The default tier must have `priority: 0`
3. The default tier must have `conditions: []`
4. Non-default tiers must have `priority > 0`
5. Non-default tiers must have at least one condition
6. Priorities must be unique within a model
7. Tier names must be unique within a model
8. Each tier must contain at least one price
9. All tiers must expose the same usage-type keys
10. Regex patterns must be valid
## Common Mistakes
- Guessing prices instead of using official provider docs
- Using MTok values directly instead of per-token values
- Forgetting the `_tier_default` suffix on the default tier ID
- Forgetting to escape regex metacharacters such as `.`
- Forgetting to refresh `updatedAt`
## Existing Model Templates
Use nearby entries as templates:
- `claude-opus-4-5-20251101` for Anthropic multi-provider patterns
- `gpt-4o` for a simple OpenAI pattern
- `gemini-2.5-pro` for a multi-tier Gemini entry
@@ -0,0 +1,115 @@
#!/usr/bin/env node
import fs from "node:fs/promises";
import path from "node:path";
const repoRoot = process.cwd();
const defaultFile = path.resolve(
repoRoot,
"worker/src/constants/default-model-prices.json",
);
const args = process.argv.slice(2);
function readOption(name) {
const index = args.indexOf(name);
if (index === -1) {
return null;
}
return args[index + 1] ?? null;
}
function readListOption(name) {
const index = args.indexOf(name);
if (index === -1) {
return [];
}
const values = [];
for (let i = index + 1; i < args.length; i += 1) {
if (args[i].startsWith("--")) {
break;
}
values.push(args[i]);
}
return values;
}
function compilePattern(rawPattern) {
let source = rawPattern;
let flags = "";
const inlineFlags = rawPattern.match(/^\(\?([dgimsuvy]*)\)/);
if (inlineFlags) {
flags = inlineFlags[1];
source = rawPattern.slice(inlineFlags[0].length);
}
return new RegExp(source, flags);
}
let pattern = readOption("--pattern");
const modelName = readOption("--model");
const accepted = readListOption("--accept");
const rejected = readListOption("--reject");
if (!pattern && !modelName) {
console.error("Pass either --pattern <regex> or --model <modelName>.");
process.exit(1);
}
if (accepted.length === 0 && rejected.length === 0) {
console.error("Provide samples with --accept and/or --reject.");
process.exit(1);
}
if (!pattern && modelName) {
const models = JSON.parse(await fs.readFile(defaultFile, "utf8"));
const model = models.find((entry) => entry.modelName === modelName);
if (!model) {
console.error(`Model not found in pricing file: ${modelName}`);
process.exit(1);
}
pattern = model.matchPattern;
}
let regex;
try {
regex = compilePattern(pattern);
} catch (error) {
console.error(`Invalid pattern: ${error.message}`);
process.exit(1);
}
const failures = [];
for (const sample of accepted) {
const matched = regex.test(sample);
console.log(`${matched ? "PASS" : "FAIL"} accept ${sample}`);
if (!matched) {
failures.push(`Expected pattern to match: ${sample}`);
}
}
for (const sample of rejected) {
const matched = regex.test(sample);
console.log(`${!matched ? "PASS" : "FAIL"} reject ${sample}`);
if (matched) {
failures.push(`Expected pattern to reject: ${sample}`);
}
}
if (failures.length > 0) {
console.error("");
for (const failure of failures) {
console.error(`- ${failure}`);
}
process.exit(1);
}
console.log("");
console.log(
`Pattern is valid for ${accepted.length + rejected.length} sample(s).`,
);
@@ -0,0 +1,150 @@
#!/usr/bin/env node
import fs from "node:fs/promises";
import path from "node:path";
const defaultFile = "worker/src/constants/default-model-prices.json";
const repoRoot = process.cwd();
const filePath = path.resolve(repoRoot, process.argv[2] ?? defaultFile);
const failures = [];
function compileMatchPattern(rawPattern, label) {
let source = rawPattern;
let flags = "";
const inlineFlags = rawPattern.match(/^\(\?([dgimsuvy]*)\)/);
if (inlineFlags) {
flags = inlineFlags[1];
source = rawPattern.slice(inlineFlags[0].length);
}
try {
return new RegExp(source, flags);
} catch (error) {
failures.push(`${label}: invalid matchPattern (${error.message})`);
return null;
}
}
function keysOfPrices(prices) {
return Object.keys(prices).sort();
}
const raw = await fs.readFile(filePath, "utf8");
const models = JSON.parse(raw);
if (!Array.isArray(models)) {
throw new Error("Expected the pricing file to be a JSON array.");
}
for (const model of models) {
const label = model.modelName ?? model.id ?? "<unknown-model>";
if (!model.id || typeof model.id !== "string") {
failures.push(`${label}: missing string id`);
}
if (!model.modelName || typeof model.modelName !== "string") {
failures.push(`${label}: missing string modelName`);
}
if (!model.matchPattern || typeof model.matchPattern !== "string") {
failures.push(`${label}: missing string matchPattern`);
} else {
compileMatchPattern(model.matchPattern, label);
}
if (Number.isNaN(Date.parse(model.createdAt ?? ""))) {
failures.push(`${label}: invalid createdAt timestamp`);
}
if (Number.isNaN(Date.parse(model.updatedAt ?? ""))) {
failures.push(`${label}: invalid updatedAt timestamp`);
}
if (!Array.isArray(model.pricingTiers) || model.pricingTiers.length === 0) {
failures.push(`${label}: pricingTiers must be a non-empty array`);
continue;
}
const defaultTiers = model.pricingTiers.filter((tier) => tier.isDefault);
if (defaultTiers.length !== 1) {
failures.push(`${label}: must have exactly one default tier`);
}
const seenPriorities = new Set();
const seenNames = new Set();
let expectedPriceKeys = null;
for (const tier of model.pricingTiers) {
const tierLabel = `${label}/${tier.name ?? tier.id ?? "<unknown-tier>"}`;
if (seenPriorities.has(tier.priority)) {
failures.push(`${tierLabel}: duplicate tier priority ${tier.priority}`);
} else {
seenPriorities.add(tier.priority);
}
if (seenNames.has(tier.name)) {
failures.push(`${tierLabel}: duplicate tier name ${tier.name}`);
} else {
seenNames.add(tier.name);
}
if (!tier.prices || typeof tier.prices !== "object") {
failures.push(`${tierLabel}: missing prices object`);
continue;
}
const priceKeys = keysOfPrices(tier.prices);
if (priceKeys.length === 0) {
failures.push(`${tierLabel}: prices object must not be empty`);
}
for (const [usageType, price] of Object.entries(tier.prices)) {
if (typeof price !== "number" || Number.isNaN(price) || price < 0) {
failures.push(`${tierLabel}: invalid price for ${usageType}`);
}
}
if (tier.isDefault) {
if (tier.priority !== 0) {
failures.push(`${tierLabel}: default tier priority must be 0`);
}
if (!Array.isArray(tier.conditions) || tier.conditions.length !== 0) {
failures.push(`${tierLabel}: default tier conditions must be []`);
}
} else {
if (!(tier.priority > 0)) {
failures.push(`${tierLabel}: non-default tier priority must be > 0`);
}
if (!Array.isArray(tier.conditions) || tier.conditions.length === 0) {
failures.push(
`${tierLabel}: non-default tiers must define at least one condition`,
);
}
}
if (!expectedPriceKeys) {
expectedPriceKeys = priceKeys.join(",");
} else if (expectedPriceKeys !== priceKeys.join(",")) {
failures.push(
`${tierLabel}: price keys must match the other tiers for ${label}`,
);
}
}
}
if (failures.length > 0) {
console.error("Pricing validation failed:\n");
for (const failure of failures) {
console.error(`- ${failure}`);
}
process.exit(1);
}
console.log(
`Validated ${models.length} pricing entries in ${path.relative(repoRoot, filePath)}.`,
);
@@ -0,0 +1,65 @@
---
name: agent-setup-maintenance
description: |
Shared workflow for editing Langfuse's repo-owned agent setup under `.agents/`.
Use when changing AGENTS files, shared skills, `.agents/config.json`,
generated shim behavior, provider discovery paths, or install-time agent sync.
---
# Agent Setup Maintenance
Use this skill when changing the shared agent setup for the repository.
## Start Here
- Read [`../../README.md`](../../README.md) for the shared config and shim model.
- Read root [`../../AGENTS.md`](../../AGENTS.md) for repo-level expectations.
- Inspect [`../../../scripts/agents/sync-agent-shims.mjs`](../../../scripts/agents/sync-agent-shims.mjs)
before changing generated outputs or provider discovery behavior.
- Inspect [`../../../scripts/postinstall.sh`](../../../scripts/postinstall.sh)
and [`../../../package.json`](../../../package.json) when changing install-time
sync behavior.
## Workflow
1. Edit the canonical files under `.agents/`, not generated provider outputs.
2. Keep root `AGENTS.md` and `CLAUDE.md` as discovery symlinks; do not turn
them back into manually maintained copies.
3. Treat tool-specific directories such as `.claude/`, `.cursor/`, `.codex/`,
`.vscode/`, and `.mcp.json` as generated discovery surfaces unless the tool
requires a truly tool-specific feature.
4. Keep root `AGENTS.md` concise and router-like. Move detailed or conditional
workflows into shared skills or package `AGENTS.md` files.
5. When adding or changing a shared skill, update `skills/README.md` and link
it from root `AGENTS.md` if it changes the default reusable workflow.
6. When shared setup behavior changes materially, update `README.md` and
contributor-facing docs in the same PR.
## Docker / Install-Time Constraint
- `pnpm install` runs in environments that may not contain the full repo source
tree.
- In Docker builds, Turbo's pruned install stage can run root `postinstall`
before `scripts/` and `.agents/` are available in the image.
- Keep install-time agent setup logic robust in those pruned contexts: skip
cleanly when the required repo-owned files are not present.
## Required Verification
Run after changing shared agent setup:
- `pnpm run agents:sync`
- `pnpm run agents:check`
Run additional verification when relevant:
- `pnpm run postinstall` when install-time behavior changes
- targeted tests for any scripts you changed
## Design Rules
- Prefer one repo-owned source of truth over duplicated provider-specific files.
- Keep shared setup tool-neutral where possible.
- Only keep provider-specific files in source control when the provider requires
a fixed discovery path or feature that cannot be expressed through the shared
setup model.
@@ -1,8 +1,3 @@
---
name: backend-dev-guidelines
description: Comprehensive backend development guide for Langfuse's Next.js 14/tRPC/Express/TypeScript monorepo. Use when creating tRPC routers, public API endpoints, BullMQ queue processors, services, or working with tRPC procedures, Next.js API routes, Prisma database access, ClickHouse analytics queries, Redis queues, OpenTelemetry instrumentation, Zod v4 validation, env.mjs configuration, tenant isolation patterns, or async patterns. Covers layered architecture (tRPC procedures → services, queue processors → services), dual database system (PostgreSQL + ClickHouse), projectId filtering for multi-tenant isolation, traceException error handling, observability patterns, and testing strategies (Jest for web, vitest for worker).
---
# Backend Development Guidelines
## Purpose
@@ -11,7 +6,7 @@ Establish consistency and best practices across Langfuse's backend packages (web
## When to Use This Skill
Automatically activates when working on:
Use this guide when working on:
- Creating or modifying tRPC routers and procedures
- Creating or modifying public API endpoints (REST)
@@ -105,7 +100,8 @@ Automatically activates when working on:
- **Worker**: Queue processors → Services → Database
- **packages/shared**: Shared code for Web and Worker
See [architecture-overview.md](architecture-overview.md) for complete details.
See [references/architecture-overview.md](references/architecture-overview.md)
for complete details.
---
@@ -359,7 +355,7 @@ const result = await instrumentAsync(
### 7. Comprehensive Testing Required
Write tests for all new features and bug fixes. See [testing-guide.md](resources/testing-guide.md) for detailed examples.
Write tests for all new features and bug fixes. See [testing-guide.md](references/testing-guide.md) for detailed examples.
**Test Types:**
@@ -403,7 +399,7 @@ expect(rows).toHaveLength(2);
- Use unique IDs (`randomUUID()`) to avoid test interference
- Clean up test data or use unique project IDs
- Tests must be independent and runnable in any order
- Never use `pruneDatabase` in tests
- Prefer scoped cleanup or unique project IDs over global reset helpers
### 8. Always Filter by projectId for Tenant Isolation
@@ -536,55 +532,46 @@ Reference existing Langfuse features for implementation patterns:
| Need to... | Read this |
| ------------------------- | ------------------------------------------------------------ |
| Understand architecture | [architecture-overview.md](resources/architecture-overview.md) |
| Create routes/controllers | [routing-and-controllers.md](resources/routing-and-controllers.md) |
| Organize business logic | [services-and-repositories.md](resources/services-and-repositories.md) |
| Create middleware | [middleware-guide.md](resources/middleware-guide.md) |
| Database access | [database-patterns.md](resources/database-patterns.md) |
| Manage config | [configuration.md](resources/configuration.md) |
| Write tests | [testing-guide.md](resources/testing-guide.md) |
| Understand architecture | [architecture-overview.md](references/architecture-overview.md) |
| Create routes/controllers | [routing-and-controllers.md](references/routing-and-controllers.md) |
| Organize business logic | [services-and-repositories.md](references/services-and-repositories.md) |
| Create middleware | [middleware-guide.md](references/middleware-guide.md) |
| Database access | [database-patterns.md](references/database-patterns.md) |
| Manage config | [configuration.md](references/configuration.md) |
| Write tests | [testing-guide.md](references/testing-guide.md) |
---
## Resource Files
## Reference Files
### [architecture-overview.md](resources/architecture-overview.md)
### [architecture-overview.md](references/architecture-overview.md)
Three-layer architecture (tRPC/Public API → Services → Data Access), request lifecycle for tRPC/Public API/Worker, Next.js 14 directory structure, dual database system (PostgreSQL + ClickHouse), separation of concerns, repository pattern for complex queries
### [routing-and-controllers.md](resources/routing-and-controllers.md)
### [routing-and-controllers.md](references/routing-and-controllers.md)
Next.js file-based routing, tRPC router patterns, Public REST API routes, layered architecture (Entry Points → Services → Repositories → Database), service layer organization, anti-patterns to avoid
### [services-and-repositories.md](resources/services-and-repositories.md)
### [services-and-repositories.md](references/services-and-repositories.md)
Service layer overview, dependency injection patterns, singleton patterns, repository pattern for data access, service design principles, caching strategies, testing services
### [middleware-guide.md](resources/middleware-guide.md)
### [middleware-guide.md](references/middleware-guide.md)
tRPC middleware (withErrorHandling, withOtelInstrumentation, enforceUserIsAuthed), seven tRPC procedure types (publicProcedure, authenticatedProcedure, protectedProjectProcedure, etc.), Public API middleware (withMiddlewares, createAuthedProjectAPIRoute), authentication patterns (NextAuth for tRPC, Basic Auth for Public API)
### [database-patterns.md](resources/database-patterns.md)
### [database-patterns.md](references/database-patterns.md)
Dual database architecture (PostgreSQL via Prisma + ClickHouse via direct client), PostgreSQL CRUD operations, ClickHouse query patterns (queryClickhouse, queryClickhouseStream, upsertClickhouse), repository pattern for complex queries, tenant isolation with projectId filtering, when to use which database
### [configuration.md](resources/configuration.md)
### [configuration.md](references/configuration.md)
Environment variable validation with Zod, package-specific configs (web/env.mjs with t3-oss/env-nextjs, worker/env.ts, shared/env.ts), NEXT_PUBLIC_LANGFUSE_CLOUD_REGION usage, LANGFUSE_EE_LICENSE_KEY for enterprise features, best practices for env management
### [testing-guide.md](resources/testing-guide.md)
### [testing-guide.md](references/testing-guide.md)
Integration tests (Public API with makeZodVerifiedAPICall), tRPC tests (createInnerTRPCContext, appRouter.createCaller), service-level tests (repository/service functions), worker tests (vitest with streams), test isolation principles, running tests (Jest for web, vitest for worker)
---
## Related Skills
- **database-verification** - Verify column names and schema consistency
- **skill-developer** - Meta-skill for creating and managing skills
---
**Skill Status**: COMPLETE ✅
**Line Count**: ~540 lines
**Progressive Disclosure**: 7 resource files ✅
**Progressive Disclosure**: 7 reference files ✅
@@ -0,0 +1,43 @@
---
name: backend-dev-guidelines
description: Shared backend guide for Langfuse's Next.js 14, tRPC, BullMQ, and TypeScript monorepo. Use when creating or reviewing tRPC routers, public REST endpoints, BullMQ queue processors, backend services, middleware, Prisma or ClickHouse data access, OpenTelemetry instrumentation, Zod validation, env configuration, or backend tests across web, worker, or packages/shared.
---
# Backend Development Guidelines
Use this skill for backend and API work across `web/`, `worker/`, and
`packages/shared/`.
## When to Apply
- Creating or modifying tRPC routers and procedures
- Creating or modifying public API endpoints
- Creating or modifying queue processors, producers, or queue-backed workflows
- Building or refactoring backend services and repositories
- Working on backend auth, middleware, validation, or observability
- Updating Prisma or ClickHouse access patterns
- Adding or fixing backend tests
## How to Read This Skill
- Start with [AGENTS.md](AGENTS.md) when the task spans multiple backend areas
or you need the end-to-end checklists.
- Read only the specific reference file that matches the work when the scope is
narrower.
## Reference Map
| Topic | Read this when | File |
| --- | --- | --- |
| Architecture and package boundaries | You need the web/worker/shared split, request flow, or queue lifecycle | [references/architecture-overview.md](references/architecture-overview.md) |
| Routing and controllers | You are writing tRPC procedures, public API routes, or queue entrypoints | [references/routing-and-controllers.md](references/routing-and-controllers.md) |
| Middleware and auth | You are changing request auth, permissions, or middleware composition | [references/middleware-guide.md](references/middleware-guide.md) |
| Services and repositories | You are placing business logic, repository code, or DI patterns | [references/services-and-repositories.md](references/services-and-repositories.md) |
| Database access | You are touching Prisma, ClickHouse, tenant filters, or query patterns | [references/database-patterns.md](references/database-patterns.md) |
| Configuration | You are adding env vars, startup config, or runtime toggles | [references/configuration.md](references/configuration.md) |
| Testing | You are adding or updating backend tests | [references/testing-guide.md](references/testing-guide.md) |
## Full Compiled Guide
Read [AGENTS.md](AGENTS.md) for the complete backend guide with checklists,
directory conventions, imports, architecture, and cross-cutting practices.
@@ -864,7 +864,7 @@ const validated = bodySchema.parse(req.body);
**Related Files:**
- [SKILL.md](../SKILL.md) - Main guide
- [../AGENTS.md](../AGENTS.md) - Main guide
- [routing-and-controllers.md](routing-and-controllers.md) - tRPC and Public API details
- [services-and-repositories.md](services-and-repositories.md) - Service patterns
- [testing-guide.md](testing-guide.md) - Testing strategies
@@ -75,7 +75,7 @@ export const env = createEnv({
// Client-side variables (exposed to browser)
client: {
NEXT_PUBLIC_LANGFUSE_CLOUD_REGION: z
.enum(["US", "EU", "STAGING", "DEV", "HIPAA"])
.enum(["US", "EU", "STAGING", "DEV", "HIPAA", "JP"])
.optional(),
NEXT_PUBLIC_SIGN_UP_DISABLED: z.enum(["true", "false"]).default("false"),
// ... client variables
@@ -274,7 +274,7 @@ const licenseKey = env.LANGFUSE_EE_LICENSE_KEY;
**Purpose:** Identifies the cloud deployment region for Langfuse Cloud.
**Type:** `"US" | "EU" | "STAGING" | "DEV" | "HIPAA" | undefined`
**Type:** `"US" | "EU" | "STAGING" | "DEV" | "HIPAA" | "JP" | undefined`
**Where Used:**
@@ -286,11 +286,12 @@ const licenseKey = env.LANGFUSE_EE_LICENSE_KEY;
**When Set:**
| Environment | Value | Purpose |
| ------------------------ | ---------------------- | ---------------------------------------------- |
|--------------------------|------------------------|------------------------------------------------|
| **Developer Laptop** | `"DEV"` or `"STAGING"` | Local development against cloud infrastructure |
| **Langfuse Cloud US** | `"US"` | Production US region |
| **Langfuse Cloud EU** | `"EU"` | Production EU region |
| **Langfuse Cloud HIPAA** | `"HIPAA"` | HIPAA-compliant region |
| **Langfuse Cloud JP** | `"JP"` | Production JP region |
| **OSS Self-Hosted** | `undefined` (not set) | Self-hosted deployments don't have region |
**Use Cases:**
@@ -558,5 +559,5 @@ langfuse/
**Related Files:**
- [SKILL.md](../SKILL.md) - Main guide
- [../AGENTS.md](../AGENTS.md) - Main guide
- [architecture-overview.md](architecture-overview.md) - Architecture patterns
@@ -655,6 +655,6 @@ LANGFUSE_CLICKHOUSE_QUERY_MAX_ATTEMPTS: z.coerce.number().positive().default(3)
**Related Files:**
- [SKILL.md](../SKILL.md) - Main backend development guidelines
- [../AGENTS.md](../AGENTS.md) - Main backend development guidelines
- [architecture-overview.md](architecture-overview.md) - System architecture
- [configuration.md](configuration.md) - Environment variable configuration
@@ -760,6 +760,6 @@ ctx.trace // TraceRecord (pre-fetched)
**Related Files:**
- [SKILL.md](../SKILL.md) - Main backend development guidelines
- [../AGENTS.md](../AGENTS.md) - Main backend development guidelines
- [architecture-overview.md](architecture-overview.md) - System architecture
- [async-and-errors.md](async-and-errors.md) - Error handling patterns
- [../AGENTS.md](../AGENTS.md) - Error handling patterns and traceException guidance
@@ -838,7 +838,7 @@ export const upsertScore = async (
**Related Files:**
- [SKILL.md](../SKILL.md) - Main backend development guidelines
- [../AGENTS.md](../AGENTS.md) - Main backend development guidelines
- [architecture-overview.md](architecture-overview.md) - System architecture
- [middleware-guide.md](middleware-guide.md) - Middleware patterns
- [database-patterns.md](database-patterns.md) - Database access patterns
@@ -871,7 +871,7 @@ describe("UserService", () => {
**Related Files:**
- [SKILL.md](SKILL.md) - Main guide
- [../AGENTS.md](../AGENTS.md) - Main guide
- [routing-and-controllers.md](routing-and-controllers.md) - Controllers that use services
- [database-patterns.md](database-patterns.md) - Prisma and repository patterns
- [complete-examples.md](complete-examples.md) - Full service/repository examples
- [testing-guide.md](testing-guide.md) - Testing service and repository code
@@ -471,7 +471,7 @@ describe("batch export test suite", () => {
1. **Test Isolation**: Each test should be independent and runnable in any order
2. **Unique IDs**: Use `randomUUID()` or unique project IDs to avoid test interference
3. **Cleanup**: Always clean up test data in service tests (or use unique project IDs)
4. **No `pruneDatabase`**: Avoid `pruneDatabase` calls, especially in `__tests__/async/` directory
4. **Avoid Global Resets**: Prefer scoped cleanup or unique project IDs over global reset helpers
### By Test Type
@@ -499,9 +499,6 @@ const { projectId } = await createOrgProjectAndApiKey();
// ❌ BAD: Shared test data between tests
const projectId = "7a88fb47-b4e2-43b8-a06c-a5ce950dc53a";
// ❌ BAD: Using pruneDatabase
await pruneDatabase();
```
---
@@ -553,6 +550,6 @@ pnpm run test --filter=worker -- --coverage
---
**Related Files:**
- [SKILL.md](../SKILL.md) - Main backend guidelines
- [../AGENTS.md](../AGENTS.md) - Main backend guidelines
- [architecture-overview.md](architecture-overview.md) - Architecture patterns
- [complete-examples.md](complete-examples.md) - Full code examples
- [services-and-repositories.md](services-and-repositories.md) - Service and repository examples
+48
View File
@@ -0,0 +1,48 @@
---
name: changelog-writing
description: |
Shared workflow for writing Langfuse changelog entries after a feature is complete.
Use when a branch is ready for merge and a changelog entry or changelog draft is needed.
---
# Changelog Writing
Use this skill when a completed feature branch needs a changelog entry.
## Workflow
1. Understand the change set.
2. Study recent changelog patterns in `../langfuse-docs/pages/changelog`.
3. Find related documentation links in `../langfuse-docs/pages`.
4. Draft a user-focused changelog entry.
5. Recommend whether an image or screenshot should be added.
## What To Gather
- The branch diff relative to `main`
- The Linear issue, if the branch name includes an `lfe-XXXX` identifier
- The affected product areas
- Relevant docs pages to link or create
## Writing Rules
- Write for users, not internal implementation detail
- Prefer second person: "you can now..."
- Focus on what changed, why it matters, and how to use it
- Match the structure and tone of recent changelog posts
- Keep technical detail only where it improves user understanding
## Output Format
Provide:
1. A short summary of what changed
2. The complete changelog post content
3. Whether an image should be added and what it should show
4. Any docs pages that should be linked or created
## Reference Files
- Changelog destination: `../langfuse-docs/pages/changelog`
- Recent changelog examples: inspect 3-5 recent files in that directory
- Existing docs: `../langfuse-docs/pages`
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,50 @@
# ClickHouse Best Practices
Agent skill providing comprehensive ClickHouse guidance for schema design, query optimization, and data ingestion.
## Installation
```bash
npx skills add ClickHouse/clickhouse-agent-skills
```
## What's Included
**28 atomic rules** organized by prefix:
| Prefix | Count | Coverage |
|--------|-------|----------|
| `schema-pk-*` | 4 | PRIMARY KEY selection, cardinality ordering |
| `schema-types-*` | 5 | Data types, LowCardinality, Nullable |
| `schema-partition-*` | 4 | Partitioning strategy, lifecycle management |
| `schema-json-*` | 1 | JSON type usage |
| `query-join-*` | 5 | JOIN algorithms, filtering, alternatives |
| `query-index-*` | 1 | Data skipping indices |
| `query-mv-*` | 2 | Incremental and refreshable MVs |
| `insert-batch-*` | 1 | Batch sizing (10K-100K rows) |
| `insert-async-*` | 2 | Async inserts, data formats |
| `insert-mutation-*` | 2 | Mutation avoidance |
| `insert-optimize-*` | 1 | OPTIMIZE FINAL avoidance |
## Trigger Phrases
This skill activates when you:
- "Create a table for..."
- "Optimize this query..."
- "Design a schema for..."
- "Why is this query slow?"
- "How should I insert data into..."
- "Should I use UPDATE or..."
## Files
| File | Purpose |
|------|---------|
| `SKILL.md` | Quick reference and decision frameworks |
| `AGENTS.md` | Complete rule reference (auto-generated) |
| `rules/*.md` | Individual rule definitions |
## Related Documentation
All rules link to official ClickHouse documentation:
- [ClickHouse Best Practices](https://clickhouse.com/docs/best-practices)
@@ -0,0 +1,234 @@
---
name: clickhouse-best-practices
description: MUST USE when reviewing ClickHouse schemas, queries, or configurations. Contains 28 rules that MUST be checked before providing recommendations. Always read relevant rule files and cite specific rules in responses.
license: Apache-2.0
metadata:
author: ClickHouse Inc
version: "0.3.0"
---
# ClickHouse Best Practices
Comprehensive guidance for ClickHouse covering schema design, query optimization, and data ingestion. Contains 28 rules across 3 main categories (schema, query, insert), prioritized by impact.
> **Official docs:** [ClickHouse Best Practices](https://clickhouse.com/docs/best-practices)
## IMPORTANT: How to Apply This Skill
**Before answering ClickHouse questions, follow this priority order:**
1. **Check for applicable rules** in the `rules/` directory
2. **If rules exist:** Apply them and cite them in your response using "Per `rule-name`..."
3. **If no rule exists:** Use the LLM's ClickHouse knowledge or search documentation
4. **If uncertain:** Use web search for current best practices
5. **Always cite your source:** rule name, "general ClickHouse guidance", or URL
**Why rules take priority:** ClickHouse has specific behaviors (columnar storage, sparse indexes, merge tree mechanics) where general database intuition can be misleading. The rules encode validated, ClickHouse-specific guidance.
### For Formal Reviews
When performing a formal review of schemas, queries, or data ingestion:
---
## Review Procedures
### For Schema Reviews (CREATE TABLE, ALTER TABLE)
**Read these rule files in order:**
1. `rules/schema-pk-plan-before-creation.md` - ORDER BY is immutable
2. `rules/schema-pk-cardinality-order.md` - Column ordering in keys
3. `rules/schema-pk-prioritize-filters.md` - Filter column inclusion
4. `rules/schema-types-native-types.md` - Proper type selection
5. `rules/schema-types-minimize-bitwidth.md` - Numeric type sizing
6. `rules/schema-types-lowcardinality.md` - LowCardinality usage
7. `rules/schema-types-avoid-nullable.md` - Nullable vs DEFAULT
8. `rules/schema-partition-low-cardinality.md` - Partition count limits
9. `rules/schema-partition-lifecycle.md` - Partitioning purpose
**Check for:**
- [ ] PRIMARY KEY / ORDER BY column order (low-to-high cardinality)
- [ ] Data types match actual data ranges
- [ ] LowCardinality applied to appropriate string columns
- [ ] Partition key cardinality bounded (100-1,000 values)
- [ ] ReplacingMergeTree has version column if used
### For Query Reviews (SELECT, JOIN, aggregations)
**Read these rule files:**
1. `rules/query-join-choose-algorithm.md` - Algorithm selection
2. `rules/query-join-filter-before.md` - Pre-join filtering
3. `rules/query-join-use-any.md` - ANY vs regular JOIN
4. `rules/query-index-skipping-indices.md` - Secondary index usage
5. `rules/schema-pk-filter-on-orderby.md` - Filter alignment with ORDER BY
**Check for:**
- [ ] Filters use ORDER BY prefix columns
- [ ] JOINs filter tables before joining (not after)
- [ ] Correct JOIN algorithm for table sizes
- [ ] Skipping indices for non-ORDER BY filter columns
### For Insert Strategy Reviews (data ingestion, updates, deletes)
**Read these rule files:**
1. `rules/insert-batch-size.md` - Batch sizing requirements
2. `rules/insert-mutation-avoid-update.md` - UPDATE alternatives
3. `rules/insert-mutation-avoid-delete.md` - DELETE alternatives
4. `rules/insert-async-small-batches.md` - Async insert usage
5. `rules/insert-optimize-avoid-final.md` - OPTIMIZE TABLE risks
**Check for:**
- [ ] Batch size 10K-100K rows per INSERT
- [ ] No ALTER TABLE UPDATE for frequent changes
- [ ] ReplacingMergeTree or CollapsingMergeTree for update patterns
- [ ] Async inserts enabled for high-frequency small batches
---
## Output Format
Structure your response as follows:
```
## Rules Checked
- `rule-name-1` - Compliant / Violation found
- `rule-name-2` - Compliant / Violation found
...
## Findings
### Violations
- **`rule-name`**: Description of the issue
- Current: [what the code does]
- Required: [what it should do]
- Fix: [specific correction]
### Compliant
- `rule-name`: Brief note on why it's correct
## Recommendations
[Prioritized list of changes, citing rules]
```
---
## Rule Categories by Priority
| Priority | Category | Impact | Prefix | Rule Count |
|----------|----------|--------|--------|------------|
| 1 | Primary Key Selection | CRITICAL | `schema-pk-` | 4 |
| 2 | Data Type Selection | CRITICAL | `schema-types-` | 5 |
| 3 | JOIN Optimization | CRITICAL | `query-join-` | 5 |
| 4 | Insert Batching | CRITICAL | `insert-batch-` | 1 |
| 5 | Mutation Avoidance | CRITICAL | `insert-mutation-` | 2 |
| 6 | Partitioning Strategy | HIGH | `schema-partition-` | 4 |
| 7 | Skipping Indices | HIGH | `query-index-` | 1 |
| 8 | Materialized Views | HIGH | `query-mv-` | 2 |
| 9 | Async Inserts | HIGH | `insert-async-` | 2 |
| 10 | OPTIMIZE Avoidance | HIGH | `insert-optimize-` | 1 |
| 11 | JSON Usage | MEDIUM | `schema-json-` | 1 |
---
## Quick Reference
### Schema Design - Primary Key (CRITICAL)
- `schema-pk-plan-before-creation` - Plan ORDER BY before table creation (immutable)
- `schema-pk-cardinality-order` - Order columns low-to-high cardinality
- `schema-pk-prioritize-filters` - Include frequently filtered columns
- `schema-pk-filter-on-orderby` - Query filters must use ORDER BY prefix
### Schema Design - Data Types (CRITICAL)
- `schema-types-native-types` - Use native types, not String for everything
- `schema-types-minimize-bitwidth` - Use smallest numeric type that fits
- `schema-types-lowcardinality` - LowCardinality for <10K unique strings
- `schema-types-enum` - Enum for finite value sets with validation
- `schema-types-avoid-nullable` - Avoid Nullable; use DEFAULT instead
### Schema Design - Partitioning (HIGH)
- `schema-partition-low-cardinality` - Keep partition count 100-1,000
- `schema-partition-lifecycle` - Use partitioning for data lifecycle, not queries
- `schema-partition-query-tradeoffs` - Understand partition pruning trade-offs
- `schema-partition-start-without` - Consider starting without partitioning
### Schema Design - JSON (MEDIUM)
- `schema-json-when-to-use` - JSON for dynamic schemas; typed columns for known
### Query Optimization - JOINs (CRITICAL)
- `query-join-choose-algorithm` - Select algorithm based on table sizes
- `query-join-use-any` - ANY JOIN when only one match needed
- `query-join-filter-before` - Filter tables before joining
- `query-join-consider-alternatives` - Dictionaries/denormalization vs JOIN
- `query-join-null-handling` - join_use_nulls=0 for default values
### Query Optimization - Indices (HIGH)
- `query-index-skipping-indices` - Skipping indices for non-ORDER BY filters
### Query Optimization - Materialized Views (HIGH)
- `query-mv-incremental` - Incremental MVs for real-time aggregations
- `query-mv-refreshable` - Refreshable MVs for complex joins
### Insert Strategy - Batching (CRITICAL)
- `insert-batch-size` - Batch 10K-100K rows per INSERT
### Insert Strategy - Async (HIGH)
- `insert-async-small-batches` - Async inserts for high-frequency small batches
- `insert-format-native` - Native format for best performance
### Insert Strategy - Mutations (CRITICAL)
- `insert-mutation-avoid-update` - ReplacingMergeTree instead of ALTER UPDATE
- `insert-mutation-avoid-delete` - Lightweight DELETE or DROP PARTITION
### Insert Strategy - Optimization (HIGH)
- `insert-optimize-avoid-final` - Let background merges work
---
## When to Apply
This skill activates when you encounter:
- `CREATE TABLE` statements
- `ALTER TABLE` modifications
- `ORDER BY` or `PRIMARY KEY` discussions
- Data type selection questions
- Slow query troubleshooting
- JOIN optimization requests
- Data ingestion pipeline design
- Update/delete strategy questions
- ReplacingMergeTree or other specialized engine usage
- Partitioning strategy decisions
---
## Rule File Structure
Each rule file in `rules/` contains:
- **YAML frontmatter**: title, impact level, tags
- **Brief explanation**: Why this rule matters
- **Incorrect example**: Anti-pattern with explanation
- **Correct example**: Best practice with explanation
- **Additional context**: Trade-offs, when to apply, references
---
## Full Compiled Document
For the complete guide with all rules expanded inline: `AGENTS.md`
Use `AGENTS.md` when you need to check multiple rules quickly without reading individual files.
@@ -0,0 +1,24 @@
# Sections
This file defines all sections, their ordering, impact levels, and descriptions.
The section ID (in parentheses) is the filename prefix used to group rules.
---
## 1. Schema Design (schema)
**Impact:** CRITICAL
**Description:** Proper schema design is foundational to ClickHouse performance. ORDER BY is immutable after table creation; wrong choices require full data migration. Includes primary key selection, data types, partitioning strategy, and JSON usage. Column types and ordering can impact query speed by orders of magnitude.
## 2. Query Optimization (query)
**Impact:** CRITICAL
**Description:** Query patterns dramatically affect performance. JOIN algorithms, filtering strategies, skipping indices, and materialized views can reduce query time from minutes to milliseconds. Pre-computed aggregations read thousands of rows instead of billions.
## 3. Insert Strategy (insert)
**Impact:** CRITICAL
**Description:** Each INSERT creates a data part. Single-row inserts overwhelm the merge process. Proper batching (10K-100K rows), async inserts for high-frequency writes, mutation avoidance, and letting background merges work are essential for stable cluster performance.
@@ -0,0 +1,28 @@
---
title: Rule Title Here
impact: CRITICAL | HIGH | MEDIUM | LOW
impactDescription: "Quantified improvement (e.g., 10x faster queries)"
tags: [tag1, tag2]
---
## Rule Title Here
**Impact: CRITICAL** (optional description)
Brief explanation of the rule and why it matters. This should be clear and concise, explaining the performance implications.
**Incorrect (description of what's wrong):**
```sql
-- Bad: description
SELECT * FROM table;
```
**Correct (description of what's right):**
```sql
-- Good: description
SELECT * FROM table;
```
Reference: [Official Docs](https://clickhouse.com/docs/best-practices/...)
@@ -0,0 +1,55 @@
---
title: Use Async Inserts for High-Frequency Small Batches
impact: HIGH
impactDescription: "Server-side buffering when client batching isn't practical"
tags: [insert, async, buffering, small-batches]
---
## Use Async Inserts for High-Frequency Small Batches
**Impact: HIGH**
When client-side batching isn't practical, async inserts buffer server-side and create larger parts automatically.
**Incorrect (small batches without async):**
```python
# Small batches without async_insert - creates too many parts
for batch in chunks(events, 100):
client.execute("INSERT INTO events VALUES", batch)
```
**Correct (enable async inserts):**
```python
# Enable async_insert with safe defaults
client.execute("SET async_insert = 1")
client.execute("SET wait_for_async_insert = 1") # Confirms durability
for batch in chunks(events, 100):
client.execute("INSERT INTO events VALUES", batch)
# Server buffers and creates larger parts automatically
```
```sql
-- Configure server-side for specific users
ALTER USER my_app_user SETTINGS
async_insert = 1,
wait_for_async_insert = 1,
async_insert_max_data_size = 10000000, -- Flush at 10MB
async_insert_busy_timeout_ms = 1000; -- Flush after 1s
```
**Flush conditions (whichever occurs first):**
- Buffer reaches `async_insert_max_data_size`
- Time threshold `async_insert_busy_timeout_ms` elapses
- Maximum insert queries accumulate
**Return modes:**
| Setting | Behavior | Use Case |
|---------|----------|----------|
| `wait_for_async_insert=1` | Waits for flush, confirms durability | **Recommended** |
| `wait_for_async_insert=0` | Fire-and-forget, unaware of errors | **Risky** - only if you accept data loss |
Reference: [Selecting an Insert Strategy](https://clickhouse.com/docs/best-practices/selecting-an-insert-strategy)
@@ -0,0 +1,54 @@
---
title: Batch Inserts Appropriately (10K-100K rows)
impact: CRITICAL
impactDescription: "Each INSERT creates a part; single-row inserts overwhelm merge process"
tags: [insert, batching, parts, performance]
---
## Batch Inserts Appropriately (10K-100K rows)
**Impact: CRITICAL**
Each INSERT creates a new data part. Single-row or small-batch inserts create thousands of tiny parts, overwhelming the merge process and causing cluster instability.
**Incorrect (single-row or tiny batches):**
```python
# Single-row inserts - creates 10,000 parts!
for event in events:
client.execute("INSERT INTO events VALUES", [event])
# Tiny batches - still too many parts
for batch in chunks(events, 100): # 100 rows per INSERT
client.execute("INSERT INTO events VALUES", batch)
```
**Correct (proper batch size):**
```python
# Ideal batch size: 10,000-100,000 rows
BATCH_SIZE = 10_000
for batch in chunks(events, BATCH_SIZE):
client.execute("INSERT INTO events VALUES", batch)
```
**Recommended batch sizes:**
| Threshold | Value |
|-----------|-------|
| Minimum | 1,000 rows |
| Ideal range | 10,000-100,000 rows |
| Insert rate (sync) | ~1 insert per second |
**Validation:**
```sql
-- Monitor part count (>3000 per partition blocks inserts)
SELECT table, count() as parts, sum(rows) as total_rows
FROM system.parts
WHERE active AND database = 'default'
GROUP BY table
ORDER BY parts DESC;
```
Reference: [Selecting an Insert Strategy](https://clickhouse.com/docs/best-practices/selecting-an-insert-strategy)
@@ -0,0 +1,29 @@
---
title: Use Native Format for Best Insert Performance
impact: MEDIUM
impactDescription: "Native format is most efficient; JSONEachRow is expensive to parse"
tags: [insert, format, Native, performance]
---
## Use Native Format for Best Insert Performance
**Impact: MEDIUM**
Data format affects insert performance. Native format is column-oriented with minimal parsing overhead.
**Performance Ranking (fastest to slowest):**
| Format | Notes |
|--------|-------|
| **Native** | Most efficient. Column-oriented, minimal parsing. Recommended. |
| **RowBinary** | Efficient row-based alternative |
| **JSONEachRow** | Easier to use but expensive to parse |
**Example:**
```python
# Use Native format for best performance
client.execute("INSERT INTO events VALUES", data, settings={'input_format': 'Native'})
```
Reference: [Selecting an Insert Strategy](https://clickhouse.com/docs/best-practices/selecting-an-insert-strategy)
@@ -0,0 +1,74 @@
---
title: Avoid ALTER TABLE DELETE
impact: CRITICAL
impactDescription: "Use lightweight DELETE, CollapsingMergeTree, or DROP PARTITION instead"
tags: [insert, mutation, DELETE, CollapsingMergeTree]
---
## Avoid ALTER TABLE DELETE
**Impact: CRITICAL**
`ALTER TABLE DELETE` is a mutation that rewrites entire data parts. Use alternatives like lightweight DELETE, CollapsingMergeTree, or DROP PARTITION.
**Incorrect (mutation delete):**
```sql
-- Mutation delete for cleanup
ALTER TABLE orders DELETE WHERE status = 'cancelled';
-- Time-based cleanup via mutation (very expensive)
ALTER TABLE sessions DELETE WHERE created_at < now() - INTERVAL 7 DAY;
```
**Correct - CollapsingMergeTree:**
```sql
CREATE TABLE orders (
order_id UInt64,
customer_id UInt64,
total Decimal(10,2),
sign Int8 -- 1 = active, -1 = deleted
)
ENGINE = CollapsingMergeTree(sign)
ORDER BY order_id;
-- Insert order
INSERT INTO orders VALUES (123, 456, 99.99, 1);
-- "Delete" by inserting with sign = -1
INSERT INTO orders VALUES (123, 456, 99.99, -1);
-- Query collapses +1 and -1 pairs
SELECT order_id, sum(total * sign) as total
FROM orders GROUP BY order_id HAVING sum(sign) > 0;
```
**Correct - Lightweight Deletes (23.3+):**
```sql
-- Marks rows, doesn't rewrite immediately
DELETE FROM orders WHERE status = 'cancelled';
-- Physical deletion happens during normal merges
```
**Correct - DROP PARTITION for Bulk Deletion:**
```sql
-- Instant deletion of old data
ALTER TABLE events DROP PARTITION '202301';
-- Much faster than:
ALTER TABLE events DELETE WHERE toYYYYMM(timestamp) = 202301;
```
**Delete strategy comparison:**
| Method | Speed | When to Use |
|--------|-------|-------------|
| ALTER DELETE | Slow | Rare corrections only |
| CollapsingMergeTree | Fast | Frequent soft deletes |
| Lightweight DELETE | Medium | Occasional deletes |
| DROP PARTITION | Instant | Bulk deletion by partition |
Reference: [Avoid Mutations](https://clickhouse.com/docs/best-practices/avoid-mutations)
@@ -0,0 +1,58 @@
---
title: Avoid ALTER TABLE UPDATE
impact: CRITICAL
impactDescription: "Mutations rewrite entire parts; use ReplacingMergeTree instead"
tags: [insert, mutation, UPDATE, ReplacingMergeTree]
---
## Avoid ALTER TABLE UPDATE
**Impact: CRITICAL**
`ALTER TABLE UPDATE` is a mutation - an asynchronous background process that rewrites entire data parts affected by the change. This is extremely expensive for frequent or large-scale operations.
**Why mutations are problematic:**
- **Write amplification:** Rewrite complete parts even for minor changes
- **Disk I/O spike:** Degrades overall cluster performance
- **No rollback:** Cannot be rolled back after submission
- **Inconsistent reads:** SELECT may read mix of mutated and unmutated parts
**Incorrect (mutation for updates):**
```sql
-- Rewrites potentially huge amounts of data
ALTER TABLE users UPDATE status = 'inactive'
WHERE last_login < now() - INTERVAL 90 DAY;
-- Frequent row updates via mutation
ALTER TABLE inventory UPDATE quantity = quantity - 1
WHERE product_id = 123;
-- If product exists across 100 parts, rewrites ALL 100 parts
```
**Correct (ReplacingMergeTree):**
```sql
-- Table design for updates
CREATE TABLE users (
user_id UInt64,
name String,
status LowCardinality(String),
updated_at DateTime DEFAULT now()
)
ENGINE = ReplacingMergeTree(updated_at)
ORDER BY user_id;
-- "Update" by inserting new version
INSERT INTO users (user_id, name, status)
VALUES (123, 'John', 'inactive');
-- Query with FINAL to get latest version
SELECT * FROM users FINAL WHERE user_id = 123;
-- Or use aggregation
SELECT user_id, argMax(status, updated_at) as status
FROM users GROUP BY user_id;
```
Reference: [Avoid Mutations](https://clickhouse.com/docs/best-practices/avoid-mutations)
@@ -0,0 +1,57 @@
---
title: Avoid OPTIMIZE TABLE FINAL
impact: HIGH
impactDescription: "Forces expensive merge of all parts; let background merges work"
tags: [insert, OPTIMIZE, merge, performance]
---
## Avoid OPTIMIZE TABLE FINAL
**Impact: HIGH**
`OPTIMIZE TABLE ... FINAL` forces immediate merge of all parts into one part per partition. This is resource-intensive and rarely necessary. ClickHouse already performs smart background merges.
**Note:** `OPTIMIZE FINAL` is not the same as `FINAL`. The `FINAL` modifier in SELECT queries may be necessary for deduplicated results in ReplacingMergeTree and is generally fine to use.
**Incorrect (OPTIMIZE FINAL after inserts):**
```sql
-- Running OPTIMIZE FINAL after every batch insert
INSERT INTO events SELECT * FROM staging_events;
OPTIMIZE TABLE events FINAL; -- Expensive and unnecessary!
-- Scheduled OPTIMIZE FINAL jobs
-- Cron: 0 * * * * clickhouse-client -q "OPTIMIZE TABLE events FINAL"
```
**Correct (let background merges work):**
```sql
-- Let background merges handle optimization
INSERT INTO events SELECT * FROM staging_events;
-- Done! ClickHouse merges automatically
-- For ReplacingMergeTree deduplication, use FINAL in queries
SELECT * FROM events FINAL WHERE user_id = 123;
-- Instead of running OPTIMIZE FINAL to deduplicate
```
**Problems with OPTIMIZE FINAL:**
- Rewrites entire partition regardless of need
- Ignores the ~150 GB part size safeguard
- Can cause memory pressure or OOM errors
- Lengthy execution time for large datasets
**When OPTIMIZE FINAL may be acceptable:**
- Finalizing data before table freezing
- Preparing data for export operations
- One-time operations, not regular workflows
**Better alternatives:**
| Need | Alternative |
|------|-------------|
| Deduplicate ReplacingMergeTree | Use `FINAL` modifier in SELECT |
| Reduce part count | Rely on background merges |
Reference: [Avoid OPTIMIZE FINAL](https://clickhouse.com/docs/best-practices/avoid-optimize-final)
@@ -0,0 +1,77 @@
---
title: Use Data Skipping Indices for Non-ORDER BY Filters
impact: HIGH
impactDescription: "Up to 60x faster queries by skipping irrelevant granules"
tags: [query, index, skipping, bloom_filter]
---
## Use Data Skipping Indices for Non-ORDER BY Filters
**Impact: HIGH**
Queries filtering on columns not in ORDER BY cannot use the primary index and result in full scans. Data skipping indices store metadata about blocks and skip granules that definitely don't match.
**Important:** Skip indices should be considered **after** optimizing data types, primary key selection, and materialized views.
**When to use:**
- High overall cardinality but low cardinality within blocks
- Rare values critical for search (error codes, specific IDs)
- Column correlates with primary key
**When NOT to use:**
- As a first optimization step
- Matching values scattered across many blocks
- Without testing on real data
**Incorrect (filtering on non-ORDER BY column):**
```sql
CREATE TABLE events (
event_type LowCardinality(String),
timestamp DateTime,
user_id UInt64 -- Not in ORDER BY
)
ENGINE = MergeTree()
ORDER BY (event_type, toDate(timestamp));
-- Query filters on user_id - scans all matching event_type
SELECT * FROM events
WHERE event_type = 'click' AND user_id = 12345;
```
**Correct (add skipping index):**
```sql
CREATE TABLE events (
event_type LowCardinality(String),
timestamp DateTime,
user_id UInt64,
INDEX idx_user_id user_id TYPE bloom_filter GRANULARITY 4
)
ENGINE = MergeTree()
ORDER BY (event_type, toDate(timestamp));
-- Or add to existing table
ALTER TABLE events ADD INDEX idx_user_id user_id TYPE bloom_filter GRANULARITY 4;
ALTER TABLE events MATERIALIZE INDEX idx_user_id;
```
**Index types:**
| Type | Best For | Example Filter |
|------|----------|----------------|
| `bloom_filter` | Equality on high-cardinality | `WHERE user_id = 123` |
| `set(N)` | Low cardinality (N unique values) | `WHERE status IN ('a','b')` |
| `minmax` | Range queries | `WHERE amount > 1000` |
| `ngrambf_v1` | Text search | `WHERE text LIKE '%term%'` |
| `tokenbf_v1` | Token search | `WHERE hasToken(text, 'word')` |
**Validation:**
```sql
EXPLAIN indexes = 1
SELECT * FROM events WHERE user_id = 12345;
-- Look for "Skip" in output showing granules skipped
```
Reference: [Use Data Skipping Indices Where Appropriate](https://clickhouse.com/docs/best-practices/use-data-skipping-indices-where-appropriate)
@@ -0,0 +1,43 @@
---
title: Choose the Right JOIN Algorithm
impact: CRITICAL
impactDescription: "Wrong algorithm causes OOM; right algorithm handles large tables efficiently"
tags: [query, JOIN, algorithm, memory]
---
## Choose the Right JOIN Algorithm
**Impact: CRITICAL**
ClickHouse's default hash join loads the RIGHT table entirely into memory. Choose the right algorithm based on table sizes and constraints.
**Algorithm selection:**
| Algorithm | Best For | Trade-off |
|-----------|----------|-----------|
| `parallel_hash` | Small-to-medium in-memory tables | Default since 24.11; fast, concurrent |
| `hash` | General purpose, all join types | Single-threaded hash table build |
| `direct` | Dictionary lookups (INNER/LEFT only) | Fastest; no hash table construction |
| `full_sorting_merge` | Tables already sorted on join key | Skips sort if pre-ordered; low memory |
| `partial_merge` | Large tables, memory-constrained | Minimized memory; slower execution |
| `grace_hash` | Large datasets, tunable memory | Flexible; disk-spilling capability |
| `auto` | Adaptive algorithm selection | Tries hash first, falls back on memory pressure |
**Example usage:**
```sql
-- Let ClickHouse choose automatically
SET join_algorithm = 'auto';
-- For large-to-large joins where memory is constrained
SET join_algorithm = 'partial_merge';
SELECT * FROM large_a JOIN large_b ON large_b.id = large_a.id;
-- When joining by primary key columns, sort-merge skips sorting step
SET join_algorithm = 'full_sorting_merge';
SELECT * FROM table_a a JOIN table_b b ON b.pk_col = a.pk_col;
```
**Note:** ClickHouse 24.12+ automatically positions smaller tables on the right side. For earlier versions, manually ensure the smaller table is on the RIGHT.
Reference: [Minimize and Optimize JOINs](https://clickhouse.com/docs/best-practices/minimize-optimize-joins)
@@ -0,0 +1,72 @@
---
title: Consider Alternatives to JOINs
impact: CRITICAL
impactDescription: "Dictionaries and denormalization shift work from query time to insert time"
tags: [query, JOIN, dictionary, denormalization]
---
## Consider Alternatives to JOINs
**Impact: CRITICAL**
Repeated JOINs to dimension tables add overhead. Dictionaries or denormalization shift computational work from query time to insert/pre-processing time.
**Incorrect (JOIN on every query):**
```sql
-- JOIN on every query
SELECT o.order_id, c.name, c.email
FROM orders o
JOIN customers c ON c.id = o.customer_id
WHERE o.created_at > '2024-01-01';
```
**Correct - Dictionary Lookup:**
```sql
-- Create dictionary
CREATE DICTIONARY customer_dict (
id UInt64,
name String,
email String
)
PRIMARY KEY id
SOURCE(CLICKHOUSE(TABLE 'customers'))
LAYOUT(HASHED())
LIFETIME(MIN 300 MAX 360);
-- Use dictGet instead of JOIN (uses direct join algorithm - fastest)
SELECT
order_id,
dictGet('customer_dict', 'name', customer_id) as customer_name,
dictGet('customer_dict', 'email', customer_id) as customer_email
FROM orders
WHERE created_at > '2024-01-01';
```
**Correct - Denormalization:**
```sql
-- Denormalized table with materialized view
CREATE MATERIALIZED VIEW orders_enriched_mv TO orders_enriched AS
SELECT
o.order_id, o.customer_id,
c.name as customer_name,
c.email as customer_email,
o.total, o.created_at
FROM orders o
JOIN customers c ON c.id = o.customer_id;
```
**Approach comparison:**
| Approach | Use Case | Performance |
|----------|----------|-------------|
| Dictionary | Frequent lookups to small dimension | Fastest (in-memory) |
| Denormalization | Analytics always need enriched data | Fast (no join at query) |
| IN subquery | Existence filtering | Often faster than JOIN |
| JOIN | Infrequent or complex joins | Acceptable |
**Critical dictionary caveat:** Dictionaries silently deduplicate duplicate keys, retaining only the final value. Only use when source has unique keys.
Reference: [Minimize and Optimize JOINs](https://clickhouse.com/docs/best-practices/minimize-optimize-joins)
@@ -0,0 +1,54 @@
---
title: Filter Tables Before Joining
impact: CRITICAL
impactDescription: "Joining full tables then filtering wastes resources"
tags: [query, JOIN, filtering, subquery]
---
## Filter Tables Before Joining
**Impact: CRITICAL**
Joining full tables then filtering wastes resources. Add filtering in `WHERE` or `JOIN ON` clauses. If automatic pushdown fails, restructure as a subquery.
**Incorrect (join then filter):**
```sql
-- Joins entire tables, then filters
SELECT o.order_id, c.name, o.total
FROM orders o
JOIN customers c ON c.id = o.customer_id
WHERE o.created_at > '2024-01-01' AND c.country = 'US';
```
**Correct (filter in subqueries before joining):**
```sql
-- Filter in subqueries before joining
SELECT o.order_id, c.name, o.total
FROM (
SELECT order_id, customer_id, total
FROM orders
WHERE created_at > '2024-01-01'
) o
JOIN (
SELECT id, name
FROM customers
WHERE country = 'US'
) c ON c.id = o.customer_id;
```
**Even better - aggregate before joining:**
```sql
SELECT c.country, o.total_revenue
FROM (
SELECT customer_id, sum(total) as total_revenue
FROM orders
WHERE created_at > '2024-01-01'
GROUP BY customer_id
) o
JOIN customers c ON c.id = o.customer_id;
```
Reference: [Minimize and Optimize JOINs](https://clickhouse.com/docs/best-practices/minimize-optimize-joins)
@@ -0,0 +1,33 @@
---
title: Optimize NULL Handling in Outer JOINs
impact: MEDIUM
impactDescription: "Default values instead of NULL reduces memory overhead"
tags: [query, JOIN, NULL, memory]
---
## Optimize NULL Handling in Outer JOINs
**Impact: MEDIUM**
Set `join_use_nulls = 0` to use default column values instead of NULL markers, reducing memory overhead compared to Nullable wrappers.
**Example:**
```sql
-- Use default values instead of NULLs for non-matching rows
SET join_use_nulls = 0;
SELECT o.order_id, c.name
FROM orders o
LEFT JOIN customers c ON c.id = o.customer_id;
-- Non-matching rows get '' for name instead of NULL
```
**When to use:**
| Setting | Behavior | Use Case |
|---------|----------|----------|
| `join_use_nulls = 0` | Default values (empty string, 0) for non-matches | When you can handle default values |
| `join_use_nulls = 1` (default) | NULL for non-matches | When you need to distinguish "no match" from "matched with default" |
Reference: [Minimize and Optimize JOINs](https://clickhouse.com/docs/best-practices/minimize-optimize-joins)
@@ -0,0 +1,40 @@
---
title: Use ANY JOIN When Only One Match Needed
impact: HIGH
impactDescription: "Returns first match only; less memory and faster execution"
tags: [query, JOIN, ANY, performance]
---
## Use ANY JOIN When Only One Match Needed
**Impact: HIGH**
Use `ANY` JOINs when you only need a single match rather than all matches. They consume less memory and execute faster.
**Incorrect (returns all matches):**
```sql
-- Returns all matching rows, uses more memory
SELECT o.order_id, c.name
FROM orders o
LEFT JOIN customers c ON c.id = o.customer_id;
```
**Correct (returns first match only):**
```sql
-- Returns only first match per row, faster and less memory
SELECT o.order_id, c.name
FROM orders o
LEFT ANY JOIN customers c ON c.id = o.customer_id;
```
**ANY JOIN types:**
| Type | Behavior |
|------|----------|
| `LEFT ANY JOIN` | At most one match from right table |
| `INNER ANY JOIN` | At most one match, only matching rows |
| `RIGHT ANY JOIN` | At most one match from left table |
Reference: [Minimize and Optimize JOINs](https://clickhouse.com/docs/best-practices/minimize-optimize-joins)
@@ -0,0 +1,68 @@
---
title: Use Incremental MVs for Real-Time Aggregations
impact: HIGH
impactDescription: "Read thousands of rows instead of billions; minimal cluster overhead"
tags: [query, materialized-view, aggregation, real-time]
---
## Use Incremental MVs for Real-Time Aggregations
**Impact: HIGH**
Incremental MVs automatically apply the view's query to new data blocks at insert time. Results are written to a target table and partial results merge over time.
**Incorrect (full aggregation on every query):**
```sql
-- Full aggregation on every dashboard load
SELECT
event_type,
toStartOfHour(timestamp) as hour,
count() as events,
uniq(user_id) as unique_users
FROM events
WHERE timestamp >= now() - INTERVAL 7 DAY
GROUP BY event_type, hour;
-- Scans 7 days of data every time (billions of rows)
```
**Correct (incremental MV with pre-aggregation):**
```sql
-- Create target table for aggregated data
CREATE TABLE events_hourly (
event_type LowCardinality(String),
hour DateTime,
events AggregateFunction(count),
unique_users AggregateFunction(uniq, UInt64)
)
ENGINE = AggregatingMergeTree()
ORDER BY (event_type, hour);
-- Create materialized view to populate incrementally
CREATE MATERIALIZED VIEW events_hourly_mv TO events_hourly AS
SELECT
event_type,
toStartOfHour(timestamp) as hour,
countState() as events,
uniqState(user_id) as unique_users
FROM events
GROUP BY event_type, hour;
-- Query the pre-aggregated data
SELECT
event_type, hour,
countMerge(events) as events,
uniqMerge(unique_users) as unique_users
FROM events_hourly
WHERE hour >= now() - INTERVAL 7 DAY
GROUP BY event_type, hour;
-- Reads thousands of rows instead of billions
```
**Key points:**
- Use `-State` functions in MV, `-Merge` functions in query
- Incremental - existing data not automatically included (backfill separately)
- Minimal cluster overhead at insert time
Reference: [Use Materialized Views](https://clickhouse.com/docs/best-practices/use-materialized-views)
@@ -0,0 +1,64 @@
---
title: Use Refreshable MVs for Complex Joins and Batch Workflows
impact: HIGH
impactDescription: "Sub-millisecond queries with periodic refresh; ideal for complex joins"
tags: [query, materialized-view, refresh, batch]
---
## Use Refreshable MVs for Complex Joins and Batch Workflows
**Impact: HIGH**
Refreshable MVs execute queries periodically on a schedule. The full query re-executes and overwrites (or appends to) the target table.
**Best for:**
- Sub-millisecond latency where minor staleness is acceptable
- Caching "top N" results or lookup tables
- Complex multi-table joins requiring denormalization
- Batch workflows and DAG dependencies
**Incorrect (expensive join on every request):**
```sql
-- Complex join executed on every request
SELECT
o.order_id, o.total,
c.name as customer_name,
p.name as product_name
FROM orders o
JOIN customers c ON o.customer_id = c.id
JOIN products p ON o.product_id = p.id
WHERE o.created_at >= now() - INTERVAL 1 DAY;
```
**Correct (refreshable MV):**
```sql
-- Create refreshable MV that runs every 5 minutes
CREATE MATERIALIZED VIEW orders_denormalized
REFRESH EVERY 5 MINUTE
ENGINE = MergeTree()
ORDER BY (created_at, order_id)
AS SELECT
o.order_id, o.created_at, o.total,
c.name as customer_name, c.segment,
p.name as product_name
FROM orders o
JOIN customers c ON o.customer_id = c.id
JOIN products p ON o.product_id = p.id
WHERE o.created_at >= now() - INTERVAL 1 DAY;
-- Query the pre-joined data (sub-millisecond)
SELECT * FROM orders_denormalized WHERE segment = 'enterprise';
```
**APPEND vs REPLACE modes:**
| Mode | Behavior | Use Case |
|------|----------|----------|
| `REPLACE` (default) | Overwrites previous contents | Current state, lookup tables |
| `APPEND` | Adds new rows to existing data | Periodic snapshots, historical accumulation |
**Critical warning:** Query should run quickly compared to refresh interval. Don't schedule every 10 seconds if the query takes 10+ seconds.
Reference: [Use Materialized Views](https://clickhouse.com/docs/best-practices/use-materialized-views)
@@ -0,0 +1,76 @@
---
title: Use JSON Type for Dynamic Schemas
impact: MEDIUM
impactDescription: "Field-level querying for semi-structured data; use typed columns for known schemas"
tags: [schema, JSON, semi-structured, flexibility]
---
## Use JSON Type for Dynamic Schemas
**Impact: MEDIUM**
ClickHouse's JSON type splits JSON objects into separate sub-columns, enabling field-level query optimization. Use it for truly dynamic data, not everything.
**Incorrect (schema bloat or opaque String):**
```sql
-- BAD: Hundreds of nullable columns for event properties
CREATE TABLE events (
event_id UUID,
prop_page_url Nullable(String),
prop_button_id Nullable(String),
-- ... 100 more nullable columns
)
-- BAD: JSON as String when you need field queries
CREATE TABLE events (
event_id UUID,
properties String -- No field-level optimization
)
```
**Correct (JSON for dynamic, typed for known):**
```sql
-- Use JSON type for dynamic properties
CREATE TABLE events (
event_id UUID DEFAULT generateUUIDv4(),
event_type LowCardinality(String),
timestamp DateTime DEFAULT now(),
properties JSON -- Flexible schema with type inference
)
ENGINE = MergeTree()
ORDER BY (event_type, timestamp);
-- Query JSON paths directly
SELECT
event_type,
properties.url as page_url,
properties.amount as purchase_amount
FROM events
WHERE event_type = 'page_view' AND properties.url = '/home';
```
**When to use JSON:**
| Scenario | Use JSON? |
|----------|-----------|
| Data structure varies unpredictably | Yes |
| Field types/schemas change over time | Yes |
| Need field-level querying | Yes |
| Fixed, known schema | No (use typed columns) |
| JSON as opaque blob (no field queries) | No (use String) |
**Optimization: specify types for known paths:**
```sql
CREATE TABLE events (
properties JSON(
url String,
amount Float64,
product_id UInt64
)
)
```
Reference: [Use JSON Where Appropriate](https://clickhouse.com/docs/best-practices/use-json-where-appropriate)
@@ -0,0 +1,50 @@
---
title: Use Partitioning for Data Lifecycle Management
impact: HIGH
impactDescription: "DROP PARTITION is instant; DELETE is expensive row-by-row scan"
tags: [schema, partitioning, TTL, data-management]
---
## Use Partitioning for Data Lifecycle Management
**Impact: HIGH**
Partitioning is **primarily a data management technique, not a query optimization tool**. It excels at:
- **Dropping data**: Remove entire partitions as single metadata operations
- **TTL retention**: Implement time-based retention policies efficiently
- **Tiered storage**: Move old partitions to cold storage
- **Archiving**: Move partitions between tables
**Incorrect (no time alignment for lifecycle):**
```sql
-- Cannot efficiently drop old data by time
CREATE TABLE events (...)
ENGINE = MergeTree()
PARTITION BY event_type -- No time alignment
ORDER BY (timestamp);
-- Slow: must scan and delete row by row
DELETE FROM events WHERE timestamp < '2023-01-01';
```
**Correct (time-based for lifecycle):**
```sql
CREATE TABLE events (
timestamp DateTime,
event_type LowCardinality(String)
)
ENGINE = MergeTree()
PARTITION BY toStartOfMonth(timestamp)
ORDER BY (event_type, timestamp)
TTL timestamp + INTERVAL 1 YEAR DELETE; -- Drops whole partitions
-- Fast: metadata-only operation
ALTER TABLE events DROP PARTITION '202301';
-- Archive to cold storage
ALTER TABLE events_archive ATTACH PARTITION '202301' FROM events;
```
Reference: [Choosing a Partitioning Key](https://clickhouse.com/docs/best-practices/choosing-a-partitioning-key)
@@ -0,0 +1,61 @@
---
title: Keep Partition Cardinality Low (100-1,000 Values)
impact: HIGH
impactDescription: "Too many partitions cause part explosion and 'too many parts' errors"
tags: [schema, partitioning, parts]
---
## Keep Partition Cardinality Low (100-1,000 Values)
**Impact: HIGH**
Too many distinct partition values create excessive data parts, eventually triggering "too many parts" errors. ClickHouse enforces limits via `max_parts_in_total` and `parts_to_throw_insert` settings.
**Incorrect (high cardinality partitioning):**
```sql
-- High cardinality = too many partitions
CREATE TABLE events (...)
ENGINE = MergeTree()
PARTITION BY user_id -- Millions of partitions!
ORDER BY (timestamp);
-- Daily partitions can grow unbounded over years
CREATE TABLE logs (...)
ENGINE = MergeTree()
PARTITION BY toDate(timestamp) -- 3650 partitions over 10 years
ORDER BY (service, timestamp);
```
**Correct (bounded cardinality):**
```sql
-- Monthly partitions = 12 per year, bounded cardinality
CREATE TABLE events (
timestamp DateTime,
event_type LowCardinality(String),
user_id UInt64
)
ENGINE = MergeTree()
PARTITION BY toStartOfMonth(timestamp)
ORDER BY (event_type, timestamp);
```
**Validation:**
```sql
-- Check partition count and health
SELECT
partition,
count() as parts,
sum(rows) as rows,
formatReadableSize(sum(bytes_on_disk)) as size
FROM system.parts
WHERE table = 'events' AND active
GROUP BY partition
ORDER BY partition;
-- Warning signs: hundreds or thousands of partitions
```
Reference: [Choosing a Partitioning Key](https://clickhouse.com/docs/best-practices/choosing-a-partitioning-key)
@@ -0,0 +1,35 @@
---
title: Understand Partition Query Performance Trade-offs
impact: MEDIUM
impactDescription: "Partition pruning helps some queries; spanning many partitions hurts others"
tags: [schema, partitioning, query, performance]
---
## Understand Partition Query Performance Trade-offs
**Impact: MEDIUM**
Partitioning can help or hurt query performance:
- **Potential improvement**: Queries filtering by partition key may benefit from partition pruning
- **Potential degradation**: Queries spanning many partitions increase total parts scanned
ClickHouse automatically builds **MinMax indexes** on partition columns. Data merges occur **within partitions only**, not across them.
**Incorrect (query scans all partitions):**
```sql
-- Query must scan all partitions
SELECT count(*) FROM events
WHERE event_type = 'click'; -- No partition pruning
```
**Correct (query prunes to single partition):**
```sql
-- Query prunes to single partition
SELECT count(*) FROM events
WHERE timestamp >= '2024-01-01' AND timestamp < '2024-02-01'
AND event_type = 'click';
```
Reference: [Choosing a Partitioning Key](https://clickhouse.com/docs/best-practices/choosing-a-partitioning-key)
@@ -0,0 +1,42 @@
---
title: Consider Starting Without Partitioning
impact: MEDIUM
impactDescription: "Add partitioning later when you have clear lifecycle requirements"
tags: [schema, partitioning, simplicity]
---
## Consider Starting Without Partitioning
**Impact: MEDIUM**
Start without partitioning and add it later only if:
- You have clear data lifecycle requirements (retention, archiving)
- Your access patterns clearly benefit from partition pruning
- You understand the cardinality implications
**Example (start simple):**
```sql
-- Start simple, no partitioning
CREATE TABLE events (
timestamp DateTime,
event_type LowCardinality(String),
user_id UInt64
)
ENGINE = MergeTree()
ORDER BY (event_type, timestamp);
-- Add partitioning later if needed for lifecycle management
-- (requires table recreation or materialized view migration)
```
**When to add partitioning:**
| Need | Add Partitioning? |
|------|-------------------|
| Time-based data retention | Yes |
| Archive old data to cold storage | Yes |
| Query performance on time ranges | Maybe (test first) |
| No specific lifecycle needs | No |
Reference: [Choosing a Partitioning Key](https://clickhouse.com/docs/best-practices/choosing-a-partitioning-key)
@@ -0,0 +1,45 @@
---
title: Order Columns by Cardinality (Low to High)
impact: CRITICAL
impactDescription: "Enables granule skipping; high-cardinality first prevents index pruning"
tags: [schema, primary-key, cardinality, ORDER BY]
---
## Order Columns by Cardinality (Low to High)
**Impact: CRITICAL**
Since the sparse primary index operates on data blocks (granules) rather than individual rows, low-cardinality leading columns create more useful index entries that can skip entire blocks. Place lower-cardinality columns before higher-cardinality ones in the ordering key.
**Incorrect (high cardinality first):**
```sql
-- UUID first means no pruning benefit
CREATE TABLE events (...)
ENGINE = MergeTree()
ORDER BY (event_id, event_type, timestamp);
-- Every granule has different event_id values, index can't skip anything
```
**Correct (low cardinality first):**
```sql
-- Low cardinality first enables pruning
CREATE TABLE events (...)
ENGINE = MergeTree()
ORDER BY (event_type, event_date, event_id);
-- Index can skip entire event_type groups
```
**Column Order Guidelines:**
| Position | Cardinality | Examples |
|----------|-------------|----------|
| 1st | Low (few distinct values) | event_type, status, country |
| 2nd | Date (coarse granularity) | toDate(timestamp) |
| 3rd+ | Medium-High | user_id, session_id |
| Last | High (if needed) | event_id, uuid |
**Tip:** Use `toDate(timestamp)` instead of raw `DateTime` columns when day-level filtering suffices - this reduces index size from 32-bit to 16-bit representations.
Reference: [Choosing a Primary Key](https://clickhouse.com/docs/best-practices/choosing-a-primary-key)
@@ -0,0 +1,52 @@
---
title: Filter on ORDER BY Columns in Queries
impact: CRITICAL
impactDescription: "Skipping prefix columns prevents index usage"
tags: [schema, primary-key, WHERE, query]
---
## Filter on ORDER BY Columns in Queries
**Impact: CRITICAL**
Even with good schema design, queries must use ORDER BY columns to benefit. Skipping prefix columns or filtering on non-ORDER BY columns prevents index usage.
**Incorrect (skips prefix or uses non-ORDER BY columns):**
```sql
-- Given: ORDER BY (tenant_id, event_type, timestamp)
-- Skips prefix columns - can't use index effectively
SELECT * FROM events WHERE event_type = 'click';
-- Filter on column not in ORDER BY - full table scan
SELECT * FROM events WHERE user_agent LIKE '%Chrome%';
```
**Correct (uses ORDER BY prefix):**
```sql
-- Given: ORDER BY (tenant_id, event_type, timestamp)
-- Full prefix match - best performance
SELECT * FROM events
WHERE tenant_id = 123 AND event_type = 'click';
-- Partial prefix - still uses index
SELECT * FROM events WHERE tenant_id = 123;
-- Range on later column after equality on earlier
SELECT * FROM events
WHERE tenant_id = 123 AND event_type = 'click' AND timestamp >= '2024-01-01';
```
**Index usage reference:**
| Filter | Index Used? |
|--------|-------------|
| `WHERE tenant_id = 123` | Full |
| `WHERE tenant_id = 123 AND event_type = 'click'` | Full |
| `WHERE event_type = 'click'` | None (skipped prefix) |
| `WHERE timestamp > '2024-01-01'` | None (skipped both) |
Reference: [Choosing a Primary Key](https://clickhouse.com/docs/best-practices/choosing-a-primary-key)
@@ -0,0 +1,64 @@
---
title: Plan PRIMARY KEY Before Table Creation
impact: CRITICAL
impactDescription: "ORDER BY is immutable; wrong choice requires full data migration"
tags: [schema, primary-key, ORDER BY]
---
## Plan PRIMARY KEY Before Table Creation
**Impact: CRITICAL** (immutable after creation)
ClickHouse's ORDER BY clause defines physical data ordering and the sparse index. Unlike other databases, **ORDER BY cannot be modified after table creation**. A wrong choice requires creating a new table and migrating all data.
**Incorrect (arbitrary ORDER BY without query analysis):**
```sql
-- Creating table without analyzing query patterns
CREATE TABLE events (
event_id UUID,
user_id UInt64,
timestamp DateTime
)
ENGINE = MergeTree()
ORDER BY (event_id); -- Chosen arbitrarily
-- Later: "Most queries filter by user_id!"
-- Cannot fix with: ALTER TABLE events MODIFY ORDER BY (user_id, timestamp)
-- ERROR: Cannot modify ORDER BY
```
**Correct (query-driven ORDER BY selection):**
```sql
-- Step 1: Document query patterns BEFORE creating table
/*
Query Analysis:
- 60% of queries: WHERE user_id = ? AND timestamp BETWEEN ? AND ?
- 25% of queries: WHERE event_type = ? AND timestamp > ?
- 15% of queries: WHERE event_id = ?
Conclusion: user_id and event_type are primary filters
*/
-- Step 2: Create table with correct ORDER BY
CREATE TABLE events (
event_id UUID DEFAULT generateUUIDv4(),
user_id UInt64,
event_type LowCardinality(String),
timestamp DateTime,
event_date Date DEFAULT toDate(timestamp)
)
ENGINE = MergeTree()
PARTITION BY toYYYYMM(event_date)
ORDER BY (user_id, event_date, event_id);
```
**Pre-creation checklist:**
- [ ] Listed top 5-10 query patterns
- [ ] Identified columns in WHERE clauses with frequency
- [ ] Prioritized columns that exclude large numbers of rows
- [ ] Ordered columns by cardinality (low first, high last)
- [ ] Limited to 4-5 key columns (typically sufficient)
Reference: [Choosing a Primary Key](https://clickhouse.com/docs/best-practices/choosing-a-primary-key)
@@ -0,0 +1,44 @@
---
title: Prioritize Filter Columns in ORDER BY
impact: CRITICAL
impactDescription: "Columns not in ORDER BY cause full table scans"
tags: [schema, primary-key, WHERE, filtering]
---
## Prioritize Filter Columns in ORDER BY
**Impact: CRITICAL**
Prioritize columns frequently used in query filters (WHERE clause), especially those that exclude large numbers of rows. Queries filtering on columns not in ORDER BY result in full table scans.
**Incorrect (ORDER BY doesn't match query patterns):**
```sql
-- If most queries filter by tenant_id:
CREATE TABLE events (...)
ENGINE = MergeTree()
ORDER BY (event_id); -- Queries by tenant_id will full-scan!
```
**Correct (ORDER BY matches filter patterns):**
```sql
-- ORDER BY matches query filter patterns
CREATE TABLE events (...)
ENGINE = MergeTree()
ORDER BY (tenant_id, event_date, event_id);
-- Query now uses primary index:
SELECT * FROM events WHERE tenant_id = 123 AND event_date >= '2024-01-01';
```
**Validation:**
```sql
-- Verify index usage
EXPLAIN indexes = 1
SELECT * FROM events WHERE tenant_id = 123;
-- Look for "PrimaryKey" with Key Condition
```
Reference: [Choosing a Primary Key](https://clickhouse.com/docs/best-practices/choosing-a-primary-key)
@@ -0,0 +1,55 @@
---
title: Avoid Nullable Unless Semantically Required
impact: HIGH
impactDescription: "Nullable adds storage overhead; use DEFAULT values instead"
tags: [schema, data-types, Nullable, DEFAULT]
---
## Avoid Nullable Unless Semantically Required
**Impact: HIGH**
Nullable columns maintain a separate UInt8 column for tracking null values, increasing storage and degrading performance. Use DEFAULT values instead when feasible.
**Incorrect (Nullable everywhere):**
```sql
CREATE TABLE users (
id Nullable(UInt64), -- IDs should never be null
name Nullable(String), -- Empty string is fine
age Nullable(UInt8), -- 0 is a valid default
login_count Nullable(UInt32) -- 0 is a valid default
)
```
**Correct (DEFAULT values, Nullable only when semantic):**
```sql
CREATE TABLE users (
id UInt64, -- Never null
name String DEFAULT '', -- Empty = unknown
age UInt8 DEFAULT 0, -- 0 = unknown
login_count UInt32 DEFAULT 0, -- 0 = never logged in
deleted_at Nullable(DateTime), -- NULL = not deleted (semantic!)
parent_id Nullable(UInt64) -- NULL = no parent (semantic!)
)
```
**When Nullable IS appropriate:**
| Use Case | Why |
|----------|-----|
| `deleted_at` | NULL = "not deleted", timestamp = "deleted at X" |
| `parent_id` | NULL = "no parent", value = "has parent" |
| `discount_percent` | NULL = "no discount", 0 = "0% discount" |
**Defaults instead of Nullable:**
| Type | Default |
|------|---------|
| String | `''` (empty string) |
| UInt*/Int* | `0` |
| DateTime | `now()` or `toDateTime(0)` |
| UUID | `generateUUIDv4()` |
Reference: [Select Data Types](https://clickhouse.com/docs/best-practices/select-data-types)
@@ -0,0 +1,58 @@
---
title: Use Enum for Finite Value Sets
impact: MEDIUM
impactDescription: "Insert-time validation and natural ordering; 1-2 bytes storage"
tags: [schema, data-types, Enum, validation]
---
## Use Enum for Finite Value Sets
**Impact: MEDIUM**
Enum types provide validation at insert time and enable queries that exploit natural ordering. Use Enum8 (up to 256 values) or Enum16 (up to 65,536 values).
**Incorrect (String without validation):**
```sql
CREATE TABLE orders (
status String -- No validation, typos like "shiped" allowed
)
-- Ordering requires CASE statements
SELECT * FROM orders ORDER BY
CASE status
WHEN 'pending' THEN 1
WHEN 'processing' THEN 2
WHEN 'shipped' THEN 3
END;
```
**Correct (Enum with validation and ordering):**
```sql
CREATE TABLE orders (
status Enum8('pending' = 1, 'processing' = 2, 'shipped' = 3, 'delivered' = 4)
)
-- Insert validation: invalid values rejected
INSERT INTO orders VALUES ('shiped'); -- ERROR: Unknown element 'shiped'
-- Natural ordering works automatically
SELECT * FROM orders ORDER BY status; -- Orders by enum value (1, 2, 3, 4)
-- Comparisons use natural order
SELECT * FROM orders WHERE status > 'processing'; -- shipped and delivered
```
**Enum Guidelines:**
| Scenario | Use |
|----------|-----|
| Fixed set of values known at schema time | Enum8/Enum16 |
| Values may change frequently | LowCardinality(String) |
| Need insert-time validation | Enum |
| Need natural ordering in queries | Enum |
| < 256 distinct values | Enum8 (1 byte) |
| 256-65,536 distinct values | Enum16 (2 bytes) |
Reference: [Select Data Types](https://clickhouse.com/docs/best-practices/select-data-types)
@@ -0,0 +1,58 @@
---
title: Use LowCardinality for Repeated Strings
impact: HIGH
impactDescription: "Dictionary encoding for <10K unique values; significant storage reduction"
tags: [schema, data-types, LowCardinality, storage]
---
## Use LowCardinality for Repeated Strings
**Impact: HIGH**
String columns with repeated values store each value repeatedly. LowCardinality uses dictionary encoding for significant storage reduction.
**Incorrect (plain String for repeated values):**
```sql
CREATE TABLE events (
country String, -- "United States" stored 500M times
browser String, -- "Chrome" stored 300M times
event_type String -- "page_view" stored 800M times
)
```
**Correct (LowCardinality for low unique counts):**
```sql
CREATE TABLE events (
country LowCardinality(String), -- ~200 unique values
browser LowCardinality(String), -- ~50 unique values
event_type LowCardinality(String) -- ~100 unique values
)
```
**When to use LowCardinality:**
| Unique Values | Recommendation |
|---------------|----------------|
| < 10,000 | Use LowCardinality |
| > 10,000 | Use regular String |
```sql
-- Check cardinality before deciding
SELECT uniq(column_name) FROM table_name;
```
**LowCardinality vs FixedString:**
Reserve `FixedString` for strictly fixed-length data (e.g., 2-char country codes). For most low-cardinality text, `LowCardinality(String)` outperforms `FixedString`.
```sql
-- FixedString: Only for truly fixed-length data
country_code FixedString(2), -- "US", "DE", "JP" - always 2 chars
-- LowCardinality: For variable-length low-cardinality strings
country_name LowCardinality(String), -- "United States", "Germany"
```
Reference: [Select Data Types](https://clickhouse.com/docs/best-practices/select-data-types)
@@ -0,0 +1,49 @@
---
title: Minimize Bit-Width for Numeric Types
impact: HIGH
impactDescription: "Smaller types reduce storage and improve cache efficiency"
tags: [schema, data-types, numeric, storage]
---
## Minimize Bit-Width for Numeric Types
**Impact: HIGH**
Select the smallest numeric type that accommodates your data range. Prefer unsigned types when negative values aren't needed.
**Incorrect (oversized types):**
```sql
CREATE TABLE metrics (
status_code Int64, -- HTTP codes are 100-599
age Int64, -- Human age fits in UInt8
year Int64, -- Years fit in UInt16
item_count Int64 -- Often small numbers
)
```
**Correct (right-sized types):**
```sql
CREATE TABLE metrics (
status_code UInt16, -- 0-65,535 (HTTP codes fit easily)
age UInt8, -- 0-255 (sufficient for age)
year UInt16, -- 0-65,535 (sufficient for years)
item_count UInt32 -- 0-4 billion (adjust based on actual max)
)
```
**Numeric Type Reference:**
| Type | Range | Bytes |
|------|-------|-------|
| UInt8 | 0 to 255 | 1 |
| UInt16 | 0 to 65,535 | 2 |
| UInt32 | 0 to 4.3 billion | 4 |
| UInt64 | 0 to 18 quintillion | 8 |
| Int8 | -128 to 127 | 1 |
| Int16 | -32,768 to 32,767 | 2 |
| Int32 | -2.1 billion to 2.1 billion | 4 |
| Int64 | -9 quintillion to 9 quintillion | 8 |
Reference: [Select Data Types](https://clickhouse.com/docs/best-practices/select-data-types)
@@ -0,0 +1,51 @@
---
title: Use Native Types Instead of String
impact: CRITICAL
impactDescription: "2-10x storage reduction; enables compression and correct semantics"
tags: [schema, data-types, storage]
---
## Use Native Types Instead of String
**Impact: CRITICAL**
Using String for all data wastes storage, prevents compression optimization, and makes comparisons slower. ClickHouse's column-oriented architecture benefits directly from optimal type selection.
**Incorrect (String for everything):**
```sql
CREATE TABLE events (
event_id String, -- "550e8400-e29b-41d4-a716-446655440000" = 36 bytes
user_id String, -- "12345" = 5 bytes (no numeric operations)
created_at String, -- "2024-01-15 10:30:00" = 19 bytes
count String, -- "42" - can't do math!
is_active String -- "true" = 4 bytes
)
```
**Correct (native types):**
```sql
CREATE TABLE events (
event_id UUID DEFAULT generateUUIDv4(), -- 16 bytes (vs 36)
user_id UInt64, -- 8 bytes, numeric ops
created_at DateTime DEFAULT now(), -- 4 bytes (vs 19)
count UInt32 DEFAULT 0, -- 4 bytes, math works
is_active Bool DEFAULT true -- 1 byte (vs 4)
)
```
**Type Selection Quick Reference:**
| Data | Use | Avoid |
|------|-----|-------|
| Sequential IDs | UInt32/UInt64 | String |
| UUIDs | UUID | String |
| Status/Category | Enum8 or LowCardinality(String) | String |
| Timestamps | DateTime | DateTime64, String |
| Dates only | Date or Date32 | DateTime, String |
| Counts | UInt8/16/32 (smallest that fits) | Int64, String |
| Money | Decimal(P,S) or Int64 (cents) | Float64, String |
| Booleans | Bool or UInt8 | String |
Reference: [Select Data Types](https://clickhouse.com/docs/best-practices/select-data-types)
+54
View File
@@ -0,0 +1,54 @@
---
name: code-review
description: |
Shared code review workflow for Langfuse. Use when reviewing a PR, branch, diff,
or local changes for correctness, regressions, risk, and missing tests.
Start with references/review-checklist.md for repo-specific review rules and
use package AGENTS.md files plus any matching shared skills when the change
touches those areas.
---
# Code Review
Use this skill when the task is to review code changes rather than implement a
feature.
## Start Here
- Read [`references/review-checklist.md`](references/review-checklist.md) for
the repo's canonical review rules.
- Read root [`AGENTS.md`](../../../AGENTS.md) and the nearest package
`AGENTS.md` for the files under review.
- If the review touches ClickHouse, also use the shared
`clickhouse-best-practices` skill.
- If the review touches backend code, also use the shared
`backend-dev-guidelines` skill where relevant.
## Review Priorities
Focus on:
- correctness bugs
- behavioral regressions
- security and tenant-isolation risks
- performance issues with real impact
- missing or weak tests for risky changes
## Output Expectations
- Findings first, ordered by severity
- File and line references for each finding
- Short summary only after findings
- If no findings, say so explicitly and mention any residual risk or coverage gaps
## Scope Guidance
Use `references/review-checklist.md` for Langfuse-specific checks such as:
- ClickHouse and Postgres migration expectations
- project-scoped tenant isolation checks
- API/Fern consistency
- banner-offset UI positioning
- environment variable access patterns
Do not duplicate those rules in ad hoc prompts or tool-specific command files.
@@ -1,4 +1,6 @@
# Code Review Instructions
# Langfuse Review Checklist
This is the canonical shared review checklist for Langfuse.
## Database Migrations
@@ -0,0 +1,60 @@
---
name: frontend-browser-review
description: |
Shared workflow for browser-based review of user-visible frontend changes in Langfuse.
Use when a change affects UI behavior, layout, styling, navigation, or browser-visible
regressions and should be checked with the Playwright MCP server before signoff.
---
# Frontend Browser Review
Use this skill when a change affects what users see or do in the browser.
## Start Here
- Read [`../../../web/AGENTS.md`](../../../web/AGENTS.md) for web-specific
entry points and test commands.
- Use the workspace `playwright` MCP server configured from the repo-owned
shared agent setup.
## When To Use It
- UI changes in `web/**`
- Layout, styling, or responsive behavior changes
- Changes to navigation or page flows
- Bug fixes where the failure mode is visible in the browser
- Final signoff for user-visible frontend work
## Review Loop
1. Start the app with `pnpm run dev:web` unless an existing local server is
already running.
2. Install Chromium with `pnpm run playwright:install` if Playwright has not
been set up on the machine yet.
3. Open the primary changed flow with the Playwright MCP server.
4. Exercise the main happy path affected by the change.
5. Check for obvious visual regressions:
- broken layout or spacing
- banner overlap or viewport anchoring issues
- missing loading, empty, or error states
- broken responsive behavior on narrow widths
6. If the page changed materially, inspect the resulting UI state and compare
it against the intended behavior from the task or existing patterns.
7. If the browser session fails, inspect traces and artifacts under
`.playwright-mcp/`.
## Output Expectations
Report:
1. What flow you reviewed
2. Whether the primary flow worked
3. Any visible regressions or follow-up risks
4. If review was blocked, exactly what prevented browser verification
## Scope Notes
- This skill complements, not replaces, targeted tests and linting.
- For implementation details, stay in `web/AGENTS.md` and package-local skills.
- Use this as the browser-signoff workflow, not as a generic frontend coding
guide.
@@ -0,0 +1,72 @@
# PNPM Upgrade Package
Use this workflow when a user wants to upgrade a dependency in the Langfuse
pnpm workspace.
## Workflow
1. Collect missing inputs.
- Ask for the package if missing.
- Ask for the target version if missing.
- If the user says `latest`, resolve the real registry latest first.
2. Run the main helper once.
- Run
`node .agents/skills/pnpm-upgrade-package/scripts/check-release-age-window.mjs <package> <targetVersion>`.
- Treat this as the single source of truth for:
- direct workspace references
- root `pnpm.overrides` / `pnpm.patchedDependencies`
- latest registry version
- latest version installable under the current release-age rules
- existing matching `minimumReleaseAgeExclude` entries
- exact dependency companions from `dependencies` and `optionalDependencies`
- exact peer dependencies that are actually installed in the workspace
3. Handle the transitive-only case before editing anything.
- If the helper shows no direct workspace references, run `pnpm why -r <package>`.
- Identify the current top-level parent that pulls the package in.
- Check whether that parent's current dependency range already permits the
requested transitive version.
- If the current parent range already covers the requested version, prefer a
lock refresh / reinstall path before changing `package.json`.
- If the current parent range does not cover the requested version, upgrade
the direct parent dependency that pulls the package in.
- If a compatible transitive package still stays pinned after the normal
refresh path, you may suggest `pnpm dedupe` to the user as an optional
manual follow-up, but do not run it automatically and do not require it.
- Do not add the transitive package directly unless the user explicitly asks.
4. Ask before changing `minimumReleaseAgeExclude`.
- Prefer `package@version` entries.
- Only use bare `package` entries after explicit approval.
- Ask about exact companion packages only when the helper says they still
need a new exclusion.
- Treat range-based dependency or peer entries as manual review.
5. Bump at the narrowest useful scope.
- `pnpm -w up <package>@<version>` for root-only changes.
- `pnpm --filter <workspace> up <package>@<version>` for one workspace.
- `pnpm -r up <package>@<version>` only when every current reference should move.
- Do not hand-edit `pnpm-lock.yaml`.
6. Validate.
- Use the nearest package `AGENTS.md` plus the root verification matrix.
- Finish with `pnpm why -r <package>`.
- If companions moved too, run `pnpm why -r <companion-package>` for them as well.
## Quick Commands
- Run the single analysis pass:
`node .agents/skills/pnpm-upgrade-package/scripts/check-release-age-window.mjs <package> <targetVersion>`
- Transitive provenance check:
`pnpm why -r <package>`
- Inspect the current parent manifest on the registry:
`npm view <parent>@<installedVersion> dependencies peerDependencies optionalDependencies --json`
- Final graph verification:
`pnpm why -r <package>`
- Bump in the root workspace:
`pnpm -w up <package>@<version>`
- Bump in one workspace:
`pnpm --filter web up <package>@<version>`
- Bump everywhere that should move together:
`pnpm -r up <package>@<version>`
@@ -0,0 +1,42 @@
---
name: pnpm-upgrade-package
description: Use when upgrading a dependency in this pnpm workspace, including requests to bump a package to a specific version, compare the registry latest version with the latest version installable under the current minimum-release-age window, or decide whether minimumReleaseAgeExclude in pnpm-workspace.yaml must change. Ask the user for the package name or target version when either is missing.
---
# PNPM Upgrade Package
Use this skill for interactive dependency bumps in Langfuse.
## Read Order
- Start with [AGENTS.md](AGENTS.md) for the end-to-end workflow.
- Run the main helper once at the start of the upgrade:
`node .agents/skills/pnpm-upgrade-package/scripts/check-release-age-window.mjs <package> [targetVersion]`
## Apply This Skill
- Ask for the package name if the user did not provide one.
- Ask for the target version if the user did not provide one.
- Run the main helper once as the first analysis step and use that single
output for scope, exclusion decisions, and the final bump.
- If the target package is not directly declared anywhere, run
`pnpm why -r <package>` to find which direct dependency brings it in, then
inspect whether the current top-level parent already allows the requested
transitive version via its dependency range.
- If the current parent range already covers the requested transitive version,
prefer a lockfile refresh / reinstall path over bumping the parent manifest.
- If the current parent range does not cover the requested transitive version,
upgrade that parent dependency instead of adding the target package directly
unless the user explicitly wants that.
- If a compatible transitive package still stays pinned after the normal
refresh path, you may suggest `pnpm dedupe` to the user as an optional manual
follow-up, but do not run it automatically and do not require it.
- Resolve the registry latest version, but do not silently upgrade to latest
unless the user asked for latest.
- Compare the target version with the latest version installable under the
current `minimumReleaseAge` window.
- Ask before adding `minimumReleaseAgeExclude` entries for the target package,
exact dependency companions from `dependencies` or `optionalDependencies`, or
locally installed exact peer dependencies.
- Finish with `pnpm why -r <package>` to confirm that only the intended version
remains in the workspace.
@@ -0,0 +1,4 @@
interface:
display_name: "PNPM Upgrade Package"
short_description: "Interactive pnpm package bump workflow"
default_prompt: "Use $pnpm-upgrade-package to upgrade a package in this pnpm workspace, asking me for the package or version if I did not provide them."
@@ -0,0 +1,418 @@
#!/usr/bin/env node
import { join } from "node:path";
import {
entryCoversVersion,
findLocalPackageReferences,
formatWorkspaceReference,
getRootPnpmControls,
readWorkspaceConfig,
} from "./lib/workspace-utils.mjs";
const args = process.argv.slice(2);
const asJson = args.includes("--json");
const positional = args.filter((arg) => !arg.startsWith("--"));
const packageName = positional[0];
const requestedTargetVersion = positional[1] ?? null;
if (!packageName) {
console.error(
"Usage: node .agents/skills/pnpm-upgrade-package/scripts/check-release-age-window.mjs <package> [targetVersion] [--json]",
);
process.exit(1);
}
const repoRoot = process.cwd();
const workspaceConfig = readWorkspaceConfig(join(repoRoot, "pnpm-workspace.yaml"));
const minimumReleaseAgeMinutes = workspaceConfig.minimumReleaseAge ?? 0;
const thresholdMs = Date.now() - minimumReleaseAgeMinutes * 60 * 1000;
const REGISTRY_FETCH_TIMEOUT_MS = 30_000;
const registryCache = new Map();
const workspaceReferenceCache = new Map();
const getWorkspaceReferences = (name) => {
if (!workspaceReferenceCache.has(name)) {
workspaceReferenceCache.set(name, findLocalPackageReferences(repoRoot, name));
}
return workspaceReferenceCache.get(name);
};
function printSectionHeader(title) {
console.log("");
console.log(title);
}
function isPrerelease(version) {
return version.includes("-");
}
function isExactVersion(spec) {
return /^\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?$/.test(spec.trim());
}
function getMatchingExcludeEntries(name, version) {
return workspaceConfig.minimumReleaseAgeExclude.filter((entry) =>
entryCoversVersion(entry, name, version),
);
}
async function fetchRegistryPackage(name) {
if (registryCache.has(name)) return registryCache.get(name);
const abortController = new AbortController();
const timeoutId = setTimeout(
() => abortController.abort(),
REGISTRY_FETCH_TIMEOUT_MS,
);
timeoutId.unref?.();
try {
const response = await fetch(
`https://registry.npmjs.org/${encodeURIComponent(name)}`,
{
headers: {
accept: "application/json",
"user-agent": "langfuse-pnpm-upgrade-package-skill",
},
signal: abortController.signal,
},
);
if (!response.ok) {
throw new Error(
`Failed to fetch ${name} from npm registry: ${response.status}`,
);
}
const metadata = await response.json();
registryCache.set(name, metadata);
return metadata;
} catch (error) {
if (error?.name === "AbortError") {
throw new Error(
`Timed out fetching ${name} from npm registry after ${REGISTRY_FETCH_TIMEOUT_MS}ms`,
{ cause: error },
);
}
throw error;
} finally {
clearTimeout(timeoutId);
}
}
function getInstallability(metadata, name, version) {
const publishedAt = metadata.time?.[version] ?? null;
const publishedAtMs = publishedAt ? Date.parse(publishedAt) : null;
const matchingExcludeEntries = getMatchingExcludeEntries(name, version);
const isYoungerThanMinimumReleaseAge =
publishedAtMs != null ? publishedAtMs > thresholdMs : null;
const isInstallableWithoutNewExclude =
isYoungerThanMinimumReleaseAge == null
? null
: !isYoungerThanMinimumReleaseAge || matchingExcludeEntries.length > 0;
return {
name,
version,
publishedAt,
isYoungerThanMinimumReleaseAge,
isInstallableWithoutNewExclude,
matchingExcludeEntries,
suggestedExclude:
isInstallableWithoutNewExclude === false ? `${name}@${version}` : null,
};
}
function selectLatestInstallableVersion(metadata, name) {
const times = metadata.time ?? {};
return (
Object.keys(metadata.versions ?? {})
.filter((version) => times[version] && !isPrerelease(version))
.sort((left, right) => Date.parse(times[right]) - Date.parse(times[left]))
.map((version) => getInstallability(metadata, name, version))
.find((candidate) => candidate.isInstallableWithoutNewExclude) ?? null
);
}
function collectManifestEntries(manifest, fields) {
const merged = new Map();
for (const field of fields) {
for (const [name, spec] of Object.entries(manifest[field] ?? {})) {
const key = `${name}:${spec}`;
const entry = merged.get(key);
if (entry) {
entry.fields.push(field);
continue;
}
merged.set(key, { name, spec, fields: [field] });
}
}
return [...merged.values()].sort((left, right) =>
left.name.localeCompare(right.name),
);
}
async function analyzeManifestEntries(entries, { includeWorkspace = false } = {}) {
const exact = [];
const range = [];
for (const entry of entries) {
const workspaceReferences = includeWorkspace
? getWorkspaceReferences(entry.name)
: null;
if (!isExactVersion(entry.spec)) {
range.push({
...entry,
...(includeWorkspace ? { workspaceReferences } : {}),
});
continue;
}
const metadata = await fetchRegistryPackage(entry.name);
const installability = getInstallability(metadata, entry.name, entry.spec);
exact.push({
...entry,
...installability,
...(includeWorkspace
? {
workspaceReferences,
isInstalledInWorkspace: workspaceReferences.length > 0,
}
: {}),
suggestedExclude:
includeWorkspace && workspaceReferences.length === 0
? null
: installability.suggestedExclude,
});
}
return { exact, range };
}
function printWorkspaceReferences(title, references) {
printSectionHeader(title);
if (references.length === 0) {
console.log("- none");
return;
}
for (const reference of references) {
console.log(`- ${formatWorkspaceReference(reference)}`);
}
}
function printRootPnpmControls(rootPnpm) {
printSectionHeader("Root pnpm controls:");
if (
rootPnpm.overrideMatches.length === 0 &&
rootPnpm.patchedDependencyMatches.length === 0
) {
console.log("- none");
return;
}
for (const match of rootPnpm.overrideMatches) {
console.log(`- override ${match.selector}: ${match.value}`);
}
for (const match of rootPnpm.patchedDependencyMatches) {
console.log(`- patched dependency ${match.selector}: ${match.value}`);
}
}
function printVersionEntries(title, entries, { includeWorkspace = false } = {}) {
printSectionHeader(title);
if (entries.length === 0) {
console.log("- none");
return;
}
for (const entry of entries) {
const status =
entry.isInstallableWithoutNewExclude == null
? "unknown"
: entry.isInstallableWithoutNewExclude
? "installable now"
: "needs exclude";
console.log(
`- ${entry.name}@${entry.version} (${status}; via ${entry.fields.join(", ")})`,
);
if (entry.publishedAt) {
console.log(` published at: ${entry.publishedAt}`);
}
if (includeWorkspace) {
console.log(
` installed in workspace: ${entry.isInstalledInWorkspace ? "yes" : "no"}`,
);
for (const reference of entry.workspaceReferences) {
console.log(` workspace reference: ${formatWorkspaceReference(reference)}`);
}
}
if (entry.matchingExcludeEntries.length > 0) {
console.log(
` matching exclude entries: ${entry.matchingExcludeEntries.join(", ")}`,
);
}
if (entry.suggestedExclude) {
console.log(` suggested exclude: ${entry.suggestedExclude}`);
}
}
}
function printRangeEntries(title, entries) {
printSectionHeader(title);
if (entries.length === 0) {
console.log("- none");
return;
}
for (const entry of entries) {
console.log(
`- ${entry.name}: ${entry.spec} (manual review; via ${entry.fields.join(", ")})`,
);
for (const reference of entry.workspaceReferences ?? []) {
console.log(` workspace reference: ${formatWorkspaceReference(reference)}`);
}
}
}
const packageMetadata = await fetchRegistryPackage(packageName);
const latestVersion = packageMetadata["dist-tags"]?.latest ?? null;
const targetVersion = requestedTargetVersion ?? latestVersion;
if (!targetVersion) {
console.error(`Could not resolve a target version for ${packageName}.`);
process.exit(1);
}
if (!packageMetadata.versions?.[targetVersion]) {
console.error(`Version ${targetVersion} was not found for ${packageName}.`);
process.exit(1);
}
const packageWorkspaceReferences = getWorkspaceReferences(packageName);
const rootPnpm = getRootPnpmControls(repoRoot, packageName);
const latestInstallableWithoutNewExclude = selectLatestInstallableVersion(
packageMetadata,
packageName,
);
const targetInstallability = getInstallability(
packageMetadata,
packageName,
targetVersion,
);
const targetManifest = packageMetadata.versions[targetVersion];
const dependencyCompanions = await analyzeManifestEntries(
collectManifestEntries(targetManifest, [
"dependencies",
"optionalDependencies",
]),
);
const peerDependencies = await analyzeManifestEntries(
collectManifestEntries(targetManifest, ["peerDependencies"]),
{ includeWorkspace: true },
);
const result = {
packageName,
targetVersion,
targetWasExplicitlyProvided: requestedTargetVersion != null,
packageWorkspaceReferences,
rootPnpm,
minimumReleaseAgeMinutes,
thresholdIso: new Date(thresholdMs).toISOString(),
latestRegistryVersion: latestVersion,
latestRegistryPublishedAt:
latestVersion != null ? packageMetadata.time?.[latestVersion] ?? null : null,
latestInstallableWithoutNewExclude,
targetPublishedAt: targetInstallability.publishedAt,
targetIsYoungerThanMinimumReleaseAge:
targetInstallability.isYoungerThanMinimumReleaseAge,
targetIsInstallableWithoutNewExclude:
targetInstallability.isInstallableWithoutNewExclude,
matchingPackageExcludeEntries: targetInstallability.matchingExcludeEntries,
suggestedPackageExclude: targetInstallability.suggestedExclude,
exactDependencyCompanions: dependencyCompanions.exact,
rangeDependencyCompanions: dependencyCompanions.range,
exactPeerDependencies: peerDependencies.exact,
rangePeerDependencies: peerDependencies.range,
};
if (asJson) {
console.log(JSON.stringify(result, null, 2));
process.exit(0);
}
console.log(`Package: ${packageName}`);
console.log(
`Target version: ${targetVersion}${
requestedTargetVersion
? ""
: " (resolved latest; still ask before bumping if version was omitted)"
}`,
);
printWorkspaceReferences(
"Target package workspace references:",
packageWorkspaceReferences,
);
printRootPnpmControls(rootPnpm);
printSectionHeader("Release-age window:");
console.log(`minimumReleaseAge: ${minimumReleaseAgeMinutes} minutes`);
console.log(`Threshold: ${result.thresholdIso}`);
console.log(`Latest registry version: ${result.latestRegistryVersion ?? "unknown"}`);
if (result.latestRegistryPublishedAt) {
console.log(`Latest registry published at: ${result.latestRegistryPublishedAt}`);
}
if (latestInstallableWithoutNewExclude) {
console.log(
`Latest installable without new exclude: ${latestInstallableWithoutNewExclude.version} (${latestInstallableWithoutNewExclude.publishedAt})`,
);
} else {
console.log("Latest installable without new exclude: none found");
}
console.log(`Target published at: ${result.targetPublishedAt ?? "unknown"}`);
console.log(
`Target installable without new exclude: ${
result.targetIsInstallableWithoutNewExclude == null
? "unknown"
: result.targetIsInstallableWithoutNewExclude
? "yes"
: "no"
}`,
);
if (result.matchingPackageExcludeEntries.length > 0) {
console.log("Matching package exclude entries:");
for (const entry of result.matchingPackageExcludeEntries) {
console.log(`- ${entry}`);
}
} else {
console.log("Matching package exclude entries: none");
}
if (result.suggestedPackageExclude) {
console.log(`Suggested package exclude: ${result.suggestedPackageExclude}`);
}
printVersionEntries(
"Exact dependency companions (dependencies + optionalDependencies):",
result.exactDependencyCompanions,
);
printRangeEntries(
"Range dependency companions (dependencies + optionalDependencies):",
result.rangeDependencyCompanions,
);
printVersionEntries("Exact peer dependencies:", result.exactPeerDependencies, {
includeWorkspace: true,
});
printRangeEntries("Range peer dependencies:", result.rangePeerDependencies);
@@ -0,0 +1,208 @@
import { existsSync, readFileSync, readdirSync } from "node:fs";
import { join, relative } from "node:path";
const packageFields = [
"dependencies",
"devDependencies",
"peerDependencies",
"optionalDependencies",
];
export function formatWorkspaceReference(reference) {
const label = reference.workspaceName
? `${reference.path} (${reference.workspaceName})`
: reference.path;
const specs = reference.matches
.map((match) => `${match.field}: ${match.spec}`)
.join(", ");
return `${label} -> ${specs}`;
}
export function readJson(path) {
return JSON.parse(readFileSync(path, "utf8"));
}
function stripInlineComment(line) {
let quote = null;
let escaped = false;
for (let index = 0; index < line.length; index += 1) {
const char = line[index];
if (escaped) {
escaped = false;
continue;
}
if (quote) {
if (char === "\\") {
escaped = true;
continue;
}
if (char === quote) {
quote = null;
}
continue;
}
if (char === "'" || char === '"') {
quote = char;
continue;
}
if (char === "#") {
return line.slice(0, index).trimEnd();
}
}
return line;
}
export function readWorkspaceConfig(path) {
const raw = readFileSync(path, "utf8");
const lines = raw.split(/\r?\n/);
let minimumReleaseAge = 0;
const minimumReleaseAgeExclude = [];
let inExcludeBlock = false;
for (const line of lines) {
const uncommented = stripInlineComment(line);
const trimmed = uncommented.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const ageMatch = trimmed.match(/^minimumReleaseAge:\s*(\d+)\s*$/);
if (ageMatch) {
minimumReleaseAge = Number(ageMatch[1]);
continue;
}
if (/^minimumReleaseAgeExclude:\s*$/.test(trimmed)) {
inExcludeBlock = true;
continue;
}
if (inExcludeBlock) {
const excludeMatch = uncommented.match(/^\s*-\s+(.+?)\s*$/);
if (excludeMatch) {
minimumReleaseAgeExclude.push(
excludeMatch[1].replace(/^['"]|['"]$/g, ""),
);
continue;
}
if (/^\S/.test(uncommented)) {
inExcludeBlock = false;
}
}
}
return { minimumReleaseAge, minimumReleaseAgeExclude };
}
export function collectPackageJsonPaths(repoRoot) {
const paths = [
"package.json",
"web/package.json",
"worker/package.json",
"ee/package.json",
];
const packagesRoot = join(repoRoot, "packages");
if (!existsSync(packagesRoot)) return paths;
const stack = [packagesRoot];
while (stack.length > 0) {
const current = stack.pop();
for (const entry of readdirSync(current, { withFileTypes: true })) {
if (
entry.name === "node_modules" ||
entry.name === "dist" ||
entry.name === ".git"
) {
continue;
}
const nextPath = join(current, entry.name);
if (entry.isDirectory()) {
stack.push(nextPath);
continue;
}
if (entry.isFile() && entry.name === "package.json") {
paths.push(relative(repoRoot, nextPath));
}
}
}
return [...new Set(paths)];
}
export function matchesPackageSelector(selector, wantedPackage) {
if (selector === wantedPackage) return true;
if (selector.startsWith(`${wantedPackage}@`)) return true;
if (selector.endsWith(`>${wantedPackage}`)) return true;
if (selector.includes(`>${wantedPackage}@`)) return true;
if (selector.endsWith("/*")) {
const prefix = selector.slice(0, -1);
return wantedPackage.startsWith(prefix);
}
return false;
}
export function entryCoversVersion(entry, wantedPackage, wantedVersion) {
if (entry === wantedPackage) return true;
if (entry.endsWith("/*")) {
const prefix = entry.slice(0, -1);
return wantedPackage.startsWith(prefix);
}
if (!entry.startsWith(`${wantedPackage}@`)) return false;
return entry
.slice(wantedPackage.length + 1)
.split("||")
.map((part) => part.trim())
.includes(wantedVersion);
}
export function findLocalPackageReferences(repoRoot, wantedPackage) {
const results = [];
for (const packageJsonPath of collectPackageJsonPaths(repoRoot)) {
const json = readJson(join(repoRoot, packageJsonPath));
const matches = [];
for (const field of packageFields) {
if (json[field]?.[wantedPackage]) {
matches.push({ field, spec: json[field][wantedPackage] });
}
}
if (matches.length > 0) {
results.push({
path: packageJsonPath,
workspaceName: json.name ?? null,
matches,
});
}
}
return results;
}
export function getRootPnpmControls(repoRoot, packageName) {
const rootPackageJson = readJson(join(repoRoot, "package.json"));
return {
overrideMatches: Object.entries(rootPackageJson.pnpm?.overrides ?? {})
.filter(([selector]) => matchesPackageSelector(selector, packageName))
.map(([selector, value]) => ({ selector, value })),
patchedDependencyMatches: Object.entries(
rootPackageJson.pnpm?.patchedDependencies ?? {},
)
.filter(([selector]) => matchesPackageSelector(selector, packageName))
.map(([selector, value]) => ({ selector, value })),
};
}
+951
View File
@@ -0,0 +1,951 @@
---
name: turborepo
description: |
Turborepo monorepo build system guidance. Triggers on: turbo.json, task pipelines,
dependsOn, caching, remote cache, the "turbo" CLI, --filter, --affected, CI optimization, environment
variables, internal packages, monorepo structure/best practices, and boundaries.
Use when user: configures tasks/workflows/pipelines, creates packages, sets up
monorepo, shares code between apps, runs changed/affected packages, debugs cache,
or has apps/packages directories.
metadata:
version: 2.8.21-canary.9
---
# Turborepo Skill
Build system for JavaScript/TypeScript monorepos. Turborepo caches task outputs and runs tasks in parallel based on dependency graph.
## IMPORTANT: Package Tasks, Not Root Tasks
**DO NOT create Root Tasks. ALWAYS create package tasks.**
When creating tasks/scripts/pipelines, you MUST:
1. Add the script to each relevant package's `package.json`
2. Register the task in root `turbo.json`
3. Root `package.json` only delegates via `turbo run <task>`
**DO NOT** put task logic in root `package.json`. This defeats Turborepo's parallelization.
```json
// DO THIS: Scripts in each package
// apps/web/package.json
{ "scripts": { "build": "next build", "lint": "eslint .", "test": "vitest" } }
// apps/api/package.json
{ "scripts": { "build": "tsc", "lint": "eslint .", "test": "vitest" } }
// packages/ui/package.json
{ "scripts": { "build": "tsc", "lint": "eslint .", "test": "vitest" } }
```
```json
// turbo.json - register tasks
{
"tasks": {
"build": { "dependsOn": ["^build"], "outputs": ["dist/**"] },
"lint": {},
"test": { "dependsOn": ["build"] }
}
}
```
```json
// Root package.json - ONLY delegates, no task logic
{
"scripts": {
"build": "turbo run build",
"lint": "turbo run lint",
"test": "turbo run test"
}
}
```
```json
// DO NOT DO THIS - defeats parallelization
// Root package.json
{
"scripts": {
"build": "cd apps/web && next build && cd ../api && tsc",
"lint": "eslint apps/ packages/",
"test": "vitest"
}
}
```
Root Tasks (`//#taskname`) are ONLY for tasks that truly cannot exist in packages (rare).
## Secondary Rule: `turbo run` vs `turbo`
**Always use `turbo run` when the command is written into code:**
```json
// package.json - ALWAYS "turbo run"
{
"scripts": {
"build": "turbo run build"
}
}
```
```yaml
# CI workflows - ALWAYS "turbo run"
- run: turbo run build --affected
```
**The shorthand `turbo <tasks>` is ONLY for one-off terminal commands** typed directly by humans or agents. Never write `turbo build` into package.json, CI, or scripts.
## Quick Decision Trees
### "I need to configure a task"
```
Configure a task?
├─ Define task dependencies → references/configuration/tasks.md
├─ Lint/check-types (parallel + caching) → Use Transit Nodes pattern (see below)
├─ Specify build outputs → references/configuration/tasks.md#outputs
├─ Handle environment variables → references/environment/RULE.md
├─ Set up dev/watch tasks → references/configuration/tasks.md#persistent
├─ Package-specific config → references/configuration/RULE.md#package-configurations
└─ Global settings (cacheDir, daemon) → references/configuration/global-options.md
```
### "My cache isn't working"
```
Cache problems?
├─ Tasks run but outputs not restored → Missing `outputs` key
├─ Cache misses unexpectedly → references/caching/gotchas.md
├─ Need to debug hash inputs → Use --summarize or --dry
├─ Want to skip cache entirely → Use --force or cache: false
├─ Remote cache not working → references/caching/remote-cache.md
└─ Environment causing misses → references/environment/gotchas.md
```
### "I want to run only changed packages"
```
Run only what changed?
├─ Changed packages + dependents (RECOMMENDED) → turbo run build --affected
├─ Custom base branch → --affected --affected-base=origin/develop
├─ Manual git comparison → --filter=...[origin/main]
└─ See all filter options → references/filtering/RULE.md
```
**`--affected` is the primary way to run only changed packages.** It automatically compares against the default branch and includes dependents.
### "I want to filter packages"
```
Filter packages?
├─ Only changed packages → --affected (see above)
├─ By package name → --filter=web
├─ By directory → --filter=./apps/*
├─ Package + dependencies → --filter=web...
├─ Package + dependents → --filter=...web
└─ Complex combinations → references/filtering/patterns.md
```
### "Environment variables aren't working"
```
Environment issues?
├─ Vars not available at runtime → Strict mode filtering (default)
├─ Cache hits with wrong env → Var not in `env` key
├─ .env changes not causing rebuilds → .env not in `inputs`
├─ CI variables missing → references/environment/gotchas.md
└─ Framework vars (NEXT_PUBLIC_*) → Auto-included via inference
```
### "I need to set up CI"
```
CI setup?
├─ GitHub Actions → references/ci/github-actions.md
├─ Vercel deployment → references/ci/vercel.md
├─ Remote cache in CI → references/caching/remote-cache.md
├─ Only build changed packages → --affected flag
├─ Skip unnecessary builds → turbo-ignore (references/cli/commands.md)
└─ Skip container setup when no changes → turbo-ignore
```
### "I want to watch for changes during development"
```
Watch mode?
├─ Re-run tasks on change → turbo watch (references/watch/RULE.md)
├─ Dev servers with dependencies → Use `with` key (references/configuration/tasks.md#with)
├─ Restart dev server on dep change → Use `interruptible: true`
└─ Persistent dev tasks → Use `persistent: true`
```
### "I need to create/structure a package"
```
Package creation/structure?
├─ Create an internal package → references/best-practices/packages.md
├─ Repository structure → references/best-practices/structure.md
├─ Dependency management → references/best-practices/dependencies.md
├─ Best practices overview → references/best-practices/RULE.md
├─ JIT vs Compiled packages → references/best-practices/packages.md#compilation-strategies
└─ Sharing code between apps → references/best-practices/RULE.md#package-types
```
### "How should I structure my monorepo?"
```
Monorepo structure?
├─ Standard layout (apps/, packages/) → references/best-practices/RULE.md
├─ Package types (apps vs libraries) → references/best-practices/RULE.md#package-types
├─ Creating internal packages → references/best-practices/packages.md
├─ TypeScript configuration → references/best-practices/structure.md#typescript-configuration
├─ ESLint configuration → references/best-practices/structure.md#eslint-configuration
├─ Dependency management → references/best-practices/dependencies.md
└─ Enforce package boundaries → references/boundaries/RULE.md
```
### "I want to enforce architectural boundaries"
```
Enforce boundaries?
├─ Check for violations → turbo boundaries
├─ Tag packages → references/boundaries/RULE.md#tags
├─ Restrict which packages can import others → references/boundaries/RULE.md#rule-types
└─ Prevent cross-package file imports → references/boundaries/RULE.md
```
## Critical Anti-Patterns
### Using `turbo` Shorthand in Code
**`turbo run` is recommended in package.json scripts and CI pipelines.** The shorthand `turbo <task>` is intended for interactive terminal use.
```json
// WRONG - using shorthand in package.json
{
"scripts": {
"build": "turbo build",
"dev": "turbo dev"
}
}
// CORRECT
{
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev"
}
}
```
```yaml
# WRONG - using shorthand in CI
- run: turbo build --affected
# CORRECT
- run: turbo run build --affected
```
### Root Scripts Bypassing Turbo
Root `package.json` scripts MUST delegate to `turbo run`, not run tasks directly.
```json
// WRONG - bypasses turbo entirely
{
"scripts": {
"build": "bun build",
"dev": "bun dev"
}
}
// CORRECT - delegates to turbo
{
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev"
}
}
```
### Using `&&` to Chain Turbo Tasks
Don't chain turbo tasks with `&&`. Let turbo orchestrate.
```json
// WRONG - turbo task not using turbo run
{
"scripts": {
"changeset:publish": "bun build && changeset publish"
}
}
// CORRECT
{
"scripts": {
"changeset:publish": "turbo run build && changeset publish"
}
}
```
### `prebuild` Scripts That Manually Build Dependencies
Scripts like `prebuild` that manually build other packages bypass Turborepo's dependency graph.
```json
// WRONG - manually building dependencies
{
"scripts": {
"prebuild": "cd ../../packages/types && bun run build && cd ../utils && bun run build",
"build": "next build"
}
}
```
**However, the fix depends on whether workspace dependencies are declared:**
1. **If dependencies ARE declared** (e.g., `"@repo/types": "workspace:*"` in package.json), remove the `prebuild` script. Turbo's `dependsOn: ["^build"]` handles this automatically.
2. **If dependencies are NOT declared**, the `prebuild` exists because `^build` won't trigger without a dependency relationship. The fix is to:
- Add the dependency to package.json: `"@repo/types": "workspace:*"`
- Then remove the `prebuild` script
```json
// CORRECT - declare dependency, let turbo handle build order
// package.json
{
"dependencies": {
"@repo/types": "workspace:*",
"@repo/utils": "workspace:*"
},
"scripts": {
"build": "next build"
}
}
// turbo.json
{
"tasks": {
"build": {
"dependsOn": ["^build"]
}
}
}
```
**Key insight:** `^build` only runs build in packages listed as dependencies. No dependency declaration = no automatic build ordering.
### Overly Broad `globalDependencies`
`globalDependencies` affects ALL tasks in ALL packages via the **global hash** — tasks cannot opt out of specific files, even with negation globs in `inputs`. Be specific.
```json
// WRONG - heavy hammer, affects all hashes
{
"globalDependencies": ["**/.env.*local"]
}
// BETTER - move to task-level inputs
{
"globalDependencies": [".env"],
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", ".env*"],
"outputs": ["dist/**"]
}
}
}
```
With `futureFlags.globalConfiguration`, this problem is reduced because `global.inputs` files are folded into each task's inputs (not the global hash). Tasks can exclude specific files:
```json
// BEST - global.inputs with per-task exclusion
{
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": [".env"]
},
"tasks": {
"build": { "outputs": ["dist/**"] },
"lint": {
"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/.env"]
}
}
}
```
### Repetitive Task Configuration
Look for repeated configuration across tasks that can be collapsed. Turborepo supports shared configuration patterns.
```json
// WRONG - repetitive env and inputs across tasks
{
"tasks": {
"build": {
"env": ["API_URL", "DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env*"]
},
"test": {
"env": ["API_URL", "DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env*"]
},
"dev": {
"env": ["API_URL", "DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env*"],
"cache": false,
"persistent": true
}
}
}
// BETTER - use globalEnv and globalDependencies for shared config
{
"globalEnv": ["API_URL", "DATABASE_URL"],
"globalDependencies": [".env*"],
"tasks": {
"build": {},
"test": {},
"dev": {
"cache": false,
"persistent": true
}
}
}
```
**When to use global vs task-level:**
- `globalEnv` / `globalDependencies` - affects ALL tasks, use for truly shared config
- Task-level `env` / `inputs` - use when only specific tasks need it
### NOT an Anti-Pattern: Large `env` Arrays
A large `env` array (even 50+ variables) is **not** a problem. It usually means the user was thorough about declaring their build's environment dependencies. Do not flag this as an issue.
### Using `--parallel` Flag
The `--parallel` flag bypasses Turborepo's dependency graph. If tasks need parallel execution, configure `dependsOn` correctly instead.
```bash
# WRONG - bypasses dependency graph
turbo run lint --parallel
# CORRECT - configure tasks to allow parallel execution
# In turbo.json, set dependsOn appropriately (or use transit nodes)
turbo run lint
```
### Package-Specific Task Overrides in Root turbo.json
When multiple packages need different task configurations, use **Package Configurations** (`turbo.json` in each package) instead of cluttering root `turbo.json` with `package#task` overrides.
```json
// WRONG - root turbo.json with many package-specific overrides
{
"tasks": {
"test": { "dependsOn": ["build"] },
"@repo/web#test": { "outputs": ["coverage/**"] },
"@repo/api#test": { "outputs": ["coverage/**"] },
"@repo/utils#test": { "outputs": [] },
"@repo/cli#test": { "outputs": [] },
"@repo/core#test": { "outputs": [] }
}
}
// CORRECT - use Package Configurations
// Root turbo.json - base config only
{
"tasks": {
"test": { "dependsOn": ["build"] }
}
}
// packages/web/turbo.json - package-specific override
{
"extends": ["//"],
"tasks": {
"test": { "outputs": ["coverage/**"] }
}
}
// packages/api/turbo.json
{
"extends": ["//"],
"tasks": {
"test": { "outputs": ["coverage/**"] }
}
}
```
**Benefits of Package Configurations:**
- Keeps configuration close to the code it affects
- Root turbo.json stays clean and focused on base patterns
- Easier to understand what's special about each package
- Works with `$TURBO_EXTENDS$` to inherit + extend arrays
**When to use `package#task` in root:**
- Single package needs a unique dependency (e.g., `"deploy": { "dependsOn": ["web#build"] }`)
- Temporary override while migrating
See `references/configuration/RULE.md#package-configurations` for full details.
### Using `../` to Traverse Out of Package in `inputs`
Don't use relative paths like `../` to reference files outside the package. Use `$TURBO_ROOT$` instead.
```json
// WRONG - traversing out of package
{
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", "../shared-config.json"]
}
}
}
// CORRECT - use $TURBO_ROOT$ for repo root
{
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", "$TURBO_ROOT$/shared-config.json"]
}
}
}
```
### Missing `outputs` for File-Producing Tasks
**Before flagging missing `outputs`, check what the task actually produces:**
1. Read the package's script (e.g., `"build": "tsc"`, `"test": "vitest"`)
2. Determine if it writes files to disk or only outputs to stdout
3. Only flag if the task produces files that should be cached
```json
// WRONG: build produces files but they're not cached
{
"tasks": {
"build": {
"dependsOn": ["^build"]
}
}
}
// CORRECT: build outputs are cached
{
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
}
}
}
```
Common outputs by framework:
- Next.js: `[".next/**", "!.next/cache/**"]`
- Vite/Rollup: `["dist/**"]`
- tsc: `["dist/**"]` or custom `outDir`
**TypeScript `--noEmit` can still produce cache files:**
When `incremental: true` in tsconfig.json, `tsc --noEmit` writes `.tsbuildinfo` files even without emitting JS. Check the tsconfig before assuming no outputs:
```json
// If tsconfig has incremental: true, tsc --noEmit produces cache files
{
"tasks": {
"typecheck": {
"outputs": ["node_modules/.cache/tsbuildinfo.json"] // or wherever tsBuildInfoFile points
}
}
}
```
To determine correct outputs for TypeScript tasks:
1. Check if `incremental` or `composite` is enabled in tsconfig
2. Check `tsBuildInfoFile` for custom cache location (default: alongside `outDir` or in project root)
3. If no incremental mode, `tsc --noEmit` produces no files
### `^build` vs `build` Confusion
```json
{
"tasks": {
// ^build = run build in DEPENDENCIES first (other packages this one imports)
"build": {
"dependsOn": ["^build"]
},
// build (no ^) = run build in SAME PACKAGE first
"test": {
"dependsOn": ["build"]
},
// pkg#task = specific package's task
"deploy": {
"dependsOn": ["web#build"]
}
}
}
```
### Environment Variables Not Hashed
```json
// WRONG: API_URL changes won't cause rebuilds
{
"tasks": {
"build": {
"outputs": ["dist/**"]
}
}
}
// CORRECT: API_URL changes invalidate cache
{
"tasks": {
"build": {
"outputs": ["dist/**"],
"env": ["API_URL", "API_KEY"]
}
}
}
```
### `.env` Files Not in Inputs
Turbo does NOT load `.env` files - your framework does. But Turbo needs to know about changes:
```json
// WRONG: .env changes don't invalidate cache
{
"tasks": {
"build": {
"env": ["API_URL"]
}
}
}
// CORRECT: .env file changes invalidate cache
{
"tasks": {
"build": {
"env": ["API_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env", ".env.*"]
}
}
}
```
### Root `.env` File in Monorepo
A `.env` file at the repo root is an anti-pattern — even for small monorepos or starter templates. It creates implicit coupling between packages and makes it unclear which packages depend on which variables.
```
// WRONG - root .env affects all packages implicitly
my-monorepo/
├── .env # Which packages use this?
├── apps/
│ ├── web/
│ └── api/
└── packages/
// CORRECT - .env files in packages that need them
my-monorepo/
├── apps/
│ ├── web/
│ │ └── .env # Clear: web needs DATABASE_URL
│ └── api/
│ └── .env # Clear: api needs API_KEY
└── packages/
```
**Problems with root `.env`:**
- Unclear which packages consume which variables
- All packages get all variables (even ones they don't need)
- Cache invalidation is coarse-grained (root .env change invalidates everything)
- Security risk: packages may accidentally access sensitive vars meant for others
- Bad habits start small — starter templates should model correct patterns
**If you must share variables**, use `globalEnv` to be explicit about what's shared, and document why.
### Strict Mode Filtering CI Variables
By default, Turborepo filters environment variables to only those in `env`/`globalEnv`. CI variables may be missing:
```json
// If CI scripts need GITHUB_TOKEN but it's not in env:
{
"globalPassThroughEnv": ["GITHUB_TOKEN", "CI"],
"tasks": { ... }
}
```
Or use `--env-mode=loose` (not recommended for production).
### Shared Code in Apps (Should Be a Package)
```
// WRONG: Shared code inside an app
apps/
web/
shared/ # This breaks monorepo principles!
utils.ts
// CORRECT: Extract to a package
packages/
utils/
src/utils.ts
```
### Accessing Files Across Package Boundaries
```typescript
// WRONG: Reaching into another package's internals
import { Button } from "../../packages/ui/src/button";
// CORRECT: Install and import properly
import { Button } from "@repo/ui/button";
```
### Too Many Root Dependencies
```json
// WRONG: App dependencies in root
{
"dependencies": {
"react": "^18",
"next": "^14"
}
}
// CORRECT: Only repo tools in root
{
"devDependencies": {
"turbo": "latest"
}
}
```
## Common Task Configurations
### Standard Build Pipeline
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**", ".next/**", "!.next/cache/**"]
},
"dev": {
"cache": false,
"persistent": true
}
}
}
```
Add a `transit` task if you have tasks that need parallel execution with cache invalidation (see below).
### Dev Task with `^dev` Pattern (for `turbo watch`)
A `dev` task with `dependsOn: ["^dev"]` and `persistent: false` in root turbo.json may look unusual but is **correct for `turbo watch` workflows**:
```json
// Root turbo.json
{
"tasks": {
"dev": {
"dependsOn": ["^dev"],
"cache": false,
"persistent": false // Packages have one-shot dev scripts
}
}
}
// Package turbo.json (apps/web/turbo.json)
{
"extends": ["//"],
"tasks": {
"dev": {
"persistent": true // Apps run long-running dev servers
}
}
}
```
**Why this works:**
- **Packages** (e.g., `@acme/db`, `@acme/validators`) have `"dev": "tsc"` — one-shot type generation that completes quickly
- **Apps** override with `persistent: true` for actual dev servers (Next.js, etc.)
- **`turbo watch`** re-runs the one-shot package `dev` scripts when source files change, keeping types in sync
**Intended usage:** Run `turbo watch dev` (not `turbo run dev`). Watch mode re-executes one-shot tasks on file changes while keeping persistent tasks running.
**Alternative pattern:** Use a separate task name like `prepare` or `generate` for one-shot dependency builds to make the intent clearer:
```json
{
"tasks": {
"prepare": {
"dependsOn": ["^prepare"],
"outputs": ["dist/**"]
},
"dev": {
"dependsOn": ["prepare"],
"cache": false,
"persistent": true
}
}
}
```
### Transit Nodes for Parallel Tasks with Cache Invalidation
Some tasks can run in parallel (don't need built output from dependencies) but must invalidate cache when dependency source code changes.
**The problem with `dependsOn: ["^taskname"]`:**
- Forces sequential execution (slow)
**The problem with `dependsOn: []` (no dependencies):**
- Allows parallel execution (fast)
- But cache is INCORRECT - changing dependency source won't invalidate cache
**Transit Nodes solve both:**
```json
{
"tasks": {
"transit": { "dependsOn": ["^transit"] },
"my-task": { "dependsOn": ["transit"] }
}
}
```
The `transit` task creates dependency relationships without matching any actual script, so tasks run in parallel with correct cache invalidation.
**How to identify tasks that need this pattern:** Look for tasks that read source files from dependencies but don't need their build outputs.
### With Environment Variables
```json
{
"globalEnv": ["NODE_ENV"],
"globalDependencies": [".env"],
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"],
"env": ["API_URL", "DATABASE_URL"]
}
}
}
```
With `futureFlags.globalConfiguration`, the same config moves global settings under `global` — and `.env` becomes a per-task input instead of a global hash input:
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"env": ["NODE_ENV"],
"inputs": [".env"]
},
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"],
"env": ["API_URL", "DATABASE_URL"]
}
}
}
```
## Reference Index
### Configuration
| File | Purpose |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [configuration/RULE.md](./references/configuration/RULE.md) | turbo.json overview, Package Configurations |
| [configuration/tasks.md](./references/configuration/tasks.md) | dependsOn, outputs, inputs, env, cache, persistent |
| [configuration/global-options.md](./references/configuration/global-options.md) | globalEnv, globalDependencies, global key, futureFlags, cacheDir, envMode |
| [configuration/gotchas.md](./references/configuration/gotchas.md) | Common configuration mistakes |
### Caching
| File | Purpose |
| --------------------------------------------------------------- | -------------------------------------------- |
| [caching/RULE.md](./references/caching/RULE.md) | How caching works, hash inputs |
| [caching/remote-cache.md](./references/caching/remote-cache.md) | Vercel Remote Cache, self-hosted, login/link |
| [caching/gotchas.md](./references/caching/gotchas.md) | Debugging cache misses, --summarize, --dry |
### Environment Variables
| File | Purpose |
| ------------------------------------------------------------- | ----------------------------------------- |
| [environment/RULE.md](./references/environment/RULE.md) | env, globalEnv, passThroughEnv |
| [environment/modes.md](./references/environment/modes.md) | Strict vs Loose mode, framework inference |
| [environment/gotchas.md](./references/environment/gotchas.md) | .env files, CI issues |
### Filtering
| File | Purpose |
| ----------------------------------------------------------- | ------------------------ |
| [filtering/RULE.md](./references/filtering/RULE.md) | --filter syntax overview |
| [filtering/patterns.md](./references/filtering/patterns.md) | Common filter patterns |
### CI/CD
| File | Purpose |
| --------------------------------------------------------- | ------------------------------- |
| [ci/RULE.md](./references/ci/RULE.md) | General CI principles |
| [ci/github-actions.md](./references/ci/github-actions.md) | Complete GitHub Actions setup |
| [ci/vercel.md](./references/ci/vercel.md) | Vercel deployment, turbo-ignore |
| [ci/patterns.md](./references/ci/patterns.md) | --affected, caching strategies |
### CLI
| File | Purpose |
| ----------------------------------------------- | --------------------------------------------- |
| [cli/RULE.md](./references/cli/RULE.md) | turbo run basics |
| [cli/commands.md](./references/cli/commands.md) | turbo run flags, turbo-ignore, other commands |
### Best Practices
| File | Purpose |
| ----------------------------------------------------------------------------- | --------------------------------------------------------------- |
| [best-practices/RULE.md](./references/best-practices/RULE.md) | Monorepo best practices overview |
| [best-practices/structure.md](./references/best-practices/structure.md) | Repository structure, workspace config, TypeScript/ESLint setup |
| [best-practices/packages.md](./references/best-practices/packages.md) | Creating internal packages, JIT vs Compiled, exports |
| [best-practices/dependencies.md](./references/best-practices/dependencies.md) | Dependency management, installing, version sync |
### Watch Mode
| File | Purpose |
| ------------------------------------------- | ----------------------------------------------- |
| [watch/RULE.md](./references/watch/RULE.md) | turbo watch, interruptible tasks, dev workflows |
### Boundaries (Experimental)
| File | Purpose |
| ----------------------------------------------------- | ----------------------------------------------------- |
| [boundaries/RULE.md](./references/boundaries/RULE.md) | Enforce package isolation, tag-based dependency rules |
## Source Documentation
This skill is based on the official Turborepo documentation at:
- Source: `apps/docs/content/docs/` in the Turborepo repository
- Live: https://turborepo.dev/docs
@@ -0,0 +1,70 @@
---
description: Load Turborepo skill for creating workflows, tasks, and pipelines in monorepos. Use when users ask to "create a workflow", "make a task", "generate a pipeline", or set up build orchestration.
---
Load the Turborepo skill and help with monorepo task orchestration: creating workflows, configuring tasks, setting up pipelines, and optimizing builds.
## Workflow
### Step 1: Load turborepo skill
```
skill({ name: 'turborepo' })
```
### Step 2: Identify task type from user request
Analyze $ARGUMENTS to determine:
- **Topic**: configuration, caching, filtering, environment, CI, or CLI
- **Task type**: new setup, debugging, optimization, or implementation
Use decision trees in SKILL.md to select the relevant reference files.
### Step 3: Read relevant reference files
Based on task type, read from `references/<topic>/`:
| Task | Files to Read |
| -------------------- | ------------------------------------------------------- |
| Configure turbo.json | `configuration/RULE.md` + `configuration/tasks.md` |
| Debug cache issues | `caching/gotchas.md` |
| Set up remote cache | `caching/remote-cache.md` |
| Filter packages | `filtering/RULE.md` + `filtering/patterns.md` |
| Environment problems | `environment/gotchas.md` + `environment/modes.md` |
| Set up CI | `ci/RULE.md` + `ci/github-actions.md` or `ci/vercel.md` |
| CLI usage | `cli/commands.md` |
### Step 4: Execute task
Apply Turborepo-specific patterns from references to complete the user's request.
**CRITICAL - When creating tasks/scripts/pipelines:**
1. **DO NOT create Root Tasks** - Always create package tasks
2. Add scripts to each relevant package's `package.json` (e.g., `apps/web/package.json`, `packages/ui/package.json`)
3. Register the task in root `turbo.json`
4. Root `package.json` only contains `turbo run <task>` - never actual task logic
**Other things to verify:**
- `outputs` defined for cacheable tasks
- `dependsOn` uses correct syntax (`^task` vs `task`)
- Environment variables in `env` key
- `.env` files in `inputs` if used
- Use `turbo run` (not `turbo`) in package.json and CI
### Step 5: Summarize
```
=== Turborepo Task Complete ===
Topic: <configuration|caching|filtering|environment|ci|cli>
Files referenced: <reference files consulted>
<brief summary of what was done>
```
<user-request>
$ARGUMENTS
</user-request>
@@ -0,0 +1,241 @@
# Monorepo Best Practices
Essential patterns for structuring and maintaining a healthy Turborepo monorepo.
## Repository Structure
### Standard Layout
```
my-monorepo/
├── apps/ # Application packages (deployable)
│ ├── web/
│ ├── docs/
│ └── api/
├── packages/ # Library packages (shared code)
│ ├── ui/
│ ├── utils/
│ └── config-*/ # Shared configs (eslint, typescript, etc.)
├── package.json # Root package.json (minimal deps)
├── turbo.json # Turborepo configuration
├── pnpm-workspace.yaml # (pnpm) or workspaces in package.json
└── pnpm-lock.yaml # Lockfile (required)
```
### Key Principles
1. **`apps/` for deployables**: Next.js sites, APIs, CLIs - things that get deployed
2. **`packages/` for libraries**: Shared code consumed by apps or other packages
3. **One purpose per package**: Each package should do one thing well
4. **No nested packages**: Don't put packages inside packages
## Package Types
### Application Packages (`apps/`)
- **Deployable**: These are the "endpoints" of your package graph
- **Not installed by other packages**: Apps shouldn't be dependencies of other packages
- **No shared code**: If code needs sharing, extract to `packages/`
```json
// apps/web/package.json
{
"name": "web",
"private": true,
"dependencies": {
"@repo/ui": "workspace:*",
"next": "latest"
}
}
```
### Library Packages (`packages/`)
- **Shared code**: Utilities, components, configs
- **Namespaced names**: Use `@repo/` or `@yourorg/` prefix
- **Clear exports**: Define what the package exposes
```json
// packages/ui/package.json
{
"name": "@repo/ui",
"exports": {
"./button": "./src/button.tsx",
"./card": "./src/card.tsx"
}
}
```
## Package Compilation Strategies
### Just-in-Time (Simplest)
Export TypeScript directly; let the app's bundler compile it.
```json
{
"name": "@repo/ui",
"exports": {
"./button": "./src/button.tsx"
}
}
```
**Pros**: Zero build config, instant changes
**Cons**: Can't cache builds, requires app bundler support
### Compiled (Recommended for Libraries)
Package compiles itself with `tsc` or bundler.
```json
{
"name": "@repo/ui",
"exports": {
"./button": {
"types": "./src/button.tsx",
"default": "./dist/button.js"
}
},
"scripts": {
"build": "tsc"
}
}
```
**Pros**: Cacheable by Turborepo, works everywhere
**Cons**: More configuration
## Dependency Management
### Install Where Used
Install dependencies in the package that uses them, not the root.
```bash
# Good: Install in the package that needs it
pnpm add lodash --filter=@repo/utils
# Avoid: Installing everything at root
pnpm add lodash -w # Only for repo-level tools
```
### Root Dependencies
Only these belong in root `package.json`:
- `turbo` - The build system
- `husky`, `lint-staged` - Git hooks
- Repository-level tooling
### Internal Dependencies
Use workspace protocol for internal packages:
```json
// pnpm/bun
{ "@repo/ui": "workspace:*" }
// npm/yarn
{ "@repo/ui": "*" }
```
## Exports Best Practices
### Use `exports` Field (Not `main`)
```json
{
"exports": {
".": "./src/index.ts",
"./button": "./src/button.tsx",
"./utils": "./src/utils.ts"
}
}
```
### Avoid Barrel Files
Don't create `index.ts` files that re-export everything:
```typescript
// BAD: packages/ui/src/index.ts
export * from './button';
export * from './card';
export * from './modal';
// ... imports everything even if you need one thing
// GOOD: Direct exports in package.json
{
"exports": {
"./button": "./src/button.tsx",
"./card": "./src/card.tsx"
}
}
```
### Namespace Your Packages
```json
// Good
{ "name": "@repo/ui" }
{ "name": "@acme/utils" }
// Avoid (conflicts with npm registry)
{ "name": "ui" }
{ "name": "utils" }
```
## Common Anti-Patterns
### Accessing Files Across Package Boundaries
```typescript
// BAD: Reaching into another package
import { Button } from "../../packages/ui/src/button";
// GOOD: Install and import properly
import { Button } from "@repo/ui/button";
```
### Shared Code in Apps
```
// BAD
apps/
web/
shared/ # This should be a package!
utils.ts
// GOOD
packages/
utils/ # Proper shared package
src/utils.ts
```
### Too Many Root Dependencies
```json
// BAD: Root has app dependencies
{
"dependencies": {
"react": "^18",
"next": "^14",
"lodash": "^4"
}
}
// GOOD: Root only has repo tools
{
"devDependencies": {
"turbo": "latest",
"husky": "latest"
}
}
```
## See Also
- [structure.md](./structure.md) - Detailed repository structure patterns
- [packages.md](./packages.md) - Creating and managing internal packages
- [dependencies.md](./dependencies.md) - Dependency management strategies
@@ -0,0 +1,246 @@
# Dependency Management
Best practices for managing dependencies in a Turborepo monorepo.
## Core Principle: Install Where Used
Dependencies belong in the package that uses them, not the root.
```bash
# Good: Install in specific package
pnpm add react --filter=@repo/ui
pnpm add next --filter=web
# Avoid: Installing in root
pnpm add react -w # Only for repo-level tools!
```
## Benefits of Local Installation
### 1. Clarity
Each package's `package.json` lists exactly what it needs:
```json
// packages/ui/package.json
{
"dependencies": {
"react": "^18.0.0",
"class-variance-authority": "^0.7.0"
}
}
```
### 2. Flexibility
Different packages can use different versions when needed:
```json
// packages/legacy-ui/package.json
{ "dependencies": { "react": "^17.0.0" } }
// packages/ui/package.json
{ "dependencies": { "react": "^18.0.0" } }
```
### 3. Better Caching
Installing in root changes workspace lockfile, invalidating all caches.
### 4. Pruning Support
`turbo prune` can remove unused dependencies for Docker images.
## What Belongs in Root
Only repository-level tools:
```json
// Root package.json
{
"devDependencies": {
"turbo": "latest",
"husky": "^8.0.0",
"lint-staged": "^15.0.0"
}
}
```
**NOT** application dependencies:
- react, next, express
- lodash, axios, zod
- Testing libraries (unless truly repo-wide)
## Installing Dependencies
### Single Package
```bash
# pnpm
pnpm add lodash --filter=@repo/utils
# npm
npm install lodash --workspace=@repo/utils
# yarn
yarn workspace @repo/utils add lodash
# bun
cd packages/utils && bun add lodash
```
### Multiple Packages
```bash
# pnpm
pnpm add jest --save-dev --filter=web --filter=@repo/ui
# npm
npm install jest --save-dev --workspace=web --workspace=@repo/ui
# yarn (v2+)
yarn workspaces foreach -R --from '{web,@repo/ui}' add jest --dev
```
### Internal Packages
```bash
# pnpm
pnpm add @repo/ui --filter=web
# This updates package.json:
{
"dependencies": {
"@repo/ui": "workspace:*"
}
}
```
## Keeping Versions in Sync
### Option 1: Tooling
```bash
# syncpack - Check and fix version mismatches
npx syncpack list-mismatches
npx syncpack fix-mismatches
# manypkg - Similar functionality
npx @manypkg/cli check
npx @manypkg/cli fix
# sherif - Rust-based, very fast
npx sherif
```
### Option 2: Package Manager Commands
```bash
# pnpm - Update everywhere
pnpm up --recursive typescript@latest
# npm - Update in all workspaces
npm install typescript@latest --workspaces
```
### Option 3: pnpm Catalogs (pnpm 9.5+)
```yaml
# pnpm-workspace.yaml
packages:
- "apps/*"
- "packages/*"
catalog:
react: ^18.2.0
typescript: ^5.3.0
```
```json
// Any package.json
{
"dependencies": {
"react": "catalog:" // Uses version from catalog
}
}
```
## Internal vs External Dependencies
### Internal (Workspace)
```json
// pnpm/bun
{ "@repo/ui": "workspace:*" }
// npm/yarn
{ "@repo/ui": "*" }
```
Turborepo understands these relationships and orders builds accordingly.
### External (npm Registry)
```json
{ "lodash": "^4.17.21" }
```
Standard semver versioning from npm.
## Peer Dependencies
For library packages that expect the consumer to provide dependencies:
```json
// packages/ui/package.json
{
"peerDependencies": {
"react": "^18.0.0",
"react-dom": "^18.0.0"
},
"devDependencies": {
"react": "^18.0.0", // For development/testing
"react-dom": "^18.0.0"
}
}
```
## Common Issues
### "Module not found"
1. Check the dependency is installed in the right package
2. Run `pnpm install` / `npm install` to update lockfile
3. Check exports are defined in the package
### Version Conflicts
Packages can use different versions - this is a feature, not a bug. But if you need consistency:
1. Use tooling (syncpack, manypkg)
2. Use pnpm catalogs
3. Create a lint rule
### Hoisting Issues
Some tools expect dependencies in specific locations. Use package manager config:
```yaml
# .npmrc (pnpm)
public-hoist-pattern[]=*eslint*
public-hoist-pattern[]=*prettier*
```
## Lockfile
**Required** for:
- Reproducible builds
- Turborepo dependency analysis
- Cache correctness
```bash
# Commit your lockfile!
git add pnpm-lock.yaml # or package-lock.json, yarn.lock
```
@@ -0,0 +1,335 @@
# Creating Internal Packages
How to create and structure internal packages in your monorepo.
## Package Creation Checklist
1. Create directory in `packages/`
2. Add `package.json` with name and exports
3. Add source code in `src/`
4. Add `tsconfig.json` if using TypeScript
5. Install as dependency in consuming packages
6. Run package manager install to update lockfile
## Package Compilation Strategies
### Just-in-Time (JIT)
Export TypeScript directly. The consuming app's bundler compiles it.
```json
// packages/ui/package.json
{
"name": "@repo/ui",
"exports": {
"./button": "./src/button.tsx",
"./card": "./src/card.tsx"
},
"scripts": {
"lint": "eslint .",
"check-types": "tsc --noEmit"
}
}
```
**When to use:**
- Apps use modern bundlers (Turbopack, webpack, Vite)
- You want minimal configuration
- Build times are acceptable without caching
**Limitations:**
- No Turborepo cache for the package itself
- Consumer must support TypeScript compilation
- Can't use TypeScript `paths` (use Node.js subpath imports instead)
### Compiled
Package handles its own compilation.
```json
// packages/ui/package.json
{
"name": "@repo/ui",
"exports": {
"./button": {
"types": "./src/button.tsx",
"default": "./dist/button.js"
}
},
"scripts": {
"build": "tsc",
"dev": "tsc --watch"
}
}
```
```json
// packages/ui/tsconfig.json
{
"extends": "@repo/typescript-config/library.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src"
},
"include": ["src"],
"exclude": ["node_modules", "dist"]
}
```
**When to use:**
- You want Turborepo to cache builds
- Package will be used by non-bundler tools
- You need maximum compatibility
**Remember:** Add `dist/**` to turbo.json outputs!
## Defining Exports
### Multiple Entrypoints
```json
{
"exports": {
".": "./src/index.ts", // @repo/ui
"./button": "./src/button.tsx", // @repo/ui/button
"./card": "./src/card.tsx", // @repo/ui/card
"./hooks": "./src/hooks/index.ts" // @repo/ui/hooks
}
}
```
### Conditional Exports (Compiled)
```json
{
"exports": {
"./button": {
"types": "./src/button.tsx",
"import": "./dist/button.mjs",
"require": "./dist/button.cjs",
"default": "./dist/button.js"
}
}
}
```
## Installing Internal Packages
### Add to Consuming Package
```json
// apps/web/package.json
{
"dependencies": {
"@repo/ui": "workspace:*" // pnpm/bun
// "@repo/ui": "*" // npm/yarn
}
}
```
### Run Install
```bash
pnpm install # Updates lockfile with new dependency
```
### Import and Use
```typescript
// apps/web/src/page.tsx
import { Button } from '@repo/ui/button';
export default function Page() {
return <Button>Click me</Button>;
}
```
## One Purpose Per Package
### Good Examples
```
packages/
├── ui/ # Shared UI components
├── utils/ # General utilities
├── auth/ # Authentication logic
├── database/ # Database client/schemas
├── eslint-config/ # ESLint configuration
├── typescript-config/ # TypeScript configuration
└── api-client/ # Generated API client
```
### Avoid Mega-Packages
```
// BAD: One package for everything
packages/
└── shared/
├── components/
├── utils/
├── hooks/
├── types/
└── api/
// GOOD: Separate by purpose
packages/
├── ui/ # Components
├── utils/ # Utilities
├── hooks/ # React hooks
├── types/ # Shared TypeScript types
└── api-client/ # API utilities
```
## Config Packages
### TypeScript Config
```json
// packages/typescript-config/package.json
{
"name": "@repo/typescript-config",
"exports": {
"./base.json": "./base.json",
"./nextjs.json": "./nextjs.json",
"./library.json": "./library.json"
}
}
```
### ESLint Config
```json
// packages/eslint-config/package.json
{
"name": "@repo/eslint-config",
"exports": {
"./base": "./base.js",
"./next": "./next.js"
},
"dependencies": {
"eslint": "^8.0.0",
"eslint-config-next": "latest"
}
}
```
## Common Mistakes
### Forgetting to Export
```json
// BAD: No exports defined
{
"name": "@repo/ui"
}
// GOOD: Clear exports
{
"name": "@repo/ui",
"exports": {
"./button": "./src/button.tsx"
}
}
```
### Wrong Workspace Syntax
```json
// pnpm/bun
{ "@repo/ui": "workspace:*" } // Correct
// npm/yarn
{ "@repo/ui": "*" } // Correct
{ "@repo/ui": "workspace:*" } // Wrong for npm/yarn!
```
### Missing from turbo.json Outputs
```json
// Package builds to dist/, but turbo.json doesn't know
{
"tasks": {
"build": {
"outputs": [".next/**"] // Missing dist/**!
}
}
}
// Correct
{
"tasks": {
"build": {
"outputs": [".next/**", "dist/**"]
}
}
}
```
## TypeScript Best Practices
### Use Node.js Subpath Imports (Not `paths`)
TypeScript `compilerOptions.paths` breaks with JIT packages. Use Node.js subpath imports instead (TypeScript 5.4+).
**JIT Package:**
```json
// packages/ui/package.json
{
"imports": {
"#*": "./src/*"
}
}
```
```typescript
// packages/ui/button.tsx
import { MY_STRING } from "#utils.ts"; // Uses .ts extension
```
**Compiled Package:**
```json
// packages/ui/package.json
{
"imports": {
"#*": "./dist/*"
}
}
```
```typescript
// packages/ui/button.tsx
import { MY_STRING } from "#utils.js"; // Uses .js extension
```
### Use `tsc` for Internal Packages
For internal packages, prefer `tsc` over bundlers. Bundlers can mangle code before it reaches your app's bundler, causing hard-to-debug issues.
### Enable Go-to-Definition
For Compiled Packages, enable declaration maps:
```json
// tsconfig.json
{
"compilerOptions": {
"declaration": true,
"declarationMap": true
}
}
```
This creates `.d.ts` and `.d.ts.map` files for IDE navigation.
### No Root tsconfig.json Needed
Each package should have its own `tsconfig.json`. A root one causes all tasks to miss cache when changed. Only use root `tsconfig.json` for non-package scripts.
### Avoid TypeScript Project References
They add complexity and another caching layer. Turborepo handles dependencies better.
@@ -0,0 +1,297 @@
# Repository Structure
Detailed guidance on structuring a Turborepo monorepo.
## Workspace Configuration
### pnpm (Recommended)
```yaml
# pnpm-workspace.yaml
packages:
- "apps/*"
- "packages/*"
```
### npm/yarn/bun
```json
// package.json
{
"workspaces": ["apps/*", "packages/*"]
}
```
## Root package.json
```json
{
"name": "my-monorepo",
"private": true,
"packageManager": "pnpm@9.0.0",
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev",
"lint": "turbo run lint",
"test": "turbo run test"
},
"devDependencies": {
"turbo": "latest"
}
}
```
Key points:
- `private: true` - Prevents accidental publishing
- `packageManager` - Enforces consistent package manager version
- **Scripts only delegate to `turbo run`** - No actual build logic here!
- Minimal devDependencies (just turbo and repo tools)
## Always Prefer Package Tasks
**Always use package tasks. Only use Root Tasks if you cannot succeed with package tasks.**
```json
// packages/web/package.json
{
"scripts": {
"build": "next build",
"lint": "eslint .",
"test": "vitest",
"typecheck": "tsc --noEmit"
}
}
// packages/api/package.json
{
"scripts": {
"build": "tsc",
"lint": "eslint .",
"test": "vitest",
"typecheck": "tsc --noEmit"
}
}
```
Package tasks enable Turborepo to:
1. **Parallelize** - Run `web#lint` and `api#lint` simultaneously
2. **Cache individually** - Each package's task output is cached separately
3. **Filter precisely** - Run `turbo run test --filter=web` for just one package
**Root Tasks are a fallback** for tasks that truly cannot run per-package:
```json
// AVOID unless necessary - sequential, not parallelized, can't filter
{
"scripts": {
"lint": "eslint apps/web && eslint apps/api && eslint packages/ui"
}
}
```
## Root turbo.json
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**", ".next/**", "!.next/cache/**"]
},
"lint": {},
"test": {
"dependsOn": ["build"]
},
"dev": {
"cache": false,
"persistent": true
}
}
}
```
With `futureFlags.globalConfiguration`, global settings move under a `global` key:
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": ["tsconfig.json"],
"env": ["CI"]
},
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**", ".next/**", "!.next/cache/**"]
},
"lint": {},
"test": {
"dependsOn": ["build"]
},
"dev": {
"cache": false,
"persistent": true
}
}
}
```
## Directory Organization
### Grouping Packages
You can group packages by adding more workspace paths:
```yaml
# pnpm-workspace.yaml
packages:
- "apps/*"
- "packages/*"
- "packages/config/*" # Grouped configs
- "packages/features/*" # Feature packages
```
This allows:
```
packages/
├── ui/
├── utils/
├── config/
│ ├── eslint/
│ ├── typescript/
│ └── tailwind/
└── features/
├── auth/
└── payments/
```
### What NOT to Do
```yaml
# BAD: Nested wildcards cause ambiguous behavior
packages:
- "packages/**" # Don't do this!
```
## Package Anatomy
### Minimum Required Files
```
packages/ui/
├── package.json # Required: Makes it a package
├── src/ # Source code
│ └── button.tsx
└── tsconfig.json # TypeScript config (if using TS)
```
### package.json Requirements
```json
{
"name": "@repo/ui", // Unique, namespaced name
"version": "0.0.0", // Version (can be 0.0.0 for internal)
"private": true, // Prevents accidental publishing
"exports": {
// Entry points
"./button": "./src/button.tsx"
}
}
```
## TypeScript Configuration
### Shared Base Config
Create a shared TypeScript config package:
```
packages/
└── typescript-config/
├── package.json
├── base.json
├── nextjs.json
└── library.json
```
```json
// packages/typescript-config/base.json
{
"compilerOptions": {
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"moduleResolution": "bundler",
"module": "ESNext",
"target": "ES2022"
}
}
```
### Extending in Packages
```json
// packages/ui/tsconfig.json
{
"extends": "@repo/typescript-config/library.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src"
},
"include": ["src"],
"exclude": ["node_modules", "dist"]
}
```
### No Root tsconfig.json
You likely don't need a `tsconfig.json` in the workspace root. Each package should have its own config extending from the shared config package.
## ESLint Configuration
### Shared Config Package
```
packages/
└── eslint-config/
├── package.json
├── base.js
├── next.js
└── library.js
```
```json
// packages/eslint-config/package.json
{
"name": "@repo/eslint-config",
"exports": {
"./base": "./base.js",
"./next": "./next.js",
"./library": "./library.js"
}
}
```
### Using in Packages
```js
// apps/web/.eslintrc.js
module.exports = {
extends: ["@repo/eslint-config/next"]
};
```
## Lockfile
A lockfile is **required** for:
- Reproducible builds
- Turborepo to understand package dependencies
- Cache correctness
Without a lockfile, you'll see unpredictable behavior.
@@ -0,0 +1,126 @@
# Boundaries
**Experimental feature** - See [RFC](https://github.com/vercel/turborepo/discussions/9435)
Full docs: https://turborepo.dev/docs/reference/boundaries
Boundaries enforce package isolation by detecting:
1. Imports of files outside the package's directory
2. Imports of packages not declared in `package.json` dependencies
## Usage
```bash
turbo boundaries
```
Run this to check for workspace violations across your monorepo.
## Tags
Tags allow you to create rules for which packages can depend on each other.
### Adding Tags to a Package
```json
// packages/ui/turbo.json
{
"tags": ["internal"]
}
```
### Configuring Tag Rules
Rules go in root `turbo.json`:
```json
// turbo.json
{
"boundaries": {
"tags": {
"public": {
"dependencies": {
"deny": ["internal"]
}
}
}
}
}
```
This prevents `public`-tagged packages from importing `internal`-tagged packages.
### Rule Types
**Allow-list approach** (only allow specific tags):
```json
{
"boundaries": {
"tags": {
"public": {
"dependencies": {
"allow": ["public"]
}
}
}
}
}
```
**Deny-list approach** (block specific tags):
```json
{
"boundaries": {
"tags": {
"public": {
"dependencies": {
"deny": ["internal"]
}
}
}
}
}
```
**Restrict dependents** (who can import this package):
```json
{
"boundaries": {
"tags": {
"private": {
"dependents": {
"deny": ["public"]
}
}
}
}
}
```
### Using Package Names
Package names work in place of tags:
```json
{
"boundaries": {
"tags": {
"private": {
"dependents": {
"deny": ["@repo/my-pkg"]
}
}
}
}
}
```
## Key Points
- Rules apply transitively (dependencies of dependencies)
- Helps enforce architectural boundaries at scale
- Catches violations before runtime/build errors
@@ -0,0 +1,153 @@
# How Turborepo Caching Works
Turborepo's core principle: **never do the same work twice**.
## The Cache Equation
```
fingerprint(inputs) → stored outputs
```
If inputs haven't changed, restore outputs from cache instead of re-running the task.
## What Determines the Cache Key
### Global Hash Inputs
These affect ALL tasks in the repo:
- `package-lock.json` / `yarn.lock` / `pnpm-lock.yaml`
- Files listed in `globalDependencies` (or `global.env` when using `globalConfiguration`)
- Environment variables in `globalEnv` (or `global.env`)
- `turbo.json` configuration
```json
{
"globalDependencies": [".env", "tsconfig.base.json"],
"globalEnv": ["CI", "NODE_ENV"]
}
```
### Task Hash Inputs
These affect specific tasks:
- All files in the package (unless filtered by `inputs`)
- `package.json` contents
- Environment variables in task's `env` key
- Task configuration (command, outputs, dependencies)
- Hashes of dependent tasks (`dependsOn`)
- Files from `global.inputs` (when using `futureFlags.globalConfiguration` — see below)
```json
{
"tasks": {
"build": {
"dependsOn": ["^build"],
"inputs": ["src/**", "package.json", "tsconfig.json"],
"env": ["API_URL"]
}
}
}
```
### How `global.inputs` Changes the Hash Equation
When `futureFlags.globalConfiguration` is enabled, `global.inputs` files are **not** part of the global hash. Instead, they are prepended to every task's `inputs` and folded into the **task hash**. This is a fundamental change from `globalDependencies`.
**With `globalDependencies` (default):**
```
task cache key = hash(global hash, task hash)
↑ includes globalDependencies file hashes
```
Changing a `globalDependencies` file invalidates **every** task, regardless of task-level `inputs`. There is no way for a task to opt out.
**With `global.inputs` (`futureFlags.globalConfiguration`):**
```
task cache key = hash(global hash, task hash)
↑ includes global.inputs file hashes (merged with task inputs)
```
`global.inputs` files are merged into each task's input globs. This means:
- Tasks can **exclude** specific global files with negation globs: `"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/tsconfig.json"]`
- The global hash is smaller (it still includes lockfile, engines, `global.env`, etc. — but not file hashes from `global.inputs`)
- The task hash correctly includes the global input file hashes alongside the task's own inputs
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": ["tsconfig.json", ".env"]
},
"tasks": {
"build": {
"outputs": ["dist/**"]
},
"lint": {
"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/tsconfig.json"]
}
}
}
```
In this example, changing `tsconfig.json` invalidates `build` (it's in the task's inputs) but **not** `lint` (which explicitly excludes it). With `globalDependencies`, both would have been invalidated.
## What Gets Cached
1. **File outputs** - files/directories specified in `outputs`
2. **Task logs** - stdout/stderr for replay on cache hit
```json
{
"tasks": {
"build": {
"outputs": ["dist/**", ".next/**"]
}
}
}
```
## Local Cache Location
```
.turbo/cache/
├── <hash1>.tar.zst # compressed outputs
├── <hash2>.tar.zst
└── ...
```
Add `.turbo` to `.gitignore`.
## Cache Restoration
On cache hit, Turborepo:
1. Extracts archived outputs to their original locations
2. Replays the logged stdout/stderr
3. Reports the task as cached (shows `FULL TURBO` in output)
## Example Flow
```bash
# First run - executes build, caches result
turbo build
# → packages/ui: cache miss, executing...
# → packages/web: cache miss, executing...
# Second run - same inputs, restores from cache
turbo build
# → packages/ui: cache hit, replaying output
# → packages/web: cache hit, replaying output
# → FULL TURBO
```
## Key Points
- Cache is content-addressed (based on input hash, not timestamps)
- Empty `outputs` array means task runs but nothing is cached
- Tasks without `outputs` key cache nothing (use `"outputs": []` to be explicit)
- Cache is invalidated when ANY input changes
@@ -0,0 +1,190 @@
# Debugging Cache Issues
## Diagnostic Tools
### `--summarize`
Generates a JSON file with all hash inputs. Compare two runs to find differences.
```bash
turbo build --summarize
# Creates .turbo/runs/<run-id>.json
```
The summary includes:
- Global hash and its inputs
- Per-task hashes and their inputs
- Environment variables that affected the hash
**Comparing runs:**
```bash
# Run twice, compare the summaries
diff .turbo/runs/<first-run>.json .turbo/runs/<second-run>.json
```
### `--dry` / `--dry=json`
See what would run without executing anything:
```bash
turbo build --dry
turbo build --dry=json # machine-readable output
```
Shows cache status for each task without running them.
### `--force`
Skip reading cache, re-execute all tasks:
```bash
turbo build --force
```
Useful to verify tasks actually work (not just cached results).
## Unexpected Cache Misses
**Symptom:** Task runs when you expected a cache hit.
### Environment Variable Changed
Check if an env var in the `env` key changed:
```json
{
"tasks": {
"build": {
"env": ["API_URL", "NODE_ENV"]
}
}
}
```
Different `API_URL` between runs = cache miss.
### .env File Changed
`.env` files aren't tracked by default. Add to `inputs`:
```json
{
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", ".env", ".env.local"]
}
}
}
```
Or use `globalDependencies` for repo-wide env files:
```json
{
"globalDependencies": [".env"]
}
```
With `futureFlags.globalConfiguration`, use `global.inputs` instead. The key difference: `global.inputs` files are folded into each task's hash individually (not the global hash), so tasks can exclude specific files with negation globs.
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": [".env"]
}
}
```
### Lockfile Changed
Installing/updating packages changes the global hash.
### Source Files Changed
Any file in the package (or in `inputs`) triggers a miss.
### turbo.json Changed
Config changes invalidate the global hash.
## Incorrect Cache Hits
**Symptom:** Cached output is stale/wrong.
### Missing Environment Variable
Task uses an env var not listed in `env`:
```javascript
// build.js
const apiUrl = process.env.API_URL; // not tracked!
```
Fix: add to task config:
```json
{
"tasks": {
"build": {
"env": ["API_URL"]
}
}
}
```
### Missing File in Inputs
Task reads a file outside default inputs:
```json
{
"tasks": {
"build": {
"inputs": [
"$TURBO_DEFAULT$",
"../../shared-config.json" // file outside package
]
}
}
}
```
## Useful Flags
```bash
# Only show output for cache misses
turbo build --output-logs=new-only
# Show output for everything (debugging)
turbo build --output-logs=full
# See why tasks are running
turbo build --verbosity=2
```
## Debugging with `globalConfiguration` Enabled
When `futureFlags.globalConfiguration` is on, `global.inputs` files appear in per-task hash inputs (not the global hash). If you're getting unexpected cache misses:
1. Check `--summarize` output — global input files will show up in the **task inputs** section, not the global hash section
2. Verify tasks aren't accidentally excluding global inputs via negation globs in `inputs`
3. Remember that toggling the `globalConfiguration` flag itself invalidates all caches (the flag value is part of the global hash)
If you're getting unexpected cache **hits** after changing a global input file, the task may be excluding that file with a negation glob. Check the task's `inputs` for `!$TURBO_ROOT$/...` patterns.
## Quick Checklist
Cache miss when expected hit:
1. Run with `--summarize`, compare with previous run
2. Check env vars with `--dry=json`
3. Look for lockfile/config changes in git
Cache hit when expected miss:
1. Verify env var is in `env` array
2. Verify file is in `inputs` array
3. Check if file is outside package directory
@@ -0,0 +1,127 @@
# Remote Caching
Share cache artifacts across your team and CI pipelines.
## Benefits
- Team members get cache hits from each other's work
- CI gets cache hits from local development (and vice versa)
- Dramatically faster CI runs after first build
- No more "works on my machine" rebuilds
## Vercel Remote Cache
Free, zero-config when deploying on Vercel. For local dev and other CI:
### Local Development Setup
```bash
# Authenticate with Vercel
npx turbo login
# Link repo to your Vercel team
npx turbo link
```
This creates `.turbo/config.json` with your team info (gitignored by default).
### CI Setup
Set these environment variables:
```bash
TURBO_TOKEN=<your-token>
TURBO_TEAM=<your-team-slug>
```
Get your token from Vercel dashboard → Settings → Tokens.
**GitHub Actions example:**
```yaml
- name: Build
run: npx turbo build
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
```
## Configuration in turbo.json
```json
{
"remoteCache": {
"enabled": true,
"signature": false
}
}
```
Options:
- `enabled`: toggle remote cache (default: true when authenticated)
- `signature`: require artifact signing (default: false)
## Artifact Signing
Verify cache artifacts haven't been tampered with:
```bash
# Set a secret key (use same key across all environments)
export TURBO_REMOTE_CACHE_SIGNATURE_KEY="your-secret-key"
```
Enable in config:
```json
{
"remoteCache": {
"signature": true
}
}
```
Signed artifacts can only be restored if the signature matches.
## Self-Hosted Options
Community implementations for running your own cache server:
- **turbo-remote-cache** (Node.js) - supports S3, GCS, Azure
- **turborepo-remote-cache** (Go) - lightweight, S3-compatible
- **ducktape** (Rust) - high-performance option
Configure with environment variables:
```bash
TURBO_API=https://your-cache-server.com
TURBO_TOKEN=your-auth-token
TURBO_TEAM=your-team
```
## Cache Behavior Control
```bash
# Disable remote cache for a run
turbo build --remote-cache-read-only # read but don't write
turbo build --no-cache # skip cache entirely
# Environment variable alternative
TURBO_REMOTE_ONLY=true # only use remote, skip local
```
## Debugging Remote Cache
```bash
# Verbose output shows cache operations
turbo build --verbosity=2
# Check if remote cache is configured
turbo config
```
Look for:
- "Remote caching enabled" in output
- Upload/download messages during runs
- "cache hit, replaying output" with remote cache indicator
@@ -0,0 +1,79 @@
# CI/CD with Turborepo
General principles for running Turborepo in continuous integration environments.
## Core Principles
### Always Use `turbo run` in CI
**Never use the `turbo <tasks>` shorthand in CI or scripts.** Always use `turbo run`:
```bash
# CORRECT - Always use in CI, package.json, scripts
turbo run build test lint
# WRONG - Shorthand is only for one-off terminal commands
turbo build test lint
```
The shorthand `turbo <tasks>` is only for one-off invocations typed directly in terminal by humans or agents. Anywhere the command is written into code (CI, package.json, scripts), use `turbo run`.
### Enable Remote Caching
Remote caching dramatically speeds up CI by sharing cached artifacts across runs.
Required environment variables:
```bash
TURBO_TOKEN=your_vercel_token
TURBO_TEAM=your_team_slug
```
### Use --affected for PR Builds
The `--affected` flag only runs tasks for packages changed since the base branch:
```bash
turbo run build test --affected
```
This requires Git history to compute what changed.
## Git History Requirements
### Fetch Depth
`--affected` needs access to the merge base. Shallow clones break this.
```yaml
# GitHub Actions
- uses: actions/checkout@v4
with:
fetch-depth: 2 # Minimum for --affected
# Use 0 for full history if merge base is far
```
### Why Shallow Clones Break --affected
Turborepo compares the current HEAD to the merge base with `main`. If that commit isn't fetched, `--affected` falls back to running everything.
For PRs with many commits, consider:
```yaml
fetch-depth: 0 # Full history
```
## Environment Variables Reference
| Variable | Purpose |
| ------------------- | ------------------------------------ |
| `TURBO_TOKEN` | Vercel access token for remote cache |
| `TURBO_TEAM` | Your Vercel team slug |
| `TURBO_REMOTE_ONLY` | Skip local cache, use remote only |
| `TURBO_LOG_ORDER` | Set to `grouped` for cleaner CI logs |
## See Also
- [github-actions.md](./github-actions.md) - GitHub Actions setup
- [vercel.md](./vercel.md) - Vercel deployment
- [patterns.md](./patterns.md) - CI optimization patterns
@@ -0,0 +1,162 @@
# GitHub Actions
Complete setup guide for Turborepo with GitHub Actions.
## Basic Workflow Structure
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Install dependencies
run: npm ci
- name: Build and Test
run: turbo run build test lint
```
## Package Manager Setup
### pnpm
```yaml
- uses: pnpm/action-setup@v3
with:
version: 9
- uses: actions/setup-node@v4
with:
node-version: 20
cache: "pnpm"
- run: pnpm install --frozen-lockfile
```
### Yarn
```yaml
- uses: actions/setup-node@v4
with:
node-version: 20
cache: "yarn"
- run: yarn install --frozen-lockfile
```
### Bun
```yaml
- uses: oven-sh/setup-bun@v1
with:
bun-version: latest
- run: bun install --frozen-lockfile
```
## Remote Cache Setup
### 1. Create Vercel Access Token
1. Go to [Vercel Dashboard](https://vercel.com/account/tokens)
2. Create a new token with appropriate scope
3. Copy the token value
### 2. Add Secrets and Variables
In your GitHub repository settings:
**Secrets** (Settings > Secrets and variables > Actions > Secrets):
- `TURBO_TOKEN`: Your Vercel access token
**Variables** (Settings > Secrets and variables > Actions > Variables):
- `TURBO_TEAM`: Your Vercel team slug
### 3. Add to Workflow
```yaml
jobs:
build:
runs-on: ubuntu-latest
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
```
## Alternative: actions/cache
If you can't use remote cache, cache Turborepo's local cache directory:
```yaml
- uses: actions/cache@v4
with:
path: .turbo
key: turbo-${{ runner.os }}-${{ hashFiles('**/turbo.json', '**/package-lock.json') }}
restore-keys: |
turbo-${{ runner.os }}-
```
Note: This is less effective than remote cache since it's per-branch.
## Complete Example
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- uses: pnpm/action-setup@v3
with:
version: 9
- uses: actions/setup-node@v4
with:
node-version: 20
cache: "pnpm"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build
run: turbo run build --affected
- name: Test
run: turbo run test --affected
- name: Lint
run: turbo run lint --affected
```
@@ -0,0 +1,145 @@
# CI Optimization Patterns
Strategies for efficient CI/CD with Turborepo.
## PR vs Main Branch Builds
### PR Builds: Only Affected
Test only what changed in the PR:
```yaml
- name: Test (PR)
if: github.event_name == 'pull_request'
run: turbo run build test --affected
```
### Main Branch: Full Build
Ensure complete validation on merge:
```yaml
- name: Test (Main)
if: github.ref == 'refs/heads/main'
run: turbo run build test
```
## Custom Git Ranges with --filter
For advanced scenarios, use `--filter` with git refs:
```bash
# Changes since specific commit
turbo run test --filter="...[abc123]"
# Changes between refs
turbo run test --filter="...[main...HEAD]"
# Changes in last 3 commits
turbo run test --filter="...[HEAD~3]"
```
## Caching Strategies
### Remote Cache (Recommended)
Best performance - shared across all CI runs and developers:
```yaml
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
```
### actions/cache Fallback
When remote cache isn't available:
```yaml
- uses: actions/cache@v4
with:
path: .turbo
key: turbo-${{ runner.os }}-${{ github.sha }}
restore-keys: |
turbo-${{ runner.os }}-${{ github.ref }}-
turbo-${{ runner.os }}-
```
Limitations:
- Cache is branch-scoped
- PRs restore from base branch cache
- Less efficient than remote cache
## Matrix Builds
Test across Node versions:
```yaml
strategy:
matrix:
node: [18, 20, 22]
steps:
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- run: turbo run test
```
## Parallelizing Across Jobs
Split tasks into separate jobs:
```yaml
jobs:
lint:
runs-on: ubuntu-latest
steps:
- run: turbo run lint --affected
test:
runs-on: ubuntu-latest
steps:
- run: turbo run test --affected
build:
runs-on: ubuntu-latest
needs: [lint, test]
steps:
- run: turbo run build
```
### Cache Considerations
When parallelizing:
- Each job has separate cache writes
- Remote cache handles this automatically
- With actions/cache, use unique keys per job to avoid conflicts
```yaml
- uses: actions/cache@v4
with:
path: .turbo
key: turbo-${{ runner.os }}-${{ github.job }}-${{ github.sha }}
```
## Conditional Tasks
Skip expensive tasks on draft PRs:
```yaml
- name: E2E Tests
if: github.event.pull_request.draft == false
run: turbo run test:e2e --affected
```
Or require label for full test:
```yaml
- name: Full Test Suite
if: contains(github.event.pull_request.labels.*.name, 'full-test')
run: turbo run test
```
@@ -0,0 +1,103 @@
# Vercel Deployment
Turborepo integrates seamlessly with Vercel for monorepo deployments.
## Remote Cache
Remote caching is **automatically enabled** when deploying to Vercel. No configuration needed - Vercel detects Turborepo and enables caching.
This means:
- No `TURBO_TOKEN` or `TURBO_TEAM` setup required on Vercel
- Cache is shared across all deployments
- Preview and production builds benefit from cache
## turbo-ignore
Skip unnecessary builds when a package hasn't changed using `turbo-ignore`.
### Installation
```bash
npx turbo-ignore
```
Or install globally in your project:
```bash
pnpm add -D turbo-ignore
```
### Setup in Vercel
1. Go to your project in Vercel Dashboard
2. Navigate to Settings > Git > Ignored Build Step
3. Select "Custom" and enter:
```bash
npx turbo-ignore
```
### How It Works
`turbo-ignore` checks if the current package (or its dependencies) changed since the last successful deployment:
1. Compares current commit to last deployed commit
2. Uses Turborepo's dependency graph
3. Returns exit code 0 (skip) if no changes
4. Returns exit code 1 (build) if changes detected
### Options
```bash
# Check specific package
npx turbo-ignore web
# Use specific comparison ref
npx turbo-ignore --fallback=HEAD~1
# Verbose output
npx turbo-ignore --verbose
```
## Environment Variables
Set environment variables in Vercel Dashboard:
1. Go to Project Settings > Environment Variables
2. Add variables for each environment (Production, Preview, Development)
Common variables:
- `DATABASE_URL`
- `API_KEY`
- Package-specific config
## Monorepo Root Directory
For monorepos, set the root directory in Vercel:
1. Project Settings > General > Root Directory
2. Set to the package path (e.g., `apps/web`)
Vercel automatically:
- Installs dependencies from monorepo root
- Runs build from the package directory
- Detects framework settings
## Build Command
Vercel auto-detects `turbo run build` when `turbo.json` exists at root.
Override if needed:
```bash
turbo run build --filter=web
```
Or for production-only optimizations:
```bash
turbo run build --filter=web --env-mode=strict
```
@@ -0,0 +1,100 @@
# turbo run
The primary command for executing tasks across your monorepo.
## Basic Usage
```bash
# Full form (use in CI, package.json, scripts)
turbo run <tasks>
# Shorthand (only for one-off terminal invocations)
turbo <tasks>
```
## When to Use `turbo run` vs `turbo`
**Always use `turbo run` when the command is written into code:**
- `package.json` scripts
- CI/CD workflows (GitHub Actions, etc.)
- Shell scripts
- Documentation
- Any static/committed configuration
**Only use `turbo` (shorthand) for:**
- One-off commands typed directly in terminal
- Ad-hoc invocations by humans or agents
```json
// package.json - ALWAYS use "turbo run"
{
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev",
"lint": "turbo run lint",
"test": "turbo run test"
}
}
```
```yaml
# CI workflow - ALWAYS use "turbo run"
- run: turbo run build --affected
- run: turbo run test --affected
```
```bash
# Terminal one-off - shorthand OK
turbo build --filter=web
```
## Running Tasks
Tasks must be defined in `turbo.json` before running.
```bash
# Single task
turbo build
# Multiple tasks
turbo run build lint test
# See available tasks (run without arguments)
turbo run
```
## Passing Arguments to Scripts
Use `--` to pass arguments through to the underlying package scripts:
```bash
turbo run build -- --sourcemap
turbo test -- --watch
turbo lint -- --fix
```
Everything after `--` goes directly to the task's script.
## Package Selection
By default, turbo runs tasks in all packages. Use `--filter` to narrow scope:
```bash
turbo build --filter=web
turbo test --filter=./apps/*
```
See `filtering/` for complete filter syntax.
## Quick Reference
| Goal | Command |
| ------------------- | -------------------------- |
| Build everything | `turbo build` |
| Build one package | `turbo build --filter=web` |
| Multiple tasks | `turbo build lint test` |
| Pass args to script | `turbo build -- --arg` |
| Preview run | `turbo build --dry` |
| Force rebuild | `turbo build --force` |
@@ -0,0 +1,297 @@
# turbo run Flags Reference
Full docs: https://turborepo.dev/docs/reference/run
## Package Selection
### `--filter` / `-F`
Select specific packages to run tasks in.
```bash
turbo build --filter=web
turbo build -F=@repo/ui -F=@repo/utils
turbo test --filter=./apps/*
```
See `filtering/` for complete syntax (globs, dependencies, git ranges).
### Task Identifier Syntax (v2.2.4+)
Run specific package tasks directly:
```bash
turbo run web#build # Build web package
turbo run web#build docs#lint # Multiple specific tasks
```
### `--affected`
Run only in packages changed since the base branch.
```bash
turbo build --affected
turbo test --affected --filter=./apps/* # combine with filter
```
**How it works:**
- Default: compares `main...HEAD`
- In GitHub Actions: auto-detects `GITHUB_BASE_REF`
- Override base: `TURBO_SCM_BASE=development turbo build --affected`
- Override head: `TURBO_SCM_HEAD=your-branch turbo build --affected`
**Requires git history** - shallow clones may fall back to running all tasks.
## Execution Control
### `--dry` / `--dry=json`
Preview what would run without executing.
```bash
turbo build --dry # human-readable
turbo build --dry=json # machine-readable
```
### `--force`
Ignore all cached artifacts, re-run everything.
```bash
turbo build --force
```
### `--concurrency`
Limit parallel task execution.
```bash
turbo build --concurrency=4 # max 4 tasks
turbo build --concurrency=50% # 50% of CPU cores
```
### `--continue`
Keep running other tasks when one fails.
```bash
turbo build test --continue
```
### `--only`
Run only the specified task, skip its dependencies.
```bash
turbo build --only # skip running dependsOn tasks
```
### `--parallel` (Discouraged)
Ignores task graph dependencies, runs all tasks simultaneously. **Avoid using this flag**—if tasks need to run in parallel, configure `dependsOn` correctly instead. Using `--parallel` bypasses Turborepo's dependency graph, which can cause race conditions and incorrect builds.
## Cache Control
### `--cache`
Fine-grained cache behavior control.
```bash
# Default: read/write both local and remote
turbo build --cache=local:rw,remote:rw
# Read-only local, no remote
turbo build --cache=local:r,remote:
# Disable local, read-only remote
turbo build --cache=local:,remote:r
# Disable all caching
turbo build --cache=local:,remote:
```
## Output & Debugging
### `--graph`
Generate task graph visualization.
```bash
turbo build --graph # opens in browser
turbo build --graph=graph.svg # SVG file
turbo build --graph=graph.png # PNG file
turbo build --graph=graph.json # JSON data
turbo build --graph=graph.mermaid # Mermaid diagram
```
### `--summarize`
Generate JSON run summary for debugging.
```bash
turbo build --summarize
# creates .turbo/runs/<run-id>.json
```
### `--output-logs`
Control log output verbosity.
```bash
turbo build --output-logs=full # all logs (default)
turbo build --output-logs=new-only # only cache misses
turbo build --output-logs=errors-only # only failures
turbo build --output-logs=none # silent
```
### `--profile`
Generate Chrome tracing profile for performance analysis.
```bash
turbo build --profile=profile.json
# open chrome://tracing and load the file
```
### `--verbosity` / `-v`
Control turbo's own log level.
```bash
turbo build -v # verbose
turbo build -vv # more verbose
turbo build -vvv # maximum verbosity
```
## Environment
### `--env-mode`
Control environment variable handling.
```bash
turbo build --env-mode=strict # only declared env vars (default)
turbo build --env-mode=loose # include all env vars in hash
```
## UI
### `--ui`
Select output interface.
```bash
turbo build --ui=tui # interactive terminal UI (default in TTY)
turbo build --ui=stream # streaming logs (default in CI)
```
---
# turbo-ignore
Full docs: https://turborepo.dev/docs/reference/turbo-ignore
Skip CI work when nothing relevant changed. Useful for skipping container setup.
## Basic Usage
```bash
# Check if build is needed for current package (uses Automatic Package Scoping)
npx turbo-ignore
# Check specific package
npx turbo-ignore web
# Check specific task
npx turbo-ignore --task=test
```
## Exit Codes
- `0`: No changes detected - skip CI work
- `1`: Changes detected - proceed with CI
## CI Integration Example
```yaml
# GitHub Actions
- name: Check for changes
id: turbo-ignore
run: npx turbo-ignore web
continue-on-error: true
- name: Build
if: steps.turbo-ignore.outcome == 'failure' # changes detected
run: pnpm build
```
## Comparison Depth
Default: compares to parent commit (`HEAD^1`).
```bash
# Compare to specific commit
npx turbo-ignore --fallback=abc123
# Compare to branch
npx turbo-ignore --fallback=main
```
---
# Other Commands
## turbo boundaries
Check workspace violations (experimental).
```bash
turbo boundaries
```
See `references/boundaries/` for configuration.
## turbo watch
Re-run tasks on file changes.
```bash
turbo watch build test
```
See `references/watch/` for details.
## turbo prune
Create sparse checkout for Docker.
```bash
turbo prune web --docker
```
## turbo link / unlink
Connect/disconnect Remote Cache.
```bash
turbo link # connect to Vercel Remote Cache
turbo unlink # disconnect
```
## turbo login / logout
Authenticate with Remote Cache provider.
```bash
turbo login # authenticate
turbo logout # log out
```
## turbo generate
Scaffold new packages.
```bash
turbo generate
```
@@ -0,0 +1,235 @@
# turbo.json Configuration Overview
Configuration reference for Turborepo. Full docs: https://turborepo.dev/docs/reference/configuration
## File Location
Root `turbo.json` lives at repo root, sibling to root `package.json`:
```
my-monorepo/
├── turbo.json # Root configuration
├── package.json
└── packages/
└── web/
├── turbo.json # Package Configuration (optional)
└── package.json
```
## Always Prefer Package Tasks Over Root Tasks
**Always use package tasks. Only use Root Tasks if you cannot succeed with package tasks.**
Package tasks enable parallelization, individual caching, and filtering. Define scripts in each package's `package.json`:
```json
// packages/web/package.json
{
"scripts": {
"build": "next build",
"lint": "eslint .",
"test": "vitest",
"typecheck": "tsc --noEmit"
}
}
// packages/api/package.json
{
"scripts": {
"build": "tsc",
"lint": "eslint .",
"test": "vitest",
"typecheck": "tsc --noEmit"
}
}
```
```json
// Root package.json - delegates to turbo
{
"scripts": {
"build": "turbo run build",
"lint": "turbo run lint",
"test": "turbo run test",
"typecheck": "turbo run typecheck"
}
}
```
When you run `turbo run lint`, Turborepo finds all packages with a `lint` script and runs them **in parallel**.
**Root Tasks are a fallback**, not the default. Only use them for tasks that truly cannot run per-package (e.g., repo-level CI scripts, workspace-wide config generation).
```json
// AVOID: Task logic in root defeats parallelization
{
"scripts": {
"lint": "eslint apps/web && eslint apps/api && eslint packages/ui"
}
}
```
## Basic Structure
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"globalEnv": ["CI"],
"globalDependencies": ["tsconfig.json"],
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
},
"dev": {
"cache": false,
"persistent": true
}
}
}
```
The `$schema` key enables IDE autocompletion and validation.
### With `futureFlags.globalConfiguration`
When the `globalConfiguration` future flag is enabled, global options move under a `global` key with cleaner names:
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": ["tsconfig.json"],
"env": ["CI"],
"ui": "tui"
},
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
}
}
}
```
See the [global options reference](./global-options.md) for the full rename mapping and behavior changes.
## Configuration Sections
**Global options** - Settings affecting all tasks:
- Without flag: `globalEnv`, `globalDependencies`, `globalPassThroughEnv`, `cacheDir`, `daemon`, `envMode`, `ui`, `remoteCache`
- With `globalConfiguration` flag: all of the above move under the `global` key (see [global options](./global-options.md))
**Task definitions** - Per-task settings in `tasks` object:
- `dependsOn`, `outputs`, `inputs`, `env`
- `cache`, `persistent`, `interactive`, `outputLogs`
## Package Configurations
Use `turbo.json` in individual packages to override root settings:
```json
// packages/web/turbo.json
{
"extends": ["//"],
"tasks": {
"build": {
"outputs": [".next/**", "!.next/cache/**"]
}
}
}
```
The `"extends": ["//"]` is required - it references the root configuration.
**When to use Package Configurations:**
- Framework-specific outputs (Next.js, Vite, etc.)
- Package-specific env vars
- Different caching rules for specific packages
- Keeping framework config close to the framework code
### Extending from Other Packages
You can extend from config packages instead of just root:
```json
// packages/web/turbo.json
{
"extends": ["//", "@repo/turbo-config"]
}
```
### Adding to Inherited Arrays with `$TURBO_EXTENDS$`
By default, array fields in Package Configurations **replace** root values. Use `$TURBO_EXTENDS$` to **append** instead:
```json
// Root turbo.json
{
"tasks": {
"build": {
"outputs": ["dist/**"]
}
}
}
```
```json
// packages/web/turbo.json
{
"extends": ["//"],
"tasks": {
"build": {
// Inherits "dist/**" from root, adds ".next/**"
"outputs": ["$TURBO_EXTENDS$", ".next/**", "!.next/cache/**"]
}
}
}
```
Without `$TURBO_EXTENDS$`, outputs would only be `[".next/**", "!.next/cache/**"]`.
**Works with:**
- `dependsOn`
- `env`
- `inputs`
- `outputs`
- `passThroughEnv`
- `with`
### Excluding Tasks from Packages
Use `extends: false` to exclude a task from a package:
```json
// packages/ui/turbo.json
{
"extends": ["//"],
"tasks": {
"e2e": {
"extends": false // UI package doesn't have e2e tests
}
}
}
```
## `turbo.jsonc` for Comments
Use `turbo.jsonc` extension to add comments with IDE support:
```jsonc
// turbo.jsonc
{
"tasks": {
"build": {
// Next.js outputs
"outputs": [".next/**", "!.next/cache/**"]
}
}
}
```
@@ -0,0 +1,239 @@
# Global Options Reference
Options that affect all tasks. Full docs: https://turborepo.dev/docs/reference/configuration
## globalEnv
Environment variables affecting all task hashes.
```json
{
"globalEnv": ["CI", "NODE_ENV", "VERCEL_*"]
}
```
Use for variables that should invalidate all caches when changed.
## globalDependencies
Files that affect all task hashes.
```json
{
"globalDependencies": ["tsconfig.json", ".env", "pnpm-lock.yaml"]
}
```
Lockfile is included by default. Add shared configs here.
## globalPassThroughEnv
Variables available to tasks but not included in hash.
```json
{
"globalPassThroughEnv": ["AWS_SECRET_KEY", "GITHUB_TOKEN"]
}
```
Use for credentials that shouldn't affect cache keys.
## cacheDir
Custom cache location. Default: `node_modules/.cache/turbo`.
```json
{
"cacheDir": ".turbo/cache"
}
```
## daemon
**Deprecated**: The daemon is no longer used for `turbo run` and this option will be removed in version 3.0. The daemon is still used by `turbo watch` and the Turborepo LSP.
## envMode
How unspecified env vars are handled. Default: `"strict"`.
```json
{
"envMode": "strict" // Only specified vars available
// or
"envMode": "loose" // All vars pass through
}
```
Strict mode catches missing env declarations.
## ui
Terminal UI mode. Default: `"stream"`.
```json
{
"ui": "tui" // Interactive terminal UI
// or
"ui": "stream" // Traditional streaming logs
}
```
TUI provides better UX for parallel tasks.
## remoteCache
Configure remote caching.
```json
{
"remoteCache": {
"enabled": true,
"signature": true,
"timeout": 30,
"uploadTimeout": 60
}
}
```
| Option | Default | Description |
| --------------- | ---------------------- | ------------------------------------------------------ |
| `enabled` | `true` | Enable/disable remote caching |
| `signature` | `false` | Sign artifacts with `TURBO_REMOTE_CACHE_SIGNATURE_KEY` |
| `preflight` | `false` | Send OPTIONS request before cache requests |
| `timeout` | `30` | Timeout in seconds for cache operations |
| `uploadTimeout` | `60` | Timeout in seconds for uploads |
| `apiUrl` | `"https://vercel.com"` | Remote cache API endpoint |
| `loginUrl` | `"https://vercel.com"` | Login endpoint |
| `teamId` | - | Team ID (must start with `team_`) |
| `teamSlug` | - | Team slug for querystring |
See https://turborepo.dev/docs/core-concepts/remote-caching for setup.
## concurrency
Default: `"10"`
Limit parallel task execution.
```json
{
"concurrency": "4" // Max 4 tasks at once
// or
"concurrency": "50%" // 50% of available CPUs
}
```
## futureFlags
Enable experimental features that will become default in future versions.
```json
{
"futureFlags": {
"errorsOnlyShowHash": true
}
}
```
### `errorsOnlyShowHash`
When using `outputLogs: "errors-only"`, show task hashes on start/completion:
- Cache miss: `cache miss, executing <hash> (only logging errors)`
- Cache hit: `cache hit, replaying logs (no errors) <hash>`
### `longerSignatureKey`
Enforce a minimum key length of 32 bytes for `TURBO_REMOTE_CACHE_SIGNATURE_KEY` when `remoteCache.signature` is enabled. Short keys weaken HMAC-SHA256 signatures. Fails the run immediately if the key is too short.
### `globalConfiguration`
Moves global configuration keys under a top-level `global` key for clarity and changes how `global.inputs` (formerly `globalDependencies`) affects task hashing.
When enabled:
- Global config keys move under `global` with cleaner names
- `global.inputs` files are **prepended to every task's inputs** instead of being folded into the global hash — tasks can opt out of specific global inputs using negation globs
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": ["tsconfig.json", ".env"],
"env": ["CI", "NODE_ENV"],
"passThroughEnv": ["AWS_SECRET_KEY"],
"ui": "tui",
"envMode": "strict",
"cacheDir": ".turbo/cache",
"remoteCache": { "enabled": true },
"concurrency": "50%"
},
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
}
}
}
```
**Key rename mapping:**
| Old (top-level) | New (`global.`) |
| -------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| `globalDependencies` | `inputs` |
| `globalEnv` | `env` |
| `globalPassThroughEnv` | `passThroughEnv` |
| `ui`, `envMode`, `cacheDir`, `daemon`, `concurrency`, `noUpdateNotifier`, `dangerouslyDisablePackageManagerCheck`, `remoteCache` | Same names under `global` |
**Behavior change for `global.inputs`:**
With `globalDependencies` (old): files are hashed into the **global hash**, which is embedded in every task's cache key. Changing any of these files invalidates all tasks — there is no opt-out.
With `global.inputs` (new): files are treated as **implicit task inputs** prepended to each task's `inputs` globs. This means:
- Tasks can exclude specific global files: `"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/tsconfig.json"]`
- The global hash no longer includes these file hashes (it still includes lockfile, engines, global env, etc.)
- Tasks with no explicit `inputs` still hash all package files plus the global inputs
See the [gotchas doc](./gotchas.md) for guidance on using `$TURBO_DEFAULT$` with `global.inputs`.
## noUpdateNotifier
Disable update notifications when new turbo versions are available.
```json
{
"noUpdateNotifier": true
}
```
## dangerouslyDisablePackageManagerCheck
Bypass the `packageManager` field requirement. Use for incremental migration.
```json
{
"dangerouslyDisablePackageManagerCheck": true
}
```
**Warning**: Unstable lockfiles can cause unpredictable behavior.
## Git Worktree Cache Sharing
When working in Git worktrees, Turborepo automatically shares local cache between the main worktree and linked worktrees.
**How it works:**
- Detects worktree configuration
- Redirects cache to main worktree's `.turbo/cache`
- Works alongside Remote Cache
**Benefits:**
- Cache hits across branches
- Reduced disk usage
- Faster branch switching
**Disabled by**: Setting explicit `cacheDir` in turbo.json.
@@ -0,0 +1,368 @@
# Configuration Gotchas
Common mistakes and how to fix them.
## #1 Root Scripts Not Using `turbo run`
Root `package.json` scripts for turbo tasks MUST use `turbo run`, not direct commands.
```json
// WRONG - bypasses turbo, no parallelization or caching
{
"scripts": {
"build": "bun build",
"dev": "bun dev"
}
}
// CORRECT - delegates to turbo
{
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev"
}
}
```
**Why this matters:** Running `bun build` or `npm run build` at root bypasses Turborepo entirely - no parallelization, no caching, no dependency graph awareness.
## #2 Using `&&` to Chain Turbo Tasks
Don't use `&&` to chain tasks that turbo should orchestrate.
```json
// WRONG - changeset:publish chains turbo task with non-turbo command
{
"scripts": {
"changeset:publish": "bun build && changeset publish"
}
}
// CORRECT - use turbo run, let turbo handle dependencies
{
"scripts": {
"changeset:publish": "turbo run build && changeset publish"
}
}
```
If the second command (`changeset publish`) depends on build outputs, the turbo task should run through turbo to get caching and parallelization benefits.
## #3 Overly Broad globalDependencies
`globalDependencies` affects hash for ALL tasks in ALL packages. Be specific.
```json
// WRONG - affects all hashes
{
"globalDependencies": ["**/.env.*local"]
}
// CORRECT - move to specific tasks that need it
{
"globalDependencies": [".env"],
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", ".env*"],
"outputs": ["dist/**"]
}
}
}
```
**Why this matters:** `**/.env.*local` matches .env files in ALL packages, causing unnecessary cache invalidation. Instead:
- Use `globalDependencies` only for truly global files (root `.env`)
- Use task-level `inputs` for package-specific .env files with `$TURBO_DEFAULT$` to preserve default behavior
With `futureFlags.globalConfiguration`, this is less of a concern because `global.inputs` acts as implicit task inputs — tasks can opt out of specific files with negation globs. But keeping the list focused is still good practice.
## #4 Repetitive Task Configuration
Look for repeated configuration across tasks that can be collapsed.
```json
// WRONG - repetitive env and inputs across tasks
{
"tasks": {
"build": {
"env": ["API_URL", "DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env*"]
},
"test": {
"env": ["API_URL", "DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env*"]
}
}
}
// BETTER - use globalEnv and globalDependencies
{
"globalEnv": ["API_URL", "DATABASE_URL"],
"globalDependencies": [".env*"],
"tasks": {
"build": {},
"test": {}
}
}
```
**When to use global vs task-level:**
- `globalEnv` / `globalDependencies` - affects ALL tasks, use for truly shared config
- Task-level `env` / `inputs` - use when only specific tasks need it
## #5 Using `../` to Traverse Out of Package in `inputs`
Don't use relative paths like `../` to reference files outside the package. Use `$TURBO_ROOT$` instead.
```json
// WRONG - traversing out of package
{
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", "../shared-config.json"]
}
}
}
// CORRECT - use $TURBO_ROOT$ for repo root
{
"tasks": {
"build": {
"inputs": ["$TURBO_DEFAULT$", "$TURBO_ROOT$/shared-config.json"]
}
}
}
```
## #6 MOST COMMON MISTAKE: Creating Root Tasks
**DO NOT create Root Tasks. ALWAYS create package tasks.**
When you need to create a task (build, lint, test, typecheck, etc.):
1. Add the script to **each relevant package's** `package.json`
2. Register the task in root `turbo.json`
3. Root `package.json` only contains `turbo run <task>`
```json
// WRONG - DO NOT DO THIS
// Root package.json with task logic
{
"scripts": {
"build": "cd apps/web && next build && cd ../api && tsc",
"lint": "eslint apps/ packages/",
"test": "vitest"
}
}
// CORRECT - DO THIS
// apps/web/package.json
{ "scripts": { "build": "next build", "lint": "eslint .", "test": "vitest" } }
// apps/api/package.json
{ "scripts": { "build": "tsc", "lint": "eslint .", "test": "vitest" } }
// packages/ui/package.json
{ "scripts": { "build": "tsc", "lint": "eslint .", "test": "vitest" } }
// Root package.json - ONLY delegates
{ "scripts": { "build": "turbo run build", "lint": "turbo run lint", "test": "turbo run test" } }
// turbo.json - register tasks
{
"tasks": {
"build": { "dependsOn": ["^build"], "outputs": ["dist/**"] },
"lint": {},
"test": {}
}
}
```
**Why this matters:**
- Package tasks run in **parallel** across all packages
- Each package's output is cached **individually**
- You can **filter** to specific packages: `turbo run test --filter=web`
Root Tasks (`//#taskname`) defeat all these benefits. Only use them for tasks that truly cannot exist in any package (extremely rare).
## #7 Tasks That Need Parallel Execution + Cache Invalidation
Some tasks can run in parallel (don't need built output from dependencies) but must still invalidate cache when dependency source code changes. Using `dependsOn: ["^taskname"]` forces sequential execution. Using no dependencies breaks cache invalidation.
**Use Transit Nodes for these tasks:**
```json
// WRONG - forces sequential execution (SLOW)
"my-task": {
"dependsOn": ["^my-task"]
}
// ALSO WRONG - no dependency awareness (INCORRECT CACHING)
"my-task": {}
// CORRECT - use Transit Nodes for parallel + correct caching
{
"tasks": {
"transit": { "dependsOn": ["^transit"] },
"my-task": { "dependsOn": ["transit"] }
}
}
```
**Why Transit Nodes work:**
- `transit` creates dependency relationships without matching any actual script
- Tasks that depend on `transit` gain dependency awareness
- Since `transit` completes instantly (no script), tasks run in parallel
- Cache correctly invalidates when dependency source code changes
**How to identify tasks that need this pattern:** Look for tasks that read source files from dependencies but don't need their build outputs.
## Missing outputs for File-Producing Tasks
**Before flagging missing `outputs`, check what the task actually produces:**
1. Read the package's script (e.g., `"build": "tsc"`, `"test": "vitest"`)
2. Determine if it writes files to disk or only outputs to stdout
3. Only flag if the task produces files that should be cached
```json
// WRONG - build produces files but they're not cached
"build": {
"dependsOn": ["^build"]
}
// CORRECT - outputs are cached
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
}
```
No `outputs` key is fine for stdout-only tasks. For file-producing tasks, missing `outputs` means Turbo has nothing to cache.
## Forgetting ^ in dependsOn
```json
// WRONG - looks for "build" in SAME package (infinite loop or missing)
"build": {
"dependsOn": ["build"]
}
// CORRECT - runs dependencies' build first
"build": {
"dependsOn": ["^build"]
}
```
The `^` means "in dependency packages", not "in this package".
## Missing persistent on Dev Tasks
```json
// WRONG - dependent tasks hang waiting for dev to "finish"
"dev": {
"cache": false
}
// CORRECT
"dev": {
"cache": false,
"persistent": true
}
```
## Package Config Missing extends
```json
// WRONG - packages/web/turbo.json
{
"tasks": {
"build": { "outputs": [".next/**"] }
}
}
// CORRECT
{
"extends": ["//"],
"tasks": {
"build": { "outputs": [".next/**"] }
}
}
```
Without `"extends": ["//"]`, Package Configurations are invalid.
## Root Tasks Need Special Syntax
To run a task defined only in root `package.json`:
```bash
# WRONG
turbo run format
# CORRECT
turbo run //#format
```
And in dependsOn:
```json
"build": {
"dependsOn": ["//#codegen"] // Root package's codegen
}
```
## Overwriting Default Inputs
```json
// WRONG - only watches test files, ignores source changes
"test": {
"inputs": ["tests/**"]
}
// CORRECT - extends defaults, adds test files
"test": {
"inputs": ["$TURBO_DEFAULT$", "tests/**"]
}
```
Without `$TURBO_DEFAULT$`, you replace all default file watching.
## Excluding `global.inputs` Without `$TURBO_DEFAULT$`
When using `futureFlags.globalConfiguration`, `global.inputs` values are prepended to every task's inputs. If you want to exclude a global input from a specific task, you **must** include `$TURBO_DEFAULT$` to preserve default file hashing.
```json
// WRONG - task hashes NO files at all (global input cancelled, no defaults)
"build": {
"inputs": ["!$TURBO_ROOT$/config.txt"]
}
// CORRECT - task hashes all package files, minus config.txt
"build": {
"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/config.txt"]
}
```
Without `$TURBO_DEFAULT$`, the only inclusion glob comes from `global.inputs`, which the negation cancels out. The task ends up with no inclusions and no default file hashing, so it hashes nothing. Changes to source files won't cause cache misses.
## Caching Tasks with Side Effects
```json
// WRONG - deploy might be skipped on cache hit
"deploy": {
"dependsOn": ["build"]
}
// CORRECT
"deploy": {
"dependsOn": ["build"],
"cache": false
}
```
Always disable cache for deploy, publish, or mutation tasks.
@@ -0,0 +1,325 @@
# Task Configuration Reference
Full docs: https://turborepo.dev/docs/reference/configuration#tasks
## dependsOn
Controls task execution order.
```json
{
"tasks": {
"build": {
"dependsOn": [
"^build", // Dependencies' build tasks first
"codegen", // Same package's codegen task first
"shared#build" // Specific package's build task
]
}
}
}
```
| Syntax | Meaning |
| ---------- | ------------------------------------ |
| `^task` | Run `task` in all dependencies first |
| `task` | Run `task` in same package first |
| `pkg#task` | Run specific package's task first |
The `^` prefix is crucial - without it, you're referencing the same package.
### Transit Nodes for Parallel Tasks
For tasks like `lint` and `check-types` that can run in parallel but need dependency-aware caching:
```json
{
"tasks": {
"transit": { "dependsOn": ["^transit"] },
"lint": { "dependsOn": ["transit"] },
"check-types": { "dependsOn": ["transit"] }
}
}
```
**DO NOT use `dependsOn: ["^lint"]`** - this forces sequential execution.
**DO NOT use `dependsOn: []`** - this breaks cache invalidation.
The `transit` task creates dependency relationships without running anything (no matching script), so tasks run in parallel with correct caching.
## outputs
Glob patterns for files to cache. **If omitted, nothing is cached.**
```json
{
"tasks": {
"build": {
"outputs": ["dist/**", "build/**"]
}
}
}
```
**Framework examples:**
```json
// Next.js
"outputs": [".next/**", "!.next/cache/**"]
// Vite
"outputs": ["dist/**"]
// TypeScript (tsc)
"outputs": ["dist/**", "*.tsbuildinfo"]
// No file outputs (lint, typecheck)
"outputs": []
```
Use `!` prefix to exclude patterns from caching.
## inputs
Files considered when calculating task hash. Defaults to all tracked files in package.
```json
{
"tasks": {
"test": {
"inputs": ["src/**", "tests/**", "vitest.config.ts"]
}
}
}
```
**Special values:**
| Value | Meaning |
| --------------------- | --------------------------------------- |
| `$TURBO_DEFAULT$` | Include default inputs, then add/remove |
| `$TURBO_ROOT$/<path>` | Reference files from repo root |
```json
{
"tasks": {
"build": {
"inputs": [
"$TURBO_DEFAULT$",
"!README.md",
"$TURBO_ROOT$/tsconfig.base.json"
]
}
}
}
```
### Interaction with `global.inputs`
When `futureFlags.globalConfiguration` is enabled, files listed in `global.inputs` are prepended to every task's `inputs`. The combined list is then used to compute the task hash.
This is different from `globalDependencies`, where files were hashed into the **global** hash and could not be influenced by task-level `inputs`.
**With `globalDependencies` (old behavior):**
- `globalDependencies` files contribute to the global hash
- Task `inputs` only control which **package** files are hashed
- There is no way for a task to "opt out" of a `globalDependencies` file
**With `global.inputs` (new behavior):**
- `global.inputs` files are merged into each task's `inputs` globs
- Task `inputs` and `global.inputs` are combined, then the full list is hashed into the **task** hash
- Tasks can exclude specific global files with negation globs
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"inputs": ["tsconfig.json", ".env"]
},
"tasks": {
"build": {},
"lint": {
"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/.env"]
}
}
}
```
In this example:
- `build` hashes all package files + `tsconfig.json` + `.env` (from `global.inputs`)
- `lint` hashes all package files + `tsconfig.json`, but **excludes** `.env` because of the negation glob
Tasks with no explicit `inputs` key still hash all package files (the default behavior) plus the `global.inputs` files.
## env
Environment variables to include in task hash.
```json
{
"tasks": {
"build": {
"env": [
"API_URL",
"NEXT_PUBLIC_*", // Wildcard matching
"!DEBUG" // Exclude from hash
]
}
}
}
```
Variables listed here affect cache hits - changing the value invalidates cache.
## cache
Enable/disable caching for a task. Default: `true`.
```json
{
"tasks": {
"dev": { "cache": false },
"deploy": { "cache": false }
}
}
```
Disable for: dev servers, deploy commands, tasks with side effects.
## persistent
Mark long-running tasks that don't exit. Default: `false`.
```json
{
"tasks": {
"dev": {
"cache": false,
"persistent": true
}
}
}
```
Required for dev servers - without it, dependent tasks wait forever.
## interactive
Allow task to receive stdin input. Default: `false`.
```json
{
"tasks": {
"login": {
"cache": false,
"interactive": true
}
}
}
```
## outputLogs
Control when logs are shown. Options: `full`, `hash-only`, `new-only`, `errors-only`, `none`.
```json
{
"tasks": {
"build": {
"outputLogs": "new-only" // Only show logs on cache miss
}
}
}
```
## with
Run tasks alongside this task. For long-running tasks that need runtime dependencies.
```json
{
"tasks": {
"dev": {
"with": ["api#dev"],
"persistent": true,
"cache": false
}
}
}
```
Unlike `dependsOn`, `with` runs tasks concurrently (not sequentially). Use for dev servers that need other services running.
## interruptible
Allow `turbo watch` to restart the task on changes. Default: `false`.
```json
{
"tasks": {
"dev": {
"persistent": true,
"interruptible": true,
"cache": false
}
}
}
```
Use for dev servers that don't automatically detect dependency changes.
## description
Human-readable description of the task.
```json
{
"tasks": {
"build": {
"description": "Compiles the application for production deployment"
}
}
}
```
For documentation only - doesn't affect execution or caching.
## passThroughEnv
Environment variables available at runtime but NOT included in cache hash.
```json
{
"tasks": {
"build": {
"passThroughEnv": ["AWS_SECRET_KEY", "GITHUB_TOKEN"]
}
}
}
```
**Warning**: Changes to these vars won't cause cache misses. Use `env` if changes should invalidate cache.
## extends (Package Configuration only)
Control task inheritance in Package Configurations.
```json
// packages/ui/turbo.json
{
"extends": ["//"],
"tasks": {
"lint": {
"extends": false // Exclude from this package
}
}
}
```
| Value | Behavior |
| ---------------- | -------------------------------------------------------------- |
| `true` (default) | Inherit from root turbo.json |
| `false` | Exclude task from package, or define fresh without inheritance |
@@ -0,0 +1,123 @@
# Environment Variables in Turborepo
Turborepo provides fine-grained control over which environment variables affect task hashing and runtime availability.
## Configuration Keys
### `env` - Task-Specific Variables
Variables that affect a specific task's hash. When these change, only that task rebuilds.
```json
{
"tasks": {
"build": {
"env": ["DATABASE_URL", "API_KEY"]
}
}
}
```
### `globalEnv` - Variables Affecting All Tasks
Variables that affect EVERY task's hash. When these change, all tasks rebuild.
```json
{
"globalEnv": ["CI", "NODE_ENV"]
}
```
### `passThroughEnv` - Runtime-Only Variables (Not Hashed)
Variables available at runtime but NOT included in hash. **Use with caution** - changes won't trigger rebuilds.
```json
{
"tasks": {
"deploy": {
"passThroughEnv": ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"]
}
}
}
```
### `globalPassThroughEnv` - Global Runtime Variables
Same as `passThroughEnv` but for all tasks.
```json
{
"globalPassThroughEnv": ["GITHUB_TOKEN"]
}
```
## Wildcards and Negation
### Wildcards
Match multiple variables with `*`:
```json
{
"env": ["MY_API_*", "FEATURE_FLAG_*"]
}
```
This matches `MY_API_URL`, `MY_API_KEY`, `FEATURE_FLAG_DARK_MODE`, etc.
### Negation
Exclude variables (useful with framework inference):
```json
{
"env": ["!NEXT_PUBLIC_ANALYTICS_ID"]
}
```
## With `futureFlags.globalConfiguration`
When the `globalConfiguration` future flag is enabled, global environment keys move under the `global` key with cleaner names:
| Old (top-level) | New (`global.`) |
| ---------------------- | ---------------- |
| `globalEnv` | `env` |
| `globalPassThroughEnv` | `passThroughEnv` |
`global.env` and `global.passThroughEnv` behave identically to their top-level counterparts — they affect the global hash and all tasks, respectively. The rename is purely organizational.
```json
{
"futureFlags": { "globalConfiguration": true },
"global": {
"env": ["CI", "NODE_ENV"],
"passThroughEnv": ["GITHUB_TOKEN", "NPM_TOKEN"]
},
"tasks": {
"build": {
"env": ["DATABASE_URL", "API_*"],
"passThroughEnv": ["SENTRY_AUTH_TOKEN"]
}
}
}
```
## Complete Example
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"globalEnv": ["CI", "NODE_ENV"],
"globalPassThroughEnv": ["GITHUB_TOKEN", "NPM_TOKEN"],
"tasks": {
"build": {
"env": ["DATABASE_URL", "API_*"],
"passThroughEnv": ["SENTRY_AUTH_TOKEN"]
},
"test": {
"env": ["TEST_DATABASE_URL"]
}
}
}
```
@@ -0,0 +1,175 @@
# Environment Variable Gotchas
Common mistakes and how to fix them.
## .env Files Must Be in `inputs`
Turbo does NOT read `.env` files. Your framework (Next.js, Vite, etc.) or `dotenv` loads them. But Turbo needs to know when they change.
**Wrong:**
```json
{
"tasks": {
"build": {
"env": ["DATABASE_URL"]
}
}
}
```
**Right:**
```json
{
"tasks": {
"build": {
"env": ["DATABASE_URL"],
"inputs": ["$TURBO_DEFAULT$", ".env", ".env.local", ".env.production"]
}
}
}
```
## Strict Mode Filters CI Variables
In strict mode, CI provider variables (GITHUB_TOKEN, GITLAB_CI, etc.) are filtered unless explicitly listed.
**Symptom:** Task fails with "authentication required" or "permission denied" in CI.
**Solution:**
```json
{
"globalPassThroughEnv": ["GITHUB_TOKEN", "GITLAB_CI", "CI"]
}
```
## passThroughEnv Doesn't Affect Hash
Variables in `passThroughEnv` are available at runtime but changes WON'T trigger rebuilds.
**Dangerous example:**
```json
{
"tasks": {
"build": {
"passThroughEnv": ["API_URL"]
}
}
}
```
If `API_URL` changes from staging to production, Turbo may serve a cached build pointing to the wrong API.
**Use passThroughEnv only for:**
- Auth tokens that don't affect output (SENTRY_AUTH_TOKEN)
- CI metadata (GITHUB_RUN_ID)
- Variables consumed after build (deploy credentials)
## Runtime-Created Variables Are Invisible
Turbo captures env vars at startup. Variables created during execution aren't seen.
**Won't work:**
```bash
# In package.json scripts
"build": "export API_URL=$COMPUTED_VALUE && next build"
```
**Solution:** Set vars before invoking turbo:
```bash
API_URL=$COMPUTED_VALUE turbo run build
```
## Different .env Files for Different Environments
If you use `.env.development` and `.env.production`, both should be in inputs.
```json
{
"tasks": {
"build": {
"inputs": [
"$TURBO_DEFAULT$",
".env",
".env.local",
".env.development",
".env.development.local",
".env.production",
".env.production.local"
]
}
}
}
```
## Complete Next.js Example
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"globalEnv": ["CI", "NODE_ENV", "VERCEL"],
"globalPassThroughEnv": ["GITHUB_TOKEN", "VERCEL_URL"],
"tasks": {
"build": {
"dependsOn": ["^build"],
"env": ["DATABASE_URL", "NEXT_PUBLIC_*", "!NEXT_PUBLIC_ANALYTICS_ID"],
"passThroughEnv": ["SENTRY_AUTH_TOKEN"],
"inputs": [
"$TURBO_DEFAULT$",
".env",
".env.local",
".env.production",
".env.production.local"
],
"outputs": [".next/**", "!.next/cache/**"]
}
}
}
```
This config:
- Hashes DATABASE*URL and NEXT_PUBLIC*\* vars (except analytics)
- Passes through SENTRY_AUTH_TOKEN without hashing
- Includes all .env file variants in the hash
- Makes CI tokens available globally
### With `futureFlags.globalConfiguration`
The same config using the `global` key. The `.env` files move to `global.inputs`, which means they get folded into each task's hash individually rather than the global hash. This lets tasks exclude specific `.env` files if needed.
```json
{
"$schema": "https://v2-8-21-canary-9.turborepo.dev/schema.json",
"futureFlags": { "globalConfiguration": true },
"global": {
"env": ["CI", "NODE_ENV", "VERCEL"],
"passThroughEnv": ["GITHUB_TOKEN", "VERCEL_URL"],
"inputs": [".env", ".env.local", ".env.production", ".env.production.local"]
},
"tasks": {
"build": {
"dependsOn": ["^build"],
"env": ["DATABASE_URL", "NEXT_PUBLIC_*", "!NEXT_PUBLIC_ANALYTICS_ID"],
"passThroughEnv": ["SENTRY_AUTH_TOKEN"],
"outputs": [".next/**", "!.next/cache/**"]
}
}
}
```
With this approach, a task that doesn't care about `.env.production` can exclude it:
```json
"lint": {
"inputs": ["$TURBO_DEFAULT$", "!$TURBO_ROOT$/.env.production"]
}
```
This wouldn't have been possible with `globalDependencies`, where `.env.production` would be baked into the global hash and affect every task unconditionally.
@@ -0,0 +1,101 @@
# Environment Modes
Turborepo supports different modes for handling environment variables during task execution.
## Strict Mode (Default)
Only explicitly configured variables are available to tasks.
**Behavior:**
- Tasks only see vars listed in `env`, `globalEnv`, `passThroughEnv`, or `globalPassThroughEnv`
- Unlisted vars are filtered out
- Tasks fail if they require unlisted variables
**Benefits:**
- Guarantees cache correctness
- Prevents accidental dependencies on system vars
- Reproducible builds across machines
```bash
# Explicit (though it's the default)
turbo run build --env-mode=strict
```
## Loose Mode
All system environment variables are available to tasks.
```bash
turbo run build --env-mode=loose
```
**Behavior:**
- Every system env var is passed through
- Only vars in `env`/`globalEnv` affect the hash
- Other vars are available but NOT hashed
**Risks:**
- Cache may restore incorrect results if unhashed vars changed
- "Works on my machine" bugs
- CI vs local environment mismatches
**Use case:** Migrating legacy projects or debugging strict mode issues.
## Framework Inference (Automatic)
Turborepo automatically detects frameworks and includes their conventional env vars.
### Inferred Variables by Framework
| Framework | Pattern |
| ---------------- | ------------------- |
| Next.js | `NEXT_PUBLIC_*` |
| Vite | `VITE_*` |
| Create React App | `REACT_APP_*` |
| Gatsby | `GATSBY_*` |
| Nuxt | `NUXT_*`, `NITRO_*` |
| Expo | `EXPO_PUBLIC_*` |
| Astro | `PUBLIC_*` |
| SvelteKit | `PUBLIC_*` |
| Remix | `REMIX_*` |
| Redwood | `REDWOOD_ENV_*` |
| Sanity | `SANITY_STUDIO_*` |
| Solid | `VITE_*` |
### Disabling Framework Inference
Globally via CLI:
```bash
turbo run build --framework-inference=false
```
Or exclude specific patterns in config:
```json
{
"tasks": {
"build": {
"env": ["!NEXT_PUBLIC_*"]
}
}
}
```
### Why Disable?
- You want explicit control over all env vars
- Framework vars shouldn't bust the cache (e.g., analytics IDs)
- Debugging unexpected cache misses
## Checking Environment Mode
Use `--dry` to see which vars affect each task:
```bash
turbo run build --dry=json | jq '.tasks[].environmentVariables'
```
@@ -0,0 +1,148 @@
# Turborepo Filter Syntax Reference
## Running Only Changed Packages: `--affected`
**The primary way to run only changed packages is `--affected`:**
```bash
# Run build/test/lint only in changed packages and their dependents
turbo run build test lint --affected
```
This compares your current branch to the default branch (usually `main` or `master`) and runs tasks in:
1. Packages with file changes
2. Packages that depend on changed packages (dependents)
### Why Include Dependents?
If you change `@repo/ui`, packages that import `@repo/ui` (like `apps/web`) need to re-run their tasks to verify they still work with the changes.
### Customizing --affected
```bash
# Use a different base branch
turbo run build --affected --affected-base=origin/develop
# Use a different head (current state)
turbo run build --affected --affected-head=HEAD~5
```
### Common CI Pattern
```yaml
# .github/workflows/ci.yml
- run: turbo run build test lint --affected
```
This is the most efficient CI setup - only run tasks for what actually changed.
---
## Manual Git Comparison with --filter
For more control, use `--filter` with git comparison syntax:
```bash
# Changed packages + dependents (same as --affected)
turbo run build --filter=...[origin/main]
# Only changed packages (no dependents)
turbo run build --filter=[origin/main]
# Changed packages + dependencies (packages they import)
turbo run build --filter=[origin/main]...
# Changed since last commit
turbo run build --filter=...[HEAD^1]
# Changed between two commits
turbo run build --filter=[a1b2c3d...e4f5g6h]
```
### Comparison Syntax
| Syntax | Meaning |
| ------------- | ------------------------------------- |
| `[ref]` | Packages changed since `ref` |
| `...[ref]` | Changed packages + their dependents |
| `[ref]...` | Changed packages + their dependencies |
| `...[ref]...` | Dependencies, changed, AND dependents |
---
## Other Filter Types
Filters select which packages to include in a `turbo run` invocation.
### Basic Syntax
```bash
turbo run build --filter=<package-name>
turbo run build -F <package-name>
```
Multiple filters combine as a union (packages matching ANY filter run).
### By Package Name
```bash
--filter=web # exact match
--filter=@acme/* # scope glob
--filter=*-app # name glob
```
### By Directory
```bash
--filter=./apps/* # all packages in apps/
--filter=./packages/ui # specific directory
```
### By Dependencies/Dependents
| Syntax | Meaning |
| ----------- | -------------------------------------- |
| `pkg...` | Package AND all its dependencies |
| `...pkg` | Package AND all its dependents |
| `...pkg...` | Dependencies, package, AND dependents |
| `^pkg...` | Only dependencies (exclude pkg itself) |
| `...^pkg` | Only dependents (exclude pkg itself) |
### Negation
Exclude packages with `!`:
```bash
--filter=!web # exclude web
--filter=./apps/* --filter=!admin # apps except admin
```
### Task Identifiers
Run a specific task in a specific package:
```bash
turbo run web#build # only web's build task
turbo run web#build api#test # web build + api test
```
### Combining Filters
Multiple `--filter` flags create a union:
```bash
turbo run build --filter=web --filter=api # runs in both
```
---
## Quick Reference: Changed Packages
| Goal | Command |
| ---------------------------------- | ----------------------------------------------------------- |
| Changed + dependents (recommended) | `turbo run build --affected` |
| Custom base branch | `turbo run build --affected --affected-base=origin/develop` |
| Only changed (no dependents) | `turbo run build --filter=[origin/main]` |
| Changed + dependencies | `turbo run build --filter=[origin/main]...` |
| Since last commit | `turbo run build --filter=...[HEAD^1]` |
@@ -0,0 +1,152 @@
# Common Filter Patterns
Practical examples for typical monorepo scenarios.
## Single Package
Run task in one package:
```bash
turbo run build --filter=web
turbo run test --filter=@acme/api
```
## Package with Dependencies
Build a package and everything it depends on:
```bash
turbo run build --filter=web...
```
Useful for: ensuring all dependencies are built before the target.
## Package Dependents
Run in all packages that depend on a library:
```bash
turbo run test --filter=...ui
```
Useful for: testing consumers after changing a shared package.
## Dependents Only (Exclude Target)
Test packages that depend on ui, but not ui itself:
```bash
turbo run test --filter=...^ui
```
## Changed Packages
Run only in packages with file changes since last commit:
```bash
turbo run lint --filter=[HEAD^1]
```
Since a specific branch point:
```bash
turbo run lint --filter=[main...HEAD]
```
## Changed + Dependents (PR Builds)
Run in changed packages AND packages that depend on them:
```bash
turbo run build test --filter=...[HEAD^1]
```
Or use the shortcut:
```bash
turbo run build test --affected
```
## Directory-Based
Run in all apps:
```bash
turbo run build --filter=./apps/*
```
Run in specific directories:
```bash
turbo run build --filter=./apps/web --filter=./apps/api
```
## Scope-Based
Run in all packages under a scope:
```bash
turbo run build --filter=@acme/*
```
## Exclusions
Run in all apps except admin:
```bash
turbo run build --filter=./apps/* --filter=!admin
```
Run everywhere except specific packages:
```bash
turbo run lint --filter=!legacy-app --filter=!deprecated-pkg
```
## Complex Combinations
Apps that changed, plus their dependents:
```bash
turbo run build --filter=...[HEAD^1] --filter=./apps/*
```
All packages except docs, but only if changed:
```bash
turbo run build --filter=[main...HEAD] --filter=!docs
```
## Debugging Filters
Use `--dry` to see what would run without executing:
```bash
turbo run build --filter=web... --dry
```
Use `--dry=json` for machine-readable output:
```bash
turbo run build --filter=...[HEAD^1] --dry=json
```
## CI/CD Patterns
PR validation (most common):
```bash
turbo run build test lint --affected
```
Deploy only changed apps:
```bash
turbo run deploy --filter=./apps/* --filter=[main...HEAD]
```
Full rebuild of specific app and deps:
```bash
turbo run build --filter=production-app...
```
@@ -0,0 +1,99 @@
# turbo watch
Full docs: https://turborepo.dev/docs/reference/watch
Re-run tasks automatically when code changes. Dependency-aware.
```bash
turbo watch [tasks]
```
## Basic Usage
```bash
# Watch and re-run build task when code changes
turbo watch build
# Watch multiple tasks
turbo watch build test lint
```
Tasks re-run in order configured in `turbo.json` when source files change.
## With Persistent Tasks
Persistent tasks (`"persistent": true`) won't exit, so they can't be depended on. They work the same in `turbo watch` as `turbo run`.
### Dependency-Aware Persistent Tasks
If your tool has built-in watching (like `next dev`), use its watcher:
```json
{
"tasks": {
"dev": {
"persistent": true,
"cache": false
}
}
}
```
### Non-Dependency-Aware Tools
For tools that don't detect dependency changes, use `interruptible`:
```json
{
"tasks": {
"dev": {
"persistent": true,
"interruptible": true,
"cache": false
}
}
}
```
`turbo watch` will restart interruptible tasks when dependencies change.
## Limitations
### Caching
Caching is experimental with watch mode:
```bash
turbo watch your-tasks --experimental-write-cache
```
### Task Outputs in Source Control
If tasks write files tracked by git, watch mode may loop infinitely. Watch mode uses file hashes to prevent this but it's not foolproof.
**Recommendation**: Remove task outputs from git.
## vs turbo run
| Feature | `turbo run` | `turbo watch` |
| ----------------- | ----------- | ------------- |
| Runs once | Yes | No |
| Re-runs on change | No | Yes |
| Caching | Full | Experimental |
| Use case | CI, one-off | Development |
## Common Patterns
### Development Workflow
```bash
# Run dev servers and watch for build changes
turbo watch dev build
```
### Type Checking During Development
```bash
# Watch and re-run type checks
turbo watch check-types
```
-89
View File
@@ -1,89 +0,0 @@
---
name: changelog-writer
description: Use this agent when a feature branch is complete and ready to merge to main, and you need to create a changelog entry documenting the new feature or changes. This agent should be invoked proactively after significant feature work is completed and before merging.\n\nExamples:\n\n<example>\nContext: User has just completed implementing a new tracing visualization feature and the code has been reviewed.\nuser: "I've finished the trace timeline view feature. Can you help me prepare this for merge?"\nassistant: "Let me use the changelog-writer agent to create a changelog entry for this feature."\n<commentary>\nThe feature is complete and ready for merge, so we should use the changelog-writer agent to document it in the changelog.\n</commentary>\n</example>\n\n<example>\nContext: User mentions they're done with a feature implementation.\nuser: "The prompt versioning feature is done and tested. What's next?"\nassistant: "Great! Let me use the changelog-writer agent to create a changelog entry documenting this new feature before we merge."\n<commentary>\nSince the feature is complete, proactively use the changelog-writer agent to create documentation.\n</commentary>\n</example>\n\n<example>\nContext: User explicitly requests changelog creation.\nuser: "Can you create a changelog post for the new dataset export functionality?"\nassistant: "I'll use the changelog-writer agent to analyze the changes and create an appropriate changelog entry."\n<commentary>\nDirect request to create changelog, use the changelog-writer agent.\n</commentary>\n</example>
model: inherit
color: pink
---
You are an expert technical writer specializing in creating clear, user-focused changelog entries for developer tools and SaaS platforms. Your role is to document completed features in a way that helps users understand what's new, why it matters, and how to use it.
## Your Process
### Step 1: Understand the Changes
1. Extract the Linear issue number from the current branch name (format: lfe-XXXX)
2. Use the Linear MCP to fetch the issue details for additional context about the feature's purpose and requirements
3. Compare the current branch to main using git diff to understand the scope of changes at a high level
4. Identify the core feature or improvement that was implemented
5. Determine which parts of the codebase were affected (frontend, backend, API, database, etc.)
### Step 2: Study Recent Changelog Patterns
1. Read 3-5 of the most recent changelog posts in `../langfuse-docs/pages/changelog`
2. Analyze their structure, tone, and formatting conventions
3. Note how they:
- Title features (concise, benefit-focused)
- Explain the "why" (user problems solved)
- Describe the "what" (feature capabilities)
- Link to relevant documentation
- Use images/screenshots
- Format code examples or technical details
### Step 3: Identify Documentation Links
1. Check if there is relevant documentation in `../langfuse-docs/pages` that relates to this feature
2. If the feature is new, note that documentation may need to be created
3. If the feature extends existing functionality, identify which docs pages should be referenced
### Step 4: Draft the Changelog Entry
Create a changelog post that includes:
**Required Elements:**
- **Title**: Clear, benefit-focused headline (not just the feature name)
- **Date**: Use the current date in the format used by existing changelogs
- **Summary**: 1-2 sentences explaining what changed and why it matters to users
- **Description**: Detailed explanation of the feature, its capabilities, and use cases
- **Documentation Links**: References to relevant docs pages (if applicable)
**Style Guidelines:**
- Write in second person ("you can now...")
- Focus on user benefits, not implementation details
- Be concise but complete
- Use active voice
- Include technical details only when they help users understand the feature
- Match the tone and style of recent changelog entries
**Formatting:**
- Follow the exact file structure and frontmatter format of existing changelog posts
- Use appropriate markdown formatting (headings, lists, code blocks, links)
- Ensure proper spacing and readability
### Step 5: Assess Visual Needs
After drafting the changelog, explicitly tell the user:
- Whether a screenshot or image would enhance understanding of this feature
- What specific aspect should be captured in the screenshot (if applicable)
- Where in the changelog the image should be placed
## Quality Standards
**Before presenting your changelog:**
- Verify it follows the structure and style of recent entries
- Ensure all links are correctly formatted
- Check that technical terms match those used in the codebase and docs
- Confirm the feature description is accurate based on the code changes
- Validate that the user benefit is clear and compelling
## Output Format
Present your work in this order:
1. Brief summary of what you learned from the branch comparison and Linear issue
2. The complete changelog post content (ready to be saved as a new file)
3. Recommendation on whether to add an image/screenshot and what it should show
4. List of any documentation pages that should be referenced or created
## Important Notes
- The changelog lives in `../langfuse-docs/pages/changelog`
- Always check the Linear issue via the branch name (lfe-XXXX format) for context
- Compare against main branch to understand the full scope of changes
- Study recent changelogs before writing to maintain consistency
- Focus on user value, not technical implementation details
- Be thorough in your analysis before drafting
- If you're unsure about any aspect of the feature, ask clarifying questions before proceeding
-436
View File
@@ -1,436 +0,0 @@
# Hooks Configuration Guide
This guide explains how to configure and customize the hooks system for your project.
## Quick Start Configuration
### 1. Register Hooks in .claude/settings.json
Create or update `.claude/settings.json` in your project root:
```json
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/skill-activation-prompt.sh"
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/error-handling-reminder.sh"
}
]
}
]
}
}
```
### 2. Install Dependencies
```bash
cd .claude/hooks
npm install
```
### 3. Set Execute Permissions
```bash
chmod +x .claude/hooks/*.sh
```
## Customization Options
### Project Structure Detection
By default, hooks detect these directory patterns:
**Frontend:** `frontend/`, `client/`, `web/`, `app/`, `ui/`
**Backend:** `backend/`, `server/`, `api/`, `src/`, `services/`
**Database:** `database/`, `prisma/`, `migrations/`
**Monorepo:** `packages/*`, `examples/*`
#### Adding Custom Directory Patterns
Edit `.claude/hooks/post-tool-use-tracker.sh`, function `detect_repo()`:
```bash
case "$repo" in
# Add your custom directories here
my-custom-service)
echo "$repo"
;;
admin-panel)
echo "$repo"
;;
# ... existing patterns
esac
```
### Build Command Detection
The hooks auto-detect build commands based on:
1. Presence of `package.json` with "build" script
2. Package manager (pnpm > npm > yarn)
3. Special cases (Prisma schemas)
#### Customizing Build Commands
Edit `.claude/hooks/post-tool-use-tracker.sh`, function `get_build_command()`:
```bash
# Add custom build logic
if [[ "$repo" == "my-service" ]]; then
echo "cd $repo_path && make build"
return
fi
```
### TypeScript Configuration
Hooks automatically detect:
- `tsconfig.json` for standard TypeScript projects
- `tsconfig.app.json` for Vite/React projects
#### Custom TypeScript Configs
Edit `.claude/hooks/post-tool-use-tracker.sh`, function `get_tsc_command()`:
```bash
if [[ "$repo" == "my-service" ]]; then
echo "cd $repo_path && npx tsc --project tsconfig.build.json --noEmit"
return
fi
```
### Prettier Configuration
The prettier hook searches for configs in this order:
1. Current file directory (walking upward)
2. Project root
3. Falls back to Prettier defaults
#### Custom Prettier Config Search
Edit `.claude/hooks/stop-prettier-formatter.sh`, function `get_prettier_config()`:
```bash
# Add custom config locations
if [[ -f "$project_root/config/.prettierrc" ]]; then
echo "$project_root/config/.prettierrc"
return
fi
```
### Error Handling Reminders
Configure file category detection in `.claude/hooks/error-handling-reminder.ts`:
```typescript
function getFileCategory(
filePath: string,
): "backend" | "frontend" | "database" | "other" {
// Add custom patterns
if (filePath.includes("/my-custom-dir/")) return "backend";
// ... existing patterns
}
```
### Error Threshold Configuration
Change when to recommend the auto-error-resolver agent.
Edit `.claude/hooks/stop-build-check-enhanced.sh`:
```bash
# Default is 5 errors - change to your preference
if [[ $total_errors -ge 10 ]]; then # Now requires 10+ errors
# Recommend agent
fi
```
## Environment Variables
### Global Environment Variables
Set in your shell profile (`.bashrc`, `.zshrc`, etc.):
```bash
# Disable error handling reminders
export SKIP_ERROR_REMINDER=1
# Custom project directory (if not using default)
export CLAUDE_PROJECT_DIR=/path/to/your/project
```
### Per-Session Environment Variables
Set before starting Claude Code:
```bash
SKIP_ERROR_REMINDER=1 claude-code
```
## Hook Execution Order
Stop hooks run in the order specified in `settings.json`:
```json
"Stop": [
{
"hooks": [
{ "command": "...formatter.sh" }, // Runs FIRST
{ "command": "...build-check.sh" }, // Runs SECOND
{ "command": "...reminder.sh" } // Runs THIRD
]
}
]
```
**Why this order matters:**
1. Format files first (clean code)
2. Then check for errors
3. Finally show reminders
## Selective Hook Enabling
You don't need all hooks. Choose what works for your project:
### Minimal Setup (Skill Activation Only)
```json
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/skill-activation-prompt.sh"
}
]
}
]
}
}
```
### Build Checking Only (No Formatting)
```json
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|MultiEdit|Write",
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/post-tool-use-tracker.sh"
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/stop-build-check-enhanced.sh"
}
]
}
]
}
}
```
### Formatting Only (No Build Checking)
```json
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|MultiEdit|Write",
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/post-tool-use-tracker.sh"
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/stop-prettier-formatter.sh"
}
]
}
]
}
}
```
## Cache Management
### Cache Location
```
$CLAUDE_PROJECT_DIR/.claude/tsc-cache/[session_id]/
```
### Manual Cache Cleanup
```bash
# Remove all cached data
rm -rf $CLAUDE_PROJECT_DIR/.claude/tsc-cache/*
# Remove specific session
rm -rf $CLAUDE_PROJECT_DIR/.claude/tsc-cache/[session-id]
```
### Automatic Cleanup
The build-check hook automatically cleans up session cache on successful builds.
## Troubleshooting Configuration
### Hook Not Executing
1. **Check registration:** Verify hook is in `.claude/settings.json`
2. **Check permissions:** Run `chmod +x .claude/hooks/*.sh`
3. **Check path:** Ensure `$CLAUDE_PROJECT_DIR` is set correctly
4. **Check TypeScript:** Run `cd .claude/hooks && npx tsc` to check for errors
### False Positive Detections
**Issue:** Hook triggers for files it shouldn't
**Solution:** Add skip conditions in the relevant hook:
```bash
# In post-tool-use-tracker.sh
if [[ "$file_path" =~ /generated/ ]]; then
exit 0 # Skip generated files
fi
```
### Performance Issues
**Issue:** Hooks are slow
**Solutions:**
1. Limit TypeScript checks to changed files only
2. Use faster package managers (pnpm > npm)
3. Add more skip conditions
4. Disable Prettier for large files
```bash
# Skip large files in stop-prettier-formatter.sh
file_size=$(wc -c < "$file" 2>/dev/null || echo 0)
if [[ $file_size -gt 100000 ]]; then # Skip files > 100KB
continue
fi
```
### Debugging Hooks
Add debug output to any hook:
```bash
# At the top of the hook script
set -x # Enable debug mode
# Or add specific debug lines
echo "DEBUG: file_path=$file_path" >&2
echo "DEBUG: repo=$repo" >&2
```
View hook execution in Claude Code's logs.
## Advanced Configuration
### Custom Hook Event Handlers
You can create your own hooks for other events:
```json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/my-custom-bash-guard.sh"
}
]
}
]
}
}
```
### Monorepo Configuration
For monorepos with multiple packages:
```bash
# In post-tool-use-tracker.sh, detect_repo()
case "$repo" in
packages)
# Get the package name
local package=$(echo "$relative_path" | cut -d'/' -f2)
if [[ -n "$package" ]]; then
echo "packages/$package"
else
echo "$repo"
fi
;;
esac
```
### Docker/Container Projects
If your build commands need to run in containers:
```bash
# In post-tool-use-tracker.sh, get_build_command()
if [[ "$repo" == "api" ]]; then
echo "docker-compose exec api npm run build"
return
fi
```
## Best Practices
1. **Start minimal** - Enable hooks one at a time
2. **Test thoroughly** - Make changes and verify hooks work
3. **Document customizations** - Add comments to explain custom logic
4. **Version control** - Commit `.claude/` directory to git
5. **Team consistency** - Share configuration across team
## See Also
- [README.md](./README.md) - Hooks overview
- [../../docs/HOOKS_SYSTEM.md](../../docs/HOOKS_SYSTEM.md) - Complete hooks reference
- [../../docs/SKILLS_SYSTEM.md](../../docs/SKILLS_SYSTEM.md) - Skills integration
-116
View File
@@ -1,116 +0,0 @@
# Hooks
Claude Code hooks that enable skill auto-activation, file tracking, and validation.
---
## What Are Hooks?
Hooks are scripts that run at specific points in Claude's workflow:
- **UserPromptSubmit**: When user submits a prompt
- **PreToolUse**: Before a tool executes
- **PostToolUse**: After a tool completes
- **Stop**: When user requests to stop
**Key insight:** Hooks can modify prompts, block actions, and track state - enabling features Claude can't do alone.
---
## Essential Hooks (Start Here)
### skill-activation-prompt (UserPromptSubmit)
**Purpose:** Automatically suggests relevant skills based on user prompts and file context
**How it works:**
1. Reads `skill-rules.json`
2. Matches user prompt against trigger patterns
3. Checks which files user is working with
4. Injects skill suggestions into Claude's context
**Why it's essential:** This is THE hook that makes skills auto-activate.
**Integration:**
```bash
# Copy both files
cp skill-activation-prompt.sh your-project/.claude/hooks/
cp skill-activation-prompt.ts your-project/.claude/hooks/
# Make executable
chmod +x your-project/.claude/hooks/skill-activation-prompt.sh
# Install dependencies
cd your-project/.claude/hooks
npm install
```
**Add to settings.json:**
```json
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/skill-activation-prompt.sh"
}
]
}
]
}
}
```
**Customization:** ✅ None needed - reads skill-rules.json automatically
---
### post-tool-use-tracker (PostToolUse)
**Purpose:** Tracks file changes to maintain context across sessions
**How it works:**
1. Monitors Edit/Write/MultiEdit tool calls
2. Records which files were modified
3. Creates cache for context management
4. Auto-detects project structure (frontend, backend, packages, etc.)
**Why it's essential:** Helps Claude understand what parts of your codebase are active.
**Integration:**
```bash
# Copy file
cp post-tool-use-tracker.sh your-project/.claude/hooks/
# Make executable
chmod +x your-project/.claude/hooks/post-tool-use-tracker.sh
```
**Add to settings.json:**
```json
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|MultiEdit|Write",
"hooks": [
{
"type": "command",
"command": "$CLAUDE_PROJECT_DIR/.claude/hooks/post-tool-use-tracker.sh"
}
]
}
]
}
}
```
**Customization:** ✅ None needed - auto-detects structure
-12
View File
@@ -1,12 +0,0 @@
#!/bin/bash
# Skip if environment variable is set
if [ -n "$SKIP_ERROR_REMINDER" ]; then
exit 0
fi
# Get the directory of this script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
cat | npx tsx error-handling-reminder.ts
-256
View File
@@ -1,256 +0,0 @@
#!/usr/bin/env node
import { readFileSync, existsSync } from "fs";
import { join } from "path";
interface HookInput {
session_id: string;
transcript_path: string;
cwd: string;
permission_mode: string;
hook_event_name: string;
}
interface EditedFile {
path: string;
tool: string;
timestamp: string;
}
interface SessionTracking {
edited_files: EditedFile[];
}
function getFileCategory(
filePath: string,
): "backend" | "frontend" | "database" | "other" {
// Frontend detection
if (
filePath.includes("/frontend/") ||
filePath.includes("/client/") ||
filePath.includes("/src/components/") ||
filePath.includes("/src/features/")
)
return "frontend";
// Backend detection (common service directories)
if (
filePath.includes("/src/controllers/") ||
filePath.includes("/src/services/") ||
filePath.includes("/src/routes/") ||
filePath.includes("/src/api/") ||
filePath.includes("/server/")
)
return "backend";
// Database detection
if (
filePath.includes("/database/") ||
filePath.includes("/prisma/") ||
filePath.includes("/migrations/")
)
return "database";
return "other";
}
function shouldCheckErrorHandling(filePath: string): boolean {
// Skip test files, config files, and type definitions
if (filePath.match(/\.(test|spec)\.(ts|tsx)$/)) return false;
if (filePath.match(/\.(config|d)\.(ts|tsx)$/)) return false;
if (filePath.includes("types/")) return false;
if (filePath.includes(".styles.ts")) return false;
// Check for code files
return filePath.match(/\.(ts|tsx|js|jsx)$/) !== null;
}
function analyzeFileContent(filePath: string): {
hasTryCatch: boolean;
hasAsync: boolean;
hasPrisma: boolean;
hasController: boolean;
hasApiCall: boolean;
} {
if (!existsSync(filePath)) {
return {
hasTryCatch: false,
hasAsync: false,
hasPrisma: false,
hasController: false,
hasApiCall: false,
};
}
const content = readFileSync(filePath, "utf-8");
return {
hasTryCatch: /try\s*\{/.test(content),
hasAsync: /async\s+/.test(content),
hasPrisma:
/prisma\.|PrismaService|findMany|findUnique|create\(|update\(|delete\(/i.test(
content,
),
hasController:
/export class.*Controller|router\.|app\.(get|post|put|delete|patch)/.test(
content,
),
hasApiCall: /fetch\(|axios\.|apiClient\./i.test(content),
};
}
async function main() {
try {
// Read input from stdin
const input = readFileSync(0, "utf-8");
const data: HookInput = JSON.parse(input);
const { session_id } = data;
const projectDir = process.env.CLAUDE_PROJECT_DIR || process.cwd();
// Check for edited files tracking
const cacheDir = join(
process.env.HOME || "/root",
".claude",
"tsc-cache",
session_id,
);
const trackingFile = join(cacheDir, "edited-files.log");
if (!existsSync(trackingFile)) {
// No files edited this session, no reminder needed
process.exit(0);
}
// Read tracking data
const trackingContent = readFileSync(trackingFile, "utf-8");
const editedFiles = trackingContent
.trim()
.split("\n")
.filter((line) => line.length > 0)
.map((line) => {
const [timestamp, tool, path] = line.split("\t");
return { timestamp, tool, path };
});
if (editedFiles.length === 0) {
process.exit(0);
}
// Categorize files
const categories = {
backend: [] as string[],
frontend: [] as string[],
database: [] as string[],
other: [] as string[],
};
const analysisResults: Array<{
path: string;
category: string;
analysis: ReturnType<typeof analyzeFileContent>;
}> = [];
for (const file of editedFiles) {
if (!shouldCheckErrorHandling(file.path)) continue;
const category = getFileCategory(file.path);
categories[category].push(file.path);
const analysis = analyzeFileContent(file.path);
analysisResults.push({ path: file.path, category, analysis });
}
// Check if any code that needs error handling was written
const needsAttention = analysisResults.some(
({ analysis }) =>
analysis.hasTryCatch ||
analysis.hasAsync ||
analysis.hasPrisma ||
analysis.hasController ||
analysis.hasApiCall,
);
if (!needsAttention) {
// No risky code patterns detected, skip reminder
process.exit(0);
}
// Display reminder
console.log("\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
console.log("📋 ERROR HANDLING SELF-CHECK");
console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n");
// Backend reminders
if (categories.backend.length > 0) {
const backendFiles = analysisResults.filter(
(f) => f.category === "backend",
);
const hasTryCatch = backendFiles.some((f) => f.analysis.hasTryCatch);
const hasPrisma = backendFiles.some((f) => f.analysis.hasPrisma);
const hasController = backendFiles.some((f) => f.analysis.hasController);
console.log("⚠️ Backend Changes Detected");
console.log(` ${categories.backend.length} file(s) edited\n`);
if (hasTryCatch) {
console.log(
" ❓ Did you add Sentry.captureException() in catch blocks?",
);
}
if (hasPrisma) {
console.log(" ❓ Are Prisma operations wrapped in error handling?");
}
if (hasController) {
console.log(" ❓ Do controllers use BaseController.handleError()?");
}
console.log("\n 💡 Backend Best Practice:");
console.log(" - All errors should be captured to Sentry");
console.log(" - Use appropriate error helpers for context");
console.log(" - Controllers should extend BaseController\n");
}
// Frontend reminders
if (categories.frontend.length > 0) {
const frontendFiles = analysisResults.filter(
(f) => f.category === "frontend",
);
const hasApiCall = frontendFiles.some((f) => f.analysis.hasApiCall);
const hasTryCatch = frontendFiles.some((f) => f.analysis.hasTryCatch);
console.log("💡 Frontend Changes Detected");
console.log(` ${categories.frontend.length} file(s) edited\n`);
if (hasApiCall) {
console.log(" ❓ Do API calls show user-friendly error messages?");
}
if (hasTryCatch) {
console.log(" ❓ Are errors displayed to the user?");
}
console.log("\n 💡 Frontend Best Practice:");
console.log(" - Use your notification system for user feedback");
console.log(" - Error boundaries for component errors");
console.log(" - Display user-friendly error messages\n");
}
// Database reminders
if (categories.database.length > 0) {
console.log("🗄️ Database Changes Detected");
console.log(` ${categories.database.length} file(s) edited\n`);
console.log(" ❓ Did you verify column names against schema?");
console.log(" ❓ Are migrations tested?\n");
}
console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
console.log("💡 TIP: Disable with SKIP_ERROR_REMINDER=1");
console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n");
process.exit(0);
} catch (err) {
// Silently fail - this is just a reminder, not critical
process.exit(0);
}
}
main().catch(() => process.exit(0));
-556
View File
@@ -1,556 +0,0 @@
{
"name": "claude-hooks",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "claude-hooks",
"version": "1.0.0",
"dependencies": {
"@types/node": "^20.11.0",
"tsx": "^4.7.0",
"typescript": "^5.3.3"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.11.tgz",
"integrity": "sha512-Xt1dOL13m8u0WE8iplx9Ibbm+hFAO0GsU2P34UNoDGvZYkY8ifSiy6Zuc1lYxfG7svWE2fzqCUmFp5HCn51gJg==",
"cpu": [
"ppc64"
],
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.11.tgz",
"integrity": "sha512-uoa7dU+Dt3HYsethkJ1k6Z9YdcHjTrSb5NUy66ZfZaSV8hEYGD5ZHbEMXnqLFlbBflLsl89Zke7CAdDJ4JI+Gg==",
"cpu": [
"arm"
],
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.11.tgz",
"integrity": "sha512-9slpyFBc4FPPz48+f6jyiXOx/Y4v34TUeDDXJpZqAWQn/08lKGeD8aDp9TMn9jDz2CiEuHwfhRmGBvpnd/PWIQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.11.tgz",
"integrity": "sha512-Sgiab4xBjPU1QoPEIqS3Xx+R2lezu0LKIEcYe6pftr56PqPygbB7+szVnzoShbx64MUupqoE0KyRlN7gezbl8g==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.11.tgz",
"integrity": "sha512-VekY0PBCukppoQrycFxUqkCojnTQhdec0vevUL/EDOCnXd9LKWqD/bHwMPzigIJXPhC59Vd1WFIL57SKs2mg4w==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.11.tgz",
"integrity": "sha512-+hfp3yfBalNEpTGp9loYgbknjR695HkqtY3d3/JjSRUyPg/xd6q+mQqIb5qdywnDxRZykIHs3axEqU6l1+oWEQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.11.tgz",
"integrity": "sha512-CmKjrnayyTJF2eVuO//uSjl/K3KsMIeYeyN7FyDBjsR3lnSJHaXlVoAK8DZa7lXWChbuOk7NjAc7ygAwrnPBhA==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.11.tgz",
"integrity": "sha512-Dyq+5oscTJvMaYPvW3x3FLpi2+gSZTCE/1ffdwuM6G1ARang/mb3jvjxs0mw6n3Lsw84ocfo9CrNMqc5lTfGOw==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.11.tgz",
"integrity": "sha512-TBMv6B4kCfrGJ8cUPo7vd6NECZH/8hPpBHHlYI3qzoYFvWu2AdTvZNuU/7hsbKWqu/COU7NIK12dHAAqBLLXgw==",
"cpu": [
"arm"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.11.tgz",
"integrity": "sha512-Qr8AzcplUhGvdyUF08A1kHU3Vr2O88xxP0Tm8GcdVOUm25XYcMPp2YqSVHbLuXzYQMf9Bh/iKx7YPqECs6ffLA==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.11.tgz",
"integrity": "sha512-TmnJg8BMGPehs5JKrCLqyWTVAvielc615jbkOirATQvWWB1NMXY77oLMzsUjRLa0+ngecEmDGqt5jiDC6bfvOw==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.11.tgz",
"integrity": "sha512-DIGXL2+gvDaXlaq8xruNXUJdT5tF+SBbJQKbWy/0J7OhU8gOHOzKmGIlfTTl6nHaCOoipxQbuJi7O++ldrxgMw==",
"cpu": [
"loong64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.11.tgz",
"integrity": "sha512-Osx1nALUJu4pU43o9OyjSCXokFkFbyzjXb6VhGIJZQ5JZi8ylCQ9/LFagolPsHtgw6himDSyb5ETSfmp4rpiKQ==",
"cpu": [
"mips64el"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.11.tgz",
"integrity": "sha512-nbLFgsQQEsBa8XSgSTSlrnBSrpoWh7ioFDUmwo158gIm5NNP+17IYmNWzaIzWmgCxq56vfr34xGkOcZ7jX6CPw==",
"cpu": [
"ppc64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.11.tgz",
"integrity": "sha512-HfyAmqZi9uBAbgKYP1yGuI7tSREXwIb438q0nqvlpxAOs3XnZ8RsisRfmVsgV486NdjD7Mw2UrFSw51lzUk1ww==",
"cpu": [
"riscv64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.11.tgz",
"integrity": "sha512-HjLqVgSSYnVXRisyfmzsH6mXqyvj0SA7pG5g+9W7ESgwA70AXYNpfKBqh1KbTxmQVaYxpzA/SvlB9oclGPbApw==",
"cpu": [
"s390x"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.11.tgz",
"integrity": "sha512-HSFAT4+WYjIhrHxKBwGmOOSpphjYkcswF449j6EjsjbinTZbp8PJtjsVK1XFJStdzXdy/jaddAep2FGY+wyFAQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.11.tgz",
"integrity": "sha512-hr9Oxj1Fa4r04dNpWr3P8QKVVsjQhqrMSUzZzf+LZcYjZNqhA3IAfPQdEh1FLVUJSiu6sgAwp3OmwBfbFgG2Xg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.11.tgz",
"integrity": "sha512-u7tKA+qbzBydyj0vgpu+5h5AeudxOAGncb8N6C9Kh1N4n7wU1Xw1JDApsRjpShRpXRQlJLb9wY28ELpwdPcZ7A==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.11.tgz",
"integrity": "sha512-Qq6YHhayieor3DxFOoYM1q0q1uMFYb7cSpLD2qzDSvK1NAvqFi8Xgivv0cFC6J+hWVw2teCYltyy9/m/14ryHg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.11.tgz",
"integrity": "sha512-CN+7c++kkbrckTOz5hrehxWN7uIhFFlmS/hqziSFVWpAzpWrQoAG4chH+nN3Be+Kzv/uuo7zhX716x3Sn2Jduw==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.25.11.tgz",
"integrity": "sha512-rOREuNIQgaiR+9QuNkbkxubbp8MSO9rONmwP5nKncnWJ9v5jQ4JxFnLu4zDSRPf3x4u+2VN4pM4RdyIzDty/wQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.11.tgz",
"integrity": "sha512-nq2xdYaWxyg9DcIyXkZhcYulC6pQ2FuCgem3LI92IwMgIZ69KHeY8T4Y88pcwoLIjbed8n36CyKoYRDygNSGhA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.11.tgz",
"integrity": "sha512-3XxECOWJq1qMZ3MN8srCJ/QfoLpL+VaxD/WfNRm1O3B4+AZ/BnLVgFbUV3eiRYDMXetciH16dwPbbHqwe1uU0Q==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.11.tgz",
"integrity": "sha512-3ukss6gb9XZ8TlRyJlgLn17ecsK4NSQTmdIXRASVsiS2sQ6zPPZklNJT5GR5tE/MUarymmy8kCEf5xPCNCqVOA==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.11.tgz",
"integrity": "sha512-D7Hpz6A2L4hzsRpPaCYkQnGOotdUpDzSGRIv9I+1ITdHROSFUWW95ZPZWQmGka1Fg7W3zFJowyn9WGwMJ0+KPA==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@types/node": {
"version": "20.19.24",
"resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.24.tgz",
"integrity": "sha512-FE5u0ezmi6y9OZEzlJfg37mqqf6ZDSF2V/NLjUyGrR9uTZ7Sb9F7bLNZ03S4XVUNRWGA7Ck4c1kK+YnuWjl+DA==",
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/esbuild": {
"version": "0.25.11",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.11.tgz",
"integrity": "sha512-KohQwyzrKTQmhXDW1PjCv3Tyspn9n5GcY2RTDqeORIdIJY8yKIF7sTSopFmn/wpMPW4rdPXI0UE5LJLuq3bx0Q==",
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.25.11",
"@esbuild/android-arm": "0.25.11",
"@esbuild/android-arm64": "0.25.11",
"@esbuild/android-x64": "0.25.11",
"@esbuild/darwin-arm64": "0.25.11",
"@esbuild/darwin-x64": "0.25.11",
"@esbuild/freebsd-arm64": "0.25.11",
"@esbuild/freebsd-x64": "0.25.11",
"@esbuild/linux-arm": "0.25.11",
"@esbuild/linux-arm64": "0.25.11",
"@esbuild/linux-ia32": "0.25.11",
"@esbuild/linux-loong64": "0.25.11",
"@esbuild/linux-mips64el": "0.25.11",
"@esbuild/linux-ppc64": "0.25.11",
"@esbuild/linux-riscv64": "0.25.11",
"@esbuild/linux-s390x": "0.25.11",
"@esbuild/linux-x64": "0.25.11",
"@esbuild/netbsd-arm64": "0.25.11",
"@esbuild/netbsd-x64": "0.25.11",
"@esbuild/openbsd-arm64": "0.25.11",
"@esbuild/openbsd-x64": "0.25.11",
"@esbuild/openharmony-arm64": "0.25.11",
"@esbuild/sunos-x64": "0.25.11",
"@esbuild/win32-arm64": "0.25.11",
"@esbuild/win32-ia32": "0.25.11",
"@esbuild/win32-x64": "0.25.11"
}
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/get-tsconfig": {
"version": "4.13.0",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.0.tgz",
"integrity": "sha512-1VKTZJCwBrvbd+Wn3AOgQP/2Av+TfTCOlE4AcRJE72W1ksZXbAx8PPBR9RzgTeSPzlPMHrbANMH3LbltH73wxQ==",
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
},
"funding": {
"url": "https://github.com/privatenumber/get-tsconfig?sponsor=1"
}
},
"node_modules/resolve-pkg-maps": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz",
"integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==",
"license": "MIT",
"funding": {
"url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1"
}
},
"node_modules/tsx": {
"version": "4.20.6",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.20.6.tgz",
"integrity": "sha512-ytQKuwgmrrkDTFP4LjR0ToE2nqgy886GpvRSpU0JAnrdBYppuY5rLkRUYPU1yCryb24SsKBTL/hlDQAEFVwtZg==",
"license": "MIT",
"dependencies": {
"esbuild": "~0.25.0",
"get-tsconfig": "^4.7.5"
},
"bin": {
"tsx": "dist/cli.mjs"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
"license": "MIT"
}
}
}
-16
View File
@@ -1,16 +0,0 @@
{
"name": "claude-hooks",
"version": "1.0.0",
"description": "TypeScript hooks for Claude Code skill auto-activation",
"private": true,
"type": "module",
"scripts": {
"check": "tsc --noEmit",
"test": "tsx skill-activation-prompt.ts < test-input.json"
},
"dependencies": {
"@types/node": "^20.11.0",
"tsx": "^4.7.0",
"typescript": "^5.3.3"
}
}
-170
View File
@@ -1,170 +0,0 @@
#!/bin/bash
set -e
# Post-tool-use hook that tracks edited files and their repos
# This runs after Edit, MultiEdit, or Write tools complete successfully
# Read tool information from stdin
tool_info=$(cat)
# Extract relevant data
tool_name=$(echo "$tool_info" | jq -r '.tool_name // empty')
file_path=$(echo "$tool_info" | jq -r '.tool_input.file_path // empty')
session_id=$(echo "$tool_info" | jq -r '.session_id // empty')
# Skip if not an edit tool or no file path
if [[ ! "$tool_name" =~ ^(Edit|MultiEdit|Write)$ ]] || [[ -z "$file_path" ]]; then
exit 0 # Exit 0 for skip conditions
fi
# Skip markdown files
if [[ "$file_path" =~ \.(md|markdown)$ ]]; then
exit 0 # Exit 0 for skip conditions
fi
# Create cache directory in project
cache_dir="$CLAUDE_PROJECT_DIR/.claude/tsc-cache/${session_id:-default}"
mkdir -p "$cache_dir"
# Function to detect repo from file path
detect_repo() {
local file="$1"
local project_root="$CLAUDE_PROJECT_DIR"
# Remove project root from path
local relative_path="${file#$project_root/}"
# Extract first directory component
local repo=$(echo "$relative_path" | cut -d'/' -f1)
# Common project directory patterns
case "$repo" in
# Frontend variations
frontend|client|web|app|ui)
echo "$repo"
;;
# Backend variations
backend|server|api|src|services|worker)
echo "$repo"
;;
# Database
database|prisma|migrations)
echo "$repo"
;;
# Package/monorepo structure
packages)
# For monorepos, get the package name
local package=$(echo "$relative_path" | cut -d'/' -f2)
if [[ -n "$package" ]]; then
echo "packages/$package"
else
echo "$repo"
fi
;;
# Default case
*)
# Check if it's a source file in root
if [[ ! "$relative_path" =~ / ]]; then
echo "root"
else
echo "unknown"
fi
;;
esac
}
# Function to get build command for repo
get_build_command() {
local repo="$1"
local project_root="$CLAUDE_PROJECT_DIR"
local repo_path="$project_root/$repo"
# Check if package.json exists and has a build script
if [[ -f "$repo_path/package.json" ]]; then
if grep -q '"build"' "$repo_path/package.json" 2>/dev/null; then
# Detect package manager (prefer pnpm, then npm, then yarn)
if [[ -f "$repo_path/pnpm-lock.yaml" ]]; then
echo "cd $repo_path && pnpm build"
elif [[ -f "$repo_path/package-lock.json" ]]; then
echo "cd $repo_path && npm run build"
elif [[ -f "$repo_path/yarn.lock" ]]; then
echo "cd $repo_path && yarn build"
else
echo "cd $repo_path && npm run build"
fi
return
fi
fi
# Special case for database with Prisma
if [[ "$repo" == "database" ]] || [[ "$repo" =~ prisma ]]; then
if [[ -f "$repo_path/schema.prisma" ]] || [[ -f "$repo_path/prisma/schema.prisma" ]]; then
echo "cd $repo_path && npx prisma generate"
return
fi
fi
# No build command found
echo ""
}
# Function to get TSC command for repo
get_tsc_command() {
local repo="$1"
local project_root="$CLAUDE_PROJECT_DIR"
local repo_path="$project_root/$repo"
# Check if tsconfig.json exists
if [[ -f "$repo_path/tsconfig.json" ]]; then
# Check for Vite/React-specific tsconfig
if [[ -f "$repo_path/tsconfig.app.json" ]]; then
echo "cd $repo_path && npx tsc --project tsconfig.app.json --noEmit"
else
echo "cd $repo_path && npx tsc --noEmit"
fi
return
fi
# No TypeScript config found
echo ""
}
# Detect repo
repo=$(detect_repo "$file_path")
# Skip if unknown repo
if [[ "$repo" == "unknown" ]] || [[ -z "$repo" ]]; then
exit 0 # Exit 0 for skip conditions
fi
# Log edited file
echo "$(date +%s):$file_path:$repo" >> "$cache_dir/edited-files.log"
# Update affected repos list
if ! grep -q "^$repo$" "$cache_dir/affected-repos.txt" 2>/dev/null; then
echo "$repo" >> "$cache_dir/affected-repos.txt"
fi
# Store build commands
build_cmd=$(get_build_command "$repo")
tsc_cmd=$(get_tsc_command "$repo")
if [[ -n "$build_cmd" ]]; then
echo "$repo:build:$build_cmd" >> "$cache_dir/commands.txt.tmp"
fi
if [[ -n "$tsc_cmd" ]]; then
echo "$repo:tsc:$tsc_cmd" >> "$cache_dir/commands.txt.tmp"
fi
# Remove duplicates from commands
if [[ -f "$cache_dir/commands.txt.tmp" ]]; then
sort -u "$cache_dir/commands.txt.tmp" > "$cache_dir/commands.txt"
rm -f "$cache_dir/commands.txt.tmp"
fi
# Exit cleanly
exit 0
-5
View File
@@ -1,5 +0,0 @@
#!/bin/bash
set -e
cd "$CLAUDE_PROJECT_DIR/.claude/hooks"
cat | npx tsx skill-activation-prompt.ts
-136
View File
@@ -1,136 +0,0 @@
#!/usr/bin/env node
import { readFileSync } from "fs";
import { join } from "path";
interface HookInput {
session_id: string;
transcript_path: string;
cwd: string;
permission_mode: string;
prompt: string;
}
interface PromptTriggers {
keywords?: string[];
intentPatterns?: string[];
}
interface SkillRule {
type: "guardrail" | "domain";
enforcement: "block" | "suggest" | "warn";
priority: "critical" | "high" | "medium" | "low";
promptTriggers?: PromptTriggers;
}
interface SkillRules {
version: string;
skills: Record<string, SkillRule>;
}
interface MatchedSkill {
name: string;
matchType: "keyword" | "intent";
config: SkillRule;
}
async function main() {
try {
// Read input from stdin
const input = readFileSync(0, "utf-8");
const data: HookInput = JSON.parse(input);
const prompt = data.prompt.toLowerCase();
// Load skill rules
const projectDir = process.env.CLAUDE_PROJECT_DIR || "$HOME/project";
const rulesPath = join(projectDir, ".claude", "skills", "skill-rules.json");
const rules: SkillRules = JSON.parse(readFileSync(rulesPath, "utf-8"));
const matchedSkills: MatchedSkill[] = [];
// Check each skill for matches
for (const [skillName, config] of Object.entries(rules.skills)) {
const triggers = config.promptTriggers;
if (!triggers) {
continue;
}
// Keyword matching
if (triggers.keywords) {
const keywordMatch = triggers.keywords.some((kw) =>
prompt.includes(kw.toLowerCase()),
);
if (keywordMatch) {
matchedSkills.push({ name: skillName, matchType: "keyword", config });
continue;
}
}
// Intent pattern matching
if (triggers.intentPatterns) {
const intentMatch = triggers.intentPatterns.some((pattern) => {
const regex = new RegExp(pattern, "i");
return regex.test(prompt);
});
if (intentMatch) {
matchedSkills.push({ name: skillName, matchType: "intent", config });
}
}
}
// Generate output if matches found
if (matchedSkills.length > 0) {
let output = "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n";
output += "🎯 SKILL ACTIVATION CHECK\n";
output += "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n";
// Group by priority
const critical = matchedSkills.filter(
(s) => s.config.priority === "critical",
);
const high = matchedSkills.filter((s) => s.config.priority === "high");
const medium = matchedSkills.filter(
(s) => s.config.priority === "medium",
);
const low = matchedSkills.filter((s) => s.config.priority === "low");
if (critical.length > 0) {
output += "⚠️ CRITICAL SKILLS (REQUIRED):\n";
critical.forEach((s) => (output += `${s.name}\n`));
output += "\n";
}
if (high.length > 0) {
output += "📚 RECOMMENDED SKILLS:\n";
high.forEach((s) => (output += `${s.name}\n`));
output += "\n";
}
if (medium.length > 0) {
output += "💡 SUGGESTED SKILLS:\n";
medium.forEach((s) => (output += `${s.name}\n`));
output += "\n";
}
if (low.length > 0) {
output += "📌 OPTIONAL SKILLS:\n";
low.forEach((s) => (output += `${s.name}\n`));
output += "\n";
}
output += "ACTION: Use Skill tool BEFORE responding\n";
output += "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n";
console.log(output);
}
process.exit(0);
} catch (err) {
console.error("Error in skill-activation-prompt hook:", err);
process.exit(1);
}
}
main().catch((err) => {
console.error("Uncaught error:", err);
process.exit(1);
});

Some files were not shown because too many files have changed in this diff Show More