Compare commits

...
1453 Commits
Author SHA1 Message Date
steffen911 c427791070 chore: release v3.96.2 2025-08-08 16:35:11 +02:00
Steffen SchmitzandGitHub 25220aef55 perf: allow sharding for trace upsert queue (#8396)
* perf: allow sharding for trace upsert queue

* chore: handle metrics for traceupsertqueue

* chore: remove outdated comment

* chore: test util upgrade

* chore: patch typing
2025-08-08 14:16:27 +00:00
Max Deichmann 8c02d5dc22 chore: release v3.96.1 2025-08-08 15:57:05 +02:00
Max DeichmannandGitHub 0b60076871 chore: upgrade form data (#8408) 2025-08-08 13:17:12 +00:00
marliessophieandGitHub b28a83531b fix(evals): display final status in peek view (#8406) 2025-08-08 13:12:38 +00:00
Steffen SchmitzandGitHub aa4d34ae66 chore: skip S3 list and legacy trace insert based on env flag (#8298)
* chore: skip S3 list and legacy trace insert based on env flag

* chore: adjust skip s3 list conditions

* chore; refactor traceUpserQueue entity assoication

* chore: reverse
2025-08-08 12:47:04 +00:00
Steffen SchmitzandGitHub cca352ad13 chore: remove trace null deletions (#8405) 2025-08-08 15:02:37 +02:00
Steffen SchmitzandGitHub 58c96cada7 chore: include boolean scores in the reference of scores-numeric (#8086) 2025-08-08 12:39:56 +00:00
Steffen SchmitzandGitHub 6f93389936 chore: adjust devcontainer setup to install deps via dockerfile (#8399)
* chore: adjust devcontainer setup to install deps via dockerfile

* Update Dockerfile
2025-08-08 12:10:05 +00:00
Max DeichmannandGitHub b8d9586490 chore: fix dev command (#8404) 2025-08-08 14:38:21 +02:00
Leo 534f5696ad chore: release v3.96.0 2025-08-08 14:20:05 +02:00
Leo WeigandandGitHub 4cebe2831c fix(onboarding): only show org onboarding for Cloud and more UX touches (#8392) 2025-08-08 14:17:49 +02:00
45a5f3b8a2 chore: upgrade slack dependency (#8402)
* chore: upgrade slack dependencies

* chore: upgrade slack deps and remove unnecessary deps from web/worker

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-08-08 14:05:41 +02:00
Max DeichmannandGitHub 8dea9b6a3a chore: remove explicit axios dependency (#8397)
remove axios
2025-08-08 11:38:42 +00:00
marliessophieandGitHub c5b781d3f6 fix(llm-connections): ensure dialog remains open when switching tabs (#8391)
* fix(llm-connections): ensure dialog remains open when switching tabs

* chore: lint
2025-08-08 09:52:48 +00:00
Leo WeigandandGitHub 07469c928d feat(cloud): new onboarding survey to better understand our users (#8374) 2025-08-08 10:37:13 +02:00
Marc KlingenandGitHub 5b10110dfe fix(auth): honor NEXT_PUBLIC_BASE_PATH in password reset verification (#8385) 2025-08-07 23:14:32 +00:00
Thorsten SpiekerandGitHub 0eb5d1c3c0 feat: pivot table sorting (#8214) 2025-08-08 00:12:09 +02:00
Max Deichmann 1585bf5e14 chore: release v3.95.2 2025-08-07 20:11:10 +02:00
Max DeichmannandGitHub dec6d6f975 chore: add openai 5 models (#8378) 2025-08-07 20:09:36 +02:00
Max DeichmannandGitHub 36fa7ea15e chore: improve typing of users (#8376)
push
2025-08-07 17:28:10 +00:00
Marlies Mayerhofer 6a4d56a5a9 chore: release v3.95.1 2025-08-07 18:32:29 +02:00
Marc KlingenandGitHub 9f0a40949e fix: check projectMembers:read for trpc members.byProjectId (#8373) 2025-08-07 17:55:57 +02:00
Steffen SchmitzandGitHub 10fdd9e172 chore: add env to move short-term reads to AMTs (#8367) 2025-08-07 15:26:11 +00:00
marliessophieandGitHub fc58add111 chore(rbac): fix types for resolveProjectRole (#8371)
* chore(rbac): fix types for resolveProjectRole

* chore: eslint
2025-08-07 15:18:22 +00:00
Steffen SchmitzandGitHub 10993838d5 perf: truncate IO in table overviews and skip internal JSON parse on TRPC (#8364)
* perf: truncate IO in table overviews and skip internal JSON parse on TRPC

* chore: fix imports

* chore: align test case
2025-08-07 15:09:35 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
148cd29e9f fix: handle invalid string lengths on traces_null table gracefully (#8369)
* fix: handle invalid string lengths on traces_null table gracefully

* Update worker/src/services/ClickhouseWriter/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: test dropped record truncation

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-07 14:25:00 +00:00
Steffen SchmitzandGitHub 7a353e99af chore: disable usage alert configuration temporarily (#8360) 2025-08-07 10:22:08 +00:00
Marc KlingenandGitHub dc52106feb chore: improve multi-tenant sso error message (#8363) 2025-08-07 12:44:06 +02:00
Marlies Mayerhofer b2f8eb7078 chore: release v3.95.0 2025-08-07 11:55:36 +02:00
026d8e2ba0 feat(annotation): support assigning users to queues (#8325)
* chore(annotation-queue): add AnnotationQueueMembership model and related database schema updates

* chore(annotation-queue): add AnnotationQueueMembership POST and DEL APIs

* feat(annotation-queue): implement user assignment functionality for annotation queues

* fix: role logic for users

* chore: rn to assignments

* fixup: rn to assignments

* chore(annotation-queue): enhance user assignment section with debounced search and display of assigned users

* feat(annotation-queue): add MultiSelectCombobox for user assignment and enhance DataTable with row class name functionality

* refactor(rbac): replace generateUserQuery with generateUserProjectRolesQuery and introduce utility functions for project role resolution

* fixup: filter query for user ids

* fixup: resolve project role test

* fix: validate user project role before adding to annotation assignment

* chore: remove code

* chore: fix

* chore: fix filter

* chore: fix filter

* tests for auth logic

* push: fix api route test

* tests for auth logic

* fix

* fix

* chore: push

* chore: rename

* docs: api types

* fix: formatting

* fix: formatting

* chore: add disabled prop to MultiSelectCombobox

* chore: rename migration

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-08-07 09:29:45 +00:00
Hassieb Pakzad 9a9f0b1743 chore: release v3.94.0 2025-08-07 10:30:22 +02:00
Hassieb PakzadandGitHub 74a510fbde feat(otel): add parsing for Vercel AI SDK (#8347)
* feat(otel): add parsing for Vercel AI SDK

* push

* push
2025-08-07 07:43:34 +00:00
Steffen SchmitzandGitHub c23b226e62 chore: move dashboard queries to new AMTs (#8308)
* chore: create background migrations for traces amt tables

* chore: add database migrations

* chore: move tests to use AMTs in one config

* chore: shift backfill from {} to map()

* chore: modify ingestion test timeouts

* chore: keep minDate updates tracked

* chore: remove android lib to save swap space

* chore: check disk space

* chore: check mnt usage

* chore: skip swap to check runner performance

* chore: add timestamp lines

* chore: handle deletions

* chore: update test cases

* chore: linting

* chore: allow opt-in creation for background migratoin record

* chore: update comparison query

* chore: concurrency config

* chore: update names

* chore: update names

* chore: update tiemout

* chore: add pre-query logs

* chore: refactor long running query into util function

* chore: refactor query exists checks

* chore: set correct tables for clustered configs

* chore: increase number of retries

* chore: extend logging

* chore: gracefully handle aborted query

* chore: handle errors during execution and raise them

* chore; skip in progress header updates

* Update migrateTracesToTracesAMTs.ts

* chore: use default clickhouse settings for ingestion

* chore: add trace_id index on traces_all_amt

* chore: update sorting behaviour of batchAction test

* chore: drop logger

* chore: update test

* chore: lint and renames

* chore: update test cases

* chore: patch score tests

* chore: add final for the checkTraceExists behaviour

* chore: remove background migration insert

* chore: move dashboard queries to new AMTs

* chore: make sure to replace all references to old traces table

* chore: revert replacement behaviour

* chore: formatting

* chore: typo

* chore: lint and refactor

* chore: update import path

* chore: patch test behaviour

* chore: remove outdated comment
2025-08-06 13:57:15 +00:00
Steffen SchmitzandGitHub 2f50b38a68 chore: adjust stalled settings for eval execution queue (#8343)
* chore: tune queue stalled checks for higher reliability

* chore: only apply stalled to eval execution queue
2025-08-06 12:29:47 +00:00
marliessophieandGitHub 451ae15e00 feat(evaluator-form): add tooltips for evaluator usage and target in edit mode (#8340)
* feat(evaluator-form): add tooltip for existing evaluator usage in edit mode

* feat(evaluator-form): enhance target data label with tooltip for edit mode

* chore: lint
2025-08-06 10:35:43 +00:00
marliessophieandGitHub e34d81578b chore(evaluator-form): prevent form submission on Enter key press; hide controls in preview (#8337)
* chore(evaluator-form): prevent form submission on Enter key press

* chore: disable select control on evaluation traces preview
2025-08-06 08:28:13 +00:00
Max DeichmannandGitHub 214c9c7ed2 perf: increase clickhouse timeouts for blob storage exports (#8328)
push
2025-08-05 22:45:49 +00:00
Max DeichmannandGitHub 961b3bfa8d chore: truncate traces i/o for the traces table (#8326)
* push

* push

* push

* something

* something
2025-08-05 20:53:40 +00:00
Marc KlingenandGitHub a1dd5b22a2 docs: improve api reference for llm-connections api (#8327) 2025-08-05 20:43:45 +02:00
Marc KlingenandGitHub 49950f9706 chore: add validation to okta issuer (#8323) 2025-08-05 17:22:40 +00:00
Max DeichmannandGitHub cfdd0fdb73 chore: refactor IOTableCell (#8324) 2025-08-05 19:26:16 +02:00
Marc KlingenandGitHub 891e7e9716 feat(models): add claude-opus-4-1-20250805 (#8322) 2025-08-05 17:19:34 +00:00
Marc KlingenandGitHub 5b407dad53 feat(posthog-integration): add langfuse_trace_id to generations and scores (#8319)
feat(posthog-integration): add langfuse_trace_id to generations and scores
2025-08-05 15:25:31 +00:00
steffen911 26e3bf9a44 chore: release v3.93.0 2025-08-05 16:56:10 +02:00
Steffen SchmitzandGitHub 1f02e364e1 chore: create background migrations for traces amt tables (#7689)
* chore: create background migrations for traces amt tables

* chore: add database migrations

* chore: move tests to use AMTs in one config

* chore: shift backfill from {} to map()

* chore: modify ingestion test timeouts

* chore: keep minDate updates tracked

* chore: remove android lib to save swap space

* chore: check disk space

* chore: check mnt usage

* chore: skip swap to check runner performance

* chore: add timestamp lines

* chore: handle deletions

* chore: update test cases

* chore: linting

* chore: allow opt-in creation for background migratoin record

* chore: update comparison query

* chore: concurrency config

* chore: update names

* chore: update names

* chore: update tiemout

* chore: add pre-query logs

* chore: refactor long running query into util function

* chore: refactor query exists checks

* chore: set correct tables for clustered configs

* chore: increase number of retries

* chore: extend logging

* chore: gracefully handle aborted query

* chore: handle errors during execution and raise them

* chore; skip in progress header updates

* Update migrateTracesToTracesAMTs.ts

* chore: use default clickhouse settings for ingestion

* chore: add trace_id index on traces_all_amt

* chore: update sorting behaviour of batchAction test

* chore: drop logger

* chore: update test

* chore: lint and renames

* chore: update test cases

* chore: patch score tests

* chore: add final for the checkTraceExists behaviour

* chore: remove background migration insert

* chore: convert traces_mt to traces_null

* chore: patch tests

* chore: switch from minMap to maxMap

* chore: mark aggregation columns on traces as do not use

* chore: patch new test

* chore: switch to maxMap

* consistent naming
2025-08-05 14:20:49 +00:00
Leo WeigandandGitHub 90dca15d86 feat(settings): improve UX for managing LLM connections (#8312) 2025-08-05 14:30:23 +02:00
Steffen SchmitzandGitHub e3f8dc8bb3 fix(blob-storage): disable auto container check for Azure buckets for costsavings (#8317) 2025-08-05 11:44:03 +00:00
Hassieb Pakzad 585ede0919 chore: release v3.92.1 2025-08-05 11:24:29 +02:00
Hassieb PakzadandGitHub 0db425d120 fix(pagination): allow limit zero again (#8315) 2025-08-05 09:22:34 +00:00
Hassieb Pakzad 09e33c3059 chore: release v3.92.0 2025-08-05 10:01:57 +02:00
Hassieb PakzadandGitHub 66226011af feat(llm-api-keys): add public api methods (#8301)
* feat(llm-api-keys): add public api LIST and PATCH endpoints

* puhs

* push

* push

* add post endpoint

* add PUT endpoint

* push

* push
2025-08-05 07:32:18 +00:00
marliessophieandGitHub 571698ab2e fix(pretty-json-view): prevent infinite recursion and optimize child row processing (#8302) 2025-08-04 18:05:45 +00:00
Steffen SchmitzandGitHub 312066f735 chore: add operation_name tag to AMT query experiments (#8294) 2025-08-04 11:25:32 +00:00
marliessophieandGitHub 5abeaf8adb fix(dataset-run-items): fix pagination for consistent ordering across sources (#8292) 2025-08-04 10:23:48 +00:00
Steffen SchmitzandGitHub fac3c732de chore: optimize queries for session AMT accesses (#8289) 2025-08-04 09:55:17 +00:00
marliessophieandGitHub 5e2e3bb5fc feat(sessions-ui): allow filtering table by scores (#7858)
* fixup(sessions-ui): allow filtering table by scores

* fix(table-definitions): update scores label from "Scores (avg)" to "Scores (numeric)"

* feat: support scores filters on sessions

* chore

* chore: fix duplicate in query

* chore: hide whitespace

* chore: fix
2025-08-04 09:36:48 +00:00
Max Deichmann 0564df8e51 chore: release v3.91.0 2025-08-04 11:33:20 +02:00
Steffen SchmitzandGitHub a2801a3be9 chore: add experiment tags to clickhouse queries (#8288) 2025-08-04 09:17:07 +00:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
075eb58ecb chore: add fern JS SDK generator (#8095)
* chore: add fern JS SDK generator

* push

* return java generator

* push

* push

* push

* Update worker/src/services/IngestionService/tests/IngestionService.integration.test.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker/src/services/IngestionService/tests/IngestionService.integration.test.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* puhs

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-04 08:49:43 +00:00
Steffen SchmitzandGitHub 674d66d179 chore: record AMT duration distribution (#8287) 2025-08-04 10:32:10 +02:00
marliessophieandGitHub 163f2a02ff fix(dataset-run-items): include run metadata for runs without dataset_run_items (#8267)
* fix(dataset-run-items): include run metadata for runs without dataset_run_items

* chore: drop name from metrics query
2025-08-04 08:12:46 +00:00
Steffen SchmitzandGitHub 075836f210 chore: upsert traces into both traces tables (#8268) 2025-08-04 09:28:46 +02:00
marliessophieandGitHub 543b6ee0f2 feat(peek-evaluator-config): add edit mode functionality (#8285) 2025-08-04 07:12:59 +00:00
marliessophieandGitHub 8c8c488e4d feat(annotation-queues): add support for batch adding observations to annotation queues (#8261) 2025-08-04 07:10:48 +00:00
Hassieb PakzadandGitHub 6a0e0a4221 feat(llm-connections): remove atla integration (#8059) 2025-08-04 05:53:24 +00:00
Max DeichmannandGitHub 1b01a267df chore: only show 10 traces for evals (#8283) 2025-08-03 20:39:51 +00:00
Thorsten SpiekerandGitHub 69a9146894 Revert "refactor: break up SlackService and move to web/worker packages" (#8276)
Revert "refactor: break up SlackService and move to web/worker packages (#8247)"

This reverts commit e452004a6a.
2025-08-03 18:59:20 +00:00
380403e8ed fix(cloud): add csp headers needed for attachment upload to Plain support chat (#7813)
Update CSP headers to allow additional S3 image and connect sources

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-03 18:52:25 +02:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
296a6c3ee6 chore: limit batch sizes of trace deletions (#8274)
* push

* Update worker/src/env.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-02 07:07:45 +00:00
Steffen SchmitzandGitHub 62904a9563 ci: debug docker build issues (#8269) 2025-08-01 18:07:42 +02:00
Steffen SchmitzandGitHub 90247ab64c chore: migrate additional queries to traces amts (#8249) 2025-08-01 15:06:16 +02:00
marliessophieandGitHub 5cf91c60d9 fix(layout): highlight nav items when viewing child routes (#8211) 2025-08-01 11:50:11 +00:00
Steffen SchmitzandGitHub 559ba6d05d chore: remove unused sessions query (#8264) 2025-08-01 13:17:32 +02:00
Steffen SchmitzandGitHub f071be69b6 build: add always() condition to run notifications on failure (#8265) 2025-08-01 12:54:30 +02:00
Max DeichmannandGitHub 52c261b422 perf: delete traces in batches (#8250) 2025-08-01 11:33:32 +02:00
Steffen SchmitzandGitHub 6f0a43ec65 build: setup slack alerts on failing CI (#8255) 2025-08-01 11:12:02 +02:00
marliessophieandGitHub 301bd6b569 feat(dataset-run-items): reads router (#8207)
* chore(dataset-run-items): GET /dataset-run-items

* fixup(dataset-run-items): GET /dataset-run-items

* chore(dataset-run-items): GET datasetRun/[runName]

* fix(dataset-run-items): return 0 for count when no records found

* fix(dataset-run-items): ensure offset is calculated only when page and limit are present

* feat(dataset-run-items): enhance dataset filtering by adding datasetId support and refactor related functions

* chore: rm comment

* refactor(dataset-run-items): rename dataset run name to ID across API and table definitions

* feat(dataset-run-items): add dataset_run_items to Clickhouse table names and refactor dataset filtering to use camelCase

* fix(dataset-run-items): update dataset filtering to ensure only the latest version of each dataset run item is retrieved

* chore: imports

* chore: fix de-duplication

* chore(dataset-router): mark runitemsByRunIdOrItemId as deprecated

* chore: refactor only, split out routes into byItemId and byRunId

* Revert "chore: refactor only, split out routes into byItemId and byRunId"

This reverts commit 2e1f2e3c14b9ed1173e2fb73b7f721c72d57bfbb.

* Revert "chore(dataset-router): mark runitemsByRunIdOrItemId as deprecated"

This reverts commit 5cb2232198d84baf7f027b00603c6fb3b9489c52.

* fixup: runitemsbyrunidoritemid

* chore: rewrite runitemsByRunIdOrItemId

* chore: rewrite runsByDatasetIdMetrics

* chore: ordering

* chore: order by created at and remove base data from query

* chore: add back name

* chore: push
2025-08-01 08:10:22 +00:00
NimarandGitHub 56fd3df2d2 fix(json-view): truncate cell after 2k chacacters (#8227)
* fix(json-view): truncate cell after 2k chacacters

* real value for truncated chars

* just rely on regex
2025-07-31 20:12:44 +00:00
Max Deichmann db433e2b72 chore: release v3.90.0 2025-07-31 20:52:35 +02:00
Thorsten SpiekerandGitHub e452004a6a refactor: break up SlackService and move to web/worker packages (#8247)
* refactor: break up SlackService and move to web/worker packages

* fix: pnpm lock file

* fix: tests

* fix
2025-07-31 18:33:03 +00:00
NimarandGitHub bce026d8b2 fix(dataset-items): add placeholder text for new items (#8228) 2025-07-31 19:11:48 +02:00
marliessophieandGitHub fbdf12bfa3 chore(dataset-run-items): add background migration for DRI (#8245) 2025-07-31 18:07:43 +02:00
marliessophieandGitHub 78f59bc543 chore: add dataset-run-items tables to clickhouse (#8243) 2025-07-31 16:50:25 +02:00
James IdzikandGitHub ea338ed83d fix: coerce LANGFUSE_INIT_PROJECT_RETENTION and other envs correctly to number (#8103)
fix: coerce init project retention, historic max eval limit envs
2025-07-31 16:01:53 +02:00
Steffen SchmitzandGitHub 8cf67fe329 chore: correctly parse input and output on traces_mt ingestion (#8242) 2025-07-31 15:21:36 +02:00
Steffen SchmitzandGitHub e3b23ea5ec feat: allow PUT operations on SCIM users endpoint (#8236)
* feat: allow PUT operations on SCIM users endpoint

* chore: make tests pass

* cleanup
2025-07-31 12:24:45 +00:00
Steffen SchmitzandGitHub 7be2791105 chore: patch docs for ingestion replay (#8238)
* chore: patch docs for ingestion replay

* chore: increase s3 parallelization
2025-07-31 11:47:10 +00:00
NimarandGitHub 683aae0069 fix(json-view): tighter ChatML validation (#8225) 2025-07-31 10:02:58 +00:00
Steffen SchmitzandGitHub 1c8ffc607d chore: write single rows into AMTs instead of pre-merged record (#8218)
* chore: write single rows into AMTs instead of pre-merged record

* chore: adjust assertion

* chore: drop debug log
2025-07-31 09:27:14 +00:00
Steffen SchmitzandGitHub 9c203e8b8a chore: allow return from aggregatingmergetrees without experiments (#8215) 2025-07-31 09:07:39 +00:00
Steffen SchmitzandGitHub c32cccce52 feat: allow redis username overwrites via env variables (#8234) 2025-07-31 09:52:03 +02:00
NimarandGitHub f1f8da5b74 fix(json-view): store formatted/json preference in local storage (#8222) 2025-07-30 19:47:22 +00:00
Max DeichmannandGitHub c23447a624 chore: improve slack links (#8219) 2025-07-30 18:31:44 +02:00
Thorsten SpiekerandGitHub 5e2225de46 fix: use different env variable in worker to construct links (#8212) 2025-07-30 17:02:11 +02:00
Nimar bb9d118853 chore: release v3.89.0 2025-07-30 16:09:51 +02:00
marliessophieandGitHub df57ff60d6 feat(dataset-run-items): support reads for public APIs (#8179)
* chore(dataset-run-items): GET /dataset-run-items

* fixup(dataset-run-items): GET /dataset-run-items

* chore(dataset-run-items): GET datasetRun/[runName]

* fix(dataset-run-items): return 0 for count when no records found

* fix(dataset-run-items): ensure offset is calculated only when page and limit are present

* feat(dataset-run-items): enhance dataset filtering by adding datasetId support and refactor related functions

* chore: rm comment

* refactor(dataset-run-items): rename dataset run name to ID across API and table definitions

* feat(dataset-run-items): add dataset_run_items to Clickhouse table names and refactor dataset filtering to use camelCase

* fix(dataset-run-items): update dataset filtering to ensure only the latest version of each dataset run item is retrieved

* chore: imports

* chore: fix de-duplication
2025-07-30 13:52:09 +00:00
NimarandGitHub bae2c5a65d fix(json-table): render links clickable (#8209)
* fix(json-table): render links clickable

* export
2025-07-30 15:48:01 +02:00
Steffen SchmitzandGitHub 3efa696513 fix(sessions): allow filter combination of environment and session_id (#8210) 2025-07-30 13:20:04 +00:00
Hassieb PakzadandGitHub 58ebf006b8 fix(llm-conenctions): drop system message from test call (#8189) 2025-07-30 11:49:31 +00:00
Thorsten SpiekerandGitHub 16743363dc fix: slack channel search component (#8208) 2025-07-30 13:56:44 +02:00
NimarandGitHub ebf0e35073 fix(json-view): smart retain collapsed/expanded state of json table view (#8191)
* handle user set expansion state in context provider

* update

* update2

* fix collapse state

* fix state bool

* final cleanup

* fix collapse all

* fix lint

* cleanup

* simplify

* remove log

* fix expand

* fix types

* lint
2025-07-30 11:21:42 +00:00
Thorsten SpiekerandGitHub 2c4799340f fix: set slack redirect uri specifically per environment (#8205) 2025-07-30 12:50:15 +02:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
5612c6a9a5 feat(cloud): add billing alerts (#7903)
* feat(cloud): add billing alerts

* chore: checkpoint state

* chore: cleanup

* chore: remove implementation plan

* chore: update send email behaviour

* chore: add usage alert config options in UI

* chore: implement stripe hook and email

* chore: optimize email

* chore: linting

* chore: use invoice hooks to retrigger usage alerts in each cycle

* chore: add log statement if usage alert got recreated

* chore: update log messages

* chore: replace cloud-billing-alerts with cloud-usage-alerts

* Update web/src/ee/features/billing/components/UsageAlerts.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-30 09:48:59 +00:00
Max DeichmannandGitHub b62962ca08 chore: move slack database migration in right order (#8203)
push
2025-07-30 11:12:28 +02:00
Thorsten SpiekerandGitHub 3a46283bf2 feat: add slack integration (#8073) 2025-07-30 10:52:57 +02:00
Marlies Mayerhofer 07801180f8 chore: release v3.88.1 2025-07-30 10:49:42 +02:00
marliessophieandGitHub 14831902ef fix(annotation-queues): remove unnecessary refetchOnMount option and ensure current item type is defined (#8199) 2025-07-30 08:28:46 +00:00
marliessophieandGitHub d377f02a49 fix(dataset-items): ensure deterministic ordering given pagination for GET dataset items api (#8200)
* fix(dataset-items): ensure deterministic ordering given pagination for GET dataset items api

* chore: push
2025-07-30 08:28:22 +00:00
Nimar e07120b163 chore: release v3.88.0 2025-07-29 20:27:11 +02:00
NimarandGitHub 506482dbe4 fix(playground): enable jumpt to playground for gemini (#8188)
* fix(playground): enable jumpt to playground for gemini

* fix exhaustive check

* correct types
2025-07-29 17:34:02 +00:00
Xichen PanandGitHub ec61f4a420 fix: set explicit ES2021 target in tsconfig to support Unicode regex (#8174)
* fix(worker): set explicit ES2021 target in tsconfig to support Unicode regex

* fix(worker): remove redundant lib config from tsconfig.json
2025-07-29 17:30:58 +00:00
marliessophieandGitHub 4db08a2960 fix(eval): trace filter preview enable dynamic sorting based on control visibility (#8187) 2025-07-29 17:11:37 +00:00
NimarandGitHub 2351d0d370 fix(markdown): don't break after list item number (#8181) 2025-07-29 12:03:49 +00:00
marliessophieandGitHub 99b3401549 feat(dataset-run-items): implement experiment service writes (#8090)
* feat(dataset-run-items): implement experiment service writes

* chore: add

* fix(experiments): update error messages for missing API key using constants

* chore(experimentCreateQueue): increase backoff delay to 10 seconds

* chore(auth): update ingestion types for improved flexibility

* chore: create traces & error-level generations for DRIs

* chore: unify timestamps

* chore: revert test to PG implementation until we add DRI migration across environments

* chore: typo

* chore: implement unified trace ID generation for ClickHouse and PostgreSQL executions to prevent duplicate traces

* chore: refactor trace creation logic for ClickHouse and PostgreSQL executions to use a unified approach

* chore: eslint

* chore: comment

* chore: rebase

* chore: rebase

* chore: rename function for clarity and fix typo in comments

* refactor: remove fetchDatasetRun function and replace with direct Prisma query for dataset run retrieval

* chore: eslint

* chore: fix test

* chore: fix test

* refactor: rename function and update comments for clarity in experimentServiceClickhouse
2025-07-29 11:14:30 +00:00
Nimar 6fb796df40 chore: release v3.87.1 2025-07-29 12:04:14 +02:00
NimarandGitHub 846d6e6c37 fix(markdown-rendering): tighter rendering (#8177)
* fix(trace-view): better markdown spacing for density

* better spacing
2025-07-29 09:55:05 +00:00
marliessophieandGitHub 1fa4fc6329 fix(annotation): mobile responsivness behaviour (#8154)
* fix(annotation): mobile behavior of annotation queue item view

* fix(ItemBadge): enhance label display with truncation and overflow handling

* fix(annotation-processing): improve layout by adjusting panel height properties for better overflow handling
2025-07-29 08:29:31 +00:00
Marlies Mayerhofer a1a48d5e7b chore: release v3.87.0 2025-07-29 00:15:18 +02:00
NimarandGitHub fcb7563763 chore: upgrade turbo to 2.5.5 (#8171) 2025-07-28 21:00:45 +00:00
Thorsten SpiekerandGitHub ae754b146d fix: scroll area for large lists of variables (#8170) 2025-07-28 21:13:56 +02:00
marliessophieandGitHub 49343b9a0c feat(annotation-queues): add support for batch adding sessions to annotation queues (#8169)
* feat(annotation-queues): add support for batch adding sessions to annotation queues

* chore: eslint types
2025-07-28 18:44:06 +00:00
Steffen SchmitzandGitHub 76bf5c0f4e fix: add queue prefix to webhook queue (#8165) 2025-07-28 16:12:48 +00:00
marliessophieandGitHub e91a29be61 fix(members-table): update table column visibility and order logic based on project context (#8164)
refactor(members-table): update column visibility and order logic based on project context
2025-07-28 14:38:14 +00:00
marliessophieandGitHub 81694363a2 fix(metrics): convert latency from milliseconds to seconds for accurate display (#8162) 2025-07-28 12:38:43 +00:00
marliessophieandGitHub 5998266680 feat(dataset-run-items): implement deletion for CH writes (#8074)
* feat(dataset): implement dataset run items deletion queue and processing

* feat(dataset): add dataset run items deletion functionality and integrate with deletion queue

* chore: revert imports

* chore: handle DRI deletion on project deletion conditionally

* fix(dataset-router): await Promise.all for dataset run items deletion

* refactor(dataset-router): streamline dataset deletion process and ensure async handling of dataset run items

* fix(env): add new environment variable for dataset run items deletion concurrency duration

* chore: rm comment

* refactor(dataset-run-items): update delete functions to use object destructuring for parameters

* fix(dataset-run-items): replace hardcoded request timeout with environment variable for deletion timeout

* refactor(dataset): rename and restructure dataset deletion functionality, replacing dataset run items deletion with a unified dataset deletion queue
2025-07-28 11:36:05 +00:00
Nimar 462e8e847d chore: release v3.86.1 2025-07-28 12:53:59 +02:00
NimarandGitHub d7c186858b fix(observations): if value is 0 show it instead of null (#8158) 2025-07-28 10:22:03 +00:00
NimarandGitHub e686aedac9 fix(trace-table): smart expand json table if not many rows (#8142)
* fix(trace-table): smart expand json table if not many rows

* clean up comments

* better performance

* cleanup

* fix lint

* clean
2025-07-28 09:58:02 +00:00
Max DeichmannandGitHub 85f75a5e8e chore: improve webhook retries (#8098)
* chore: improve webhook retries

* chore: improve webhook retries

* chore: improve webhook retries

* chore: improve webhook retries

* fixes

* push

* fixes

* fixes
2025-07-26 13:26:15 +00:00
Max DeichmannandGitHub 1ea643300d chore: remove ingestion via bullmq (#8143) 2025-07-26 00:13:53 +02:00
596486c1ec chore: ingest secondary ingestion queue data (#8137)
* push

* add duration tracking

* push

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2025-07-25 17:13:03 +00:00
Hassieb PakzadandGitHub cb65391c2d fix(model-selector): correct combinations provider model (#8135)
* fix(model-selector): correct combinations provider model

* push

* push

* push

* push

* push
2025-07-25 16:29:06 +00:00
marliessophieandGitHub 419e07260d fix(evaluator): update button label based on form mode (#8134) 2025-07-25 14:06:59 +00:00
marliessophieandGitHub 825f030e81 feat(annotation): support annotation on sessions (#8130)
* feat(session): integrate CreateNewAnnotationQueueItem component into session page

* refactor(annotation-queues): enhance QueueItemRowData structure to support multiple object types

* chore: refactor to extract common functionality

* fixup: refactor to extract common functionality

* chore: handle session annotation type in annotation processing UI

* feat(annotation-queues): add object type column and enhance source display in AnnotationQueueItemsTable

* chore: add session annotation ui

* feat(annotation-queues): add tooltip for detailed view toggle and enhance session item handling

* chore: small improvement

* feat(annotation-queues): implement intersection observer for trace visibility and update pagination logic

* fix(annotation-queues): update current item type retrieval and improve session item handling logic

* chore: eslint

* chore: nits
2025-07-25 13:32:54 +00:00
Max DeichmannandGitHub 34ded9c927 chore: adjust sampling metric (#8132) 2025-07-25 11:53:15 +02:00
Hassieb PakzadandGitHub c935d4ae73 chore: add sampling to ingestion pipeline (#8128) 2025-07-25 11:26:30 +02:00
Hassieb PakzadandGitHub e7283ac06e fix(ingestion): cached model not found (#8122) 2025-07-25 06:51:18 +00:00
Max DeichmannandGitHub 20d0106626 chore: fix date parsing for eval retries (#8119) 2025-07-25 02:00:48 +02:00
Max DeichmannandGitHub 849591fdf2 chore: improve evals logging (#8118) 2025-07-25 01:34:52 +02:00
Max DeichmannandGitHub 0c7b50e564 chore: fix eval event envelope timestmp (#8117) 2025-07-25 01:10:40 +02:00
Max DeichmannandGitHub 37b4f43351 chore: increase observability and delay of eval executions (#8116) 2025-07-25 00:08:54 +02:00
marliessophieandGitHub 7a3b0379e5 fix(experiments): update loading condition in CreateExperimentsForm to check for datasetId (#8115) 2025-07-24 21:11:42 +00:00
marliessophieandGitHub f458d7626c chore: add sessions type for annotation queue (#8107) 2025-07-24 17:11:47 +00:00
Max DeichmannandGitHub 258dde4691 fix: fix automation updates (#8108)
* fix: fix automation updates

* fix: fix automation updates

* fix: fix automation updates
2025-07-24 16:35:33 +00:00
marliessophieandGitHub f15246d9df feat(experiments): support remote experiment trigger (#8094)
* fixup(experiments): support trigger for remote experiment run

* chore: remove instructions

* chore: rename webhook > remote experiment server side

* chore: rename webhook > remote experiment client side

* style: design review

* chore: lint

* fix(experiments): change URL validation to use z.url() in RemoteExperimentUpsertForm

* chore: rename

* feat(datasets): add transformation function for datasets and update API responses
2025-07-24 16:12:59 +00:00
NimarandGitHub edb43a24ab fix(trace): show linebreaks in only text markdown view (#8106) 2025-07-24 16:08:00 +00:00
Steffen SchmitzandGitHub c45ae1b140 chore: adjust getSessionsTable query for AMTs (#8102)
* chore: adjust getSessionsTable query for AMTs

* chore: add additional index on traces_all_amt
2025-07-24 14:56:43 +00:00
Ted KimandGitHub 90eeeeaf9d fix: remove 'ON CLUSTER' instruction from unclustered migration (#7959) 2025-07-24 16:49:00 +02:00
Nimar 66accb563c chore: release v3.86.0 2025-07-24 16:30:43 +02:00
NimarandGitHub 9f2e8906d0 fix(trace-table): expanded state and make it fast! (#8099)
* preserve expanded state

* lazy load table for better performance

* fix broken commit

* pending: verbose state tracking working again

* fix states

* fix lint

* cleanup

* cleaner even

* fix build
2025-07-24 14:24:47 +00:00
c5d61b7ed2 chore: Remove add window button and its imports (#8097)
* Remove Add Window button from playground page

Co-authored-by: max <max@langfuse.com>

* feat(playground): remove Add Window button from header

- Remove Add Window button from playground page header
- Clean up unused imports (Plus icon, MULTI_WINDOW_CONFIG)
- Remove unused isAddWindowDisabled variable
- Preserve addWindow function for copy functionality
- Fix linting warnings for unused imports

Users can still add windows via the copy button on individual windows.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-24 13:05:27 +00:00
Max DeichmannandGitHub f95dd872e6 chore: fix output box for playground (#8096) 2025-07-24 12:38:11 +00:00
marliessophieandGitHub 04339741a9 chore(dataset-run-items): adjust converter for background migration (#8075)
* chore: reorder converter

* chore: treat numbers and booleans as strings

* chore: adjust metadata conversion

* Revert "chore: adjust metadata conversion"

This reverts commit d5ca3a5cb1b6e5972e5a321ab4884c589b938087.
2025-07-24 11:38:48 +00:00
marliessophieandGitHub 9c85e46996 feat(table): persist column sizes in local storage (#7816)
* feat(table): persist column sizing in localStorage

* chore: apply to all tables

* chore: push

* chore: add string
2025-07-24 11:37:16 +00:00
Steffen SchmitzandGitHub 4a7236451f chore: bump clickhouse client to 1.12.0 (#8092) 2025-07-24 11:20:21 +00:00
Steffen SchmitzandGitHub 5cb5204181 chore: migrate remaining trace reads to AMT (#8056)
* chore: migrate remaining trace reads to AMT

* chore: revert box ticks

* chore: revert streamed exports

* chore: confirm trace count by project in creation interval

* chore: confirm trace count since created

* chore: confirm getTracesByIdsForAnyProject

* chore: session table simplification

* chore: refactor user metrics

* chore: update
2025-07-24 10:49:44 +00:00
NimarandGitHub c19066afa0 fix(playground): api key link should show llm connection (#8080) 2025-07-24 09:43:03 +00:00
Max DeichmannandGitHub bbb9dc285d chore: remove delay for otel (#8085) 2025-07-24 11:16:59 +02:00
Steffen SchmitzandGitHub b750acba06 feat(admin-api): add okta role assignment capabilities (#8084)
* feat(admin-api): add okta role assignment capabilities

* chore: lint
2025-07-24 08:21:36 +00:00
Max DeichmannandGitHub bb06e079eb chore: upgrade prism (#8083)
* chore: upgrade prismjs

* fix
2025-07-24 07:49:50 +00:00
Max DeichmannandGitHub 5f711780e0 chore: upgrade prismjs (#8077) 2025-07-23 17:20:53 +00:00
Max DeichmannandGitHub 0f58ea1ebf chore: remove trivy (#8076) 2025-07-23 17:11:59 +00:00
Nimar ec70ca3951 chore: release v3.85.2 2025-07-23 18:20:34 +02:00
NimarandGitHub 142d3a1612 fix(trace-view): render new lines in json table view (#8072) 2025-07-23 16:09:32 +00:00
Max DeichmannandGitHub 1dee7092b3 chore: configure posthog SDK (#8067)
* chore: configure posthog SDK

* chore: add score values to scores

* update posthog integration

* update posthog integration
2025-07-23 15:27:06 +00:00
Steffen SchmitzandGitHub 83b64f5e77 perf: extend model cache duration and invalidate on model changes (#8053) 2025-07-23 16:39:04 +02:00
Steffen SchmitzandGitHub 48b05247c8 fix(admin-api): handle Okta provisioning for POST and PATCH (#8064) 2025-07-23 14:15:10 +00:00
Max DeichmannandGitHub 9732262466 chore: fix node types discrepancies across packages (#8060)
* chore: fix node types discrepancies across packages

* chore: fix node types discrepancies across packages
2025-07-23 12:23:43 +00:00
Max DeichmannandGitHub e51e2df2dc chore: fix snyk again (#8050)
* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2025-07-23 12:04:01 +00:00
Nimar 74d1dfe5ac chore: release v3.85.1 2025-07-23 13:55:34 +02:00
NimarandGitHub 5633a8867e fix(datasets-table): don't table render json here (#8058) 2025-07-23 11:52:50 +00:00
6a8bdae22e feat: multi window playground (#7661)
* docs: add technical specs for multi window/prompt playground

* docs: add requirements and tech summary for sharing

* feat: add multi-window playground architecture and state isolation

* feat: make playground UI multi-window capable

* fix: multi window playground caching

* fix: jump to playground button adds windows to playground using stable id

* fix: avoid race condition in writing to cache before navigating to playground

* feat: add collapsible, more compact sections to non-model config

* feat: add compact version of model picker/params component

* feat: add ability to copy window states to multi playground

* fix: solve header crowding for thin windows in playground

* feat: make entire playground config section collapsible

* feat: change SaveToPrompt button to align with other window buttons

* feat: switch buttons in page header to icon-only when too small

* fix: adjust button labels to fit button size

* fix: update the execution window status correctly

* docs: complete remaining task items

* refactor: simplify playground state management hooks

* fix: make the model configuration section prettier

* docs: remove agent work files

* refactor: call register functions on the registry directly

* chore: refactored some components and compacted the design even more

* chore: move add message buttons

* chore: start align model param settings popover when compact

* chore: small UI spacing improvements

* fix: scrolling bug

* fix: handle config headers when window gets small

* chore: rename button hover

* feat: move CMD+Enter to execute all button

* feat: give user the choice to jump to fresh/existing playground

* push

* chore: fix

* chore: fix

* push

* push

* fixes

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-07-23 10:01:43 +00:00
Steffen SchmitzandGitHub dbf994f9bb perf: replace prisma upsert with raw query (#8036)
* perf: replace prisma upsert with raw query

* chore: drop prsima import

* chore: lint
2025-07-23 10:00:06 +00:00
Steffen SchmitzandGitHub 4d5c288d1e perf: initialize clickhouse skip read map on startup (#8039)
* perf: initialize clickhouse skip read map on startup

* chore: linting

* chore: patch skip cache init
2025-07-23 09:29:44 +00:00
Hassieb PakzadandGitHub cf119fb9cc fix(otel): parse trace name if as_root is true (#8008)
* fix(otel): parse trace name if as_root is true

* push

* push

* push
2025-07-23 09:13:54 +00:00
Hassieb PakzadandGitHub 5af8c4e400 fix(otel): parse public attribute for traces (#8049) 2025-07-23 10:18:31 +02:00
Max DeichmannandGitHub 835a19bbd2 fix: fix snyk uploads (#8044) 2025-07-22 20:48:38 +00:00
Max DeichmannandGitHub ae91a98ad9 fix: use drop down for prompt full text search (#8043)
* fix: use drop down for prompt full text search

* fix: use drop down for prompt full text search

* fix: use drop down for prompt full text search

* fix: use drop down for prompt full text search

* fix: use drop down for prompt full text search
2025-07-22 20:33:25 +00:00
NimarandGitHub e38cc24205 chore: add default claude settings (#8042) 2025-07-22 19:12:19 +00:00
marliessophieandGitHub 7ab45ec685 feat(dataset-run-items): migrate from postgres to clickhouse; write to CH API-only (#7912)
* feat(migrations): add dataset_run_items table in clickhouse

* feat(dataset): add environment variables and utility for pg/ch execution

* chore: re-write `runitemsByRunIdOrItemId` to support both clickhouse and postgres execution

* chore: use `runItemsByRunId` or `runItemsByItemId` rather than  `runitemsByRunIdOrItemId`

* chore: rename env variables

* chore: reflect dri as ch data is seeder

* fixup: dataset run item types and converters

* chore: background migration

* chore: rewrite dataset-router routes

* chore(dataset): dual-write strategy for dataset run items to support ClickHouse and PostgreSQL

* chore: add utilities and converters

* chore: rewrite GET api/public/dataset-run-items

* feat(dataset): implement dataset run items deletion queue and processing

* chore: rewrite DELETE api/public/datasets/{datasetName}/runs/{runName}

* chore: rewrite GET api/public/datasets/{datasetName}/runs/{runName}

* fixup: rewrite GET api/public/datasets/{datasetName}/runs/{runName}

* chore(schema): add error to DRI schema

* fixup(ingestion): support dataset run items

* fixup: rewrite POST /api/public/dataset-run-items​

* fixup: rewrite POST /api/public/dataset-run-items​ --amend

* fixup(ingestion): support dataset run items --amend

* fixup(experiments): preliminary implementation, for illustrative purposes only

* fixup(schema): indexes

* fixup(schema): remove indexes, add `output`

* chore: support DRI ingestion

* chore: eslint, writes related

* chore: rewrite experiment service

* chore: rm file

* chore: cascade delete dataset run items

* chore: add clustered migrations

* chore: update background migration

* chore: handle experiment job errors

* chore: remove CH read implementation

* chore: remove md file

* chore: remove CH read implementation II

* chore: fix typing

* chore: fix migration types

* fix: lint/imports

* fix: import errors

* chore: imports and comments

* chore: remove only from test

* fix: validateDatasetRunAndFetch

* fix: validateDatasetRunAndFetch id

* fixup: attempt fix of test

* fix: test

* chore: test validate logging

* chore: adjust logs

* chore: revert cascade delete

* Revert "feat(dataset): implement dataset run items deletion queue and processing"

This reverts commit 9f725f053386ded41e3617efaacee128d5ac023f.

* chore: delete imports

* revert: experiment service changes

* chore: adjust comment

* rename: env variables to incl experiment

* chore: remove unused variable eslint disables from datasetExecution and types files

* refactor: remove enrichedDatasetRunItem function and integrate its logic directly into IngestionService

* chore: remove TODO comment regarding dataset run item authorization

* refactor: replace Date constructor with parseClickhouseUTCDateTimeFormat for date parsing in dataset run item conversion

* chore: remove deprecated flag for runitemsByRunIdOrItemId method

* refactor: remove getDatasetRunItemsByRunId function to streamline dataset run item retrieval

* refactor: add back run properties in dataset runs mapping

* chore: revert changes to delete-dataset-run API

* refactor: remove validateDatasetItemAndFetch function and replace its usage with direct Prisma query in IngestionService

* refactor: delete validateCreateDatasetRunItemBodyAndFetch function and replace its logic with direct Prisma queries in dataset-run-items API

* refactor: move createOrFetchDatasetRun function to a new dataset-runs API file and maintain unique constraint handling

* refactor: remove validateDatasetRunAndFetch function and replace its usage with direct Prisma queries in dataset-run-items and IngestionService

* chore: nits

* refactor: streamline dataset run item creation by consolidating validation and execution logic

* chore: fix imports

* fix: types

* fix: simplify ingestion schema

* refactor: enhance ingestion schema creation to support public and internal environments

* chore: extend immutable keys list DRI

* chore: use same id for clickhouse and postgres

* chore: remove background migration trigger

* chore: move dataset run items migration into readme

* chore: remove creation of DRI in seeder

* chore: lint
2025-07-22 16:20:47 +00:00
Nimar b101bf48d3 chore: release v3.85.0 2025-07-22 16:47:56 +02:00
NimarandGitHub 4c4f0ff3c0 feat(tracing): pretty display json as collapsible table (#7938)
* feat(tracing): pretty display json as collapsible table

* show empty list and unwrap single item containing objects

* enable toggle for code or pretty view

* clean up

* fix type casting

* some more fixes

* expand row with click anywhere

* fix alignment

* remove dead code

* cleanup

* make it more clean

* don't unwrap initial items

* collapse expand all

* fixing collapse button

* don't render for chatml

* better markdown check

* reduce col width

* make table borderless

* display null / empty string for empty value

* smaller text

* make table more compact

* make preview items grey & italic

* linebreak path column

* table headers inherit color

* cleanup

* external collapsed state; also move button into code view

* remove code view switch

* show empty dicts also as empty

* always show objects as tables

* extract markdown into helper function

* review fixes

* update
2025-07-22 14:41:11 +00:00
Steffen SchmitzandGitHub a42ca96225 chore: expose email_from_address and smtp_connection_url in docker-compose (#8033)
chore: expose email_from_address and smtp_connection_url in docker-compose.yml
2025-07-22 13:26:03 +00:00
marliessophieandGitHub f63420827b feat(prompts/playground): allow arbitrary message sorting in messages (#8031) 2025-07-22 12:41:39 +00:00
Steffen SchmitzandGitHub e6fd056ae5 chore: migrate analytic routes to new AMTs (#8029)
* chore: migrate analytic routes to new AMTs

* chore: reduce new shared code

* chore: cleanup

* chore: cleanup
2025-07-22 12:39:48 +00:00
Steffen SchmitzandGitHub 4708fe4f47 chore: migrate additional session routes to new AMTs (#8027)
* chore: migrate additional session routes to new AMTs

* chore: remove unused function

* chore: remove unused code

* chore: remove outdated row in readme

* chore: update assertions
2025-07-22 12:24:57 +00:00
marliessophieandGitHub 362246b616 chore(datasets): update button label from "Rename" to "Edit" in DatasetActionButton component (#8030) 2025-07-22 12:21:45 +00:00
Steffen SchmitzandGitHub b3de5fd71a chore: migrate traces table to AMTs (#7853)
* chore: migrate traces table to AMTs

* chore: fix linting errors

* chore: ensure inplace update

* chore: add updated query

* chore: use final on query
2025-07-21 14:12:25 +00:00
Hassieb PakzadandGitHub ad236ecc99 fix(model-params): increase max tokens limit (#8005) 2025-07-21 15:12:37 +02:00
NimarandGitHub beb2063dcb fix: add select all to multi select dropdown (#8004) 2025-07-21 12:53:09 +00:00
Hassieb PakzadandGitHub 4de10298dc fix(otel): parse only trace io on trace update (#8003) 2025-07-21 12:11:04 +00:00
Danushka FernandoandGitHub f9924975fa feat: expose a key prefix option for Redis (#7958) 2025-07-21 14:01:14 +02:00
Steffen SchmitzandGitHub 98774cbd61 fix(sessions): infer session env from trace (#7996) 2025-07-21 10:44:17 +00:00
Steffen SchmitzandGitHub 80361ef53f fix(sessions): align duration unit with UI (#7998)
* fix(sessions): align duration unit with UI

* chore: update tests
2025-07-21 09:40:17 +00:00
Steffen SchmitzandGitHub 8350507434 chore: add missing blob-storage export entitlement (#7990) 2025-07-21 08:09:22 +00:00
marliessophieandGitHub 0a976edb70 fix(prompts): commitMessage error handling in NewPromptForm (#7984) 2025-07-20 16:13:39 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Max Deichmann
6a67c93010 chore(deps): bump exponential-backoff from 3.1.1 to 3.1.2 (#7412)
Bumps [exponential-backoff](https://github.com/coveooss/exponential-backoff) from 3.1.1 to 3.1.2.
- [Commits](https://github.com/coveooss/exponential-backoff/compare/v3.1.1...v3.1.2)

---
updated-dependencies:
- dependency-name: exponential-backoff
  dependency-version: 3.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-07-20 09:33:39 +00:00
Max DeichmannandGitHub e51ee05d9c chore: upgrade express 5 (#7978)
* chore: upgrade express 5

* chore: upgrade express 5
2025-07-20 09:13:22 +00:00
Max Deichmann 565c561715 chore: release v3.84.0 2025-07-20 09:45:00 +02:00
Max DeichmannandGitHub d1338ad5e1 chore: move dev dependencies to dev dependencies web container (#7977)
* fix

* fix

* fix

* fix

* chore: move dev dependencies to dev dependencies web container
2025-07-19 19:15:48 +00:00
Max DeichmannandGitHub 52d4fe37de chore: remove unused tsx (#7976)
* fix

* fix

* fix

* fix
2025-07-19 19:15:40 +00:00
Max DeichmannandGitHub 67ef7abb63 chore: full text search for prompts (#7973)
* fix

* fix

* fix
2025-07-19 18:55:58 +00:00
Max DeichmannandGitHub 727ae71a93 fix: fix codemirror upgrade (#7975)
fix
2025-07-19 17:57:49 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a8eabbc96d chore(deps): bump the radix-ui group with 23 updates (#5562)
Bumps the radix-ui group with 23 updates:

| Package | From | To |
| --- | --- | --- |
| [@radix-ui/react-accordion](https://github.com/radix-ui/primitives) | `1.2.1` | `1.2.3` |
| [@radix-ui/react-alert-dialog](https://github.com/radix-ui/primitives) | `1.1.2` | `1.1.6` |
| [@radix-ui/react-avatar](https://github.com/radix-ui/primitives) | `1.1.1` | `1.1.3` |
| [@radix-ui/react-checkbox](https://github.com/radix-ui/primitives) | `1.1.2` | `1.1.4` |
| [@radix-ui/react-collapsible](https://github.com/radix-ui/primitives) | `1.1.1` | `1.1.3` |
| [@radix-ui/react-dialog](https://github.com/radix-ui/primitives) | `1.1.2` | `1.1.6` |
| [@radix-ui/react-dropdown-menu](https://github.com/radix-ui/primitives) | `2.1.2` | `2.1.6` |
| [@radix-ui/react-hover-card](https://github.com/radix-ui/primitives) | `1.1.2` | `1.1.6` |
| @radix-ui/react-icons | `1.3.0` | `1.3.2` |
| [@radix-ui/react-label](https://github.com/radix-ui/primitives) | `2.1.0` | `2.1.2` |
| [@radix-ui/react-popover](https://github.com/radix-ui/primitives) | `1.1.2` | `1.1.6` |
| [@radix-ui/react-progress](https://github.com/radix-ui/primitives) | `1.1.1` | `1.1.2` |
| [@radix-ui/react-radio-group](https://github.com/radix-ui/primitives) | `1.2.1` | `1.2.3` |
| [@radix-ui/react-scroll-area](https://github.com/radix-ui/primitives) | `1.2.0` | `1.2.3` |
| [@radix-ui/react-select](https://github.com/radix-ui/primitives) | `2.1.2` | `2.1.6` |
| [@radix-ui/react-separator](https://github.com/radix-ui/primitives) | `1.1.0` | `1.1.2` |
| [@radix-ui/react-slider](https://github.com/radix-ui/primitives) | `1.2.1` | `1.2.3` |
| [@radix-ui/react-slot](https://github.com/radix-ui/primitives) | `1.1.0` | `1.1.2` |
| [@radix-ui/react-switch](https://github.com/radix-ui/primitives) | `1.1.1` | `1.1.3` |
| [@radix-ui/react-tabs](https://github.com/radix-ui/primitives) | `1.1.1` | `1.1.3` |
| [@radix-ui/react-toggle](https://github.com/radix-ui/primitives) | `1.1.0` | `1.1.2` |
| [@radix-ui/react-toggle-group](https://github.com/radix-ui/primitives) | `1.1.0` | `1.1.2` |
| [@radix-ui/react-tooltip](https://github.com/radix-ui/primitives) | `1.1.3` | `1.1.8` |


Updates `@radix-ui/react-accordion` from 1.2.1 to 1.2.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-alert-dialog` from 1.1.2 to 1.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-avatar` from 1.1.1 to 1.1.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-checkbox` from 1.1.2 to 1.1.4
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-collapsible` from 1.1.1 to 1.1.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-dialog` from 1.1.2 to 1.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-dropdown-menu` from 2.1.2 to 2.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-hover-card` from 1.1.2 to 1.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-icons` from 1.3.0 to 1.3.2

Updates `@radix-ui/react-label` from 2.1.0 to 2.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-popover` from 1.1.2 to 1.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-progress` from 1.1.1 to 1.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-radio-group` from 1.2.1 to 1.2.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-scroll-area` from 1.2.0 to 1.2.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-select` from 2.1.2 to 2.1.6
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-separator` from 1.1.0 to 1.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-slider` from 1.2.1 to 1.2.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-slot` from 1.1.0 to 1.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-switch` from 1.1.1 to 1.1.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-tabs` from 1.1.1 to 1.1.3
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-toggle` from 1.1.0 to 1.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-toggle-group` from 1.1.0 to 1.1.2
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

Updates `@radix-ui/react-tooltip` from 1.1.3 to 1.1.8
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

---
updated-dependencies:
- dependency-name: "@radix-ui/react-accordion"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-alert-dialog"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-avatar"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-checkbox"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-collapsible"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-dialog"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-dropdown-menu"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-hover-card"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-icons"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-label"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-popover"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-progress"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-radio-group"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-scroll-area"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-select"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-separator"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-slider"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-slot"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-switch"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-tabs"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-toggle"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-toggle-group"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
- dependency-name: "@radix-ui/react-tooltip"
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: radix-ui
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-19 17:21:00 +00:00
Max DeichmannandGitHub 51befcad0f chore: upgrade codemirror (#7972)
fix
2025-07-19 17:13:38 +00:00
Max DeichmannandGitHub ee8b5e4998 chore: package upgrade vitest (#7971)
fix
2025-07-19 17:13:14 +00:00
Max DeichmannandGitHub 9bb9f7fa39 chore: remove ineffective pnpm configs (#7970) 2025-07-19 17:04:43 +00:00
Thorsten SpiekerandGitHub 4cc10840ab feat: automation secrets (#7807) 2025-07-18 17:20:48 +02:00
Hassieb Pakzad 311cd900fb chore: release v3.83.0 2025-07-18 14:23:53 +02:00
Hassieb PakzadandGitHub 92f14fd297 feat(llm-connections): default credential provider chain for Bedrock when self-hosted (#7947)
* feat(llm-connections): default credential provider chain for Bedrock when self-hosted

* push

* push
2025-07-18 12:14:59 +00:00
Max DeichmannandGitHub 42306e5247 chore: fix snyk configuration (#7944) 2025-07-18 11:09:07 +02:00
5bb1b6de54 fix(playground): scroll variables if many are present (#7946)
* Refactor Variables rendering with improved sorting and key management

Co-authored-by: nimar <nimar@langfuse.com>

* fix(playground): make placeholders and variables scroll individually

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-17 15:06:38 +00:00
Marc KlingenandGitHub dae0ee6165 fix(ui): main menu icons clickable when collapsed (#7942) 2025-07-17 12:10:02 +00:00
Nimar 31fb304a5e chore: release v3.82.0 2025-07-17 10:03:10 +02:00
NimarandGitHub d6dac50734 fix(playground): use playground with tool calls form langgraph traces (#7923)
* fix(playground): test with toolcalls from langgraph traces

* delete comment

* fix ts error

* remove superfluous comment
2025-07-16 18:36:03 +00:00
Max DeichmannandGitHub 0cda8e2e21 fix: add timeouts to webhooks (#7924)
* fix

* fix

* fix
2025-07-16 20:01:07 +02:00
Hassieb PakzadandGitHub 3f4fd2ff5e feat(model-prices): match cross-region inference profile for claude via bedrock (#7918) 2025-07-16 16:19:32 +00:00
Steffen SchmitzandGitHub 17f0a10b23 feat: add level filter param to observations api (#7905)
* feat: add level filter param to observations api

* chore: update fern reference

* chore: generate api spec

* chore: patch

* chore: extend timeout
2025-07-16 12:24:40 +00:00
Max DeichmannandGitHub 6c80b5b4a0 chore: add linter rule for react icons (#7910)
* push

* perf: import react-icons from one sub path
2025-07-16 13:51:48 +02:00
Max DeichmannandGitHub 91b56518c6 perf: import react-icons from one sub path (#7907) 2025-07-16 10:00:00 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
257c4136fe chore: expand logging for large entity truncation (#7900)
* chore: expand logging for large entity truncation

* Update worker/src/services/ClickhouseWriter/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: fix tests

* chore: patch table name typing

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-16 07:50:54 +00:00
94e5f4b2e7 perf: optimize shared package rebuilds for dev speed b069 (#7895)
* Update turbo.json to use @langfuse/shared#dev instead of #build

Co-authored-by: max <max@langfuse.com>

* Add incremental build script and update turbo.json dev dependencies

Co-authored-by: max <max@langfuse.com>

* Remove build:dev script and update turbo.json dependencies

Co-authored-by: max <max@langfuse.com>

* push

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-15 20:43:13 +00:00
Steffen SchmitzandGitHub f09c363aaf perf: truncate very large records on insertion (#7894)
* perf: truncate very large records on insertion

* chore: skip stringify
2025-07-15 19:09:31 +00:00
d0c317b423 chore(ingestions): add s3 download size hist and span attribute (#7887)
* chore(ingestions): add s3 download size hist and span attribute

* push

* Apply suggestion from @maxdeichmann

Co-authored-by: Max Deichmann <m.deichmann@tum.de>

* Apply suggestion from @maxdeichmann

Co-authored-by: Max Deichmann <m.deichmann@tum.de>

* Apply suggestion from @hassiebp

* push

* push

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-07-15 18:18:13 +00:00
Max DeichmannandGitHub 5bcb6b5b38 chore: remove markdown react-icon (#7893)
push
2025-07-15 17:59:39 +00:00
Max DeichmannandGitHub 3806cb72a1 chore: remove react icons as much as possible (#7889)
* fixes

* fixes

* fixes

* fixes

* push

* fixes

* fixes
2025-07-15 16:02:20 +00:00
Max DeichmannandGitHub cbfc78d0e3 chore: fix automations scrolling (#7884)
fixes
2025-07-15 13:26:26 +00:00
marliessophieandGitHub 5d87047577 chore(layout): restructure main navigation (#7875)
* chore(layout): restructure main navigation

* chore: refactor page-header tabs component

* chore: push

* chore: push

* chore: push

* chore: push

* chore: move sessions to main nav

* chore: update test

* chore: test
2025-07-15 11:52:57 +00:00
Steffen SchmitzandGitHub 7a3e5ba0c8 fix: fallback to auto for falsy region value in blob storage integration (#7879) 2025-07-15 09:33:14 +00:00
Steffen SchmitzandGitHub df1ec4d81e fix(otel): handle missing scope name gracefully (#7872) 2025-07-15 07:21:57 +00:00
steffen911 68a884c5ed chore: release v3.81.1 2025-07-15 08:41:30 +02:00
Steffen SchmitzandGitHub 7bcfb9a57e fix: adjust trace session indexes for faster API accesses (#7864) 2025-07-15 06:30:40 +00:00
Hassieb PakzadandGitHub 0e360759ec fix(playground): overflow on large description (#7863) 2025-07-14 17:43:51 +02:00
steffen911 0510a352a3 chore: release v3.81.0 2025-07-14 15:22:22 +02:00
Steffen SchmitzandGitHub 7854a7e41f perf: enable API Key and Prompt caching in redis by default (#7854)
* perf: enable API Key and Prompt caching in redis by default

* chore: update sync prompt test

* chore: disable prompt caching for sync web tests

* chore: enable prompt caching in sync tests

* chore: disable cachking for v1 pipeline

* chore Use cacheEnabled as pure test overwrite

* chore; undefined check

* chore: log body

* chore: patch promptCache test

* chore: fmt

* remove console log
2025-07-14 13:20:37 +00:00
Hassieb PakzadandGitHub f0fee7b0ce chore: remove turbo flag from dev command (#7859) 2025-07-14 13:49:44 +02:00
marliessophieandGitHub 52fd2b9ed8 chore(nav): remove all close/open transition delays for improved user experience (#7856) 2025-07-14 09:57:22 +00:00
Hassieb PakzadandGitHub 3ba75e57a0 fix(otel): keep hex ids in place in id parsing (#7827) 2025-07-14 09:54:43 +00:00
marliessophieandGitHub a42cf72d2a chore(experiments): enhance error handling in LLM call and implement retry logic for experiment creation jobs (#7632)
* chore(experiments): enhance error handling in LLM call and implement retry logic for experiment creation jobs

- Added try-catch block in callLLM function to handle specific API errors and return appropriate ApiError responses.
- Implemented retry logic for experiment creation jobs in experimentQueue, allowing retries for rate-limited and server errors, with a delay mechanism based on job age.

* chore: push

* refactor: introduce delay utility for job processing

* chore: abstract standardized error handling for LLM operations

* refactor: reorganize utilities and enhance error handling for job processing

* chore: push

* chore: typing

* chore: unify error message

* fix(retry-handler): change executeTakeFirst to executeTakeFirstOrThrow for better error handling
2025-07-14 09:35:59 +00:00
Steffen SchmitzandGitHub 35171cc77d chore: add projectId to prices table (#7817) 2025-07-14 09:34:11 +02:00
Steffen SchmitzandGitHub 6a5f221567 fix(auditlog): patch llmTool and llmSchema records in auditLog (#7826)
* fix(auditlog): patch llmTool and llmSchema records in auditLog

* chore: linting

* chore: remove redundant L
2025-07-14 06:55:29 +00:00
c9c61c4b8e ci: all-ci-passed also checks for cancelled status (#7845)
Update pipeline workflow to handle cancelled jobs in deployment status

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-13 15:25:23 +00:00
Max DeichmannandGitHub 8b6186f191 chore: use --turbo always in web dev (#7842) 2025-07-13 11:07:56 +00:00
Steffen SchmitzandGitHub 73fc0e71a3 chore: remove local claude settings file (#7828)
chore: remove local cloud settings file
2025-07-11 15:12:34 +00:00
Steffen SchmitzandGitHub 40152d1f3c chore: forbid html elements on names and descriptions via trpc routes (#7815)
* chore: forbid html elements on names and descriptions via trpc routes

* chore: patch test cases

* chore: patches
2025-07-11 14:39:59 +00:00
Max Deichmann 6cfb78e06e chore: release v3.80.1 2025-07-11 14:26:20 +02:00
Max DeichmannandGitHub 4f01e773b3 fix: navigation history automations (#7822)
* fix: navigation history automations

* fix: navigation history automations
2025-07-11 12:21:43 +00:00
Max DeichmannandGitHub 2a83fdbc9a fix: remove automations cud for members (#7820)
* fix: hipaa scaling

* fix: move webhook cud operations to admin
2025-07-11 11:40:14 +00:00
Max DeichmannandGitHub 84dd3fa63b chore: set default webhook selector without setting the browser history (#7819)
fix: hipaa scaling
2025-07-11 11:24:39 +00:00
Max DeichmannandGitHub fdf70cdde9 fix: remove S3 delete marker in S3 backfill (#7812)
push
2025-07-10 20:44:22 +00:00
Marc KlingenandGitHub 07f0125e7c fix: stripe webhook should also match subscriptions which were not created via checkout session (#7810)
* wip

* attempted fix

* typo
2025-07-10 19:29:17 +00:00
NimarandGitHub bacd44abb1 fix(prompts): prompt folder pagination fix (#7776)
* remove folder detection from frontend

* filter in be

* update

* update

* caps

* cleanup

* cleanup

* clarify

* fix

* robust

* dedup

* smiplif

* simplift

* update
2025-07-10 17:16:33 +00:00
Marc KlingenandGitHub cb032047d3 fix: when projects are queued for deletion, hide them and block access from UI (#7809)
fix: when projects are queued for deletion, hide them and block access
2025-07-10 15:35:50 +00:00
Max DeichmannandGitHub 3ecbcc1670 fix: webhook UI (#7808) 2025-07-10 14:48:08 +00:00
Steffen SchmitzandGitHub 85774535ae chore: create script to refill arbitrary queue events (#7806) 2025-07-10 13:29:55 +00:00
Steffen SchmitzandGitHub db15742d29 chore: add migration checks to confirm that required env variables are present (#7802) 2025-07-10 12:42:19 +00:00
cc1c847728 README: update language-specific README files (#7798)
* Update README.cn.md with Terraform templates and minor text changes

Co-authored-by: jannik <jannik@langfuse.com>

* Update cloud deployment and integration sections in README files

Co-authored-by: jannik <jannik@langfuse.com>

* Changes from background composer bc-f4eefbe6-9e55-4dc0-be21-0fc7d81c9079

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-10 09:37:24 +00:00
Max DeichmannandGitHub a6bf28495c security: upgrade alpine image (#7796)
push
2025-07-10 09:13:58 +00:00
marliessophieandGitHub 0391b9a263 chore(prompt-experiments): set langfuse-native environment (#7573)
* chore(prompt-experiments): set langfuse-native environment

* refactor: no longer export ingestionEvent schema directly

* chore: rm line number

* chore(ingestion): refactor processEventBatch to accept options object

* chore(ingestion): refactor ingestion schemas for public and internal environments

- Introduced separate schemas for public and internal environments.
- Updated environment name validation logic and error messages.
- Refactored event schemas to utilize the new schema structure.
- Deprecated direct export of `ingestionEvent` schema in favor of factory method for better environment handling.

* chore: push

* chore: type environment schema as string

* chore: push

* chore: fix import statements

* chore: update langfuse-langchain to version 3.38.3

* chore: prettier

* chore: revert prettier
2025-07-10 08:23:28 +00:00
Max DeichmannandGitHub 4fc2e33058 fix: gracefully handle missing config for webhooks (#7787)
* fix: gracefully handle missing config for webhooks

* fix: gracefully handle missing config for webhooks
2025-07-09 18:34:08 +00:00
Max DeichmannandGitHub 9a6fa8747b fix: remove swap settings from release gha (#7786)
fix: remove swap settings from releasegha
2025-07-09 18:18:52 +00:00
Max Deichmann bd0b204d18 chore: release v3.80.0 2025-07-09 19:30:09 +02:00
Max DeichmannandGitHub 94586e3f62 fix: fix webhook storage (#7783) 2025-07-09 17:20:49 +00:00
steffen911 63349976fa chore: rename migration folder 2025-07-09 18:02:21 +02:00
Max DeichmannandGitHub 8d092d0e18 chore: remove prompt logs (#7779)
* security: upgrade golang-migrate

* push

* push

* push
2025-07-09 15:45:43 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
c8c031e0bd feat(blob-integration): allow custom start date for blob storage exports (#7745)
* chore: formatting

* chore: add migration file and patches

* format

* chore: add test case

* chore: patch test case

* chore; frontend cleanup

* Apply suggestion from @ellipsis-dev[bot]

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update .devcontainer.json

* chore: test

* patch azure

* chore: revert

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-09 15:44:22 +00:00
Steffen SchmitzandGitHub b616b0d260 fix(dashboard): allow score numeric value filter and metric (#7771)
* fix(dashboard): allow score numeric value filter and metric

* revert

* chore: drop debug log
2025-07-09 13:33:55 +00:00
Hassieb PakzadandGitHub 91d929646e chore(prompt-experiments): upgrade langfuse-langchain (#7770) 2025-07-09 13:18:14 +00:00
NimarandGitHub bfdbd93997 feat(playground): allow arbitrary values for placeholders in playground (#7768)
* allow arbitrary values for placeholders in playground

* update playground

* fix lint

* re-add removed functions

* stringify if arbitrary

* safety first!
2025-07-09 12:14:57 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2916b4228d chore: improve retry script for ingestion events (#7769)
* chore: improve retry script for ingestion events

* chore: move file

* Update worker/src/scripts/replayIngestionEvents/README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-09 11:30:31 +00:00
e5a04751e1 fix: Ensure json output for single value csv import (#7763)
* Ensure CSV parsing always returns JSON object for single columns

Co-authored-by: marc <marc@langfuse.com>

* Update web/src/features/datasets/lib/csvHelpers.ts

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-09 10:57:11 +00:00
bb554b38af chore: remove webhook button from single prompt page (#7762)
* Changes from background composer bc-61bc6129-a444-40fd-9e03-d397fc1c9ee1

* fix lint

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-09 10:57:05 +00:00
marliessophieandGitHub cfc8161473 chore(experimentService): extend logging (#7759)
* chore(experimentService): extend logging

* chore: eslint

* chore: revert
2025-07-09 08:49:41 +00:00
Steffen SchmitzandGitHub aecd8eff2d chore: add backoff retries to clickhouse writer (#7761) 2025-07-09 08:39:54 +00:00
Nimar 74c59f58c2 chore: release v3.79.1 2025-07-09 09:55:32 +02:00
NimarandGitHub e073ec9a21 fix(traces): jumptoplayground possible for langgraph traces with tools (#7744)
* fix jumptoplayground for langgraph

* update type
2025-07-09 07:38:31 +00:00
NimarandGitHub 5920f3a6c7 chore(dx): add prettier to CI (#7735)
* add prettier to CI

* escape

* use an array

* make CI simple again

* debug

* debug

* debug2

* debug again
2025-07-08 08:54:55 +00:00
c2a0b381d4 feat: Add gen_ai.conversation.id support (#7738)
Changes from background composer bc-11040caa-e818-42ef-8bb2-4b004ae862a7

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-08 06:45:45 +00:00
7598c95fc3 fix(prompts): prompts with many labels look good (#7731)
* Add TruncatedLabels component for flexible label display with hover

Co-authored-by: nimar <nimar@langfuse.com>

* Refactor: Improve code formatting and consistency across components

Co-authored-by: nimar <nimar@langfuse.com>

* add many labelled prompt

* add many labelled prompt

* remove superfluous code

* do 8 labels here

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-07 15:16:08 +00:00
Steffen SchmitzandGitHub 35f6f5f0fc feat(dashboard): allow breakdown by month (#7725)
* feat(dashboard): allow breakdown by month

* chore: add aliases to the queries

* chore: add test case and formatting

* chore: adjust test behaviour
2025-07-07 15:07:39 +00:00
da23a5ad0b chore(ui-markdown-viewer): stop interpreting $currency symbol as latex math block marker (#7723)
Remove remark-math plugin from MarkdownViewer

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-07 14:38:08 +00:00
Max DeichmannandGitHub 5adac14049 chore: print response body webhook errors (#7728)
* security: upgrade golang-migrate

* push

* push
2025-07-07 13:36:45 +00:00
Nimar 234f131493 chore: release v3.79.0 2025-07-07 15:24:20 +02:00
Nimar c87e3f88fc chore: release v3.78.3 2025-07-07 15:17:43 +02:00
b0aeb8c454 fix(prompts): handle placeholder messages in diff view (#7726)
* Improve prompt diff display with formatted content for placeholders

Co-authored-by: nimar <nimar@langfuse.com>

* tmp

* Update PromptVersionDiffDialog.tsx

* fix diff

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-07 13:15:14 +00:00
NimarandGitHub a71ac105b9 chore(dx): fix prettier version in ee (#7722)
* chore(dx): fix prettier in ee

* fix tailwind

* update formatting

* cache formattgin
2025-07-07 12:09:29 +00:00
Steffen SchmitzandGitHub 225d64d0f8 build: add registry prefix to docker-compose files (#7719) 2025-07-07 11:48:33 +00:00
Steffen SchmitzandGitHub 327018eb09 build: remove swap settings from pipeline config (#7717) 2025-07-07 11:29:05 +00:00
fe8ba75ff3 chore(dx): prettier for all (#7692)
* add prettier command

* add pre-commit hook

* sync prettier config with .vscode config

* add line

* fix husky

* update for less changes

* upgrade to prettier 3.4 for bug fixes

* ignore no-undef in TS (also worker!)

* changes to prettier

* adhere to style

* make prettier 3.4 default for bug fix

* add prettier ignore

* reset to standard

* update to 3.6.2

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-07-07 11:00:45 +00:00
Jannik MaierhöferandGitHub 74e8c0fe58 chore(ui): change link on traces and observations table to data model docs page (#7713) 2025-07-07 09:15:09 +00:00
marliessophieandGitHub 4f57af8ea1 refactor(datasets-compare): implement context for managing selected resource metrics (#7660)
* refactor(datasets-compare): implement context for managing selected resource metrics

* chore: push

* chore: add dialog header wrapper
2025-07-07 09:12:17 +00:00
Max DeichmannandGitHub 4132335c96 security: upgrade golang-migrate (#7712)
* security: upgrade golang-migrate

* push
2025-07-07 08:52:53 +00:00
Steffen SchmitzandGitHub 67dae75e3a chore: add timestamp column to AMTs for backwards compat (#7710)
* chore: add timestamp column to AMTs for backwards compat

* chore; expand allowed claude functions
2025-07-07 08:12:30 +00:00
ed0a07a958 fix(cloud): update CSP to allow for file upload in support chat (#7703)
Update CSP to allow connections to AWS S3 domains

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-06 12:45:09 +02:00
Steffen SchmitzandGitHub 0a63cc61fc chore: fix timestamp ordering for amt table (#7700)
* chore: fix timestamp ordering for amt table

* chore: update comment
2025-07-05 13:50:53 +00:00
Max DeichmannandGitHub c385121406 chore: improve auto selection of automations page (#7698)
* chore: add webhook throughput alert

* chore: add webhook throughput alert

* chore: add webhook throughput alert
2025-07-05 08:28:45 +00:00
Max DeichmannandGitHub 7d7932e024 fix: do not store output of webhook automations (#7697)
* push

* fixes
2025-07-05 08:07:46 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
b26362cbca feat: webhook for prompt version changes (#7012)
* push

* fix

* fix

* fix

* fix

* fix

* push

* fix

* push

* push

* push

* push

* fix

* fix

* push

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fixes

* Update .cursor/rules/global.mdc

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update web/src/features/rbac/constants/projectAccessRights.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* push

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* push

* change event prcessing

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* Update worker/src/__tests__/webhooks.test.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* push

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fix tests

* Update web/src/__tests__/async/prompts.v2.servertest.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* data model

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-05 07:02:30 +00:00
NimarandGitHub 9b8095f8f5 chore(dx): unit tests run against their own db (#7671)
* chore(dx): unit tests run against their own db

* fix clickhouse

* clean up a bit

* close

* update only use different postgres

* update with command

* remove CH
2025-07-04 14:50:27 +00:00
Hassieb PakzadandGitHub 8b76617019 fix(dataset-run-items-api-spec): correct response type (#7690) 2025-07-04 15:20:54 +02:00
93e50f4720 feat(prompts): Add prompt filtering by config (#7688)
* Add config column to prompts table for JSON filtering

Co-authored-by: marc <marc@langfuse.com>

* remove plan

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-04 13:09:07 +00:00
Hassieb PakzadandGitHub c1388c8699 chore: upgrade langfuse-langchain (#7687) 2025-07-04 12:37:00 +00:00
Steffen SchmitzandGitHub 43fc305d5b fix: create virtual timestamp column in AMT query to support all filters (#7676)
* fix: create virtual timestamp column in AMT query to support all filters

* chore: add test logging

* chore: skip database prune for async tests

* chore: cleanup

* chore: drop console logs
2025-07-04 12:06:43 +00:00
marliessophieandGitHub ebd4cf3e8a chore(sentry): restrict loggerLink to development environment only for less noise in sentry (#7684) 2025-07-04 12:00:23 +00:00
NimarandGitHub 5dfc821575 fix: run prettier on my changes (#7685)
format all
2025-07-04 11:52:37 +00:00
marliessophieandGitHub 3965a5174f chore(sentry): only report server side errors (5xx) to sentry; skip reporting client side errors (#7682) 2025-07-04 11:09:46 +00:00
Max DeichmannandGitHub df237a11d5 chore: refactor prompts test (#7680)
* push

* fixes

* fixes

* fixes
2025-07-04 09:55:27 +00:00
marliessophieandGitHub 373cefdcb5 chore(sentry): do not report expected API key errors to sentry (#7678) 2025-07-04 08:57:37 +00:00
Steffen SchmitzandGitHub b92a8c6d81 fix(dashboard): adjust filters on view change and expose bad request errors (#7675)
* fix(dashboard): adjust filters on view change and expose bad request errors

* chore: skip error typing
2025-07-04 08:52:27 +00:00
Steffen SchmitzandGitHub 68422d404c chore(dashboard): remove breakdowns by 1:N dimensional relations (#7674) 2025-07-04 07:28:42 +00:00
Steffen SchmitzandGitHub d42c4e5e2f perf: start to experimentally shift reads to new aggregatingmergetrees (#7631)
* perf: start to experimentally shift reads to new aggregatingmergetrees

* chore: linting

* chore: more linting

* chore: create checklist for trace table view migrations

* feat: shift by id queries onto new AMTs

* chore: cover traces for public api

* chore: add notes for maxMap

* chore: incorporate feedback into query declarations

* chore: cleanup queries

* chore: adjust reads to new layout

* chore: update ingestion pipeline to make use of nullable fields

* chore: add sampling

* chore: update sampling decision attribute

* chore: fix limit for getTracesByIds

* chore: spacing

* chore: linting
2025-07-03 17:30:10 +00:00
0d2586e4f2 feat(ui): Add trace download button, exports trace and observation metadata as json (#7668)
* Add download trace as JSON button to TracePage

Co-authored-by: marc <marc@langfuse.com>

* Add JSON download functionality for trace details

Co-authored-by: marc <marc@langfuse.com>

* Checkpoint before follow-up message

* Refactor trace download button placement and add conditional rendering

Co-authored-by: marc <marc@langfuse.com>

* Refactor trace download to export trace and observations data

Co-authored-by: marc <marc@langfuse.com>

* fix

* push

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-03 17:28:00 +00:00
Hassieb PakzadandGitHub ff40e79143 feat(llm-connections): add gemini 2.5 flash lite and gemini 2.5 prices (#7666)
* feat(llm-connections): add gemini 2.5 flash lite and gemini 2.5 prices

* push

* add reasoning
2025-07-03 16:25:11 +00:00
marliessophieandGitHub bd04f6b186 fix(nav): implement responsive navigation with DropdownMenu for mobile (#7665)
* feat(nav): implement responsive navigation with DropdownMenu for mobile

* chore: push
2025-07-03 16:06:44 +00:00
Max DeichmannandGitHub d107f98654 perf: increase ingestion queue retry attempts (#7649)
* push

* push
2025-07-03 12:55:37 +00:00
Hassieb PakzadandGitHub de9fa8199a feat(llm-connections): add gemini 2.5 pro and flash GA (#7659) 2025-07-03 12:33:32 +00:00
Marlies Mayerhofer 0e1498a094 chore: release v3.78.2 2025-07-03 13:37:47 +02:00
marliessophieandGitHub 38f020e482 fix(scores): update score validation logic to accept inclusive ranges for score config validation (#7650)
* fix(scores): update score validation logic to accept inclusive ranges for score config validation

* chore: typo
2025-07-03 09:22:33 +00:00
Hassieb PakzadandGitHub 07e1d4e960 fix(otel): trace update events on present metadata (#7648) 2025-07-03 08:36:40 +00:00
Hassieb PakzadandGitHub 602d00a5bb chore: upgrade langfuse-langchain (#7647) 2025-07-03 08:16:41 +00:00
7395bd8d83 fix: loading state of csv import button (#7640)
* Disable import button during processing and show loading state

Co-authored-by: marc <marc@langfuse.com>

* prettier

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-03 00:12:27 +00:00
Marc KlingenandGitHub 63d164ad52 chore: improve cursor background agent configuration (#7641)
* fix

* push

* test
2025-07-03 01:41:14 +02:00
Thorsten SpiekerandGitHub f1b915c8b7 fix: remove invalid metrics and dimensions from pivot table widget on view change (#7636)
* fix: remove invalid metrics and dimensions from pivot table widget on view change

* linting
2025-07-02 19:01:26 +02:00
Steffen SchmitzandGitHub e3886b94c1 chore: set static backoff for query clickhouse retries (#7635) 2025-07-02 17:35:43 +02:00
Steffen SchmitzandGitHub 44376c2a9f chore: ignore all pending linting errors and enforce zero warnings (#7634) 2025-07-02 14:32:35 +00:00
marliessophieandGitHub 2e85c9ac15 chore(evals): add QUEUE_ERROR_MESSAGES constants for improved error handling (#7630)
* chore(evals): add QUEUE_ERROR_MESSAGES constants for improved error handling

* chore: push
2025-07-02 14:01:15 +00:00
Steffen SchmitzandGitHub 7cc81deb65 chore: retry socket hang ups for CH queries (#7628) 2025-07-02 13:52:26 +00:00
Steffen SchmitzandGitHub a13ddd41ae exp: write observation and score info to traces_mt (#7590)
* exp: write observation and score info to traces_mt

* chore: update declarations for experimental traces table

* chore: change anyLast pattern to working condition

* chore: revert anyLast setup

* chore: revert

* chore: cleanup queries
2025-07-02 11:53:38 +00:00
Nimar ebe85f2a67 chore: release v3.78.1 2025-07-02 12:15:50 +02:00
NimarandGitHub c7b62adbab feat(experiments) support for placeholders as message histories (#7615)
* run experiments with placeholders as well

* add missing file

* fix experiements

* run outside of loop

* add tests

* fix tests

* make tests clean again

* also test that variables resovle first
2025-07-02 10:04:51 +00:00
66d4926fd0 feat: add tags to search on prompts table (#7619)
* Enhance prompt search to include tags with case-insensitive matching

Co-authored-by: marc <marc@langfuse.com>

* drop md

* prettier

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-01 20:37:57 +00:00
b26f5c512f refactor: pivot table dropdowns (#7611)
---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-07-01 21:21:46 +02:00
marliessophieandGitHub 9227eaab9c feat(deleteButton): add custom delete message for running evaluator (#7617) 2025-07-01 18:33:41 +00:00
Nimar 3debac082a chore: release v3.78.0 2025-07-01 18:34:30 +02:00
fb43e03171 feat(prompts): Add search bar to prompts table (#7603)
* Add search functionality to prompts table with query param support

Co-authored-by: marc <marc@langfuse.com>

* prettier

* fix

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-01 14:16:45 +00:00
marliessophieandGitHub dd04da64ba fix(prisma): update JobExecution and DefaultLlmModel relations to include onUpdate and onDelete behaviors to match DB constraints (#7602)
* fix(prisma): update JobExecution and DefaultLlmModel relations to include onUpdate and onDelete behaviors to match DB constraints

* chore(prisma): update foreign key onUpdate behavior for job_template_id in JobExecution model

* Revert "chore(prisma): update foreign key onUpdate behavior for job_template_id in JobExecution model"

This reverts commit e65c12c9c36a1f8732560e665b1d8d7422c530f0.
2025-07-01 14:07:42 +00:00
b94cd9880a chore: fix typos (#7604)
* Fix typos: GitHub brand name and "check out" verb phrase

Co-authored-by: marc <marc@langfuse.com>

* Checkpoint before follow-up message

* Fix GitHub brand name and typos across documentation and UI

Co-authored-by: marc <marc@langfuse.com>

* Changes from background composer bc-bdbd909e-6932-4489-b125-e66b1d0f5ebe

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-07-01 13:54:09 +00:00
NimarandGitHub 0f2d2b8850 fix(prompts): don't allow placeholders and variable with same name on same prompt version (#7598)
* no naming collisions between prompt and placeholder

* tests for variable placeholder name collision

* fix test spelling

* also show conflict in playhground
2025-07-01 13:39:35 +00:00
steffen911 69db85cfe4 chore: release v3.77.0 2025-07-01 13:37:53 +02:00
Thorsten SpiekerandGitHub b445471f98 feat(dashboard): add pivot table widget type (#7546) 2025-07-01 13:36:54 +02:00
2ce014980b feat(posthog-export): Add environment attribute to PostHog exports (#7581)
* Add environment to PostHog event tracking for traces and observations

Co-authored-by: marc <marc@langfuse.com>

* use object envs instead of trace env

* fix

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-30 22:47:14 +00:00
Nimar 40ad6761cc chore: release v3.76.0 2025-06-30 19:13:49 +02:00
bd36669c6a feat(prompts): prompt chat message placeholders in backend and api (#7388)
* add first unit test

* first test passes

* share types

* fix tests

* update docs for placeholder

* improve tests

* add message compile

* update types

* add backend remaining

* fix playground

* review suggest

* add playground

* code review

* fix spelling

* fix validation in FE

* fix validation

* add seed data

* allow placeholdered prompt to playgrounf

* add tooltip

* fix build

* code feedback

* use zod to validate

* Update CLAUDE.md

* chore: remove prefix from user_id column (#7477)

* chore: add .env values for sdk integration tests (#7479)

* fix(data-tables): table peeks and full links should respect custom basePaths (#7483)

* strip url

* fix for dataset peek

* add to cursor rules

* chore: release v3.75.1

* feat: line breaks on evaluation run tooltip hints (#7486)

* Fix comment display to preserve whitespace and line breaks

* include in seeder

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* chore: reduce number of listed S3 files in event processing (#7487)

* chore(cloud): apply "cloud:demo" tier to demo org tenant in plain sup… (#7489)

chore(cloud): apply "cloud:demo" tier to demo org tenant in plain support

* make the api beautiful again

* update api

* reorg import

* fix improt

* prefill message placeholders

* remove todos

* fix prompt mapping

---------

Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
Co-authored-by: Marc Klingen <git@marcklingen.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-30 16:54:07 +00:00
Steffen SchmitzandGitHub 98cc1bb7a1 exp: add aggregating merge trees experiment for traces tables (#7574)
* exp: add aggregating merge trees experiment for traces tables

* chore: lint
2025-06-30 16:46:50 +00:00
marliessophieandGitHub 3dcdbd99b3 fix(evals): show loading state of detail buttons only if preview is enabled (#7570)
fix(evals): show loading state of detail buttons only if show preview is enabled
2025-06-30 14:40:17 +00:00
Hassieb PakzadandGitHub 1a5c5f6383 fix(otel): skip multiproject check for non-python (#7568) 2025-06-30 13:38:14 +00:00
Steffen SchmitzandGitHub 867f6c7484 feat(dashboard): allow filtering by score value in custom dashboards (#7567)
* feat(dashboard): allow filtering by score value in custom dashboards

* chore: remove console log
2025-06-30 13:16:14 +00:00
NimarandGitHub 1097f1b8d5 chore(dx): show different favicon for dev mode - easier to find tab (#7550)
add dev favicon
2025-06-30 12:00:25 +00:00
Marc KlingenandGitHub a73156bbe5 fix: warn if missing encryption_key when updating posthog integration (#7560)
* fix: warn if missing encryption_key when updating posthog integration

* push
2025-06-30 11:20:22 +00:00
marliessophieandGitHub 7a9c62b106 refactor(evals): optimize variable extraction logic in EvaluationPromptPreview and useExtractVariables hooks (#7409)
* refactor(evals): optimize variable extraction logic in EvaluationPromptPreview and useExtractVariables hooks

* style: improve preview header

* chore: eslint
2025-06-30 10:57:37 +00:00
marliessophieandGitHub 70cc910ee9 fix(datasets): allow users to update metadata and description while keeping name constant (#7562)
* feat: add whitelisted name support for unique name validation in dataset form

* fix: pass metadata to update dataset form
2025-06-30 09:55:00 +00:00
Steffen SchmitzandGitHub 64a71f059f chore: add trace update compatibility properties (#7561) 2025-06-30 09:28:23 +00:00
Max DeichmannandGitHub f5a0c7cfed chore: move prompt types into shared package (#7555)
* push

* fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2025-06-29 10:38:14 +00:00
Marc KlingenandGitHub 2472d2f6da chore(cloud): remove posthog groups to reduce costs (#7552) 2025-06-27 20:01:38 +00:00
Max DeichmannandGitHub dd0446f4dd chore: add python sdk v3 notification (#7543)
* fixes

* push
2025-06-27 14:08:47 +00:00
Max DeichmannandGitHub 7263a1554c chore: fix admin api (#7530)
* fixes

* fixes
2025-06-27 11:10:00 +00:00
Max DeichmannandGitHub 2c25c27b51 chore: skip s3 LIST for projects configured in env (#7541)
push
2025-06-27 11:09:50 +00:00
Steffen SchmitzandGitHub f2b92e1f23 chore: only consider getWaitingCount for queue length (#7542) 2025-06-27 09:44:37 +00:00
Steffen SchmitzandGitHub 593c0a567c perf: execute eval variable parsing sequentially (#7540)
* perf: execute eval variable parsing sequentially

* chore: skip caching for dataset items
2025-06-27 09:24:02 +00:00
Steffen SchmitzandGitHub ac1f465634 perf: cache eval entities during variable extraction (#7538) 2025-06-27 07:56:01 +00:00
NimarandGitHub 083b1357a1 fix: long trace names shouldn't break at icons (#7382)
* fix breaks

* fix

* undo
2025-06-26 18:27:36 +00:00
Nimar 30e594e839 chore: release v3.75.4 2025-06-26 19:25:59 +02:00
NimarGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2b8f583c44 fix(prompts): custom label creation popover too wide (#7529)
* fix UI

* disable button if nothing changed

* Update web/src/features/prompts/components/SetPromptVersionLabels/index.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-26 17:20:27 +00:00
steffen911 ef3656a8d3 chore: release v3.75.3 2025-06-26 14:57:02 +02:00
Steffen SchmitzandGitHub 5cf2411e74 feat: include all values from scoreConfig in filterOptions (#7524) 2025-06-26 12:54:00 +00:00
Steffen SchmitzandGitHub 6d9956ae6e fix(dashboard): correctly display name filters on widget edit (#7522) 2025-06-26 12:29:39 +00:00
NimarandGitHub 37d1ac7ff8 chore: update contributing.md with recent state (#7510) 2025-06-25 17:50:07 +00:00
Marc KlingenandGitHub 339633e8de chore(cloud): color of upgrade notification (#7508)
* add danger color to sidebar notification

* add icon when sidebar is collapsed

* fix
2025-06-25 17:05:33 +00:00
Nimar a31af76ba1 chore: release v3.75.2 2025-06-25 19:01:15 +02:00
Steffen SchmitzandGitHub dfd527a740 fix: show falsy values in IOTableCell unless null/undefined (#7507) 2025-06-25 16:53:23 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
d65bdaa761 fix(dashboards): represent small numbers in scientific notation (#7506)
* fix(dashboards): represent small numbers in scientific notation

* Update web/src/utils/numbers.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-25 16:09:17 +00:00
NimarandGitHub ef08a514af chore: enable sentry replays (#7498)
* Update sentry.client.config.ts

* simpify

* fix TS build - complexify
2025-06-25 10:11:15 +00:00
NimarandGitHub 3b3d9de206 chore(deps): upgrade turborepo 1.13 to 2.5.4+ (#7492)
* upgrade turbo

* add web dev turbo runner

* upgrade docker

* fix linter

* set env mode to loose (as in turbo v1) so we don't have to double specify env vars
2025-06-25 09:48:24 +00:00
Max DeichmannandGitHub f1ca851b0d chore: gracefully handle failed evals (#7500)
push
2025-06-25 09:14:16 +00:00
Steffen SchmitzandGitHub c1c6035778 chore: propagate additional properties on logs and spans (#7497) 2025-06-25 08:11:44 +00:00
Marc KlingenandGitHub d2bf002ea2 chore: improve support widget CTAs (#7494) 2025-06-25 00:08:32 +00:00
Marc KlingenandGitHub 88f9008558 chore(cloud): apply "cloud:demo" tier to demo org tenant in plain sup… (#7489)
chore(cloud): apply "cloud:demo" tier to demo org tenant in plain support
2025-06-24 20:05:22 +00:00
Steffen SchmitzandGitHub 586e852375 chore: reduce number of listed S3 files in event processing (#7487) 2025-06-24 18:18:35 +00:00
94e49dbc4e feat: line breaks on evaluation run tooltip hints (#7486)
* Fix comment display to preserve whitespace and line breaks

* include in seeder

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-24 18:07:55 +00:00
Nimar d37e676ea6 chore: release v3.75.1 2025-06-24 19:29:19 +02:00
NimarandGitHub 626b36e225 fix(data-tables): table peeks and full links should respect custom basePaths (#7483)
* strip url

* fix for dataset peek

* add to cursor rules
2025-06-24 17:21:03 +00:00
NimarandGitHub 62cd67f431 chore: add .env values for sdk integration tests (#7479) 2025-06-24 15:37:44 +00:00
Steffen SchmitzandGitHub 09f1bb9f7f chore: remove prefix from user_id column (#7477) 2025-06-24 15:19:45 +00:00
Steffen SchmitzandGitHub d6963297a7 fix: handle dataset queries with cached traces in evals (#7475)
* fix: handle dataset queries with cached traces in evals

* chore: replace all validatedfilter calls in Inmemorytacefitlerlogic
2025-06-24 14:35:17 +00:00
Steffen SchmitzandGitHub 06b9e88d3a chore: update lossless-json to 4.1.1 (#7473) 2025-06-24 12:55:53 +00:00
Hassieb PakzadandGitHub 39feb65d90 fix(llm-connection-vertex): use zodV3 for structured output calls (#7468)
* fix(llm-connection-vertex): use zodV3 for structured output calls

* fix
2025-06-24 12:07:56 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
b85511a3fe perf: cache eval trace check across multiple configs (#7407)
* perf: cache eval trace check across multiple configs

* chore: update conditions

* chore: udpate conditions

* chore: remove unused eval test

* chore: update filter stuff

* chore: reduce unused test properties

* chore: filters

* chore: add missing in memory filter types

* Update packages/shared/src/server/services/InMemoryFilterService.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: comments

* chore: add metrics for eval config fetches

* chore: linting

* chore: fix tests

* chore: linting

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-24 12:07:54 +00:00
marliessophieandGitHub 88e47faec4 refactor(dialog): update button layouts in forms for consistent styling (#7470) 2025-06-24 11:04:48 +00:00
steffen911 9291f8e110 chore: release v3.75.0 2025-06-24 11:18:52 +02:00
Steffen SchmitzandGitHub fad5d047e7 fix: avoid ambigious identifiers for full-text search (#7465) 2025-06-24 10:46:04 +02:00
5071009dfb fix: export datasets (#7455)
* feat: add mapDatasetItems table definition

* feat: add dataset_items table to database read stream

* feat: add export button to dataset items table

* feat: add dataset items batch export tests

* fix: remove unnecessary filtering attempt

* fix: remove unused imports, refactor use of b utton component

* push fixes

* fix: only export dataset items for the displayed dataset

* fix: set htmlSourcePath to empty string for dataset items with no source

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-06-23 19:21:37 +00:00
Marc KlingenandGitHub 8dfb023a55 chore(cloud): update in-product pricing page for new graduated pricing (#7393)
* chore(cloud): update in-product pricing page for new graduated pricing

* rename to units

* push
2025-06-23 21:14:53 +02:00
Marc KlingenandGitHub 870adf284a chore(cloud): apply pro rate limits to core plan (tmp) (#7392) 2025-06-23 19:00:04 +00:00
Hassieb PakzadandGitHub 164f81f249 fix(trace-ui): remove unnecessary json key (#7454) 2025-06-23 17:07:24 +00:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
d43f30a40f feat(llm-api-keys): allow configuring GCP location for VertexAI (#7367)
* feat(llm-api-keys): allow configuring GCP location for VertexAI

* Update web/src/features/public-api/components/CreateLLMApiKeyForm.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-23 14:04:29 +00:00
Hassieb PakzadandGitHub 4243be58f6 fix(models): zodResolver ignoring modelName (#7451) 2025-06-23 12:11:22 +00:00
88074ea024 feat: add ability to export dataset items (#7199)
* feat: add mapDatasetItems table definition

* feat: add dataset_items table to database read stream

* feat: add export button to dataset items table

* feat: add dataset items batch export tests

* fix: remove unnecessary filtering attempt

* fix: remove unused imports, refactor use of b utton component

* push fixes

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-06-23 11:09:10 +00:00
marliessophieandGitHub a31add749f fix(playground): do not report playground errors to sentry (#7448) 2025-06-23 10:48:32 +00:00
NimarandGitHub 8d19030391 fix(data-tables): pagination not adjusted by enter key (#7425)
fix pagination not accessible by enter key
2025-06-23 10:47:21 +00:00
marliessophieandGitHub 8bab408098 fix(observation): filter out null values from model parameters in ObservationPreview (#7447) 2025-06-23 10:27:44 +00:00
marliessophieandGitHub 9f2dd6190d fix(scores): add environment field to upsert annotation score (#7446) 2025-06-23 10:13:56 +00:00
steffen911 89a339a6bf chore: release v3.74.0 2025-06-23 11:45:18 +02:00
Steffen SchmitzandGitHub ca9c64099f fix(exports): propagate search state to export (#7445) 2025-06-23 09:33:35 +00:00
NimarGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
5a10257b64 add claude.md file (#7439)
* add claude.md file

* spelling

* Update CLAUDE.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-22 21:51:41 +00:00
ClemoandGitHub 6ee95d5cac docs: update readme 2025-06-20 16:23:46 -07:00
marliessophieandGitHub 0a9e322d26 feat(sidebar): persist collapsed state (#7424)
* feat(nav): integrate HoverCard component for improved navigation item display

* refactor(nav): enhance HoverCard integration and sidebar styles for improved UX

* chore(sidebar): replace cookie-based state management with local storage for sidebar state persistence

* chore: push

* refactor(nav): wrap HoverCardContent in Portal for improved rendering context

* style: mark hovered item as active
2025-06-20 15:11:34 +00:00
marliessophieandGitHub faee716eba chore(seeder): tweak session and prompt data (#7423)
* chore(seeder): tweak session and prompt data

* chore: push
2025-06-20 12:42:26 +00:00
Nimar 03880ba4c6 chore: release v3.73.1 2025-06-20 13:18:11 +02:00
NimarandGitHub 2626c1a57c fix(prompts): bottom pagination buttons didn't respond (#7421)
fix bottom pagination buttons don't work
2025-06-20 09:51:45 +00:00
marliessophieandGitHub 31810bb2b7 fix(button): set default button type to "button" if not specified (#7406) 2025-06-19 14:51:14 +00:00
marliessophieandGitHub b5c6727db7 fix(evals): use jsonpath-plus for JSONPath extraction in frontend preview (#7404)
* fix(evals): use jsonpath-plus for JSONPath extraction in frontend preview

* refactor(evals): remove

* chore: perf

* fix: imports
2025-06-19 14:43:28 +00:00
marliessophieandGitHub e1d0b5e173 chore(seeder): represent correct langfuse data model in seed data (#7389)
* chore(CreateLLMApiKeyForm): replace Input with PasswordInput for improved UX

* chore: revert

* chore(data): add seed data and constants for datasets, prompts, and traces

- Introduced new JSON files for chat ML and nested JSON data.
- Created seed constants for datasets and prompts to streamline data generation.
- Updated seed scripts to utilize new constants and improve dataset creation logic.
- Enhanced Clickhouse preparation scripts to support new dataset structures and trace generation.
- Added utility functions for generating unique trace IDs.

This commit lays the groundwork for improved data handling and observability in the Langfuse application.

* chore(data): remove deprecated prompt files and enhance seed constants

* chore(prisma): remove unique constraint on projectId, datasetId, and input in DatasetItem model

* chore: push

* chore(seeder): introduce seeder abstraction

* chore: typos

* chore: restructure files

* chore: restructure files

* chore: fix imports

* chore: fix imports

* chore: lint

* chore: simplify

* chore: ensure eval data integrity

* chore: update metric name calculation in ClickHouse query builder

* refactor: use test-utils for clickhouse inserts
2025-06-19 13:23:13 +00:00
Steffen SchmitzandGitHub f147f344d6 chore: add devcontainer.json (#7402)
* chore: add devcontainer.json

* chore: add pnpm install

* chore: replace tab with space

* chore: node version

* chore: add host requirements

* chore: revert hostrequirements

* Update .devcontainer.json

* Update .devcontainer.json

* add claude code
2025-06-19 12:33:44 +00:00
Steffen SchmitzandGitHub 9c0d16fb3a chore: upgrade prisma to 6.10.1 (#7401) 2025-06-19 07:30:10 +00:00
Steffen SchmitzandGitHub 20d11f1a4d chore: move trpc unauthed logging to info/warn (#7391) 2025-06-18 18:21:11 +02:00
Steffen SchmitzandGitHub 89eedcd807 chore: track clickhouse calls in ingestion-pipeline as clients & auth error logging (#7390) 2025-06-18 17:26:54 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ef5efe530e chore(deps): bump @react-email/components from 0.0.42 to 0.1.0 (#7371)
Bumps [@react-email/components](https://github.com/resend/react-email/tree/HEAD/packages/components) from 0.0.42 to 0.1.0.
- [Release notes](https://github.com/resend/react-email/releases)
- [Changelog](https://github.com/resend/react-email/blob/canary/packages/components/CHANGELOG.md)
- [Commits](https://github.com/resend/react-email/commits/@react-email/components@0.1.0/packages/components)

---
updated-dependencies:
- dependency-name: "@react-email/components"
  dependency-version: 0.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-18 10:43:41 +00:00
NimarandGitHub 7918faa942 fix: center align dataset description vertically (#7383)
center align vertically
2025-06-18 09:53:56 +00:00
Steffen SchmitzandGitHub 6210ca3e14 chore: add otel conventional attributes to clickhouse db spans (#6550)
* chore: add otel conventional attributes to clickhouse db spans

* chore: use db.system instead of db.system.name

* chore: set span kind client for clickhouse queries
2025-06-18 09:32:28 +00:00
steffen911 ff8c44c25f chore: release v3.73.0 2025-06-18 10:08:46 +02:00
marliessophieandGitHub 20480321ab chore(evals): re-introduce delay input field for evaluation timing (#7357)
* chore(evals): re-introduce delay input field for evaluation timing

* chore(evals): enforce minimum value for delay
2025-06-18 07:49:22 +00:00
Steffen SchmitzandGitHub d39855a2e3 feat: select fields in get traces api route (#7356)
* feat: select fields in get traces api route

* chore: add implementation

* chore: rounding

* chore: remove planning file

* chore: align tests

* chore: cleanup

* chore: lint

* chore: add zod verifications again
2025-06-18 07:01:02 +00:00
Marc KlingenandGitHub 9fd03082ea feat(dashboard): add csv download from ui and loading state (#7364)
* feat(web): add download dropdown for dashboard charts

* csv escaping

* fix

* add loading indicator

* add loading spinner and fix 0 state during loading of BigNumber

* fix name DownloadButton

* refactor

* add success confirm
2025-06-17 16:18:52 +00:00
marliessophieandGitHub d6941f903b chore(CreateLLMApiKeyForm): replace Input with PasswordInput for improved UX (#7359)
* chore(CreateLLMApiKeyForm): replace Input with PasswordInput for improved UX

* chore: revert
2025-06-17 13:59:40 +00:00
Nimar 125db51dd1 chore: release v3.72.1 2025-06-17 13:21:19 +02:00
NimarandGitHub 7f40a32aa9 fix(prompts): undefined prompt name when clicking on metrics (#7358)
* fix undefined prompt name when clicking on metrics

* better comment

* only remove last metrics segement

* pass promptName as prop

* add note for bug

* update comment

* fix spelling

* add noreferer
2025-06-17 10:21:30 +00:00
Hassieb Pakzad 6f3050af0f chore: release v3.72.0 2025-06-17 12:21:49 +02:00
Hassieb PakzadandGitHub 926c2b8e45 feat(playground): allow non-streaming responses (#7344)
* feat(playground): allow non-streaming responses

* push

* push

* push

* push

* push
2025-06-17 12:21:02 +02:00
Marc KlingenandGitHub 9e0bd8ed75 docs: Update network diagram (#7355)
docs: fix newline characters in network diagram
2025-06-17 09:12:03 +00:00
Hassieb PakzadandGitHub d0cca331ed fix(otel): force modelparam values to be strings (#7337)
* fix(otel): force modelparam values to be strings

* fix
2025-06-17 09:03:11 +00:00
Max DeichmannandGitHub 1f55032c34 fix: gracefully fail exceeded context length for evals (#7343)
fixes
2025-06-16 19:02:44 +00:00
marliessophieandGitHub 52967de1ac fix(slider): handle undefined or null values in slider input (#7341) 2025-06-16 18:52:13 +00:00
marliessophieandGitHub 9ca8614644 fix(BigNumber): adjust decimal formatting to limit to a maximum of 3 decimal places (#7332) 2025-06-16 15:25:24 +00:00
Jannik MaierhöferandGitHub 80714c6e5e chore(ui): change wording for consistency (#7016) 2025-06-16 14:36:28 +00:00
Nimar b7eb46bd4b chore: release v3.71.0 2025-06-16 16:20:48 +02:00
NimarandGitHub ca3791e66d feat(prompts): virtual folders for prompt management (#7262) 2025-06-16 16:01:15 +02:00
Hassieb PakzadandGitHub 652036e8d3 chore(otel): add metrics for shallow trace events and sdk batches (#7325)
* chore(otel): add metrics for shallow trace events and sdk batches

* push
2025-06-16 13:19:49 +00:00
marliessophieandGitHub 9054df5685 refactor(peek): remove urlPathname parameter from peek navigation and state hooks (#7292)
* refactor(peek): remove urlPathname parameter from peek navigation and state hooks

* chore: eslint

* chore: eslint

* chore: eslint
2025-06-16 11:24:40 +00:00
steffen911 30ca47b667 chore: release v3.70.0 2025-06-16 13:14:22 +02:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
ce1c1a5016 feat: allow sharded ingestion queue (#7240)
* feat: create outline for redis cluster mode setup

* chore: fix compilation problems

* chore: add redis-cluster docker compose setup

* chore: make cluster config compile

* chore: remove unused env var declarations

* chore: remove experimental queue length metrics flag

* chore: include queue name into prefix

* chore: lint

* chore: update node config

* chore: lint

* chore: tty to skip model match cache

* chore: execute test flush on all nodes

* chore: add aws specific dns lookup

* chore: add ioredis debug logs for tests

* chore: optimize setup for ci

* chore: update example config

* chore: secret

* chore: cleanup

* chore: remove redis cluster testing script

* feat: allow sharded ingestion queue

* Update packages/shared/src/server/redis/ingestionQueue.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: patch

* chore: remove redundent getQueue

* chore: lint

* chore: skip queue metric collection

* chore: release v3.67.1-0

* chore: linting

* chore: release v3.67.1-1

* chore: release v3.67.1-2

* chore: queue metric collection

* chore: update docs

* chore: add docs

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-16 10:28:55 +00:00
Steffen SchmitzandGitHub ed01cb7d91 chore: upgrade langchain components (#7320) 2025-06-16 10:18:59 +00:00
marliessophieandGitHub 17364919af fix(observation-ui): enhance badge rendering with text truncation and improved value display (#7321) 2025-06-16 10:03:20 +00:00
Hassieb PakzadandGitHub 0c58edcf5e feat(otel): parse trace attributes from trace metadata (#7314)
* feat(otel): parse trace attributes from trace metadata

* add test
2025-06-16 08:02:37 +00:00
Steffen SchmitzandGitHub 90d7c7df98 feat: add redis cluster mode setup (#7162)
* feat: create outline for redis cluster mode setup

* chore: fix compilation problems

* chore: add redis-cluster docker compose setup

* chore: make cluster config compile

* chore: remove unused env var declarations

* chore: remove experimental queue length metrics flag

* chore: include queue name into prefix

* chore: lint

* chore: update node config

* chore: lint

* chore: tty to skip model match cache

* chore: execute test flush on all nodes

* chore: add aws specific dns lookup

* chore: add ioredis debug logs for tests

* chore: optimize setup for ci

* chore: update example config

* chore: secret

* chore: cleanup

* chore: remove redis cluster testing script
2025-06-16 07:51:08 +00:00
Max DeichmannandGitHub 07fbae3da7 security: upgrade react-syntax-highlighter (#7313) 2025-06-16 07:35:15 +00:00
steffen911 b4bd8208c3 chore: release v3.69.0 2025-06-16 09:35:37 +02:00
Max DeichmannandGitHub c8b282dc82 chore: upgrade react email (#6358)
* chore: upgrade react email

* chore: upgrade react email

* chore: upgrade react email

* chore: upgrade react email

* upgrade

* upgrade
2025-06-15 13:05:52 +00:00
Max DeichmannandGitHub e2922aa787 fix: fix select all UI bug (#7312) 2025-06-15 12:13:25 +00:00
Max DeichmannandGitHub e8690eeb0b chore: clean up scripts (#7306)
chore: clean up
2025-06-13 23:05:34 +00:00
Max DeichmannandGitHub a695d58e8e chore: improve search tooltip (#7305)
push
2025-06-13 22:59:11 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
b097a98444 chore(deps): bump next from 14.2.26 to 14.2.30 (#7301)
Bumps [next](https://github.com/vercel/next.js) from 14.2.26 to 14.2.30.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.26...v14.2.30)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.30
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-13 19:12:39 +00:00
Max DeichmannandGitHub 87c6d45a01 security: upgrade release-it (#7300) 2025-06-13 17:27:23 +00:00
d5a79b1830 chore(ui) Update LLM connection API key message (#7278)
Update API key description based on cloud or self-hosted deployment

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-13 16:22:56 +00:00
Steffen SchmitzandGitHub 40b967c258 fix: add auth headers to ctx within middleware, update not found logging (#7295) 2025-06-13 11:56:14 +00:00
marliessophieandGitHub b6b7a7c670 chore(dialog): move close button to DialogHeader for improved accessibility (#7294)
refactor(dialog): move close button to DialogHeader for improved accessibility
2025-06-13 11:08:07 +00:00
Hassieb PakzadandGitHub 346e787784 fix(prompts-api): make updates via API concurrency safe (#7287)
* fix(prompts-api): make updates via API concurrency safe

* push
2025-06-13 10:04:57 +00:00
Steffen SchmitzandGitHub 8e67f851b0 fix: keep zod/v3 schema for langchainjs interactions (#7293)
Patches https://github.com/langchain-ai/langchainjs/issues/8357
2025-06-13 09:33:40 +00:00
marliessophieandGitHub 8b416da2ef fix(PopoverFilterBuilder): ensure filter buttons do not submit forms (#7290) 2025-06-13 09:29:59 +00:00
marliessophieandGitHub 88237dcb30 fix(evals): add missing Dialog Body and Footer components (#7289) 2025-06-13 09:12:35 +00:00
Steffen SchmitzandGitHub 8a0b3962ac chore: upgrade zod to v4 (#7271)
* chore: upgrade zod to v4

* chore: docs

* chore: switch enums to non-deprecated api

* chore: remove unnecessary any

* chore: reverts

* chore: cleanup

* chore: cleanup

* chore: prompts filter tags

* chore: fix sync tests

* chore: patch tests
2025-06-13 08:04:27 +00:00
marliessophieandGitHub 2f6414bbf4 fix(dataset-items): add sorting by id for deterministic ordering (#7280) 2025-06-13 07:34:48 +00:00
c7aa992586 feat: add export functionality to audit log table (#7202)
Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-06-12 20:48:58 +02:00
marliessophieandGitHub c2b5923e88 fix(NewDatasetItemForm): add JSON formatting utility for input, output, and metadata field (#7265)
* fix(NewDatasetItemForm): add JSON formatting utility for input, output, and metadata fields

* perf(NewDatasetItemFromExistingObject): conditionally render NewDatasetItemForm based on form state

* fix(NewDatasetItemForm): update JSON value check to handle undefined case
2025-06-12 17:57:22 +00:00
marliessophieandGitHub fc411d0c5c style(CommentList): reduce font size for author username (#7277) 2025-06-12 17:57:08 +00:00
marliessophieandGitHub 3fac8d2608 style(sessions): align header pills (#7282) 2025-06-12 16:38:55 +00:00
marliessophieandGitHub cd413fa887 style(TagButton): reduce size of spinner icon (#7281) 2025-06-12 16:29:11 +00:00
Marc KlingenandGitHub be7cc835be fix(trpcErrorToast): enhance error handling for 5xx server errors wit… (#7279)
fix(trpcErrorToast): enhance error handling for 5xx server errors with user-friendly message
2025-06-12 16:23:43 +00:00
marliessophieandGitHub b68896d14a refactor(DataTableToolbar): reposition PopoverFilterBuilder for improved UX (#7275) 2025-06-12 15:22:58 +00:00
Marc Klingen aae1909466 chore: release v3.68.0 2025-06-12 17:45:04 +02:00
Steffen SchmitzandGitHub 3d7960f43e fix: propagate searchQuery to deleteMany call (#7274)
* fix: propagate searchQuery to deleteMany call

* chore: address feedback

* chore: propagate filters from frontend
2025-06-12 14:16:35 +00:00
Hassieb PakzadandGitHub a121e24b79 feat(otel-ingestion): handle long-running traces (#7241)
* feat(otel-ingestion): handle long-running traces

* rename

* push

* push

* add tests

* add test

* use logger and update cache key

* refactor to OtelIngestionProcessor

* fix

* fix

* fix

* push

* fix graph view
2025-06-12 12:06:17 +00:00
7279a924ed fix: add llm_api_key_update to events, fix linting & call chain (#7160)
* feat: add ability to update llm api keys

* fix: add llm_api_key_update to events, fix linting & call chain

* fix: security issue, deleting last headers, typing

* fix: remove unused variable

* fix: remove unused variable

* refactor: align LlmApiKeys type with shared package across the components

* push fixes

* push fixes

* push fixes

* push fixes

* push fixes

* push fixes

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-06-12 11:57:38 +00:00
Max DeichmannandGitHub a10951b7e5 chore: allow trace deletions on entire history (#7252)
push
2025-06-12 11:57:04 +00:00
marliessophieandGitHub e456b08f92 fix(trace-graph): validate node existence before selection and handle errors gracefully (#7272) 2025-06-12 11:46:02 +00:00
85130cecb3 feat(cloud): 2-step sign in to make enterprise sso easier (#7250)
* chore: update `pnpm dx-f` to require no confirmation

* chore: setup cursor background agent

* Implement two-step login flow with SSO domain detection

* Simplify SSO fallback logic in sign-in page

* Conditionally render credentials form based on auth provider config

* Reorder SSO buttons and error display in sign-in page layout

* Checkpoint before follow-up message

* add focus password

* skip two step if no sso configured

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-12 09:52:29 +00:00
marliessophieandGitHub fbb15e26ec fix(dialog): adjust dialog component styling to prevent overflow issues (#7264)
* fix(dialog): adjust dialog component styling to prevent overflow issues

* refactor(dialog): simplify dialog footer styling across multiple components
2025-06-12 08:36:33 +00:00
Marc KlingenandGitHub dd203a1f18 chore: setup cursor background agent (#7251)
* chore: update `pnpm dx-f` to require no confirmation

* chore: setup cursor background agent

* push
2025-06-11 22:06:15 +00:00
Marc KlingenandGitHub 06c25f9ec3 chore: update pnpm dx-f to require no confirmation (#7249) 2025-06-11 21:23:07 +00:00
marliessophieandGitHub 7072ca2786 fix(popover): update max-width calculation for popover component to use fit-content (#7248) 2025-06-11 20:30:12 +00:00
marliessophieandGitHub cb1e5f4c19 fix(TracesTable): stabilize control column definition (#7247)
* fix(TracesTable): stabilize control column definition

- Ensured conditional rendering of the bookmarked column remains intact while simplifying the structure.

* refactor(TracesTable): enhance column visibility and order identifiers for control states
2025-06-11 19:36:46 +00:00
Steffen SchmitzandGitHub d4a28aa164 chore: upgrade clickhouse client dependency (#7246) 2025-06-11 18:40:43 +00:00
marliessophieandGitHub fda5fcdd0c refactor(dialog): improve dialog component layout and styling for billing settings (#7245)
- Updated dialog header to use a title component and improved button layout for clarity.
- Adjusted positioning of the close button for better alignment.
- Enhanced header and footer with rounded corners for a more polished look.
2025-06-11 16:39:42 +00:00
Marc KlingenandGitHub 24131e0791 feat(cloud): add stripe productid of cloud enterprise plan (#7244) 2025-06-11 16:14:16 +00:00
Marc KlingenandGitHub ffe132403e chore(cloud): increase user limit on core plan (#7243) 2025-06-11 15:51:39 +00:00
marliessophieandGitHub cce3bf35e1 fix(CreateExperimentsForm): clear modelConfig errors when valid model parameters are set and adjust layout spacing (#7233) 2025-06-11 10:34:06 +00:00
marliessophieandGitHub eccfec36e8 feat(MarkdownViewer): add URL sanitization using DOMPurify for safe link rendering (#7232) 2025-06-11 09:31:29 +00:00
Hassieb PakzadandGitHub a71faa4e60 feat(model-prices): add o1-pro (#7229) 2025-06-11 08:41:26 +00:00
b7362258c6 fix(ui): enable clipboard copying for self-hosted instances (#7227)
* fix(ui): Enable clipboard copying in insecure browser contexts

* Fixes #5541

* Adds clipboard utility file to manage copy interactions

* Implements fallback method for scenarios lacking Clipboard API
  support in insecure contexts (e.g., HTTP resources not delivered locally)

* Suggested review comments addressed

* chore: eslint

* chore: apply copy functionality to new occurrences

---------

Co-authored-by: akshitvijay <akshitthevijay@gmail.com>
2025-06-11 08:35:08 +00:00
Hassieb PakzadandGitHub 5eb1d98985 fix(graph-view): do not JSON parse node names (#7228) 2025-06-11 10:13:06 +02:00
Marlies Mayerhofer 1c574b2a5d chore: release v3.67.0 2025-06-11 01:39:36 +02:00
marliessophieandGitHub 6db9b87b6a chore(models): adjust price of o3 to reflected updated openai pricing (#7223) 2025-06-10 22:30:07 +00:00
marliessophieandGitHub 3e3ccb65ff feat(models): add o3-pro for cost tracking (#7222)
* feat(models): add o3-pro for playground and costs

* chore: remove from playground as langchain does not support o3 model for completions
2025-06-10 22:23:28 +00:00
marliessophieandGitHub 0d53b443e5 style(CreateScoreConfigButton): add padding to confirmation message for improved readability (#7220) 2025-06-10 16:26:16 +00:00
marliessophieandGitHub 1b795a3dcf chore(evals): show warning if API key was deleted (#7053)
* chore(evals): show warning if API key was deleted

* chore: fix typo
2025-06-10 15:44:36 +00:00
Hassieb PakzadandGitHub 2e94bc9e09 fix(graph-view): handle missing __start__ node for langgraph>=0.4 (#7215) 2025-06-10 15:44:12 +00:00
marliessophieandGitHub 8b94e8a3bd fix(annotation): handle deletes gracefully and protect against race conditions (#6848) 2025-06-10 15:00:43 +00:00
marliessophieandGitHub 91730a2eaf refactor(dialogs): dialog components with improved structure and styling (#7218) 2025-06-10 14:40:26 +00:00
marliessophieandGitHub 4f1a2337be fix(datasets): prevent delete modal from closing on mouse navigation event (#7163) 2025-06-10 14:30:05 +00:00
Hassieb PakzadandGitHub 4cf5aa03ec fix(trpc-prompts-api): handle prompt not found case on setLabels (#7216) 2025-06-10 13:00:23 +00:00
Steffen SchmitzandGitHub 922eafd4f6 chore: refactor chart display names into single util (#7209) 2025-06-10 07:29:06 +00:00
Hassieb PakzadandGitHub 9067f80206 fix(evals): do not throw errors if eval model does not consistently return JSON (#7187) 2025-06-08 14:05:51 +00:00
Steffen SchmitzandGitHub a7ca1fb269 chore: show frontend name for histogram chart type (#7181) 2025-06-06 15:09:54 +00:00
Hassieb PakzadandGitHub b51f321d22 feat(quickstart+readme): add python SDK v3 examples (#7157) 2025-06-06 16:36:16 +02:00
Hassieb PakzadandGitHub dd048a64a0 chore: upgrade langchain (#7178)
* chore: upgrade langchain

* push
2025-06-06 11:41:08 +00:00
ClemoandGitHub c6c19de0e8 docs: update readme, add terraform templates 2025-06-06 11:25:16 +02:00
marliessophieandGitHub 5491f176e1 chore(datasets): simplify DatasetAnalytics component interaction and integrate local storage for chart metrics (#7161)
* chore(datasets): simplify DatasetAnalytics component interaction and integrate local storage for chart metrics

* chore: eslint
2025-06-05 16:10:06 +00:00
marliessophieandGitHub 7a28473ebb perf(traces-table): improve performance (#7126)
* fixup: limit render duration of traces table

* fixup: reduce render cycles through data flows and memoization

* refactor: enhance performance and structure of table components

- Removed unnecessary console logs in `data-table.tsx`.
- Refactored `TablePeekView` to use a more structured props type and memoization for improved performance.
- Updated `usePeekData` to reduce stale time from 5 minutes to 1 minute.
- Replaced `IOTableCell` with `MemoizedIOTableCell` in various components to optimize rendering.
- Adjusted `peekView` configurations in `observations.tsx` and `traces.tsx` for consistency and clarity.

* chore: eslint
2025-06-05 14:16:10 +00:00
Steffen SchmitzandGitHub 005ce7318c chore: allow bullmq-api for non-cloud deployments (#7156) 2025-06-05 12:22:18 +00:00
marliessophieandGitHub 448c56e114 feat(dialog): add sticky header and footer (#7154)
* chore(ui-components): add dialog body

* style(popover): update width styling to use CSS variable for dynamic sizing

* feat(dialog): add DialogBody and DialogFooter components to enhance dialog structure

* chore: eslint

* chore: adjust modal height
2025-06-05 12:05:31 +00:00
Hassieb PakzadandGitHub 1d6e498e2b fix(otel): read resourceAttributes for release attribute (#7155) 2025-06-05 11:38:13 +00:00
Steffen SchmitzandGitHub 33ab717ae0 chore: include LANGFUSE_USE_AZURE_BLOB in docker-compose options (#7153) 2025-06-05 08:29:05 +00:00
Steffen SchmitzandGitHub ec563b775b feat(dashboards): add histogram chart type (#7140)
* feat(dashboards): add histogram chart type

* chore: update querybuilder behaviour

* chore: lint

* chore: push

* chore: format numbers nicely

* chore: store histogram bins in chartConfig

* chore: change order

* chore: add histogram function to metrics api

* chore: docs

* chore: generate docs
2025-06-05 07:43:35 +00:00
marliessophieandGitHub de427b5572 chore(evals): implement update confirmation for default evaluation model (#7137)
* chore(evals): implement update confirmation for default evaluation model

* chore: adjust alert to confirm

* chore: adjust alert to confirm
2025-06-04 17:08:44 +00:00
Marc Klingen 95921fa5da chore: release v3.66.1 2025-06-04 11:09:25 +02:00
Marc KlingenandGitHub 9e4d352366 fix(ee): pro plan label (#7136)
fix(ee): self-hosted pro plan label
2025-06-04 08:57:50 +00:00
steffen911 02449cbe0a chore: release v3.66.0 2025-06-04 09:24:50 +02:00
Steffen SchmitzandGitHub 616c68a87b feat: add support for new OTEL semantic event types (#7135) 2025-06-04 07:06:07 +00:00
Steffen SchmitzandGitHub 319b22ae78 chore: disentangle batch actions from batch exports (#7092)
* chore: disentangle batch actions from batch exports

* chore: revert

* chore: revert

* chore: revert
2025-06-04 06:53:52 +00:00
Max Deichmann b919562eed chore: release v3.65.3 2025-06-04 01:17:23 +02:00
Max DeichmannandGitHub 91c01e7363 chore: allow annotation queues on FOSS (#7130)
chore: move features
2025-06-03 22:17:02 +00:00
Max DeichmannandGitHub 48ec1bccae fix: fix full text search selector (#7129)
* push

* fix: annotation queue count

* chore: move features

* chore: move features

* chore: move features
2025-06-03 22:04:49 +00:00
steffen911 66aa2fd48e chore: release v3.65.2 2025-06-03 22:02:11 +02:00
Steffen SchmitzandGitHub f822bf1049 chore: add opt-out flag from queue length metric checks (#7123) 2025-06-03 19:21:35 +00:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
96ec24aad8 fix(otel): set trace io from root span (#7127)
* fix(otel): set trace io from root span

* Update web/src/features/otel/server/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-03 16:48:58 +00:00
steffen911 068fbd0cc6 chore: release v3.65.1 2025-06-03 11:33:30 +02:00
Steffen SchmitzandGitHub 7c8b76d573 security: bump tar-fs, rollup version (#7116)
* security: bump tar-fs version

* chore: bump rollup version
2025-06-03 08:13:54 +00:00
Max Deichmann e4e93d64bb chore: release v3.65.0 2025-06-03 10:09:46 +02:00
Max DeichmannandGitHub f5cf8693e9 chore: move core features to OSS (#7091)
* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* chore: move features

* changes to pro plan

* chore: move features
2025-06-03 08:07:45 +00:00
marliessophieandGitHub 5beef28214 style(scores): limit height of grouped scores hovercard (#7106) 2025-06-02 10:59:46 +00:00
marliessophieandGitHub d93712c6ab chore: sort dataset (run) items by created_at and id (#7105)
* `created_at` is not unique, we require a deterministic order of items to ensure data consistency
2025-06-02 10:46:46 +00:00
Hassieb PakzadandGitHub b1b0a966be fix(projects-api-docs): retentionDays to optional (#7103) 2025-06-02 11:49:25 +02:00
b5edadb203 fix: update Google AI Studio model name to preview version (#7086) (#7102)
Co-authored-by: Faris Ibrahim <faris@traba.work>
2025-06-02 11:23:51 +02:00
marliessophieandGitHub 3090882590 fix(default-model): allow set up flow to add LLM API key (#7101) 2025-06-02 07:58:37 +00:00
Max DeichmannandGitHub 787f5d8d10 chore: remove popup from count for historic eval elements (#7098)
* chore: move features

* chore: move features

* chore: move features
2025-05-30 21:25:50 +00:00
Jannik MaierhöferandGitHub 5fa142eda4 chore(ui): changed wording on checkout page (#7045) 2025-05-30 10:05:15 +00:00
steffen911 d96865b019 chore: release v3.64.0 2025-05-30 11:25:55 +02:00
Steffen SchmitzandGitHub 8d808923a1 feat: allow host credentials for blob storage S3 integration (#7072)
* feat: allow host credentials for blob storage S3 integration

* chore: compilation errors
2025-05-30 09:07:52 +00:00
Max DeichmannandGitHub 0f056c1c2d fix: add error message for missing encryption key for self-hosters (#7090)
push
2025-05-30 08:40:50 +00:00
Max DeichmannandGitHub 094a523795 perf: use trace identifier stream to batch delete traces (#7081) 2025-05-29 20:08:27 +02:00
Marc KlingenandGitHub c60e58a568 feat(ui): misc improvements to dashboards including mobile view, big number chart, truncate title/description, enforce min height/width of widgets (#7082)
* truncate title and description to prevent wraps

* remove unnecessary padding from charts

* improve bignumber chart

* add min h/w

* mobile resizing

* fix
2025-05-29 15:58:38 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
6147e9c518 perf: only read trace identifiers for historic eval read stream (#7080)
* push

* Update packages/shared/src/server/services/traces-ui-table-service.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* perf: add exact timestamp match for eval batch actions

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-29 14:35:39 +00:00
Max DeichmannandGitHub f0408b208e perf: add exact timestamp match for eval batch actions (#7077)
* perf: add exact timestamp match for eval batch actions

* perf: add exact timestamp match for eval batch actions

* perf: add exact timestamp match for eval batch actions
2025-05-29 10:54:13 +00:00
Steffen SchmitzandGitHub 55e0bdcac0 feat: allow validation and run now for blob storage integrations (#7064)
* feat: allow validation and run now for blob storage integrations

* chore: error handling and formatting

* lint

* chore: linting
2025-05-28 16:47:23 +00:00
Hassieb PakzadandGitHub 05e15c9f53 fix(otel): update trace name only for root spans (#7061)
* fix(otel): update trace name only for root spans

* push
2025-05-28 15:30:37 +00:00
Max DeichmannandGitHub ce6249b266 perf: data retention concurrency (#7065) 2025-05-28 17:36:20 +02:00
Max DeichmannandGitHub 83e91e5bf5 chore: add logging to retry queue (#7062)
fix: do not wrap traces toolbar
2025-05-28 13:58:46 +00:00
Hassieb PakzadandGitHub edf073cc18 feat(api-models): return model prices on public API (#7046)
* feat(api-models): return model prices on public API

* add tests

* add openapi spec

* return map instead of list
2025-05-28 13:24:20 +00:00
Hassieb Pakzad 65343a8e9b chore: release v3.63.1 2025-05-28 14:17:40 +02:00
Max DeichmannandGitHub 0978055bdc chore: rename dead letter retry queue (#7058)
* push

* push

* push
2025-05-28 11:54:11 +00:00
Max DeichmannandGitHub 76aaf2a3d0 chore: upgrade clickhouse node client (#6806)
* push

* fix: do not wrap traces toolbar

* fix: do not wrap traces toolbar

* fix: do not wrap traces toolbar
2025-05-28 07:57:14 +00:00
Max DeichmannandGitHub 8ed61bf705 perf: increase retry batch action queue (#7049)
push
2025-05-27 19:59:47 +00:00
Steffen SchmitzandGitHub 45436530b4 chore: add analytic views in ClickHouse (#7034)
* chore: add analytic views in ClickHouse

* chore: add additional properties

* chore: add filtering and order changes
2025-05-27 15:59:12 +00:00
Hassieb PakzadandGitHub 7a3f01fc0a fix(playground): resolve prompts when jumping to playground (#7041)
* fix(playground): resolve prompts when jumping to playground

* push
2025-05-27 14:36:40 +00:00
marliessophieandGitHub 468fc4d079 chore(evals): display default model on peek (#7042) 2025-05-27 14:19:11 +00:00
Hassieb PakzadandGitHub 945d13dac0 fix(playground): update tools and schemas if name already exists (#7039)
* fix(playground): update tools and schemas if name already exists

* push
2025-05-27 13:38:59 +00:00
Marc KlingenandGitHub 3646eaa5a8 feat(model): add regex for claude-sonnet-4@20250514 (#7035) 2025-05-27 00:50:10 +00:00
marliessophieandGitHub feb5af11b8 style: default evaluation model management components (#7028) 2025-05-26 16:57:08 +00:00
marliessophieandGitHub 3711af5663 fix: prompt variables of SQL Semantic Equivalence evaluator (#7029) 2025-05-26 14:56:35 +00:00
marliessophieandGitHub 9ec798f393 feat(evaluation): allow filtering by log status (#7018) 2025-05-26 14:43:57 +00:00
Hassieb PakzadandGitHub f71fd43d91 fix(llm-tool-dialog): do not propagate textarea keystrokes up (#7023)
* fix(llm-tool-dialog): do not propagate textarea keystrokes up

* push
2025-05-26 13:51:51 +00:00
marliessophieandGitHub 0303596420 fix(scores): conditionally render metadata in score badges (#7020) 2025-05-26 11:31:50 +00:00
marliessophieandGitHub cfaccc46d3 chore: ensure eval config modal closes after successful delete (#7015)
* chore: ensure eval config modal closes after successful delete

* chore: eslint
2025-05-26 10:21:47 +00:00
Max DeichmannandGitHub 907f764adc fix: do not wrap traces toolbar (#7019)
* fix: do not wrap traces toolbar

* fix: do not wrap traces toolbar
2025-05-26 10:18:23 +00:00
Hassieb PakzadandGitHub 652e9e3fe5 fix(otel): set trace name only if root span or attr provided (#7017) 2025-05-26 09:59:59 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
48673e43d7 feat: allow propagation of request headers through logs (#6979)
* feat: allow propagation of request headers through logs

* chore: propagate under correct name

* chore: take all header values

* chore: allow more properties in context propagation

* chore: update trpc propagation

* Update packages/shared/src/env.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: trace header props for ingestion api

* chore: rename env variable

* chore: remove unused comments

* chore: remove outdated addUserToSpan method

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-26 07:33:02 +00:00
Marlies Mayerhofer a136ac8184 chore: release v3.63.0 2025-05-24 14:05:07 +02:00
marliessophieandGitHub 9ebf62fc9c chore: do not alert on missing API keys or model (#7007)
* chore: change error string for LLM API key not found

* chore: push

* chore: do not alert on missing default model
2025-05-24 07:16:17 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
cf52dce773 chore: fix dataset eval executions for historic evals (#7001)
* push

* fix

* push

* fix

* fix

* fix

* fix

* fix

* fix

* Update worker/src/ee/evaluation/evalService.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix

* fix

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-23 17:44:21 +00:00
marliessophieandGitHub 0a6ecf1cba feat: evaluator library (#6956)
* chore: make `project_id` nullable on `eval_templates`

* feat: add script for managed langfuse evaluators

* feat: restyle running evaluator configs table

* feat: add peek view for running evals

* fixup: add peek view for evaluator library

* fixup: add configure button to table

* feat: rework eval creation flow

* feat: add option to edit eval from table

* fixup: configure evaluator flow with traces data preview

* style: eval configuration UI elements

* chore: add support for current langfuse templates

* chore: reference templateId on jobExecution

* style: add maintainer icon across

* chore: extract WorkerEvaluationVariableService

* fixup: support live data mapping interface

* fixup: revamp datasets eval ui

* fixup: add clone functionality

* chore: rename online evaluation -> llm-as-a-judge

* feat: add evaluator selection in experiment creation flow

* chore: refactor into hooks in experiment form

* feat: add ragas evaluator prompt templates

* chore: show evaluator prompt preview by default

* fix: eval table filtering

* feat: add input box to slider

* fix: clean up running eval peek view and logs

* fix: ensure preview variables are not wrapped in quotes in evaluation prompt preview

* feat: support navigating through traces table

* feat: support default model workflow

* fix: typos in managed evaluators

* fix: do not allow selection of evaluators requiring default to run

* fix: type and build errors

* chore: rename table to default_llm_model

* chore: rm empty file

* fixup: complete rename

* chore: push

* style: eval set up form

* style: separate new evaluator form out into two steps

* chore: ensure we only run evals if default model is set

* chore: add `partner` col to `eval_templates`

* style: new eval template creation

* chore: add proper mapping for dataset evals

* feat: also put evaluation selection on datasets page

* feat: datasets also run on historic experiments after creating mapping

* revert: 09ab242dc12c8647bc716c84d180bc8e631d4919

* chore: fix eslint and ts

* chore: fix warnings

* fix: ensure we inactivate all job configs that rely on default eval model

* fix: tests

* chore: fix eslint

* chore: wrap in tx

* chore: fix default model checkbox logic

* chore: improve wording on delete model

* fix: eslint

* chore: adjust migration order

* style: push ui improvements

* style: sort templates by partner

* fix: eslint
2025-05-23 17:10:47 +00:00
Steffen SchmitzandGitHub c359900b59 perf: skip S3 list calls for OTEL observations (#6995)
* perf: skip S3 list calls for OTEL observations

* chore: reuse prefix definition
2025-05-23 14:04:40 +00:00
Steffen SchmitzandGitHub 720ed7a36f feat: add new /api/public/organizations/projects route to list projects (#6997)
* feat: add new /api/public/organizations/projects route to list projects

* chore: address typos
2025-05-23 13:56:07 +00:00
Marc KlingenandGitHub 1dbefb0994 feat(cloud): push orgs as tenants to plain (#6986)
* fix: plain initialization issues via updatequeue

* apply to the show cb

* chore

* catch errors

* feat(cloud): push orgs as tenants to plain

* add PLAIN_CARDS_API_TOKEN

* fix

* fix

* limit to cloud

* fix
2025-05-23 13:17:45 +00:00
Jannik MaierhöferandGitHub abfdd2366a chore: lw3-5 notification (#6992) 2025-05-23 10:06:02 +00:00
Marc KlingenandGitHub 1f9bb37f3b fix(cloud): plain.com initialization issues via update queue (#6866)
* fix: plain initialization issues via updatequeue

* apply to the show cb

* chore

* catch errors
2025-05-22 23:00:52 +00:00
Marc Klingen 00afab9725 chore: release v3.62.1 2025-05-22 20:24:52 +02:00
Hassieb PakzadandGitHub d04e808515 feat(cost-tracking+llm-keys): add claude 4 sonnet and opus (#6982) 2025-05-22 17:56:45 +00:00
Jannik MaierhöferandGitHub ee04912f0e chore: lw3-4 notification (#6977) 2025-05-22 14:34:55 +00:00
steffen911 7f04a54c0c chore: release v3.62.0 2025-05-22 16:31:39 +02:00
Steffen SchmitzandGitHub cce11ef6b5 perf: drop unused observation_media index (#6978) 2025-05-22 14:27:21 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
1b03f537e2 feat: allow AWS S3 SSE configuration for S3 interactions (#6965)
* feat: allow AWS S3 SSE configuration for S3 interactions

* Update packages/shared/src/env.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: whitelist allowed SSE settings

* dummy

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-22 12:32:12 +00:00
Hassieb PakzadandGitHub dba114e4a3 feat(otel): add support for observations of type 'event' (#6972) 2025-05-22 12:27:14 +00:00
Steffen SchmitzandGitHub fefe3bc3bd fix: dynamically calc max value for home line charts (#6966)
* fix: dynamically calc max value for home line charts

* chore: wrap in useMemo

* dummy
2025-05-22 12:21:18 +00:00
Steffen SchmitzandGitHub 800cc299cb fix: use quorum writes for temporary dataset tables (#6958) 2025-05-21 15:21:58 +00:00
Hassieb PakzadandGitHub 1bdaed4314 fix(ui-jump-to-playground): look up llm api keys when model parsing (#6957)
* fix(ui-jump-to-playground): look up llm api keys when model parsing

* push

* push
2025-05-21 14:18:37 +00:00
Hassieb PakzadandGitHub 71dde233c5 fix(ui-jump-to-playground): allow response format in modelParams (#6955) 2025-05-21 14:13:05 +02:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
3b80afdf2d chore: reduce minimum project retention to 3 days (#6954)
* chore: reduce minimum project retention to 3 days

* Update web/src/features/projects/components/ConfigureRetention.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: update test cases

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-21 12:08:52 +00:00
Steffen SchmitzandGitHub b69f0bda9d chore: bump metrics-api rate limits and deprecate daily metrics (#6950)
* chore: bump metrics-api rate limits

* chore: remove docs for metrics/daily

* chore: remove flaky assertion
2025-05-21 10:26:36 +00:00
Hassieb PakzadandGitHub 9b9a58ae45 fix(prompt-experiments): use resolved prompts (#6940)
* fix(prompt-experiments): use resolved prompts

* fix test
2025-05-21 09:39:37 +00:00
Marc Klingen 775e6a8bfc chore: release v3.61.0 2025-05-21 11:12:29 +02:00
5253014fa6 chore: lw3-3 product hunt notification (#6942)
* chore: lw3-3 product hunt notification

* reverse order

* add created date

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-05-21 09:37:55 +02:00
Marc KlingenandGitHub 1571de133b chore: update pipcat otel attribute mapping (#6944)
based on this change: https://github.com/pipecat-ai/pipecat/pull/1859
2025-05-20 23:50:45 +00:00
Marc KlingenandGitHub 9202f4ec79 docs: add note that docker ENTRYPOINT is covered by semver (#6941) 2025-05-20 23:27:18 +00:00
Steffen SchmitzandGitHub 949e3d1181 chore(dashboards): add more langfuse managed templates (#6939)
* chore(dashboards): add more langfuse managed templates

* chore: update descriptions

* chore: dashboards

* chore: remove wrong dashboard

* chore: commit

* chore: push

* chore: typo

* chore: drop projectId etc from default

* chore: remove beta label

* chore: fix number chart type naming
2025-05-20 17:32:08 +00:00
Jannik MaierhöferGitHubMarc Klingenellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2d280329c5 chore: update product notification - lw3-2 (#6929)
* chore: update product notification: launch week day 2

* update

* Update web/src/components/nav/sidebar-notifications.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* update

* push

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-20 17:24:29 +00:00
Steffen SchmitzandGitHub 6e1886055a feat(dashboards): add langfuse managed dashboards (#6877)
* feat(dashboards): add langfuse managed dashboard templates

* chore: typing

* chore: update dashboards
2025-05-20 14:49:21 +00:00
Marc KlingenandGitHub 39e4a7533a feat(dashboards): add big number chart type (#6933) 2025-05-20 15:46:20 +02:00
Steffen SchmitzandGitHub e8787e3f38 chore(dashboards): expand measures for dashboards (#6932)
* chore(dashboards): expand measures for dashboards

* chore: add testcase

* chore: define inputTokens and outputTokens

* chore: add timePerOutputToken and costs

* chore: fix definition and handling of tag based breakdowns
2025-05-20 13:43:20 +00:00
Max DeichmannandGitHub 59055cd311 chore: add metrics to measure by-id recency (#6930)
push
2025-05-20 14:59:55 +02:00
Marc KlingenandGitHub 7712b63b95 fix(auth): only apply login domain enforcement to multi-tenant sso (#6928) 2025-05-20 10:33:35 +02:00
marliessophieandGitHub f25d466388 fix(tables-ui): truncate name columns in traces and observations table (#6925) 2025-05-20 07:55:46 +00:00
Max DeichmannandGitHub b9e18f59d8 chore: upgrade zod (#6921)
push
2025-05-19 20:59:03 +00:00
Steffen SchmitzandGitHub 09f3343db2 fix(dashboards): update resize migration to handle empty widget list correctly (#6918)
fix(dashboards): update migration to handle empty widget list correctly
2025-05-19 15:15:32 +00:00
Steffen SchmitzandGitHub 5ab8e4219f feat(dashboard): make dashboard widgets draggable (#6891)
* feat(dashboard): make dashboard widgets draggable

* chore: do not update on layout change

* chore: fix size

* chore: remove the preventCollision flag

* chore: linting

* chore: get row height dynamically

* chore: make chart resizable

* chore: styling for resizes

* chore: add migration to align y values
2025-05-19 14:48:39 +00:00
Steffen SchmitzandGitHub 40359f0e3f chore(dashboards): order metrics and dimensions in widgetform (#6917)
* chore(dashboards): make metrics and dimensions in widgetform searchable

* chore: only sort without search
2025-05-19 14:18:48 +00:00
Marc KlingenandGitHub 817b6d37bc feat(ui): implement optional sidebar notification expiry (#6916)
* feat(web): add expiry for sidebar notifications

* Update web/src/components/nav/sidebar-notifications.tsx

* Update web/src/components/nav/sidebar-notifications.tsx
2025-05-19 13:57:04 +00:00
Hassieb Pakzad 8bb73822d3 chore: release v3.60.1 2025-05-19 14:50:33 +02:00
Jannik MaierhöferGitHubMarc Klingenellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
fa53ba3c1a chore: add lw3-1 notification (#6912)
* chore: add lw3-1 notification

* Update web/src/components/nav/sidebar-notifications.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-19 09:56:02 +00:00
Hassieb PakzadandGitHub 086a764083 chore(media): make ID generation fully deterministic based on hash (#6900) 2025-05-19 11:52:55 +02:00
Steffen SchmitzandGitHub 79a9f3c3e3 perf: add mediaId index to efficiently delete media items (#6909)
* perf: add mediaId index to efficiently delete media items

* chore: only add index if they do not exist yet
2025-05-19 09:30:16 +00:00
Max Deichmann abd46f2d4d chore: release v3.60.0 2025-05-19 11:16:27 +02:00
marliessophieandGitHub dfd9565a3f chore(experiments): improve delete experiment confirm message (#6908) 2025-05-19 07:41:27 +00:00
Marc KlingenandGitHub ebb8a5a13f chore(ui): trim password reset otp (#6905) 2025-05-18 20:01:26 +00:00
Marc KlingenandGitHub 44104596f6 fix(auth): add domain check for multi-tenant sso providers (#6904) 2025-05-18 19:54:40 +00:00
Marc KlingenandGitHub 17f6260671 feat: otel span mapping for pipecat traces (#6903) 2025-05-18 17:41:49 +00:00
Marc KlingenandGitHub 0a2bfe5ab5 fix(ui): peek view header should stay single line on mobile screens (#6902) 2025-05-18 17:23:32 +00:00
Marc KlingenandGitHub 8bd1a4da3e chore(ui): hide thread refs in support chat (#6901)
feat(web): hide thread refs in support chat
2025-05-18 13:28:49 +00:00
Marc KlingenGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
31b230d3e0 feat(auth): switch to OTP based password reset instead of magic link (#6898)
* feat(auth): add otp-based password reset

* fix

* fix lint and margins

* Update web/src/features/auth-credentials/components/ResetPasswordButton.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update web/src/server/auth.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* simplify

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-18 13:04:40 +00:00
Marc KlingenandGitHub dbe71249e7 chore: ignore HEAD request for auth api endpoints (#6899) 2025-05-18 12:04:20 +00:00
Marc KlingenandGitHub a9d0486edd chore: Update AGENTS guide (#6897)
docs: mention conventional commits
2025-05-18 13:45:29 +02:00
Marc KlingenandGitHub 2bf506a436 fix(ui): hover state color of chat message types (#6895) 2025-05-18 10:38:17 +00:00
Steffen SchmitzandGitHub edca91c12a chore(dashboard): auto-generate widget name from selected props (#6890)
* chore(dashboard): auto-generate widget name from selected props

* chore: lint
2025-05-17 13:52:52 +00:00
Steffen SchmitzandGitHub a11e7ddfee feat(dashboard): include dashboards in cmdk menu (#6889) 2025-05-17 12:28:26 +00:00
Steffen SchmitzandGitHub cb9eb23f36 chore(dashboard): show description on hover in selection tables (#6888) 2025-05-17 12:25:43 +00:00
Steffen SchmitzandGitHub 5e942f0208 feat(dashboards): add langfuse managed read-only dashboard support (#6873)
* chore: add owner to project and return langfuse dashboards

* chore: show owner for dashborads in table

* chore: snapshot

* chore: snapshot

* chore: allow widget clone

* chore: snapshot

* chore: linting

* chore: use project instead of user

* chore: show error message on clone failure

* chore: indicate existing dashboard widget relationship

* chore: show only project dashboards in selection for adding widgets

* chore: make full dashboard row clickable

* chore: add clone button to langfuse dashboards
2025-05-17 12:02:50 +00:00
Max DeichmannandGitHub 8e0e139848 chore: remove clickhouse session id (#6849)
* push

* push
2025-05-16 15:09:08 +00:00
Hassieb Pakzad acc1227962 chore: release v3.59.1 2025-05-16 16:57:30 +02:00
Hassieb PakzadandGitHub d9682511c1 perf(otel): do not pass deep parsed metadata (#6878)
* revert previous commit

* perf(otel): do not pass deep parsed metadata
2025-05-16 14:55:16 +00:00
Hassieb Pakzad 00e0b51bc4 chore: release v3.59.0 2025-05-16 14:06:50 +02:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
e63cf84e4f feat(otel): add Langfuse attribute parsing (#6692)
* feat(otel): add Langfuse attribute parsing

* fix

* push

* add safeguard for cross project spans

* fix test

* add extract completionStartTime

* fix type issues

* handle trace io

* fix

* Update web/src/features/otel/server/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-16 12:03:01 +00:00
Steffen SchmitzandGitHub 63d1051ac2 perf: increase socket lifetime and connections for clickhouse client (#6870) 2025-05-16 08:54:38 +00:00
Steffen SchmitzandGitHub 0e056eb072 chore: increase default timeframe in custom dashboards to 7d (#6871) 2025-05-16 05:45:37 +00:00
Steffen SchmitzandGitHub 87f7095966 chore: reduce title size in widget item doc hover (#6869) 2025-05-16 05:26:57 +00:00
steffen911 5eefe997ae chore: release v3.58.0 2025-05-16 06:39:23 +02:00
Marc KlingenandGitHub e251e60b53 fix: plain chat availability check should not rely on window.Plain but only on env (#6867) 2025-05-16 05:36:07 +02:00
Marc KlingenandGitHub adc05eeef4 feat(ui-ee): show/hide product modules in main menu via LANGFUSE_UI_VISIBLE/HIDDEN_PRODUCT_MODULES (#6841)
* feat(ui): show/hide product modules in main menu

* prod example

* add dashboards

* improve comment
2025-05-16 00:20:17 +00:00
Steffen SchmitzandGitHub 702143fbe2 perf: skip zod validation for otel event batches (#6861)
* perf: skip zod validation for otel event batches

* chore: remove unnecessary casting
2025-05-15 16:47:30 +00:00
Steffen SchmitzandGitHub 06643f30e5 feat(dashboard): add documentation hovers onto widget properties (#6856)
* feat(dashboard): add documentation hovers onto widget properties

* chore: fix typing

* chore: remove open and close delays of hovercards
2025-05-15 14:31:20 +00:00
Steffen SchmitzandGitHub 2e496d5429 chore(dashboard): make hovers in dark mode more readable (#6858) 2025-05-15 13:54:44 +00:00
Steffen SchmitzandGitHub 5b62df9122 feat(dashboard): add new widget to dashboard on save (#6854)
* feat(dashboard): add new widget to dashboard on save

* chore: linting
2025-05-15 13:15:08 +00:00
Steffen SchmitzandGitHub fd189dc66b fix(dashboard): map traceRelease and traceVersion correctly on breakdown (#6853) 2025-05-15 11:39:51 +00:00
Steffen SchmitzandGitHub 10a3644dcf chore(dashboard): auto-save dashboard on any changes (#6820)
* chore(dashboard): auto-save dashboard on any changes

* chore: linting

* chore: switch timeouts to useDebounce

* chore: linting
2025-05-15 09:48:02 +00:00
Max DeichmannandGitHub 8ca000b225 chore: reduce worker logs (#6851) 2025-05-15 11:33:33 +02:00
Steffen SchmitzandGitHub 47d41b6a42 chore: change logging pattern around redis retries (#6843) 2025-05-15 07:34:00 +00:00
Steffen SchmitzandGitHub 1d66408c18 fix(dashboard): avoid invalid filter name on view switches (#6829)
* fix(dashboard): avoid invalid filter name on view switches

* chore: add test case
2025-05-15 07:18:41 +00:00
Max DeichmannandGitHub 44d1727dd0 chore: reduce error logs for llm 429 errors (#6836) 2025-05-15 00:37:58 +02:00
Max DeichmannandGitHub bfb2878719 chore: reduce log level for prompt service (#6835)
fix
2025-05-14 22:19:00 +00:00
Marc KlingenandGitHub 67e6b4ce87 chore(cloud): increases api rate limits (#6783) 2025-05-14 23:03:49 +02:00
Max DeichmannandGitHub dcc5bf9c6e fix: fix traceid filter for observations by id clickhouse query (#6828) 2025-05-14 16:10:23 +02:00
Steffen SchmitzandGitHub db357101b7 feat(dashboards): add support for cloning existing dashboards (#6824)
* feat(dashboards): add support for cloning existing dashboards

* chore: move buttons into a dedicated dropdown menu

* chore: allow updates to dashboard name and description

* chore: lint

* chore: removal double capturing of dashboard delete

* chore: track deletions
2025-05-14 12:50:26 +00:00
Steffen SchmitzandGitHub afbd45b9c3 chore(metrics-api): return bad request errors for invalid query (#6826) 2025-05-14 11:01:10 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
b6b37212ce perf: reduce project deletion concurrency (#6823)
* push

* Update worker/src/app.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-14 10:30:46 +00:00
Steffen SchmitzandGitHub 76a090241e feat(dashboard): add newly created widget to dashboard on save (#6819)
* feat(dashboard): add newly created widget to dashboard on save

* chore: linting
2025-05-14 09:25:23 +00:00
Max DeichmannandGitHub c53abf6aeb perf: also add trace_id to observations search (#6822)
push
2025-05-14 09:16:49 +00:00
Hassieb Pakzad 5cbfd922f9 chore: release v3.57.2 2025-05-14 09:55:07 +02:00
Hassieb PakzadandGitHub 0be269d519 fix(ui-tool-calls): handle parsing errors gracefully (#6817) 2025-05-14 07:52:47 +00:00
Max DeichmannandGitHub d0eeb44b90 fix: fail gracefully for batch actions in case eval config does not exist (#6809)
* push

* fix
2025-05-13 15:03:57 +00:00
Marlies Mayerhofer 30ced3ab42 chore: release v3.57.1 2025-05-13 16:52:57 +02:00
Max DeichmannandGitHub 2ed6b5a40f chore: parameterize clickhouse deletion timeouts (#6801) 2025-05-13 16:20:13 +02:00
Steffen SchmitzandGitHub 6eaef75f20 fix(dashboard): render localized time strings for LineCharts (#6799)
* fix(dashboard): render localized time strings for LineCharts

* chore: use memo
2025-05-13 13:57:07 +00:00
Max DeichmannandGitHub 31c36565c3 chore: fix clickhouse timeout settings (#6798) 2025-05-13 15:30:04 +02:00
Steffen SchmitzandGitHub d1a1ca7744 fix(dashboard): increase precision of numbers on hover cards (#6797) 2025-05-13 12:54:52 +00:00
Steffen SchmitzandGitHub 2dac75d203 chore(dashboard): use more granular time buckets on home dashboard (#6795) 2025-05-13 12:25:39 +00:00
Steffen SchmitzandGitHub fab90b36a6 chore: remove queryPlayground page (#6796) 2025-05-13 12:20:12 +00:00
Steffen SchmitzandGitHub 597bd6c96b fix(dashboard): warn on slow chart render and ask users to confirm (#6794) 2025-05-13 12:03:02 +00:00
Max DeichmannandGitHub ba3bcc9295 fix: Increase memory usage for deploy (#6793)
Increase memory usage for deploy
2025-05-13 10:32:29 +00:00
marliessophieandGitHub 5f65929a5e fix(session-ui): handling mine type as enum string equivalent (#6792) 2025-05-13 09:40:01 +00:00
Steffen SchmitzandGitHub f1344a4c1a chore: remove unused env variable (#6789) 2025-05-13 08:44:02 +00:00
Steffen SchmitzandGitHub 4e3b9a82f7 fix: do not refresh blobstorage form on browser navigation (#6788) 2025-05-13 08:12:28 +00:00
Max DeichmannandGitHub 7b803619fd fix: fix metadata json selector (#6784) 2025-05-12 21:11:21 +02:00
marliessophieandGitHub 1ef51b1644 feat(datasets): support GET dataset-run-items (#6780)
* feat(datasets): support `GET` dataset run items

* chore: add fern api types

* chore: eslint
2025-05-12 17:55:26 +00:00
Hassieb PakzadandGitHub fee9e34b7e chore(ui-graph-view): allow graph view in impersonation mode (#6782) 2025-05-12 17:37:32 +00:00
Hassieb PakzadandGitHub 1354db489f fix(ui-graph-view): fetch graph data separately (#6781)
* fix(ui-graph-view): fetch graph data separately

* fix

* push

* refactor

* fix

* fix lint
2025-05-12 16:33:33 +00:00
steffen911 740a25150c chore: release v3.57.0 2025-05-12 17:49:49 +02:00
Marc KlingenandGitHub 47c2d65d12 chore(ui): improve ux of full-text search input (#6772)
* width

* change to dropdown

* fix
2025-05-12 15:08:12 +00:00
Steffen SchmitzandGitHub d94e7c1c06 feat: add new metrics api endpoint (#6778)
* feat: add metric api endpoint

* feat: add implementation and documentation

* chore: add defaults for timeDimension and ordering
2025-05-12 14:17:40 +00:00
Steffen SchmitzandGitHub 38ed98a76d feat: add org API keys to organization settings (#6729)
* feat: add org API keys to organization settings

* chore: linting suggestions

* chore: fix typing

* feedback: do not call hooks conditionally
2025-05-12 11:25:01 +00:00
Max DeichmannandGitHub 92567b765b chore: reduce amount of ingested dd spans (#6775)
* push

* fix
2025-05-12 09:47:37 +00:00
Marc KlingenandGitHub b37396893a chore: improve auth error message (#6773) 2025-05-12 04:02:11 +00:00
Steffen SchmitzandGitHub 0fd1eebbff fix: align signed download URL with hostname (#6764) 2025-05-11 12:24:49 +00:00
Max DeichmannandGitHub f478bb5763 refactor: refactor observations by id repo function (#6768)
fix
2025-05-10 20:12:13 +00:00
Max Deichmann 1de48b42ba chore: release v3.56.0 2025-05-09 19:14:08 +02:00
Max DeichmannandGitHub 4c78450924 chore: do not fetch metadata for trace detail initial load (#6763)
* fix

* fix

* fix
2025-05-09 14:59:22 +00:00
Hassieb PakzadandGitHub e360c13763 fix(ui-graph-view): fix metadata parsing in isLanggraphTrace (#6762) 2025-05-09 12:25:44 +00:00
Max DeichmannandGitHub a03a451088 fix: full text search (#6759) 2025-05-09 10:45:33 +02:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
e3fa322ecf feat: full text search (#6343)
* feat: full text search

* push

* feat: full text search

* feat: full text search

* feat: full text search

* feat: full text search

* feat: full text search

* fix

* push

* fix: fix json limits and scope to single bodies only

* fix

* fix

* jumping UI

* jumping UI

* jumping UI

* push

* Update web/src/components/table/data-table-toolbar.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-09 08:41:27 +00:00
2b20c9bdc2 fix(ui) langgraph agent graph (#6753)
* fix(ui) langgraph agent graph

* push

* push

* push

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2025-05-09 06:50:26 +00:00
Max DeichmannandGitHub 860d05a3f4 chore: add sentry release tag (#6743) 2025-05-08 13:30:50 +02:00
Marlies Mayerhofer 398174d3a4 chore: release v3.55.0 2025-05-08 12:55:54 +02:00
Max DeichmannandGitHub 040ff26853 chore: allow js-profiling in browser via header (#6740)
push
2025-05-08 09:38:37 +00:00
marliessophieandGitHub 78924f42e2 chore(run-scores): show only populated run-level scores (#6739)
* chore(run-scores): show only populated run-level scores

* chore: early return
2025-05-08 09:10:02 +00:00
Jannik MaierhöferandGitHub 515186e9a2 feat: add otel attribute parsing for google adk (#6736) 2025-05-08 04:20:46 +00:00
Jannik MaierhöferandGitHub af863bcbd9 fix(otel): avoid logging model name as model param (#6737) 2025-05-08 04:19:38 +00:00
marliessophieandGitHub 4e3f97d56e chore(table-view-presets): conditionally apply generations filter; add avatar fallback; set beta badge (#6734)
* chore(table-view-presets): conditionally apply generations filter

* chore: add fallback avatar; set beta badge
2025-05-07 22:20:16 +00:00
marliessophieandGitHub 159f7560a4 feat(scores): support run level scores (#6676)
* feat(scores): support run level scores

* style: show run level scores on runs table

* chore: fix type

* chore: tests

* chore: rename runId -> datasetRunId

* chore: generate fern api docs

* style: improve UI/UX naming

* style: improve UI
2025-05-07 17:56:43 +00:00
marliessophieandGitHub fa77bc9daf chore(table-views): handle nullish avatar (#6730) 2025-05-07 19:21:32 +02:00
marliessophieandGitHub 23e55cd90a feat(tables): support saving and sharing table views (#6595)
* style(traces-table): re-order columns

* style(observations-table): re-order columns

* feat: add peek view to observations view

* style: revamp column visibility filter

* chore(saved-views): add data model

* chore(saved-views): add CRUD operations

* fixup: ui for saved filter views

* fixup: add create table views functionality

* chore: support update and delete operations in UI

* feat: generate permalink

* feat: allow permalink to resolve

* chore: save view in session storage too

* style: show image of author

* chore: remove duplicated columns

* chore: push

* chore: refactor

* chore: rename

* chore: move types

* feat: add support on session, scores, observations tables

* chore: ensure invalid ordering and filtering conditions are ignored

* chore: add posthog events

* tests: add validation unit tests

* chore: remove testing readme

* chore: fix test

* chore: remove async

* chore: rename saved table views -> table view presets

* chore: reformulate

* chore: adjust

* feat: support table views on datasets

* chore: ui for viewers
2025-05-07 15:42:49 +00:00
marliessophieandGitHub 5e39ac49f0 feat(scores): add link to session on session scores table (#6728) 2025-05-07 11:50:46 +00:00
marliessophieandGitHub 75b4c6da1e style(observations): correctly display costs as sums (#6725) 2025-05-07 09:16:50 +00:00
Steffen SchmitzandGitHub 03a8eeaf08 feat: allow metadata filtering in dashboard builder (#6724)
* feat: allow metadata filtering in dashboard builder

* chore: update query type for metadata
2025-05-07 09:02:36 +00:00
marliessophieandGitHub 0da9738243 fix: persist changes in evals template edit on tab change (#6716) 2025-05-07 08:02:26 +00:00
Marc KlingenandGitHub 8d2d242375 chore(ui): add border to trace timeline bars to increase contrast (#6723)
* chore(ui): add border to trace timeline bars to increase contrast

* fix
2025-05-07 03:36:43 +00:00
Marc KlingenandGitHub 7b5231db88 chore: show full user email on hover (#6722)
chore(ui): show full user email address on hover
2025-05-07 03:23:14 +00:00
Marc KlingenandGitHub 2925bc77fc chore(ui): increase contrast ratio of annotation selection (#6721)
chore(ui): The selection in `Annotate` window should have higher-contrast color

Fixes langfuse/langfuse#6720
2025-05-07 03:14:00 +00:00
Max DeichmannandGitHub cfc8d68a57 fix: increase io limit (#6714)
* push

* fix: fix json limits and scope to single bodies only

* push

* fix: fix json limits and scope to single bodies only

* fix: fix json limits and scope to single bodies only

* fix: fix json limits and scope to single bodies only

* fix: fix json limits and scope to single bodies only

* fix: fix json limits and scope to single bodies only
2025-05-06 21:34:03 +00:00
Steffen SchmitzandGitHub aa38a2cbb8 feat: add totalTokens and totalCost to dashboard builder (#6715) 2025-05-06 19:20:07 +00:00
marliessophieandGitHub 78958ac5f9 fix(score-analytics): css on categorical charts (#6710) 2025-05-06 16:04:03 +00:00
Marlies Mayerhofer 98506db2b0 chore: release v3.54.1 2025-05-06 16:54:54 +02:00
Steffen SchmitzandGitHub 1ecf3a2a60 fix: allow protected keys in metadata object (#6701) 2025-05-06 11:37:14 +00:00
marliessophieandGitHub bca360b738 fix(security): remove hyperlinks from organization invitation emails (#6703) 2025-05-06 09:43:16 +00:00
Steffen SchmitzandGitHub 8c44e03cc3 chore: remove non-production note on worker in CONTRIBUTING.md (#6626) 2025-05-06 06:36:47 +00:00
marliessophieandGitHub 5487dc36c9 fix: navigation issue with organization/project members pages (#6695) 2025-05-05 18:51:55 +00:00
marliessophieandGitHub 4e2fa5d794 chore(peek): memoize table body for enhanced front-end performance (#6690)
* chore: memoize table body for enhanced front end performance

* chore: push

* chore: refactor

* style: speed up animation

* chore: re-render on table state changes

* chore: import function directly

* chore: refactor to use timestamp

* chore: push

* chore: push
2025-05-05 18:04:56 +00:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
8b013c6712 fix(trace-ui): usage breakdown on hover (#6694)
* fix(trace-ui): usage breakdown on hover

* Update web/src/components/table/use-cases/traces.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-05 17:36:03 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
76fae866ea fix: improve axis labeling in widgets (#6691)
* fix: improve axis labeling in widgets

* Update web/src/features/widgets/chart-library/utils.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-05 15:16:42 +00:00
Max DeichmannandGitHub 0e0bf5cfbc chore: improve front end performance (#6683)
* push

* push
2025-05-02 22:30:52 +00:00
Max Deichmann 81872c1fdc chore: release v3.54.0 2025-05-01 17:24:48 +02:00
Max DeichmannandGitHub c06af525bf chore: disable dlx retry for oss (#6665) 2025-05-01 16:33:59 +02:00
Max DeichmannandGitHub cd30f0c89c chore: add eval execution metric (#6664)
push
2025-05-01 16:32:18 +02:00
marliessophieandGitHub 5bc4f1cbac chore: add copy ID button to trace/observation preview (#6651) 2025-04-30 09:39:42 +00:00
marliessophieandGitHub d0f7663ea2 feat(datasets): search by name on table (#6645)
* feat: add search by name to datasets

* chore: apply search query on count
2025-04-29 14:24:58 +00:00
Max DeichmannandGitHub 7496e2aeb0 chore: increase trace id delete observability (#6635) 2025-04-28 22:49:24 +02:00
marliessophieandGitHub d39b0214b8 chore(markdown): introduce type guards for markdown rendering (#6631) 2025-04-28 14:50:15 +00:00
Max Deichmann 9957c51d66 chore: release v3.53.0 2025-04-28 12:19:51 +02:00
0f06a066df feat(scores): support session level scores (#6367)
* chore: make changes to `scores` data structure

* fixup: migrations

* chore: update CH score types and schemas

* feat: support session level scores

* chore: add mock session level scores

* feat(session-scores): create v2 APIs; extract score and trace api service

* chore: clean up todos

* eslint

* eslint

* chore: reorder migrations after rebase

* chore: order unclustered migrations after rebase

* chore: fixes after rebase

* chore

* revert changes to .json file

* chore

* chore: move session_id column

* typo

* chore: migration adjustment

* chore: styling adjustments

* chore: materialize index

* chore: fix types after rebase

* chore: fix types after rebase

* chore: feedback

* chore: refactor scores types file structure

* chore: fix lint

* chore: add api types

* chore: rename

* fix: wait for clickhouse mutations to succeed (#6629)

* fix: wait for clickhouse mutations to succeed

* push

* fix: wait for clickhouse mutations to succeed

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-04-28 09:51:22 +00:00
marliessophieandGitHub ff8ccf1d61 chore(ui): add support for input_text and output_text in markdown rendering (#6625) 2025-04-28 08:39:15 +00:00
Max DeichmannandGitHub eeff4a4f68 feat: add dlx retry queue and service (#6621)
* feat: add dlx retry queue and service

* feat: add dlx retry queue and service

* feat: add dlx retry queue and service

* feat: add dlx retry queue and service

* feat: add dlx retry queue and service
2025-04-27 10:15:24 +00:00
Hassieb PakzadandGitHub 876d43e317 perf(api-traces): increase observations payload limit to 10mb (#6615)
* perf(api-traces): increase observations payload limit to 10mb

* add test
2025-04-25 18:47:05 +02:00
Hassieb PakzadandGitHub 3da7f3c8b2 perf(trace-api): increase observations payload limit to 6mb (#6614) 2025-04-25 15:11:48 +00:00
Hassieb PakzadandGitHub 6e5d0cee7b fix(api-prompts): use middleware to send response when updating labels (#6611)
* fix(api-prompts): use middleware to send response when updating labels

* fix lint error
2025-04-25 14:57:45 +00:00
Steffen SchmitzandGitHub 9f1fafd1bb build: prepare deployment for HIPAA compatibility (#6613) 2025-04-25 14:26:04 +00:00
Steffen SchmitzandGitHub a2cbcbc585 Revert "perf: race byId queries for traces and observations (#6555)" (#6612)
* Revert "perf: race byId queries for traces and observations (#6555)"

This reverts commit 591fb98a

* chore: compare on toDate starttime
2025-04-25 13:29:25 +00:00
Steffen SchmitzandGitHub e7aa0fbe08 feat: add support for score name filters on dashboard (#6608) 2025-04-25 09:52:23 +00:00
Hassieb PakzadandGitHub 30e608244a test(trace-api): fix observations too large test (#6607) 2025-04-25 11:32:59 +02:00
Hassieb PakzadandGitHub 3ec5476c04 perf(api-traces): increase byId size limit to 5MB (#6604) 2025-04-25 10:11:01 +02:00
Max DeichmannandGitHub 8c46c79e4f perf: reduce optimistic lookback for byid queries (#6593)
* perf: reduce optimistic lookback for byid queries

* perf: reduce optimistic lookback for byid queries
2025-04-24 15:58:13 +00:00
Hassieb PakzadandGitHub 231ee5873c perf(public-api-traces): raise early error if observations are too large (#6592) 2025-04-24 14:25:01 +00:00
Steffen SchmitzandGitHub 7a74f7da9d perf: refactor clickhouseClient creation to singletons (#6584)
* perf: refactor clickhouseClient creation to singletons

* chore: align remaining client usages

* chore: update test usage
2025-04-24 14:03:53 +00:00
Steffen SchmitzandGitHub 82bf0baab7 perf: return only on API if no trace found (#6587) 2025-04-24 14:29:26 +02:00
Steffen SchmitzandGitHub 591fb98afb perf: race byId queries for traces and observations (#6555)
* perf: race byId queries for traces and observations

* chore: update tests
2025-04-24 10:02:04 +00:00
marliessophieandGitHub e919893c2b fix(ui): add deprecated col mapping on traces table for cached filter states (#6583)
* fix(ui): add deprecated col mapping on traces table for cached filter states

* chore: improve comments

* chore: fix

* chore: push
2025-04-24 07:01:54 +00:00
Max DeichmannandGitHub 237076bf92 chore: improve user identification in observability (#6575) 2025-04-23 20:11:27 +00:00
Steffen SchmitzandGitHub c7b09e7843 perf: include timestamp on trace retrieval on sessions and datasetrunitemstable (#6549) 2025-04-23 16:03:12 +00:00
marliessophieandGitHub 277e7b7501 feat(scores): support categorical filters across tables (#6484)
* fixup(scores): add categorical scores filter to traces table

* chore: support categorical scores filter on traces table

* chore: throw error if categories options filter is used in postgres

* chore: eslint

* feat: support categorical scores filter on observation scores

* chore: adjust typing

* style: filter builder

* chore: rename function
2025-04-23 07:30:43 +00:00
a01d12c064 fix: use source entity environment for annotation scores (#6545)
* fix: use source entity environment for annotation scores

* chore: resolve typo

* perf: remove redundant trace fetch for traces.byIdWithObservationsAndScores

* Revert "perf: remove redundant trace fetch for traces.byIdWithObservationsAndScores"

This reverts commit 544d241f0ddeccc9e1b7c1cd8f33a63d7253ae6c.

* fix: include `environment` in `handleCommentUpdate` call

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-04-22 16:14:36 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>Steffen Schmitz
e1ae79e0d1 perf: soft delete event logs clickhouse (#6533)
* chore: mograte event_log table to new blob storage table

* push

* push

* push

* fix

* fix

* Update packages/shared/clickhouse/migrations/clustered/00011_add_blob_storage_file_log.up.sql

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker/src/backgroundMigrations/migrateEventLogToBlobStorageRefTable.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* push

* push

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* push

* fix

* fix

* fix

* push

* fix

* fix

* fix

* fix

* fix

* fix

* push

* push

* Update packages/shared/src/server/data-deletion/ingestionFileDeletion.ts

Co-authored-by: Steffen Schmitz <steffen@langfuse.com>

* push

* fix

* fix

* fix

* push

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
2025-04-22 13:43:10 +00:00
marliessophieandGitHub 9804739ff1 chore(observations): show model name in table column (#6552) 2025-04-22 12:39:17 +00:00
Steffen SchmitzandGitHub 2d1f104a1c perf: remove redundant trace fetch for traces.byIdWithObservationsAndScores (#6548)
* perf: remove redundant trace fetch for traces.byIdWithObservationsAndScores

* chore: include fromTimestamp in enforceTraceAccess util
2025-04-22 11:38:31 +00:00
Max DeichmannandGitHub 2a871d1e75 chore: migrate event_log table to new blob storage table (#6534) 2025-04-22 12:24:55 +02:00
marliessophieandGitHub 4c749298e6 chore(prompts): throw 400 if prompt name type mismatched (#6547)
chore(prompts): throw 400 if prompt name type mismatched
2025-04-22 10:17:52 +00:00
marliessophieandGitHub b207712bcd chore(model-costs): support alias for all models (#6542)
* chore(model-costs): support alias for all models

* chore: remove from playground
2025-04-22 09:29:46 +00:00
Hassieb PakzadandGitHub 472b053c44 fix(llm-keys): drop top_p from test call (#6544) 2025-04-22 09:22:26 +00:00
steffen911 5b77e5835b chore: release v3.52.0 2025-04-22 10:00:35 +02:00
Steffen SchmitzandGitHub 657bb63c3e chore: make custom dashboards available for self-hosters (#6541) 2025-04-22 07:33:56 +00:00
Steffen SchmitzandGitHub b101e7c558 feat: allow optional metadata param on org / project CRUD API (#6524)
* feat: allow optional metadata param on org / project CRUD API

* chore: drop debugger statements

* chore: remove more debuggers

* chore: update tests

* chore: add metadata to fern spec
2025-04-22 07:30:02 +00:00
Steffen SchmitzandGitHub bac6b13145 fix: handle data retention on clustered clickhouse deployments (#6538) 2025-04-22 07:27:09 +00:00
Steffen SchmitzandGitHub 6817cdab0a chore: confirm pagination behaviour for scores-api (#6537)
* chore: confirm pagination behaviour for scores-api

* chore: use correct logger for score errors

* chore: inline error
2025-04-22 07:22:31 +00:00
Steffen SchmitzandGitHub 2d4708921c feat(dashboard): allow filtering by observation name (#6511) 2025-04-18 10:12:44 +00:00
marliessophieandGitHub 756b2f7501 chore(ui-forms): ensure name uniqueness enforced on form-level also (#6503)
* chore(ui-forms): ensure name uniqueness enforced on form-level also

* chore: nits
2025-04-17 14:13:01 +00:00
marliessophieandGitHub 2a685edbfa fix(tables): rename usage column to tokens column (#6498) 2025-04-17 11:58:12 +00:00
Steffen SchmitzandGitHub c1c22a9b9b fix: parse negative integer signs on otel timestamps (#6497) 2025-04-17 11:23:55 +00:00
Marlies Mayerhofer ecd3afb4c9 chore: release v3.51.2 2025-04-17 11:40:15 +02:00
marliessophieandGitHub 362902a69c fix(llm-keys): ensure default openai model supports all model params (#6496) 2025-04-17 11:38:05 +02:00
Marlies Mayerhofer ab14bac8df chore: release v3.51.1 2025-04-17 09:11:58 +02:00
Steffen SchmitzandGitHub 3e92e6b74d chore: add query tags to health check queries (#6493)
* chore: add query tags to health check queries

* chore: update type

* chore: switch to domain_ts instead of created_at
2025-04-17 05:34:59 +00:00
Steffen SchmitzandGitHub 54f3c224a7 chore: configure global posthog processing rate limit (#6492) 2025-04-17 04:44:34 +00:00
marliessophieandGitHub e39dda87f9 chore(cost-tracking): add support for openai o3 and o4-mini models (#6491) 2025-04-16 21:43:38 +00:00
Max DeichmannandGitHub 0b8ad96b18 fix: do not report usage to stripe for projects which have been deleted (#6468)
push
2025-04-16 13:14:58 +00:00
marliessophieandGitHub a4ba66f329 perf(datasets): split metrics and core data into two endpoints (#6481)
* perf(datasets): split metrics and core data into two endpoints

* chore: nit
2025-04-16 10:21:36 +00:00
Marc KlingenandGitHub 9c473be745 chore: Update CONTRIBUTING.md (#6476) 2025-04-16 01:46:33 +00:00
Marc KlingenandGitHub 4da0b9874a fix: preserve new lines when displaying score comments on single trace view (#6475) 2025-04-16 00:24:43 +00:00
Marc KlingenandGitHub 8fd6951a19 fix: window not available at build time (#6472) 2025-04-15 21:11:25 +02:00
Marc KlingenandGitHub b30734f43c fix: plain not available on some page loads (#6471) 2025-04-15 20:53:32 +02:00
Max DeichmannandGitHub 293215d1d4 chore: more error logging improvements for project deletions (#6467)
* fix

* push
2025-04-15 16:41:54 +00:00
Max DeichmannandGitHub ead31c8ca4 chore: improve logging for project deletion queue (#6465)
fix
2025-04-15 16:26:40 +00:00
marliessophieandGitHub 870336a5ed fix(prompts): prompt label popover overflows screen bounds (#6463) 2025-04-15 14:21:29 +00:00
steffen911 40792d6587 chore: release v3.51.0 2025-04-15 15:58:59 +02:00
Steffen SchmitzandGitHub 820de6f30f chore: refactor admin-api handling into ee folder (#6462)
* chore: refactor admin-api handling into ee folder

* chore: lint

* chore: add a license key for testing
2025-04-15 13:41:18 +00:00
marliessophieandGitHub 39fb6d99d4 feat(tables): reorder traces and observations column order; re-style visibility selector; peek on observations table (#6442)
* style(traces-table): re-order columns

* style(observations-table): re-order columns

* feat: add peek view to observations view

* style: revamp column visibility filter

* chore: typo

* chore: css
2025-04-15 13:27:55 +00:00
marliessophieandGitHub c156a961a2 chore(cost-tracking): add support for openai 4.1 models (#6459)
* chore(cost-tracking): add support for openai 4.1 models

* chore: add tokenizer config

* chore: add gpt-4.1 prefix
2025-04-15 11:45:57 +00:00
steffen911 8907056b39 chore: release v3.50.0 2025-04-15 10:42:48 +02:00
Steffen SchmitzandGitHub 7ad6749058 feat: expose project retention via the projects API (#6456)
* feat: expose project retention via the projects API

* chore: adjust docs

* chore: remove unnecessary assertion
2025-04-15 07:55:44 +00:00
Steffen SchmitzandGitHub 2e40330df8 feat: allow management of project api keys via API (#6457)
* feat: allow management of project api keys via API

* chore: add api docs
2025-04-15 07:37:51 +00:00
Jason E. JensenandGitHub 3f36fc1191 fix(users): adjust pagination calculation for user retrieval (#6453) 2025-04-15 08:42:27 +02:00
Steffen SchmitzandGitHub 82d7cb9769 chore: expand project and org name limit to 60 chars (#6455) 2025-04-15 06:22:41 +00:00
Steffen SchmitzandGitHub 9bc75b5081 chore: add reference to full Langfuse API spec (#6454)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* feat: allow org api key management via admin api

* feat: allow the creation of projects using an org api key

* chore: allow deletion of projects via API

* feat: create outline of SCIM Users support

* feat: implement SCIM style user endpoints

* chore: allow set password on user

* chore: lint

* feat: add endpoints to manage project and organization memberships

* chore: add memberships api docs

* chore: reject project membership calls if entitlement is missing

* chore: add team plan to test org

* chore: add reference to full Langfuse API spec
2025-04-15 06:07:05 +00:00
Steffen SchmitzandGitHub 3b4811599d feat: add endpoints to manage project and organization memberships (#6449)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* feat: allow org api key management via admin api

* feat: allow the creation of projects using an org api key

* chore: allow deletion of projects via API

* feat: create outline of SCIM Users support

* feat: implement SCIM style user endpoints

* chore: allow set password on user

* chore: lint

* feat: add endpoints to manage project and organization memberships

* chore: add memberships api docs

* chore: reject project membership calls if entitlement is missing

* chore: add team plan to test org
2025-04-15 05:51:11 +00:00
Steffen SchmitzandGitHub d6eee1687e feat: create outline of SCIM Users support (#6439)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* feat: allow org api key management via admin api

* feat: allow the creation of projects using an org api key

* chore: allow deletion of projects via API

* feat: create outline of SCIM Users support

* feat: implement SCIM style user endpoints

* chore: allow set password on user

* chore: lint

* chore: add scim api docs

* chore: remove outdated generated artifacts
2025-04-15 05:31:22 +00:00
Steffen SchmitzandGitHub 4b36ff8752 feat: allow the creation and deletion of projects using an org api key (#6438)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* feat: allow org api key management via admin api

* feat: allow the creation of projects using an org api key

* chore: allow deletion of projects via API

* chore: add documentation for new project routes

* chore: cleanup

* chore: error updates and handling

* chore: reduce reduntant error handling
2025-04-14 20:05:14 +00:00
Steffen SchmitzandGitHub 64827a0c9d feat: allow org api key management via admin api (#6437)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* feat: allow org api key management via admin api

* chore: add defs

* chore: remove admin api
2025-04-14 19:34:46 +00:00
Steffen SchmitzandGitHub fc84da9508 feat: add new organization routes (#6430)
* feat: add new organizations apiKey route

* chore: implement additional org routes

* chore: add API docs

* chore: checkin openapi spec

* chore: update tests

* chore: move admin-api spec to organizations-api

* chore: add postman collection
2025-04-14 19:00:01 +00:00
Marc KlingenandGitHub 3ca321d9f8 feat(cloud): switch from crisp to plain for in-app chat support (#6433)
* feat(cloud): switch from crisp to plain for in-app chat support

* push

* fix lint

* push
2025-04-14 16:49:56 +02:00
marliessophieandGitHub 66df08e232 chore(evals): fall back to bullmq job.data.timestamp allowing for successful retries (#6450)
* chore(evals): fall back to bullmq `job.data.timestamp` allowing for successful retries

* chore: test

* chore: test

* docs: add diagram and docs

* chore: typo

* chore: typo
2025-04-14 14:33:22 +00:00
Hassieb PakzadandGitHub af3cf6d860 fix(llm-tools): do not strip additional properties in schema (#6445) 2025-04-14 11:02:25 +02:00
marliessophieandGitHub 0edecde6e3 style(compare-runs): label on run metrics visibility settings (#6405) 2025-04-14 08:34:41 +00:00
marliessophieandGitHub 5a2fef1fe9 feat(compare-runs): Allow for variable row height on compare view (#6400)
* feat(dataset-compare-runs): Allow for variable row height on compare view

* chore: import as type
2025-04-14 08:21:47 +00:00
Steffen SchmitzandGitHub 28bada800d chore: add test cases for projects api (#6429)
* chore: add test cases for projects api

* chore: lint

* chore: align project name with pipeline
2025-04-11 15:06:53 +00:00
Steffen SchmitzandGitHub 663d67cbd6 fix: update access check for projects api (#6428) 2025-04-11 16:00:32 +02:00
Steffen SchmitzandGitHub bd651e81b7 chore: refactor authentication for admin apis (#6417)
* chore: refactor authentication for admin apis

* chore: lint

* chore: address feedback

* feat: add post organizations endpoint

* chore: update tests

* chore: update api key data model

* chore: align code with new scoped api keys

* chore: typing

* chore: align tests with new logic

* chore: remove ratelimitoverrides

* chore: remove accesslevel from queue message

* chore: add additional check to ingestion endpoint

* chore: add test for legacy api key format in redis

* chore: map all api keys in auditlog
2025-04-11 12:37:38 +00:00
Steffen SchmitzandGitHub f1547dab58 fix: avoid write after end errors on otel metrics endpoint (#6419) 2025-04-10 14:09:12 +00:00
Steffen SchmitzandGitHub 34b470f1fe fix: allow sessions filter on widgets (#6418) 2025-04-10 14:04:16 +00:00
steffen911 040b720dbf chore: release v3.49.1 2025-04-10 15:04:52 +02:00
Steffen SchmitzandGitHub d56e06d6c9 Revert "fix: access all replicas to create dataset score metrics (#6407) (#6416)"
Revert "fix: access all replicas to create dataset score metrics (#6407)"

This reverts commit a3b407ac92.
2025-04-10 15:04:39 +02:00
steffen911 08af244179 chore: release v3.49.0 2025-04-10 14:39:07 +02:00
c821d06099 chore: disable error log checks in playwright tests (#6414)
* push

* dummy

* chore: lint

---------

Co-authored-by: steffen911 <steffen@langfuse.com>
2025-04-10 12:11:58 +00:00
Steffen SchmitzandGitHub 7ed5f71154 fix(dashboards): support all dashboard filters across views (#6413) 2025-04-10 11:59:46 +00:00
Steffen SchmitzandGitHub 8c44314f56 chore: add dummy route that accepts all otel metrics calls (#6408) 2025-04-10 11:37:54 +00:00
Steffen SchmitzandGitHub a3b407ac92 fix: access all replicas to create dataset score metrics (#6407)
* fix: access all replicas to create dataset score metrics

* fix: access all replicas to create dataset score metrics

* chore: linting

* dummy
2025-04-10 09:31:03 +00:00
marliessophieandGitHub b043cf2600 style(compare-runs): remove duplicate run column selector (#6403) 2025-04-10 09:12:29 +00:00
Steffen SchmitzandGitHub d7f75a5e62 build: update end to end tests after dashboard change (#6406) 2025-04-09 19:28:34 +00:00
Steffen SchmitzandGitHub 9951364ff0 feat(dashboards): add beta preview for customizable dashboards (#6376) 2025-04-09 18:14:51 +02:00
David Alexander PfeifferandGitHub 03cf7bccfc feat(evals): Extend job configuration filter options; Evaluators can run based on environments (#6039)
* added environment filter option to traces

* WIP on multi-select ui inconsistency

* fixed overflow issue in multi select where an item would not take the full width automatically; also changed to overflow-x-auto

* fixed invalid DOM nesting of div within p (FormDescription) that resulted in a hydration error

* fix multiselect to cut overflowing text with ellipsis and take full name as titlte so you can hover over it, removed unnecessary css classes

* WIP on create evaluator page ui issue with filter builder

* Revert "WIP on create evaluator page ui issue with filter builder"

This reverts commit fd6dd507f2df3c5fb404f4e38a2983e8ef6405ef.

* hide empty string from multi-select options, as supporting empty values would require further work - out of scope for now; also checked for side effects, with current usage all good

* Added test suite for eval filtering

* Resolved comment: removed getEnvironmentsForProject from traces router and instead fetched environments inside evaluator-form via api.projects.environmentFilterOptions

* move eval filtering tests into separate test file

* removed redundancy via fixtures, prepared tests for concurrency

* filtering tests run concurrently

* extracted traceFilterOptions fetching into separate hook as it is used in both the evaluator form and the traces table, also fixed traces table since the environment was now missing there

* useTraceFilterOptions Hook after merge

* Make TraceOptions keys partial so that we can update filter options asynchronously

* default to empty object if response undefined

* exclude environment filter builder column from traces table since toolbar already has env

* removed useTraceFilterOptions hook as use case changed such that extraction into hook is no longer warranted
2025-04-09 13:40:52 +00:00
Steffen SchmitzandGitHub 4e737ea6cf fix: update column mapping for categorical score charts (#6399) 2025-04-09 13:39:21 +00:00
marliessophieandGitHub d604ce3ceb feat(run-compare): add detail navigation (#6398)
* feat(run-compare): add detail navigation

* nits

* typo

* eslint
2025-04-09 13:26:41 +00:00
Steffen SchmitzandGitHub 2c2daf6bb4 chore: remove beta flag from blob storage integration (#6397) 2025-04-09 09:45:56 +00:00
Hassieb PakzadandGitHub 8ebafda8a4 fix(prompt-ui): internal type on review chat prompt (#6391) 2025-04-08 21:22:22 +02:00
Steffen SchmitzandGitHub af7384b549 fix: propagate userId correctly to downstream tables (#6389) 2025-04-08 14:14:18 +00:00
Hassieb PakzadandGitHub 9768c2d171 perf(batch-exports): make page size configurable and set to 500 (#6385) 2025-04-08 12:59:44 +02:00
David Alexander PfeifferandGitHub f5aec02277 fix(scores/trpc): byid on deleted score resulted in 5xx error instead of 404 (#6384) 2025-04-08 09:58:31 +00:00
David Alexander PfeifferandGitHub b9cf01dd28 feat: add metadata field to scores (#6046) 2025-04-07 22:13:21 +02:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
7c42cf4357 perf: retry failed S3 deletions (#6378)
* push

* Update packages/shared/src/server/services/StorageService.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-07 18:19:19 +00:00
Max DeichmannandGitHub 72f4b88d15 perf: Increase batch export timeouts (#6375)
* push

* push

* push

* fix: flaky playwright

* fix

* push

* push
2025-04-07 13:26:23 +00:00
Max DeichmannandGitHub 99ce7c5093 chore: Increase project deletion retries (#6372)
* fix: flaky playwright

* push

* push
2025-04-07 12:31:09 +00:00
Max DeichmannandGitHub fcb7b82ba4 chore: improve project deletion logs (#6373)
push
2025-04-07 12:30:51 +00:00
Max DeichmannandGitHub ac5017c9ae chore: increase batch export retries and improve observability (#6374)
push
2025-04-07 12:17:57 +00:00
Max DeichmannandGitHub 00ffddad0f chore: Increase delete queue processing observability (#6371)
* fix: flaky playwright

* push
2025-04-07 11:34:37 +00:00
Steffen SchmitzandGitHub 28bf40769e chore: clarify titles in dashboard latency charts (#6370) 2025-04-07 11:33:13 +00:00
Hassieb PakzadandGitHub b65b0e25fc fix(playground): handle undefined content with toolCalls (#6364) 2025-04-07 09:13:31 +00:00
Max DeichmannandGitHub 62f860d9ef fix: flaky playwright (#6365) 2025-04-07 09:02:33 +00:00
Steffen SchmitzandGitHub 3989480a94 fix: update dashboard column accesses after query builder updates (#6363)
* fix: update dashboard column accesses after query builder updates

* chore: modify count
2025-04-07 08:50:09 +00:00
Max DeichmannandGitHub 4a6afddbed chore: create scores domain (#6360)
* push

* security: upgrade next

* fix

* push

* fix

* fix

* fix

* fix

* push

* push

* feat: full text search

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* push

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* security: upgrade nextjs 14.2.26

* push

* push

* playwright

* push

* fix

* remove

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* chore: create score domain

* chore: create score domain

* chore: create score domain

* chore: create score domain

* chore: create score domain
2025-04-07 07:19:04 +00:00
Max DeichmannandGitHub 4c6e8964e3 chore: refactor observations domain (#6353)
* push

* security: upgrade next

* fix

* push

* fix

* fix

* fix

* fix

* push

* push

* feat: full text search

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* push

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* chore: refactor observations domain

* security: upgrade nextjs 14.2.26

* push

* push

* playwright

* push

* fix

* remove

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* chore: create score domain
2025-04-07 07:03:38 +00:00
Max DeichmannandGitHub cb900c61f8 chore: add traces domain layer (#6253)
* push

* security: upgrade next

* fix

* push

* fix

* fix

* fix

* fix

* push

* push

* feat: full text search
2025-04-07 06:42:03 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
9bc745d647 chore: upgrade playwright tests (#6359)
* push

* push

* fix

* fix

* Update web/src/__e2e__/create-project.spec.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix

* fix

* fix

* fix

* fix

* fix

* Update web/src/__e2e__/create-project.spec.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-06 16:41:54 +00:00
Max DeichmannandGitHub 1b6e9e1c8b chore: upgrade tiktoken (#6357) 2025-04-06 12:23:24 +00:00
Max DeichmannandGitHub ce21cf34ae fix: fix model table provided_model_name (#6356)
* fix: fix model table

* fix: fix model table

* fix: fix model table
2025-04-06 12:15:54 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
bf063a72a0 chore(deps): bump next from 14.2.25 to 14.2.26 (#6316)
Bumps [next](https://github.com/vercel/next.js) from 14.2.25 to 14.2.26.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.25...v14.2.26)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.26
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-05 22:05:30 +00:00
5c2ac78eca feat: add new widget creator page (#6322)
* feat: add new widget creator page

* chore: add widget configuration box to creation screen

* chore: make chart title and description editable

* chore: auto-create cases

* chore: dynamically render chart based on output properties

* chore: linting

* chore: coloring

* chore: allow time series breakdowns

* chore: add support for additional chart types

* chore: conditionally show aggregation and row limit

* chore: add more separation between data selection and visualization in chart builder

* chore: add filter and time selector

* chore: make widget form overflow vertically

* chore: linting

* chore: reuse chart colors

* style: revert padding changes to card.tsx

* style: revert changing card headings to div element in card.tsx

* chore: refactoring and feedback

* chore: remove selected view restrictions

* chore: chart formatting

* chore: lint

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-04-04 14:18:33 +00:00
steffen911 c6dadda96f chore: release v3.48.1 2025-04-04 11:47:13 +02:00
Steffen SchmitzandGitHub f831d1ac21 fix: improve parsing of tags for otel endpoint (#6345) 2025-04-04 09:44:29 +00:00
steffen911 1ca4ec7e28 chore: release v3.48.0 2025-04-03 19:04:59 +02:00
Steffen SchmitzandGitHub a9ca40aa81 fix: include tag filtering support on dashboard (#6338) 2025-04-03 16:58:05 +00:00
Marc KlingenandGitHub 8b0588a786 fix(auth): block duplicate membership invites that lead to sign in issues (#6337) 2025-04-03 16:24:54 +00:00
marliessophieandGitHub cc79604302 style(ui): reduce text size of trace/observation name on detail (#6329) 2025-04-03 13:41:28 +00:00
marliessophieandGitHub f12532c0a2 chore(ui): minor peek ui improvements (#6328)
chore(ui): minor peek ui wording improvements
2025-04-03 12:08:54 +00:00
Steffen SchmitzandGitHub 1c1519555a fix: blob storage export UI consistency (#6327)
* fix: blob storage export UI consistency

* chore: only set field value
2025-04-03 11:39:34 +00:00
SebiandGitHub 8954cf0c82 feat(auth): scope cookies to path for custom BASE_PATH deployments (#6320)
extend cookie paths
2025-04-03 11:10:36 +00:00
marliessophieandGitHub 849428abee feat(dataset-runs): improve UI/UX of runs and compare page (#6296)
* fixup: dataset io cell

* fixup: restructure dataset compare view

* fix: react dom error

* chore: fix margins

* chore: remove dataset details popover

* feat: make dataset runs table resizeable

* fix: react dom error

* fix: show more fraction digits on cost chart for experiment runs

* chore: improve design of observation tree peek view

* chore: refactor peek view pattern

* chore: implement new peek view pattern for compare view

* chore: remove zustand dep

* push

* perf: improve dataset compare view

* chore: eslint

* eslint

* push

* lint

* chore

* push
2025-04-03 10:57:35 +00:00
Hassieb PakzadandGitHub c014612cca feat(playground): allow uppercase in tool / schema names (#6326)
* feat(playground): allow uppercase in tool / schema names

* refactor
2025-04-03 09:59:04 +00:00
Hassieb PakzadandGitHub 803057b5eb fix(localStorage): memoize properly to avoid navigation issues (#6323) 2025-04-03 09:01:42 +00:00
Steffen SchmitzandGitHub 2152447932 perf: reduce search interval for trace existence check (#6315)
* perf: reduce search interval for trace existence check

* chore: add final again
2025-04-03 07:28:08 +00:00
marliessophieandGitHub 70e0b036e5 chore(batch-action): make entitlement prop optional (#5884)
* chore(batch-action): make `entitlement` prop optional

* push
2025-04-02 17:00:54 +00:00
Steffen SchmitzandGitHub bef666a694 feat: add json and jsonl support for scheduled blob exports (#6313)
* feat: add json and jsonl support for scheduled blob exports

* chore: revert schedule and add migration
2025-04-02 15:38:54 +00:00
Hassieb PakzadandGitHub 601a22d42c feat(playground): add tool parsing from Langchain generations (#6314) 2025-04-02 15:26:52 +00:00
Steffen SchmitzandGitHub 5fdde72d64 chore: track dlq length (#6312) 2025-04-02 12:54:06 +00:00
Hassieb Pakzad f9fc778ee7 chore: release v3.47.0 2025-04-02 14:01:27 +02:00
Hassieb PakzadandGitHub 975ef81bb5 feat(prompts): add protected labels (#6306) 2025-04-02 13:57:31 +02:00
Steffen SchmitzandGitHub 28e61fae0b chore: remove obsolete dashboard queries (#6303) 2025-04-02 09:32:41 +00:00
Steffen SchmitzandGitHub 0534ccacb2 perf: reduce input validation for dataset items (#6307) 2025-04-01 19:25:23 +00:00
Steffen SchmitzandGitHub 0c5089fd2b chore: pin dev clickhouse version to 24.3 (#6304)
* chore: pin dev clickhouse version to 24.3

* chore: only add input format flag for cloud

* chore: whitelist accepted cloud regions

* chore: disable test
2025-04-01 15:48:18 +00:00
Max DeichmannandGitHub 37c4c1422c perf: remove sentry tunnel config (#6305)
* fix

* fix
2025-04-01 13:01:48 +00:00
014da40fbf fix(ui): TableWithMetadataWrapper uses contain-layout on its root div so that EvaluatorDetail overflow issue is mitigated (#5982)
* fix: (workaround) evals page overflows when rendering right hand side metadata bar

This is more of a workaround, we should probably disable the overflow on the main wrapper div in layout.tsx and enable it only where needed - or find out why the overflow inside the metadata sidebar trigggers the scrollbar to pop up - some info on what I found:

- The body and other divs close to the root do not increase their size.
- The navigation sidebar does increase its vertical height, but limiting that does not resolve the main scrollbar.
- Limiting specifically the wrapper div in layout.tsx with overflow-hidden does resolve this, but globally disabling it requires most pages to be refactored.

* Revert "fix: (workaround) evals page overflows when rendering right hand side metadata bar"

This reverts commit 313a1c0d8a885a04974399f850f7f9aee58fd714.

* fix(ui): TableWithMetadataWrapper uses contain-layout on its root div so that EvaluatorDetail overflow issue is mitigated

Checked for side effects on the other components using TableWithMetadataWrapper and all work as expected.

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-04-01 10:38:42 +00:00
Hassieb PakzadandGitHub 2317334bff feat(playground): persist selected model (#6292)
* feat(playground): persist selected model

* push

* push
2025-04-01 08:27:48 +00:00
Steffen SchmitzandGitHub 9ed449288b chore: add batch export env variables to docker-compose.yml (#6300) 2025-04-01 06:54:23 +00:00
Steffen SchmitzandGitHub 3b4800c37a chore: migrate score charts to new query builder (#6294) 2025-04-01 08:47:37 +02:00
Hassieb PakzadandGitHub b2ffd05ff8 fix(prompt-form): move prompt reference button (#6289) 2025-03-31 15:09:15 +00:00
Steffen SchmitzandGitHub e4594db953 chore: migrate latency charts to new query builder (#6284)
* chore: migrate latency charts to new query builder

* chore: convert model latencies chart

* chore: typing
2025-03-31 13:00:05 +00:00
marliessophieandGitHub 4b733c4da8 chore(annotation-scores): revert to synchronous delete; upsert on create route (#6282)
* chore(annotation-scores): revert to synchronous delete; upsert on create route

* fix: ensure scores data not only marked as stale but refreshed on score update
2025-03-31 12:36:29 +00:00
Hassieb PakzadandGitHub 378ac1c1a4 feat(llm-connections): add gemini-2.5-pro-exp-03-25 (#6286) 2025-03-31 11:57:18 +00:00
steffen911 9233c7c2d2 chore: release v3.46.0 2025-03-31 12:10:23 +02:00
Steffen SchmitzandGitHub c2170c0c8d feat: add support for GCS buckets (#6269)
* feat: add support for GCS buckets

* chore: cleanup

* chore: trim gcs credentials before parsing

* chore: add logging for redis shutdown in tests

* chore: update timeout for trace delete test

* chore: bump trace delete concurrency for tests
2025-03-31 10:04:49 +00:00
Max DeichmannandGitHub 7faf0e009f chore: revert unintended worker concurrency changes (#6275)
push
2025-03-29 14:42:12 +01:00
Max DeichmannandGitHub 86318ab50b perf: reduce worker deletion concurrency (#6274)
push
2025-03-29 14:39:17 +01:00
Max DeichmannandGitHub 48d280433a perf: reduce delete concurrency (#6273)
push
2025-03-29 14:27:38 +01:00
marliessophieandGitHub 45f29eaad8 fix(patch): hide scores tab on trace/observation preview (#6270)
fix(patch): hide scores tab on peek view
2025-03-28 20:34:52 +00:00
2bca2899f4 feat(evals, ui): enable deletion of evaluator via UI, DeleteButton refactor (#6247)
* feat(evals): enable deletion of evaluators and templates via UI

* Added tests for the evals job and template deletion trpc endpoints

* evals-trpc test refactor, it had a side effect on other tests since it was pruning the db, now instead creates a new org and project per tests and cleans them up after all tests are done

* removed test comments as those are now implemented

* fix wrong wording

* added template version to delete confirmation on template details delete action

* DeleteButton refactoring and moved deletion of templates to LFE-4573

* moving the call to captureDeleteSuccess inside the successful branch of executeDeleteMutation

* Display lock icon when action button in icon mode is unauthorized

* fix: quietly delete scheduled evals for deleted job executions

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-03-28 18:22:21 +00:00
marliessophieandGitHub af6d0f0086 fix(ui): trace timeline width calculation error (#6268)
fix(ui): timeline width
2025-03-28 15:44:01 +00:00
Steffen SchmitzandGitHub 3802b1849d chore: migrate user chart to new query builder (#6262) 2025-03-28 14:28:03 +00:00
Steffen SchmitzandGitHub 19d2ee06e9 chore: update list of allowed blob storage export intervals (#6266) 2025-03-28 14:26:44 +00:00
Hassieb PakzadandGitHub b670abc6bc feat(playground): add tool call support (#6069) 2025-03-28 15:21:51 +01:00
Hassieb PakzadandGitHub 9395e94670 fix(prompts): handle resolvePromptGraph on deleted prompt (#6264) 2025-03-28 14:49:15 +01:00
Hassieb PakzadandGitHub 684e52e11f chore(docker-compose): update default LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT (#6265) 2025-03-28 14:49:00 +01:00
marliessophieandGitHub 4048e807af fix(prompt-ui): state stuck in showing prompt diff cycle (#6258) 2025-03-28 09:31:19 +00:00
Hassieb PakzadandGitHub 0b173f47f7 fix(prompts): dollar sign in replaceValue breaking resolution (#6256) 2025-03-27 23:39:38 +01:00
Max DeichmannandGitHub 15cad50991 chore: add logger for trpc route (#6255) 2025-03-27 21:48:28 +01:00
marliessophieandGitHub 747bfae721 chore(eval-count-tooltip): refactor to extract business logic into hook (#6251)
* chore(eval-count-tooltip): refactor to extract business logic into hook

* eslint
2025-03-27 17:45:50 +00:00
marliessophieandGitHub a01ec32715 feat(evals): allow adding custom value to tags filter (#6250) 2025-03-27 16:22:08 +00:00
Max DeichmannandGitHub 780fd577c4 chore: set sentry profiling sample rate (#6249)
push
2025-03-27 15:49:33 +00:00
marliessophieandGitHub cd33442168 fix(ui): detail navigation on users > traces page (#6248) 2025-03-27 15:39:49 +00:00
Steffen SchmitzandGitHub 6582dbc645 chore: migrate observation by model charts to new query builder (#6245) 2025-03-27 14:50:39 +00:00
Max DeichmannandGitHub 055eb06a4f chore: add sentry remove dd env variables in build step (#6246) 2025-03-27 15:40:26 +01:00
marliessophieandGitHub daeaf0a986 fix(trace-timeline): overflow and scroll behaviour (#6216)
* fix(trace-timeline): overflow and scroll behaviour

* fix(trace-timeline): show maximum of 3 scores and remainder in hover card

* push

* chore: refactor

* chore

* chore
2025-03-27 13:53:55 +00:00
Steffen SchmitzandGitHub f199dcb61c chore: remove warnings on missing domain timestamp (#6235) 2025-03-27 12:09:33 +01:00
Max DeichmannandGitHub 0ec0cca442 chore: enable sentry tunnel (#6234) 2025-03-27 11:38:53 +01:00
Steffen SchmitzandGitHub f4c75b896b chore: bump timeout for posthog export queries (#6233) 2025-03-27 09:57:04 +00:00
Steffen SchmitzandGitHub 16c76b2722 perf: reduce memory usage for posthog export via join_algo overwrite (#6230) 2025-03-27 10:17:44 +01:00
Steffen SchmitzandGitHub 64f6051048 chore: move the traces timeseries chart to new query builder (#6226) 2025-03-26 17:02:03 +00:00
Max DeichmannandGitHub 96dfd86b51 chore: improve sentry setup (#6224) 2025-03-26 16:20:14 +01:00
Steffen SchmitzandGitHub b1aaf57ad5 chore: migrate ModelCostTable to new query mechanism (#6222)
* chore: migrate ModelCostTable to new query mechanism

* chore: lint
2025-03-26 14:58:43 +00:00
steffen911 fae508d820 chore: release v3.45.2 2025-03-26 09:59:47 +01:00
Steffen SchmitzandGitHub a6e581d349 fix: use correct column name for filtering traces bar chart (#6215) 2025-03-26 08:56:09 +00:00
Max DeichmannandGitHub 9e77738a2f fix: fix by model charts (#6208)
* fix

* push

* push

* push
2025-03-25 20:56:00 +00:00
Marc Klingen d15c0a27f5 chore: release v3.45.1 2025-03-25 19:49:44 +01:00
Marc KlingenandGitHub a8626841b5 chore(cloud): update blobstorage integration entitlement (#6209) 2025-03-25 18:40:57 +00:00
marliessophieandGitHub 5945a93b7d fix(evaluator-config): preview execution of dataset run items (#6206)
* chore(evaluator-config): preview execution of dataset run items

* push
2025-03-25 17:44:11 +00:00
Steffen SchmitzandGitHub 9c447de584 chore: auto-close stale issues (#6207) 2025-03-25 17:21:44 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
48df0e04de fix: fix model charts (#6202)
* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* Update packages/shared/src/server/repositories/dashboards.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update web/src/__tests__/async/repositories/dashboard-repository.servertest.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix

* fix

* fix

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-25 17:12:50 +00:00
Steffen SchmitzandGitHub 0897ac753b feat: add custom-query backend (#6006)
* feat: add self-serve dashboard backend poc

* chore: data model thoughts

* chore: create initial query builder example and test case

* chore: handle empty dimensions and metrics

* chore: lint

* chore: extend filter conditions to account for timestamps

* chore: refactor query builder into sub-functions

* chore: use template queries for user-supplied values

* chore: add query builder tests

* chore: handle time dimension

* chore: add trpc endpoint to execute custom clickhouse query

* chore: add a query playground component to test a couple of queries

* chore: handle multiple joins

* chore: add observations table to data model

* chore: fix bug in data model

* chore: add test case to compare with old dashboard results

* chore: add scaffold for additional views

* expand the users view and add segments to filter scores subviews

* chore: add test cases for score views

* chore: drop users and sessions for now

* chore: add scores aggregate and observations cost tests

* chore: add sql injection tests

* chore: lint

* chore: fix dashboard test cases

* chore: drop users query test

* chore: add order by logic

* chore: fill timeseries values

* chore: use new query function on dashboard

* chore: convert TracesBarListChart.tsx to new query endpoint

* chore: make chart data compile

* chore: update tests

* chore: add tags to custom queries

* chore: typing

* chore: suffix join condition with sql

* chore: add typing in queryBuidler

* chore: limit playground to cloud admin users

* chore: add util to map legacy dashboard columns to new model

* chore: separate time filter state for TracesBarListChart.tsx

* chore: pass timestamps directly into chart components

* chore: add userId and sessionId on observations

* chore: pick auto time granularity based on hours
2025-03-25 14:54:37 +00:00
Marc KlingenandGitHub d00c63100e chore(cloud): remove rate limits from projects api as it is used by auth_check in sdks (#6193)
* chore(cloud): remove rate limits from projects api as it is used by auth_check in sdks

* address feedback

* improve comment
2025-03-25 14:52:23 +00:00
steffen911 e8970bc11c chore: release v3.45.0 2025-03-25 15:52:33 +01:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
eecb253093 feat: add blob storage integration (#6143)
* feat: add page to configure blob storage integration

* chore: remove blob storage logo and include three config options

* chore: add blob storage integration table

* chore: update router for blobstorage

* chore: placeholder for password and type update

* chore: create scaffolding for blobstorage queue processors

* chore: add scheduling logic for blob storage integration

* chore: prepare setup in processing job

* chore: implement extraction fucntion

* chore: add test case and fix generator setup

* chore: lint

* chore: overwrite default region to empty

* chore: remove details from auditlog and add beta

* Update web/src/pages/project/[projectId]/settings/integrations/blobstorage.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker/src/__tests__/blobStorageIntegrationProcessing.test.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: correct env name in tests

* Update worker/src/ee/integrations/blobstorage/handleBlobStorageIntegrationProjectJob.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: update doc urls

* chore: open docs in new tab

* chore: ensure consistent block id length azure

* chore: revert test change

* chore: remove dummy error log

* chore: type annotation

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-25 13:39:33 +00:00
marliessophieGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
4e4fbb5101 fix(prompt-experiments): show form error if name is duplicated (#6189)
* fix(prompt-experiments): show form error if name is duplicated

* chore: fix typo

* Update web/src/ee/features/experiments/hooks/useExperimentNameValidation.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* eslint

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-25 09:07:16 +00:00
marliessophieandGitHub bb4823aa27 fix(evaluator-templates): control editing state (#6187) 2025-03-24 13:29:19 +00:00
marliessophieandGitHub 174d96c100 feat(evals): preview number of historic items to be evaluated (#6116)
* feat(evals): preview number of historic items to be evaluated

* refactor: split up large evaluator form file

* chore: drop optional pagination

* chore: fix eslint

* chore: rename test
2025-03-24 13:05:50 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
8e59b544c9 chore(deps): bump next from 14.2.21 to 14.2.25 (#6175)
Bumps [next](https://github.com/vercel/next.js) from 14.2.21 to 14.2.25.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.21...v14.2.25)

---
updated-dependencies:
- dependency-name: next
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-24 12:57:26 +00:00
Hassieb PakzadandGitHub cffd84f0e7 chore(azure-llm-connection): update api-version to support reasoning models (#6186) 2025-03-24 14:06:55 +01:00
Hassieb PakzadandGitHub 845faee385 fix(cost-tracking): null values in OpenAI usage schema (#6184) 2025-03-24 13:42:20 +01:00
Hassieb PakzadGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
e45c4af30f chore(prompts): add error logs for resolvePromptGraph (#6181)
* chore(prompts): add error logs for resolvePromptGraph

* Update packages/shared/src/server/services/PromptService/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-24 11:31:29 +00:00
Marc KlingenandGitHub 616544a48c chore: apply prettier (#6176) 2025-03-24 09:46:13 +00:00
steffen911 75d64b38b3 chore: release v3.44.0 2025-03-24 09:35:15 +01:00
Steffen SchmitzandGitHub 65cdd75282 chore: drop queue backup option in bullmq (#6141)
* chore: drop queue backup option in bullmq

* chore: lint
2025-03-24 08:26:32 +00:00
Steffen SchmitzandGitHub 2744028f4c feat: extract top-level metadata from langfuse.metadata (#6173)
* feat: extract top-level metadata from langfuse.metadata

* chore: extract metadata based on prefix

* chore: add support for resourceAttributes
2025-03-24 08:09:04 +00:00
Max DeichmannandGitHub 3c02c54de2 fix: fix model paring from redis (#6157)
* push

* fix

* push

* fix

* fix
2025-03-21 18:42:06 +00:00
marliessophieandGitHub 974d2c1007 fix(ui): reference errors in prompt detail screen (#6126)
* chore(ui): convert `action-button` to use forwardRef for compatibility w/ buttons

* fix(ui): ensure prompt item ref points to `command-item`
2025-03-21 17:07:21 +00:00
Max DeichmannandGitHub 008045c771 perf: cache model not found (#6148)
* push

* push

* fix
2025-03-21 14:57:17 +00:00
Marc KlingenandGitHub c6790c6c0e chore(ui): do not auto-generate new api keys in setup-tracing step (#6149)
* chore(ui): do not auto-generate new api keys in setup-tracing step

* fix
2025-03-21 14:17:01 +00:00
Marc KlingenandGitHub 909445ce5b fix(prompts): trim prompt names (#6145)
* fix(prompts): trim prompt names

* push
2025-03-21 13:38:04 +00:00
Max DeichmannandGitHub f4ed4a2635 chore: improve model match observability (#6147)
push
2025-03-21 13:18:39 +00:00
Max DeichmannandGitHub d6206b5269 chore: increase model match observability (#6144) 2025-03-21 13:45:36 +01:00
Max DeichmannandGitHub 62fa43aa78 feat: cache LLM models in Redis (#6129)
* push

* fix

* fix

* fix

* push

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* push

* push

* push

* fix

* fix

* fix

* fix

* push
2025-03-21 11:26:11 +00:00
Steffen SchmitzandGitHub 3c9239a4ab chore: update .env.prod.example (#6140) 2025-03-21 10:24:48 +00:00
marliessophieandGitHub 476cddb4e1 chore(ui): clamp page-header title to one line (#6139) 2025-03-21 10:01:45 +00:00
Steffen SchmitzandGitHub a6c1678da8 feat: add additional input/output parsing for pydantic via OTel (#6138) 2025-03-21 10:00:33 +00:00
Steffen SchmitzandGitHub 129805ebde fix: safely parse big numbers in api responses (#6136)
* fix: safely parse big numbers in api responses

* feat: add additional input/output parsing for pydantic via OTel

* Revert "feat: add additional input/output parsing for pydantic via OTel"

This reverts commit 71d0f3abd951702393aac7ab79a875b888f089cf.
2025-03-21 09:30:24 +00:00
marliessophieandGitHub 59eafb42c3 fix(ui): require annotation queue selection for bulk action (#6127) 2025-03-20 21:08:57 +00:00
marliessophieandGitHub e4695ac717 fix(ui): dom nesting error data-table-row-height-switch (#6125) 2025-03-20 20:12:52 +00:00
marliessophieandGitHub eebd6ea0e9 fix(ui): dom nesting error page-header (#6124) 2025-03-20 20:06:05 +00:00
Marc KlingenandGitHub b4a8b08d3d chore(cloud): add posthog for projects and orgs (#6117)
* chore(cloud): add posthog for projects and orgs

* push
2025-03-20 18:07:17 +00:00
Max DeichmannandGitHub ed44b663ff chore: add dd env variables to github build command (#6119) 2025-03-20 17:36:48 +01:00
Marc KlingenandGitHub 720ebe861e chore(cloud): cache response of cloudStatus api for 5 minutes (#6115)
chore(cloud): keep response of cloudStatus api cached across views for 5 minutes
2025-03-20 14:25:54 +00:00
Max DeichmannandGitHub 9a6f8fd75d refactor: refactor naming of database batch exports (#6111)
* refactor: refactor naming of database batch exports

* refactor: refactor naming of database batch exports
2025-03-20 12:56:38 +00:00
Steffen SchmitzandGitHub f0da1cfa30 chore: reduce cloudwatch flush frequency and batch metrics (#6108)
* chore: reduce cloudwatch flush frequency and batch metrics

* chore: simplify metrics reset

* chore: keep 30s flush

* chore: correct CW metrics
2025-03-20 12:05:07 +00:00
Marc KlingenandGitHub 0c9ebb3da7 feat(cloud): show status page menu item during incidents (#6100)
* feat(cloud): show status page menu item during incidents

* add to support menu
2025-03-20 09:59:49 +00:00
Hassieb PakzadandGitHub 3d16d1ac57 fix(model-params-ui): duplicate model names leading to rendering issues (#6107) 2025-03-20 10:40:59 +01:00
Marc KlingenandGitHub fa462a8bc4 chore(ui): improve padding in peek view header (#6099) 2025-03-19 18:59:29 +00:00
David Alexander PfeifferandGitHub 695b65fe0d chore(tests): fully unpin node version from temporary fix #6063 (#6096) 2025-03-19 15:38:36 +00:00
Hassieb PakzadGitHubjake goldenellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
955388e846 feat(llm-connections): add atla adapter (#6088)
* feat: enable Atla integration (#6022)

* push

* push

* Update packages/shared/src/server/llm/fetchLLMCompletion.ts

* Update web/src/features/public-api/components/CreateLLMApiKeyForm.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Revert "Update web/src/features/public-api/components/CreateLLMApiKeyForm.tsx"

This reverts commit efb2d3b359a85680224876481421d31bfbd44fd3.

---------

Co-authored-by: jake golden <jackson.golden@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-19 12:36:12 +00:00
David Alexander PfeifferandGitHub c6864ed8d9 fix(tests): disable --experimental-require-module (default since Node 20.19.0) since it breaks jest-based testing (#6075)
* pin node to problematic version for working on a fix

* added cross-env and disabled the feature flag causing the testing issue - long term fix should probably be switching to vitest

* unpin node version since problematic feature is now disabled fo affected tests
2025-03-19 11:04:59 +00:00
marliessophieandGitHub 917516fa91 chore(ui): control for more specific onClick actions in table with peek view (#6086) 2025-03-19 10:25:03 +00:00
David Alexander PfeifferandGitHub 19017a6709 fix(ui): OpenAiMessageView now renders messages where content is an empty string instead of ignoring them (#6068)
* fix(ui): IOPreview now renders messages where content is an empty string instead of ignoring them

* removed debug logs

* OpenAiMessageView first filters valid messages to render and then executes the "show x more ..." logic, this way we ensure that there can never be a mismatch between messages items and the amount of fields we support displaying

* Render empty string quotation marks if content is empty but not null or undefined

* Change length check to falsy check for conditional display of empty quotes if content is empty

* cleaner falsy check for rendering empty string quotation marks
2025-03-19 09:41:32 +00:00
marliessophieandGitHub ca5e6569f9 fix(peek-view): ensure onClickRow action smoothly keeps peek view open; onClick of select cells independent of peek view state (#6079)
* fix(ui): stop propagation on traces table checkbox

* fix(peek-ui): handle onRowClick action correctly for peek view

* feat(peek): do not close upon any screen button or checkbox clicks

* push
2025-03-19 00:41:36 +00:00
marliessophieandGitHub 101942b98b fix(ui): padding on single prompt page (#6070) 2025-03-18 18:35:32 +00:00
marliessophieandGitHub 1188a21b02 feat(traces): add peek view on traces table (#6048)
* feat: adjust table to new style

* style: filter and column options

* fixup: add peek view

* feat: add detail navigation

* chore: remove duplicated code

* chore: style settings tables

* chore: pin first column on traces table

* style: adjust animation speed of peek view

* fix: padding on score configs table

* style: onClick cursor in table

* fix: eslint

* style

* chore: fix toggle for timeline

* push

* style: add padding
2025-03-18 17:25:52 +00:00
Marc KlingenandGitHub ec72dd840d chore(ui): render min-level hint in a single line (#6051) 2025-03-18 14:55:54 +00:00
steffen911 c2d800ff0e chore: release v3.43.0 2025-03-18 15:15:41 +01:00
Steffen SchmitzandGitHub dcf159ef82 chore: lock dd-trace to 5.36.0 (#6064) 2025-03-18 14:13:30 +00:00
Marc KlingenandGitHub 998984b9ed chore(cloud): improved pricing/packaging (#5664)
* wip

* simplify

* actionbutton for upgrade prompt

* count events for free plan limit

* add sso settings

* rename lite to core

* add product id

* nit

* nits

* fix rate limit test

* ci run on teams plan

* fix

* fix names of pipeline steps

* test tmp

* add better error logs to makeAPICall

* push

* clone to throw better errors

* improve error

* improve error

* fix

* remove comment

* refactor

* push

* next attempt

* nit

* create new auth for failing test suites

* push

* fix rate limit test
2025-03-18 13:47:03 +00:00
Marc KlingenandGitHub 022b234e57 chore: pin node 20.18.3 to avoid issue introduced by 20.19.0 (#6063)
* test commit

* chore: pin node 20.18.3 to avoid issue introduced by 20.19.0

* docker

* revert test commit

* push
2025-03-18 13:30:58 +00:00
b8777aa5d1 chore: better error message when uploading existing dataset item (#6053)
chore: better docstring when uploading existing dataset item to different dataset

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-18 13:07:11 +00:00
Steffen SchmitzandGitHub 2e84feeab7 feat: support external S3 endpoint for batch exports (#6055)
* feat: add media and export external endpoint config

* chore: add docs string

* chore: add external endpoint support for azure

* chore: drop feature for media
2025-03-18 10:57:14 +00:00
Steffen SchmitzandGitHub a508485f88 chore: enable datadog profiling for web (#6020)
* chore: enable datadog profiling for web

* chore: update dockerfile
2025-03-18 07:26:53 +00:00
marliessophieandGitHub d8bb283a44 fix(ui): make models detail page scrollable (#6050) 2025-03-17 19:48:27 +00:00
Steffen SchmitzandGitHub 75ec05e206 Delete .devcontainer/devcontainer.json 2025-03-17 15:01:41 +01:00
Steffen SchmitzandGitHub 19f710abe2 chore: add devcontainer config 2025-03-17 14:56:35 +01:00
David Alexander PfeifferandGitHub ad2d6bee66 chore(ui): remove padding of "Additional Input" field on OpenAiMessageView (#6037) 2025-03-17 10:06:32 +00:00
David Alexander PfeifferandGitHub 40d0acae5f chore(ui): alignment and sizing consitency in ObservationTree / GroupedScoreBadge (#6031)
* Unified design of grouped score badge with trace detail score badges, except that text is no longer medium/semibold

* Made padding of root item of ObservationTree consistent with other items
2025-03-17 09:58:23 +00:00
Marc KlingenandGitHub 4d387cc35c chore(ui): standardize the prompt variable list preview across views (#6027)
* chore(ui): standardize the prompt variable list preview across views

* use ctx prisma
2025-03-17 09:34:44 +00:00
690885c900 fix(ui): fixed scrollbar appearing on IO preview mode switcher in trace and observation details (#5995)
* fixed scrollbar appearing on IO preview mode switcher in trace and observation details

* fix

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-14 19:32:46 +00:00
David Alexander PfeifferandGitHub ef2baa94be fix(ui): fixed misalignment of badges in trace and observation details (#5994)
* fixed alignment issues of badges wrapped in links in trace and observation detail views

* Fixed tooltip of aggregated usage data badge (in trace details) rendering even if there is no underlying data
2025-03-14 19:19:14 +00:00
db6b442832 feat: Add environment tag on trace / observation detail views (#5985)
* feat: Add environment tag on trace / observation detail views

* shorten label

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-14 19:17:19 +00:00
Hassieb Pakzad 8231ebae4a chore: release v3.42.1 2025-03-14 17:05:33 +01:00
Marc KlingenandGitHub 326e8d3189 chore(ui): always show tag buttons in table cells instead of only on hover (#6025) 2025-03-14 15:23:32 +00:00
Hassieb PakzadandGitHub d2001d110d chore(docker-compose): bind services to localhost (#6023) 2025-03-14 14:16:36 +01:00
Hassieb PakzadandGitHub 3f81c6b9d5 feat(cost-tracking): add OpenAIResponseUsageSchema support (#6003) 2025-03-14 12:08:28 +00:00
Marc KlingenandGitHub 991f837ee5 fix(ui): hide jump to playground button when not entitled (#6014) 2025-03-14 00:01:40 +00:00
Marc Klingen 062a7f27b8 chore: release v3.42.0 2025-03-13 19:23:52 +01:00
Steffen SchmitzandGitHub 924b326db6 chore: remove unused prisma query builder (#6005) 2025-03-13 17:21:23 +00:00
Andres CarrilloandGitHub 2780e66c13 feat(auth): add custom Gitlab SSO URL option (#5984) (#6004) 2025-03-13 17:50:16 +01:00
Marc KlingenandGitHub c2f51b8d2d feat(ui): add csv/json export for scores table (#5949)
* feat(ui): add csv/json export for scores table

* Merge branch 'main' into marc/lfe-3466-export-scores

* push

* add test

* fix cols

* clean up diff

* fix
2025-03-13 16:23:30 +00:00
Steffen SchmitzandGitHub 3a3021a176 chore: reduce host port mapping for internal docker compose components (#6002) 2025-03-13 15:36:34 +00:00
Marc KlingenandGitHub 5dd5a39f90 feat(api): add api for annotation queues (#5945)
* feat(api): add api for annotation queues

* fix

* push

* strict zod interfaces

* fix

* fix spec and types

* add additional tests
2025-03-13 15:24:19 +00:00
Steffen SchmitzandGitHub 9ee4544e1d docs: add security notes to docker compose deployment (#5999) 2025-03-13 14:27:10 +00:00
Hassieb PakzadandGitHub 136f8a3e83 fix(llm-connections): show bedrock error (#6000) 2025-03-13 15:06:42 +01:00
Hassieb Pakzad eecf0d680b chore: release v3.41.1 2025-03-13 11:07:07 +01:00
Hassieb PakzadandGitHub d50dbed797 feat(llm-connections): allow extraHeaders for azure (#5996) 2025-03-13 09:46:38 +00:00
Steffen SchmitzandGitHub bb1dbdf824 chore: reproduce clickhouse missing surrogate key error (#5980)
* chore: reproduce clickhouse missing surrogate key error

* chore: add fix candidate

* chore: adjust test cases

* chore: adjust test case
2025-03-13 07:47:31 +00:00
Hassieb Pakzad 0991d44525 chore: release v3.41.0 2025-03-12 18:28:25 +01:00
Max DeichmannandGitHub 449c9360ac fix: search past observations with correct timestamp in evals (#5986)
fix: use readonly for the column definitions
2025-03-12 16:41:14 +00:00
Hassieb PakzadandGitHub cc4a88d6da fix(prompts): block label deletion if dependent exists (#5983)
* fix(prompts): block label deletion if dependent exists

* fix

* Update web/src/features/prompts/server/actions/updatePrompts.ts
2025-03-12 16:05:46 +00:00
Steffen SchmitzandGitHub 11b2411a5b chore: prevent sorting by environment in UI tables (#5978) 2025-03-12 13:45:25 +00:00
d8a5ebc42e chore(cloud): Make UI recognize cloud:enterprise plan and add corresponding access rights and rate limits (#5975)
* chore(cloud): Make UI recognize cloud:enterprise plan and add corresponding access rights and rate limits

- label on project selector
- rate limits same as team
- access rights same as team
- no stripe product id for this (yet), cannot be purchased self-serve in product

* remove comments

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-12 13:44:01 +00:00
Steffen SchmitzandGitHub 9042b005b8 feat: extract logfire.msg as span and trace name (#5977) 2025-03-12 13:23:33 +00:00
Hassieb PakzadandGitHub 7b70b5b678 feat(prompts): add composability (#5882) 2025-03-12 13:26:00 +01:00
Marc KlingenandGitHub c3ce043d47 chore: remove unnecessary comment (#5961) 2025-03-11 13:20:18 +00:00
Marc KlingenandGitHub 8c5737ebea chore: rename batch exports from generation to observation as table was generalized (#5950)
* chore: rename batch exports from generation to observation as table was generalized

* fix test
2025-03-11 13:15:36 +00:00
steffen911 cb928bf3fc chore: release v3.40.0 2025-03-11 13:49:18 +01:00
Steffen SchmitzandGitHub dc8a997324 feat: add score deletion to UI (#5956)
* feat: enable score deletion via Langfuse UI

* chore: add test cases

* chore: typo

* chore: implement deleteAnnotationScore

* chore: lint

* feat: add score deletion to UI

* chore: add trpc tests

* chore: allow multi-score deletion from table view

* chore: typoes and mock resets

* chore: update mock handling
2025-03-11 12:44:17 +00:00
Steffen SchmitzandGitHub 31da4bc5a5 chore: refactor score deletions into async queue (#5939)
* feat: enable score deletion via Langfuse UI

* chore: add test cases

* chore: typo

* chore: implement deleteAnnotationScore

* chore: lint
2025-03-11 08:02:13 +00:00
Max DeichmannandGitHub c0189eecad fix: use readonly for the column definitions (#5948)
* fix: use readonly for the column definitions

* fix: use readonly for the column definitions
2025-03-10 22:13:28 +00:00
Marc KlingenandGitHub a3f85e75dc docs: add zod type strict() to public api cursor rule (#5947) 2025-03-10 21:56:41 +00:00
Marc KlingenandGitHub 88ee2f3823 docs: improve public api cursor rule (#5946) 2025-03-10 21:35:38 +00:00
Max DeichmannandGitHub 0b485ad5d8 fix: fix eval status on details page (#5944)
* fix: fix eval status on details page

* push

* fix: fix eval status on details page

* fix: fix eval status on details page
2025-03-10 20:18:04 +00:00
Marc KlingenandGitHub 4e95550411 feat(datasets): show run name on runitems table (#5943)
* feat(datasets): show run name on runitems table

* perf

* push
2025-03-10 19:50:00 +00:00
Marc KlingenandGitHub ba9e26181b chore: remove models page notification (#5942) 2025-03-10 19:28:48 +00:00
Marc KlingenandGitHub 289a7210ee chore(cloud): add more data masking for posthog (#5941) 2025-03-10 19:27:36 +00:00
Max Deichmann fe0c4dc98d chore: release v3.39.0 2025-03-10 17:09:30 +01:00
Hassieb PakzadandGitHub 61c2ec268a perf(playground): memoizate ChatMessages (#5937)
* perf(playground): memoizate ChatMessages

* push

* add callbacks
2025-03-10 16:07:28 +00:00
Max DeichmannandGitHub 1df2781b59 fix: correctly extract jsonpath form observations for evals (#5938)
* fix: correctly extract jsonpath form observations for evals

* fix: correctly extract jsonpath form observations for evals
2025-03-10 15:42:44 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5478e870b3 chore(deps): bump axios from 1.7.7 to 1.8.2 (#5915)
Bumps [axios](https://github.com/axios/axios) from 1.7.7 to 1.8.2.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.7.7...v1.8.2)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-10 14:49:50 +00:00
Max DeichmannandGitHub 80c298fa1e chore: limit event id length (#5934) 2025-03-10 14:48:51 +00:00
Steffen SchmitzandGitHub 330cca6978 feat: add trace delete API endpoints (#5931)
* feat: add trace delete API endpoints

* chore: add api docs

* chore: update docs

* chore: typo

* chore: update audit logs

* chore: lint

* chore: update fern docs

* chore: fix return type
2025-03-10 14:13:33 +00:00
Steffen SchmitzandGitHub 286b78a1fe feat: add api key actions to auditlog table (#5932)
* feat: add api key actions to auditlog table

* chore: propagate audit logs to all relevant API routes

* chore: lint
2025-03-10 12:35:10 +00:00
Marc KlingenGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
f0c37c8309 chore: add initial cursor rules (#5926)
* chore: add some initial cursor rules

* Update .cursor/rules/frontend-features.mdc

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update .cursor/rules/public-api.mdc

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-09 22:54:01 +00:00
Marc KlingenandGitHub 326b27cdda fix: ui config in localstorage should persist signing out (#5916)
* fix: ui config in localstorage should persist signing out

* push

* push
2025-03-07 17:50:04 +00:00
Marc KlingenandGitHub 886e352c39 fix: select/highlight text in Compare Dataset view (#5914)
* fix: select/highlight text in Compare Dataset view

* clean diff
2025-03-07 15:59:53 +00:00
steffen911 1cd76a49f1 chore: release v3.38.0 2025-03-07 16:48:21 +01:00
Marc KlingenandGitHub 0639b78468 feat(datasets): add multi-delete for dataset runs table (#5912) 2025-03-07 15:29:50 +00:00
Steffen SchmitzandGitHub 2d6e51e219 chore: test processing of special characters in S3 (#5907)
* chore: test processing of special characters in S3

* chore: reject carriage-return character in IDs

* Update ingestion-api.servertest.ts
2025-03-07 15:20:11 +00:00
Yohan GonçalvesandGitHub 1ed6358738 feat: add customizable claim mapping for SSO (#5858) 2025-03-07 16:19:51 +01:00
4fc339dbac chore: remove redundant readme files (#5900)
chore: remoe redundant readme files

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-06 19:55:34 +00:00
Steffen SchmitzandGitHub 1e64d97b75 feat: create new environment materialized views for quick lookups (#5895)
* feat: create new environment materialized views for quick lookups

* chore: update repository

* chore: lint

* chore: add final to environments query

* chore: always include default in array
2025-03-06 19:09:28 +00:00
Steffen SchmitzandGitHub 4d06482d78 fix: enforce ints on usageDetails (#5889)
* fix: enforce ints on usageDetails

* chore: make test actually test a  generation

* chore: lint
2025-03-06 17:05:10 +00:00
Marc KlingenandGitHub 603e5ca7c9 chore(ui): add env label on env multi-select (#5890) 2025-03-06 16:37:22 +00:00
Steffen SchmitzandGitHub 61c6560f24 chore: return 200 instead of 207 as OTel success status (#5883) 2025-03-06 14:58:40 +00:00
Steffen SchmitzandGitHub 6bff022a88 chore: log dropped data items correctly in clickhouse writer (#5876)
* chore: log dropped data items correctly in clickhouse writer

* chore: update test case
2025-03-06 14:43:11 +00:00
Steffen SchmitzandGitHub 1f7cac2dd8 feat: add environment column to UI tables (#5877)
* feat: add environment column to UI tables

* feat: add environment on sessions table

* chore: add to observations

* chore: add to scores

* chore: add to users table

* chore: lint
2025-03-06 14:08:41 +00:00
Hassieb PakzadandGitHub 49bccb8fa0 fix(ui-graph-view): handle single top most node (#5881) 2025-03-06 15:15:03 +01:00
Marc KlingenandGitHub fea8a3d8d1 chore: add noindex header (#5880) 2025-03-06 13:46:00 +00:00
Max DeichmannandGitHub 08fcdb68be chore: fix contributingmd file (#5879) 2025-03-06 14:34:46 +01:00
f18a120bd8 feat: add environment filter (#5765)
* feat: add environment filter on dashboard

* chore: add a useEffect and dashboard columns for filtering

* chore: add dashboard filters

* fixup: add environment filter hook and barebone code

* chore: add traces table filter

* chore: add sessions environment filter

* chore: do observations, scores, users

* chore make filter bar dynamic and change UI placement

* chore: map environment column specially in dashboard

* chore: add column

* chore: lint

* chore: remove overflow on multi-select

* chore: only load downstream calls once environment is available

* chore: only use isLoading condition

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-03-06 10:45:18 +00:00
Max DeichmannandGitHub 2b3ca85344 chore: fix contributingmd file (#5870)
* chore: fix contributingmd file

* chore: fix contributingmd file

* chore: fix contributingmd file

* chore: fix contributingmd file

* chore: fix contributingmd file
2025-03-06 08:03:39 +00:00
Hassieb PakzadandGitHub 8b408858c4 feat(batch-exports): increase row limit to 1.5M (#5872) 2025-03-06 08:47:56 +01:00
89ad92c0ff chore: consistent casing of README files (#5865)
* docs: fix broken link in CONTRIBUTING.md

* chore: rename files to fix casing

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-03-05 23:23:54 +00:00
Marc Klingen 16e1d19917 chore: release v3.37.0 2025-03-05 23:51:35 +01:00
Marc KlingenandGitHub 18d5ba1bfa feat(auth): add AUTH_WORKOS_ORGANIZATION_ID and AUTH_WORKOS_CONNECTION_ID envs (#5866) 2025-03-05 21:31:45 +00:00
Steffen SchmitzandGitHub 95d6b9fe13 chore: add name mappings to sync prisma schema with reality (#5863) 2025-03-05 16:56:47 +00:00
Max DeichmannandGitHub 23183a71c1 chore: add missing clickhouse writer metrics (#5851)
* chore: add missing clickhouse writer metrics

* chore: add missing clickhouse writer metrics

* Update worker/src/services/ClickhouseWriter/index.ts
2025-03-05 16:37:00 +00:00
5eb0a275a5 feat: add S3_CONCURRENT_READS and S3_CONCURRENT_WRITES config options (#5824)
Co-authored-by: Steffen Schmitz <steffen@langfuse.com>
2025-03-05 16:58:26 +01:00
Steffen SchmitzandGitHub 4c5366fa20 perf: reduce trace scans for scores counts (#5860)
* perf: reduce trace scans for sessions counts

* chore: add countAll test

* chore: add session for testing
2025-03-05 15:18:10 +00:00
Marc Klingen 4ebaf0a24a chore: release v3.36.0 2025-03-05 13:32:42 +01:00
Marc KlingenandGitHub b98b90e635 feat(auth): add workos as idp (#5828)
* feat(auth): add workos as idp

* add button for org and connection based sign in

* fix

* remove client auth method and checks

* remove unnecessary diff
2025-03-05 12:29:26 +00:00
Marc Klingen 36f11a645f chore: release v3.35.1 2025-03-04 22:32:19 +01:00
Marc KlingenandGitHub 3c8b26aa79 chore(api): update api reference and postman collection (#5842) 2025-03-04 22:31:38 +01:00
Marc Klingen 86840e4119 chore: release v3.35.0 2025-03-04 22:23:24 +01:00
Steffen SchmitzandGitHub 4b412fb6ca feat: map pydantic logfire events to langfuse datamodel (#5841)
* feat: map pydantic logfire events to langfuse datamodel

* chore: map events to empty if parsing fails
2025-03-04 20:58:42 +00:00
Steffen SchmitzandGitHub 271f75aa17 chore: reduce default batch size of background migrations (#5838)
* chore: reduce default batch size of background migrations

* chore: update the cli defaults
2025-03-04 15:58:11 +00:00
Marc KlingenandGitHub 669ad4cb92 feat(datasets): deletion of dataset runs via api (#5834)
* public api

* improve public api
2025-03-04 13:41:27 +00:00
Steffen SchmitzandGitHub 9b237d4041 fix: detect openinference LLM calls as generations (#5833)
feat: detect openinference LLM calls as generations
2025-03-04 11:13:42 +00:00
marliessophieandGitHub 15b4ca01ec fix(ui): overlapping status message on trace detail page (#5830) 2025-03-04 09:07:29 +00:00
Marc KlingenandGitHub 196b5319bf feat(datasets): allow for deletion of dataset items in ui and public API (#5816)
* push

* move into drop down menu

* fix

* move copy to the right side

* fern generate

* use trpc errors
2025-03-03 14:03:18 +00:00
Steffen SchmitzandGitHub d88e0456b8 perf: reduce memory consumption on observations query (#5811)
* perf: reduce memory consumption on observations query

* chore: drop skip index list for lint
2025-03-03 13:35:40 +00:00
Marc KlingenandGitHub 78463a9ce3 feat(datasets): copy items between datasets (#5800)
* wip

* push

* push
2025-03-03 12:51:06 +00:00
marliessophieandGitHub 0bd7219eba style(ui): trace view alignment improvements (#5789)
* style(ui): trace view alignment improvements

* fix: test
2025-03-03 12:25:15 +00:00
Steffen SchmitzandGitHub a307270d8c chore: upgrade fern and add java sdk example (#5814) 2025-03-03 11:09:41 +00:00
Max DeichmannandGitHub f1555190c1 fix: use aes-gcm 12 bytes for IV (#5764)
* fix: use aes-gcm 12 bytes for IV

* fix: use aes-gcm 12 bytes for IV

* fix: use aes-gcm 12 bytes for IV
2025-03-03 10:32:22 +00:00
Max DeichmannandGitHub 56c77e0de8 chore: reduce log level for frequent logs (#5804) 2025-03-02 17:54:07 +00:00
Marc KlingenandGitHub 48cbbb694f feat(ui): add LLM connections settings page separate from api keys (#5798) 2025-03-01 19:04:33 +00:00
Marc KlingenandGitHub 168dfd2caa feat(datasets): add archive/unarchive functionality for dataset items on single item view (#5797)
* feat(datasets): add archive/unarchive functionality for dataset items on single item view

* imrpove copy
2025-03-01 19:04:23 +00:00
Marc KlingenandGitHub ae66dd9e6f fix(ui): add playsInline attribute to video players (#5795) 2025-03-01 18:33:46 +00:00
Marc KlingenandGitHub d081aab38f feat(datasets): support adding dataset items to multiple datasets (#5796)
* feat(datasets): support adding dataset items to multiple datasets

* feat(datasets): optimize bulk dataset item creation with createMany
2025-03-01 18:09:46 +00:00
Marc KlingenandGitHub a86f0ab263 fix(cloud): only active evals should count towards plan limit (#5793) 2025-03-01 12:42:39 +00:00
Marc KlingenandGitHub 738df01ab7 feat(ui): add settings deep links to cmd+k (#5680)
* proejct settings

* push

* add org and small fixes
2025-03-01 10:12:04 +00:00
marliessophieandGitHub 549eac6fd3 fix(ui): overflow behaviour on trace timeline (#5792) 2025-03-01 07:50:55 +00:00
Marc KlingenandGitHub 6d125ec274 chore(ui): remove redundant className props from splash screens (#5791) 2025-03-01 00:44:46 +00:00
Marc KlingenandGitHub 4f54577f83 chore(ui): improve perf of sessions hasAny (#5783)
chore: improve performance of sessions hasAny
2025-03-01 00:31:29 +00:00
Steffen SchmitzandGitHub 1ec1bdf849 feat: add environment properties on api routes (#5781)
* feat: add environment properties on api routes

* chore: add env filter to traces

* chore: add observation env and daily metrics env filters

* chore: add env to score endpoints

* chore: filter sessions by environment

* chore: filter traces by environment

* chore: expand trace test assertions

* chore: lint
2025-02-28 16:12:18 +00:00
Hassieb PakzadandGitHub ed5a53b10e fix(ui): llm api key form model name delete (#5786) 2025-02-28 15:57:18 +00:00
marliessophieandGitHub 5b1d9b1dd1 style(page-header): margins (#5784) 2025-02-28 14:42:19 +00:00
marliessophieandGitHub adf4508ffd feat(traces): redesign single trace UI (#5718)
* feat(traces-ui): refactor preview for traces and observations

* fix: typo in use prompt code

* feat(ui): add redesigned timeline

* feat: support all settings on timeline view

* feat: support langgraph trace

* style: langgraph view

* chore: fix other usages of single trace components

* style: comment icon color

* fix: eslint

* style: optimise for reponsive design

* chore: refactor common functions to helpers

* chore(globals): remove comments

* style: expand clickable area; adjust search string

* style: small improvements

* fix: styling of trace preview page

* fix: eslint

* fix: remove duplicated cost in observation preview

* fix: session IO formatting

* style

* style

* style: final margins
2025-02-28 13:37:07 +00:00
Steffen SchmitzandGitHub 45f8b2e97b chore: skip tracing for LLM Api key routes (#5780) 2025-02-28 10:49:40 +00:00
marliessophieandGitHub 39c49b65a4 fix(ui): overlap on dataset runs screen (#5778) 2025-02-28 10:15:26 +00:00
steffen911 d905623cb3 chore: release v3.34.1 2025-02-28 11:11:43 +01:00
Steffen SchmitzandGitHub a87a3505b8 chore: add env.HOSTNAME support for worker container (#5779) 2025-02-28 10:06:41 +00:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
591de20a72 feat: simplify otel input/output and accept string token values (#5777)
* feat: simplify otel input/output and accept string token values

* chore: typo

* Update web/src/features/otel/server/index.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* chore: also beautify event attribute fallback

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-28 09:49:49 +00:00
Marc Klingen 33d3ab80cb chore: release v3.34.0 2025-02-27 22:54:18 +01:00
Marc KlingenandGitHub 6b9773caec feat(models): add openai gpt-4.5-preview (#5772) 2025-02-27 21:47:24 +00:00
Steffen SchmitzandGitHub c90b477d40 fix: disable tracing for resetPassword route (#5763) 2025-02-27 15:36:36 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
0f448ccdde fix: always return 422 for signup error handler (#5761)
* fix: always return 422 for signup error handler

* Update web/src/features/auth-credentials/server/signupApiHandler.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update web/src/features/auth-credentials/server/signupApiHandler.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-27 15:33:41 +00:00
marliessophieandGitHub 74f93aad79 chore: move dashboard button to the right, unify eval headers (#5762)
chore: move dashboard button to the right
2025-02-27 15:15:40 +00:00
Max DeichmannandGitHub 9650e87f64 fix: standardize way to get sso blocked domains (#5760)
fix: standardise way to get sso blocked domains
2025-02-27 15:15:10 +00:00
Steffen SchmitzandGitHub c5fc30a9d0 feat: map otel environment config to langfuse environment (#5757) 2025-02-27 14:46:43 +00:00
steffen911 4a26f47971 chore: release v3.33.1 2025-02-27 15:34:24 +01:00
Steffen SchmitzGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
e3a9e61d07 perf: filter relevant observation for data set run item stats (#5752)
* perf: filter relevant observation for data set run item stats

* Update web/src/features/datasets/server/service.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-27 14:08:07 +00:00
Hassieb Pakzad a12c2a3a97 fix(model-prices): bedrock claude 3-7 version 2025-02-27 13:14:05 +01:00
Hassieb PakzadandGitHub 80681e08af fix(model-prices): bedrock claude sonnet 3-7 version (#5753) 2025-02-27 13:10:08 +01:00
steffen911 4e97494390 chore: release v3.33.0 2025-02-27 11:58:10 +01:00
Steffen SchmitzandGitHub 346e5fe7b6 feat: map environment to score within eval service (#5749)
* feat: map environment to score within eval service

* chore: adjust tests

* chore: drop unused imports
2025-02-27 10:50:36 +00:00
Steffen SchmitzandGitHub 16482f588a fix: create traces via otel for empty parent span buffer (#5748)
* fix: create traces via otel for empty parent span buffer

* chore: test range for session duration
2025-02-27 10:10:51 +00:00
Marc KlingenandGitHub f047e21e13 chore: improved domain handling and early return in signupApiHandler (#5744) 2025-02-26 21:56:20 +00:00
marliessophieandGitHub 534e832c13 fix(ui): overlapping variables for chat prompt (#5740) 2025-02-26 17:47:03 +00:00
marliessophieandGitHub 5dbfec24f0 fix(ui): experiment from the prompt screen -> window does not fit screen (#5738) 2025-02-26 17:21:05 +00:00
Hassieb PakzadandGitHub 700705be84 chore(llm-keys): update bedrock form description (#5737) 2025-02-26 17:58:49 +01:00
Hassieb Pakzad 611c31ce54 chore: release v3.32.1 2025-02-26 17:06:23 +01:00
Hassieb PakzadandGitHub bdb5ddadd9 fix(ingestion): environment to immutable keys (#5736) 2025-02-26 16:01:26 +00:00
Hassieb PakzadandGitHub 0fd1f065bd fix(api): add environment to ingestion spec (#5735) 2025-02-26 16:02:49 +01:00
Marc KlingenandGitHub 8aa063303a chore(ui): move "configure tracing" to the right in dashboard header (#5731) 2025-02-26 13:59:21 +00:00
Marc KlingenandGitHub d0ff8a87fe chore(ui): minor improvements to onboarding pages (#5730)
chore: minor improvements to onboarding pages
2025-02-26 13:56:39 +00:00
marliessophieandGitHub 0e61ffd621 style(prompts): always show edit label button for prompt versions (#5729) 2025-02-26 13:41:01 +00:00
steffen911 9f20926bc4 chore: release v3.32.0 2025-02-26 14:36:38 +01:00
Steffen SchmitzandGitHub 81a232e4a4 fix: fallback to "default" for missing environment (#5728) 2025-02-26 13:34:33 +00:00
Marc KlingenandGitHub c7e7d62f47 feat(ui): add onboarding screens for core features (#5706)
* feat: add in-ui-onboarding for tracing

* feat(ui): add onboarding screens for core features

* add annotation queues

* move videos to static, add eval onboarding

* push

* push

* push

* push

* push
2025-02-26 13:13:43 +00:00
steffen911 2aa8971dff chore: release v3.31.0 2025-02-26 13:53:58 +01:00
Steffen SchmitzandGitHub 85f4cb0c13 fix: show retention settings for self-hosters (#5727) 2025-02-26 12:34:14 +00:00
Hassieb PakzadandGitHub 4bbba9d983 feat(api): add environment property to traces, scores, obs (#5724) 2025-02-26 13:32:03 +01:00
Steffen SchmitzandGitHub 0b7dd5a1a7 feat: process environment in ingestion pipeline (#5671)
* feat: process environment in ingestion pipeline

* chore: remove unnecessary status

* chore: typing

* chore: more typing

* chore: score env prop

* chore: adjust apis to pass tests

* chore: adjust ingestion service test

* chore: restrict allowed prefix
2025-02-26 12:18:38 +00:00
Steffen SchmitzandGitHub b5cad2693f fix: exclude clickhouse default date from session duration calc (#5726) 2025-02-26 09:51:52 +00:00
Max Deichmann 58d85d66b3 chore: release v3.30.0 2025-02-26 10:30:59 +01:00
Max DeichmannandGitHub 55a5a896f1 chore: track distribution of ingested event body size (#5723)
push
2025-02-26 09:05:08 +00:00
Steffen SchmitzandGitHub 5437beec48 Revert "fix: return nulls for left-joins for correct session durations (#5697)" (#5717)
This reverts commit ce967bac12.
2025-02-25 18:14:26 +01:00
Steffen SchmitzandGitHub 418e33a333 fix: only delete media items with no remaining references (#5713)
* fix: only delete media items with no remaining references

* chore: expand filter condition
2025-02-25 15:18:02 +00:00
Hassieb PakzadandGitHub e9bbd4488f feat(playground+evals): add Google AI Studio support (#5711) 2025-02-25 14:48:22 +01:00
Steffen SchmitzandGitHub 668ca24d5e fix: support gzip encoding on otel endpoint (#5710)
* fix: support gzip encoding on otel endpoint

* chore: check for includes
2025-02-25 10:41:43 +00:00
Hassieb PakzadandGitHub 7c746c7cfc feat(models): add claude-3-7-sonnet-20250219 (#5709) 2025-02-25 10:07:41 +00:00
Hassieb Pakzad de0eb76153 chore: release v3.29.1 2025-02-24 20:53:46 +01:00
Hassieb PakzadandGitHub 85088574b2 fix(public-api): list observations with name filter (#5700) 2025-02-24 20:52:58 +01:00
Steffen SchmitzandGitHub ce967bac12 fix: return nulls for left-joins for correct session durations (#5697)
* fix: return nulls for left-joins for correct session durations

* chore: comment updates

* chore: adjust trace_count
2025-02-24 15:57:47 +00:00
Steffen SchmitzandGitHub b8d0a4bbe8 chore: remove prisma references to traces, observations, scores (#5672)
* chore: remove prisma references to traces, observations, scores

* chore: separate enum and type name

* chore: type overwrites

* chore: chore

* chore: drop views

* chore: reduce diff

* chore: diff

* chore: types gen

* chore: lint

* chore: adjust tests

* chore: remove unused objects in test utils

* chore: adjust tests

* chore: score type parsing

* chore: fix otelmapping

* chore: make observation level internal
2025-02-24 15:43:25 +00:00
marliessophieandGitHub 1e7aeb8fc5 chore(llmCompletion): support system/developer messages for o3 (#5661) 2025-02-24 09:25:33 +00:00
marliessophieandGitHub 9d5cc39cd4 style(ui): improvements to prompt details screen (#5678)
* fix: header

* fix: search always show current version

* chore: adjust code block

* chore: re-add copy in code editor if no title

* chore: do not match current version always

* adjust margins

* chore: remove breadcrumb

* fix: typo

* fix: final margin

* style: eval template

* small design nits
2025-02-22 09:16:44 +00:00
Marc KlingenandGitHub 5c5dfcbac6 chore(ui): improve consistency of prompt and eval template forms (#5679)
* chore(ui): consistency in prompt form labels

* eval prompt form
2025-02-21 18:10:12 +00:00
marliessophieandGitHub 628d180051 feat(ui): new prompt detail screen; adjust design of IO and message view components (#5537)
* style(ui): adjust design of IO and message view components

* chore: adjust final style and fix imports

* style

* nit: show "null" if no output in dataset compare view

* style: add header and tertiary colors

* chore: add Header levels

* chore: add Timeline component

* feat: make JSONView optionally scrollable

* chore: refactor TagManager styling

* style: button variants for header

* chore: ensure chatml message overflow behaves within chat message

* feat: new prompt screen outline

* chore: refactor `extractVariables` to return count

* chore: ensure robust mobile behaviour

* style: small component adjustments

* chore: refactor style of `Comment` button

* feat: add search functionality to prompt nodes

* style: restyle tags

* revert: variable extraction count

* style: prompts page overflow

* chore: control label behaviour

* labels: show buttons on hover

* fix: formatting of titles

* eslint

* push

* push

* fix: overflows

* push

* diabled view
2025-02-21 13:46:59 +00:00
Max Deichmann 57708f0219 chore: release v3.29.0 2025-02-21 13:42:45 +01:00
Max DeichmannandGitHub 5d30aefcb3 fix: do not allow to deactivate eval configs which are only on existing objects (#5675)
* fix: do not allow to deactivate eval configs which rano on historic traces only

* fix: do not allow to deactivate eval configs which rano on historic traces only

* fix: do not allow to deactivate eval configs which rano on historic traces only

* fix: do not allow to deactivate eval configs which rano on historic traces only

* fix: do not allow to deactivate eval configs which rano on historic traces only
2025-02-21 12:40:18 +00:00
Max DeichmannandGitHub dd14e11535 perf: reduce eval create queue concurrency (#5665) 2025-02-20 19:24:38 +00:00
Max DeichmannandGitHub 5f87980013 fix: do not return all executions for eval config list trpc (#5663)
* fix: do not return all executions for eval config list trpc

* fix: do not return all executions for eval config list trpc

* fix: do not return all executions for eval config list trpc

* fix: do not return all executions for eval config list trpc
2025-02-20 17:56:38 +00:00
Max DeichmannandGitHub 2edc2487ae perf: limit concurrency for eval job creations (#5660)
* push

* push
2025-02-20 16:05:41 +00:00
Steffen SchmitzandGitHub 18508187f6 chore: add environment column to trace_sessions (#5656)
* chore: add environment column to trace_sessions

* chore: update test
2025-02-20 15:06:39 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
eb5ee957e6 feat: apply evals on history of datasets + traces (#5488)
* push

* push

* push

* push

* fix

* fix

* add tests

* add tests

* add tests

* add tests

* add tests

* add tests

* fic

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* push

* push

* Update web/src/ee/features/evals/components/evaluator-form.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* push

* fix: fix dompurify

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-20 13:59:14 +00:00
Steffen SchmitzandGitHub 582f150b72 chore: add environment columns in clickhouse (#5649)
* chore: add environment columns in clickhouse

* chore: replace logger.warning with logger.warn
2025-02-20 13:05:35 +00:00
Steffen SchmitzandGitHub 7ce4e24cab chore: replace logger.warning with logger.warn (#5650) 2025-02-20 12:12:31 +00:00
marliessophieandGitHub b2d3080790 chore(prompts): support developer message (#5639)
* chore(prompts): support developer message

* chore: fix eslint config

* push
2025-02-20 09:40:32 +00:00
steffen911 3590608d30 chore: increase event log deletion timeout to 5min 2025-02-19 15:56:11 +01:00
Steffen SchmitzandGitHub 78e6c882ba chore: reduce attempts and increase delay on trace delete errors (#5637) 2025-02-19 14:24:28 +00:00
Steffen SchmitzandGitHub d5ea44b33c perf: only validate response schema on dev servers (#5630) 2025-02-19 13:11:42 +00:00
Steffen SchmitzandGitHub 8b601eed20 fix: re-enable trace deletion and remove trace S3 files (#5594)
* fix: re-enable trace deletion and remove trace S3 files

* chore: lint

* chore: lint

* chore: increase query timeout

* chore: reduce query result to only return event log

* chore: keep condition pattern

* chore: revert entitlement changes

* chore: move permission check

* chore: explain semi join and shift entitlement comment
2025-02-19 12:45:37 +00:00
Steffen SchmitzandGitHub caebc92051 chore: bump static page generation timeout (#5621) 2025-02-18 14:44:33 +00:00
Steffen SchmitzandGitHub e41f1c5e5b build: notify slack on failed deployment (#5617) 2025-02-18 14:40:29 +01:00
Steffen SchmitzandGitHub c419fd69fa chore: remove superfluous container_name overwrites (#5615) 2025-02-18 13:03:45 +00:00
Steffen SchmitzandGitHub 485dc94444 feat: map langfuse data model to further otel attributes (#5614)
* feat: map langfuse data model to further otel attributes

* chore: expand tests
2025-02-18 11:29:17 +00:00
Steffen SchmitzandGitHub c233c7fa8d feat: parse mlflow attributes on otel spans (#5609)
* feat: parse mlflow attributes on otel spans

* chore: correct failing test
2025-02-18 09:07:12 +00:00
marliessophieandGitHub 0263b40987 perf(sessions): display only first 10 users; show rest in paginated popover (#5599)
* perf(sessions): show only up to 10 initial users and the rest in popover

* nit: fix header on traces page
2025-02-17 19:01:01 +00:00
Marc KlingenandGitHub 44621d0d1f chore: remove the unnecessary file (#5598)
chore: remove unnecessary file
2025-02-17 18:34:01 +00:00
marliessophieandGitHub 5bd891ddc7 style(traces): style traces delete button in more menu in table (#5596) 2025-02-17 14:59:12 +00:00
marliessophieandGitHub 8e02f86fa7 chore(page-header): add new tabsComponent (#5593)
* chore(page-header): add tabs component

* style: replace tabs component on datasets pages

* style: replace tabs component on evals pages

* nit: fix dataset item page formatting

* style: replace tabs component on prompts pages

* chore: adjust col behaviour on header

* eslint
2025-02-17 13:34:57 +00:00
Max DeichmannandGitHub d7582c66ee security: upgrade jsonpath plus (#5591) 2025-02-17 12:41:12 +00:00
marliessophieandGitHub 75bef08f0b style(page-header): optimise for mobile view (#5590)
* chore: fix demo banner

* fix(page-header): info icon relative positioning

* fix: header title wrap behaviour

* fix(page-header): have info icon follow title

* style: move dataset level buttons to more menu

* push

* push
2025-02-17 12:36:33 +00:00
Max Deichmann 5f7b50cea8 chore: release v3.28.3 2025-02-17 09:56:07 +01:00
Steffen SchmitzandGitHub a9757d144f chore: fallback to LLM http status for /api/chatCompletion (#5554) 2025-02-17 08:42:49 +00:00
Steffen SchmitzandGitHub 12fb26388c fix: accept further content type formats on otel endpoint (#5584) 2025-02-17 08:41:14 +00:00
Max Deichmann 95d9aef705 chore: release v3.28.2 2025-02-15 13:29:06 +01:00
Max Deichmann 15fea028c2 chore: release v3.28.1 2025-02-15 01:04:28 +01:00
Max DeichmannandGitHub ac6e94ee3c security: upgrade release-it (#5558)
* security: upgrade release-it

* security: upgrade release-it

* security: upgrade release-it
2025-02-14 18:59:53 +00:00
Max DeichmannandGitHub e7332bd1e5 security: upgrade dompurify (#5557) 2025-02-14 19:39:59 +01:00
Max Deichmann fe8b2ba7ae chore: release v3.28.0 2025-02-14 19:20:25 +01:00
marliessophieandGitHub 652e96ee9d fix(header): breakpoint header behaviour (#5555)
fix(header): fix breakpoint header behaviour
2025-02-14 17:49:13 +00:00
Steffen SchmitzandGitHub 40ce493b06 fix: change numeric type parsing for sessions table filters (#5553) 2025-02-14 16:21:43 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
aa7ba1a9c5 chore: add dashboard time filter test case (#5526)
* add tests

* Update web/src/__tests__/async/repositories/dashboard-repository.servertest.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-14 15:22:15 +00:00
Jannik MaierhöferandGitHub 869b027110 docs: fix gif on Readme (#5502) 2025-02-14 15:18:05 +00:00
Hassieb PakzadandGitHub 1243d9022e feat(trace-ui): add graph view (#5511)
* feat(trace-ui): add graph view

* push

* rename

* fix build error
2025-02-14 15:08:15 +00:00
marliessophieandGitHub d988c07d46 style(header): adjust minor paddings and relative scroll behaviour (#5551)
* style(header): relative positioning on scroll page

* style: add padding to container layout

* chore: rm top margin
2025-02-14 14:27:46 +00:00
0a53682ed6 chore: Update README.ja.md (#5545)
Update README.ja.md

modification in Japanese and put Japanese support info.

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-02-14 14:24:15 +00:00
Steffen SchmitzandGitHub ac517c1849 chore: enhance ingestion api documentation (#5550) 2025-02-14 14:17:45 +00:00
Iaroslav VolkovandGitHub 2778be5abb feat: add TLS env variable support for Redis (#5393) 2025-02-14 13:28:15 +01:00
56e4b92fd7 feat(ui): add new langfuse header (#5451)
* chore: extract PageContainer component

* chore: add ItemBadge component

* style: redesign publish trace switch

* style: redesign detail page nav

* chore: extract copybutton

* chore: wrap all pages in `PageContainer`

* chore: add settings-container; style nits

* chore: simplify langfuse header

* chore: replace session icon in itembadge

* chore: mv lg:container to settings-container level

* chore: refactor component logic in ProjectOverview

* style: make project button primary

* style: adjust light mode sidebar colors

* fix: rm unneccessary class

* fix: eslint

* chore: update readme

* chore: adjust page header to <h2/>

* chore: adjust colors

* style: button color

* style: refactor public/private highlight

* push

* chore: rename components

* style: add breadcrumb to pages

* style: add sidebar border

* style: adjust sharing logic

* push

* chore: eslint

* fix: header alignment

* push

* push

* eslint

---------

Co-authored-by: Max Deichmann <m.deichmann@tum.de>
2025-02-14 10:39:42 +00:00
Steffen SchmitzandGitHub 613126bbfb chore: add clickhouse tags to queries (#5535) (#5538)
* chore: add clickhouse tags to queries (#5535)

* chore: add clickhouse tags to queries

* chore: linting

* chore: add dashboard tags

(cherry picked from commit 95d67d65d7)

* chore: move tags to json

* chore: move to json format

* chore: fix build

* chore: remove redundant fallback
2025-02-14 10:09:08 +00:00
Max Deichmann d8738e7594 chore: release v3.27.2 2025-02-13 23:01:58 +01:00
Max DeichmannandGitHub b4f7aded4f chore: increase static page generation timeout nextjs (#5539)
* add tests

* push
2025-02-13 21:59:17 +00:00
Max Deichmann dd1ac129d1 chore: release v3.27.1 2025-02-13 20:19:28 +01:00
steffen911 7fd8c03aca Revert "chore: add clickhouse tags to queries (#5535)"
This reverts commit 95d67d65d7.
2025-02-13 19:03:13 +01:00
Steffen SchmitzandGitHub 95d67d65d7 chore: add clickhouse tags to queries (#5535)
* chore: add clickhouse tags to queries

* chore: linting

* chore: add dashboard tags
2025-02-13 15:59:54 +00:00
Hassieb PakzadandGitHub 8b7b3570e8 fix(prompts-api): handle unique constraint errors (#5534) 2025-02-13 15:34:34 +00:00
Hassieb PakzadandGitHub 7fa738693a fix(model-prices): move gemini prices to Google AI values (#5533) 2025-02-13 14:46:23 +00:00
Hassieb PakzadandGitHub b554068713 fix(batch-exports): handle retries on stalled jobs for killed workers (#5532) 2025-02-13 14:42:50 +00:00
Marc Klingen 3396cc86a2 chore: release v3.27.0 2025-02-13 15:22:28 +01:00
Marc KlingenandGitHub e41ca6baa1 fix(env): standardize AZURE_AD auth environment variables and add all to .env.prod.example (#5530)
* fix(env): standardize AZURE_AD auth environment variables and add all to .env.prod.example

* push
2025-02-13 14:05:15 +00:00
marliessophieandGitHub 2001630e4d perf(tables): truncate large I/O content preview in tables (#5531)
* perf(tables): truncate large I/O content preview in tables

* push
2025-02-13 14:01:58 +00:00
Steffen SchmitzandGitHub e9da31f26a chore: delay project delete queue to allow for remaining events to be processed (#5528) 2025-02-13 12:58:59 +00:00
Steffen SchmitzandGitHub 6dd1a12bfc chore: reduce authed request logging to debug (#5527) 2025-02-13 12:24:09 +00:00
marliessophieandGitHub 922f846ce8 chore: re-enable sentry; reverts #5505 (#5524)
* chore: re-enable sentry

* chore: revert back to original version

* chore: revert removing client config
2025-02-13 09:21:13 +00:00
marliessophieandGitHub 7f16fc353d fix(datasets): broken link in compare view (#5521) 2025-02-13 09:01:39 +00:00
marliessophieandGitHub becae4c720 chore: revert changes to session storage hook from #5314 (#5513) 2025-02-12 21:45:45 +00:00
Hassieb PakzadandGitHub fede3480d1 perf(exports): increase max row limit to 1M (#5512) 2025-02-12 22:01:14 +01:00
Steffen SchmitzandGitHub 9bf5b829a7 fix: delete event files from s3 after retention period end (#5509)
* fix: delete event files from s3 after retention period end

* chore: lint
2025-02-12 17:03:36 +00:00
marliessophieandGitHub 92d5591171 perf(tables): disable link prefetching in viewport, only prefetch on link hover (#5508) 2025-02-12 16:59:29 +00:00
40f7849444 fix: disable sentry (#5505)
* fix: disable sentry

* chore: remove usages of sentry

* dep: remove sentry dependency

* Revert "dep: remove sentry dependency"

This reverts commit 63fb207fce4fafc17993cc6be9905db3b4e5713c.

* dep: remove sentry dependency

* chore: lint

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-02-12 16:44:00 +00:00
Steffen SchmitzandGitHub 9f21b1044f chore: upgrade dd-trace-js (#5506) 2025-02-12 16:36:55 +01:00
Steffen SchmitzandGitHub 030e759c0d fix: delete media and event files from S3 on project delete (#5504)
* fix: delete media and event files from S3 on project delete

* chore: update comment

* chore: test cleanup
2025-02-12 14:07:24 +00:00
Steffen SchmitzandGitHub 90a0b67ea2 chore: propagate otel trace context to clickhouse spans (#5501) 2025-02-12 12:45:13 +00:00
Steffen SchmitzandGitHub 731958ca11 chore: add additional attributes to ingestion spans (#5499) 2025-02-12 10:25:12 +00:00
marliessophieandGitHub 9d37cc4d34 chore(csv-upload): reduce min chunk size (#5496)
* chore(csv-upload): reduce min chunk size

* push
2025-02-12 09:15:09 +00:00
marliessophieandGitHub 55e0ff4169 chore: hide trace deletion button on cloud until enabled again (#5495)
* chore: hide trace deletion button on cloud until enabled again

* docs: add reasoning
2025-02-12 08:39:35 +00:00
Steffen SchmitzandGitHub 304c4fcb7e chore: link to docs for batch export config errors (#5484) 2025-02-11 16:14:29 +00:00
Steffen SchmitzandGitHub 8ede19d263 feat: log all created s3 event files in clickhouse (#5476)
* feat: log all created s3 event files in clickhouse

* chore: remove empty module

* chore: bucket path for clickhouse events

* chore: add test

* chore: lint
2025-02-11 15:38:04 +00:00
Steffen SchmitzandGitHub eda925c686 fix: map token filters for ui table to fitting clickhouse types (#5482) 2025-02-11 13:59:19 +00:00
Jannik MaierhöferandGitHub 57e8de852e docs: add translations of README for Chinese, Japanese and Korean (#5479) 2025-02-11 13:34:23 +00:00
ClemoandGitHub 7d77c36817 Update LICENSE
add clarification
2025-02-11 13:06:38 +01:00
c821d1b2fe docs: new readme (#5446)
* docs: readme rework

* banner update

* docs: minor edits

* add star us cta

* add feature overview image

* delete image and careers

* push

---------

Co-authored-by: Clemo <121163007+clemra@users.noreply.github.com>
Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-02-11 12:04:24 +00:00
Steffen SchmitzandGitHub 0ea80605b1 chore: drop postgres events table (#5472)
* chore: drop postgres events table

* chore: celanup

* chore: drop events table
2025-02-11 11:45:43 +00:00
Steffen SchmitzandGitHub 05e16e7d00 chore: remove event_log table (#5471)
* chore: remove event_log table

* chore: add fileKey for score events
2025-02-11 11:28:41 +00:00
Max DeichmannandGitHub 67876d4e4f fix: add clickhouse final for traces grouped by name dashboard (#5474)
fix
2025-02-11 10:58:51 +00:00
marliessophieandGitHub 179751db91 style(dashboard): round to two decimals on model latencies chart (#5473) 2025-02-11 10:35:08 +00:00
marliessophieandGitHub e2164c2f00 fix(dataset-upsert): filter on dataset evals triggered in trace-upsert queue (#5353) 2025-02-11 09:38:28 +00:00
marliessophieandGitHub 778dc24324 chore(datasets): reduce max payload size in csv upload (#5466) 2025-02-11 09:02:39 +00:00
Steffen SchmitzandGitHub d48da06274 chore: instrument api authentication (#5465) 2025-02-11 08:54:49 +00:00
Steffen SchmitzandGitHub 185cb10d7c feat(otel): map traceloop.entity.input/output to input/output (#5460)
* feat(otel): map traceloop.entity.input/output to input/output in langfuse

* chore: workaround to have `null` detections parsed in snyk

* chore: add workaround source note

* chore: formatting

* chore: add replace for worker
2025-02-11 08:43:13 +00:00
Steffen SchmitzandGitHub 92d5f3524b chore: enable tracing on all platforms (#5461) 2025-02-11 07:41:16 +00:00
Steffen SchmitzandGitHub e490147d73 feat(otel): accept application/json content type spans (#5450)
* feat(otel): accept application/json content type spans

* chore: apply suggestion
2025-02-11 07:22:02 +00:00
Max Deichmann 5a7ba1fcbc chore: release v3.26.0 2025-02-10 22:44:15 +01:00
Max DeichmannandGitHub 8204fb2a24 fix: fix cost calculation discrepancies in dashboards (#5448)
* fix: fix charts

* security: upgrade jsonpath-plus

* security: upgrade jsonpath-plus

* fix

* fix

* fix

* fix

* push

* push
2025-02-10 21:13:05 +00:00
Steffen SchmitzandGitHub 2559a388aa feat(otel): add input/output and name on traces (#5449) 2025-02-10 17:22:44 +00:00
Max DeichmannandGitHub 27e1785362 security: upgrade jsonpath-plus (#5445)
* security: upgrade jsonpath-plus

* push
2025-02-10 15:25:15 +00:00
Hassieb PakzadandGitHub 3fc9cee53d fix(model-prices): gemini 2 vertex AI prices (#5444) 2025-02-10 14:28:03 +00:00
Steffen SchmitzandGitHub 0dfefd2793 perf: skip processing of already seen events in worker (#5437)
* perf: skip processing of already seen events in worker

* chore: add queue delay to make use of seen events processing
2025-02-10 09:45:45 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
25bc0fc660 perf: improve trpc session.id performance (#5431)
* perf: improve trpc session.id performance

* perf: improve trpc session.id performance

* push

* perf: improve trpc session.id performance

* Update web/src/server/api/routers/sessions.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-09 10:28:36 +00:00
Max DeichmannandGitHub ccd5ab97a4 perf: improve performance for trace.id trpc route (#5430)
* push

* fix
2025-02-08 10:54:38 +00:00
Steffen SchmitzandGitHub bf68a7edd5 feat: map additional properties for otel llm spans (#5422) 2025-02-07 15:40:20 +00:00
marliessophieandGitHub 2a697a9048 feat(tables): add select all interface to tables (#5314)
* feat(ui): extract generalized `TableActionMenu`

* chore(select-all): add queue infra

* chore(select-all): add `POST` selectAll trpc route

* fixup: register queue

* chore: improve state management

* fixup: handle select all job

* chore: handle streamed records in actions

* chore(ui): add `DataTableSelectAllBanner`

* chore: handle multi-select in interface

* chore: skip processed chunks on retries

* chore: clear select all state on route nav

* chore: extract select-column components

* feat: query options for create action

* refactor: rename router

* chore: add annotation queue item processing in select all job

* fix: TS typo

* fix: sync session storage across modals and tabs

* chore: block select all if job on table for project is in progress

* chore: fix blocking select all jobs

* chore: make specific to table name

* push

* revert: commited by mistake

* remove dead code

* fixup: test

* push

* push

* chore: simplify `TableSelectionManager`

* chore(ui): refactor `SelectAll` -> `BulkAction`

* chore(fixup): refactor `SelectAll` -> `BatchAction`

* chore: set a deterministic `jobId` on redis

* chore: drop progress indicator

* chore: refactor remove selectAll route and extend existing routes instead

* push

* chore: remove dead code

* chore: refactor extract `processPostgresTraceDelete`

* push

* fix test

* chore: add batch action capability to annotation queue

* eslint

* push

* push

* push

* push

* push

* push

* push

* fix: add trace entitlement check to trace.deleteMany

* style: show note if batch action is in progress

* push

* fix imports

* cleanup
2025-02-07 10:21:44 +00:00
marliessophieandGitHub 87a961be47 fix(gemini): pricing units (#5420)
* fix(gemini): pricing units

* push
2025-02-07 10:03:52 +00:00
Max DeichmannandGitHub 3a52c59f2a fix: do not run snyk in merge groups (#5415)
fix
2025-02-06 22:55:23 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1722703623 chore(deps): bump jsonpath-plus from 10.0.7 to 10.2.0 (#5411)
Bumps [jsonpath-plus](https://github.com/s3u/JSONPath) from 10.0.7 to 10.2.0.
- [Release notes](https://github.com/s3u/JSONPath/releases)
- [Changelog](https://github.com/JSONPath-Plus/JSONPath/blob/main/CHANGES.md)
- [Commits](https://github.com/s3u/JSONPath/compare/v10.0.7...v10.2.0)

---
updated-dependencies:
- dependency-name: jsonpath-plus
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-06 21:32:03 +00:00
Max DeichmannandGitHub 129c94c455 security: upgrade jsonpath-plus (#5409)
* fix

* fix
2025-02-06 14:24:19 +00:00
Steffen SchmitzandGitHub 23a62e3a03 chore: deduplicate postgres event list by bucketPath (#5407) 2025-02-06 13:56:46 +00:00
steffen911 d2a8d4e4b1 chore: release v3.25.0 2025-02-06 14:29:41 +01:00
Steffen SchmitzandGitHub 431f8ed941 chore: fetch file list for worker processing from s3 event log (#5405) 2025-02-06 13:08:35 +00:00
Iaroslav VolkovandGitHub f50b69cbc8 feat: add custom gid and uid support (#5355) 2025-02-06 14:06:07 +01:00
Max DeichmannandGitHub cdde86d96e fix: improve working for eval jsonpath selector (#5406)
* fix

* push
2025-02-06 12:54:02 +00:00
marliessophieandGitHub 9e84150ce0 feat(model): add playground and eval support for Gemini 2.0 Flash, Gemini 2.0 Flash-Lite, and Gemini 2.0 Pro (#5403)
* feat(model): add playground and eval support for Gemini 2.0 Flash, Gemini 2.0 Flash-Lite, and Gemini 2.0 Pro

* chore: add costs
2025-02-06 11:51:38 +00:00
Max DeichmannandGitHub a6fbceff84 feat: use json-path to select from tracing data for evals (#5402)
* push

* push

* push

* error handling
2025-02-06 11:02:57 +00:00
Max DeichmannandGitHub 3ca6dd48c0 fix: return empty results in dashboard if from is larger than to (#5389)
push
2025-02-05 16:36:22 +00:00
marliessophieandGitHub 03e80bac76 fix(ui-markdown): fallback to json view if markdown parsing is unsucessful (#5386)
fix(ui-markdown): fallback to json view if markdown parsing is unsuccessful
2025-02-05 15:13:10 +00:00
Steffen SchmitzandGitHub 005ba31740 chore: remove unused imports (#5388) 2025-02-05 15:00:49 +00:00
Max DeichmannandGitHub 727797e0ec perf: do not load all models by default in the dashboards (#5387)
* fix

* push

* fix
2025-02-05 14:31:51 +00:00
Steffen SchmitzandGitHub 123f513199 chore: optionally log s3 event bucket uploads in postgres (#5381)
* chore: optionally log s3 event bucket uploads in postgres

* chore: reduce uniqueness constraints and separate eventId from eventLog id field
2025-02-05 14:05:45 +00:00
89896c70ee fix: model aggregation for daily metrics (#5384)
* fix: model aggregation for daily metrics

* fix query

* remove comment

---------

Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com>
2025-02-05 11:23:02 +00:00
marliessophieandGitHub e52f9dd1e2 fix(dataset-evals): reference job.observationId rather than sourceObservationId (#5350)
* fix(dataset-evals): reference `job.observationId` rather than `sourceObservationId`

* chore: typo
2025-02-05 10:53:36 +00:00
marliessophieandGitHub ace69419b3 fix(ui): input changes lost on focus in dataset item edit (#5380)
* fix(ui): input changes lost on focus in dataset item edit

* push
2025-02-05 09:21:01 +00:00
marliessophieandGitHub 6f3cc8982a style(dataset-analytics): decrease chart height and width (#5379) 2025-02-05 08:55:09 +00:00
Steffen SchmitzandGitHub c612109948 chore: fix flaky traces-api test (#5370) 2025-02-04 16:43:03 +00:00
Marc Klingen 2359d7f9ae chore: release v3.24.1 2025-02-04 15:55:02 +01:00
Steffen SchmitzandGitHub 1baa0aaaed chore: decrease ingestion queue delay outside of date boundaries (#5364)
* chore: decrease ingestion queue delay outside of date boundaries

* chore: check for non-null delay instead of truthy
2025-02-04 13:33:29 +00:00
c4186102eb fix: fix flaky dataset test (#5366)
* fix(dataset-evals): reference `job.observationId` rather than `sourceObservationId`

* chore: typo

* push

* push

---------

Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
2025-02-04 13:29:27 +00:00
Steffen SchmitzandGitHub 016993c117 fix: instantiate the ratelimit redis client on first use to avoid connection errors (#5362) 2025-02-04 07:23:44 +00:00
Marc Klingen 909bdde9d8 chore: release v3.24.0 2025-02-03 20:54:10 +01:00
Marc KlingenandGitHub 6f074dc59f feat(ui): add local iso date with utc on hover and ms precision on traces (#5359)
* feat(ui): add local iso date with utc on hover and ms precision on traces

* fixes
2025-02-03 19:48:06 +00:00
Hassieb PakzadandGitHub b8bc474b83 fix(playground): allow o3 (#5357) 2025-02-03 16:16:31 +00:00
Marc KlingenandGitHub 334d34ac15 chore: add cmd_k_menu:opened event for main menu "go to..." button (#5351) 2025-02-03 13:13:55 +00:00
marliessophieandGitHub 1efb37f3c1 fix(filter-ui): support mouse wheel scroll behaviour on desktop and mobile (#5346) 2025-02-03 10:26:23 +00:00
Steffen SchmitzandGitHub 5fc87cabeb chore: upgrade prisma to version 6 (#5266)
* chore: upgrade prisma to version 6

* chore: update dockerfile

* chore: upgrade prisma
2025-02-03 10:03:07 +00:00
marliessophieandGitHub a14fb003e6 fix(score-analytics): display categorical data in dashboard (#5341) 2025-02-03 08:36:24 +00:00
Max DeichmannandGitHub e1521b4ef1 fix: add bedrock throttling exception to retries for evals (#5338)
push
2025-02-02 22:52:13 +00:00
Hassieb PakzadandGitHub fd8756ff4c chore(media): add retries on media record write (#5336) 2025-02-02 09:14:11 +01:00
Marc KlingenandGitHub d69808747c chore(models): updated price for o1-mini (#5335) 2025-02-01 13:58:44 +00:00
Max DeichmannandGitHub 854e2c60c8 fix: fix observation frontend exception (#5334)
* fix

* push

* fix

* push
2025-02-01 10:57:40 +00:00
Hassieb PakzadandGitHub 4567ed473e chore(media-api): add instrumentation (#5333) 2025-02-01 10:00:08 +01:00
Steffen SchmitzandGitHub 2abc9fed7a fix: handle multiple number types in otel parser (#5323)
* chore: add more otel logging

* chore: logging

* chore: enhance log message

* chore: handle number timestamps correctly

* chore: handle non-object intvalue
2025-02-01 07:19:55 +00:00
Marc KlingenandGitHub f6de06d695 chore: remove polynomial regex from MUSTACHE_REGEX (#5328)
fix: optimize MUSTACHE_REGEX to remove polynomial regex
2025-01-31 21:53:42 +00:00
Marc Klingen 02b9220c14 chore: release v3.23.0 2025-01-31 22:09:29 +01:00
Marc KlingenandGitHub e15a26d1de feat(models): add o3-mini and o3-mini-2025-01-31 (#5327) 2025-01-31 22:08:50 +01:00
Max Deichmann 1c5c072eed chore: release v3.22.0 2025-01-31 20:00:46 +01:00
Max DeichmannandGitHub b805b9c93f perf: improve observation list endpoint CH performance (#5325)
fix
2025-01-31 18:56:49 +00:00
Hassieb PakzadandGitHub da147ea5aa fix(media-api): handle retries on existing UUIDs (#5320) 2025-01-31 18:07:29 +01:00
Marc KlingenandGitHub 8e15c9770b chore(ui): improve sizing of chat message role button (#5321) 2025-01-31 16:12:04 +00:00
Steffen SchmitzandGitHub 799c7cf940 feat: create otel/v1/traces endpoint to accept otel spans (#5241)
* feat: create otel/v1/traces endpoint to accept otel spans

* chore: add comments

* chore: otel test

* chore: fallback for buffers

* chore: attribute parsing

* chore: empty fallback on attributes

* chore: remove otel collector

* chore: remove otlp-transformer dependency

* chore: remove unrelated test case

* chore: remove header logging

* chore: add traceloop example

* chore: update lockfile

* chore: revert lock

* chore: reduce delta

* chore: add otel specific rate limits

* chore: reuse ingestion rate-limit config

* chore: document generated code

* chore: test setup

* chore: adjust result parsing

* chore: typing

* chore: otel typing
2025-01-31 15:58:43 +00:00
Steffen SchmitzandGitHub 64d784e16e fix: retry failed connections in RateLimiter and disable offlinequeue (#5317) 2025-01-31 14:58:57 +00:00
Marc KlingenandGitHub 096467d7dd feat(ui): open cmd+k menu from main menu (#5318)
* feat(ui): open cmd+k menu from main menu

* fix lint and naming

* move provider to fix issue
2025-01-31 14:51:13 +00:00
Marc KlingenandGitHub dc995ff377 fix(ui): deduplicate prompt variables in variable preview (#5316) 2025-01-31 13:46:57 +00:00
Marc KlingenandGitHub 935aa20dda fix(ui): correct value for accent-foreground color (#5315) 2025-01-31 13:43:51 +00:00
Steffen SchmitzandGitHub 543ee3fb77 fix: disable autopipelining for rate limits to avoid uncaught exceptions (#5300)
* fix: disable autopipelining for rate limits to avoid uncaught exceptions

* chore: adjust

* chore: add shutdown logic for ratelimitservice

* chore: adjust tests

* chore: test setup
2025-01-31 10:13:13 +00:00
Max Deichmann 95c6047724 chore: release v3.21.0 2025-01-30 23:31:27 +01:00
Max DeichmannandGitHub 6e31e559f0 fix: parse inputs and outputs correctly when adding to datasets (#5307)
* fix

* fix

* fix
2025-01-30 22:18:31 +00:00
Marc KlingenandGitHub 4ec1fc53d1 feat(ui): customize codemirror theme (#5305)
* remove existing theme

* init themes

* push

* doesnt work

* hide gutters when no line numbers

* push

* push

* fix import
2025-01-30 22:13:29 +00:00
Max DeichmannandGitHub 7e24f5f42c chore: download 20 S3 ingestion events at a time (#5306) 2025-01-30 21:54:58 +01:00
Marc KlingenandGitHub fe00afa627 fix(ui): hide project-level routes in project overview main menu, disable active project in cmd+k menu (#5303) 2025-01-30 20:10:19 +00:00
Max DeichmannandGitHub 3e5dc4a551 fix: pass tracing input / output as string via trpc (#5302)
* fix

* fix

* fix

* fix
2025-01-30 19:37:46 +00:00
Marc KlingenandGitHub fa75fb9245 feat(ui): use codemirror for text area input, highlighting/linting of prompt variables (#5296)
* abstract editor to support "text" mode in addition to "json" mode

* always use CodeMirrorEditor for prompts

* minheight on editor

* upgrade codemirror and add language package

* add linting for prompts and switch to githubDark theme

* package

* use codemirror for chat prompts

* imrpove scroll behavior of new dataset form

* improve dataset item editor view

* push

* remove dep pinning

* align editor linting with current variable definition

* defauklt value to value
2025-01-30 19:36:40 +00:00
Max Deichmann a640a6a139 chore: release v3.20.0 2025-01-30 19:50:57 +01:00
Hassieb PakzadandGitHub b0c12f02ec fix(llm-api-key): overwrite default top_p on key test (#5301) 2025-01-30 18:47:28 +01:00
Max DeichmannandGitHub 774efa4eaa perf: batch read single S3 events (#5298)
* push

* fix

* push

* fix
2025-01-30 17:13:40 +00:00
Max DeichmannandGitHub a866cba228 chore: improve 429 error handling (#5282) 2025-01-30 16:46:38 +00:00
Marc KlingenandGitHub 92b6963347 feat: add posthog events to cmd+k menu (#5288)
* feat: add posthog events to cmd+k menu

* ellipsis fix
2025-01-30 17:27:33 +01:00
Steffen SchmitzandGitHub ba55837b71 chore: move create export job log to internal lib (#5295) 2025-01-30 15:50:27 +00:00
Marc KlingenandGitHub 966efa1b92 docs(tracing-users): add header tooltips to user table (#5290)
docs: add header tooltips to user table
2025-01-30 15:12:37 +00:00
Marc KlingenandGitHub dbe5b93a8a fix(ui): do not close dialogs on outside interactions by default (#5294) 2025-01-30 15:11:52 +00:00
Marc KlingenandGitHub 22371b4c32 feat(prompts): scroll selected prompt version into view (#5293)
* feat(prompts): scroll prompt versions independently from prompt itself

* feat(prompts): scroll selected prompt version into view
2025-01-30 15:00:23 +00:00
marliessophieandGitHub 8d1c381d46 fix(media-ui): fetch i/o media only for authed project member (#5284) 2025-01-30 15:57:49 +01:00
Marc KlingenandGitHub 1efce634e7 feat(prompts): scroll prompt versions independently from prompt itself (#5292) 2025-01-30 14:20:17 +00:00
Steffen SchmitzandGitHub ec46e7d68e fix: correct default timestamp order for traces table (#5291)
* fix: correct default timestamp order for traces table

* chore: correct order

* chore: typo
2025-01-30 14:18:58 +00:00
Hassieb PakzadandGitHub b302e8eb96 chore(evals): handle invalid JSON error if maxTokens too low (#5289) 2025-01-30 14:50:10 +01:00
marliessophieandGitHub e04e21bf13 fix(rbac): load trace data in shared sessions (#5283)
* fix(rbac): load trace data in shared sessions

* chore: refactor and simplify
2025-01-30 13:12:16 +00:00
Marc KlingenandGitHub 273bcddc0f feat: switch between projects/orgs via cmd+k menu (#5285)
* filter out dynamic routes

* add project deeplinks
2025-01-30 13:10:53 +00:00
Max Deichmann e2099de653 chore: release v3.19.0 2025-01-30 11:46:17 +01:00
Max DeichmannandGitHub 435a09f7df feat: add more x-axis chart ticks (#5220)
* push

* push

* push

* chore: release v2.58.0

* push

* push

* push

* fix

* push

* push

* fix
2025-01-30 10:06:28 +00:00
Steffen SchmitzandGitHub 037dbd8c9d feat: add LANGFUSE_INIT_PROJECT_RETENTION support to init data retention (#5281)
* feat: accept LANGFUSE_INIT_PROJECT_RETENTION setting

* feat: add LANGFUSE_INIT_PROJECT_RETENTION support to init data retention
2025-01-30 09:52:30 +00:00
Marc KlingenandGitHub 02a3d91c56 feat(ui): add cmd+k menu (#5276)
* rename existing command exports

* move existing to input-command

* add comment

* wip cmd k menu for navigation

* fix types and paths of nested menu items

* add support for nested menu items

* filter correctly
2025-01-30 09:38:27 +00:00
Steffen SchmitzandGitHub 8821ad5b30 chore: remove data retention beta label and allow self-host enterprise plan (#5279)
* chore: remove data retention beta label

* chore: allow data-retention for self-host enterprise
2025-01-30 09:28:45 +00:00
Marc KlingenandGitHub ae1ba493aa chore(deps): bump cmdk 1.0.4 (#5275) 2025-01-30 00:01:04 +00:00
Max DeichmannandGitHub 5d2735ee14 security: upgrade sentry (#5269)
push
2025-01-29 17:48:29 +00:00
marliessophieandGitHub 38f3336ad5 style(prompts): add note explaining prompt commit message purpose (#5268) 2025-01-29 16:19:33 +00:00
marliessophieandGitHub 6224d7ad0f fix(dataset-analytics): latency label should denote seconds (s) (#5267) 2025-01-29 15:03:39 +00:00
marliessophieandGitHub 30c8fc2239 fix(experiments): double brace in json syntax mistaken as variable (#5264)
* fix(experiments): double brace in json syntax mistaken as variable

* push
2025-01-29 13:33:51 +00:00
marliessophieandGitHub edbf16f3a6 fix(experiments): change logger.warning to logger.error call (#5263)
fix(experiments): remove invalid `logger.warning` call from service
2025-01-29 12:41:42 +00:00
marliessophieandGitHub 8d050717a9 chore: support uploads of CSV files with just one column (#5262)
* chore: support uploads of CSV files with just one column

* chore: increase preview file byte size

* push
2025-01-29 10:45:18 +00:00
74703f62c4 ci: lock in current lint warning levels (#2648)
* ci(worker): enforce linting

* chore: lock current linting levels

* chore: lock current lint warning levels

---------

Co-authored-by: steffen911 <steffen@langfuse.com>
2025-01-29 08:53:19 +00:00
Max DeichmannandGitHub 09603a1f23 security: upgrade golang github.com/golang-jwt/jwt/v4 (#5249)
* push

* push
2025-01-28 16:53:48 +00:00
Steffen SchmitzandGitHub 34eb3e3d76 perf: reduce redundant parsing operations within ingestionservice (#5243)
* perf: reduce redundant parsing operations within ingestionservice

* chore: utils

* chore: move trace record parsing

* chore: cleanup

* chore: access input output correctly

* chore: avoid mutations

* chore: test cases

* chore: typo
2025-01-28 16:42:04 +00:00
Hassieb PakzadandGitHub b307a75a9a feat(messages-ui): allow removing system message (#5248) 2025-01-28 17:44:41 +01:00
marliessophieandGitHub 9f20b3e1cd feat(prompts): support commit_message (#5197)
* chore(prompts-ui): add nullable `commit_message` col

* chore(prompts-ui): add commit message to prompts

* chore(prompts-api): add commit message to prompts v2 api

* chore: rename migration

* chore: extract commit message length const

* fix: legacy types

* push

* chore: refactor types to not extend legancy, but create base types

* feat: add commit message to prompt diff view between versions

* chore: migration final in order

* push
2025-01-28 16:32:05 +00:00
Hassieb PakzadandGitHub 4ae24f97f5 fix(ui-model-table): collapsed view (#5246) 2025-01-28 16:40:04 +01:00
Hassieb PakzadandGitHub 58858de930 chore(llm-keys): rename vertex-ai to google-vertex-ai (#5244) 2025-01-28 16:11:16 +01:00
Max Deichmann b8a9630731 chore: release v3.18.0 2025-01-27 23:56:51 +01:00
Max DeichmannandGitHub aec58b1149 fix: use correct path for prompt versions api (#5236)
* push

* push
2025-01-27 22:54:50 +00:00
marliessophieandGitHub d3634917d4 chore(datasets): increase csv upload limit to 10MB (#5234) 2025-01-27 21:33:23 +00:00
Max DeichmannandGitHub 52fa9eb1ff fix: align naming of prompt patch docs (#5233)
push
2025-01-27 16:12:52 +00:00
Marc KlingenandGitHub e27f0bdc1d fix(cloud): fallback to clikchouse when usage meter cannot be fetched from stripe (#5232)
* fix(cloud): fallback to clikchouse when usage meter cannot be fetched from stripe

* use error
2025-01-27 15:28:51 +00:00
marliessophieandGitHub 0576f01641 fix(experiments): improve error handling (#5209)
* fixup: execute trace creation callback handler

* chore: create trace for experimentation on error

* push

* chore: adjust test to new behaviour

* push

* push
2025-01-27 15:23:15 +00:00
Hassieb PakzadandGitHub 9f6fccfbeb fix(evals): remove system message (#5231) 2025-01-27 16:07:39 +01:00
Marc KlingenandGitHub 7cb9b75897 fix(ui): tabIndex on sign in page to skip password reset and cloud region info (#5230) 2025-01-27 14:36:46 +00:00
Marc KlingenandGitHub 5857a9e61d feat(ui): scroll deeplinked observation into view in trace tree (#5228)
* refactor component to single out the node

* push
2025-01-27 14:22:14 +00:00
Hassieb Pakzad a9a9136010 chore: release v3.17.1 2025-01-27 15:32:23 +01:00
Hassieb PakzadandGitHub e5c45e381e fix(llm-api-key): extra header JSON parsing (#5229) 2025-01-27 15:19:24 +01:00
Steffen SchmitzandGitHub cb00d463c7 feat: remove data beyond retention limit from clickhouse (#5226)
* feat: remove data beyond retention limit from clickhouse

* chore: lint

* chore: log statement

* chore: typo
2025-01-27 13:37:56 +00:00
Steffen SchmitzandGitHub e18ac03b44 perf: use final for traces and observations api if no skip indexes are filtered (#5223)
* perf: use final for traces and observations api if no skip indexes are filtered

* chore: operator order
2025-01-27 13:16:10 +00:00
Marc Klingen d54cbc6c9b chore: release v3.17.0 2025-01-27 13:38:42 +01:00
Marc KlingenandGitHub 332f6658d9 ci: run codespell also on v2 branch and prs (#5224) 2025-01-27 13:20:58 +01:00
Hassieb PakzadandGitHub 25a559293b fix(media): base id gen on project ID + content hash (#5222) 2025-01-27 11:57:47 +01:00
Max DeichmannandGitHub 9ff8643489 chore: add client side logging for auth errors (#5221)
push
2025-01-27 10:03:02 +00:00
Steffen SchmitzandGitHub dd7e4f344e fix: align input/output and metadata parsing between GET /traces and /traces/:id (#5218)
* fix: align input/output and metadata parsing between GET /traces and /traces/:id

* chore: lint

* chore: lint
2025-01-27 09:53:33 +00:00
Marc KlingenandGitHub fe089e15f2 fix(openapi): upgrade openapi spec generator to correctly specify /ingestion 207 status code (#5217) 2025-01-27 10:44:33 +01:00
Marc KlingenandGitHub 9f8793e26a feat(auth): make checks and auth method configurable across SSO providers (#5203)
push
2025-01-27 09:12:58 +00:00
Hassieb PakzadandGitHub a1cb4c727b fix(media): remove transactions (#5211) 2025-01-25 23:44:34 +01:00
Hassieb PakzadandGitHub b39eb5445a fix(media): reduce tx level to RepeateableRead (#5210) 2025-01-25 21:09:14 +01:00
Hassieb PakzadandGitHub b611301658 fix(media-api): increase tx maxWait (#5208) 2025-01-25 16:12:34 +01:00
Hassieb PakzadandGitHub c3fd8a74dc fix(trace-timeline): first token label (#5202) 2025-01-24 16:38:47 +01:00
steffen911 aff8421dde chore: release v3.16.0 2025-01-24 16:12:52 +01:00
Steffen SchmitzandGitHub c442c4290e feat: add proxy support for oauth flows (#5198) 2025-01-24 14:32:46 +00:00
Max Deichmann 01bc983b97 chore: release v3.15.0 2025-01-24 15:02:59 +01:00
Max DeichmannandGitHub e2015bc52b feat: update prompt versions (#5106)
* feat: update prompt versions

* push

* fix: return 404 error instead of 500 for not found observations trpc route

* fix

* fix

* fix

* fix

* fix

* feat: update prompts and invalidate cache

* feat: update prompts and invalidate cache

* feat: update prompts and invalidate cache

* feat: update prompts and invalidate cache
2025-01-24 13:40:59 +00:00
Max DeichmannandGitHub 91589d20c6 chore: add error logs for trpc add prompts label (#5195)
push
2025-01-24 12:45:00 +00:00
50f9444c36 feat(auth): Add AUTH_CUSTOM_ID_TOKEN environment variable (#5193)
* Add AUTH_CUSTOM_ID_TOKEN environment variable

* add to multi-tenant-sso (ee)

* add comment

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-01-24 11:14:39 +00:00
marliessophieandGitHub 4432223fd2 fix(comments-api): increase comment character limit to 3,000 (#5194) 2025-01-24 10:59:25 +00:00
marliessophieandGitHub e3d58ff920 feat(traces): display number of all log kinds on trace table (#5165)
* feat(traces): display number of all log kinds on trace table

* chore: lint

* tests: adjust

* push

* chore: rename `computed_level` to `aggregated_level`

* chore: adjust ui labels

* chore: include level counts in export
2025-01-24 09:57:21 +00:00
Hassieb PakzadandGitHub ded00e6fdf feat(trace-timeline): add time to first token (#5185) 2025-01-24 10:02:05 +01:00
Max DeichmannandGitHub 3fd1cc021c perf: add timestamp filter to observations join in eval queue (#5192)
* push

* chore: disable secondary queue
2025-01-23 23:50:40 +00:00
Steffen SchmitzandGitHub 4b2e68017b feat: add support for media asset deletion based on retention settings (#5181)
* feat: allow data retention configuration in project settings

* chore: remove prefix

* chore: hide data retention on cloud

* chore: lint

* feat: add support for media asset deletion based on retention settings

* chore: fix naming

* chore: increase expiry schedule

* chore: add logging

* chore: revert to intended schedule
2025-01-23 16:54:33 +00:00
Marc KlingenandGitHub a52f87c64a chore(cloud): add login issue banner, allow to open support chat (#5183) 2025-01-23 16:07:03 +00:00
Hassieb Pakzad 7500575a96 chore: release v3.14.0 2025-01-23 16:24:10 +01:00
Steffen SchmitzandGitHub 105284d756 feat: allow data retention configuration in project settings (#5178)
* feat: allow data retention configuration in project settings

* chore: remove prefix

* chore: hide data retention on cloud

* chore: lint

* chore: disable box on missing entitlement and add integer enforcement
2025-01-23 14:42:24 +00:00
Hassieb PakzadandGitHub 29f7ace2ce feat(llm-api-keys): allow for extra headers (#5167) 2025-01-23 14:59:00 +01:00
Hassieb PakzadandGitHub 8fd17b52d1 fix(prompt-diff-dialog): keep open on cursor leaving window (#5176) 2025-01-23 10:44:07 +01:00
marliessophieandGitHub 484af10463 feat(prompts): support in UI prompt duplication (#5110)
* feat(prompts): support in UI prompt duplication

* chore(ui-ds): make `children` in `ActionButton` optional

* fix: extract custom hook for prompt name validation

* eslint

* fix: insert correct prompt count

* chore: validate prompt name uniqueness; attach latest label for single version

* chore: refactor duplication approach to use createMany and save on network-roundtrips

* chore: revert changes on action button

* push
2025-01-22 17:53:09 +00:00
Hassieb PakzadandGitHub 33d43ca9bd fix(diff-view-ui): add line breaks (#5168) 2025-01-22 17:27:58 +01:00
Hassieb PakzadandGitHub 3658620540 feat(prompts): add diff view on new prompt version (#5163) 2025-01-22 15:54:06 +01:00
Steffen SchmitzandGitHub 83e1c5a3df chore: use a distribution metric for langfuse.ingestion.count_files (#5164)
* chore: use a distribution metric for langfuse.ingestion.count_files

* chore: update name

* chore: update name

* chore: remove outdated import
2025-01-22 11:53:31 +00:00
Steffen SchmitzandGitHub 5efcef593f chore: add additional metrics into ingestion pipeline (#5160)
* chore: add additional metrics into ingestion pipeline

* chore: spellcheck
2025-01-22 11:03:56 +00:00
Marc KlingenandGitHub 6c84c54a4a feat(cloud): switch between available plans (#5153)
* feat(cloud): switch between available plans

* push

* inline comments

* entitlement check

* improve error message

* remove unused ids

* fix spellcheck
2025-01-22 10:38:25 +00:00
Max DeichmannandGitHub 9b3d77e1de chore: disable secondary queue (#5156) 2025-01-21 23:25:56 +01:00
steffen911 f10e896545 chore: release v3.13.0 2025-01-21 17:10:01 +01:00
0047cbb367 fix: use unix timestamp for batch export file name to avoid invalid characters (#5092)
Co-authored-by: hopkinx <hopkinx@yanxihuadeMac-mini.local>
2025-01-21 17:08:33 +01:00
Marc KlingenandGitHub 8df120c186 feat(ee): add audit log view to project settings (#5150)
* add audit logs table

* push

* fix pagination zod schema

* copy

* add check that user is in org

* simplify
2025-01-21 15:10:22 +00:00
Max DeichmannandGitHub a12b5b5006 fix: correctly upgrade katex (#5151) 2025-01-21 14:57:48 +00:00
marliessophieandGitHub 91f360ae29 chore(models-ui): move models into settings (#5124)
* chore(models-ui): move models into settings

* style: add notification to manage models in settings

* chore: mv button
2025-01-21 14:54:58 +00:00
Steffen SchmitzandGitHub 026f7f1b4b chore: allow skipping ingestion CH reads for projects created after cutoff date (#5116)
* chore: allow skipping ingestion CH reads for projects created after cutoff date

* chore: add in memory cache for project skips
2025-01-21 14:08:36 +00:00
Max DeichmannandGitHub 29ad2965e6 security: upgrade katex (#5149) 2025-01-21 14:01:37 +00:00
Steffen SchmitzandGitHub ae3f84125b chore: remove legacy/batched ingestion pipeline code (#5121)
* chore: remove legacy/batched ingestion pipeline code

* chore: remove js-tiktoken dependency

* chore: partially revert updates

* chore: typing in gitHubEnterpriseProvider.ts

* chore: remove model-match test
2025-01-21 12:57:43 +00:00
Steffen SchmitzandGitHub 507a0eb7f4 chore: skip re-builds for same tree hash (#5135)
* chore: skip re-builds for same tree hash

* dummy

* chore: include do not skip

* chore: skip push
2025-01-21 11:58:10 +00:00
Max DeichmannandGitHub 64b29bce24 fix: observation table type alignment (#5146) 2025-01-21 11:27:50 +00:00
marliessophieandGitHub 60d6c5359b feat(dataset-runs): add product analytics events (#5145)
* feat(dataset-runs): add product analytics events

* push

* push
2025-01-21 11:25:45 +00:00
Steffen SchmitzandGitHub 7f94a29c48 chore: reduce default write interval for CH queue to 1000 (#5144) 2025-01-21 11:19:44 +00:00
steffen911 a26f16b090 chore: release v3.12.0
CI/CD / lint (push) Waiting to run
CI/CD / test-docker-build (push) Waiting to run
CI/CD / tests-web-sync (node20, pg12) (push) Waiting to run
CI/CD / tests-web-sync (node20, pg15) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / e2e-tests (push) Waiting to run
CI/CD / e2e-server-tests (push) Waiting to run
CI/CD / all-ci-passed (push) Blocked by required conditions
CI/CD / push-docker-image (push) Blocked by required conditions
release.yml / release (push) Waiting to run
2025-01-21 12:28:38 +01:00
Steffen SchmitzandGitHub b34da5a0de feat: add support for DATABASE_ARGS config (#5143) 2025-01-21 11:14:39 +00:00
marliessophieandGitHub 9684768346 chore(traces): remove unsued code pulling trace data from prisma [deprecated] (#5139) 2025-01-21 10:24:27 +00:00
Steffen SchmitzandGitHub 1c51a319fd chore: remove postgres fallback for generations table calls (#5137) 2025-01-21 10:55:54 +01:00
Steffen SchmitzandGitHub 9dd7aa833b chore: remove queryClickhouse toggles (#5123)
* chore: remove queryClickhouse toggles

* chore: linting
2025-01-21 08:01:57 +00:00
Marc KlingenandGitHub e14f8587af revert: ask for github star during onboarding (#5127)
Revert "feat(onboarding): add GitHub star prompt with badge (#5125)"

This reverts commit ed743c1f76.
2025-01-20 21:04:24 +00:00
ed743c1f76 feat(onboarding): add GitHub star prompt with badge (#5125)
* feat(onboarding): add GitHub star prompt with badge

* ui improvements

* add posthog tracking

---------

Co-authored-by: Marc Klingen <git@marcklingen.com>
2025-01-20 19:27:46 +00:00
Hassieb PakzadandGitHub 0ee21fb98e fix(llm-api-keys): improve form dialog for bedrock (#5122) 2025-01-20 18:28:09 +01:00
Hassieb PakzadandGitHub 78d74ccb4e fix(trace-view): show disabled jump to playground instead of hiding (#5120) 2025-01-20 18:05:38 +01:00
Hassieb PakzadandGitHub 4cb5af71d3 fix(ingestion): allow nullish tokens_details in openai usage schema (#5117) 2025-01-20 18:03:51 +01:00
steffen911 7111ebfbeb chore: release v3.11.1
CI/CD / lint (push) Waiting to run
CI/CD / test-docker-build (push) Waiting to run
CI/CD / tests-web-sync (node20, pg12) (push) Waiting to run
CI/CD / tests-web-sync (node20, pg15) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / e2e-tests (push) Waiting to run
CI/CD / e2e-server-tests (push) Waiting to run
CI/CD / all-ci-passed (push) Blocked by required conditions
CI/CD / push-docker-image (push) Blocked by required conditions
release.yml / release (push) Waiting to run
2025-01-20 17:02:28 +01:00
Steffen SchmitzandGitHub a9506b40ab fix: allow field overwrites with null in ingestion api (#5114)
* fix: allow field overwrites with `null` in ingestion api

* chore: add undefined check

* chore: check undefined does not overwrite value

* chore: explicit undefined check
2025-01-20 15:24:09 +00:00
Max Deichmann 3cd82c97cf chore: release v3.11.0
CI/CD / lint (push) Waiting to run
CI/CD / test-docker-build (push) Waiting to run
CI/CD / tests-web-sync (node20, pg12) (push) Waiting to run
CI/CD / tests-web-sync (node20, pg15) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / e2e-tests (push) Waiting to run
CI/CD / e2e-server-tests (push) Waiting to run
CI/CD / all-ci-passed (push) Blocked by required conditions
CI/CD / push-docker-image (push) Blocked by required conditions
release.yml / release (push) Waiting to run
2025-01-20 15:49:51 +01:00
Max DeichmannandGitHub f55bee2d36 fix: route all links to new observations table (#5112) 2025-01-20 15:32:54 +01:00
Max DeichmannandGitHub 5391dd90f4 fix: correct the rate limiter header return types (#5111) 2025-01-20 15:16:56 +01:00
Max DeichmannandGitHub a1e670a729 feat: convert generations to observations table (#5099)
* feat: convert generations to observations table

* feat: add type col to generations table

* feat: add type col to generations table

* feat: add type col to generations table

* feat: add type col to generations table

* feat: add type col to generations table

* fix: fix
2025-01-20 13:40:41 +00:00
Max DeichmannandGitHub b6e21fc44b fix: return 404 error instead of 500 for not found observations trpc … (#5105)
fix: return 404 error instead of 500 for not found observations trpc route
2025-01-19 14:23:15 +01:00
Max DeichmannandGitHub eb59082bf7 fix: flaky session export test (#5103) 2025-01-18 19:03:57 +00:00
Max Deichmann 950f36b87d chore: release v3.10.0
CI/CD / lint (push) Waiting to run
CI/CD / test-docker-build (push) Waiting to run
CI/CD / tests-web-sync (node20, pg12) (push) Waiting to run
CI/CD / tests-web-sync (node20, pg15) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-web-async (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-web-async (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode) (push) Waiting to run
CI/CD / tests-worker (node20, pg12, mode-azure) (push) Waiting to run
CI/CD / tests-worker (node20, pg15, mode-azure) (push) Waiting to run
CI/CD / e2e-tests (push) Waiting to run
CI/CD / e2e-server-tests (push) Waiting to run
CI/CD / all-ci-passed (push) Blocked by required conditions
CI/CD / push-docker-image (push) Blocked by required conditions
release.yml / release (push) Waiting to run
2025-01-18 19:55:56 +01:00
Max DeichmannandGitHub c55b70d119 fix: fix session ui table ordering (#5102) 2025-01-18 19:55:26 +01:00
Marc KlingenandGitHub c1109c91f1 feat(ui): single user view, simpler metrics, link to dashboard (#5097) 2025-01-17 21:14:42 +00:00
Marc KlingenandGitHub 3b737934f9 chore(cloud): run analytics jobs daily (#5096) 2025-01-17 18:14:22 +00:00
Steffen SchmitzandGitHub a12f3f675e fix: update ordering for get traces calls to avoid stale results (#5090)
* fix: update ordering for get traces calls to avoid stale results

* chore: explanation

* chore: remove stale test

* chore: lint

* chore: update comment

* chore: cleanup
2025-01-17 16:30:07 +00:00
Marc KlingenandGitHub 30d5755c24 fix(ui): support menu button width (#5094)
push
2025-01-17 15:04:49 +00:00
Marc KlingenandGitHub 39762cb1c0 feat(cloud): export billing meter backup to s3 (#5091) 2025-01-17 15:22:41 +01:00
marliessophieandGitHub 8ac4176aaf feat(datasets): upload dataset via csv import (#4656)
* fixup: add csv import for datasets

* feat: include DnD logic

* add: fastest-levenshtein dep

* fixup: upload & parse .csv file

* fixup: uploading file

* fix: allow uploading .csv file

* chore: add csv-parse

* eslint

* chore: include project id in temp storage logic

* chore: access checks

* push

* chore: use s3 signed url to upload .csv file temporarily

fixup: env credentials

* deps: fix circular dep

* fixup

* create import bucket for uploads

* push

* push

* push

* remove s3 config for import

* remove server side processing

* fix: metadata display for items

* style: fix css styling and add progress bar

* push

* push

* push

* refactor: extract helper function

* wrap in try catch block

* fix: button logic in case of no access

* analytics: add posthog events
2025-01-17 13:51:25 +00:00
marliessophieandGitHub f87af4f675 feat(model): add playground and eval support for gemini-2.0-flash-exp (#5087) 2025-01-17 13:19:06 +00:00
Max Deichmann bc323772c1 chore: release v3.9.0 2025-01-17 12:26:06 +01:00
Marc KlingenandGitHub fcef737dc0 feat(cloud): snapshot usage metering to postgres db (#5075) 2025-01-17 12:13:46 +01:00
marliessophieandGitHub 4bf77d68ed feat(model): add support for o1 model costs (#5073)
* feat(model): add support for o1 model costs

* order cost list by ID

* push

* update contributing

* push
2025-01-17 10:23:25 +00:00
Steffen SchmitzandGitHub d7ec924ed8 chore: remove outdated postgres executions (#5069) 2025-01-17 11:10:20 +01:00
Marc KlingenandGitHub cefaf56613 chore(cloud): region details (#5080) 2025-01-17 00:18:00 +01:00
Max DeichmannandGitHub 6aff9b0f4a fix: quickly recover from metering queue errors (#5077)
fix: quickly reciver from metering queue errors
2025-01-16 22:11:55 +01:00
Max DeichmannandGitHub 815de149aa fix: retry stripe apis (#5076)
push
2025-01-16 21:57:05 +01:00
Max DeichmannandGitHub bc8923485e fix: add race condition check to billing queue (#5074) 2025-01-16 20:40:59 +01:00
Max DeichmannandGitHub 1a60425692 fix: retry stripe in metering logic and meter again if error ocurred (#5072) 2025-01-16 19:42:10 +01:00
marliessophieandGitHub 1d3ac98123 style(ui): improve observation type label colors for readbility, accessibility (#5071)
* style(ui): adjust muted colors for readbility, accessibility

* style(ui): adjust comment labels given type

* margins

* push
2025-01-16 16:27:47 +00:00
Max DeichmannandGitHub de67ec1a02 chore: add cloud meter event trigger (#5068)
* push

* push

* feat: add metering event
2025-01-16 15:22:59 +00:00
marliessophieandGitHub 9510995a67 fix(ui-tables): populate filter operator by default (#5067) 2025-01-16 14:21:48 +00:00
Marc KlingenandGitHub 4623d76155 fix(cloud): remove race condition check in metering cron job (#5066) 2025-01-16 11:31:00 +01:00
Steffen SchmitzandGitHub 6a6abc6bd8 chore: skip zod parse for json parsing (#5063)
* chore: skip zod parse for json parsing

* chore: lint

* chore: add JsonNested type

* chore: add one more test for invalid json
2025-01-16 09:32:41 +00:00
Max DeichmannandGitHub ddee827e22 chore: update architecture diagram (#5055) 2025-01-15 22:07:16 +00:00
Max DeichmannandGitHub e327db8bbd chore: correct data region (#5054) 2025-01-15 22:01:58 +00:00
Max DeichmannandGitHub 9229d759e1 chore: add project_id to get clickhouse query ingestion (#5051) 2025-01-15 20:09:53 +01:00
Hassieb PakzadandGitHub d8d8ab7060 fix(jump-to-playground): hide button if message parsing will fail (#5048) 2025-01-15 16:55:58 +01:00
Steffen SchmitzandGitHub b851359923 chore: remove legacy v3-migration env variables (#5041)
* chore: remove legacy v3-migration env variables

* chore: linting

* chore: move legacy files to current env

* chore: remove outdated tests

* chore: add async ingestion test

* chore: reduce delay for tests

* chore: lint

* chore: skip scores and enable redis on CI

* chore: remove outdated test classes
2025-01-15 15:39:20 +00:00
marliessophieandGitHub b36ea74314 fix(ui-annoation): bug when re-submitting score comments on categorical scores (#5045)
fix(ui-annoation): re-submit score comments on categorical scores
2025-01-15 15:12:46 +00:00
Hassieb PakzadandGitHub b2f103549a feat(trace-tree-iu): add hint for hidden obs (#5044) 2025-01-15 14:47:39 +01:00
marliessophieandGitHub f660b80438 style(ui-tables): fix position of No results. text if table has no data (#5043)
* style(ui-tables): fix position of `No results.` text if table has no data

* fix: dom nesting
2025-01-15 13:18:23 +00:00
Steffen SchmitzandGitHub 5bbfffef50 chore: allow any value for input/output in ingestion endpoint (#5040)
* chore: allow any value for input/output in ingestion endpoint

* chore: add explanation
2025-01-15 10:36:13 +00:00
Hassieb PakzadandGitHub cb6f95dc93 feat(trace-detail-ui): filter by minObservationlvl (#5039) 2025-01-15 11:40:57 +01:00
Max Deichmann 8345ce48fc chore: release v3.8.0 2025-01-15 10:26:15 +01:00
Max DeichmannandGitHub f71d28900a perf: use streaming for observation and trace list endpoints (#5038)
push
2025-01-15 10:25:28 +01:00
Hassieb PakzadandGitHub 14eaf62e45 fix(ingestion): overwriting of observation level (#5027) 2025-01-15 09:43:41 +01:00
Steffen SchmitzandGitHub f8e61c2659 chore: fallback to error.status for eval error handling (#5037) 2025-01-15 07:40:36 +00:00
marliessophieandGitHub 3b205b45ce chore(ui-dashboard): save selected score graphs in the dashboard to localStorage (project-bound) (#5029)
* chore(ui-dashboard): save selected score graphs in the dashboard to `localStorage` (project-bound)

* add comment

* push
2025-01-14 19:14:17 +00:00
Max DeichmannandGitHub 7260c44486 fix: correctly order sessions by time (#5028)
* fix: correctly order sessions by time

* fix: correctly order sessions by time

* fix: correctly order sessions by time
2025-01-14 19:00:53 +00:00
marliessophieandGitHub 92ec2b60e2 feat(ui-tabels): Count of group selection on column visibility (#5024) 2025-01-14 16:26:01 +00:00
Steffen SchmitzandGitHub a856488f2d chore: upgrade bullmq to 5.34.10 (#5022)
* chore: upgrade bullmq to 5.34.10

* chore: adjust tests
2025-01-14 13:59:55 +00:00
Steffen SchmitzandGitHub a18f09fd8d fix: process status codes on eval execution errors (#5013)
* fix: process status codes on eval execution errors

* chore: error handling
2025-01-14 12:29:44 +00:00
Hassieb PakzadandGitHub 2a95dd1da0 fix(openapi-models): modelusageunit to optional (#5015) 2025-01-14 10:25:07 +01:00
Hassieb PakzadandGitHub 04ebd5994f fix(fern-models): unit to optional commons.model (#5014) 2025-01-14 10:22:32 +01:00
Hassieb PakzadandGitHub 268b741a3a chore(models): update fern types startDate unit (#5004) 2025-01-13 19:53:41 +01:00
Steffen SchmitzandGitHub 8b686e9b31 fix: fail open on redis errors in ratelimitservice (#5001) 2025-01-13 17:29:17 +00:00
Max Deichmann e802a5c23a chore: release v3.7.1 2025-01-13 14:39:24 +01:00
Max DeichmannandGitHub 5180e2382e fix: drop dataset run item temp tables from all CH nodes (#4999) 2025-01-13 14:38:11 +01:00
Max DeichmannandGitHub 2cae1253d3 fix: fix session table sorting (#4995)
* fix: fix session table sorting

* push
2025-01-13 12:18:37 +00:00
ClemoandGitHub be2414a82c docs: fix typo 2025-01-13 10:54:43 +01:00
ClemoandGitHub 95e63c6f78 docs: Update careers 2025-01-13 10:53:54 +01:00
Marc Klingen 32b6502d13 chore: release v3.7.0 2025-01-12 09:59:41 +01:00
Marc KlingenandGitHub 1bc33c7b92 fix: telemetry error "invalid input syntax for type interval" (#4991) 2025-01-12 09:57:46 +01:00
Marc KlingenandGitHub 4967b20478 fix(ui): support menu forwardref error, memo list of options, keys for separators (#4989)
fix(ui): support menu forwardref error
2025-01-12 03:33:37 +00:00
Marc KlingenandGitHub 1130894cdc chore: remove old launch week notifications (#4986) 2025-01-12 03:18:13 +00:00
Marc KlingenandGitHub af276e9e78 feat(cloud): add banner for status page announcement (#4988)
* wip

* fix(ui): "chatisvisble not a function"

* csp

* feat(cloud): add banner for status page announcement
2025-01-12 03:17:45 +00:00
Marc KlingenandGitHub d69dbd81a2 fix(ui): chatIsVisible rendering issues (#4987)
fix(ui): "chatisvisble not a function"
2025-01-12 02:43:25 +00:00
Marc KlingenandGitHub 7e807c0de8 feat(annotation-queue): show "add to dataset" button on trace preview (#4985) 2025-01-12 02:24:57 +00:00
Marc KlingenandGitHub cfa2ac6da9 chore(ui): improve button labels for playground and experiments on prompt view (#4984) 2025-01-12 02:11:49 +00:00
Marc KlingenandGitHub 952a550847 feat(ui): add support dropdown menu to replace /support page (#4983) 2025-01-12 02:05:46 +00:00
Marc KlingenandGitHub 6199903117 chore(ui): add descriptions to the setup onboarding flow (#4982) 2025-01-12 00:00:19 +00:00
Marc KlingenandGitHub 88cdb8999d chore(ui): do not collapse chat messages on prompt view (#4981) 2025-01-11 23:48:35 +00:00
Marc KlingenGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
9ad2ef1f91 fix(ui): docs menu item should open in new tab (#4980)
* fix(ui): docs menu item should open in new tab

* Update web/src/components/nav-main.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-01-11 21:11:37 +00:00
Marc KlingenandGitHub 7a9717cb5e chore(ui): remove introducing orgs banner (#4979) 2025-01-11 20:59:51 +00:00
Marc KlingenandGitHub ce31a806c7 chore(cloud): improve ui of cloud region selection, remove login error notice (#4978)
chore(cloud): imrpove ui of cloud region selection, remove login error notice
2025-01-11 18:03:20 +00:00
Max DeichmannandGitHub e7e6cf1b99 feat: script to replay missing events (#4966)
* feat: script to replay missing events

* feat: script to replay missing events
2025-01-10 15:40:49 +00:00
Steffen SchmitzandGitHub 9f76457f22 chore: stream clickhouse results to posthog to reduce memory consumption (#4967)
* chore: stream clickhouse results to posthog to reduce memory consumption

* chore: end span after streaming all results

* chore: tracing for clickhouse streaming
2025-01-10 13:42:03 +00:00
1447 changed files with 211543 additions and 53639 deletions
+14
View File
@@ -0,0 +1,14 @@
{
"permissions": {
"allow": [
"Bash(find:*)",
"Bash(rg:*)",
"Bash(grep:*)",
"Bash(ls:*)",
"Bash(cat:*)",
"Bash(head:*)",
"Bash(tail:*)"
],
"deny": []
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
[codespell]
skip = .git,*.pdf,*.svg,package-lock.json,*.prisma,pnpm-lock.yaml
ignore-words-list = afterall,vertx,notIn
ignore-words-list = afterall,vertx,notIn,alue
+54
View File
@@ -0,0 +1,54 @@
FROM node:20
# ---------- System packages --------------------------------------------------
# The buildpack-deps base already ships git, build-essential, python, etc.
# Add a few extra tools handy during Langfuse development.
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
openssl \
wget \
curl \
ca-certificates \
postgresql-client \
redis-tools \
less nano \
sudo \
&& rm -rf /var/lib/apt/lists/*
# ---------- Docker -----------------------------------------------------------
# Install Docker for background agents that need container capabilities
RUN curl -fsSL https://get.docker.com -o get-docker.sh && \
sh get-docker.sh && \
rm get-docker.sh
# ---------- pnpm -------------------------------------------------------------
# Langfuse monorepo relies on pnpm 9.5.0 (see CONTRIBUTING.md)
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable && \
corepack prepare pnpm@9.5.0 --activate
# ---------- golang-migrate ----------------------------------------------------
# CLI used for database migrations during development.
ENV MIGRATE_VERSION=4.18.3
RUN wget -qO- "https://github.com/golang-migrate/migrate/releases/download/v${MIGRATE_VERSION}/migrate.linux-amd64.tar.gz" \
| tar -xz -C /usr/local/bin && \
chmod +x /usr/local/bin/migrate
# ---------- Non-root user -----------------------------------------------------
# Create non-root user with sudo privileges and docker group access
RUN useradd -ms /bin/bash ubuntu && \
usermod -aG sudo ubuntu && \
usermod -aG docker ubuntu && \
echo "ubuntu ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
# Pre-create pnpm store and set correct ownership to avoid first-run cost & permission issues
RUN pnpm store path > /dev/null && \
chown -R ubuntu:ubuntu /pnpm
USER ubuntu
WORKDIR /home/ubuntu
ENV HOME=/home/ubuntu
# Container starts with a bash shell ready for hacking.
CMD ["bash"]
+14
View File
@@ -0,0 +1,14 @@
{
"install": "cp .env.dev.example .env && pnpm i",
"build": {
"context": ".",
"dockerfile": "Dockerfile"
},
"start": "sudo service docker start",
"terminals": [
{
"name": "dev server",
"command": "pnpm run dx-f"
}
]
}
+6
View File
@@ -0,0 +1,6 @@
---
description: How to manage authorization and RBAC within full-stack langfuse features
globs: web/
alwaysApply: true
---
See [README.md](mdc:web/src/features/rbac/README.md) for details
+7
View File
@@ -0,0 +1,7 @@
---
description: How to enforce entitlements across full-stack features
globs: web/**
alwaysApply: true
---
Check [README.md](mdc:web/src/features/entitlements/README.md) to learn more about entitlements
+33
View File
@@ -0,0 +1,33 @@
---
description: How we build new full-stack features including frontend APIs, file structure, and components
globs: web/**
alwaysApply: true
---
## Framework
- Next.js
- Pages router
## File Structure
- We generally put all code related to a net-new feature into a folder within web/src/features.
- Check out other features to learn about the common structure.
## API for frontend features
- We use TRPC.io to power full-stack features of the Langfuse Frontend
- Entry point for all trpc routes: [root.ts](mdc:web/src/server/api/root.ts)
- Authentication, see [authorization-and-rbac.mdc](mdc:.cursor/rules/authorization-and-rbac.mdc)
- Entitlements, see [entitlements.mdc](mdc:.cursor/rules/entitlements.mdc)
## Components
- We use Shadcn/ui
- Components in `@/src/components/ui`
- If a component is not installed yet, ask the user to install it for you
- When creating new custom components that generalize, we add them with a generalizable naming to `@/src/components`
## Styling
- We use Tailwind CSS
- We use a standard color palette which automatically handles light/dark mode, see [globals.css](mdc:web/src/styles/globals.css)
+9
View File
@@ -0,0 +1,9 @@
---
description:
globs:
alwaysApply: true
---
# General rules
- Linting in this repo only works if the development server is running
- Always run the full mono-repo via `pnpm run dx` (use `pnpm dx-f` when you run this in a background agent). Thereby the database will also be seeded.
+20
View File
@@ -0,0 +1,20 @@
---
description:
globs:
alwaysApply: true
---
## Project setup
- This project is a Turborepo monorepo
- We build two containers, web (./web) and worker (./worker)
- We have shared code between these two in the shared package (./packages/shared). For that package, we have different entry points [package.json](mdc:packages/shared/package.json).
## Domain layer
The most important domain objects are in [observations.ts](mdc:packages/shared/src/domain/observations.ts), [traces.ts](mdc:packages/shared/src/domain/traces.ts), [scores.ts](mdc:packages/shared/src/domain/scores.ts).
## Database schema
We use Postgres and Clickhouse.
- The postgres schema is in [schema.prisma](mdc:packages/shared/prisma/schema.prisma)
- The clickhouse schema is in [0001_traces.up.sql](mdc:packages/shared/clickhouse/migrations/clustered/0001_traces.up.sql), [0002_observations.up.sql](mdc:packages/shared/clickhouse/migrations/clustered/0002_observations.up.sql), [0003_scores.up.sql](mdc:packages/shared/clickhouse/migrations/clustered/0003_scores.up.sql)
+17
View File
@@ -0,0 +1,17 @@
---
description: How to create new public api routes for Langfuse
globs:
alwaysApply: false
---
# Implementation
- All public api routes are in /web/src/pages/api/public
- New api routes should follow these guidelines:
- Use [withMiddlewares.ts](mdc:web/src/features/public-api/server/withMiddlewares.ts) as a wrapper
- Define types of api request and response in /web/src/features/public-api/types, use strict() for all zod objects
- Add end-to-end test, similar to [datasets-api.servertest.ts](mdc:web/src/__tests__/async/datasets-api.servertest.ts)
- Add fern configuration in /fern, learn more about structure here: https://buildwithfern.com/learn/api-definition/fern/overview
- Prompt user to regenerate the OpenAPI spec via the fern CLI
- Pagination starts at 1, query typing defined in publicApiPaginationZod, return meta in paginationMetaResponseZod
- For tests, please look at [this directory](mdc:web/src/__tests__/async/) for examples
+9
View File
@@ -0,0 +1,9 @@
---
description:
globs:
alwaysApply: true
---
# Writing tests
- When writing tests, focus on decoupling each `it` or `test` block to ensure that they can run independently and concurrently. Tests must never depend on the action or outcome of previous or subsequent tests.
- When writing tests, especially in the __tests__/async directory, ensure that you avoid `pruneDatabase` calls.
+13
View File
@@ -0,0 +1,13 @@
# Dev container Dockerfile
FROM mcr.microsoft.com/vscode/devcontainers/typescript-node:20-bookworm
# Install golang-migrate for database migrations
RUN curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz && \
chmod +x migrate && \
mv migrate /usr/local/bin/migrate
# Install pnpm globally
RUN npm install -g pnpm@9.5.0
# Install Claude Code CLI
RUN npm install -g @anthropic-ai/claude-code
+8
View File
@@ -0,0 +1,8 @@
{
"name": "langfuse-development",
"build": {
"dockerfile": "Dockerfile"
},
"forwardPorts": [3000, 5432, 6379, 8123, 9000],
"postCreateCommand": "cp .env.dev.example .env && pnpm i"
}
+1
View File
@@ -68,6 +68,7 @@ LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE=true
LANGFUSE_S3_EVENT_UPLOAD_PREFIX=events/
LANGFUSE_USE_AZURE_BLOB=true
LANGFUSE_AZURE_SKIP_CONTAINER_CHECK=false
# Set during docker build of application
# Used to disable environment verification at build time
@@ -45,7 +45,7 @@ LANGFUSE_S3_BATCH_EXPORT_ENDPOINT=http://localhost:9090
LANGFUSE_S3_BATCH_EXPORT_FORCE_PATH_STYLE=true
LANGFUSE_S3_BATCH_EXPORT_PREFIX=exports/
# # S3 Media Upload LOCAL
# S3 Media Upload LOCAL
LANGFUSE_S3_MEDIA_UPLOAD_BUCKET=langfuse
LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID=minio
LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY=miniosecret
@@ -72,7 +72,11 @@ LANGFUSE_S3_EVENT_UPLOAD_PREFIX=events/
REDIS_HOST="127.0.0.1"
REDIS_PORT=6379
REDIS_AUTH="myredissecret"
REDIS_AUTH="bitnami"
REDIS_CLUSTER_ENABLED="true"
REDIS_CLUSTER_NODES="127.0.0.1:6370,127.0.0.1:6371,127.0.0.1:6372,127.0.0.1:6373,127.0.0.1:6374,127.0.0.1:6375"
LANGFUSE_INGESTION_QUEUE_SHARD_COUNT=8
LANGFUSE_TRACE_UPSERT_QUEUE_SHARD_COUNT=4
# openssl rand -hex 32 used only here
ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000
@@ -80,7 +84,10 @@ ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000
# speeds up local development by not executing init scripts on server startup
NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT="false"
LANGFUSE_READ_FROM_POSTGRES_ONLY=true
LANGFUSE_RETURN_FROM_CLICKHOUSE=false
LANGFUSE_READ_DASHBOARDS_FROM_CLICKHOUSE=false
LANGFUSE_READ_FROM_CLICKHOUSE_ONLY=false
# Use the following settings to enforce running the new AMTs during the tests
LANGFUSE_EXPERIMENT_INSERT_INTO_AGGREGATING_MERGE_TREES="true"
LANGFUSE_EXPERIMENT_COMPARE_READ_FROM_AGGREGATING_MERGE_TREES="true"
LANGFUSE_EXPERIMENT_WHITELISTED_PROJECT_IDS="7a88fb47-b4e2-43b8-a06c-a5ce950dc53a"
LANGFUSE_EXPERIMENT_ADD_QUERY_RESULT_TO_SPAN_PROJECT_IDS="7a88fb47-b4e2-43b8-a06c-a5ce950dc53a"
LANGFUSE_EXPERIMENT_RETURN_NEW_RESULT="true"
LANGFUSE_EXPERIMENT_SAMPLING_RATE=1
+10 -1
View File
@@ -75,7 +75,16 @@ REDIS_PORT=6379
REDIS_AUTH="myredissecret"
# openssl rand -hex 32 used only here
ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000
ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000000
# speeds up local development by not executing init scripts on server startup
NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT="false"
# For SDK integration tests to pass, decrease the ingestion queue delay by uncommenting the env vars:
# LANGFUSE_INGESTION_QUEUE_DELAY_MS=10
# LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS=10
# Slack credentials for development
SLACK_CLIENT_ID=your_slack_client_id
SLACK_CLIENT_SECRET=your_slack_client_secret
SLACK_STATE_SECRET=your_slack_state_secret
+74 -47
View File
@@ -65,37 +65,61 @@ OTEL_SERVICE_NAME="langfuse"
# AUTH_GOOGLE_CLIENT_SECRET=
# AUTH_GOOGLE_ALLOW_ACCOUNT_LINKING=false
# AUTH_GOOGLE_ALLOWED_DOMAINS=langfuse.com,google.com # optional allowlist of workspace domains that can sign in via Google
# AUTH_GOOGLE_CLIENT_AUTH_METHOD=
# AUTH_GOOGLE_CHECKS=
# AUTH_GITHUB_CLIENT_ID=
# AUTH_GITHUB_CLIENT_SECRET=
# AUTH_GITHUB_ALLOW_ACCOUNT_LINKING=false
# AUTH_GITHUB_CLIENT_AUTH_METHOD=
# AUTH_GITHUB_CHECKS=
# AUTH_GITHUB_ENTERPRISE_CLIENT_ID=
# AUTH_GITHUB_ENTERPRISE_CLIENT_SECRET=
# AUTH_GITHUB_ENTERPRISE_BASE_URL=
# AUTH_GITHUB_ENTERPRISE_ALLOW_ACCOUNT_LINKING=false
# AUTH_GITHUB_ENTERPRISE_CLIENT_AUTH_METHOD=
# AUTH_GITHUB_ENTERPRISE_CHECKS=
# AUTH_GITLAB_CLIENT_ID=
# AUTH_GITLAB_CLIENT_SECRET=
# AUTH_GITLAB_ALLOW_ACCOUNT_LINKING=false
# AUTH_GITLAB_ISSUER=
# AUTH_GITLAB_CLIENT_AUTH_METHOD=
# AUTH_GITLAB_CHECKS=
# AUTH_GITLAB_URL=
# AUTH_AZURE_AD_CLIENT_ID=
# AUTH_AZURE_AD_CLIENT_SECRET=
# AUTH_AZURE_AD_TENANT_ID=
# AUTH_AZURE_ALLOW_ACCOUNT_LINKING=false
# AUTH_AZURE_AD_ALLOW_ACCOUNT_LINKING=false
# AUTH_AZURE_AD_CLIENT_AUTH_METHOD=
# AUTH_AZURE_AD_CHECKS=
# AUTH_OKTA_CLIENT_ID=
# AUTH_OKTA_CLIENT_SECRET=
# AUTH_OKTA_ISSUER=
# AUTH_OKTA_ALLOW_ACCOUNT_LINKING=false
# AUTH_OKTA_CLIENT_AUTH_METHOD=
# AUTH_OKTA_CHECKS=
# AUTH_AUTH0_CLIENT_ID=
# AUTH_AUTH0_CLIENT_SECRET=
# AUTH_AUTH0_ISSUER=
# AUTH_AUTH0_ALLOW_ACCOUNT_LINKING=false
# AUTH_AUTH0_CLIENT_AUTH_METHOD=
# AUTH_AUTH0_CHECKS=
# AUTH_COGNITO_CLIENT_ID=
# AUTH_COGNITO_CLIENT_SECRET=
# AUTH_COGNITO_ISSUER=
# AUTH_COGNITO_ALLOW_ACCOUNT_LINKING=false
# AUTH_COGNITO_CLIENT_AUTH_METHOD=
# AUTH_COGNITO_CHECKS=
# AUTH_KEYCLOAK_CLIENT_ID=
# AUTH_KEYCLOAK_CLIENT_SECRET=
# AUTH_KEYCLOAK_ISSUER=
# AUTH_KEYCLOAK_ALLOW_ACCOUNT_LINKING=false
# AUTH_KEYCLOAK_CLIENT_AUTH_METHOD=
# AUTH_KEYCLOAK_CHECKS=
# AUTH_WORKOS_CLIENT_ID=
# AUTH_WORKOS_CLIENT_SECRET=
# AUTH_WORKOS_ALLOW_ACCOUNT_LINKING=false
# AUTH_WORKOS_ORGANIZATION_ID=
# AUTH_WORKOS_CONNECTION_ID=
# AUTH_CUSTOM_CLIENT_ID=
# AUTH_CUSTOM_CLIENT_SECRET=
# AUTH_CUSTOM_ISSUER=
@@ -103,6 +127,9 @@ OTEL_SERVICE_NAME="langfuse"
# AUTH_CUSTOM_SCOPE="openid email profile" # optional
# AUTH_CUSTOM_CLIENT_AUTH_METHOD="client_secret_basic" # optional
# AUTH_CUSTOM_ALLOW_ACCOUNT_LINKING=false
# AUTH_CUSTOM_ID_TOKEN=false # optional, default is true
# AUTH_CUSTOM_CLIENT_AUTH_METHOD=
# AUTH_CUSTOM_CHECKS=
# Transactional email, optional
# Defines the email address to use as the from address.
@@ -130,10 +157,6 @@ OTEL_SERVICE_NAME="langfuse"
# LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID=
# LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=
# Exports are streamed to S3 in pages to avoid memory issues
# The page size can be adjusted if needed to optimize performance
# DB_EXPORT_PAGE_SIZE=1000
# Automated provisioning of default resources
# LANGFUSE_INIT_ORG_ID=org-id
# LANGFUSE_INIT_ORG_NAME=org-name
@@ -145,7 +168,37 @@ OTEL_SERVICE_NAME="langfuse"
# LANGFUSE_INIT_USER_NAME=User Name
# LANGFUSE_INIT_USER_PASSWORD=password
# Redis configuration
# REDIS_HOST=
# REDIS_PORT=
# REDIS_AUTH=
# REDIS_USERNAME=default
# REDIS_CONNECTION_STRING=
# REDIS_ENABLE_AUTO_PIPELINING=
# Redis Cluster configuration (optional)
# REDIS_CLUSTER_ENABLED=false
# REDIS_CLUSTER_NODES=redis-node1:6379,redis-node2:6379,redis-node3:6379
# Cache configuration
# LANGFUSE_CACHE_API_KEY_ENABLED=
# LANGFUSE_CACHE_API_KEY_TTL_SECONDS=
# LANGFUSE_CACHE_PROMPT_ENABLED=
# LANGFUSE_CACHE_PROMPT_TTL_SECONDS=
# Clickhouse configuration
# CLICKHOUSE_URL=
# CLICKHOUSE_CLUSTER_NAME=default
# CLICKHOUSE_DB=default
# CLICKHOUSE_USER=
# CLICKHOUSE_PASSWORD=
# CLICKHOUSE_CLUSTER_ENABLED=true
# Ingestion configuration
# LANGFUSE_INGESTION_QUEUE_DELAY_MS=
# LANGFUSE_INGESTION_CLICKHOUSE_WRITE_BATCH_SIZE=
# LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS=
# LANGFUSE_INGESTION_CLICKHOUSE_MAX_ATTEMPTS=
### START Enterprise Edition Configuration
@@ -163,21 +216,12 @@ OTEL_SERVICE_NAME="langfuse"
# LANGFUSE_UI_DEFAULT_BASE_URL_OPENAI=https://api.openai.com/v1
# LANGFUSE_UI_DEFAULT_BASE_URL_ANTHROPIC=https://api.anthropic.com
# LANGFUSE_UI_DEFAULT_BASE_URL_AZURE_OPENAI=https://{instanceName}.openai.azure.com/openai/deployments
# LANGFUSE_UI_VISIBLE_PRODUCT_MODULES=
# LANGFUSE_UI_HIDDEN_PRODUCT_MODULES=
### END Enterprise Edition Configuration
### START Envs to be deprecated in Langfuse v3.0
# Disable the expensive analytics queries and related features
# LANGFUSE_DISABLE_EXPENSIVE_POSTGRES_QUERIES="true"
### END Envs to be deprecated in Langfuse v3.0
### START Langfuse Cloud Config
# Used for Langfuse Cloud deployments
# Not recommended for self-hosted deployments as these are NOT COVERED BY SEMANTIC VERSIONING
@@ -209,24 +253,16 @@ OTEL_SERVICE_NAME="langfuse"
# NEXT_PUBLIC_DEMO_ORG_ID=
# NEXT_PUBLIC_DEMO_PROJECT_ID=
# Crisp chat
# NEXT_PUBLIC_CRISP_WEBSITE_ID=
# Plain Chat
# NEXT_PUBLIC_PLAIN_APP_ID=
# PLAIN_AUTHENTICATION_SECRET=
# PLAIN_API_KEY=
# PLAIN_CARDS_API_TOKEN=
# Admin API
# ADMIN_API_KEY=
# Redis
# REDIS_HOST=
# REDIS_PORT=
# REDIS_AUTH=
# REDIS_CONNECTION_STRING=
# REDIS_ENABLE_AUTO_PIPELINING=
# Cache configuration
# LANGFUSE_CACHE_API_KEY_ENABLED=
# LANGFUSE_CACHE_API_KEY_TTL_SECONDS=
# LANGFUSE_CACHE_PROMPT_ENABLED=
# LANGFUSE_CACHE_PROMPT_TTL_SECONDS=
# LANGFUSE_CACHE_MODEL_MATCH_ENABLED=
# LANGFUSE_CACHE_MODEL_MATCH_TTL_SECONDS=
# Rate limiting
# LANGFUSE_RATE_LIMITS_ENABLED=
@@ -235,24 +271,15 @@ OTEL_SERVICE_NAME="langfuse"
# STRIPE_SECRET_KEY=
# STRIPE_WEBHOOK_SIGNING_SECRET=
# Betterstack Status Page
# BETTERSTACK_UPTIME_API_KEY=
# BETTERSTACK_UPTIME_STATUS_PAGE_ID=
### END Langfuse Cloud Config
## START Langfuse V3 Ingestion
### START Langfuse CI Config
# Clickhouse
# CLICKHOUSE_MIGRATION_URL=
# CLICKHOUSE_URL=
# CLICKHOUSE_USER=
# CLICKHOUSE_PASSWORD=
# CLICKHOUSE_DB=
# LANGFUSE_INIT_ORG_CLOUD_PLAN=
# Ingestion
# LANGFUSE_INGESTION_QUEUE_DELAY_MS=
# LANGFUSE_INGESTION_CLICKHOUSE_WRITE_BATCH_SIZE=
# LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS=
# LANGFUSE_INGESTION_CLICKHOUSE_MAX_ATTEMPTS=
# LANGFUSE_LEGACY_INGESTION_WORKER_CONCURRENCY=
# QUEUE_CONSUMER_LEGACY_INGESTION_QUEUE_IS_ENABLED="true"
## END Langfuse V3 Ingestion
### END Langfuse CI Config
+12
View File
@@ -0,0 +1,12 @@
# Test Environment Configuration
# Copy this file to .env.test for test database isolation
# Only overrides specific test variables - other values inherited from .env
# PostgreSQL - Test Database
DIRECT_URL="postgresql://postgres:postgres@localhost:5432/langfuse_test"
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/langfuse_test"
# ClickHouse - Use Default Database for now, nothing set
# Redis - Test Database (database 1 for isolation)
REDIS_CONNECTION_STRING="redis://:myredissecret@127.0.0.1:6379/1"
+1 -1
View File
@@ -28,7 +28,7 @@ Fixes # (issue)
<!-- Remove bullet points below that don't apply to you -->
- I haven't read the [contributing guide](https://github.com/langfuse/langfuse/blob/main/CONTRIBUTING.md)
- My code doesn't follow the style guidelines of this project (`npm run prettier`)
- My code doesn't follow the style guidelines of this project (`pnpm run format`)
- I haven't commented my code, particularly in hard-to-understand areas
- I haven't checked if my PR needs changes to the documentation
- I haven't checked if my changes generate no new warnings (`npm run lint`)
+1
View File
@@ -36,6 +36,7 @@ updates:
patterns:
- "express"
- "@types/express"
- "@types/express-serve-static-core"
observability:
patterns:
- "dd-trace"
+2 -1
View File
@@ -52,10 +52,11 @@ jobs:
--build-arg NEXT_PUBLIC_BUILD_ID=${{ github.sha }} \
--build-arg NEXT_PUBLIC_POSTHOG_KEY=${{ vars.NEXT_PUBLIC_POSTHOG_KEY }} \
--build-arg NEXT_PUBLIC_POSTHOG_HOST=${{ vars.NEXT_PUBLIC_POSTHOG_HOST }} \
--build-arg NEXT_PUBLIC_CRISP_WEBSITE_ID=${{ vars.NEXT_PUBLIC_CRISP_WEBSITE_ID }} \
--build-arg NEXT_PUBLIC_PLAIN_APP_ID=${{ vars.NEXT_PUBLIC_PLAIN_APP_ID }} \
--build-arg SENTRY_AUTH_TOKEN=${{ secrets.SENTRY_AUTH_TOKEN }} \
--build-arg SENTRY_ORG=${{ vars.SENTRY_ORG }} \
--build-arg SENTRY_PROJECT=${{ vars.SENTRY_PROJECT }} \
--build-arg NEXT_PUBLIC_LANGFUSE_TRACING_SAMPLE_RATE=${{ vars.NEXT_PUBLIC_LANGFUSE_TRACING_SAMPLE_RATE }} \
.
docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG
- name: Render AWS ECS Task Definition
+6 -2
View File
@@ -3,9 +3,13 @@ name: Codespell
on:
push:
branches: [main]
branches:
- "main"
tags:
- "v*"
pull_request:
branches: [main]
branches:
- "**"
merge_group:
permissions:
+2 -1
View File
@@ -21,6 +21,7 @@ on:
- staging
- prod-eu
- prod-us
- prod-hipaa
required: true
concurrency:
@@ -78,7 +79,7 @@ jobs:
return `["staging"]`
}
if (context.ref === "refs/heads/production") {
return `["prod-eu", "prod-us"]`
return `["prod-eu", "prod-us", "prod-hipaa"]`
}
}
return "[]"
+143 -40
View File
@@ -17,8 +17,22 @@ concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
pre-job:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
timeout-minutes: 15
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5
with:
do_not_skip: '["workflow_dispatch"]'
lint:
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
@@ -38,9 +52,55 @@ jobs:
- name: lint web
run: pnpm run lint
prettier-check:
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: pnpm/action-setup@v3
with:
version: 9.5.0
- uses: actions/setup-node@v4
with:
node-version: 20
cache: "pnpm"
cache-dependency-path: "pnpm-lock.yaml"
- name: install dependencies
run: |
pnpm i
- name: Load default env
run: |
cp .env.dev.example .env
- name: Check formatting on changed files
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE_SHA=${{ github.event.pull_request.base.sha }}
else
BASE_SHA=$(git merge-base origin/main HEAD)
fi
echo "Checking files changed from $BASE_SHA to HEAD"
# Get changed files
CHANGED_FILES=$(git diff --name-only $BASE_SHA HEAD -- '*.js' '*.jsx' '*.ts' '*.tsx' '*.css' | tr '\n' ' ')
if [ -n "$CHANGED_FILES" ] && [ "$CHANGED_FILES" != " " ]; then
echo "Files to check: $CHANGED_FILES"
pnpm prettier --check --experimental-cli $CHANGED_FILES
else
echo "No JS/TS/CSS files changed - skipping prettier check"
fi
test-docker-build:
timeout-minutes: 20
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -54,7 +114,9 @@ jobs:
run: echo "NEXT_PUBLIC_BUILD_ID=$(git rev-parse --short HEAD)" >> $GITHUB_ENV
- name: Build and run both images from compose
run: |
docker compose -f docker-compose.build.yml up -d
docker compose --progress plain --verbose -f docker-compose.build.yml build --print > /tmp/bake.json
docker buildx bake -f /tmp/bake.json
docker compose --progress plain -f docker-compose.build.yml up -d
sleep 5 # Wait for PostgreSQL to accept connections
- name: Ensure no unhealthy status
run: |
@@ -72,22 +134,21 @@ jobs:
timeout 10 bash -c 'until curl -f http://localhost:3000/api/public/health; do sleep 2; done'
tests-web-sync:
timeout-minutes: 20
timeout-minutes: 30
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
name: tests-web-sync (node${{ matrix.node-version }}, pg${{ matrix.postgres-version }})
strategy:
matrix:
node-version: [20]
postgres-version: [12, 15]
steps:
- name: Set Swap Space
uses: pierotofy/set-swap-space@master
with:
swap-size-gb: 10
- uses: actions/checkout@v4
- name: Install golang-migrate for Clickhouse migrations
run: |
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.16.2/migrate.linux-amd64.tar.gz | tar xvz
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz
sudo mv migrate /usr/bin/migrate
which migrate
- uses: pnpm/action-setup@v3
@@ -111,16 +172,18 @@ jobs:
- name: Load default env
run: |
cp .env.dev.example .env
grep -v -e '^LANGFUSE_S3_BATCH_EXPORT_ENABLED=' -e '^REDIS_HOST=' -e '^NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT=' .env.dev.legacy.example > .env
echo "LANGFUSE_POSTGRES_INGESTION_ENABLED=true" >> .env
- name: Run + migrate
grep -v -e '^LANGFUSE_S3_BATCH_EXPORT_ENABLED=' -e '^NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT=' .env.dev.example > .env
echo "LANGFUSE_INGESTION_QUEUE_DELAY_MS=1" >> .env
echo "LANGFUSE_CACHE_PROMPT_ENABLED=false" >> .env
echo "LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS=1" >> .env
- name: Run dev containers
run: |
docker compose -f docker-compose.dev.yml up -d
sleep 5 # Wait for PostgreSQL to accept connections
docker compose ps
env:
POSTGRES_VERSION: ${{ matrix.postgres-version }}
- name: Seed DB
- name: Migrate DB
run: |
pnpm run db:migrate
pnpm --filter=shared ch:up
@@ -131,6 +194,7 @@ jobs:
env:
LANGFUSE_INIT_ORG_ID: "seed-org-id"
LANGFUSE_INIT_ORG_NAME: "Seed Org"
LANGFUSE_INIT_ORG_CLOUD_PLAN: "Team"
LANGFUSE_INIT_PROJECT_ID: "7a88fb47-b4e2-43b8-a06c-a5ce950dc53a"
LANGFUSE_INIT_PROJECT_NAME: "Seed Project"
LANGFUSE_INIT_PROJECT_PUBLIC_KEY: "pk-lf-1234567890"
@@ -144,23 +208,22 @@ jobs:
run: pnpm --filter=web run test-client
tests-web-async:
timeout-minutes: 20
timeout-minutes: 30
runs-on: ubuntu-latest
name: tests-web-async (node${{ matrix.node-version }}, pg${{ matrix.postgres-version }}, mode${{ matrix.blob-provider }})
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
name: tests-web-async (node${{ matrix.node-version }}, pg${{ matrix.postgres-version }}, mode${{ matrix.deploy-mode }})
strategy:
matrix:
node-version: [20]
postgres-version: [12, 15]
blob-provider: ["", "-azure"]
deploy-mode: ["", "-azure", "-redis-cluster"]
steps:
- name: Set Swap Space
uses: pierotofy/set-swap-space@master
with:
swap-size-gb: 10
- uses: actions/checkout@v4
- name: Install golang-migrate for Clickhouse migrations
run: |
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.16.2/migrate.linux-amd64.tar.gz | tar xvz
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz
sudo mv migrate /usr/bin/migrate
which migrate
- uses: pnpm/action-setup@v3
@@ -183,16 +246,21 @@ jobs:
pnpm install
- name: Load default env
run: |
cp .env.dev${{ matrix.blob-provider }}.example .env
grep -v -e '^LANGFUSE_S3_BATCH_EXPORT_ENABLED=' -e '^REDIS_HOST=' -e '^NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT=' .env.dev${{ matrix.blob-provider }}.example > .env
- name: Run + migrate
cp .env.dev${{ matrix.deploy-mode }}.example .env
grep -v -e '^LANGFUSE_S3_BATCH_EXPORT_ENABLED=' -e '^NEXT_PUBLIC_LANGFUSE_RUN_NEXT_INIT=' .env.dev${{ matrix.deploy-mode }}.example > .env
echo "LANGFUSE_INGESTION_QUEUE_DELAY_MS=1" >> .env
echo "LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS=1" >> .env
echo "LANGFUSE_TRACE_DELETE_CONCURRENCY=100" >> .env
echo "ADMIN_API_KEY=admin-api-key" >> .env
echo "LANGFUSE_EE_LICENSE_KEY=langfuse_ee_test" >> .env
- name: Run dev containers
run: |
docker compose -f docker-compose.dev${{ matrix.blob-provider }}.yml up -d
docker compose -f docker-compose.dev${{ matrix.deploy-mode }}.yml up -d
sleep 5 # Wait for PostgreSQL to accept connections
docker compose ps
env:
POSTGRES_VERSION: ${{ matrix.postgres-version }}
- name: Seed DB
- name: Migrate DB
run: |
pnpm run db:migrate
pnpm --filter=shared ch:up
@@ -203,6 +271,7 @@ jobs:
env:
LANGFUSE_INIT_ORG_ID: "seed-org-id"
LANGFUSE_INIT_ORG_NAME: "Seed Org"
LANGFUSE_INIT_ORG_CLOUD_PLAN: "Team"
LANGFUSE_INIT_PROJECT_ID: "7a88fb47-b4e2-43b8-a06c-a5ce950dc53a"
LANGFUSE_INIT_PROJECT_NAME: "Seed Project"
LANGFUSE_INIT_PROJECT_PUBLIC_KEY: "pk-lf-1234567890"
@@ -216,17 +285,16 @@ jobs:
tests-worker:
timeout-minutes: 20
runs-on: ubuntu-latest
name: tests-worker (node${{ matrix.node-version }}, pg${{ matrix.postgres-version }}, mode${{ matrix.blob-provider }})
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
name: tests-worker (node${{ matrix.node-version }}, pg${{ matrix.postgres-version }}, mode${{ matrix.deploy-mode }})
strategy:
matrix:
node-version: [20]
postgres-version: [12, 15]
blob-provider: ["", "-azure"]
deploy-mode: ["", "-azure", "-redis-cluster"]
steps:
- name: Set Swap Space
uses: pierotofy/set-swap-space@master
with:
swap-size-gb: 10
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
@@ -248,17 +316,17 @@ jobs:
pnpm install
- name: Install golang-migrate for Clickhouse migrations
run: |
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.16.2/migrate.linux-amd64.tar.gz | tar xvz
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz
sudo mv migrate /usr/bin/migrate
which migrate
- name: Load default env
run: |
cp .env.dev${{ matrix.blob-provider }}.example .env
cp .env.dev${{ matrix.blob-provider }}.example web/.env
cp .env.dev${{ matrix.blob-provider }}.example worker/.env
cp .env.dev${{ matrix.deploy-mode }}.example .env
cp .env.dev${{ matrix.deploy-mode }}.example web/.env
cp .env.dev${{ matrix.deploy-mode }}.example worker/.env
- name: Run + migrate
run: |
docker compose -f docker-compose.dev${{ matrix.blob-provider }}.yml up -d
docker compose -f docker-compose.dev${{ matrix.deploy-mode }}.yml up -d
sleep 5 # Wait for PostgreSQL to accept connections
docker compose ps
- name: Ensure no unhealthy status
@@ -281,6 +349,9 @@ jobs:
e2e-tests:
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
@@ -304,14 +375,28 @@ jobs:
run: |
cp .env.dev.example .env
cp .env.dev.example web/.env
- name: Install golang-migrate for Clickhouse migrations
run: |
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz
sudo mv migrate /usr/bin/migrate
which migrate
- name: Run + migrate
run: |
docker compose -f docker-compose.dev.yml up -d
docker compose ps
sleep 5 # Wait for PostgreSQL to accept connections
- name: Ensure Docker dependencies are healthy
run: |
if docker-compose ps | grep "(unhealthy)"; then
echo "One or more services are unhealthy"
exit 1
else
echo "All services are healthy"
fi
- name: Seed DB
run: |
pnpm run db:migrate
pnpm --filter=shared run ch:up
pnpm run db:seed
- name: Build
run: pnpm run build
@@ -322,6 +407,9 @@ jobs:
e2e-server-tests:
runs-on: ubuntu-latest
needs:
- pre-job
if: needs.pre-job.outputs.should_skip != 'true'
steps:
- uses: actions/checkout@v4
- name: Login to Docker Hub
@@ -343,14 +431,12 @@ jobs:
pnpm install
- name: Install golang-migrate for Clickhouse migrations
run: |
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.16.2/migrate.linux-amd64.tar.gz | tar xvz
curl -L https://github.com/golang-migrate/migrate/releases/download/v4.18.3/migrate.linux-amd64.tar.gz | tar xvz
sudo mv migrate /usr/bin/migrate
which migrate
- name: Load default env
run: |
cp .env.dev.example .env
echo "LANGFUSE_CACHE_API_KEY_ENABLED=true" >> .env
echo "LANGFUSE_CACHE_PROMPT_ENABLED=true" >> .env
- name: Run + migrate
run: |
docker compose -f docker-compose.dev.yml up -d
@@ -388,6 +474,7 @@ jobs:
needs:
[
lint,
prettier-check,
tests-web-sync,
tests-worker,
e2e-tests,
@@ -398,13 +485,21 @@ jobs:
if: always()
steps:
- name: Successful deploy
if: ${{ !(contains(needs.*.result, 'failure')) }}
if: ${{ !(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')) }}
run: exit 0
working-directory: .
- name: Failing deploy
if: ${{ contains(needs.*.result, 'failure') }}
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: exit 1
working-directory: .
- name: Notify Slack
uses: ravsamhq/notify-slack-action@v2
if: always() && github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/'))
with:
status: ${{ job.status }}
notify_when: "failure"
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
push-docker-image:
needs: all-ci-passed
@@ -501,3 +596,11 @@ jobs:
platforms: |
linux/amd64
${{ startsWith(github.ref, 'refs/tags/') && 'linux/arm64' || '' }}
- name: Notify Slack
uses: ravsamhq/notify-slack-action@v2
if: always()
with:
status: ${{ job.status }}
notify_when: "failure"
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+10 -33
View File
@@ -1,51 +1,28 @@
name: Snyk Container
on:
pull_request:
branches:
- "**"
push:
branches:
- main
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
branches: ["main"]
jobs:
snyk:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build a Docker image
run: docker compose -f docker-compose.build.yml up -d
- uses: actions/checkout@v3
- name: Run Snyk to check Docker image for vulnerabilities (langfuse-web)
continue-on-error: true
- name: Scan web image with Snyk
uses: snyk/actions/docker@master
continue-on-error: true # let upload step always run
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: langfuse-langfuse-web
args: --file=web/Dockerfile
image: langfuse/langfuse # pulled from Docker Hub
args: --severity-threshold=high # (any extra CLI flags, but NO --file)
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: snyk.sarif
category: web
- name: Run Snyk to check Docker image for vulnerabilities (langfuse-worker)
continue-on-error: true
- name: Scan worker image with Snyk
uses: snyk/actions/docker@master
continue-on-error: true # let upload step always run
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: langfuse-langfuse-worker
args: --file=worker/Dockerfile
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: snyk.sarif
category: worker
image: langfuse/langfuse-worker # pulled from Docker Hub
args: --severity-threshold=high # (any extra CLI flags, but NO --file)
+22
View File
@@ -0,0 +1,22 @@
name: Close inactive issues
on:
schedule:
- cron: "30 1 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v9
with:
days-before-issue-stale: 30
days-before-issue-close: 14
stale-issue-label: "stale"
stale-issue-message: "This issue is stale because it has been open for 30 days with no activity."
close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen if the issue persists."
days-before-pr-stale: -1
days-before-pr-close: -1
repo-token: ${{ secrets.GITHUB_TOKEN }}
+11 -5
View File
@@ -4,9 +4,11 @@
/node_modules
/.pnp
.pnp.js
migrate
# testing
/coverage
/certs/
# database
/prisma/db.sqlite
@@ -38,8 +40,9 @@ yarn-error.log*
.env*
!.env.dev.example
!.env.dev-azure.example
!.env.dev-redis-cluster.example
!.env.prod.example
!.env.dev.legacy.example
!.env.test.example
# vercel
.vercel
@@ -47,19 +50,22 @@ yarn-error.log*
# typescript
*.tsbuildinfo
/generated/typescript-server
/generated
# openapi spec that is copied during build
/public/openapi*.yml
# vscode
.devcontainer
node_modules
**/node_modules
**/dist
**/.next/*
**/.turbo/*
.yarn
.turbo
.turbo
web/test-results/*
# local config files
*.local.*
+4 -3
View File
@@ -11,12 +11,13 @@ if [ "$current_branch" = "$protected_branch" ]; then
echo "🚨 You are about to commit to the $protected_branch branch. Are you sure? (y/n)"
read -r answer < /dev/tty
if [ "$answer" != "${answer#[Yy]}" ]; then
exit 0 # Commit will proceed
# Commit approved, check formatting. On files changed, block commit
pnpm run format:check
else
echo "Commit to $protected_branch branch has been canceled."
exit 1 # Commit will be blocked
fi
fi
# If not the protected branch, proceed with the commit
exit 0
# If not the protected branch, check formatting (on changed files, block)
pnpm run format:check
+2
View File
@@ -0,0 +1,2 @@
# Generated code
packages/shared/prisma/generated
+18
View File
@@ -0,0 +1,18 @@
# Codex Guidelines for Langfuse
Langfuse is an **open source LLM engineering** platform for developing, monitoring, evaluating and debugging AI applications. See the README for more details.
## Linting
- Run `pnpm run lint` to lint all packages.
- Fix issues automatically with `pnpm run lint:fix`.
## Tests
- Codex cannot run the test suite because it depends on Docker-based infrastructure that is unavailable in this environment.
- When writing tests, focus on decoupling each `it` or `test` block to ensure that they can run independently and concurrently. Tests must never depend on the action or outcome of previous or subsequent tests.
- When writing tests, especially in the __tests__/async directory, ensure that you avoid `pruneDatabase` calls.
## Cursor Rules
- Additional folder-specific rules live in `.cursor/rules/`.
## Commits
- Follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) when crafting commit messages.
+197
View File
@@ -0,0 +1,197 @@
# CLAUDE.md
## Project Overview
Langfuse is an open-source LLM engineering platform that helps teams collaboratively develop, monitor, evaluate, and debug AI applications.
The main feature areas are tracing, evals and prompt management. Langfuse consists of the web application (this repo), documentation, python SDK and javascript/typescript SDK.
This repo contains the web application, worker, and supporting packages but notably not the JS nor Python client SDKs.
## Repository Structure
High level structure. There are more folders (eg for hooks etc).
```
langfuse/
├── web/ # Next.js 14 frontend/backend application
│ ├── src/
│ │ ├── components/ # Reusable UI components (shadcn/ui)
│ │ ├── features/ # Feature-specific code organized by domain
│ │ ├── pages/ # Next.js pages (Pages Router)
│ │ └── server/ # tRPC API routes and server logic
│ └── public/ # Static assets
├── worker/ # Express.js background job processor
│ └── src/
│ ├── queues/ # BullMQ job queues
│ └── services/ # Background processing services
├── packages/
│ ├── shared/ # Shared types, schemas, and utilities
│ │ ├── prisma/ # Database schema and migrations
│ │ └── src/ # Shared TypeScript code
│ ├── config-eslint/ # ESLint configuration
│ └── config-typescript/ # TypeScript configuration
├── ee/ # Enterprise Edition features
├── fern/ # API documentation and OpenAPI specs
├── generated/ # Auto-generated client code
└── scripts/ # Development and deployment scripts
```
## Repository Architecture
This is a **pnpm + Turbo monorepo** with the following key packages:
### Core Applications
- **`/web/`** - Next.js 14 application (Pages Router) providing both frontend UI and backend APIs
- **`/worker/`** - Express.js background job processing server
- **`/packages/shared/`** - Shared database schema, types, and utilities
### Supporting Packages
- **`/ee/`** - Enterprise Edition features (separate licensing)
- **`/packages/config-eslint/`** - Shared ESLint configuration
- **`/packages/config-typescript/`** - Shared TypeScript configuration
## Development Commands
### Development
```sh
pnpm i # Install dependencies
pnpm run dev # Start all services (web + worker)
pnpm run dev:web # Web app only (localhost:3000) - **used in most cases!**
pnpm run dev:worker # Worker only
pnpm run dx # Full initial setup: install deps, reset DBs, resets node modules, seed data, start dev. USE SPARINGLY AS IT WIPES THE DATABASE & node_modules
```
### Database Management
database commands are to be run in the `packages/shared/` folder.
```sh
pnpm run db:generate # Build prisma models
pnpm run db:migrate # Run Prisma migrations
pnpm run db:reset # Reset and reseed databases
pnpm run db:seed # Seed with example data
```
### Infrastructure
```sh
pnpm run infra:dev:up # Start Docker services (PostgreSQL, ClickHouse, Redis, MinIO)
pnpm run infra:dev:down # Stop Docker services
```
### Building
```sh
pnpm --filter=PACKAGE_NAME run build # Runs the build command, will show real typescript errors etc.
```
### Testing in Web Package
The web package uses JEST for unit tests.
Depending on the file location (sync, async)
`web` related tests must go into the `web/src/__tests__/` folder.
```sh
pnpm test-sync --testPathPattern="$FILE_LOCATION_PATTERN" --testNamePattern="$TEST_NAME_PATTERN"
pnpm test-async --testPathPattern="$FILE_LOCATION_PATTERN" --testNamePattern="$TEST_NAME_PATTERN"
```
### Testing in the Worker Package
The worker uses `vitest` for unit tests.
```sh
pnpm run test --filter=worker -- $TEST_FILE_NAME -t "$TEST_NAME"
```
### Utilities
```bash
pnpm run format # Format code across entire project
pnpm run nuke # Remove all node_modules, build files, wipe database, docker containers. **USE WITH CAUTION**
```
## Technology Stack
### Web Application (`/web/`)
- **Framework**: Next.js 14 (Pages Router)
- **APIs**: tRPC (type-safe client-server communication) + REST APIs for public access
- **Authentication**: NextAuth.js/Auth.js
- **Database**: Prisma ORM with PostgreSQL
- **Analytics Database**: ClickHouse (high-volume trace data)
- **Validation**: Zod schemas, we use zodv4 (always import from `zod/v4`)
- **Styling**: Tailwind CSS with CSS variables for theming
- **Components**: shadcn/ui (Radix UI primitives)
- **State Management**: TanStack Query (React Query) + tRPC
- **Charts**: Tremor, Recharts
### Worker Application (`/worker/`)
- **Framework**: Express.js
- **Queue System**: BullMQ with Redis
- **Purpose**: Async processing (data ingestion, evaluations, exports, integrations)
### Infrastructure
- **Primary Database**: PostgreSQL (via Prisma ORM)
- **Analytics Database**: ClickHouse
- **Cache/Queues**: Redis
- **Blob Storage**: MinIO/S3
## Development Guidelines
### Frontend Features
- All new features go in `/web/src/features/[feature-name]/`
- Use tRPC for full-stack features (entry point: `web/src/server/api/root.ts`)
- Follow existing feature structure for consistency
- Use shadcn/ui components from `@/src/components/ui`
- Custom reusable components go in `@/src/components`
### Public API Development
- All public API routes in `/web/src/pages/api/public`
- Use `withMiddlewares.ts` wrapper
- Define types in `/web/src/features/public-api/types` with strict Zod v4 objects
- Add end-to-end tests (see `datasets-api.servertest.ts`)
- Manually update Fern API specs in `/fern/`, then regenerate OpenAPI spec via Fern CLI
### Authorization & RBAC
- Check `/web/src/features/rbac/README.md` for authorization patterns
- Implement proper entitlements checking (see `/web/src/features/entitlements/README.md`)
### Database
- **Dual database system**: PostgreSQL (primary) + ClickHouse (analytics)
- Use `golang-migrate` CLI for database migrations
- All database operations go through Prisma ORM for PostgreSQL
- Foreign key relationships may not be enforced in schema to allow unordered ingestion
### Testing
- Jest for API tests, Playwright for E2E tests
- For backend/API changes, tests must pass before pushes
- Add tests for new API endpoints and features
- When writing tests, focus on decoupling each `it` or `test` block to ensure that they can run independently and concurrently. Tests must never depend on the action or outcome of previous or subsequent tests.
- When writing tests, especially in the __tests__/async directory, ensure that you avoid `pruneDatabase` calls.
### Code Conventions
- **Pages Router** (not App Router)
- Follow conventional commits on main branch
- Use CSS variables for theming (supports auto dark/light mode)
- TypeScript throughout
- Zod v4 for all input validation
## Environment Setup
- **Node.js**: Version 20 (specified in `.nvmrc`)
- **Package Manager**: pnpm v9.5.0
- **Database Dependencies**: Docker for local PostgreSQL, ClickHouse, Redis, MinIO
- **Environment**: Copy `.env.dev.example` to `.env`
## Login for Development
When running locally with seed data:
- Username: `demo@langfuse.com`
- Password: `password`
- Demo project URL: `http://localhost:3000/project/7a88fb47-b4e2-43b8-a06c-a5ce950dc53a`
## Linear MCP
To get a project, use the `get_project` capability with the full project name as it is in the title.
- bad: message-placeholder-in-chat-messages-2beb6f02ec48
- good: Message placeholder in chat messages
## Front-end Tips
### Window Location Handling
- Whenever you want to use or do use window.location..., ensure that you also add proper handling for a custom basePath
## TypeScript Best Practices
- In TypeScript, if possible, don't use the `any` type
## General Coding Guidelines
- For easier code reviews, prefer not to move functions etc around within a file unless necessary or instructed to do so
## Development Tips
- Before trying to build the package, try running the linter once first
+120 -155
View File
@@ -27,7 +27,7 @@ The maintainers are available on [Discord](https://langfuse.com/discord) in case
## Making a change
_Before making any significant changes, please [open an issue](https://github.com/langfuse/langfuse/issues)._ Discussing your proposed changes ahead of time will make the contribution process smooth for everyone. Large changes that were not discussed in an issue may be rejected.
_Before making any significant changes, please [open an issue](https://github.com/langfuse/langfuse/issues)._ Discussing your proposed changes ahead of time will make the contribution process smooth for everyone. Changes that were not discussed in an issue may be rejected.
Once we've discussed your changes and you've got your code ready, make sure that tests are passing and open your pull request.
@@ -42,7 +42,7 @@ A good first step is to search for open [issues](https://github.com/langfuse/lan
- NextAuth.js / Auth.js
- tRPC: Frontend APIs
- Prisma ORM
- Zod
- Zod v4
- Tailwind CSS
- shadcn/ui tailwind components (using Radix and tanstack)
- Fern: generate OpenAPI spec and Pydantic models
@@ -53,124 +53,35 @@ A good first step is to search for open [issues](https://github.com/langfuse/lan
### Architecture Overview
**Langfuse v2**
```mermaid
flowchart TB
subgraph s4["Clients"]
subgraph s2["langfuse/langfuse-python"]
Python["Python low-level SDK"]
Decorator["observe() decorator"] -->|extends| Python
OAI["OpenAI drop-in replacement"] -->|extends| Python
Llamaindex["LlamaIndex Integration"] -->|extends| Python
LCPYTHON["Langchain Python Integration"] -->|extends| Python
Langflow -->|uses| LCPYTHON
LiteLLM -->|uses| Python
end
subgraph s3["langfuse/langfuse-js"]
JS["JS SDK"]
LCJS["Langchain JS Integration"] -->|extends| JS
Flowise -->|uses| LCJS
end
end
DB[Postgres Database]
Redis[Redis]
subgraph s1["Application (langfuse/langfuse/web)"]
API[Public HTTP API]
G[TRPC API]
I[NextAuth]
H[React Frontend]
Prisma[Prisma ORM]
H --> G
H --> I
G --> I
G --- Prisma
API --- Prisma
I --- Prisma
end
Prisma --- DB
JS --- API
Python --- API
```
**Langfuse v3 (work in progress, not released yet)**
> [!NOTE]
> Infrastructure will change in Langfuse version 3.0. More in the [GitHub Discussions](https://github.com/orgs/langfuse/discussions/1902).
> `langfuse/langfuse/worker` is under active development and not recommended for production use in Langfuse 2.x.
```mermaid
flowchart TB
subgraph s4["Clients"]
subgraph s2["langfuse/langfuse-python"]
Python["Python low-level SDK"]
Decorator["observe() decorator"] -->|extends| Python
OAI["OpenAI drop-in replacement"] -->|extends| Python
Llamaindex["LlamaIndex Integration"] -->|extends| Python
LCPYTHON["Langchain Python Integration"] -->|extends| Python
Langflow -->|uses| LCPYTHON
LiteLLM -->|uses| Python
end
subgraph s3["langfuse/langfuse-js"]
JS["JS SDK"]
LCJS["Langchain JS Integration"] -->|extends| JS
Flowise -->|uses| LCJS
end
end
DB[Postgres Database]
Redis[Redis Cache/Queue]
Clickhouse[Clickhouse Database]
subgraph s1["Application (langfuse/langfuse/web)"]
API[Public HTTP API]
G[TRPC API]
I[NextAuth]
H[React Frontend]
ORM
H --> G
H --> I
G --> I
G --- ORM
API --- ORM
I --- ORM
end
subgraph s5["Application (langfuse/langfuse/worker)"]
Worker
end
Worker --- DB
Worker --- Redis
Worker --- Clickhouse
ORM --- DB
ORM --- Redis
ORM --- Clickhouse
JS --- API
Python --- API
```
See this [diagram](https://langfuse.com/self-hosting#architecture) for an overview of the architecture.
### Network Overview
> [!NOTE]
> This will change in Langfuse version 3.0. More in the [GitHub Discussions](https://github.com/orgs/langfuse/discussions/1902).
```mermaid
flowchart LR
Browser ---|Web UI & TRPC API| App
Integrations/SDKs ---|Public HTTP API| App
subgraph i1["Application Network"]
App["Langfuse Application"]
end
subgraph i2["Database Network"]
DB["Postgres Database"]
end
App --- DB
flowchart TB
User["UI, API, SDKs"]
subgraph vpc["VPC"]
Web["Web Server<br/>(langfuse/langfuse)"]
Worker["Async Worker<br/>(langfuse/worker)"]
Postgres["Postgres - OLTP<br/>(Transactional Data)"]
Cache["Redis/Valkey<br/>(Cache, Queue)"]
Clickhouse["Clickhouse - OLAP<br/>(Observability Data)"]
S3["S3 / Blob Storage<br/>(Raw events, multi-modal attachments)"]
end
LLM["LLM API/Gateway<br/>(optional)"]
User --> Web
Web --> S3
Web --> Postgres
Web --> Cache
Web --> Clickhouse
Web -.->|"optional for playground"| LLM
Cache --> Worker
Worker --> Clickhouse
Worker --> Postgres
Worker --> S3
Worker -.->|"optional for evals"| LLM
```
### Database Overview
@@ -186,7 +97,7 @@ Full database schema: [packages/shared/prisma/schema.prisma](packages/shared/pri
We built a monorepo using [pnpm](https://pnpm.io/motivation) and [turbo](https://turbo.build/repo/docs) to manage the dependencies and build process. The monorepo contains the following packages:
- `web`: is the main application package providing Frontend and Backend APIs for Langfuse.
- `worker` (no production yet): contains an application for asynchronous processing of tasks. This package is not yet used in production.
- `worker`: contains an application for asynchronous processing of tasks.
- `packages`:
- `shared`: contains shared code between the above packages.
- `config-eslint`: contains eslint configurations which are shared between the above packages.
@@ -205,53 +116,51 @@ Requirements
**Steps**
1. Fork the repository and clone it locally
2. Run the development database
1. Install [golang-migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate#migrate-cli) as CLI
2. Fork the repository and clone it locally
```bash
pnpm run infra:dev:up
git clone https://github.com/langfuse/langfuse.git
cd langfuse
```
3. Create an env file
3. Install dependencies and set up pre-commit hooks
```bash
pnpm install
pnpm run prepare # Sets up Husky pre-commit hooks for code formatting
```
4. Create an env file
```bash
cp .env.dev.example .env
```
4. Install dependencies
4. Run the entire infrastructure in dev mode. **Note**: if you have an existing database, this command wipes it. Also, this will fail on the very first run. Please run it again.
```bash
pnpm install
pnpm run dx # first run only (resets db, docker containers, etc...)
pnpm run dev # any subsequent runs
```
5. Run the migrations
You will be asked whether you want to reset Postgres and ClickHouse. Confirm both with 'Y' and press enter.
All database migrations and configs are in the `shared` package.
5. Open the web app in your browser to start using Langfuse:
```bash
pnpm --filter=shared run db:migrate
- [Sign up page, http://localhost:3000](http://localhost:3000)
- [Demo project, http://localhost:3000/project/7a88fb47-b4e2-43b8-a06c-a5ce950dc53a](http://localhost:3000/project/7a88fb47-b4e2-43b8-a06c-a5ce950dc53a)
# Optional: seed the database
# pnpm run db:seed
# pnpm run db:seed:examples
# pnpm --filter=shared run db:seed:load
```
6. Log in as a test user:
6. Start the development server
- Username: `demo@langfuse.com`
- Password: `password`
```bash
pnpm run dev
```
To get comprehensive example data, you can use the `seed` command:
7. Open the web app in the browser:
http://localhost:3000
8. Log in as a test user (after you ran `db:seed` command):
Username: demo@langfuse.com
Password: password
```sh
pnpm run db:seed:examples
```
## Monorepo quickstart
@@ -302,23 +211,65 @@ Requirements
On the main branch, we adhere to the best practices of [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). All pull requests and branches are squash-merged to maintain a clean and readable history. This approach ensures the addition of a conventional commit message when merging contributions.
## Test the public API
## Running Unit Tests
The API is tested using Jest. With the development server running, you can run the tests with:
All tests run in the CI and must pass before merging.
All tests run against a running langfuse instance and **write/delete real data from the database**.
Run all
### Test Database Setup
Per default, the tests use the local development database. Therefore, wiping your data in the process.
For proper test isolation, create a `.env.test` file in the root directory:
```bash
npm run test
cp .env.test.example .env.test
```
Run interactively in watch mode
Then, a different PostgreSQL and Redis are used for the tests.
The `.env.test` file only overrides the set values and falls back on `.env` for all undefined values.
```bash
npm run test:watch
- **PostgreSQL**: Uses separate `langfuse_test` database for isolation
- **ClickHouse**: Uses shared `default` database for now
- **Redis**: Uses database 1 instead of 0 for isolation (Redis data is not cleaned between tests)
Tests automatically create the PostgreSQL test database if it doesn't exist and clean up data between runs.
### Tests in the `web` package (public API)
We're using Jest with in the `web` package. Therefore, if you want to provide an argument to the test runner, do it directly without an intermittent `--`.
There are three types of unit tests:
- `test-sync`
- `test-async`
- `test-client`
To run a specific test, for example the test: `"should handle special characters in prompt names"` in `prompts.v2.servertest.ts`, run:
```sh
cd web # or with --filter=web
pnpm test-sync --testPathPattern="prompts\.v2\.servertest" --testNamePattern="should handle special characters in prompt names"
```
These tests are also run in CI.
To run all tests:
```sh
pnpm run test
```
Run interactively in watch mode (not recommended!)
```sh
pnpm run test:watch
```
### Tests in the `worker` package
For the `worker` package, we're using `vitest` to run unit tests.
```sh
pnpm run test --filter=worker -- FILE_YOU_WANT_TO_TEST.ts -t "test name"
```
## CI/CD
@@ -397,7 +348,7 @@ The background color of the following component will be `hsl(var(--primary))` an
| --primary-accent | Primary accent color used for branding | Layout |
| --hover-primary-accent | Primary accent color used for hover effects for links | SignIn and AuthCloudRegionSwitch |
| --muted-green | Muted green for Event label | ObservationTree |
| --muted-orange | Muted orange for Generation label | ObservationTree |
| --muted-magenta | Muted magenta for Generation label | ObservationTree |
| --muted-blue | Muted blue for Span label | ObservationTree |
| --muted-gray | Muted gray for disabled status badges | StatusBadge |
| --accent-light-green | Light green accent for background of output and assistant messages | IOPreview, Generations, Traces |
@@ -439,13 +390,27 @@ You can update the default AI models and prices by adding or updating an entry i
Please note that
- prices are in USD
- the list is ordered by ID, so make sure to keep this order
- the list is ordered by ID, so make sure to keep this order and insert new models at the end of the list
- the `updated_at` field must be updated with the current date in ISO 8601 format. Otherwise, the change will be ignored.
### Transition period until V3 release
Until the V3 release, both the JSON record must be updated **and** a migration must be created to continue supporting self-hosted users. Note that the migration must updated both the `models` as well as the `prices` table accordingly.
## Updating the OpenAPI Specs & fern SDKs
We maintain the API specifications manually to guarantee a high degree of understandability. If you made changes to the API, please update the respective `.yml` files in `fern/apis/...`.
To generate the respective `openapi.yml` files which power the online API reference & SDKs, run:
```sh
npx fern-api generate --api server # for the server API
npx fern-api generate --api client # for the client API
npx fern-api generate --api organizations # for the organizations API
```
**Note:** You need a signed in fern account to run those commands.
## License
Langfuse is MIT licensed, except for `ee/` folder. See [LICENSE](LICENSE) and [docs](https://langfuse.com/docs/open-source) for more details.
+355
View File
@@ -0,0 +1,355 @@
![Langfuse GitHub Banner](https://langfuse.com/images/docs/github-readme/github-banner.png)
<div align="center">
<div>
<h3>
<a href="https://langfuse.com/cn">
<strong>🇨🇳 🤝 🪢</strong>
</a> ·
<a href="https://cloud.langfuse.com">
<strong>Langfuse Cloud</strong>
</a> ·
<a href="https://langfuse.com/docs/deployment/self-host">
<strong>自托管</strong>
</a> ·
<a href="https://langfuse.com/demo">
<strong>演示</strong>
</a>
</h3>
</div>
<div>
<a href="https://langfuse.com/docs"><strong>文档</strong></a> ·
<a href="https://langfuse.com/issues"><strong>报告问题</strong></a> ·
<a href="https://langfuse.com/ideas"><strong>功能请求</strong></a> ·
<a href="https://langfuse.com/changelog"><strong>更新日志</strong></a> ·
<a href="https://langfuse.com/roadmap"><strong>路线图</strong></a> ·
</div>
<br/>
<span>Langfuse 使用 <a href="https://github.com/orgs/langfuse/discussions"><strong>GitHub Discussions</strong></a> 作为支持和功能请求的平台。</span>
<br/>
<span><b>我们正在招聘。</b> <a href="https://langfuse.com/careers"><strong>加入我们</strong></a>,从事产品工程和技术市场职位。</span>
<br/>
<br/>
<div>
</div>
</div>
<p align="center">
<a href="https://github.com/langfuse/langfuse/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-E11311.svg" alt="MIT License">
</a>
<a href="https://www.ycombinator.com/companies/langfuse">
<img src="https://img.shields.io/badge/Y%20Combinator-W23-orange" alt="Y Combinator W23">
</a>
<a href="https://hub.docker.com/u/langfuse" target="_blank">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/langfuse/langfuse?labelColor=%20%23FDB062&logo=Docker&labelColor=%20%23528bff">
</a>
<a href="https://pypi.python.org/pypi/langfuse">
<img src="https://img.shields.io/pypi/dm/langfuse?logo=python&logoColor=white&label=pypi%20langfuse&color=blue" alt="langfuse Python package on PyPi">
</a>
<a href="https://www.npmjs.com/package/langfuse">
<img src="https://img.shields.io/npm/dm/langfuse?logo=npm&logoColor=white&label=npm%20langfuse&color=blue" alt="langfuse npm package">
</a>
<br/>
<a href="https://discord.com/invite/7NXusRtqYU" target="_blank">
<img src="https://img.shields.io/discord/1111061815649124414?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="在 Discord 上聊天">
</a>
<a href="https://twitter.com/intent/follow?screen_name=langfuse" target="_blank">
<img src="https://img.shields.io/twitter/follow/langfuse?logo=X&color=%20%23f5f5f5"
alt="在 X (Twitter) 上关注">
</a>
<a href="https://www.linkedin.com/company/langfuse/" target="_blank">
<img src="https://custom-icon-badges.demolab.com/badge/LinkedIn-0A66C2?logo=linkedin-white&logoColor=fff"
alt="在 LinkedIn 上关注">
</a>
<a href="https://github.com/langfuse/langfuse/graphs/commit-activity" target="_blank">
<img alt="过去一个月的提交" src="https://img.shields.io/github/commit-activity/m/langfuse/langfuse?labelColor=%20%2332b583&color=%20%2312b76a">
</a>
<a href="https://github.com/langfuse/langfuse/" target="_blank">
<img alt="已关闭的问题" src="https://img.shields.io/github/issues-search?query=repo%3Alangfuse%2Flangfuse%20is%3Aclosed&label=issues%20closed&labelColor=%20%237d89b0&color=%20%235d6b98">
</a>
<a href="https://github.com/langfuse/langfuse/discussions/" target="_blank">
<img alt="讨论帖数量" src="https://img.shields.io/github/discussions/langfuse/langfuse?labelColor=%20%239b8afb&color=%20%237a5af8">
</a>
</p>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>
<a href="./README.cn.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
<a href="./README.ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-d9d9d9"></a>
<a href="./README.kr.md"><img alt="README in Korean" src="https://img.shields.io/badge/한국어-d9d9d9"></a>
</p>
Langfuse 是一个 **开源 LLM 工程** 平台。它帮助团队协作 **开发、监控、评估** 以及 **调试** AI 应用。Langfuse 可在几分钟内 **自托管**,并且经过 **实战考验**
[![Langfuse 概览视频](https://github.com/user-attachments/assets/3926b288-ff61-4b95-8aa1-45d041c70866)](https://langfuse.com/watch-demo)
## ✨ 核心特性
![Langfuse 概览](https://langfuse.com/images/docs/github-readme/github-feature-overview.png)
- [LLM 应用可观察性](https://langfuse.com/docs/tracing):为你的应用插入仪表代码,并开始将追踪数据传送到 Langfuse,从而追踪 LLM 调用及应用中其他相关逻辑(如检索、嵌入或代理操作)。检查并调试复杂日志及用户会话。试试互动的 [演示](https://langfuse.com/docs/demo) 看看效果。
- [提示管理](https://langfuse.com/docs/prompts/get-started) 帮助你集中管理、版本控制并协作迭代提示。得益于服务器和客户端的高效缓存,你可以在不增加延迟的情况下反复迭代提示。
- [评估](https://langfuse.com/docs/scores/overview) 是 LLM 应用开发流程的关键组成部分,Langfuse 能够满足你的多样需求。它支持 LLM 作为"裁判"、用户反馈收集、手动标注以及通过 API/SDK 实现自定义评估流程。
- [数据集](https://langfuse.com/docs/datasets/overview) 为评估你的 LLM 应用提供测试集和基准。它们支持持续改进、部署前测试、结构化实验、灵活评估,并能与 LangChain、LlamaIndex 等框架无缝整合。
- [LLM 试玩平台](https://langfuse.com/docs/playground) 是用于测试和迭代提示及模型配置的工具,缩短反馈周期,加速开发。当你在追踪中发现异常结果时,可以直接跳转至试玩平台进行调整。
- [综合 API](https://langfuse.com/docs/api):Langfuse 常用于驱动定制化的 LLMOps 工作流程,同时利用 Langfuse 提供的构建模块和 API。我们提供 OpenAPI 规格、Postman 集合以及针对 Python 和 JS/TS 的类型化 SDK。
## 📦 部署 Langfuse
![Langfuse 部署选项](https://langfuse.com/images/docs/github-readme/github-deployment-options.png)
### Langfuse Cloud
由 Langfuse 团队管理的部署,提供慷慨的免费额度(爱好者计划),无需信用卡。
<div align="center">
<a href="https://cloud.langfuse.com" target="_blank">
<img alt="注册 Langfuse Cloud" src="https://img.shields.io/badge/»%20Sign%20up%20for%20Langfuse%20Cloud-8A2BE2?&color=orange">
</a>
</div>
### 自托管 Langfuse
在你自己的基础设施上运行 Langfuse:
- [本地(docker compose](https://langfuse.com/self-hosting/local):使用 Docker Compose 在你的机器上于 5 分钟内运行 Langfuse。
````bash:README.md/docker-compose
# 获取最新的 Langfuse 仓库副本
git clone https://github.com/langfuse/langfuse.git
cd langfuse
# 运行 Langfuse 的 docker compose
docker compose up
`````
- [KubernetesHelm](https://langfuse.com/self-hosting/kubernetes-helm):使用 Helm 在 Kubernetes 集群上部署 Langfuse。这是推荐的生产环境部署方式。
- [虚拟机](https://langfuse.com/self-hosting/docker-compose):使用 Docker Compose 在单台虚拟机上部署 Langfuse。
- Terraform 模板: [AWS](https://langfuse.com/self-hosting/aws)、[Azure](https://langfuse.com/self-hosting/azure)、[GCP](https://langfuse.com/self-hosting/gcp)
请参阅 [自托管文档](https://langfuse.com/self-hosting) 了解更多关于架构和配置选项的信息。
## 🔌 集成
![Langfuse 集成](https://langfuse.com/images/docs/github-readme/github-integrations.png)
### 主要集成:
| 集成 | 支持语言/平台 | 描述 |
| ---------------------------------------------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- |
| [SDK](https://langfuse.com/docs/sdk) | Python, JS/TS | 使用 SDK 进行手动仪表化,实现全面灵活性。 |
| [OpenAI](https://langfuse.com/docs/integrations/openai) | Python, JS/TS | 通过直接替换 OpenAI SDK 实现自动仪表化。 |
| [Langchain](https://langfuse.com/docs/integrations/langchain) | Python, JS/TS | 通过传入回调处理器至 Langchain 应用实现自动仪表化。 |
| [LlamaIndex](https://langfuse.com/docs/integrations/llama-index/get-started) | Python | 通过 LlamaIndex 回调系统实现自动仪表化。 |
| [Haystack](https://langfuse.com/docs/integrations/haystack) | Python | 通过 Haystack 内容追踪系统实现自动仪表化。 |
| [LiteLLM](https://langfuse.com/docs/integrations/litellm) | Python, JS/TS (仅代理) | 允许使用任何 LLM 替代 GPT。支持 Azure、OpenAI、Cohere、Anthropic、Ollama、VLLM、Sagemaker、HuggingFace、Replicate100+ LLMs)。 |
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | 基于 TypeScript 的工具包,帮助开发者使用 React、Next.js、Vue、Svelte 和 Node.js 构建 AI 驱动的应用。 |
| [API](https://langfuse.com/docs/api) | | 直接调用公共 API。提供 OpenAPI 规格。 |
| [Google VertexAI 和 Gemini](https://langfuse.com/docs/integrations/google-vertex-ai) | 模型 | 在 Google 上运行基础模型和微调模型。 |
### 与 Langfuse 集成的软件包:
| 名称 | 类型 | 描述 |
| ---------------------------------------------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------- |
| [Instructor](https://langfuse.com/docs/integrations/instructor) | 库 | 用于获取结构化 LLM 输出(JSON、Pydantic)的库。 |
| [DSPy](https://langfuse.com/docs/integrations/dspy) | 库 | 一个系统性优化语言模型提示和权重的框架。 |
| [Amazon Bedrock](https://langfuse.com/docs/integrations/amazon-bedrock) | 模型 | 在 AWS 上运行基础模型和微调模型。 |
| [Mirascope](https://langfuse.com/docs/integrations/mirascope) | 库 | 构建 LLM 应用的 Python 工具包。 |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | 模型(本地) | 在你的机器上轻松运行开源 LLM。 |
| [AutoGen](https://langfuse.com/docs/integrations/autogen) | 代理框架 | 用于构建分布式代理的开源 LLM 平台。 |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | 聊天/代理界面 | 基于 JS/TS 的无代码构建器,用于定制化 LLM 流程。 |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | 聊天/代理界面 | 基于 Python 的 LangChain 用户界面,采用 react-flow 设计,提供便捷的实验与原型构建体验。 |
| [Dify](https://langfuse.com/docs/integrations/dify) | 聊天/代理界面 | 带有无代码构建器的开源 LLM 应用开发平台。 |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | 聊天/代理界面 | 自托管的 LLM 聊天网页界面,支持包括自托管和本地模型在内的多种 LLM 运行器。 |
| [Promptfoo](https://langfuse.com/docs/integrations/promptfoo) | 工具 | 开源 LLM 测试平台。 |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | 聊天/代理界面 | 开源聊天机器人平台。 |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | 平台 | 开源语音 AI 平台。 |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | 代理 | 构建分布式代理的开源 LLM 平台。 |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | 聊天/代理界面 | 开源 Python 库,可用于构建类似聊天 UI 的网页界面。 |
| [Goose](https://langfuse.com/docs/integrations/goose) | 代理 | 构建分布式代理的开源 LLM 平台。 |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | 代理 | 开源 AI 代理框架。 |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | 代理 | 多代理框架,用于实现代理之间的协作与工具调用。 |
## 🚀 快速入门
为你的应用增加仪表代码,并开始将追踪数据上传到 Langfuse,从而记录 LLM 调用及应用中其他相关逻辑(如检索、嵌入或代理操作)。
### 1️⃣ 创建新项目
1. [创建 Langfuse 账户](https://cloud.langfuse.com/auth/sign-up) 或 [自托管](https://langfuse.com/self-hosting)
2. 创建一个新项目
3. 在项目设置中创建新的 API 凭证
### 2️⃣ 记录你的第一个 LLM 调用
使用 [<code>@observe()</code> 装饰器](https://langfuse.com/docs/sdk/python/decorators) 可轻松跟踪任何 Python LLM 应用。在本快速入门中,我们还使用了 Langfuse 的 [OpenAI 集成](https://langfuse.com/docs/integrations/openai) 来自动捕获所有模型参数。
> [!提示]
> 不使用 OpenAI?请访问 [我们的文档](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse) 了解如何记录其他模型和框架。
安装依赖:
````bash
pip install langfuse openai
````
配置环境变量(创建名为 **.env** 的文件):
````bash:.env
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 欧盟区域
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 美洲区域
````
创建示例代码(文件名:**main.py**):
```python:main.py
from langfuse import observe
from langfuse.openai import openai # OpenAI 集成
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
````
### 3️⃣ 在 Langfuse 中查看追踪记录
在 Langfuse 中查看你的语言模型调用及其他应用逻辑。
![示例追踪记录](https://langfuse.com/images/docs/github-readme/github-example-trace.png)
_[Langfuse 中的公共示例追踪](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!提示]
>
> [了解更多](https://langfuse.com/docs/tracing) 关于 Langfuse 中的追踪,或试试 [互动演示](https://langfuse.com/docs/demo)。
## ⭐️ 给我们加星
![为 Langfuse 加星](https://langfuse.com/images/docs/github-readme/github-star-howto.gif)
## 💭 支持
查找问题答案:
- 我们的 [文档](https://langfuse.com/docs) 是查找答案的最佳起点。内容全面,我们投入大量时间进行维护。你也可以通过 GitHub 提出文档修改建议。
- [Langfuse 常见问题](https://langfuse.com/faq) 解答了最常见的问题。
- 使用 "Ask AI" 立即获取问题答案。
支持渠道:
- **在 GitHub Discussions 的 [公共问答](https://github.com/orgs/langfuse/discussions/categories/support) 中提出任何问题。** 请尽量提供详细信息(如代码片段、截图、背景信息)以帮助我们理解你的问题。
- 在 GitHub Discussions 中 [提出功能请求](https://github.com/orgs/langfuse/discussions/categories/ideas)。
- 在 GitHub Issues 中 [报告 Bug](https://github.com/langfuse/langfuse/issues)。
- 对于时效性较强的问题,请通过应用内聊天小部件联系我们。
## 🤝 贡献
欢迎你的贡献!
- 在 GitHub Discussions 中为 [想法](https://github.com/orgs/langfuse/discussions/categories/ideas)投票。
- 提出并评论 [问题](https://github.com/langfuse/langfuse/issues)。
- 提交 PR —— 详情请参见 [CONTRIBUTING.md](CONTRIBUTING.md),了解如何搭建开发环境。
## 🥇 许可证
除 `ee` 文件夹外,本仓库采用 MIT 许可证。详情请参见 [LICENSE](LICENSE) 以及 [文档](https://langfuse.com/docs/open-source)。
## ⭐️ 星标历史
<a href="https://star-history.com/#langfuse/langfuse&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="星标历史图表" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" style="border-radius: 15px;" />
</picture>
</a>
## ❤️ 使用 Langfuse 的开源项目
以下是使用 Langfuse 的顶级开源 Python 项目,按星标数排名([来源](https://github.com/langfuse/langfuse-docs/blob/main/components-mdx/dependents)):
| 仓库 | 星数 |
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----: |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/127165244?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langgenius](https://github.com/langgenius) / [dify](https://github.com/langgenius/dify) | 54865 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [open-webui](https://github.com/open-webui/open-webui) | 51531 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/131470832?s=40&v=4" width="20" height="20" alt=""> &nbsp; [lobehub](https://github.com/lobehub) / [lobe-chat](https://github.com/lobehub/lobe-chat) | 49003 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85702467?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langflow-ai](https://github.com/langflow-ai) / [langflow](https://github.com/langflow-ai/langflow) | 39093 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt=""> &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 37368 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/139558948?s=40&v=4" width="20" height="20" alt=""> &nbsp; [chatchat-space](https://github.com/chatchat-space) / [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | 32486 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/128289781?s=40&v=4" width="20" height="20" alt=""> &nbsp; [FlowiseAI](https://github.com/FlowiseAI) / [Flowise](https://github.com/FlowiseAI/Flowise) | 32448 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/31035808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mindsdb](https://github.com/mindsdb) / [mindsdb](https://github.com/mindsdb/mindsdb) | 26931 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/119600397?s=40&v=4" width="20" height="20" alt=""> &nbsp; [twentyhq](https://github.com/twentyhq) / [twenty](https://github.com/twentyhq/twenty) | 24195 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog](https://github.com/PostHog/posthog) | 22618 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/121462774?s=40&v=4" width="20" height="20" alt=""> &nbsp; [BerriAI](https://github.com/BerriAI) / [litellm](https://github.com/BerriAI/litellm) | 15151 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/179202840?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mediar-ai](https://github.com/mediar-ai) / [screenpipe](https://github.com/mediar-ai/screenpipe) | 11037 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/105877416?s=40&v=4" width="20" height="20" alt=""> &nbsp; [formbricks](https://github.com/formbricks) / [formbricks](https://github.com/formbricks/formbricks) | 9386 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/76263028?s=40&v=4" width="20" height="20" alt=""> &nbsp; [anthropics](https://github.com/anthropics) / [courses](https://github.com/anthropics/courses) | 8385 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/78410652?s=40&v=4" width="20" height="20" alt=""> &nbsp; [GreyDGL](https://github.com/GreyDGL) / [PentestGPT](https://github.com/GreyDGL/PentestGPT) | 7374 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt=""> &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5391 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/137907881?s=40&v=4" width="20" height="20" alt=""> &nbsp; [promptfoo](https://github.com/promptfoo) / [promptfoo](https://github.com/promptfoo/promptfoo) | 4976 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/157326433?s=40&v=4" width="20" height="20" alt=""> &nbsp; [onlook-dev](https://github.com/onlook-dev) / [onlook](https://github.com/onlook-dev/onlook) | 4141 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/7250217?s=40&v=4" width="20" height="20" alt=""> &nbsp; [Canner](https://github.com/Canner) / [WrenAI](https://github.com/Canner/WrenAI) | 2526 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/11855343?s=40&v=4" width="20" height="20" alt=""> &nbsp; [pingcap](https://github.com/pingcap) / [autoflow](https://github.com/pingcap/autoflow) | 2061 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85268109?s=40&v=4" width="20" height="20" alt=""> &nbsp; [MLSysOps](https://github.com/MLSysOps) / [MLE-agent](https://github.com/MLSysOps/MLE-agent) | 1161 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [pipelines](https://github.com/open-webui/pipelines) | 1100 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18422723?s=40&v=4" width="20" height="20" alt=""> &nbsp; [alishobeiri](https://github.com/alishobeiri) / [thread](https://github.com/alishobeiri/thread) | 1074 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/125468716?s=40&v=4" width="20" height="20" alt=""> &nbsp; [topoteretes](https://github.com/topoteretes) / [cognee](https://github.com/topoteretes/cognee) | 971 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/188657705?s=40&v=4" width="20" height="20" alt=""> &nbsp; [bRAGAI](https://github.com/bRAGAI) / [bRAG-langchain](https://github.com/bRAGAI/bRAG-langchain) | 823 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169500408?s=40&v=4" width="20" height="20" alt=""> &nbsp; [opslane](https://github.com/opslane) / [opslane](https://github.com/opslane/opslane) | 677 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/151867818?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dynamiq-ai](https://github.com/dynamiq-ai) / [dynamiq](https://github.com/dynamiq-ai/dynamiq) | 639 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/48585267?s=40&v=4" width="20" height="20" alt=""> &nbsp; [theopenconversationkit](https://github.com/theopenconversationkit) / [tock](https://github.com/theopenconversationkit/tock) | 514 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/20493493?s=40&v=4" width="20" height="20" alt=""> &nbsp; [andysingal](https://github.com/andysingal) / [llm-course](https://github.com/andysingal/llm-course) | 394 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/132396805?s=40&v=4" width="20" height="20" alt=""> &nbsp; [phospho-app](https://github.com/phospho-app) / [phospho](https://github.com/phospho-app/phospho) | 384 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/178644984?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sentient-engineering](https://github.com/sentient-engineering) / [agent-q](https://github.com/sentient-engineering/agent-q) | 370 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/168552753?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sql-agi](https://github.com/sql-agi) / [DB-GPT](https://github.com/sql-agi/DB-GPT) | 324 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog-foss](https://github.com/PostHog/posthog-foss) | 305 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/154247157?s=40&v=4" width="20" height="20" alt=""> &nbsp; [vespperhq](https://github.com/vespperhq) / [vespper](https://github.com/vespperhq/vespper) | 304 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/185116535?s=40&v=4" width="20" height="20" alt=""> &nbsp; [block](https://github.com/block) / [goose](https://github.com/block/goose) | 295 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/609489?s=40&v=4" width="20" height="20" alt=""> &nbsp; [aorwall](https://github.com/aorwall) / [moatless-tools](https://github.com/aorwall/moatless-tools) | 291 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2357342?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dmayboroda](https://github.com/dmayboroda) / [minima](https://github.com/dmayboroda/minima) | 221 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/66303003?s=40&v=4" width="20" height="20" alt=""> &nbsp; [RobotecAI](https://github.com/RobotecAI) / [rai](https://github.com/RobotecAI/rai) | 172 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/148684274?s=40&v=4" width="20" height="20" alt=""> &nbsp; [i-am-alice](https://github.com/i-am-alice) / [3rd-devs](https://github.com/i-am-alice/3rd-devs) | 148 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/171735272?s=40&v=4" width="20" height="20" alt=""> &nbsp; [8090-inc](https://github.com/8090-inc) / [xrx-sample-apps](https://github.com/8090-inc/xrx-sample-apps) | 138 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/104478511?s=40&v=4" width="20" height="20" alt=""> &nbsp; [babelcloud](https://github.com/babelcloud) / [LLM-RGB](https://github.com/babelcloud/LLM-RGB) | 135 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/15125613?s=40&v=4" width="20" height="20" alt=""> &nbsp; [souzatharsis](https://github.com/souzatharsis) / [tamingLLMs](https://github.com/souzatharsis/tamingLLMs) | 129 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169401942?s=40&v=4" width="20" height="20" alt=""> &nbsp; [LibreChat-AI](https://github.com/LibreChat-AI) / [librechat.ai](https://github.com/LibreChat-AI/librechat.ai) | 128 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/51827949?s=40&v=4" width="20" height="20" alt=""> &nbsp; [deepset-ai](https://github.com/deepset-ai) / [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) | 126 |
## 🔒 安全与隐私
我们非常重视数据安全和隐私。更多信息请参阅我们的 [安全与隐私](https://langfuse.com/security) 页面。
### 遥测
默认情况下,Langfuse 会自动将自托管实例的基础使用统计数据上传至集中服务器(PostHog)。
这有助于我们:
1. 了解 Langfuse 的使用情况,并改进最关键的功能。
2. 跟踪整体使用数据,以便内部及外部(例如筹款)报告。
所有数据均不会与第三方共享,也不包含任何敏感信息。我们对这一过程保持高度透明,你可以在 [此处](/web/src/features/telemetry/index.ts) 查看我们收集的具体数据。
你可以通过设置 `TELEMETRY_ENABLED=false` 来选择退出。
```
+359
View File
@@ -0,0 +1,359 @@
![Langfuse GitHub Banner](https://langfuse.com/images/docs/github-readme/github-banner.png)
<div align="center">
<div>
<h3>
<a href="https://langfuse.com/jp">
<strong>🇯🇵 🤝 🪢</strong>
</a> ·
<a href="https://cloud.langfuse.com">
<strong>Langfuse Cloud</strong>
</a> ·
<a href="https://langfuse.com/docs/deployment/self-host">
<strong>セルフホスティング</strong>
</a> ·
<a href="https://langfuse.com/demo">
<strong>デモ</strong>
</a>
</h3>
</div>
<div>
<a href="https://langfuse.com/docs"><strong>ドキュメント</strong></a> ·
<a href="https://langfuse.com/issues"><strong>バグ報告</strong></a> ·
<a href="https://langfuse.com/ideas"><strong>機能リクエスト</strong></a> ·
<a href="https://langfuse.com/changelog"><strong>変更履歴</strong></a> ·
<a href="https://langfuse.com/roadmap"><strong>ロードマップ</strong></a> ·
</div>
<br/>
<span>Langfuseは、サポートと機能リクエストのために <a href="https://github.com/orgs/langfuse/discussions"><strong>GitHub Discussions</strong></a> を利用しています。</span>
<br/>
<span><b>We're hiring.</b> <a href="https://langfuse.com/careers"><strong>チームに加わる</strong></a> (製品エンジニアリングおよびテクニカルGTMのポジション)への応募をお待ちしています。</span>
<br/>
<br/>
<div>
</div>
</div>
<p align="center">
<a href="https://github.com/langfuse/langfuse/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-E11311.svg" alt="MIT License">
</a>
<a href="https://www.ycombinator.com/companies/langfuse"><img src="https://img.shields.io/badge/Y%20Combinator-W23-orange" alt="Y Combinator W23"></a>
<a href="https://hub.docker.com/u/langfuse" target="_blank">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/langfuse/langfuse?labelColor=%20%23FDB062&logo=Docker&labelColor=%20%23528bff"></a>
<a href="https://pypi.python.org/pypi/langfuse"><img src="https://img.shields.io/pypi/dm/langfuse?logo=python&logoColor=white&label=pypi%20langfuse&color=blue" alt="langfuse Python package on PyPi"></a>
<a href="https://www.npmjs.com/package/langfuse"><img src="https://img.shields.io/npm/dm/langfuse?logo=npm&logoColor=white&label=npm%20langfuse&color=blue" alt="langfuse npm package"></a>
<br/>
<a href="https://discord.com/invite/7NXusRtqYU" target="_blank">
<img src="https://img.shields.io/discord/1111061815649124414?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://twitter.com/intent/follow?screen_name=langfuse" target="_blank">
<img src="https://img.shields.io/twitter/follow/langfuse?logo=X&color=%20%23f5f5f5"
alt="follow on X(Twitter)"></a>
<a href="https://www.linkedin.com/company/langfuse/" target="_blank">
<img src="https://custom-icon-badges.demolab.com/badge/LinkedIn-0A66C2?logo=linkedin-white&logoColor=fff"
alt="follow on LinkedIn"></a>
<a href="https://github.com/langfuse/langfuse/graphs/commit-activity" target="_blank">
<img alt="Commits last month" src="https://img.shields.io/github/commit-activity/m/langfuse/langfuse?labelColor=%20%2332b583&color=%20%2312b76a"></a>
<a href="https://github.com/langfuse/langfuse/" target="_blank">
<img alt="Issues closed" src="https://img.shields.io/github/issues-search?query=repo%3Alangfuse%2Flangfuse%20is%3Aclosed&label=issues%20closed&labelColor=%20%237d89b0&color=%20%235d6b98"></a>
<a href="https://github.com/langfuse/langfuse/discussions/" target="_blank">
<img alt="Discussion posts" src="https://img.shields.io/github/discussions/langfuse/langfuse?labelColor=%20%239b8afb&color=%20%237a5af8"></a>
</p>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>
<a href="./README.cn.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
<a href="./README.ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-d9d9d9"></a>
<a href="./README.kr.md"><img alt="README in Korean" src="https://img.shields.io/badge/한국어-d9d9d9"></a>
</p>
Langfuseは**オープンソースのLLMエンジニアリング**プラットフォームです。
チームが共同でAIアプリケーションを**開発、監視、評価**、および**デバッグ**するのを支援します。
Langfuseは**数分でセルフホスト可能**で、**多くの実績を持つ**システムです。
[![Langfuse Overview Video](https://github.com/user-attachments/assets/3926b288-ff61-4b95-8aa1-45d041c70866)](https://langfuse.com/watch-demo)
## ✨ コア機能
![Langfuse Overview](https://langfuse.com/images/docs/github-readme/github-feature-overview.png)
- **[LLMアプリケーションの可観測性](https://langfuse.com/docs/tracing):**
アプリケーションにインストゥルメンテーションを導入し、Langfuseへトレースを取り込むことで、LLM呼び出しやリトリーバル、埋め込み、エージェントアクションなどの関連ロジックを追跡できます。
複雑なログやユーザーセッションを解析・デバッグできます。
インタラクティブな[デモ](https://langfuse.com/docs/demo)で動作を確認してください。
- **[プロンプト管理](https://langfuse.com/docs/prompts/get-started):**
プロンプトを一元管理し、バージョン管理しながら共同で改善を行えます。
サーバーおよびクライアント側で強力なキャッシングを行うため、アプリケーションのレイテンシを増やすことなくプロンプトの改良が可能です。
- **[評価](https://langfuse.com/docs/scores/overview):**
評価はLLMアプリケーション開発ワークフローの要であり、Langfuseは多様なニーズに対応します。
LLMを判定者として用いる方法、ユーザーフィードバックの収集、手動によるラベリング、API/SDKを通じたカスタム評価パイプラインをサポートします。
- **[データセット](https://langfuse.com/docs/datasets/overview):**
LLMアプリケーション評価用のテストセットやベンチマークを構築できます。
継続的な改善、事前デプロイテスト、構造化された実験、柔軟な評価、さらにLangChainやLlamaIndexなどとのシームレスな統合をサポートします。
- **[LLMプレイグラウンド](https://langfuse.com/docs/playground):**
プロンプトやモデル設定のテスト・反復作業を支援するツールで、フィードバックループを短縮し開発を加速します。
トレースで不具合が見つかった場合、直接プレイグラウンドへ飛び、迅速に改善できます。
- **[包括的なAPI](https://langfuse.com/docs/api):**
LangfuseはAPIを通じて提供されるビルディングブロックを用い、カスタムLLMOpsワークフローの基盤として頻繁に利用されます。
OpenAPI仕様、Postmanコレクション、PythonやJS/TS向けの型付きSDKが利用可能です。
## 📦 Langfuseのデプロイ
![Langfuse Deployment Options](https://langfuse.com/images/docs/github-readme/github-deployment-options.png)
### Langfuse Cloud
Langfuseチームによるマネージドデプロイメント。充実した無料プラン(ホビープラン)で、クレジットカード不要です。
<div align="center">
<a href="https://cloud.langfuse.com" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/»%20Sign%20up%20for%20Langfuse%20Cloud-8A2BE2?&color=orange">
</a>
</div>
### セルフホスティング Langfuse
自身のインフラ上でLangfuseを実行できます:
- **[Local (docker compose)](https://langfuse.com/self-hosting/local):**
Docker Composeを使用して、たった5分で自分のマシン上でLangfuseを実行できます.
```bash
# 最新のLangfuseリポジトリのコピーを取得
git clone https://github.com/langfuse/langfuse.git
cd langfuse
# Langfuseのdocker composeを起動
docker compose up
```
- **[Kubernetes (Helm)](https://langfuse.com/self-hosting/kubernetes-helm):**
Helmを使用してKubernetesクラスター上でLangfuseを実行します。
こちらが推奨される本番環境でのデプロイ方法です。
- **[VM](https://langfuse.com/self-hosting/docker-compose):**
Docker Composeを使用して、単一の仮想マシン上でLangfuseを実行します。
- Terraform テンプレート: [AWS](https://langfuse.com/self-hosting/aws), [Azure](https://langfuse.com/self-hosting/azure), [GCP](https://langfuse.com/self-hosting/gcp)
[セルフホスティングのドキュメント](https://langfuse.com/self-hosting)を参照し、アーキテクチャや設定オプションの詳細をご確認ください。
## 🔌 インテグレーション
![Langfuse Integrations](https://langfuse.com/images/docs/github-readme/github-integrations.png)
### 主なインテグレーション:
| インテグレーション | 対応言語・環境 | 説明 |
| ----------------------------------------------------------------------------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| [SDK](https://langfuse.com/docs/sdk) | Python, JS/TS | SDKを利用して手動でインストゥルメンテーションを実装し、完全な柔軟性を提供します。 |
| [OpenAI](https://langfuse.com/docs/integrations/openai) | Python, JS/TS | OpenAI SDKのドロップイン置換による自動インストゥルメンテーションを実現します。 |
| [Langchain](https://langfuse.com/docs/integrations/langchain) | Python, JS/TS | Langchainアプリケーションにコールバックハンドラーを渡すことで自動的に計測します。 |
| [LlamaIndex](https://langfuse.com/docs/integrations/llama-index/get-started) | Python | LlamaIndexのコールバックシステムを介して自動的にインストゥルメントします。 |
| [Haystack](https://langfuse.com/docs/integrations/haystack) | Python | Haystackのコンテンツトレースシステムを利用した自動インストゥルメンテーションを実現します。 |
| [LiteLLM](https://langfuse.com/docs/integrations/litellm) | Python, JS/TS (proxy only)| GPTのドロップイン置換として任意のLLMを使用できます。Azure、OpenAI、Cohere、Anthropic、Ollama、VLLM、Sagemaker、HuggingFace、Replicate100+ LLM)に対応。 |
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | React、Next.js、Vue、Svelte、Node.jsを使用してAI搭載アプリケーションの構築を支援するTypeScriptツールキットです。 |
| [API](https://langfuse.com/docs/api) | | 公開APIを直接呼び出すことが可能です。OpenAPI仕様も利用できます。 |
### Langfuseと統合されているパッケージ:
| 名前 | タイプ | 説明 |
| ------------------------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------- |
| [Instructor](https://langfuse.com/docs/integrations/instructor) | ライブラリ | 構造化されたLLM出力(JSON、Pydantic)を取得するためのライブラリ |
| [DSPy](https://langfuse.com/docs/integrations/dspy) | ライブラリ | LLMプロンプトや重み付けを体系的に最適化するためのフレームワーク |
| [Mirascope](https://langfuse.com/docs/integrations/mirascope) | ライブラリ | LLMアプリケーション構築用のPythonツールキット |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | モデル(ローカル) | オープンソースLLMを手軽にローカルで実行するためのツール |
| [Amazon Bedrock](https://langfuse.com/docs/integrations/amazon-bedrock) | モデル | AWS上でファウンデーションモデルやファインチューニング済みモデルを実行 |
| [Google VertexAI and Gemini](https://langfuse.com/docs/integrations/google-vertex-ai) | モデル | Google上でファウンデーションモデルやファインチューニング済みモデルを実行 |
| [AutoGen](https://langfuse.com/docs/integrations/autogen) | エージェントフレームワーク | 分散型エージェント構築のためのオープンソースLLMプラットフォーム |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | チャット/エージェント UI | JS/TSのノーコードビルダーで、カスタマイズ可能なLLMフローを構築 |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | チャット/エージェント UI | PythonベースのUIで、react-flowを用いてLangChainの実験やプロトタイピングを容易に実現 |
| [Dify](https://langfuse.com/docs/integrations/dify) | チャット/エージェント UI | ノーコードでLLMアプリ開発が可能なオープンソースプラットフォーム |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | チャット/エージェント UI | 自前ホストおよびローカルモデルに対応するLLMチャットWeb UI |
| [Promptfoo](https://langfuse.com/docs/integrations/promptfoo) | ツール | オープンソースのLLMテストプラットフォーム |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | チャット/エージェント UI | オープンソースのチャットボットプラットフォーム |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | プラットフォーム | オープンソースの音声AIプラットフォーム |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | エージェント | 分散型エージェント構築のためのオープンソースLLMプラットフォーム |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | チャット/エージェント UI | チャットUIなどのWebインターフェース構築のためのオープンソースPythonライブラリ |
| [Goose](https://langfuse.com/docs/integrations/goose) | エージェント | 分散型エージェント構築のためのオープンソースLLMプラットフォーム |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | エージェント | オープンソースのAIエージェントフレームワーク |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | エージェント | エージェントの協調とツール利用を実現するマルチエージェントフレームワーク |
## 🚀 クイックスタート
アプリケーションにインストゥルメンテーションを導入し、LLM呼び出しやリトリーバル、埋め込み、エージェントアクションなどの動作をLangfuseに記録しましょう。
複雑なログやユーザーセッションの解析・デバッグが可能になります。
### 1️⃣ 新規プロジェクトの作成
1. [Langfuseアカウント作成](https://cloud.langfuse.com/auth/sign-up) または [セルフホスト](https://langfuse.com/self-hosting)
2. 新規プロジェクトを作成
3. プロジェクト設定で新しいAPIクレデンシャルを作成
### 2️⃣ 初めてのLLM呼び出しのログ記録
[`@observe()` デコレーター](https://langfuse.com/docs/sdk/python/decorators)を利用することで、任意のPython製LLMアプリケーションのトレースが簡単に行えます。
このクイックスタートでは、Langfuseの[OpenAI統合](https://langfuse.com/docs/integrations/openai)を使用して、全てのモデルパラメータを自動で取得します。
> [!TIP]
> OpenAIを利用していない場合は、[こちらのドキュメント](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse)で、他のモデルやフレームワークのログ記録方法をご確認ください。
```bash
pip install langfuse openai
```
```bash filename=".env"
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EUリージョン
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 USリージョン
```
```python:/@observe()/ /from langfuse.openai import openai/ filename="main.py"
from langfuse import observe
from langfuse.openai import openai # OpenAI統合
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
```
### 3️⃣ Langfuseでトレースを確認する
Langfuse上で、LLM呼び出しおよびその他のアプリケーションロジックのトレースを確認できます。
![Example trace in Langfuse](https://langfuse.com/images/docs/github-readme/github-example-trace.png)
_[Langfuseの公開トレース例](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!TIP]
>
> Langfuseでのトレースの詳細については、[こちら](https://langfuse.com/docs/tracing)をご参照いただくか、[インタラクティブデモ](https://langfuse.com/docs/demo)でお試しください。
## ⭐️ Star Langfuse
![Star Langfuse](https://langfuse.com/images/docs/github-readme/github-star-howto.gif)
## 💭 サポート
質問の回答をお探しの場合は:
- 当社の[ドキュメント](https://langfuse.com/docs)は、回答を探すための最良の出発点です。内容が充実しており、継続的なメンテナンスに努めています。GitHubを通じてドキュメントへの修正提案も可能です。
- よくある質問は[Langfuse FAQ](https://langfuse.com/faq)にまとめられています。
- [Ask AI](https://langfuse.com/docs/ask-ai)を利用すれば、質問に対して即座に回答を得ることができます。
- 日本語のサポートや決済, 請求書払いなどをお求めの場合は、日本のリセラー (https://gao-ai.com) にご相談ください。
サポートチャネル:
- **GitHub Discussionsの[パブリックQ&A](https://github.com/orgs/langfuse/discussions/categories/support)で質問してください。**
質問には、コードスニペット、スクリーンショット、背景情報など、できるだけ詳細な情報を含めるとスムーズな対応が可能です。
- GitHub Discussionsで[機能リクエスト](https://github.com/orgs/langfuse/discussions/categories/ideas)を投稿してください。
- GitHub Issuesにて[バグ報告](https://github.com/langfuse/langfuse/issues)を行ってください。
- 緊急の問い合わせの場合は、アプリ内チャットウィジェットでご連絡ください。
## 🤝 貢献
皆様からの貢献を歓迎します!
- GitHub Discussionsの[アイデア](https://github.com/orgs/langfuse/discussions/categories/ideas)に投票してください。
- [Issues](https://github.com/langfuse/langfuse/issues)を作成・コメントしてください。
- プルリクエストを送信してください。開発環境のセットアップ方法については[CONTRIBUTING.md](CONTRIBUTING.md)をご参照ください。
## 🥇 ライセンス
このリポジトリは、`ee`フォルダを除き、MITライセンスの下で公開されています。
詳細は[LICENSE](LICENSE)および[オープンソースに関するドキュメント](https://langfuse.com/docs/open-source)をご確認ください。
## ⭐️ スターの履歴
<a href="https://star-history.com/#langfuse/langfuse&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" style="border-radius: 15px;" />
</picture>
</a>
## ❤️ Langfuseを利用しているオープンソースプロジェクト
Langfuseを利用している主要なオープンソースPythonプロジェクト(スター数順): ([出典](https://github.com/langfuse/langfuse-docs/blob/main/components-mdx/dependents))
| リポジトリ | スター |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----: |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/127165244?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langgenius](https://github.com/langgenius) / [dify](https://github.com/langgenius/dify) | 54865 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [open-webui](https://github.com/open-webui/open-webui) | 51531 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/131470832?s=40&v=4" width="20" height="20" alt=""> &nbsp; [lobehub](https://github.com/lobehub) / [lobe-chat](https://github.com/lobehub/lobe-chat) | 49003 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85702467?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langflow-ai](https://github.com/langflow-ai) / [langflow](https://github.com/langflow-ai/langflow) | 39093 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt=""> &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 37368 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/139558948?s=40&v=4" width="20" height="20" alt=""> &nbsp; [chatchat-space](https://github.com/chatchat-space) / [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | 32486 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/128289781?s=40&v=4" width="20" height="20" alt=""> &nbsp; [FlowiseAI](https://github.com/FlowiseAI) / [Flowise](https://github.com/FlowiseAI/Flowise) | 32448 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/31035808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mindsdb](https://github.com/mindsdb) / [mindsdb](https://github.com/mindsdb/mindsdb) | 26931 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/119600397?s=40&v=4" width="20" height="20" alt=""> &nbsp; [twentyhq](https://github.com/twentyhq) / [twenty](https://github.com/twentyhq/twenty) | 24195 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog](https://github.com/PostHog/posthog) | 22618 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/121462774?s=40&v=4" width="20" height="20" alt=""> &nbsp; [BerriAI](https://github.com/BerriAI) / [litellm](https://github.com/BerriAI/litellm) | 15151 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/179202840?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mediar-ai](https://github.com/mediar-ai) / [screenpipe](https://github.com/mediar-ai/screenpipe) | 11037 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/105877416?s=40&v=4" width="20" height="20" alt=""> &nbsp; [formbricks](https://github.com/formbricks) / [formbricks](https://github.com/formbricks/formbricks) | 9386 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/76263028?s=40&v=4" width="20" height="20" alt=""> &nbsp; [anthropics](https://github.com/anthropics) / [courses](https://github.com/anthropics/courses) | 8385 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/78410652?s=40&v=4" width="20" height="20" alt=""> &nbsp; [GreyDGL](https://github.com/GreyDGL) / [PentestGPT](https://github.com/GreyDGL/PentestGPT) | 7374 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt=""> &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5391 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/137907881?s=40&v=4" width="20" height="20" alt=""> &nbsp; [promptfoo](https://github.com/promptfoo) / [promptfoo](https://github.com/promptfoo/promptfoo) | 4976 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/157326433?s=40&v=4" width="20" height="20" alt=""> &nbsp; [onlook-dev](https://github.com/onlook-dev) / [onlook](https://github.com/onlook-dev/onlook) | 4141 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/7250217?s=40&v=4" width="20" height="20" alt=""> &nbsp; [Canner](https://github.com/Canner) / [WrenAI](https://github.com/Canner/WrenAI) | 2526 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/11855343?s=40&v=4" width="20" height="20" alt=""> &nbsp; [pingcap](https://github.com/pingcap) / [autoflow](https://github.com/pingcap/autoflow) | 2061 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85268109?s=40&v=4" width="20" height="20" alt=""> &nbsp; [MLSysOps](https://github.com/MLSysOps) / [MLE-agent](https://github.com/MLSysOps/MLE-agent) | 1161 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [pipelines](https://github.com/open-webui/pipelines) | 1100 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18422723?s=40&v=4" width="20" height="20" alt=""> &nbsp; [alishobeiri](https://github.com/alishobeiri) / [thread](https://github.com/alishobeiri/thread) | 1074 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/125468716?s=40&v=4" width="20" height="20" alt=""> &nbsp; [topoteretes](https://github.com/topoteretes) / [cognee](https://github.com/topoteretes/cognee) | 971 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/188657705?s=40&v=4" width="20" height="20" alt=""> &nbsp; [bRAGAI](https://github.com/bRAGAI) / [bRAG-langchain](https://github.com/bRAGAI/bRAG-langchain) | 823 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169500408?s=40&v=4" width="20" height="20" alt=""> &nbsp; [opslane](https://github.com/opslane) / [opslane](https://github.com/opslane/opslane) | 677 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/151867818?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dynamiq-ai](https://github.com/dynamiq-ai) / [dynamiq](https://github.com/dynamiq-ai/dynamiq) | 639 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/48585267?s=40&v=4" width="20" height="20" alt=""> &nbsp; [theopenconversationkit](https://github.com/theopenconversationkit) / [tock](https://github.com/theopenconversationkit/tock) | 514 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/20493493?s=40&v=4" width="20" height="20" alt=""> &nbsp; [andysingal](https://github.com/andysingal) / [llm-course](https://github.com/andysingal/llm-course) | 394 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/132396805?s=40&v=4" width="20" height="20" alt=""> &nbsp; [phospho-app](https://github.com/phospho-app) / [phospho](https://github.com/phospho-app/phospho) | 384 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/178644984?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sentient-engineering](https://github.com/sentient-engineering) / [agent-q](https://github.com/sentient-engineering/agent-q) | 370 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/168552753?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sql-agi](https://github.com/sql-agi) / [DB-GPT](https://github.com/sql-agi/DB-GPT) | 324 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog-foss](https://github.com/PostHog/posthog-foss) | 305 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/154247157?s=40&v=4" width="20" height="20" alt=""> &nbsp; [vespperhq](https://github.com/vespperhq) / [vespper](https://github.com/vespperhq/vespper) | 304 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/185116535?s=40&v=4" width="20" height="20" alt=""> &nbsp; [block](https://github.com/block) / [goose](https://github.com/block/goose) | 295 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/609489?s=40&v=4" width="20" height="20" alt=""> &nbsp; [aorwall](https://github.com/aorwall) / [moatless-tools](https://github.com/aorwall/moatless-tools) | 291 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2357342?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dmayboroda](https://github.com/dmayboroda) / [minima](https://github.com/dmayboroda/minima) | 221 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/66303003?s=40&v=4" width="20" height="20" alt=""> &nbsp; [RobotecAI](https://github.com/RobotecAI) / [rai](https://github.com/RobotecAI/rai) | 172 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/148684274?s=40&v=4" width="20" height="20" alt=""> &nbsp; [i-am-alice](https://github.com/i-am-alice) / [3rd-devs](https://github.com/i-am-alice/3rd-devs) | 148 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/171735272?s=40&v=4" width="20" height="20" alt=""> &nbsp; [8090-inc](https://github.com/8090-inc) / [xrx-sample-apps](https://github.com/8090-inc/xrx-sample-apps) | 138 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/104478511?s=40&v=4" width="20" height="20" alt=""> &nbsp; [babelcloud](https://github.com/babelcloud) / [LLM-RGB](https://github.com/babelcloud/LLM-RGB) | 135 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/15125613?s=40&v=4" width="20" height="20" alt=""> &nbsp; [souzatharsis](https://github.com/souzatharsis) / [tamingLLMs](https://github.com/souzatharsis/tamingLLMs) | 129 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169401942?s=40&v=4" width="20" height="20" alt=""> &nbsp; [LibreChat-AI](https://github.com/LibreChat-AI) / [librechat.ai](https://github.com/LibreChat-AI/librechat.ai) | 128 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/51827949?s=40&v=4" width="20" height="20" alt=""> &nbsp; [deepset-ai](https://github.com/deepset-ai) / [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) | 126 |
## 🔒 セキュリティとプライバシー
データのセキュリティとプライバシーは非常に重要です。
詳細につきましては、[セキュリティとプライバシー](https://langfuse.com/security)ページをご参照ください。
### テレメトリー
デフォルトでは、Langfuseは以下の目的でセルフホストされたインスタンスの基本的な使用統計情報を中央サーバ(PostHog)へ自動的に報告します。
1. Langfuseの利用状況を把握し、最も重要な機能の改善に役立てる
2. 内部および外部(例:資金調達)のレポートのために全体の利用状況を追跡する
収集されたデータは第三者と共有されず、機微な情報は一切含まれていません。
当社はこの点について極力透明性を保っており、収集される具体的なデータの詳細は[こちら](/web/src/features/telemetry/index.ts)で確認できます。
`TELEMETRY_ENABLED=false` を設定することで、テレメトリーの報告をオプトアウトできます.
+340
View File
@@ -0,0 +1,340 @@
![Langfuse GitHub Banner](https://langfuse.com/images/docs/github-readme/github-banner.png)
<div align="center">
<div>
<h3>
<a href="https://langfuse.com/kr">
<strong>🇰🇷 🤝 🪢</strong>
</a> ·
<a href="https://cloud.langfuse.com">
<strong>Langfuse Cloud</strong>
</a> ·
<a href="https://langfuse.com/docs/deployment/self-host">
<strong>셀프 호스트</strong>
</a> ·
<a href="https://langfuse.com/demo">
<strong>데모</strong>
</a>
</h3>
</div>
<div>
<a href="https://langfuse.com/docs"><strong>문서</strong></a> ·
<a href="https://langfuse.com/issues"><strong>버그 신고</strong></a> ·
<a href="https://langfuse.com/ideas"><strong>기능 요청</strong></a> ·
<a href="https://langfuse.com/changelog"><strong>변경 내역</strong></a> ·
<a href="https://langfuse.com/roadmap"><strong>로드맵</strong></a> ·
</div>
<br/>
<span>Langfuse는 지원 및 기능 요청을 위해 <a href="https://github.com/orgs/langfuse/discussions"><strong>GitHub Discussions</strong></a>를 사용합니다.</span>
<br/>
<span><b>채용 중입니다.</b> <a href="https://langfuse.com/careers"><strong>함께 하세요</strong></a> 제품 엔지니어링 및 기술 go-to-market 역할의 인재를 찾고 있습니다.</span>
<br/>
<br/>
<div>
</div>
</div>
<p align="center">
<a href="https://github.com/langfuse/langfuse/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-E11311.svg" alt="MIT License">
</a>
<a href="https://www.ycombinator.com/companies/langfuse"><img src="https://img.shields.io/badge/Y%20Combinator-W23-orange" alt="Y Combinator W23"></a>
<a href="https://hub.docker.com/u/langfuse" target="_blank">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/langfuse/langfuse?labelColor=%20%23FDB062&logo=Docker&labelColor=%20%23528bff"></a>
<a href="https://pypi.python.org/pypi/langfuse"><img src="https://img.shields.io/pypi/dm/langfuse?logo=python&logoColor=white&label=pypi%20langfuse&color=blue" alt="langfuse Python package on PyPi"></a>
<a href="https://www.npmjs.com/package/langfuse"><img src="https://img.shields.io/npm/dm/langfuse?logo=npm&logoColor=white&label=npm%20langfuse&color=blue" alt="langfuse npm package"></a>
<br/>
<a href="https://discord.com/invite/7NXusRtqYU" target="_blank">
<img src="https://img.shields.io/discord/1111061815649124414?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://twitter.com/intent/follow?screen_name=langfuse" target="_blank">
<img src="https://img.shields.io/twitter/follow/langfuse?logo=X&color=%20%23f5f5f5"
alt="follow on X(Twitter)"></a>
<a href="https://www.linkedin.com/company/langfuse/" target="_blank">
<img src="https://custom-icon-badges.demolab.com/badge/LinkedIn-0A66C2?logo=linkedin-white&logoColor=fff"
alt="follow on LinkedIn"></a>
<a href="https://github.com/langfuse/langfuse/graphs/commit-activity" target="_blank">
<img alt="Commits last month" src="https://img.shields.io/github/commit-activity/m/langfuse/langfuse?labelColor=%20%2332b583&color=%20%2312b76a"></a>
<a href="https://github.com/langfuse/langfuse/" target="_blank">
<img alt="Issues closed" src="https://img.shields.io/github/issues-search?query=repo%3Alangfuse%2Flangfuse%20is%3Aclosed&label=issues%20closed&labelColor=%20%237d89b0&color=%20%235d6b98"></a>
<a href="https://github.com/langfuse/langfuse/discussions/" target="_blank">
<img alt="Discussion posts" src="https://img.shields.io/github/discussions/langfuse/langfuse?labelColor=%20%239b8afb&color=%20%237a5af8"></a>
</p>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>
<a href="./README.cn.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
<a href="./README.ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-d9d9d9"></a>
<a href="./README.kr.md"><img alt="README in Korean" src="https://img.shields.io/badge/한국어-d9d9d9"></a>
</p>
Langfuse는 **오픈 소스 LLM 엔지니어링** 플랫폼입니다.
팀이 협업하여 AI 애플리케이션을 **개발, 모니터링, 평가** 및 **디버그**할 수 있도록 도와줍니다.
Langfuse는 몇 분 안에 셀프 호스팅할 수 있으며, 검증된(battle-tested) 솔루션입니다.
[![Langfuse Overview Video](https://github.com/user-attachments/assets/3926b288-ff61-4b95-8aa1-45d041c70866)](https://langfuse.com/watch-demo)
## ✨ 주요 기능
![Langfuse Overview](https://langfuse.com/images/docs/github-readme/github-feature-overview.png)
- **LLM 애플리케이션 관측**
앱에 계측(instrumentation)을 추가하여 Langfuse로 trace 데이터를 수집함으로써, 검색, 임베딩, 또는 에이전트 동작과 같은 LLM 호출 및 기타 관련 로직을 추적할 수 있습니다. 복잡한 로그와 사용자 세션을 확인 및 디버깅 해보세요. 인터랙티브 데모를 통해 실제 작동 예를 확인할 수 있습니다.
- **프롬프트 관리**
프롬프트를 중앙에서 관리하고 버전 관리하며 협업으로 수정할 수 있도록 도와줍니다. 서버와 클라이언트 측의 강력한 캐싱 덕분에 애플리케이션에 지연(latency)을 추가하지 않고도 프롬프트를 반복 개선할 수 있습니다.
- **평가**
LLM 애플리케이션 개발 워크플로우에서 핵심적인 역할을 하며, Langfuse는 여러분의 필요에 맞게 유연하게 대응합니다. LLM을 심사자로 활용하는 기능, 사용자 피드백 수집, 수동 라벨링 및 API/SDK를 통한 맞춤 평가 파이프라인을 지원합니다.
- **데이터셋**
LLM 애플리케이션 평가를 위한 테스트 세트와 벤치마크를 제공하여, 지속적인 개선, 배포 전 테스트, 구조화된 실험, 유연한 평가 및 LangChain과 LlamaIndex와 같은 프레임워크와의 원활한 통합을 지원합니다.
- **LLM 플레이그라운드**
프롬프트와 모델 구성에 대해 테스트 및 반복 개선할 수 있는 도구로, 피드백 루프를 단축하여 개발 속도를 높여줍니다. trace에서 이상한 결과가 발생하면 플레이그라운드로 바로 이동해 개선할 수 있습니다.
- **종합 API**
Langfuse는 API를 통해 제공되는 구성 요소들을 활용하여 맞춤형 LLMOps 워크플로우를 강화하는 데 자주 사용됩니다. OpenAPI 명세, Postman 컬렉션, Python 및 JS/TS용 타입드 SDK가 제공됩니다.
## 📦 Langfuse 배포
![Langfuse Deployment Options](https://langfuse.com/images/docs/github-readme/github-deployment-options.png)
### Langfuse Cloud
Langfuse 팀이 관리하는 배포 방식으로, 후한 무료 플랜(취미 플랜)을 제공하며 신용카드가 필요하지 않습니다.
<div align="center">
<a href="https://cloud.langfuse.com" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/»%20Sign%20up%20for%20Langfuse%20Cloud-8A2BE2?&color=orange">
</a>
</div>
### Langfuse 셀프 호스트
자체 인프라에서 Langfuse를 실행하세요:
- [로컬 (docker compose)](https://langfuse.com/self-hosting/local): Docker Compose를 사용하여 본인의 컴퓨터에서 5분 안에 Langfuse를 실행할 수 있습니다.
```bash
# 최신 Langfuse 저장소 클론
git clone https://github.com/langfuse/langfuse.git
cd langfuse
# langfuse docker compose 실행
docker compose up
```
- [Kubernetes (Helm)](https://langfuse.com/self-hosting/kubernetes-helm): Helm을 사용해 Kubernetes 클러스터에서 Langfuse를 실행합니다. 이는 권장되는 프로덕션 배포 방식입니다.
- [VM](https://langfuse.com/self-hosting/docker-compose): Docker Compose를 사용해 단일 가상 머신에서 Langfuse를 실행합니다.
- Terraform 템플릿: [AWS](https://langfuse.com/self-hosting/aws), [Azure](https://langfuse.com/self-hosting/azure), [GCP](https://langfuse.com/self-hosting/gcp)
자세한 내용은 [자체 호스팅 문서](https://langfuse.com/self-hosting)를 참조하세요.
## 🔌 통합 기능
![Langfuse Integrations](https://langfuse.com/images/docs/github-readme/github-integrations.png)
### 주요 통합:
| 통합 | 지원 | 설명 |
| --------------------------------------------------------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| [SDK](https://langfuse.com/docs/sdk) | Python, JS/TS | SDK를 사용하여 완전한 유연성을 갖춘 수동 계측(manual instrumentation)을 수행합니다. |
| [OpenAI](https://langfuse.com/docs/integrations/openai) | Python, JS/TS | OpenAI SDK의 드롭인 대체(drop-in replacement)를 통해 자동 계측(automated instrumentation)을 수행합니다. |
| [Langchain](https://langfuse.com/docs/integrations/langchain) | Python, JS/TS | Langchain 애플리케이션에 callback 핸들러를 전달하여 자동 계측합니다. |
| [LlamaIndex](https://langfuse.com/docs/integrations/llama-index/get-started) | Python | LlamaIndex 콜백 시스템을 통한 자동 계측을 지원합니다. |
| [Haystack](https://langfuse.com/docs/integrations/haystack) | Python | Haystack 콘텐츠 추적 시스템을 통한 자동 계측을 지원합니다. |
| [LiteLLM](https://langfuse.com/docs/integrations/litellm) | Python, JS/TS (proxy only) | GPT의 드롭인 대체품으로 어떤 LLM도 사용할 수 있습니다. Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate 등 100개 이상의 LLM 지원. |
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | React, Next.js, Vue, Svelte, Node.js와 함께 AI 기반 애플리케이션 구축을 돕는 TypeScript 툴킷입니다. |
| [API](https://langfuse.com/docs/api) | | 공개 API를 직접 호출합니다. OpenAPI 명세가 제공됩니다. |
### Langfuse와 통합된 패키지:
| 이름 | 유형 | 설명 |
| ----------------------------------------------------------------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------- |
| [Instructor](https://langfuse.com/docs/integrations/instructor) | 라이브러리 | 구조화된 LLM 출력을(JSON, Pydantic) 얻기 위한 라이브러리입니다. |
| [DSPy](https://langfuse.com/docs/integrations/dspy) | 라이브러리 | 언어 모델 프롬프트와 가중치를 체계적으로 최적화하는 프레임워크입니다. |
| [Mirascope](https://langfuse.com/docs/integrations/mirascope) | 라이브러리 | LLM 애플리케이션 구축을 위한 Python 툴킷입니다. |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | 모델 (로컬) | 자신의 컴퓨터에서 오픈 소스 LLM을 손쉽게 실행할 수 있습니다. |
| [Amazon Bedrock](https://langfuse.com/docs/integrations/amazon-bedrock) | 모델 | AWS에서 기본 및 파인튜닝된 모델을 실행합니다. |
| [Google VertexAI and Gemini](https://langfuse.com/docs/integrations/google-vertex-ai) | 모델 | Google에서 기본 및 파인튜닝된 모델을 실행합니다. |
| [AutoGen](https://langfuse.com/docs/integrations/autogen) | 에이전트 프레임워크 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | 채팅/에이전트 UI | 맞춤형 LLM 플로우를 위한 JS/TS 코드 없는(no-code) 빌더입니다. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | 채팅/에이전트 UI | react-flow를 활용하여 실험 및 프로토타이핑을 손쉽게 할 수 있도록 디자인된 LangChain용 Python 기반 UI입니다. |
| [Dify](https://langfuse.com/docs/integrations/dify) | 채팅/에이전트 UI | 코드 없는 빌더와 함께 제공되는 오픈 소스 LLM 애플리케이션 개발 플랫폼입니다. |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | 채팅/에이전트 UI | 셀프 호스팅 및 로컬 모델 등 다양한 LLM 실행기를 지원하는 셀프 호스팅 LLM 채팅 웹 UI입니다. |
| [Promptfoo](https://langfuse.com/docs/integrations/promptfoo) | 도구 | 오픈 소스 LLM 테스트 플랫폼입니다. |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | 채팅/에이전트 UI | 오픈 소스 챗봇 플랫폼입니다. |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | 플랫폼 | 오픈 소스 음성 AI 플랫폼입니다. |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | 에이전트 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | 채팅/에이전트 UI | 채팅 UI와 같은 웹 인터페이스 구축을 위한 오픈 소스 Python 라이브러리입니다. |
| [Goose](https://langfuse.com/docs/integrations/goose) | 에이전트 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | 에이전트 | 오픈 소스 AI 에이전트 프레임워크입니다. |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | 에이전트 | 에이전트 간 협업 및 도구 사용을 위한 다중 에이전트 프레임워크입니다. |
## 🚀 빠른 시작
앱에 계측을 추가하고 Langfuse에 trace 데이터를 수집하여, LLM 호출 및 검색, 임베딩, 에이전트 동작과 같은 애플리케이션 로직을 추적해보세요. 복잡한 로그와 사용자 세션을 확인하여 디버깅할 수 있습니다.
### 1️⃣ 새 프로젝트 생성
1. [Langfuse 계정 생성](https://cloud.langfuse.com/auth/sign-up) 또는 [셀프 호스트](https://langfuse.com/self-hosting)
2. 새 프로젝트를 생성합니다.
3. 프로젝트 설정에서 새로운 API 자격 증명을 생성합니다.
### 2️⃣ 첫 번째 LLM 호출 기록하기
[`@observe()` 데코레이터](https://langfuse.com/docs/sdk/python/decorators)를 사용하면 Python LLM 애플리케이션의 추적이 매우 간편해집니다. 이 빠른 시작 예제에서는 Langfuse [OpenAI 통합](https://langfuse.com/docs/integrations/openai)을 사용하여 모든 모델 파라미터를 자동으로 캡처합니다.
> [!TIP]
> OpenAI를 사용하지 않으시다면, 다른 모델 및 프레임워크의 로그 기록 방법은 [문서](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse)를 참조하세요.
```bash
pip install langfuse openai
```
```bash filename=".env"
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EU region
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 US region
```
```python:main.py
from langfuse import observe
from langfuse.openai import openai # OpenAI integration
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
```
### 3️⃣ Langfuse에서 trace 확인하기
Langfuse에서 LLM 호출 및 애플리케이션의 기타 로직에 대한 trace를 확인할 수 있습니다.
![Example trace in Langfuse](https://langfuse.com/images/docs/github-readme/github-example-trace.png)
_[Langfuse의 공개 예제 trace](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!TIP]
>
> Langfuse의 trace에 대해 더 알아보거나 [인터랙티브 데모](https://langfuse.com/docs/demo)에서 직접 체험해보세요.
## ⭐️ 별을 눌러주세요
![Star Langfuse](https://langfuse.com/images/docs/github-readme/github-star-howto.gif)
## 💭 지원
질문에 대한 답변을 찾는 방법:
- 우리의 [문서](https://langfuse.com/docs)는 답을 찾기 위한 최적의 장소입니다. 문서가 매우 포괄적이며, 유지보수에 많은 노력을 기울이고 있습니다. GitHub를 통해 문서 수정 제안도 가능합니다.
- [Langfuse FAQ](https://langfuse.com/faq)에서는 가장 흔한 질문에 대해 답변하고 있습니다.
- 질문에 즉각적인 답변이 필요하다면 [Ask AI](https://langfuse.com/docs/ask-ai)를 사용해보세요.
지원 채널:
- **GitHub Discussions의 [공개 Q&A](https://github.com/orgs/langfuse/discussions/categories/support)** 에 질문을 남겨주세요. 가능한 한 많은 세부 사항(예: 코드 스니펫, 스크린샷, 배경 정보)을 포함해 질문해 주시기 바랍니다.
- [기능 요청](https://github.com/orgs/langfuse/discussions/categories/ideas)을 남겨주세요.
- [버그 신고](https://github.com/langfuse/langfuse/issues)는 GitHub Issues를 통해 해주세요.
- 긴급한 문의는 앱 내 채팅 위젯을 통해 연락 바랍니다.
## 🤝 기여하기
여러분의 기여를 환영합니다!
- GitHub Discussions의 [아이디어](https://github.com/orgs/langfuse/discussions/categories/ideas)에 투표해보세요.
- GitHub Issues에서 이슈를 제기하고 댓글을 남겨주세요.
- PR을 제출하세요 – 개발 환경 설정 방법 등 자세한 내용은 [CONTRIBUTING.md](CONTRIBUTING.md)를 참조하세요.
## 🥇 라이선스
이 저장소는 `ee` 폴더를 제외하고 MIT 라이선스가 적용됩니다. 자세한 내용은 [LICENSE](LICENSE)와 [문서](https://langfuse.com/docs/open-source)를 확인하세요.
## ⭐️ 별(Star) 히스토리
<a href="https://star-history.com/#langfuse/langfuse&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" style="border-radius: 15px;" />
</picture>
</a>
## ❤️ Langfuse를 사용하는 오픈 소스 프로젝트
별(star) 수를 기준으로 순위가 매겨진 Langfuse를 사용하는 상위 오픈 소스 Python 프로젝트들 ([출처](https://github.com/langfuse/langfuse-docs/blob/main/components-mdx/dependents)):
| 저장소 | 별 |
| :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---: |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/127165244?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langgenius](https://github.com/langgenius) / [dify](https://github.com/langgenius/dify) | 54865 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [open-webui](https://github.com/open-webui/open-webui) | 51531 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/131470832?s=40&v=4" width="20" height="20" alt=""> &nbsp; [lobehub](https://github.com/lobehub) / [lobe-chat](https://github.com/lobehub/lobe-chat) | 49003 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85702467?s=40&v=4" width="20" height="20" alt=""> &nbsp; [langflow-ai](https://github.com/langflow-ai) / [langflow](https://github.com/langflow-ai/langflow) | 39093 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt=""> &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 37368 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/139558948?s=40&v=4" width="20" height="20" alt=""> &nbsp; [chatchat-space](https://github.com/chatchat-space) / [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | 32486 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/128289781?s=40&v=4" width="20" height="20" alt=""> &nbsp; [FlowiseAI](https://github.com/FlowiseAI) / [Flowise](https://github.com/FlowiseAI/Flowise) | 32448 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/31035808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mindsdb](https://github.com/mindsdb) / [mindsdb](https://github.com/mindsdb/mindsdb) | 26931 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/119600397?s=40&v=4" width="20" height="20" alt=""> &nbsp; [twentyhq](https://github.com/twentyhq) / [twenty](https://github.com/twentyhq/twenty) | 24195 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog](https://github.com/PostHog/posthog) | 22618 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/121462774?s=40&v=4" width="20" height="20" alt=""> &nbsp; [BerriAI](https://github.com/BerriAI) / [litellm](https://github.com/BerriAI/litellm) | 15151 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/179202840?s=40&v=4" width="20" height="20" alt=""> &nbsp; [mediar-ai](https://github.com/mediar-ai) / [screenpipe](https://github.com/mediar-ai/screenpipe) | 11037 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/105877416?s=40&v=4" width="20" height="20" alt=""> &nbsp; [formbricks](https://github.com/formbricks) / [formbricks](https://github.com/formbricks/formbricks) | 9386 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/76263028?s=40&v=4" width="20" height="20" alt=""> &nbsp; [anthropics](https://github.com/anthropics) / [courses](https://github.com/anthropics/courses) | 8385 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/78410652?s=40&v=4" width="20" height="20" alt=""> &nbsp; [GreyDGL](https://github.com/GreyDGL) / [PentestGPT](https://github.com/GreyDGL/PentestGPT) | 7374 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt=""> &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5391 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/137907881?s=40&v=4" width="20" height="20" alt=""> &nbsp; [promptfoo](https://github.com/promptfoo) / [promptfoo](https://github.com/promptfoo/promptfoo) | 4976 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/157326433?s=40&v=4" width="20" height="20" alt=""> &nbsp; [onlook-dev](https://github.com/onlook-dev) / [onlook](https://github.com/onlook-dev/onlook) | 4141 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/7250217?s=40&v=4" width="20" height="20" alt=""> &nbsp; [Canner](https://github.com/Canner) / [WrenAI](https://github.com/Canner/WrenAI) | 2526 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/11855343?s=40&v=4" width="20" height="20" alt=""> &nbsp; [pingcap](https://github.com/pingcap) / [autoflow](https://github.com/pingcap/autoflow) | 2061 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/85268109?s=40&v=4" width="20" height="20" alt=""> &nbsp; [MLSysOps](https://github.com/MLSysOps) / [MLE-agent](https://github.com/MLSysOps/MLE-agent) | 1161 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/158137808?s=40&v=4" width="20" height="20" alt=""> &nbsp; [open-webui](https://github.com/open-webui) / [pipelines](https://github.com/open-webui/pipelines) | 1100 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18422723?s=40&v=4" width="20" height="20" alt=""> &nbsp; [alishobeiri](https://github.com/alishobeiri) / [thread](https://github.com/alishobeiri/thread) | 1074 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/125468716?s=40&v=4" width="20" height="20" alt=""> &nbsp; [topoteretes](https://github.com/topoteretes) / [cognee](https://github.com/topoteretes/cognee) | 971 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/188657705?s=40&v=4" width="20" height="20" alt=""> &nbsp; [bRAGAI](https://github.com/bRAGAI) / [bRAG-langchain](https://github.com/bRAGAI/bRAG-langchain) | 823 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169500408?s=40&v=4" width="20" height="20" alt=""> &nbsp; [opslane](https://github.com/opslane) / [opslane](https://github.com/opslane/opslane) | 677 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/151867818?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dynamiq-ai](https://github.com/dynamiq-ai) / [dynamiq](https://github.com/dynamiq-ai/dynamiq) | 639 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/48585267?s=40&v=4" width="20" height="20" alt=""> &nbsp; [theopenconversationkit](https://github.com/theopenconversationkit) / [tock](https://github.com/theopenconversationkit/tock) | 514 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/20493493?s=40&v=4" width="20" height="20" alt=""> &nbsp; [andysingal](https://github.com/andysingal) / [llm-course](https://github.com/andysingal/llm-course) | 394 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/132396805?s=40&v=4" width="20" height="20" alt=""> &nbsp; [phospho-app](https://github.com/phospho-app) / [phospho](https://github.com/phospho-app/phospho) | 384 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/178644984?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sentient-engineering](https://github.com/sentient-engineering) / [agent-q](https://github.com/sentient-engineering/agent-q) | 370 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/168552753?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sql-agi](https://github.com/sql-agi) / [DB-GPT](https://github.com/sql-agi/DB-GPT) | 324 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/60330232?s=40&v=4" width="20" height="20" alt=""> &nbsp; [PostHog](https://github.com/PostHog) / [posthog-foss](https://github.com/PostHog/posthog-foss) | 305 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/154247157?s=40&v=4" width="20" height="20" alt=""> &nbsp; [vespperhq](https://github.com/vespperhq) / [vespper](https://github.com/vespperhq/vespper) | 304 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/185116535?s=40&v=4" width="20" height="20" alt=""> &nbsp; [block](https://github.com/block) / [goose](https://github.com/block/goose) | 295 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/609489?s=40&v=4" width="20" height="20" alt=""> &nbsp; [aorwall](https://github.com/aorwall) / [moatless-tools](https://github.com/aorwall/moatless-tools) | 291 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2357342?s=40&v=4" width="20" height="20" alt=""> &nbsp; [dmayboroda](https://github.com/dmayboroda) / [minima](https://github.com/dmayboroda/minima) | 221 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/66303003?s=40&v=4" width="20" height="20" alt=""> &nbsp; [RobotecAI](https://github.com/RobotecAI) / [rai](https://github.com/RobotecAI/rai) | 172 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/148684274?s=40&v=4" width="20" height="20" alt=""> &nbsp; [i-am-alice](https://github.com/i-am-alice) / [3rd-devs](https://github.com/i-am-alice/3rd-devs) | 148 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/171735272?s=40&v=4" width="20" height="20" alt=""> &nbsp; [8090-inc](https://github.com/8090-inc) / [xrx-sample-apps](https://github.com/8090-inc/xrx-sample-apps) | 138 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/104478511?s=40&v=4" width="20" height="20" alt=""> &nbsp; [babelcloud](https://github.com/babelcloud) / [LLM-RGB](https://github.com/babelcloud/LLM-RGB) | 135 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/15125613?s=40&v=4" width="20" height="20" alt=""> &nbsp; [souzatharsis](https://github.com/souzatharsis) / [tamingLLMs](https://github.com/souzatharsis/tamingLLMs) | 129 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169401942?s=40&v=4" width="20" height="20" alt=""> &nbsp; [LibreChat-AI](https://github.com/LibreChat-AI) / [librechat.ai](https://github.com/LibreChat-AI/librechat.ai) | 128 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/51827949?s=40&v=4" width="20" height="20" alt=""> &nbsp; [deepset-ai](https://github.com/deepset-ai) / [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) | 126 |
## 🔒 보안 & 개인정보 보호
우리는 데이터 보안과 개인정보 보호를 매우 중요하게 생각합니다. 자세한 내용은 [Security and Privacy](https://langfuse.com/security) 페이지를 참조하세요.
### 텔레메트리
기본적으로 Langfuse는 자체 호스팅 인스턴스의 기본 사용 통계를 중앙 서버(PostHog)로 자동 보고합니다.
이를 통해:
1. Langfuse가 어떻게 사용되는지 이해하고, 가장 관련성 높은 기능을 개선할 수 있습니다.
2. 내부 및 외부(예: 자금 조달) 보고를 위한 전체 사용량을 추적할 수 있습니다.
수집된 데이터는 제3자와 공유되지 않으며 민감한 정보를 포함하지 않습니다. 이에 대해 매우 투명하게 공개하고 있으며, 수집되는 정확한 데이터는 [여기](/web/src/features/telemetry/index.ts)에서 확인할 수 있습니다.
환경 변수 `TELEMETRY_ENABLED=false`를 설정하여 옵트아웃할 수 있습니다.
+200 -110
View File
@@ -1,113 +1,134 @@
![Langfuse GitHub Banner](https://github.com/langfuse/langfuse/assets/121163007/6035f0f3-d691-4963-b5d0-10cf506e9d42)
<div align="center"><h1>Langfuse: Open Source LLM Engineering Platform</h1></div>
<div align="center"><h4>LLM Observability, Prompt Management, LLM Evaluations,<br/>Datasets, LLM Metrics, and Prompt Playground</h4></div>
![Langfuse GitHub Banner](https://langfuse.com/images/docs/github-readme/github-banner.png)
<div align="center">
<div>
<h3>
<a href="https://langfuse.com/blog/2025-06-04-open-sourcing-langfuse-product">
<strong>Langfuse Is Doubling Down On Open Source</strong>
</a> <br> <br>
<a href="https://cloud.langfuse.com">
<strong>Sign up</strong>
<strong>Langfuse Cloud</strong>
</a> ·
<a href="https://langfuse.com/docs/deployment/self-host">
<strong>Self Host</strong>
</a> ·
<a href="https://langfuse.com/demo">
<strong>Demo (live data)</strong>
<strong>Demo</strong>
</a>
</h3>
</div>
<div>
<a href="https://langfuse.com/docs"><strong>Docs</strong></a> ·
<a href="https://langfuse.com/issues"><strong>Report Bug</strong></a> ·
<a href="https://langfuse.com/ideas"><strong>Feature Request</strong></a> ·
<a href="https://langfuse.com/changelog"><strong>Changelog</strong></a> ·
<a href="https://langfuse.com/roadmap"><strong>Roadmap</strong></a> ·
<a href="https://langfuse.com/discord"><strong>Discord</strong></a>
</div>
<span>Langfuse uses <a href="https://github.com/orgs/langfuse/discussions"><strong>Github Discussions</strong></a> for Support and Feature Requests.</span>
<br/>
<span>We're hiring. <a href="https://langfuse.com/careers"><strong>Join us</strong></a> in Product Engineering and Developer Relations.</span>
<span>Langfuse uses <a href="https://github.com/orgs/langfuse/discussions"><strong>GitHub Discussions</strong></a> for Support and Feature Requests.</span>
<br/>
<span><b>We're hiring.</b> <a href="https://langfuse.com/careers"><strong>Join us</strong></a> in product engineering and technical go-to-market roles.</span>
<br/>
<br/>
<div>
<a href="https://github.com/langfuse/langfuse/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-red.svg?style=flat-square" alt="MIT License"></a>
<a href="https://www.ycombinator.com/companies/langfuse"><img src="https://img.shields.io/badge/Y%20Combinator-W23-orange?style=flat-square" alt="Y Combinator W23"></a>
<a href="https://github.com/langfuse/langfuse/pkgs/container/langfuse"><img alt="Docker Image" src="https://img.shields.io/badge/docker-langfuse-blue?logo=Docker&logoColor=white&style=flat-square"></a>
<a href="https://pypi.python.org/pypi/langfuse"><img src="https://img.shields.io/pypi/dm/langfuse?style=flat-square&logo=python&logoColor=white&label=pypi%20langfuse&color=blue" alt="langfuse Python package on PyPi"></a>
<a href="https://www.npmjs.com/package/langfuse"><img src="https://img.shields.io/npm/dm/langfuse?style=flat-square&logo=npm&logoColor=white&label=npm%20langfuse&color=blue" alt="langfuse npm package"></a>
</div>
</div>
</br>
## Langfuse Overview
<p align="center">
<a href="https://github.com/langfuse/langfuse/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-E11311.svg" alt="MIT License">
</a>
<a href="https://www.ycombinator.com/companies/langfuse"><img src="https://img.shields.io/badge/Y%20Combinator-W23-orange" alt="Y Combinator W23"></a>
<a href="https://hub.docker.com/u/langfuse" target="_blank">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/langfuse/langfuse?labelColor=%20%23FDB062&logo=Docker&labelColor=%20%23528bff"></a>
<a href="https://pypi.python.org/pypi/langfuse"><img src="https://img.shields.io/pypi/dm/langfuse?logo=python&logoColor=white&label=pypi%20langfuse&color=blue" alt="langfuse Python package on PyPi"></a>
<a href="https://www.npmjs.com/package/langfuse"><img src="https://img.shields.io/npm/dm/langfuse?logo=npm&logoColor=white&label=npm%20langfuse&color=blue" alt="langfuse npm package"></a>
<br/>
<a href="https://discord.com/invite/7NXusRtqYU" target="_blank">
<img src="https://img.shields.io/discord/1111061815649124414?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://twitter.com/intent/follow?screen_name=langfuse" target="_blank">
<img src="https://img.shields.io/twitter/follow/langfuse?logo=X&color=%20%23f5f5f5"
alt="follow on X(Twitter)"></a>
<a href="https://www.linkedin.com/company/langfuse/" target="_blank">
<img src="https://custom-icon-badges.demolab.com/badge/LinkedIn-0A66C2?logo=linkedin-white&logoColor=fff"
alt="follow on LinkedIn"></a>
<a href="https://github.com/langfuse/langfuse/graphs/commit-activity" target="_blank">
<img alt="Commits last month" src="https://img.shields.io/github/commit-activity/m/langfuse/langfuse?labelColor=%20%2332b583&color=%20%2312b76a"></a>
<a href="https://github.com/langfuse/langfuse/" target="_blank">
<img alt="Issues closed" src="https://img.shields.io/github/issues-search?query=repo%3Alangfuse%2Flangfuse%20is%3Aclosed&label=issues%20closed&labelColor=%20%237d89b0&color=%20%235d6b98"></a>
<a href="https://github.com/langfuse/langfuse/discussions/" target="_blank">
<img alt="Discussion posts" src="https://img.shields.io/github/discussions/langfuse/langfuse?labelColor=%20%239b8afb&color=%20%237a5af8"></a>
</p>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-d9d9d9"></a>
<a href="./README.cn.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
<a href="./README.ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-d9d9d9"></a>
<a href="./README.kr.md"><img alt="README in Korean" src="https://img.shields.io/badge/한국어-d9d9d9"></a>
</p>
Langfuse is an **open source LLM engineering** platform. It helps teams collaboratively
**develop, monitor, evaluate,** and **debug** AI applications. Langfuse can be **self-hosted in minutes** and is **battle-tested**.
[![Langfuse Overview Video](https://github.com/user-attachments/assets/3926b288-ff61-4b95-8aa1-45d041c70866)](https://langfuse.com/watch-demo)
### Develop
## ✨ Core Features
- **LLM Observability:** Instrument your app and start ingesting traces to Langfuse ([Quickstart](https://langfuse.com/docs/get-started), [Integrations](https://langfuse.com/docs/integrations) [Tracing](https://langfuse.com/docs/tracing))
- **Langfuse UI:** Inspect and debug complex logs ([Demo](https://langfuse.com/docs/demo), [Tracing](https://langfuse.com/docs/tracing))
- **Prompt Management:** Manage, version and deploy prompts from within Langfuse ([Prompt Management](https://langfuse.com/docs/prompts/get-started))
- **Prompt Engineering:** Test and iterate on your prompts with the [LLM Playground](https://langfuse.com/docs/playground)
![Langfuse Overview](https://langfuse.com/images/docs/github-readme/github-feature-overview.png)
### Monitor
- [LLM Application Observability](https://langfuse.com/docs/tracing): Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions. Try the interactive [demo](https://langfuse.com/docs/demo) to see this in action.
- **LLM Analytics:** Track metrics (cost, latency, quality) and gain insights from dashboards & data exports ([Analytics](https://langfuse.com/docs/analytics))
- **LLM Evaluations:** Collect and calculate scores for your LLM completions ([Scores & Evaluations](https://langfuse.com/docs/scores))
- Run ([Model-based evaluations](https://langfuse.com/docs/scores/model-based-evals)) and LLM-as-a-Judge within Langfuse
- Collect user feedback ([User Feedback](https://langfuse.com/docs/scores/user-feedback))
- Manually score LLM outputs in Langfuse ([Manual Scores](https://langfuse.com/docs/scores/manually))
- [Prompt Management](https://langfuse.com/docs/prompts/get-started) helps you centrally manage, version control, and collaboratively iterate on your prompts. Thanks to strong caching on server and client side, you can iterate on prompts without adding latency to your application.
### Test
- [Evaluations](https://langfuse.com/docs/scores/overview) are key to the LLM application development workflow, and Langfuse adapts to your needs. It supports LLM-as-a-judge, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs.
- **Experiments:** Track and test app behaviour before deploying a new version
- Datasets let you test expected in and output pairs and benchmark performance before deploying ([Datasets](https://langfuse.com/docs/datasets))
- Track versions and releases in your application ([Experimentation](https://langfuse.com/docs/experimentation), [Prompt Management](https://langfuse.com/docs/prompts))
- [Datasets](https://langfuse.com/docs/datasets/overview) enable test sets and benchmarks for evaluating your LLM application. They support continuous improvement, pre-deployment testing, structured experiments, flexible evaluation, and seamless integration with frameworks like LangChain and LlamaIndex.
## Get started
- [LLM Playground](https://langfuse.com/docs/playground) is a tool for testing and iterating on your prompts and model configurations, shortening the feedback loop and accelerating development. When you see a bad result in tracing, you can directly jump to the playground to iterate on it.
- [Comprehensive API](https://langfuse.com/docs/api): Langfuse is frequently used to power bespoke LLMOps workflows while using the building blocks provided by Langfuse via the API. OpenAPI spec, Postman collection, and typed SDKs for Python, JS/TS are available.
## 📦 Deploy Langfuse
![Langfuse Deployment Options](https://langfuse.com/images/docs/github-readme/github-deployment-options.png)
### Langfuse Cloud
Managed deployment by the Langfuse team, generous free-tier (hobby plan), no credit card required.
Managed deployment by the Langfuse team, generous free-tier, no credit card required.
**[» Langfuse Cloud](https://cloud.langfuse.com)**
<div align="center">
<a href="https://cloud.langfuse.com" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/»%20Sign%20up%20for%20Langfuse%20Cloud-8A2BE2?&color=orange">
</a>
</div>
## Self-Hosting Open Source LLM Observability with Langfuse
### Self-Host Langfuse
### Localhost (docker)
Run Langfuse on your own infrastructure:
```bash
# Clone repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse
- [Local (docker compose)](https://langfuse.com/self-hosting/local): Run Langfuse on your own machine in 5 minutes using Docker Compose.
# Run server and database
docker compose up -d
```
```bash
# Get a copy of the latest Langfuse repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse
[→ Learn more about deploying locally](https://langfuse.com/docs/deployment/local)
# Run the langfuse docker compose
docker compose up
```
- [VM](https://langfuse.com/self-hosting/docker-compose): Run Langfuse on a single Virtual Machine using Docker Compose.
- [Kubernetes (Helm)](https://langfuse.com/self-hosting/kubernetes-helm): Run Langfuse on a Kubernetes cluster using Helm. This is the preferred production deployment.
- Terraform Templates: [AWS](https://langfuse.com/self-hosting/aws), [Azure](https://langfuse.com/self-hosting/azure), [GCP](https://langfuse.com/self-hosting/gcp)
### Self-host (docker)
See [self-hosting documentation](https://langfuse.com/self-hosting) to learn more about architecture and configuration options.
Langfuse is simple to self-host and keep updated. It currently requires only a single docker container and a postgres database.
[→ Self Hosting Instructions](https://langfuse.com/docs/deployment/self-host)
## 🔌 Integrations
Templated deployments: [Railway, GCP, AWS, Azure, Kubernetes and others](https://langfuse.com/docs/deployment/self-host#platform-specific-information)
![Langfuse Integrations](https://langfuse.com/images/docs/github-readme/github-integrations.png)
## Get Started
### API Keys
You need a Langfuse public and secret key to get started. Sign up [here](https://cloud.langfuse.com) and find them in your project settings.
### Ingesting Data · Instrumenting Your Application · LLM Observability with Langfuse
Note: We recommend using our fully async, typed [SDKs](https://langfuse.com/docs/sdk) that allow you to instrument any LLM application with any underlying model. They are available in [Python (Decorators)](https://langfuse.com/docs/sdk/python) & [JS/TS](https://langfuse.com/docs/sdk/typescript). The SDKs will always be the most fully featured and stable way to ingest data into Langfuse.
See the [→ Quickstart](https://langfuse.com/docs/get-started) to integrate Langfuse.
### LLM Observability Integrations
### Main Integrations:
| Integration | Supports | Description |
| ---------------------------------------------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
@@ -120,77 +141,129 @@ See the [→ Quickstart](https://langfuse.com/docs/get-started) to integrate Lan
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js. |
| [API](https://langfuse.com/docs/api) | | Directly call the public API. OpenAPI spec available. |
Packages integrated with Langfuse:
### Packages integrated with Langfuse:
| Name | Description |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| [Instructor](https://langfuse.com/docs/integrations/instructor) | Library to get structured LLM outputs (JSON, Pydantic) |
| [Dify](https://langfuse.com/docs/integrations/dify) | Open source LLM app development platform with no-code builder. |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | Easily run open source LLMs on your own machine. |
| [Mirascope](https://langfuse.com/docs/integrations/mirascope) | Python toolkit for building LLM applications. |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | JS/TS no-code builder for customized LLM flows. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. |
| Name | Type | Description |
| ----------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------- |
| [Instructor](https://langfuse.com/docs/integrations/instructor) | Library | Library to get structured LLM outputs (JSON, Pydantic) |
| [DSPy](https://langfuse.com/docs/integrations/dspy) | Library | Framework that systematically optimizes language model prompts and weights |
| [Mirascope](https://langfuse.com/docs/integrations/mirascope) | Library | Python toolkit for building LLM applications. |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | Model (local) | Easily run open source LLMs on your own machine. |
| [Amazon Bedrock](https://langfuse.com/docs/integrations/amazon-bedrock) | Model | Run foundation and fine-tuned models on AWS. |
| [AutoGen](https://langfuse.com/docs/integrations/autogen) | Agent Framework | Open source LLM platform for building distributed agents. |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | Chat/Agent&nbsp;UI | JS/TS no-code builder for customized LLM flows. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | Chat/Agent&nbsp;UI | Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. |
| [Dify](https://langfuse.com/docs/integrations/dify) | Chat/Agent&nbsp;UI | Open source LLM app development platform with no-code builder. |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | Chat/Agent&nbsp;UI | Self-hosted LLM Chat web ui supporting various LLM runners including self-hosted and local models. |
| [Promptfoo](https://langfuse.com/docs/integrations/promptfoo) | Tool | Open source LLM testing platform. |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | Chat/Agent&nbsp;UI | Open source chatbot platform. |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | Platform | Open source voice AI platform. |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | Agents | Open source LLM platform for building distributed agents. |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | Chat/Agent&nbsp;UI | Open source Python library to build web interfaces like Chat UI. |
| [Goose](https://langfuse.com/docs/integrations/goose) | Agents | Open source LLM platform for building distributed agents. |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | Agents | Open source AI agents framework. |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | Agents | Multi agent framework for agent collaboration and tool use. |
## Questions and feedback
## 🚀 Quickstart
### Ideas and roadmap
Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions.
- [Roadmap](https://langfuse.com/roadmap)
- [GitHub Discussions](https://github.com/orgs/langfuse/discussions)
- [Feature Requests](https://langfuse.com/ideas)
### 1️⃣ Create new project
### Support and feedback
1. [Create Langfuse account](https://cloud.langfuse.com/auth/sign-up) or [self-host](https://langfuse.com/self-hosting)
2. Create a new project
3. Create new API credentials in the project settings
In order of preference the best way to communicate with us:
### 2️⃣ Log your first LLM call
- [GitHub Discussions](https://github.com/orgs/langfuse/discussions) (preferred): Contribute [ideas](https://langfuse.com/ideas), [support requests](https://langfuse.com/gh-support) and [report bugs](https://langfuse.com/issues)
- [Discord](https://langfuse.com/discord): community support
- Privately: contact at langfuse dot com
The [`@observe()` decorator](https://langfuse.com/docs/sdk/python/decorators) makes it easy to trace any Python LLM application. In this quickstart we also use the Langfuse [OpenAI integration](https://langfuse.com/docs/integrations/openai) to automatically capture all model parameters.
## Contributing to Langfuse
> [!TIP]
> Not using OpenAI? Visit [our documentation](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse) to learn how to log other models and frameworks.
- Vote on [Ideas](https://github.com/orgs/langfuse/discussions/categories/ideas)
- Raise and comment on [Issues](https://github.com/langfuse/langfuse/issues)
```bash
pip install langfuse openai
```
```bash filename=".env"
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EU region
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 US region
```
```python /@observe()/ /from langfuse.openai import openai/ filename="main.py"
from langfuse import observe
from langfuse.openai import openai # OpenAI integration
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
```
### 3️⃣ See traces in Langfuse
See your language model calls and other application logic in Langfuse.
![Example trace in Langfuse](https://langfuse.com/images/docs/github-readme/github-example-trace.png)
_[Public example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!TIP]
>
> [Learn more](https://langfuse.com/docs/tracing) about tracing in Langfuse or play with the [interactive demo](https://langfuse.com/docs/demo).
## ⭐️ Star Us
![star-langfuse-on-github](https://github.com/user-attachments/assets/79a1d816-d229-4526-aecc-097d4a19f1ad)
## 💭 Support
Finding an answer to your question:
- Our [documentation](https://langfuse.com/docs) is the best place to start looking for answers. It is comprehensive, and we invest significant time into maintaining it. You can also suggest edits to the docs via GitHub.
- [Langfuse FAQs](https://langfuse.com/faq) where the most common questions are answered.
- Use "[Ask AI](https://langfuse.com/docs/ask-ai)" to get instant answers to your questions.
Support Channels:
- **Ask any question in our [public Q&A](https://github.com/orgs/langfuse/discussions/categories/support) on GitHub Discussions.** Please include as much detail as possible (e.g. code snippets, screenshots, background information) to help us understand your question.
- [Request a feature](https://github.com/orgs/langfuse/discussions/categories/ideas) on GitHub Discussions.
- [Report a Bug](https://github.com/langfuse/langfuse/issues) on GitHub Issues.
- For time-sensitive queries, ping us via the in-app chat widget.
## 🤝 Contributing
Your contributions are welcome!
- Vote on [Ideas](https://github.com/orgs/langfuse/discussions/categories/ideas) in GitHub Discussions.
- Raise and comment on [Issues](https://github.com/langfuse/langfuse/issues).
- Open a PR - see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to setup a development environment.
## License
## 🥇 License
This repository is MIT licensed, except for the `ee` folders. See [LICENSE](LICENSE) and [docs](https://langfuse.com/docs/open-source) for more details.
## Misc
### GET API to export your data
[**GET routes**](https://langfuse.com/docs/integrations/api) to use data in downstream applications (e.g. embedded analytics). You can also access them conveniently via the SDKs ([docs](https://langfuse.com/docs/query-traces)).
### Security & Privacy
We take data security and privacy seriously. Please refer to our [Security and Privacy](https://langfuse.com/security) page for more information.
### Telemetry
By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).
This helps us to:
1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.
None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect [here](/web/src/features/telemetry/index.ts).
You can opt-out by setting `TELEMETRY_ENABLED=false`.
### Star History
## ⭐️ Star History
<a href="https://star-history.com/#langfuse/langfuse&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" style="border-radius: 15px;" />
</picture>
</a>
### Open Source Projects Using Langfuse
## ❤️ Open Source Projects Using Langfuse
Top open-source Python projects that use Langfuse, ranked by stars ([Source](https://github.com/langfuse/langfuse-docs/blob/main/components-mdx/dependents)):
@@ -240,3 +313,20 @@ Top open-source Python projects that use Langfuse, ranked by stars ([Source](htt
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/15125613?s=40&v=4" width="20" height="20" alt=""> &nbsp; [souzatharsis](https://github.com/souzatharsis) / [tamingLLMs](https://github.com/souzatharsis/tamingLLMs) | 129 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/169401942?s=40&v=4" width="20" height="20" alt=""> &nbsp; [LibreChat-AI](https://github.com/LibreChat-AI) / [librechat.ai](https://github.com/LibreChat-AI/librechat.ai) | 128 |
| <img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/51827949?s=40&v=4" width="20" height="20" alt=""> &nbsp; [deepset-ai](https://github.com/deepset-ai) / [haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations) | 126 |
## 🔒 Security & Privacy
We take data security and privacy seriously. Please refer to our [Security and Privacy](https://langfuse.com/security) page for more information.
### Telemetry
By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).
This helps us to:
1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.
None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect [here](/web/src/features/telemetry/index.ts).
You can opt-out by setting `TELEMETRY_ENABLED=false`.
-13
View File
@@ -1,13 +0,0 @@
_We are Hiring_
Join us in scaling Langfuse in Berlin, Germany. We are an open source company, we hire in person, we are only hiring technical talent.
_Open Roles_
- Product Engineer, 70-130k EUR, 0.25-0.75% Equity, https://www.ycombinator.com/companies/langfuse/jobs/aAvmoFB-product-engineer
- Developer Advocate, 60-110k EUR, 0.25-0.5% Equity, https://www.ycombinator.com/companies/langfuse/jobs/uHysbKH-developer-advocate-devrel
_More Info_
- https://langfuse.com/careers
- https://langfuse.com/docs
- https://langfuse.com/changelog
Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

+8 -7
View File
@@ -48,6 +48,10 @@ services:
REDIS_HOST: ${REDIS_HOST:-redis}
REDIS_PORT: ${REDIS_PORT:-6379}
REDIS_AUTH: ${REDIS_AUTH:-myredissecret}
REDIS_TLS_ENABLED: ${REDIS_TLS_ENABLED:-false}
REDIS_TLS_CA: ${REDIS_TLS_CA:-/certs/ca.crt}
REDIS_TLS_CERT: ${REDIS_TLS_CERT:-/certs/redis.crt}
REDIS_TLS_KEY: ${REDIS_TLS_KEY:-/certs/redis.key}
restart: always
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/api/public/health"]
@@ -74,10 +78,8 @@ services:
retries: 3
clickhouse:
image: clickhouse/clickhouse-server
image: docker.io/clickhouse/clickhouse-server
user: "101:101"
container_name: clickhouse
hostname: clickhouse
environment:
CLICKHOUSE_DB: default
CLICKHOUSE_USER: clickhouse
@@ -96,8 +98,7 @@ services:
start_period: 1s
minio:
image: minio/minio
container_name: minio
image: docker.io/minio/minio
entrypoint: sh
# create the 'langfuse' bucket before starting the service
command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
@@ -117,7 +118,7 @@ services:
start_period: 1s
redis:
image: redis:7
image: docker.io/redis:7
restart: always
command: >
--requirepass ${REDIS_AUTH:-myredissecret}
@@ -130,7 +131,7 @@ services:
retries: 10
postgres:
image: postgres:${POSTGRES_VERSION:-latest}
image: docker.io/postgres:${POSTGRES_VERSION:-latest}
restart: always
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
+3 -6
View File
@@ -1,9 +1,7 @@
services:
clickhouse:
image: clickhouse/clickhouse-server
image: docker.io/clickhouse/clickhouse-server:24.3
user: "101:101"
container_name: clickhouse
hostname: clickhouse
environment:
CLICKHOUSE_DB: default
CLICKHOUSE_USER: clickhouse
@@ -19,7 +17,6 @@ services:
azurite:
image: mcr.microsoft.com/azure-storage/azurite
container_name: azurite
command: azurite-blob --blobHost 0.0.0.0
ports:
- "10000:10000"
@@ -27,7 +24,7 @@ services:
- langfuse_azurite_data:/data
redis:
image: redis:7.2.4
image: docker.io/redis:7.2.4
restart: always
command: >
--requirepass ${REDIS_AUTH:-myredissecret}
@@ -35,7 +32,7 @@ services:
- 6379:6379
postgres:
image: postgres:${POSTGRES_VERSION:-latest}
image: docker.io/postgres:${POSTGRES_VERSION:-latest}
restart: always
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
+146
View File
@@ -0,0 +1,146 @@
services:
clickhouse:
image: docker.io/clickhouse/clickhouse-server:24.3
user: "101:101"
environment:
CLICKHOUSE_DB: default
CLICKHOUSE_USER: clickhouse
CLICKHOUSE_PASSWORD: clickhouse
volumes:
- langfuse_clickhouse_data:/var/lib/clickhouse
- langfuse_clickhouse_logs:/var/log/clickhouse-server
ports:
- 127.0.0.1:8123:8123
- 127.0.0.1:9000:9000
depends_on:
- postgres
minio:
image: docker.io/minio/minio
entrypoint: sh
# create the 'langfuse' bucket before starting the service
command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
environment:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: miniosecret
ports:
- 127.0.0.1:9090:9000
- 127.0.0.1:9091:9001
volumes:
- langfuse_minio_data:/data
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 1s
timeout: 5s
retries: 5
start_period: 1s
postgres:
image: docker.io/postgres:${POSTGRES_VERSION:-latest}
restart: always
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 3s
timeout: 3s
retries: 10
command: ["postgres", "-c", "log_statement=all"]
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
ports:
- 127.0.0.1:5432:5432
volumes:
- langfuse_postgres_data:/var/lib/postgresql/data
# Redis Cluster. Requires a Unix host to work for network_mode: host. I.e. tests will fail on windows and mac with this setup.
redis-node-0:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-0:/bitnami/redis/data
environment:
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6370
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
redis-node-1:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-1:/bitnami/redis/data
environment:
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6371
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
redis-node-2:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-2:/bitnami/redis/data
environment:
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6372
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
redis-node-3:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-3:/bitnami/redis/data
environment:
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6373
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
redis-node-4:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-4:/bitnami/redis/data
environment:
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6374
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
redis-node-5:
image: docker.io/bitnami/redis-cluster:8.0
network_mode: host
volumes:
- redis-cluster_data-5:/bitnami/redis/data
depends_on:
- redis-node-0
- redis-node-1
- redis-node-2
- redis-node-3
- redis-node-4
environment:
REDISCLI_AUTH: bitnami
REDIS_CLUSTER_REPLICAS: 1
REDIS_PASSWORD: bitnami
REDIS_PORT_NUMBER: 6375
REDIS_NODES: 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375
REDIS_CLUSTER_CREATOR: yes
volumes:
langfuse_postgres_data:
driver: local
langfuse_clickhouse_data:
driver: local
langfuse_clickhouse_logs:
driver: local
langfuse_minio_data:
driver: local
redis-cluster_data-0:
driver: local
redis-cluster_data-1:
driver: local
redis-cluster_data-2:
driver: local
redis-cluster_data-3:
driver: local
redis-cluster_data-4:
driver: local
redis-cluster_data-5:
driver: local
+10 -13
View File
@@ -1,9 +1,7 @@
services:
clickhouse:
image: clickhouse/clickhouse-server
image: docker.io/clickhouse/clickhouse-server:24.3
user: "101:101"
container_name: clickhouse
hostname: clickhouse
environment:
CLICKHOUSE_DB: default
CLICKHOUSE_USER: clickhouse
@@ -12,14 +10,13 @@ services:
- langfuse_clickhouse_data:/var/lib/clickhouse
- langfuse_clickhouse_logs:/var/log/clickhouse-server
ports:
- "8123:8123"
- "9000:9000"
- 127.0.0.1:8123:8123
- 127.0.0.1:9000:9000
depends_on:
- postgres
minio:
image: minio/minio
container_name: minio
image: docker.io/minio/minio
entrypoint: sh
# create the 'langfuse' bucket before starting the service
command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
@@ -27,8 +24,8 @@ services:
MINIO_ACCESS_KEY: minio
MINIO_SECRET_KEY: miniosecret
ports:
- "9090:9000"
- "9091:9001"
- 127.0.0.1:9090:9000
- 127.0.0.1:9091:9001
volumes:
- langfuse_minio_data:/data
healthcheck:
@@ -39,15 +36,15 @@ services:
start_period: 1s
redis:
image: redis:7.2.4
image: docker.io/redis:7.2.4
restart: always
command: >
--requirepass ${REDIS_AUTH:-myredissecret}
ports:
- 6379:6379
- 127.0.0.1:6379:6379
postgres:
image: postgres:${POSTGRES_VERSION:-latest}
image: docker.io/postgres:${POSTGRES_VERSION:-latest}
restart: always
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
@@ -60,7 +57,7 @@ services:
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
ports:
- 5432:5432
- 127.0.0.1:5432:5432
volumes:
- langfuse_postgres_data:/var/lib/postgresql/data
+49 -34
View File
@@ -1,6 +1,11 @@
# Make sure to update the credential placeholders with your own secrets.
# We mark them with # CHANGEME in the file below.
# In addition, we recommend to restrict inbound traffic on the host to langfuse-web (port 3000) and minio (port 9090) only.
# All other components are bound to localhost (127.0.0.1) to only accept connections from the local machine.
# External connections from other machines will not be able to reach these services directly.
services:
langfuse-worker:
image: langfuse/langfuse-worker:3
image: docker.io/langfuse/langfuse-worker:3
restart: always
depends_on: &langfuse-depends-on
postgres:
@@ -12,48 +17,64 @@ services:
clickhouse:
condition: service_healthy
ports:
- "3030:3030"
- 127.0.0.1:3030:3030
environment: &langfuse-worker-env
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres
SALT: "mysalt"
ENCRYPTION_KEY: "0000000000000000000000000000000000000000000000000000000000000000" # generate via `openssl rand -hex 32`
NEXTAUTH_URL: http://localhost:3000
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres # CHANGEME
SALT: "mysalt" # CHANGEME
ENCRYPTION_KEY: "0000000000000000000000000000000000000000000000000000000000000000" # CHANGEME: generate via `openssl rand -hex 32`
TELEMETRY_ENABLED: ${TELEMETRY_ENABLED:-true}
LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES: ${LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES:-true}
CLICKHOUSE_MIGRATION_URL: ${CLICKHOUSE_MIGRATION_URL:-clickhouse://clickhouse:9000}
CLICKHOUSE_URL: ${CLICKHOUSE_URL:-http://clickhouse:8123}
CLICKHOUSE_USER: ${CLICKHOUSE_USER:-clickhouse}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-clickhouse}
CLICKHOUSE_PASSWORD: ${CLICKHOUSE_PASSWORD:-clickhouse} # CHANGEME
CLICKHOUSE_CLUSTER_ENABLED: ${CLICKHOUSE_CLUSTER_ENABLED:-false}
LANGFUSE_USE_AZURE_BLOB: ${LANGFUSE_USE_AZURE_BLOB:-false}
LANGFUSE_S3_EVENT_UPLOAD_BUCKET: ${LANGFUSE_S3_EVENT_UPLOAD_BUCKET:-langfuse}
LANGFUSE_S3_EVENT_UPLOAD_REGION: ${LANGFUSE_S3_EVENT_UPLOAD_REGION:-auto}
LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID: ${LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID:-minio}
LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY:-miniosecret}
LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY:-miniosecret} # CHANGEME
LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT: ${LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT:-http://minio:9000}
LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE: ${LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE:-true}
LANGFUSE_S3_EVENT_UPLOAD_PREFIX: ${LANGFUSE_S3_EVENT_UPLOAD_PREFIX:-events/}
LANGFUSE_S3_MEDIA_UPLOAD_BUCKET: ${LANGFUSE_S3_MEDIA_UPLOAD_BUCKET:-langfuse}
LANGFUSE_S3_MEDIA_UPLOAD_REGION: ${LANGFUSE_S3_MEDIA_UPLOAD_REGION:-auto}
LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID: ${LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID:-minio}
LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY:-miniosecret}
LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: ${LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT:-http://minio:9000}
LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY: ${LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY:-miniosecret} # CHANGEME
LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT: ${LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT:-http://localhost:9090}
LANGFUSE_S3_MEDIA_UPLOAD_FORCE_PATH_STYLE: ${LANGFUSE_S3_MEDIA_UPLOAD_FORCE_PATH_STYLE:-true}
LANGFUSE_S3_MEDIA_UPLOAD_PREFIX: ${LANGFUSE_S3_MEDIA_UPLOAD_PREFIX:-media/}
LANGFUSE_S3_BATCH_EXPORT_ENABLED: ${LANGFUSE_S3_BATCH_EXPORT_ENABLED:-false}
LANGFUSE_S3_BATCH_EXPORT_BUCKET: ${LANGFUSE_S3_BATCH_EXPORT_BUCKET:-langfuse}
LANGFUSE_S3_BATCH_EXPORT_PREFIX: ${LANGFUSE_S3_BATCH_EXPORT_PREFIX:-exports/}
LANGFUSE_S3_BATCH_EXPORT_REGION: ${LANGFUSE_S3_BATCH_EXPORT_REGION:-auto}
LANGFUSE_S3_BATCH_EXPORT_ENDPOINT: ${LANGFUSE_S3_BATCH_EXPORT_ENDPOINT:-http://minio:9000}
LANGFUSE_S3_BATCH_EXPORT_EXTERNAL_ENDPOINT: ${LANGFUSE_S3_BATCH_EXPORT_EXTERNAL_ENDPOINT:-http://localhost:9090}
LANGFUSE_S3_BATCH_EXPORT_ACCESS_KEY_ID: ${LANGFUSE_S3_BATCH_EXPORT_ACCESS_KEY_ID:-minio}
LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY: ${LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY:-miniosecret} # CHANGEME
LANGFUSE_S3_BATCH_EXPORT_FORCE_PATH_STYLE: ${LANGFUSE_S3_BATCH_EXPORT_FORCE_PATH_STYLE:-true}
LANGFUSE_INGESTION_QUEUE_DELAY_MS: ${LANGFUSE_INGESTION_QUEUE_DELAY_MS:-}
LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS: ${LANGFUSE_INGESTION_CLICKHOUSE_WRITE_INTERVAL_MS:-}
REDIS_HOST: ${REDIS_HOST:-redis}
REDIS_PORT: ${REDIS_PORT:-6379}
REDIS_AUTH: ${REDIS_AUTH:-myredissecret}
REDIS_AUTH: ${REDIS_AUTH:-myredissecret} # CHANGEME
REDIS_TLS_ENABLED: ${REDIS_TLS_ENABLED:-false}
REDIS_TLS_CA: ${REDIS_TLS_CA:-/certs/ca.crt}
REDIS_TLS_CERT: ${REDIS_TLS_CERT:-/certs/redis.crt}
REDIS_TLS_KEY: ${REDIS_TLS_KEY:-/certs/redis.key}
EMAIL_FROM_ADDRESS: ${EMAIL_FROM_ADDRESS:-}
SMTP_CONNECTION_URL: ${SMTP_CONNECTION_URL:-}
langfuse-web:
image: langfuse/langfuse:3
image: docker.io/langfuse/langfuse:3
restart: always
depends_on: *langfuse-depends-on
ports:
- "3000:3000"
- 3000:3000
environment:
<<: *langfuse-worker-env
NEXTAUTH_URL: http://localhost:3000
NEXTAUTH_SECRET: mysecret
NEXTAUTH_SECRET: mysecret # CHANGEME
LANGFUSE_INIT_ORG_ID: ${LANGFUSE_INIT_ORG_ID:-}
LANGFUSE_INIT_ORG_NAME: ${LANGFUSE_INIT_ORG_NAME:-}
LANGFUSE_INIT_PROJECT_ID: ${LANGFUSE_INIT_PROJECT_ID:-}
@@ -63,27 +84,21 @@ services:
LANGFUSE_INIT_USER_EMAIL: ${LANGFUSE_INIT_USER_EMAIL:-}
LANGFUSE_INIT_USER_NAME: ${LANGFUSE_INIT_USER_NAME:-}
LANGFUSE_INIT_USER_PASSWORD: ${LANGFUSE_INIT_USER_PASSWORD:-}
LANGFUSE_SDK_CI_SYNC_PROCESSING_ENABLED: ${LANGFUSE_SDK_CI_SYNC_PROCESSING_ENABLED:-false}
LANGFUSE_READ_FROM_POSTGRES_ONLY: ${LANGFUSE_READ_FROM_POSTGRES_ONLY:-false}
LANGFUSE_READ_FROM_CLICKHOUSE_ONLY: ${LANGFUSE_READ_FROM_CLICKHOUSE_ONLY:-true}
LANGFUSE_RETURN_FROM_CLICKHOUSE: ${LANGFUSE_RETURN_FROM_CLICKHOUSE:-true}
clickhouse:
image: clickhouse/clickhouse-server
image: docker.io/clickhouse/clickhouse-server
restart: always
user: "101:101"
container_name: clickhouse
hostname: clickhouse
environment:
CLICKHOUSE_DB: default
CLICKHOUSE_USER: clickhouse
CLICKHOUSE_PASSWORD: clickhouse
CLICKHOUSE_PASSWORD: clickhouse # CHANGEME
volumes:
- langfuse_clickhouse_data:/var/lib/clickhouse
- langfuse_clickhouse_logs:/var/log/clickhouse-server
ports:
- "8123:8123"
- "9000:9000"
- 127.0.0.1:8123:8123
- 127.0.0.1:9000:9000
healthcheck:
test: wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1
interval: 5s
@@ -92,18 +107,17 @@ services:
start_period: 1s
minio:
image: minio/minio
image: docker.io/minio/minio
restart: always
container_name: minio
entrypoint: sh
# create the 'langfuse' bucket before starting the service
command: -c 'mkdir -p /data/langfuse && minio server --address ":9000" --console-address ":9001" /data'
environment:
MINIO_ROOT_USER: minio
MINIO_ROOT_PASSWORD: miniosecret
MINIO_ROOT_PASSWORD: miniosecret # CHANGEME
ports:
- "9090:9000"
- "9091:9001"
- 9090:9000
- 127.0.0.1:9091:9001
volumes:
- langfuse_minio_data:/data
healthcheck:
@@ -114,12 +128,13 @@ services:
start_period: 1s
redis:
image: redis:7
image: docker.io/redis:7
restart: always
# CHANGEME: row below to secure redis password
command: >
--requirepass ${REDIS_AUTH:-myredissecret}
ports:
- 6379:6379
- 127.0.0.1:6379:6379
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 3s
@@ -127,7 +142,7 @@ services:
retries: 10
postgres:
image: postgres:${POSTGRES_VERSION:-latest}
image: docker.io/postgres:${POSTGRES_VERSION:-latest}
restart: always
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
@@ -136,10 +151,10 @@ services:
retries: 10
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PASSWORD: postgres # CHANGEME
POSTGRES_DB: postgres
ports:
- 5432:5432
- 127.0.0.1:5432:5432
volumes:
- langfuse_postgres_data:/var/lib/postgresql/data
+25 -11
View File
@@ -1,24 +1,36 @@
Langfuse Enterprise License (the “Enterprise License”)
Copyright (c) 2024 Langfuse GmbH ('Langfuse')
START NOTE
Langfuse is an open core project. Langfuse's core
is permissively licensed (MIT license).
Certain parts of the periphery of Langfuse are commercially
licensed and governed by this Enterprise License.
For the avoidance of doubt, this license does not apply
to the core of Langfuse as it is defined in its license in
the directory "/LICENSE" (as opposed to this file "ee/LICENSE")
which can be used and run without infringing the below license
and its licensed materials.
END NOTE
With regard to the Langfuse Software:
START OF LICENSE
Langfuse Enterprise license (the “Enterprise License” or "EE license")
Copyright (c) 2023-2025 Langfuse GmbH
With regard to the Langfuse Enterprise Software:
This software and associated documentation files (the "Software") may only be
used in production, if you (and any entity that you represent) have agreed to,
and are in compliance with, the Langfuse Terms of Service, available
used, if you (and any entity that you represent) have agreed to,
and are in compliance with, the applicable Langfuse Terms of Service, available
at https://langfuse.com/terms (the “Enterprise Terms”), or other
agreement governing the use of the Software, as agreed by you and Langfuse,
and otherwise have a valid Langfuse Enterprise license.
and otherwise have a valid Langfuse Enterprise License.
Subject to the foregoing sentence, you are free to
modify this Software and publish patches to the Software. You agree that Langfuse
and/or its licensors (as applicable) retain all right, title and interest in and
to all such modifications and/or patches, and all such modifications and/or
patches may only be used, copied, modified, displayed, distributed, or otherwise
exploited with a valid Langfuse Enterprise license.
Notwithstanding the foregoing, you may copy and modify
exploited with a valid Langfuse Enterprise License. Notwithstanding the foregoing, you may copy and modify
the Software for development and testing purposes, without requiring a
subscription. You agree that Langfuse and/or its licensors (as applicable) retain
subscription. You agree that Langfuse GmbH and/or its licensors (as applicable) retain
all right, title and interest in and to all such modifications. You are not
granted any other rights beyond what is expressly stated herein. Subject to the
foregoing, it is forbidden to copy, merge, publish, distribute, sublicense,
@@ -33,5 +45,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
For all third party components incorporated into the Langfuse Software, those
components are licensed under the original license provided by the owner of the
components are licensed under the original license provided by the owner of the
applicable component.
END OF LICENSE
View File
+4 -10
View File
@@ -26,10 +26,10 @@
"dependencies": {
"@langfuse/shared": "workspace:*",
"@opentelemetry/api": ">=1.0.0 <1.10.0",
"axios": "^1.7.7",
"next": "^14.2.21",
"https-proxy-agent": "^7.0.6",
"next": "^14.2.30",
"next-auth": "^4.24.11",
"zod": "^3.23.8"
"zod": "^3.25.62"
},
"devDependencies": {
"@repo/eslint-config": "workspace:*",
@@ -40,15 +40,9 @@
"eslint-config-prettier": "^9.1.0",
"eslint-config-standard": "^17.1.0",
"eslint-plugin-prettier": "^5.1.3",
"prettier": "^3.3.3",
"prettier": "^3.6.2",
"ts-node": "^10.9.2",
"tsc-watch": "^6.2.0",
"typescript": "^5.4.5"
},
"pnpm": {
"overrides": {
"jsonpath-plus": "10.0.7",
"nanoid": "^3.3.8"
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
import { z } from "zod";
import { z } from "zod/v4";
import { removeEmptyEnvVariables } from "@langfuse/shared";
const EnvSchema = z.object({
+14 -58
View File
@@ -14,13 +14,16 @@ types:
CreateScoreRequest:
properties:
id: optional<string>
traceId: string
traceId: optional<string>
sessionId: optional<string>
observationId: optional<string>
datasetRunId: optional<string>
name: string
value:
type: CreateScoreValue
docs: The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId: optional<string>
comment: optional<string>
metadata: optional<unknown>
dataType:
type: optional<ScoreDataType>
docs: When set, must match the score value's type. If not set, will be inferred from the score value or config
@@ -64,62 +67,15 @@ types:
dataType: "BOOLEAN"
configId: "1234-5678-90ab-cdef"
traceId: "cdef-1234-5678-90ab"
BaseScore:
properties:
id: string
traceId: string
name: string
source: ScoreSource
observationId: optional<string>
timestamp: datetime
createdAt: datetime
updatedAt: datetime
authorUserId: optional<string>
comment: optional<string>
configId:
type: optional<string>
docs: Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
NumericScore:
extends: BaseScore
properties:
value:
type: double
docs: The numeric value of the score
BooleanScore:
extends: BaseScore
properties:
value:
type: double
docs: The numeric value of the score. Equals 1 for "True" and 0 for "False"
stringValue:
type: string
docs: The string representation of the score value. Is inferred from the numeric value and equals "True" or "False"
CategoricalScore:
extends: BaseScore
properties:
value:
type: optional<double>
docs: Only defined if a config is linked. Represents the numeric category mapping of the stringValue
stringValue:
type: string
docs: The string representation of the score value. If no config is linked, can be any string. Otherwise, must map to a config category
Score:
discriminant: "dataType"
union:
NUMERIC:
type: NumericScore
docs: "Score with NUMERIC data type"
CATEGORICAL:
type: CategoricalScore
docs: "Score with CATEGORICAL data type"
BOOLEAN:
type: BooleanScore
docs: "Score with BOOLEAN data type"
ScoreSource:
enum:
- ANNOTATION
- API
- EVAL
- value:
name: "contextrelevant"
value: "not relevant"
sessionId: "abyt-1234-5678-80ab"
- value:
name: "hallucination"
value: 0
datasetRunId: "7891-5678-90ab-hijk"
ScoreDataType:
enum:
- NUMERIC
+1 -1
View File
@@ -12,7 +12,7 @@ groups:
# namespaceExport: Langfuse
# allowCustomFetcher: true
- name: fernapi/fern-openapi
version: 0.0.26
version: 0.1.7
output:
location: local-file-system
path: ../../../web/public/generated/api-client
@@ -0,0 +1,32 @@
name: langfuse-organizations
docs: |
## General Notes
This admin API is only available on self-hosted instances, not on Langfuse Cloud.
An administrator must set the `ADMIN_API_KEY` environment variable to use this API.
## Authentication
Authenticate with the API by setting the Authorization header to `Bearer $ADMIN_API_KEY` where
`$ADMIN_API_KEY` is the API key set via the container environment.
## Exports
- OpenAPI spec: https://cloud.langfuse.com/generated/organizations-api/openapi.yml
- Postman collection: https://cloud.langfuse.com/generated/organizations-postman/collection.json
## Looking for the Langfuse API?
Are you looking for the full Langfuse API?
Check out our [API reference](https://api.reference.langfuse.com/#overview).
error-discrimination:
strategy: status-code
auth: bearer
imports:
commons: commons.yml
errors:
- commons.Error
- commons.UnauthorizedError
- commons.AccessDeniedError
- commons.MethodNotAllowedError
@@ -0,0 +1,13 @@
errors:
Error:
status-code: 400
type: string
UnauthorizedError:
status-code: 401
type: string
AccessDeniedError:
status-code: 403
type: string
MethodNotAllowedError:
status-code: 405
type: string
@@ -0,0 +1,170 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
service:
auth: true
base-path: /api/admin
endpoints:
getAll:
docs: Get all organizations
method: GET
path: /organizations
response: Organizations
errors:
- commons.UnauthorizedError
- commons.MethodNotAllowedError
create:
docs: Create a new organization
method: POST
path: /organizations
request:
name: CreateOrganizationRequest
body:
properties:
name: string
metadata:
type: optional<map<string, unknown>>
docs: Optional metadata for the organization
response: Organization
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
getById:
docs: Get organization by ID
method: GET
path: /organizations/{organizationId}
path-parameters:
organizationId: string
response: Organization
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
update:
docs: Update an organization
method: PUT
path: /organizations/{organizationId}
path-parameters:
organizationId: string
request:
name: UpdateOrganizationRequest
body:
properties:
name: string
metadata:
type: optional<map<string, unknown>>
docs: Optional metadata for the organization
response: Organization
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
delete:
docs: Delete an organization
method: DELETE
path: /organizations/{organizationId}
path-parameters:
organizationId: string
response: DeleteOrganizationResponse
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
# API Key endpoints
getApiKeys:
docs: Get all API keys for an organization
method: GET
path: /organizations/{organizationId}/apiKeys
path-parameters:
organizationId: string
response: ApiKeyList
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
createApiKey:
docs: Create a new API key for an organization
method: POST
path: /organizations/{organizationId}/apiKeys
path-parameters:
organizationId: string
request:
name: CreateApiKeyRequest
body:
properties:
note:
type: optional<string>
docs: Optional note for the API key
response: ApiKeyResponse
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
deleteApiKey:
docs: Delete an API key for an organization
method: DELETE
path: /organizations/{organizationId}/apiKeys/{apiKeyId}
path-parameters:
organizationId: string
apiKeyId: string
response: DeleteOrganizationResponse
errors:
- commons.Error
- commons.UnauthorizedError
- commons.MethodNotAllowedError
types:
Organizations:
docs: List of organizations
properties:
organizations: list<Organization>
Organization:
docs: Organization object
properties:
id: string
name: string
createdAt: datetime
metadata:
type: map<string, unknown>
docs: Metadata for the organization
DeleteOrganizationResponse:
docs: Response for successful organization deletion
properties:
success: boolean
ApiKeyList:
docs: List of API keys for an organization
properties:
apiKeys: list<ApiKeySummary>
ApiKeySummary:
docs: Summary of an API key
properties:
id: string
createdAt: datetime
expiresAt: optional<datetime>
lastUsedAt: optional<datetime>
note: optional<string>
publicKey: string
displaySecretKey: string
ApiKeyResponse:
docs: Response for API key creation
properties:
id: string
createdAt: datetime
publicKey: string
secretKey: string
displaySecretKey: string
note: optional<string>
+17
View File
@@ -0,0 +1,17 @@
default-group: local
groups:
local:
generators:
- name: fernapi/fern-openapi
version: 0.1.7
output:
location: local-file-system
path: ../../../web/public/generated/organizations-api
config:
namespaceExport: Langfuse
allowCustomFetcher: true
- name: fernapi/fern-postman
version: 0.0.45
output:
location: local-file-system
path: ../../../web/public/generated/organizations-postman
@@ -0,0 +1,201 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
pagination: ./utils/pagination.yml
service:
auth: true
base-path: /api/public
endpoints:
listQueues:
docs: Get all annotation queues
method: GET
path: /annotation-queues
request:
name: GetAnnotationQueuesRequest
query-parameters:
page:
type: optional<integer>
docs: page number, starts at 1
limit:
type: optional<integer>
docs: limit of items per page
response: PaginatedAnnotationQueues
getQueue:
docs: Get an annotation queue by ID
method: GET
path: /annotation-queues/{queueId}
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
response: AnnotationQueue
listQueueItems:
docs: Get items for a specific annotation queue
method: GET
path: /annotation-queues/{queueId}/items
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
request:
name: GetAnnotationQueueItemsRequest
query-parameters:
status:
type: optional<AnnotationQueueStatus>
docs: Filter by status
page:
type: optional<integer>
docs: page number, starts at 1
limit:
type: optional<integer>
docs: limit of items per page
response: PaginatedAnnotationQueueItems
getQueueItem:
docs: Get a specific item from an annotation queue
method: GET
path: /annotation-queues/{queueId}/items/{itemId}
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
itemId:
type: string
docs: The unique identifier of the annotation queue item
response: AnnotationQueueItem
createQueueItem:
docs: Add an item to an annotation queue
method: POST
path: /annotation-queues/{queueId}/items
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
request: CreateAnnotationQueueItemRequest
response: AnnotationQueueItem
updateQueueItem:
docs: Update an annotation queue item
method: PATCH
path: /annotation-queues/{queueId}/items/{itemId}
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
itemId:
type: string
docs: The unique identifier of the annotation queue item
request: UpdateAnnotationQueueItemRequest
response: AnnotationQueueItem
deleteQueueItem:
docs: Remove an item from an annotation queue
method: DELETE
path: /annotation-queues/{queueId}/items/{itemId}
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
itemId:
type: string
docs: The unique identifier of the annotation queue item
response: DeleteAnnotationQueueItemResponse
createQueueAssignment:
docs: Create an assignment for a user to an annotation queue
method: POST
path: /annotation-queues/{queueId}/assignments
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
request: AnnotationQueueAssignmentRequest
response: CreateAnnotationQueueAssignmentResponse
deleteQueueAssignment:
docs: Delete an assignment for a user to an annotation queue
method: DELETE
path: /annotation-queues/{queueId}/assignments
path-parameters:
queueId:
type: string
docs: The unique identifier of the annotation queue
request: AnnotationQueueAssignmentRequest
response: DeleteAnnotationQueueAssignmentResponse
types:
AnnotationQueueStatus:
enum:
- PENDING
- COMPLETED
AnnotationQueueObjectType:
enum:
- TRACE
- OBSERVATION
AnnotationQueue:
properties:
id: string
name: string
description: optional<string>
scoreConfigIds: list<string>
createdAt: datetime
updatedAt: datetime
AnnotationQueueItem:
properties:
id: string
queueId: string
objectId: string
objectType: AnnotationQueueObjectType
status: AnnotationQueueStatus
completedAt: optional<datetime>
createdAt: datetime
updatedAt: datetime
PaginatedAnnotationQueues:
properties:
data: list<AnnotationQueue>
meta: pagination.MetaResponse
PaginatedAnnotationQueueItems:
properties:
data: list<AnnotationQueueItem>
meta: pagination.MetaResponse
CreateAnnotationQueueItemRequest:
properties:
objectId: string
objectType: AnnotationQueueObjectType
status:
type: optional<AnnotationQueueStatus>
docs: Defaults to PENDING for new queue items
UpdateAnnotationQueueItemRequest:
properties:
status:
type: optional<AnnotationQueueStatus>
DeleteAnnotationQueueItemResponse:
properties:
success: boolean
message: string
AnnotationQueueAssignmentRequest:
properties:
userId: string
DeleteAnnotationQueueAssignmentResponse:
properties:
success: boolean
CreateAnnotationQueueAssignmentResponse:
properties:
userId: string
queueId: string
projectId: string
+88 -7
View File
@@ -38,6 +38,9 @@ types:
public:
type: optional<boolean>
docs: Public traces are accessible via url without login
environment:
type: optional<string>
docs: The environment from which this trace originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
TraceWithDetails: # GET /traces
extends: Trace
properties:
@@ -72,13 +75,16 @@ types:
type: list<ObservationsView>
docs: List of observations
scores:
type: list<Score>
type: list<ScoreV1>
docs: List of scores
Session:
properties:
id: string
createdAt: datetime
projectId: string
environment:
type: optional<string>
docs: The environment from which this session originated.
SessionWithTraces:
extends: Session
properties:
@@ -145,6 +151,9 @@ types:
costDetails:
type: optional<map<string, double>>
docs: The cost details of the observation. Key is the name of the cost metric, value is the cost in USD. The total key is the sum of all (non-total) cost metrics or the total value ingested.
environment:
type: optional<string>
docs: The environment from which this observation originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
ObservationsView:
extends: Observation
@@ -231,7 +240,7 @@ types:
properties:
value: double
label: string
BaseScore:
BaseScoreV1:
properties:
id: string
traceId: string
@@ -243,12 +252,77 @@ types:
updatedAt: datetime
authorUserId: optional<string>
comment: optional<string>
metadata: optional<unknown>
configId:
type: optional<string>
docs: Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
queueId:
type: optional<string>
docs: Reference an annotation queue on a score. Populated if the score was initially created in an annotation queue.
environment:
type: optional<string>
docs: The environment from which this score originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
NumericScoreV1:
extends: BaseScoreV1
properties:
value:
type: double
docs: The numeric value of the score
BooleanScoreV1:
extends: BaseScoreV1
properties:
value:
type: double
docs: The numeric value of the score. Equals 1 for "True" and 0 for "False"
stringValue:
type: string
docs: The string representation of the score value. Is inferred from the numeric value and equals "True" or "False"
CategoricalScoreV1:
extends: BaseScoreV1
properties:
value:
type: optional<double>
docs: Only defined if a config is linked. Represents the numeric category mapping of the stringValue
stringValue:
type: string
docs: The string representation of the score value. If no config is linked, can be any string. Otherwise, must map to a config category
ScoreV1:
discriminant: "dataType"
union:
NUMERIC:
type: NumericScoreV1
docs: "Score with NUMERIC data type"
CATEGORICAL:
type: CategoricalScoreV1
docs: "Score with CATEGORICAL data type"
BOOLEAN:
type: BooleanScoreV1
docs: "Score with BOOLEAN data type"
BaseScore:
properties:
id: string
traceId: optional<string>
sessionId: optional<string>
observationId: optional<string>
datasetRunId: optional<string>
name: string
source: ScoreSource
timestamp: datetime
createdAt: datetime
updatedAt: datetime
authorUserId: optional<string>
comment: optional<string>
metadata: optional<unknown>
configId:
type: optional<string>
docs: Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
queueId:
type: optional<string>
docs: Reference an annotation queue on a score. Populated if the score was initially created in an annotation queue.
environment:
type: optional<string>
docs: The environment from which this score originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
NumericScore:
extends: BaseScore
properties:
@@ -378,18 +452,18 @@ types:
type: string
startDate:
docs: Apply only to generations which are newer than this ISO date.
type: optional<date>
type: optional<datetime>
unit:
docs: Unit used by this model.
type: ModelUsageUnit
type: optional<ModelUsageUnit>
inputPrice:
docs: Price (USD) per input unit
docs: Deprecated. See 'prices' instead. Price (USD) per input unit
type: optional<double>
outputPrice:
docs: Price (USD) per output unit
docs: Deprecated. See 'prices' instead. Price (USD) per output unit
type: optional<double>
totalPrice:
docs: Price (USD) per total unit. Cannot be set if input or output price is set.
docs: Deprecated. See 'prices' instead. Price (USD) per total unit. Cannot be set if input or output price is set.
type: optional<double>
tokenizerId:
docs: Optional. Tokenizer to be applied to observations which match to this model. See docs for more details.
@@ -399,6 +473,13 @@ types:
type: optional<unknown>
isLangfuseManaged:
type: boolean
prices:
docs: Price (USD) by usage type
type: map<string, ModelPrice>
ModelPrice:
properties:
price: double
# Utilities
ModelUsageUnit:
@@ -37,8 +37,21 @@ service:
type: optional<integer>
docs: limit of items per page
response: PaginatedDatasetItems
delete:
docs: Delete a dataset item and all its run items. This action is irreversible.
method: DELETE
path: /dataset-items/{id}
path-parameters:
id:
type: string
response: DeleteDatasetItemResponse
types:
DeleteDatasetItemResponse:
properties:
message:
type: string
docs: Success message after deletion
CreateDatasetItemRequest:
properties:
datasetName: string
@@ -1,6 +1,7 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
pagination: ./utils/pagination.yml
service:
auth: true
base-path: /api/public
@@ -11,6 +12,23 @@ service:
path: /dataset-run-items
request: CreateDatasetRunItemRequest
response: commons.DatasetRunItem
list:
method: GET
docs: List dataset run items
path: /dataset-run-items
request:
name: ListDatasetRunItemsRequest
query-parameters:
datasetId: string
runName: string
page:
type: optional<integer>
docs: page number, starts at 1
limit:
type: optional<integer>
docs: limit of items per page
response: PaginatedDatasetRunItems
types:
CreateDatasetRunItemRequest:
properties:
@@ -26,3 +44,7 @@ types:
traceId:
type: optional<string>
docs: traceId should always be provided. For compatibility with older SDK versions it can also be inferred from the provided observationId.
PaginatedDatasetRunItems:
properties:
data: list<commons.DatasetRunItem>
meta: pagination.MetaResponse
+11
View File
@@ -41,6 +41,14 @@ service:
datasetName: string
runName: string
response: commons.DatasetRunWithItems
deleteRun:
method: DELETE
docs: Delete a dataset run and all its run items. This action is irreversible.
path: /datasets/{datasetName}/runs/{runName}
path-parameters:
datasetName: string
runName: string
response: DeleteDatasetRunResponse
getRuns:
method: GET
docs: Get dataset runs
@@ -72,3 +80,6 @@ types:
properties:
data: list<commons.DatasetRun>
meta: pagination.MetaResponse
DeleteDatasetRunResponse:
properties:
message: string
+107 -9
View File
@@ -8,7 +8,15 @@ service:
endpoints:
batch:
docs: |
Batched ingestion for Langfuse Tracing. If you want to use tracing via the API, such as to build your own Langfuse client implementation, this is the only API route you need to implement.
Batched ingestion for Langfuse Tracing.
If you want to use tracing via the API, such as to build your own Langfuse client implementation, this is the only API route you need to implement.
Within each batch, there can be multiple events.
Each event has a type, an id, a timestamp, metadata and a body.
Internally, we refer to this as the "event envelope" as it tells us something about the event but not the trace.
We use the event id within this envelope to deduplicate messages to avoid processing the same event twice, i.e. the event id should be unique per request.
The event.body.id is the ID of the actual trace and will be used for updates and will be visible within the Langfuse App.
I.e. if you want to update a trace, you'd use the same body id, but separate event IDs.
Notes:
- Introduction to data model: https://langfuse.com/docs/tracing-data-model
@@ -29,6 +37,70 @@ service:
response:
type: IngestionResponse
status-code: 207
examples:
# Trace Create Request
- request:
batch:
- id: abcdef-1234-5678-90ab
timestamp: "2022-01-01T00:00:00.000Z"
type: "trace-create"
body:
id: abcdef-1234-5678-90ab
timestamp: "2022-01-01T00:00:00.000Z"
environment: "production"
name: "My Trace"
userId: "1234-5678-90ab-cdef"
input: "My input"
output: "My output"
sessionId: "1234-5678-90ab-cdef"
release: "1.0.0"
version: "1.0.0"
metadata: "My metadata"
tags: ["tag1", "tag2"]
public: true
response:
body:
successes:
- id: abcdef-1234-5678-90ab
status: 201
errors: []
# Observation Create Request
- request:
batch:
- id: abcdef-1234-5678-90ab
timestamp: "2022-01-01T00:00:00.000Z"
type: "span-create"
body:
id: abcdef-1234-5678-90ab
traceId: "1234-5678-90ab-cdef"
startTime: "2022-01-01T00:00:00.000Z"
environment: "test"
response:
body:
successes:
- id: abcdef-1234-5678-90ab
status: 201
errors: []
# Score Create Request
- request:
batch:
- id: abcdef-1234-5678-90ab
timestamp: "2022-01-01T00:00:00.000Z"
type: "score-create"
body:
id: abcdef-1234-5678-90ab
traceId: "1234-5678-90ab-cdef"
name: "My Score"
value: 0.9
environment: "default"
response:
body:
successes:
- id: abcdef-1234-5678-90ab
status: 201
errors: []
types:
IngestionEvent:
@@ -99,6 +171,7 @@ types:
statusMessage: optional<string>
parentObservationId: optional<string>
version: optional<string>
environment: optional<string>
CreateEventBody:
extends: OptionalObservationBody
@@ -128,7 +201,7 @@ types:
modelParameters: optional<map<string, commons.MapValue>>
usage: optional<IngestionUsage>
usageDetails: optional<UsageDetails>
costDetails: optional<map<string, float>>
costDetails: optional<map<string, double>>
promptName: optional<string>
promptVersion: optional<integer>
@@ -141,7 +214,7 @@ types:
usage: optional<IngestionUsage>
promptName: optional<string>
usageDetails: optional<UsageDetails>
costDetails: optional<map<string, float>>
costDetails: optional<map<string, double>>
promptVersion: optional<integer>
ObservationBody:
@@ -163,6 +236,7 @@ types:
level: optional<commons.ObservationLevel>
statusMessage: optional<string>
parentObservationId: optional<string>
environment: optional<string>
TraceBody:
properties:
@@ -177,6 +251,7 @@ types:
version: optional<string>
metadata: optional<unknown>
tags: optional<list<string>>
environment: optional<string>
public:
type: optional<boolean>
docs: Make trace publicly accessible via url
@@ -188,13 +263,17 @@ types:
ScoreBody:
properties:
id: optional<string>
traceId: string
traceId: optional<string>
sessionId: optional<string>
observationId: optional<string>
datasetRunId: optional<string>
name: string
environment: optional<string>
value:
type: commons.CreateScoreValue
docs: The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId: optional<string>
comment: optional<string>
metadata: optional<unknown>
dataType:
type: optional<commons.ScoreDataType>
docs: When set, must match the score value's type. If not set, will be inferred from the score value or config
@@ -238,6 +317,14 @@ types:
dataType: "BOOLEAN"
configId: "1234-5678-90ab-cdef"
traceId: "cdef-1234-5678-90ab"
- value:
name: "contextrelevant"
value: "not relevant"
sessionId: "abyt-1234-5678-80ab"
- value:
name: "hallucination"
value: 0
datasetRunId: "7891-5678-90ab-hijk"
BaseEvent:
properties:
@@ -318,16 +405,27 @@ types:
successes: list<IngestionSuccess>
errors: list<IngestionError>
OpenAIUsageSchema:
OpenAICompletionUsageSchema:
docs: OpenAI Usage schema from (Chat-)Completion APIs
properties:
prompt_tokens: integer
completion_tokens: integer
total_tokens: integer
prompt_tokens_details: optional<map<string, integer>>
completion_tokens_details: optional<map<string, integer>>
prompt_tokens_details: optional<map<string, optional<integer>>>
completion_tokens_details: optional<map<string, optional<integer>>>
OpenAIResponseUsageSchema:
docs: OpenAI Usage schema from Response API
properties:
input_tokens: integer
output_tokens: integer
total_tokens: integer
input_tokens_details: optional<map<string, optional<integer>>>
output_tokens_details: optional<map<string, optional<integer>>>
UsageDetails:
discriminated: false
union:
- map<string, integer>
- OpenAIUsageSchema
- OpenAICompletionUsageSchema
- OpenAIResponseUsageSchema
@@ -0,0 +1,102 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
pagination: ./utils/pagination.yml
service:
auth: true
base-path: /api/public
endpoints:
list:
method: GET
docs: Get all LLM connections in a project
path: /llm-connections
request:
name: GetLlmConnectionsRequest
query-parameters:
page:
type: optional<integer>
docs: page number, starts at 1
limit:
type: optional<integer>
docs: limit of items per page
response: PaginatedLlmConnections
upsert:
method: PUT
docs: Create or update an LLM connection. The connection is upserted on provider.
path: /llm-connections
request: UpsertLlmConnectionRequest
response: LlmConnection
types:
LlmConnection:
docs: LLM API connection configuration (secrets excluded)
properties:
id: string
provider:
type: string
docs: Provider name (e.g., 'openai', 'my-gateway'). Must be unique in project, used for upserting.
adapter:
type: string
docs: The adapter used to interface with the LLM
displaySecretKey:
type: string
docs: Masked version of the secret key for display purposes
baseURL:
type: optional<string>
docs: Custom base URL for the LLM API
customModels:
type: list<string>
docs: List of custom model names available for this connection
withDefaultModels:
type: boolean
docs: Whether to include default models for this adapter
extraHeaderKeys:
type: list<string>
docs: Keys of extra headers sent with requests (values excluded for security)
createdAt: datetime
updatedAt: datetime
PaginatedLlmConnections:
properties:
data: list<LlmConnection>
meta: pagination.MetaResponse
UpsertLlmConnectionRequest:
docs: Request to create or update an LLM connection (upsert)
properties:
provider:
type: string
docs: Provider name (e.g., 'openai', 'my-gateway'). Must be unique in project, used for upserting.
adapter:
type: LlmAdapter
docs: The adapter used to interface with the LLM
secretKey:
type: string
docs: Secret key for the LLM API.
baseURL:
type: optional<string>
docs: Custom base URL for the LLM API
customModels:
type: optional<list<string>>
docs: List of custom model names
withDefaultModels:
type: optional<boolean>
docs: Whether to include default models. Default is true.
extraHeaders:
type: optional<map<string, string>>
docs: Extra headers to send with requests
LlmAdapter:
enum:
- value: anthropic
name: Anthropic
- value: openai
name: OpenAI
- value: azure
name: Azure
- value: bedrock
name: Bedrock
- value: google-vertex-ai
name: GoogleVertexAI
- value: google-ai-studio
name: GoogleAIStudio
+55 -58
View File
@@ -6,67 +6,64 @@ service:
auth: true
base-path: /api/public
endpoints:
daily:
docs: Get daily metrics of the Langfuse project
metrics:
docs: Get metrics from the Langfuse project using a query object
method: GET
path: /metrics/daily
path: /metrics
request:
name: GetDailyMetricsRequest
name: GetMetricsRequest
query-parameters:
page:
type: optional<integer>
docs: page number, starts at 1
limit:
type: optional<integer>
docs: limit of items per page
traceName:
type: optional<string>
docs: Optional filter by the name of the trace
userId:
type: optional<string>
docs: Optional filter by the userId associated with the trace
tags:
type: optional<string>
allow-multiple: true
docs: Optional filter for metrics where traces include all of these tags
fromTimestamp:
type: optional<datetime>
docs: Optional filter to only include traces and observations on or after a certain datetime (ISO 8601)
toTimestamp:
type: optional<datetime>
docs: Optional filter to only include traces and observations before a certain datetime (ISO 8601)
response: DailyMetrics
query:
type: string
docs: |
JSON string containing the query parameters with the following structure:
```json
{
"view": string, // Required. One of "traces", "observations", "scores-numeric", "scores-categorical"
"dimensions": [ // Optional. Default: []
{
"field": string // Field to group by, e.g. "name", "userId", "sessionId"
}
],
"metrics": [ // Required. At least one metric must be provided
{
"measure": string, // What to measure, e.g. "count", "latency", "value"
"aggregation": string // How to aggregate, e.g. "count", "sum", "avg", "p95", "histogram"
}
],
"filters": [ // Optional. Default: []
{
"column": string, // Column to filter on
"operator": string, // Operator, e.g. "=", ">", "<", "contains"
"value": any, // Value to compare against
"type": string, // Data type, e.g. "string", "number", "stringObject"
"key": string // Required only when filtering on metadata
}
],
"timeDimension": { // Optional. Default: null. If provided, results will be grouped by time
"granularity": string // One of "minute", "hour", "day", "week", "month", "auto"
},
"fromTimestamp": string, // Required. ISO datetime string for start of time range
"toTimestamp": string, // Required. ISO datetime string for end of time range
"orderBy": [ // Optional. Default: null
{
"field": string, // Field to order by
"direction": string // "asc" or "desc"
}
],
"config": { // Optional. Query-specific configuration
"bins": number, // Optional. Number of bins for histogram (1-100), default: 10
"row_limit": number // Optional. Row limit for results (1-1000)
}
}
```
response: MetricsResponse
types:
DailyMetrics:
MetricsResponse:
properties:
data:
type: list<DailyMetricsDetails>
docs: A list of daily metrics, only days with ingested data are included.
meta: pagination.MetaResponse
DailyMetricsDetails:
properties:
date: date
countTraces: integer
countObservations: integer
totalCost:
type: double
docs: Total model cost in USD
usage: list<UsageByModel>
UsageByModel:
docs: Daily usage of a given model. Usage corresponds to the unit set for the specific model (e.g. tokens).
properties:
model: optional<string>
inputUsage:
type: integer
docs: Total number of generation input units (e.g. tokens)
outputUsage:
type: integer
docs: Total number of generation output units (e.g. tokens)
totalUsage:
type: integer
docs: Total number of generation total units (e.g. tokens)
countTraces: integer
countObservations: integer
totalCost:
type: double
docs: Total model cost in USD
type: list<map<string, unknown>>
docs: |
The metrics data. Each item in the list contains the metric values and dimensions requested in the query.
Format varies based on the query parameters.
Histograms will return an array with [lower, upper, height] tuples.
+1 -1
View File
@@ -58,7 +58,7 @@ types:
type: optional<datetime>
unit:
docs: Unit used by this model.
type: commons.ModelUsageUnit
type: optional<commons.ModelUsageUnit>
inputPrice:
docs: Price (USD) per input unit
type: optional<double>
+8 -1
View File
@@ -32,10 +32,17 @@ service:
userId: optional<string>
type: optional<string>
traceId: optional<string>
level:
type: optional<commons.ObservationLevel>
docs: Optional filter for observations with a specific level (e.g. "DEBUG", "DEFAULT", "WARNING", "ERROR").
parentObservationId: optional<string>
environment:
type: optional<string>
allow-multiple: true
docs: Optional filter for observations where the environment is one of the provided values.
fromStartTime:
type: optional<datetime>
docs: Retrieve only observations with a start_time or or after this datetime (ISO 8601).
docs: Retrieve only observations with a start_time on or after this datetime (ISO 8601).
toStartTime:
type: optional<datetime>
docs: Retrieve only observations with a start_time before this datetime (ISO 8601).
@@ -0,0 +1,80 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
pagination: ./utils/pagination.yml
service:
auth: true
base-path: /api/public
endpoints:
getOrganizationMemberships:
docs: Get all memberships for the organization associated with the API key (requires organization-scoped API key)
method: GET
path: /organizations/memberships
response: MembershipsResponse
updateOrganizationMembership:
docs: Create or update a membership for the organization associated with the API key (requires organization-scoped API key)
method: PUT
path: /organizations/memberships
request: MembershipRequest
response: MembershipResponse
getProjectMemberships:
docs: Get all memberships for a specific project (requires organization-scoped API key)
method: GET
path: /projects/{projectId}/memberships
path-parameters:
projectId: string
response: MembershipsResponse
updateProjectMembership:
docs: Create or update a membership for a specific project (requires organization-scoped API key). The user must already be a member of the organization.
method: PUT
path: /projects/{projectId}/memberships
path-parameters:
projectId: string
request: MembershipRequest
response: MembershipResponse
getOrganizationProjects:
docs: Get all projects for the organization associated with the API key (requires organization-scoped API key)
method: GET
path: /organizations/projects
response: OrganizationProjectsResponse
types:
MembershipRole:
enum:
- OWNER
- ADMIN
- MEMBER
- VIEWER
MembershipRequest:
properties:
userId: string
role: MembershipRole
MembershipResponse:
properties:
userId: string
role: MembershipRole
email: string
name: string
MembershipsResponse:
properties:
memberships: list<MembershipResponse>
OrganizationProject:
properties:
id: string
name: string
metadata: optional<map<string, unknown>>
createdAt: datetime
updatedAt: datetime
OrganizationProjectsResponse:
properties:
projects: list<OrganizationProject>
+121
View File
@@ -12,6 +12,85 @@ service:
path: /projects
response: Projects
create:
docs: Create a new project (requires organization-scoped API key)
method: POST
path: /projects
request:
name: CreateProjectRequest
body:
properties:
name: string
metadata:
type: optional<map<string, unknown>>
docs: Optional metadata for the project
retention:
type: integer
docs: Number of days to retain data. Must be 0 or at least 3 days. Requires data-retention entitlement for non-zero values. Optional.
response: Project
update:
docs: Update a project by ID (requires organization-scoped API key).
method: PUT
path: /projects/{projectId}
path-parameters:
projectId: string
request:
name: UpdateProjectRequest
body:
properties:
name: string
metadata:
type: optional<map<string, unknown>>
docs: Optional metadata for the project
retention:
type: integer
docs: Number of days to retain data. Must be 0 or at least 3 days. Requires data-retention entitlement for non-zero values. Optional.
response: Project
delete:
docs: Delete a project by ID (requires organization-scoped API key). Project deletion is processed asynchronously.
method: DELETE
path: /projects/{projectId}
path-parameters:
projectId: string
response:
status-code: 202
type: ProjectDeletionResponse
# API Key endpoints
getApiKeys:
docs: Get all API keys for a project (requires organization-scoped API key)
method: GET
path: /projects/{projectId}/apiKeys
path-parameters:
projectId: string
response: ApiKeyList
createApiKey:
docs: Create a new API key for a project (requires organization-scoped API key)
method: POST
path: /projects/{projectId}/apiKeys
path-parameters:
projectId: string
request:
name: CreateApiKeyRequest
body:
properties:
note:
type: optional<string>
docs: Optional note for the API key
response: ApiKeyResponse
deleteApiKey:
docs: Delete an API key for a project (requires organization-scoped API key)
method: DELETE
path: /projects/{projectId}/apiKeys/{apiKeyId}
path-parameters:
projectId: string
apiKeyId: string
response: ApiKeyDeletionResponse
types:
Projects:
properties:
@@ -21,3 +100,45 @@ types:
properties:
id: string
name: string
metadata:
type: map<string, unknown>
docs: Metadata for the project
retentionDays:
type: optional<integer>
docs: Number of days to retain data. Null or 0 means no retention. Omitted if no retention is configured.
ProjectDeletionResponse:
properties:
success: boolean
message: string
ApiKeyList:
docs: List of API keys for a project
properties:
apiKeys: list<ApiKeySummary>
ApiKeySummary:
docs: Summary of an API key
properties:
id: string
createdAt: datetime
expiresAt: optional<datetime>
lastUsedAt: optional<datetime>
note: optional<string>
publicKey: string
displaySecretKey: string
ApiKeyResponse:
docs: Response for API key creation
properties:
id: string
createdAt: datetime
publicKey: string
secretKey: string
displaySecretKey: string
note: optional<string>
ApiKeyDeletionResponse:
docs: Response for API key deletion
properties:
success: boolean
@@ -0,0 +1,27 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
prompts: ./prompts.yml
pagination: ./utils/pagination.yml
service:
auth: true
base-path: /api/public/v2
endpoints:
update:
docs: Update labels for a specific prompt version
method: PATCH
path: /prompts/{name}/versions/{version}
path-parameters:
name:
type: string
docs: The name of the prompt
version:
type: integer
docs: Version of the prompt to update
request:
name: UpdatePromptRequest
body:
properties:
newLabels:
type: list<string>
docs: New labels for the prompt version. Labels are unique across versions. The "latest" label is reserved and managed by Langfuse.
response: prompts.Prompt
+24 -2
View File
@@ -82,7 +82,7 @@ types:
CreateChatPromptRequest:
properties:
name: string
prompt: list<ChatMessage>
prompt: list<ChatMessageWithPlaceholders>
config: optional<unknown>
labels:
type: optional<list<string>>
@@ -90,6 +90,9 @@ types:
tags:
type: optional<list<string>>
docs: List of tags to apply to all versions of this prompt.
commitMessage:
type: optional<string>
docs: Commit message for this prompt version.
CreateTextPromptRequest:
properties:
@@ -102,6 +105,9 @@ types:
tags:
type: optional<list<string>>
docs: List of tags to apply to all versions of this prompt.
commitMessage:
type: optional<string>
docs: Commit message for this prompt version.
Prompt:
union:
@@ -119,6 +125,17 @@ types:
tags:
type: list<string>
docs: List of tags. Used to filter via UI and API. The same across versions of a prompt.
commitMessage:
type: optional<string>
docs: Commit message for this prompt version.
resolutionGraph:
type: optional<map<string, unknown>>
docs: The dependency resolution graph for the current prompt. Null if prompt has no dependencies.
ChatMessageWithPlaceholders:
union:
chatmessage: ChatMessage
placeholder: PlaceholderMessage
ChatMessage:
properties:
@@ -127,6 +144,11 @@ types:
content:
type: string
PlaceholderMessage:
properties:
name:
type: string
TextPrompt:
extends: BasePrompt
properties:
@@ -135,4 +157,4 @@ types:
ChatPrompt:
extends: BasePrompt
properties:
prompt: list<ChatMessage>
prompt: list<ChatMessageWithPlaceholders>
+199
View File
@@ -0,0 +1,199 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
commons: ./commons.yml
service:
auth: true
base-path: /api/public/scim
endpoints:
getServiceProviderConfig:
docs: Get SCIM Service Provider Configuration (requires organization-scoped API key)
method: GET
path: /ServiceProviderConfig
response: ServiceProviderConfig
getResourceTypes:
docs: Get SCIM Resource Types (requires organization-scoped API key)
method: GET
path: /ResourceTypes
response: ResourceTypesResponse
getSchemas:
docs: Get SCIM Schemas (requires organization-scoped API key)
method: GET
path: /Schemas
response: SchemasResponse
listUsers:
docs: List users in the organization (requires organization-scoped API key)
method: GET
path: /Users
request:
name: ListUsersRequest
query-parameters:
filter:
docs: Filter expression (e.g. userName eq "value")
type: optional<string>
startIndex:
docs: 1-based index of the first result to return (default 1)
type: optional<integer>
count:
docs: Maximum number of results to return (default 100)
type: optional<integer>
response: ScimUsersListResponse
createUser:
docs: Create a new user in the organization (requires organization-scoped API key)
method: POST
path: /Users
request:
name: CreateUserRequest
body:
properties:
userName:
docs: User's email address (required)
type: string
name:
docs: User's name information
type: ScimName
emails:
docs: User's email addresses
type: optional<list<ScimEmail>>
active:
docs: Whether the user is active
type: optional<boolean>
password:
docs: Initial password for the user
type: optional<string>
response: ScimUser
getUser:
docs: Get a specific user by ID (requires organization-scoped API key)
method: GET
path: /Users/{userId}
path-parameters:
userId: string
response: ScimUser
deleteUser:
docs: Remove a user from the organization (requires organization-scoped API key). Note that this only removes the user from the organization but does not delete the user entity itself.
method: DELETE
path: /Users/{userId}
path-parameters:
userId: string
response: EmptyResponse
types:
ServiceProviderConfig:
properties:
schemas: list<string>
documentationUri: string
patch: ScimFeatureSupport
bulk: BulkConfig
filter: FilterConfig
changePassword: ScimFeatureSupport
sort: ScimFeatureSupport
etag: ScimFeatureSupport
authenticationSchemes: list<AuthenticationScheme>
meta: ResourceMeta
ScimFeatureSupport:
properties:
supported: boolean
BulkConfig:
properties:
supported: boolean
maxOperations: integer
maxPayloadSize: integer
FilterConfig:
properties:
supported: boolean
maxResults: integer
ResourceMeta:
properties:
resourceType: string
location: string
AuthenticationScheme:
properties:
name: string
description: string
specUri: string
type: string
primary: boolean
ResourceTypesResponse:
properties:
schemas: list<string>
totalResults: integer
Resources: list<ResourceType>
ResourceType:
properties:
schemas: optional<list<string>>
id: string
name: string
endpoint: string
description: string
schema: string
schemaExtensions: list<SchemaExtension>
meta: ResourceMeta
SchemaExtension:
properties:
schema: string
required: boolean
SchemasResponse:
properties:
schemas: list<string>
totalResults: integer
Resources: list<SchemaResource>
SchemaResource:
properties:
id: string
name: string
description: string
attributes: list<unknown>
meta: ResourceMeta
ScimUsersListResponse:
properties:
schemas: list<string>
totalResults: integer
startIndex: integer
itemsPerPage: integer
Resources: list<ScimUser>
ScimUser:
properties:
schemas: list<string>
id: string
userName: string
name: ScimName
emails: list<ScimEmail>
meta: UserMeta
UserMeta:
properties:
resourceType: string
created: optional<string>
lastModified: optional<string>
ScimName:
properties:
formatted: optional<string>
ScimEmail:
properties:
primary: boolean
value: string
type: string
EmptyResponse:
docs: Empty response for 204 No Content responses
properties: {}
+112
View File
@@ -0,0 +1,112 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/fern-api/fern/main/fern.schema.json
imports:
pagination: ./utils/pagination.yml
commons: ./commons.yml
service:
auth: true
base-path: /api/public/v2
endpoints:
get:
docs: Get a list of scores (supports both trace and session scores)
method: GET
path: /scores
request:
name: GetScoresRequest
query-parameters:
page:
type: optional<integer>
docs: Page number, starts at 1.
limit:
type: optional<integer>
docs: Limit of items per page. If you encounter api issues due to too large page sizes, try to reduce the limit.
userId:
type: optional<string>
docs: Retrieve only scores with this userId associated to the trace.
name:
type: optional<string>
docs: Retrieve only scores with this name.
fromTimestamp:
type: optional<datetime>
docs: Optional filter to only include scores created on or after a certain datetime (ISO 8601)
toTimestamp:
type: optional<datetime>
docs: Optional filter to only include scores created before a certain datetime (ISO 8601)
environment:
type: optional<string>
allow-multiple: true
docs: Optional filter for scores where the environment is one of the provided values.
source:
type: optional<commons.ScoreSource>
docs: Retrieve only scores from a specific source.
operator:
type: optional<string>
docs: Retrieve only scores with <operator> value.
value:
type: optional<double>
docs: Retrieve only scores with <operator> value.
scoreIds:
type: optional<string>
docs: Comma-separated list of score IDs to limit the results to.
configId:
type: optional<string>
docs: Retrieve only scores with a specific configId.
queueId:
type: optional<string>
docs: Retrieve only scores with a specific annotation queueId.
dataType:
type: optional<commons.ScoreDataType>
docs: Retrieve only scores with a specific dataType.
traceTags:
type: optional<string>
allow-multiple: true
docs: Only scores linked to traces that include all of these tags will be returned.
response: GetScoresResponse
get-by-id:
docs: Get a score (supports both trace and session scores)
method: GET
path: /scores/{scoreId}
path-parameters:
scoreId:
type: string
docs: The unique langfuse identifier of a score
response: commons.Score
types:
GetScoresResponseTraceData:
properties:
userId:
type: optional<string>
docs: The user ID associated with the trace referenced by score
tags:
type: optional<list<string>>
docs: A list of tags associated with the trace referenced by score
environment:
type: optional<string>
docs: The environment of the trace referenced by score
GetScoresResponseDataNumeric:
extends: commons.NumericScore
properties:
trace: optional<GetScoresResponseTraceData>
GetScoresResponseDataCategorical:
extends: commons.CategoricalScore
properties:
trace: optional<GetScoresResponseTraceData>
GetScoresResponseDataBoolean:
extends: commons.BooleanScore
properties:
trace: optional<GetScoresResponseTraceData>
GetScoresResponseData:
discriminant: dataType
union:
NUMERIC: GetScoresResponseDataNumeric
CATEGORICAL: GetScoresResponseDataCategorical
BOOLEAN: GetScoresResponseDataBoolean
GetScoresResponse:
properties:
data: list<GetScoresResponseData>
meta: pagination.MetaResponse
+12 -99
View File
@@ -7,73 +7,13 @@ service:
base-path: /api/public
endpoints:
create:
docs: Create a score
docs: Create a score (supports both trace and session scores)
method: POST
path: /scores
request: CreateScoreRequest
response: CreateScoreResponse
get:
docs: Get a list of scores
method: GET
path: /scores
request:
name: GetScoresRequest
query-parameters:
page:
type: optional<integer>
docs: Page number, starts at 1.
limit:
type: optional<integer>
docs: Limit of items per page. If you encounter api issues due to too large page sizes, try to reduce the limit.
userId:
type: optional<string>
docs: Retrieve only scores with this userId associated to the trace.
name:
type: optional<string>
docs: Retrieve only scores with this name.
fromTimestamp:
type: optional<datetime>
docs: Optional filter to only include scores created on or after a certain datetime (ISO 8601)
toTimestamp:
type: optional<datetime>
docs: Optional filter to only include scores created before a certain datetime (ISO 8601)
source:
type: optional<commons.ScoreSource>
docs: Retrieve only scores from a specific source.
operator:
type: optional<string>
docs: Retrieve only scores with <operator> value.
value:
type: optional<double>
docs: Retrieve only scores with <operator> value.
scoreIds:
type: optional<string>
docs: Comma-separated list of score IDs to limit the results to.
configId:
type: optional<string>
docs: Retrieve only scores with a specific configId.
queueId:
type: optional<string>
docs: Retrieve only scores with a specific annotation queueId.
dataType:
type: optional<commons.ScoreDataType>
docs: Retrieve only scores with a specific dataType.
traceTags:
type: optional<list<string>>
allow-multiple: true
docs: Only scores linked to traces that include all of these tags will be returned.
response: GetScoresResponse
get-by-id:
docs: Get a score
method: GET
path: /scores/{scoreId}
path-parameters:
scoreId:
type: string
docs: The unique langfuse identifier of a score
response: commons.Score
delete:
docs: Delete a score
docs: Delete a score (supports both trace and session scores)
method: DELETE
path: /scores/{scoreId}
path-parameters:
@@ -84,13 +24,19 @@ types:
CreateScoreRequest:
properties:
id: optional<string>
traceId: string
traceId: optional<string>
sessionId: optional<string>
observationId: optional<string>
datasetRunId: optional<string>
name: string
value:
type: commons.CreateScoreValue
docs: The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId: optional<string>
comment: optional<string>
metadata: optional<unknown>
environment:
type: optional<string>
docs: The environment of the score. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
dataType:
type: optional<commons.ScoreDataType>
docs: The data type of the score. When passing a configId this field is inferred. Otherwise, this field must be passed or will default to numeric.
@@ -113,10 +59,12 @@ types:
dataType: "NUMERIC"
configId: "9203-4567-89ab-cdef"
traceId: "cdef-1234-5678-90ab"
environment: "test"
- value:
name: "toxicity"
value: "not toxic"
traceId: "cdef-1234-5678-90ab"
environment: "production"
- value:
name: "correctness"
value: "partially correct"
@@ -139,38 +87,3 @@ types:
id:
type: string
docs: The id of the created object in Langfuse
GetScoresResponseTraceData:
properties:
userId:
type: optional<string>
docs: The user ID associated with the trace referenced by score
tags:
type: optional<list<string>>
docs: A list of tags associated with the trace referenced by score
GetScoresResponseDataNumeric:
extends: commons.NumericScore
properties:
trace: GetScoresResponseTraceData
GetScoresResponseDataCategorical:
extends: commons.CategoricalScore
properties:
trace: GetScoresResponseTraceData
GetScoresResponseDataBoolean:
extends: commons.BooleanScore
properties:
trace: GetScoresResponseTraceData
GetScoresResponseData:
discriminant: dataType
union:
NUMERIC: GetScoresResponseDataNumeric
CATEGORICAL: GetScoresResponseDataCategorical
BOOLEAN: GetScoresResponseDataBoolean
GetScoresResponse:
properties:
data: list<GetScoresResponseData>
meta: pagination.MetaResponse
+4
View File
@@ -25,6 +25,10 @@ service:
toTimestamp:
type: optional<datetime>
docs: Optional filter to only include sessions created before a certain datetime (ISO 8601)
environment:
type: optional<string>
allow-multiple: true
docs: Optional filter for sessions where the environment is one of the provided values.
response: PaginatedSessions
get:
docs: Get a session. Please note that `traces` on this endpoint are not paginated, if you plan to fetch large sessions, consider `GET /api/public/traces?sessionId=<sessionId>`
+31
View File
@@ -15,6 +15,15 @@ service:
type: string
docs: The unique langfuse identifier of a trace
response: commons.TraceWithFullDetails
delete:
docs: Delete a specific trace
method: DELETE
path: /traces/{traceId}
path-parameters:
traceId:
type: string
docs: The unique langfuse identifier of the trace to delete
response: DeleteTraceResponse
list:
docs: Get list of traces
method: GET
@@ -50,13 +59,35 @@ service:
release:
type: optional<string>
docs: Optional filter to only include traces with a certain release.
environment:
type: optional<string>
allow-multiple: true
docs: Optional filter for traces where the environment is one of the provided values.
fields:
type: optional<string>
docs: "Comma-separated list of fields to include in the response. Available field groups are 'core' (always included), 'io' (input, output, metadata), 'scores', 'observations', 'metrics'. If not provided, all fields are included. Example: 'core,scores,metrics'"
response: Traces
deleteMultiple:
docs: Delete multiple traces
method: DELETE
path: /traces
request:
name: DeleteTracesRequest
body:
properties:
traceIds:
type: list<string>
docs: List of trace IDs to delete
response: DeleteTraceResponse
types:
Traces:
properties:
data: list<commons.TraceWithDetails>
meta: pagination.MetaResponse
DeleteTraceResponse:
properties:
message: string
Sort:
properties:
id: string
+27 -21
View File
@@ -1,12 +1,14 @@
# yaml-language-server: $schema=https://schema.buildwithfern.dev/generators-yml.json
default-group: local
groups:
local:
generators:
- name: fernapi/fern-openapi
version: 0.0.31
version: 0.1.7
output:
location: local-file-system
path: ../../../web/public/generated/api
- name: fernapi/fern-python-sdk
version: 2.16.0
output:
@@ -19,28 +21,32 @@ groups:
pydantic_config:
require_optional_fields: false
use_str_enums: false
# - name: fernapi/fern-java-sdk
# version: 2.20.1
# output:
# location: local-file-system
# path: ../../../../langfuse-java/src/main/java/com/langfuse/client/
# config:
# client-class-name: LangfuseClient
- name: fernapi/fern-typescript-node-sdk
version: 2.6.1
output:
location: local-file-system
path: ../../../generated/typescript
config:
namespaceExport: LangfuseAPI
outputSourceFiles: true
skipResponseValidation: true
fetchSupport: native
formDataSupport: Node18
fileResponseType: binary-response
streamType: web
omitFernHeaders: true
- name: fernapi/fern-postman
version: 0.0.45
output:
location: local-file-system
path: ../../../web/public/generated/postman
# published:
# generators:
# - name: fernapi/fern-python-sdk
# version: 0.3.7
# output:
# location: pypi
# url: pypi.buildwithfern.com
# package-name: finto-fern-langfuse
# config:
# namespaceExport: Langfuse
# allowCustomFetcher: true
# - name: fernapi/fern-typescript-node-sdk
# version: 0.7.1
# output:
# location: npm
# url: npm.buildwithfern.com
# package-name: "@finto-fern/langfuse-node"
# config:
# namespaceExport: Langfuse
# allowCustomFetcher: true
+3 -3
View File
@@ -1,4 +1,4 @@
{
"organization": "finto",
"version": "0.43.7"
}
"organization": "langfuse",
"version": "0.56.0"
}
-1
View File
@@ -1 +0,0 @@
python/
+21 -11
View File
@@ -1,6 +1,6 @@
{
"name": "langfuse",
"version": "3.6.2",
"version": "3.96.2",
"author": "engineering@langfuse.com",
"license": "MIT",
"private": true,
@@ -17,26 +17,30 @@
"db:seed": "turbo run db:seed",
"db:seed:examples": "turbo run db:seed:examples",
"nuke": "bash ./scripts/nuke.sh",
"dx": "pnpm i && pnpm run infra:dev:prune && pnpm run infra:dev:up --pull always && pnpm --filter=shared run db:reset && pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"dx-f": "pnpm i && pnpm run infra:dev:prune && pnpm run infra:dev:up --pull always && pnpm --filter=shared run db:reset -f && pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"dx:skip-infra": "pnpm i && pnpm --filter=shared run db:reset && pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"dx": "pnpm i && pnpm run infra:dev:prune && pnpm run infra:dev:up --pull always && pnpm --filter=shared run db:reset:test && pnpm --filter=shared run db:reset && pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"dx-f": "pnpm i && pnpm run infra:dev:prune && pnpm run infra:dev:up --pull always && pnpm --filter=shared run db:reset:test && pnpm --filter=shared run db:reset -f && SKIP_CONFIRM=1 pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"dx:skip-infra": "pnpm i && pnpm --filter=shared run db:reset:test && pnpm --filter=shared run db:reset && pnpm --filter=shared run ch:reset && pnpm --filter=shared run db:seed:examples && pnpm run dev",
"build": "turbo run build",
"start": "turbo run start",
"dev": "turbo run dev",
"dev:worker": "turbo run dev --filter=worker",
"dev:web": "turbo run dev --filter=web",
"dev:web-turbo": "turbo run dev --filter=web -- --turbo",
"lint": "turbo run lint",
"format": "prettier --write \"**/*.{js,jsx,ts,tsx,css}\" --experimental-cli",
"format:check": "prettier --check \"**/*.{js,jsx,ts,tsx,css}\" --experimental-cli",
"test": "turbo run test",
"models:migrate": "turbo run models:migrate",
"release": "dotenv -e ../.env -- release-it",
"prepare": "husky"
},
"devDependencies": {
"@release-it/bumper": "^6.0.1",
"@release-it/bumper": "^7.0.5",
"braces": "3.0.3",
"dotenv-cli": "^7.4.2",
"husky": "^9.0.11",
"prettier": "^3.3.3",
"release-it": "^17.3.0",
"turbo": "^1.13.4"
"prettier": "^3.6.2",
"release-it": "^19.0.3",
"turbo": "^2.5.5"
},
"release-it": {
"git": {
@@ -84,8 +88,14 @@
"packageManager": "pnpm@9.5.0",
"pnpm": {
"overrides": {
"jsonpath-plus": "10.0.7",
"nanoid": "^3.3.8"
"nanoid": "^3.3.8",
"katex": "^0.16.21",
"tar-fs": "^2.1.2",
"rollup@^4.0.0": "^4.22.4",
"@types/node-fetch": "^2.6.13"
},
"patchedDependencies": {
"next-auth@4.24.11": "patches/next-auth@4.24.11.patch"
}
}
}
+8
View File
@@ -35,5 +35,13 @@ module.exports = {
{
files: ["*.js?(x)", "*.ts?(x)"],
},
{
files: ["*.ts", "*.mts", "*.cts", "*.tsx"],
// no-undef doesn't make sense in TS, see:
// https://typescript-eslint.io/troubleshooting/faqs/eslint/#i-get-errors-from-the-no-undef-rule-about-global-variables-not-being-defined-even-though-there-are-no-typescript-errors
rules: {
"no-undef": "off",
},
},
],
};
+1 -1
View File
@@ -13,7 +13,7 @@
"@vercel/style-guide": "^6.0.0",
"eslint-config-next": "^14.2.15",
"eslint-config-prettier": "^9.1.0",
"eslint-config-turbo": "^1.13.4",
"eslint-config-turbo": "^2.5.5",
"eslint-plugin-only-warn": "^1.1.0",
"typescript": "^5.4.5"
}
@@ -0,0 +1 @@
DROP TABLE event_log ON CLUSTER default;
@@ -0,0 +1,19 @@
CREATE TABLE event_log ON CLUSTER default
(
`id` String,
`project_id` String,
`entity_type` String,
`entity_id` String,
`event_id` Nullable(String),
`bucket_name` String,
`bucket_path` String,
`created_at` DateTime64(3) DEFAULT now(),
`updated_at` DateTime64(3) DEFAULT now()
) ENGINE = MergeTree()
ORDER BY (
project_id,
entity_type,
entity_id
);
@@ -0,0 +1,3 @@
ALTER TABLE traces ON CLUSTER default DROP COLUMN IF EXISTS environment;
ALTER TABLE observations ON CLUSTER default DROP COLUMN IF EXISTS environment;
ALTER TABLE scores ON CLUSTER default DROP COLUMN IF EXISTS environment;
@@ -0,0 +1,3 @@
ALTER TABLE traces ON CLUSTER default ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE observations ON CLUSTER default ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
ALTER TABLE scores ON CLUSTER default ADD COLUMN environment LowCardinality(String) DEFAULT 'default' AFTER project_id;
@@ -0,0 +1,7 @@
-- Drop materialized views
DROP VIEW IF EXISTS project_environments_traces_mv ON CLUSTER default;
DROP VIEW IF EXISTS project_environments_observations_mv ON CLUSTER default;
DROP VIEW IF EXISTS project_environments_scores_mv ON CLUSTER default;
-- Drop the project_environments table
DROP TABLE IF EXISTS project_environments ON CLUSTER default;
@@ -0,0 +1,30 @@
-- Create the project_environments table with AggregatingMergeTree engine
CREATE TABLE project_environments ON CLUSTER default (
`project_id` String,
`environments` SimpleAggregateFunction(groupUniqArrayArray, Array(String))
) ENGINE = ReplicatedAggregatingMergeTree
ORDER BY (project_id);
-- Create materialized view for traces
CREATE MATERIALIZED VIEW project_environments_traces_mv ON CLUSTER default TO project_environments AS
SELECT
project_id,
groupUniqArray(environment) AS environments
FROM traces
GROUP BY project_id;
-- Create materialized view for observations
CREATE MATERIALIZED VIEW project_environments_observations_mv ON CLUSTER default TO project_environments AS
SELECT
project_id,
groupUniqArray(environment) AS environments
FROM observations
GROUP BY project_id;
-- Create materialized view for scores
CREATE MATERIALIZED VIEW project_environments_scores_mv ON CLUSTER default TO project_environments AS
SELECT
project_id,
groupUniqArray(environment) AS environments
FROM scores
GROUP BY project_id;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default DROP COLUMN IF EXISTS metadata;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default ADD COLUMN metadata Map(LowCardinality(String), String) AFTER comment;
@@ -0,0 +1 @@
DROP TABLE blob_storage_file_log ON CLUSTER default;
@@ -0,0 +1,21 @@
CREATE TABLE blob_storage_file_log ON CLUSTER default (
`id` String,
`project_id` String,
`entity_type` String,
`entity_id` String,
`event_id` String,
`bucket_name` String,
`bucket_path` String,
`created_at` DateTime64(3) DEFAULT now(),
`updated_at` DateTime64(3) DEFAULT now(),
event_ts DateTime64(3),
is_deleted UInt8,
) ENGINE = ReplicatedReplacingMergeTree(event_ts, is_deleted)
ORDER BY (
project_id,
entity_type,
entity_id,
event_id
);
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default DROP COLUMN IF EXISTS session_id SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default ADD COLUMN session_id Nullable(String) AFTER trace_id SETTINGS mutations_sync = 2;
@@ -0,0 +1,2 @@
ALTER TABLE scores ON CLUSTER default ADD INDEX IF NOT EXISTS idx_project_trace_observation (project_id, trace_id, observation_id) TYPE bloom_filter(0.001) GRANULARITY 1 SETTINGS mutations_sync = 2;
ALTER TABLE scores ON CLUSTER default MATERIALIZE INDEX IF EXISTS idx_project_trace_observation SETTINGS mutations_sync = 2;
@@ -0,0 +1,2 @@
ALTER TABLE scores ON CLUSTER default DROP INDEX IF EXISTS idx_project_trace_observation SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default MODIFY COLUMN trace_id Nullable(String) SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default MODIFY COLUMN trace_id Nullable(String) SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default DROP INDEX IF EXISTS idx_project_trace_observation SETTINGS mutations_sync = 2;
@@ -0,0 +1,2 @@
ALTER TABLE scores ON CLUSTER default ADD INDEX IF NOT EXISTS idx_project_trace_observation (project_id, trace_id, observation_id) TYPE bloom_filter(0.001) GRANULARITY 1 SETTINGS mutations_sync = 2;
ALTER TABLE scores ON CLUSTER default MATERIALIZE INDEX IF EXISTS idx_project_trace_observation SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default DROP INDEX IF EXISTS idx_project_session SETTINGS mutations_sync = 2;
@@ -0,0 +1,2 @@
ALTER TABLE scores ON CLUSTER default ADD INDEX IF NOT EXISTS idx_project_session (project_id, session_id) TYPE bloom_filter(0.001) GRANULARITY 1 SETTINGS mutations_sync = 2;
ALTER TABLE scores ON CLUSTER default MATERIALIZE INDEX IF EXISTS idx_project_session SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default DROP COLUMN IF EXISTS dataset_run_id SETTINGS mutations_sync = 2;
@@ -0,0 +1 @@
ALTER TABLE scores ON CLUSTER default ADD COLUMN dataset_run_id Nullable(String) AFTER session_id SETTINGS mutations_sync = 2;

Some files were not shown because too many files have changed in this diff Show More