* fixup: limit render duration of traces table
* fixup: reduce render cycles through data flows and memoization
* refactor: enhance performance and structure of table components
- Removed unnecessary console logs in `data-table.tsx`.
- Refactored `TablePeekView` to use a more structured props type and memoization for improved performance.
- Updated `usePeekData` to reduce stale time from 5 minutes to 1 minute.
- Replaced `IOTableCell` with `MemoizedIOTableCell` in various components to optimize rendering.
- Adjusted `peekView` configurations in `observations.tsx` and `traces.tsx` for consistency and clarity.
* chore: eslint
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* chore: move features
* changes to pro plan
* chore: move features
* truncate title and description to prevent wraps
* remove unnecessary padding from charts
* improve bignumber chart
* add min h/w
* mobile resizing
* fix
2025-05-29 15:58:38 +00:00
Max DeichmannGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* perf: add exact timestamp match for eval batch actions
* perf: add exact timestamp match for eval batch actions
* perf: add exact timestamp match for eval batch actions
* chore: make `project_id` nullable on `eval_templates`
* feat: add script for managed langfuse evaluators
* feat: restyle running evaluator configs table
* feat: add peek view for running evals
* fixup: add peek view for evaluator library
* fixup: add configure button to table
* feat: rework eval creation flow
* feat: add option to edit eval from table
* fixup: configure evaluator flow with traces data preview
* style: eval configuration UI elements
* chore: add support for current langfuse templates
* chore: reference templateId on jobExecution
* style: add maintainer icon across
* chore: extract WorkerEvaluationVariableService
* fixup: support live data mapping interface
* fixup: revamp datasets eval ui
* fixup: add clone functionality
* chore: rename online evaluation -> llm-as-a-judge
* feat: add evaluator selection in experiment creation flow
* chore: refactor into hooks in experiment form
* feat: add ragas evaluator prompt templates
* chore: show evaluator prompt preview by default
* fix: eval table filtering
* feat: add input box to slider
* fix: clean up running eval peek view and logs
* fix: ensure preview variables are not wrapped in quotes in evaluation prompt preview
* feat: support navigating through traces table
* feat: support default model workflow
* fix: typos in managed evaluators
* fix: do not allow selection of evaluators requiring default to run
* fix: type and build errors
* chore: rename table to default_llm_model
* chore: rm empty file
* fixup: complete rename
* chore: push
* style: eval set up form
* style: separate new evaluator form out into two steps
* chore: ensure we only run evals if default model is set
* chore: add `partner` col to `eval_templates`
* style: new eval template creation
* chore: add proper mapping for dataset evals
* feat: also put evaluation selection on datasets page
* feat: datasets also run on historic experiments after creating mapping
* revert: 09ab242dc12c8647bc716c84d180bc8e631d4919
* chore: fix eslint and ts
* chore: fix warnings
* fix: ensure we inactivate all job configs that rely on default eval model
* fix: tests
* chore: fix eslint
* chore: wrap in tx
* chore: fix default model checkbox logic
* chore: improve wording on delete model
* fix: eslint
* chore: adjust migration order
* style: push ui improvements
* style: sort templates by partner
* fix: eslint
* chore(dashboards): expand measures for dashboards
* chore: add testcase
* chore: define inputTokens and outputTokens
* chore: add timePerOutputToken and costs
* chore: fix definition and handling of tag based breakdowns
* feat(dashboard): make dashboard widgets draggable
* chore: do not update on layout change
* chore: fix size
* chore: remove the preventCollision flag
* chore: linting
* chore: get row height dynamically
* chore: make chart resizable
* chore: styling for resizes
* chore: add migration to align y values
* chore: add owner to project and return langfuse dashboards
* chore: show owner for dashborads in table
* chore: snapshot
* chore: snapshot
* chore: allow widget clone
* chore: snapshot
* chore: linting
* chore: use project instead of user
* chore: show error message on clone failure
* chore: indicate existing dashboard widget relationship
* chore: show only project dashboards in selection for adding widgets
* chore: make full dashboard row clickable
* chore: add clone button to langfuse dashboards
* feat(dashboards): add support for cloning existing dashboards
* chore: move buttons into a dedicated dropdown menu
* chore: allow updates to dashboard name and description
* chore: lint
* chore: removal double capturing of dashboard delete
* chore: track deletions
* feat: full text search
* push
* feat: full text search
* feat: full text search
* feat: full text search
* feat: full text search
* feat: full text search
* fix
* push
* fix: fix json limits and scope to single bodies only
* fix
* fix
* jumping UI
* jumping UI
* jumping UI
* push
* Update web/src/components/table/data-table-toolbar.tsx
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* push
* fix: fix json limits and scope to single bodies only
* push
* fix: fix json limits and scope to single bodies only
* fix: fix json limits and scope to single bodies only
* fix: fix json limits and scope to single bodies only
* fix: fix json limits and scope to single bodies only
* fix: fix json limits and scope to single bodies only
* chore: memoize table body for enhanced front end performance
* chore: push
* chore: refactor
* style: speed up animation
* chore: re-render on table state changes
* chore: import function directly
* chore: refactor to use timestamp
* chore: push
* chore: push
* feat: add dlx retry queue and service
* feat: add dlx retry queue and service
* feat: add dlx retry queue and service
* feat: add dlx retry queue and service
* feat: add dlx retry queue and service
* feat: add new organizations apiKey route
* chore: implement additional org routes
* chore: add API docs
* chore: checkin openapi spec
* chore: update tests
* feat: allow org api key management via admin api
* feat: allow the creation of projects using an org api key
* chore: allow deletion of projects via API
* feat: create outline of SCIM Users support
* feat: implement SCIM style user endpoints
* chore: allow set password on user
* chore: lint
* feat: add endpoints to manage project and organization memberships
* chore: add memberships api docs
* chore: reject project membership calls if entitlement is missing
* chore: add team plan to test org
* chore: add reference to full Langfuse API spec
* feat: add new organizations apiKey route
* chore: implement additional org routes
* chore: add API docs
* chore: checkin openapi spec
* chore: update tests
* feat: allow org api key management via admin api
* feat: allow the creation of projects using an org api key
* chore: allow deletion of projects via API
* feat: create outline of SCIM Users support
* feat: implement SCIM style user endpoints
* chore: allow set password on user
* chore: lint
* feat: add endpoints to manage project and organization memberships
* chore: add memberships api docs
* chore: reject project membership calls if entitlement is missing
* chore: add team plan to test org
* feat: add new organizations apiKey route
* chore: implement additional org routes
* chore: add API docs
* chore: checkin openapi spec
* chore: update tests
* feat: allow org api key management via admin api
* feat: allow the creation of projects using an org api key
* chore: allow deletion of projects via API
* feat: create outline of SCIM Users support
* feat: implement SCIM style user endpoints
* chore: allow set password on user
* chore: lint
* chore: add scim api docs
* chore: remove outdated generated artifacts
* feat: add new organizations apiKey route
* chore: implement additional org routes
* chore: add API docs
* chore: checkin openapi spec
* chore: update tests
* feat: allow org api key management via admin api
* feat: allow the creation of projects using an org api key
* chore: allow deletion of projects via API
* chore: add documentation for new project routes
* chore: cleanup
* chore: error updates and handling
* chore: reduce reduntant error handling
* chore(evals): fall back to bullmq `job.data.timestamp` allowing for successful retries
* chore: test
* chore: test
* docs: add diagram and docs
* chore: typo
* chore: typo
* chore: refactor authentication for admin apis
* chore: lint
* chore: address feedback
* feat: add post organizations endpoint
* chore: update tests
* chore: update api key data model
* chore: align code with new scoped api keys
* chore: typing
* chore: align tests with new logic
* chore: remove ratelimitoverrides
* chore: remove accesslevel from queue message
* chore: add additional check to ingestion endpoint
* chore: add test for legacy api key format in redis
* chore: map all api keys in auditlog
* added environment filter option to traces
* WIP on multi-select ui inconsistency
* fixed overflow issue in multi select where an item would not take the full width automatically; also changed to overflow-x-auto
* fixed invalid DOM nesting of div within p (FormDescription) that resulted in a hydration error
* fix multiselect to cut overflowing text with ellipsis and take full name as titlte so you can hover over it, removed unnecessary css classes
* WIP on create evaluator page ui issue with filter builder
* Revert "WIP on create evaluator page ui issue with filter builder"
This reverts commit fd6dd507f2df3c5fb404f4e38a2983e8ef6405ef.
* hide empty string from multi-select options, as supporting empty values would require further work - out of scope for now; also checked for side effects, with current usage all good
* Added test suite for eval filtering
* Resolved comment: removed getEnvironmentsForProject from traces router and instead fetched environments inside evaluator-form via api.projects.environmentFilterOptions
* move eval filtering tests into separate test file
* removed redundancy via fixtures, prepared tests for concurrency
* filtering tests run concurrently
* extracted traceFilterOptions fetching into separate hook as it is used in both the evaluator form and the traces table, also fixed traces table since the environment was now missing there
* useTraceFilterOptions Hook after merge
* Make TraceOptions keys partial so that we can update filter options asynchronously
* default to empty object if response undefined
* exclude environment filter builder column from traces table since toolbar already has env
* removed useTraceFilterOptions hook as use case changed such that extraction into hook is no longer warranted
* chore: pin dev clickhouse version to 24.3
* chore: only add input format flag for cloud
* chore: whitelist accepted cloud regions
* chore: disable test
* fix: (workaround) evals page overflows when rendering right hand side metadata bar
This is more of a workaround, we should probably disable the overflow on the main wrapper div in layout.tsx and enable it only where needed - or find out why the overflow inside the metadata sidebar trigggers the scrollbar to pop up - some info on what I found:
- The body and other divs close to the root do not increase their size.
- The navigation sidebar does increase its vertical height, but limiting that does not resolve the main scrollbar.
- Limiting specifically the wrapper div in layout.tsx with overflow-hidden does resolve this, but globally disabling it requires most pages to be refactored.
* Revert "fix: (workaround) evals page overflows when rendering right hand side metadata bar"
This reverts commit 313a1c0d8a885a04974399f850f7f9aee58fd714.
* fix(ui): TableWithMetadataWrapper uses contain-layout on its root div so that EvaluatorDetail overflow issue is mitigated
Checked for side effects on the other components using TableWithMetadataWrapper and all work as expected.
---------
Co-authored-by: Marlies Mayerhofer <74332854+marliessophie@users.noreply.github.com>
* chore(annotation-scores): revert to synchronous delete; upsert on create route
* fix: ensure scores data not only marked as stale but refreshed on score update
* feat: add support for GCS buckets
* chore: cleanup
* chore: trim gcs credentials before parsing
* chore: add logging for redis shutdown in tests
* chore: update timeout for trace delete test
* chore: bump trace delete concurrency for tests
* feat(evals): enable deletion of evaluators and templates via UI
* Added tests for the evals job and template deletion trpc endpoints
* evals-trpc test refactor, it had a side effect on other tests since it was pruning the db, now instead creates a new org and project per tests and cleans them up after all tests are done
* removed test comments as those are now implemented
* fix wrong wording
* added template version to delete confirmation on template details delete action
* DeleteButton refactoring and moved deletion of templates to LFE-4573
* moving the call to captureDeleteSuccess inside the successful branch of executeDeleteMutation
* Display lock icon when action button in icon mode is unauthorized
* fix: quietly delete scheduled evals for deleted job executions
---------
Co-authored-by: Max Deichmann <m.deichmann@tum.de>
* fix(trace-timeline): overflow and scroll behaviour
* fix(trace-timeline): show maximum of 3 scores and remainder in hover card
* push
* chore: refactor
* chore
* chore
* feat: add self-serve dashboard backend poc
* chore: data model thoughts
* chore: create initial query builder example and test case
* chore: handle empty dimensions and metrics
* chore: lint
* chore: extend filter conditions to account for timestamps
* chore: refactor query builder into sub-functions
* chore: use template queries for user-supplied values
* chore: add query builder tests
* chore: handle time dimension
* chore: add trpc endpoint to execute custom clickhouse query
* chore: add a query playground component to test a couple of queries
* chore: handle multiple joins
* chore: add observations table to data model
* chore: fix bug in data model
* chore: add test case to compare with old dashboard results
* chore: add scaffold for additional views
* expand the users view and add segments to filter scores subviews
* chore: add test cases for score views
* chore: drop users and sessions for now
* chore: add scores aggregate and observations cost tests
* chore: add sql injection tests
* chore: lint
* chore: fix dashboard test cases
* chore: drop users query test
* chore: add order by logic
* chore: fill timeseries values
* chore: use new query function on dashboard
* chore: convert TracesBarListChart.tsx to new query endpoint
* chore: make chart data compile
* chore: update tests
* chore: add tags to custom queries
* chore: typing
* chore: suffix join condition with sql
* chore: add typing in queryBuidler
* chore: limit playground to cloud admin users
* chore: add util to map legacy dashboard columns to new model
* chore: separate time filter state for TracesBarListChart.tsx
* chore: pass timestamps directly into chart components
* chore: add userId and sessionId on observations
* chore: pick auto time granularity based on hours
* fix(prompt-experiments): show form error if name is duplicated
* chore: fix typo
* Update web/src/ee/features/experiments/hooks/useExperimentNameValidation.tsx
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* eslint
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat(evals): preview number of historic items to be evaluated
* refactor: split up large evaluator form file
* chore: drop optional pagination
* chore: fix eslint
* chore: rename test
* fix: safely parse big numbers in api responses
* feat: add additional input/output parsing for pydantic via OTel
* Revert "feat: add additional input/output parsing for pydantic via OTel"
This reverts commit 71d0f3abd951702393aac7ab79a875b888f089cf.
* pin node to problematic version for working on a fix
* added cross-env and disabled the feature flag causing the testing issue - long term fix should probably be switching to vitest
* unpin node version since problematic feature is now disabled fo affected tests
* fix(ui): IOPreview now renders messages where content is an empty string instead of ignoring them
* removed debug logs
* OpenAiMessageView first filters valid messages to render and then executes the "show x more ..." logic, this way we ensure that there can never be a mismatch between messages items and the amount of fields we support displaying
* Render empty string quotation marks if content is empty but not null or undefined
* Change length check to falsy check for conditional display of empty quotes if content is empty
* cleaner falsy check for rendering empty string quotation marks
* fix(ui): stop propagation on traces table checkbox
* fix(peek-ui): handle onRowClick action correctly for peek view
* feat(peek): do not close upon any screen button or checkbox clicks
* push
* wip
* simplify
* actionbutton for upgrade prompt
* count events for free plan limit
* add sso settings
* rename lite to core
* add product id
* nit
* nits
* fix rate limit test
* ci run on teams plan
* fix
* fix names of pipeline steps
* test tmp
* add better error logs to makeAPICall
* push
* clone to throw better errors
* improve error
* improve error
* fix
* remove comment
* refactor
* push
* next attempt
* nit
* create new auth for failing test suites
* push
* fix rate limit test
* feat: add media and export external endpoint config
* chore: add docs string
* chore: add external endpoint support for azure
* chore: drop feature for media
* Unified design of grouped score badge with trace detail score badges, except that text is no longer medium/semibold
* Made padding of root item of ObservationTree consistent with other items
* fixed alignment issues of badges wrapped in links in trace and observation detail views
* Fixed tooltip of aggregated usage data badge (in trace details) rendering even if there is no underlying data
* chore(cloud): Make UI recognize cloud:enterprise plan and add corresponding access rights and rate limits
- label on project selector
- rate limits same as team
- access rights same as team
- no stripe product id for this (yet), cannot be purchased self-serve in product
* remove comments
---------
Co-authored-by: Marc Klingen <git@marcklingen.com>
* feat(auth): add workos as idp
* add button for org and connection based sign in
* fix
* remove client auth method and checks
* remove unnecessary diff
* fix: header
* fix: search always show current version
* chore: adjust code block
* chore: re-add copy in code editor if no title
* chore: do not match current version always
* adjust margins
* chore: remove breadcrumb
* fix: typo
* fix: final margin
* style: eval template
* small design nits
* fix: do not allow to deactivate eval configs which rano on historic traces only
* fix: do not allow to deactivate eval configs which rano on historic traces only
* fix: do not allow to deactivate eval configs which rano on historic traces only
* fix: do not allow to deactivate eval configs which rano on historic traces only
* fix: do not allow to deactivate eval configs which rano on historic traces only
* fix: do not return all executions for eval config list trpc
* fix: do not return all executions for eval config list trpc
* fix: do not return all executions for eval config list trpc
* fix: do not return all executions for eval config list trpc
* chore: fix demo banner
* fix(page-header): info icon relative positioning
* fix: header title wrap behaviour
* fix(page-header): have info icon follow title
* style: move dataset level buttons to more menu
* push
* push
* abstract editor to support "text" mode in addition to "json" mode
* always use CodeMirrorEditor for prompts
* minheight on editor
* upgrade codemirror and add language package
* add linting for prompts and switch to githubDark theme
* package
* use codemirror for chat prompts
* imrpove scroll behavior of new dataset form
* improve dataset item editor view
* push
* remove dep pinning
* align editor linting with current variable definition
* defauklt value to value
* rename existing command exports
* move existing to input-command
* add comment
* wip cmd k menu for navigation
* fix types and paths of nested menu items
* add support for nested menu items
* filter correctly
* feat(prompts): support in UI prompt duplication
* chore(ui-ds): make `children` in `ActionButton` optional
* fix: extract custom hook for prompt name validation
* eslint
* fix: insert correct prompt count
* chore: validate prompt name uniqueness; attach latest label for single version
* chore: refactor duplication approach to use createMany and save on network-roundtrips
* chore: revert changes on action button
* push
# Snyk cannot upload results in merge group. Hence, we only run on PRs and when pushingon the main branch https://github.com/github/codeql-action/issues/1572
concurrency:
group:${{ github.workflow }}-${{ github.ref }}
@@ -29,6 +29,13 @@ jobs:
image:langfuse-langfuse-web
args:--file=web/Dockerfile
# Workaround for https://github.com/github/codeql-action/issues/2187
- name:Replace security-severity undefined for license-related findings
Langfuse is an **open source LLM engineering** platform for developing, monitoring, evaluating and debugging AI applications. See the README for more details.
## Linting
- Run `pnpm run lint` to lint all packages.
- Fix issues automatically with `pnpm run lint:fix`.
## Tests
- Codex cannot run the test suite because it depends on Docker-based infrastructure that is unavailable in this environment.
## Cursor Rules
- Additional folder-specific rules live in `.cursor/rules/`.
## Commits
- Follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) when crafting commit messages.
@@ -27,7 +27,7 @@ The maintainers are available on [Discord](https://langfuse.com/discord) in case
## Making a change
_Before making any significant changes, please [open an issue](https://github.com/langfuse/langfuse/issues)._ Discussing your proposed changes ahead of time will make the contribution process smooth for everyone. Large changes that were not discussed in an issue may be rejected.
_Before making any significant changes, please [open an issue](https://github.com/langfuse/langfuse/issues)._ Discussing your proposed changes ahead of time will make the contribution process smooth for everyone. Changes that were not discussed in an issue may be rejected.
Once we've discussed your changes and you've got your code ready, make sure that tests are passing and open your pull request.
@@ -53,117 +53,10 @@ A good first step is to search for open [issues](https://github.com/langfuse/lan
See this [diagram](https://langfuse.com/self-hosting#architecture) for an overview of the architecture.
### Network Overview
> [!NOTE]
> This will change in Langfuse version 3.0. More in the [GitHub Discussions](https://github.com/orgs/langfuse/discussions/1902).
```mermaid
flowchart LR
Browser ---|Web UI & TRPC API| App
@@ -190,7 +83,7 @@ Full database schema: [packages/shared/prisma/schema.prisma](packages/shared/pri
We built a monorepo using [pnpm](https://pnpm.io/motivation) and [turbo](https://turbo.build/repo/docs) to manage the dependencies and build process. The monorepo contains the following packages:
-`web`: is the main application package providing Frontend and Backend APIs for Langfuse.
-`worker` (no production yet): contains an application for asynchronous processing of tasks. This package is not yet used in production.
-`worker`: contains an application for asynchronous processing of tasks.
-`packages`:
-`shared`: contains shared code between the above packages.
-`config-eslint`: contains eslint configurations which are shared between the above packages.
@@ -209,11 +102,12 @@ Requirements
**Steps**
1.Fork the repository and clone it locally
2.Run the development database
1.Install [golang-migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate#migrate-cli) as CLI
앱에 계측(instrumentation)을 추가하여 Langfuse로 trace 데이터를 수집함으로써, 검색, 임베딩, 또는 에이전트 동작과 같은 LLM 호출 및 기타 관련 로직을 추적할 수 있습니다. 복잡한 로그와 사용자 세션을 확인 및 디버깅 해보세요. 인터랙티브 데모를 통해 실제 작동 예를 확인할 수 있습니다.
- **프롬프트 관리**
프롬프트를 중앙에서 관리하고 버전 관리하며 협업으로 수정할 수 있도록 도와줍니다. 서버와 클라이언트 측의 강력한 캐싱 덕분에 애플리케이션에 지연(latency)을 추가하지 않고도 프롬프트를 반복 개선할 수 있습니다.
- **평가**
LLM 애플리케이션 개발 워크플로우에서 핵심적인 역할을 하며, Langfuse는 여러분의 필요에 맞게 유연하게 대응합니다. LLM을 심사자로 활용하는 기능, 사용자 피드백 수집, 수동 라벨링 및 API/SDK를 통한 맞춤 평가 파이프라인을 지원합니다.
- **데이터셋**
LLM 애플리케이션 평가를 위한 테스트 세트와 벤치마크를 제공하여, 지속적인 개선, 배포 전 테스트, 구조화된 실험, 유연한 평가 및 LangChain과 LlamaIndex와 같은 프레임워크와의 원활한 통합을 지원합니다.
- **LLM 플레이그라운드**
프롬프트와 모델 구성에 대해 테스트 및 반복 개선할 수 있는 도구로, 피드백 루프를 단축하여 개발 속도를 높여줍니다. trace에서 이상한 결과가 발생하면 플레이그라운드로 바로 이동해 개선할 수 있습니다.
- **종합 API**
Langfuse는 API를 통해 제공되는 구성 요소들을 활용하여 맞춤형 LLMOps 워크플로우를 강화하는 데 자주 사용됩니다. OpenAPI 명세, Postman 컬렉션, Python 및 JS/TS용 타입드 SDK가 제공됩니다.
| [LlamaIndex](https://langfuse.com/docs/integrations/llama-index/get-started) | Python | LlamaIndex 콜백 시스템을 통한 자동 계측을 지원합니다. |
| [Haystack](https://langfuse.com/docs/integrations/haystack) | Python | Haystack 콘텐츠 추적 시스템을 통한 자동 계측을 지원합니다. |
| [LiteLLM](https://langfuse.com/docs/integrations/litellm) | Python, JS/TS (proxy only) | GPT의 드롭인 대체품으로 어떤 LLM도 사용할 수 있습니다. Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate 등 100개 이상의 LLM 지원. |
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | React, Next.js, Vue, Svelte, Node.js와 함께 AI 기반 애플리케이션 구축을 돕는 TypeScript 툴킷입니다. |
| [API](https://langfuse.com/docs/api) | | 공개 API를 직접 호출합니다. OpenAPI 명세가 제공됩니다. |
| [Ollama](https://langfuse.com/docs/integrations/ollama) | 모델 (로컬) | 자신의 컴퓨터에서 오픈 소스 LLM을 손쉽게 실행할 수 있습니다. |
| [Amazon Bedrock](https://langfuse.com/docs/integrations/amazon-bedrock) | 모델 | AWS에서 기본 및 파인튜닝된 모델을 실행합니다. |
| [AutoGen](https://langfuse.com/docs/integrations/autogen) | 에이전트 프레임워크 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [Flowise](https://langfuse.com/docs/integrations/flowise) | 채팅/에이전트 UI | 맞춤형 LLM 플로우를 위한 JS/TS 코드 없는(no-code) 빌더입니다. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | 채팅/에이전트 UI | react-flow를 활용하여 실험 및 프로토타이핑을 손쉽게 할 수 있도록 디자인된 LangChain용 Python 기반 UI입니다. |
| [Dify](https://langfuse.com/docs/integrations/dify) | 채팅/에이전트 UI | 코드 없는 빌더와 함께 제공되는 오픈 소스 LLM 애플리케이션 개발 플랫폼입니다. |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | 채팅/에이전트 UI | 셀프 호스팅 및 로컬 모델 등 다양한 LLM 실행기를 지원하는 셀프 호스팅 LLM 채팅 웹 UI입니다. |
| [Promptfoo](https://langfuse.com/docs/integrations/promptfoo) | 도구 | 오픈 소스 LLM 테스트 플랫폼입니다. |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | 채팅/에이전트 UI | 오픈 소스 챗봇 플랫폼입니다. |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | 플랫폼 | 오픈 소스 음성 AI 플랫폼입니다. |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | 에이전트 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | 채팅/에이전트 UI | 채팅 UI와 같은 웹 인터페이스 구축을 위한 오픈 소스 Python 라이브러리입니다. |
| [Goose](https://langfuse.com/docs/integrations/goose) | 에이전트 | 분산 에이전트 구축을 위한 오픈 소스 LLM 플랫폼입니다. |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | 에이전트 | 오픈 소스 AI 에이전트 프레임워크입니다. |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | 에이전트 | 에이전트 간 협업 및 도구 사용을 위한 다중 에이전트 프레임워크입니다. |
## 🚀 빠른 시작
앱에 계측을 추가하고 Langfuse에 trace 데이터를 수집하여, LLM 호출 및 검색, 임베딩, 에이전트 동작과 같은 애플리케이션 로직을 추적해보세요. 복잡한 로그와 사용자 세션을 확인하여 디버깅할 수 있습니다.
### 1️⃣ 새 프로젝트 생성
1. [Langfuse 계정 생성](https://cloud.langfuse.com/auth/sign-up) 또는 [셀프 호스트](https://langfuse.com/self-hosting)
2. 새 프로젝트를 생성합니다.
3. 프로젝트 설정에서 새로운 API 자격 증명을 생성합니다.
### 2️⃣ 첫 번째 LLM 호출 기록하기
[`@observe()` 데코레이터](https://langfuse.com/docs/sdk/python/decorators)를 사용하면 Python LLM 애플리케이션의 추적이 매우 간편해집니다. 이 빠른 시작 예제에서는 Langfuse [OpenAI 통합](https://langfuse.com/docs/integrations/openai)을 사용하여 모든 모델 파라미터를 자동으로 캡처합니다.
> [!TIP]
> OpenAI를 사용하지 않으시다면, 다른 모델 및 프레임워크의 로그 기록 방법은 [문서](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse)를 참조하세요.
```bash
pip install langfuse openai
```
```bash filename=".env"
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EU region
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 US region
```
```python:main.py
from langfuse import observe
from langfuse.openai import openai # OpenAI integration
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
```
### 3️⃣ Langfuse에서 trace 확인하기
Langfuse에서 LLM 호출 및 애플리케이션의 기타 로직에 대한 trace를 확인할 수 있습니다.

_[Langfuse의 공개 예제 trace](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!TIP]
>
> Langfuse의 trace에 대해 더 알아보거나 [인터랙티브 데모](https://langfuse.com/docs/demo)에서 직접 체험해보세요.
- 우리의 [문서](https://langfuse.com/docs)는 답을 찾기 위한 최적의 장소입니다. 문서가 매우 포괄적이며, 유지보수에 많은 노력을 기울이고 있습니다. GitHub를 통해 문서 수정 제안도 가능합니다.
- [Langfuse FAQ](https://langfuse.com/faq)에서는 가장 흔한 질문에 대해 답변하고 있습니다.
- 질문에 즉각적인 답변이 필요하다면 [Ask AI](https://langfuse.com/docs/ask-ai)를 사용해보세요.
지원 채널:
- **GitHub Discussions의 [공개 Q&A](https://github.com/orgs/langfuse/discussions/categories/support)** 에 질문을 남겨주세요. 가능한 한 많은 세부 사항(예: 코드 스니펫, 스크린샷, 배경 정보)을 포함해 질문해 주시기 바랍니다.
<span>Langfuse uses <a href="https://github.com/orgs/langfuse/discussions"><strong>Github Discussions</strong></a> for Support and Feature Requests.</span>
<br/>
<span>We're hiring. <a href="https://langfuse.com/careers"><strong>Join us</strong></a> in Product Engineering and Developer Relations.</span>
<span><b>We're hiring.</b> <a href="https://langfuse.com/careers"><strong>Join us</strong></a> in product engineering and technical go-to-market roles.</span>
- **LLM Observability:** Instrument your app and start ingesting traces to Langfuse ([Quickstart](https://langfuse.com/docs/get-started), [Integrations](https://langfuse.com/docs/integrations) [Tracing](https://langfuse.com/docs/tracing))
- **Langfuse UI:** Inspect and debug complex logs ([Demo](https://langfuse.com/docs/demo), [Tracing](https://langfuse.com/docs/tracing))
- **Prompt Management:** Manage, version and deploy prompts from within Langfuse ([Prompt Management](https://langfuse.com/docs/prompts/get-started))
- **Prompt Engineering:** Test and iterate on your prompts with the [LLM Playground](https://langfuse.com/docs/playground)
- [LLM Application Observability](https://langfuse.com/docs/tracing): Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions. Try the interactive [demo](https://langfuse.com/docs/demo) to see this in action.
-**LLM Analytics:** Track metrics (cost, latency, quality) and gain insights from dashboards & data exports ([Analytics](https://langfuse.com/docs/analytics))
- **LLM Evaluations:** Collect and calculate scores for your LLM completions ([Scores & Evaluations](https://langfuse.com/docs/scores))
- Run ([Model-based evaluations](https://langfuse.com/docs/scores/model-based-evals)) and LLM-as-a-Judge within Langfuse
- Collect user feedback ([User Feedback](https://langfuse.com/docs/scores/user-feedback))
- Manually score LLM outputs in Langfuse ([Manual Scores](https://langfuse.com/docs/scores/manually))
-[Prompt Management](https://langfuse.com/docs/prompts/get-started) helps you centrally manage, version control, and collaboratively iterate on your prompts. Thanks to strong caching on server and client side, you can iterate on prompts without adding latency to your application.
### Test
- [Evaluations](https://langfuse.com/docs/scores/overview) are key to the LLM application development workflow, and Langfuse adapts to your needs. It supports LLM-as-a-judge, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs.
-**Experiments:** Track and test app behaviour before deploying a new version
- Datasets let you test expected in and output pairs and benchmark performance before deploying ([Datasets](https://langfuse.com/docs/datasets))
- Track versions and releases in your application ([Experimentation](https://langfuse.com/docs/experimentation), [Prompt Management](https://langfuse.com/docs/prompts))
-[Datasets](https://langfuse.com/docs/datasets/overview) enable test sets and benchmarks for evaluating your LLM application. They support continuous improvement, pre-deployment testing, structured experiments, flexible evaluation, and seamless integration with frameworks like LangChain and LlamaIndex.
## Get started
- [LLM Playground](https://langfuse.com/docs/playground) is a tool for testing and iterating on your prompts and model configurations, shortening the feedback loop and accelerating development. When you see a bad result in tracing, you can directly jump to the playground to iterate on it.
- [Comprehensive API](https://langfuse.com/docs/api): Langfuse is frequently used to power bespoke LLMOps workflows while using the building blocks provided by Langfuse via the API. OpenAPI spec, Postman collection, and typed SDKs for Python, JS/TS are available.
[→ Learn more about deploying locally](https://langfuse.com/docs/deployment/local)
# Run the langfuse docker compose
docker compose up
```
- [VM](https://langfuse.com/self-hosting/docker-compose): Run Langfuse on a single Virtual Machine using Docker Compose.
- [Kubernetes (Helm)](https://langfuse.com/self-hosting/kubernetes-helm): Run Langfuse on a Kubernetes cluster using Helm. This is the preferred production deployment.
You need a Langfuse public and secret key to get started. Sign up [here](https://cloud.langfuse.com) and find them in your project settings.
### Ingesting Data · Instrumenting Your Application · LLM Observability with Langfuse
Note: We recommend using our fully async, typed [SDKs](https://langfuse.com/docs/sdk) that allow you to instrument any LLM application with any underlying model. They are available in [Python (Decorators)](https://langfuse.com/docs/sdk/python) & [JS/TS](https://langfuse.com/docs/sdk/typescript). The SDKs will always be the most fully featured and stable way to ingest data into Langfuse.
See the [→ Quickstart](https://langfuse.com/docs/get-started) to integrate Langfuse.
@@ -120,77 +138,129 @@ See the [→ Quickstart](https://langfuse.com/docs/get-started) to integrate Lan
| [Vercel AI SDK](https://langfuse.com/docs/integrations/vercel-ai-sdk) | JS/TS | TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js. |
| [API](https://langfuse.com/docs/api) | | Directly call the public API. OpenAPI spec available. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. |
| [Langflow](https://langfuse.com/docs/integrations/langflow) | Chat/Agent UI | Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows. |
| [Dify](https://langfuse.com/docs/integrations/dify) | Chat/Agent UI | Open source LLM app development platform with no-code builder. |
| [OpenWebUI](https://langfuse.com/docs/integrations/openwebui) | Chat/Agent UI | Self-hosted LLM Chat web ui supporting various LLM runners including self-hosted and local models. |
| [LobeChat](https://langfuse.com/docs/integrations/lobechat) | Chat/Agent UI | Open source chatbot platform. |
| [Vapi](https://langfuse.com/docs/integrations/vapi) | Platform | Open source voice AI platform. |
| [Inferable](https://langfuse.com/docs/integrations/other/inferable) | Agents | Open source LLM platform for building distributed agents. |
| [Gradio](https://langfuse.com/docs/integrations/other/gradio) | Chat/Agent UI | Open source Python library to build web interfaces like Chat UI. |
| [Goose](https://langfuse.com/docs/integrations/goose) | Agents | Open source LLM platform for building distributed agents. |
| [smolagents](https://langfuse.com/docs/integrations/smolagents) | Agents | Open source AI agents framework. |
| [CrewAI](https://langfuse.com/docs/integrations/crewai) | Agents | Multi agent framework for agent collaboration and tool use. |
## Questions and feedback
## 🚀 Quickstart
### Ideas and roadmap
Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions.
- [Discord](https://langfuse.com/discord): community support
- Privately: contact at langfuse dot com
The [`@observe()` decorator](https://langfuse.com/docs/sdk/python/decorators) makes it easy to trace any Python LLM application. In this quickstart we also use the Langfuse [OpenAI integration](https://langfuse.com/docs/integrations/openai) to automatically capture all model parameters.
## Contributing to Langfuse
> [!TIP]
> Not using OpenAI? Visit [our documentation](https://langfuse.com/docs/get-started#log-your-first-llm-call-to-langfuse) to learn how to log other models and frameworks.
- Vote on [Ideas](https://github.com/orgs/langfuse/discussions/categories/ideas)
- Raise and comment on [Issues](https://github.com/langfuse/langfuse/issues)
```bash
pip install langfuse openai
```
```bash filename=".env"
LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EU region
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 US region
from langfuse.openai import openai # OpenAI integration
@observe()
def story():
return openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Langfuse?"}],
).choices[0].message.content
@observe()
def main():
return story()
main()
```
### 3️⃣ See traces in Langfuse
See your language model calls and other application logic in Langfuse.

_[Public example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/2cec01e3-3dc2-472f-afcf-3b968cf0c1f4?timestamp=2025-02-10T14%3A27%3A30.275Z&observation=cb5ff844-07ef-41e6-b8e2-6c64344bc13b)_
> [!TIP]
>
> [Learn more](https://langfuse.com/docs/tracing) about tracing in Langfuse or play with the [interactive demo](https://langfuse.com/docs/demo).
- Our [documentation](https://langfuse.com/docs) is the best place to start looking for answers. It is comprehensive, and we invest significant time into maintaining it. You can also suggest edits to the docs via GitHub.
- [Langfuse FAQs](https://langfuse.com/faq) where the most common questions are answered.
- Use "[Ask AI](https://langfuse.com/docs/ask-ai)" to get instant answers to your questions.
Support Channels:
- **Ask any question in our [public Q&A](https://github.com/orgs/langfuse/discussions/categories/support) on GitHub Discussions.** Please include as much detail as possible (e.g. code snippets, screenshots, background information) to help us understand your question.
- [Request a feature](https://github.com/orgs/langfuse/discussions/categories/ideas) on GitHub Discussions.
- [Report a Bug](https://github.com/langfuse/langfuse/issues) on GitHub Issues.
- For time-sensitive queries, ping us via the in-app chat widget.
## 🤝 Contributing
Your contributions are welcome!
- Vote on [Ideas](https://github.com/orgs/langfuse/discussions/categories/ideas) in GitHub Discussions.
- Raise and comment on [Issues](https://github.com/langfuse/langfuse/issues).
- Open a PR - see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to setup a development environment.
## License
## 🥇 License
This repository is MIT licensed, except for the `ee` folders. See [LICENSE](LICENSE) and [docs](https://langfuse.com/docs/open-source) for more details.
## Misc
### GET API to export your data
[**GET routes**](https://langfuse.com/docs/integrations/api) to use data in downstream applications (e.g. embedded analytics). You can also access them conveniently via the SDKs ([docs](https://langfuse.com/docs/query-traces)).
### Security & Privacy
We take data security and privacy seriously. Please refer to our [Security and Privacy](https://langfuse.com/security) page for more information.
### Telemetry
By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).
This helps us to:
1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.
None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect [here](/web/src/features/telemetry/index.ts).
You can opt-out by setting `TELEMETRY_ENABLED=false`.
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=langfuse/langfuse&type=Date" style="border-radius: 15px;" />
</picture>
</a>
### Open Source Projects Using Langfuse
## ❤️ Open Source Projects Using Langfuse
Top open-source Python projects that use Langfuse, ranked by stars ([Source](https://github.com/langfuse/langfuse-docs/blob/main/components-mdx/dependents)):
@@ -240,3 +310,20 @@ Top open-source Python projects that use Langfuse, ranked by stars ([Source](htt
We take data security and privacy seriously. Please refer to our [Security and Privacy](https://langfuse.com/security) page for more information.
### Telemetry
By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).
This helps us to:
1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.
None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect [here](/web/src/features/telemetry/index.ts).
You can opt-out by setting `TELEMETRY_ENABLED=false`.
Join us in building out Langfuse in Berlin, Germany. Langfuse is the open source LLM engineering platform: we build tooling to help developers [build & improve LLM applications](https://langfuse.com/docs).
We are an open source company, we hire in person (4+ days a week), we only hire excellent technical talent. Find more information on our [careers page](https://langfuse.com/careers)
docs:The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId:optional<string>
comment:optional<string>
metadata:optional<unknown>
dataType:
type:optional<ScoreDataType>
docs:When set, must match the score value's type. If not set, will be inferred from the score value or config
@@ -64,62 +67,15 @@ types:
dataType:"BOOLEAN"
configId:"1234-5678-90ab-cdef"
traceId:"cdef-1234-5678-90ab"
BaseScore:
properties:
id:string
traceId:string
name:string
source:ScoreSource
observationId:optional<string>
timestamp:datetime
createdAt:datetime
updatedAt:datetime
authorUserId:optional<string>
comment:optional<string>
configId:
type:optional<string>
docs:Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
NumericScore:
extends:BaseScore
properties:
value:
type:double
docs:The numeric value of the score
BooleanScore:
extends:BaseScore
properties:
value:
type:double
docs:The numeric value of the score. Equals 1 for "True" and 0 for "False"
stringValue:
type:string
docs:The string representation of the score value. Is inferred from the numeric value and equals "True" or "False"
CategoricalScore:
extends:BaseScore
properties:
value:
type:optional<double>
docs:Only defined if a config is linked. Represents the numeric category mapping of the stringValue
stringValue:
type:string
docs:The string representation of the score value. If no config is linked, can be any string. Otherwise, must map to a config category
docs:Public traces are accessible via url without login
environment:
type:optional<string>
docs:The environment from which this trace originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
TraceWithDetails:# GET /traces
extends:Trace
properties:
@@ -72,13 +75,16 @@ types:
type:list<ObservationsView>
docs:List of observations
scores:
type:list<Score>
type:list<ScoreV1>
docs:List of scores
Session:
properties:
id:string
createdAt:datetime
projectId:string
environment:
type:optional<string>
docs:The environment from which this session originated.
SessionWithTraces:
extends:Session
properties:
@@ -145,6 +151,9 @@ types:
costDetails:
type:optional<map<string, double>>
docs:The cost details of the observation. Key is the name of the cost metric, value is the cost in USD. The total key is the sum of all (non-total) cost metrics or the total value ingested.
environment:
type:optional<string>
docs:The environment from which this observation originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
ObservationsView:
extends:Observation
@@ -231,7 +240,7 @@ types:
properties:
value:double
label:string
BaseScore:
BaseScoreV1:
properties:
id:string
traceId:string
@@ -243,12 +252,77 @@ types:
updatedAt:datetime
authorUserId:optional<string>
comment:optional<string>
metadata:optional<unknown>
configId:
type:optional<string>
docs:Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
queueId:
type:optional<string>
docs:Reference an annotation queue on a score. Populated if the score was initially created in an annotation queue.
environment:
type:optional<string>
docs:The environment from which this score originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
NumericScoreV1:
extends:BaseScoreV1
properties:
value:
type:double
docs:The numeric value of the score
BooleanScoreV1:
extends:BaseScoreV1
properties:
value:
type:double
docs:The numeric value of the score. Equals 1 for "True" and 0 for "False"
stringValue:
type:string
docs:The string representation of the score value. Is inferred from the numeric value and equals "True" or "False"
CategoricalScoreV1:
extends:BaseScoreV1
properties:
value:
type:optional<double>
docs:Only defined if a config is linked. Represents the numeric category mapping of the stringValue
stringValue:
type:string
docs:The string representation of the score value. If no config is linked, can be any string. Otherwise, must map to a config category
ScoreV1:
discriminant:"dataType"
union:
NUMERIC:
type:NumericScoreV1
docs:"Score with NUMERIC data type"
CATEGORICAL:
type:CategoricalScoreV1
docs:"Score with CATEGORICAL data type"
BOOLEAN:
type:BooleanScoreV1
docs:"Score with BOOLEAN data type"
BaseScore:
properties:
id:string
traceId:optional<string>
sessionId:optional<string>
observationId:optional<string>
datasetRunId:optional<string>
name:string
source:ScoreSource
timestamp:datetime
createdAt:datetime
updatedAt:datetime
authorUserId:optional<string>
comment:optional<string>
metadata:optional<unknown>
configId:
type:optional<string>
docs:Reference a score config on a score. When set, config and score name must be equal and value must comply to optionally defined numerical range
queueId:
type:optional<string>
docs:Reference an annotation queue on a score. Populated if the score was initially created in an annotation queue.
environment:
type:optional<string>
docs:The environment from which this score originated. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
NumericScore:
extends:BaseScore
properties:
@@ -383,13 +457,13 @@ types:
docs:Unit used by this model.
type:optional<ModelUsageUnit>
inputPrice:
docs:Price (USD) per input unit
docs:Deprecated. See 'prices' instead. Price (USD) per input unit
type:optional<double>
outputPrice:
docs:Price (USD) per output unit
docs:Deprecated. See 'prices' instead. Price (USD) per output unit
type:optional<double>
totalPrice:
docs:Price (USD) per total unit. Cannot be set if input or output price is set.
docs:Deprecated. See 'prices' instead. Price (USD) per total unit. Cannot be set if input or output price is set.
type:optional<double>
tokenizerId:
docs:Optional. Tokenizer to be applied to observations which match to this model. See docs for more details.
Batched ingestion for Langfuse Tracing. If you want to use tracing via the API, such as to build your own Langfuse client implementation, this is the only API route you need to implement.
Batched ingestion for Langfuse Tracing.
If you want to use tracing via the API, such as to build your own Langfuse client implementation, this is the only API route you need to implement.
Within each batch, there can be multiple events.
Each event has a type, an id, a timestamp, metadata and a body.
Internally, we refer to this as the "event envelope" as it tells us something about the event but not the trace.
We use the event id within this envelope to deduplicate messages to avoid processing the same event twice, i.e. the event id should be unique per request.
The event.body.id is the ID of the actual trace and will be used for updates and will be visible within the Langfuse App.
I.e. if you want to update a trace, you'd use the same body id, but separate event IDs.
Notes:
- Introduction to data model: https://langfuse.com/docs/tracing-data-model
docs:The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId:optional<string>
comment:optional<string>
metadata:optional<unknown>
dataType:
type:optional<commons.ScoreDataType>
docs:When set, must match the score value's type. If not set, will be inferred from the score value or config
@@ -238,6 +317,14 @@ types:
dataType:"BOOLEAN"
configId:"1234-5678-90ab-cdef"
traceId:"cdef-1234-5678-90ab"
- value:
name:"contextrelevant"
value:"not relevant"
sessionId:"abyt-1234-5678-80ab"
- value:
name:"hallucination"
value:0
datasetRunId:"7891-5678-90ab-hijk"
BaseEvent:
properties:
@@ -318,16 +405,27 @@ types:
successes:list<IngestionSuccess>
errors:list<IngestionError>
OpenAIUsageSchema:
OpenAICompletionUsageSchema:
docs:OpenAI Usage schema from (Chat-)Completion APIs
docs:Get all memberships for the organization associated with the API key (requires organization-scoped API key)
method:GET
path:/organizations/memberships
response:MembershipsResponse
updateOrganizationMembership:
docs:Create or update a membership for the organization associated with the API key (requires organization-scoped API key)
method:PUT
path:/organizations/memberships
request:MembershipRequest
response:MembershipResponse
getProjectMemberships:
docs:Get all memberships for a specific project (requires organization-scoped API key)
method:GET
path:/projects/{projectId}/memberships
path-parameters:
projectId:string
response:MembershipsResponse
updateProjectMembership:
docs:Create or update a membership for a specific project (requires organization-scoped API key). The user must already be a member of the organization.
method:PUT
path:/projects/{projectId}/memberships
path-parameters:
projectId:string
request:MembershipRequest
response:MembershipResponse
getOrganizationProjects:
docs:Get all projects for the organization associated with the API key (requires organization-scoped API key)
docs:Get SCIM Service Provider Configuration (requires organization-scoped API key)
method:GET
path:/ServiceProviderConfig
response:ServiceProviderConfig
getResourceTypes:
docs:Get SCIM Resource Types (requires organization-scoped API key)
method:GET
path:/ResourceTypes
response:ResourceTypesResponse
getSchemas:
docs:Get SCIM Schemas (requires organization-scoped API key)
method:GET
path:/Schemas
response:SchemasResponse
listUsers:
docs:List users in the organization (requires organization-scoped API key)
method:GET
path:/Users
request:
name:ListUsersRequest
query-parameters:
filter:
docs:Filter expression (e.g. userName eq "value")
type:optional<string>
startIndex:
docs:1-based index of the first result to return (default 1)
type:optional<integer>
count:
docs:Maximum number of results to return (default 100)
type:optional<integer>
response:ScimUsersListResponse
createUser:
docs:Create a new user in the organization (requires organization-scoped API key)
method:POST
path:/Users
request:
name:CreateUserRequest
body:
properties:
userName:
docs:User's email address (required)
type:string
name:
docs:User's name information
type:ScimName
emails:
docs:User's email addresses
type:optional<list<ScimEmail>>
active:
docs:Whether the user is active
type:optional<boolean>
password:
docs:Initial password for the user
type:optional<string>
response:ScimUser
getUser:
docs:Get a specific user by ID (requires organization-scoped API key)
method:GET
path:/Users/{userId}
path-parameters:
userId:string
response:ScimUser
deleteUser:
docs:Remove a user from the organization (requires organization-scoped API key). Note that this only removes the user from the organization but does not delete the user entity itself.
docs:Create a score (supports both trace and session scores)
method:POST
path:/scores
request:CreateScoreRequest
response:CreateScoreResponse
get:
docs:Get a list of scores
method:GET
path:/scores
request:
name:GetScoresRequest
query-parameters:
page:
type:optional<integer>
docs:Page number, starts at 1.
limit:
type:optional<integer>
docs:Limit of items per page. If you encounter api issues due to too large page sizes, try to reduce the limit.
userId:
type:optional<string>
docs:Retrieve only scores with this userId associated to the trace.
name:
type:optional<string>
docs:Retrieve only scores with this name.
fromTimestamp:
type:optional<datetime>
docs:Optional filter to only include scores created on or after a certain datetime (ISO 8601)
toTimestamp:
type:optional<datetime>
docs:Optional filter to only include scores created before a certain datetime (ISO 8601)
source:
type:optional<commons.ScoreSource>
docs:Retrieve only scores from a specific source.
operator:
type:optional<string>
docs:Retrieve only scores with <operator> value.
value:
type:optional<double>
docs:Retrieve only scores with <operator> value.
scoreIds:
type:optional<string>
docs:Comma-separated list of score IDs to limit the results to.
configId:
type:optional<string>
docs:Retrieve only scores with a specific configId.
queueId:
type:optional<string>
docs:Retrieve only scores with a specific annotation queueId.
dataType:
type:optional<commons.ScoreDataType>
docs:Retrieve only scores with a specific dataType.
traceTags:
type:optional<list<string>>
allow-multiple:true
docs:Only scores linked to traces that include all of these tags will be returned.
response:GetScoresResponse
get-by-id:
docs:Get a score
method:GET
path:/scores/{scoreId}
path-parameters:
scoreId:
type:string
docs:The unique langfuse identifier of a score
response:commons.Score
delete:
docs:Delete a score
docs:Delete a score (supports both trace and session scores)
method:DELETE
path:/scores/{scoreId}
path-parameters:
@@ -84,13 +24,19 @@ types:
CreateScoreRequest:
properties:
id:optional<string>
traceId:string
traceId:optional<string>
sessionId:optional<string>
observationId:optional<string>
datasetRunId:optional<string>
name:string
value:
type:commons.CreateScoreValue
docs:The value of the score. Must be passed as string for categorical scores, and numeric for boolean and numeric scores. Boolean score values must equal either 1 or 0 (true or false)
observationId:optional<string>
comment:optional<string>
metadata:optional<unknown>
environment:
type:optional<string>
docs:The environment of the score. Can be any lowercase alphanumeric string with hyphens and underscores that does not start with 'langfuse'.
dataType:
type:optional<commons.ScoreDataType>
docs:The data type of the score. When passing a configId this field is inferred. Otherwise, this field must be passed or will default to numeric.
@@ -113,10 +59,12 @@ types:
dataType:"NUMERIC"
configId:"9203-4567-89ab-cdef"
traceId:"cdef-1234-5678-90ab"
environment:"test"
- value:
name:"toxicity"
value:"not toxic"
traceId:"cdef-1234-5678-90ab"
environment:"production"
- value:
name:"correctness"
value:"partially correct"
@@ -139,38 +87,3 @@ types:
id:
type:string
docs:The id of the created object in Langfuse
GetScoresResponseTraceData:
properties:
userId:
type:optional<string>
docs:The user ID associated with the trace referenced by score
tags:
type:optional<list<string>>
docs:A list of tags associated with the trace referenced by score
docs:Optional filter to only include sessions created before a certain datetime (ISO 8601)
environment:
type:optional<string>
allow-multiple:true
docs:Optional filter for sessions where the environment is one of the provided values.
response:PaginatedSessions
get:
docs:Get a session. Please note that `traces` on this endpoint are not paginated, if you plan to fetch large sessions, consider `GET /api/public/traces?sessionId=<sessionId>`
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.