mirror of
https://github.com/hanzoai/insights.git
synced 2026-11-03 00:52:32 +00:00
Mechanically re-derived: bin/debrand runs over a fresh import of posthog/posthog@fa1a9a60d4 (2026-08-06). The pipeline (.hanzo/workflows), the Dockerfiles and bin/docker* entrypoints, compose files, LLM.md and the squashed initial migration are restored from the shipped tree, same as the previous import. The prune's import-closure guard kept 21 products the new core now imports; dropping them is a refactor now, recorded rather than forced.
677 lines
36 KiB
YAML
677 lines
36 KiB
YAML
# MCP tool definition — tool entries are scaffolded from the OpenAPI schema.
|
|
# The tool list and operation IDs are kept in sync automatically:
|
|
# pnpm --filter=@hanzo/mcp run scaffold-yaml -- --sync-all
|
|
#
|
|
# To enable a tool, set enabled: true and add required scopes + annotations.
|
|
# All other fields (title, description, enrich_url, etc.) are yours to configure.
|
|
category: Replay vision
|
|
feature: replay_vision
|
|
url_prefix: /replay-vision
|
|
ui_apps: {}
|
|
tools:
|
|
vision-actions-create:
|
|
operation: vision_actions_create
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Create vision action
|
|
description: >
|
|
Create a vision action, an "and then…" automation over a scanner's observations. Two modes: `group_summary`
|
|
(a scheduled report synthesized from the observations matching `selection`; set the cadence via
|
|
`trigger_config` `{rrule, timezone}`, an iCal RRULE without DTSTART, e.g. `FREQ=DAILY`) and `alert`
|
|
(evaluated continuously on the scanner's sweep; requires `alert_config`: `every_match` notifies about each
|
|
new matching observation, `on_breach` needs a `threshold` and fires when the `metric` (`count`, or
|
|
`avg_score` for scorer scanners) crosses it over `window_days` of 1/3/7/14/30). `selection` narrows which
|
|
observations count (verdict/tags/score bounds). Deliver to Slack or a webhook via `delivery_config`, e.g.
|
|
`[{type: 'slack', integration_id, channel}]`. Find the workspace's `integration_id` with `integrations-list`
|
|
(filter kind=slack) and the channel ID with `integrations-channels-retrieve`; without a delivery target,
|
|
results only appear in-app. Names must be unique within the team.
|
|
exclude_params:
|
|
- id
|
|
- next_run_at
|
|
- last_run_at
|
|
- hog_flow_id
|
|
- created_at
|
|
- created_by
|
|
- updated_at
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-delete:
|
|
operation: vision_actions_destroy
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:write
|
|
annotations:
|
|
readOnly: false
|
|
destructive: true
|
|
idempotent: true
|
|
title: Delete vision action
|
|
description: >
|
|
Permanently delete a vision action by ID, along with its run history (past group summaries and alert
|
|
firings). Its delivery destinations stop firing. To stop an action without losing its history, set `enabled:
|
|
false` via `vision-actions-update` instead.
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-list:
|
|
operation: vision_actions_list
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
list: true
|
|
title: List vision actions
|
|
description: >
|
|
List the vision actions (the "and then…" automations that run over a scanner's observations) for the current
|
|
project. A vision action in `mode: group_summary` produces the group summary shown on the "Summaries and
|
|
alerts" tab — one report synthesized from up to `max_observations` observations of its bound scanner; `mode:
|
|
alert` delivers only when its alert condition holds. Each row carries the bound `scanner`, `mode`,
|
|
`trigger_type`, cadence, and `is_scanner_digest` (the built-in daily digest). Pass `?scanner=<id>` to filter
|
|
to one scanner. To read a specific action's produced reports, list its runs with `vision-actions-runs-list`.
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-retrieve:
|
|
operation: vision_actions_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get a vision action
|
|
description: >
|
|
Get a single vision action by ID. Returns its configuration: the bound `scanner`, `mode`
|
|
(group_summary/alert), `selection` (the observation-targeting predicate — verdict/tags/score bounds/window),
|
|
`synthesis_config`, `alert_config`, cadence, and delivery targets. Use `vision-actions-runs-list` +
|
|
`vision-actions-runs-retrieve` to read the reports it has actually produced.
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-run-create:
|
|
operation: vision_actions_run_create
|
|
enabled: false
|
|
vision-actions-runs-list:
|
|
operation: vision_actions_runs_list
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
list: true
|
|
title: List a vision action's runs
|
|
description: >
|
|
List the run history for one vision action (nested under the action). Each run is one execution that
|
|
produced a group summary or evaluated an alert. Rows are lightweight — `status`
|
|
(running/completed/failed/skipped), `scheduled_at`, `observation_count` (how many observations fed the
|
|
summary), `error_reason`, and `is_recovery` — without the report body. Fetch a completed run with
|
|
`vision-actions-runs-retrieve` to get the synthesized report and the observations it cited.
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-runs-retrieve:
|
|
operation: vision_actions_runs_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get a vision action run's full group summary
|
|
description: >
|
|
Get one vision action run in full: `synthesized_markdown` (the group-summary report, which cites its source
|
|
observations inline with `[obs N]` markers) and `observations` — the recordings the run included, in summary
|
|
order, each carrying the 1-based `index` that the report's `[obs N]` markers reference plus that
|
|
observation's title and outcome. This is the tool for auditing a group summary: for each `[obs N]` in the
|
|
markdown, `observations[N-1]` is the observation it cites, so you can check whether the cited observation
|
|
actually supports the claim. Empty `synthesized_markdown`/`observations` means the run skipped, failed, or
|
|
predates observation tracking.
|
|
feature_flag: replay-vision-actions
|
|
vision-actions-update:
|
|
operation: vision_actions_partial_update
|
|
enabled: true
|
|
scopes:
|
|
- vision_action:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: true
|
|
title: Update vision action
|
|
description: >
|
|
Partially update a vision action by ID. Send only the fields you want to change. Common changes: toggle
|
|
`enabled` to pause/resume it, tune the `alert_config` threshold, change the `trigger_config` cadence, or
|
|
point `delivery_config` at a different Slack channel or webhook. Nested objects and lists you send replace
|
|
the stored value wholesale (e.g. `delivery_config` is the full new list), so fetch the action with
|
|
`vision-actions-retrieve` first and resend the parts you want to keep. `mode` cross-field rules apply as on
|
|
create: switching to `alert` requires an `alert_config`.
|
|
exclude_params:
|
|
- id
|
|
- next_run_at
|
|
- last_run_at
|
|
- hog_flow_id
|
|
- created_at
|
|
- created_by
|
|
- updated_at
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision-actions
|
|
vision-observations-create-task-create:
|
|
operation: vision_observations_create_task_create
|
|
enabled: false
|
|
vision-observations-label-create:
|
|
operation: vision_observations_label_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: true
|
|
title: Rate a scanner observation
|
|
description: >
|
|
Set or update the shared thumbs up/down rating on a replay scanner observation: `is_correct` true means the
|
|
scanner got the session right (thumbs up), false means it got it wrong (thumbs down), plus optional written
|
|
`feedback` explaining why (applies to both directions). One shared rating per observation, team-wide, last
|
|
write wins; re-posting updates it in place. These ratings power the scanner's calibration tab: the
|
|
ratings-over-time chart and the AI prompt recommendations (see
|
|
`vision-scanners-prompt-suggestions-generate`). Find observation IDs via `vision-scanners-observations-list`
|
|
(filter `labeled=false` for unrated ones) or `vision-observations-list` for a session.
|
|
feature_flag: replay-vision
|
|
vision-observations-label-destroy:
|
|
operation: vision_observations_label_destroy
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: true
|
|
idempotent: true
|
|
title: Remove a scanner observation rating
|
|
description: >
|
|
Remove the shared thumbs up/down rating (and its feedback) from a replay scanner observation, returning it
|
|
to unrated. Team-wide: this clears the rating for everyone.
|
|
feature_flag: replay-vision
|
|
vision-observations-list:
|
|
operation: vision_observations_list
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
enrich_url: '{session_id}'
|
|
list: true
|
|
title: List a session's replay vision observations
|
|
description: >
|
|
List every replay vision observation recorded for a single session, across all scanners the caller can read.
|
|
This is the "what has Replay Vision found about this session?" lookup — pair it with the session-recording
|
|
MCP tools or the investigating-replay skill while investigating a recording. The `session_id` query
|
|
parameter is REQUIRED; without it the request is rejected. Each result includes the scanner that produced
|
|
it, the observation `status` (pending/running/succeeded/failed/ineligible), the `error_reason` for terminal
|
|
non-success statuses, and the `scanner_result` (LLM output) on success. To list observations for one
|
|
specific scanner over time instead, use `vision-scanners-observations-list`.
|
|
agent_note: >
|
|
Each observation's `_insightsUrl` opens the recording it analysed.
|
|
`scanner_result.model_output.reasoning_segments` interleaves prose with `chip` segments, and a chip's
|
|
`timestamp_ms` is the recording-relative offset of the moment being cited — append `?t=<seconds>`
|
|
(`timestamp_ms` / 1000, rounded down) to that URL to seek straight to it. When you report a finding to
|
|
someone, deep-link the one or two moments it turns on rather than only describing them.
|
|
feature_flag: replay-vision
|
|
url_prefix: /replay
|
|
vision-observations-retrieve:
|
|
operation: vision_observations_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
enrich_url: '{session_id}'
|
|
title: Get a session's replay vision observation
|
|
description: >
|
|
Get a single replay vision observation by ID, scoped to the scanners the caller can read. Returns the full
|
|
`scanner_snapshot` (the scanner's configuration at run time) and `scanner_result` (the LLM output, with any
|
|
event citations). Use `vision-observations-list` to find observation IDs for a session first.
|
|
agent_note: >
|
|
`_insightsUrl` opens the recording this observation analysed.
|
|
`scanner_result.model_output.reasoning_segments` interleaves prose with `chip` segments, and a chip's
|
|
`timestamp_ms` is the recording-relative offset of the moment being cited — append `?t=<seconds>`
|
|
(`timestamp_ms` / 1000, rounded down) to that URL to seek straight to it. When you report a finding to
|
|
someone, deep-link the one or two moments it turns on rather than only describing them.
|
|
feature_flag: replay-vision
|
|
url_prefix: /replay
|
|
vision-observations-retry-create:
|
|
operation: vision_observations_retry_create
|
|
enabled: false
|
|
vision-quota-retrieve:
|
|
operation: environment_vision_quota_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get replay vision quota
|
|
description: >
|
|
Get the organization's monthly Replay vision credit budget (1 credit = $0.01; observations cost credits by
|
|
model). Returns `credit_limit` (null when uncapped), `credits_used` (in-flight + succeeded observations
|
|
counted against it), `remaining`, `exhausted`, `projected_monthly_credits`, and the
|
|
`period_start`/`period_end` of the current calendar-month window. Check `remaining`/`exhausted` before
|
|
creating a scanner or calling `vision-scanners-scan-session`: over budget, a scanner's scheduled sweeps
|
|
silently skip their observations until the next period, while `vision-scanners-scan-session` is rejected
|
|
outright with a 402. Pair with `vision-scanners-estimate-create` to size a new scanner against the remaining
|
|
budget.
|
|
feature_flag: replay-vision
|
|
vision-scanners-affected-cohort-create:
|
|
operation: vision_scanners_affected_cohort_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- cohort:write
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Save a scanner's affected users as a cohort
|
|
description: >
|
|
Create a static cohort of the users a scanner matched within the trailing window (default 30 days). Monitors
|
|
count verdict-yes observations and take no qualifiers; classifiers require `tag`; scorers require
|
|
`min_score` and/or `max_score`. The cohort is a dated snapshot and does not live-update; calling again
|
|
creates a new cohort. Use it anywhere cohorts work: funnel breakdowns, retention, survey targeting,
|
|
experiment exclusion. Fails when the window has no users or none resolve to a person profile.
|
|
feature_flag: replay-vision
|
|
vision-scanners-bulk-observe-create:
|
|
operation: vision_scanners_bulk_observe_create
|
|
enabled: false
|
|
vision-scanners-create:
|
|
operation: vision_scanners_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Create replay vision scanner
|
|
description: >
|
|
Create a replay vision scanner. Pick a `scanner_type` — `monitor` (open-ended observations), `classifier`
|
|
(assign a tag from a fixed label set), `scorer` (numeric rubric), or `summarizer` (free-text summary, which
|
|
always emits facet embeddings) — and fill `scanner_config` to match: all types need a `prompt`; classifiers
|
|
also need `tags`; scorers also need `scale`; summarizers optionally set `length`. Unknown `scanner_config`
|
|
keys are rejected. `query` is a `RecordingsQuery` shape that defines which sessions the scanner watches —
|
|
`date_from` and `date_to` are ignored (the schedule controls time). `sampling_rate` is a 0..1 random
|
|
downsample applied after the query matches (default 1.0). `model` chooses the LLM, and sets what each
|
|
observation costs in credits. Names must be unique within the team. Before creating a broad scanner (a
|
|
permissive `query` and/or high `sampling_rate`), first call `vision-scanners-estimate-create` to project its
|
|
monthly credit spend and `vision-quota-retrieve` to check the org's remaining budget — an enabled scanner
|
|
sweeps matching recordings every 5 minutes, and creation itself does not check quota, so a too-broad scanner
|
|
can exhaust the monthly budget on its first sweeps. To get notified about what the scanner finds (a Slack
|
|
alert or a scheduled summary), create a vision action bound to it with `vision-actions-create`. For a
|
|
scanner scoped to one experiment's exposed sessions, load the `scanning-experiments-with-replay-vision`
|
|
skill first — it derives the query from the experiment's exposure criteria.
|
|
exclude_params:
|
|
- id
|
|
- scanner_version
|
|
- last_swept_at
|
|
- created_at
|
|
- created_by
|
|
- updated_at
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision
|
|
vision-scanners-creators-retrieve:
|
|
operation: vision_scanners_creators_retrieve
|
|
enabled: false
|
|
vision-scanners-delete:
|
|
operation: vision_scanners_destroy
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
annotations:
|
|
readOnly: false
|
|
destructive: true
|
|
idempotent: true
|
|
title: Delete replay vision scanner
|
|
description: >
|
|
Permanently delete a replay vision scanner by ID. The scanner stops scanning immediately and its scheduled
|
|
Temporal jobs are removed by the reconciler. Past observations recorded by this scanner are deleted along
|
|
with it. Use sparingly — to stop a scanner without losing its history, set `enabled: false` via
|
|
`vision-scanners-update` instead.
|
|
feature_flag: replay-vision
|
|
vision-scanners-estimate-create:
|
|
operation: vision_scanners_estimate_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Estimate replay vision scanner volume
|
|
description: >
|
|
Preview how many observations a proposed scanner would generate before saving it. Send a `query`
|
|
(`RecordingsQuery` shape; `date_from`/`date_to` are ignored — the estimate uses a fixed 30-day lookback) and
|
|
an optional `sampling_rate` and `model`. Returns `matched_sessions_in_window`, the `window_days` it was
|
|
measured over, `estimated_observations_per_month`, `credits_per_observation` (the price at that model),
|
|
`estimated_credits_per_month`, and `other_enabled_scanners_monthly_credits`. Enabled scanners sweep every 5
|
|
minutes, so a broad query can exhaust the monthly budget quickly — run this and compare the *credit* figures
|
|
against `remaining` from `vision-quota-retrieve` before `vision-scanners-create`. Comparing observation
|
|
counts against `remaining` understates the cost: `remaining` is denominated in credits, and one observation
|
|
costs several.
|
|
feature_flag: replay-vision
|
|
vision-scanners-get:
|
|
operation: vision_scanners_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get replay vision scanner
|
|
description: >
|
|
Get a single replay vision scanner by ID. Returns the full configuration including `scanner_config` (the
|
|
type-specific prompt and response schema parameters), the `RecordingsQuery` used to select sessions, and
|
|
operational fields like `enabled`, `sampling_rate`, `provider`, `model`, `emits_signals`, `scanner_version`,
|
|
and `last_swept_at`.
|
|
feature_flag: replay-vision
|
|
vision-scanners-impact-retrieve:
|
|
operation: vision_scanners_impact_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get a scanner's impact counts
|
|
description: >
|
|
Affected sessions and users for a scanner over a trailing window, counted from its observations. Monitors
|
|
count verdict-yes observations and take no qualifiers; classifiers require `tag`; scorers require
|
|
`min_score` and/or `max_score`. `sessions_without_user` reports affected sessions that carried no distinct
|
|
ID, so the user count reads honestly against the session count.
|
|
feature_flag: replay-vision
|
|
vision-scanners-inline-scan-create:
|
|
operation: vision_scanners_inline_scan_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: true
|
|
title: Scan sessions against a prompt
|
|
description: >
|
|
Answer a one-off question about session recordings you already have in hand, without creating a scanner
|
|
first. Pass the `session_ids` and a `prompt`; nothing is saved and nothing runs on a schedule. Prefer this
|
|
over `vision-scanners-create` whenever you have specific sessions and a question about them — a scanner is
|
|
for a standing watch over future recordings, and creating one to answer a single question leaves a scheduled
|
|
sweep behind. `scanner_type` defaults to `monitor` (open-ended); pass `classifier`, `scorer`, or
|
|
`summarizer` with the matching `scanner_config` (`tags`, `scale`, optional `length`) when you want
|
|
structured output. Results are not returned inline: the response gives a `scan_id`, and you read the
|
|
observations from `vision-observations-list` with that scanner id, polling until each session's observation
|
|
leaves `pending`/`running`. Asking the same question again returns the same `scan_id` and reuses the answers
|
|
it already has, so re-asking is cheap and sessions already scanned come back as `already_scanned` rather
|
|
than being charged twice. Scans cost credits per session at the same rate a scanner does; check
|
|
`vision-quota-retrieve` first if the batch is large, since sessions past the monthly limit come back as
|
|
`skipped_quota` instead of failing the call.
|
|
feature_flag: replay-vision
|
|
vision-scanners-list:
|
|
operation: vision_scanners_list
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
list: true
|
|
title: List replay vision scanners
|
|
description: >
|
|
List all replay vision scanners in the current project. A scanner watches a slice of session recordings
|
|
(defined by its `query` and `sampling_rate`) and runs one of four LLM analyses against each matching
|
|
session: `monitor` (open-ended observations), `classifier` (tag from a fixed label set), `scorer` (numeric
|
|
rubric), or `summarizer` (free-text summary, with facet embeddings for downstream search). Each scanner
|
|
returns its current configuration, enabled state, sampling rate, and `last_swept_at` timestamp.
|
|
feature_flag: replay-vision
|
|
vision-scanners-observations-create-task-create:
|
|
operation: vision_scanners_observations_create_task_create
|
|
enabled: false
|
|
vision-scanners-observations-get:
|
|
operation: vision_scanners_observations_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
enrich_url: '{session_id}'
|
|
title: Get scanner observation
|
|
description: >
|
|
Get a single scanner observation by ID. Returns the full `scanner_snapshot` (the scanner's configuration as
|
|
it was at run time, for reproducibility across `scanner_version` bumps) and the full `scanner_result`
|
|
including any event citations linking the LLM output back to specific events in the originating session.
|
|
Pair with the session-recording MCP tools or the investigating-replay skill to inspect the underlying
|
|
recording.
|
|
agent_note: >
|
|
`_insightsUrl` opens the recording this observation analysed.
|
|
`scanner_result.model_output.reasoning_segments` interleaves prose with `chip` segments, and a chip's
|
|
`timestamp_ms` is the recording-relative offset of the moment being cited — append `?t=<seconds>`
|
|
(`timestamp_ms` / 1000, rounded down) to that URL to seek straight to it. When you report a finding to
|
|
someone, deep-link the one or two moments it turns on rather than only describing them.
|
|
feature_flag: replay-vision
|
|
url_prefix: /replay
|
|
vision-scanners-observations-label-create:
|
|
operation: vision_scanners_observations_label_create
|
|
enabled: false
|
|
vision-scanners-observations-label-destroy:
|
|
operation: vision_scanners_observations_label_destroy
|
|
enabled: false
|
|
vision-scanners-observations-list:
|
|
operation: vision_scanners_observations_list
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
enrich_url: '{session_id}'
|
|
list: true
|
|
title: List scanner observations
|
|
description: >
|
|
List the observations recorded by a single scanner. Each observation is one scan of one session and includes
|
|
its `status` (pending/running/succeeded/failed/ineligible), `error_reason` (set on terminal non-success
|
|
statuses, e.g. `too_short`/`no_recording` for ineligible or `provider_rejected`/`validation_failed` for
|
|
failed — useful for triaging why a scan produced no result), `triggered_by` (`schedule` or `on_demand`),
|
|
`workflow_id`, the `scanner_result` (the LLM output validated against the scanner's response schema for that
|
|
scanner_type), and the `session_id` of the recording it analysed. Use this to inspect what a scanner has
|
|
found over time; to see every scanner's observations for one session, use `vision-observations-list`.
|
|
agent_note: >
|
|
Each observation's `_insightsUrl` opens the recording it analysed.
|
|
`scanner_result.model_output.reasoning_segments` interleaves prose with `chip` segments, and a chip's
|
|
`timestamp_ms` is the recording-relative offset of the moment being cited — append `?t=<seconds>`
|
|
(`timestamp_ms` / 1000, rounded down) to that URL to seek straight to it. When you report a finding to
|
|
someone, deep-link the one or two moments it turns on rather than only describing them.
|
|
feature_flag: replay-vision
|
|
url_prefix: /replay
|
|
vision-scanners-observations-retry-create:
|
|
operation: vision_scanners_observations_retry_create
|
|
enabled: false
|
|
vision-scanners-observations-stats:
|
|
operation: vision_scanners_observations_stats_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get scanner observation stats
|
|
description: >
|
|
Aggregate statistics over one scanner's observations. Accepts the same filters as
|
|
`vision-scanners-observations-list` plus `recent_days` (default 14, clamped 1..365) for the windowed series.
|
|
Returns `status_counts` (totals and success rate), `coverage` (distinct sessions scanned), `labels` (thumbs
|
|
up/down rating totals, a per-day accuracy series bucketed by scan day, per-day rating activity, and
|
|
`version_markers`: each prompt version's first day, prompt text, and rating counts), plus per-type
|
|
distributions (`monitor` verdict counts, `classifier` tag rankings, `scorer` score summary and histogram).
|
|
Use it to judge scanner quality and rating coverage, e.g. before generating a prompt suggestion with
|
|
`vision-scanners-prompt-suggestions-generate`.
|
|
feature_flag: replay-vision
|
|
vision-scanners-partial-update:
|
|
operation: vision_scanners_partial_update
|
|
enabled: false
|
|
vision-scanners-prompt-suggestions-apply:
|
|
operation: vision_scanners_prompt_suggestions_apply_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Apply prompt suggestion
|
|
description: >
|
|
Apply a prompt suggestion to its scanner: writes the `suggested_prompt` into the scanner's config (bumping
|
|
`scanner_version`) and marks the suggestion applied. Only the current (pending or dismissed) suggestion can
|
|
be applied, and only while the scanner's prompt is unchanged since it was generated. Otherwise the call is
|
|
rejected and you should generate a fresh one. This is a team-wide change: the scanner scans with the new
|
|
prompt from its next sweep onward.
|
|
feature_flag: replay-vision
|
|
vision-scanners-prompt-suggestions-current:
|
|
operation: vision_scanners_prompt_suggestions_current_retrieve
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:read
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: true
|
|
destructive: false
|
|
idempotent: true
|
|
title: Get current prompt suggestion
|
|
description: >
|
|
Get a scanner's newest AI prompt-rewrite suggestion, generated from the team's thumbs up/down ratings.
|
|
Returns `suggestion` (null when none has been generated yet) with its `suggested_prompt`, the `base_prompt`
|
|
it rewrote (for diffing), the `rationale`, and `status` (pending/applied/dismissed/superseded). Also returns
|
|
`stale` (true when ratings changed since it was generated, so regenerating may give a better rewrite) and
|
|
`rated_count` (rated observations available to generate from).
|
|
feature_flag: replay-vision
|
|
vision-scanners-prompt-suggestions-dismiss:
|
|
operation: vision_scanners_prompt_suggestions_dismiss_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: true
|
|
title: Dismiss prompt suggestion
|
|
description: >
|
|
Dismiss a prompt suggestion without applying it. The scanner's prompt is unchanged and the suggestion stays
|
|
in history. A dismissed suggestion can still be applied later with
|
|
`vision-scanners-prompt-suggestions-apply` as long as the scanner's prompt hasn't changed since it was
|
|
generated.
|
|
feature_flag: replay-vision
|
|
vision-scanners-prompt-suggestions-evaluate-create:
|
|
operation: vision_scanners_prompt_suggestions_evaluate_create
|
|
enabled: false
|
|
vision-scanners-prompt-suggestions-generate:
|
|
operation: vision_scanners_prompt_suggestions_generate_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Generate prompt suggestion from ratings
|
|
description: >
|
|
Generate a fresh AI prompt-rewrite suggestion for a scanner from the team's current thumbs up/down ratings
|
|
and their written feedback. Requires at least one rated observation (check `rated_count` on
|
|
`vision-scanners-prompt-suggestions-current`; create ratings with `vision-observations-label-create`). The
|
|
previous pending suggestion becomes superseded. The LLM call runs inline, so expect the request to take a
|
|
while. Review the returned suggestion, then apply it with `vision-scanners-prompt-suggestions-apply` or
|
|
dismiss it with `vision-scanners-prompt-suggestions-dismiss`.
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision
|
|
vision-scanners-prompt-suggestions-list:
|
|
operation: vision_scanners_prompt_suggestions_list
|
|
enabled: false
|
|
vision-scanners-scan-session:
|
|
operation: vision_scanners_observe_create
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: false
|
|
title: Run scanner against a session now
|
|
description: >
|
|
Trigger a scanner against a specific session immediately, bypassing the sampling rate and any scheduled
|
|
sweep. Returns 202 with a `workflow_id`; the resulting observation appears under
|
|
`vision-scanners-observations-list` once the Temporal workflow finishes (typically several minutes —
|
|
rasterising the recording into video and calling the LLM is slow). Useful for ad-hoc scanning during an
|
|
investigation: pick a scanner that matches the question you want answered, and point it at the session
|
|
you're examining. A scanner can observe a given session only once — re-triggering it on a session that
|
|
already has an observation (even a failed or ineligible one) is a no-op and will not produce a fresh scan.
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision
|
|
vision-scanners-stats-retrieve:
|
|
operation: vision_scanners_stats_retrieve
|
|
enabled: false
|
|
vision-scanners-suggest-tags-create:
|
|
operation: vision_scanners_suggest_tags_create
|
|
enabled: false
|
|
vision-scanners-update:
|
|
operation: vision_scanners_partial_update
|
|
enabled: true
|
|
scopes:
|
|
- replay_scanner:write
|
|
- session_recording:read
|
|
annotations:
|
|
readOnly: false
|
|
destructive: false
|
|
idempotent: true
|
|
title: Update replay vision scanner
|
|
description: >
|
|
Partially update a replay vision scanner by ID. Send only the fields you want to change. Common changes:
|
|
toggle `enabled` to pause/resume scanning, adjust `sampling_rate`, tweak the `scanner_config` prompt or
|
|
tags, or refine the `query` to watch a different slice of sessions. `scanner_type` is locked after creation
|
|
— to change it, delete and recreate. Editing any config field bumps `scanner_version`; existing observations
|
|
keep a snapshot of the previous config for reproducibility. If you widen the `query` or raise
|
|
`sampling_rate`, re-check projected volume with `vision-scanners-estimate-create` and remaining budget with
|
|
`vision-quota-retrieve` first — the scanner resumes sweeping every 5 minutes at the new, broader scope.
|
|
exclude_params:
|
|
- id
|
|
- scanner_version
|
|
- last_swept_at
|
|
- created_at
|
|
- created_by
|
|
- updated_at
|
|
requires_ai_consent: true
|
|
feature_flag: replay-vision
|