insights/docs/published/handbook/engineering/ai/sandboxed-agents.md

title, sidebar, showTitle

title	sidebar	showTitle
Using sandboxed agents	Docs	true

Sandboxed agents are background AI agents that run in isolated cloud containers with access to Insights data, GitHub repositories, and code execution. Use them when your feature needs an autonomous agent that reads Insights data, writes code, and produces artifacts like PRs or reports.

For simpler LLM calls (summarization, translation, classification), skip this page and use the LLM gateway (get_llm_client(product=..., team_id=...)) directly — it's simpler and doesn't need a sandbox.

When to use what

Example	Solution
Signals team building an enrichment pipeline that generates reports from Insights analytics data	Sandboxed agent (this page)
Conversations team building a support agent that queries Insights and customer documentation	Sandboxed agent (this page)
AI observability summarizing a funnel, generating a natural-language insight title	LLM gateway via get_llm_client(product=..., team_id=...)
Not sure	Ask in #team-insights-ai

Rule of thumb: if the LLM needs to do things (query data, read files, create branches, open PRs), use a sandboxed agent. If it just needs to answer a question given some context you already have, use the LLM gateway.

How it works

A sandboxed agent runs inside an isolated cloud container (Modal in production, Docker locally). The system provisions the sandbox, clones a GitHub repo, starts an agent server, and waits for the agent to finish.

Your product code
    │
    │  Task.create_and_run(...)
    ▼
Temporal workflow (process-task)
    │
    ├── 1. Create scoped OAuth token
    ├── 2. Provision sandbox (Modal / Docker)
    ├── 3. Clone repository
    ├── 4. Start agent server
    ├── 5. Wait for completion (heartbeat-extended timeout)
    └── 6. Cleanup sandbox

The agent inside the sandbox gets:

• A scoped OAuth access token for the Insights API (6-hour TTL)
• A GitHub installation token for repo operations
• Access to the Insights MCP server for querying data
• Code execution capabilities within the sandbox

Creating a sandboxed agent

Use Task.create_and_run() to launch a sandboxed agent from your product code:

from products.tasks.backend.models import Task

task = Task.create_and_run(
    team=team,
    title="Generate weekly signal report",
    description="Analyze error trends and generate a summary report with recommendations.",
    origin_product=Task.OriginProduct.ERROR_TRACKING,  # or your product's origin
    user_id=user.id,
    insights_mcp_scopes="read_only",  # or "full" if the agent needs write access
)

Parameters

Parameter	Required	Description
team	Yes	The team this task belongs to
title	Yes	Human-readable task title
description	Yes	Detailed description of what the agent should do
origin_product	Yes	Which product created this task (see Task.OriginProduct choices)
user_id	Yes	User ID — used for feature flag validation and creating the scoped API key
repository	Yes	GitHub repo in org/repo format (e.g., insights/insights-js)
insights_mcp_scopes	No	Scope preset or explicit scope list (default: "full")
create_pr	No	Whether the agent should create a PR (default: True)
mode	No	Execution mode (default: "background")
slack_thread_context	No	Slack thread context for agents triggered from Slack
start_workflow	No	Whether to start the Temporal workflow immediately (default: True)
sandbox_environment_id	No	ID of a SandboxEnvironment to apply network restrictions (see below)

Adding a new origin product

If your product doesn't have an OriginProduct entry yet, add one to Task.OriginProduct in products/tasks/backend/models.py:

class OriginProduct(models.TextChoices):
    ERROR_TRACKING = "error_tracking", "Error Tracking"
    # ...
    YOUR_PRODUCT = "your_product", "Your Product"

Then create and run a Django migration.

Fine-grained access tokens

Every sandboxed agent gets a scoped OAuth access token that controls what Insights resources it can access. Tokens expire after 6 hours and are scoped to a single team.

Scope presets

Use the insights_mcp_scopes parameter to control access:

Preset	What it grants	When to use
"read_only"	Read access to actions, cohorts, dashboards, experiments, feature flags, insights, queries, surveys, etc.	Agent only needs to read data for analysis or reporting
"full" (default)	Read + write access to all MCP-exposed resources	Agent needs to create or modify Insights resources (e.g., create feature flags, update experiments)

Custom scopes

For more granular control, pass an explicit list of scopes instead of a preset:

Task.create_and_run(
    # ...
    insights_mcp_scopes=["query:read", "feature_flag:read", "experiment:read"],
)

Available read scopes: action:read, cohort:read, dashboard:read, error_tracking:read, event_definition:read, experiment:read, feature_flag:read, insight:read, project:read, query:read, survey:read, and others.

Available write scopes: action:write, cohort:write, dashboard:write, experiment:write, feature_flag:write, insight:write, survey:write, and others.

Internal scopes (task:write, llm_gateway:read) are always added automatically.

See insights/temporal/oauth.py for the full list.

Principle of least privilege: default to "read_only" unless your agent genuinely needs to create or modify resources. This limits blast radius if the agent misbehaves.

Insights MCP server

The sandbox comes with access to the Insights MCP server, which exposes Insights resources as tools the agent can call — listing feature flags, running InsightsQL queries, searching session recordings, etc.

The MCP server is ready to use today. For details on available tools, see Implementing MCP tools.

Skills

Skills are job-to-be-done templates that teach agents how to compose MCP tools into workflows. They provide domain knowledge, query patterns, and step-by-step guidance.

Skills are pre-installed in the sandbox base image and available to all sandboxed agents. They're copied to three discovery locations during image build:

• /scripts/plugins/insights/skills/ – plugin discovery
• ~/.agents/skills/ – Codex agent discovery
• ~/.claude/skills/ – Claude Code CLI discovery

For details on writing skills, see Writing skills.

Multi-turn sessions

MultiTurnSession provides a structured API for building custom multi-turn research agents. Use it when your agent needs multiple conversation turns with schema-validated responses – for example, a discovery pass followed by per-item research, then assessment and summarization.

Mental model

1. Start – MultiTurnSession.start(prompt, context, model=Shape) launches a sandbox, sends the first prompt, and returns a validated Pydantic model.
2. Follow up – session.send_followup(prompt, Shape) sends additional prompts within the same sandbox session. Each response is validated against the provided schema. The session retries once on empty responses.
3. End – session.end() signals the sandbox workflow to shut down.

Example

from products.tasks.backend.logic.services.custom_prompt_multi_turn_runner import MultiTurnSession
from products.tasks.backend.logic.services.custom_prompt_internals import CustomPromptSandboxContext

# 1. Start: discovery turn
session, candidates = await MultiTurnSession.start(
    prompt="Find up to 10 items to investigate.",
    context=context,  # CustomPromptSandboxContext
    model=DiscoveryResult,
    branch="master",
    step_name="my_discovery",
)

# 2. Follow up: research each item
for item in candidates.items:
    finding = await session.send_followup(
        f"Research {item.name} in detail.",
        FindingResult,
        label=f"research_{item.name}",
    )

# 3. Follow up: summarize
summary = await session.send_followup(
    "Summarize all findings.",
    SummaryResult,
    label="summary",
)

# 4. End the session
await session.end()

Reference implementation

See products/tasks/backend/logic/services/mts_example/ for a complete working example. It runs a multi-turn agent that discovers "cursed" identifiers in a repo, researches each one, and produces output in the shape Signals consumes:

discovery → research ×N → actionability → priority? → presentation

Run it locally (DEBUG only):

DEBUG=1 python manage.py demo_mts_example --team-id <id> --user-id <id>

See the example README for details on adapting it to your own use case.

Code execution

Agents run inside an isolated sandbox with full code execution capabilities. They can:

• Read, write, and execute files in the cloned repository
• Install dependencies (npm, pip, etc.)
• Run tests, linters, and build tools
• Create git branches and pull requests (commits must be signed — see below)
• Execute arbitrary shell commands within the container

Git commit signing

Direct git commit and git push commands are blocked in the sandbox to ensure all commits are properly signed by GitHub. A PATH shim (git-guard.sh at /opt/insights/bin/git) intercepts these subcommands while passing all other git operations through to the real binary.

If an agent attempts to run git commit or git push, it will see:

git commit is disabled in Insights Desktop: commits must be signed.
To commit: stage changes with 'git add', then call the git_signed_commit tool.
To force-push after a rebase/conflict fix: call the git_signed_rewrite tool.

Agents should stage changes with git add, then use the git_signed_commit tool to create signed commits. For force-pushing after a rebase or conflict resolution, use the git_signed_rewrite tool instead.

Debugging escape hatch: Set INSIGHTS_ALLOW_UNSIGNED_GIT=1 in the sandbox environment to bypass this restriction. This is intended for debugging only and should not be used in production.

Sandbox isolation

	Production (Modal)	Local dev (Docker)
Isolation	gVisor kernel-level sandboxing	Standard Docker container
Network	Configurable via SandboxEnvironment	Host network via host.docker.internal
Image	ghcr.io/insights/insights-sandbox-base	Local Dockerfile build
Auth	Modal connect token	No token needed

Runtime selection (gVisor vs Modal VM)

Production sandboxes run on one of two Modal runtimes, chosen per run in get_task_processing_context (_resolve_modal_vm_sandbox) and forked in provision_sandbox:

• gVisor (SandboxTemplate.DEFAULT_BASE) — the historical default: a gVisor kernel-sandboxed container.
• Modal VM (SandboxTemplate.VM_BASE) — a kernel microVM that also bakes in Docker-in-Docker, so the agent can run nested containers. Custom base images layer on this base, and it is what image-builder runs execute on.

Selection is driven by the tasks-modal-vm-sandbox flag's JSON payload, which carries two origin allowlists and an optional default image:

• origin_products — origins allowed on the VM runtime when a custom image is resolved for the run (custom images cannot run under gVisor).
• default_base_origin_products — origins that default to the bare VM base image even without a custom image. This is the knob for making the VM runtime the default for standard cloud runs; we widen it origin-by-origin (and, later, the flag's release condition) as the rollout expands.
• default_custom_image — a Modal image name that VM runs fall back to when no custom image was picked. Because the flag's payload variants are org-targeted, this routes which default VM image an org gets: Insights's own org points at the prebaked dev-stack image (below), everyone else keeps the plain VM base. A user- or environment-selected custom image always wins over this default, and provisioning falls back to the plain VM base if the named image is missing.

The prebaked dev-stack image

insightscli start on a fresh VM pays for multi-gigabyte docker pulls and the full Django + persons + Datastore migration history — and dead-ends anyway, because the lean VM base lacks the dev toolchain flox provides on dev machines (brotli, phrocs, Go, Rust). For runs on the Insights monorepo we bake all of that ahead of time: the bake-dev-stack-image Temporal workflow boots a plain VM-base sandbox, runs bake-insights-dev-stack.sh inside it (install the dev toolchain, pre-pull the dev compose images, bring the stack up, run the Django and Rust-driven migrations, shut down cleanly), snapshots the filesystem, and publishes it under the fixed Modal image name insights-dev-stack (see products/tasks/backend/logic/services/dev_stack_image.py). It is dispatched on two cadences, both gated per region on the tasks-dev-stack-image-bake flag via a region person property: a nightly full rebake that keeps the heavy state (migrations, docker pulls) close to master, and — mirroring the custom-image refresh fanout — a ten-minute sweep that rebakes as soon as the VM base image digest moves (e.g. an agent-server release), at most once per new digest. python manage.py bake_dev_stack_image triggers a bake manually and bypasses the flag. Pointing an org's default_custom_image payload key at that name gives its VM runs warm docker state and already-migrated databases, so a task-time insightscli start only applies the migrations that landed since the last bake. The pnpm store and Playwright's Chromium are prewarmed too: pnpm install --frozen-lockfile --prefer-offline is a fast linking pass and browser installs are no-ops. Build outputs (node_modules, Storybook dist, Vite/Turbo caches) are deliberately not baked — the bake's checkout is deleted before the snapshot — so frontend builds always run from the task's own source. The bake must run on the real VM runtime — dockerd cannot run inside Modal's gVisor image builder — which is why it is a sandbox filesystem snapshot rather than a spec-built image.

At task time the restored image is not self-starting: the sandbox runtime rewrites /etc/hosts at boot and dockerd does not autostart. Run the baked bootstrap-dev-stack helper first (restores the compose host aliases and starts dockerd — the bake manifest at /opt/insights/dev-stack-bake.json names it under bootstrap), then from the checkout run uv sync, source .venv/bin/activate, insightscli start -y -d, and insightscli wait. Detached mode is required — the sandbox has no TTY, and running phrocs under a pseudo-TTY makes it balloon in memory until it is OOM-killed — and the detached start returns while the stack is still booting, so insightscli wait is what blocks until every process reports ready.

Provisioning also fires that helper detached as soon as the sandbox is up (best-effort, only on runs that booted the Insights-published insights-dev-stack image itself — never a user-authored custom image, and never a filesystem-snapshot restore, whose filesystem a prior run could have altered; directory-snapshot resumes only mount the workspace, so they keep the warmup), so the dockerd warmup overlaps the repo clone and the environment is usually ready by the agent's first command. Running bootstrap-dev-stack again is still the right first step — it is the synchronization point, blocking until the warmup completes.

Runs with a restricted-egress SandboxEnvironment (a custom domain allowlist) always stay on gVisor — Modal's outbound domain allowlist is a gVisor-only feature. The use_modal_vm_sandbox run-state key force-selects the VM runtime for trusted server-created runs (image builders) and is never accepted from client input.

Network access

Network access is configured per-team via SandboxEnvironment:

• Trusted — only allows access to a default set of trusted domains (GitHub, npm, PyPI, etc.)
• Full — unrestricted network access
• Custom — explicit allowlist of domains, optionally including the trusted defaults

To apply network restrictions from your product code, create a SandboxEnvironment and pass its ID to Task.create_and_run:

from products.tasks.backend.models import SandboxEnvironment, Task

# 1. Create an environment (once, or look up an existing one)
env = SandboxEnvironment.objects.create(
    team=team,
    created_by=user,
    name="Restricted agent env",
    network_access_level="custom",  # "full" | "trusted" | "custom"
    allowed_domains=["github.com", "api.example.com"],
    include_default_domains=True,  # merge GitHub, npm, PyPI defaults
)

# 2. Pass its ID when creating the task
task = Task.create_and_run(
    team=team,
    title="My restricted task",
    description="...",
    origin_product=Task.OriginProduct.YOUR_PRODUCT,
    user_id=user.id,
    repository="org/repo",
    sandbox_environment_id=str(env.id),
)

The temporal workflow resolves the allowed domains at execution time from the environment, so updates to the environment take effect on the next run. Domain restrictions are enforced at the syscall level by agentsh via ptrace — the agent cannot bypass them through proxy settings or DNS tricks.

Environments can also be managed via the REST API (SandboxEnvironmentViewSet) or the Insights Desktop settings UI.

Custom base images

Teams can bake their own tools and dependencies into a custom base image (SandboxCustomImage) and select it as a cloud environment's base via SandboxEnvironment.custom_image. Custom images always layer on top of the published VM sandbox base — agent tooling, git guard, and the VM runtime are always present — and the whole mechanism is gated on the Modal VM runtime being available: the sandbox_custom_images API returns 403 (and the Insights Desktop UI hides the feature) unless the tasks-modal-vm-sandbox flag is enabled for the org with user_created in its origin_products payload allowlist, since custom-image sandboxes cannot run under gVisor.

The flow, driven from the Insights Desktop Environments → Cloud tab:

1. Creating an image spawns an interactive image-builder agent task (custom_image_builder_id in the run state, VM runtime forced) that iterates inside the real VM base and maintains a declarative spec (SandboxImageSpec: apt_packages, run_commands, env) at /tmp/workspace/image-spec.yaml.
2. "Save & build" reads the spec from the builder sandbox (or accepts it inline via POST /v1/projects/:id/sandbox_custom_images/:id/build/), then the build-sandbox-image Temporal workflow runs an LLM security scan of the spec, builds it layered on the VM base, and publishes it as a Modal named image (Image.publish() / Image.from_name()).
3. Runs using an environment with a ready custom image provision their sandbox from the published image (SandboxConfig.custom_image_name), falling back to the standard base if the image can't be loaded. Repo-setup snapshots are skipped for custom-image runs; resume snapshots still apply.

Local development

To set up sandboxed agents for local development:

1. Create a personal dev GitHub App (see the Cloud runs setup guide for details)
2. Run python manage.py setup_background_agents
3. Run insightscli start

The setup command is idempotent and handles:

• Writing required env vars (OIDC_RSA_PRIVATE_KEY, SANDBOX_JWT_PRIVATE_KEY, DEBUG, SANDBOX_PROVIDER, SANDBOX_MCP_URL) to your .env
• Creating the Array OAuth application
• Enabling the tasks feature flag for all teams
• Building the agent skills bundle

For advanced setup options (Modal sandboxes, local agent packages, MCP), see the Cloud runs setup guide.

Tip: Set SANDBOX_REPO_MOUNT_MAP to bind-mount local repositories into the Docker container and skip cloning from GitHub. Format: SANDBOX_REPO_MOUNT_MAP=org/repo:/local/path (e.g., SANDBOX_REPO_MOUNT_MAP=Insights/insights:~/Developer/insights). This can significantly reduce sandbox startup time for large repos.

Questions?

If you're unsure whether a sandboxed agent is the right fit for your use case, ask in #team-insights-ai on Slack.