Files
insights/.semgrep/rules/security/data-imports-http-transport.yaml
zeekay 1b367e2b62 chore: rebase the tree onto upstream master
The tree was a debranded snapshot of upstream taken around 2026-02-20, and
upstream has moved about five months since. This replaces it with upstream
master, mechanically re-derived: bin/debrand does the rename, and the
pipeline and squashed migrations are restored from the shipped tree.

Not a merge. The previous import replaced the tree wholesale in a single
commit, so there is no shared content history to merge against. Reproducing
the rename on the new tree is what actually converges.

What survives: .hanzo/workflows, the Dockerfiles and bin/docker* entrypoints,
compose files, LLM.md, and insights/migrations. The squashed initial
migration is load-bearing — the live database records it by name, so
upstream's ~918 individual migrations cannot come back.

Nine products are dropped as unreferenced. Twenty-one more that we do not run
are kept anyway because the core app still imports them; unpicking those is a
refactor and does not belong in a rebase.

This base does not build yet, and the reasons are not in the rename:
upstream now wants Python 3.13.13 against our 3.12.12 and Django 5.2 against
our 4.2, and the dependency manifests still name upstream packages that we
republish under our own names. Those are decisions, not renames.
2026-08-01 15:44:40 -07:00

105 lines
4.4 KiB
YAML

# Warehouse-source HTTP calls must go through `make_tracked_session()` so they
# participate in HTTP logging, metrics, and sample capture (see
# products/warehouse_sources/backend/temporal/data_imports/sources/common/http/). Direct `requests.<verb>`
# / `requests.Session()` / `httpx.Client(...)` calls bypass the transport.
# Type and exception imports (`from requests import Response`,
# `from requests.exceptions import ...`) remain allowed — only the network-call
# patterns are banned.
rules:
- id: data-imports-http-transport-requests-session
pattern: requests.Session(...)
message: |
Use `make_tracked_session(...)` from
`products.warehouse_sources.backend.temporal.data_imports.sources.common.http` instead of
`requests.Session()`. The tracked session attaches `team_id`,
`source_type`, schema/job IDs to logs and metrics for every
outbound request, and participates in opt-in sample capture.
If a vendor SDK constructs its own `Session` and exposes no hook
to inject ours, add `# nosemgrep: data-imports-http-transport-requests-session`
on the line above with a brief reason.
languages: [python]
severity: ERROR
metadata:
category: best-practice
subcategory: audit
confidence: HIGH
paths:
include:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/**'
exclude:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/common/http/**'
- '**/test/**'
- '**/tests/**'
- '**/test_*.py'
- id: data-imports-http-transport-requests-verb
pattern-either:
- pattern: requests.get(...)
- pattern: requests.post(...)
- pattern: requests.put(...)
- pattern: requests.patch(...)
- pattern: requests.delete(...)
- pattern: requests.head(...)
- pattern: requests.options(...)
- pattern: requests.request(...)
message: |
Use `make_tracked_session(...)` from
`products.warehouse_sources.backend.temporal.data_imports.sources.common.http` instead of the
module-level `requests.<verb>` shortcut. Each tracked session
participates in HTTP logging, metrics, and sample capture.
If a vendor SDK constructs its own client and exposes no hook to
inject our session, add `# nosemgrep: data-imports-http-transport-requests-verb`
on the line above with a brief reason.
languages: [python]
severity: ERROR
metadata:
category: best-practice
subcategory: audit
confidence: HIGH
paths:
include:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/**'
exclude:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/common/http/**'
- '**/test/**'
- '**/tests/**'
- '**/test_*.py'
- id: data-imports-http-transport-httpx-client
pattern-either:
- pattern: httpx.Client(...)
- pattern: httpx.AsyncClient(...)
- pattern: httpx.get(...)
- pattern: httpx.post(...)
- pattern: httpx.put(...)
- pattern: httpx.patch(...)
- pattern: httpx.delete(...)
- pattern: httpx.head(...)
- pattern: httpx.options(...)
- pattern: httpx.request(...)
message: |
`httpx` is not currently routed through the warehouse-source
tracked transport. Use `make_tracked_session(...)` (a
`requests.Session`) from
`products.warehouse_sources.backend.temporal.data_imports.sources.common.http` instead.
If you must use `httpx` here for some reason, add a
`# nosemgrep: data-imports-http-transport-httpx-client` comment
and explain the reason.
languages: [python]
severity: ERROR
metadata:
category: best-practice
subcategory: audit
confidence: HIGH
paths:
include:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/**'
exclude:
- '**/products/warehouse_sources/backend/temporal/data_imports/sources/common/http/**'
- '**/test/**'
- '**/tests/**'
- '**/test_*.py'