feat(single-binary): zip front router + reverse-proxy to in-process Blockscout-rs services

Make the explorer ONE Go binary fronted by github.com/hanzoai/zip (Fiber v3 /
fasthttp) that runs the Blockscout-rs Rust services IN-PROCESS via the existing
cgo FFI staticlib and reverse-proxies to each. Least-work: the Rust services are
not rewritten/ported — their real entrypoints run on Tokio threads inside the
process, keeping their own HTTP servers on loopback.

Front router (front.go): zip owns the public listener; routes in order
  /api/<prefix>/*  -> httputil.ReverseProxy -> 127.0.0.1:<svc_port> (prefix stripped)
  /*               -> the existing net/http explorer mux via zip.AdaptNetHTTP
Security headers + CORS become zip middleware (replaces withSecurity).

Config (services: block, ServicesConfig/resolveServices in config.go+services.go):
single source of truth for enabled services, loopback ports (base 8050,
HTTP=base+2n/gRPC=+1), URL prefixes, and per-service settings JSON; both the
proxy and the FFI launcher read it. startFFIServices now takes resolvedService
and feeds each service its server addr (+ DB url when set).

Status: sig-provider WIRED + PROVEN end-to-end — one process owns :8090 and
:8050, curl /api/sig/health -> {"status":"SERVING"} and the real REST API
/api/sig/api/v1/abi/function -> 200. visualizer + smart-contract-verifier are
stateless and ready to wire (recipe in ffi/README.md). stats +
multichain-aggregator stay DISABLED: blockscout-service-launcher's DB layer is
Postgres-only (initialize_postgres pins DatabaseBackend::Postgres) — no SQLite
without a Rust rewrite — and stats also needs a populated Blockscout indexer
Postgres. The Go side already passes database.url+migrations through, so they
enable with config alone once a Postgres exists.

Build: make single (cargo build ffi/ + go build -tags ffi) unchanged; default
go build ./... is the Go-native binary (ffi_off.go stub, no cgo). go build ./...
and go test ./... pass; make single links the sig-provider symbol and boots.
This commit is contained in:
zooqueen
2026-06-23 21:20:29 -07:00
parent f3bf55b0b9
commit 7156dfdd7a
11 changed files with 510 additions and 61 deletions
+28
View File
@@ -51,6 +51,34 @@ explorer Deployment.
Per env: 1 Pod, 2 containers, 1 Service exposing `:80 / :8090 / :3000`.
## Single-binary mode — zip front + Blockscout-rs in-process (cgo FFI)
The explorer can be built as ONE Go binary that runs the Blockscout-rs Rust
services IN-PROCESS (each a Tokio thread, no subprocess) and reverse-proxies to
them through `github.com/hanzoai/zip` (Fiber v3 / fasthttp) as the front router.
LEAST-WORK: the Rust services are NOT rewritten/ported — their real entrypoints
run via the `ffi/` staticlib (`lux_explorer_ffi`, crate-type staticlib) over
cgo, keeping their own HTTP servers on loopback.
- **Build:** `make single` (default `make build` / `go build ./...` is the
Go-native binary, no cgo — `ffi_off.go` stub). Scope services with
`make single FFI_FEATURES=sig-provider,visualizer`.
- **Front router (`front.go`):** zip owns the public listener. Routes, in order:
`/api/<prefix>/*` → `httputil.ReverseProxy` → `127.0.0.1:<svc_port>`
(prefix stripped); `/*` → the existing net/http explorer mux via
`zip.AdaptNetHTTP`. Security headers + CORS are zip middleware.
- **Config (`services:` in chains.yaml, `ServicesConfig`/`resolveServices`):**
one source of truth for enabled services, loopback ports (base 8050,
HTTP=base+2n/gRPC=+1), prefixes, and per-service settings JSON. The FFI
launcher (`ffi_on.go startFFIServices`) and the proxy both read it.
- **Status:** sig-provider WIRED + PROVEN end-to-end (one process owns :8090
and :8050; `curl /api/sig/health → {"status":"SERVING"}`). visualizer +
smart-contract-verifier are stateless and ready to wire (recipe in
`ffi/README.md`). stats + multichain-aggregator stay DISABLED:
`blockscout-service-launcher` DB layer is Postgres-only (no SQLite without a
Rust rewrite), and stats also needs a populated Blockscout indexer Postgres.
See `ffi/README.md` for the per-service table + exact code citations.
Manifests:
~/work/lux/universe/k8s/lux-mainnet/explorer.yaml
~/work/lux/universe/k8s/lux-mainnet/explore-ingress.yaml
+27
View File
@@ -262,3 +262,30 @@ chains:
rpc: ${ATOM_RPC:-https://cosmos-rpc.publicnode.com}
coin: ATOM
enabled: false
# In-process Blockscout-rs services (single-binary `-tags ffi` build only).
# Each runs as a Tokio thread inside the explorer process; the zip front router
# reverse-proxies /api/<prefix>/* -> http://127.0.0.1:<http_port>. In the
# default (non-FFI) build this whole block is inert. See ffi/README.md.
services:
enabled: false # master switch; true only in the `make single` binary
list:
# sig-provider — stateless 4byte/ABI signature lookup. WIRED + PROVEN.
- name: sig-provider
prefix: sig # -> /api/sig/* (e.g. /api/sig/health, /api/sig/api/v1/abi/function)
http_port: 8050 # loopback; gRPC auto = http_port+1
# --- ready to wire (stateless, no DB) — add the dep+feature+service! in ffi/ ---
# - name: visualizer # -> /api/visualizer/*
# http_port: 8052
# - name: smart-contract-verifier # -> /api/verifier/*
# http_port: 8054
# --- Postgres-only (NOT sqlite-able via blockscout-service-launcher) ---
# stats + multichain-aggregator need a real Postgres (stats also needs a
# populated Blockscout indexer Postgres to read). Set database_url to a
# postgresql:// DSN to enable; migrations run automatically.
# - name: stats
# http_port: 8056
# database_url: ${STATS_DB:-postgresql://localhost/stats}
# - name: multichain-aggregator
# http_port: 8058
# database_url: ${MULTICHAIN_DB:-postgresql://localhost/multichain}
+50 -6
View File
@@ -10,12 +10,56 @@ import (
// Config is the top-level explorer configuration loaded from chains.yaml.
type Config struct {
DataDir string `yaml:"data_dir"`
HTTPAddr string `yaml:"http_addr"`
StaticDir string `yaml:"static_dir"` // overlay dir for SPA; empty = embedded only
BrandDefault Brand `yaml:"brand_default"`
Networks []Network `yaml:"networks"`
Chains []ChainConfig `yaml:"chains"`
DataDir string `yaml:"data_dir"`
HTTPAddr string `yaml:"http_addr"`
StaticDir string `yaml:"static_dir"` // overlay dir for SPA; empty = embedded only
BrandDefault Brand `yaml:"brand_default"`
Networks []Network `yaml:"networks"`
Chains []ChainConfig `yaml:"chains"`
Services ServicesConfig `yaml:"services"`
}
// ServicesConfig configures the in-process Blockscout-rs services that the
// `-tags ffi` build runs as goroutine-driven Tokio threads inside this one
// binary. The zip front router reverse-proxies each enabled service under
// /api/<prefix>/* → http://127.0.0.1:<port>. In the default (non-FFI) build
// nothing is launched and these proxies are not mounted (see services.go).
type ServicesConfig struct {
// Enabled is the master switch. When false (default) no in-process
// service is started and no proxy is mounted — the explorer is exactly
// the Go-native binary it has always been.
Enabled bool `yaml:"enabled"`
// Services lists each in-process service. Order is irrelevant; the
// reverse-proxy prefix is derived from Prefix (default = Name).
Services []ServiceConfig `yaml:"list"`
}
// ServiceConfig is one in-process Blockscout-rs service.
type ServiceConfig struct {
// Name is the FFI service key (e.g. "sig-provider", "stats"). It selects
// which lux_explorer_start_<name> entrypoint runs and keys the settings
// passed to it.
Name string `yaml:"name"`
// Prefix is the URL path segment the front router proxies under
// (/api/<prefix>/*). Empty => derived from Name (sig-provider => "sig").
Prefix string `yaml:"prefix"`
// HTTPPort is the localhost port the service's own HTTP server binds.
// Empty => a default is assigned per service (see servicePort).
HTTPPort int `yaml:"http_port"`
// GRPCPort is the localhost port for the service's gRPC server (the
// launcher always binds one; we give it a distinct port to avoid
// collisions). Empty => HTTPPort+1.
GRPCPort int `yaml:"grpc_port"`
// DatabaseURL is the sea-orm connection string for DB-backed services
// (stats, multichain-aggregator). A sqlite URL keeps the single binary
// self-contained. Empty for stateless services (sig-provider, visualizer,
// smart-contract-verifier).
DatabaseURL string `yaml:"database_url"`
// Enabled toggles this individual service. Default true when listed.
Enabled *bool `yaml:"enabled"`
// Settings is an optional raw settings overlay merged into the JSON sent
// to the service (service-specific keys beyond addr/db).
Settings map[string]any `yaml:"settings"`
}
// ChainConfig defines a single chain plus per-chain customization. The
+54 -3
View File
@@ -105,6 +105,57 @@ Confirmed upstream entrypoints (all take a `serde::Deserialize` `Settings`):
| multichain-aggregator | `multichain_aggregator_server` | `run(Settings)` |
| visualizer | `visualizer_server` | `run(Settings)` |
DB-backed services (stats, multichain-aggregator) also need their database URL
in the config JSON and migrations run — pass that through `startFFIServices`'s
per-service config map.
## Front router (zip) — how the binary is fronted
The single binary's **public** listener is [`github.com/hanzoai/zip`](../../../hanzo/hanzoai/zip)
(Fiber v3 / fasthttp), wired in `front.go`. Each in-process service keeps its
own HTTP server bound to **loopback** (`127.0.0.1:<port>`, assigned from
`servicePortBase = 8050`, HTTP=base+2n / gRPC=base+2n+1); zip is the only thing
exposed. Routing, in registration order:
```
/api/<prefix>/* → httputil.ReverseProxy → http://127.0.0.1:<svc_http_port> (mountServiceProxies, services.go)
/* → the existing Go-native explorer *http.ServeMux (zip.AdaptNetHTTP, front.go)
```
The proxy strips the `/api/<prefix>` mount prefix, so the upstream sees its own
native path (e.g. `/api/sig/health` → sig-provider `/health`;
`/api/sig/api/v1/abi/function` → `/api/v1/abi/function`). Enabled services +
ports are configured under `services:` in chains.yaml
(`ServicesConfig` → `resolveServices` is the single source of truth both the
proxy and this FFI launcher read). Default prefixes:
`sig-provider→sig`, `smart-contract-verifier→verifier`, `stats→stats`,
`multichain-aggregator→multichain`, `visualizer→visualizer`.
Proven end-to-end (sig-provider, one process owning both :8090 and the service
port): `curl /api/sig/health → {"status":"SERVING"}`, matching a direct hit on
the in-process service port.
## Database: SQLite status per service (verified against explorer-rs)
The brief asked to run DB-backed services on **SQLite** if sea-orm allows it
with minimal effort. It does **not**, here — documented exactly:
| service | DB? | SQLite-able? | status | what it needs |
|-------------------------|-----|--------------|----------|---------------|
| sig-provider | no | n/a | **WIRED + PROVEN** | nothing (stateless) |
| visualizer | no | n/a | ready to wire | nothing (stateless) — clean next target |
| smart-contract-verifier | no | n/a | ready to wire | nothing (stateless) |
| stats | yes | **no** | disabled | its **own** Postgres + a populated **Blockscout indexer Postgres** to read from |
| multichain-aggregator | yes | **no** | disabled | its own Postgres (+ optional replica) |
Why no SQLite: `blockscout_service_launcher::database` is **Postgres-only** —
`initialize_postgres::<Migrator>()` pins `sea_orm::DatabaseBackend::Postgres`
and formats `postgresql://…` DSNs (`libs/blockscout-service-launcher/src/database.rs`
lines 39/67/294). `stats-server` calls that directly (`server.rs:206 init_stats_db`)
**and** opens a second connection to a Blockscout indexer DB
(`server.rs:223 connect_to_indexer_db_common`). Forcing SQLite would mean
patching the launcher's backend selection + every migration — outside "minimal
effort" and explicitly a Rust rewrite the brief forbids. So stats /
multichain-aggregator stay disabled. The Go side already passes a
`database.url` (with `create_database` + `run_migrations`) through
`buildSettingsJSON` whenever `ServiceConfig.DatabaseURL` is set, so the day a
Postgres is provided these turn on with config alone — no code change.
`ServiceConfig.DatabaseURL` in chains.yaml feeds `database.url` in the settings
JSON `startFFIServices` sends; migrations run when present.
+5 -2
View File
@@ -13,5 +13,8 @@ package main
// which variant is running.
const ffiEnabled = false
// startFFIServices is a no-op in the non-FFI build.
func startFFIServices(_ map[string]string) {}
// startFFIServices is a no-op in the non-FFI build: the in-process Rust
// services are simply not launched, so the front router has nothing local to
// proxy to. (The proxies are still mounted by main.go when services.enabled is
// set, and will 502 until a service answers — see services.go.)
func startFFIServices(_ []resolvedService) {}
+33 -15
View File
@@ -60,25 +60,43 @@ func ffiResult(code C.int) string {
}
}
// startFFIServices launches the in-process Rust explorer services. configJSON
// keys the per-service settings (empty => upstream defaults). It is called once
// from main() at startup in the ffi build; the non-FFI build's stub is a no-op.
// ffiStarters maps an FFI service name to its C entrypoint. Adding a service
// is one line here plus its service!{…} block in ffi/src/lib.rs and its
// dep+feature in ffi/Cargo.toml (see ffi/README.md). Disabled services still
// export their symbol and return LUX_FFI_ERR_DISABLED, so this table can list
// every service unconditionally — the Go binary links identically regardless
// of which cargo features were compiled in.
var ffiStarters = map[string]func(*C.char) C.int{
"sig-provider": func(c *C.char) C.int { return C.lux_explorer_start_sig_provider(c) },
// Scale-out (uncomment as the matching service!{…} is enabled in ffi/):
// "smart-contract-verifier": func(c *C.char) C.int { return C.lux_explorer_start_smart_contract_verifier(c) },
// "stats": func(c *C.char) C.int { return C.lux_explorer_start_stats(c) },
// "multichain-aggregator": func(c *C.char) C.int { return C.lux_explorer_start_multichain_aggregator(c) },
// "visualizer": func(c *C.char) C.int { return C.lux_explorer_start_visualizer(c) },
}
// startFFIServices launches each resolved in-process Rust service by calling
// its C entrypoint with the settings JSON resolveServices rendered (server
// addr/port + optional DB url). It is called once from main() at startup in
// the ffi build; the non-FFI build's stub is a no-op.
//
// Each starter returns immediately after spawning the service's worker thread,
// so this does not block the Go HTTP listener.
func startFFIServices(configJSON map[string]string) {
start := func(name string, fn func(*C.char) C.int) {
cfg := configJSON[name]
// so this does not block the zip HTTP listener.
func startFFIServices(svcs []resolvedService) {
for _, s := range svcs {
fn, ok := ffiStarters[s.Name]
if !ok {
log.Printf("[ffi] no entrypoint for service %q (rebuild ffi with its feature?)", s.Name)
continue
}
var cstr *C.char
if cfg != "" {
cstr = C.CString(cfg)
defer C.free(unsafe.Pointer(cstr))
if s.SettingsJSON != "" {
cstr = C.CString(s.SettingsJSON)
}
code := fn(cstr)
log.Printf("[ffi] start %s -> %s (%d)", name, ffiResult(code), int(code))
if cstr != nil {
C.free(unsafe.Pointer(cstr))
}
log.Printf("[ffi] start %s on :%d -> %s (%d)", s.Name, s.HTTPPort, ffiResult(code), int(code))
}
start("sig-provider", func(c *C.char) C.int { return C.lux_explorer_start_sig_provider(c) })
// As services are scaled out in ffi/, add their starters here, e.g.:
// start("stats", func(c *C.char) C.int { return C.lux_explorer_start_stats(c) })
}
+65
View File
@@ -0,0 +1,65 @@
package main
// front.go — zip is the FRONT HTTP router of the explorer binary.
//
// One zip.App owns the public listener. It composes two things, in order:
//
// 1. /api/<prefix>/* reverse-proxies to each in-process Blockscout-rs
// service (mountServiceProxies, services.go). Registered FIRST so Fiber's
// in-order route matching sends these to the Rust services.
//
// 2. /* catch-all → the existing Go-native explorer (the *http.ServeMux
// built in main.go: SPA, indexer, graph, realtime, admin). Adapted onto
// zip via zip.AdaptNetHTTP — the stdlib mux is mounted whole, unchanged.
//
// This is the LEAST-WORK front: the explorer's net/http routes are reused
// verbatim, the Rust services keep their own HTTP servers, and zip just
// routes + proxies. The single seam to the in-process Rust world is the
// reverse-proxy target host:port — nothing in the Go HTTP layer knows or
// cares that the upstream is a cgo-launched Tokio thread.
import (
"net/http"
"github.com/hanzoai/zip"
luxlog "github.com/luxfi/log"
)
// buildFrontApp constructs the zip front router. base is the fully-wired
// explorer mux (SPA + indexer + graph + realtime + admin) from main.go; svcs
// are the resolved in-process services to reverse-proxy. The returned App owns
// the public listener (App.Listen).
func buildFrontApp(base http.Handler, svcs []resolvedService) *zip.App {
app := zip.New(zip.Config{
AppName: "lux-explorer",
ServerHeader: "explorer",
DisableStartupMessage: true,
Logger: luxlog.New("module", "explorer"),
})
// Baseline security headers + permissive CORS, mirroring the previous
// withSecurity wrapper. Runs on every request including the proxied ones.
app.Use(func(c *zip.Ctx) error {
c.SetHeader("X-Content-Type-Options", "nosniff")
c.SetHeader("X-Frame-Options", "DENY")
c.SetHeader("Referrer-Policy", "strict-origin-when-cross-origin")
c.SetHeader("Access-Control-Allow-Origin", "*")
c.SetHeader("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
c.SetHeader("Access-Control-Allow-Headers", "Content-Type, Authorization")
if c.Method() == "OPTIONS" {
return c.NoContent(204)
}
return c.Continue()
})
// (1) Per-service reverse proxies — registered before the catch-all so
// /api/<prefix>/* wins over /*.
mountServiceProxies(app, svcs)
// (2) The whole existing explorer mux as the catch-all. zip.AdaptNetHTTP
// turns the *http.ServeMux into a zip handler; All("/*") makes it the
// fallback for everything the proxies above didn't claim.
app.All("/*", zip.AdaptNetHTTP(base))
return app
}
+10 -1
View File
@@ -4,8 +4,10 @@ go 1.26.4
require (
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674
github.com/hanzoai/zip v0.0.0-00010101000000-000000000000
github.com/luxfi/graph v1.2.6
github.com/luxfi/indexer v1.4.6
github.com/luxfi/log v1.4.3
github.com/luxfi/mdns v0.1.1
github.com/mattn/go-sqlite3 v2.0.3+incompatible
gopkg.in/yaml.v3 v3.0.1
@@ -13,6 +15,7 @@ require (
require (
filippo.io/hpke v0.4.0 // indirect
github.com/andybalholm/brotli v1.2.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
@@ -22,6 +25,9 @@ require (
github.com/go-chi/chi/v5 v5.3.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gofiber/fiber/v3 v3.2.0 // indirect
github.com/gofiber/schema v1.7.1 // indirect
github.com/gofiber/utils/v2 v2.0.4 // indirect
github.com/google/flatbuffers v25.12.19+incompatible // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/mux v1.8.1 // indirect
@@ -48,7 +54,6 @@ require (
github.com/luxfi/crypto v1.19.17 // indirect
github.com/luxfi/database v1.19.0 // indirect
github.com/luxfi/ids v1.2.13 // indirect
github.com/luxfi/log v1.4.3 // indirect
github.com/luxfi/math v1.4.1 // indirect
github.com/luxfi/math/big v0.1.0 // indirect
github.com/luxfi/metric v1.5.8 // indirect
@@ -75,6 +80,8 @@ require (
github.com/rs/xid v1.6.0 // indirect
github.com/tetratelabs/wazero v1.12.0 // indirect
github.com/tinylib/msgp v1.6.4 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.70.0 // indirect
github.com/wasilibs/go-re2 v1.11.0 // indirect
github.com/wasilibs/wazero-helpers v0.0.0-20250123031827-cd30c44769bb // indirect
github.com/zeebo/xxh3 v1.1.0 // indirect
@@ -100,3 +107,5 @@ require (
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.51.0 // indirect
)
replace github.com/hanzoai/zip => ../../../hanzo/hanzoai/zip
+20 -4
View File
@@ -22,6 +22,8 @@ github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 h1:oygO0locgZJ
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.3.0 h1:wQlqotpyjYPjJz+Noh5bRu7Snmydk8SKC5Z6u1CR20Y=
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.3.0/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
github.com/andybalholm/brotli v1.2.1 h1:R+f5xP285VArJDRgowrfb9DqL18yVK0gKAW/F+eTWro=
github.com/andybalholm/brotli v1.2.1/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/aws/aws-sdk-go-v2 v1.41.0 h1:tNvqh1s+v0vFYdA1xq0aOJH+Y5cRyZ5upu6roPgPKd4=
github.com/aws/aws-sdk-go-v2 v1.41.0/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 h1:489krEF9xIGkOaaX3CE/Be2uWjiXrkCH6gUX+bZA/BU=
@@ -84,6 +86,8 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/fxamacker/cbor/v2 v2.9.1 h1:2rWm8B193Ll4VdjsJY28jxs70IdDsHRWgQYAI80+rMQ=
github.com/fxamacker/cbor/v2 v2.9.1/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
github.com/go-chi/chi/v5 v5.3.0 h1:halUjDxhshgXHMrao5bB8eNBXo/rnzwr8m5m36glehM=
github.com/go-chi/chi/v5 v5.3.0/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
@@ -91,6 +95,12 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/gofiber/fiber/v3 v3.2.0 h1:g9+09D320foINPpCnR3ibQ5oBEFHjAWRRfDG1te54u8=
github.com/gofiber/fiber/v3 v3.2.0/go.mod h1:FHOsc2Db7HhHpsE62QAaJlXVV1pNkbZEptZ4jtti7m4=
github.com/gofiber/schema v1.7.1 h1:oSJBKdgP8JeIME4TQSAqlNKTU2iBB+2RNmKi8Nsc+TI=
github.com/gofiber/schema v1.7.1/go.mod h1:A/X5Ffyru4p9eBdp99qu+nzviHzQiZ7odLT+TwxWhbk=
github.com/gofiber/utils/v2 v2.0.4 h1:WwAxUA7L4MW2DjdEHF234lfqvBqd2vYYuBtA9TJq2ec=
github.com/gofiber/utils/v2 v2.0.4/go.mod h1:GGERKU3Vhj5z6hS8YKvxL99A54DjOvTFZ0cjZnG4Lj4=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
@@ -179,14 +189,10 @@ github.com/luxfi/database v1.19.0 h1:sySH5eS8ihJtyc+yb0LgTignGwJ9H4W9bv+nboCuIm0
github.com/luxfi/database v1.19.0/go.mod h1:o1ccjgVKMYbVBfi+6+sDvx4swGFgGnCGFQ4gL/oJ+kY=
github.com/luxfi/geth v1.16.98 h1:w187TtKuGStf3tm2bshuHVKBv2Frjx0lT54kQVXyNHA=
github.com/luxfi/geth v1.16.98/go.mod h1:6kEzSExdk9CPQDPXALt6P3HfQqBq7KF1Jrrv9gBpxbU=
github.com/luxfi/graph v1.2.3 h1:cgHiBU8vBO4b39HnVr1UewEAeP3Bx+JZBHlfacTvDZQ=
github.com/luxfi/graph v1.2.3/go.mod h1:NbKTixz6Miunyh+7ER3G389mzUKV22QIVPjpcTnKZuY=
github.com/luxfi/graph v1.2.6 h1:iapa/PB2UqLzHyAT5bM8173PfRISDJSbBYFuVcjcPJY=
github.com/luxfi/graph v1.2.6/go.mod h1:NbKTixz6Miunyh+7ER3G389mzUKV22QIVPjpcTnKZuY=
github.com/luxfi/ids v1.2.13 h1:lEotq0WUpMLcc/5X5MbE3n73jdfNo8IlFvTDfUXg+9A=
github.com/luxfi/ids v1.2.13/go.mod h1:QWjJvggX/nRkmrz1AGSaF2Y6mnOvz3g7lFOzs3Zd+JM=
github.com/luxfi/indexer v1.4.5 h1:vfYgM/hnNujjS47B7/VKAxFJHsFhhME0SnLaAAaYuwA=
github.com/luxfi/indexer v1.4.5/go.mod h1:pz5W+DqKx0La2uyq1mV5Kpi0gXXNhmVpZTZRmhw+fX4=
github.com/luxfi/indexer v1.4.6 h1:DytagtI2+99jMD+4JU3MRQuLdZiZjtHOCi8ukSehz2g=
github.com/luxfi/indexer v1.4.6/go.mod h1:pz5W+DqKx0La2uyq1mV5Kpi0gXXNhmVpZTZRmhw+fX4=
github.com/luxfi/log v1.4.3 h1:xkUKRWvQ4ZwvlUC2e0/RTtHYZOYSMvSQ9W9lbjwBmiI=
@@ -268,6 +274,8 @@ github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0t
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/shamaton/msgpack/v3 v3.1.0 h1:jsk0vEAqVvvS9+fTZ5/EcQ9tz860c9pWxJ4Iwecz8gU=
github.com/shamaton/msgpack/v3 v3.1.0/go.mod h1:DcQG8jrdrQCIxr3HlMYkiXdMhK+KfN2CitkyzsQV4uc=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -283,10 +291,18 @@ github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZ
github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0=
github.com/tinylib/msgp v1.6.4 h1:mOwYbyYDLPj35mkA2BjjYejgJk9BuHxDdvRnb6v2ZcQ=
github.com/tinylib/msgp v1.6.4/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.70.0 h1:LAhMGcWk13QZWm85+eg8ZBNbrq5mnkWFGbHMUJHIdXA=
github.com/valyala/fasthttp v1.70.0/go.mod h1:oDZEHHkJ/Buyklg6uURmYs19442zFSnCIfX3j1FY3pE=
github.com/wasilibs/go-re2 v1.11.0 h1:9j4uCAJyO3IJHygeOdUJNzm6sX3P1+3xuKTbCITQrvc=
github.com/wasilibs/go-re2 v1.11.0/go.mod h1:cDOlK/pNYGbGKRNa+NnXDJ1c8R63tWjnTjFKzoSbLv0=
github.com/wasilibs/wazero-helpers v0.0.0-20250123031827-cd30c44769bb h1:gQ+ZV4wJke/EBKYciZ2MshEouEHFuinB85dY3f5s1q8=
github.com/wasilibs/wazero-helpers v0.0.0-20250123031827-cd30c44769bb/go.mod h1:jMeV4Vpbi8osrE/pKUxRZkVaA0EX7NZN0A9/oRzgpgY=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
+23 -30
View File
@@ -119,15 +119,19 @@ func main() {
go registry.hub.Run(ctx)
go supervisor.Wait(ctx)
// In the `-tags ffi` build this launches the Blockscout-rs explorer
// services (sig-provider, …) in-process via cgo on their own Tokio
// runtime threads — one binary runs everything. In the default build
// startFFIServices is a no-op (see ffi_off.go). Per-service settings are
// keyed by service name; empty => the service's upstream defaults / env.
if ffiEnabled {
log.Printf("[explorer] ffi: starting in-process Rust services")
// Resolve the in-process Blockscout-rs services from config: concrete
// ports, URL prefixes, and per-service settings JSON. This is the single
// source of truth both the FFI launcher and the zip reverse-proxy read
// from (services.go). Empty when services.enabled is false.
svcs := resolveServices(cfg.Services)
// In the `-tags ffi` build this launches each resolved service in-process
// via cgo on its own Tokio runtime thread — one binary runs everything.
// In the default build startFFIServices is a no-op (see ffi_off.go).
if ffiEnabled && len(svcs) > 0 {
log.Printf("[explorer] ffi: starting %d in-process Rust service(s)", len(svcs))
}
startFFIServices(map[string]string{})
startFFIServices(svcs)
frontend, err := NewFrontend(cfg, registry)
if err != nil {
@@ -188,39 +192,28 @@ func main() {
supervisor.MountRoutes(mux)
frontend.Mount(mux)
log.Printf("[explorer] %s listening %s data=%s chains=%d mdns=%v %s",
version, cfg.HTTPAddr, cfg.DataDir, registry.Count(), *enableMDNS, fingerprint())
// zip is the FRONT router (front.go): it reverse-proxies /api/<prefix>/*
// to each in-process service and serves everything else from the explorer
// mux built above. When no services are configured the front app is just
// the mux behind zip's security middleware — identical behaviour, one
// router.
app := buildFrontApp(mux, svcs)
log.Printf("[explorer] %s listening %s data=%s chains=%d services=%d mdns=%v %s",
version, cfg.HTTPAddr, cfg.DataDir, registry.Count(), len(svcs), *enableMDNS, fingerprint())
server := &http.Server{Addr: cfg.HTTPAddr, Handler: withSecurity(mux)}
go func() {
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
if err := app.Listen(cfg.HTTPAddr); err != nil {
log.Fatalf("[explorer] server: %v", err)
}
}()
<-ctx.Done()
shutCtx, shutCancel := context.WithTimeout(context.Background(), 10*time.Second)
defer shutCancel()
server.Shutdown(shutCtx)
_ = app.ShutdownWithContext(shutCtx)
log.Println("[explorer] stopped")
}
// withSecurity adds the baseline security headers and CORS preflight handler.
func withSecurity(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("X-Frame-Options", "DENY")
w.Header().Set("Referrer-Policy", "strict-origin-when-cross-origin")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next.ServeHTTP(w, r)
})
}
func env(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
+195
View File
@@ -0,0 +1,195 @@
package main
// services.go — front-router wiring for the in-process Blockscout-rs services.
//
// This file is build-tag agnostic: it is compiled into BOTH the default
// (Go-native) binary and the `-tags ffi` single binary. It owns three
// decomplected concerns, none of which touch cgo:
//
// 1. RESOLVE — turn ServicesConfig (chains.yaml) into a normalized slice of
// resolvedService with concrete ports, prefixes, and a settings JSON.
// 2. SETTINGS — build the JSON each service's upstream `Settings` deserializes
// (server.http.addr / server.grpc.addr, and DB url where applicable).
// 3. PROXY — mount a reverse-proxy on the zip front App at
// /api/<prefix>/* → http://127.0.0.1:<http_port>.
//
// The actual in-process LAUNCH (cgo → liblux_explorer_ffi.a) is the ONLY part
// that differs by build tag and lives in ffi_on.go / ffi_off.go. That split is
// the single seam between "what runs in-process" (Rust, cgo) and "how it is
// fronted" (Go, zip) — orthogonal and complete.
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"strings"
"github.com/hanzoai/zip"
)
// servicePortBase is where auto-assigned in-process service HTTP ports start.
// Each service gets HTTP=base+2n, gRPC=base+2n+1, so they never collide with
// each other or with the explorer's own listener (:8090 by default). The
// upstream blockscout-service-launcher default is 0.0.0.0:8050 (http) /
// :8051 (grpc); we keep that family but bind to 127.0.0.1 (loopback only —
// the front router is the sole public surface) and fan out per service.
const servicePortBase = 8050
// defaultPrefix maps an FFI service name to its default URL prefix. The prefix
// is the path segment the SPA / clients hit: /api/<prefix>/*. Kept short and
// stable so front-end calls don't churn when a service is renamed upstream.
var defaultPrefix = map[string]string{
"sig-provider": "sig",
"smart-contract-verifier": "verifier",
"stats": "stats",
"multichain-aggregator": "multichain",
"visualizer": "visualizer",
}
// resolvedService is a ServiceConfig with every field made concrete: ports
// assigned, prefix derived, and the settings JSON the FFI entrypoint will
// receive already rendered. Both the proxy layer and the FFI launch layer
// consume this — one resolution, two uses (DRY).
type resolvedService struct {
Name string
Prefix string
HTTPPort int
GRPCPort int
SettingsJSON string
}
// resolveServices normalizes ServicesConfig into concrete resolvedService
// records. Disabled or empty-named entries are dropped. Ports auto-assign in
// listing order from servicePortBase when not pinned. This is the single
// source of truth both the proxy and the FFI launcher read from.
func resolveServices(cfg ServicesConfig) []resolvedService {
if !cfg.Enabled {
return nil
}
out := make([]resolvedService, 0, len(cfg.Services))
next := servicePortBase
for _, s := range cfg.Services {
if s.Name == "" {
continue
}
if s.Enabled != nil && !*s.Enabled {
continue
}
httpPort := s.HTTPPort
grpcPort := s.GRPCPort
if httpPort == 0 {
httpPort = next
}
if grpcPort == 0 {
grpcPort = httpPort + 1
}
// Advance the auto-allocator past whatever this service consumed so
// the next unpinned service doesn't collide.
if httpPort >= next {
next = httpPort + 2
}
prefix := s.Prefix
if prefix == "" {
if p, ok := defaultPrefix[s.Name]; ok {
prefix = p
} else {
prefix = s.Name
}
}
out = append(out, resolvedService{
Name: s.Name,
Prefix: strings.Trim(prefix, "/"),
HTTPPort: httpPort,
GRPCPort: grpcPort,
SettingsJSON: buildSettingsJSON(s, httpPort, grpcPort),
})
}
return out
}
// buildSettingsJSON renders the JSON the service's upstream `Settings`
// deserializes. Every wrapped service is launcher-based and shares the
// `server.http.addr` / `server.grpc.addr` shape; DB-backed services also take
// `database.url`. We bind every service to 127.0.0.1 (loopback) so the zip
// front router is the only thing exposed. Service-specific overrides from
// ServiceConfig.Settings are merged last so they win.
func buildSettingsJSON(s ServiceConfig, httpPort, grpcPort int) string {
m := map[string]any{
"server": map[string]any{
"http": map[string]any{
"enabled": true,
"addr": fmt.Sprintf("127.0.0.1:%d", httpPort),
},
"grpc": map[string]any{
// gRPC stays enabled but on loopback; the front router only
// proxies HTTP. Disabling it entirely is service-dependent, so
// we just bind it somewhere harmless.
"enabled": true,
"addr": fmt.Sprintf("127.0.0.1:%d", grpcPort),
},
},
// Quiet the per-service metrics listener onto loopback too; it defaults
// to 0.0.0.0:6060 which would collide if two services start.
"metrics": map[string]any{
"enabled": false,
"addr": fmt.Sprintf("127.0.0.1:%d", grpcPort+1),
},
}
if s.DatabaseURL != "" {
// sea-orm services (stats, multichain-aggregator) read database.url.
// A sqlite URL (sqlite:///path?mode=rwc) keeps the binary self-contained.
m["database"] = map[string]any{
"url": s.DatabaseURL,
"create_database": true,
"run_migrations": true,
}
}
for k, v := range s.Settings {
m[k] = v
}
b, err := json.Marshal(m)
if err != nil {
// map[string]any of strings/ints/bools never fails to marshal; keep a
// safe fallback so a typo can't crash startup.
return "{}"
}
return string(b)
}
// mountServiceProxies registers a reverse-proxy on the zip front App for each
// resolved in-process service: /api/<prefix>/* → http://127.0.0.1:<http_port>.
//
// We strip the /api/<prefix> mount prefix before forwarding so the upstream
// service sees the path it expects (e.g. /api/v1/abi/... not
// /api/sig/api/v1/abi/...). zip.AdaptNetHTTP wraps the stdlib
// httputil.ReverseProxy onto a zip route — the simplest thing that works, per
// the brief: no bespoke fasthttp proxy, no escape hatch.
//
// Mounted via app.Mount which registers `All(prefix+"/*")`. These are
// registered BEFORE the catch-all explorer mux (see buildFrontApp) so Fiber's
// in-order matching routes /api/<prefix>/* to the service and everything else
// to the explorer.
func mountServiceProxies(app *zip.App, svcs []resolvedService) {
for _, s := range svcs {
target := &url.URL{Scheme: "http", Host: fmt.Sprintf("127.0.0.1:%d", s.HTTPPort)}
mountPrefix := "/api/" + s.Prefix
rp := httputil.NewSingleHostReverseProxy(target)
// Strip the /api/<prefix> mount prefix so the upstream sees its own
// native path. NewSingleHostReverseProxy's default Director only sets
// the host; we extend it to rewrite the path.
base := rp.Director
stripped := mountPrefix
rp.Director = func(req *http.Request) {
base(req)
p := strings.TrimPrefix(req.URL.Path, stripped)
if p == "" {
p = "/"
}
req.URL.Path = p
req.Host = target.Host
}
app.Mount(mountPrefix, rp)
}
}