graph/LLM.md

graph

Org: luxfi · Ecosystem: lux · Path: /Users/a/work/lux/luxfi/graph Origin: https://github.com/luxfi/graph.git

Discovery

This file (CLAUDE.md) is the canonical agent-facing readme; LLM.md is a symlink to it. Update either name and both stay in sync.

Where to look first

• README.md — human-facing overview (if present)
• package.json / Cargo.toml / pyproject.toml / go.mod — language & deps
• .github/workflows/ — CI surface
• docs/ — extended docs (if present)

A token's totalSupply is WHOLE TOKENS, and a named row is not a finished one

Two things about Token that a reader has to know before touching erc20.go:

• One unit. SeedTokenData.TotalSupply (and Staked) is whole tokens as an exact decimal string, never base units. The contract reports base units and publishNativeSupply reports a genesis figure already whole; both land in the same field, and a client that scales one and not the other is a valuation 10^18 out. wholeUnits does the conversion, with integer division and the remainder — float64 gives up around 2^53, four orders of magnitude below a two-trillion-token supply. The exchange multiplies this by a price directly.
• One predicate. settled(row, addr) — a real symbol AND a supply — decides whether a row still has anything to ask the chain. token() is the only resolver: store first, contract for the rest, memoised per address for the process, persisted once. BackfillTokens asks the same predicate at every start, so a row is completed by starting rather than by remembering a flag. Keying "finished" on the symbol alone is what left every token indexed before v1.2.25 supply-less forever: the row had a name, so nothing asked again, and market cap and fully diluted value printed a dash however much it traded.

resolve folds a read over the stored row and never erases: a reverting call must not downgrade a real symbol, and SeedToken is INSERT OR REPLACE, so a write built only from the contract read zeroes the value locked, volume, price and trade count the valuation pass put on that same row.

🪤 The sqlite driver is hanzoai/csqlite, opened as "sqlite3". Not upstream mattn (a second copy of the same amalgamation — every sqlite3_* symbol twice, a link failure on darwin), and not the hanzoai/sqlite wrapper, which registers the name "sqlite" — as does modernc.org/sqlite, which arrives here with hanzoai/replicate next door in storage. Two packages under one name is a panic in database/sql init, before main runs: v1.2.28 through v1.2.30 could not start at all.

A swap's timestamp is a TIME, and the poll is what makes it one

eth_getLogs says which block a log was in and never when it was mined. Until v1.2.17 the swap handlers wrote the block NUMBER into timestamp, so trade history dated every fill to 1970 and no day could be computed from a swap at all. stampTimes (blocks.go) now reads one batch of headers per poll and hangs the time on the logEntry, and poll returns an error rather than advancing past a block whose header did not answer — a block is indexed when it has been read whole.

SeedSwapData.Block is what tells an old row from a new one: no swap is mined in block 0, so Block == 0 means "the number in Timestamp is a block number", which is exactly what healSwapTimes needs to look the real time up. That is Dated(), and both the heal and the day series ask it. The heal runs once at Run and stops when every row has a time.

A swap's columns are its queryable projection, and all-time volume is a SUM

swaps is (id, data JSON, timestamp, pool, amountUSD). The three scalars are copies of fields inside data, promoted so the database can answer questions Go would otherwise answer by reading the table into memory: a window (timestamp), a pool's own history (pool), and what the protocol has traded (amountUSD). SeedSwap and ValueSwaps write both homes; Store.Traded() is the only reader of the sum. 🪤 There is no json_extract escape hatch — this driver's amalgamation is built without JSON1, which is why the column had to exist at all.

A database written before the column gets it in Init, filled once from the rows themselves; the ALTER failing is how a later start knows not to do it again. Arriving empty would have zeroed every trade older than the current window — the fix erasing the number it exists to correct.

A pool's volume and the protocol's are one question sliced two ways, so TradedByPool() answers both and the total is the fold. Asked separately they drifted fifty times apart — the protocol summing every trade ever indexed while each pool summed the window a pass had just read.

The trade count comes from the same read as the volume beside it. Pool, token and factory counts are all folds of TradedByPool(); bumpSwapCounts and bumpFactory are gone. A tally kept as events arrive drifts on every restart and re-index — the protocol claimed 16,062 trades over pools holding 14,827, and a token reported $74,100 of volume next to a count of zero. txCount is therefore TRADES, not every interaction: mints and burns no longer count toward it, because a reader comparing that column to the volume column is owed the number that produced it. What a create handler owns is the registration; the counts are derived, like poolCount before them.

Total volume is not the pass's total. maxValuedSwaps bounds what one valuation pass values, because the swap table grows forever. Summing that window answers for the newest trades only. Summing the day series does not escape it either: those rows are folded from the same window, so their sum IS the window — a fix that looked right and changed nothing. Factory.txCount is a separate question (every interaction, mints and burns included) and stays with the handlers that see them.

A pass reads BOTH ENDS of what it has not seen. The head (RecentSwapsRaw) keeps arriving trades priced. The tail (SwapsAfter) walks forward from PricedThrough, a mark in meta, so a table longer than any window is covered a batch at a time and then never asked about again. It is a mark and not a "which rows are still zero" query on purpose: a trade whose tokens reach no stablecoin can NEVER be priced, and that query would hand back the same unpriceable trades every pass forever. Having looked is monotone; having valued is not. The mark moves only after ValueSwaps returns without error.

wholeDays drops the day each read's own cut falls in, because a day row is REPLACED by the pass that writes it and half a day would land where a whole one was. A read that came back short of its limit was not cut and keeps everything — that condition is the whole test; without it the pass discards its oldest real day on every chain small enough to fit in one window.

The day series (series.go) is a SECOND FOLD over the valuation pass's trades

tokenDayDatas / poolDayDatas / uniswapDayDatas are rolled out of the swaps already on disk, which is why history appears the moment a build ships instead of starting that day. valuedTrades reads and values the swap window ONCE; valueSwaps folds it into running totals and writeSeries folds the same stream into days. Do not add a second read or a second pricing path — valuation.go owns pricing (see its header).

• What is exact. A pool's candle is its own execution ratio amount0/amount1. A token's is that ratio carried to USD through the OTHER leg, so it is exact against a stablecoin and exact in shape against a volatile one. Pricing a token off its OWN leg yields a flat line at today's price — that mutation is covered by a test.
• What is not. Value locked is not in the trades. Each day records it while it is today; a day that passed before the series existed reports "", never "0.00" (fmtHeld) — zero would draw a cliff to no liquidity.
• Write only what moved. A pass recomputes the whole history every 30s; idx.written remembers each cell as persisted so the steady state is one write per subject that traded. Rewriting everything is the O(rows) storm that once stopped this pass from finishing.
• Rows are entities (storage.TokenDay / PoolDay / FactoryDay) built from the engine structs, so the json tags ARE the wire. Store.Entities reads a whole type then filters/orders/limits — a SQL LIMIT before the filter would answer with the first N rows rather than the first N matches.
• The client draws nothing under three points; fewer renders "Missing chart data", the same as an error. pairDayDatas and the hour series stay stubs — no client asks for them.

where learned two things a chart needs: a filter naming a reference ({token: "0x…"}) compares against the referenced entity's id, and an exact text match ignores case, because an address arrives checksummed as often as lower-cased.

DEX indexing has TWO orthogonal sources (EVM 0x9999 logs + native D-Chain CLOB)

The dex subgraph (markets/fills/orders/orderbook) can be fed from either or both of:

1. EVM 0x9999 settlement logs — the cross-chain SETTLEMENT view. handleDEXFill/ handleInitializeV4 derive Market/Fill from DEXFill/Initialize logs emitted by the 0x9999 PoolManager on an EVM chain (C-Chain). Source = indexer.go eth_getLogs.
2. Native D-Chain CLOB read RPC — the TRADING view. On a native-DEX deployment the trade engine IS the D-Chain (dexvm): a trade is a consensus state transition at Block.Verify, NOT an EVM event, so it NEVER appears on the EVM RPC. CLOBSource (indexer/clob.go) polls the committed-state read RPC …/v1/chain/<D>/dex/clob_get_{markets,trades,orders} (GET/JSON, READ-ONLY, zero consensus impact — see luxfi/dex pkg/dchain/read.go) and writes the SAME Market/Fill/Order entities the resolvers serve. Enabled by Config.DexRPC (graph) / dex_rpc: per-chain (explorer chains.yaml); empty = source 1 only.

The two sources are ORTHOGONAL — distinct loop, distinct protocol, distinct endpoint; they share only the store (the sink) and the entity shapes (the contract). Running the CLOB source NEVER regresses AMM/EVM indexing (the amm subgraph never gets dex_rpc). Both Markets key by poolID, so when both run they converge on one Market per pool: writeMarket refreshes the live CLOB book summary and MERGES through any EVM-accrued volume24h/tradeCount/lastPrice it does not own.

Markets serve on the same host under distinct slugs (.../cchain/amm/graphql, .../cchain/dex/graphql).

Native CLOB source notes (clob.go)

• Fills are a GLOBAL feed. The 80-byte on-chain trade row carries no poolID (frozen GPU ABI), so clob_get_trades is a global height-ordered log. writeTrade records every fill as a Fill (the trade-history view) keyed height:seq (deterministic → idempotent re-read). Per-market figures (open orders, depth, best bid/ask, remaining) come from clob_get_markets' authoritative per-market book summary — we do NOT fabricate a per-market volume by mis-attributing the global log.
• Incremental. drainTrades pages by since=lastHeight+1; a steady chain costs one small query per tick. Symbol decode: a bound market's hex symbol (4c55… = ASCII LUX/LUSD) → human pair; an unbound poolID stays hex.

Native DEX indexing (0x9999) — the EVM-settlement source detail

• Single Fill source. A settled fill is recorded ONLY from the authoritative DEXFill event in handleDEXFill. handleSwapV4 is the AMM-side view and does NOT write a Fill. Do not add a second fill source.
• Emitter-scoped. handleDEXFill drops any DEXFill not emitted by the configured PoolManager (0x9999) — !isPoolManager(l.Address) returns early. This is the anti-spoof control; keep it.
• Topic0 hashes (indexer/events.go) are keccak256 of the EXACT signatures the luxfi/precompile emits (see ~/work/lux/precompile/dex/events.go). The Lux Swap carries a trailing uint24 fee — NOT the upstream Uniswap-v4 signature. DEXFill = keccak256("DEXFill(bytes32,address,uint256,uint256)"). If you change a signature in the precompile, recompute the hash here or it matches nothing.
• Re-genesis self-heal rewinds the cursor only on a confirmed re-genesis (changed genesis-block hash) after a hysteresis streak — never on a transient low-head blip.

Deployment topology — TWO exchange-api images (do not conflate)

• ghcr.io/luxfi/graph (this repo, Dockerfile → graphd) backs the explorer. Built by .github/workflows/docker.yml on v* tags, self-hosted ARC pool.
• ghcr.io/luxfi/exchange-api has TWO independent source/version lines:
  • v2.2.x ← repo luxfi/exchange-api (TypeScript, /v1/graphql with amm/dex routing + DEX_SUBGRAPH_URL). Deployed in lux-explorer. This is the markets API.
  • v5.x ← repo luxfi/exchange Dockerfile.api (server.js, REST /v1/tokens|pools|trades|price + /subgraph/*). Deployed in lux-ns. These are NOT the same codebase. A change to one does not reach the other's image.

Sibling repos

See the org-level LLM.md at /Users/a/work/lux/luxfi/LLM.md for the full inventory of sibling repos and inter-repo dependencies.