One request the edge permits was a thousand questions it did not

A price is a fan-out. The edge limits requests — thirty a second from one
address — and each may name fifty tokens, read at two sizes against two
numéraires across every arm. Nothing bounded what came out the other side, and
what comes out lands on public endpoints we hold no account with; being refused
there takes quoting down for everybody, on every chain.

So the budget is sixteen questions in flight against one chain for the whole
process, rather than eight per request multiplied by however many requests
arrive. It is taken around a single reading and not around a whole price,
because a price against the hub waits for the hub's own price, and a slot held
across that wait deadlocks the first time every slot is held by something
waiting on the hub.

Measured: one request of 50 cold Ethereum tokens went from 10.1s to 8.6s, and
four chains of 40 at once from 10.7s to 9.3s — and returned the same counts on
a repeat, where the same burst before this returned a different set of prices
each time.

Co-authored-by: Hanzo Dev <[email protected]>

LLM.md

@@ -419,8 +419,19 @@ Cross-chain, same minute, against an independent source:
 wrong number. `venue` and `via` go missing with it, because a number nobody can
 trace is a number nobody can check.
 
-Cost: 50 cold tokens on Ethereum in 10.1s, the same 50 warm in 36ms. The two
+**The bound is on what leaves, not on what arrives.** The edge limits requests —
-sizes are asked concurrently, so the second one costs calls and not latency.
+thirty a second from one address — and each may name fifty tokens read at two
+sizes against two numéraires across every arm, so one permitted request becomes
+up to a thousand eth_calls landing on public endpoints we hold no account with.
+`readingsInFlight` is sixteen questions in flight against ONE chain for the
+whole process, not per request, so a second screen shares them instead of
+doubling them.
+
+Cost, measured: 50 cold Ethereum tokens in 8.6s, the same 50 warm in 36ms, and
+160 tokens across four chains at once in 9.3s — and the same counts on a repeat
+run, where before the bound a four-chain burst returned a different set of
+prices each time. The two sizes are asked concurrently, so the second one costs
+calls and not latency.
 
 A reading stands for a minute (`markTTL`) and `asOf` says when it was taken; a
 chain that could not be read is remembered for ten seconds, not a minute,

api/openapi.yaml

@@ -192,7 +192,8 @@ paths:
             Repeated or comma-separated, up to fifty, and answered in the order
             asked. Fifty because each one is a fan-out of eth_calls against a
             public endpoint; a screen wanting more asks again, and by then most
-            of them are already read.
+            of them are already read. Fifty cold tokens take about nine seconds
+            and the same fifty again take milliseconds.
       responses:
         '200':
           description: One entry per token asked, in that order.

pkg/gateway/price.go

@@ -24,6 +24,20 @@ import (
 // $0.036834 and /v1/trade/quote for the same size implies $0.036806. A screen
 // that says one thing and a panel that pays another is worse than either.
 
+// readingsInFlight is how many questions this process will have in flight
+// against ONE chain at a time.
+//
+// A price is a fan-out, and the edge limits REQUESTS: thirty a second from one
+// address, each of which may name fifty tokens, each read at two sizes against
+// two numéraires across every arm. One permitted request becomes up to a
+// thousand eth_calls, and they land on public endpoints we hold no account
+// with. Getting our egress refused there takes quoting down for everybody, so
+// the bound is on what leaves this process rather than on what arrives at it.
+//
+// Sixteen is what a single request already used, so one screen is no slower;
+// what changes is that a second screen shares them instead of doubling them.
+const readingsInFlight = 16
+
 // markSmall and markLarge are the two sizes a reading is taken at, in dollars.
 //
 // Not one whole token: one whole token is a fifty-cent trade in one market and
@@ -319,6 +333,19 @@ func (s *Server) readAgainst(ctx context.Context, chain ChainID, num Token, numU
 		return nil, nil
 	}
 
+	// Wait for room to ask. Taken here and not around a whole reading, because
+	// a reading against the hub waits for the hub's own reading, and a slot
+	// held across that wait is a deadlock the first time every slot is held by
+	// something waiting on the hub.
+	if room := s.reading[chain]; room != nil {
+		select {
+		case room <- struct{}{}:
+			defer func() { <-room }()
+		case <-ctx.Done():
+			return nil, ctx.Err()
+		}
+	}
+
 	held, err := venues.QueryAllVenues(ctx, VenueQuoteRequest{
 		TokenIn:  num.Address,
 		TokenOut: token.Address,

pkg/gateway/server.go

@@ -24,6 +24,9 @@ type Server struct {
 	quotes     *quoteCache   // optional short-lived cache of venue quotes
 	catalog    *Catalog      // the tokens the chains above can trade
 	marks      *marks        // what each of them is worth, briefly kept
+	// reading bounds how many questions are in flight against each chain, for
+	// the whole process rather than for one request.
+	reading map[ChainID]chan struct{}
 }
 
 // ServerConfig holds server configuration
@@ -129,6 +132,10 @@ func NewServer(router *Router, cfg ServerConfig, opts ...ServerOption) *Server {
 	// a way to set it wrongly.
 	s.catalog = NewCatalog(s.chains.Chains())
 	s.marks = newMarks()
+	s.reading = map[ChainID]chan struct{}{}
+	for _, id := range s.chains.Chains() {
+		s.reading[id] = make(chan struct{}, readingsInFlight)
+	}
 
 	s.registerRoutes()
 	return s

pkg/gateway/server_catalog.go

@@ -24,9 +24,6 @@ const (
 	// a screen wanting two hundred asks four times, and by the second one most
 	// of them are already read.
 	pricesAtOnce = 50
-	// And this many at a time, so a cold page is a steady stream of calls
-	// rather than five hundred at once.
-	pricesInFlight = 8
 	// The whole fan-out is bounded well inside the server's own write timeout,
 	// so a chain that has gone slow answers rather than holding the connection
 	// until it is cut. A token that ran out of time counts as a chain that
@@ -123,17 +120,16 @@ func (s *Server) handlePrice(w http.ResponseWriter, r *http.Request) {
 	unreachable := 0
 	var mu sync.Mutex
 	var wg sync.WaitGroup
-	inFlight := make(chan struct{}, pricesInFlight)
 
+	// Every token at once. What they wait on is the chain's own budget of
+	// questions in flight, which is one number for the whole process — a bound
+	// per request would still multiply by the number of requests.
 	ctx, done := context.WithTimeout(s.requestContext(r), pricesWithin)
 	defer done()
 	for i, t := range tokens {
 		wg.Add(1)
 		go func(i int, t Token) {
 			defer wg.Done()
-			inFlight <- struct{}{}
-			defer func() { <-inFlight }()
-
 			mk := s.markOf(ctx, chain, t)
 			mu.Lock()
 			out[i] = mk.price(chain, t)