mirror of
https://github.com/luxfi/bridge.git
synced 2026-09-24 00:26:46 +00:00
don't rate-limit /health — it makes the kubelet kill the pod
Liveness probe failed: HTTP probe failed with statuscode: 429
Container bridge-server failed liveness probe, will be restarted
Observed in lux-bridge as a restart every few minutes. `trust proxy` is set, so
real traffic keys on the client, but probes arrive straight at the pod IP with no
X-Forwarded-For and therefore all share one key. Readiness every 5s plus liveness
every 10s is ~270 requests per 15-minute window against a limit of 100, so
/health starts answering 429 and the platform reads that as a dead process.
Health is infrastructure asking whether the process is alive. Rate-limiting the
answer only teaches the kubelet to kill it.
Surfaced by removing the cors-patch ConfigMap: production's entry point had been
a March build of server.ts that predates this limiter, so the limiter had never
actually run in this deployment until now.
Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
@@ -66,6 +66,17 @@ try {
|
||||
max: 100, // limit each IP to 100 requests per window
|
||||
standardHeaders: true,
|
||||
legacyHeaders: false,
|
||||
// The kubelet is not a client. Its probes arrive straight at the pod IP with
|
||||
// no X-Forwarded-For, so `trust proxy` cannot spread them the way it does
|
||||
// real traffic — every probe from a node shares one key. Readiness every 5s
|
||||
// plus liveness every 10s is ~270 requests per window against a limit of
|
||||
// 100, so /health starts answering 429, the liveness probe reads that as
|
||||
// dead, and the container is killed and restarted on a loop:
|
||||
// Liveness probe failed: HTTP probe failed with statuscode: 429
|
||||
// Observed in lux-bridge as a restart every few minutes. Health is
|
||||
// infrastructure asking whether the process is alive; rate-limiting the
|
||||
// answer only teaches the platform to kill it.
|
||||
skip: (req) => req.path === '/health',
|
||||
}))
|
||||
app.use(express.urlencoded({ extended: true }))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user