Files

180 KiB

Code map

Navigation index for agents and new contributors: concept → file → entry points → invariants. Start here instead of grepping the tree. If you had to grep for something that belongs in this map, add the row in the same change (see AGENTS.md).

  • High level purpose and samples: README.md
  • Known defects, with IDs referenced from code comments: FINDINGS.md
  • Larger planned work: ROADMAP.md
  • Generated API reference (make docs): api/

Module layout

gserver/            gRPC + gRPC-Web + REST server (cmux), identity + authz wiring
  credentials/      gRPC credential bundle (TLS + Bearer/DPoP per-RPC tokens)
  roles/            identity provider: JWT, DPoP, AWS STS, TLS/SPIFFE → roles
restserver/         httprouter REST server with fixed middleware chain
  authz/            path/role access control tree (HTTP handler + gRPC interceptors)
  ready/            readiness gate middleware
  telemetry/        request logger, request metrics, ResponseCapture
xhttp/
  correlation/      correlation ID across HTTP, gRPC metadata, context, logs
  header/           header name and content-type constants
  httperror/        structured API errors, HTTP ↔ gRPC code mapping
  identity/         caller identity + client IP in request contexts
  marshal/          JSON response writing (gzip, ?pp), bounded bodies and strict decoding
  limits/           Shared timeout and body-size defaults
pkg/
  retriable/        HTTP client: retry policy, failover, Bearer/DPoP, nonces, token storage
  rpcclient/        gRPC client builder from Config
  redisclient/      prefixed go-redis wrapper, owner-bound lock, rate limiter
  cache/            Provider abstraction: memory, Redis, proxy; pub/sub
  revocation/       JWT revocation list over cache.Provider
  location/         client location from IP: Provider, ipwho.is lookup, per-network LRU cache
  tasks/            cron-like scheduler
  tlsconfig/        tls.Config from files, KeypairReloader, HTTPTransport
  transport/        TLS listener (eager handshake, CRL), keepalive listener, TLSInfo
  appinit/          logging, metrics (Prometheus/CloudWatch), CPU profiler; config/
  discovery/        in-process service registry by interface
  crlcache/         Verifier interface for revocation checks
  streamctx/        override grpc.ServerStream context
metricskey/         metric descriptors
tests/              testutils (free ports, JSON compare), mockappcontainer (dig builder)

Internal dependency direction

Arrows point from importer to imported. Nothing under xhttp/ or pkg/ imports restserver or gserver, except pkg/retriable, pkg/rpcclient, pkg/cache and pkg/redisclient, which import gserver/credentials (token types) or gserver (TLSInfo) only.

Package Imports from this module
gserver gserver/credentials, gserver/roles, pkg/discovery, pkg/streamctx, pkg/transport, restserver/authz, restserver/ready, restserver/telemetry, xhttp/*, metricskey
gserver/roles gserver/credentials, xhttp/header, xhttp/identity
gserver/credentials none
restserver restserver/authz, restserver/ready, restserver/telemetry, xhttp/*
restserver/authz restserver/telemetry, xhttp/httperror, xhttp/identity, xhttp/marshal
restserver/ready xhttp/httperror, xhttp/marshal
restserver/telemetry metricskey, xhttp/header, xhttp/identity
xhttp/correlation pkg/streamctx, xhttp/header
xhttp/httperror xhttp/correlation, xhttp/header
xhttp/identity pkg/streamctx, xhttp/header, xhttp/httperror, xhttp/marshal
xhttp/marshal xhttp/header, xhttp/httperror, xhttp/limits
pkg/retriable gserver/credentials, pkg/tlsconfig, xhttp/correlation, xhttp/header, xhttp/httperror
pkg/rpcclient gserver/credentials, pkg/retriable, xhttp/httperror
pkg/redisclient, pkg/cache pkg/tlsconfig (and gserver.TLSInfo for config)
pkg/revocation pkg/cache
pkg/location xhttp/header
pkg/transport pkg/crlcache, pkg/tlsconfig, xhttp/limits
pkg/appinit pkg/appinit/config, metricskey, xhttp/marshal; pkg/appinit/config imports xhttp/limits
tests/mockappcontainer gserver, pkg/discovery
xhttp/header, xhttp/limits, pkg/tlsconfig, pkg/tasks, pkg/discovery, pkg/crlcache, pkg/streamctx, metricskey, tests/testutils none

Concept index

Concept Package File Symbol
gRPC/REST/gRPC-Web server bootstrap gserver server.go Start, GServer
listen URL schemes (http/https/unix/unixs) gserver serve.go configureListeners
cmux protocol multiplexing (HTTP1 vs HTTP2) gserver serve.go serveCtx.serve
gRPC-Web translation / trailer frame gserver grpc_web_response.go grpcWebResponse, buildTrailerFrame
gRPC-Web text (base64) gserver grpc_web_response.go writeTextPayload
gRPC-Web gzip / X-GRPC-Stream gserver serve.go grpcHandlerFunc
gzip acceptance / quality values xhttp/marshal marshal.go AcceptsGzip
gRPC DPoP proof URI gserver/credentials, gserver/roles credentials.go, roles.go GetRequestMetadata, IdentityFromContext (grpcDPoPURI)
CORS (REST, rs/cors) gserver, restserver serve.go, router.go configureHandlers, corsHandler, CORSOptions, NewRouterWithCORS
CORS (gRPC-Web headers) gserver serve.go, grpc_web_response.go grpcHandlerFunc, grpcWebResponse.prepareHeaders, mergeVaryHeader
rate limit (tollbooth, HTTP and gRPC) gserver ratelimit.go, serve.go, config.go newRateLimiter, rateLimiter.httpHandler, rateLimiter.unaryInterceptor, rateLimiter.streamInterceptor, RateLimit, RateLimit.Validate, RateLimit.GetLogRejections
trusted proxy CIDRs / forwarding headers xhttp/identity, gserver, restserver realip.go, config.go, server.go ParseTrustedProxies, WithTrustedProxies, NewTrustedProxyHandler, Config.Validate, HTTPServer.WithTrustedProxies
client IP resolved once per request xhttp/identity realip.go NewTrustedProxyHandler, ClientIPFromRequest, forwarding
HTTP middleware chain order gserver, restserver serve.go, server.go configureHandlers, HTTPServer.NewMux
gRPC interceptor chain gserver serve.go grpcServer
panic recovery gserver, xhttp/correlation, xhttp/identity serve.go, correlation.go, ctx.go recoverHTTPPanic, panicInterceptor, NewAuthUnaryInterceptor
panicking REST handler metrics restserver/telemetry request_metrics.go, response_capture.go NewRequestMetrics, ResponseCapture
request validation (Validate()) gserver serve.go NewRequestValidationUnaryInterceptor, validate, Validator
keepalive (gRPC server) gserver serve.go, config.go configureListeners, KeepAliveCfg
keepalive (gRPC client) pkg/rpcclient config.go Config.DialKeepAliveTime/Timeout
keepalive (TCP listener) pkg/transport keepalive_listener.go NewKeepAliveListener, keepAliveConfig
listener Accept errors / temporary retry pkg/transport, gserver keepalive_listener.go, listener_tls.go, listener.go keepaliveListener.Accept, tlsListener.forwardAcceptError, backoffListener
TLS server config / client cert auth gserver, pkg/transport config.go, tls.go TLSInfo, TLSInfo.ServerTLSWithReloader (custom policy: VerifyConnection on its config)
TLS reload / certificate rotation pkg/tlsconfig reloader.go KeypairReloader, GetKeypairFunc
TLS min version / ALPN defaults pkg/tlsconfig tlsconfig.go NewServerTLSFromFiles, NewClientTLSFromFiles
mutual TLS client (GetClientCertificate) pkg/tlsconfig tlsconfig.go NewClientTLSWithReloader
cipher suites by name pkg/tlsconfig cipher_suites.go UpdateCipherSuites, GetCipherSuite
OCSP staple loading pkg/tlsconfig tls.go X509KeyPairWithOCSP, LoadX509KeyPairWithOCSP
reloading http.RoundTripper pkg/tlsconfig tlsconfig.go HTTPTransport, NewHTTPTransportWithReloader
TLS listener / eager handshake pkg/transport listener_tls.go NewTLSListener
CRL / revocation check on client certs pkg/transport, pkg/crlcache listener_tls.go, crlcache.go TLSInfo.CRLVerifier, Verifier
handshake failure callback pkg/transport tls.go TLSInfo.HandshakeFailure
graceful shutdown / shutdown timeout gserver, restserver server.go, config.go Server.Close, Config.Timeout.Shutdown (shutdownTimeout), HTTPServer.StopHTTP, WithShutdownTimeout
server lifecycle events restserver server.go OnEvent, ServerEvent
REST bind address helpers restserver config.go GetPort, GetHostName
REST server config access restserver server.go HTTPServer.Config, HTTPConfig
custom middleware / interceptors gserver, restserver options.go, server.go WithMiddleware, WithUnaryServerInterceptor, WithMuxFactory
service registration (REST/gRPC) gserver, restserver server.go, service.go RouteRegistrator, GRPCRegistrator, ServiceFactory, Service
service registry / DI lookup pkg/discovery discovery.go Discovery.Register/Find
dig container for tests tests/mockappcontainer builder.go NewBuilder
readiness gate gserver, restserver, restserver/ready server.go, ready.go IsReady, NewServiceStatusVerifier
slow request logging / gRPC metrics gserver logs.go WarnUnaryRequestLatency, logRequest, callerOf
caller (client IP, role) in gRPC logs gserver logs.go, serve.go callLog, identityHandoffUnaryInterceptor, identityHandoffStreamInterceptor, grpcServer
log field names remote/peer/agent restserver/telemetry, xhttp/identity, pkg/transport requestlogger.go, ctx.go, keepalive_listener.go NoAgent, RequestLogger.ServeHTTP, NewContextHandler, createIdentityContext, setKeepAlive
log field names in gserver lines gserver logs.go, ratelimit.go, serve.go logRequest, logError, logRejection, grpcHandlerFunc
request logging (HTTP) restserver/telemetry requestlogger.go NewRequestLogger
log skip paths restserver/telemetry, gserver requestlogger.go, config.go LoggerSkipPath, ShouldSkip, Config.SkipLogPaths
HTTP request metrics (nested role handoff) restserver/telemetry, restserver, gserver request_metrics.go, server.go, serve.go NewRequestMetrics, NewMux, configureHandlers, rateLimiter.httpHandler
metric route/method/role labels restserver/telemetry, restserver route.go, request_metrics.go, router.go SetRoute, requestLabels, UnknownRoute, OtherMethod, proxyHandle
response capture, Started/Unwrap/Hijack/ReadFrom restserver/telemetry response_capture.go ResponseCapture
metric descriptors metricskey describe.go HTTPReqPerf, GRPCReqPerf, *ByRole, StatsVersion, HealthLogErrors
metrics init (prometheus / cloudwatch) pkg/appinit metrics.go Metrics
metrics config YAML pkg/appinit/config config.go Metrics
logging flags / log rotation pkg/appinit init.go LogConfig, Logs
CPU profiling pkg/appinit init.go, cpu_profiler.go CPUProfiler
static response headers gserver config.go Config.HTTPHeaders, Config.Validate
max message size (gRPC) gserver, pkg/rpcclient options.go, client.go MaxRecvMsgSize, MaxSendMsgSize, defaultMaxCallSendMsgSize
HTTP body limits xhttp/marshal, gserver, restserver limits.go, config.go, server.go LimitRequestBody, DecodeBody, Config.MaxRequestBody, WithMaxRequestBody, MaxRequestSize
Shared server timeouts xhttp/limits, gserver, restserver, pkg/appinit/config limits.go, config.go, server.go Timeouts, Config.Timeouts, WithTimeouts, Prometheus.Timeouts
Handler deadline (server-side request bound) xhttp/limits, gserver, restserver, pkg/appinit limits.go, deadline.go, serve.go, server.go, prometheus.go Timeouts.Handler, Timeouts.HandlerContext, Timeouts.ApplyHandler, deadlineTapHandle, capGRPCTimeout, setStreamWriteDeadline, deadlineUnaryInterceptor, deadlineStreamInterceptor, deadlineError, prometheusHandler
TLS gRPC stream read deadline gserver serve.go grpcHandlerFunc, clearStreamReadDeadline
header copy / trailer prefix handling gserver header.go copyHeader, replaceInKeys
header name constants xhttp/header headers.go XCorrelationID, Authorization, Cookie, XCSRFToken, ...
correlation ID (HTTP) xhttp/correlation correlation.go NewHandler, ID
correlation ID (gRPC) xhttp/correlation correlation.go NewAuthUnaryInterceptor, WithMetaFromContext
correlation ID (client side) pkg/retriable retriable.go convertRequest
gRPC stream context override pkg/streamctx streamctx.go WithContext
structured API error xhttp/httperror errors.go Error, New, Wrap
error → HTTP status xhttp/httperror errors.go Status
gRPC ↔ HTTP code mapping xhttp/httperror codes.go, rpc.go codeStatus, statusCode, HTTPStatusFromRPC, restoredHTTPStatus
multi-error (validation) xhttp/httperror many.go ManyError.Add
gRPC error with code/correlation details xhttp/httperror rpc.go Error.GRPCStatus, NewFromPb, CorrelationID
OAuth error mapping xhttp/httperror codes.go FromOAuth
JSON response / error writing xhttp/marshal marshal.go WriteJSON
generic 5xx message (no internal text) xhttp/marshal marshal.go WriteJSON
shared error values / non-mutating writer xhttp/httperror, restserver/ready errors.go, many.go, ready.go Error.WriteHTTPResponse, ManyError.WriteHTTPResponse
gzip responses xhttp/marshal marshal.go WriteJSON
pretty print (?pp) xhttp/marshal json.go PrettyPrintSetting
JSON body decoding (strict) xhttp/marshal json.go DecodeBody, DecoderHandle
caller identity xhttp/identity identity.go Identity, NewIdentity, AuthMethod
identity middleware (HTTP) xhttp/identity ctx.go NewContextHandler
identity mapper error response xhttp/identity ctx.go identityError
identity interceptor (gRPC) xhttp/identity ctx.go NewAuthUnaryInterceptor, NewStreamServerInterceptor
request context (identity/IP/UA) xhttp/identity ctx.go FromRequest, FromContext
X-Forwarded-For / X-Real-Ip xhttp/identity realip.go ClientIPFromRequest, ClientIPFromGRPC
client IP as string ("" when none) xhttp/identity realip.go, ctx.go IPString, RequestContext.ClientIP, RequestContext.ClientAddr
X-Forwarded-Proto restserver, xhttp/identity server.go, realip.go GetServerURL, ForwardedProto
basic auth parsing xhttp/identity basicauth.go BasicAuthFromRequest
test identity injection xhttp/identity identity.go WithTestIdentity
identity provider / role mapping gserver/roles roles.go New, IdentityProvider
JWT role mapping gserver/roles roles.go, config.go jwtIdentity, JWTIdentityMap
DPoP verification (server) gserver/roles roles.go dpopIdentity
AWS STS caller identity (AWS4 token) gserver/roles roles.go awsIdentity, ValidateSTSPresignedURL, ParseSTSTokenExpiration
STS lookup caches and coalescing gserver/roles roles.go awsCallerIdentity, leadAWSLookup, lookupAWS, cacheableSTSFailure
TLS / SPIFFE role mapping gserver/roles roles.go tlsIdentity, GenericIdentityMap
SkipAuthPaths gserver/roles roles.go, config.go IdentityFromRequest, IdentityMap.SkipAuthPaths
cookie auth / CSRF double submit gserver/roles roles.go, config.go enforceCSRFCookieAndHeader, enforceCSRFMetadata, checkCSRF, CookiesConfig
strict auth mode gserver/roles config.go IdentityMap.Strict
guest role gserver/roles, xhttp/identity roles.go, identity.go GuestRoleName
path/role authorization restserver/authz authz.go Provider.Allow, AllowAny, AllowAnyRole
authz YAML config restserver/authz authz.go Config
authz HTTP handler / gRPC interceptors restserver/authz authz.go NewHandler, NewUnaryInterceptor, NewStreamServerInterceptor
authz denial status (401 vs 403) restserver/authz, xhttp/httperror authz.go, codes.go denied, codeStatus, statusCode
CORS preflight termination before authz restserver, gserver server.go, router.go, serve.go NewMux, newCORS, corsHandler
role mapper restserver/authz authz.go SetRoleMapper, SetGRPCRoleMapper
gRPC client credentials bundle gserver/credentials credentials.go NewBundle, Bundle, bundle.NewWithMode
authorization metadata key gserver/credentials credentials.go TokenFieldNameGRPC
token refresh / caller identity (client) gserver/credentials, pkg/retriable credentials.go, retriable.go CallerIdentity, Token.Expired, refreshToken, WithCallerIdentity
DPoP proof (client) gserver/credentials, pkg/retriable credentials.go, retriable.go Bundle.WithDPoP, Client.Do
OAuth fixed token gserver/credentials oauth.go NewOauthAccess
gRPC client construction pkg/rpcclient client.go New, NewFromURL
gRPC client TLS endpoint validation pkg/rpcclient client.go New, Config.TLS
gRPC blocking dial timeout pkg/rpcclient client.go Client.dial, Config.DialTimeout
retry policy / backoff pkg/retriable retriable.go Policy, DefaultPolicy, ShouldRetry, DefaultShouldRetryFactory, waitRetry
Retry-After / 429 retries (client) pkg/retriable retriable.go RetryAfterShouldRetryFactory, parseRetryAfter
HTTP client transport (TLS, DNS, clone) pkg/retriable retriable.go WithTLS, WithDNSServer, WithTransport, transportClone, replaceTransport
error response body bound (client) pkg/retriable retriable.go DecodeResponse, drainResponseBody, maxErrorBodySize
non-retriable errors (TLS/DNS) pkg/retriable retriable.go DefaultNonRetriableErrors
request timeout (HTTP client) pkg/retriable retriable.go Policy.RequestTimeout, WithTimeout, RequestPolicy, cancelOnClose
rewindable request body pkg/retriable request.go Request, NewRequest
header propagation via context pkg/retriable retriable.go WithHeaders, PropagateHeadersFromRequest
User-Agent / X-CLIENT-IP pkg/retriable retriable.go WithUserAgent
Bearer token storage (.auth_token) pkg/retriable storage.go Storage, SaveAuthToken, LoadAuthToken
token from env var pkg/retriable, pkg/rpcclient config.go CheckAuthTokenFromEnv, LoadAuthTokenOrFromEnv
DPoP key files (.jwk) pkg/retriable storage.go Storage.SaveKey/LoadKey/ListKeys, KeyInfo
nonce / Replay-Nonce pkg/retriable nonce.go NonceProvider, NonceContext, DefaultReplayNonceHeader, Client.WithNonce
custom DNS server pkg/retriable retriable.go WithDNSServer
multi-client YAML config pkg/retriable config.go Config, LoadFactory, Factory
request dump (debug) pkg/retriable dumpreq.go DumpRequestOut
Redis connection / TLS / auth pkg/redisclient redisclient.go NewRedisClient, Config, parseURL (URL kept out of logs/errors)
Redis key prefix / .. containment pkg/redisclient redisclient.go WithPrefix, Key, SubKey, pattern, escapeGlob
Redis value marshalling pkg/redisclient marshal.go Marshal, UnmarshalStringCmd
distributed lock (owner token) pkg/redisclient redisclient.go TryLock, ReleaseLock, IsLocked, releaseLockScript, setNX, coordKey, remainingTTL
rate limit (Redis window) pkg/redisclient redisclient.go TryAcquireRateLimit, GetRateLimitRemainingTime, setNX, coordKey, remainingTTL
bounded set / hash (eviction) pkg/redisclient redisclient.go SAddWithEviction, HSetWithEviction, boundedSetScript, checkEviction
key scan pkg/redisclient redisclient.go ScanKeys, Keys
cache provider abstraction pkg/cache cache.go Provider
cache TTL defaults pkg/cache cache.go DefaultTTL, KeepTTL
cache expiry / cleanup pkg/cache memory.go CleanExpired, NowFunc
read-through cache pkg/cache cache.go GetOrSet
pub/sub (cache) pkg/cache memory.go, redis.go Publish, Subscribe, Subscription
cache namespacing pkg/cache proxy.go, keys.go NewProxyProvider, namespace
cache key layout / .. containment pkg/cache keys.go namespace.key, namespace.rel, namespace.pattern, namespace.listed
cache key glob (Redis-compatible) pkg/cache keys.go, memory.go, redis.go globMatch, escapeGlob, memProv.ListKeys, redisProv.ListKeys, deprecated Keys
cache key listing pages (SCAN cursor) pkg/cache cache.go, memory.go, redis.go, proxy.go Provider.ListKeys, defaultListCount, listedKey, pageHeap
not-found sentinel pkg/cache, pkg/redisclient cache.go, redisclient.go ErrNotFound, IsNotFoundError
closed-subscription sentinel pkg/cache cache.go ErrClosed, closedSub
slow-subscriber policy (pub/sub) pkg/cache cache.go, memory.go, redis.go subscriberBufferSize, memProv.Publish, redisProv.Subscribe
Redis subscription confirmation pkg/cache redis.go, cache.go confirmSubscription, subscribeTimeout, failedSub
JWT token revocation pkg/revocation revocaction.go New, Validate, Revoke
revoked-token snapshot pkg/revocation revocaction.go load, reload, snapshotTTL, reloadPageSize, remember
revocation cache key pkg/revocation revocaction.go cacheKey
revocation test clock pkg/revocation revocaction.go TimeNowFn
client location from IP (geolocation) pkg/location location.go, ipwhois.go Provider, Place, NewIPWhoIs, IPLocation
location cache per network (/24, /48) pkg/location cache.go NewCache, CacheConfig, Cache.Location
public IP check (no lookup for private) pkg/location location.go IsPublic, ParsePublicIP, parseHostAddr, nonPublicPrefixes
location quota (HTTP 429, Retry-After) pkg/location ipwhois.go ErrQuotaExceeded, retryAfter, IPWhoIs.backOff, quotaUntil
cron / periodic tasks pkg/tasks scheduler.go NewScheduler, Scheduler
schedule string format pkg/tasks task.go ParseSchedule
task run lock / overlap prevention pkg/tasks task.go task.Run, WithRunTimeout
task status publishing pkg/tasks scheduler.go Publisher
test clock override pkg/tasks, pkg/cache task.go, cache.go TimeNow, NowFunc
free port for tests tests/testutils testutils.go CreateBindAddr, CreateURL

Packages

github.com/effective-security/porto/gserver

Purpose: Combined gRPC + gRPC-Web + REST server. Start opens one cmux listener per unique address, runs a gRPC server and an http.Server (REST router) on each, translates gRPC-Web to gRPC on TLS listeners, and wires the fixed middleware/interceptor chains.

Files:

  • doc.go — package overview and usage example.
  • server.go — Server/GServer, Start (resolveDependencies, abort), service registry, Close, error channel.
  • listener.go — backoffListener: waits after a failed root Accept before cmux retries it.
  • ratelimit.go — rateLimiter (one tollbooth limiter per listener): newRateLimiter, httpHandler (REST limiter and its rejection telemetry), unaryInterceptor/streamInterceptor (allowGRPC, grpcRequest), key and clientIP (limiter key), logRejection (RateLimit.LogRejections line).
  • serve.go — configureListeners (URL schemes, TLS, keepalive, accept backoff; returns the shared transport.TLSInfo the server owns), serveCtx.serve (cmux split, one newRateLimiter per listener) and serveCtx.closeServers, configureHandlers (HTTP chain), corsHandler (REST/preflight CORS response headers; an empty origin list allows none, and a disallowed actual request still reaches the handler), grpcServer (interceptor chain, with the listener's limiter), grpcHandlerFunc (gRPC / gRPC-Web / REST mux on TLS listeners), NewRequestValidationUnaryInterceptor, panicInterceptor.
  • grpc_web_response.go — grpcWebResponse: http.ResponseWriter that rewrites gRPC responses into gRPC-Web framing (trailer frame, base64 for -text, pooled optional gzip for unary).
  • header.go — copyHeader and option helpers used for gRPC → gRPC-Web header/trailer translation.
  • logs.go — gRPC request logging/metrics interceptors (newLogUnaryInterceptor, newLogStreamServerInterceptor, logRequest), the caller handoff from identity back to them (callLog, withCallLog, identityHandoffUnaryInterceptor, identityHandoffStreamInterceptor, callerOf) and logError (the API_ERROR/INTERNAL_ERROR line).
  • deadline.go — the handler deadline of gRPC and gRPC-Web calls (Config.Timeouts.Handler): deadlineTapHandle (the stream deadline of the native HTTP/2 transport, plaintext listeners), capGRPCTimeout with encodeGRPCTimeout/decodeGRPCTimeout (the grpc-timeout cap grpcHandlerFunc applies before ServeHTTP on TLS listeners), setStreamWriteDeadline with deadlineWriteGrace (the per-stream write deadline that releases a send blocked on the client's window on TLS listeners), deadlineUnaryInterceptor, deadlineStreamInterceptor (bound the call context with limits.Timeouts.HandlerContext) and deadlineError (a plain context error returned by the handler becomes DeadlineExceeded/Canceled).
  • config.go — Config (Config.Validate, Config.shutdownTimeout), TLSInfo, KeepAliveCfg, CORS (CORS.Validate), RateLimit (RateLimit.Validate), SwaggerCfg.
  • options.go — WithMiddleware, WithUnaryServerInterceptor, WithStreamServerInterceptor, MaxRecvMsgSize, MaxSendMsgSize.

Entry points:

  • Start(name, cfg, container, factories, opts...) (GServer, error) — the only constructor; starts serving immediately.
  • GServer — AddService, Service, IsReady, Err, Close, Discovery.
  • ServiceFactory, Service, RouteRegistrator, GRPCRegistrator, StartSubcriber — plug-in contracts.
  • Config YAML keys: listen_urls, server_tls{cert,key,trusted_ca,client_ca,client_cert_auth,cipher_suites}, services, identity_map, authz, cors, rate_limit, trusted_proxy_cidrs, timeouts{header,read,idle,handshake,handler}, max_request_body, timeout.shutdown (timeout.request is its deprecated former name; Validate rejects both set), keep_alive{min_time,interval,timeout}, max_recv_msg_size, max_send_msg_size, http_headers, logger_skip_paths, prom_grpc, debug_logs.

Invariants:

  • Start returns errors, starting with the Config.Validate checks (TrustedProxyCIDRs, CORS.Validate, RateLimit.Validate, Timeout.Shutdown and Timeout.Request both set, static CORS headers); newServer calls the unexported Config.validate, which also returns the parsed TrustedProxyCIDRs policy so it is parsed once. When Start fails after newServer created services or opened listeners, abort closes the serve channels and calls Close, so services, listeners and the TLS reloader are released; configureListeners closes the listeners it opened and stops the reloader on its own errors. configureHandlers panics through the logger if the authz handler cannot be built. gRPC handler panics are recovered by the unary and stream interceptors of xhttp/correlation and the unary one of xhttp/identity into codes.Internal with message unhandled exception; identity uses httperror.NewGrpcFromCtx, while correlation returns a gRPC status directly because httperror imports correlation. panicInterceptor catches panics outside them. The log interceptors register their recording defer before calling the handler, count escaping panics as Internal and leave recovery to the surrounding interceptor; ordinary errors use status.Code (Unknown for plain errors), and wrapped httperror causes are found with errors.As. A REST handler's panic is counted by the outer metrics handler of configureHandlers (500 unless the handler sent a status) before it reaches recoverHTTPPanic in grpcHandlerFunc (TLS listeners: 500 only before the response starts, otherwise an abort; http.ErrAbortHandler passes through without logging) or net/http (plaintext listeners, which aborts the connection).
  • Process-global: package logger; WarnUnaryRequestLatency. The gRPC gzip compressor is registered by blank import.
  • Log field names: remote is the client IP resolved under the trusted proxy policy (identity.IPString, "" for a Unix socket peer) in every request line (REST access line and 429, gRPC logRequest and logError lines, the cors_not_allowed line and the two per-request DebugLogs lines of grpcHandlerFunc, and the xhttp/identity WARNING lines); peer is the socket address (host:port, @ for a Unix socket peer) in the logRejection, cors_not_allowed and per-request DebugLogs lines and in pkg/transport; agent is the User-Agent, telemetry.NoAgent when missing, in the access, gRPC, logRejection, cors_not_allowed and DebugLogs lines. The DebugLogs line written after a gRPC-Web response has only the method and headers. The xhttp/marshal API_ERROR line has no client IP (marshal cannot import xhttp/identity) and logs an empty agent when missing; join it to the access line on ctx.
  • Every name in Config.Services must exist in the factory map; factories run through dig.Container.Invoke. The container must provide discovery.Discovery and, when JWT/DPoP is enabled, jwt.Parser.
  • Listen URL schemes: http, https, unix, unixs; no scheme → https when TLS is configured. The same address listed twice yields one listener serving both secure and insecure.
  • Network limits: Config.Timeouts uses xhttp/limits defaults (header 10s, read 30s, idle 60s, handshake/detection 10s); zero selects defaults and negatives disable individual limits. cmux detection and transport.TLSInfo.HandshakeTimeout use Handshake. Both HTTP servers limit bodies to MaxRequestBody (default 10 MiB), including gRPC/gRPC-Web over TLS. Native plaintext gRPC keeps its per-message limits. On TLS, grpcHandlerFunc clears the per-stream read deadline for native gRPC over HTTP/2 (clearStreamReadDeadline), so client and bidi streams may outlive Timeouts.Read; MaxRequestBody still bounds each stream in total because grpc-go's ServeHTTP transport buffers request data without flow control (ROADMAP 12), and a stream over it fails with Unavailable. Raise or disable read/body limits for large uploads and long request streams. The read deadlines end when the request has been read; the server-side bound on a request's execution is Timeouts.Handler (deadline.go, limits.Timeouts.ApplyHandler for REST), off by default: the context of every REST request and gRPC/gRPC-Web call, streams included, expires after it unless the client asked for an earlier deadline, which is kept (context.WithTimeout keeps the earlier parent deadline). The REST deadline wraps identity extraction and the handler, inside correlation; the gRPC one runs inside the log interceptor, so a handler that returns its expired context error (no gRPC status) is answered, logged and counted as DeadlineExceeded (Canceled when the client went away) with the handler's error as the cause, while errors carrying a status are unchanged. For gRPC the deadline is also the transport stream deadline, so a stream handler blocked in RecvMsg or SendMsg (a client that stopped sending without closing its send side, a client that stopped reading but kept the connection open) is released; streamctx.WithContext replaces only Context, so the interceptors alone could not do that. On plaintext listeners deadlineTapHandle (grpc.InTapHandle, an experimental grpc-go API) returns the bounded context the transport then uses as the stream context (it cancels the parent when the stream ends, which stops the child's timer, so no cancel is kept): both a blocked receive and a blocked send fail with DeadlineExceeded, and that status reaches the client. On TLS listeners grpcHandlerFunc caps the grpc-timeout request header (capGRPCTimeout: missing or longer than Handler → Handler in the wire format, rounded up to the finest unit that fits 8 digits; shorter kept; malformed left for grpc-go to reject) before ServeHTTP, which makes it the stream deadline while the request context stays alive, so a blocked receive fails with DeadlineExceeded; bounding the request context instead makes grpc-go close the transport and the client gets Unavailable: server stopped. That deadline does not release a send: grpc-go's ServeHTTP transport performs it as a net/http write that waits for the client's flow-control window, so for HTTP/2 requests grpcHandlerFunc also sets a per-stream write deadline of Handler + deadlineWriteGrace (1s) through http.ResponseController (setStreamWriteDeadline); when it expires net/http resets the stream (RST_STREAM INTERNAL_ERROR), the send fails with Unavailable, the handler returns, and the client sees Internal, since no trailers can follow an aborted write. The grace leaves a handler that returned at the deadline time to deliver its status; a response still being written after it is cut the same way, which plaintext listeners do not do. gRPC-Web over HTTP/1.1 gets no write deadline (it would be the connection's and outlive the request). grpc-go decodes a unary request before the interceptors run, so the bound starts after the decode. Handlers that ignore their context outlive the deadline, as with a client deadline, and a REST handler blocked writing to a client that stopped reading is not released (no response write deadline). Timeout.Shutdown (former name Timeout.Request) is shutdown-only.
  • Plain listeners: cmux routes HTTP/2 → gRPC (h2c), HTTP/1 → REST; gRPC-Web and HTTPHeaders are only handled on TLS listeners, where everything goes through http.Server and grpcHandlerFunc selects by Content-Type.
  • HTTP chain (outer → inner): trusted proxy policy → body limiter (bounds reads, writes no response) → rate limit → correlation → handler deadline (if Timeouts.Handler is set) → metrics → CORS (if enabled) → identity → metrics (role handoff, records nothing) → request logger → authz (if configured) → readiness → WithMiddleware → router. On TLS listeners grpcHandlerFunc sits between the body limiter and the rate limit and sends gRPC and gRPC-Web to the gRPC server, whose responses the HTTP metrics never see. CORS answers preflights before authz; with options_pass_through, OPTIONS requests are authorized like any other method. The outer metrics handler counts identity rejections and CORS preflights with the guest role. Rate limit (ratelimit.go): each listener has one rateLimiter (newRateLimiter in serveCtx.serve) shared by its REST handler (httpHandler) and the interceptors of its gRPC servers, so plaintext (h2c) gRPC, which cmux sends past the HTTP handler, is limited, and a gRPC call on a TLS listener draws one token. Buckets are tollbooth's: client address, request path and Metods; a gRPC call is checked as the POST to its full method name (grpcRequest: the socket peer, with the x-forwarded-for, x-real-ip and user-agent metadata as headers). The default key is the client IP resolved under the trusted proxy policy (clientIP: identity.IPString of identity.ClientIPFromRequest, "" when the peer is not an IP, as for a Unix socket peer @; grpcRequest puts the policy in the call context): tollbooth checks a shallow request copy whose RemoteAddr is that bare IP (limitRequest), so X-Rate-Limit-Request-Remote-Addr echoes that client IP rather than the socket peer (tollbooth then folds an IPv6 address to its /64 prefix for the bucket); a request without a client IP, including every Unix socket caller, is not limited. Explicit headers_ip_lookups reads the named headers (metadata for gRPC) as sent, overrides that policy, and must be used only when a proxy overwrites them. A rejected REST request never reaches the chain above: it is answered like tollbooth.LimitHandler (its OnLimitReached is unused) behind its own correlation → metrics (guest, unknown) → request logger (SkipLogPaths apply), so a 429 has an X-Correlation-ID and an access-log line whose remote is that client IP. A rejected gRPC or gRPC-Web call returns an httperror ResourceExhausted with the tollbooth message and the correlation ID; the limiter interceptor runs after the log interceptor, which logs it with the client IP resolved under the trusted proxy policy and counts it in the gRPC metrics with the guest role, and before identity, so a rejected call does no identity work. grpc-go reads and decodes a unary request before any interceptor runs, so a rejected call still costs its decode (bounded by MaxRecvMsgSize and MaxRequestBody); calls to unregistered methods (Unimplemented) and HTTP/1.1 application/grpc requests on TLS (505) are answered by grpc-go without drawing a token. A unary call draws its token before NewRequestValidationUnaryInterceptor, so a request that fails Validate is limited, logged and counted like any other error; a panic in Validate fails the call with Internal (validate recovers it before it leaves the log interceptor). RateLimit.LogRejections (log_rejections, off by default; GetLogRejections) adds a WARNING line per rejection (logRejection, reason rate_limit) in the request context, with the limiter key (rateLimiter.key, canonicalized like the bucket: an IPv6 address as its /64 prefix), the socket peer, the User-Agent and the forwarding headers as received; its ctx is the X-Correlation-ID and access-line ctx of a 429 and the request ID of a gRPC error (unary gRPC log lines carry the same ctx; stream log lines have none, because the stream log interceptor runs outside correlation). Unary gRPC chain: panic recovery → correlation → log → handler deadline (if Timeouts.Handler is set) → rate limit (if enabled) → validation → identity → identity handoff → authz (only when configured) → prometheus (opt) → custom. Stream chain: log → correlation → handler deadline (if set) → rate limit (if enabled) → identity → identity handoff → authz (only when configured) → prometheus → custom. gRPC log lines and metrics (logs.go): the log interceptor runs before identity, so it puts a callLog in the call context (streams get it through streamctx.WithContext) and the handoff interceptor stores the identity RequestContext in it; logRequest then logs remote (that context's client IP) and labels GRPCReqByRole with its role. A call rejected before or by identity (rate limit, validation, identity errors) is logged with the client IP that callerOf resolves under the server's trusted proxy policy and counted as guest; its logError line has the same remote. Before this handoff (P-105) every gRPC line had an empty remote and every call counted as guest. agent is the user-agent metadata, or telemetry.NoAgent when missing; SkipLogPaths matches the metadata as sent (empty when missing, unlike the REST request logger, which matches NoAgent), and a skipped call records no gRPC metrics either.
  • Headers read: Content-Type, Origin, Accept-Encoding, X-GRPC-Stream. For gRPC-Web, CORS headers are written only when CORS.GetEnabled() and an allowed Origin is present. Nonempty AllowedOrigins uses rs/cors matching, including its origin patterns; explicit * allows all origins and an empty list allows none. Start rejects an enabled * with AllowCredentials (CORS.Validate) and, while CORS is enabled, any Access-Control-* name in HTTPHeaders (Config.Validate, case-insensitive), so static headers cannot bypass the CORS policy; gRPC-Web never sends Access-Control-Allow-Credentials with Access-Control-Allow-Origin: *. A disallowed gRPC-Web POST gets HTTP 403; a disallowed preflight gets no allow-origin header. REST CORS is not request or CSRF protection: rs/cors still runs the handler for a disallowed actual request and only omits CORS response headers. Access-Control-Expose-Headers merges configured and service names with response and gRPC trailer names without case-insensitive duplicates; service response metadata cannot override server CORS policy headers. Reflected and denied origins add Vary: Origin; prepareHeaders merges service Vary header and trailer metadata without dropping it. Other headers written: Content-Encoding: gzip, Config.HTTPHeaders.
  • gRPC-Web compression uses marshal.AcceptsGzip, honoring exact coding names, quality values and all Accept-Encoding header lines; X-GRPC-Stream disables compression. serveGRPCHTTP completes a ServeHTTP transport that returned HTTP 200 without a gRPC status (including shutdown before or after response headers) with Unavailable; non-200 HTTP errors from grpc-go are preserved.
  • Compressed gRPC-Web responses borrow a gzip writer from a pool; Close is idempotent because both finishRequest and the handler's deferred cleanup call it. Writers are reset before reuse and release the previous response writer on return to the pool. After Close, body writes to a compressed response return errWriteAfterClose and late trailer writes are logged and dropped, so raw bytes never follow the gzip footer.
  • Close ordering (teardown, run once under closeOnce, so services are closed once and a repeated or concurrent Close returns after the first teardown finished): stopc → per listener, each bounded by Config.shutdownTimeout (Timeout.Shutdown, else its former name Timeout.Request, else defaultShutdownTimeout, 10s; 3s before v1.0.x): TLS http.Shutdown (drains REST and gRPC over ServeHTTP) then grpc.Stop; plaintext concurrent http.Shutdown and grpc.GracefulStop, with grpc.Stop on timeout → http.Close to close remaining HTTP connections → listener contexts canceled → services Close() → listeners closed → transport.TLSInfo.Close (stops the certificate reloader). Idle HTTP keep-alive connections are always closed. Handlers that ignore cancellation may outlive the deadline; teardown does not wait for plaintext GracefulStop's handler wait after forcing Stop, so services that release a blocked handler from Close can finish cleanup. HTTP shutdown/close errors are intentionally ignored because Close has no error result. Err() is buffered and never closed. Close ranges over each listener's serversC, so that channel is always closed: serve closes it (closeServers, once-guarded) after publishing its servers or, deferred, when it fails; abort closes it for a server that never served. serve starts no gRPC or HTTP server until every fallible step (the TLS listener) succeeded, because a gRPC server blocked in Accept on a cmux listener cannot be stopped until m.Serve runs. Its deferred cleanup stops the gRPC servers only for those setup failures (serving is false): closing a cmux child listener (GracefulStop on plaintext, http.Shutdown on TLS) closes the root listener, so m.Serve returns an error while Close is still draining.
  • An enabled RateLimit must pass RateLimit.Validate: positive RequestsPerSecond (tollbooth would otherwise admit one request per client and reject the rest); ExpirationTTL zero (10 minutes, defaultRateLimitTTL) or at least minRateLimitTTL (1s), because tollbooth recreates an expired bucket with a full burst; HeadersIPLookups limited to the names tollbooth compares exactly (RemoteAddr, X-Forwarded-For, X-Real-IP; rateIPLookups); and Metods entries that are single upper-case HTTP method tokens (isUpperMethodToken, RFC 9110 tchar), since tollbooth compares methods exactly and a non-matching entry limits nothing.
  • Keepalive: MaxConnectionIdle fixed at 5m; enforcement MinTime only if >0; Time/Timeout only if both >0. These gRPC transport settings apply to plaintext (h2c) gRPC only; TLS gRPC is served through http.Server and ServeHTTP (ROADMAP 12).
  • Accept errors: configureListeners wraps every root listener (TCP after transport.NewKeepAliveListener, and Unix) in backoffListener (listener.go), which waits 5ms doubling to 1s after each failed Accept (reset by a successful one, cut short by Close, logged as WARNING accept_failed) and returns the error unchanged; an error wrapping net.ErrClosed returns at once. cmux then retries temporary errors (EMFILE) without spinning and stops on the others.

Tests: shutdown_test.go covers HTTP keep-alive closure, draining before service cleanup on plaintext and TLS, and cancellation after timeout; panic_test.go covers metrics for escaping unary/stream panics, wrapped errors and partial-response aborts over TLS; serve_test.go covers gzip negotiation. grpc_test.go also checks shutdown status before/after TLS response headers and a handler released by service Close. dpop_test.go (TestGRPCDPoP: stored-key pkg/rpcclient authenticates unary and streaming calls with real signed access tokens over TCP TLS and Unix TLS, including TLS server-name and grpc.WithAuthority overrides; hostless overrides fail before handlers run); server_test.go and example_test.go start real servers on random ports using tests/mockappcontainer and tests/testutils.CreateURL (the example binds 127.0.0.1:0 and a Unix socket in a short os.MkdirTemp directory); metrics_test.go (white-box, process-global in-memory sink, not parallel) checks the exact HTTP metrics of configureHandlers (role handoff, identity 401 and CORS preflight as guest) and of both REST limiter variants (rateLimiter.httpHandler: 429 with guest and unknown), and TestPanicCounted the metrics of panicking routes behind grpcHandlerFunc (500 with route and role, or the status already sent), with a fake roles.IdentityProvider; TLS fixtures in testdata/test-server{,-key,-rootca}.pem; lifecycle_test.go (white-box; its goroutine-counting tests run sequentially because they count KeypairReloader and server accept loops process-wide) checks that Close, a failed Start (missing discovery, identity provider error, failed or missing service factory), a failed configureListeners and a failed serve release services, listeners, the reloader and the serve channel, that a failed serve starts no server, and that overlapping Close calls (run under -race) close services once; grpc_web_test.go and grpc_web_bench_test.go unit-test the gRPC-Web writer with httptest.ResponseRecorder; serve_test.go drives grpcHandlerFunc with a bare grpc.NewServer(); limits_test.go checks protocol detection, stalled headers/bodies and HTTP body limits on plain and TLS listeners, including HTTP/2 unknown-length bodies and stalled request streams; it registers a client-streaming porto.test.Upload service from a hand-written grpc.ServiceDesc to check that TLS gRPC streams outlive Timeouts.Read but stay within MaxRequestBody, and checks that a 413 carries CORS and correlation headers. grpc_test.go (black box) serves a hand-written echo service on a plaintext listener and checks the unary chain end to end (validation before the option interceptors, which see the caller identity; panics in Validate and handlers; MaxRecvMsgSize/MaxSendMsgSize from Config and options), the exact gRPC metrics (process-global sink, not parallel) and that Close drains active RPCs for Timeout.Request before stopping them on plaintext and TLS listeners (TestCloseStopsStreamsAfterRequestTimeout; startEcho serves TLS when ServerTLS is set); server_test.go also covers readiness, notFoundHandler, Discovery and Start failures that release the listener. calllog_test.go (white-box) serves a hand-written porto.test.CallLog service with a trusted loopback peer and compares whole JSON log lines (logBuffer.lines drops time and duration and replaces the correlation ID): TestLogRequestCaller (grpcServer with headerIdentity, whose role comes from the X-Test-Role header or metadata) checks the logRequest lines and GRPCReqByRole roles of admitted and identity-rejected unary and stream calls and the identity access_denied lines; TestLogRequestSkipNoAgent checks NoAgent and that SkipLogPaths matches the User-Agent as sent; TestLogRequestRemoteListeners (Start, plaintext and TLS) checks remote of admitted and rate-limited calls and their logError line; TestGRPCHandlerLogFields (TLS) checks the cors_not_allowed and DebugLogs lines. They install process-global log and metrics sinks and start TLS reloaders, so none is parallel. rate_proxy_test.go (white-box) checks the limiter keys of both variants with trusted and spoofed forwarding, Unix socket peers (not limited), Metods, gRPC keys and per-path buckets from metadata (allowGRPC), the interceptors, and the 429 access line, X-Correlation-ID and LogRejections lines for HTTP and gRPC (TestRateLimitRejectionLog installs a process-global log formatter, so it is not parallel); ratelimit_test.go (black box, startEcho on plaintext and TLS listeners) checks trusted-proxy keys, per-path buckets and one token per TLS call (TestGRPCRateLimit), the exact gRPC metrics of a ResourceExhausted rejection with no HTTP metric (TestGRPCRateLimitMetrics, not parallel), and on TLS a gRPC-Web ResourceExhausted with CORS headers and a REST 429 with a correlation ID (TestGRPCWebRateLimit). listener_test.go (white-box) checks the backoff schedule and reset, that errors are returned unchanged, that net.ErrClosed and Close end the wait, that cmux behind backoffListener makes few Accept calls during an EMFILE window and serves afterwards, and that configureListeners wraps TCP and Unix listeners and rejects malformed URLs, https without TLS and missing certificates. deadline_test.go (black box, startEcho on plaintext and TLS listeners; the echo service's echoWait, echoWaitStatus and echoDeadline values and its only REST route, GET /v1/deadline) checks that Timeouts.Handler bounds unary calls, server streams, gRPC-Web calls and REST requests, keeps an earlier client deadline, caps a later or missing one, answers a handler's wrapped context error with DeadlineExceeded and keeps a status error, that a Collect client-stream handler blocked in RecvMsg behind a client that stopped sending is released with DeadlineExceeded on both listener types (TestHandlerTimeoutStalledReceive, the server-side error on echoServer.collectErr), that a Flood server-stream handler blocked in SendMsg behind a client that stopped reading (a static 64 KiB stream window through dialEcho with grpc.WithInitialWindowSize, so the block is deterministic) is released after the deadline while the connection stays open (TestHandlerTimeoutBlockedSend, the server-side error on echoServer.floodErr: DeadlineExceeded on plaintext, Unavailable on TLS where the client then sees Internal), and that nothing is bounded when unset; deadline_unit_test.go (white box) covers every deadlineError branch, the grpc-timeout codec and cap, and Config.shutdownTimeout with both names.

github.com/effective-security/porto/gserver/credentials

Purpose: gRPC credential helpers: a grpccredentials.Bundle (TLS transport plus per-RPC authorization metadata with static, refreshable or DPoP-signed tokens) and a fixed-token PerRPCCredentials.

Files:

  • doc.go — overview and client example.
  • credentials.go — Config, Token, CallerIdentity, Bundle, NewBundle, transport wrapper, perRPCCredential (refreshToken, leadRefresh, finishRefresh, fetchToken, copyToken), TimeISO8601, CacheTTL, TokenFieldNameGRPC.
  • oauth.go — NewOauthAccess.

Entry points: NewBundle(Config) Bundle (TransportCredentials(), PerRPCCredentials(), UpdateAuthToken, WithCallerIdentity, WithDPoP); NewOauthAccess(token); Token.Expired.

Invariants:

  • Process-global vars: TokenFieldNameGRPC = "authorization", CacheTTL = 5m (also the AWS cache TTL in roles), package logger.
  • Metadata written: authorization: "<TokenType> <AccessToken>" and dpop: <proof> when the type is DPoP and a signer is set. DPoP signs POST https://<authority><full method path>: the authority comes from grpc-go's absolute HTTPS credential audience (including TLS server-name/authority overrides), and the full method comes from credentials.RequestInfo. A missing/invalid audience or method returns an error. Each RPC signs a fresh proof; default port 443 may be omitted by grpc-go.
  • RequireTransportSecurity() is always true; NewWithMode returns an error for every mode (the grpclb and RLS balancers request it; a copy would send the token to the balancer).
  • perRPCCredential.mu guards the token, DPoP signer, provider and the in-flight refresh; GetRequestMetadata snapshots them under RLock and reads the signer again after a refresh, because a provider may install the signer bound to the new token with WithDPoP (also for RPCs that shared the refresh).
  • Token refresh happens lazily inside GetRequestMetadata when Expired() (less than one minute left). refreshToken makes one GetCallerIdentity call for all concurrent RPCs, with the context of the RPC that started it; waiters take its token or error and return their own context error when that ends first. A live waiter makes the next call when the shared one failed after its RPC's context ended (tokenRefresh.canceled) or when WithCallerIdentity replaced the provider (superseded: the running call's result is discarded, also for its own RPC). leadRefresh releases the waiters with caller identity provider panicked if the provider panics and lets the panic continue, so a recovered panic cannot leave the refresh pending. The stored token is a copy with Expires always set (CacheTTL from now when the provider sets none); UpdateAuthToken also stores a copy of Expires. Provider errors are wrapped as unable to get caller identity, and a nil token is an error. Without a provider an expired non-empty token is still sent; an empty token sends no metadata.
  • DPoP proof errors are wrapped as unable to sign DPoP proof.

Tests: credentials_test.go and dpop_test.go, no network; TestPerRPCDPoPURI covers absolute proof URIs, IPv6/ports and invalid or missing audience/method inputs; the refresh tests run in testing/synctest bubbles (shared call and shared error, short-lived token, canceled first caller, waiter deadline, provider panic, provider replaced during a refresh, provider installing the DPoP signer during a refresh) and TestPerRPCConcurrentSetters races the setters against RPCs under -race.

github.com/effective-security/porto/gserver/roles

Purpose: Authenticates HTTP and gRPC callers (AWS STS presigned tokens, DPoP JWT, bearer JWT via header or cookie, TLS client cert with SPIFFE SAN) and maps them to roles, producing identity.Identity.

Files:

  • doc.go — overview, example, YAML layout of identity_map.
  • config.go — IdentityMap (debug_logs, strict, cookies{auth,csrf,domain}, skip_auth, tls, jwt, jwt_dpop, aws), GenericIdentityMap, JWTIdentityMap, AWSIdentityMap, CookiesConfig.
  • roles.go — IdentityProvider, New, IdentityFromRequest / IdentityFromContext, per-method verifiers (awsIdentity, dpopIdentity, jwtIdentity, tlsIdentity), CSRF checks (enforceCSRFCookieAndHeader, enforceCSRFMetadata, checkCSRF), STS lookups (awsCallerIdentity, leadAWSLookup, cachedAWS, lookupAWS, cacheableSTSFailure, stsErrorCode, withoutURL), dumpDM, ValidateSTSPresignedURL, ParseSTSTokenExpiration, CallerIdentity, role-name constants.

Entry points:

  • New(*IdentityMap, jwt.Parser) (IdentityProvider, error) — errors if JWT/DPoP is enabled without a parser, or JWT is enabled with cookies.auth but no cookies.csrf.
  • IdentityProvider.IdentityFromRequest (HTTP) and IdentityFromContext(ctx, uri) (gRPC) — the mappers gserver passes to identity.NewContextHandler / identity.NewAuthUnaryInterceptor.
  • Constants GuestRoleName, TLSUserRoleName, JWTUserRoleName, DPoPUserRoleName, AWSUserRoleName, Default{Subject,Role,Tenant}Claim.

Invariants:

  • Never panics; returns errors only in Strict mode; otherwise falls through to the next method and finally to guest.
  • Evaluation order: SkipAuthPaths (exact path, HTTP only) → AWS4 → DPoP → Bearer header, else cookie cookies.auth with the CSRF double-submit check (checkCSRF: X-CSRF-Token equals the cookies.csrf cookie, constant time) on unsafe HTTP methods and on every gRPC call (enforceCSRFMetadata; gRPC-Web headers arrive as metadata) → TLS peer cert (first cert must carry exactly one spiffe:// URI SAN). A scheme without a space is treated as Bearer. A failed CSRF check or cookie JWT falls through and never returns an error, even in Strict; CSRF errors never contain token values.
  • Headers/metadata read: Authorization, DPoP, Cookie, X-CSRF-Token (header.XCSRFToken; metadata authorization, dpop, cookie, x-csrf-token); header.Authority (:authority) for the gRPC DPoP URI; gRPC peer TLS state via peer.FromContext. With debug_logs, dumpDM logs the incoming metadata with authorization, proxy-authorization, cookie, dpop and x-csrf-token values replaced by [REDACTED].
  • Role lookup: reverse map claim value → role built once in New (last role wins); fallback default_authenticated_role. JWT claims default to sub/email/tenant. AWS subject form "<account>:<type>/<name>" or full ARN; allowed_accounts filter.
  • AWS4 tokens are base64url presigned STS URLs. ValidateSTSPresignedURL requires https, no userinfo, an AWS STS host (global, regional, FIPS, China, or VPC endpoint under amazonaws.com) and Action=GetCallerIdentity before any request is made; the lookup uses a dedicated client (provider.sts, stsHTTPClient) with a 10s timeout and a 64 KiB body cap. The presigned URL is a bearer credential: failure logs name only the host, and withoutURL drops the URL from *url.Error causes (parse and transport errors) before they are wrapped. Successful lookups are cached in a 100-entry LRU (awsCache) keyed by URL for credentials.CacheTTL as read by New (awsTTL; not positive: no expiry, until evicted). Failures caused by the URL are cached for awsFailureTTL (30s) in a separate 100-entry LRU (awsFailures). Both are plain lru.Cache values with a per-entry expiry checked on read (an expired entry is ignored until replaced or evicted), so New starts no goroutine and IdentityProvider needs no Close. Bad tokens cannot evict successes: STS 4xx except 408, 429 and the throttling codes in transientSTSErrorCodes (STS throttles with HTTP 400; stsErrorCode reads the JSON or XML error code), and an undecodable 200 body. Transport errors, timeouts, 408, 429, throttling and 5xx are not cached. A cached failure returns cached STS lookup failure: <error>. Concurrent lookups of one uncached URL share one STS call (awsLookups under lookupMu), made with the context of the request that started it, so a reset request still cancels its call; results are cached before the entry is removed. A waiter returns unable to get Caller Identity from AWS: <context error> when its context ends first and makes the next call when the shared one failed after its request's context ended; leadAWSLookup releases the waiters with STS lookup panicked if the lookup panics and lets the panic continue on the request.
  • DPoP proofs are verified against https://<host><path> for HTTP and POST https://<incoming :authority><full method path> for gRPC. gRPC requires exactly one authority of the form host[:port] with a non-empty hostname, parsed from its escaped wire form (grpcDPoPURI; an IPv6 zone arrives as %25); missing, ambiguous, hostless or malformed authority, an authority with userinfo, path, query or fragment, relative legacy proofs and authority/method/key mismatches fail authentication (error in Strict, otherwise fallback). xpki normalizes host case and default HTTPS port 443; method path case is significant. Forwarded headers never supply the gRPC authority. A JWT with cnf is rejected on the bearer path.

Tests: dpop_test.go (TestGRPCDPoPBinding: real signed access tokens and proofs, successful IPv6, IPv6-zone and default-port binding, rejected non-host[:port] authorities and authority/method/key failures in Strict and fallback modes); roles_test.go (mock JWT parser, createPeerContext, TestValidateSTSPresignedURL, cookie CSRF over gRPC metadata, New cookie validation), csrf_test.go, internal_test.go, aws_test.go (white-box: replaces provider.sts with a counting RoundTripper to check the success and failure caches including throttling codes, success expiry (TestAWSIdentitySuccessExpires, in a testing/synctest bubble, which also fails if New leaves a goroutine behind), failure expiry, that failures do not evict successes, the shared lookup, canceled waiters and leaders, a panicking lookup and URL-free parse errors), aws_errors_test.go (token decoding, expiry, host, account and ARN rejections with STS call counts; Strict versus guest on HTTP and gRPC) and strict_test.go (Strict versus guest for DPoP and SPIFFE failures, SkipAuthPaths, ApplicableFor*). No network.

github.com/effective-security/porto/restserver

Purpose: HTTP/HTTPS REST server hosting Service implementations behind an httprouter mux with a fixed middleware chain.

Files:

  • server.go — HTTPServer lifecycle (New/StartHTTP/StopHTTP), middleware assembly in NewMux, lifecycle events, GetServerURL helpers.
  • router.go — Router interface over httprouter, Params, Handle, CORSOptions (rs/cors adapter including wrapAllowOriginRequestFunc); proxyHandle records each route's registered path with telemetry.SetRoute for the metrics uri label.
  • config.go — Config / TLSInfoConfig contracts, GetPort, GetHostName.
  • service.go — Service interface.

Entry points:

  • New(version, ipaddr, Config, *tls.Config) (*HTTPServer, error) — nil TLS means plain HTTP.
  • WithAuthz / WithIdentityProvider / WithTrustedProxies / WithCORS / WithShutdownTimeout / WithTimeouts / WithMaxRequestBody / WithMuxFactory — configure before StartHTTP; all but WithMuxFactory return the server for chaining.
  • AddService, StartHTTP, StopHTTP, NewMux, IsReady, OnEvent; Router methods; GetServerURL, GetServerBaseURL.

Invariants:

  • StartHTTP binds HTTP and HTTPS listeners synchronously and returns bind errors; serving remains asynchronous. A failed bind can be retried, but a successfully started instance cannot be restarted.
  • AddService panics on duplicate names; NewMux panics if the authz handler cannot be built.
  • Chain (outer → inner): trusted proxy policy → correlation → handler deadline (if Timeouts.Handler is set) → body limiter (bounds reads, writes no response) → metrics → identity → metrics (role handoff, records nothing) → request logger → CORS (if set) → authz (if set) → ready → router. The outer metrics handler also counts identity mapper rejections, with the guest role. CORS (newCORS, shared with NewRouterWithCORS) sits outside authz so rs/cors answers preflights before authorization and denied or not-ready responses carry CORS headers; with OptionsPassthrough an OPTIONS request continues into authz. NewRouterWithCORS still wraps only the router and is for custom mux factories, which must keep CORS outside authz themselves. WithTrustedProxies takes a policy from identity.ParseTrustedProxies (CIDR errors are returned there); nil trusts no proxy. StartHTTP applies the policy, the body limit and the handler deadline to custom mux factories too. Default identity mapper is identity.GuestIdentityMapper; logger granularity is time.Millisecond.
  • StopHTTP: mark unready → wait for Started callbacks → broadcast Stopping → Shutdown with shutdownTimeout (default 5s) → Service.Close() for all → broadcast Stopped. Calls before start do nothing; concurrent and repeated calls wait for the first shutdown. A timeout still closes services after Shutdown returns. Lifecycle callbacks must not call StopHTTP synchronously.
  • serving is an atomic.Bool; IsReady reads services under lock.RLock. HTTPServer.Config() and HTTPConfig() return the same configuration. Event handlers are copied under lock.RLock before callbacks run, so callbacks may register handlers.
  • GetPort uses net.SplitHostPort for host:port and defaults bare IPv6 literals to port 443. GetHostName returns IPv6 hosts without brackets; GetServerURL and GetServerBaseURL rebuild host:port with net.JoinHostPort.
  • GetServerURL accepts X-Forwarded-Proto only for a configured trusted peer and only when the value is http or https. 404 → JSON not_found. MaxRequestSize is enforced (10 MiB). WithMaxRequestBody overrides it, including for custom muxes; zero selects the default, negative disables it.
  • WithTimeouts applies shared HTTP defaults (header 10s, read 30s, idle 60s); zero selects defaults, negative disables an individual deadline. Native net/http TLS handshakes use the smaller positive header/read timeout; Handshake is for cmux/eager transport listeners. No write deadline is imposed. A positive Handler bounds the context of every request (limits.Timeouts.ApplyHandler, inside correlation), keeping an earlier client deadline; it is off by default, and a handler that ignores its context outlives it.

Tests: rest_test.go suite builds CA/server/client chains with xpki/testca in a temp dir; server_test.go and service_test.go use random ports via tests/testutils and example_test.go binds 127.0.0.1:0; router_test.go CORS and route-template metric labels (Test_RouterRouteMetrics, process-global sink, not parallel); example_test.go uses testdata/test-server*.pem; limits_test.go checks real stalled headers/bodies, idle connections and custom-mux body limits, that a default-mux 413 from DecodeBody carries router CORS and correlation headers, and (TestHandlerTimeout) that Timeouts.Handler bounds the request context of the default and a custom mux and leaves it unbounded when unset; server_test.go TestServer_CORSPreflightAuthz checks that preflights are answered before authz, that OPTIONS is otherwise authorized (CORS disabled or OptionsPassthrough), and that denied responses carry CORS headers.

github.com/effective-security/porto/restserver/authz

Purpose: path-segment/role access control tree for HTTP handlers and gRPC interceptors, configurable from YAML/JSON.

Files: authz.go — Config, Provider (tree build/walk/clone, NewHandler, interceptors, access logging); doc.go.

Entry points: New(*Config) parsing allow entries "/path:role1,role2"; Allow, AllowAny, AllowAnyRole, SetRoleMapper, SetGRPCRoleMapper, Clone; NewHandler(delegate), NewUnaryInterceptor(), NewStreamServerInterceptor(); interfaces HTTPAuthz, GRPCAuthz.

Invariants:

  • Paths must start with / or walkPath panics; match is by whole segment on the deepest configured node; AllowAny beats roles; guest and empty roles are never allowed by Allow/AllowAnyRole.
  • NewHandler snapshots through Clone; interceptors use the live provider. An empty tree denies everything.
  • Every method is checked, including OPTIONS and CORS preflights: the preflight headers are caller-controlled, so a CORS middleware that answers preflights must run outside the authz handler (restserver.NewMux and gserver do; custom muxes must too). With OptionsPassthrough, OPTIONS requests reach authz and need a permitted role. Denial (denied) is httperror.Unauthorized (401 / codes.Unauthenticated) for an empty or guest role and httperror.Forbidden (403 / codes.PermissionDenied) for any other role; the message is <role> role not allowed (an empty role is reported as guest), the request ID is carried in request_id, and the HTTP handler writes the error from checkAccess as is (no double wrapping).
  • Mutators are unsynchronized: configure before serving.
  • Config tags: allow, allow_any, allow_any_role, log_allowed_any, log_allowed, log_denied, log_level, logger_skip_paths[{path,agent}].

Tests: authz_test.go — tree walk, YAML load, handler and interceptor allow/deny with identity.WithTestIdentity.

github.com/effective-security/porto/restserver/ready

Purpose: readiness gate middleware returning JSON 503 not_ready until ServiceStatus.IsReady() is true.

Files: ready.go — ServiceStatus, NewServiceStatusVerifier.

Invariants: IsReady is called on every request and must be cheap and thread-safe. The 503 not_ready error is built per request with httperror.NotReady, so the request_id in the body is always the current request's; nothing is shared between requests.

github.com/effective-security/porto/restserver/telemetry

Purpose: request logging and request metrics middleware plus the ResponseCapture writer.

Files: requestlogger.go (RequestLogger, NewRequestLogger, LoggerSkipPath, ShouldSkip, WithLoggerSkipPaths), request_metrics.go (NewRequestMetrics, UnknownRoute, OtherMethod), route.go (SetRoute, requestLabels: the per-request holder of the route and the role reported by a nested metrics handler), response_capture.go (ResponseCapture, Started).

Invariants:

  • NewRequestLogger panics on a nil handler; a nil logger returns the handler unwrapped; a granularity below 1ns is clamped to 1ns (the duration is logged in nanoseconds).
  • Log fields: method, path, status, bytes, duration, remote (identity.IPString(identity.ClientIPFromRequest(r)), the IP cached by identity.NewTrustedProxyHandler when present, "" for a Unix socket peer), agent (NoAgent, no-agent, without a User-Agent; ShouldSkip matches the same value); INFO via ContextKV. gserver logs gRPC calls with the same remote and agent names.
  • Metric tags: verb, status, uri, role; every request, including 404s, records both HTTPReqPerf and HTTPReqByRole. No tag value is copied from the request: uri is the route template recorded with SetRoute (the restserver Router records its registered path, e.g. /v1/users/:id) or UnknownRoute (unknown) for unmatched paths and responses produced before or instead of a route handler (identity mapper 401s, gserver CORS preflights and rate-limit 429s, authz 401/403, ready 503, restserver CORS preflights, httprouter's 405, automatic OPTIONS and redirects, and gserver WithMiddleware handlers that answer without calling SetRoute); verb is the method when it is one of the 9 standard net/http methods, else OtherMethod (_OTHER). NewRequestMetrics puts a label holder (requestLabels) in the request context; a NewRequestMetrics that finds one (nested) records nothing and stores the role of the identity in its request context (identity.FromContext) in it, so each request is recorded once, by the outermost handler. SetRoute without a holder is a no-op, the last call wins, and the holder is atomic because http.TimeoutHandler can leave the handler running after the metrics are recorded. Custom routers behind NewRequestMetrics must call SetRoute with the registered pattern, never with a value from the request. The role tag is whatever the identity mapper returned, as reported by the innermost nested handler; without one (or when the request never reached it) it is the role in the outer request's context, guest when none was stored. Both servers place one metrics handler outside identity.NewContextHandler and one inside it, so identity rejections are counted as guest; custom mux factories need the same pair to count them. The outer handler records in a defer, so a handler that panics (or calls runtime.Goexit) is counted with the status it sent, or 500 when it sent none (ResponseCapture.wroteHeader); the panic, http.ErrAbortHandler included, is not recovered and reaches the server with its original stack.
  • LoggerSkipPath tags path, agent; * wildcard; agent is a substring match.
  • ResponseCapture captures the status net/http sends: the first final WriteHeader status (1xx other than 101, codes outside 100-999, which net/http rejects with a panic, and later calls are ignored), or 200 once Write, a ReadFrom that copied a byte, or a flush the chain supports sent the header; Started() reports that a final status was sent or the connection was successfully hijacked; informational responses alone do not start a response.
  • ResponseCapture implements Unwrap, so http.ResponseController (deadlines, full duplex, flush, hijack) reaches the delegate, and http.Flusher, FlushError and http.Hijacker through http.ResponseController: when the delegate chain lacks the feature Flush is a no-op, and FlushError (which ResponseController.Flush calls) and Hijack return an error wrapping http.ErrNotSupported (HTTP/2 cannot hijack). Captured status and size exclude bytes written to a hijacked connection; fields are unsynchronized (one handler goroutine). ReadFrom (io.ReaderFrom) counts the copied bytes and calls the delegate's ReadFrom when it has one (so io.Copy, http.ServeContent and http.FileServer keep the sendfile path of *http.response through both captures), else io.Copy into the delegate; like Write, it returns the delegate's error unwrapped.

Tests: request_metrics_test.go (internal; exact label sets through a process-global in-memory sink, so those tests are not parallel; Test_RequestMetricsNested checks the role handoff, including rejections before the nested handler, an empty role and three levels; Test_RequestMetricsTimeoutRace overlaps SetRoute with the metrics read, run with -race; Test_RequestMetricsPanic checks the status recorded for panics before and after a status, body, flush or 1xx, http.ErrAbortHandler and runtime.Goexit, and that the panic value propagates; Test_RequestMetricsPanicServer checks on a real httptest.Server that net/http still logs the panicking handler's stack), requestlogger_test.go (internal; log lines through the global xlog formatter), response_capture_test.go (black box; ResponseController, flush, hijack and http.ServeContent through the logger and metrics chain on a real httptest.Server, ReadFrom delegation and counting, TestResponseCapture_Status for the captured status, and BenchmarkResponseCapture_ServeFile for the sendfile path; serveChain waits for the chain to return, because a hijacked response reaches the client first).

github.com/effective-security/porto/xhttp/correlation

Purpose: correlation/request ID propagation across HTTP, gRPC metadata, contexts and logs.

Files: correlation.go.

Entry points: NewHandler(delegate), NewAuthUnaryInterceptor(), NewStreamServerInterceptor(), ID(ctx), Value(ctx), WithID(ctx), WithMetaFromContext(ctx), WithMetaFromRequest(r), NewFromContext(ctx), Correlator, IDSize, CorrelationIDgRPCHeaderName.

Invariants:

  • Incoming ID sources (HTTP): X-Correlation-ID, then X-Request-ID; (gRPC) x-correlation-id, x-request-id; truncated to 12 chars; otherwise random.
  • Writes the response header X-Correlation-ID and the xlog KV ctx.
  • WithMetaFromRequest copies x-*, grpc-*, authorization, date, timestamp headers into incoming and outgoing MD (forwards credentials).
  • gRPC metadata keys are lowercase; WithMetaFromRequest forwards the resolved (possibly truncated) correlation ID under CorrelationIDgRPCHeaderName and gives any x-request-id alias the same value.
  • NewFromContext detaches from cancellation. gRPC interceptors recover panics into codes.Internal with message unhandled exception.

github.com/effective-security/porto/xhttp/header

Purpose: constants for HTTP header names and content-type values, including Authority (:authority) for gRPC DPoP binding, Vary for response negotiation, XForwardedFor, XRealIP, XForwardedProto for proxy headers, Origin, AccessControlRequestMethod for CORS preflight requests, RetryAfter for 429/503 retries, XCSRFToken for the gserver/roles cookie CSRF check, and XClientHostname, XClientIP (informational, set by pkg/retriable WithUserAgent). GRPCStatus and GRPCMessage name the gRPC status trailers and GRPCTimeout the client deadline header that gserver caps on TLS listeners. No imports.

github.com/effective-security/porto/xhttp/httperror

Purpose: structured API errors with HTTP status, code, gRPC status and correlation ID; JSON writer; HTTP ↔ gRPC code mapping; classification helpers.

Files: errors.go (Error, constructors, Wrap/WrapWithCtx, Is*, Status, WriteHTTPResponse), codes.go (Code*, httpCode, codeStatus, statusCode, HTTPStatusFromRPC, FromOAuth), many.go (ManyError), rpc.go (NewGrpc, NewGrpcFromCtx, NewFromPb, GRPCStatus, CorrelationID; status details and restoredHTTPStatus), encoder.go.

Invariants:

  • Wire JSON {code, message, request_id?}; ManyError adds errors{}.
  • WithContext/WithCause mutate the receiver. WriteHTTPResponse (both Error and ManyError) never does: when RequestID is empty, the body is written from a per-request copy carrying the correlation ID, so package-level Error values can be shared across requests. ManyError.WriteHTTPResponse snapshots the fields and clones Errors under the Add lock (snapshot) and releases it before writing, so a slow client never blocks Add or other writers.
  • errMsg (behind Wrap/WrapWithCtx) never panics: a non-string first value with further values is formatted with %+v and space-joined instead of being used as a format. It does not forward the values to a fmt print function, so vet does not treat Wrap as a print wrapper.
  • RequestTooLarge returns HTTP 413 with request_too_large, sent over gRPC as ResourceExhausted.
  • Mapping: Unauthenticated ↔ 401 unauthorized (CodeUnauthorized) and PermissionDenied ↔ 403 forbidden (CodeForbidden), so 401/403 round-trip through gRPC; Canceled/DeadlineExceeded → 408; ResourceExhausted → 429; FailedPrecondition → 400 bad_request (google.rpc.Code; *Error.RPCStatus keeps the gRPC code); CodeTimeout → DeadlineExceeded; conflict → AlreadyExists; precondition_failed → FailedPrecondition; unknown codes → RPCStatus 0.
  • Error.GRPCStatus and ManyError.GRPCStatus attach string details @httperror.code (the Code) and @correlation.id (the RequestID) when set; a status with code OK carries no details. NewFromPb, Wrap and Status convert a status with the Google mapping, except that a code listed in restoredHTTPStatus (only request_too_large → 413) is restored when its detail matches the gRPC code. Other codes keep the lossy mapping.
  • Wrap classification is substring-based ("invalid"/"bad"/"400" → 400; "not found"/"404" → 404; "timeout"/"deadline"/"cancel" → 408).
  • Is compares Code and Message; Unwrap skips one wrapper level of the cause.

github.com/effective-security/porto/xhttp/identity

Purpose: caller identity (role/subject/tenant/claims/auth method) and connection facts (client IP, UA, target) in request contexts for HTTP and gRPC.

Files: identity.go (Identity, NewIdentity, AuthMethod, guest mappers, WithTestIdentity), ctx.go (RequestContext, FromContext/FromRequest/AddToContext, NewContextHandler, NewAuthUnaryInterceptor/NewStreamServerInterceptor), realip.go (TrustedProxies, ParseTrustedProxies, WithTrustedProxies, NewTrustedProxyHandler, ClientIPFromRequest, ClientIPFromGRPC, IPString, ForwardedProto), basicauth.go.

Invariants:

  • Never returns a nil identity: guest (GuestRoleName = "guest", MethodNone) when missing. Mapper error (identityError) → logged, then a generic HTTP 401 unauthorized JSON / gRPC codes.Unauthenticated with message invalid identity; the mapper's error text is never sent. A non-nil *httperror.Error in the mapper error's chain is the mapper's client-safe answer and is returned unmodified when it has an HTTP status (over gRPC only when its code also has a gRPC mapping; otherwise a plain Unauthenticated status with its message); one without an HTTP status, or a typed nil, gets the generic 401.
  • An existing RequestContext in ctx is respected (test injection). FromRequest fills in an empty target and user agent and an invalid ClientAddr from the request, in place, but leaves a context marked resolved unchanged: NewContextHandler and createIdentityContext set it, because they built the context from the request, so concurrent FromRequest calls in a served handler never write (TestFromRequestConcurrent, run with -race). WithTestIdentity sets both clientAddr and clientIP from the local IP and is not resolved.
  • Client IP type: ClientIPFromRequest and ClientIPFromGRPC return netip.Addr. A caller whose peer is not an IP (empty RemoteAddr, a Unix socket peer @, a gRPC call without a peer) has no client IP: the zero Addr (IsValid false). IPString formats it as ""; netip.Addr.String would return "invalid IP", which as a rate limit key put every such caller in one bucket, so every log remote, the gserver limiter key and RequestContext.ClientIP go through IPString. RequestContext keeps the Addr (ClientAddr, also on the Context interface) and its IPString (ClientIP); every place that sets one sets the other.
  • Without a trusted proxy policy, client IP is the socket peer and forwarded headers are ignored. ParseTrustedProxies validates CIDRs; WithTrustedProxies stores the immutable policy in context (replacing any policy and cached IP); NewTrustedProxyHandler applies it to HTTP and resolves the client IP once per request. The context value (forwarding) holds the policy plus the resolved IP and the RemoteAddr it was resolved for; ClientIPFromRequest returns the cached IP when RemoteAddr still matches, so header changes after the trust boundary are ignored and a request copy with another peer is resolved again. A request with an empty or non-IP RemoteAddr has no client IP and returns the zero Addr (the pre-B05 fallback to netutil.GetLocalIP was removed; before the netip.Addr change a Unix socket peer returned @). For trusted peers, XFF is walked right to left to the first untrusted address, including private addresses, or to the leftmost entry when every hop is trusted; malformed values fall back to the peer. IPv4-mapped IPv6 addresses are returned as IPv4, including the socket peer when it is the answer (forwardedClient checks trust on the mapped form, then unmaps). IPv6 zones (fe80::1%eth0) are ignored when matching trusted CIDRs, because netip.Prefix.Contains rejects zoned addresses; returned addresses keep their zone. X-Real-Ip is used only when XFF is absent. gRPC metadata follows the same rules. Proxies must overwrite client supplied forwarding headers. ForwardedProto accepts only http or https from a trusted peer.
  • Unary gRPC interceptor recovers panics; the stream one does not. Only the first trusted argument of the gRPC interceptors is used; nil keeps a policy already in the context.
  • Rejections are logged at WARNING with the client IP as remote: identityMapper (HTTP, with target) and access_denied (gRPC, with method). createIdentityContext resolves the client IP once, for both its WARNING line and the RequestContext. The package logger is registered as repo github.com/effective-security/porto/xhttp, package context (the pkg field of its lines).

Tests: ctx_test.go (white-box; TestRejectedIdentityLogsRemote installs a process-global JSON log formatter and is not parallel; TestFromRequestClientIP checks ClientIP/ClientAddr for a Unix socket peer and a mapped peer; TestFromRequestConcurrent is the -race check above), identity_test.go (Test_WithTestIdentityDirect checks the local IP survives FromRequest), realip_test.go (trust policy, forwarding headers, Unix socket and mapped peers, cached client IP, IPString), basicauth_test.go.

github.com/effective-security/porto/xhttp/marshal

Purpose: JSON response writing (with errors, gzip, pretty-print) and strict JSON request decoding via ugorji codec.

Files: limits.go (LimitRequestBody, shared body-limit policy), marshal.go (WriteJSON, WritePlainJSON, WriteHTTPResponse, NewRequest, AcceptsGzip), json.go (PrettyPrintSetting, NewEncoder, EncodeBytes, DecodeBytes, Decode, DecodeBody, DecoderHandle).

Invariants:

  • First non-nil body wins; WriteHTTPResponse implementers write themselves; other errors are converted with httperror.NewFromPb (500 unexpected, or the gRPC status mapping) and, when the result is 5xx, sent with the generic http.StatusText message while the original error is kept as the cause and logged; 4xx conversions keep their message. Explicit httperror values are written as given. Non-404 errors are logged with the caller's file and line (no recursion through WriteJSON).
  • Success: 200, application/json; exported AcceptsGzip(http.Header) is shared with gRPC-Web; gzip for payloads of at least 1 KiB when Accept-Encoding allows gzip with positive quality (a missing q means 1, a malformed or out-of-range q means 0; explicit gzip denial overrides *), pretty when ?pp. Vary: Accept-Encoding is merged with existing values; gzip writers are pooled. r must be non-nil.
  • Decoding is strict (ErrorIfNoField), maps → map[string]any. DecodeBody defaults to a 10 MiB body cap via http.MaxBytesReader; LimitRequestBody overrides it (zero = default, negative = disabled). LimitRequestBody never writes a response, so it may wrap CORS and telemetry middleware: a body whose known Content-Length exceeds the limit is replaced by oversizedBody, which returns *http.MaxBytesError on the first read without reading any bytes, and net/http closes the connection when the unread remainder is too large to drain. The request always reaches the handler. DecodeBody consumes the remaining bounded body after decoding so trailing data cannot bypass the cap; with the limit disabled the remainder is left unread. Overflow returns/writes a structured 413; malformed JSON remains 400. Other body readers must handle *http.MaxBytesError. Decode/DecodeBytes remain unbounded.
  • Encoder handles are package globals initialised in init.

Tests: marshal_test.go covers negotiation, the compression threshold, Vary, decompressed payloads, and log line numbers derived from runtime.Caller; marshal_bench_test.go measures compressed responses. limits_test.go covers exact/unknown-length/oversized/trailing bodies, defaults and overrides; a counting reader checks that known oversized bodies are never read.

github.com/effective-security/porto/xhttp/limits

Purpose: shared conservative server timeout and request-body defaults, and the optional handler deadline, without internal dependencies.

Files: doc.go, limits.go (Timeouts, WithDefaults, ApplyHTTP, HandlerContext, ApplyHandler, default constants).

Invariants: header 10s, read 30s, idle 60s, handshake/detection 10s, body 10 MiB. Zero selects defaults; negatives explicitly disable individual limits. Header applies to HTTP/1.x through net/http.ReadHeaderTimeout; net/http has no per-stream HTTP/2 header deadline. HTTP timeout application preserves negative values to avoid net/http fallback deadlines; no response write deadline is set. Call before serving. Native net/http TLS handshakes use the smaller positive header/read deadline; Handshake applies to cmux and eager transport listeners. Handler has no default: zero and negative leave requests bounded only by the client's deadline. It is not a net/http setting, so ApplyHTTP ignores it; HandlerContext bounds a context with context.WithTimeout (an earlier parent deadline is kept) and ApplyHandler wraps an http.Handler with it, returning the handler itself when Handler is not positive. Servers (gserver, restserver, pkg/appinit Prometheus) call ApplyHandler on their chains; gserver uses HandlerContext in its gRPC interceptors.

Tests: limits_test.go covers defaults, overrides, explicit disabling, write-timeout preservation, HandlerContext (unset, no parent deadline, earlier and later parent deadlines, expiry) and ApplyHandler (unset returns the handler itself).

github.com/effective-security/porto/pkg/retriable

Purpose: HTTP client with retry policy, JSON marshalling, context header propagation, Bearer/DPoP authorization, replay nonces and file-based token/key storage. See also pkg/retriable/README.md.

Files: doc.go, retriable.go (Client, options, Policy, Request/Do, DecodeResponse, context header helpers), client.go (Head/Get/Post/Put/Delete), config.go (Config/ClientConfig/RequestPolicy/TLSInfo, Factory, LoadClient, SetAuthorization, HostFolderName), storage.go (Storage, AuthToken, ParseAuthToken, KeyInfo), nonce.go (NonceProvider), request.go (Request, Requestor), dumpreq.go (DumpRequestOut).

Entry points: New(cfg, opts...), Default(host), LoadClient(file), NewFactory/LoadFactory → CreateClient/ForHost; Client.Request/RequestURL/HeadTo, Get/Post/Put/Delete/Head, Do; SetAuthorization(); Policy, DefaultPolicy(); WithHeaders(ctx, map), PropagateHeadersFromRequest; Storage.*.

Invariants:

  • Errors, not panics. WithTLS/WithDNSServer install a modified Clone of the current *http.Transport (of http.DefaultTransport, pooled to 100 connections per host, when none is set) and never modify a supplied transport; on another http.RoundTripper (or a non-*http.Transport http.DefaultTransport) they record configErr, which New and Do return (closing the request body) until WithTransport replaces the transport (fail closed). replaceTransport closes the idle connections of a replaced transport the client created (ownTransport), never of one passed to WithTransport; connections in use at that moment close when idle on HTTP/1.1, otherwise after the transport's IdleConnTimeout (90s for http.DefaultTransport clones). New applies ClientConfig.TLS with WithTLS after the options, to the transport they leave, unless tlsSet is still true (set by WithTLS, cleared by WithTransport), so only a WithTLS option that no later WithTransport replaced wins; on a non-*http.Transport from WithTransport New fails closed. WithTransport treats a nil *http.Transport as nil (http.DefaultTransport).
  • DEBUG request dumps redact Authorization, DPoP, Cookie, and Proxy-Authorization without mutating the request; Client.Do omits request bodies from its DEBUG dump.
  • Client setters lock mutable fields; Do snapshots headers, policy, name, signer, and a copy of the *http.Client (so transport setters are safe during requests). Caller-identity refresh (callerToken, leadRefresh, finishRefresh) is shared by concurrent requests, with context-aware waiting and a fast path for valid tokens or no provider; changing the provider invalidates an in-flight refresh. A panicking provider releases the waiters with caller identity provider panicked and the panic continues on the request that made the call; the next request calls the provider again. Direct mutation of exported Name, Policy, Config, or the returned HTTPClient() still requires caller synchronization.
  • ShouldRetry order: ctx done → TotalRetryLimit → NonRetriableErrors → Retries[0]; status < 400 succeeds; 404 stops; 429 → Retries[429], else LimitExceeded; other 4xx stop; 5xx → Retries[status]. DefaultShouldRetryFactory(limit, ...) retries while retries <= limit (limit+1 retries). DefaultPolicy: total 5; connection errors 4x/2s; 502/503 5x/1s; 429 3x after Retry-After (1s without it; > 30s stops with LimitExceeded); no timeout. RetryAfterShouldRetryFactory parses delay-seconds or an HTTP date (past → 0, overflow saturates). A request: config block (RequestPolicy) replaces only non-zero values (cmp.Or); negative retry_limit disables retries.
  • Do waits between attempts with waitRetry (timer + ctx.Done()), returning "<METHOD> <redacted URL>: waiting to retry: <ctx err>"; while ctx is live, a retry whose wait would pass its deadline is skipped and the last response returned unread (after the deadline, the ctx error is returned). Retried bodies are drained up to 64 KiB (maxErrorBodySize) and closed; DecodeResponse reads at most 64 KiB of an error body (and returns read errors); DEBUG dumps log headers plus that bounded body.
  • RequestTimeout applies to Request and helpers, not Do; executeRequest wraps the response body in cancelOnClose, so closing the body (as Request/HeadTo do) releases the timeout context, and a failed request releases it at once.
  • RequestURL derives host (re-escaped, for IPv6 zones) and RequestURI() (+ fragment) from the parsed URL; it rejects URLs without a scheme, opaque URLs and URLs with user info.
  • NonceProvider.NonceContext(ctx) bounds a nonce fetch by ctx; Nonce() (jose.NonceSource) returns a cached nonce or fetches with a 30s timeout (nonceFetchTimeout). The LIFO cache holds 64 nonces; when full, the oldest half is dropped.
  • WithUserAgent reads os.Hostname and netutil.GetLocalIP once (no network wait; localIP is a test seam) and omits values it cannot resolve.
  • Headers written: X-Correlation-ID, Authorization, DPoP, User-Agent, X-CLIENT-HOSTNAME, X-CLIENT-IP; context headers override client headers.
  • Authorization only for https:// / unixs:// hosts; token formats: opaque or access_token=&exp=&dpop_jkt= (optional refresh_token, aliases id_token/token). ParseAuthToken always sets TokenType to Bearer and ignores token_type; clients select DPoP whenever dpop_jkt is present.
  • DPoP proofs are signed separately for each retry attempt; a DPoP authorization header without a configured signer returns an error.
  • Config YAML: host, tls{cert,key,trusted_ca}, request{retry_limit,timeout}, storage_folder; ~ and $VAR are expanded; LoadFactory appends HostFolderName(host) to storage_folder.
  • NewStorage expands a leading ~ (alone or before a separator) with os.UserHomeDir, then user.Current().HomeDir, falling back to the literal folder; no ~user or $VAR expansion (config loaders expand $VAR with resolve.ExpandPath first).
  • Storage files: .auth_token and <thumbprint>.jwk are written through mode-0600 temporary files and atomically replace existing files or symlinks. A newly created credentials folder uses mode 0700; existing directory permissions stay intact. An empty StorageFolder uses the working directory without changing its mode. A public-only DPoP key is rejected by SetAuthorization. ListKeys skips unreadable files and subdirectories without losing later siblings; a missing or unreadable root produces an empty list with nil error. Clean intentionally ignores removal errors.

Tests: retriable_test.go (httptest servers; TestMain writes a test CA/client cert under $TMPDIR/test-retriable), policy_test.go (config defaults, 429/Retry-After, ctx-aware backoff, body draining/bounds, timeout release, RequestURL, transport cloning, ClientConfig.TLS precedence over WithTransport/WithTLS against an httptest TLS server (TestConfigTLSWithTransport), idle-connection release (server ConnState counting) and fail-closed errors, a setter/request race; TestTransportSettersWithoutHTTPDefaultTransport swaps http.DefaultTransport and must stay non-parallel), safety_test.go (credential redaction, concurrent refresh, a panicking provider, a provider replaced during its refresh (normal return and panic), retry signing, storage permissions, ~ expansion with t.Setenv), config_test.go with testdata/*.yaml, nonce_test.go (cache order, fetch context via a fake HTTPClient), internal_test.go (needs outbound DNS to 8.8.8.8; the localIP seam test is non-parallel), storage_test.go (NewKeyInfo algorithms, ListKeys filtering and unreadable subdirectory/root handling, ParseAuthToken, write failures with exact errors and no temporary files left; permission tests skip as root), clientconfig_test.go (token from env versus file, NewForHost, LoadClient, SetAuthorization; t.Setenv tests non-parallel), request_test.go (NewRequest and body kinds, errors before sending, nil contexts, ShouldRetry on transport errors) and dumpreq_test.go (DumpRequestOut body handling and errors).

github.com/effective-security/porto/pkg/rpcclient

Purpose: build a *grpc.ClientConn plus default call options from a Config (TLS bundle, Bearer/DPoP per-RPC credentials, keepalive, message limits, optional blocking dial).

Files: doc.go, client.go (Client, New/NewFromURL, newClient, dial), config.go (Config, token loading, Storage).

Invariants:

  • HTTP(S) scheme prefixes are stripped and :443 is appended when no host port is given. unixs:///path is converted to gRPC's unix:///path resolver target; unix:///path is preserved.
  • When cfg.TLS != nil, the endpoint must start with https:// or unixs://; New otherwise returns an error. A nil TLS config uses insecure.NewCredentials().
  • Defaults: WaitForReady(true), send limit 10 MiB, recv limit math.MaxInt32; Config.CallOptions replaces them.
  • DialTimeout > 0 waits for connectivity.Ready; otherwise lazy. Uses grpc.NewClient. Dial cancellation and timeout errors wrap the context error. Stored public-only DPoP keys return an error; failed construction cancels its context.
  • Config has no yaml/json tags.

Tests: server_test.go starts an in-process gRPC health server on loopback (plain, or TLS with the httptest certificate) that records the metadata of the last call; credentials_test.go (Bearer and DPoP metadata, proofs signed by the stored key, New rejections), dial_test.go (target normalization on lazy clients, a canceled dial, dial timeout, call options and size limits, Close) and config_test.go (token from env versus file, Storage/SetStorage).

github.com/effective-security/porto/pkg/redisclient

Purpose: prefixed, JSON-marshalling wrapper over go-redis v9 with helpers for lists/sets/hashes/zsets, bounded collections, an owner-bound distributed lock and a per-window rate limiter.

Files: doc.go, redisclient.go (Config, Provider/DistributedLock/RateLimiter, RedisClient, parseURL, escapeGlob, setNX, the Lua scripts boundedSetScript/boundedHashScript/releaseLockScript), marshal.go.

Entry points: New(*Config) (root client owning the connection), NewWithClient, NewRedisClient; WithPrefix, Key, SubKey; Get/Set/Del, L*, S*, SAddWithEviction, H*, HSetWithEviction, Z*, ScanKeys/Keys; TryLock/ReleaseLock/IsLocked (lock:<key>), TryAcquireRateLimit/GetRateLimitRemainingTime (ratelimit:<key>).

Invariants:

  • Get/HGet wrap ErrNotFound with a stack; IsNotFoundError uses only errors.Is. Sorted-set errors include the key and wrap the go-redis error; SIsMember formats any member with %v. UnmarshalStringCmd checks the command error before changing any destination, including *string.
  • Prefix is /<trimmed, path-cleaned>/; Key joins it with the key cleaned as a rooted path (path.Join(prefix, path.Clean("/"+key))), so .. cannot leave the prefix and keys whose .. segments stay below their own first segment keep the layout of the earlier plain path.Join; without a prefix the key is unchanged. SubKey inverts Key (the prefix itself maps to ""). Keys/ScanKeys clean the glob pattern like a key, escape the prefix's glob metacharacters (pattern, globPrefix) and return keys relative to the prefix. Prefixes do not nest.
  • Only the root client's Close closes the connection; it returns the close error, is idempotent (redis.ErrClosed from a second close is ignored) and keeps the embedded client, so later commands on the root and its WithPrefix children fail with redis.ErrClosed instead of panicking.
  • Config YAML: server, ttl, client_tls{cert, key, trusted_ca}, user, password; password overrides URL credentials. NewRedisClient logs only the parsed address and DB at INFO; parseURL drops the URL from a *url.Error (it may embed a password) and keeps the reason.
  • Lock: TryLock stores a crypto/rand.Text token with SET NX PX and reads PTTL in the same MULTI/EXEC (setNX); it returns the token or "" plus the holder's remaining TTL (at least 1ms, 0 for a key without expiry). ReleaseLock(key, token) is a Lua compare-and-delete; an empty token is an error. Locks written by earlier versions hold a timestamp and are never released by a token (they expire, except zero-timeout ones, which need a manual DEL). Lock and rate-limit keys go through coordKey, which, under a prefix, rejects a key whose .. segments leave the lock:/ratelimit: sub-namespace (without a prefix keys are not cleaned).
  • Rate limiter: TryAcquireRateLimit is SET NX PX on ratelimit:<key> via setNX: one call per window wins, denied calls write nothing, the window runs on the server clock, and an existing key of any type (the earlier sorted set) denies until it expires. An earlier version polling a new window gets WRONGTYPE but its pipelined EXPIRE resets the window (mixed versions fail closed). GetRateLimitRemainingTime is PTTL through remainingTTL: a live key reports at least 1ms (PTTL says 0 during its last millisecond), a missing key or one without expiry 0; the lock's remaining time uses the same conversion. Lock timeouts and windows below 1ms (minExpiry) are errors.
  • Eviction helpers run one Lua script each: LLEN and SADD/HSET first (a wrong-type key fails before any write), append to the order list only when the member or field is new, after LREM of its stale entries (O(list length); re-adds keep their position), then pop and remove until the list holds at most the limit. The limit must be positive and the two keys must differ (compared after Key). A duplicate written by an earlier version for a member still in the set can evict it once.
  • Scripts run through go-redis Script.Run (EVALSHA with EVAL fallback), so the server must allow Lua scripting.
  • Keys uses KEYS (tests only); ScanKeys uses SCAN with COUNT 100 (scanBatch), limit <= 0 means 1000 (defaultScanLimit), and may return more than limit.

Tests: testcontainers redis:8.2 with password redis; needs Docker. Test_Redis covers key cleaning and glob-escaped prefixes (KeyNamespace), bounded collections (SetWithEvictionReAdd, SetWithEvictionStaleEntries, *Errors checking that a wrong-type key writes nothing, HashWithEvictionUpdate), owner-bound locks (DistributedLock, LockExpiredOwner, LockArguments, ConcurrentLocking, CoordinationKeyNamespace) and the rate limiter (RateLimiterPolling with concurrent pollers asserting from each allowed call's client-clock span that windows start at least one window apart, RateLimiterLegacyWindow, ConcurrentRateLimiting). ttl_test.go (internal) covers remainingTTL. TestClose and TestConfigSecrets (log capture with xlog.InstallFormatter) need no Docker: clients connect lazily to an unreachable address; client_errors_test.go adds the other Docker-free tests (every Provider method on a closed client returns redis.ErrClosed, NewRedisClient credentials and TLS, Marshal/UnmarshalStringCmd tables).

github.com/effective-security/porto/pkg/cache

Purpose: Provider cache abstraction (Set/Get with TTL, Delete, paged ListKeys and the deprecated Keys, pub/sub) with memory, Redis and prefixing proxy backends.

Files: doc.go, cache.go (Provider, Subscription, Config, RedisConfig, DefaultTTL, KeepTTL, NowFunc, GetOrSet, ErrNotFound, ErrClosed, subscriberBufferSize, defaultListCount, failedSub, closedSub), keys.go (namespace key/pattern/rel mapping, escapeGlob, globMatch port of the Redis matcher), memory.go (memProv with its mutex-guarded subs registry and closed flag, msub, listedKey, pageHeap), redis.go (redisProv with the same registry, rsub, confirmSubscription, closePubSub, subscribeTimeout), proxy.go, export_test.go (test-only SubscriberBufferSize, DefaultListCount), example_test.go (ExampleProvider_listKeys; go/doc attaches ExampleT_suffix to an interface type only with a lowercase suffix, never as a method example).

Invariants:

  • Errors, not panics; Get requires a non-nil pointer; memory and Redis misses wrap ErrNotFound with a stack. IsNotFoundError uses only errors.Is; unrelated "not found" messages are not misses and GetOrSet returns them without calling the getter or writing.
  • TTL 0 → DefaultTTL (memory, 30m, package var) or RedisConfig.TTL (default 1h); KeepTTL (-1) → no expiry.
  • Memory provider is unbounded; expiry enforced on Get and CleanExpired; NowFunc is process-global.
  • Keys: every provider maps keys through namespace (keys.go): the stored name is path.Join(prefix, path.Clean("/"+key)), so .. cannot leave the prefix and in-prefix keys keep the earlier path.Join layout; an empty memory or Redis prefix becomes "/". A proxy cleans its own prefix as a rooted path without the leading slash ("/a/" and "../a" become "a"), prefixes each key and lets the parent clean it again. Keys builds a Redis glob from the escaped prefix and the pattern cleaned like a key (namespace.pattern), and returns names relative to the provider without a leading slash (namespace.rel, "" for the prefix itself); namespace.listed skips names that are not the stored name of their relative key (written by other clients, or outside a prefix that cleans to .) and lists the key "" only for an empty pattern (* would match it under /). A proxy strips its prefix from its parent's result. path.Clean does not know glob syntax, so .. or // inside [...] is cleaned too. The memory provider matches with globMatch, a byte-wise port of Redis stringmatchlen (* and ? match /, sets, ranges, escapes, the skip-longer-matches bound and the 1000-level nesting limit; stringmatchlen's * does not match the empty string, while KEYS special-cases a lone *, which a namespaced glob is only under a prefix cleaning to ., where listed drops what it adds; range bounds compare unsigned bytes, so ranges mixing bytes >= 0x80 with ASCII can differ from Redis builds with signed char), skips expired entries and returns a non-nil slice.
  • ListKeys(ctx, pattern, cursor, count) lists one page with the same matching, namespacing and listed filtering as Keys, and returns the next cursor ("" at the end; start with ""); count <= 0 means defaultListCount (100). Redis runs one SCAN per call (cursor: the decimal SCAN cursor, strconv.ParseUint, a bad one fails with invalid cursor: ...; COUNT count, so a page may be empty with a non-empty cursor, may exceed count, and may repeat a key; a key present for the whole iteration is listed). The memory provider returns ctx.Err() (with a stack) when ctx is already done, then walks the whole sync.Map per call (Range) and keeps the count+1 smallest live matches whose stored name sorts after the cursor in a bounded max-heap (pageHeap of listedKey; a name at or above the heap's largest is skipped before the glob match once the heap is full; the bound is checked as len-1 > count, so math.MaxInt does not overflow), sorts those and returns at most count: the cursor is the stored name of the last key returned (never empty, so "" starts the iteration), the page is full unless it is the last, and each key is listed once per iteration even under concurrent writes. A full iteration walks the store once per page (100k entries, count 1000: about 1s; Keys about 60ms). A proxy passes the cursor and count through unchanged and strips its prefix. Cursors are opaque and not portable between providers. Keys is deprecated (Deprecated: on Provider.Keys) and still runs KEYS; nothing outside tests in this module calls it.
  • GetOrSet writes a successful getter result with TTL 0 (the provider default), storing the dereferenced value so Redis string/byte reads remain compatible; a failed write leaves the destination unchanged and returns an error. Cache misses require a pointer to a concrete destination type because interface values do not round-trip reliably across providers. Concurrent misses may call the getter more than once.
  • Pub/sub channels are not prefixed and delivery is at most once. Publish never waits for a subscriber: each subscriber buffers subscriberBufferSize (100) undelivered messages; the memory provider drops the message for a subscriber whose buffer is full, and Redis subscriptions read through go-redis PubSub.Channel (one reader and one health-check goroutine per subscription, automatic reconnect and resubscribe, a message is dropped after its reader has waited one minute on a full channel). ReceiveMessage starts no goroutine and returns promptly with ctx.Err() or, after Close, ErrClosed; Close is idempotent. Redis Subscribe waits for the server's subscription confirmation (bounded by ctx; without it, connecting is bounded by the go-redis DialTimeout, attempted at most twice, and the confirmation read by subscribeTimeout, 10s) so a publish after Subscribe returns is delivered; a subscription that could not be established is returned as a failedSub whose ReceiveMessage returns the wrapped error and whose Close is a no-op. go-redis does not watch ctx.Done while reading, so confirmSubscription runs the read on a goroutine and closes the PubSub on cancellation to unblock it, reporting ctx.Err() (or context.DeadlineExceeded when the socket deadline derived from ctx fired first) with the go-redis error attached as a secondary error. Redis ReceiveMessage does not report connection loss: go-redis reconnects and resubscribes in the background and messages published meanwhile are lost; redisProv.Close first closes every live subscription (tracked in redisProv.subs, a map of *rsub guarded by redisProv.mu together with the closed flag; rsub.Close unregisters itself), then the client, so pending receives return ErrClosed and queued messages are discarded. Subscribe checks closed before the network round trip and registers under the lock after PubSub.Channel started its goroutines; when Close won the race the subscription is closed on the spot and closedSub, a failedSub reporting ErrClosed ("provider closed"), is returned, so no go-redis health-check goroutine can outlive Close. Close also closes the connection of a Subscribe still waiting for its confirmation (go-redis closes the pub/sub connections it tracks with the client), so a failed confirmation re-checks closed and returns closedSub with the go-redis error ("use of closed network connection") as a secondary error. go-redis tracks a pub/sub connection only once its setup (HELLO, CLIENT SETINFO) is done, so Subscribe also re-checks closed right after client.Subscribe returns and closes that connection itself; only a redial inside the confirmation read, when client.Subscribe left no connection (its dial or setup failed, or the SUBSCRIBE write and the reconnect failed), can still wait for its confirmation after Close (until ctx is done or subscribeTimeout), and it then returns closedSub too. memProv uses the same scheme (memProv.mu guards its subs map of *msub and closed; Publish delivers under the read lock), so a memory Subscribe during or after Close also returns closedSub; the in-process store stays usable after Close. If the Redis client is closed some other way, the go-redis reader closes the message channel and ReceiveMessage returns ErrClosed after closing the PubSub itself, which stops its health-check goroutine (only PubSub.Close does). rsub.Close also starts a goroutine that drains the go-redis channel until the reader closes it: the reader's send into a full channel only watches its one-minute timeout, so without the drain a slow subscriber's reader survived Close for up to a minute. rsub.Close may wait for an in-flight go-redis redial (DialTimeout) during an outage. Both subscribers check a done channel closed by Close (via sync.Once) before reading and again after a message was selected, so buffered messages are discarded after Close on both providers and a delivery that races Close returns ErrClosed; the memory subscriber channel is not closed (Close signals done, and Publish, which delivers under memProv.mu, stops sending once msub.Close unregistered it). Memory Publish returns ctx.Err() when ctx is already done.
  • Config YAML: provider: redis|memory, redis{...}; no factory reads provider. NewRedisProvider drops the URL from a *url.Error so a malformed URL's password is not echoed.

Tests: keys_test.go (internal) covers namespace, escapeGlob and globMatch tables and backtracking bounds; TestProvider/keys parity stores the same keys through Redis and memory providers (and proxies over them) under a prefix with glob metacharacters beside a sibling it must not match, and compares Keys for ~45 patterns against the real server, and ListKeys paged with counts 1, 3 and the default against Keys (Redis results deduplicated; memory pages full except the last, in key order, without repeats; helpers listPages, flatten, unique); redis namespace and memory namespace check that .. keys and proxy prefixes stay inside their namespace; redis dot prefix checks that Keys and ListKeys under a prefix cleaning to . list no name Get cannot read, and TestMemProv_KeysRoot the key "" under "/" and a named prefix for both; redis list keys (in Redis DB 1, alone: SCAN COUNT bounds the keys a call looks at, matching or not, so page sizes depend on what else is stored) checks that one ListKeys call is one SCAN (250 keys need several pages at counts 10, 0 and -1, the first page holds at least count or DefaultListCount keys, one page at 1000), invalid cursors (also through a proxy) and the error of a closed client; cache_test.go runs the same matrix against redis (testcontainers), memory and proxy(memory), including exact relative Keys names, read-through persistence and pubsubTest (prompt cancel and close, no message stolen by an abandoned receive, broadcast to two subscribers, and a go-redis goroutine-leak check that counts (*channel).init* stacks); receives run in goroutines that report over channels, never require off the test goroutine. TestRedisSubscribeUnreachable and TestRedisSubscribeContext (a fake RESP server that never confirms SUBSCRIBE, to test ctx cancellation and deadline during the confirmation wait) need no Docker; TestProvider/redis provider closed and memory provider closed cover ErrClosed for a pending receive and a queued message plus goroutine release after the provider is closed, redis subscribe during close races eight Subscribe calls against Close (every result reports ErrClosed, none live, no goroutine left) and checks the rejection after Close, TestRedisSubscribeDuringClose (no Docker: the fake server's onCommand hook signals the SUBSCRIBE it swallowed) closes the provider during the confirmation wait and expects ErrClosed, TestRedisSubscribeCloseDuringSetup (the hook holds the HELLO reply) closes it while the connection is set up and expects ErrClosed without the confirmation wait, and TestProvider/redis slow subscriber closed overflows the buffer, waits until the go-redis reader is blocked in its send select (readerBlockedOnSend, stack inspection) and asserts the reader exits promptly after Close. export_test.go exposes subscriberBufferSize as SubscriberBufferSize to the black-box tests. get_or_set_test.go covers default TTL, hit reuse, type errors, and write failures; memory_test.go covers CleanExpired (and Keys/ListKeys skipping an expired entry before it is cleaned), TestMemProv_ListKeys (exact pages and stored-name cursors, a full last page, keys added and deleted between pages, the default count, math.MaxInt, a proxy, a canceled ctx), TestMemProv_ListKeysPageBound (the page bound on 64 fresh providers, because sync.Map visit order is random per map), TestMemProv_ListKeysConcurrent (pages while writers churn keys around 200 stable ones: each listed exactly once; run with -race), the slow-subscriber drop, Publish with a done ctx, a publish/receive/close race (run with -race) and TestMemProv_SubscribeDuringClose (Subscribe and Publish racing provider Close: every result reports ErrClosed, and a later Subscribe is rejected).

github.com/effective-security/porto/pkg/revocation

Purpose: JWT revocation list. New returns a jwt.Revocation that stores revoked token IDs in a cache.Provider and checks them from Validate. Distinct from certificate revocation (pkg/crlcache).

Files: doc.go, revocaction.go (New, provider, TimeNowFn, cacheKey).

Entry points: New(cache.Provider) jwt.Revocation — Validate(ctx, token, claims) and Revoke(ctx, token, claims).

Invariants:

  • Returns errors; does not panic. Validate refreshes the snapshot (load), then rejects an expired exp (token expired at: ...) before a revoked ID (token revoked). A missing exp is not treated as expired.
  • Cache key (cacheKey): /revoked/<sub>/<id>. id is the jti claim, or certutil.SHA1Base64 of the raw token when jti is empty. reload pages through /revoked/* with ListKeys (count reloadPageSize, 1000, per call) until the cursor is empty and keeps the segment after the last /; an error on any page fails the reload without publishing; a key a Redis SCAN repeats lands in the map once, and a key added during the scan may be missed (see local).
  • revoked is a published snapshot. remember and reload replace the map; they never write an entry into the live map (a concurrent Validate would crash on a map read/write).
  • snapshotTTL is 5 minutes. One in-flight scan (inflight) serves every Validate that arrives while the snapshot is stale; waiters share that scan's error. A failed scan is not kept: the next Validate tries again.
  • local is the set of IDs this process has stored. reload merges it into the new snapshot, so a scan that started before Revoke wrote the key cannot drop the ID. An ID present in the scan is removed from local.
  • Revoke with exp already in the past returns nil and stores nothing. A missing exp is stored for 24 hours. Otherwise the cache value is exp in RFC3339 and the TTL is exp - now. Revoke then remembers the ID on the local snapshot.
  • Process-global: package logger; TimeNowFn (tests override the clock).

Tests: revocation_test.go (same package; it reads provider, snapshotTTL and TimeNowFn). Memory provider only, no Docker. TestRevokeReplacesSnapshot checks that a revoke publishes a new map. TestConcurrentValidateAndRevoke races revoke and validate. TestReloadPages stores 2*reloadPageSize+1 revoked IDs through the cache and expects one scan of three pages to find them all; TestReloadPageErrorKeepsSnapshot fails the second page (gatingCache.pageErr) and expects the error, an unchanged snapshot and loadedAt, and a retry on the next Validate. TestReloadKeepsRevokeStartedBeforeStore, TestConcurrentValidateSharesReload and TestReloadErrorIsSharedAndRetried run in testing/synctest behind a gating ListKeys (it counts scans, i.e. calls from the empty cursor, and pages, and holds the first page) so a scan can be held open.

github.com/effective-security/porto/pkg/location

Purpose: geographic location (city, region, country) of a client IP, for example to show where a sign-in came from. Independent of xhttp/identity: callers pass the netip.Addr from identity.ClientIPFromRequest or RequestContext.ClientAddr.

Files: doc.go, location.go (Provider, Place and Place.String, IsPublic, ParsePublicIP, parseHostAddr, canonical, globalUnicastIPv6, nonPublicPrefixes), ipwhois.go (IPWhoIs, IPWhoIsConfig, NewIPWhoIs, IPLocation, ErrQuotaExceeded, DefaultIPWhoIsURL, DefaultTimeout, DefaultUserAgent, noLocationMessages, retryAfter, quotaUntil, backOff, displayURL, drainBody), cache.go (Cache, CacheConfig, NewCache, DefaultCacheTTL, NoExpiry, DefaultIPv4Bits, DefaultIPv6Bits, lookup, resolve), export_test.go (test-only RetryAfter).

Entry points: NewIPWhoIs(IPWhoIsConfig) (*IPWhoIs, error); NewCache(Provider, CacheConfig) (*Cache, error); Provider.Location(ctx, netip.Addr) (*Place, error); IPLocation(ctx, baseURL, addr) *Place for a one-off lookup.

Invariants:

  • Provider.Location contract: a *Place for a known address; nil, nil for no location (the address is not public, or ipwho.is answered "success": false with "Invalid IP address" or "Reserved range", noLocationMessages, compared without case); an error for a failed lookup (network, timeout, non-200 status, any other "success": false message, undecodable or oversized answer), which may succeed later. Constructors return errors, never panic.
  • Addresses are canonicalized (canonical: IPv4-mapped IPv6 as IPv4, zone dropped) before the public check, the lookup and the cache key. IsPublic refuses non-global-unicast and private (RFC 1918, RFC 4193) addresses, IPv6 outside 2000::/3 (globalUnicastIPv6: NAT64, SIIT and IPv4-compatible embeddings, site-local, discard-only) and the special-purpose nonPublicPrefixes (0.0.0.0/8, CGNAT 100.64.0.0/10, 192.0.0.0/24, 192.88.99.0/24, documentation and benchmarking ranges, 240.0.0.0/4, 2001::/23, 2001:db8::/32, 6to4 2002::/16, 3fff::/20); such addresses are never sent to the service or cached. ParsePublicIP accepts an address, [IPv6], IPv4:port or [IPv6]:port (netip.ParseAddr, then parseHostAddr with netip.ParseAddrPort rules); unbalanced brackets, [IPv4] and a malformed port are rejected. IPv4 embedded in 6to4, NAT64 or Teredo addresses is not unwrapped, so those clients have no location.
  • IPWhoIs sends GET <base>/<ip>?fields=success,message,city,region,country,country_code with Accept: application/json and the configured User-Agent. Other query parameters of BaseURL (an API key) are kept; fields is replaced. A transport or url.Parse error is unwrapped from *url.Error, and NewIPWhoIs errors show the base URL without query, fragment and password (displayURL), so errors and logs never repeat the query. One http.Client per provider (on http.DefaultTransport) with Timeout (default 3s) and no redirects (a 3xx is a status error); the answer is capped at 4096 bytes (ipWhoIsMaxBody), and every response, including a status error or an oversized answer, is drained up to that size before it is closed (drainBody, best effort: a remainder of at most 4096 decompressed bytes that arrives within the client Timeout and ctx), so its connection is idle again when Location returns; a status error with a slow body therefore waits for it, up to those bounds. The transport drains a body closed early only in the background, and a following lookup would often dial. A no-location answer is logged at DEBUG (reason=no_location, ip, message).
  • Quota: HTTP 429 returns an error wrapping ErrQuotaExceeded and sets retryAt (backOff, guarded by IPWhoIs.mu; it only moves later) from Retry-After (retryAfter: seconds or HTTP date, capped at 24h; seconds are clamped before they are converted, so no value overflows; missing, invalid (including a number beyond int64) or non-positive waits 1 minute). Until then every lookup fails fast with ErrQuotaExceeded ("skipped until ") without a request (quotaUntil). ipwho.is documents 429 with Retry-After for its free per-IP daily limit.
  • IPLocation intentionally swallows errors: it logs them at WARNING (reason=ip_location, ip, err) and returns nil. It returns nil without a lookup or log for an address that is not public. It builds a new IPWhoIs per call, so it has no cache and no 429 backoff.
  • Cache keys answers by network: addr.Prefix(IPv4Bits) (default /24) or addr.Prefix(IPv6Bits) (default /48); the first address of a network is the one looked up. It is a size-bounded golang-lru/v2 LRU (lru.Cache, which locks itself, unlike simplelru; Cache.mu guards only inflight; no background goroutine; Cache starts none); TTL is checked on read (get; default 24h, NoExpiry keeps entries until evicted, time.Now, so testing/synctest controls it). get Peeks first and promotes (Get) only a live entry, so an expired entry whose refresh fails is evicted before live ones. Places and nil (no location) answers are kept; errors are not, so the next call retries. Each call returns its own copy of the Place (clone), and the stored entry is a copy of what the provider returned.
  • Concurrent misses for one network share one lookup (inflight, guarded by Cache.mu), run on the ctx of the caller that started it; no lookup is detached from its caller, so callers must give a ctx that allows for the provider's latency: a lookup cut short by ctx is an error and is not cached, and with deadlines shorter than the provider the cache never fills. Later callers wait for it or for their own ctx (ctx.Err() wrapped) and share its result; when it failed with a ctx error (context.Canceled/DeadlineExceeded in the chain) after its caller's ctx was done (lookup.canceled), a waiter whose ctx is live loops and looks the network up again, one lookup at a time. Any other error is shared, even if the first caller's ctx was done by then. resolve stores the answer before it leaves inflight, and a caller re-checks the LRU under mu, so a finished lookup is not repeated. A panicking provider still releases the waiters (resolve's deferred cleanup; they get "location lookup of did not complete") and the panic propagates to the first caller.
  • Process-global: package logger (github.com/effective-security/porto/pkg, location).

Tests: black box (package location_test; export_test.go exposes retryAfter), no Docker or network. location_test.go (Place.String, ParsePublicIP and IsPublic tables, including IPv6 special-purpose ranges); ipwhois_test.go (httptest servers: request path, query (fields replaced, key kept), headers, IPv6 without zone, JSON nulls, body limit, known and unknown success:false messages, status/redirect/decode errors, caller deadline, client timeout, unreachable without the URL query in the error; TestIPWhoIsQuota (real 1s Retry-After, waited from when the 429 lookup returned), TestRetryAfter table; TestIPWhoIsReusesConnection makes three lookups per answer kind and asserts the exact error, three requests and one server connection; the first answer holds back its last bytes for 150ms (slowTail, longer than the transport's 50ms background drain), so a missing drainBody fails without -race; it is not parallel, because httptest.Server.Close closes the idle connections of http.DefaultTransport; lookupServer answers a 302 with Location: /moved, which answers a place, so following a redirect fails the redirect rows; TestIPLocationLogsErrors installs a process-global JSON log formatter and is not parallel); cache_test.go (per-network sharing and custom prefix lengths, no-location kept, errors retried, copies, LRU eviction, expired entries not promoted and TTL in testing/synctest, shared lookups including waiter cancellation, a first caller that gives up (retried) or whose lookup fails on its own (shared), and provider panic in synctest, and TestCacheConcurrent, which races 16 goroutines over a cache smaller than the working set; run with -race); example_test.go (ExampleNewCache, run against an httptest server).

github.com/effective-security/porto/pkg/tasks

Purpose: in-process cron-like scheduler; runs functions on interval/daily/weekly schedules parsed from a human-friendly string.

Files: doc.go, task.go (Task, Schedule, ParseSchedule, run/next-run logic), scheduler.go (Scheduler, Publisher, options).

Entry points: NewScheduler(opts...) with Add/Start/Stop; NewTask(format), NewTaskAtIntervals, NewTaskDaily, NewTaskOnWeekday, New(*Schedule); bind with Task.Do(name, fn, args...); ParseSchedule ("every N unit [hh:mm]", " [hh:mm]", "hh:mm"); Publisher.

Invariants:

  • Panics: NewTaskDaily/NewTaskOnWeekday on a bad hh:mm, Task.Do on a non-func or wrong arity. Errors: NewTask/ParseSchedule/UpdateSchedule, Start (already running); Stop succeeds when already stopped.
  • Globals: TimeNow, location via SetGlobalLocation, package logger.
  • Start spawns one ticker goroutine; each due task runs in its own goroutine with a per-task run lock and WithRunTimeout (default 1s); callback panics are recovered and logged. Tick = WithTickerInterval or min(1s, shortest Duration/10). Runnable tasks are sorted by a snapshot of next-run times using slices.SortFunc.
  • scheduler.lock guards membership and lifecycle; Task has a separate RWMutex for callback, publisher, and run state. List copies the task slice; Task.Schedule deep-copies LastRunAt and returns a snapshot. New(*Schedule) copies its input. Use Add/Clear and SetNextRun/UpdateSchedule to mutate state. Custom Task implementations must provide their own concurrency safety.
  • Stop closes the current run's quit channel under the lifecycle lock, is idempotent, and does not wait for task callbacks or for the ticker goroutine, but no *task is dispatched after Stop returned: runPending(quit) re-checks running and the run's quit under the lifecycle lock and dispatches in the same critical section, and a *task is dispatched with tryRun, which acquires the run lock without waiting and marks the task running (begin) before starting execute in a goroutine, so a run dispatched before Stop is already visible to IsRunning and RunCount (the callback itself may begin just after Stop returned); a custom Task implementation is dispatched with go Run(), so its run may begin, and become visible, after Stop returned; the guarantee is documented on Scheduler.Stop for tasks created by this package only (holding the lock until a custom Run began would run arbitrary code under it). A tick that was ready when quit closed may still win the select, harmlessly. Start may restart after Stop. Publisher callbacks run outside scheduler and task locks (execute publishes from the run goroutine).
  • Task ID: WithID or uuid.NewV7().

Tests: time-based (task_test.go, scheduler_test.go) and overlapping reads, writes, stops, and callbacks (concurrency_test.go); task_bench_test.go measures Run; package race tests pass.

github.com/effective-security/porto/pkg/tlsconfig

Purpose: build tls.Config from PEM files and keep the keypair fresh via a polling reloader; reloading RoundTripper and cipher-suite name mapping.

Files: doc.go, tlsconfig.go (NewServerTLSFromFiles, NewClientTLSFromFiles, NewClientTLSWithReloader, HTTPTransport), tls.go (X509KeyPairWithOCSP, LoadX509KeyPairWithOCSP), reloader.go (KeypairReloader), cipher_suites.go (UpdateCipherSuites, GetCipherSuite).

Invariants:

  • Defaults: MinVersion TLS 1.2, NextProtos h2 + http/1.1; rootsFile is used for both RootCAs and ClientCAs on the server; CA files with no valid certificates return an error.
  • Expired certificates fail initial load or reload; a failed reload keeps the previous pair. If the current pair later expires, TLS callbacks return errors and Keypair returns nil. Expiry within one hour is logged as a warning.
  • One poll goroutine per reloader (mtime polling plus an hourly forced reload); handlers run in their own goroutines; the goroutine snapshots timestamps under RLock; Close waits for an active reload, then closes stopChan. A second Close or Reload after Close returns an error; the poll goroutine does not log that closed error (errReloaderClosed).
  • Reload marks itself in progress under the write lock, sleeps and loads outside the lock, then swaps the pair under the lock. A Reload that finds another in progress waits on reloadDone, then loads again, so a nil return means the files were read after the call started. File mtimes are stat'ed before each load attempt, so a write during the load triggers the next poll; count is atomic.
  • HTTPTransport clones a supplied http.Transport, installs a fixed TLSClientConfig with GetClientCertificate, and closes idle connections on reload and Close. RoundTrip never mutates the active TLS config.
  • pkg/transport.TLSInfo.ServerTLSWithReloader clears the static Certificates slice after installing GetCertificate so clients without SNI still receive the current, expiry-checked pair.
  • OCSP staple file: <certfile-minus-ext>.ocsp; ignored if expired or unparsable, error if revoked.
  • Cipher suite names (cipher_suites.go) are derived at init from the Go release the binary is built with: tls.CipherSuites entries usable below TLS 1.3, plus the TLS_ECDHE_{RSA,ECDSA}_WITH_CHACHA20_POLY1305 alias constants. UpdateCipherSuites rejects names in tls.InsecureCipherSuites (Go 1.27: RC4, 3DES, CBC-SHA256 and every TLS_RSA_* key-exchange suite) and TLS 1.3 names with distinct errors (an alias is classified by its canonical name), and leaves the config unchanged on error; GetCipherSuite reports only accepted names. The list only restricts the enabled suites: Go ignores the order of tls.Config.CipherSuites, and PreferServerCipherSuites has no effect, so cipher_suites sets no preference. A toolchain upgrade that moves a suite to the insecure list makes configs naming it fail.
  • Test hook: package var makeTicker.

Tests: generate certs with xpki/testca; reloader tests rewrite files on a 100ms poll, and b02_test.go covers expiry, lock availability, Close versus Reload, Reload waiting for an active reload, concurrent reloads, no error log for a poll during Close (non-parallel: replaces makeTicker and the xlog formatter), stable transport config, and invalid CA bundles; tls_test.go has inline PEM fixtures; cipher_suites_test.go compares the accepted table with tls.CipherSuites/tls.InsecureCipherSuites and table-tests UpdateCipherSuites errors; keypair_test.go (black box) covers key types (key equality compares key material, not RSA precomputation internals), key/certificate mismatches, OCSP staples, client configs and HTTPTransport, and reloader_poll_test.go (internal, replaces makeTicker) the poll loop with deleted files and validateCertificate.

github.com/effective-security/porto/pkg/transport

Purpose: server-side listeners: eager-handshake TLS listener with CRL check, and TCP keepalive listeners; TLSInfo config → tls.Config with reloader.

Files: doc.go, transport.go (logger), tls.go (TLSInfo: ServerTLSWithReloader, Config, Close), listener_tls.go (NewTLSListener), keepalive_listener.go (NewKeepAliveListener).

Invariants:

  • TLSInfo fields: CertFile/KeyFile/TrustedCAFile/ClientCAFile/ClientAuthType/CipherSuites/CRLVerifier/HandshakeFailure/HandshakeTimeout, all used. The unenforced etcd fields (AllowedCN, AllowedHostname, EmptyCN, ServerName, InsecureSkipVerify, SkipClientSANVerify) were removed in B15; other client-certificate policy is set with VerifyConnection on the config returned by ServerTLSWithReloader before NewTLSListener serves (the listener uses the same *tls.Config); Close drops that config, so set it again after a rebuild.
  • CRL check only sees VerifiedChains (needs ClientAuth ≥ VerifyClientCertIfGiven); Revoked → reject; verify error or Unknown → log and allow (fail-open).
  • Eager handshakes have a default 10s deadline (TLSInfo.HandshakeTimeout, zero = default, negative = disabled), cleared before returning a successful connection. Deadline and handshake errors go to HandshakeFailure and the connection is closed.
  • Accept errors of the wrapped listener are returned unwrapped by both listener types: net/http, grpc and cmux type-assert net.Error to retry temporary errors (EMFILE). The TLS listener passes each such error to one Accept call (its accept loop waits for that call) and keeps accepting; it stops only when the wrapped listener returns net.ErrClosed or Close is called, and Accept then returns an error wrapping net.ErrClosed. A wrapped listener that reports its closure with another error (cmux: ErrListenerClosed, ErrServerClosed) keeps the loop running until Close; net/http and grpc close their listener when such an error stops them. A repeated Close neither blocks nor panics and returns the wrapped listener's close error. After handing an error to Accept, the loop calls the wrapped Accept again at once and holds the result for the next call, so the caller sees an error that is one of its backoff periods old and recovers from an error storm up to one backoff period (net/http: 1s) later than on a raw listener. NewTLSListener closes l for a nil or empty TLSInfo but leaves it open when ServerTLSWithReloader fails.
  • Keepalive listeners call SetKeepAliveConfig with keepAliveConfig (idle 30s, interval 15s, 9 probes: a dead peer is dropped after 165s). Connections without SetKeepAliveConfig (non-TCP) are returned unchanged; a SetKeepAliveConfig error is logged at DEBUG and swallowed so a reset peer cannot fail Accept (net.TCPListener's own accept ignores it too).
  • Reloader interval is 5 minutes (reloadInterval); TLSInfo.Close stops it. ServerTLSWithReloader builds into locals and caches the config and reloader only after every step succeeded, so a failed call (load, expiry, cipher list, reloader) caches nothing and a retry reads the files again.
  • ServerTLSWithReloader clears static certificates after installing GetCertificate, so handshakes without SNI use the current pair and receive expiry errors.

Tests: tls_test.go init() builds a CA/intermediate/server chain with xpki/testca in a per-process os.MkdirTemp directory (fixtureDir, removed by TestMain, so concurrent test processes do not overwrite each other's certificates) and verifies a handshake without SNI uses the callback; listener tests start restserver over the listener with a fake verifier; limits_test.go checks stalled handshakes and application I/O after the handshake deadline; b15_test.go covers failed-build retries (bad cipher list, expired certificate), unwrapped Accept errors, serving after EMFILE through net/http (both listener types) and cmux (keepalive listener), Close while the accept loop waits to forward an error, the keepalive config passed to fake connections (including non-TCP and failing ones), and VerifyConnection enforcement; b15_linux_test.go reads the keepalive socket options of a real connection.

github.com/effective-security/porto/pkg/appinit and pkg/appinit/config

Purpose: service bootstrap: logging setup, metrics pipeline (Prometheus/CloudWatch), CPU profiler; config.Metrics is the YAML/JSON struct.

Files: init.go (LogConfig, Flags, Logs, CPUProfiler), metrics.go (Metrics, cloudWatchRunner, runtimeStatsCloser, metricsClosers), prometheus.go (bounded, closable HTTP endpoint; prometheusHandler builds its handler for a registerer and gatherer), cpu_profiler.go (startCPUProfile, cpuProfileCloser, cpuProfileRunning, removeCPUProfile); config/config.go (Metrics, Prometheus, CloudWatch).

Invariants:

  • Process-global side effects: xlog.SetFormatter (or the logrotate formatter installed by logrotate.Initialize, which Logs keeps and only adjusts), metrics.NewGlobal, default Prometheus registry, xlog.OnError hook, package vars promSink/cwSink (once per process).
  • provider is comma-separated: prometheus | cloudwatch | inmem; empty or disabled → (nil, nil). A provider without its config block is an error. Provider choices are validated before binding or registering sinks. Prometheus binds synchronously before sink initialization; bind errors are returned without initializing the sink, later initialization errors release the listener and reset the Prometheus and CloudWatch sinks created by the call, so a failed call can be retried; the Prometheus endpoint and the CloudWatch Run goroutine start only after initialization succeeds. Its HTTP server uses shared timeout/body defaults (prometheus.timeouts, prometheus.max_request_body) and, with prometheus.timeouts.handler set, answers a scrape that takes longer with 503 through promhttp.HandlerOpts.Timeout (prometheusHandler; an http.TimeoutHandler, which also bounds the request context): promhttp gathers without consulting the request context, so limits.Timeouts.ApplyHandler alone did not end a slow scrape, and the collection still finishes in the background. The returned closer closes active connections and the listener and waits for Serve to exit; concurrent/repeated endpoint closes are safe. Unexpected serve errors are logged, never fatal. The endpoint closer ignores an already-closed listener error. The CloudWatch closer (cloudWatchRunner) cancels the context of cloudwatch.Sink.Run and waits for it to return; Run publishes the last interval first (bounded by the sink's 10s shutdown timeout, errors logged, not returned). An interval is lost when the cancellation aborts a periodic publish in flight or meets a tick Run has not handled yet (Data empties the sink with WithCleanup before the publish fails); a Run that already stopped on expired or missing credentials makes Close return at once. Fixing either needs cloudwatch.Sink.Run changes in the metrics module. With runtime_metrics and a Prometheus or CloudWatch sink, runtimeStatsCloser signals the collector of the global *metrics.Metrics to stop without waiting for it (emission keeps working), before the CloudWatch runner closes; a closer is therefore returned even for Prometheus without an address. inmem alone installs no global instance, so runtime_metrics starts nothing. Closers close in order: Prometheus endpoint, runtime collector, CloudWatch runner; each is idempotent and safe for concurrent use. Sinks remain initialized once per process after Close; initialize Metrics serially. Reads env NODE_NAME for the node tag.
  • CPUProfiler keeps the process-global cpuProfileRunning flag while a profile it started runs: a second call fails with CPU profile already running before os.Create, so it cannot truncate the running profile's file. A profile started otherwise (direct pprof, go test -cpuprofile) makes pprof.StartCPUProfile fail after the file was created; the file is then closed and removed (if that profile writes to the same path, its file is truncated and removed), and the flag is cleared only after the removal, so a concurrent call on the same path cannot start a profile that the removal unlinks. The closer stops the profile, closes the file, clears the flag and returns the close error; a second Close returns an error.
  • Nil closers are normal; callers must nil-check.
  • Test hook: package var removeCPUProfile (failed-start file removal).
  • LogDir /dev/null discards; set → logrotate (10 MB / 10 days, buffered) with stderr as an extra sink when LogStd.
  • Config keys: disabled, provider, prefix, prefix_for_number_labels, prometheus{addr,expiration,timeouts{header,read,idle,handshake,handler},max_request_body}, runtime_metrics, cloudwatch{aws_region,aws_endpoint,namespace,publish_interval,with_sample_count}, global_tags, allowed_prefixes, blocked_prefixes. B21 removed the unused add_tags/replace_tags (lenient decoders ignore them, strict ones reject them); aws_endpoint replaced the untagged AwsEndpoint/awsendpoint keys.

Tests: init_test.go (log branches with t.TempDir, asserts the rotating file receives output; CPUProfiler no-op and create errors), cpu_profiles_test.go (profile lifecycle, a second call while a profile runs, a profile started outside CPUProfiler, file closed or removed on every path, a call during failed-start removal fails as already running, and a removal error is returned with the start error (non-parallel: replaces removeCPUProfile); CPU profiling is process-global, so serial, and skipped under go test -cpuprofile). metrics_test.go covers Prometheus bind errors/retry, Prometheus and CloudWatch sink-failure cleanup, provider validation, scrapes, header deadlines and concurrent close; prometheus_test.go checks prometheusHandler on private registries (a scrape without the handler timeout, a fast one within it, and a blocking collector answered 503 within the timeout); TestMetricsCloudWatchCloseStopsRun points aws_endpoint at an httptest fake CloudWatch (static credentials from env, no shared AWS config) and checks that Close stops Run after its final publish and stops the runtime stats collector (goroutine stack counts, waiting with Eventually for the goroutines to start). config/config_test.go covers the YAML/JSON keys; metrics_config_test.go covers provider validation errors, GlobalTags, Prometheus with CloudWatch, and the release of the Prometheus sink and endpoint when CloudWatch fails. Global-state tests run serially.

github.com/effective-security/porto/pkg/discovery

Purpose: tiny in-process service registry resolved by interface type (gserver exposes services to handlers through it).

Entry points: New(), Register(server, svc), Find(server, *iface), ForEach(*iface, fn).

Invariants: key <server>/<concrete type>; duplicate or nil Register → error; Find requires a non-nil pointer-to-interface, server "" means any; map iteration order makes multiple matches arbitrary. An RWMutex protects registration and lookup; ForEach snapshots matching entries before invoking callbacks so callbacks may register services. The caller owns synchronization of the destination interface value.

github.com/effective-security/porto/pkg/crlcache

Purpose: Verifier{Update() error; Verify(crt, issuer) (int, error)} returning ocsp.Good/Revoked/Unknown, consumed by transport.TLSInfo. No implementation here; implementations must be concurrency-safe.

github.com/effective-security/porto/pkg/streamctx

Purpose: wrap grpc.ServerStream to override Context() in stream interceptors. If ss is already a wrapper its context is replaced in place and the same object returned.

Tests: streamctx_test.go checks the in-place replacement and that every stream method reaches the wrapped stream (a recording stub).

github.com/effective-security/porto/metricskey

Purpose: metrics.Describe descriptors: HTTPReqPerf, HTTPReqByRole, GRPCReqPerf, GRPCReqByRole, StatsVersion, HealthLogErrors; Metrics slice. Names http_requests_perf, http_requests_role, rpc_requests_perf, rpc_requests_role, version, log_errors.

github.com/effective-security/porto/tests/testutils and tests/mockappcontainer

Purpose: test helpers. testutils: CreateURL, CreateBindAddr (panic if no free port; TOCTOU), JSON, CompareJSON. mockappcontainer: NewBuilder().WithConfig(*gserver.Config).WithJwtSigner(...).WithJwtParser(...).WithDiscovery(...).Container() builds a dig.Container; dig.Provide errors are discarded. builder_test.go checks that each With* provides its value; testutils/testutils_test.go covers the free-port helper's documented panic on an invalid host.

Build, test and CI

  • go.mod targets Go 1.27; the standard uuid package is used.
  • make lint = gofmt with rewrites, go vet, govulncheck, golangci-lint (revive exported + staticcheck all, comment presets excluded).
  • make test needs Docker for pkg/redisclient and pkg/cache; pkg/retriable internal_test.go needs outbound DNS.
  • make covtest writes coverage.out; CI (.github/workflows/unittest.yml) runs make lint and make build covtest, and its coverage status requires more than 90% total coverage (MIN_TESTCOV); every package with statements is above 90% (B24). CI does not run -race.
  • make docs regenerates Documentation/api/*.md with gomarkdoc (one file per non-test package).
  • pkg/tasks and the other packages pass go test -race.