180 KiB
Code map
Navigation index for agents and new contributors: concept → file → entry points → invariants. Start here instead of grepping the tree. If you had to grep for something that belongs in this map, add the row in the same change (see AGENTS.md).
- High level purpose and samples: README.md
- Known defects, with IDs referenced from code comments: FINDINGS.md
- Larger planned work: ROADMAP.md
- Generated API reference (
make docs): api/
Module layout
gserver/ gRPC + gRPC-Web + REST server (cmux), identity + authz wiring
credentials/ gRPC credential bundle (TLS + Bearer/DPoP per-RPC tokens)
roles/ identity provider: JWT, DPoP, AWS STS, TLS/SPIFFE → roles
restserver/ httprouter REST server with fixed middleware chain
authz/ path/role access control tree (HTTP handler + gRPC interceptors)
ready/ readiness gate middleware
telemetry/ request logger, request metrics, ResponseCapture
xhttp/
correlation/ correlation ID across HTTP, gRPC metadata, context, logs
header/ header name and content-type constants
httperror/ structured API errors, HTTP ↔ gRPC code mapping
identity/ caller identity + client IP in request contexts
marshal/ JSON response writing (gzip, ?pp), bounded bodies and strict decoding
limits/ Shared timeout and body-size defaults
pkg/
retriable/ HTTP client: retry policy, failover, Bearer/DPoP, nonces, token storage
rpcclient/ gRPC client builder from Config
redisclient/ prefixed go-redis wrapper, owner-bound lock, rate limiter
cache/ Provider abstraction: memory, Redis, proxy; pub/sub
revocation/ JWT revocation list over cache.Provider
location/ client location from IP: Provider, ipwho.is lookup, per-network LRU cache
tasks/ cron-like scheduler
tlsconfig/ tls.Config from files, KeypairReloader, HTTPTransport
transport/ TLS listener (eager handshake, CRL), keepalive listener, TLSInfo
appinit/ logging, metrics (Prometheus/CloudWatch), CPU profiler; config/
discovery/ in-process service registry by interface
crlcache/ Verifier interface for revocation checks
streamctx/ override grpc.ServerStream context
metricskey/ metric descriptors
tests/ testutils (free ports, JSON compare), mockappcontainer (dig builder)
Internal dependency direction
Arrows point from importer to imported. Nothing under xhttp/ or pkg/
imports restserver or gserver, except pkg/retriable, pkg/rpcclient,
pkg/cache and pkg/redisclient, which import gserver/credentials
(token types) or gserver (TLSInfo) only.
| Package | Imports from this module |
|---|---|
gserver |
gserver/credentials, gserver/roles, pkg/discovery, pkg/streamctx, pkg/transport, restserver/authz, restserver/ready, restserver/telemetry, xhttp/*, metricskey |
gserver/roles |
gserver/credentials, xhttp/header, xhttp/identity |
gserver/credentials |
none |
restserver |
restserver/authz, restserver/ready, restserver/telemetry, xhttp/* |
restserver/authz |
restserver/telemetry, xhttp/httperror, xhttp/identity, xhttp/marshal |
restserver/ready |
xhttp/httperror, xhttp/marshal |
restserver/telemetry |
metricskey, xhttp/header, xhttp/identity |
xhttp/correlation |
pkg/streamctx, xhttp/header |
xhttp/httperror |
xhttp/correlation, xhttp/header |
xhttp/identity |
pkg/streamctx, xhttp/header, xhttp/httperror, xhttp/marshal |
xhttp/marshal |
xhttp/header, xhttp/httperror, xhttp/limits |
pkg/retriable |
gserver/credentials, pkg/tlsconfig, xhttp/correlation, xhttp/header, xhttp/httperror |
pkg/rpcclient |
gserver/credentials, pkg/retriable, xhttp/httperror |
pkg/redisclient, pkg/cache |
pkg/tlsconfig (and gserver.TLSInfo for config) |
pkg/revocation |
pkg/cache |
pkg/location |
xhttp/header |
pkg/transport |
pkg/crlcache, pkg/tlsconfig, xhttp/limits |
pkg/appinit |
pkg/appinit/config, metricskey, xhttp/marshal; pkg/appinit/config imports xhttp/limits |
tests/mockappcontainer |
gserver, pkg/discovery |
xhttp/header, xhttp/limits, pkg/tlsconfig, pkg/tasks, pkg/discovery, pkg/crlcache, pkg/streamctx, metricskey, tests/testutils |
none |
Concept index
| Concept | Package | File | Symbol |
|---|---|---|---|
| gRPC/REST/gRPC-Web server bootstrap | gserver | server.go | Start, GServer |
| listen URL schemes (http/https/unix/unixs) | gserver | serve.go | configureListeners |
| cmux protocol multiplexing (HTTP1 vs HTTP2) | gserver | serve.go | serveCtx.serve |
| gRPC-Web translation / trailer frame | gserver | grpc_web_response.go | grpcWebResponse, buildTrailerFrame |
| gRPC-Web text (base64) | gserver | grpc_web_response.go | writeTextPayload |
gRPC-Web gzip / X-GRPC-Stream |
gserver | serve.go | grpcHandlerFunc |
| gzip acceptance / quality values | xhttp/marshal | marshal.go | AcceptsGzip |
| gRPC DPoP proof URI | gserver/credentials, gserver/roles | credentials.go, roles.go | GetRequestMetadata, IdentityFromContext (grpcDPoPURI) |
| CORS (REST, rs/cors) | gserver, restserver | serve.go, router.go | configureHandlers, corsHandler, CORSOptions, NewRouterWithCORS |
| CORS (gRPC-Web headers) | gserver | serve.go, grpc_web_response.go | grpcHandlerFunc, grpcWebResponse.prepareHeaders, mergeVaryHeader |
| rate limit (tollbooth, HTTP and gRPC) | gserver | ratelimit.go, serve.go, config.go | newRateLimiter, rateLimiter.httpHandler, rateLimiter.unaryInterceptor, rateLimiter.streamInterceptor, RateLimit, RateLimit.Validate, RateLimit.GetLogRejections |
| trusted proxy CIDRs / forwarding headers | xhttp/identity, gserver, restserver | realip.go, config.go, server.go | ParseTrustedProxies, WithTrustedProxies, NewTrustedProxyHandler, Config.Validate, HTTPServer.WithTrustedProxies |
| client IP resolved once per request | xhttp/identity | realip.go | NewTrustedProxyHandler, ClientIPFromRequest, forwarding |
| HTTP middleware chain order | gserver, restserver | serve.go, server.go | configureHandlers, HTTPServer.NewMux |
| gRPC interceptor chain | gserver | serve.go | grpcServer |
| panic recovery | gserver, xhttp/correlation, xhttp/identity | serve.go, correlation.go, ctx.go | recoverHTTPPanic, panicInterceptor, NewAuthUnaryInterceptor |
| panicking REST handler metrics | restserver/telemetry | request_metrics.go, response_capture.go | NewRequestMetrics, ResponseCapture |
request validation (Validate()) |
gserver | serve.go | NewRequestValidationUnaryInterceptor, validate, Validator |
| keepalive (gRPC server) | gserver | serve.go, config.go | configureListeners, KeepAliveCfg |
| keepalive (gRPC client) | pkg/rpcclient | config.go | Config.DialKeepAliveTime/Timeout |
| keepalive (TCP listener) | pkg/transport | keepalive_listener.go | NewKeepAliveListener, keepAliveConfig |
| listener Accept errors / temporary retry | pkg/transport, gserver | keepalive_listener.go, listener_tls.go, listener.go | keepaliveListener.Accept, tlsListener.forwardAcceptError, backoffListener |
| TLS server config / client cert auth | gserver, pkg/transport | config.go, tls.go | TLSInfo, TLSInfo.ServerTLSWithReloader (custom policy: VerifyConnection on its config) |
| TLS reload / certificate rotation | pkg/tlsconfig | reloader.go | KeypairReloader, GetKeypairFunc |
| TLS min version / ALPN defaults | pkg/tlsconfig | tlsconfig.go | NewServerTLSFromFiles, NewClientTLSFromFiles |
mutual TLS client (GetClientCertificate) |
pkg/tlsconfig | tlsconfig.go | NewClientTLSWithReloader |
| cipher suites by name | pkg/tlsconfig | cipher_suites.go | UpdateCipherSuites, GetCipherSuite |
| OCSP staple loading | pkg/tlsconfig | tls.go | X509KeyPairWithOCSP, LoadX509KeyPairWithOCSP |
reloading http.RoundTripper |
pkg/tlsconfig | tlsconfig.go | HTTPTransport, NewHTTPTransportWithReloader |
| TLS listener / eager handshake | pkg/transport | listener_tls.go | NewTLSListener |
| CRL / revocation check on client certs | pkg/transport, pkg/crlcache | listener_tls.go, crlcache.go | TLSInfo.CRLVerifier, Verifier |
| handshake failure callback | pkg/transport | tls.go | TLSInfo.HandshakeFailure |
| graceful shutdown / shutdown timeout | gserver, restserver | server.go, config.go | Server.Close, Config.Timeout.Shutdown (shutdownTimeout), HTTPServer.StopHTTP, WithShutdownTimeout |
| server lifecycle events | restserver | server.go | OnEvent, ServerEvent |
| REST bind address helpers | restserver | config.go | GetPort, GetHostName |
| REST server config access | restserver | server.go | HTTPServer.Config, HTTPConfig |
| custom middleware / interceptors | gserver, restserver | options.go, server.go | WithMiddleware, WithUnaryServerInterceptor, WithMuxFactory |
| service registration (REST/gRPC) | gserver, restserver | server.go, service.go | RouteRegistrator, GRPCRegistrator, ServiceFactory, Service |
| service registry / DI lookup | pkg/discovery | discovery.go | Discovery.Register/Find |
| dig container for tests | tests/mockappcontainer | builder.go | NewBuilder |
| readiness gate | gserver, restserver, restserver/ready | server.go, ready.go | IsReady, NewServiceStatusVerifier |
| slow request logging / gRPC metrics | gserver | logs.go | WarnUnaryRequestLatency, logRequest, callerOf |
| caller (client IP, role) in gRPC logs | gserver | logs.go, serve.go | callLog, identityHandoffUnaryInterceptor, identityHandoffStreamInterceptor, grpcServer |
log field names remote/peer/agent |
restserver/telemetry, xhttp/identity, pkg/transport | requestlogger.go, ctx.go, keepalive_listener.go | NoAgent, RequestLogger.ServeHTTP, NewContextHandler, createIdentityContext, setKeepAlive |
| log field names in gserver lines | gserver | logs.go, ratelimit.go, serve.go | logRequest, logError, logRejection, grpcHandlerFunc |
| request logging (HTTP) | restserver/telemetry | requestlogger.go | NewRequestLogger |
| log skip paths | restserver/telemetry, gserver | requestlogger.go, config.go | LoggerSkipPath, ShouldSkip, Config.SkipLogPaths |
| HTTP request metrics (nested role handoff) | restserver/telemetry, restserver, gserver | request_metrics.go, server.go, serve.go | NewRequestMetrics, NewMux, configureHandlers, rateLimiter.httpHandler |
| metric route/method/role labels | restserver/telemetry, restserver | route.go, request_metrics.go, router.go | SetRoute, requestLabels, UnknownRoute, OtherMethod, proxyHandle |
| response capture, Started/Unwrap/Hijack/ReadFrom | restserver/telemetry | response_capture.go | ResponseCapture |
| metric descriptors | metricskey | describe.go | HTTPReqPerf, GRPCReqPerf, *ByRole, StatsVersion, HealthLogErrors |
| metrics init (prometheus / cloudwatch) | pkg/appinit | metrics.go | Metrics |
| metrics config YAML | pkg/appinit/config | config.go | Metrics |
| logging flags / log rotation | pkg/appinit | init.go | LogConfig, Logs |
| CPU profiling | pkg/appinit | init.go, cpu_profiler.go | CPUProfiler |
| static response headers | gserver | config.go | Config.HTTPHeaders, Config.Validate |
| max message size (gRPC) | gserver, pkg/rpcclient | options.go, client.go | MaxRecvMsgSize, MaxSendMsgSize, defaultMaxCallSendMsgSize |
| HTTP body limits | xhttp/marshal, gserver, restserver | limits.go, config.go, server.go | LimitRequestBody, DecodeBody, Config.MaxRequestBody, WithMaxRequestBody, MaxRequestSize |
| Shared server timeouts | xhttp/limits, gserver, restserver, pkg/appinit/config | limits.go, config.go, server.go | Timeouts, Config.Timeouts, WithTimeouts, Prometheus.Timeouts |
| Handler deadline (server-side request bound) | xhttp/limits, gserver, restserver, pkg/appinit | limits.go, deadline.go, serve.go, server.go, prometheus.go | Timeouts.Handler, Timeouts.HandlerContext, Timeouts.ApplyHandler, deadlineTapHandle, capGRPCTimeout, setStreamWriteDeadline, deadlineUnaryInterceptor, deadlineStreamInterceptor, deadlineError, prometheusHandler |
| TLS gRPC stream read deadline | gserver | serve.go | grpcHandlerFunc, clearStreamReadDeadline |
| header copy / trailer prefix handling | gserver | header.go | copyHeader, replaceInKeys |
| header name constants | xhttp/header | headers.go | XCorrelationID, Authorization, Cookie, XCSRFToken, ... |
| correlation ID (HTTP) | xhttp/correlation | correlation.go | NewHandler, ID |
| correlation ID (gRPC) | xhttp/correlation | correlation.go | NewAuthUnaryInterceptor, WithMetaFromContext |
| correlation ID (client side) | pkg/retriable | retriable.go | convertRequest |
| gRPC stream context override | pkg/streamctx | streamctx.go | WithContext |
| structured API error | xhttp/httperror | errors.go | Error, New, Wrap |
| error → HTTP status | xhttp/httperror | errors.go | Status |
| gRPC ↔ HTTP code mapping | xhttp/httperror | codes.go, rpc.go | codeStatus, statusCode, HTTPStatusFromRPC, restoredHTTPStatus |
| multi-error (validation) | xhttp/httperror | many.go | ManyError.Add |
| gRPC error with code/correlation details | xhttp/httperror | rpc.go | Error.GRPCStatus, NewFromPb, CorrelationID |
| OAuth error mapping | xhttp/httperror | codes.go | FromOAuth |
| JSON response / error writing | xhttp/marshal | marshal.go | WriteJSON |
| generic 5xx message (no internal text) | xhttp/marshal | marshal.go | WriteJSON |
| shared error values / non-mutating writer | xhttp/httperror, restserver/ready | errors.go, many.go, ready.go | Error.WriteHTTPResponse, ManyError.WriteHTTPResponse |
| gzip responses | xhttp/marshal | marshal.go | WriteJSON |
pretty print (?pp) |
xhttp/marshal | json.go | PrettyPrintSetting |
| JSON body decoding (strict) | xhttp/marshal | json.go | DecodeBody, DecoderHandle |
| caller identity | xhttp/identity | identity.go | Identity, NewIdentity, AuthMethod |
| identity middleware (HTTP) | xhttp/identity | ctx.go | NewContextHandler |
| identity mapper error response | xhttp/identity | ctx.go | identityError |
| identity interceptor (gRPC) | xhttp/identity | ctx.go | NewAuthUnaryInterceptor, NewStreamServerInterceptor |
| request context (identity/IP/UA) | xhttp/identity | ctx.go | FromRequest, FromContext |
X-Forwarded-For / X-Real-Ip |
xhttp/identity | realip.go | ClientIPFromRequest, ClientIPFromGRPC |
| client IP as string ("" when none) | xhttp/identity | realip.go, ctx.go | IPString, RequestContext.ClientIP, RequestContext.ClientAddr |
X-Forwarded-Proto |
restserver, xhttp/identity | server.go, realip.go | GetServerURL, ForwardedProto |
| basic auth parsing | xhttp/identity | basicauth.go | BasicAuthFromRequest |
| test identity injection | xhttp/identity | identity.go | WithTestIdentity |
| identity provider / role mapping | gserver/roles | roles.go | New, IdentityProvider |
| JWT role mapping | gserver/roles | roles.go, config.go | jwtIdentity, JWTIdentityMap |
| DPoP verification (server) | gserver/roles | roles.go | dpopIdentity |
| AWS STS caller identity (AWS4 token) | gserver/roles | roles.go | awsIdentity, ValidateSTSPresignedURL, ParseSTSTokenExpiration |
| STS lookup caches and coalescing | gserver/roles | roles.go | awsCallerIdentity, leadAWSLookup, lookupAWS, cacheableSTSFailure |
| TLS / SPIFFE role mapping | gserver/roles | roles.go | tlsIdentity, GenericIdentityMap |
SkipAuthPaths |
gserver/roles | roles.go, config.go | IdentityFromRequest, IdentityMap.SkipAuthPaths |
| cookie auth / CSRF double submit | gserver/roles | roles.go, config.go | enforceCSRFCookieAndHeader, enforceCSRFMetadata, checkCSRF, CookiesConfig |
| strict auth mode | gserver/roles | config.go | IdentityMap.Strict |
| guest role | gserver/roles, xhttp/identity | roles.go, identity.go | GuestRoleName |
| path/role authorization | restserver/authz | authz.go | Provider.Allow, AllowAny, AllowAnyRole |
| authz YAML config | restserver/authz | authz.go | Config |
| authz HTTP handler / gRPC interceptors | restserver/authz | authz.go | NewHandler, NewUnaryInterceptor, NewStreamServerInterceptor |
| authz denial status (401 vs 403) | restserver/authz, xhttp/httperror | authz.go, codes.go | denied, codeStatus, statusCode |
| CORS preflight termination before authz | restserver, gserver | server.go, router.go, serve.go | NewMux, newCORS, corsHandler |
| role mapper | restserver/authz | authz.go | SetRoleMapper, SetGRPCRoleMapper |
| gRPC client credentials bundle | gserver/credentials | credentials.go | NewBundle, Bundle, bundle.NewWithMode |
| authorization metadata key | gserver/credentials | credentials.go | TokenFieldNameGRPC |
| token refresh / caller identity (client) | gserver/credentials, pkg/retriable | credentials.go, retriable.go | CallerIdentity, Token.Expired, refreshToken, WithCallerIdentity |
| DPoP proof (client) | gserver/credentials, pkg/retriable | credentials.go, retriable.go | Bundle.WithDPoP, Client.Do |
| OAuth fixed token | gserver/credentials | oauth.go | NewOauthAccess |
| gRPC client construction | pkg/rpcclient | client.go | New, NewFromURL |
| gRPC client TLS endpoint validation | pkg/rpcclient | client.go | New, Config.TLS |
| gRPC blocking dial timeout | pkg/rpcclient | client.go | Client.dial, Config.DialTimeout |
| retry policy / backoff | pkg/retriable | retriable.go | Policy, DefaultPolicy, ShouldRetry, DefaultShouldRetryFactory, waitRetry |
Retry-After / 429 retries (client) |
pkg/retriable | retriable.go | RetryAfterShouldRetryFactory, parseRetryAfter |
| HTTP client transport (TLS, DNS, clone) | pkg/retriable | retriable.go | WithTLS, WithDNSServer, WithTransport, transportClone, replaceTransport |
| error response body bound (client) | pkg/retriable | retriable.go | DecodeResponse, drainResponseBody, maxErrorBodySize |
| non-retriable errors (TLS/DNS) | pkg/retriable | retriable.go | DefaultNonRetriableErrors |
| request timeout (HTTP client) | pkg/retriable | retriable.go | Policy.RequestTimeout, WithTimeout, RequestPolicy, cancelOnClose |
| rewindable request body | pkg/retriable | request.go | Request, NewRequest |
| header propagation via context | pkg/retriable | retriable.go | WithHeaders, PropagateHeadersFromRequest |
User-Agent / X-CLIENT-IP |
pkg/retriable | retriable.go | WithUserAgent |
Bearer token storage (.auth_token) |
pkg/retriable | storage.go | Storage, SaveAuthToken, LoadAuthToken |
| token from env var | pkg/retriable, pkg/rpcclient | config.go | CheckAuthTokenFromEnv, LoadAuthTokenOrFromEnv |
DPoP key files (.jwk) |
pkg/retriable | storage.go | Storage.SaveKey/LoadKey/ListKeys, KeyInfo |
nonce / Replay-Nonce |
pkg/retriable | nonce.go | NonceProvider, NonceContext, DefaultReplayNonceHeader, Client.WithNonce |
| custom DNS server | pkg/retriable | retriable.go | WithDNSServer |
| multi-client YAML config | pkg/retriable | config.go | Config, LoadFactory, Factory |
| request dump (debug) | pkg/retriable | dumpreq.go | DumpRequestOut |
| Redis connection / TLS / auth | pkg/redisclient | redisclient.go | NewRedisClient, Config, parseURL (URL kept out of logs/errors) |
Redis key prefix / .. containment |
pkg/redisclient | redisclient.go | WithPrefix, Key, SubKey, pattern, escapeGlob |
| Redis value marshalling | pkg/redisclient | marshal.go | Marshal, UnmarshalStringCmd |
| distributed lock (owner token) | pkg/redisclient | redisclient.go | TryLock, ReleaseLock, IsLocked, releaseLockScript, setNX, coordKey, remainingTTL |
| rate limit (Redis window) | pkg/redisclient | redisclient.go | TryAcquireRateLimit, GetRateLimitRemainingTime, setNX, coordKey, remainingTTL |
| bounded set / hash (eviction) | pkg/redisclient | redisclient.go | SAddWithEviction, HSetWithEviction, boundedSetScript, checkEviction |
| key scan | pkg/redisclient | redisclient.go | ScanKeys, Keys |
| cache provider abstraction | pkg/cache | cache.go | Provider |
| cache TTL defaults | pkg/cache | cache.go | DefaultTTL, KeepTTL |
| cache expiry / cleanup | pkg/cache | memory.go | CleanExpired, NowFunc |
| read-through cache | pkg/cache | cache.go | GetOrSet |
| pub/sub (cache) | pkg/cache | memory.go, redis.go | Publish, Subscribe, Subscription |
| cache namespacing | pkg/cache | proxy.go, keys.go | NewProxyProvider, namespace |
cache key layout / .. containment |
pkg/cache | keys.go | namespace.key, namespace.rel, namespace.pattern, namespace.listed |
| cache key glob (Redis-compatible) | pkg/cache | keys.go, memory.go, redis.go | globMatch, escapeGlob, memProv.ListKeys, redisProv.ListKeys, deprecated Keys |
| cache key listing pages (SCAN cursor) | pkg/cache | cache.go, memory.go, redis.go, proxy.go | Provider.ListKeys, defaultListCount, listedKey, pageHeap |
| not-found sentinel | pkg/cache, pkg/redisclient | cache.go, redisclient.go | ErrNotFound, IsNotFoundError |
| closed-subscription sentinel | pkg/cache | cache.go | ErrClosed, closedSub |
| slow-subscriber policy (pub/sub) | pkg/cache | cache.go, memory.go, redis.go | subscriberBufferSize, memProv.Publish, redisProv.Subscribe |
| Redis subscription confirmation | pkg/cache | redis.go, cache.go | confirmSubscription, subscribeTimeout, failedSub |
| JWT token revocation | pkg/revocation | revocaction.go | New, Validate, Revoke |
| revoked-token snapshot | pkg/revocation | revocaction.go | load, reload, snapshotTTL, reloadPageSize, remember |
| revocation cache key | pkg/revocation | revocaction.go | cacheKey |
| revocation test clock | pkg/revocation | revocaction.go | TimeNowFn |
| client location from IP (geolocation) | pkg/location | location.go, ipwhois.go | Provider, Place, NewIPWhoIs, IPLocation |
| location cache per network (/24, /48) | pkg/location | cache.go | NewCache, CacheConfig, Cache.Location |
| public IP check (no lookup for private) | pkg/location | location.go | IsPublic, ParsePublicIP, parseHostAddr, nonPublicPrefixes |
| location quota (HTTP 429, Retry-After) | pkg/location | ipwhois.go | ErrQuotaExceeded, retryAfter, IPWhoIs.backOff, quotaUntil |
| cron / periodic tasks | pkg/tasks | scheduler.go | NewScheduler, Scheduler |
| schedule string format | pkg/tasks | task.go | ParseSchedule |
| task run lock / overlap prevention | pkg/tasks | task.go | task.Run, WithRunTimeout |
| task status publishing | pkg/tasks | scheduler.go | Publisher |
| test clock override | pkg/tasks, pkg/cache | task.go, cache.go | TimeNow, NowFunc |
| free port for tests | tests/testutils | testutils.go | CreateBindAddr, CreateURL |
Packages
github.com/effective-security/porto/gserver
Purpose: Combined gRPC + gRPC-Web + REST server. Start opens one cmux
listener per unique address, runs a gRPC server and an http.Server (REST
router) on each, translates gRPC-Web to gRPC on TLS listeners, and wires the
fixed middleware/interceptor chains.
Files:
doc.go— package overview and usage example.server.go—Server/GServer,Start(resolveDependencies,abort), service registry,Close, error channel.listener.go—backoffListener: waits after a failed rootAcceptbefore cmux retries it.ratelimit.go—rateLimiter(one tollbooth limiter per listener):newRateLimiter,httpHandler(REST limiter and its rejection telemetry),unaryInterceptor/streamInterceptor(allowGRPC,grpcRequest),keyandclientIP(limiter key),logRejection(RateLimit.LogRejectionsline).serve.go—configureListeners(URL schemes, TLS, keepalive, accept backoff; returns the sharedtransport.TLSInfothe server owns),serveCtx.serve(cmux split, onenewRateLimiterper listener) andserveCtx.closeServers,configureHandlers(HTTP chain),corsHandler(REST/preflight CORS response headers; an empty origin list allows none, and a disallowed actual request still reaches the handler),grpcServer(interceptor chain, with the listener's limiter),grpcHandlerFunc(gRPC / gRPC-Web / REST mux on TLS listeners),NewRequestValidationUnaryInterceptor,panicInterceptor.grpc_web_response.go—grpcWebResponse:http.ResponseWriterthat rewrites gRPC responses into gRPC-Web framing (trailer frame, base64 for-text, pooled optional gzip for unary).header.go—copyHeaderand option helpers used for gRPC → gRPC-Web header/trailer translation.logs.go— gRPC request logging/metrics interceptors (newLogUnaryInterceptor,newLogStreamServerInterceptor,logRequest), the caller handoff from identity back to them (callLog,withCallLog,identityHandoffUnaryInterceptor,identityHandoffStreamInterceptor,callerOf) andlogError(theAPI_ERROR/INTERNAL_ERRORline).deadline.go— the handler deadline of gRPC and gRPC-Web calls (Config.Timeouts.Handler):deadlineTapHandle(the stream deadline of the native HTTP/2 transport, plaintext listeners),capGRPCTimeoutwithencodeGRPCTimeout/decodeGRPCTimeout(thegrpc-timeoutcapgrpcHandlerFuncapplies beforeServeHTTPon TLS listeners),setStreamWriteDeadlinewithdeadlineWriteGrace(the per-stream write deadline that releases a send blocked on the client's window on TLS listeners),deadlineUnaryInterceptor,deadlineStreamInterceptor(bound the call context withlimits.Timeouts.HandlerContext) anddeadlineError(a plain context error returned by the handler becomesDeadlineExceeded/Canceled).config.go—Config(Config.Validate,Config.shutdownTimeout),TLSInfo,KeepAliveCfg,CORS(CORS.Validate),RateLimit(RateLimit.Validate),SwaggerCfg.options.go—WithMiddleware,WithUnaryServerInterceptor,WithStreamServerInterceptor,MaxRecvMsgSize,MaxSendMsgSize.
Entry points:
Start(name, cfg, container, factories, opts...) (GServer, error)— the only constructor; starts serving immediately.GServer—AddService,Service,IsReady,Err,Close,Discovery.ServiceFactory,Service,RouteRegistrator,GRPCRegistrator,StartSubcriber— plug-in contracts.ConfigYAML keys:listen_urls,server_tls{cert,key,trusted_ca,client_ca,client_cert_auth,cipher_suites},services,identity_map,authz,cors,rate_limit,trusted_proxy_cidrs,timeouts{header,read,idle,handshake,handler},max_request_body,timeout.shutdown(timeout.requestis its deprecated former name;Validaterejects both set),keep_alive{min_time,interval,timeout},max_recv_msg_size,max_send_msg_size,http_headers,logger_skip_paths,prom_grpc,debug_logs.
Invariants:
Startreturns errors, starting with theConfig.Validatechecks (TrustedProxyCIDRs,CORS.Validate,RateLimit.Validate,Timeout.ShutdownandTimeout.Requestboth set, static CORS headers);newServercalls the unexportedConfig.validate, which also returns the parsedTrustedProxyCIDRspolicy so it is parsed once. WhenStartfails afternewServercreated services or opened listeners,abortcloses the serve channels and callsClose, so services, listeners and the TLS reloader are released;configureListenerscloses the listeners it opened and stops the reloader on its own errors.configureHandlerspanics through the logger if the authz handler cannot be built. gRPC handler panics are recovered by the unary and stream interceptors ofxhttp/correlationand the unary one ofxhttp/identityintocodes.Internalwith messageunhandled exception; identity useshttperror.NewGrpcFromCtx, while correlation returns a gRPC status directly becausehttperrorimports correlation.panicInterceptorcatches panics outside them. The log interceptors register their recording defer before calling the handler, count escaping panics as Internal and leave recovery to the surrounding interceptor; ordinary errors usestatus.Code(Unknown for plain errors), and wrappedhttperrorcauses are found witherrors.As. A REST handler's panic is counted by the outer metrics handler ofconfigureHandlers(500 unless the handler sent a status) before it reachesrecoverHTTPPanicingrpcHandlerFunc(TLS listeners: 500 only before the response starts, otherwise an abort;http.ErrAbortHandlerpasses through without logging) or net/http (plaintext listeners, which aborts the connection).- Process-global: package logger;
WarnUnaryRequestLatency. The gRPC gzip compressor is registered by blank import. - Log field names:
remoteis the client IP resolved under the trusted proxy policy (identity.IPString, "" for a Unix socket peer) in every request line (REST access line and 429, gRPClogRequestandlogErrorlines, thecors_not_allowedline and the two per-requestDebugLogslines ofgrpcHandlerFunc, and thexhttp/identityWARNING lines);peeris the socket address (host:port,@for a Unix socket peer) in thelogRejection,cors_not_allowedand per-requestDebugLogslines and inpkg/transport;agentis the User-Agent,telemetry.NoAgentwhen missing, in the access, gRPC,logRejection,cors_not_allowedandDebugLogslines. TheDebugLogsline written after a gRPC-Web response has only the method and headers. Thexhttp/marshalAPI_ERRORline has no client IP (marshalcannot importxhttp/identity) and logs an emptyagentwhen missing; join it to the access line onctx. - Every name in
Config.Servicesmust exist in the factory map; factories run throughdig.Container.Invoke. The container must providediscovery.Discoveryand, when JWT/DPoP is enabled,jwt.Parser. - Listen URL schemes:
http,https,unix,unixs; no scheme → https when TLS is configured. The same address listed twice yields one listener serving both secure and insecure. - Network limits:
Config.Timeoutsusesxhttp/limitsdefaults (header 10s, read 30s, idle 60s, handshake/detection 10s); zero selects defaults and negatives disable individual limits. cmux detection andtransport.TLSInfo.HandshakeTimeoutuse Handshake. Both HTTP servers limit bodies toMaxRequestBody(default 10 MiB), including gRPC/gRPC-Web over TLS. Native plaintext gRPC keeps its per-message limits. On TLS,grpcHandlerFuncclears the per-stream read deadline for native gRPC over HTTP/2 (clearStreamReadDeadline), so client and bidi streams may outliveTimeouts.Read;MaxRequestBodystill bounds each stream in total because grpc-go'sServeHTTPtransport buffers request data without flow control (ROADMAP 12), and a stream over it fails withUnavailable. Raise or disable read/body limits for large uploads and long request streams. The read deadlines end when the request has been read; the server-side bound on a request's execution isTimeouts.Handler(deadline.go,limits.Timeouts.ApplyHandlerfor REST), off by default: the context of every REST request and gRPC/gRPC-Web call, streams included, expires after it unless the client asked for an earlier deadline, which is kept (context.WithTimeoutkeeps the earlier parent deadline). The REST deadline wraps identity extraction and the handler, inside correlation; the gRPC one runs inside the log interceptor, so a handler that returns its expired context error (no gRPC status) is answered, logged and counted asDeadlineExceeded(Canceledwhen the client went away) with the handler's error as the cause, while errors carrying a status are unchanged. For gRPC the deadline is also the transport stream deadline, so a stream handler blocked inRecvMsgorSendMsg(a client that stopped sending without closing its send side, a client that stopped reading but kept the connection open) is released;streamctx.WithContextreplaces onlyContext, so the interceptors alone could not do that. On plaintext listenersdeadlineTapHandle(grpc.InTapHandle, an experimental grpc-go API) returns the bounded context the transport then uses as the stream context (it cancels the parent when the stream ends, which stops the child's timer, so no cancel is kept): both a blocked receive and a blocked send fail withDeadlineExceeded, and that status reaches the client. On TLS listenersgrpcHandlerFunccaps thegrpc-timeoutrequest header (capGRPCTimeout: missing or longer thanHandler→Handlerin the wire format, rounded up to the finest unit that fits 8 digits; shorter kept; malformed left for grpc-go to reject) beforeServeHTTP, which makes it the stream deadline while the request context stays alive, so a blocked receive fails withDeadlineExceeded; bounding the request context instead makes grpc-go close the transport and the client getsUnavailable: server stopped. That deadline does not release a send: grpc-go'sServeHTTPtransport performs it as a net/http write that waits for the client's flow-control window, so for HTTP/2 requestsgrpcHandlerFuncalso sets a per-stream write deadline ofHandler+deadlineWriteGrace(1s) throughhttp.ResponseController(setStreamWriteDeadline); when it expires net/http resets the stream (RST_STREAMINTERNAL_ERROR), the send fails withUnavailable, the handler returns, and the client seesInternal, since no trailers can follow an aborted write. The grace leaves a handler that returned at the deadline time to deliver its status; a response still being written after it is cut the same way, which plaintext listeners do not do. gRPC-Web over HTTP/1.1 gets no write deadline (it would be the connection's and outlive the request). grpc-go decodes a unary request before the interceptors run, so the bound starts after the decode. Handlers that ignore their context outlive the deadline, as with a client deadline, and a REST handler blocked writing to a client that stopped reading is not released (no response write deadline).Timeout.Shutdown(former nameTimeout.Request) is shutdown-only. - Plain listeners: cmux routes HTTP/2 → gRPC (h2c), HTTP/1 → REST; gRPC-Web and
HTTPHeadersare only handled on TLS listeners, where everything goes throughhttp.ServerandgrpcHandlerFuncselects byContent-Type. - HTTP chain (outer → inner): trusted proxy policy → body limiter (bounds reads, writes no response) → rate limit → correlation → handler deadline (if
Timeouts.Handleris set) → metrics → CORS (if enabled) → identity → metrics (role handoff, records nothing) → request logger → authz (if configured) → readiness →WithMiddleware→ router. On TLS listenersgrpcHandlerFuncsits between the body limiter and the rate limit and sends gRPC and gRPC-Web to the gRPC server, whose responses the HTTP metrics never see. CORS answers preflights before authz; withoptions_pass_through,OPTIONSrequests are authorized like any other method. The outer metrics handler counts identity rejections and CORS preflights with the guest role. Rate limit (ratelimit.go): each listener has onerateLimiter(newRateLimiterinserveCtx.serve) shared by its REST handler (httpHandler) and the interceptors of its gRPC servers, so plaintext (h2c) gRPC, which cmux sends past the HTTP handler, is limited, and a gRPC call on a TLS listener draws one token. Buckets are tollbooth's: client address, request path andMetods; a gRPC call is checked as the POST to its full method name (grpcRequest: the socket peer, with thex-forwarded-for,x-real-ipanduser-agentmetadata as headers). The default key is the client IP resolved under the trusted proxy policy (clientIP:identity.IPStringofidentity.ClientIPFromRequest, "" when the peer is not an IP, as for a Unix socket peer@;grpcRequestputs the policy in the call context): tollbooth checks a shallow request copy whoseRemoteAddris that bare IP (limitRequest), soX-Rate-Limit-Request-Remote-Addrechoes that client IP rather than the socket peer (tollbooth then folds an IPv6 address to its /64 prefix for the bucket); a request without a client IP, including every Unix socket caller, is not limited. Explicitheaders_ip_lookupsreads the named headers (metadata for gRPC) as sent, overrides that policy, and must be used only when a proxy overwrites them. A rejected REST request never reaches the chain above: it is answered liketollbooth.LimitHandler(itsOnLimitReachedis unused) behind its own correlation → metrics (guest,unknown) → request logger (SkipLogPathsapply), so a 429 has anX-Correlation-IDand an access-log line whoseremoteis that client IP. A rejected gRPC or gRPC-Web call returns anhttperrorResourceExhaustedwith the tollbooth message and the correlation ID; the limiter interceptor runs after the log interceptor, which logs it with the client IP resolved under the trusted proxy policy and counts it in the gRPC metrics with the guest role, and before identity, so a rejected call does no identity work. grpc-go reads and decodes a unary request before any interceptor runs, so a rejected call still costs its decode (bounded byMaxRecvMsgSizeandMaxRequestBody); calls to unregistered methods (Unimplemented) and HTTP/1.1application/grpcrequests on TLS (505) are answered by grpc-go without drawing a token. A unary call draws its token beforeNewRequestValidationUnaryInterceptor, so a request that failsValidateis limited, logged and counted like any other error; a panic inValidatefails the call withInternal(validaterecovers it before it leaves the log interceptor).RateLimit.LogRejections(log_rejections, off by default;GetLogRejections) adds a WARNING line per rejection (logRejection, reasonrate_limit) in the request context, with the limiter key (rateLimiter.key, canonicalized like the bucket: an IPv6 address as its /64 prefix), the socket peer, the User-Agent and the forwarding headers as received; itsctxis theX-Correlation-IDand access-linectxof a 429 and the request ID of a gRPC error (unary gRPC log lines carry the samectx; stream log lines have none, because the stream log interceptor runs outside correlation). Unary gRPC chain: panic recovery → correlation → log → handler deadline (ifTimeouts.Handleris set) → rate limit (if enabled) → validation → identity → identity handoff → authz (only when configured) → prometheus (opt) → custom. Stream chain: log → correlation → handler deadline (if set) → rate limit (if enabled) → identity → identity handoff → authz (only when configured) → prometheus → custom. gRPC log lines and metrics (logs.go): the log interceptor runs before identity, so it puts acallLogin the call context (streams get it throughstreamctx.WithContext) and the handoff interceptor stores the identityRequestContextin it;logRequestthen logsremote(that context's client IP) and labelsGRPCReqByRolewith its role. A call rejected before or by identity (rate limit, validation, identity errors) is logged with the client IP thatcallerOfresolves under the server's trusted proxy policy and counted as guest; itslogErrorline has the sameremote. Before this handoff (P-105) every gRPC line had an emptyremoteand every call counted as guest.agentis theuser-agentmetadata, ortelemetry.NoAgentwhen missing;SkipLogPathsmatches the metadata as sent (empty when missing, unlike the REST request logger, which matchesNoAgent), and a skipped call records no gRPC metrics either. - Headers read:
Content-Type,Origin,Accept-Encoding,X-GRPC-Stream. For gRPC-Web, CORS headers are written only whenCORS.GetEnabled()and an allowedOriginis present. NonemptyAllowedOriginsusesrs/corsmatching, including its origin patterns; explicit*allows all origins and an empty list allows none.Startrejects an enabled*withAllowCredentials(CORS.Validate) and, while CORS is enabled, anyAccess-Control-*name inHTTPHeaders(Config.Validate, case-insensitive), so static headers cannot bypass the CORS policy; gRPC-Web never sendsAccess-Control-Allow-CredentialswithAccess-Control-Allow-Origin: *. A disallowed gRPC-Web POST gets HTTP 403; a disallowed preflight gets no allow-origin header. REST CORS is not request or CSRF protection:rs/corsstill runs the handler for a disallowed actual request and only omits CORS response headers.Access-Control-Expose-Headersmerges configured and service names with response and gRPC trailer names without case-insensitive duplicates; service response metadata cannot override server CORS policy headers. Reflected and denied origins addVary: Origin;prepareHeadersmerges serviceVaryheader and trailer metadata without dropping it. Other headers written:Content-Encoding: gzip,Config.HTTPHeaders. - gRPC-Web compression uses
marshal.AcceptsGzip, honoring exact coding names, quality values and allAccept-Encodingheader lines;X-GRPC-Streamdisables compression.serveGRPCHTTPcompletes a ServeHTTP transport that returned HTTP 200 without a gRPC status (including shutdown before or after response headers) withUnavailable; non-200 HTTP errors from grpc-go are preserved. - Compressed gRPC-Web responses borrow a gzip writer from a pool;
Closeis idempotent because bothfinishRequestand the handler's deferred cleanup call it. Writers are reset before reuse and release the previous response writer on return to the pool. AfterClose, body writes to a compressed response returnerrWriteAfterCloseand late trailer writes are logged and dropped, so raw bytes never follow the gzip footer. - Close ordering (
teardown, run once undercloseOnce, so services are closed once and a repeated or concurrentClosereturns after the first teardown finished):stopc→ per listener, each bounded byConfig.shutdownTimeout(Timeout.Shutdown, else its former nameTimeout.Request, elsedefaultShutdownTimeout, 10s; 3s before v1.0.x): TLShttp.Shutdown(drains REST and gRPC overServeHTTP) thengrpc.Stop; plaintext concurrenthttp.Shutdownandgrpc.GracefulStop, withgrpc.Stopon timeout →http.Closeto close remaining HTTP connections → listener contexts canceled → servicesClose()→ listeners closed →transport.TLSInfo.Close(stops the certificate reloader). Idle HTTP keep-alive connections are always closed. Handlers that ignore cancellation may outlive the deadline; teardown does not wait for plaintext GracefulStop's handler wait after forcing Stop, so services that release a blocked handler from Close can finish cleanup. HTTP shutdown/close errors are intentionally ignored because Close has no error result.Err()is buffered and never closed.Closeranges over each listener'sserversC, so that channel is always closed:servecloses it (closeServers, once-guarded) after publishing its servers or, deferred, when it fails;abortcloses it for a server that never served.servestarts no gRPC or HTTP server until every fallible step (the TLS listener) succeeded, because a gRPC server blocked inAccepton a cmux listener cannot be stopped untilm.Serveruns. Its deferred cleanup stops the gRPC servers only for those setup failures (servingis false): closing a cmux child listener (GracefulStopon plaintext,http.Shutdownon TLS) closes the root listener, som.Servereturns an error whileCloseis still draining. - An enabled
RateLimitmust passRateLimit.Validate: positiveRequestsPerSecond(tollbooth would otherwise admit one request per client and reject the rest);ExpirationTTLzero (10 minutes,defaultRateLimitTTL) or at leastminRateLimitTTL(1s), because tollbooth recreates an expired bucket with a full burst;HeadersIPLookupslimited to the names tollbooth compares exactly (RemoteAddr,X-Forwarded-For,X-Real-IP;rateIPLookups); andMetodsentries that are single upper-case HTTP method tokens (isUpperMethodToken, RFC 9110 tchar), since tollbooth compares methods exactly and a non-matching entry limits nothing. - Keepalive:
MaxConnectionIdlefixed at 5m; enforcementMinTimeonly if >0;Time/Timeoutonly if both >0. These gRPC transport settings apply to plaintext (h2c) gRPC only; TLS gRPC is served throughhttp.ServerandServeHTTP(ROADMAP 12). - Accept errors:
configureListenerswraps every root listener (TCP aftertransport.NewKeepAliveListener, and Unix) inbackoffListener(listener.go), which waits 5ms doubling to 1s after each failedAccept(reset by a successful one, cut short byClose, logged as WARNINGaccept_failed) and returns the error unchanged; an error wrappingnet.ErrClosedreturns at once. cmux then retries temporary errors (EMFILE) without spinning and stops on the others.
Tests: shutdown_test.go covers HTTP keep-alive closure, draining before service cleanup on plaintext and TLS, and cancellation after timeout; panic_test.go covers metrics for escaping unary/stream panics, wrapped errors and partial-response aborts over TLS; serve_test.go covers gzip negotiation. grpc_test.go also checks shutdown status before/after TLS response headers and a handler released by service Close. dpop_test.go (TestGRPCDPoP: stored-key pkg/rpcclient authenticates unary and streaming calls with real signed access tokens over TCP TLS and Unix TLS, including TLS server-name and grpc.WithAuthority overrides; hostless overrides fail before handlers run); server_test.go and example_test.go start real servers on random ports using tests/mockappcontainer and tests/testutils.CreateURL (the example binds 127.0.0.1:0 and a Unix socket in a short os.MkdirTemp directory); metrics_test.go (white-box, process-global in-memory sink, not parallel) checks the exact HTTP metrics of configureHandlers (role handoff, identity 401 and CORS preflight as guest) and of both REST limiter variants (rateLimiter.httpHandler: 429 with guest and unknown), and TestPanicCounted the metrics of panicking routes behind grpcHandlerFunc (500 with route and role, or the status already sent), with a fake roles.IdentityProvider; TLS fixtures in testdata/test-server{,-key,-rootca}.pem; lifecycle_test.go (white-box; its goroutine-counting tests run sequentially because they count KeypairReloader and server accept loops process-wide) checks that Close, a failed Start (missing discovery, identity provider error, failed or missing service factory), a failed configureListeners and a failed serve release services, listeners, the reloader and the serve channel, that a failed serve starts no server, and that overlapping Close calls (run under -race) close services once; grpc_web_test.go and grpc_web_bench_test.go unit-test the gRPC-Web writer with httptest.ResponseRecorder; serve_test.go drives grpcHandlerFunc with a bare grpc.NewServer(); limits_test.go checks protocol detection, stalled headers/bodies and HTTP body limits on plain and TLS listeners, including HTTP/2 unknown-length bodies and stalled request streams; it registers a client-streaming porto.test.Upload service from a hand-written grpc.ServiceDesc to check that TLS gRPC streams outlive Timeouts.Read but stay within MaxRequestBody, and checks that a 413 carries CORS and correlation headers. grpc_test.go (black box) serves a hand-written echo service on a plaintext listener and checks the unary chain end to end (validation before the option interceptors, which see the caller identity; panics in Validate and handlers; MaxRecvMsgSize/MaxSendMsgSize from Config and options), the exact gRPC metrics (process-global sink, not parallel) and that Close drains active RPCs for Timeout.Request before stopping them on plaintext and TLS listeners (TestCloseStopsStreamsAfterRequestTimeout; startEcho serves TLS when ServerTLS is set); server_test.go also covers readiness, notFoundHandler, Discovery and Start failures that release the listener. calllog_test.go (white-box) serves a hand-written porto.test.CallLog service with a trusted loopback peer and compares whole JSON log lines (logBuffer.lines drops time and duration and replaces the correlation ID): TestLogRequestCaller (grpcServer with headerIdentity, whose role comes from the X-Test-Role header or metadata) checks the logRequest lines and GRPCReqByRole roles of admitted and identity-rejected unary and stream calls and the identity access_denied lines; TestLogRequestSkipNoAgent checks NoAgent and that SkipLogPaths matches the User-Agent as sent; TestLogRequestRemoteListeners (Start, plaintext and TLS) checks remote of admitted and rate-limited calls and their logError line; TestGRPCHandlerLogFields (TLS) checks the cors_not_allowed and DebugLogs lines. They install process-global log and metrics sinks and start TLS reloaders, so none is parallel. rate_proxy_test.go (white-box) checks the limiter keys of both variants with trusted and spoofed forwarding, Unix socket peers (not limited), Metods, gRPC keys and per-path buckets from metadata (allowGRPC), the interceptors, and the 429 access line, X-Correlation-ID and LogRejections lines for HTTP and gRPC (TestRateLimitRejectionLog installs a process-global log formatter, so it is not parallel); ratelimit_test.go (black box, startEcho on plaintext and TLS listeners) checks trusted-proxy keys, per-path buckets and one token per TLS call (TestGRPCRateLimit), the exact gRPC metrics of a ResourceExhausted rejection with no HTTP metric (TestGRPCRateLimitMetrics, not parallel), and on TLS a gRPC-Web ResourceExhausted with CORS headers and a REST 429 with a correlation ID (TestGRPCWebRateLimit). listener_test.go (white-box) checks the backoff schedule and reset, that errors are returned unchanged, that net.ErrClosed and Close end the wait, that cmux behind backoffListener makes few Accept calls during an EMFILE window and serves afterwards, and that configureListeners wraps TCP and Unix listeners and rejects malformed URLs, https without TLS and missing certificates. deadline_test.go (black box, startEcho on plaintext and TLS listeners; the echo service's echoWait, echoWaitStatus and echoDeadline values and its only REST route, GET /v1/deadline) checks that Timeouts.Handler bounds unary calls, server streams, gRPC-Web calls and REST requests, keeps an earlier client deadline, caps a later or missing one, answers a handler's wrapped context error with DeadlineExceeded and keeps a status error, that a Collect client-stream handler blocked in RecvMsg behind a client that stopped sending is released with DeadlineExceeded on both listener types (TestHandlerTimeoutStalledReceive, the server-side error on echoServer.collectErr), that a Flood server-stream handler blocked in SendMsg behind a client that stopped reading (a static 64 KiB stream window through dialEcho with grpc.WithInitialWindowSize, so the block is deterministic) is released after the deadline while the connection stays open (TestHandlerTimeoutBlockedSend, the server-side error on echoServer.floodErr: DeadlineExceeded on plaintext, Unavailable on TLS where the client then sees Internal), and that nothing is bounded when unset; deadline_unit_test.go (white box) covers every deadlineError branch, the grpc-timeout codec and cap, and Config.shutdownTimeout with both names.
github.com/effective-security/porto/gserver/credentials
Purpose: gRPC credential helpers: a grpccredentials.Bundle (TLS transport plus per-RPC authorization metadata with static, refreshable or DPoP-signed tokens) and a fixed-token PerRPCCredentials.
Files:
doc.go— overview and client example.credentials.go—Config,Token,CallerIdentity,Bundle,NewBundle, transport wrapper,perRPCCredential(refreshToken,leadRefresh,finishRefresh,fetchToken,copyToken),TimeISO8601,CacheTTL,TokenFieldNameGRPC.oauth.go—NewOauthAccess.
Entry points: NewBundle(Config) Bundle (TransportCredentials(), PerRPCCredentials(), UpdateAuthToken, WithCallerIdentity, WithDPoP); NewOauthAccess(token); Token.Expired.
Invariants:
- Process-global vars:
TokenFieldNameGRPC = "authorization",CacheTTL = 5m(also the AWS cache TTL inroles), package logger. - Metadata written:
authorization: "<TokenType> <AccessToken>"anddpop: <proof>when the type isDPoPand a signer is set. DPoP signsPOST https://<authority><full method path>: the authority comes from grpc-go's absolute HTTPS credential audience (including TLS server-name/authority overrides), and the full method comes fromcredentials.RequestInfo. A missing/invalid audience or method returns an error. Each RPC signs a fresh proof; default port 443 may be omitted by grpc-go. RequireTransportSecurity()is always true;NewWithModereturns an error for every mode (the grpclb and RLS balancers request it; a copy would send the token to the balancer).perRPCCredential.muguards the token, DPoP signer, provider and the in-flight refresh;GetRequestMetadatasnapshots them underRLockand reads the signer again after a refresh, because a provider may install the signer bound to the new token withWithDPoP(also for RPCs that shared the refresh).- Token refresh happens lazily inside
GetRequestMetadatawhenExpired()(less than one minute left).refreshTokenmakes oneGetCallerIdentitycall for all concurrent RPCs, with the context of the RPC that started it; waiters take its token or error and return their own context error when that ends first. A live waiter makes the next call when the shared one failed after its RPC's context ended (tokenRefresh.canceled) or whenWithCallerIdentityreplaced the provider (superseded: the running call's result is discarded, also for its own RPC).leadRefreshreleases the waiters withcaller identity provider panickedif the provider panics and lets the panic continue, so a recovered panic cannot leave the refresh pending. The stored token is a copy withExpiresalways set (CacheTTLfrom now when the provider sets none);UpdateAuthTokenalso stores a copy ofExpires. Provider errors are wrapped asunable to get caller identity, and a nil token is an error. Without a provider an expired non-empty token is still sent; an empty token sends no metadata. - DPoP proof errors are wrapped as
unable to sign DPoP proof.
Tests: credentials_test.go and dpop_test.go, no network; TestPerRPCDPoPURI covers absolute proof URIs, IPv6/ports and invalid or missing audience/method inputs; the refresh tests run in testing/synctest bubbles (shared call and shared error, short-lived token, canceled first caller, waiter deadline, provider panic, provider replaced during a refresh, provider installing the DPoP signer during a refresh) and TestPerRPCConcurrentSetters races the setters against RPCs under -race.
github.com/effective-security/porto/gserver/roles
Purpose: Authenticates HTTP and gRPC callers (AWS STS presigned tokens, DPoP JWT, bearer JWT via header or cookie, TLS client cert with SPIFFE SAN) and maps them to roles, producing identity.Identity.
Files:
doc.go— overview, example, YAML layout ofidentity_map.config.go—IdentityMap(debug_logs,strict,cookies{auth,csrf,domain},skip_auth,tls,jwt,jwt_dpop,aws),GenericIdentityMap,JWTIdentityMap,AWSIdentityMap,CookiesConfig.roles.go—IdentityProvider,New,IdentityFromRequest/IdentityFromContext, per-method verifiers (awsIdentity,dpopIdentity,jwtIdentity,tlsIdentity), CSRF checks (enforceCSRFCookieAndHeader,enforceCSRFMetadata,checkCSRF), STS lookups (awsCallerIdentity,leadAWSLookup,cachedAWS,lookupAWS,cacheableSTSFailure,stsErrorCode,withoutURL),dumpDM,ValidateSTSPresignedURL,ParseSTSTokenExpiration,CallerIdentity, role-name constants.
Entry points:
New(*IdentityMap, jwt.Parser) (IdentityProvider, error)— errors if JWT/DPoP is enabled without a parser, or JWT is enabled withcookies.authbut nocookies.csrf.IdentityProvider.IdentityFromRequest(HTTP) andIdentityFromContext(ctx, uri)(gRPC) — the mappers gserver passes toidentity.NewContextHandler/identity.NewAuthUnaryInterceptor.- Constants
GuestRoleName,TLSUserRoleName,JWTUserRoleName,DPoPUserRoleName,AWSUserRoleName,Default{Subject,Role,Tenant}Claim.
Invariants:
- Never panics; returns errors only in
Strictmode; otherwise falls through to the next method and finally to guest. - Evaluation order:
SkipAuthPaths(exact path, HTTP only) →AWS4→DPoP→Bearerheader, else cookiecookies.authwith the CSRF double-submit check (checkCSRF:X-CSRF-Tokenequals thecookies.csrfcookie, constant time) on unsafe HTTP methods and on every gRPC call (enforceCSRFMetadata; gRPC-Web headers arrive as metadata) → TLS peer cert (first cert must carry exactly onespiffe://URI SAN). A scheme without a space is treated asBearer. A failed CSRF check or cookie JWT falls through and never returns an error, even inStrict; CSRF errors never contain token values. - Headers/metadata read:
Authorization,DPoP,Cookie,X-CSRF-Token(header.XCSRFToken; metadataauthorization,dpop,cookie,x-csrf-token);header.Authority(:authority) for the gRPC DPoP URI; gRPC peer TLS state viapeer.FromContext. Withdebug_logs,dumpDMlogs the incoming metadata withauthorization,proxy-authorization,cookie,dpopandx-csrf-tokenvalues replaced by[REDACTED]. - Role lookup: reverse map
claim value → rolebuilt once inNew(last role wins); fallbackdefault_authenticated_role. JWT claims default tosub/email/tenant. AWS subject form"<account>:<type>/<name>"or full ARN;allowed_accountsfilter. - AWS4 tokens are base64url presigned STS URLs.
ValidateSTSPresignedURLrequireshttps, no userinfo, an AWS STS host (global, regional, FIPS, China, or VPC endpoint underamazonaws.com) andAction=GetCallerIdentitybefore any request is made; the lookup uses a dedicated client (provider.sts,stsHTTPClient) with a 10s timeout and a 64 KiB body cap. The presigned URL is a bearer credential: failure logs name only the host, andwithoutURLdrops the URL from*url.Errorcauses (parse and transport errors) before they are wrapped. Successful lookups are cached in a 100-entry LRU (awsCache) keyed by URL forcredentials.CacheTTLas read byNew(awsTTL; not positive: no expiry, until evicted). Failures caused by the URL are cached forawsFailureTTL(30s) in a separate 100-entry LRU (awsFailures). Both are plainlru.Cachevalues with a per-entry expiry checked on read (an expired entry is ignored until replaced or evicted), soNewstarts no goroutine andIdentityProviderneeds noClose. Bad tokens cannot evict successes: STS 4xx except 408, 429 and the throttling codes intransientSTSErrorCodes(STS throttles with HTTP 400;stsErrorCodereads the JSON or XML error code), and an undecodable 200 body. Transport errors, timeouts, 408, 429, throttling and 5xx are not cached. A cached failure returnscached STS lookup failure: <error>. Concurrent lookups of one uncached URL share one STS call (awsLookupsunderlookupMu), made with the context of the request that started it, so a reset request still cancels its call; results are cached before the entry is removed. A waiter returnsunable to get Caller Identity from AWS: <context error>when its context ends first and makes the next call when the shared one failed after its request's context ended;leadAWSLookupreleases the waiters withSTS lookup panickedif the lookup panics and lets the panic continue on the request. - DPoP proofs are verified against
https://<host><path>for HTTP andPOST https://<incoming :authority><full method path>for gRPC. gRPC requires exactly one authority of the form host[:port] with a non-empty hostname, parsed from its escaped wire form (grpcDPoPURI; an IPv6 zone arrives as%25); missing, ambiguous, hostless or malformed authority, an authority with userinfo, path, query or fragment, relative legacy proofs and authority/method/key mismatches fail authentication (error inStrict, otherwise fallback). xpki normalizes host case and default HTTPS port 443; method path case is significant. Forwarded headers never supply the gRPC authority. A JWT withcnfis rejected on the bearer path.
Tests: dpop_test.go (TestGRPCDPoPBinding: real signed access tokens and proofs, successful IPv6, IPv6-zone and default-port binding, rejected non-host[:port] authorities and authority/method/key failures in Strict and fallback modes); roles_test.go (mock JWT parser, createPeerContext, TestValidateSTSPresignedURL, cookie CSRF over gRPC metadata, New cookie validation), csrf_test.go, internal_test.go, aws_test.go (white-box: replaces provider.sts with a counting RoundTripper to check the success and failure caches including throttling codes, success expiry (TestAWSIdentitySuccessExpires, in a testing/synctest bubble, which also fails if New leaves a goroutine behind), failure expiry, that failures do not evict successes, the shared lookup, canceled waiters and leaders, a panicking lookup and URL-free parse errors), aws_errors_test.go (token decoding, expiry, host, account and ARN rejections with STS call counts; Strict versus guest on HTTP and gRPC) and strict_test.go (Strict versus guest for DPoP and SPIFFE failures, SkipAuthPaths, ApplicableFor*). No network.
github.com/effective-security/porto/restserver
Purpose: HTTP/HTTPS REST server hosting Service implementations behind an httprouter mux with a fixed middleware chain.
Files:
server.go—HTTPServerlifecycle (New/StartHTTP/StopHTTP), middleware assembly inNewMux, lifecycle events,GetServerURLhelpers.router.go—Routerinterface over httprouter,Params,Handle,CORSOptions(rs/cors adapter includingwrapAllowOriginRequestFunc);proxyHandlerecords each route's registered path withtelemetry.SetRoutefor the metrics uri label.config.go—Config/TLSInfoConfigcontracts,GetPort,GetHostName.service.go—Serviceinterface.
Entry points:
New(version, ipaddr, Config, *tls.Config) (*HTTPServer, error)— nil TLS means plain HTTP.WithAuthz/WithIdentityProvider/WithTrustedProxies/WithCORS/WithShutdownTimeout/WithTimeouts/WithMaxRequestBody/WithMuxFactory— configure beforeStartHTTP; all butWithMuxFactoryreturn the server for chaining.AddService,StartHTTP,StopHTTP,NewMux,IsReady,OnEvent;Routermethods;GetServerURL,GetServerBaseURL.
Invariants:
StartHTTPbinds HTTP and HTTPS listeners synchronously and returns bind errors; serving remains asynchronous. A failed bind can be retried, but a successfully started instance cannot be restarted.AddServicepanics on duplicate names;NewMuxpanics if the authz handler cannot be built.- Chain (outer → inner): trusted proxy policy → correlation → handler deadline (if
Timeouts.Handleris set) → body limiter (bounds reads, writes no response) → metrics → identity → metrics (role handoff, records nothing) → request logger → CORS (if set) → authz (if set) → ready → router. The outer metrics handler also counts identity mapper rejections, with the guest role. CORS (newCORS, shared withNewRouterWithCORS) sits outside authz sors/corsanswers preflights before authorization and denied or not-ready responses carry CORS headers; withOptionsPassthroughanOPTIONSrequest continues into authz.NewRouterWithCORSstill wraps only the router and is for custom mux factories, which must keep CORS outside authz themselves.WithTrustedProxiestakes a policy fromidentity.ParseTrustedProxies(CIDR errors are returned there); nil trusts no proxy.StartHTTPapplies the policy, the body limit and the handler deadline to custom mux factories too. Default identity mapper isidentity.GuestIdentityMapper; logger granularity istime.Millisecond. StopHTTP: mark unready → wait for Started callbacks → broadcast Stopping →ShutdownwithshutdownTimeout(default 5s) →Service.Close()for all → broadcast Stopped. Calls before start do nothing; concurrent and repeated calls wait for the first shutdown. A timeout still closes services afterShutdownreturns. Lifecycle callbacks must not callStopHTTPsynchronously.servingis anatomic.Bool;IsReadyreads services underlock.RLock.HTTPServer.Config()andHTTPConfig()return the same configuration. Event handlers are copied underlock.RLockbefore callbacks run, so callbacks may register handlers.GetPortusesnet.SplitHostPortfor host:port and defaults bare IPv6 literals to port 443.GetHostNamereturns IPv6 hosts without brackets;GetServerURLandGetServerBaseURLrebuild host:port withnet.JoinHostPort.GetServerURLacceptsX-Forwarded-Protoonly for a configured trusted peer and only when the value ishttporhttps. 404 → JSONnot_found.MaxRequestSizeis enforced (10 MiB).WithMaxRequestBodyoverrides it, including for custom muxes; zero selects the default, negative disables it.WithTimeoutsapplies shared HTTP defaults (header 10s, read 30s, idle 60s); zero selects defaults, negative disables an individual deadline. Native net/http TLS handshakes use the smaller positive header/read timeout; Handshake is for cmux/eager transport listeners. No write deadline is imposed. A positiveHandlerbounds the context of every request (limits.Timeouts.ApplyHandler, inside correlation), keeping an earlier client deadline; it is off by default, and a handler that ignores its context outlives it.
Tests: rest_test.go suite builds CA/server/client chains with xpki/testca in a temp dir; server_test.go and service_test.go use random ports via tests/testutils and example_test.go binds 127.0.0.1:0; router_test.go CORS and route-template metric labels (Test_RouterRouteMetrics, process-global sink, not parallel); example_test.go uses testdata/test-server*.pem; limits_test.go checks real stalled headers/bodies, idle connections and custom-mux body limits, that a default-mux 413 from DecodeBody carries router CORS and correlation headers, and (TestHandlerTimeout) that Timeouts.Handler bounds the request context of the default and a custom mux and leaves it unbounded when unset; server_test.go TestServer_CORSPreflightAuthz checks that preflights are answered before authz, that OPTIONS is otherwise authorized (CORS disabled or OptionsPassthrough), and that denied responses carry CORS headers.
github.com/effective-security/porto/restserver/authz
Purpose: path-segment/role access control tree for HTTP handlers and gRPC interceptors, configurable from YAML/JSON.
Files: authz.go — Config, Provider (tree build/walk/clone, NewHandler, interceptors, access logging); doc.go.
Entry points: New(*Config) parsing allow entries "/path:role1,role2"; Allow, AllowAny, AllowAnyRole, SetRoleMapper, SetGRPCRoleMapper, Clone; NewHandler(delegate), NewUnaryInterceptor(), NewStreamServerInterceptor(); interfaces HTTPAuthz, GRPCAuthz.
Invariants:
- Paths must start with
/orwalkPathpanics; match is by whole segment on the deepest configured node;AllowAnybeats roles;guestand empty roles are never allowed byAllow/AllowAnyRole. NewHandlersnapshots throughClone; interceptors use the live provider. An empty tree denies everything.- Every method is checked, including
OPTIONSand CORS preflights: the preflight headers are caller-controlled, so a CORS middleware that answers preflights must run outside the authz handler (restserver.NewMuxandgserverdo; custom muxes must too). WithOptionsPassthrough,OPTIONSrequests reach authz and need a permitted role. Denial (denied) ishttperror.Unauthorized(401 /codes.Unauthenticated) for an empty orguestrole andhttperror.Forbidden(403 /codes.PermissionDenied) for any other role; the message is<role> role not allowed(an empty role is reported asguest), the request ID is carried inrequest_id, and the HTTP handler writes the error fromcheckAccessas is (no double wrapping). - Mutators are unsynchronized: configure before serving.
- Config tags:
allow,allow_any,allow_any_role,log_allowed_any,log_allowed,log_denied,log_level,logger_skip_paths[{path,agent}].
Tests: authz_test.go — tree walk, YAML load, handler and interceptor allow/deny with identity.WithTestIdentity.
github.com/effective-security/porto/restserver/ready
Purpose: readiness gate middleware returning JSON 503 not_ready until ServiceStatus.IsReady() is true.
Files: ready.go — ServiceStatus, NewServiceStatusVerifier.
Invariants: IsReady is called on every request and must be cheap and thread-safe. The 503 not_ready error is built per request with httperror.NotReady, so the request_id in the body is always the current request's; nothing is shared between requests.
github.com/effective-security/porto/restserver/telemetry
Purpose: request logging and request metrics middleware plus the ResponseCapture writer.
Files: requestlogger.go (RequestLogger, NewRequestLogger, LoggerSkipPath, ShouldSkip, WithLoggerSkipPaths), request_metrics.go (NewRequestMetrics, UnknownRoute, OtherMethod), route.go (SetRoute, requestLabels: the per-request holder of the route and the role reported by a nested metrics handler), response_capture.go (ResponseCapture, Started).
Invariants:
NewRequestLoggerpanics on a nil handler; a nil logger returns the handler unwrapped; a granularity below 1ns is clamped to 1ns (the duration is logged in nanoseconds).- Log fields: method, path, status, bytes, duration, remote (
identity.IPString(identity.ClientIPFromRequest(r)), the IP cached byidentity.NewTrustedProxyHandlerwhen present, "" for a Unix socket peer), agent (NoAgent,no-agent, without a User-Agent;ShouldSkipmatches the same value); INFO viaContextKV.gserverlogs gRPC calls with the sameremoteandagentnames. - Metric tags: verb, status, uri, role; every request, including 404s, records both
HTTPReqPerfandHTTPReqByRole. No tag value is copied from the request: uri is the route template recorded withSetRoute(therestserverRouter records its registered path, e.g./v1/users/:id) orUnknownRoute(unknown) for unmatched paths and responses produced before or instead of a route handler (identity mapper 401s,gserverCORS preflights and rate-limit 429s, authz 401/403, ready 503,restserverCORS preflights, httprouter's 405, automaticOPTIONSand redirects, andgserverWithMiddlewarehandlers that answer without callingSetRoute); verb is the method when it is one of the 9 standard net/http methods, elseOtherMethod(_OTHER).NewRequestMetricsputs a label holder (requestLabels) in the request context; aNewRequestMetricsthat finds one (nested) records nothing and stores the role of the identity in its request context (identity.FromContext) in it, so each request is recorded once, by the outermost handler.SetRoutewithout a holder is a no-op, the last call wins, and the holder is atomic becausehttp.TimeoutHandlercan leave the handler running after the metrics are recorded. Custom routers behindNewRequestMetricsmust callSetRoutewith the registered pattern, never with a value from the request. The role tag is whatever the identity mapper returned, as reported by the innermost nested handler; without one (or when the request never reached it) it is the role in the outer request's context, guest when none was stored. Both servers place one metrics handler outsideidentity.NewContextHandlerand one inside it, so identity rejections are counted as guest; custom mux factories need the same pair to count them. The outer handler records in adefer, so a handler that panics (or callsruntime.Goexit) is counted with the status it sent, or 500 when it sent none (ResponseCapture.wroteHeader); the panic,http.ErrAbortHandlerincluded, is not recovered and reaches the server with its original stack. LoggerSkipPathtagspath,agent;*wildcard; agent is a substring match.ResponseCapturecaptures the status net/http sends: the first finalWriteHeaderstatus (1xx other than 101, codes outside 100-999, which net/http rejects with a panic, and later calls are ignored), or 200 onceWrite, aReadFromthat copied a byte, or a flush the chain supports sent the header;Started()reports that a final status was sent or the connection was successfully hijacked; informational responses alone do not start a response.ResponseCaptureimplementsUnwrap, sohttp.ResponseController(deadlines, full duplex, flush, hijack) reaches the delegate, andhttp.Flusher,FlushErrorandhttp.Hijackerthroughhttp.ResponseController: when the delegate chain lacks the featureFlushis a no-op, andFlushError(whichResponseController.Flushcalls) andHijackreturn an error wrappinghttp.ErrNotSupported(HTTP/2 cannot hijack). Captured status and size exclude bytes written to a hijacked connection; fields are unsynchronized (one handler goroutine).ReadFrom(io.ReaderFrom) counts the copied bytes and calls the delegate'sReadFromwhen it has one (soio.Copy,http.ServeContentandhttp.FileServerkeep the sendfile path of*http.responsethrough both captures), elseio.Copyinto the delegate; likeWrite, it returns the delegate's error unwrapped.
Tests: request_metrics_test.go (internal; exact label sets through a process-global in-memory sink, so those tests are not parallel; Test_RequestMetricsNested checks the role handoff, including rejections before the nested handler, an empty role and three levels; Test_RequestMetricsTimeoutRace overlaps SetRoute with the metrics read, run with -race; Test_RequestMetricsPanic checks the status recorded for panics before and after a status, body, flush or 1xx, http.ErrAbortHandler and runtime.Goexit, and that the panic value propagates; Test_RequestMetricsPanicServer checks on a real httptest.Server that net/http still logs the panicking handler's stack), requestlogger_test.go (internal; log lines through the global xlog formatter), response_capture_test.go (black box; ResponseController, flush, hijack and http.ServeContent through the logger and metrics chain on a real httptest.Server, ReadFrom delegation and counting, TestResponseCapture_Status for the captured status, and BenchmarkResponseCapture_ServeFile for the sendfile path; serveChain waits for the chain to return, because a hijacked response reaches the client first).
github.com/effective-security/porto/xhttp/correlation
Purpose: correlation/request ID propagation across HTTP, gRPC metadata, contexts and logs.
Files: correlation.go.
Entry points: NewHandler(delegate), NewAuthUnaryInterceptor(), NewStreamServerInterceptor(), ID(ctx), Value(ctx), WithID(ctx), WithMetaFromContext(ctx), WithMetaFromRequest(r), NewFromContext(ctx), Correlator, IDSize, CorrelationIDgRPCHeaderName.
Invariants:
- Incoming ID sources (HTTP):
X-Correlation-ID, thenX-Request-ID; (gRPC)x-correlation-id,x-request-id; truncated to 12 chars; otherwise random. - Writes the response header
X-Correlation-IDand the xlog KVctx. WithMetaFromRequestcopiesx-*,grpc-*,authorization,date,timestampheaders into incoming and outgoing MD (forwards credentials).- gRPC metadata keys are lowercase;
WithMetaFromRequestforwards the resolved (possibly truncated) correlation ID underCorrelationIDgRPCHeaderNameand gives anyx-request-idalias the same value. NewFromContextdetaches from cancellation. gRPC interceptors recover panics intocodes.Internalwith messageunhandled exception.
github.com/effective-security/porto/xhttp/header
Purpose: constants for HTTP header names and content-type values, including Authority (:authority) for gRPC DPoP binding, Vary for response negotiation, XForwardedFor, XRealIP, XForwardedProto for proxy headers, Origin, AccessControlRequestMethod for CORS preflight requests, RetryAfter for 429/503 retries, XCSRFToken for the gserver/roles cookie CSRF check, and XClientHostname, XClientIP (informational, set by pkg/retriable WithUserAgent). GRPCStatus and GRPCMessage name the gRPC status trailers and GRPCTimeout the client deadline header that gserver caps on TLS listeners. No imports.
github.com/effective-security/porto/xhttp/httperror
Purpose: structured API errors with HTTP status, code, gRPC status and correlation ID; JSON writer; HTTP ↔ gRPC code mapping; classification helpers.
Files: errors.go (Error, constructors, Wrap/WrapWithCtx, Is*, Status, WriteHTTPResponse), codes.go (Code*, httpCode, codeStatus, statusCode, HTTPStatusFromRPC, FromOAuth), many.go (ManyError), rpc.go (NewGrpc, NewGrpcFromCtx, NewFromPb, GRPCStatus, CorrelationID; status details and restoredHTTPStatus), encoder.go.
Invariants:
- Wire JSON
{code, message, request_id?};ManyErroraddserrors{}. WithContext/WithCausemutate the receiver.WriteHTTPResponse(bothErrorandManyError) never does: whenRequestIDis empty, the body is written from a per-request copy carrying the correlation ID, so package-levelErrorvalues can be shared across requests.ManyError.WriteHTTPResponsesnapshots the fields and clonesErrorsunder theAddlock (snapshot) and releases it before writing, so a slow client never blocksAddor other writers.errMsg(behindWrap/WrapWithCtx) never panics: a non-string first value with further values is formatted with%+vand space-joined instead of being used as a format. It does not forward the values to afmtprint function, so vet does not treatWrapas a print wrapper.RequestTooLargereturns HTTP 413 withrequest_too_large, sent over gRPC asResourceExhausted.- Mapping:
Unauthenticated↔ 401unauthorized(CodeUnauthorized) andPermissionDenied↔ 403forbidden(CodeForbidden), so 401/403 round-trip through gRPC;Canceled/DeadlineExceeded→ 408;ResourceExhausted→ 429;FailedPrecondition→ 400bad_request(google.rpc.Code;*Error.RPCStatuskeeps the gRPC code);CodeTimeout→DeadlineExceeded;conflict→AlreadyExists;precondition_failed→FailedPrecondition; unknown codes → RPCStatus 0. Error.GRPCStatusandManyError.GRPCStatusattach string details@httperror.code(the Code) and@correlation.id(the RequestID) when set; a status with code OK carries no details.NewFromPb,WrapandStatusconvert a status with the Google mapping, except that a code listed inrestoredHTTPStatus(onlyrequest_too_large→ 413) is restored when its detail matches the gRPC code. Other codes keep the lossy mapping.Wrapclassification is substring-based ("invalid"/"bad"/"400" → 400; "not found"/"404" → 404; "timeout"/"deadline"/"cancel" → 408).Iscompares Code and Message;Unwrapskips one wrapper level of the cause.
github.com/effective-security/porto/xhttp/identity
Purpose: caller identity (role/subject/tenant/claims/auth method) and connection facts (client IP, UA, target) in request contexts for HTTP and gRPC.
Files: identity.go (Identity, NewIdentity, AuthMethod, guest mappers, WithTestIdentity), ctx.go (RequestContext, FromContext/FromRequest/AddToContext, NewContextHandler, NewAuthUnaryInterceptor/NewStreamServerInterceptor), realip.go (TrustedProxies, ParseTrustedProxies, WithTrustedProxies, NewTrustedProxyHandler, ClientIPFromRequest, ClientIPFromGRPC, IPString, ForwardedProto), basicauth.go.
Invariants:
- Never returns a nil identity: guest (
GuestRoleName = "guest",MethodNone) when missing. Mapper error (identityError) → logged, then a generic HTTP 401unauthorizedJSON / gRPCcodes.Unauthenticatedwith messageinvalid identity; the mapper's error text is never sent. A non-nil*httperror.Errorin the mapper error's chain is the mapper's client-safe answer and is returned unmodified when it has an HTTP status (over gRPC only when its code also has a gRPC mapping; otherwise a plainUnauthenticatedstatus with its message); one without an HTTP status, or a typed nil, gets the generic 401. - An existing
RequestContextin ctx is respected (test injection).FromRequestfills in an empty target and user agent and an invalidClientAddrfrom the request, in place, but leaves a context markedresolvedunchanged:NewContextHandlerandcreateIdentityContextset it, because they built the context from the request, so concurrentFromRequestcalls in a served handler never write (TestFromRequestConcurrent, run with-race).WithTestIdentitysets bothclientAddrandclientIPfrom the local IP and is notresolved. - Client IP type:
ClientIPFromRequestandClientIPFromGRPCreturnnetip.Addr. A caller whose peer is not an IP (emptyRemoteAddr, a Unix socket peer@, a gRPC call without a peer) has no client IP: the zeroAddr(IsValidfalse).IPStringformats it as "";netip.Addr.Stringwould return "invalid IP", which as a rate limit key put every such caller in one bucket, so every logremote, thegserverlimiter key andRequestContext.ClientIPgo throughIPString.RequestContextkeeps theAddr(ClientAddr, also on theContextinterface) and itsIPString(ClientIP); every place that sets one sets the other. - Without a trusted proxy policy, client IP is the socket peer and forwarded headers are ignored.
ParseTrustedProxiesvalidates CIDRs;WithTrustedProxiesstores the immutable policy in context (replacing any policy and cached IP);NewTrustedProxyHandlerapplies it to HTTP and resolves the client IP once per request. The context value (forwarding) holds the policy plus the resolved IP and theRemoteAddrit was resolved for;ClientIPFromRequestreturns the cached IP whenRemoteAddrstill matches, so header changes after the trust boundary are ignored and a request copy with another peer is resolved again. A request with an empty or non-IPRemoteAddrhas no client IP and returns the zeroAddr(the pre-B05 fallback tonetutil.GetLocalIPwas removed; before thenetip.Addrchange a Unix socket peer returned@). For trusted peers, XFF is walked right to left to the first untrusted address, including private addresses, or to the leftmost entry when every hop is trusted; malformed values fall back to the peer. IPv4-mapped IPv6 addresses are returned as IPv4, including the socket peer when it is the answer (forwardedClientchecks trust on the mapped form, then unmaps). IPv6 zones (fe80::1%eth0) are ignored when matching trusted CIDRs, becausenetip.Prefix.Containsrejects zoned addresses; returned addresses keep their zone.X-Real-Ipis used only when XFF is absent. gRPC metadata follows the same rules. Proxies must overwrite client supplied forwarding headers.ForwardedProtoaccepts onlyhttporhttpsfrom a trusted peer. - Unary gRPC interceptor recovers panics; the stream one does not. Only the first
trustedargument of the gRPC interceptors is used; nil keeps a policy already in the context. - Rejections are logged at WARNING with the client IP as
remote:identityMapper(HTTP, withtarget) andaccess_denied(gRPC, withmethod).createIdentityContextresolves the client IP once, for both its WARNING line and theRequestContext. The package logger is registered as repogithub.com/effective-security/porto/xhttp, packagecontext(thepkgfield of its lines).
Tests: ctx_test.go (white-box; TestRejectedIdentityLogsRemote installs a process-global JSON log formatter and is not parallel; TestFromRequestClientIP checks ClientIP/ClientAddr for a Unix socket peer and a mapped peer; TestFromRequestConcurrent is the -race check above), identity_test.go (Test_WithTestIdentityDirect checks the local IP survives FromRequest), realip_test.go (trust policy, forwarding headers, Unix socket and mapped peers, cached client IP, IPString), basicauth_test.go.
github.com/effective-security/porto/xhttp/marshal
Purpose: JSON response writing (with errors, gzip, pretty-print) and strict JSON request decoding via ugorji codec.
Files: limits.go (LimitRequestBody, shared body-limit policy), marshal.go (WriteJSON, WritePlainJSON, WriteHTTPResponse, NewRequest, AcceptsGzip), json.go (PrettyPrintSetting, NewEncoder, EncodeBytes, DecodeBytes, Decode, DecodeBody, DecoderHandle).
Invariants:
- First non-nil body wins;
WriteHTTPResponseimplementers write themselves; other errors are converted withhttperror.NewFromPb(500unexpected, or the gRPC status mapping) and, when the result is 5xx, sent with the generichttp.StatusTextmessage while the original error is kept as the cause and logged; 4xx conversions keep their message. Explicithttperrorvalues are written as given. Non-404 errors are logged with the caller's file and line (no recursion throughWriteJSON). - Success: 200,
application/json; exportedAcceptsGzip(http.Header)is shared with gRPC-Web; gzip for payloads of at least 1 KiB whenAccept-Encodingallows gzip with positive quality (a missingqmeans 1, a malformed or out-of-rangeqmeans 0; explicit gzip denial overrides*), pretty when?pp.Vary: Accept-Encodingis merged with existing values; gzip writers are pooled.rmust be non-nil. - Decoding is strict (
ErrorIfNoField), maps →map[string]any.DecodeBodydefaults to a 10 MiB body cap viahttp.MaxBytesReader;LimitRequestBodyoverrides it (zero = default, negative = disabled).LimitRequestBodynever writes a response, so it may wrap CORS and telemetry middleware: a body whose known Content-Length exceeds the limit is replaced byoversizedBody, which returns*http.MaxBytesErroron the first read without reading any bytes, and net/http closes the connection when the unread remainder is too large to drain. The request always reaches the handler. DecodeBody consumes the remaining bounded body after decoding so trailing data cannot bypass the cap; with the limit disabled the remainder is left unread. Overflow returns/writes a structured 413; malformed JSON remains 400. Other body readers must handle*http.MaxBytesError.Decode/DecodeBytesremain unbounded. - Encoder handles are package globals initialised in
init.
Tests: marshal_test.go covers negotiation, the compression threshold, Vary, decompressed payloads, and log line numbers derived from runtime.Caller; marshal_bench_test.go measures compressed responses. limits_test.go covers exact/unknown-length/oversized/trailing bodies, defaults and overrides; a counting reader checks that known oversized bodies are never read.
github.com/effective-security/porto/xhttp/limits
Purpose: shared conservative server timeout and request-body defaults, and the optional handler deadline, without internal dependencies.
Files: doc.go, limits.go (Timeouts, WithDefaults, ApplyHTTP, HandlerContext, ApplyHandler, default constants).
Invariants: header 10s, read 30s, idle 60s, handshake/detection 10s, body 10 MiB. Zero selects defaults; negatives explicitly disable individual limits. Header applies to HTTP/1.x through net/http.ReadHeaderTimeout; net/http has no per-stream HTTP/2 header deadline. HTTP timeout application preserves negative values to avoid net/http fallback deadlines; no response write deadline is set. Call before serving. Native net/http TLS handshakes use the smaller positive header/read deadline; Handshake applies to cmux and eager transport listeners. Handler has no default: zero and negative leave requests bounded only by the client's deadline. It is not a net/http setting, so ApplyHTTP ignores it; HandlerContext bounds a context with context.WithTimeout (an earlier parent deadline is kept) and ApplyHandler wraps an http.Handler with it, returning the handler itself when Handler is not positive. Servers (gserver, restserver, pkg/appinit Prometheus) call ApplyHandler on their chains; gserver uses HandlerContext in its gRPC interceptors.
Tests: limits_test.go covers defaults, overrides, explicit disabling, write-timeout preservation, HandlerContext (unset, no parent deadline, earlier and later parent deadlines, expiry) and ApplyHandler (unset returns the handler itself).
github.com/effective-security/porto/pkg/retriable
Purpose: HTTP client with retry policy, JSON marshalling, context header propagation, Bearer/DPoP authorization, replay nonces and file-based token/key storage. See also pkg/retriable/README.md.
Files: doc.go, retriable.go (Client, options, Policy, Request/Do, DecodeResponse, context header helpers), client.go (Head/Get/Post/Put/Delete), config.go (Config/ClientConfig/RequestPolicy/TLSInfo, Factory, LoadClient, SetAuthorization, HostFolderName), storage.go (Storage, AuthToken, ParseAuthToken, KeyInfo), nonce.go (NonceProvider), request.go (Request, Requestor), dumpreq.go (DumpRequestOut).
Entry points: New(cfg, opts...), Default(host), LoadClient(file), NewFactory/LoadFactory → CreateClient/ForHost; Client.Request/RequestURL/HeadTo, Get/Post/Put/Delete/Head, Do; SetAuthorization(); Policy, DefaultPolicy(); WithHeaders(ctx, map), PropagateHeadersFromRequest; Storage.*.
Invariants:
- Errors, not panics.
WithTLS/WithDNSServerinstall a modifiedCloneof the current*http.Transport(ofhttp.DefaultTransport, pooled to 100 connections per host, when none is set) and never modify a supplied transport; on anotherhttp.RoundTripper(or a non-*http.Transporthttp.DefaultTransport) they recordconfigErr, whichNewandDoreturn (closing the request body) untilWithTransportreplaces the transport (fail closed).replaceTransportcloses the idle connections of a replaced transport the client created (ownTransport), never of one passed toWithTransport; connections in use at that moment close when idle on HTTP/1.1, otherwise after the transport'sIdleConnTimeout(90s forhttp.DefaultTransportclones).NewappliesClientConfig.TLSwithWithTLSafter the options, to the transport they leave, unlesstlsSetis still true (set byWithTLS, cleared byWithTransport), so only aWithTLSoption that no laterWithTransportreplaced wins; on a non-*http.TransportfromWithTransportNewfails closed.WithTransporttreats a nil*http.Transportas nil (http.DefaultTransport). - DEBUG request dumps redact
Authorization,DPoP,Cookie, andProxy-Authorizationwithout mutating the request;Client.Doomits request bodies from its DEBUG dump. - Client setters lock mutable fields;
Dosnapshots headers, policy, name, signer, and a copy of the*http.Client(so transport setters are safe during requests). Caller-identity refresh (callerToken,leadRefresh,finishRefresh) is shared by concurrent requests, with context-aware waiting and a fast path for valid tokens or no provider; changing the provider invalidates an in-flight refresh. A panicking provider releases the waiters withcaller identity provider panickedand the panic continues on the request that made the call; the next request calls the provider again. Direct mutation of exportedName,Policy,Config, or the returnedHTTPClient()still requires caller synchronization. ShouldRetryorder: ctx done →TotalRetryLimit→NonRetriableErrors→Retries[0]; status < 400 succeeds; 404 stops; 429 →Retries[429], elseLimitExceeded; other 4xx stop; 5xx →Retries[status].DefaultShouldRetryFactory(limit, ...)retries whileretries <= limit(limit+1 retries).DefaultPolicy: total 5; connection errors 4x/2s; 502/503 5x/1s; 429 3x afterRetry-After(1s without it; > 30s stops withLimitExceeded); no timeout.RetryAfterShouldRetryFactoryparses delay-seconds or an HTTP date (past → 0, overflow saturates). Arequest:config block (RequestPolicy) replaces only non-zero values (cmp.Or); negativeretry_limitdisables retries.Dowaits between attempts withwaitRetry(timer +ctx.Done()), returning"<METHOD> <redacted URL>: waiting to retry: <ctx err>"; while ctx is live, a retry whose wait would pass its deadline is skipped and the last response returned unread (after the deadline, the ctx error is returned). Retried bodies are drained up to 64 KiB (maxErrorBodySize) and closed;DecodeResponsereads at most 64 KiB of an error body (and returns read errors); DEBUG dumps log headers plus that bounded body.RequestTimeoutapplies toRequestand helpers, notDo;executeRequestwraps the response body incancelOnClose, so closing the body (asRequest/HeadTodo) releases the timeout context, and a failed request releases it at once.RequestURLderives host (re-escaped, for IPv6 zones) andRequestURI()(+ fragment) from the parsed URL; it rejects URLs without a scheme, opaque URLs and URLs with user info.NonceProvider.NonceContext(ctx)bounds a nonce fetch by ctx;Nonce()(jose.NonceSource) returns a cached nonce or fetches with a 30s timeout (nonceFetchTimeout). The LIFO cache holds 64 nonces; when full, the oldest half is dropped.WithUserAgentreadsos.Hostnameandnetutil.GetLocalIPonce (no network wait;localIPis a test seam) and omits values it cannot resolve.- Headers written:
X-Correlation-ID,Authorization,DPoP,User-Agent,X-CLIENT-HOSTNAME,X-CLIENT-IP; context headers override client headers. - Authorization only for
https:///unixs://hosts; token formats: opaque oraccess_token=&exp=&dpop_jkt=(optionalrefresh_token, aliasesid_token/token).ParseAuthTokenalways sets TokenType to Bearer and ignorestoken_type; clients select DPoP wheneverdpop_jktis present. - DPoP proofs are signed separately for each retry attempt; a DPoP authorization header without a configured signer returns an error.
- Config YAML:
host, tls{cert,key,trusted_ca}, request{retry_limit,timeout}, storage_folder;~and$VARare expanded;LoadFactoryappendsHostFolderName(host)tostorage_folder. NewStorageexpands a leading~(alone or before a separator) withos.UserHomeDir, thenuser.Current().HomeDir, falling back to the literal folder; no~useror$VARexpansion (config loaders expand$VARwithresolve.ExpandPathfirst).- Storage files:
.auth_tokenand<thumbprint>.jwkare written through mode-0600 temporary files and atomically replace existing files or symlinks. A newly created credentials folder uses mode 0700; existing directory permissions stay intact. An emptyStorageFolderuses the working directory without changing its mode. A public-only DPoP key is rejected bySetAuthorization.ListKeysskips unreadable files and subdirectories without losing later siblings; a missing or unreadable root produces an empty list with nil error.Cleanintentionally ignores removal errors.
Tests: retriable_test.go (httptest servers; TestMain writes a test CA/client cert under $TMPDIR/test-retriable), policy_test.go (config defaults, 429/Retry-After, ctx-aware backoff, body draining/bounds, timeout release, RequestURL, transport cloning, ClientConfig.TLS precedence over WithTransport/WithTLS against an httptest TLS server (TestConfigTLSWithTransport), idle-connection release (server ConnState counting) and fail-closed errors, a setter/request race; TestTransportSettersWithoutHTTPDefaultTransport swaps http.DefaultTransport and must stay non-parallel), safety_test.go (credential redaction, concurrent refresh, a panicking provider, a provider replaced during its refresh (normal return and panic), retry signing, storage permissions, ~ expansion with t.Setenv), config_test.go with testdata/*.yaml, nonce_test.go (cache order, fetch context via a fake HTTPClient), internal_test.go (needs outbound DNS to 8.8.8.8; the localIP seam test is non-parallel), storage_test.go (NewKeyInfo algorithms, ListKeys filtering and unreadable subdirectory/root handling, ParseAuthToken, write failures with exact errors and no temporary files left; permission tests skip as root), clientconfig_test.go (token from env versus file, NewForHost, LoadClient, SetAuthorization; t.Setenv tests non-parallel), request_test.go (NewRequest and body kinds, errors before sending, nil contexts, ShouldRetry on transport errors) and dumpreq_test.go (DumpRequestOut body handling and errors).
github.com/effective-security/porto/pkg/rpcclient
Purpose: build a *grpc.ClientConn plus default call options from a Config (TLS bundle, Bearer/DPoP per-RPC credentials, keepalive, message limits, optional blocking dial).
Files: doc.go, client.go (Client, New/NewFromURL, newClient, dial), config.go (Config, token loading, Storage).
Invariants:
- HTTP(S) scheme prefixes are stripped and
:443is appended when no host port is given.unixs:///pathis converted to gRPC'sunix:///pathresolver target;unix:///pathis preserved. - When
cfg.TLS != nil, the endpoint must start withhttps://orunixs://;Newotherwise returns an error. A nil TLS config usesinsecure.NewCredentials(). - Defaults:
WaitForReady(true), send limit 10 MiB, recv limitmath.MaxInt32;Config.CallOptionsreplaces them. DialTimeout > 0waits forconnectivity.Ready; otherwise lazy. Usesgrpc.NewClient. Dial cancellation and timeout errors wrap the context error. Stored public-only DPoP keys return an error; failed construction cancels its context.Confighas no yaml/json tags.
Tests: server_test.go starts an in-process gRPC health server on loopback (plain, or TLS with the httptest certificate) that records the metadata of the last call; credentials_test.go (Bearer and DPoP metadata, proofs signed by the stored key, New rejections), dial_test.go (target normalization on lazy clients, a canceled dial, dial timeout, call options and size limits, Close) and config_test.go (token from env versus file, Storage/SetStorage).
github.com/effective-security/porto/pkg/redisclient
Purpose: prefixed, JSON-marshalling wrapper over go-redis v9 with helpers for lists/sets/hashes/zsets, bounded collections, an owner-bound distributed lock and a per-window rate limiter.
Files: doc.go, redisclient.go (Config, Provider/DistributedLock/RateLimiter, RedisClient, parseURL, escapeGlob, setNX, the Lua scripts boundedSetScript/boundedHashScript/releaseLockScript), marshal.go.
Entry points: New(*Config) (root client owning the connection), NewWithClient, NewRedisClient; WithPrefix, Key, SubKey; Get/Set/Del, L*, S*, SAddWithEviction, H*, HSetWithEviction, Z*, ScanKeys/Keys; TryLock/ReleaseLock/IsLocked (lock:<key>), TryAcquireRateLimit/GetRateLimitRemainingTime (ratelimit:<key>).
Invariants:
Get/HGetwrapErrNotFoundwith a stack;IsNotFoundErroruses onlyerrors.Is. Sorted-set errors include the key and wrap the go-redis error;SIsMemberformats any member with%v.UnmarshalStringCmdchecks the command error before changing any destination, including*string.- Prefix is
/<trimmed, path-cleaned>/;Keyjoins it with the key cleaned as a rooted path (path.Join(prefix, path.Clean("/"+key))), so..cannot leave the prefix and keys whose..segments stay below their own first segment keep the layout of the earlier plainpath.Join; without a prefix the key is unchanged.SubKeyinvertsKey(the prefix itself maps to "").Keys/ScanKeysclean the glob pattern like a key, escape the prefix's glob metacharacters (pattern,globPrefix) and return keys relative to the prefix. Prefixes do not nest. - Only the root client's
Closecloses the connection; it returns the close error, is idempotent (redis.ErrClosedfrom a second close is ignored) and keeps the embedded client, so later commands on the root and itsWithPrefixchildren fail withredis.ErrClosedinstead of panicking. - Config YAML:
server, ttl, client_tls{cert, key, trusted_ca}, user, password;passwordoverrides URL credentials.NewRedisClientlogs only the parsed address and DB at INFO;parseURLdrops the URL from a*url.Error(it may embed a password) and keeps the reason. - Lock:
TryLockstores acrypto/rand.Texttoken withSET NX PXand readsPTTLin the same MULTI/EXEC (setNX); it returns the token or "" plus the holder's remaining TTL (at least 1ms, 0 for a key without expiry).ReleaseLock(key, token)is a Lua compare-and-delete; an empty token is an error. Locks written by earlier versions hold a timestamp and are never released by a token (they expire, except zero-timeout ones, which need a manualDEL). Lock and rate-limit keys go throughcoordKey, which, under a prefix, rejects a key whose..segments leave thelock:/ratelimit:sub-namespace (without a prefix keys are not cleaned). - Rate limiter:
TryAcquireRateLimitisSET NX PXonratelimit:<key>viasetNX: one call per window wins, denied calls write nothing, the window runs on the server clock, and an existing key of any type (the earlier sorted set) denies until it expires. An earlier version polling a new window getsWRONGTYPEbut its pipelinedEXPIREresets the window (mixed versions fail closed).GetRateLimitRemainingTimeisPTTLthroughremainingTTL: a live key reports at least 1ms (PTTL says 0 during its last millisecond), a missing key or one without expiry 0; the lock's remaining time uses the same conversion. Lock timeouts and windows below 1ms (minExpiry) are errors. - Eviction helpers run one Lua script each:
LLENandSADD/HSETfirst (a wrong-type key fails before any write), append to the order list only when the member or field is new, afterLREMof its stale entries (O(list length); re-adds keep their position), then pop and remove until the list holds at most the limit. The limit must be positive and the two keys must differ (compared afterKey). A duplicate written by an earlier version for a member still in the set can evict it once. - Scripts run through go-redis
Script.Run(EVALSHA with EVAL fallback), so the server must allow Lua scripting. KeysusesKEYS(tests only);ScanKeysusesSCANwith COUNT 100 (scanBatch), limit <= 0 means 1000 (defaultScanLimit), and may return more than limit.
Tests: testcontainers redis:8.2 with password redis; needs Docker. Test_Redis covers key cleaning and glob-escaped prefixes (KeyNamespace), bounded collections (SetWithEvictionReAdd, SetWithEvictionStaleEntries, *Errors checking that a wrong-type key writes nothing, HashWithEvictionUpdate), owner-bound locks (DistributedLock, LockExpiredOwner, LockArguments, ConcurrentLocking, CoordinationKeyNamespace) and the rate limiter (RateLimiterPolling with concurrent pollers asserting from each allowed call's client-clock span that windows start at least one window apart, RateLimiterLegacyWindow, ConcurrentRateLimiting). ttl_test.go (internal) covers remainingTTL. TestClose and TestConfigSecrets (log capture with xlog.InstallFormatter) need no Docker: clients connect lazily to an unreachable address; client_errors_test.go adds the other Docker-free tests (every Provider method on a closed client returns redis.ErrClosed, NewRedisClient credentials and TLS, Marshal/UnmarshalStringCmd tables).
github.com/effective-security/porto/pkg/cache
Purpose: Provider cache abstraction (Set/Get with TTL, Delete, paged ListKeys and the deprecated Keys, pub/sub) with memory, Redis and prefixing proxy backends.
Files: doc.go, cache.go (Provider, Subscription, Config, RedisConfig, DefaultTTL, KeepTTL, NowFunc, GetOrSet, ErrNotFound, ErrClosed, subscriberBufferSize, defaultListCount, failedSub, closedSub), keys.go (namespace key/pattern/rel mapping, escapeGlob, globMatch port of the Redis matcher), memory.go (memProv with its mutex-guarded subs registry and closed flag, msub, listedKey, pageHeap), redis.go (redisProv with the same registry, rsub, confirmSubscription, closePubSub, subscribeTimeout), proxy.go, export_test.go (test-only SubscriberBufferSize, DefaultListCount), example_test.go (ExampleProvider_listKeys; go/doc attaches ExampleT_suffix to an interface type only with a lowercase suffix, never as a method example).
Invariants:
- Errors, not panics;
Getrequires a non-nil pointer; memory and Redis misses wrapErrNotFoundwith a stack.IsNotFoundErroruses onlyerrors.Is; unrelated "not found" messages are not misses andGetOrSetreturns them without calling the getter or writing. - TTL 0 →
DefaultTTL(memory, 30m, package var) orRedisConfig.TTL(default 1h);KeepTTL(-1) → no expiry. - Memory provider is unbounded; expiry enforced on
GetandCleanExpired;NowFuncis process-global. - Keys: every provider maps keys through
namespace(keys.go): the stored name ispath.Join(prefix, path.Clean("/"+key)), so..cannot leave the prefix and in-prefix keys keep the earlierpath.Joinlayout; an empty memory or Redis prefix becomes "/". A proxy cleans its own prefix as a rooted path without the leading slash ("/a/"and"../a"become"a"), prefixes each key and lets the parent clean it again.Keysbuilds a Redis glob from the escaped prefix and the pattern cleaned like a key (namespace.pattern), and returns names relative to the provider without a leading slash (namespace.rel, "" for the prefix itself);namespace.listedskips names that are not the stored name of their relative key (written by other clients, or outside a prefix that cleans to.) and lists the key "" only for an empty pattern (*would match it under/). A proxy strips its prefix from its parent's result.path.Cleandoes not know glob syntax, so..or//inside[...]is cleaned too. The memory provider matches withglobMatch, a byte-wise port of Redisstringmatchlen(*and?match/, sets, ranges, escapes, the skip-longer-matches bound and the 1000-level nesting limit;stringmatchlen's*does not match the empty string, while KEYS special-cases a lone*, which a namespaced glob is only under a prefix cleaning to., wherelisteddrops what it adds; range bounds compare unsigned bytes, so ranges mixing bytes >= 0x80 with ASCII can differ from Redis builds with signedchar), skips expired entries and returns a non-nil slice. ListKeys(ctx, pattern, cursor, count)lists one page with the same matching, namespacing andlistedfiltering asKeys, and returns the next cursor ("" at the end; start with ""); count <= 0 meansdefaultListCount(100). Redis runs oneSCANper call (cursor: the decimal SCAN cursor,strconv.ParseUint, a bad one fails withinvalid cursor: ...; COUNT count, so a page may be empty with a non-empty cursor, may exceed count, and may repeat a key; a key present for the whole iteration is listed). The memory provider returnsctx.Err()(with a stack) when ctx is already done, then walks the wholesync.Mapper call (Range) and keeps the count+1 smallest live matches whose stored name sorts after the cursor in a bounded max-heap (pageHeapoflistedKey; a name at or above the heap's largest is skipped before the glob match once the heap is full; the bound is checked aslen-1 > count, somath.MaxIntdoes not overflow), sorts those and returns at most count: the cursor is the stored name of the last key returned (never empty, so "" starts the iteration), the page is full unless it is the last, and each key is listed once per iteration even under concurrent writes. A full iteration walks the store once per page (100k entries, count 1000: about 1s;Keysabout 60ms). A proxy passes the cursor and count through unchanged and strips its prefix. Cursors are opaque and not portable between providers.Keysis deprecated (Deprecated:onProvider.Keys) and still runsKEYS; nothing outside tests in this module calls it.GetOrSetwrites a successful getter result with TTL 0 (the provider default), storing the dereferenced value so Redis string/byte reads remain compatible; a failed write leaves the destination unchanged and returns an error. Cache misses require a pointer to a concrete destination type because interface values do not round-trip reliably across providers. Concurrent misses may call the getter more than once.- Pub/sub channels are not prefixed and delivery is at most once.
Publishnever waits for a subscriber: each subscriber bufferssubscriberBufferSize(100) undelivered messages; the memory provider drops the message for a subscriber whose buffer is full, and Redis subscriptions read through go-redisPubSub.Channel(one reader and one health-check goroutine per subscription, automatic reconnect and resubscribe, a message is dropped after its reader has waited one minute on a full channel).ReceiveMessagestarts no goroutine and returns promptly withctx.Err()or, afterClose,ErrClosed;Closeis idempotent. RedisSubscribewaits for the server's subscription confirmation (bounded by ctx; without it, connecting is bounded by the go-redisDialTimeout, attempted at most twice, and the confirmation read bysubscribeTimeout, 10s) so a publish afterSubscribereturns is delivered; a subscription that could not be established is returned as afailedSubwhoseReceiveMessagereturns the wrapped error and whoseCloseis a no-op. go-redis does not watchctx.Donewhile reading, soconfirmSubscriptionruns the read on a goroutine and closes thePubSubon cancellation to unblock it, reportingctx.Err()(orcontext.DeadlineExceededwhen the socket deadline derived from ctx fired first) with the go-redis error attached as a secondary error. RedisReceiveMessagedoes not report connection loss: go-redis reconnects and resubscribes in the background and messages published meanwhile are lost;redisProv.Closefirst closes every live subscription (tracked inredisProv.subs, a map of*rsubguarded byredisProv.mutogether with theclosedflag;rsub.Closeunregisters itself), then the client, so pending receives returnErrClosedand queued messages are discarded.Subscribechecksclosedbefore the network round trip and registers under the lock afterPubSub.Channelstarted its goroutines; whenClosewon the race the subscription is closed on the spot andclosedSub, afailedSubreportingErrClosed("provider closed"), is returned, so no go-redis health-check goroutine can outliveClose.Closealso closes the connection of aSubscribestill waiting for its confirmation (go-redis closes the pub/sub connections it tracks with the client), so a failed confirmation re-checksclosedand returnsclosedSubwith the go-redis error ("use of closed network connection") as a secondary error. go-redis tracks a pub/sub connection only once its setup (HELLO, CLIENT SETINFO) is done, soSubscribealso re-checksclosedright afterclient.Subscribereturns and closes that connection itself; only a redial inside the confirmation read, whenclient.Subscribeleft no connection (its dial or setup failed, or the SUBSCRIBE write and the reconnect failed), can still wait for its confirmation afterClose(until ctx is done orsubscribeTimeout), and it then returnsclosedSubtoo.memProvuses the same scheme (memProv.muguards itssubsmap of*msubandclosed;Publishdelivers under the read lock), so a memorySubscribeduring or afterClosealso returnsclosedSub; the in-process store stays usable afterClose. If the Redis client is closed some other way, the go-redis reader closes the message channel andReceiveMessagereturnsErrClosedafter closing thePubSubitself, which stops its health-check goroutine (onlyPubSub.Closedoes).rsub.Closealso starts a goroutine that drains the go-redis channel until the reader closes it: the reader's send into a full channel only watches its one-minute timeout, so without the drain a slow subscriber's reader survivedClosefor up to a minute.rsub.Closemay wait for an in-flight go-redis redial (DialTimeout) during an outage. Both subscribers check adonechannel closed byClose(viasync.Once) before reading and again after a message was selected, so buffered messages are discarded afterCloseon both providers and a delivery that racesClosereturnsErrClosed; the memory subscriber channel is not closed (Closesignalsdone, andPublish, which delivers undermemProv.mu, stops sending oncemsub.Closeunregistered it). MemoryPublishreturnsctx.Err()when ctx is already done. - Config YAML:
provider: redis|memory,redis{...}; no factory readsprovider.NewRedisProviderdrops the URL from a*url.Errorso a malformed URL's password is not echoed.
Tests: keys_test.go (internal) covers namespace, escapeGlob and globMatch tables and backtracking bounds; TestProvider/keys parity stores the same keys through Redis and memory providers (and proxies over them) under a prefix with glob metacharacters beside a sibling it must not match, and compares Keys for ~45 patterns against the real server, and ListKeys paged with counts 1, 3 and the default against Keys (Redis results deduplicated; memory pages full except the last, in key order, without repeats; helpers listPages, flatten, unique); redis namespace and memory namespace check that .. keys and proxy prefixes stay inside their namespace; redis dot prefix checks that Keys and ListKeys under a prefix cleaning to . list no name Get cannot read, and TestMemProv_KeysRoot the key "" under "/" and a named prefix for both; redis list keys (in Redis DB 1, alone: SCAN COUNT bounds the keys a call looks at, matching or not, so page sizes depend on what else is stored) checks that one ListKeys call is one SCAN (250 keys need several pages at counts 10, 0 and -1, the first page holds at least count or DefaultListCount keys, one page at 1000), invalid cursors (also through a proxy) and the error of a closed client; cache_test.go runs the same matrix against redis (testcontainers), memory and proxy(memory), including exact relative Keys names, read-through persistence and pubsubTest (prompt cancel and close, no message stolen by an abandoned receive, broadcast to two subscribers, and a go-redis goroutine-leak check that counts (*channel).init* stacks); receives run in goroutines that report over channels, never require off the test goroutine. TestRedisSubscribeUnreachable and TestRedisSubscribeContext (a fake RESP server that never confirms SUBSCRIBE, to test ctx cancellation and deadline during the confirmation wait) need no Docker; TestProvider/redis provider closed and memory provider closed cover ErrClosed for a pending receive and a queued message plus goroutine release after the provider is closed, redis subscribe during close races eight Subscribe calls against Close (every result reports ErrClosed, none live, no goroutine left) and checks the rejection after Close, TestRedisSubscribeDuringClose (no Docker: the fake server's onCommand hook signals the SUBSCRIBE it swallowed) closes the provider during the confirmation wait and expects ErrClosed, TestRedisSubscribeCloseDuringSetup (the hook holds the HELLO reply) closes it while the connection is set up and expects ErrClosed without the confirmation wait, and TestProvider/redis slow subscriber closed overflows the buffer, waits until the go-redis reader is blocked in its send select (readerBlockedOnSend, stack inspection) and asserts the reader exits promptly after Close. export_test.go exposes subscriberBufferSize as SubscriberBufferSize to the black-box tests. get_or_set_test.go covers default TTL, hit reuse, type errors, and write failures; memory_test.go covers CleanExpired (and Keys/ListKeys skipping an expired entry before it is cleaned), TestMemProv_ListKeys (exact pages and stored-name cursors, a full last page, keys added and deleted between pages, the default count, math.MaxInt, a proxy, a canceled ctx), TestMemProv_ListKeysPageBound (the page bound on 64 fresh providers, because sync.Map visit order is random per map), TestMemProv_ListKeysConcurrent (pages while writers churn keys around 200 stable ones: each listed exactly once; run with -race), the slow-subscriber drop, Publish with a done ctx, a publish/receive/close race (run with -race) and TestMemProv_SubscribeDuringClose (Subscribe and Publish racing provider Close: every result reports ErrClosed, and a later Subscribe is rejected).
github.com/effective-security/porto/pkg/revocation
Purpose: JWT revocation list. New returns a jwt.Revocation that stores revoked token IDs in a cache.Provider and checks them from Validate. Distinct from certificate revocation (pkg/crlcache).
Files: doc.go, revocaction.go (New, provider, TimeNowFn, cacheKey).
Entry points: New(cache.Provider) jwt.Revocation — Validate(ctx, token, claims) and Revoke(ctx, token, claims).
Invariants:
- Returns errors; does not panic.
Validaterefreshes the snapshot (load), then rejects an expiredexp(token expired at: ...) before a revoked ID (token revoked). A missingexpis not treated as expired. - Cache key (
cacheKey):/revoked/<sub>/<id>.idis thejticlaim, orcertutil.SHA1Base64of the raw token whenjtiis empty.reloadpages through/revoked/*withListKeys(countreloadPageSize, 1000, per call) until the cursor is empty and keeps the segment after the last/; an error on any page fails the reload without publishing; a key a Redis SCAN repeats lands in the map once, and a key added during the scan may be missed (seelocal). revokedis a published snapshot.rememberandreloadreplace the map; they never write an entry into the live map (a concurrentValidatewould crash on a map read/write).snapshotTTLis 5 minutes. One in-flight scan (inflight) serves everyValidatethat arrives while the snapshot is stale; waiters share that scan's error. A failed scan is not kept: the nextValidatetries again.localis the set of IDs this process has stored.reloadmerges it into the new snapshot, so a scan that started beforeRevokewrote the key cannot drop the ID. An ID present in the scan is removed fromlocal.Revokewithexpalready in the past returns nil and stores nothing. A missingexpis stored for 24 hours. Otherwise the cache value isexpin RFC3339 and the TTL isexp - now.Revokethenremembers the ID on the local snapshot.- Process-global: package logger;
TimeNowFn(tests override the clock).
Tests: revocation_test.go (same package; it reads provider, snapshotTTL and TimeNowFn). Memory provider only, no Docker. TestRevokeReplacesSnapshot checks that a revoke publishes a new map. TestConcurrentValidateAndRevoke races revoke and validate. TestReloadPages stores 2*reloadPageSize+1 revoked IDs through the cache and expects one scan of three pages to find them all; TestReloadPageErrorKeepsSnapshot fails the second page (gatingCache.pageErr) and expects the error, an unchanged snapshot and loadedAt, and a retry on the next Validate. TestReloadKeepsRevokeStartedBeforeStore, TestConcurrentValidateSharesReload and TestReloadErrorIsSharedAndRetried run in testing/synctest behind a gating ListKeys (it counts scans, i.e. calls from the empty cursor, and pages, and holds the first page) so a scan can be held open.
github.com/effective-security/porto/pkg/location
Purpose: geographic location (city, region, country) of a client IP, for example to show where a sign-in came from. Independent of xhttp/identity: callers pass the netip.Addr from identity.ClientIPFromRequest or RequestContext.ClientAddr.
Files: doc.go, location.go (Provider, Place and Place.String, IsPublic, ParsePublicIP, parseHostAddr, canonical, globalUnicastIPv6, nonPublicPrefixes), ipwhois.go (IPWhoIs, IPWhoIsConfig, NewIPWhoIs, IPLocation, ErrQuotaExceeded, DefaultIPWhoIsURL, DefaultTimeout, DefaultUserAgent, noLocationMessages, retryAfter, quotaUntil, backOff, displayURL, drainBody), cache.go (Cache, CacheConfig, NewCache, DefaultCacheTTL, NoExpiry, DefaultIPv4Bits, DefaultIPv6Bits, lookup, resolve), export_test.go (test-only RetryAfter).
Entry points: NewIPWhoIs(IPWhoIsConfig) (*IPWhoIs, error); NewCache(Provider, CacheConfig) (*Cache, error); Provider.Location(ctx, netip.Addr) (*Place, error); IPLocation(ctx, baseURL, addr) *Place for a one-off lookup.
Invariants:
Provider.Locationcontract: a*Placefor a known address;nil, nilfor no location (the address is not public, or ipwho.is answered"success": falsewith "Invalid IP address" or "Reserved range",noLocationMessages, compared without case); an error for a failed lookup (network, timeout, non-200 status, any other"success": falsemessage, undecodable or oversized answer), which may succeed later. Constructors return errors, never panic.- Addresses are canonicalized (
canonical: IPv4-mapped IPv6 as IPv4, zone dropped) before the public check, the lookup and the cache key.IsPublicrefuses non-global-unicast and private (RFC 1918, RFC 4193) addresses, IPv6 outside2000::/3(globalUnicastIPv6: NAT64, SIIT and IPv4-compatible embeddings, site-local, discard-only) and the special-purposenonPublicPrefixes(0.0.0.0/8, CGNAT100.64.0.0/10,192.0.0.0/24,192.88.99.0/24, documentation and benchmarking ranges,240.0.0.0/4,2001::/23,2001:db8::/32, 6to42002::/16,3fff::/20); such addresses are never sent to the service or cached.ParsePublicIPaccepts an address,[IPv6],IPv4:portor[IPv6]:port(netip.ParseAddr, thenparseHostAddrwithnetip.ParseAddrPortrules); unbalanced brackets,[IPv4]and a malformed port are rejected. IPv4 embedded in 6to4, NAT64 or Teredo addresses is not unwrapped, so those clients have no location. IPWhoIssendsGET <base>/<ip>?fields=success,message,city,region,country,country_codewithAccept: application/jsonand the configured User-Agent. Other query parameters ofBaseURL(an API key) are kept;fieldsis replaced. A transport orurl.Parseerror is unwrapped from*url.Error, andNewIPWhoIserrors show the base URL without query, fragment and password (displayURL), so errors and logs never repeat the query. Onehttp.Clientper provider (onhttp.DefaultTransport) withTimeout(default 3s) and no redirects (a 3xx is a status error); the answer is capped at 4096 bytes (ipWhoIsMaxBody), and every response, including a status error or an oversized answer, is drained up to that size before it is closed (drainBody, best effort: a remainder of at most 4096 decompressed bytes that arrives within the clientTimeoutand ctx), so its connection is idle again whenLocationreturns; a status error with a slow body therefore waits for it, up to those bounds. The transport drains a body closed early only in the background, and a following lookup would often dial. A no-location answer is logged at DEBUG (reason=no_location,ip,message).- Quota: HTTP 429 returns an error wrapping
ErrQuotaExceededand setsretryAt(backOff, guarded byIPWhoIs.mu; it only moves later) fromRetry-After(retryAfter: seconds or HTTP date, capped at 24h; seconds are clamped before they are converted, so no value overflows; missing, invalid (including a number beyond int64) or non-positive waits 1 minute). Until then every lookup fails fast withErrQuotaExceeded("skipped until ") without a request (quotaUntil). ipwho.is documents 429 withRetry-Afterfor its free per-IP daily limit. IPLocationintentionally swallows errors: it logs them at WARNING (reason=ip_location,ip,err) and returns nil. It returns nil without a lookup or log for an address that is not public. It builds a newIPWhoIsper call, so it has no cache and no 429 backoff.Cachekeys answers by network:addr.Prefix(IPv4Bits)(default /24) oraddr.Prefix(IPv6Bits)(default /48); the first address of a network is the one looked up. It is a size-boundedgolang-lru/v2LRU (lru.Cache, which locks itself, unlikesimplelru;Cache.muguards onlyinflight; no background goroutine;Cachestarts none); TTL is checked on read (get; default 24h,NoExpirykeeps entries until evicted,time.Now, sotesting/synctestcontrols it).getPeeks first and promotes (Get) only a live entry, so an expired entry whose refresh fails is evicted before live ones. Places andnil(no location) answers are kept; errors are not, so the next call retries. Each call returns its own copy of thePlace(clone), and the stored entry is a copy of what the provider returned.- Concurrent misses for one network share one lookup (
inflight, guarded byCache.mu), run on the ctx of the caller that started it; no lookup is detached from its caller, so callers must give a ctx that allows for the provider's latency: a lookup cut short by ctx is an error and is not cached, and with deadlines shorter than the provider the cache never fills. Later callers wait for it or for their own ctx (ctx.Err()wrapped) and share its result; when it failed with a ctx error (context.Canceled/DeadlineExceededin the chain) after its caller's ctx was done (lookup.canceled), a waiter whose ctx is live loops and looks the network up again, one lookup at a time. Any other error is shared, even if the first caller's ctx was done by then.resolvestores the answer before it leavesinflight, and a caller re-checks the LRU undermu, so a finished lookup is not repeated. A panicking provider still releases the waiters (resolve's deferred cleanup; they get "location lookup of did not complete") and the panic propagates to the first caller. - Process-global: package logger (
github.com/effective-security/porto/pkg,location).
Tests: black box (package location_test; export_test.go exposes retryAfter), no Docker or network. location_test.go (Place.String, ParsePublicIP and IsPublic tables, including IPv6 special-purpose ranges); ipwhois_test.go (httptest servers: request path, query (fields replaced, key kept), headers, IPv6 without zone, JSON nulls, body limit, known and unknown success:false messages, status/redirect/decode errors, caller deadline, client timeout, unreachable without the URL query in the error; TestIPWhoIsQuota (real 1s Retry-After, waited from when the 429 lookup returned), TestRetryAfter table; TestIPWhoIsReusesConnection makes three lookups per answer kind and asserts the exact error, three requests and one server connection; the first answer holds back its last bytes for 150ms (slowTail, longer than the transport's 50ms background drain), so a missing drainBody fails without -race; it is not parallel, because httptest.Server.Close closes the idle connections of http.DefaultTransport; lookupServer answers a 302 with Location: /moved, which answers a place, so following a redirect fails the redirect rows; TestIPLocationLogsErrors installs a process-global JSON log formatter and is not parallel); cache_test.go (per-network sharing and custom prefix lengths, no-location kept, errors retried, copies, LRU eviction, expired entries not promoted and TTL in testing/synctest, shared lookups including waiter cancellation, a first caller that gives up (retried) or whose lookup fails on its own (shared), and provider panic in synctest, and TestCacheConcurrent, which races 16 goroutines over a cache smaller than the working set; run with -race); example_test.go (ExampleNewCache, run against an httptest server).
github.com/effective-security/porto/pkg/tasks
Purpose: in-process cron-like scheduler; runs functions on interval/daily/weekly schedules parsed from a human-friendly string.
Files: doc.go, task.go (Task, Schedule, ParseSchedule, run/next-run logic), scheduler.go (Scheduler, Publisher, options).
Entry points: NewScheduler(opts...) with Add/Start/Stop; NewTask(format), NewTaskAtIntervals, NewTaskDaily, NewTaskOnWeekday, New(*Schedule); bind with Task.Do(name, fn, args...); ParseSchedule ("every N unit [hh:mm]", " [hh:mm]", "hh:mm"); Publisher.
Invariants:
- Panics:
NewTaskDaily/NewTaskOnWeekdayon a bad hh:mm,Task.Doon a non-func or wrong arity. Errors:NewTask/ParseSchedule/UpdateSchedule,Start(already running);Stopsucceeds when already stopped. - Globals:
TimeNow, location viaSetGlobalLocation, package logger. Startspawns one ticker goroutine; each due task runs in its own goroutine with a per-task run lock andWithRunTimeout(default 1s); callback panics are recovered and logged. Tick =WithTickerIntervalor min(1s, shortest Duration/10). Runnable tasks are sorted by a snapshot of next-run times usingslices.SortFunc.scheduler.lockguards membership and lifecycle;Taskhas a separate RWMutex for callback, publisher, and run state.Listcopies the task slice;Task.Scheduledeep-copiesLastRunAtand returns a snapshot.New(*Schedule)copies its input. UseAdd/ClearandSetNextRun/UpdateScheduleto mutate state. CustomTaskimplementations must provide their own concurrency safety.Stopcloses the current run's quit channel under the lifecycle lock, is idempotent, and does not wait for task callbacks or for the ticker goroutine, but no*taskis dispatched afterStopreturned:runPending(quit)re-checksrunningand the run'squitunder the lifecycle lock and dispatches in the same critical section, and a*taskis dispatched withtryRun, which acquires the run lock without waiting and marks the task running (begin) before startingexecutein a goroutine, so a run dispatched beforeStopis already visible toIsRunningandRunCount(the callback itself may begin just afterStopreturned); a customTaskimplementation is dispatched withgo Run(), so its run may begin, and become visible, afterStopreturned; the guarantee is documented onScheduler.Stopfor tasks created by this package only (holding the lock until a customRunbegan would run arbitrary code under it). A tick that was ready whenquitclosed may still win the select, harmlessly.Startmay restart afterStop. Publisher callbacks run outside scheduler and task locks (executepublishes from the run goroutine).- Task ID:
WithIDoruuid.NewV7().
Tests: time-based (task_test.go, scheduler_test.go) and overlapping reads, writes, stops, and callbacks (concurrency_test.go); task_bench_test.go measures Run; package race tests pass.
github.com/effective-security/porto/pkg/tlsconfig
Purpose: build tls.Config from PEM files and keep the keypair fresh via a polling reloader; reloading RoundTripper and cipher-suite name mapping.
Files: doc.go, tlsconfig.go (NewServerTLSFromFiles, NewClientTLSFromFiles, NewClientTLSWithReloader, HTTPTransport), tls.go (X509KeyPairWithOCSP, LoadX509KeyPairWithOCSP), reloader.go (KeypairReloader), cipher_suites.go (UpdateCipherSuites, GetCipherSuite).
Invariants:
- Defaults: MinVersion TLS 1.2, NextProtos h2 + http/1.1;
rootsFileis used for bothRootCAsandClientCAson the server; CA files with no valid certificates return an error. - Expired certificates fail initial load or reload; a failed reload keeps the previous pair. If the current pair later expires, TLS callbacks return errors and
Keypairreturns nil. Expiry within one hour is logged as a warning. - One poll goroutine per reloader (mtime polling plus an hourly forced reload); handlers run in their own goroutines; the goroutine snapshots timestamps under
RLock;Closewaits for an active reload, then closesstopChan. A secondCloseorReloadafterClosereturns an error; the poll goroutine does not log that closed error (errReloaderClosed). Reloadmarks itself in progress under the write lock, sleeps and loads outside the lock, then swaps the pair under the lock. AReloadthat finds another in progress waits onreloadDone, then loads again, so a nil return means the files were read after the call started. File mtimes are stat'ed before each load attempt, so a write during the load triggers the next poll;countis atomic.HTTPTransportclones a suppliedhttp.Transport, installs a fixedTLSClientConfigwithGetClientCertificate, and closes idle connections on reload andClose.RoundTripnever mutates the active TLS config.pkg/transport.TLSInfo.ServerTLSWithReloaderclears the staticCertificatesslice after installingGetCertificateso clients without SNI still receive the current, expiry-checked pair.- OCSP staple file:
<certfile-minus-ext>.ocsp; ignored if expired or unparsable, error if revoked. - Cipher suite names (
cipher_suites.go) are derived atinitfrom the Go release the binary is built with:tls.CipherSuitesentries usable below TLS 1.3, plus theTLS_ECDHE_{RSA,ECDSA}_WITH_CHACHA20_POLY1305alias constants.UpdateCipherSuitesrejects names intls.InsecureCipherSuites(Go 1.27: RC4, 3DES, CBC-SHA256 and everyTLS_RSA_*key-exchange suite) and TLS 1.3 names with distinct errors (an alias is classified by its canonical name), and leaves the config unchanged on error;GetCipherSuitereports only accepted names. The list only restricts the enabled suites: Go ignores the order oftls.Config.CipherSuites, andPreferServerCipherSuiteshas no effect, socipher_suitessets no preference. A toolchain upgrade that moves a suite to the insecure list makes configs naming it fail. - Test hook: package var
makeTicker.
Tests: generate certs with xpki/testca; reloader tests rewrite files on a 100ms poll, and b02_test.go covers expiry, lock availability, Close versus Reload, Reload waiting for an active reload, concurrent reloads, no error log for a poll during Close (non-parallel: replaces makeTicker and the xlog formatter), stable transport config, and invalid CA bundles; tls_test.go has inline PEM fixtures; cipher_suites_test.go compares the accepted table with tls.CipherSuites/tls.InsecureCipherSuites and table-tests UpdateCipherSuites errors; keypair_test.go (black box) covers key types (key equality compares key material, not RSA precomputation internals), key/certificate mismatches, OCSP staples, client configs and HTTPTransport, and reloader_poll_test.go (internal, replaces makeTicker) the poll loop with deleted files and validateCertificate.
github.com/effective-security/porto/pkg/transport
Purpose: server-side listeners: eager-handshake TLS listener with CRL check, and TCP keepalive listeners; TLSInfo config → tls.Config with reloader.
Files: doc.go, transport.go (logger), tls.go (TLSInfo: ServerTLSWithReloader, Config, Close), listener_tls.go (NewTLSListener), keepalive_listener.go (NewKeepAliveListener).
Invariants:
TLSInfofields:CertFile/KeyFile/TrustedCAFile/ClientCAFile/ClientAuthType/CipherSuites/CRLVerifier/HandshakeFailure/HandshakeTimeout, all used. The unenforced etcd fields (AllowedCN,AllowedHostname,EmptyCN,ServerName,InsecureSkipVerify,SkipClientSANVerify) were removed in B15; other client-certificate policy is set withVerifyConnectionon the config returned byServerTLSWithReloaderbeforeNewTLSListenerserves (the listener uses the same*tls.Config);Closedrops that config, so set it again after a rebuild.- CRL check only sees
VerifiedChains(needs ClientAuth ≥VerifyClientCertIfGiven); Revoked → reject; verify error or Unknown → log and allow (fail-open). - Eager handshakes have a default 10s deadline (
TLSInfo.HandshakeTimeout, zero = default, negative = disabled), cleared before returning a successful connection. Deadline and handshake errors go to HandshakeFailure and the connection is closed. - Accept errors of the wrapped listener are returned unwrapped by both listener types: net/http, grpc and cmux type-assert
net.Errorto retry temporary errors (EMFILE). The TLS listener passes each such error to oneAcceptcall (its accept loop waits for that call) and keeps accepting; it stops only when the wrapped listener returnsnet.ErrClosedorCloseis called, andAcceptthen returns an error wrappingnet.ErrClosed. A wrapped listener that reports its closure with another error (cmux:ErrListenerClosed,ErrServerClosed) keeps the loop running untilClose; net/http and grpc close their listener when such an error stops them. A repeatedCloseneither blocks nor panics and returns the wrapped listener's close error. After handing an error toAccept, the loop calls the wrappedAcceptagain at once and holds the result for the next call, so the caller sees an error that is one of its backoff periods old and recovers from an error storm up to one backoff period (net/http: 1s) later than on a raw listener.NewTLSListenercloseslfor a nil or emptyTLSInfobut leaves it open whenServerTLSWithReloaderfails. - Keepalive listeners call
SetKeepAliveConfigwithkeepAliveConfig(idle 30s, interval 15s, 9 probes: a dead peer is dropped after 165s). Connections withoutSetKeepAliveConfig(non-TCP) are returned unchanged; aSetKeepAliveConfigerror is logged at DEBUG and swallowed so a reset peer cannot failAccept(net.TCPListener's own accept ignores it too). - Reloader interval is 5 minutes (
reloadInterval);TLSInfo.Closestops it.ServerTLSWithReloaderbuilds into locals and caches the config and reloader only after every step succeeded, so a failed call (load, expiry, cipher list, reloader) caches nothing and a retry reads the files again. ServerTLSWithReloaderclears static certificates after installingGetCertificate, so handshakes without SNI use the current pair and receive expiry errors.
Tests: tls_test.go init() builds a CA/intermediate/server chain with xpki/testca in a per-process os.MkdirTemp directory (fixtureDir, removed by TestMain, so concurrent test processes do not overwrite each other's certificates) and verifies a handshake without SNI uses the callback; listener tests start restserver over the listener with a fake verifier; limits_test.go checks stalled handshakes and application I/O after the handshake deadline; b15_test.go covers failed-build retries (bad cipher list, expired certificate), unwrapped Accept errors, serving after EMFILE through net/http (both listener types) and cmux (keepalive listener), Close while the accept loop waits to forward an error, the keepalive config passed to fake connections (including non-TCP and failing ones), and VerifyConnection enforcement; b15_linux_test.go reads the keepalive socket options of a real connection.
github.com/effective-security/porto/pkg/appinit and pkg/appinit/config
Purpose: service bootstrap: logging setup, metrics pipeline (Prometheus/CloudWatch), CPU profiler; config.Metrics is the YAML/JSON struct.
Files: init.go (LogConfig, Flags, Logs, CPUProfiler), metrics.go (Metrics, cloudWatchRunner, runtimeStatsCloser, metricsClosers), prometheus.go (bounded, closable HTTP endpoint; prometheusHandler builds its handler for a registerer and gatherer), cpu_profiler.go (startCPUProfile, cpuProfileCloser, cpuProfileRunning, removeCPUProfile); config/config.go (Metrics, Prometheus, CloudWatch).
Invariants:
- Process-global side effects:
xlog.SetFormatter(or the logrotate formatter installed bylogrotate.Initialize, whichLogskeeps and only adjusts),metrics.NewGlobal, default Prometheus registry,xlog.OnErrorhook, package varspromSink/cwSink(once per process). provideris comma-separated:prometheus | cloudwatch | inmem; empty or disabled →(nil, nil). A provider without its config block is an error. Provider choices are validated before binding or registering sinks. Prometheus binds synchronously before sink initialization; bind errors are returned without initializing the sink, later initialization errors release the listener and reset the Prometheus and CloudWatch sinks created by the call, so a failed call can be retried; the Prometheus endpoint and the CloudWatchRungoroutine start only after initialization succeeds. Its HTTP server uses shared timeout/body defaults (prometheus.timeouts,prometheus.max_request_body) and, withprometheus.timeouts.handlerset, answers a scrape that takes longer with 503 throughpromhttp.HandlerOpts.Timeout(prometheusHandler; anhttp.TimeoutHandler, which also bounds the request context): promhttp gathers without consulting the request context, solimits.Timeouts.ApplyHandleralone did not end a slow scrape, and the collection still finishes in the background. The returned closer closes active connections and the listener and waits for Serve to exit; concurrent/repeated endpoint closes are safe. Unexpected serve errors are logged, never fatal. The endpoint closer ignores an already-closed listener error. The CloudWatch closer (cloudWatchRunner) cancels the context ofcloudwatch.Sink.Runand waits for it to return;Runpublishes the last interval first (bounded by the sink's 10s shutdown timeout, errors logged, not returned). An interval is lost when the cancellation aborts a periodic publish in flight or meets a tickRunhas not handled yet (Dataempties the sink withWithCleanupbefore the publish fails); aRunthat already stopped on expired or missing credentials makesClosereturn at once. Fixing either needscloudwatch.Sink.Runchanges in the metrics module. Withruntime_metricsand a Prometheus or CloudWatch sink,runtimeStatsClosersignals the collector of the global*metrics.Metricsto stop without waiting for it (emission keeps working), before the CloudWatch runner closes; a closer is therefore returned even for Prometheus without an address.inmemalone installs no global instance, soruntime_metricsstarts nothing. Closers close in order: Prometheus endpoint, runtime collector, CloudWatch runner; each is idempotent and safe for concurrent use. Sinks remain initialized once per process after Close; initialize Metrics serially. Reads envNODE_NAMEfor thenodetag.CPUProfilerkeeps the process-globalcpuProfileRunningflag while a profile it started runs: a second call fails withCPU profile already runningbeforeos.Create, so it cannot truncate the running profile's file. A profile started otherwise (directpprof,go test -cpuprofile) makespprof.StartCPUProfilefail after the file was created; the file is then closed and removed (if that profile writes to the same path, its file is truncated and removed), and the flag is cleared only after the removal, so a concurrent call on the same path cannot start a profile that the removal unlinks. The closer stops the profile, closes the file, clears the flag and returns the close error; a secondClosereturns an error.- Nil closers are normal; callers must nil-check.
- Test hook: package var
removeCPUProfile(failed-start file removal). LogDir/dev/nulldiscards; set → logrotate (10 MB / 10 days, buffered) with stderr as an extra sink whenLogStd.- Config keys:
disabled, provider, prefix, prefix_for_number_labels, prometheus{addr,expiration,timeouts{header,read,idle,handshake,handler},max_request_body}, runtime_metrics, cloudwatch{aws_region,aws_endpoint,namespace,publish_interval,with_sample_count}, global_tags, allowed_prefixes, blocked_prefixes. B21 removed the unusedadd_tags/replace_tags(lenient decoders ignore them, strict ones reject them);aws_endpointreplaced the untaggedAwsEndpoint/awsendpointkeys.
Tests: init_test.go (log branches with t.TempDir, asserts the rotating file receives output; CPUProfiler no-op and create errors), cpu_profiles_test.go (profile lifecycle, a second call while a profile runs, a profile started outside CPUProfiler, file closed or removed on every path, a call during failed-start removal fails as already running, and a removal error is returned with the start error (non-parallel: replaces removeCPUProfile); CPU profiling is process-global, so serial, and skipped under go test -cpuprofile). metrics_test.go covers Prometheus bind errors/retry, Prometheus and CloudWatch sink-failure cleanup, provider validation, scrapes, header deadlines and concurrent close; prometheus_test.go checks prometheusHandler on private registries (a scrape without the handler timeout, a fast one within it, and a blocking collector answered 503 within the timeout); TestMetricsCloudWatchCloseStopsRun points aws_endpoint at an httptest fake CloudWatch (static credentials from env, no shared AWS config) and checks that Close stops Run after its final publish and stops the runtime stats collector (goroutine stack counts, waiting with Eventually for the goroutines to start). config/config_test.go covers the YAML/JSON keys; metrics_config_test.go covers provider validation errors, GlobalTags, Prometheus with CloudWatch, and the release of the Prometheus sink and endpoint when CloudWatch fails. Global-state tests run serially.
github.com/effective-security/porto/pkg/discovery
Purpose: tiny in-process service registry resolved by interface type (gserver exposes services to handlers through it).
Entry points: New(), Register(server, svc), Find(server, *iface), ForEach(*iface, fn).
Invariants: key <server>/<concrete type>; duplicate or nil Register → error; Find requires a non-nil pointer-to-interface, server "" means any; map iteration order makes multiple matches arbitrary. An RWMutex protects registration and lookup; ForEach snapshots matching entries before invoking callbacks so callbacks may register services. The caller owns synchronization of the destination interface value.
github.com/effective-security/porto/pkg/crlcache
Purpose: Verifier{Update() error; Verify(crt, issuer) (int, error)} returning ocsp.Good/Revoked/Unknown, consumed by transport.TLSInfo. No implementation here; implementations must be concurrency-safe.
github.com/effective-security/porto/pkg/streamctx
Purpose: wrap grpc.ServerStream to override Context() in stream interceptors. If ss is already a wrapper its context is replaced in place and the same object returned.
Tests: streamctx_test.go checks the in-place replacement and that every stream method reaches the wrapped stream (a recording stub).
github.com/effective-security/porto/metricskey
Purpose: metrics.Describe descriptors: HTTPReqPerf, HTTPReqByRole, GRPCReqPerf, GRPCReqByRole, StatsVersion, HealthLogErrors; Metrics slice. Names http_requests_perf, http_requests_role, rpc_requests_perf, rpc_requests_role, version, log_errors.
github.com/effective-security/porto/tests/testutils and tests/mockappcontainer
Purpose: test helpers. testutils: CreateURL, CreateBindAddr (panic if no free port; TOCTOU), JSON, CompareJSON. mockappcontainer: NewBuilder().WithConfig(*gserver.Config).WithJwtSigner(...).WithJwtParser(...).WithDiscovery(...).Container() builds a dig.Container; dig.Provide errors are discarded. builder_test.go checks that each With* provides its value; testutils/testutils_test.go covers the free-port helper's documented panic on an invalid host.
Build, test and CI
go.modtargets Go 1.27; the standarduuidpackage is used.make lint= gofmt with rewrites,go vet,govulncheck,golangci-lint(reviveexported+ staticcheckall, comment presets excluded).make testneeds Docker forpkg/redisclientandpkg/cache;pkg/retriableinternal_test.goneeds outbound DNS.make covtestwritescoverage.out; CI (.github/workflows/unittest.yml) runsmake lintandmake build covtest, and its coverage status requires more than 90% total coverage (MIN_TESTCOV); every package with statements is above 90% (B24). CI does not run-race.make docsregeneratesDocumentation/api/*.mdwith gomarkdoc (one file per non-test package).pkg/tasksand the other packages passgo test -race.