Files
7dcd87768b feat(ee): add Routable as a dedicated EE connector (#711)
* feat(ee): add Routable as a dedicated EE connector
Replaces the standalone connector-routable "protocol conversion" service
that exposed Routable through the Generic Connector. The dedicated plugin
uses native PSP types (PSPAccount, PSPBalance, PSPPayment,
PSPPaymentInitiation) instead of metadata-blob workarounds.
Capabilities: FETCH_ACCOUNTS, FETCH_BALANCES, FETCH_EXTERNAL_ACCOUNTS,
FETCH_PAYMENTS, CREATE_TRANSFER, CREATE_PAYOUT. Webhooks, bank-account
creation, and reversals are deferred to follow-up PRs. See
ee/plugins/routable/MAPPING.md for the full Routable<->Formance field
mapping and the per-payout metadata key surface.

* chore(openapi): regenerate openapi.yaml and api docs for routable connector

* fix(openapi): wrap $ref in allOf for nullable PSU fields

kin-openapi's stricter OpenAPI 3.1 enforcement now rejects nullable: true
as a sibling of $ref. Wrap the PSU contactDetails/address $refs in allOf
so nullable becomes a sibling of the parent schema. Same semantics,
unblocks just validate-openapi (CI Dirty step).

* fix(openapi): drop nullable: true sibling on PSU $ref fields

* fix(ee/routable): address CodeRabbit findings

- payments cursor: hold the cycle lower bound immutable across all pages of
  payables and receivables. The previous design advanced LastSeenAt after
  every page, so page 2 used a tighter floor than page 1 and could silently
  skip rows. Cycle now uses CycleLowerBound (frozen) and accumulates
  CycleMaxSeen separately; max is promoted to the next cycle's floor only
  when receivables exhausts. Legacy state migrates on decode.
- client.do: only append the Routable error envelope when the response
  carried one. Transport errors no longer get a misleading "routable api
  error: empty body" suffix.
- scheme: trim whitespace before classifying delivery_method.
- status test: assert PAYMENT_STATUS_REFUNDED is treated as terminal.

* fix(routable): update metadata handling and enforce required fields

- Updated `line_item_description` to ensure it always has a non-empty value, as required by Routable's v1 schema.
- Clarified that `memo` is read-only and not forwarded during payable creation, directing users to use `line_item_description` instead.
- Adjusted `send_on` to always be emitted as JSON null when sending immediately, complying with Routable's requirements.
- Enhanced error handling in the client to only append the error envelope when content is present, improving log clarity.

* feat(routable): add comprehensive field mapping documentation and metadata handling

- Introduced MAPPINGS.md as the authoritative reference for the Routable EE connector, detailing the translation of Routable API objects to Formance PSP types.
- Added self-describing metadata keys for payments to facilitate correlation between Routable and Formance systems, including `payment_initiation_reference` and `payable_id`.
- Implemented tests to ensure the accuracy of metadata handling for payables and receivables, confirming the integrity of the correlation contract.

* feat(routable): production hardening for ~200k tx/wk
Address Quentin/CodeRabbit feedback on #711 and the production-readiness
gaps specific to ~20 tx/min sustained throughput.
Critical correctness:
- Empty-cycle cursor regression: CycleLowerBound no longer regresses
  to epoch when both phases of a sync cycle return zero rows. Would have
  triggered a multi-million-row replay storm at this volume.
Engine extension (cross-connector, backward-compatible):
- New plugins.RateLimitedError{RetryAfter time.Duration} wraps the
  existing ErrUpstreamRatelimit sentinel. The activity error translator
  honours the hint in temporal ApplicationError.NextRetryDelay (clamped
  to the configured static floor), falling back to the static delay when
  no hint is present. Every existing connector still satisfies
  errors.Is(_, ErrUpstreamRatelimit) — no caller changes required.
Routable client:
- Parses IETF draft-ietf-httpapi-ratelimit-headers RateLimit + RFC 9110
  Retry-After per the draft §6 precedence rule (Retry-After wins, else
  smallest t across listed policies, else nil). Treats 5xx with a
  RateLimit header as rate-limit-driven (per IETF §6).
- HTTP transport pool tuned for sustained ~100-200 req/min:
  MaxIdleConnsPerHost=64, IdleConnTimeout=90s. Default
  MaxIdleConnsPerHost=2 would have churned TCP/TLS handshakes at this
  volume.
- PAGE_SIZE bumped from 50 to Routable's documented max of 100 — halves
  pagination request count at steady state.
- Credential probe in Install (GET /v1/settings/accounts?page=1) fails
  fast on 401/403 instead of letting the worker discover bad creds on
  the first FETCH_ACCOUNTS run.
Plugin architecture:
- Mappers extracted into ee/plugins/routable/mappers/ (amounts, scheme,
  status, metadata + 5 mapping files) per Quentin's review. Pure
  functions, no plugin state. Plugin package keeps orchestration only.
- initiatePayable lifted into payable_create.go; payouts.go and
  transfers.go are now thin orchestrators sharing the same plumbing.
- scheme.go covers every documented Routable delivery_method (ach
  family, wire, international_wire, check, payout, wallet) with OTHER
  fallback documented for rails the Formance enum has no dedicated
  constant for.
- status.go drops legacy connector-routable mention and the redundant
  case "" branch.
Tests pinned across the contract:
- Idempotency-Key=pi.Reference on createPayout AND createTransfer
  (cross-flow regression guard).
- Pagination invariants: empty-cycle preservation, partial-cycle
  promotion, multi-page floor immutability, boundary-row replay across
  cycles, JSON round-trip resumability, legacy-state migration.
- 4xx terminal vs 5xx transient classification end-to-end (8 status
  codes); 503 with vs without RateLimit header; 429 with/without hint;
  Retry-After delta-seconds + HTTP-date + past-date clamp.
- 404 ErrPayableNotFound treated as transient across the eventual-
  consistency window.
- Currency precision JPY (0), USD/EUR (2), KWD/BHD/OMR (3), VND (0).
- nil DestinationAccount + nil SourceAccount validation.
Docs (MAPPINGS.md):
- §1.1 concurrency/immutability statement (Plugin instances are
  immutable post-construction).
- §1.2 credential validation (install-time probe).
- §3.6 cursor invariants — three explicit invariants (resumable,
  lossless, tiebreaker) replacing the deprecated lastSeenAt language.
- §5.5 payout-under-retries lifecycle mermaid + rate-limit hint
  explainer.
- §6 ops capacity & cost model — Routable RPS budget, Temporal
  workflow/activity volumes, Cloud Action estimate (~24M/month ⇒ ~$600
  at list pricing), and the four levers operators can pull (polling
  period, page size, webhooks follow-up, per-capability schedules
  follow-up).

* test(routable): add direct unit tests (mappers 31% -> 90%, client 62% -> 70%, plugins/errors 12% -> 22%)

* fix(routable): branch on 201/202 HTTP status (not on body) for payable creation, validate inputs at the choke point, trim whitespace credentials

* chore(routable): trim doc-comments duplicating MAPPINGS.md

* chore(routable): scrub rate-limit narrative from MAPPINGS.md and client.go doc-comments

Removes the documentation references to the IETF/RFC 9110 rate-limit hint
parsing that lived briefly in this branch. The shared parser was lifted into
the engine's httpwrapper package in #715; this PR no longer touches the engine
or any internal/ code.

* chore(routable): regenerate SDK gen.lock after rebase onto main

* fix(routable): non-retriable validation errors, PI-Payment linkage on terminal failures, status_changed_at timestamps, externally_paid mapping

* fix(routable): always return Payment from pollPayableStatus; resync MAPPINGS volume model

* fix(routable): inline time.Now() in external_accounts, return req.State unchanged on throttle skip

* Add PayoutsPerSecond() to Routable connector

---------

Co-authored-by: Crimson Thompson <[email protected]>
2026-05-20 16:28:19 +00:00

372 lines
13 KiB
Go

package client
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// newTestServer spins up a Routable-shaped httptest server that asserts the
// authentication header the client must send and routes requests through a
// per-test handler. Returning a fresh Client wired to the test server keeps
// each test isolated.
func newTestServer(t *testing.T, handler http.HandlerFunc) (Client, *httptest.Server) {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "Bearer test-key" {
t.Errorf("expected Bearer auth header, got %q", got)
}
handler(w, r)
}))
return New("routable-test", "test-key", srv.URL), srv
}
func TestListAccounts(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/settings/accounts" {
t.Errorf("unexpected path: %s", r.URL.Path)
}
if r.URL.Query().Get("page") != "1" || r.URL.Query().Get("page_size") != "25" {
t.Errorf("unexpected pagination: %s", r.URL.RawQuery)
}
_, _ = io.WriteString(w, `{"object":"List","results":[{"id":"acc_1","name":"Operating","type_details":{"available_amount":"100.00"},"created_at":"2025-01-01T00:00:00Z"}],"links":{"next":"/v1/settings/accounts?page=2"}}`)
})
defer srv.Close()
resp, err := c.ListAccounts(context.Background(), 1, 25)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(resp.Results) != 1 || resp.Results[0].ID != "acc_1" {
t.Fatalf("unexpected results: %+v", resp.Results)
}
if !resp.Links.HasMore() {
t.Fatalf("expected HasMore=true")
}
}
func TestGetPayableNotFound(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
_, _ = io.WriteString(w, `{"object":"Error","message":"not found"}`)
})
defer srv.Close()
_, err := c.GetPayable(context.Background(), "pa_missing")
if !errors.Is(err, ErrPayableNotFound) {
t.Fatalf("expected ErrPayableNotFound, got %v", err)
}
}
func TestCreatePayableSendsIdempotencyKey(t *testing.T) {
var captured struct {
Key string
Body CreatePayableRequest
}
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
captured.Key = r.Header.Get("Idempotency-Key")
_ = json.NewDecoder(r.Body).Decode(&captured.Body)
w.WriteHeader(http.StatusCreated)
_, _ = io.WriteString(w, `{"object":"Payable","id":"pa_1","status":"pending","amount":"10.00","currency_code":"USD","created_at":"2025-01-01T00:00:00Z"}`)
})
defer srv.Close()
req := CreatePayableRequest{
Type: "ach",
DeliveryMethod: "ach_standard",
PayToCompany: "co_1",
WithdrawFromAccount: "acc_1",
Amount: "10.00",
CurrencyCode: "USD",
LineItems: []PayableLineItem{{UnitPrice: "10.00", Amount: "10.00", Quantity: 1}},
ActingTeamMember: "tm_1",
IdempotencyKey: "pi_42",
}
resp, status, err := c.CreatePayable(context.Background(), req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if status != http.StatusCreated {
t.Errorf("status = %d, want 201", status)
}
if resp.ID != "pa_1" {
t.Fatalf("unexpected payable id: %s", resp.ID)
}
if captured.Key != "pi_42" {
t.Fatalf("expected idempotency key 'pi_42', got %q", captured.Key)
}
if captured.Body.PayToCompany != "co_1" {
t.Fatalf("body not forwarded: %+v", captured.Body)
}
}
func TestCreatePayableValidatesInputBeforeNetwork(t *testing.T) {
c := New("routable-test", "test-key", "http://invalid")
if _, _, err := c.CreatePayable(context.Background(), CreatePayableRequest{Type: ""}); err == nil {
t.Fatal("expected validation error before any network call")
}
}
// 201 vs 202: the upstream HTTP status determines whether the caller
// reads the full payable (201) or just the polling ID (202). The plugin
// branches on this — the test pins both shapes so a regression in the
// status propagation is caught at the client boundary.
func TestCreatePayableReturnsStatusCode(t *testing.T) {
t.Parallel()
t.Run("201 Created with full payable model", func(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusCreated)
_, _ = io.WriteString(w, `{"object":"Payable","id":"pa_sync","status":"completed","amount":"10.00","currency_code":"USD","created_at":"2025-01-01T00:00:00Z"}`)
})
defer srv.Close()
req := CreatePayableRequest{
Type: "ach", DeliveryMethod: "ach_standard",
PayToCompany: "co_1", WithdrawFromAccount: "acc_1",
Amount: "10.00", CurrencyCode: "USD",
LineItems: []PayableLineItem{{UnitPrice: "10.00", Amount: "10.00", Quantity: 1, Description: "x"}},
ActingTeamMember: "tm_1", IdempotencyKey: "pi_sync",
}
got, status, err := c.CreatePayable(context.Background(), req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if status != http.StatusCreated {
t.Errorf("status = %d, want 201", status)
}
if got.ID != "pa_sync" || got.Amount != "10.00" {
t.Errorf("unexpected payable: %+v", got)
}
})
t.Run("202 Accepted with id-only payload", func(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusAccepted)
_, _ = io.WriteString(w, `{"id":"pa_async","status":"pending"}`)
})
defer srv.Close()
req := CreatePayableRequest{
Type: "ach", DeliveryMethod: "ach_standard",
PayToCompany: "co_1", WithdrawFromAccount: "acc_1",
Amount: "10.00", CurrencyCode: "USD",
LineItems: []PayableLineItem{{UnitPrice: "10.00", Amount: "10.00", Quantity: 1, Description: "x"}},
ActingTeamMember: "tm_1", IdempotencyKey: "pi_async",
}
got, status, err := c.CreatePayable(context.Background(), req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if status != http.StatusAccepted {
t.Errorf("status = %d, want 202", status)
}
if got.ID != "pa_async" {
t.Errorf("ID = %q, want pa_async", got.ID)
}
if got.Amount != "" || got.CurrencyCode != "" {
t.Errorf("202 body should not carry amount/currency, got %+v", got)
}
})
}
func TestErrorSuffixOnlyWhenAPIBodyPresent(t *testing.T) {
t.Parallel()
t.Run("with API body", func(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = io.WriteString(w, `{"object":"Error","code":"invalid","message":"bad amount"}`)
})
defer srv.Close()
_, err := c.ListAccounts(context.Background(), 1, 25)
if err == nil {
t.Fatal("expected error")
}
if !strings.Contains(err.Error(), "bad amount") {
t.Fatalf("expected error to surface API message, got %q", err.Error())
}
})
t.Run("without API body", func(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
// no JSON body
})
defer srv.Close()
_, err := c.ListAccounts(context.Background(), 1, 25)
if err == nil {
t.Fatal("expected error")
}
// We must not pollute the message with the placeholder "empty body" suffix.
if strings.Contains(err.Error(), "routable api error: empty body") {
t.Fatalf("error should not include placeholder suffix, got %q", err.Error())
}
})
}
func TestListPayablesPassesStatusChangedAtGte(t *testing.T) {
when := time.Date(2025, 1, 2, 3, 4, 5, 0, time.UTC)
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
got := r.URL.Query().Get("status_changed_at.gte")
if !strings.HasPrefix(got, "2025-01-02T03:04:05") {
t.Errorf("missing/bad status_changed_at.gte: %q", got)
}
_, _ = io.WriteString(w, `{"object":"List","results":[]}`)
})
defer srv.Close()
if _, err := c.ListPayables(context.Background(), 1, 50, when); err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestListReceivablesPassesStatusChangedAtGte(t *testing.T) {
when := time.Date(2025, 1, 2, 3, 4, 5, 0, time.UTC)
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/receivables" {
t.Errorf("path = %q, want /v1/receivables", r.URL.Path)
}
if got := r.URL.Query().Get("status_changed_at.gte"); !strings.HasPrefix(got, "2025-01-02T03:04:05") {
t.Errorf("missing/bad status_changed_at.gte: %q", got)
}
_, _ = io.WriteString(w, `{"object":"List","results":[{"id":"re_1","status":"pending","amount":"5.00","currency_code":"USD","created_at":"2025-01-02T00:00:00Z"}]}`)
})
defer srv.Close()
resp, err := c.ListReceivables(context.Background(), 1, 50, when)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(resp.Results) != 1 || resp.Results[0].ID != "re_1" {
t.Fatalf("unexpected results: %+v", resp.Results)
}
}
// ListReceivables without the gte filter must NOT send the empty
// status_changed_at.gte query param (Routable rejects it).
func TestListReceivablesOmitsEmptyGte(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Has("status_changed_at.gte") {
t.Errorf("empty gte must be omitted, got %q", r.URL.RawQuery)
}
_, _ = io.WriteString(w, `{"object":"List","results":[]}`)
})
defer srv.Close()
if _, err := c.ListReceivables(context.Background(), 1, 50, time.Time{}); err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestListCompanies(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/companies" {
t.Errorf("path = %q, want /v1/companies", r.URL.Path)
}
if r.URL.Query().Get("page") != "2" || r.URL.Query().Get("page_size") != "50" {
t.Errorf("unexpected pagination: %s", r.URL.RawQuery)
}
_, _ = io.WriteString(w, `{"object":"List","results":[{"id":"co_1","display_name":"Acme","created_at":"2025-01-01T00:00:00Z"}],"links":{"next":"/v1/companies?page=3"}}`)
})
defer srv.Close()
resp, err := c.ListCompanies(context.Background(), 2, 50)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(resp.Results) != 1 || resp.Results[0].ID != "co_1" {
t.Fatalf("unexpected results: %+v", resp.Results)
}
if !resp.Links.HasMore() {
t.Fatalf("expected HasMore=true")
}
}
func TestGetAccount(t *testing.T) {
c, srv := newTestServer(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/settings/accounts/acc_1" {
t.Errorf("path = %q, want /v1/settings/accounts/acc_1", r.URL.Path)
}
_, _ = io.WriteString(w, `{"id":"acc_1","name":"Operating","currency_code":"USD","created_at":"2025-01-01T00:00:00Z","type_details":{"available_amount":"1000.00"}}`)
})
defer srv.Close()
got, err := c.GetAccount(context.Background(), "acc_1")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got.ID != "acc_1" || got.Name != "Operating" {
t.Fatalf("unexpected account: %+v", got)
}
}
// GetAccount with empty id must short-circuit before any network call —
// guards against a bug where the engine handed us a zero-value reference.
func TestGetAccountRejectsEmptyID(t *testing.T) {
c := New("routable-test", "test-key", "http://invalid-on-purpose")
_, err := c.GetAccount(context.Background(), "")
if err == nil {
t.Fatal("expected error for empty id")
}
}
// CreatePayable validation: every required field that's missing must
// surface a distinct error message before the request hits the wire.
func TestValidateCreatePayableRequiredFields(t *testing.T) {
base := CreatePayableRequest{
Type: "ach",
DeliveryMethod: "ach_standard",
PayToCompany: "co_1",
WithdrawFromAccount: "acc_1",
Amount: "10.00",
LineItems: []PayableLineItem{{UnitPrice: "10.00", Amount: "10.00", Quantity: 1}},
ActingTeamMember: "tm_1",
}
cases := []struct {
name string
mut func(*CreatePayableRequest)
want string
}{
{"missing type", func(r *CreatePayableRequest) { r.Type = "" }, "type is required"},
{"missing delivery_method", func(r *CreatePayableRequest) { r.DeliveryMethod = "" }, "delivery_method is required"},
{"missing pay_to_company", func(r *CreatePayableRequest) { r.PayToCompany = "" }, "pay_to_company is required"},
{"missing withdraw_from_account", func(r *CreatePayableRequest) { r.WithdrawFromAccount = "" }, "withdraw_from_account is required"},
{"missing amount", func(r *CreatePayableRequest) { r.Amount = "" }, "amount is required"},
{"empty line_items", func(r *CreatePayableRequest) { r.LineItems = nil }, "line item is required"},
{"missing acting_team_member", func(r *CreatePayableRequest) { r.ActingTeamMember = "" }, "acting_team_member is required"},
}
c := New("routable-test", "test-key", "http://invalid-on-purpose")
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := base
tc.mut(&req)
_, _, err := c.CreatePayable(context.Background(), req)
if err == nil {
t.Fatalf("expected validation error for %s", tc.name)
}
if !strings.Contains(err.Error(), tc.want) {
t.Errorf("error = %q, want contains %q", err.Error(), tc.want)
}
})
}
}
// GetPayable must reject empty IDs before any network call (mirror of
// GetAccount).
func TestGetPayableRejectsEmptyID(t *testing.T) {
c := New("routable-test", "test-key", "http://invalid-on-purpose")
_, err := c.GetPayable(context.Background(), "")
if err == nil {
t.Fatal("expected error for empty id")
}
}