mirror of
https://github.com/hanzo-fi/payments.git
synced 2026-10-30 20:31:20 +00:00
65fa8df73f
* fix(assets): EN-1011 align asset regex with Ledger The asset regex in internal/utils/assets had drifted from Ledger's canonical pattern (formancehq/ledger/pkg/assets/asset.go) after the Fireblocks (#639) and hyphen (#649) PRs, accepting strings that Ledger later rejects. Roll the pattern back to Ledger's exact shape: [A-Z][A-Z0-9]{0,16}(_[A-Z]{1,16})?(\/\d{1,6})? This drops support for multi-underscore identifiers, hyphens, and digit-containing suffix segments. Connectors emitting non-canonical asset strings (Fireblocks today) must canonicalize at their boundary; see the follow-up Fireblocks commit on this branch. * feat(fireblocks): EN-1011 emit canonical asset + rich metadata The connector used to surface the raw Fireblocks `legacyId` as the Formance asset string (`USDT_ERC20/6`, `MATIC_POLYGON_AMOY/18`, …), which Ledger rejects and which leaks Fireblocks-isms downstream. Rework the plugin so the canonical asset is derived from `displaySymbol` (sanitised to fit the Ledger regex) and the wealth of context Fireblocks provides ends up where it belongs: * Extend `client.Asset` to capture `displaySymbol`, `displayName`, `blockchainId`, `assetClass`, `onchain.{symbol,name,address,standards}`, `metadata.{scope,verified,deprecated,features}`. * Replace the `map[legacyId]int` cache with `map[legacyIdUpper]assetInfo` carrying the canonical asset string, precision, blockchainId, legacyId and a prebuilt `com.fireblocks.spec/` metadata slice. * `buildAssetInfo` filters NFT/SFT/VIRTUAL + deprecated, picks decimals by `assetClass` (FIAT → top-level, NATIVE/FT → onchain), and drops assets without a legacyId (vaults reference legacyIds only — UUIDs would never round-trip per Fireblocks' own guidance). * `sanitizeSymbol` / `canonicalAsset` produce a Ledger-valid asset (`1INCH` → `INCH`, `xDAI` → `XDAI`, oversized symbols truncated to 17). * `fetchNextBalances` aggregates same-canonical-asset entries inside a vault (USDT_ERC20 + USDT_TRX → one `USDT/6` balance with summed amount) and validates per-entry, dropping invalid balances with a log instead of poisoning the whole batch. * `fetchNextPayments` looks up via the cache, copies the asset's metadata slice onto the payment, renames the existing tx metadata keys to the `com.fireblocks.spec/` namespace (`tx_hash`, `network_fee`, `note`, `sub_status`, `destination_ids`) and validates per-entry. * `fetchNextAccounts` surfaces `customer_ref_id`, `hidden_on_ui`, `auto_fuel` onto `PSPAccount.Metadata` when set. Existing tests are updated to the new field name / metadata keys; expanded coverage lives in the next commit. * test(fireblocks): EN-1011 cover sanitization, metadata, and skip paths Expand the Fireblocks test surface to lock the new behaviour: * `assets_sync_test.go` (new): table-driven specs for `sanitizeSymbol` (1INCH/xDAI/punctuation/oversize/empty), `canonicalAsset`, and `buildAssetInfo` covering FIAT vs FT decimals selection, the `displaySymbol` → `onchain.symbol` → `legacyId` fallback chain, digit-prefix sanitisation, verified+features metadata, and the deprecated / NFT / SFT / VIRTUAL / no-decimals / empty-symbol skip paths. * `balances_test.go`: aggregation of USDT_ERC20+USDT_TRX into a single `USDT/6` balance with summed amount, plus case-insensitive xDAI matching. * `payments_test.go`: asset-cache metadata (legacy_id, contract_address, token_standard, blockchain_id) is copied onto each payment alongside the namespaced tx_hash / network_fee / note / sub_status fields. * `accounts_test.go`: customer_ref_id / hidden_on_ui / auto_fuel surface on PSPAccount.Metadata when present, and stays nil otherwise. * fix(fireblocks): EN-1011 fetch full transaction history on first sync Fireblocks' /v1/transactions endpoint silently defaults `after` to "the past 90 days" when the parameter is omitted. On a fresh install this caused the connector to miss every transaction older than 90 days and then advance state past the rest, never backfilling them. Always send `after`, pinning it to 1 ms past epoch on the first call. Subsequent polls keep using the cursor we track in state, so behaviour is unchanged beyond the first poll. * feat(fireblocks): EN-1011 segregate testnet assets via blockchain.onchain.test Without this, the canonicalisation step happily collapses testnet and mainnet holdings of the same symbol onto the same Formance asset. The local sandbox already proved this: a vault holding both ETH_TEST5 (Sepolia) and ETH-AETH_SEPOLIA aggregated under `ETH/18`, and the same asset would have collided with real ETH. Use Fireblocks' own authoritative signal — `Blockchain.onchain.test` exposed on `/v1/blockchains` — rather than legacyId heuristics: * Add `client.Blockchain` + `ListBlockchains` (cursor-paginated, same shape as ListAssets). * `loadAssets` now also calls ListBlockchains and builds a `map[blockchainId]bool` keyed by the blockchain UUID. The lookup is in-memory and bounded (~50 blockchains). * `buildAssetInfo` / `canonicalAsset` take an `isTestnet` bool. For testnet assets the canonical asset gets a `_TEST` suffix (`ETH_TEST/18`, `SOL_TEST/9`, …). Base is trimmed when needed so the whole identifier still fits Ledger's 17-char cap before the suffix. * `com.fireblocks.spec/testnet=true` is stamped onto the per-asset metadata slice so it propagates to every payment. * Tests: extend canonicalAsset, buildAssetInfo, and the install spec with the new mocked `ListBlockchains` call and a Sepolia entry. * fix(fireblocks): EN-1011 preserve full base length for testnet canonical assets canonicalAsset was trimming the base to 12 chars before appending `_TEST`, producing collisions between distinct long-symbol testnet assets that shared a 12-char prefix. Ledger's regex `[A-Z][A-Z0-9]{0,16}(_[A-Z]{1,16})?(/\d{1,6})?` permits a 17-char base plus the `_TEST` suffix independently — sanitizeSymbol already caps the base at 17 chars, so dropping the inner trim is safe. Addresses CodeRabbit review comment on #720. * refactor(fireblocks): EN-1011 trim comments and modernise helpers * Trim AI-flavoured / stale / coaching comments across the plugin to match sibling EE connector style (bitstamp, coinbaseprime). * Switch `buildPaymentMetadata` to `maps.Copy` and `appendUnique` to `slices.Contains` (Go 1.21+ idioms). * Drop the unused `boolStr` helper; inline `"true"` literals in account metadata, matching sibling connectors. No behavioural change. * test(fireblocks): EN-1011 lock 90-day fix, refresh, aggregation, status mapping * `client/transactions_test.go` (new): httptest-based specs pinning `after=1` on the first sync, `after=<state>` thereafter, the rest of the query shape (`limit`, `orderBy=createdAt`, `sort=ASC`), the happy-path round-trip, and the server-error wrap. * `client/blockchains_test.go` (new): two-page cursor pagination, initial request has no `pageCursor`, second carries the cursor; plus server-error wrap. * `client/client_testing_test.go` (new): shared `newTestClient` helper. * `assets_sync_test.go`: `ensureAssetsFresh` specs covering the no-op (fresh cache), first-load (empty cache), and TTL-stale refresh. * `balances_test.go`: three-chain aggregation (USDT_ERC20 + USDT_TRX + USDT_BSC -> USDT/6) and duplicate-legacyId-row aggregation. * `payments_test.go`: contract that `Metadata` is non-nil even when the cached asset has no metadata slice (so `destination_ids` writes are safe), plus a `DescribeTable` covering every documented Fireblocks status -> `PaymentStatus` mapping. Coverage deltas: ensureAssetsFresh 38.5% -> 92.3%, ListTransactions / ListBlockchains 0% -> ~92%, matchPaymentStatus 50% -> 100%, appendUnique 67% -> 100%. No behavioural change to production code. * refactor(fireblocks): EN-1011 PR #720 review feedback * accounts.go: rewrite buildAccountMetadata doc to state "non-zero" (CustomerRefID has the same treatment as the bools). * assets_sync.go: emit an error-level log when loadAssets returns 0 entries, defensive against a future Fireblocks legacyId surface change. * balances.go: switch user-visible "vault" log wording to "account", matching the Formance/Fireblocks console UX. * client/{accounts,assets,blockchains,transactions}.go: surface the Fireblocks error body in wrapped client errors via a small wrap helper on fireblocksError. * balances_test.go: add a regression spec proving same-symbol different-precision assets stay in separate balance rows. --------- Co-authored-by: Thierry Coopman <[email protected]> Co-authored-by: Cursor <[email protected]>
Formance Payments 
Getting started
Payments works as a standalone binary, the latest of which can be downloaded from the releases page.
You can move the binary to any executable path, such as to /usr/local/bin. Installing it locally using Docker is also
possible, and probably easier for testing purposes as it comes prepackaged with all the dependencies.
Note that you need to set up the STACK_PUBLIC_URL env variable to set up a publicly available URL so that webhooks
and redirects from the Payment Service Providers (PSP) can be sent to the application.
You can use NGrok for that, for example. If you don't plan on using any connectors with webhooks,
you could simply set it as localhost (or anything, really).
$ git clone [email protected]:formancehq/payments.git
$ cd payments
$ just compile-plugins
$ STACK_PUBLIC_URL=http://localhost docker compose up
Debugging
You can also use the docker-compose.dev.yml file to run the application with Delve and Air, which allow debugging and live reloading.
Use console as a frontend
The payment application comes with a console frontend when deploying through docker-compose (with or without debugging). You can access it at http://localhost:3000/formance/localhost?region=localhost.
What is it?
Basically, a framework.
A framework to ingest payin and payout coming from different payment providers (PSP).
The framework contains connectors. Each connector is basically a translator for a PSP. Translator, because the main role of a connector is to translate specific PSP payin/payout formats to a generalized format used at Formance.
Because it is a framework, it is extensible. Please follow the guide below if you want to add your connector.
Contribute
Please see the following documents:
- Connector development tutorial: CONTRIBUTING.md
- General development guidelines: CONTRIBUTING_GUIDE.md