mirror of
https://github.com/hanzo-fi/payments.git
synced 2026-10-30 20:31:20 +00:00
* fix(assets): EN-1011 align asset regex with Ledger The asset regex in internal/utils/assets had drifted from Ledger's canonical pattern (formancehq/ledger/pkg/assets/asset.go) after the Fireblocks (#639) and hyphen (#649) PRs, accepting strings that Ledger later rejects. Roll the pattern back to Ledger's exact shape: [A-Z][A-Z0-9]{0,16}(_[A-Z]{1,16})?(\/\d{1,6})? This drops support for multi-underscore identifiers, hyphens, and digit-containing suffix segments. Connectors emitting non-canonical asset strings (Fireblocks today) must canonicalize at their boundary; see the follow-up Fireblocks commit on this branch. * feat(fireblocks): EN-1011 emit canonical asset + rich metadata The connector used to surface the raw Fireblocks `legacyId` as the Formance asset string (`USDT_ERC20/6`, `MATIC_POLYGON_AMOY/18`, …), which Ledger rejects and which leaks Fireblocks-isms downstream. Rework the plugin so the canonical asset is derived from `displaySymbol` (sanitised to fit the Ledger regex) and the wealth of context Fireblocks provides ends up where it belongs: * Extend `client.Asset` to capture `displaySymbol`, `displayName`, `blockchainId`, `assetClass`, `onchain.{symbol,name,address,standards}`, `metadata.{scope,verified,deprecated,features}`. * Replace the `map[legacyId]int` cache with `map[legacyIdUpper]assetInfo` carrying the canonical asset string, precision, blockchainId, legacyId and a prebuilt `com.fireblocks.spec/` metadata slice. * `buildAssetInfo` filters NFT/SFT/VIRTUAL + deprecated, picks decimals by `assetClass` (FIAT → top-level, NATIVE/FT → onchain), and drops assets without a legacyId (vaults reference legacyIds only — UUIDs would never round-trip per Fireblocks' own guidance). * `sanitizeSymbol` / `canonicalAsset` produce a Ledger-valid asset (`1INCH` → `INCH`, `xDAI` → `XDAI`, oversized symbols truncated to 17). * `fetchNextBalances` aggregates same-canonical-asset entries inside a vault (USDT_ERC20 + USDT_TRX → one `USDT/6` balance with summed amount) and validates per-entry, dropping invalid balances with a log instead of poisoning the whole batch. * `fetchNextPayments` looks up via the cache, copies the asset's metadata slice onto the payment, renames the existing tx metadata keys to the `com.fireblocks.spec/` namespace (`tx_hash`, `network_fee`, `note`, `sub_status`, `destination_ids`) and validates per-entry. * `fetchNextAccounts` surfaces `customer_ref_id`, `hidden_on_ui`, `auto_fuel` onto `PSPAccount.Metadata` when set. Existing tests are updated to the new field name / metadata keys; expanded coverage lives in the next commit. * test(fireblocks): EN-1011 cover sanitization, metadata, and skip paths Expand the Fireblocks test surface to lock the new behaviour: * `assets_sync_test.go` (new): table-driven specs for `sanitizeSymbol` (1INCH/xDAI/punctuation/oversize/empty), `canonicalAsset`, and `buildAssetInfo` covering FIAT vs FT decimals selection, the `displaySymbol` → `onchain.symbol` → `legacyId` fallback chain, digit-prefix sanitisation, verified+features metadata, and the deprecated / NFT / SFT / VIRTUAL / no-decimals / empty-symbol skip paths. * `balances_test.go`: aggregation of USDT_ERC20+USDT_TRX into a single `USDT/6` balance with summed amount, plus case-insensitive xDAI matching. * `payments_test.go`: asset-cache metadata (legacy_id, contract_address, token_standard, blockchain_id) is copied onto each payment alongside the namespaced tx_hash / network_fee / note / sub_status fields. * `accounts_test.go`: customer_ref_id / hidden_on_ui / auto_fuel surface on PSPAccount.Metadata when present, and stays nil otherwise. * fix(fireblocks): EN-1011 fetch full transaction history on first sync Fireblocks' /v1/transactions endpoint silently defaults `after` to "the past 90 days" when the parameter is omitted. On a fresh install this caused the connector to miss every transaction older than 90 days and then advance state past the rest, never backfilling them. Always send `after`, pinning it to 1 ms past epoch on the first call. Subsequent polls keep using the cursor we track in state, so behaviour is unchanged beyond the first poll. * feat(fireblocks): EN-1011 segregate testnet assets via blockchain.onchain.test Without this, the canonicalisation step happily collapses testnet and mainnet holdings of the same symbol onto the same Formance asset. The local sandbox already proved this: a vault holding both ETH_TEST5 (Sepolia) and ETH-AETH_SEPOLIA aggregated under `ETH/18`, and the same asset would have collided with real ETH. Use Fireblocks' own authoritative signal — `Blockchain.onchain.test` exposed on `/v1/blockchains` — rather than legacyId heuristics: * Add `client.Blockchain` + `ListBlockchains` (cursor-paginated, same shape as ListAssets). * `loadAssets` now also calls ListBlockchains and builds a `map[blockchainId]bool` keyed by the blockchain UUID. The lookup is in-memory and bounded (~50 blockchains). * `buildAssetInfo` / `canonicalAsset` take an `isTestnet` bool. For testnet assets the canonical asset gets a `_TEST` suffix (`ETH_TEST/18`, `SOL_TEST/9`, …). Base is trimmed when needed so the whole identifier still fits Ledger's 17-char cap before the suffix. * `com.fireblocks.spec/testnet=true` is stamped onto the per-asset metadata slice so it propagates to every payment. * Tests: extend canonicalAsset, buildAssetInfo, and the install spec with the new mocked `ListBlockchains` call and a Sepolia entry. * fix(fireblocks): EN-1011 preserve full base length for testnet canonical assets canonicalAsset was trimming the base to 12 chars before appending `_TEST`, producing collisions between distinct long-symbol testnet assets that shared a 12-char prefix. Ledger's regex `[A-Z][A-Z0-9]{0,16}(_[A-Z]{1,16})?(/\d{1,6})?` permits a 17-char base plus the `_TEST` suffix independently — sanitizeSymbol already caps the base at 17 chars, so dropping the inner trim is safe. Addresses CodeRabbit review comment on #720. * refactor(fireblocks): EN-1011 trim comments and modernise helpers * Trim AI-flavoured / stale / coaching comments across the plugin to match sibling EE connector style (bitstamp, coinbaseprime). * Switch `buildPaymentMetadata` to `maps.Copy` and `appendUnique` to `slices.Contains` (Go 1.21+ idioms). * Drop the unused `boolStr` helper; inline `"true"` literals in account metadata, matching sibling connectors. No behavioural change. * test(fireblocks): EN-1011 lock 90-day fix, refresh, aggregation, status mapping * `client/transactions_test.go` (new): httptest-based specs pinning `after=1` on the first sync, `after=<state>` thereafter, the rest of the query shape (`limit`, `orderBy=createdAt`, `sort=ASC`), the happy-path round-trip, and the server-error wrap. * `client/blockchains_test.go` (new): two-page cursor pagination, initial request has no `pageCursor`, second carries the cursor; plus server-error wrap. * `client/client_testing_test.go` (new): shared `newTestClient` helper. * `assets_sync_test.go`: `ensureAssetsFresh` specs covering the no-op (fresh cache), first-load (empty cache), and TTL-stale refresh. * `balances_test.go`: three-chain aggregation (USDT_ERC20 + USDT_TRX + USDT_BSC -> USDT/6) and duplicate-legacyId-row aggregation. * `payments_test.go`: contract that `Metadata` is non-nil even when the cached asset has no metadata slice (so `destination_ids` writes are safe), plus a `DescribeTable` covering every documented Fireblocks status -> `PaymentStatus` mapping. Coverage deltas: ensureAssetsFresh 38.5% -> 92.3%, ListTransactions / ListBlockchains 0% -> ~92%, matchPaymentStatus 50% -> 100%, appendUnique 67% -> 100%. No behavioural change to production code. * refactor(fireblocks): EN-1011 PR #720 review feedback * accounts.go: rewrite buildAccountMetadata doc to state "non-zero" (CustomerRefID has the same treatment as the bools). * assets_sync.go: emit an error-level log when loadAssets returns 0 entries, defensive against a future Fireblocks legacyId surface change. * balances.go: switch user-visible "vault" log wording to "account", matching the Formance/Fireblocks console UX. * client/{accounts,assets,blockchains,transactions}.go: surface the Fireblocks error body in wrapped client errors via a small wrap helper on fireblocksError. * balances_test.go: add a regression spec proving same-symbol different-precision assets stay in separate balance rows. --------- Co-authored-by: Thierry Coopman <[email protected]> Co-authored-by: Cursor <[email protected]>
244 lines
6.5 KiB
Go
244 lines
6.5 KiB
Go
package fireblocks
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/formancehq/payments/ee/plugins/fireblocks/client"
|
|
)
|
|
|
|
// assetInfo is the per-asset cache entry built from /v1/assets at refresh time.
|
|
type assetInfo struct {
|
|
Asset string // canonical Formance asset, e.g. "USDT/6"
|
|
Precision int
|
|
BlockchainID string
|
|
LegacyID string
|
|
Metadata map[string]string // copied onto every PSPPayment using this asset
|
|
}
|
|
|
|
func (p *Plugin) ensureAssetsFresh(ctx context.Context) error {
|
|
p.assetsMu.RLock()
|
|
needsRefresh := p.assets == nil || time.Since(p.assetsLastSync) >= assetRefreshInterval
|
|
p.assetsMu.RUnlock()
|
|
if !needsRefresh {
|
|
return nil
|
|
}
|
|
|
|
p.assetsRefreshMu.Lock()
|
|
defer p.assetsRefreshMu.Unlock()
|
|
|
|
p.assetsMu.RLock()
|
|
needsRefresh = p.assets == nil || time.Since(p.assetsLastSync) >= assetRefreshInterval
|
|
p.assetsMu.RUnlock()
|
|
if !needsRefresh {
|
|
return nil
|
|
}
|
|
|
|
return p.loadAssets(ctx)
|
|
}
|
|
|
|
// lookupAsset resolves a Fireblocks legacyId (case-insensitive) against the cache.
|
|
func (p *Plugin) lookupAsset(legacyID string) (assetInfo, bool) {
|
|
p.assetsMu.RLock()
|
|
defer p.assetsMu.RUnlock()
|
|
info, ok := p.assets[strings.ToUpper(legacyID)]
|
|
return info, ok
|
|
}
|
|
|
|
func (p *Plugin) loadAssets(ctx context.Context) error {
|
|
blockchains, err := p.client.ListBlockchains(ctx)
|
|
if err != nil {
|
|
return fmt.Errorf("listing blockchains: %w", err)
|
|
}
|
|
testnetByBlockchain := make(map[string]bool, len(blockchains))
|
|
for _, b := range blockchains {
|
|
if b.Onchain != nil && b.Onchain.Test {
|
|
testnetByBlockchain[b.ID] = true
|
|
}
|
|
}
|
|
|
|
rawAssets, err := p.client.ListAssets(ctx)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
out := make(map[string]assetInfo, len(rawAssets))
|
|
var skipped int
|
|
for _, a := range rawAssets {
|
|
info, ok := buildAssetInfo(a, testnetByBlockchain[a.BlockchainID])
|
|
if !ok {
|
|
p.logger.Infof("skipping fireblocks asset legacyId=%q id=%q class=%q",
|
|
a.LegacyID, a.ID, a.AssetClass)
|
|
skipped++
|
|
continue
|
|
}
|
|
if a.LegacyID == "" {
|
|
// Vaults reference assets by legacyId only.
|
|
skipped++
|
|
continue
|
|
}
|
|
out[strings.ToUpper(a.LegacyID)] = info
|
|
}
|
|
|
|
p.assetsMu.Lock()
|
|
p.assets = out
|
|
p.assetsLastSync = time.Now()
|
|
p.assetsMu.Unlock()
|
|
|
|
if len(out) == 0 {
|
|
p.logger.Errorf("loaded 0 fireblocks assets (%d skipped) — Fireblocks may have changed the legacyId surface or the workspace is empty",
|
|
skipped)
|
|
} else {
|
|
p.logger.Infof("loaded %d fireblocks assets (%d skipped, %d testnet blockchains)",
|
|
len(out), skipped, len(testnetByBlockchain))
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// buildAssetInfo turns a Fireblocks Asset into the cached entry. Returns
|
|
// ok=false for assets that must not be ingested (deprecated, NFT/SFT/VIRTUAL,
|
|
// unknown decimals, sanitization yields empty symbol). isTestnet stamps the
|
|
// canonical asset with a `_TEST` suffix so testnet holdings never aggregate
|
|
// with their mainnet equivalents.
|
|
func buildAssetInfo(a client.Asset, isTestnet bool) (assetInfo, bool) {
|
|
if a.Metadata != nil && a.Metadata.Deprecated {
|
|
return assetInfo{}, false
|
|
}
|
|
|
|
switch a.AssetClass {
|
|
case client.AssetClassNFT, client.AssetClassSFT, client.AssetClassVirtual:
|
|
return assetInfo{}, false
|
|
}
|
|
|
|
precision, ok := pickPrecision(a)
|
|
if !ok {
|
|
return assetInfo{}, false
|
|
}
|
|
|
|
symbol := a.DisplaySymbol
|
|
if symbol == "" && a.Onchain != nil {
|
|
symbol = a.Onchain.Symbol
|
|
}
|
|
if symbol == "" {
|
|
// Last-resort fallback so we keep some signal; legacyId is the only
|
|
// other string guaranteed to identify the asset.
|
|
symbol = a.LegacyID
|
|
}
|
|
|
|
asset := canonicalAsset(symbol, precision, isTestnet)
|
|
if asset == "" {
|
|
return assetInfo{}, false
|
|
}
|
|
|
|
return assetInfo{
|
|
Asset: asset,
|
|
Precision: precision,
|
|
BlockchainID: a.BlockchainID,
|
|
LegacyID: a.LegacyID,
|
|
Metadata: buildAssetMetadata(a, isTestnet),
|
|
}, true
|
|
}
|
|
|
|
// pickPrecision selects decimals based on assetClass: FIAT uses the top-level
|
|
// `decimals`, NATIVE/FT/unknown fall through to onchain. Negative or absent
|
|
// values cause a skip.
|
|
func pickPrecision(a client.Asset) (int, bool) {
|
|
if a.AssetClass == client.AssetClassFiat {
|
|
if a.Decimals != nil && *a.Decimals >= 0 {
|
|
return *a.Decimals, true
|
|
}
|
|
// Some FIAT assets ship decimals via onchain too; try that next.
|
|
}
|
|
if a.Onchain != nil && a.Onchain.Decimals >= 0 {
|
|
return a.Onchain.Decimals, true
|
|
}
|
|
if a.Decimals != nil && *a.Decimals >= 0 {
|
|
return *a.Decimals, true
|
|
}
|
|
return 0, false
|
|
}
|
|
|
|
// sanitizeSymbol turns an arbitrary Fireblocks displaySymbol into a Ledger-
|
|
// valid base: uppercase ASCII letters/digits only, first char must be a
|
|
// letter, capped at 17 chars. Returns "" if no valid prefix can be derived.
|
|
func sanitizeSymbol(symbol string) string {
|
|
var b strings.Builder
|
|
b.Grow(len(symbol))
|
|
seenLetter := false
|
|
for _, r := range strings.ToUpper(symbol) {
|
|
if b.Len() >= 17 {
|
|
break
|
|
}
|
|
switch {
|
|
case r >= 'A' && r <= 'Z':
|
|
seenLetter = true
|
|
b.WriteRune(r)
|
|
case r >= '0' && r <= '9':
|
|
if seenLetter {
|
|
b.WriteRune(r)
|
|
}
|
|
}
|
|
}
|
|
return b.String()
|
|
}
|
|
|
|
// canonicalAsset assembles "<sanitized>[_TEST]/<precision>" (or the bare base
|
|
// when precision is 0). isTestnet appends a `_TEST` segment so testnet and
|
|
// mainnet assets never collide downstream. Ledger's regex permits the base
|
|
// (capped at 17 chars by sanitizeSymbol) and the suffix independently.
|
|
func canonicalAsset(symbol string, precision int, isTestnet bool) string {
|
|
s := sanitizeSymbol(symbol)
|
|
if s == "" {
|
|
return ""
|
|
}
|
|
if isTestnet {
|
|
s += "_TEST"
|
|
}
|
|
if precision == 0 {
|
|
return s
|
|
}
|
|
return fmt.Sprintf("%s/%d", s, precision)
|
|
}
|
|
|
|
// buildAssetMetadata captures the Fireblocks-side context that's worth
|
|
// surfacing alongside each balance/payment. Optional fields are omitted when
|
|
// empty / default so the map stays small.
|
|
func buildAssetMetadata(a client.Asset, isTestnet bool) map[string]string {
|
|
m := map[string]string{}
|
|
setIfPresent := func(key, value string) {
|
|
if value != "" {
|
|
m[MetadataPrefix+key] = value
|
|
}
|
|
}
|
|
|
|
setIfPresent("legacy_id", a.LegacyID)
|
|
setIfPresent("asset_uuid", a.ID)
|
|
setIfPresent("display_name", a.DisplayName)
|
|
setIfPresent("display_symbol", a.DisplaySymbol)
|
|
setIfPresent("blockchain_id", a.BlockchainID)
|
|
setIfPresent("asset_class", a.AssetClass)
|
|
|
|
if a.Onchain != nil {
|
|
setIfPresent("contract_address", a.Onchain.Address)
|
|
if len(a.Onchain.Standards) > 0 {
|
|
m[MetadataPrefix+"token_standard"] = strings.Join(a.Onchain.Standards, ",")
|
|
}
|
|
}
|
|
|
|
if a.Metadata != nil {
|
|
if a.Metadata.Verified {
|
|
m[MetadataPrefix+"verified"] = "true"
|
|
}
|
|
if len(a.Metadata.Features) > 0 {
|
|
m[MetadataPrefix+"features"] = strings.Join(a.Metadata.Features, ",")
|
|
}
|
|
}
|
|
if isTestnet {
|
|
m[MetadataPrefix+"testnet"] = "true"
|
|
}
|
|
|
|
return m
|
|
}
|