mirror of
https://github.com/we-promise/sure.git
synced 2026-09-03 22:01:16 +00:00
fd6f4ff078ea30751069e2de59f4fbdf2510c57a
3235
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Juan José Mata
|
fd6f4ff078 |
Add live AI checks to system health (#3155)
* Add live AI checks to system health Give super admins a dedicated AI status view with bounded liveness probes for LLMs, vector stores, pgvector, and embedding endpoints. Record sanitized failures in both the system debug log and Rails logger, and document the recommended local configuration.\n\nCloses #3145 * Fix AI health CI checks * Address AI health review feedback * Correct Ollama model preload guidance * Distinguish OpenAI-compatible providers * Make Ollama startup readiness explicit * Recognize Cloudflare AI endpoints |
||
|
buzzromainandClaude Opus 5
|
c26b16d36f |
chore(goals): drop copy-pasted duplicates and an unused lock-key helper (#3159)
`Account#goal_earmarked_total` and `Account#free_to_earmark` were each defined twice, byte for byte, with no `private` boundary or singleton class between the blocks — a copy-paste artifact where the second definition silently overwrote the first. Keep one copy of each. `Goal.advisory_lock_key_for` had no caller anywhere in app/; the only reference was a test asserting the dead helper was deterministic. Remove both. No behavior change. Claude-Session: https://claude.ai/code/session_01DJ1npaGEHr6t2HW1rYZdt4 Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Brandon WolfandClaude Fable 5
|
41d4db38dc |
docs(api): sync openapi.yaml with merchants import and transfer fee fields (#2961)
* docs(api): sync openapi.yaml with merchants import and transfer fee fields Regenerate the OpenAPI document from the request specs already on main (bundle exec rake rswag:specs:swaggerize). Purely additive: documents the POST /api/v1/merchants CSV import endpoint, the MerchantImportResult schema, and the transfer source/destination fee fields that were intentionally left out of #2823. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): describe merchants CSV multipart body as an object schema type: file is a Swagger 2 idiom that is invalid under OpenAPI 3.0.3; declare the multipart part as an object with a required binary file property (matching params[:file]) and regenerate the document. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Juan José Mata
|
37a65be4b8 | Fix Mintlify docs workflow (#3156) | ||
|
buzzromainandClaude Opus 5
|
721fc394ec |
fix(securities): give prefixed crypto tickers their logo back (#3151)
Security#crypto_base_asset only stripped a fiat suffix — "BTCUSD" gave "BTC" — so it answered nil for the "CRYPTO:BTC" form, which is the form every crypto integration writes: the on-chain wallets, Kraken, CoinStats and Binance all store the prefix. display_logo_url feeds that nil to the crypto branch, so none of those securities carried a logo at all. Provider::BinancePublic already parses every shape it accepts — the pair form, the prefixed pair, the bare base asset and the USD stablecoins — so this delegates instead of growing a second parser next to it. The parsing moves to a class method for that; the private instance method stays as a delegator, so its own callers and their tests are untouched. Verified across the four documented forms: CRYPTO:BTC and BTCUSD both give BTC, CRYPTO:ETH gives ETH, CRYPTO:USDT gives USDT. A logo still needs BRAND_FETCH_CLIENT_ID configured — this fixes the parsing, not the hosting requirement. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
github-actions[bot]andgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
6a9c30a172 |
Bump version to next iteration after v0.7.4-alpha.8 release (#3144)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Sure Admin (bot)
|
79c826c0e3 |
Add Sure client overview docs (#2832)
* Add Sure client overview docs * docs: address client API comments |
||
|
GFRandClaude Sonnet 5
|
0252787dc0 |
fix(enable-banking): use real merchant instead of POS terminal line for name (#2968)
* fix(enable-banking): use real merchant instead of POS terminal line for name
Some ASPSPs (e.g. BankDirekt/Raiffeisen in Austria) return
remittance_information as a multi-element array where the first line is a
generic card terminal descriptor (\"POS 45,13 AT D6 31.07. 10:27\")
and a later line holds the real merchant. EnableBankingEntry::Processor
always used the first array element, so transaction names showed the
terminal string instead of the merchant.
primary_remittance_information now skips lines that look like a technical
terminal booking (POS/ATM + amount, or a trailing date+time stamp) and
prefers the first descriptive line, falling back to the original element
when nothing better is available. It also strips known small-merchant
payment-processor prefixes (SumUp, Square, iZettle, PayPal) from the
selected line.
Fixes #2935
Disclosure: this fix was written by Claude Code, verified against the
reporter's real (decrypted) Enable Banking payload and against test-stack
Rails test / RuboCop / Brakeman runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(enable-banking): require both technical-line signals together
Address CodeRabbit/Codex review feedback on #2968: the POS/ATM+amount
prefix and the trailing date+time suffix were OR'd, so either alone
could misclassify a legitimate line as technical. A line embedding the
merchant right after the amount (e.g. "POS 45,13 BILLA DANKT ...") or
a legitimate descriptor that happens to end in a timestamp (e.g.
"Invoice paid 31.07. 10:27") would have been wrongly skipped.
Both signals are now required together in a single anchored pattern,
matching every real technical line observed in production while no
longer misclassifying either of the scenarios above.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(enable-banking): tighten date-stamp regex and clean up the merchant line
Addresses two review threads on this PR:
1. jjmata (PR review): technical_remittance_line?'s date-stamp check required
exactly 2-digit day/month (\d{2}[./]\d{2}), so an un-padded date ("1.07."
instead of "01.07.") wasn't recognized as technical and the line would
resurface as the transaction name -- reproducing the original #2935 bug for
that date shape. Now accepts 1-2 digits for both.
2. john-frandsen (issue #2935 comment): suggested cleaning up the merchant line
further (e.g. "BILLA DANKT 0007114 SIEGENDORF 7011" -> "Billa"). Checked
point 1 (structured remittance fields) against Enable Banking's own API
docs -- no such field exists there, not applicable. Points 2/3/5 already
match current behavior. Point 4 (loyalty-marker cleanup) implemented as two
layers:
- Primary: match the line against merchants the family already knows
(Family#known_merchant_names) -- self-maintaining, no pattern-guessing,
and now also assigns the transaction's merchant when matched (previously
out of scope for blank-counterparty EB transactions). Case-insensitive,
regex-escaped, longest-match-wins, with a minimum length guard against
spurious short-name matches.
- Fallback (no known merchant yet): remove only the "DANKT"/"DANKE"
thank-you marker word itself, not a directional truncation -- the marker
can precede or follow the merchant name depending on phrasing ("X DANKT"
vs. "DANKE ... bei X"), so truncating at it risked deleting the real
merchant name in one of the two phrasings.
Verified against the full test suite, RuboCop, and Brakeman on a test-stack
Rails instance (0 RuboCop offenses, 0 Brakeman warnings; full-suite failures
present on that instance are pre-existing/environmental and unrelated to
these files).
Disclosure: this fix (investigation, implementation, and tests) was written
by Claude Code.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(enable-banking): address CodeRabbit/Codex review on merchant-line cleanup
Follow-up to
|
||
|
buzzromainandClaude Opus 5
|
0d0003b032 |
fix(onchain-wallets): show linked wallets on the accounts page, and drop the tracking row on unlink (#3136)
Two gaps in #3081 after it merged, both raised there by @jjmata's review of #2191 — credit to that PR for identifying them. **Linked wallets were invisible on /accounts.** The feature creates real Sure accounts and they count towards net worth, but the accounts page never showed them: `Account.manual` excludes anything carrying a provider link, and unlike the twenty other providers there was no on-chain section to claim them. A family whose only connection was a wallet saw the empty state on the one page meant to list their accounts. The controller now loads the items, the view renders them, and the empty-state condition counts them. The card is rendered by name rather than as a collection, because the model's default partial slot is already the provider settings row; renaming it would be the tidier convention but costs eight locale files of churn for a follow-up fix. **A generic unlink left the tracking row behind.** The dedicated disconnect flow is careful, but `AccountsController#unlink` destroys the AccountProvider directly, and an OnchainWalletAccount holds that link rather than being held by it. Orphaned, it stops syncing — the syncer only reads linked rows — while its partial unique index still holds the (item, chain, address, asset) slot, so linking that same asset again would collide with a row nothing displays. It is destroyed with its link now, following the CoinStats precedent in the same model, with a guard against the recursion that precedent lacks. The regression test is the one asked for explicitly: the Sure account and its holdings survive as manual while the tracking row and the provider link go. **The card only renders accounts the viewer may see.** Found in review of this branch. An item is surfaced as soon as ONE of its accounts is accessible, so rendering them all showed a member given access to one wallet account the names and balances of the others. Reproduced before fixing, with two real addresses under one item and a member shared into only one: the unshared account's name appeared on /accounts. Non-admins now get the accessible subset and the address count derives from it; admins keep the whole item, which is the rule visible_provider_items already applies. That pattern is not specific to this card — six existing providers pass `item.accounts` unfiltered to the same partial, which filters nothing. In the same reproduction the unshared name appeared twice, once from a CoinStats card over the same accounts. Raised separately for the maintainers; only this card is changed here. Both figures are computed on the item off the preloaded associations and prepared per card by the controller, the way `_coinstats_sync_stats_map` already is: a first attempt did it in the template with a `linked` scope, which opened a fresh relation and cost a query per row. Measured on /accounts with 1 then 5 wallet items: 52->79 queries became 50->69, so 6.75 per extra item became 4.75. Verified beyond the suite: a real Bitcoin address linked, then both flows driven over real HTTP — `GET /accounts` renders the card, and `DELETE /accounts/:id/unlink` leaves the account behind reporting `manual: true`. Integration parity was checked rather than assumed: on-chain now appears everywhere CoinStats does, the nightly family sync picks it up through its own reflection over `*_items` associations, and it is already exposed to the mobile client via /api/v1/accounts and to /api/v1/provider_connections. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sure Admin (bot)
|
81c613f64a |
Improve async AI rule error reporting (#2271)
* Improve async AI rule error reporting * fix(rule-runs): preserve failures on job completion * fix(rule-runs): deduplicate failure messages |
||
|
Josh
|
ca66346dc5 |
feat: add safe admin user removal (#3131)
* feat: add safe admin user removal * fix: address user removal review findings * fix: close remaining user removal review gaps * fix: handle deleted users during session creation * fix: fail closed when session creation fails * fix: reject token issuance for inactive users |
||
|
|
735b62d9c7 |
Track self-custody wallets natively: Bitcoin, EVM and Solana (#3081)
* feat(onchain-wallets): foundation for self-custody wallet tracking
Adds the schema, models and the normalised contract every chain will
produce, with no chain implemented yet.
The central constraint of a multi-chain integration is that `case chain`
must not spread through the importer, processor, controller and views.
So a chain adapter's only job is to turn an address into an
Onchain::Snapshot — a list of Onchain::Assets and Onchain::Movements —
and Onchain::Chains is the single source of truth for which chains exist,
how their addresses are validated, what their native asset is, and which
adapter to instantiate. Everything downstream is written once.
Two tables:
- onchain_wallet_items: the family-level connection. Keyless by
default; the only credential is an optional Etherscan key, encrypted
via `encrypts`.
- onchain_wallet_accounts: one row per asset, per address, per chain.
Uniqueness uses three partial unique indexes, one per asset kind, because
the identity of an asset depends on its kind: a native coin is identified
by its address alone while a token is identified by its contract/mint. A
single index over all columns would treat two native rows with a NULL
contract_address as distinct and let duplicates through — the model test
proves the rejection comes from the database by saving with
`validate: false`.
The schema also leaves room for extended-key (xpub) wallets later: an HD
wallet is a set of derived addresses under one item, which the current
uniqueness key already allows, so adding it needs no destructive change.
db/schema.rb is hand-edited to add only the two new tables: regenerating
it with the Rails version now in the Gemfile reorders every column in the
file, which is out of scope for this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): chain-agnostic importer, processor and syncer
The whole pipeline is written once here and never branches on chain: it
consumes Onchain::Snapshots, so a chain is only ever a registry key and an
adapter.
Importer: refreshes every tracked address and records a digest of what it
saw (quantity plus movements, no timestamps) on each row. It deliberately
never creates rows — real wallets are full of spam airdrops, so a newly
seen token becomes trackable only when the user ticks it. An asset that
disappears from a wallet goes to zero rather than going stale.
Syncer: reprocesses only the rows whose digest changed. Two consecutive
syncs of an idle wallet write nothing at all — no row updates, no
holdings, no entries, and no queued account syncs. Both the importer and
syncer tests for that were checked against the pre-fix behaviour: with the
content_hash guard removed they fail.
Processor: writes the holding, the account balance and the movements.
Movements materialise two ways. When that day's price is known, a signed
trade (positive = Buy, negative = Sell) so cost basis and the value chart
reconstruct back to acquisition. Otherwise a display-only entry with
amount 0 and excluded: true, raw movement preserved in `extra` — visible,
but not inventing a value that would distort the account's history. Prices
are matched on the exact day for trades, because valuing a two-year-old
transfer at today's price would fabricate a cost basis.
Onchain::SecurityResolver binds a "CRYPTO:<SYMBOL>" ticker straight to the
crypto price provider instead of going through provider search, where
"USDC" can come back as a EUR-quoted pair and then need FX to repair. It
reuses an existing Security for the ticker whatever its MIC, so one asset
never splits into two records. Symbol normalisation for bridged
stablecoins and the price backfill follow in the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): canonical asset symbols and price backfill
Two things stood between a linked wallet and a correct valuation.
Bridged and wrapped variants. The same dollar is USDC on Ethereum, USDC.e
on Arbitrum and USDbC on Base; the same ether is ETH natively and WETH
once wrapped. Left alone each variant becomes its own Security, so one
gets priced and the others sit at zero, and the same asset held on two
chains reports two different values. Onchain::AssetSymbol maps the
variants that are redeemable 1:1 onto the canonical asset, so pricing them
as that asset is exact rather than approximate.
Missing price history. A Security created at link time has none, so on the
first sync every movement would fall back to a display-only entry and the
cost basis would never reconstruct — the feature would look broken exactly
when the user first looks at it. The processor now backfills the window in
one batched provider call rather than one call per movement date, includes
today so a wallet whose movements all predate the sync window still gets a
current valuation, and treats a provider failure as a logged warning: the
holding and the quantity are still written.
All of it is a no-op when no crypto price provider is enabled, which is
the case the settings panel has to warn about before linking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): Bitcoin adapter
Bitcoin has no account balances: an address owns unspent outputs, so the
balance is everything ever paid to it minus everything spent from it, and
mempool totals count — a broadcast-but-unconfirmed spend has already left
the wallet as far as its owner is concerned. Movements are the net effect
of each transaction on the address, so a self-transfer nets to zero and
produces no entry.
All three address formats are accepted (Base58 P2PKH/P2SH, bech32
segwit, bech32m taproot) with the character-set exclusions each encoding
actually has, and a malformed address is rejected before any request is
made — the test relies on WebMock failing the run if a request escapes.
Single address, not extended keys. A Bitcoin wallet is normally an HD
wallet: one xpub derives thousands of addresses and change goes to derived
ones, so tracking a single address under-reports such a wallet. Extended
keys would need BIP32 derivation (a dependency this codebase does not
want) or a descriptor-indexing backend. The limitation is stated in the
adapter, will be stated in the linking UI, and an xpub is rejected as an
address rather than silently treated as one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): EVM adapter for six networks, two backends
One adapter class serves every EVM network: a network's identity — label,
native coin, explorer URL, whether a family-supplied Etherscan key applies
— is data in the chain registry, so adding a network is an entry there
rather than a branch here.
The unit tracked is the (chain, address) couple, carried by the unique
index. A 0x address is valid on all six networks and holds different
balances on each, so detection asks each candidate network whether the
address is worth tracking there. That probe is exactly one request and
never reads paginated history: Blockscout's address summary carries the
coin balance and the token/transfer flags together, which is also why a
wallet holding only ERC-20 tokens with zero native balance is still found.
An explorer being down means "not detected here", not an error the user has
to interpret — a dead indexer must not break linking.
Two interchangeable backends behind Provider::EvmExplorer: keyless
Blockscout by default for every network, and Etherscan when the family
configured a key on a network the registry enables it for (today Ethereum
only), where a key buys nothing but a higher rate limit. Etherscan
deliberately does not implement the activity probe: its one-request answer
is the native balance, which reports "nothing here" for a token-only
wallet, so detection stays on the indexer that can answer correctly.
Zero-balance token rows are dropped — real wallets are full of spam dust —
while the native asset is always reported, even at zero, because a wallet
that spent everything still has a history worth keeping.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): Solana adapter
A Solana wallet does not hold its tokens. Each SPL token sits in its own
token account, owned by the wallet but addressed separately, so balances
come from enumerating those accounts across both token programs rather
than from reading the wallet address — and because one wallet can own
several accounts for the same mint, they are summed into one position.
Emptied token accounts are left behind on chain by design and are dropped.
RPC gives mints, not metadata. Well-known mints get their real symbol;
anything else is labelled with its mint in a form that deliberately cannot
pass for a ticker, so security resolution declines it and the asset is
tracked by quantity rather than priced as some unrelated coin that happens
to share a name.
Activity is one request. Bitcoin's Base58 addresses fall inside Solana's
address shape, so the two are told apart by asking the node: it rejects a
non-32-byte key, which reads as "not here" rather than an error.
The Snapshot it returns has the same shape as Bitcoin's and the EVM
adapter's — asserted in the test — so nothing downstream changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): settings panel and linking flow
Linking is three steps: paste an address, confirm which network, choose
what to track.
The network step exists because address formats are not unique to a chain.
Candidates come from the address shape, and when there is more than one each
is asked whether the address is worth tracking there — one bounded request
each. When several answer yes, or none does, the user picks from a list that
marks which ones showed activity. Silently keeping the first match would
link the wrong network and surface later as a sync bug.
The token step imports nothing that was not ticked. Assets whose symbol a
price provider can quote are pre-checked; spam airdrops, whose "symbol" is
usually an advertisement, are listed unticked and can still be tracked by
quantity. Quantities and metadata are re-read from the chain when the
selection is applied, so a tampered selection can only change which assets
are tracked, never what they claim to hold. Previewing an address creates
no connection record, so an abandoned flow leaves nothing behind.
The panel and both modal steps carry a price-provider warning: with no
crypto market data enabled every wallet is valued at zero, which users
report as a broken sync rather than a missing setting, so it is said before
linking and links to where to fix it. The Bitcoin single-address limit and
"never enter a seed phrase" are stated in the linking UI too.
Errors are separated by kind. A rate limit or an unreachable explorer gets
its own localized message. Anything else is a bug: the user sees a generic
message and the class and message go to DebugLogEntry, never into the
response — asserted in the tests, which also check the exception text does
not appear in the body.
Adapters now translate their data source's errors into Onchain::Chains
errors, so the controller rescues two chain-agnostic types instead of
carrying a list of every explorer's error classes — which would have put
per-chain knowledge back into the controller.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): manage, review tokens, change address, disconnect
Four actions, because two would make the token choice irrevocable.
Review tokens reopens the selection screen with the address left alone —
deliberately without an address field. Without it the only way to untick a
token would be to change the address, which is a different operation
entirely. Assets the chain no longer reports stay listed and ticked, so
they can be dropped once they are gone.
Disconnect one asset is a per-row action with a button next to every asset.
A destroy route no view calls is dead code: the feature does not exist
until it has a button, so the test asserts one form per asset rather than
one per wallet.
Change address updates the existing rows instead of recreating them, so the
accounts, holdings, entries and balance history all survive — verified by
asserting a pre-existing balance is still there afterwards, and checked
against the recreate-instead-of-update behaviour, which fails it. The
content digest is cleared so the next sync reprocesses even if the new
address happens to hold the same amount, and an account still carrying its
generated name is renamed to match.
Disconnect wallet drops every asset at one address and leaves other
addresses alone.
Disconnecting never destroys an account: the provider link goes, holdings
are detached, and what the user can see stays as a manual account that
stops updating — the same contract every other provider's unlink has here.
The duplicate-address guard covers initial linking as well as address
changes. Without it, re-linking an already-tracked address would become the
unofficial way to add a token, quietly creating a second set of rows for
the same wallet; both guards are checked against that pre-fix behaviour.
Every lookup and mutation is scoped through Current.family, including the
per-row disconnect, which cannot reach a row belonging to another
connection.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(onchain-wallets): hosting guide for self-custody wallet tracking
docs/hosting/onchain-wallets.md covers what the feature reads, which
endpoint serves each network and how to point it at your own instance, the
optional Etherscan key and why it is optional, the per-sync request cost and
where history is capped, and the management actions.
Two things get stated plainly because they generate the support traffic:
prices come from a separate market data setting, so with no crypto-capable
provider enabled every wallet is tracked by quantity and valued at zero; and
Bitcoin is one address at a time, which under-reports an HD wallet whose
funds are spread across derived addresses.
Every locale key the feature uses is checked to resolve, including the
pluralised ones.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): keep EVM balances on the keyless indexer when a key is set
Configuring an Etherscan key moved every read onto Etherscan, including
balances. That was wrong: Etherscan has no free endpoint that enumerates an
address's tokens, so its token balances are summed from transfer history —
which cannot see a rebasing token's current balance, and silently
under-reports any wallet whose history exceeds the page cap. A user adding a
key to fix a rate limit would have quietly traded it for wrong balances.
The two reads are now separated by what each backend can actually answer.
Balances and activity detection always go to the keyless indexer, whose
address summary answers both in one request — so adding a key buys nothing
there and cannot cost anything either. History, the paginated and
rate-limited half, is where a key helps and is the only thing it changes.
Provider::Etherscan drops its balance methods and refuses token_balances
with the reason, rather than offering an approximation that reads as a fact.
The chain-agnostic error mapping now accepts several error families per role,
since one snapshot can involve both backends.
Checked against the pre-fix behaviour: with balances routed through the
keyed backend, the new test reports 1 USDC held instead of the 7 the indexer
sees, and Etherscan raises on the balance call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): name verified SPL tokens so they can be priced
Solana RPC returns mints, not names, so every SPL token outside the
hard-coded handful showed up as "SPL:abcd…wxyz" — which security resolution
correctly declines, leaving the asset tracked by quantity and valued at zero.
For a wallet holding anything beyond USDC that was most of its value.
Names now come from Jupiter's keyless token search, asked once per snapshot
for every mint at once, cached 24 hours per mint.
Only mints the list reports as *verified* are trusted. Anyone can mint a
token calling itself USDC; naming an unverified one would hand it the real
dollar's price and value dust at thousands. Unverified and unknown mints keep
the placeholder, and the test for that asserts the spam token is not named
USDC. Misses are cached as well, because spam wallets hold many mints that
will still be unvouched-for tomorrow.
Metadata is a naming nicety, not the wallet: a list that is down or rate
limiting degrades to placeholders instead of failing the snapshot, and the
hard-coded mints stay as an offline floor. Assets and movements resolve from
the same lookup, so a token cannot be called two different things within one
snapshot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): enable crypto pricing in one click, and log zero valuations
The warning said what was wrong and where to fix it, but still made the user
navigate to another settings page and pick the right provider out of a list.
On a self-hosted instance an admin can now fix it from the warning itself:
the crypto provider is appended to the enabled securities providers, leaving
the others alone — enabling crypto prices must not turn off whatever prices
the user's equities, and the test fails if it does. On a managed instance the
providers are the operator's setting, so the button is not offered and the
action refuses.
The other half is diagnosis. Until now a holding valued at zero for this
reason looked exactly like a holding whose price simply had not been fetched
yet: nothing recorded it, so support had to infer it from a screenshot. The
processor now records it via DebugLogEntry — but only for this cause, since a
price merely missing for today is ordinary and already covered by the
backfill.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): upgrade display-only movements once their price is known
A wallet linked before market data covered its history got the worst of both
worlds: its transfers landed as zero-amount excluded entries, and nothing ever
brought them back. The syncer only reprocesses assets whose on-chain state
changed, and a two-year-old transfer never changes — so the cost basis stayed
broken for exactly the wallets that were linked earliest.
perform_post_sync now runs a repair over every linked asset, not just the ones
that moved: what changed is the price history, which no chain read can report.
It reads prices from the database only, makes no network call, and does
nothing when there is nothing to upgrade.
An upgraded entry keeps its external_id, so it is the same transfer rather
than a duplicate. The entry is destroyed and rewritten because an Entry cannot
change entryable type in place and import_trade refuses an id already held by
a Transaction. Entries from other providers are never touched — matched by
source and by this asset's own id prefix.
Checked against the pre-fix behaviour: with perform_post_sync empty again, the
syncer test fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(onchain-wallets): make history truncation visible and its depth configurable
History was capped silently. A wallet with more transfers than one sync reads
looked exactly like a wallet whose history was fully imported — the only trace
was a Rails log line on Bitcoin, and nothing at all on the EVM and Solana
paths. A user reconciling their cost basis had no way to tell the difference
between "this is all there was" and "we stopped reading".
Each source now reports whether it stopped on the budget or on the end of the
history. That travels on the Snapshot, so it stays chain-agnostic; the flag is
recorded on the affected rows, the manage screen says the history is
incomplete for that address, and the importer records it once per address per
sync — only when something changed, so an idle wallet with deep history does
not log the same line every night. The message also states what is not
affected: balances come from an address summary, never from history.
The depth itself is a hosting decision, not a property of a chain, so it moves
out of the providers into Onchain::HistoryBudget and is settable with
ONCHAIN_HISTORY_MAX_PAGES (default 10, clamped to 200). Adapters inject it, so
the providers stay unaware of the policy and the budget is testable without
touching the environment.
What this does not do is resume where it stopped. Reading older history across
syncs needs a per-wallet cursor and changes what the content digest means —
which is what guarantees an idle wallet writes nothing — so it is a change of
its own rather than a rider on this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): read Blockscout token balances the way the API accepts
Verified against the live API: /api/v2/addresses/{address}/token-balances
rejects a `type` filter with HTTP 422 — the filter exists on token-transfers,
not here. Every EVM snapshot was therefore failing in production and
surfacing as "the explorer could not be reached", while the tests passed
because the stub encoded the same wrong assumption. No unit test can catch a
mistaken API contract; only asking the API can.
Without the filter the endpoint answers with every token standard the address
holds — ERC-20, ERC-721, ERC-1155, ERC-404 — so ERC-20 is now selected
client-side. An NFT row carries value "1" and no decimals, so it would have
been imported as a fungible balance of one token, priced by whatever its
symbol happened to resemble.
The tests now stub the endpoint the way it really answers and assert we ask
without a query, so re-adding the filter fails the run. Each token's
market-cap signal is carried through for the next commit, which has to decide
what to do about an address holding thousands of tokens.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): bound how many tokens one address can surface
Probing the live APIs turned up the other thing the stubs were hiding: real
addresses are airdrop dumping grounds. A well-known Ethereum address returns
7,924 ERC-20 balances (3.2 MB), and a comparable Solana one 2,801 token
accounts. Nothing bounded either. That meant a review screen with thousands of
rows nobody can use, and on Solana — where names are looked up in batches of
50 — around 56 extra requests per sync just to label an airdrop dump.
One read now surfaces at most 200 tokens per address
(ONCHAIN_MAX_TOKENS_PER_ADDRESS, clamped to 5,000). The native coin is never
affected, and anything already tracked keeps syncing regardless of the cap.
What survives the cap has to be both sensible and stable. On EVM the tokens
are ranked by the market cap the indexer already reports, so real assets stay
and airdrops fall off the end. Solana RPC gives no such signal, so the order
is the mint address: arbitrary, but identical between two reads of an unchanged
address — an unstable order would reshuffle the wallet, change the content
digest, and rewrite holdings every night. There is a test for that on both
chains. On Solana the cap is applied before metadata lookup, so it bounds the
requests as well as the rows.
The cap is not silent: it is recorded on the affected rows, stated in the token
review screen and in Manage wallets, and reported alongside history truncation
in the debug log with the limit that applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(onchain-wallets): walk the linking and management flow in a browser
The linking flow stacks three Turbo frame navigations — the provider drawer,
the linking modal on top of it, then the token review in that same modal —
and no controller test exercises any of that. Driving it in a real browser
found two things worth keeping:
The panel is reached two different ways. With nothing linked yet it is a card
under "Available" that opens the drawer; once a wallet exists the provider
moves to "Your connections", where the panel renders inline inside a
disclosure and there is no link to click at all. Only the first path had ever
been exercised.
Both paths now are, along with the parts of the flow that only exist in a
browser: assets arriving pre-ticked or not according to whether they can be
priced, review tokens reopening the selection with no address field present,
and disconnecting one asset leaving its account behind. One test per flow —
the branching cases stay in the controller test, where they cost a fraction
of the time.
Verified visually at each step: the warning banner, the Bitcoin
single-address note, the three-asset review with the spam token unticked, the
four management actions, and the accounts landing under Crypto with the wallet
subtype.
Full system suite green the documented way (DISABLE_PARALLELIZATION=true):
94 runs, 385 assertions, 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): report a timed-out data source as unreachable
Reading a real Solana wallet turned up the hole: the public RPC timed out,
Net::ReadTimeout escaped every layer untranslated, and the user was told
something had gone wrong with Sure — the generic message reserved for our own
bugs — with a DebugLogEntry filed as if it were one. A public endpoint being
slow is the most ordinary failure this feature has.
All five clients now translate transport failures into their own ApiError:
timeouts, refused or reset connections, unreachable hosts, TLS errors, and a
response that is not JSON. The adapters already map a provider's own errors
onto Onchain::Chains::UnreachableError, so a timeout now reads as "the public
explorer could not be reached", the message that tells the user to retry,
while genuine bugs keep the generic one. During chain detection it goes back
to meaning "not detected here", so a slow explorer still cannot break linking.
The translated message carries only the error class, never the original
message, because a transport error's message contains the full URL and these
get logged.
Checked against the pre-fix behaviour on Bitcoin: with the translation
removed, the timeout test fails with the raw exception instead of the chain
error.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): name on-chain trades after the asset, not "shares"
Running a real Bitcoin address through the app showed every imported transfer
as "Buy 0.000003 shares of CRYPTO:BTC". That name comes from the shared trade
helper, which is written for equities; a wallet does not hold shares, and the
internal ticker is not what the user calls the coin. Trades are now named
"Buy 0.000003 BTC", through i18n like the display-only entries already were.
Also drops the sync status_text calls. Sync has no such attribute in this
schema, so every one of them was a guarded no-op and the four locale strings
they referenced could never render — dead weight copied from another
provider's syncer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): keep balances when a source refuses history
Reading real wallets on the free Solana endpoint exposed two problems, one of
which I introduced.
The budget. Making history depth configurable scaled Solana's transaction cap
from 25 to 250, and on Solana one transaction is one RPC call rather than one
page — so the same nominal depth became an order of magnitude more expensive
and a sync went from seconds to minutes. The per-transaction budget is now its
own constant, scaled proportionally off the page knob so one setting still
moves both, with a test that pins it far below the paginated row count.
The bigger one: a failure while reading history threw away the balances too.
A balance is one bounded request and is what a wallet fundamentally is; history
is paginated, far more expensive, and the first thing a throttled endpoint
refuses. On the free Solana endpoint, which routinely throttles getTransaction,
that meant a wallet showed nothing at all rather than showing what it holds —
permanently, not transiently.
History is now best effort: when the data source refuses it, the balances are
still recorded and the history is marked incomplete, which the manage screen
already surfaces. Anything that is not the data source failing still raises, so
a bug here cannot be swallowed. Verified live: the same Solana wallet that
failed entirely now reports 52.06 SOL and its SPL tokens with the history
flagged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): pre-tick what is worth something, not what parses as a ticker
Linking a real airdropped address showed 200 of its 201 assets arriving
pre-ticked, one click away from 200 accounts — the opposite of what reviewing
tokens before import is for. The pre-tick rule was "the symbol looks like a
ticker", and airdrops use perfectly plausible short symbols (0XBTC, 4CHAN), so
it selected nearly everything.
Assets now carry whether the data source treats them as notable, and only those
are pre-ticked. Two attempts at that signal, decided by measuring the real
address rather than guessing:
- Blockscout's `reputation` is "ok" for all 6,669 tokens it holds. Useless.
- Market-cap presence looks strong on the full list (365 of 6,669) but is
useless after the cap, which already ranks by market cap — hence every
surfaced token having one.
- Holding value discriminates: of the 365 priced tokens, 112 are worth more
than a dollar.
So on EVM networks a token is notable when the indexer can price it and the
holding is worth more than a dollar; on Solana when the verified token list
vouches for the mint; the native coin always. Pre-ticked count on that address
drops from 200 to 72, and everything else stays one click away.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): warn when nothing can convert USD prices into the family currency
A family whose currency is not USD needs two settings, not one, and only the
first was covered. The single provider that prices bare crypto symbols quotes in
USD, so valuing a wallet in EUR needs an exchange rate on top — and Sure's
default exchange rate provider requires an API key, so a self-hosted install
without one has no FX at all.
Tested against a real address in a EUR family: every wallet came out at zero,
with 275 transfers recorded as unpriced, and nothing anywhere said why. That is
the same support ticket the crypto-provider banner exists to prevent, arriving
through the other door.
Onchain::Pricing answers "can an on-chain asset be valued in this currency, and
if not, why" with the two reasons separately. The linking UI states whichever
applies — naming the currency for the FX one, and pointing at Frankfurter, which
needs no key — and a USD family never sees that warning because it needs no
conversion. The processor records the reasons when a holding lands at zero, so
support sees "exchange_rate" rather than guessing.
Verified live afterwards: with EXCHANGE_RATE_PROVIDER=frankfurter, the same EUR
family values the wallet at 416,198,342.25 EUR at 0.86363 USD/EUR, with all 275
transfers priced in EUR.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): let a movement become a trade after being display-only
Syncing a real wallet twice — once before its prices were reachable, once after
— raised ArgumentError from the shared importer: an Entry cannot change
entryable type in place, and the display-only Transaction already held the
external_id the trade needed.
That is the ordinary case, not an edge one. A wallet linked before market data
covers its history gets display-only entries on the first sync, and the first
sync that can price them dies. Worse, it dies inside perform_sync, so the repair
pass that exists precisely to upgrade those entries — and which runs in
perform_post_sync, afterwards — was never reached. The account stayed stuck.
Both paths now go through one writer that discards a stale display-only entry
before the trade takes over its identity, so the entry keeps being the same
transfer rather than becoming a duplicate. Checked against the pre-fix
behaviour: without the discard, the new test raises the original ArgumentError.
Found by running a EUR family through two syncs on real data, which is the only
way the two price states occur in order.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): write a trade and drop its display-only entry atomically
Replacing a display-only entry with the trade it became is one change, but it
was two writes outside a transaction. A failure between them left the account
with neither: the transfer disappeared until some later sync happened to rewrite
it, which for an idle wallet could be never.
The repair pass already wrapped this; the sync path did not, and that is the one
that runs first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): tell apart two transfers of one token in one transaction
A token transfer was identified by its transaction hash and contract, which are
not unique together: a swap router or a batch payout routinely emits several
transfers of the same token involving the same address within one transaction.
The second overwrote the first, so a transfer disappeared from the account
without a trace.
Both EVM backends report the log index — the field that makes an event unique
inside a transaction — and neither was using it. It is now part of the
identifier, with the contract kept only as a fallback for an instance that does
not report one.
Reported by review on #3081; confirmed against the live Blockscout payload,
which carries log_index. The test fails against the previous identifier.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): canonicalise an address per chain before anything keys off it
Addresses were only stripped, never canonicalised, and what counts as the same
address differs by chain — so the duplicate guard could be walked straight past
and one Bitcoin case was worse than a duplicate.
On EVM, hex is hex: 0xABC… and 0xabc… are one wallet, but they linked as two,
each with its own accounts and holdings for the same balance.
On Bitcoin, bech32 is case-insensitive and canonically lowercase, and the API
reports outputs that way. An uppercase bech32 address passed validation, gave a
correct balance from the address summary, and matched no output at all — so the
wallet silently had zero movements and no cost basis, with nothing anywhere
saying why.
Canonicalisation is now the adapter's answer, since only the chain knows whether
case carries identity: EVM and bech32 fold, Base58 and Solana are left exactly
as given. The controller applies it as soon as the chain is known and before the
duplicate check, and on an address change too.
Reported by review on #3081. Both tests fail without the folding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): stop folding the case of Solana mints
Contract identifiers were downcased everywhere on the assumption that a contract
address is hex. That holds for ERC-20 but not for an SPL mint, which is a Base58
public key where case is part of the value: the stored mint was an unusable copy
of the real one, and two distinct mints could collide once folded into the same
string — one wallet's balance landing on the other's row.
Whether case carries identity is a property of the token kind, so it is now
answered in one place and applied consistently: by the asset's identity key, by
the column, and by the two comparisons in Onchain::Snapshot. A Movement no longer
folds anything on its own, because a movement does not know its asset's kind.
That also removes the duplication review flagged: the asset key was derived in
three places and only one of them folded, so the review screen and the linker
could disagree about what identifies an asset. There is one definition now,
OnchainWalletAccount#asset_key.
Reported by review on #3081. The test fails with the unconditional fold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): isolate a failing address, and let a late symbol land
Two problems in the importer, both reported by review on #3081.
One unreachable address took the whole connection down with it: the loop over a
family's addresses had no rescue, so a Bitcoin explorer being throttled left an
untouched Ethereum wallet unsynced as well. Each address is now recorded and
skipped on its own — a row we failed to read keeps its previous quantity rather
than being zeroed, since we did not learn that it holds nothing — and only a
connection whose every address failed is reported as a failed sync rather than a
quiet success.
The content digest covered quantity and movements but not the metadata written
alongside them, so a Solana mint that later gained a real symbol from the token
list produced the same digest as before: the row was never rewritten and its
placeholder label became permanent. That silently undid the token-naming work.
The digest now covers everything the update writes.
Both tests fail against the previous behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(onchain-wallets): use DS::Button for the wallet actions
Five hand-built button_to controls carried their own utility-class strings for
what DS::Button already does — sync, disconnect a connection, stop tracking one
asset, disconnect a wallet, enable crypto prices. They drifted from the design
system on size, hover and destructive treatment, and the confirm prompts were
wired by hand.
The browser test walks the two that matter, so the behaviour is unchanged; this
is the styling and the confirm handling moving to the component that owns them.
Reported by review on #3081, along with the raw `bg-amber-600` on the provider
badge — left as it is, deliberately: all 23 other providers set a raw palette
class for their badge, there is no functional token for a brand colour, and
changing one entry would make it the only different one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): four smaller findings from the review on #3081
**A failed read no longer leaves an empty connection.** `link_wallet` created the
family's connection before fetching the address, so an explorer being down left a
connection with no wallets showing in the panel as connected. Reading needs no
saved connection — which is exactly why previewing uses an unsaved one — so it is
now created only once the read succeeds.
**Assets that could not be tracked are named instead of blamed on the user.**
`success?` is `created.positive?`, and the only failure message was "pick at least
one asset to track" — so a user who ticked three assets and hit three failed
creates was told to tick something, sending them back to tick the same three.
The linker already collected which assets failed; the message now says so, and a
partial failure is reported alongside what did get tracked instead of being
dropped silently. Same fix in the token revision action, which had the same shape.
**A malformed stored amount costs its movement, not the asset.** `BigDecimal()`
on a stored payload raises, and one unparseable row would fail the whole asset's
processing — including the repair pass. These payloads are written by this code,
so it takes a row that survived a format change, but the containment is two lines.
**No dead-end link.** The warning offered "open market data settings" to
everyone, while that page is gated to self-hosted admins — the same gate the
enable button already had. Both controls are now behind it, and the warning text
stands on its own for everyone else.
Each has a test that fails against the previous behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): reject non-finite amounts and quantities
The guard added an hour ago for malformed stored amounts was incomplete for the
very case it was written for: BigDecimal parses "NaN", "Infinity" and
"-Infinity", and none of them is zero, so all three sailed through
`parse_amount` into trade materialisation. Verified rather than assumed — the
test fails without the check with PG::NumericValueOutOfRange, so the infinity
reached the insert.
Fixed at both layers that write numbers. Movements: a non-finite amount skips its
movement, like any other unparseable one. Quantities: `normalize` treats
non-finite as unknown and writes zero, because Postgres numeric stores NaN
happily and one NaN quantity would turn every total that reads it into NaN.
Reported by review on #3081.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): derive the content digest from what actually gets written
Third time the digest missed a field it was supposed to cover: first the symbol,
so a Solana placeholder that gained a real name was never rewritten; now the
truncation flags, so a wallet whose history became complete — or started being
capped — kept showing the old completeness in the UI, since the early return
fires before `extra` is touched.
Rather than add a third field to a hand-picked list, the digest is now taken from
exactly the attribute hash that is about to be written. The two cannot drift
apart, which is what kept going wrong.
That needs two things to hold. Movements are sorted before the payload is built,
so the order a source happened to list them in is not mistaken for a change. And
the hash is canonicalised before hashing, because jsonb does not preserve key
order: `extra` written as {history, assets} comes back as {assets, history}, and
hashing that raw made an idle wallet's digest flip every other sync — the tests
caught it, and it is now covered by one asserting that reordered movements are
not a change.
Reported by review on #3081. Both halves checked against the pre-fix behaviour:
without the flags two tests fail, without the canonicalisation three.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(onchain-wallets): state the pricing coverage limit, and that DeFi is unseen
The hosting guide explained how to configure prices but never said what the
crypto provider actually covers. It quotes by symbol, and a symbol is not a
token's identity — measured on a real Ethereum address, two of its ten largest
token positions were quoted and the other eight, including holdings worth
roughly $406k, $141k and $74k, showed zero with their quantities tracked
correctly.
That reads as a broken sync unless it is written down, so it is now the first
limitation in the list, with the practical rule a user needs: a zero next to a
token you know is worth something means the provider does not list it, not that
the balance is wrong. Troubleshooting gains the matching entry, separating it
from the two configuration causes that produce a zero across every wallet.
Also records that DeFi positions — staked ETH, LP tokens, lending, Solana stake
accounts — are invisible, which was missing from the list entirely. A wallet
holding most of its value in a staking protocol reports a fraction of it.
Docs only; no code touched, so the suite was not re-run — the last run on this
tree was green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): do not zero an asset the token cap never reached
A tracked asset missing from a snapshot was always read as "the wallet no
longer holds it" and set to zero. That is right for a complete read, and wrong
for a capped one: at most 200 tokens per address are surfaced, so a tracked
token can be absent simply because the read stopped before reaching it. Its
balance was then wiped — a real holding removed from the user's net worth, with
a holding of zero written and the account balance reduced to match.
It is reachable: on Solana the surfaced set is ordered by mint address, which is
arbitrary, so a genuinely held USDC position on an airdropped wallet can fall
outside the cap and be zeroed on the next sync.
Absence of evidence is not evidence of absence — the same distinction this code
already makes for an address it could not read, where the row keeps its previous
quantity. When the snapshot is capped and the asset is not in it, the row now
keeps what was last known and only its completeness flags are refreshed.
Reported by review on #3081. The test fails against the previous behaviour, and
the "an asset that disappeared is set to zero" case still passes: a complete read
that no longer lists an asset still zeroes it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(onchain-wallets): rename the local `token` variables the secret scan trips on
Pipelock failed the PR with eleven "Credential in URL (high)" findings, all on
the same shape: a local variable named `token` being assigned. `token = ...`
is what a leaked credential looks like to a scanner, and the rule is a
reasonable one to keep.
Renamed rather than excluded. Excluding paths or adding `# pipelock:ignore`
would have kept the pattern and blunted the check for everyone; the new names
are at least as clear — `token_data` for the raw hash from an indexer,
`metadata` for the resolved symbol/name pair, `token_asset` for an
Onchain::Asset in tests.
Verified with the scanner itself rather than by inference: the same pipelock
2.8.0 the workflow pins, run locally over the branch diff, now reports "No
secrets found in diff". It also caught one site the CI log had truncated away
(the system test), which is why the local run was worth setting up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): bound detection latency, and three review findings
Detection runs on the request thread but asked each candidate chain with a
sync's patience: a 30s timeout, three retries and exponential backoff, per
chain. A 0x address is a candidate on six networks, so one rate-limited
explorer could hold the page for minutes. Detection now reads with its own
budget - one short attempt, no retry, ONCHAIN_DETECTION_TIMEOUT to raise it -
while syncs keep the patient one. A chain that cannot answer in time is
reported as "no activity", which is the screen an ambiguous answer already
produces.
Also from review:
- A full page of Solana signatures is now reported as incomplete history.
The adapter passes no cursor, so a page that fills means the read stopped
short of the address's history; counting only against the transaction
budget called that complete.
- A reused CRYPTO: security with no price provider is bound to the crypto
one. A blank provider falls back to whichever is enabled first, and only
the crypto provider quotes a bare coin symbol, so the holding valued at
zero and read as a broken sync. A provider another integration chose is
left alone: the CRYPTO: prefix is shared with the exchange integrations.
- The chain select's label reached the form builder as an HTML attribute
instead of a label option, so no <label> was associated with the field.
Each regression test was checked against the pre-fix code: the two detection
tests fail with four requests instead of one, and the truncation test fails
by reporting complete history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): let the table refuse a token row with no contract
The partial unique indexes key a token on its contract address, and NULLs
are distinct to Postgres, so a token row that reached the table without one
would slip past its index and duplicate freely. The model already refuses
it, but a direct write does not go through the model, and this repo puts
simple guarantees like this in the database.
Added to the existing migration rather than a new one: the table is created
by this branch, so the constraint belongs with it, and this leaves the
schema version untouched.
The regression test writes with validate: false, which is how the sibling
tests prove a guarantee comes from the table rather than from the model
above it. Without the constraint it fails with "expected but nothing was
raised".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(onchain-wallets): detect a token-only Solana wallet, and pin asset_kind
Two findings from the review of the previous commits.
Detection read only the wallet's lamports, so a wallet emptied of SOL but
still holding SPL tokens answered "nothing here". Each token account carries
its own rent, so an empty wallet address is not an empty wallet. The token
accounts are asked only once the balance comes back zero, so the ordinary
case still costs the one request this probe is meant to be, and accounts
left behind empty do not count as activity.
The narrow blast radius is worth stating: has_activity? only runs when an
address matches more than one chain, and when no candidate answers the user
is asked to choose rather than turned away. So this was a worse screen, not
a rejected wallet.
Separately, the check constraint accepted any asset_kind that carried a
contract address. Each partial unique index names its kind, so a row with
any other one is keyed by nothing and duplicates freely. Adding a token kind
already means adding its index here, so pinning the three in the table adds
no coupling that the indexes did not already have.
Both regression tests fail on the previous code. The third test - emptied
token accounts are not activity - passes either way by design: it guards the
new branch rather than testing it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(onchain-wallets): cover the asset_kind constraint that shipped without it
The constraint landed in
|
||
|
|
6d6dac44f3 |
fix(i18n): remove duplicate locale keys and guard with a regression test (#2789)
* fix(i18n): remove duplicate locale keys that shadow translations
YAML resolves duplicate keys by silently letting the last occurrence
win, so every duplicated key hides an earlier definition without any
warning:
- transactions/{en,de,nl,hu}.yml defined `transactions.merge_duplicate`
twice: once as a stale flat string ("Yes, merge them") and once as the
`success`/`failure` mapping the controllers actually read. The flat
string is dead either way (the show view reads
`transactions.show.merge_duplicate`), so drop it.
- securities/fr.yml listed five providers (tiingo, eodhd, alpha_vantage,
mfapi, binance_public) twice inside `securities.providers`.
- settings/api_keys/hu.yml defined
`settings.api_keys.created.usage_instructions_title` twice.
- imports/en.yml defined `imports.create.ndjson_uploaded` twice.
All removals keep the currently-winning value, so rendered output is
unchanged; this only makes the files match what I18n already loads.
Add a non-skipped I18nTest case that walks the raw Psych AST of every
file under config/locales and fails on same-level duplicate keys, so
shadowed translations can't sneak back in.
Fixes #1506
Fixes #1502
* Fix linter / merge errors
* fix(i18n): repair duplicate locale regression test
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: rsnetworkinginc <rsnetworkinginc@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: sure-admin <sure-admin@splashblot.com>
|
||
|
Sure Admin (bot)
|
381ede89d9 | ci: skip chrome install in unit tests (#3125) | ||
|
|
4e010493c7 |
feat(up): map Up category slugs to Sure categories on import (#2487)
* feat(up): map Up category slugs to Sure categories on import UpEntry::Processor captured Up's category slug into extra but never applied it, so Up transactions imported uncategorised even though the user had already tagged them in the Up app. Add UpAccount::Transactions::CategoryTaxonomy + CategoryMatcher, mirroring PlaidAccount::Transactions::CategoryMatcher: map Up's child category slugs onto the family's existing/default Sure categories by alias, and wire the matcher through UpAccount::Transactions::Processor into UpEntry::Processor. The category is applied via the adapter's enrich_attribute, so a category the user has set or locked is preserved on re-sync. High-confidence mappings only. Up-specific categories with no honest Sure default (Booze, Pets, Apps & Games, Life Admin, Technology, ...) intentionally stay uncategorised for the user's own rules / AI, since a wrong auto-category is worse than none. Adds a matcher unit test and processor wiring tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(up): match category slugs as strings in CategoryMatcher Up category ids are string slugs; compare them against the taxonomy keys as strings so the lookup does not depend on the keys being symbols. No behaviour change (the "slug": hash syntax already produces symbol keys that matched the symbolized input, covered by the matcher unit test), but it removes a subtle footgun and reads clearer. Flagged by the Codex review on the PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): make category import non-destructive; word-boundary the alias match Per review feedback: do not bootstrap Sure's default categories during a sync. family_categories now returns the family's existing categories without creating defaults, so a family that has none (deliberately cleared, or pre-onboarding) gets uncategorised transactions rather than having the full default set silently created. Matching resumes once the user sets up categories through the normal UI flow. Also word-boundary the "and" stripping in the matcher normalization so it strips only the standalone conjunction, not "and" inside a word (e.g. errand). Adds a processor test for the non-destructive guarantee. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Gavin Matthews <matthews.gav@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Juan José MataandClaude
|
4d1d33f91d |
perf: memoize Family#balance_sheet and sync status lookups per request (#2553)
Family#balance_sheet built a new BalanceSheet on every call. The application layout renders the account sidebar twice per page (desktop + mobile) with three tab panels each, and each panel asks the family for its balance sheet - so the account, sync-status and exchange-rate queries behind it ran up to six times per request. - Memoize Family#balance_sheet per user id (Current.family is the same instance for the whole request, and jobs/controllers use short-lived Family objects, so staleness is not a concern). - Memoize BalanceSheet::SyncStatusMonitor#syncing_account_ids in the instance: it is called once per account row, and each call was a Rails.cache round-trip (or a full re-query with the test null store). Measured on the test suite probes: ReportsController#index view time 230ms -> 155ms, AccountsController#show 299ms -> 233ms. Claude-Session: https://claude.ai/code/session_01RpZe2ajeGkPRRBHfaJTfUB Co-authored-by: Claude <noreply@anthropic.com> |
||
|
super
|
1a5d04d527 |
feat(rules): add a Transaction tag condition filter (#2558)
* feat(rules): add a Transaction tag condition filter Rules could already set tags via the set_transaction_tags action but had no way to match transactions by an existing tag. Add a select-type transaction_tag condition filter (mirroring transaction_category) with the standard "Equal to" and "Is empty" operators, registered on the transaction rule resource so it surfaces in the rule builder automatically. Also remap the tag UUID<->name in family data export/import for the new condition, matching how the transaction_category/transaction_merchant conditions and the set_transaction_tags action are already handled, so tag-based rules survive an export/import round-trip. Closes #2557 * fix(rules): match transaction_tag via EXISTS so compound tag conditions work Addresses review feedback on #2558: the tag filter joined transactions.tags and predicated on tags.id, which broke two compound cases: - two ANDed tag conditions collapsed to `tags.id = a AND tags.id = b` on the same joined alias and could never match, even when the transaction had both tags; - OR / multi-tag matches returned a transaction once per tagging row, inflating counts and making rule actions iterate duplicate transactions. Use a correlated EXISTS subquery per condition instead. Each condition is independent (fixes the AND case) and no join is added, so rows are never multiplied (fixes the OR duplication) and prepare adds nothing, keeping branches structurally compatible inside a compound OR. Add tests for both. |
||
|
|
6439a731ab |
feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1906)
* feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1481) When the Sidekiq worker container isn't running — the most common Docker Compose misconfiguration in self-hosted setups — every background job silently never executes. Balance calculations, net-worth updates, and account syncs stall. The UI shows zeros and "No balance data available for this date" without explaining why (#1481, #1047). Per jjmata's resolution on the issue, this PR ships both halves of the fix in one pass: 1. A user-facing nudge banner that appears on every authenticated page when Sidekiq isn't processing jobs. Tells the user their data may be stale; doesn't pretend zeros are real. 2. An admin-only deep link from that banner into a new `/settings/admin/system_health` page (super-admin gated, matching the existing admin namespace contract) showing live Sidekiq state: process count, last heartbeat, max queue latency, job counters, and per-queue depth. ## What changed - New `SidekiqHealth` PORO (`app/models/sidekiq_health.rb`) eagerly loads ProcessSet + Queue + Stats in one pass and exposes `healthy?` plus a stable `reason` symbol (`:redis_unreachable`, `:no_worker_processes`, `:stale_heartbeat`, `:queue_backed_up`). Any Redis/Sidekiq failure during the eager load is caught and surfaced as `:redis_unreachable` so a degraded broker never crashes the layout. - `ApplicationController#current_sidekiq_health` memoizes a single instance per request via `helper_method` so the layout, banner partial, and any controller checks share one Redis round-trip. - New `app/views/shared/_sidekiq_health_banner.html.erb` rendered from `_htmldoc.html.erb` when `Current.user` is present and the health check is failing. Banner shows the user-facing message to everyone; the "View system health" CTA + reason detail are gated on `Current.user&.super_admin?`. - New `Admin::SystemHealthController#show` (inherits the existing `Admin::BaseController`, so super-admin gating is enforced for free) + view rendering status, counters, and per-queue breakdown. - Routes: `resource :system_health, only: :show` inside the existing `namespace :admin`. - Settings nav: new "System health" entry under the Advanced section, gated on `super_admin?` to match `sso_providers_label` and `users_label`. - i18n: new `shared.sidekiq_health_banner.*` keys (title, body, CTA, per-reason explanations) and a full `admin.system_health.show.*` namespace for the new admin page. English-only, matching how `ds.pill.*` and other DS keys are scoped. ## Why - jjmata: "Let's take both approaches ... a nudge about 'data unavailable' which hyperlinks to the admin UI if you are an admin only (not for other types of users) sounds like the best path forward. **Any takers for the PR?**" (#1481) - smurfpandey: "We can add a section in Settings for superadmins to see 'health' of the application/host." - The detection signal is conservative on purpose: - `PROCESS_HEARTBEAT_TIMEOUT = 2.minutes` tolerates deploy restarts and brief Redis blips without flapping. - `LATENCY_THRESHOLD = 5.minutes` is well above the sync-job tail under default `config/sidekiq.yml` concurrency. ## Validation This worktree runs on Windows without a local Ruby toolchain, so I could not run `bin/rubocop`, `bundle exec erb_lint`, `bin/brakeman`, or `bin/rails test` locally. CI will run the full matrix on the PR: - `lint` — `bin/rubocop -f github` - `lint_js` — `npm run lint` (no JS touched, should be green) - `scan_ruby` — `bin/brakeman --no-pager` - `scan_js` — `bin/importmap audit` - `test_unit` — `bin/rails test` (includes 7 new tests under `test/models/sidekiq_health_test.rb` and 4 new under `test/controllers/admin/system_health_controller_test.rb`) - `test_system` — `DISABLE_PARALLELIZATION=true bin/rails test:system` - `pipelock` — secret + agent-security diff scan Manual checks done in this worktree: - Re-read `CONTRIBUTING.md` and `.cursor/rules/project-conventions.mdc`. PORO under `app/models/` per Convention 2. No new gem dependency per Convention 1. Banner uses semantic tokens (`bg-warning/10`, `text-warning`) per the design-system rules. No `lucide_icon` direct call — uses the `icon` helper per CLAUDE.md. - Confirmed `Sidekiq::ProcessSet` / `Sidekiq::Queue` / `Sidekiq::Stats` are the same APIs Sidekiq 7+ exposes (we're on Sidekiq 8.x per the `Gemfile.lock` comment in `config/initializers/sidekiq.rb`). - Tests stub `Sidekiq::ProcessSet.new` / `Sidekiq::Queue.all` / `Sidekiq::Stats.new` so the suite doesn't need Redis populated. - The admin route lives inside the existing `namespace :admin` so `Admin::BaseController#require_super_admin!` enforces auth — no new authorization surface added. ## Notes - No public API endpoints, no rswag specs, no OpenAPI changes. - No migrations, no model changes outside the new PORO. - No background jobs touched. - English-only locale entry, mirroring the `ds.*` / `admin.invitations.*` precedent in this repo. Other locales fall back to English. - Detection thresholds are constants on `SidekiqHealth` so they're easy to tune from a follow-up PR if the defaults turn out to flap on any real-world deployment. - The banner positions itself at `top-20` (below the impersonation / super-admin bars) and uses `z-40` (below the `z-50` notification tray). Single-screen overlap with mobile flash toasts is acceptable for V1. Refs: #1481, #1047 * fix(self-hosting): address review on Sidekiq health PR (#1481) - `Admin::SystemHealthController#show` now reads from the request-memoized `current_sidekiq_health` instead of building a fresh `SidekiqHealth.new`, so the controller and the layout banner share one Redis round-trip. - `SidekiqHealth#reason` now treats `last_heartbeat_at.nil?` the same as a stale beat: a registered process that hasn't published a heartbeat is not "healthy". Previously the check short-circuited on the nil guard and silently fell through to the queue-latency branch. Added a unit test covering the `ProcessSet` entry with `"beat" => nil` case. - Settings nav: switched the "System health" entry's icon from `activity` to `heart-pulse` so it no longer duplicates the LLM Usage icon. - Routes: dropped the redundant `controller: "system_health"` option from the `resource :system_health` declaration — Rails infers `Admin::SystemHealthController` from the namespace, matching the style of the sibling `:sso_providers`, `:users`, `:invitations`, and `:families` admin resources. * fix(self-hosting): scope + cache Sidekiq health, admin-only banner (#1481) Addresses the second round of maintainer review on the Sidekiq health PR. - Skip the check entirely in managed mode. `current_sidekiq_health` returns `nil` unless `Rails.application.config.app_mode.self_hosted?`, so authenticated requests in managed deployments add zero Redis round-trips for this feature. - Cache the snapshot across requests via `SidekiqHealth.current` (Rails.cache, TTL `CACHE_TTL` = 60s default, env-overridable). The per-request memoization on `ApplicationController` is preserved on top, so even back-to-back self-hosted pages share one fetch. - Make thresholds operator-tunable. `PROCESS_HEARTBEAT_TIMEOUT`, `LATENCY_THRESHOLD`, and the new `CACHE_TTL` read from `SIDEKIQ_HEALTH_HEARTBEAT_TIMEOUT`, `SIDEKIQ_HEALTH_LATENCY_THRESHOLD`, and `SIDEKIQ_HEALTH_CACHE_TTL` env vars (seconds), with the previous values as defaults. Comments now explain the tuning rationale. - Gate the banner on `Current.user&.super_admin?` at the layout level rather than rendering a vague warning to family members who can't act on it. The partial no longer carries an internal admin check since the call site does it; non-admins see nothing. - Replace the hard-coded `top-20` offset with a computed offset based on which impersonation bars are visible (`top-4` / `top-20` / `top-36`) so the banner doesn't collide with the super-admin or approval bars when both are stacked above it. - `Admin::SystemHealthController#show` now bypasses the cache (`SidekiqHealth.expire_cache!` + `SidekiqHealth.new`) so an operator who just restarted the worker sees fresh state instead of a stale 60-second snapshot. Also lets the page render in managed mode where `current_sidekiq_health` is nil. - Tests: add coverage for `.current` cache reuse and `.expire_cache!` forcing a re-query, swapping `Rails.cache` to a MemoryStore since the test env defaults to `:null_store`. * fix(self-hosting): route singular resource + drop assert_same on cached snapshot (#1481) Two CI failures surfaced once the full pipeline ran on this branch for the first time (it was gated on contributor approval until d04b78e): - Admin system-health controller tests returned 404. Singular `resource :system_health` in `config/routes.rb` makes Rails infer `Admin::SystemHealthsController` (it pluralizes the controller name even for singular resources), but the controller file is named `system_health_controller.rb` / `Admin::SystemHealthController`. Restore the explicit `controller: "system_health"` override that the previous "address review" commit dropped on the (mistaken) premise that Rails would infer it from the namespace — the sibling admin routes all use plural `resources` so they round-trip cleanly, this one doesn't. Comment now spells the gotcha out so the next reviewer doesn't try to "simplify" it again. - `SidekiqHealthTest#test_current_memoizes_across_calls_inside_the_cache_TTL` used `assert_same` on the two returns from `SidekiqHealth.current`. `ActiveSupport::Cache::MemoryStore` defaults to `dup_values: true` and Marshals on read, so a cache hit returns an `==`-equal but `equal?`-different instance. Replace the identity check with the behavioral assertion we actually care about: re-stub `ProcessSet` to raise on the second call, then assert the second `current` return is still healthy (proving Redis was not re-queried). * fix(i18n): drop redundant inline default on system_health nav label (#1481) `system_health_label` is already defined in config/locales/views/settings/en.yml, so the inline `default: "System health"` was a hard-coded English string in the template (DS Drift Patrol Rule 5). Use the bare locale lookup like the sibling nav entries. --------- Co-authored-by: John Baillie <johnbaillie2007@gmail.com> Co-authored-by: Khaostica <256858950+Khaostica@users.noreply.github.com> |
||
|
William Wei MingandCursor
|
8cffeaaefd |
fix(ds): resolve remaining DS Drift findings (#1971) (#2978)
* fix(ds): resolve remaining DS Drift findings from #1971 Migrate leftover preference currency badges and the admin invitation delete control to DS::Pill / DS::Button. Drop a redundant Enable Banking beta_label default now that the locale key exists. Closes #1971 Co-authored-by: Cursor <cursoragent@cursor.com> * chore(ds): drop duplicate admin invite button from #1971 PR Leave the invitation delete DS::Button migration to #2979, which also touches admin/users/index.html.erb, to avoid a same-hunk merge conflict. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(ds): drop duplicate Enable Banking beta_label from #1971 PR #2977 already removes the same t(..., default:) on select_bank; leave that hunk there so the two PRs don't touch the identical line. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(ds): drop redundant DS::Pill size: :sm defaults size: :sm is already the initializer default; remove the explicit arg from the preferences currency pills per review. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
5a2bf02b13 |
fix(simplefin): stop repair_stale_linkages from hijacking a live linkage on a same-name twin (#3116)
* fix(simplefin): stop repair_stale_linkages from hijacking a live linkage on a same-name twin repair_stale_linkages matched purely on case-insensitive display name, so two distinct upstream accounts sharing a name (e.g. two "CHECKING (0001)" accounts at the same institution) caused the unlinked twin to silently steal the linked account's AccountProvider, merge in its transactions, and overwrite its balance on every subsequent sync. Thread the upstream account_id set already computed during account discovery through to repair_stale_linkages so it only treats a linked account as stale when its account_id is actually absent upstream, and skip ambiguous multi-way name matches instead of picking the first one. Fixes #2852 * fix(simplefin): clear stale upstream_account_ids and log skipped repairs - Clear simplefin_item.upstream_account_ids at the start of each perform_account_discovery run so a later discovery that finds zero accounts can't reuse IDs from a prior run on the same SimplefinItem instance (CodeRabbit review finding). - Capture skipped stale-linkage repairs via DebugLogEntry so operators can see them in /settings/debug, not just the raw Rails log (Codex review finding). - Fix two pre-existing SimplefinAccount::Transactions::ProcessorInvestmentTest tests broken by the new upstream_account_ids nil-guard: they called process_accounts directly without going through the Importer, so they now set upstream_account_ids explicitly to simulate a legitimate "old account_id genuinely absent upstream" repair. - Add regression coverage for both fixes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
pro3958andagentloop
|
fcb46be19f |
fix(transactions): re-render form when creating a transaction with no account (#2777)
TransactionsController#create looked up the account with accessible_accounts.find(params.dig(:entry, :account_id)). With no account selected the id is blank, find raises RecordNotFound, and StoreLocation's rescue_from turns that into head :not_found — the 404 on /transactions the reporter saw. Switch to find_by(id:) and, when it returns nil, rebuild the entry, run validation, and re-render :new with 422, matching the existing validation-failure branch. This covers a blank, missing, or invalid account_id, so the user gets the form back with errors instead of a dead button. Fixes #2566 Co-authored-by: agentloop <agentloop@localhost> |
||
|
|
b0ecb919b0 |
fix(lunchflow): refresh stored transaction on pending to posted (#2778)
* fix(lunchflow): refresh stored transaction on pending to posted LunchflowItem::Importer keyed stored raw transactions but treated them as immutable snapshots. When Lunchflow flipped a transaction from pending to posted under a stable ID, fetch_and_store_transactions skipped it as a duplicate and kept the stale pending snapshot. The processor then re-imported it with isPending:true, so ProviderImportAdapter#import_transaction never reached its pending-clearing branch and the entry stayed stuck with a "Pending" badge. Index stored transactions by key (the Lunchflow ID, or a content hash for the blank IDs Lunchflow returns for some pendings) and refresh the stored snapshot in place when the upstream payload actually changed, while still deduplicating by key to prevent unbounded growth. Fixes #2735 * fix(lunchflow): preserve identical same-response blank-ID transactions The stored snapshot was keyed with a Hash of key -> transaction, so two rows in one sync response that share a content hash (Lunchflow returns blank IDs for some pending transactions, and two genuinely distinct identical purchases hash the same) collapsed into a single entry. The second row hit the existing-key branch as though it were a duplicate, dropping a real transaction before LunchflowEntry::Processor could apply its collision suffix. Pool the existing snapshot into one bucket per key and match incoming rows one-for-one (shift), so same-response collisions each claim their own slot or count as new. This preserves every real transaction while still deduplicating across syncs (re-syncing the same pair stays at two, not four) and refreshing pending -> posted transitions. Adds a regression test for two identical blank-ID rows in the same response. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
|
964817748e |
fix: keep excluded entries visible in account activity (#2772)
* fix: keep excluded entries visible in account activity AccountsController#show built the activity list with @account.entries.where(excluded: false), which hard-hid every excluded entry. Trades only appear in the account activity feed, not the global /transactions page, so once a trade was excluded from analytics there was no row to click and no way to toggle exclusion back off. The balance still counted the trade, so the row vanished from the list but remained in the balance tooltip. Use the excluding_split_parents scope instead, matching Transaction::Search. Excluded entries render greyed-out and can be re-included via the existing exclude toggle in the drawer; only split parents stay hidden. The account list is now consistent with both the global transactions list and the balance calculation. Fixes #2612 * Fix account activity test lint --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
|
29c0a369d0 |
fix(transfers): isolate concurrent transfer match in a savepoint (#2769)
* fix(transfers): isolate concurrent transfer match in a savepoint auto_match_transfers! opens one Transfer.transaction and, per candidate, calls Transfer.find_or_create_by! while rescuing RecordNotUnique. The transfers table has a composite unique index on (inflow_transaction_id, outflow_transaction_id), so when two syncs of the same family run at once the losing insert raises the unique violation. On PostgreSQL a failed statement aborts the whole surrounding transaction. Rescuing the Ruby exception does not clear that state, so the next update! raises PG::InFailedSqlTransaction and every following candidate is dropped. Run the per-candidate insert in its own savepoint via Transfer.transaction(requires_new: true), extracted into a private find_or_create_transfer! helper. A lost race now rolls back only to the savepoint; the outer transaction stays healthy and the loop keeps matching. The same race surfacing through the uniqueness validation (RecordInvalid with :taken) is treated as already-created; any other validation failure is re-raised. Fixes #2471 * fix(transfers): only swallow the uniqueness race for the exact pair The rescue treated a :taken on inflow_transaction_id or outflow_transaction_id as proof that this candidate's transfer was created. But the uniqueness validations are per-column, so two same-amount candidates racing (one committing (inflow, outflow_a) while another tries (inflow, outflow_b)) raise :taken on inflow_transaction_id even though no Transfer exists for (inflow, outflow_b). The caller then marked outflow_b as matched with no Transfer behind it. Confirm the exact (inflow_transaction_id, outflow_transaction_id) row exists before accepting the race; otherwise return nil and skip the candidate. Non-:taken validation failures still re-raise. The RecordNotUnique path (composite index) already implies the exact pair — it now returns that row for the same reason. * test(transfers): assert matching continues past a skipped collision The concurrent-race test had no surviving candidate, so a regression that stopped processing after the skipped collision would still pass. Add a second, non-conflicting candidate and assert its transfer is created and both entries are marked. * test(transfers): pass insert! attributes as an explicit hash Ruby 3 treats insert!(inflow_transaction_id: ..., outflow_transaction_id: ...) as keyword arguments, so ActiveRecord's insert!(attributes) got zero positional args and raised ArgumentError (given 0, expected 1). Wrap the attributes in { } so they are the positional attributes hash. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
|
d57c4301f2 |
fix(import): tolerate null Rule names and orphaned rejected transfers (#2775)
* fix(import): tolerate null Rule names and orphaned rejected transfers Importing a full all.ndjson export aborted on data that is actually valid. The preflight listed name as a required field for Rule, but rules.name is nullable and the model allows it, so a single rule with "name": null blocked the entire import. Separately, a RejectedTransfer whose referenced transaction had been deleted raised a hard missing_reference error in preflight and a MissingReferenceError in strict mode, even though the importer already had a skip path for it. Require Rule.id instead of Rule.name in preflight, matching the field the importer actually needs. Treat RejectedTransfer references as advisory: a missing referenced transaction becomes a warning, and the importer resolves the references with required: false so the orphaned row is skipped and counted instead of raising. Fixes #2721 * fix(import): keep SureImport preflight warnings as strings at the API boundary The orphaned-RejectedTransfer path emits warnings as {code, message} hashes via add_warning, but the published OpenAPI contract documents /api/v1/imports/preflight warnings as strings. sure_import_preflight_payload copied them through unchanged, so the endpoint returned a heterogeneous array once that path was exercised and contract-generated clients could fail to deserialize. Map warnings to their human-readable message at the API boundary so the array stays homogeneous strings, matching the documented schema. The internal {code, message} shape is unchanged. Adds a regression test. * i18n(import): localize missing-reference preflight messages The missing-reference warning and error are user-facing (returned in Result#payload[:warnings]/[:errors]) but were hard-coded. Move the full templates to config/locales/models/sure_import/preflight/en.yml and interpolate line, type, field, and value, per the i18n coding guideline. Warning key and error behavior are unchanged, and the rendered text is identical. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
github-actions[bot]andgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
0575b78e60 |
Bump version to next iteration after v0.7.4-alpha.7 release (#3123)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Josh
|
2f821e2567 |
chore(security): update Pipelock integration to 3.4.0 (#3122)
* chore(security): update Pipelock integration to 3.4.0 * fix(ci): validate shipped Pipelock configs * fix(security): isolate external assistant profile * fix(ci): build Helm dependencies before validation * fix(ci): strengthen Pipelock contract checks |
||
|
rottenit
|
9930b721b3 |
Add inline category creation to transaction form (#3088)
* Add inline category creation to transaction form * Address category selector review feedback * Fix category system test regression * Address category selector review feedback * Use Rails field ID for category selector |
||
|
GFRandGerald
|
07ce130405 |
fix: respect SURE_IMPORT_MAX_NDJSON_SIZE_MB in Sure import GUI upload (#3111)
The GUI upload paths (imports_controller#create_sure_import and Import::UploadsController#update_sure_import_upload) checked file size against the hardcoded SureImport::MAX_NDJSON_SIZE constant instead of SureImport.max_ndjson_size, so self-hosted admins raising SURE_IMPORT_MAX_NDJSON_SIZE_MB had no effect on the GUI — only the API upload paths honored it. Removes the now-unused constant. Fixes #3010. Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> |
||
|
GFRandGerald
|
da30477775 |
i18n: remove dead transfers.form exchange-rate keys from ru/zh-CN locales (#3109)
The views only ever call shared.exchange_rate_tabs.*, not transfers.form. calculate_rate_tab/convert_tab/exchange_rate/exchange_rate_help — en/fr were already cleaned up (PR #1501), ru/zh-CN still carried the dead keys. Fixes #1504 Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> |
||
|
|
f0a0013da9 |
fix(charts): round series values to the currency's display precision (#3091)
* fix(charts): round chart values to the currency's display precision Charts are drawn from the serialized amounts, not from the formatted strings, so a sub-unit residue plotted as a visible move between two points that both read $0.00, and the trend between them reported a change. Round in `Series#as_json` so the series values stay exact for insights, goals and the assistant. Also hide the percentage when the previous value is zero, since that makes it infinite. * fix(charts): round the two payloads that bypass Series#as_json `NetWorthBreakdownSeriesBuilder` builds its payload by hand, so the reports chart never went through the rounding added in `Series#as_json` and still plotted raw amounts: adjacent points printing the same value rendered a visible move, and the tooltip it inherits reported a change between them. `Series#trend` had the same gap on the server side. It is rendered right above the chart by `UI::Account::Chart` and by the reports summary, so an account going from 0 to a sub-cent residue showed a coloured $0.00 change next to a flat line. Rounding the trend can make a previously finite percentage infinite, so guard the three views that render `percent_formatted` without checking, as `shared/_trend_change` already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(charts): address net worth review comments * fix(charts): tighten rounded trend handling * fix(reports): restore positive sign in print trend --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
GFRandGerald
|
0ff5b0f274 |
fix: allow saving a budget without any category allocation (#3101)
The "Save" button in the budget setup wizard was hard-disabled via Budget#allocations_valid?, which requires allocated_spending > 0. This meant a user could not finish creating a budget that only sets an overall monthly amount without splitting it across categories, even though the Budget model itself supports that (only start_date/ end_date are required) and the rest of the app already treats an unallocated amount as "Uncategorized" (donut chart, show page). The wizard's own "X" close button confirmed the inconsistency: it navigates to the same budget show page with zero validation, so the unallocated state was already reachable, just not via the intended confirm action. Change the button's disabled condition to only require an initialized budget and no over-allocation, dropping the allocated_spending > 0 requirement. allocations_valid? itself is left untouched since it still correctly drives the "fully allocated" donut/nav state elsewhere. Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> |
||
|
Abhinav Dhiman
|
aafefab65a |
fix(process-pdf-job): discard on RuntimeError and guard failed status (#2453)
* fix(process-pdf-job): discard on RuntimeError and guard failed status Add discard_on(RuntimeError) to drop the job immediately instead of exhausting 25 Sidekiq retries on deterministic errors. The block logs job_id and message for observability. Widen the early-return guard from status == "complete" to also cover "failed", preventing re-processing of already-failed imports. * fix(process-pdf-job): discard on Provider::Error instead of RuntimeError * fix(process-pdf-job): log error class name instead of message in discard handler |
||
|
GFRandGerald
|
fa8769f792 |
fix: support DD/MM/YY date format for CSV imports (#3110)
Adds "DD/MM/YY" as a CSV-only date format for transaction and account balance imports, addressing #1530. Kept out of Family::DATE_FORMATS (the global date preference) since a prior PR (#531) adding it there was rejected by a maintainer: 2-digit years are ambiguous (Ruby's %y assumes 1969-2068) and could silently misparse historical or future-dated transactions. Restricting it to Import::CSV_ONLY_DATE_FORMATS keeps it available where the user can see and verify a parsed preview against their own CSV data, per the maintainer's suggested approach in that PR's discussion. Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> |
||
|
Juan José MataandClaude Opus 5
|
8ca65ffd27 |
Reconcile PDF statement imports against transactions that already exist (#3105)
* Reset to main, keeping only the account matcher improvements Backs out the LLM-driven reconciliation work (PR #1382's approach and the two commits hardening it). That approach compares whole-statement aggregates, which is all-or-nothing: when 18 of 20 transactions have already synced the totals disagree, the import proceeds, and 18 duplicates are created. Reconciliation is a row-level problem and belongs in the import path, where TransactionImport already solves it via Account::ProviderImportAdapter. Kept from that work, because it stands on its own: - AccountMatcher gains a hint-based class-level entry point so callers without an AccountStatement row can score against the same rules. The instance path used by AccountStatement#assign_account_match is unchanged. - It also refuses to guess between equally-confident candidates rather than letting max_by take whichever the scan reached first. Account names are not unique within a family, so that tie was reachable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Reconcile PDF statement imports against transactions that already exist Reconciliation is a row-level problem. Comparing whole-statement totals is all-or-nothing: when 18 of 20 transactions have already synced the totals disagree, the import proceeds, and 18 duplicates are created. This matches each extracted transaction against what the account already holds, so only genuinely new transactions are ever offered for import and the rest are marked reconciled. TransactionImport#import! already does this for CSV via Account::ProviderImportAdapter. PdfImport#import! called it zero times and built one Transaction per row unconditionally -- the only import path in the repo with no duplicate protection. Reconciliation state follows Quicken's uncleared / cleared / reconciled, but only the last state is stored: - "Cleared" means the institution acknowledged the transaction, which is exactly what entries.source and entries.external_id already record. It stays accurate on its own, because the adapter stamps both onto a manual entry when a provider transaction claims it, so a hand-entered transaction that later appears in a download becomes cleared with no extra bookkeeping. Deriving it also keeps it non-editable, which is right: it is a fact about provenance, not an opinion. - "Reconciled" means a statement was matched against the transaction. Nothing can derive that, so entries gains reconciled_at and reconciled_by_statement_id. It is a judgement, so it can be set and unset, and it survives the statement being deleted (the FK nullifies, the timestamp stays). Matching: - find_duplicate_transaction grows include_provider_entries, which is what makes this work for Provider-backed accounts -- the existing where(external_id: nil) filter hid synced transactions from every import path, so this gap affected CSV imports equally. Default stays false: provider sync must not claim another provider's entry. - It also grows date_window, because a statement's posting date routinely differs by a day or two from the date a provider recorded. Nearest date wins. - Name is deliberately not matched on: statement descriptions and provider names for the same transaction rarely agree. The adapter makes the same choice for sync. - Candidates are built as real Import::Row objects so matching uses the same signed_amount and date_iso the import itself would write, rather than a second interpretation of signage that could drift. - Matching is per-account, so with no account assigned every row is offered and re-judged on assignment; reassigning also releases the previous account's reconciliations. - A row whose date or amount will not parse is offered for import rather than dropped, so nothing disappears silently. - import! re-checks at publish, since a sync can land between review and publish, and new transactions are born reconciled: the statement is their evidence. Provider-backed accounts are now offered in the import target picker. The manual-only restriction existed because importing into a synced account would duplicate what sync brought in, which is precisely what this removes. Also fixes a bug this uncovered on main: extract_transactions stored the extractor's symbol-keyed hash, while every reader digs with strings. jsonb keeps the hash as assigned until reload and ProcessPdfJob never reloads, so has_extracted_transactions? was false and PDF imports generated zero rows. The existing tests missed it -- one uses a YAML fixture, the other stubs the extractor with string keys. Supersedes #1382. Refs #1379. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Fix three review findings in statement reconciliation All three confirmed against the code before fixing. Publish-time recheck consumed the same entry twice. import! started its exclusion list empty, so a statement carrying two same-amount transactions against an account holding only one would re-match the surviving row against the entry row generation had already consumed -- silently dropping a genuinely new transaction instead of creating it. Seed the exclusions with the entries this statement already reconciled; newly synced entries are still caught, since only already-reconciled ones are excluded. A regeneration that emptied the row set left the import stranded. In the normal upload flow the account is assigned after extraction, so assign_account! regenerates -- and if everything then reconciled, rows_count went to zero while status stayed pending. _pdf_import.html.erb renders pending-with-no-rows as the processing screen, and process_with_ai_later cannot restart because ai_processed? is already true, so the import was stuck with no way forward. Status now follows the same rule ProcessPdfJob applies after initial processing, in both directions: no rows completes it, rows returning sends it back to pending. Guarded by data_committed? so a published import is never reopened. Unevaluatable rows went only to the Rails log. AGENTS.md asks for DebugLogEntry.capture on recoverable import failures so they surface in /settings/debug with structured context. Capture family, account, import, statement, row number and the raw date/amount that would not parse. Adds regression coverage for each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Fix row regeneration collision and over-broad reconciliation release test_unit caught one error in 6462 tests, and it was real. Row regeneration collided on the second call. insert_all! bypasses ActiveRecord, so the rows association is never populated with what it wrote. Calling generate_rows_from_extracted_data twice on the same in-memory record -- which assign_account! now does after ProcessPdfJob has already generated once -- made rows.destroy_all clear a stale empty collection, delete nothing, and then collide on (import_id, source_row_number). Reload before destroying, and reset the association after inserting so sync_mappings and the view read what was actually written. Releasing reconciliations was scoped to the statement, not the account. A statement is evidence for exactly one account at a time but can back more than one import, so reassigning an account cleared reconciliations another account still relied on. Scoped to the account being moved away from; a blank scope releases nothing, which is correct because nothing is reconciled while no account is assigned. The second finding was raised by CodeRabbit. Its other flagged risk -- entries being marked reconciled before the import is published -- is deliberate and stays: the statement is the evidence, reconciling is reversible via unmark_reconciled!, and deferring it to publish would leave a fully reconciled statement with nothing to publish and therefore nothing ever marked. Adds regression coverage for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Address review nitpicks: model validation, scope placement, mock style, lock-safe DDL All four checked against the repo's own conventions before applying. Mirror the check constraint as a model validation. Assigning reconciled_by_statement without reconciled_at raised StatementInvalid rather than a validation error. CLAUDE.md Convention 5 asks for exactly this pairing -- the constraint in the database, an ActiveRecord validation for form-friendly errors. Stop hijacking the pending-scope comment. The reconciliation scopes were inserted directly under "Pending transaction scopes", so that header read as documentation for them and the provider note below read as a continuation of reconciled_by. Given the reconciliation scopes their own header. Use OpenStruct for the provider response double, per "Always prefer OpenStruct when creating mock instances". Verified OpenStruct.new(success?: true) responds to success?, and ostruct is already a dependency used elsewhere in test/. Make the migration lock-safe. entries is the largest table in the app: both indexes now build concurrently, and the check constraint is added unvalidated then validated separately so VALIDATE takes only SHARE UPDATE EXCLUSIVE instead of holding ACCESS EXCLUSIVE for a full scan. This follows existing practice -- 13 migrations already use disable_ddl_transaction! and 11 use algorithm: :concurrently, with add_offline_reason_to_securities combining add_column and a concurrent index in one migration exactly like this. The suggested follow-up migration for validation was not needed: validating in the same non- transactional migration gets the same lock behavior without a second file, and the repo has no validate: false precedent in 400 migrations. schema.rb is unchanged: a validated constraint and a concurrently-built index dump identically. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Add the two review-requested regression tests Covers the invalid-amount half of "malformed rows are offered, not dropped" -- the existing test only exercised an unparseable date. Asserts the raw value is stored verbatim rather than coerced to 0, so the review step shows the user what the statement actually said. Also covers the Entry validation added in |
||
|
GFRandGerald
|
ecf00f8a0f |
fix(enable_banking): recognize N26's PERIOD_INVALID error shape as a retryable period rejection (#3112)
N26 (via Enable Banking) rejects an out-of-range transaction period with
{"code": "PERIOD_INVALID", "detail": "dateFrom=...,dateTo=..."} instead of
the {"error": "WRONG_TRANSACTIONS_PERIOD"} shape the retry ladder from
#2992 already handles. wrong_transactions_period? never matched, so the
sync failed outright instead of retrying with a shorter window. Also
guard corrected_date_from against a non-hash detail payload, which would
otherwise raise NoMethodError for this ASPSP's plain-string detail.
Fixes #1262
Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com>
|
||
|
Victor Dusart
|
ca876fa02a |
fix(ui): keep the account chart view selector in sync on reload (#3121)
The selected view lives only in the request params, so a reload renders the
default "Total value" chart. Browsers restore form control state on reload,
which left the selector showing the previously picked view while the chart
below it showed the total value. Opting the select out of autocomplete makes
it fall back to the server-rendered default alongside the chart.
Also switch the selector labels and the empty state to absolute i18n keys.
Lazy `t(".")` lookups resolve against a lowercased scope (`ui.account.chart`)
while the locale files declare the `UI:` namespace, so they silently rendered
titleized key names.
|
||
|
|
9d225a21e6 |
fix: disable Mark as Recurring button when a manual recurring transaction already exists (#3103)
* fix: disable "Mark as Recurring" button when a manual recurring transaction already exists Previously the button was always clickable and only failed after a POST, showing "A manual recurring transaction already exists for this pattern". Extract the lookup into Transaction#existing_manual_recurring_transaction (reused by the controller guard) so the view can disable the button ahead of time and show the reason inline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: address CodeRabbit review feedback on PR #3103 Move the existing_manual_recurring lookup out of the show view and into the controller so rendering no longer runs an Active Record query in-template, and strengthen the "no match" model test with near-match recurring transactions that individually differ by account, merchant, amount, currency, and manual flag. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: move mark-recurring presentation state fully into controller, fix stale state on failed update Address CodeRabbit follow-up on PR #3103: - Compute the mark-recurring button's subtitle text/class, href, disabled state, title, and class entirely in TransactionsController (via a shared assign_mark_recurring_state helper) instead of deriving them with ternaries in the view. - Populate that state before TransactionsController#update re-renders :show on a failed entry update, so the button doesn't incorrectly appear enabled when a matching manual recurring transaction exists. - Add a controller test covering the failed-update render path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: use blank name instead of blank date to trigger validation failure in mark-recurring test The CI test_unit run flagged a real bug in the test itself: TransactionsController#entry_params strips blank :date/:amount before update, so date: "" never reached model validation and the update succeeded (302) instead of failing (422) as the test expected. Use a blank :name instead, which isn't stripped, and add DOM assertions (disabled button, no mark_as_recurring form action) per CodeRabbit's follow-up review. Verified against a live NAS Rails console reproduction (bypassing the test stack's broken fixtures) that the failed-update render now correctly shows the button as disabled with the "already exists" message and no action link. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: use transaction id instead of entry id in mark-recurring route assertion mark_as_recurring is a member route on the transactions resource keyed by the Transaction's id, not the Entry's id (Entry uses delegated_type, so Entry and its Transaction entryable have distinct ids). The prior assertion built the path from `entry`, which could produce a different URL than the one actually rendered, so the "no href" check could pass even if the button leaked a link. Use entry.entryable so the assertion matches the real route. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: refresh mark-recurring state on turbo_stream update, avoid unconditional query, keep DS::Button href Addresses jjmata's review on PR #3103: - Extract the "Mark as Recurring" block into a dom_id-wrapped partial and replace it in the successful update turbo_stream response, so inline edits that change whether the transaction matches an existing manual recurring transaction are reflected immediately instead of only on the next full page render. - Skip the existing_manual_recurring_transaction lookup entirely when the block won't be rendered (no edit permission, or split-child entry), avoiding an unconditional extra query on every transaction show/failed update. - Keep href present on the DS::Button and only toggle disabled, matching the established pattern elsewhere in the app, instead of nulling href (which flips the component to a bare <button> and leaks a stray method="post" attribute). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brandon
|
25a9011f14 |
feat(ai): analytical tool-set upgrade for the builtin assistant (#3064)
* fix(assistant): survive tool failures with error and hint results instead of aborting the turn
A tool exception used to raise FunctionExecutionError out of the responder
loop, turning the whole turn into a generic chat error banner. An unknown
tool name was worse: the rescue block itself crashed (fn.name on nil).
Tool failures now come back to the model as data ({error, hint}) so the
conversation survives and the model can retry once with corrected
arguments. The catch-all branch logs and tells the model not to retry.
FunctionExecutionError remains defined for API compatibility.
* fix(assistant): strict schemas declare every property as required
get_categories and get_tags declared an optional page property while
inheriting strict mode, which is invalid under strict function calling
(every property must be listed in required). Both now opt out of strict
mode like every other paginated tool, and gain a page_size param
(1..100) while being touched.
A registry-walking test asserts the invariant for every current and
future tool, preview tools included.
* fix(assistant): HistoryTrimmer always keeps the newest turn
Trimming iterates newest-first and stopped at the first group over
budget. When the newest group alone exceeded the budget, everything was
dropped, including the user message the model was being asked to
answer, and the provider received only the system prompt. The newest
group now always survives.
* perf(assistant): compact AI time series and make account history opt-in
get_accounts shipped a 5-year monthly series for every account on every
call, roughly 60 formatted money strings per account, which dominates
the tool payload for multi-account families and swamps small
self-hosted context windows. The series is now opt-in
(include_balance_series) and bounded by a named period (series_period,
default last_365_days).
to_ai_time_series states the currency once and emits numeric values
instead of formatting every point; the system prompt already tells the
model how to render currency.
get_accounts also now returns account ids (they are what other tools
accept as account_ids filters) and respects the visible scope, so
hidden accounts no longer leak into responses.
* refactor(assistant): id and name filters replace user-data enums in get_transactions
The schema inlined every account, category, merchant, and tag name as
enum values on every request. That grows without bound with family
data, defeats provider prompt caching (definitions change whenever a
name does), and is the pattern that made empty-enum pruning necessary
in the first place.
Filters are now plain string arrays documented as exact names from the
sibling get_* tools, which Transaction::Search already resolves
server-side, plus an account_ids UUID filter. New params: page_size
(1..100), sort_by amount, types (income/expense/transfer, the way to
exclude transfers), and statuses (pending/confirmed).
The three now-unused enum helpers are removed from the base class;
family_tag_names stays for update_tag, which still identifies tags by
name.
* feat(assistant): add get_merchants and get_recurring_transactions
Merchants were unreachable: names appeared nowhere and
update_transaction's merchant_id had no source of ids, making it
unusable. get_merchants lists id, exact name, and source, scoped
through available_merchants_for so merchants seen only in accounts
hidden from the user never leak.
Recurring transactions had a model, an Upcoming view, and no assistant
reach. get_recurring_transactions lists detected and manual recurring
items (status filter defaulting to active, optional
upcoming_within_days window) with per-currency totals of active
non-transfer items, answering subscription and upcoming-bill questions
directly instead of via transaction paging.
* feat(assistant): flexible periods on get_balance_sheet and trends on get_income_statement
get_balance_sheet was hard-wired to five years of monthly history with
no parameters, although Period supports arbitrary ranges and the chart
builder takes any interval. It now accepts a named period or custom
dates plus an interval, with a 400-point cap so a day-granularity
request over a decade returns an error instead of a giant series. The
default call is byte-compatible with the old shape. The balance sheet
object is also memoized; it was being constructed four times per call.
get_income_statement gains the analysis surface the assistant lacked:
group_by month for a monthly income/expenses/net series (capped at 36
buckets), compare_previous_period for an equal-length prior window
with absolute and percent deltas, and account_ids to scope totals to
specific accounts via IncomeStatement#totals_for. Category breakdowns
are family-wide by construction, so the account-filtered view omits
them and says why. Unknown or inaccessible account ids come back as a
soft failure naming the ids so the model can correct itself.
* feat(assistant): preview reads for insights and valuations
The Insights feed is generated nightly with pre-computed numbers, and
the chat assistant could not read a word of it. get_insights returns
the visible feed (type filter, acknowledged toggle, limit) without
marking anything read; an assistant read is not the user viewing the
feed. It sits in PREVIEW_FUNCTION_CLASSES because the feature itself is
preview-gated, which also keeps it off the default /mcp surface.
record_valuation was write-only: an agent recording provenance-cited
valuations had no way to audit what it wrote or find dates already
carrying a value. get_valuations lists valuation entries newest first
with kind and the citation notes, scoped to accessible visible
accounts.
* feat(assistant): cache-stable system prompt with session context
The prompt interpolated currency mid-text and the date near the end, so
no two requests shared a cacheable prefix, and it told the model
nothing about the family: not one account name, not a single category.
Models opened most conversations blind, either wandering through tools
or answering without data.
The prompt is now STATIC_INSTRUCTIONS, a frozen constant that is
byte-identical for every request (providers discount an
exactly-repeated prefix; tool definitions are also stable now that
schemas carry no user data), followed by a trailing Session context
block holding everything volatile: date, date format, currency details,
an account roster with balances, and category names.
The static half gains a request-classification rule (CHAT / LOOKUP /
ANALYSIS), a reuse-what-you-have rule with an explicit re-fetch
carve-out, specific-tool preference, and the error/hint retry-once
rule that pairs with the tool soft-fail contract.
Context stays cheap by construction: the roster collapses to per-type
counts beyond 25 accounts, categories to a count beyond 60 names, and
both collapse whenever the configured context window is under 4096
(the self-hosted default is 2048), via the new Assistant::TokenBudget
helper. Intro chats are untouched.
* feat(assistant): raise tool-round cap to 8 with a no-tools grace turn; instructions-aware history budget
Five rounds was tight for a tool surface that now supports real
analysis chains, and hitting the cap raised ToolCallLimitError, which
surfaced to the user as a dead chat with an error banner. The default
is now eight rounds (env override unchanged), and on the final
permitted round the follow-up request offers no tools, so the model
must answer in text with whatever it gathered. The limit error remains
as a defensive backstop.
The generic-path history budget reserved a flat 256 tokens for a
system prompt that already estimates well past that; the trimmer now
budgets against the actual instructions when available.
LLM_MAX_RESPONSE_TOKENS was reserved in budget math but never sent to
the provider. It is now sent (max_tokens on chat completions,
max_output_tokens on the Responses API) only when explicitly
configured via ENV or a stored Setting; stock installs keep today's
uncapped behavior.
* test(evals): chat golden v2 exercising the real prompt and registry
The eval runner scored a fiction: hardcoded instructions and four fake
permissive tool schemas, so a prompt or registry regression could
sail through green. It now runs STATIC_INSTRUCTIONS plus a fixed
synthetic session context and builds definitions from
Assistant.function_classes against a reference user (classes whose
schema cannot build are skipped with a log line, never faked).
chat_golden_v2 adds routing scenarios the upgrade cares about: CHAT
classification must use no tools, aggregates route to
get_income_statement / get_balance_sheet rather than transaction
paging, and the new analytical tools are selected with sensible
params. The dataset header documents the harness's single-shot
limitation.
* docs(ai,mcp): current tool tables, responder loop, prompt structure, timeout math
Both docs listed 7 tools against a registry of 19, in three separate
drift-prone copies. mcp.md now carries the canonical tables (default +
preview); ai.md links to them from the MCP section, keeps one grouped
functions list for the architecture chapter, and replaces its stale
hardcoded registry snippet with a pointer to assistant.rb.
The architecture section gains the contracts contributors need when
adding a function: the responder loop (rounds vs calls, cap 8, the
no-tools grace turn) and the error/hint soft-failure convention, plus
the prompt's static/session-context split and its collapse gates.
Timeout guidance is recomputed for the new default cap.
* fix(assistant): address automated review findings
Codex and CodeRabbit findings on the initial push, all verified before
changing anything:
- AI time series rounded every value to two decimals, which turns
0.001 BTC into 0.0; values now round to the currency's own precision
(BTC 8, CLF 4, OMR 3).
- get_income_statement validated account_ids against all visible
accounts, but totals_for excludes hidden, excluded-from-reports and
tax-advantaged accounts, so those ids produced silent zeros. Ids now
validate against income_statement.eligible_accounts and the soft
failure explains eligibility.
- get_recurring_transactions computed totals from the displayed rows,
so past the 200-row cap the value labeled a total was partial. Totals
now aggregate over the full filtered scope in SQL, and the response
carries total_results and a truncated flag. The upcoming_within_days
window also starts at today, matching its documentation; overdue
items appear in unwindowed calls.
- get_valuations silently dropped a malformed date filter and presented
unfiltered data as filtered; malformed dates now return invalid_date.
- get_balance_sheet returned a generic failure for a reversed custom
range because Period's own validation raises past the Date::Error
rescue; it now returns the structured invalid_date error.
- get_insights documents that its family-wide scope matches the web
feed exactly (InsightsController serves Current.family.insights to
every member), so the tool exposes nothing the /insights page does
not already show the same user.
- Tests: limit clamp proven against more insights than the cap,
Setting fallbacks stubbed in the provider budget tests, currency
precision and reversed-range regression tests added.
* refactor(assistant): apply reviewer nitpicks
- order declares type alongside its enum, matching sort_by
- page-size clamp deduplicated into the base class (MAX_PAGE_SIZE +
shared resolved_page_size); dead per-tool copies removed
- get_accounts preloads balance rows only when the series is requested
- get_income_statement validates the bucket count before running any
aggregation work
Deliberately unchanged: the balance sheet's monthly_history key. The
default response shape stays byte-compatible for existing MCP
consumers, and the nested series already states its interval.
* fix(assistant): second-round review findings on get_valuations
- A reversed date range (start after end) now returns the structured
invalid_date error instead of presenting an empty result as filtered
data, matching get_balance_sheet's handling.
- Page numbers are normalized before pagination: Pagy raises on zero,
negative or non-numeric pages. The fix lands as a shared
resolved_page helper on the base class and applies to every
paginated tool (categories, tags, merchants, transactions, holdings,
valuations), since all shared the same page-or-1 pattern; schemas
declare minimum: 1.
* fix(assistant): round series amounts as BigDecimal before Float conversion
Converting to Float first can perturb the value at the requested
precision; round the exact decimal, then convert for JSON.
* fix(assistant): address maintainer review findings
- get_accounts no longer fails the whole listing when one account's
start date lies beyond the requested period (start_date derives from
the first entry, which can be future-dated); that account simply has
no series. The unrescued Period.custom was reachable exactly there.
- The balances preload is gone: the series goes through
Balance::ChartSeriesBuilder, which runs its own query keyed by
account ids, so the eager-loaded rows were loaded and discarded.
- Provider::Openai#context_window now delegates to
Assistant::TokenBudget, removing the duplicated ENV > Setting >
default precedence so prompt assembly and the provider can never
disagree about the window.
* fix(ai): final no-tools round uses tool_choice none instead of dropping tools
Anthropic rejects requests whose messages contain tool_use blocks when
no tools are defined, so re-requesting with an empty tool list made the
final-round grace die in a provider 400 on Anthropic models. The final
round now sends the real tool definitions with tool_choice none, which
both providers accept, and the model answers in prose as intended.
* fix(assistant): scope every income statement read to the requesting user
get_income_statement validated account_ids against the user-scoped statement
but computed every total from an unscoped one. IncomeStatement falls back to
Current.user, which is nil in the assistant job and the MCP endpoint, so the
unscoped reads dropped the included_in_finances_for filter and reported
family-wide totals next to ids that had been checked against a narrower set.
Route all reads through one memoized user-scoped statement, the idiom
get_balance_sheet already uses. Also lets the per-instance memoization in
IncomeStatement apply across the eligibility check and the totals.
Adds a regression test that fails without the change, plus a companion test
asserting eligibility and totals agree on scope. Guard the strictness walk
against an empty registry so it cannot silently assert nothing.
|
||
|
Ellion BlessanandClaude Sonnet 5
|
e69894adb9 |
fix(providers): capture API error response body in PDF processor span output (#2937)
* fix(providers): capture API error response body in PDF processor span output Anthropic and OpenAI PDF processing errors only logged the exception message, dropping the parsed response body that usually explains the failure. Add safe_error_body to both providers' UsageRecorder concerns and include it in the langfuse span output on failure, with tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 * fix(providers): allowlist PDF processor error fields sent to Langfuse safe_error_body forwarded the entire upstream error body into the langfuse span output. For custom OpenAI-compatible providers/proxies (and the analogous Anthropic path), that body can echo request content from the financial document being processed. Replace it with safe_error_detail, which extracts only type/message/code/request_id instead of the raw body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 * test(openai): cover request_id extraction in PDF processor error_detail The safe_error_detail request_id path (error.response_headers) had no test coverage. Stub response_headers with x-request-id and assert it appears in error_detail. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Abhinav Dhiman
|
e3d46021c2 |
fix: memory leak in sidekiq (#1940)
* perf(sync): reduce per-job memory peak in Balance/Holding materialization
Profiling of SyncJob (StackProf object mode + Sidekiq memory middleware)
showed peaks of 600k-1.1M live heap slots per job and ~196k retained
ActiveModel::Attribute::FromUser objects post-GC, driven by full-history
in-memory accumulation in the balance/holding sync pipeline.
Changes:
- Replace Holding.new / Balance.new in calculators with lightweight
Struct-based HoldingData / BalanceData. Skips AR attribute sets,
belongs_to proxies, dirty tracking, type casting, and callbacks
that were never used (upsert_all bypasses validations/callbacks
anyway). Eliminates ~30% of allocations and the bulk of retained
ActiveModel::Attribute::* instances.
- Build upsert payloads directly from struct fields instead of
Holding/Balance#attributes.slice(...).
- Batch upsert_all in PERSIST_BATCH_SIZE (2,000) slices in both
Balance::Materializer and Holding::Materializer so the intermediate
attribute-hash array is bounded instead of holding the full
multi-year history alongside the calculator output.
- Replace account.holdings.reload with account.holdings.reset in
Holding::Materializer. Same cache invalidation, no eager re-query;
the next consumer (Balance::SyncCache) loads on demand.
- Mutate entries in place in Balance::SyncCache#converted_entries
instead of Entry#dup. The instances are scoped to the throwaway
sync-cache and never persisted, so dup'ing was producing tens of
thousands of unused FromUser/FromDatabase attribute wrappers per
sync.
All persist paths run inside the existing Balance.transaction wrapper,
so batched upserts retain transactional atomicity. No production caller
of Balance::SyncCache or Holding::Materializer reuses the affected
instances outside the materializer's lifetime.
Test coverage: balance/{sync_cache,materializer,forward_calculator,
reverse_calculator} and holding/{materializer,forward_calculator,
reverse_calculator} plus account/syncer and sync (82 runs, 2,548
assertions, 0 failures).
* refactor(holding): stream materializer upserts to bound peak memory
Replace full-array accumulation + each_slice in Materializer#persist_holdings
with two flush-on-fill buffers (holdings_buffer_to_upsert_with_cost /
holdings_buffer_to_upsert_without_cost) that upsert and clear at
PERSIST_BATCH_SIZE, keeping peak RSS bounded to ~2x batch size.
Also add assert_not_nil guards in ReverseCalculatorTest before
dereferencing calculated.find results to surface clear failures
instead of NoMethodError.
* refactor(balance): promote BalanceData to Balance namespace and document sync mutation safety
- Extract Balance::BalanceData struct into its own file (app/models/balance/balance_data.rb)
so it is discoverable without knowing it lived inside BaseCalculator
- Remove inline Struct definition from Balance::BaseCalculator; update build_balance
to reference Balance::BalanceData explicitly (required because class Foo::Bar syntax
does not nest Foo in constant lookup)
- Add comment to SyncCache#converted_entries clarifying that to_a materialises
independent AR instances with no identity map active, making in-place mutation safe
- Update all Balance::BaseCalculator::BalanceData references in materializer_test
* test(balance): use to_h instead of attributes on BalanceData struct in waypoint test
* perf(balance,market_data): replace sort_by + first/last with minmax_by and push account-entry join to SQL
- Balance::Materializer#purge_stale_balances: replace sort_by(&:date) + first/last with minmax_by(&:date) to avoid full sort when only min/max are needed
- MarketDataImporter: replace Entry.group(:account_id).minimum(:date) (which loads all account IDs into a Hash) with a LEFT JOIN subquery that computes MIN(date) per account in SQL and exposes it as first_entry_date on the Account relation
* test(balance): use BalanceData struct in materializer purge test
* fix(holding): carry cost_basis forward onto gap-filled dates
|
||
|
Sure Admin (bot)
|
5c18086089 |
docs: refresh Sure MCP and external AI setup (#2608)
* docs: explain self-hosted onboarding modes * docs: refresh MCP and external AI docs * docs: correct MCP auth and tool accuracy * docs: address MCP review comments |
||
|
Atlasandsure-admin
|
eb382b8899 |
Fix onboarding country and currency defaults (#3070)
* fix: derive onboarding currency from country * feat: use browser locale for onboarding defaults * Fix onboarding currency defaults * Preserve saved onboarding currency during hydration --------- Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
|
fd01a5b3dc |
fix(accounts): remember transaction page size across account navigation (#3084)
* fix(accounts): remember transaction page size across account navigation per_page was only ever read from the current request's query string, so switching accounts always reset the activity feed back to 10 entries. Reuses TransactionsController's existing prev_transaction_page_params session store so the chosen page size applies consistently on both the account detail page and the global transactions page. Closes #3082 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(accounts): validate stored per_page and preserve it across filtered requests Addresses review feedback on #3084: safe_per_page now validates the stored default against the allowed values (a raw stored value like "1" was previously passed through unchecked), TransactionsController no longer wipes the remembered per_page when a request supplies filters but omits per_page, and Session#prev_transaction_page_params normalizes a NULL value to an empty hash. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(transactions): use stored per_page as pagy fallback on filtered requests store_params! already preserved the previously-selected per_page in the session when a filtered request (e.g. dashboard money-flow links) omitted it, but TransactionsController#index still called safe_per_page with its hardcoded default of 10, so the stored preference was never applied to the actual page rendered. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
sentry[bot]andsentry[bot] <39604003+sentry[bot]@users.noreply.github.com>
|
3c299a76f0 |
fix(demo): Scope budget auto-fill to family entries to prevent FK violations (#3078)
Co-authored-by: sentry[bot] <39604003+sentry[bot]@users.noreply.github.com> |
||
|
|
e5750a6c09 |
feat(budgets): add per-user personal budgets with strict isolation (#2891)
* feat(budgets): add per-user personal budgets with strict isolation Families can now opt into personal budgets (toggleable via family settings): each family member gets their own budget for a given period instead of sharing a single family-wide budget. - Add families.personal_budgets flag and budgets.user_id, with partial unique indexes so shared budgets (user_id IS NULL) and personal budgets (user_id IS NOT NULL) can't collide. - Budget.find_or_bootstrap scopes lookup/creation by user when the family has personal_budgets enabled. - Scope most_recent_initialized_budget (used to seed a new budget from the prior period) by user_id so one user's copy-forward never bleeds into another user's budget. - budgets.user_id cascades on user deletion so personal budgets don't outlive their owner. * feat(budgets): enforce user-specific budget ownership and cascade deletion * feat(budgets): display user name for personal budgets in budget card on the plan section * feat(budgets): enhance personal budgets display for admins with preview feature indication * feat(budgets): enforce user-specific budget and category visibility for personal budgets * feat(budgets): create budget section titles and add translations notice in preferences * feat(budgets): let household and personal budgets coexist with sharing Previously enabling personal_budgets made the shared household budget unreachable. Budget.find_or_bootstrap now takes an explicit household: flag so both can be resolved independently for the same period, with a new household_budget_enabled family setting to opt out of the household side and keep personal budgets only. Adds a BudgetShare model (read_only/read_write) so a member can grant another family member access to their personal budget, enforced via Budget#viewable_by?/editable_by? across BudgetsController, BudgetCategoriesController, PlansController, and the read-only API. Preferences gains a Budget sharing card (gated on preview access like the rest of the personal budgets UI) and an owner switcher pill ( Household / mine / shared-with-me) appears on the budget page and the Plan hub card. Also fixes personal budgets showing the same "actual spending" as the household budget: actual spending/income now scope to the budget owner's own accounts instead of the viewer's full accessible set, via a new accounts: override on IncomeStatement. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(budgets): enhance budget switcher with icons and improved styling * feat(budgets): remove user name display from budget card and header * feat(budgets): remove unique index on taggable_type and taggable_id in taggings * feat(budgets): enhance budget sharing functionality and improve UI elements * Collapse personal budget migrations --------- Signed-off-by: JulienGourmet <69808509+jubbakka@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
github-actions[bot]andgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
8cfd4c9c7d |
Bump version to next iteration after v0.7.4-alpha.6 release (#3075)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Juan José MataandClaude
|
9dfcf9c823 |
Remove unfinished Cloudflare Workers PR preview deployments (#3074)
The Cloudflare Containers preview stack was never finished and no open PR carries the `preview-cf` label, so nothing is actively using it. Meanwhile the deploy workflow fires on every completed "Pull Request" run and spins up a runner just to evaluate its gate and skip. Removed entirely: - `workers/preview/` — the Worker (Durable Object container orchestration), wrangler config, npm manifests, and the `deploy/` resolver, config renderer, and log redaction helpers - `Dockerfile.preview` — Rails image with embedded PostgreSQL/Redis and the inline preview entrypoint - `.github/workflows/preview-deploy.yml` and `preview-cleanup.yml` - `bin/preview_deploy_security_check.rb` — CI guard for the above workflows - `test/javascript/preview_deploy/` — tests for the deleted deploy helpers Workflow cleanup: - `pr.yml`: drop the `preview_image` job. Also drop the `labeled` trigger type, which existed only so labelling `preview-cf` re-triggered the build and would otherwise re-run full CI on every label change. - `ci.yml`: drop the preview hardening validation step. - `pipelock.yml`: drop the exclude-paths entries for the deleted files. Deliberately untouched: the per-user preview *feature* gate (`PreviewGateable`, `config/locales/views/preview/`), Cloudflare R2 Active Storage config, and the Cloudflare Workers AI demo banner strings — all unrelated to preview deployments. Claude-Session: https://claude.ai/code/session_017TrvqNZqqzN74X7PmznJ5X Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Scott HughesandClaude Opus 5
|
5bba880e66 |
Fix SnapTrade holdings by using the /positions/all endpoint (#3043)
* Fix SnapTrade holdings by using the /positions/all endpoint SnapTrade returns HTTP 410 Gone on /positions, /holdings and /options for apps registered after their 2026 cutoff, so newly connected accounts import their balance but never any holdings. The importer rescues the error and the sync still reports success, which makes it look like the brokerage simply holds nothing. Switch get_positions to the documented replacement, /positions/all, and teach the shared payload helpers to read its flat `instrument` object alongside the legacy nested `symbol.symbol` shape. Fixes #3029 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Filter unsupported instrument kinds before they reach the payload Derivatives skipped by HoldingsProcessor still landed in raw_holdings_payload, where calculate_holdings_value sums units * price over every entry. Any non-zero sum then displaces SnapTrade's own figure in calculate_total_balance, so an options-only account could report a balance derived from per-contract units against a per-share price. Move the denylist to Provider::Snaptrade#get_positions so unsupported kinds never enter the payload at all, keeping holdings, balance, currency detection and cash-equivalent handling consistent with one filter. Also raise when the positions response carries no results array, so a partial or schema-changed response leaves the previous snapshot in place instead of overwriting it with nothing. An empty results array remains a legitimately empty account. Note that tax_lots[].cost_basis is documented as a whole-lot total, unlike the per-share field this reads. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
e97bc7aaf5 |
i18n: complete Spanish (es) translations (#2669)
* i18n: complete Spanish (es) translations
Add 23 new es.yml files and complete 59 existing ones, covering
2,547 previously missing keys across config/locales (views, models,
breadcrumbs, defaults, mailers). Also fix 11 pre-existing strings
that dropped %{...} interpolations from the English source.
Validated: all es.yml parse, every es key set is a superset of its
en.yml counterpart, and all English interpolations are preserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address Spanish i18n review feedback
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: webprojectxeventos-sudo <webprojectxeventos-sudo@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: sure-admin <sure-admin@splashblot.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
|