0xRozier 662ec1fc92 Fix/issue 954 enable banking duplicate transactions (#988)
* fix: deduplicate Enable Banking API transactions with different entry_reference IDs (#954)

Enable Banking API sometimes returns the same logical transaction multiple
times with different entry_reference values in a single response. This causes
duplicate entries because the existing ID-based deduplication treats them as
distinct transactions.

Add content-based deduplication that compares (date, amount, currency,
creditor, debtor, remittance_information, status) to detect and remove these
API-level duplicates before storing them. The first occurrence is kept.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add Enable Banking processor and importer deduplication tests (#954)

Add tests for:
- EnableBankingEntry::Processor: verifies entry_reference fallback for
  external_id, idempotent re-processing, string key handling
- EnableBankingItem::Importer: verifies content-based deduplication removes
  API-level duplicates while preserving legitimate distinct transactions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle nil values in remittance_information array for dedup key (#954)

Call compact and map(&:to_s) before sort.join when remittance_information
is an array, preventing ArgumentError when it contains nil elements.
Also document the known limitation of content-based deduplication
collapsing genuinely distinct identical transactions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add coverage for nil values in remittance_information array (#954)

Verify that deduplication handles remittance_information arrays containing
nil elements without raising ArgumentError, and correctly treats arrays
with different nil positions but same non-nil content as duplicates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prefer transaction_id over content-based dedup to preserve legit duplicates (#954)

When transaction_id is present, use it as the dedup key instead of falling
back to content-based deduplication. This preserves legitimately distinct
transactions with identical content (e.g. two laundromat payments of the
same amount on the same day) while still deduplicating the Enable Banking
duplicate entry_reference issue when transaction_id is nil.

Addresses review feedback from @jjmata about legitimate duplicate
transactions being incorrectly collapsed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use composite key for dedup instead of transaction_id alone (#954)

Per the Enable Banking API docs, transaction_id is not guaranteed to be
unique. Include it as one component of the composite content key rather
than using it as the sole dedup criterion. This preserves transactions
with non-unique transaction_ids but different content, while still
deduplicating true API-level duplicates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add value_date fallback coverage for dedup key (#954)

build_transaction_content_key falls back to value_date when booking_date
is absent. This test exercises that path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: document known limitation of content-based dedup (#954)

When transaction_id is nil for both transactions, pure content comparison
applies, which could theoretically collapse two genuinely distinct
transactions with identical fields. Document this trade-off inline for
future maintainers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add credit_debit_indicator to dedup composite key (#954)

transaction_amount.amount is always positive in the Enable Banking API,
with direction encoded separately in credit_debit_indicator (CRDT/DBIT).
Without it in the composite key, a payment and a same-day refund of the
same amount to the same merchant would produce identical keys, silently
dropping one transaction.

- Add credit_debit_indicator to build_transaction_content_key
- Add test for payment + same-day refund scenario
- Update docstring to document the rationale

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 20:31:25 +02:00
2026-01-30 00:34:46 +01:00
2025-05-20 13:31:05 -05:00
2024-08-23 10:06:24 -04:00
2024-08-23 10:06:24 -04:00
2026-03-24 19:11:58 +01:00
2026-03-24 19:11:58 +01:00
2025-08-05 23:35:01 +02:00
2025-09-24 00:19:51 +02:00

Ask DeepWiki View performance data on Skylight Dosu

sure_shot

Deutsch | Español | Français | 日本語 | 한국어 | Português | Русский | 中文

Sure: The personal finance app for everyone

Get involved: DiscordWebsiteIssues

Important

This repository is a community fork of the now-abandoned Maybe Finance project.
Learn more in their final release doc.

Backstory

The Maybe Finance team spent most of 20212022 building a full-featured personal finance and wealth management app. It even included an “Ask an Advisor” feature that connected users with a real CFP/CFA — all included with your subscription.

The business end of things didn't work out, and so they stopped developing the app in mid-2023.

After spending nearly $1 million on development (employees, contractors, data providers, infra, etc.), the team open-sourced the app. Their goal was to let users self-host it for free — and eventually launch a hosted version for a small fee.

They actually did launch that hosted version … briefly.

That also didnt work out — at least not as a sustainable B2C business — so now here we are: hosting a community-maintained fork to keep the codebase alive and see where this can go next.

Join us!

Hosting Sure

Sure is a fully working personal finance app that can be self hosted with Docker.

Forking and Attribution

This repo is a community fork of the archived Maybe Finance repo. Youre free to fork it under the AGPLv3 license — but wed love it if you stuck around and contributed here instead.

To stay compliant and avoid trademark issues:

  • Be sure to include the original AGPLv3 license and clearly state in your README that your fork is based on Maybe Finance but is not affiliated with or endorsed by Maybe Finance Inc.
  • "Maybe" is a trademark of Maybe Finance Inc. and therefore, use of it is NOT allowed in forked repositories (or the logo)

Performance Issues

With data-heavy apps, inevitably, there are performance issues. We've set up a public dashboard showing the problematic requests seen on the demo site, along with the stacktraces to help debug them.

https://www.skylight.io/app/applications/s6PEZSKwcklL/recent/6h/endpoints

Any contributions that help improve performance are very much welcome.

Local Development Setup

If you are trying to self-host the app, read this guide to get started.

The instructions below are for developers to get started with contributing to the app.

Requirements

  • See .ruby-version file for required Ruby version
  • PostgreSQL >9.3 (latest stable version recommended)
  • Redis > 5.4 (latest stable version recommended)

Getting Started

cd sure
cp .env.local.example .env.local
bin/setup
bin/dev

# Optionally, load demo data
rake demo_data:default

Visit http://localhost:3000 to view the app.

If you loaded the optional demo data, log in with these credentials:

  • Email: user@example.com
  • Password: Password1!

For further instructions, see guides below.

Setup Guides

One-click

Run on PikaPods

Deploy on Railway

License and Trademarks

Maybe and Sure are both distributed under an AGPLv3 license.

  • "Maybe" is a trademark of Maybe Finance, Inc.
  • "Sure" is not, and refers to this community fork.
Description
No description provided
Readme AGPL-3.0 76 MiB
Languages
Ruby 73%
HTML 16.8%
Dart 6%
JavaScript 3.3%
CSS 0.5%
Other 0.3%