mirror of
https://github.com/we-promise/sure.git
synced 2026-09-05 23:01:26 +00:00
* feat(bills): schema and domain core for the bills subsystem First of three chunks carved out of #3083. This one carries the schema and the domain layer: no bills pages, no calendar feed, no assistant tools. Nothing here is reachable from the UI yet, so it changes no user-visible behavior on its own. Schema, in a single migration with a full down: - recurrence_rules, recurring_occurrences, recurring_allocations, recurring_price_changes and recurring_match_rejections - bill columns on recurring_transactions (bill_type, payment_url, autopay, notes, anchor and end conditions, weekend adjustment, dedup scope) - the four data backfills, in their original order Domain layer: - Schedule, the pure date PORO every cadence resolves through, and FrequencyPreset for the labels - OccurrenceGenerator, Matcher, Allocator, PriceChangeDetector, Classifier, DeclaredBill, HistoryBackfiller and PaycheckPlanner - Pipeline, tying detection to generation, plus the nightly job and rake task Existing detection code changed in three places, each a bug this schema exposes: - Cleaner used a flat two-month staleness threshold, which silently retired every quarterly and annual series - SubscriptionAuditGenerator used a flat 45-day overdue threshold, meaningless at both ends of the frequency range - CashFlowWarningGenerator read one projected entry per series, which only equalled the monthly amount because every series was monthly; weekly bills were under-counted fourfold in its 30-day projection The JSON API travels with the model rather than the UI, because the status enum widens here. The API accepts only active and inactive on write; suggested, paused and ended are lifecycle states owned by detection, so the documented enum stays truthful. Uniqueness keys gain dedup_scope alongside amount, never instead of it: a series that is not price-forked carries a blank scope, so amount is what keeps two different prices apart. Suite 7,550 runs, 0 failures. Rubocop and brakeman clean. Eager loading verified, and the migration reverses and re-applies. Includes the first review round: orphan repair matches income and refuses coincidental twins, session imports persist occurrence mappings across chunks, semimonthly anchors canonicalize, classifier keywords match whole words, and the down refuses rather than failing when price-forked rows exist. * Address second review round Bound the cross-currency default allocation by the entry leftover and the occurrence remainder, matching the same-currency path. Let keyword stems carry a suffix again after the word-boundary fix silenced them. Skip an incoherent recurrence rule row instead of rolling back the whole import. Check rollback collisions per restored index so a refusal cannot land after the bills tables are dropped. Replay the closed_at test through a real second import. Preload the orphan repair associations and move the allocator errors to locale keys. * Match index NULL semantics in the rollback collision checks GROUP BY treats NULLs as equal but the restored unique indexes do not: account_id is nullable and indexed, so two accountless rows can never collide under any of them. Excluding NULL accounts keeps the guard from refusing a rollback PostgreSQL can perform. Verified live both ways: accountless duplicates roll back, a real collision still refuses. * Address maintainer review Scope the payable debt-destination subquery to the row and its family instead of scanning every account in the installation. Batch the cash flow generator remaining-amount sums into one grouped query, matching the two sibling sites. Enforce both window bounds in the after_count branch so a future-anchored plan cannot leak past the requested end date. Skip the explicit regeneration when the day column change will fire the model callback anyway. Add the missing locale entry for the allocation currency validation.
121 lines
4.3 KiB
Ruby
121 lines
4.3 KiB
Ruby
class RecurringTransaction
|
|
# Rule-based first guess at what a detected charge is: a subscription (a
|
|
# service auto-charging a card on file) or a bill (an obligation you push
|
|
# money at). Transparent by design -- keyword lists and three heuristics, no
|
|
# scoring -- and only a default, since Kind stays user-editable and detection
|
|
# never reclassifies after creation.
|
|
#
|
|
# Category is inherited from the most common one across the cluster's
|
|
# entries.
|
|
class Classifier
|
|
# Services that are subscriptions essentially always.
|
|
SUBSCRIPTION_KEYWORDS = %w[
|
|
netflix spotify hulu disney hbo hbo\ max paramount peacock crunchyroll
|
|
youtube prime audible kindle icloud apple.com/bill google\ one
|
|
playstation xbox nintendo twitch patreon substack onlyfans
|
|
dropbox github openai anthropic claude chatgpt midjourney canva adobe
|
|
microsoft\ 365 office\ 365 notion slack zoom 1password lastpass
|
|
bitwarden nordvpn expressvpn sirius pandora tidal deezer duolingo
|
|
headspace calm grammarly peloton planet\ fitness la\ fitness
|
|
grok x.ai xai
|
|
].freeze
|
|
|
|
# Wording that marks classic push-payment obligations: utilities,
|
|
# telecom, insurance, housing, taxes.
|
|
BILL_KEYWORDS = %w[
|
|
electric power energy gas water sewer utility utilit* insurance insur*
|
|
mortgage rent lease loan hoa property tax comcast xfinity spectrum
|
|
cox centurylink frontier at&t verizon t-mobile tmobile mint\ mobile
|
|
wireless phone internet interest finance\ charge
|
|
].freeze
|
|
|
|
# Buy-now-pay-later and financing: a fixed run of payments, then done.
|
|
INSTALLMENT_KEYWORDS = %w[klarna affirm afterpay sezzle zip\ pay uplift].freeze
|
|
|
|
# Push-payment fingerprints in raw descriptors.
|
|
ACH_MARKERS = %w[ach web\ pmt webpmt billpay bill\ pay online\ pmt e-pay epay].freeze
|
|
|
|
# Above this, a flat recurring charge is more likely rent-or-service
|
|
# than a streaming plan.
|
|
SUBSCRIPTION_AMOUNT_CEILING = BigDecimal("150")
|
|
|
|
Result = Data.define(:bill_type, :category_id, :autopay)
|
|
|
|
def self.classify(name:, entries:, account: nil)
|
|
new(name: name, entries: entries, account: account).classify
|
|
end
|
|
|
|
attr_reader :name, :entries, :account
|
|
|
|
def initialize(name:, entries:, account: nil)
|
|
@name = name.to_s.downcase
|
|
@entries = entries
|
|
@account = account
|
|
end
|
|
|
|
def classify
|
|
kind =
|
|
if matches?(INSTALLMENT_KEYWORDS)
|
|
"installment"
|
|
elsif subscription?
|
|
"subscription"
|
|
else
|
|
"bill"
|
|
end
|
|
|
|
Result.new(
|
|
bill_type: kind,
|
|
category_id: modal_category_id,
|
|
# Subscriptions and BNPL plans are auto-charges: on the list, not a
|
|
# task.
|
|
autopay: kind != "bill"
|
|
)
|
|
end
|
|
|
|
private
|
|
def subscription?
|
|
return false if matches?(BILL_KEYWORDS) || matches?(ACH_MARKERS)
|
|
return true if matches?(SUBSCRIPTION_KEYWORDS)
|
|
|
|
# No name signal: an identical-to-the-cent modest charge on a credit
|
|
# card is the shape of a card-on-file service.
|
|
flat_amounts? && modest_amount? && credit_card_account?
|
|
end
|
|
|
|
# Whole words only. A substring test let "max" claim Maxwell Plumbing
|
|
# and "gas" claim a gastropub, and a false subscription match also set
|
|
# autopay, which removes the row from the needs-action list. A trailing
|
|
# "*" marks a stem that may carry a suffix, so "utilit*" still reaches
|
|
# "UTILITIES" without a bare stem matching inside unrelated words.
|
|
def matches?(keywords)
|
|
keywords.any? do |keyword|
|
|
if keyword.end_with?("*")
|
|
name.match?(/\b#{Regexp.escape(keyword.delete_suffix('*'))}\w*/)
|
|
else
|
|
name.match?(/\b#{Regexp.escape(keyword)}\b/)
|
|
end
|
|
end
|
|
end
|
|
|
|
def flat_amounts?
|
|
amounts = entries.map { |entry| entry.amount.abs }
|
|
amounts.uniq.size == 1
|
|
end
|
|
|
|
def modest_amount?
|
|
entries.first.amount.abs <= SUBSCRIPTION_AMOUNT_CEILING
|
|
end
|
|
|
|
def credit_card_account?
|
|
account&.accountable_type == "CreditCard"
|
|
end
|
|
|
|
def modal_category_id
|
|
entries.filter_map { |entry| entry.entryable.try(:category_id) }
|
|
.tally
|
|
.max_by { |_category_id, count| count }
|
|
&.first
|
|
end
|
|
end
|
|
end
|