QT/sure
1
0
Fork 0
mirror of https://github.com/we-promise/sure.git synced 2026-04-19 12:04:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

Full .ndjson import / reorganize UI with Financial Tools / Raw Data tabs (#1208)

Browse Source 
* Reorganize import UI with Financial Tools / Raw Data tabs

Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix import controller test to account for YNAB coming soon entry

The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix system tests to click Raw Data tab before selecting import type

Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* feat: Add bulk import for NDJSON export files

Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)

Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter

* fix: Fix category import and test query issues

- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax

* Linter errors

* fix: Add default color for tags when not provided in import

* fix: Add default kind for transactions when not provided in import

* Fix test

* Fix tests

* Fix remaining merge conflicts from PR 766 cherry-pick

Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Import Sure `.ndjson`

* Remove `.ndjson` import from raw data

* Fix support for Sure "bulk" import from old branch

* Linter

* Fix CI test

* Fix more CI tests

* Fix tests

* Fix tests / move PDF import to first tab

* Remove redundant title

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Juan José Mata
2026-03-23 14:27:41 +01:00
committed by GitHub
parent 998ea6f7c5
commit 2595885eb7
32 changed files with 1960 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
app/controllers/import/uploads_controller.rb
View File

@@ -16,6 +16,8 @@ class Import::UploadsController < ApplicationController
def update
if @import.is_a?(QifImport)
handle_qif_upload
elsif @import.is_a?(SureImport)
update_sure_import_upload
elsif csv_valid?(csv_str)
@import.account = Current.family.accounts.find_by(id: import_account_id)
@import.assign_attributes(raw_file_str: csv_str, col_sep: upload_params[:col_sep])
@@ -23,13 +25,54 @@ class Import::UploadsController < ApplicationController
redirect_to import_configuration_path(@import, template_hint: true), notice: "CSV uploaded successfully."
else
flash.now[:alert] = "Must be valid CSV with headers and at least one row of data"
render :show, status: :unprocessable_entity
update_csv_import
end
end
private
def update_csv_import
if csv_valid?(csv_str)
@import.account = Current.family.accounts.find_by(id: params.dig(:import, :account_id))
@import.assign_attributes(raw_file_str: csv_str, col_sep: upload_params[:col_sep])
@import.save!(validate: false)
redirect_to import_configuration_path(@import, template_hint: true), notice: t("imports.create.csv_uploaded")
else
flash.now[:alert] = t("import.uploads.show.csv_invalid", default: "Must be valid CSV with headers and at least one row of data")
render :show, status: :unprocessable_entity
end
end
def update_sure_import_upload
uploaded = upload_params[:ndjson_file]
unless uploaded.present?
flash.now[:alert] = t("import.uploads.sure_import.ndjson_invalid", default: "Must be valid NDJSON with at least one record")
render :show, status: :unprocessable_entity
return
end
if uploaded.size > SureImport::MAX_NDJSON_SIZE
flash.now[:alert] = t("imports.create.file_too_large", max_size: SureImport::MAX_NDJSON_SIZE / 1.megabyte)
render :show, status: :unprocessable_entity
return
end
content = uploaded.read
uploaded.rewind
if ndjson_valid?(content)
uploaded.rewind
@import.ndjson_file.attach(uploaded)
@import.sync_ndjson_rows_count!
redirect_to import_path(@import), notice: t("imports.create.ndjson_uploaded")
else
flash.now[:alert] = t("import.uploads.sure_import.ndjson_invalid", default: "Must be valid NDJSON with at least one record")
render :show, status: :unprocessable_entity
end
end
def set_import
@import = Current.family.imports.find(params[:import_id])
end
@@ -71,8 +114,23 @@ class Import::UploadsController < ApplicationController
end
end
def ndjson_valid?(str)
return false if str.blank?
# Check at least first line is valid NDJSON
first_line = str.lines.first&.strip
return false if first_line.blank?
begin
record = JSON.parse(first_line)
record.key?("type") && record.key?("data")
rescue JSON::ParserError
false
end
end
def upload_params
params.require(:import).permit(:raw_file_str, :import_file, :col_sep)
params.require(:import).permit(:raw_file_str, :import_file, :ndjson_file, :col_sep)
end
def import_account_id

40
app/controllers/imports_controller.rb
View File

@@ -49,6 +49,11 @@ class ImportsController < ApplicationController
return
end
if file.present? && sure_import_request?
create_sure_import(file)
return
end
# Handle PDF file uploads - process with AI
if file.present? && Import::ALLOWED_PDF_MIME_TYPES.include?(file.content_type)
unless valid_pdf_file?(file)
@@ -85,6 +90,7 @@ class ImportsController < ApplicationController
# Stream reading is not fully applicable here as we store the raw string in the DB,
# but we have validated size beforehand to prevent memory exhaustion from massive files.
import.update!(raw_file_str: file.read)
redirect_to import_configuration_path(import), notice: t("imports.create.csv_uploaded")
else
redirect_to import_upload_path(import)
@@ -200,6 +206,40 @@ class ImportsController < ApplicationController
params.dig(:import, :type) == "DocumentImport"
end
def sure_import_request?
params.dig(:import, :type) == "SureImport"
end
def create_sure_import(file)
if file.size > SureImport::MAX_NDJSON_SIZE
redirect_to new_import_path, alert: t("imports.create.file_too_large", max_size: SureImport::MAX_NDJSON_SIZE / 1.megabyte)
return
end
ext = File.extname(file.original_filename.to_s).downcase
unless ext.in?(%w[.ndjson .json])
redirect_to new_import_path, alert: t("imports.create.invalid_ndjson_file_type")
return
end
content = file.read
file.rewind
unless SureImport.valid_ndjson_first_line?(content)
redirect_to new_import_path, alert: t("imports.create.invalid_ndjson_file_type")
return
end
import = Current.family.imports.create!(type: "SureImport")
import.ndjson_file.attach(
io: StringIO.new(content),
filename: file.original_filename,
content_type: file.content_type
)
import.sync_ndjson_rows_count!
redirect_to import_path(import), notice: t("imports.create.ndjson_uploaded")
end
def valid_pdf_file?(file)
header = file.read(5)
file.rewind

7
app/helpers/imports_helper.rb
View File

@@ -35,7 +35,12 @@ module ImportsHelper
accounts: DryRunResource.new(label: "Accounts", icon: "layers", text_class: "text-orange-500", bg_class: "bg-orange-500/5"),
categories: DryRunResource.new(label: "Categories", icon: "shapes", text_class: "text-blue-500", bg_class: "bg-blue-500/5"),
tags: DryRunResource.new(label: "Tags", icon: "tags", text_class: "text-violet-500", bg_class: "bg-violet-500/5"),
rules: DryRunResource.new(label: "Rules", icon: "workflow", text_class: "text-green-500", bg_class: "bg-green-500/5")
rules: DryRunResource.new(label: "Rules", icon: "workflow", text_class: "text-green-500", bg_class: "bg-green-500/5"),
merchants: DryRunResource.new(label: "Merchants", icon: "store", text_class: "text-amber-500", bg_class: "bg-amber-500/5"),
trades: DryRunResource.new(label: "Trades", icon: "arrow-left-right", text_class: "text-emerald-500", bg_class: "bg-emerald-500/5"),
valuations: DryRunResource.new(label: "Valuations", icon: "trending-up", text_class: "text-pink-500", bg_class: "bg-pink-500/5"),
budgets: DryRunResource.new(label: "Budgets", icon: "wallet", text_class: "text-indigo-500", bg_class: "bg-indigo-500/5"),
budget_categories: DryRunResource.new(label: "Budget Categories", icon: "pie-chart", text_class: "text-teal-500", bg_class: "bg-teal-500/5")
}
map[key]

474
app/models/family/data_importer.rb Normal file
View File

@@ -0,0 +1,474 @@
class Family::DataImporter
SUPPORTED_TYPES = %w[Account Category Tag Merchant Transaction Trade Valuation Budget BudgetCategory Rule].freeze
ACCOUNTABLE_TYPES = Accountable::TYPES.freeze
def initialize(family, ndjson_content)
@family = family
@ndjson_content = ndjson_content
@id_mappings = {
accounts: {},
categories: {},
tags: {},
merchants: {},
budgets: {},
securities: {}
}
@created_accounts = []
@created_entries = []
end
def import!
records = parse_ndjson
Import.transaction do
# Import in dependency order
import_accounts(records["Account"] || [])
import_categories(records["Category"] || [])
import_tags(records["Tag"] || [])
import_merchants(records["Merchant"] || [])
import_transactions(records["Transaction"] || [])
import_trades(records["Trade"] || [])
import_valuations(records["Valuation"] || [])
import_budgets(records["Budget"] || [])
import_budget_categories(records["BudgetCategory"] || [])
import_rules(records["Rule"] || [])
end
{ accounts: @created_accounts, entries: @created_entries }
end
private
def parse_ndjson
records = Hash.new { |h, k| h[k] = [] }
@ndjson_content.each_line do |line|
next if line.strip.empty?
begin
record = JSON.parse(line)
type = record["type"]
next unless SUPPORTED_TYPES.include?(type)
records[type] << record
rescue JSON::ParserError
# Skip invalid lines
end
end
records
end
def import_accounts(records)
records.each do |record|
data = record["data"]
old_id = data["id"]
accountable_data = data["accountable"] || {}
accountable_type = data["accountable_type"]
# Skip if accountable type is not valid
next unless ACCOUNTABLE_TYPES.include?(accountable_type)
# Build accountable
accountable_class = accountable_type.constantize
accountable = accountable_class.new
accountable.subtype = accountable_data["subtype"] if accountable.respond_to?(:subtype=) && accountable_data["subtype"]
# Copy any other accountable attributes
safe_accountable_attrs = %w[subtype locked_attributes]
safe_accountable_attrs.each do |attr|
if accountable.respond_to?("#{attr}=") && accountable_data[attr].present?
accountable.send("#{attr}=", accountable_data[attr])
end
end
account = @family.accounts.build(
name: data["name"],
balance: data["balance"].to_d,
cash_balance: data["cash_balance"]&.to_d || data["balance"].to_d,
currency: data["currency"] || @family.currency,
accountable: accountable,
subtype: data["subtype"],
institution_name: data["institution_name"],
institution_domain: data["institution_domain"],
notes: data["notes"],
status: "active"
)
account.save!
# Set opening balance if we have a historical balance
if data["balance"].present?
manager = Account::OpeningBalanceManager.new(account)
manager.set_opening_balance(balance: data["balance"].to_d)
end
@id_mappings[:accounts][old_id] = account.id
@created_accounts << account
end
end
def import_categories(records)
# First pass: create all categories without parent relationships
parent_mappings = {}
records.each do |record|
data = record["data"]
old_id = data["id"]
parent_id = data["parent_id"]
# Store parent relationship for second pass
parent_mappings[old_id] = parent_id if parent_id.present?
category = @family.categories.build(
name: data["name"],
color: data["color"] || Category::UNCATEGORIZED_COLOR,
classification_unused: data["classification_unused"] || data["classification"] || "expense",
lucide_icon: data["lucide_icon"] || "shapes"
)
category.save!
@id_mappings[:categories][old_id] = category.id
end
# Second pass: establish parent relationships
parent_mappings.each do |old_id, old_parent_id|
new_id = @id_mappings[:categories][old_id]
new_parent_id = @id_mappings[:categories][old_parent_id]
next unless new_id && new_parent_id
category = @family.categories.find(new_id)
category.update!(parent_id: new_parent_id)
end
end
def import_tags(records)
records.each do |record|
data = record["data"]
old_id = data["id"]
tag = @family.tags.build(
name: data["name"],
color: data["color"] || Tag::COLORS.sample
)
tag.save!
@id_mappings[:tags][old_id] = tag.id
end
end
def import_merchants(records)
records.each do |record|
data = record["data"]
old_id = data["id"]
merchant = @family.merchants.build(
name: data["name"],
color: data["color"],
logo_url: data["logo_url"]
)
merchant.save!
@id_mappings[:merchants][old_id] = merchant.id
end
end
def import_transactions(records)
records.each do |record|
data = record["data"]
# Map account ID
new_account_id = @id_mappings[:accounts][data["account_id"]]
next unless new_account_id
account = @family.accounts.find(new_account_id)
# Map category ID (optional)
new_category_id = nil
if data["category_id"].present?
new_category_id = @id_mappings[:categories][data["category_id"]]
end
# Map merchant ID (optional)
new_merchant_id = nil
if data["merchant_id"].present?
new_merchant_id = @id_mappings[:merchants][data["merchant_id"]]
end
# Map tag IDs (optional)
new_tag_ids = []
if data["tag_ids"].present?
new_tag_ids = Array(data["tag_ids"]).map { |old_tag_id| @id_mappings[:tags][old_tag_id] }.compact
end
transaction = Transaction.new(
category_id: new_category_id,
merchant_id: new_merchant_id,
kind: data["kind"] || "standard"
)
entry = Entry.new(
account: account,
date: Date.parse(data["date"].to_s),
amount: data["amount"].to_d,
name: data["name"] || "Imported transaction",
currency: data["currency"] || account.currency,
notes: data["notes"],
excluded: data["excluded"] || false,
entryable: transaction
)
entry.save!
# Add tags through the tagging association
new_tag_ids.each do |tag_id|
transaction.taggings.create!(tag_id: tag_id)
end
@created_entries << entry
end
end
def import_trades(records)
records.each do |record|
data = record["data"]
# Map account ID
new_account_id = @id_mappings[:accounts][data["account_id"]]
next unless new_account_id
account = @family.accounts.find(new_account_id)
# Resolve or create security
ticker = data["ticker"]
next unless ticker.present?
security = find_or_create_security(ticker, data["currency"])
trade = Trade.new(
security: security,
qty: data["qty"].to_d,
price: data["price"].to_d,
currency: data["currency"] || account.currency
)
entry = Entry.new(
account: account,
date: Date.parse(data["date"].to_s),
amount: data["amount"].to_d,
name: "#{data["qty"].to_d >= 0 ? 'Buy' : 'Sell'} #{ticker}",
currency: data["currency"] || account.currency,
entryable: trade
)
entry.save!
@created_entries << entry
end
end
def import_valuations(records)
records.each do |record|
data = record["data"]
# Map account ID
new_account_id = @id_mappings[:accounts][data["account_id"]]
next unless new_account_id
account = @family.accounts.find(new_account_id)
valuation = Valuation.new
entry = Entry.new(
account: account,
date: Date.parse(data["date"].to_s),
amount: data["amount"].to_d,
name: data["name"] || "Valuation",
currency: data["currency"] || account.currency,
entryable: valuation
)
entry.save!
@created_entries << entry
end
end
def import_budgets(records)
records.each do |record|
data = record["data"]
old_id = data["id"]
budget = @family.budgets.build(
start_date: Date.parse(data["start_date"].to_s),
end_date: Date.parse(data["end_date"].to_s),
budgeted_spending: data["budgeted_spending"]&.to_d,
expected_income: data["expected_income"]&.to_d,
currency: data["currency"] || @family.currency
)
budget.save!
@id_mappings[:budgets][old_id] = budget.id
end
end
def import_budget_categories(records)
records.each do |record|
data = record["data"]
# Map budget ID
new_budget_id = @id_mappings[:budgets][data["budget_id"]]
next unless new_budget_id
# Map category ID
new_category_id = @id_mappings[:categories][data["category_id"]]
next unless new_category_id
budget = @family.budgets.find(new_budget_id)
budget_category = budget.budget_categories.build(
category_id: new_category_id,
budgeted_spending: data["budgeted_spending"].to_d,
currency: data["currency"] || budget.currency
)
budget_category.save!
end
end
def import_rules(records)
records.each do |record|
data = record["data"]
rule = @family.rules.build(
name: data["name"],
resource_type: data["resource_type"] || "transaction",
active: data["active"] || false,
effective_date: data["effective_date"].present? ? Date.parse(data["effective_date"].to_s) : nil
)
# Build conditions
(data["conditions"] || []).each do |condition_data|
build_rule_condition(rule, condition_data)
end
# Build actions
(data["actions"] || []).each do |action_data|
build_rule_action(rule, action_data)
end
rule.save!
end
end
def build_rule_condition(rule, condition_data, parent: nil)
value = resolve_rule_condition_value(condition_data)
condition = if parent
parent.sub_conditions.build(
condition_type: condition_data["condition_type"],
operator: condition_data["operator"],
value: value
)
else
rule.conditions.build(
condition_type: condition_data["condition_type"],
operator: condition_data["operator"],
value: value
)
end
# Handle nested sub_conditions for compound conditions
(condition_data["sub_conditions"] || []).each do |sub_condition_data|
build_rule_condition(rule, sub_condition_data, parent: condition)
end
condition
end
def build_rule_action(rule, action_data)
value = resolve_rule_action_value(action_data)
rule.actions.build(
action_type: action_data["action_type"],
value: value
)
end
def resolve_rule_condition_value(condition_data)
condition_type = condition_data["condition_type"]
value = condition_data["value"]
return value unless value.present?
# Map category names to IDs
if condition_type == "transaction_category"
category = @family.categories.find_by(name: value)
category ||= @family.categories.create!(
name: value,
color: Category::UNCATEGORIZED_COLOR,
classification_unused: "expense",
lucide_icon: "shapes"
)
return category.id
end
# Map merchant names to IDs
if condition_type == "transaction_merchant"
merchant = @family.merchants.find_by(name: value)
merchant ||= @family.merchants.create!(name: value)
return merchant.id
end
value
end
def resolve_rule_action_value(action_data)
action_type = action_data["action_type"]
value = action_data["value"]
return value unless value.present?
# Map category names to IDs
if action_type == "set_transaction_category"
category = @family.categories.find_by(name: value)
category ||= @family.categories.create!(
name: value,
color: Category::UNCATEGORIZED_COLOR,
classification_unused: "expense",
lucide_icon: "shapes"
)
return category.id
end
# Map merchant names to IDs
if action_type == "set_transaction_merchant"
merchant = @family.merchants.find_by(name: value)
merchant ||= @family.merchants.create!(name: value)
return merchant.id
end
# Map tag names to IDs
if action_type == "set_transaction_tags"
tag = @family.tags.find_by(name: value)
tag ||= @family.tags.create!(name: value)
return tag.id
end
value
end
def find_or_create_security(ticker, currency)
# Check cache first
cache_key = "#{ticker}:#{currency}"
return @id_mappings[:securities][cache_key] if @id_mappings[:securities][cache_key]
security = Security.find_by(ticker: ticker.upcase)
security ||= Security.create!(
ticker: ticker.upcase,
name: ticker.upcase
)
@id_mappings[:securities][cache_key] = security
security
end
end

2
app/models/import.rb
View File

@@ -9,7 +9,7 @@ class Import < ApplicationRecord
DOCUMENT_TYPES = %w[bank_statement credit_card_statement investment_statement financial_document contract other].freeze
TYPES = %w[TransactionImport TradeImport AccountImport MintImport CategoryImport RuleImport PdfImport QifImport].freeze
TYPES = %w[TransactionImport TradeImport AccountImport MintImport CategoryImport RuleImport PdfImport QifImport SureImport].freeze
SIGNAGE_CONVENTIONS = %w[inflows_positive inflows_negative]
SEPARATORS = [ [ "Comma (,)", "," ], [ "Semicolon (;)", ";" ] ].freeze

132
app/models/sure_import.rb Normal file
View File

@@ -0,0 +1,132 @@
class SureImport < Import
MAX_NDJSON_SIZE = 10.megabytes
ALLOWED_NDJSON_CONTENT_TYPES = %w[
application/x-ndjson
application/ndjson
application/json
application/octet-stream
text/plain
].freeze
has_one_attached :ndjson_file, dependent: :purge_later
class << self
# Counts JSON lines by top-level "type" (used for dry-run summaries and row limits).
def ndjson_line_type_counts(content)
return {} unless content.present?
counts = Hash.new(0)
content.each_line do |line|
next if line.strip.empty?
begin
record = JSON.parse(line)
counts[record["type"]] += 1 if record["type"]
rescue JSON::ParserError
# Skip invalid lines
end
end
counts
end
def dry_run_totals_from_ndjson(content)
counts = ndjson_line_type_counts(content)
{
accounts: counts["Account"] || 0,
categories: counts["Category"] || 0,
tags: counts["Tag"] || 0,
merchants: counts["Merchant"] || 0,
transactions: counts["Transaction"] || 0,
trades: counts["Trade"] || 0,
valuations: counts["Valuation"] || 0,
budgets: counts["Budget"] || 0,
budget_categories: counts["BudgetCategory"] || 0,
rules: counts["Rule"] || 0
}
end
def valid_ndjson_first_line?(str)
return false if str.blank?
first_line = str.lines.first&.strip
return false if first_line.blank?
begin
record = JSON.parse(first_line)
record.key?("type") && record.key?("data")
rescue JSON::ParserError
false
end
end
end
def requires_csv_workflow?
false
end
def column_keys
[]
end
def required_column_keys
[]
end
def mapping_steps
[]
end
def csv_template
nil
end
def dry_run
return {} unless uploaded?
self.class.dry_run_totals_from_ndjson(ndjson_blob_string)
end
def import!
importer = Family::DataImporter.new(family, ndjson_blob_string)
result = importer.import!
result[:accounts].each { |account| accounts << account }
result[:entries].each { |entry| entries << entry }
end
def uploaded?
return false unless ndjson_file.attached?
self.class.valid_ndjson_first_line?(ndjson_blob_string)
end
def configured?
uploaded?
end
def cleaned?
configured?
end
def publishable?
cleaned? && dry_run.values.sum.positive?
end
def max_row_count
100_000
end
# Row total for max-row enforcement (counts every parsed line with a "type", including unsupported types).
def sync_ndjson_rows_count!
return unless ndjson_file.attached?
total = self.class.ndjson_line_type_counts(ndjson_blob_string).values.sum
update_column(:rows_count, total)
end
private
def ndjson_blob_string
ndjson_file.download.force_encoding(Encoding::UTF_8)
end
end

79
app/views/import/confirms/show.html.erb
View File

@@ -2,32 +2,65 @@
<%= render "imports/nav", import: @import %>
<% end %>
<%= content_for :previous_path, import_clean_path(@import) %>
<%= content_for :previous_path, @import.is_a?(SureImport) ? imports_path : import_clean_path(@import) %>
<% step_idx = (params[:step] || "1").to_i - 1 %>
<% step_mapping_class = @import.mapping_steps[step_idx] %>
<% if @import.is_a?(SureImport) %>
<div class="space-y-6 mx-auto max-w-md mb-6">
<div class="text-center space-y-2">
<h1 class="text-3xl text-primary font-medium"><%= t("import.confirms.sure_import.title") %></h1>
<p class="text-secondary text-sm"><%= t("import.confirms.sure_import.description") %></p>
</div>
<div class="space-y-12 mx-auto max-w-md mb-6">
<div class="flex justify-center items-center gap-2">
<% @import.mapping_steps.each_with_index do |step_mapping_class, idx| %>
<% is_active = step_idx == idx %>
<%= link_to url_for(step: idx + 1), class: "w-5 h-[3px] #{is_active ? 'bg-gray-900' : 'bg-gray-100'} rounded-xl hover:bg-gray-300 transition-colors duration-200" do %>
<span class="sr-only">Step <%= idx + 1 %></span>
<div class="bg-container shadow-border-xs rounded-xl p-6 space-y-4">
<h3 class="text-sm font-medium text-secondary uppercase"><%= t("import.confirms.sure_import.summary") %></h3>
<% dry_run = @import.dry_run %>
<% sure_summary_empty = dry_run.values.none?(&:positive?) %>
<% if sure_summary_empty %>
<p class="text-secondary text-sm text-center py-4"><%= t("import.confirms.sure_import.empty_summary") %></p>
<% else %>
<ul class="space-y-3">
<% dry_run.each do |key, count| %>
<% next if count.zero? %>
<li class="flex items-center justify-between">
<span class="text-primary"><%= key.to_s.humanize %></span>
<span class="text-secondary font-medium"><%= count %></span>
</li>
<% end %>
</ul>
<% end %>
<% end %>
</div>
<div class="flex flex-col gap-2">
<%= button_to t("import.confirms.sure_import.publish_button"), publish_import_path(@import), method: :post, class: "btn btn--primary w-full", disabled: !@import.publishable? %>
<%= link_to t("import.confirms.sure_import.cancel"), imports_path, class: "btn btn--ghost w-full text-center" %>
</div>
</div>
<% else %>
<% step_idx = (params[:step] || "1").to_i - 1 %>
<% step_mapping_class = @import.mapping_steps[step_idx] %>
<div class="space-y-12 mx-auto max-w-md mb-6">
<div class="flex justify-center items-center gap-2">
<% @import.mapping_steps.each_with_index do |step_mapping_class, idx| %>
<% is_active = step_idx == idx %>
<%= link_to url_for(step: idx + 1), class: "w-5 h-[3px] #{is_active ? 'bg-gray-900' : 'bg-gray-100'} rounded-xl hover:bg-gray-300 transition-colors duration-200" do %>
<span class="sr-only">Step <%= idx + 1 %></span>
<% end %>
<% end %>
</div>
<div class="text-center space-y-2">
<h1 class="text-3xl text-primary font-medium">
<%= t(".#{step_mapping_class.name.demodulize.underscore}_title", import_type: @import.type.underscore.humanize) %>
</h1>
<p class="text-secondary text-sm">
<%= t(".#{step_mapping_class.name.demodulize.underscore}_description", import_type: @import.type.underscore.humanize, product_name: product_name) %>
</p>
</div>
</div>
<div class="text-center space-y-2">
<h1 class="text-3xl text-primary font-medium">
<%= t(".#{step_mapping_class.name.demodulize.underscore}_title", import_type: @import.type.underscore.humanize) %>
</h1>
<p class="text-secondary text-sm">
<%= t(".#{step_mapping_class.name.demodulize.underscore}_description", import_type: @import.type.underscore.humanize, product_name: product_name) %>
</p>
<div class="max-w-3xl mx-auto flex flex-col items-center">
<%= render partial: "import/confirms/mappings", locals: { import: @import, mapping_class: step_mapping_class, step_idx: step_idx } %>
</div>
</div>
<div class="max-w-3xl mx-auto flex flex-col items-center">
<%= render partial: "import/confirms/mappings", locals: { import: @import, mapping_class: step_mapping_class, step_idx: step_idx } %>
</div>
<% end %>

44
app/views/import/uploads/show.html.erb
View File

@@ -4,7 +4,49 @@
<%= content_for :previous_path, imports_path %>
<% if @import.is_a?(QifImport) %>
<% if @import.is_a?(SureImport) %>
<div class="space-y-4" data-controller="drag-and-drop-import">
<!-- Overlay -->
<%= render "imports/drag_drop_overlay", title: t("import.uploads.sure_import.drop_title"), subtitle: t("import.uploads.sure_import.drop_subtitle") %>
<div class="space-y-4 mx-auto max-w-md">
<div class="text-center space-y-2">
<h1 class="text-3xl text-primary font-medium"><%= t("import.uploads.sure_import.title") %></h1>
<p class="text-secondary text-sm"><%= t("import.uploads.sure_import.description") %></p>
</div>
<%= styled_form_with model: @import, scope: :import, url: import_upload_path(@import), multipart: true, class: "space-y-2", data: { drag_and_drop_import_target: "form" } do |form| %>
<div class="flex flex-col items-center justify-center w-full h-64 border border-secondary border-dashed rounded-xl cursor-pointer" data-controller="file-upload" data-action="click->file-upload#triggerFileInput" data-file-upload-target="uploadArea">
<div class="flex flex-col items-center justify-center pt-5 pb-6">
<div data-file-upload-target="uploadText" class="flex flex-col items-center">
<%= icon("database", size: "lg", class: "mb-4 mx-auto") %>
<p class="mb-2 text-md text-gray text-center">
<span class="font-medium text-primary"><%= t("import.uploads.sure_import.browse") %></span> <%= t("import.uploads.sure_import.browse_hint") %>
</p>
</div>
<div class="flex flex-col gap-4 items-center hidden mb-2" data-file-upload-target="fileName">
<span class="text-primary">
<%= icon("file-text", size: "lg", color: "current") %>
</span>
<p class="text-md font-medium text-primary"></p>
</div>
<%= form.file_field :ndjson_file, class: "hidden", accept: ".ndjson,.json", "data-auto-submit-form-target": "auto", "data-file-upload-target": "input", "data-drag-and-drop-import-target": "input" %>
</div>
</div>
<%= form.submit t("import.uploads.sure_import.upload_button"), disabled: @import.complete? %>
<% end %>
</div>
<div class="flex justify-center">
<span class="text-secondary text-sm">
<%= t("import.uploads.sure_import.hint_html") %>
</span>
</div>
</div>
<% elsif @import.is_a?(QifImport) %>
<%# ── QIF upload – fixed format, account required ── %>
<div class="space-y-4 mx-auto max-w-md">
<div class="text-center space-y-2">

7
app/views/imports/_nav.html.erb
View File

@@ -1,6 +1,11 @@
<%# locals: (import:) %>
<% steps = if import.is_a?(PdfImport)
<% steps = if import.is_a?(SureImport)
[
{ name: t("imports.steps.upload", default: "Upload"), path: import_upload_path(import), is_complete: import.uploaded?, step_number: 1 },
{ name: t("imports.steps.confirm", default: "Confirm"), path: import_path(import), is_complete: import.complete?, step_number: 2 }
]
elsif import.is_a?(PdfImport)
# PDF imports have a simplified flow: Upload -> Confirm
# Upload/Configure/Clean are always complete for processed PDF imports
[

43
app/views/imports/_ready.html.erb
View File

@@ -1,4 +1,7 @@
<%# locals: (import:) %>
<% dry_run = import.dry_run %>
<% resources_with_counts = dry_run.select { |_, count| count > 0 }.filter_map { |key, count| [dry_run_resource(key), count] if dry_run_resource(key) } %>
<% import_summary_empty = resources_with_counts.empty? %>
<div class="text-center space-y-2 mb-4 mx-auto max-w-md">
<h1 class="text-3xl text-primary font-medium"><%= t(".title") %></h1>
@@ -8,32 +11,40 @@
<div class="mx-auto max-w-2xl space-y-4">
<div class="bg-container-inset rounded-xl p-1 space-y-1">
<div class="flex justify-between items-center text-xs font-medium text-secondary uppercase px-5 py-3">
<p>item</p>
<p class="justify-self-end">count</p>
<p><%= t(".summary_item_label") %></p>
<p class="justify-self-end"><%= t(".summary_count_label") %></p>
</div>
<div class="bg-container shadow-border-xs rounded-lg text-sm">
<% import.dry_run.each do |key, count| %>
<% resource = dry_run_resource(key) %>
<% if import_summary_empty %>
<div class="px-5 py-8 text-center space-y-2">
<p class="text-primary text-sm"><%= t(".empty_summary") %></p>
</div>
<% else %>
<% resources_with_counts.each_with_index do |(resource, count), index| %>
<div class="flex items-center justify-between gap-2 bg-container px-5 py-3 rounded-lg">
<div class="flex items-center gap-3">
<%= tag.div class: class_names(resource.bg_class, resource.text_class, "w-8 h-8 rounded-full flex justify-center items-center") do %>
<%= icon resource.icon, color: "current" %>
<% end %>
<div class="flex items-center justify-between gap-2 bg-container px-5 py-3 rounded-lg">
<div class="flex items-center gap-3">
<%= tag.div class: class_names(resource.bg_class, resource.text_class, "w-8 h-8 rounded-full flex justify-center items-center") do %>
<%= icon resource.icon, color: "current" %>
<% end %>
<p><%= resource.label %></p>
</div>
<p><%= resource.label %></p>
<p class="justify-self-end"><%= count %></p>
</div>
<p class="justify-self-end"><%= count %></p>
</div>
<% if key != import.dry_run.keys.last %>
<%= render "shared/ruler" %>
<% unless index == resources_with_counts.length - 1 %>
<%= render "shared/ruler" %>
<% end %>
<% end %>
<% end %>
</div>
</div>
<%= render DS::Button.new(text: "Publish import", href: publish_import_path(import), full_width: true) %>
<% if import_summary_empty %>
<%= render DS::Button.new(text: t(".back_to_imports"), href: imports_path, variant: :secondary, full_width: true) %>
<% else %>
<%= render DS::Button.new(text: t(".publish_import"), href: publish_import_path(import), method: :post, full_width: true) %>
<% end %>
</div>

304
app/views/imports/new.html.erb
View File

@@ -5,11 +5,10 @@
<% has_accounts = Current.family.accounts.any? %>
<% requires_account_message = t(".requires_account") %>
<div class="rounded-xl bg-container-inset p-1">
<h3 class="uppercase text-secondary text-xs font-medium px-3 py-1.5"><%= t(".sources") %></h3>
<ul class="bg-container shadow-border-xs rounded-lg">
<li>
<% if @pending_import.present? && (params[:type].nil? || params[:type] == @pending_import.type) %>
<% if @pending_import.present? && params[:type].nil? %>
<div class="rounded-xl bg-container-inset p-1 mb-4">
<ul class="bg-container shadow-border-xs rounded-lg">
<li>
<%= link_to import_path(@pending_import), class: "flex items-center justify-between p-4 group cursor-pointer", data: { turbo: false } do %>
<div class="flex items-center gap-2">
<div class="bg-orange-500/5 rounded-md w-8 h-8 flex items-center justify-center">
@@ -26,109 +25,208 @@
<%= render "shared/ruler" %>
</li>
<% end %>
</ul>
</div>
<% end %>
<% if params[:type].nil? || params[:type] == "TransactionImport" %>
<%= render "imports/import_option",
type: "TransactionImport",
icon_name: "file-spreadsheet",
icon_bg_class: "bg-indigo-500/5",
icon_text_class: "text-indigo-500",
label: t(".import_transactions"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<%
import_type = params[:type].presence || @pending_import&.type
active_tab = import_type.present? && !import_type.in?(%w[MintImport QifImport SureImport DocumentImport PdfImport]) ? "raw_data" : "financial_tools"
%>
<%= render DS::Tabs.new(active_tab: active_tab) do |tabs| %>
<% tabs.with_nav do |nav| %>
<% nav.with_btn(id: "financial_tools", label: t(".tab_financial_tools")) %>
<% nav.with_btn(id: "raw_data", label: t(".tab_raw_data")) %>
<% end %>
<% if params[:type].nil? || params[:type] == "TradeImport" %>
<%= render "imports/import_option",
type: "TradeImport",
icon_name: "square-percent",
icon_bg_class: "bg-yellow-500/5",
icon_text_class: "text-yellow-500",
label: t(".import_portfolio"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<% if params[:type].nil? || params[:type] == "AccountImport" %>
<%= render "imports/import_option",
type: "AccountImport",
icon_name: "building",
icon_bg_class: "bg-violet-500/5",
icon_text_class: "text-violet-500",
label: t(".import_accounts"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "CategoryImport" %>
<%= render "imports/import_option",
type: "CategoryImport",
icon_name: "shapes",
icon_bg_class: "bg-blue-500/5",
icon_text_class: "text-blue-500",
label: t(".import_categories"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "RuleImport" %>
<%= render "imports/import_option",
type: "RuleImport",
icon_name: "workflow",
icon_bg_class: "bg-green-500/5",
icon_text_class: "text-green-500",
label: t(".import_rules"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "MintImport" || params[:type] == "TransactionImport" %>
<%= render "imports/import_option",
type: "MintImport",
image: "mint-logo.jpeg",
label: t(".import_mint"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<% if params[:type].nil? || params[:type] == "QifImport" %>
<%= render "imports/import_option",
type: "QifImport",
icon_name: "file-clock",
icon_bg_class: "bg-teal-500/5",
icon_text_class: "text-teal-500",
label: t(".import_qif"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<% if (params[:type].nil? || params[:type].in?(%w[DocumentImport PdfImport])) && @document_upload_extensions.any? %>
<li>
<%= styled_form_with url: imports_path, scope: :import, multipart: true, class: "w-full" do |form| %>
<%= form.hidden_field :type, value: "DocumentImport" %>
<label class="flex items-center justify-between p-4 group cursor-pointer w-full">
<div class="flex items-center gap-2">
<div class="bg-red-500/5 rounded-md w-8 h-8 flex items-center justify-center">
<span class="text-red-500">
<%= icon("file-text", color: "current") %>
<% tabs.with_panel(tab_id: "financial_tools") do %>
<div class="rounded-xl bg-container-inset p-1">
<ul class="bg-container shadow-border-xs rounded-lg">
<% if @pending_import.present? && params[:type].present? && params[:type].in?(%w[MintImport QifImport SureImport DocumentImport PdfImport]) %>
<li>
<%= link_to import_path(@pending_import), class: "flex items-center justify-between p-4 group cursor-pointer", data: { turbo: false } do %>
<div class="flex items-center gap-2">
<div class="bg-orange-500/5 rounded-md w-8 h-8 flex items-center justify-center">
<span class="text-orange-500">
<%= icon("loader", color: "current") %>
</span>
</div>
<span class="text-sm text-primary group-hover:text-secondary">
<%= t(".resume", type: @pending_import.type.titleize) %>
</span>
</div>
<div class="text-left">
<span class="text-sm text-primary group-hover:text-secondary block">
<%= t(".import_file") %>
</span>
<span class="text-xs text-secondary">
<%= t(".import_file_description") %>
</span>
</div>
</div>
<%= icon("chevron-right") %>
<%= form.file_field :import_file, accept: @document_upload_extensions.join(","), class: "hidden", onchange: "this.form.submit()" %>
</label>
<%= icon("chevron-right") %>
<% end %>
<%= render "shared/ruler" %>
</li>
<% end %>
<%= render "shared/ruler" %>
</li>
<% end %>
</ul>
</div>
<% if params[:type].nil? || params[:type] == "SureImport" %>
<li>
<%= styled_form_with url: imports_path, scope: :import, multipart: true, class: "w-full", data: { turbo: false } do |form| %>
<%= form.hidden_field :type, value: "SureImport" %>
<label class="flex items-center justify-between p-4 group cursor-pointer w-full">
<div class="flex items-center gap-2">
<div class="bg-cyan-500/5 rounded-md w-8 h-8 flex items-center justify-center">
<span class="text-cyan-500">
<%= icon("file-braces", color: "current") %>
</span>
</div>
<div class="text-left">
<span class="text-sm text-primary group-hover:text-secondary block">
<%= t(".import_sure") %>
</span>
<span class="text-xs text-secondary">
<%= t(".import_sure_description") %>
</span>
</div>
</div>
<%= icon("chevron-right") %>
<%= form.file_field :import_file, accept: ".ndjson,.json", class: "hidden", onchange: "this.form.submit()" %>
</label>
<% end %>
<%= render "shared/ruler" %>
</li>
<% end %>
<% if params[:type].nil? || params[:type] == "MintImport" || params[:type] == "TransactionImport" %>
<%= render "imports/import_option",
type: "MintImport",
image: "mint-logo.jpeg",
label: t(".import_mint"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "QifImport" %>
<%= render "imports/import_option",
type: "QifImport",
icon_name: "file-clock",
icon_bg_class: "bg-teal-500/5",
icon_text_class: "text-teal-500",
label: t(".import_qif"),
enabled: true %>
<% end %>
<%= render "imports/import_option",
type: "TransactionImport",
icon_name: "bar-chart-2",
icon_bg_class: "bg-gray-500/5",
icon_text_class: "text-gray-400",
label: t(".import_ynab"),
enabled: false,
disabled_message: t(".coming_soon") %>
<% if (params[:type].nil? || params[:type].in?(%w[DocumentImport PdfImport])) && @document_upload_extensions.any? %>
<li>
<%= styled_form_with url: imports_path, scope: :import, multipart: true, class: "w-full", data: { turbo: false } do |form| %>
<%= form.hidden_field :type, value: "DocumentImport" %>
<label class="flex items-center justify-between p-4 group cursor-pointer w-full">
<div class="flex items-center gap-2">
<div class="bg-red-500/5 rounded-md w-8 h-8 flex items-center justify-center">
<span class="text-red-500">
<%= icon("file-text", color: "current") %>
</span>
</div>
<div class="text-left">
<span class="text-sm text-primary group-hover:text-secondary block">
<%= t(".import_file") %>
</span>
<span class="text-xs text-secondary">
<%= t(".import_file_description") %>
</span>
</div>
</div>
<%= icon("chevron-right") %>
<%= form.file_field :import_file, accept: @document_upload_extensions.join(","), class: "hidden", onchange: "this.form.submit()" %>
</label>
<% end %>
<%= render "shared/ruler" %>
</li>
<% end %>
</ul>
</div>
<% end %>
<% tabs.with_panel(tab_id: "raw_data") do %>
<div class="rounded-xl bg-container-inset p-1">
<ul class="bg-container shadow-border-xs rounded-lg">
<% if @pending_import.present? && params[:type].present? && !params[:type].in?(%w[MintImport QifImport SureImport DocumentImport PdfImport]) %>
<li>
<%= link_to import_path(@pending_import), class: "flex items-center justify-between p-4 group cursor-pointer", data: { turbo: false } do %>
<div class="flex items-center gap-2">
<div class="bg-orange-500/5 rounded-md w-8 h-8 flex items-center justify-center">
<span class="text-orange-500">
<%= icon("loader", color: "current") %>
</span>
</div>
<span class="text-sm text-primary group-hover:text-secondary">
<%= t(".resume", type: @pending_import.type.titleize) %>
</span>
</div>
<%= icon("chevron-right") %>
<% end %>
<%= render "shared/ruler" %>
</li>
<% end %>
<% if params[:type].nil? || params[:type] == "TransactionImport" %>
<%= render "imports/import_option",
type: "TransactionImport",
icon_name: "file-spreadsheet",
icon_bg_class: "bg-indigo-500/5",
icon_text_class: "text-indigo-500",
label: t(".import_transactions"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<% if params[:type].nil? || params[:type] == "TradeImport" %>
<%= render "imports/import_option",
type: "TradeImport",
icon_name: "square-percent",
icon_bg_class: "bg-yellow-500/5",
icon_text_class: "text-yellow-500",
label: t(".import_portfolio"),
enabled: has_accounts,
disabled_message: requires_account_message %>
<% end %>
<% if params[:type].nil? || params[:type] == "AccountImport" %>
<%= render "imports/import_option",
type: "AccountImport",
icon_name: "building",
icon_bg_class: "bg-violet-500/5",
icon_text_class: "text-violet-500",
label: t(".import_accounts"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "CategoryImport" %>
<%= render "imports/import_option",
type: "CategoryImport",
icon_name: "shapes",
icon_bg_class: "bg-blue-500/5",
icon_text_class: "text-blue-500",
label: t(".import_categories"),
enabled: true %>
<% end %>
<% if params[:type].nil? || params[:type] == "RuleImport" %>
<%= render "imports/import_option",
type: "RuleImport",
icon_name: "workflow",
icon_bg_class: "bg-green-500/5",
icon_text_class: "text-green-500",
label: t(".import_rules"),
enabled: true %>
<% end %>
</ul>
</div>
<% end %>
<% end %>
<% end %>
<% end %>