mirror of
https://github.com/we-promise/sure.git
synced 2026-04-07 14:31:25 +00:00
* Feat: Add QIF (Quicken Interchange Format) import functionality - Add the ability to import QIF files for users coming from Quicken - Includes categories and tags - Comprehensive tests for QifImport, including parsing, row generation, and import functionality. - Ensure handling of hierarchical categories (ex "Home:Home Improvement" is imported as Parent:Child) * Fix QIF import issues raised in code review - Fix two-digit year windowing in QIF date parser (e.g. '99 → 1999, not 2099) - Fix ArgumentError from invalid `undef: :raise` encoding option - Nil-safe `leaf_category_name` with blank guard and `.to_s` coercion - Memoize `qif_account_type` to avoid re-parsing the full QIF file - Add strong parameters (`selection_params`) to QifCategorySelectionsController - Wrap all mutations in DB transactions in uploads and category-selections controllers - Skip unchanged tag rows (only write rows where tags actually differ) - Replace hardcoded strings with i18n keys across QIF views and nav - Fix potentially colliding checkbox/label IDs in category selection view - Improve keyboard accessibility: use semantic `<label>` for file picker area Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix QIF import test count and Brakeman mass assignment warning - Update ImportsControllerTest to expect 4 disabled import options (was 3), accounting for the new QIF import type added in this branch - Remove :account_id from upload_params permit list; it was never accessed through strong params (always via params.dig with Current.family scope), so this resolves the Brakeman high-confidence mass assignment warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix: QIF import security, safety, and i18n issues raised in code review - Added french, spanish and german translations for newly added i18n keys - Replace params.dig(:import, :account_id) with a proper strong-params accessor (import_account_id) in UploadsController to satisfy Rails parameter filtering requirements - Guard ImportsController#show against QIF imports reaching the publish screen before a file has been uploaded, preventing an unrescued error on publish - Gate the QIF "Clean" nav step link on import.uploaded? to prevent routing to CleansController with an unconfigured import (which would raise "Unknown import type: QifImport" via ImportsHelper) - Replace hard-coded "txn" pluralize calls in the category/tag selection view with t(".txn_count") and add pluralization keys to the locale file - Localize all hard-coded strings in the QIF upload section of uploads/show.html.erb and add corresponding en.yml keys - Convert the CSV upload drop zone from a clickable <div> (JS-only) to a semantic <label> element, making it keyboard-accessible without JavaScript * Fix: missing translations keys * Add icon mapping and random color assignment to new categories * fix a lint issue * Add a warning about splits and some plumbing for future support. Updated locales. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
209 lines
6.4 KiB
Ruby
209 lines
6.4 KiB
Ruby
class ImportsController < ApplicationController
|
|
include SettingsHelper
|
|
|
|
before_action :set_import, only: %i[show update publish destroy revert apply_template]
|
|
|
|
def update
|
|
# Handle both pdf_import[account_id] and import[account_id] param formats
|
|
account_id = params.dig(:pdf_import, :account_id) || params.dig(:import, :account_id)
|
|
|
|
if account_id.present?
|
|
account = Current.family.accounts.find_by(id: account_id)
|
|
unless account
|
|
redirect_back_or_to import_path(@import), alert: t("imports.update.invalid_account", default: "Account not found.")
|
|
return
|
|
end
|
|
@import.update!(account: account)
|
|
end
|
|
|
|
redirect_to import_path(@import), notice: t("imports.update.account_saved", default: "Account saved.")
|
|
end
|
|
|
|
def publish
|
|
@import.publish_later
|
|
|
|
redirect_to import_path(@import), notice: "Your import has started in the background."
|
|
rescue Import::MaxRowCountExceededError
|
|
redirect_back_or_to import_path(@import), alert: "Your import exceeds the maximum row count of #{@import.max_row_count}."
|
|
end
|
|
|
|
def index
|
|
@pagy, @imports = pagy(Current.family.imports.where(type: Import::TYPES).ordered, limit: safe_per_page)
|
|
@breadcrumbs = [
|
|
[ t("breadcrumbs.home"), root_path ],
|
|
[ t("breadcrumbs.imports"), imports_path ]
|
|
]
|
|
render layout: "settings"
|
|
end
|
|
|
|
def new
|
|
@pending_import = Current.family.imports.ordered.pending.first
|
|
@document_upload_extensions = document_upload_supported_extensions
|
|
end
|
|
|
|
def create
|
|
file = import_params[:import_file]
|
|
|
|
if file.present? && document_upload_request?
|
|
create_document_import(file)
|
|
return
|
|
end
|
|
|
|
# Handle PDF file uploads - process with AI
|
|
if file.present? && Import::ALLOWED_PDF_MIME_TYPES.include?(file.content_type)
|
|
unless valid_pdf_file?(file)
|
|
redirect_to new_import_path, alert: t("imports.create.invalid_pdf")
|
|
return
|
|
end
|
|
create_pdf_import(file)
|
|
return
|
|
end
|
|
|
|
type = params.dig(:import, :type).to_s
|
|
type = "TransactionImport" unless Import::TYPES.include?(type)
|
|
|
|
account = Current.family.accounts.find_by(id: params.dig(:import, :account_id))
|
|
import = Current.family.imports.create!(
|
|
type: type,
|
|
account: account,
|
|
date_format: Current.family.date_format,
|
|
)
|
|
|
|
if file.present?
|
|
if file.size > Import::MAX_CSV_SIZE
|
|
import.destroy
|
|
redirect_to new_import_path, alert: t("imports.create.file_too_large", max_size: Import::MAX_CSV_SIZE / 1.megabyte)
|
|
return
|
|
end
|
|
|
|
unless Import::ALLOWED_CSV_MIME_TYPES.include?(file.content_type)
|
|
import.destroy
|
|
redirect_to new_import_path, alert: t("imports.create.invalid_file_type")
|
|
return
|
|
end
|
|
|
|
# Stream reading is not fully applicable here as we store the raw string in the DB,
|
|
# but we have validated size beforehand to prevent memory exhaustion from massive files.
|
|
import.update!(raw_file_str: file.read)
|
|
redirect_to import_configuration_path(import), notice: t("imports.create.csv_uploaded")
|
|
else
|
|
redirect_to import_upload_path(import)
|
|
end
|
|
end
|
|
|
|
def show
|
|
unless @import.requires_csv_workflow?
|
|
redirect_to import_upload_path(@import), alert: t("imports.show.finalize_upload") unless @import.uploaded?
|
|
return
|
|
end
|
|
|
|
if !@import.uploaded?
|
|
redirect_to import_upload_path(@import), alert: t("imports.show.finalize_upload")
|
|
elsif !@import.publishable?
|
|
redirect_to import_confirm_path(@import), alert: t("imports.show.finalize_mappings")
|
|
end
|
|
end
|
|
|
|
def revert
|
|
@import.revert_later
|
|
redirect_to imports_path, notice: "Import is reverting in the background."
|
|
end
|
|
|
|
def apply_template
|
|
if @import.suggested_template
|
|
@import.apply_template!(@import.suggested_template)
|
|
redirect_to import_configuration_path(@import), notice: "Template applied."
|
|
else
|
|
redirect_to import_configuration_path(@import), alert: "No template found, please manually configure your import."
|
|
end
|
|
end
|
|
|
|
def destroy
|
|
@import.destroy
|
|
|
|
redirect_to imports_path, notice: "Your import has been deleted."
|
|
end
|
|
|
|
private
|
|
def set_import
|
|
@import = Current.family.imports.includes(:account).find(params[:id])
|
|
end
|
|
|
|
def import_params
|
|
params.require(:import).permit(:import_file)
|
|
end
|
|
|
|
def create_pdf_import(file)
|
|
if file.size > Import::MAX_PDF_SIZE
|
|
redirect_to new_import_path, alert: t("imports.create.pdf_too_large", max_size: Import::MAX_PDF_SIZE / 1.megabyte)
|
|
return
|
|
end
|
|
|
|
pdf_import = Current.family.imports.create!(type: "PdfImport")
|
|
pdf_import.pdf_file.attach(file)
|
|
pdf_import.process_with_ai_later
|
|
|
|
redirect_to import_path(pdf_import), notice: t("imports.create.pdf_processing")
|
|
end
|
|
|
|
def create_document_import(file)
|
|
adapter = VectorStore.adapter
|
|
unless adapter
|
|
redirect_to new_import_path, alert: t("imports.create.document_provider_not_configured")
|
|
return
|
|
end
|
|
|
|
if file.size > Import::MAX_PDF_SIZE
|
|
redirect_to new_import_path, alert: t("imports.create.document_too_large", max_size: Import::MAX_PDF_SIZE / 1.megabyte)
|
|
return
|
|
end
|
|
|
|
filename = file.original_filename.to_s
|
|
ext = File.extname(filename).downcase
|
|
supported_extensions = adapter.supported_extensions.map(&:downcase)
|
|
|
|
unless supported_extensions.include?(ext)
|
|
redirect_to new_import_path, alert: t("imports.create.invalid_document_file_type")
|
|
return
|
|
end
|
|
|
|
if ext == ".pdf"
|
|
unless valid_pdf_file?(file)
|
|
redirect_to new_import_path, alert: t("imports.create.invalid_pdf")
|
|
return
|
|
end
|
|
|
|
create_pdf_import(file)
|
|
return
|
|
end
|
|
|
|
family_document = Current.family.upload_document(
|
|
file_content: file.read,
|
|
filename: filename
|
|
)
|
|
|
|
if family_document
|
|
redirect_to new_import_path, notice: t("imports.create.document_uploaded")
|
|
else
|
|
redirect_to new_import_path, alert: t("imports.create.document_upload_failed")
|
|
end
|
|
end
|
|
|
|
def document_upload_supported_extensions
|
|
adapter = VectorStore.adapter
|
|
return [] unless adapter
|
|
|
|
adapter.supported_extensions.map(&:downcase).uniq.sort
|
|
end
|
|
|
|
def document_upload_request?
|
|
params.dig(:import, :type) == "DocumentImport"
|
|
end
|
|
|
|
def valid_pdf_file?(file)
|
|
header = file.read(5)
|
|
file.rewind
|
|
header&.start_with?("%PDF-")
|
|
end
|
|
end
|