Files
sure/app/models/rule_import.rb
T
457698f75b feat(rules): support multiple tags in the set transaction tags action (#3397)
* feat(rules): support multiple tags in the set transaction tags action

Fixes #3353. Reuses the existing DS::TagSelect multi-select tag picker
(made generic via attribute:/show_label:) instead of a native
<select multiple>, so the UX matches the rest of the app. Multiple
tag ids are stored as a comma-separated string in the existing
value column, keeping single-tag rows backward compatible with no
migration. Also closes a read-modify-write race in
SetTransactionTags#execute via with_lock.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(rules): address automated review findings on multi-tag actions

- Fix data export/import: multi-tag actions were exported/imported as
  one opaque comma string, losing all but a bogus combined tag on
  restore. Each tag id is now resolved/reconstructed independently,
  with a backward-compatible scalar value_ref for single-tag actions.
- Fix N+1 in Rule::Action#value_display (options queried once per tag).
- Add aria-label to DS::TagSelect's trigger button when show_label is
  false, so the control keeps an accessible name.
- Localize the "to" label in rule action rows (rules.actions.to_label).
- Use a monotonic counter instead of Date.now() for nested form
  indices, closing a same-millisecond collision window.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(rules): batch tag lookups in multi-tag import resolution

Avoids one find_by query per tag name when reconstructing multi-tag
rule actions during import.

* fix(rules): resolve jjmata review findings on multi-tag action

- rules_controller.js: prefix the JS-side nested-form index counter
  with "new_" so it can never collide with the numeric indexes Rails
  assigns to already-persisted conditions/actions on an edit form.
- data_exporter.rb: key the value_ref scalar/array decision off the
  number of tag ids on the action, not the number that still resolve,
  so a partially-orphaned multi-tag action keeps round-tripping as an
  array.
- rule_import.rb: split comma-separated set_transaction_tags values
  into individual tag names during CSV rule import, matching the
  batched resolution already used by Family::DataImporter.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(rules): CSV-quote multi-tag names so commas don't split them

CodeRabbit flagged that a tag name containing a comma (e.g. "Food,
Dining") would be silently split into two tags when round-tripped
through the comma-separated multi-tag value/CSV formats used by
Family::DataExporter, Family::DataImporter, and RuleImport.

Add Rule::Action.encode_multi_value_names/.decode_multi_value_names,
backed by Ruby's CSV line quoting, and use them at all three call
sites instead of a plain join(",")/split(","). A single name without
a comma round-trips byte-identical to before, so existing exports and
CSV rule templates are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: GFR <248542187+gfr-free@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 08:13:59 +02:00

392 lines
12 KiB
Ruby

class RuleImport < Import
def import!
transaction do
rows.each do |row|
create_or_update_rule_from_row(row)
end
end
end
# Rules hang off the family, not the import — see
# Import#committed_by_named_records?. Nameless rows build unconditionally and
# carry no stable key, so a file of only nameless rows has no commit signal.
def data_committed?
committed_by_named_records?(family.rules)
end
def column_keys
%i[name resource_type active effective_date conditions actions]
end
def required_column_keys
%i[resource_type conditions actions]
end
def mapping_steps
[]
end
def dry_run
{ rules: rows_count }
end
def csv_template
csv_string = CSV.generate do |csv|
csv << %w[name resource_type* active effective_date conditions* actions*]
csv << [
"Categorize groceries",
"transaction",
"true",
"2024-01-01",
'[{"condition_type":"transaction_name","operator":"like","value":"grocery"}]',
'[{"action_type":"set_transaction_category","value":"Groceries"}]'
]
csv << [
"Auto-categorize transactions",
"transaction",
"true",
"",
'[{"condition_type":"transaction_name","operator":"like","value":"amazon"}]',
'[{"action_type":"auto_categorize"}]'
]
end
CSV.parse(csv_string, headers: true)
end
def generate_rows_from_csv
rows.destroy_all
csv_rows.each.with_index(1) do |row, index|
normalized_row = normalize_rule_row(row)
rows.create!(
source_row_number: index,
name: normalized_row[:name].to_s.strip,
resource_type: normalized_row[:resource_type].to_s.strip,
active: parse_boolean(normalized_row[:active]),
effective_date: normalized_row[:effective_date].to_s.strip,
conditions: normalized_row[:conditions].to_s.strip,
actions: normalized_row[:actions].to_s.strip,
currency: default_currency
)
end
end
def parsed_csv
@parsed_csv ||= Import.parse_csv_str(raw_file_str, col_sep: col_sep)
end
private
def normalize_rule_row(row)
fields = row.fields
name, resource_type, active, effective_date = fields[0..3]
conditions, actions = extract_conditions_and_actions(fields[4..])
{
name: row["name"].presence || name,
resource_type: row["resource_type"].presence || resource_type,
active: row["active"].presence || active,
effective_date: row["effective_date"].presence || effective_date,
conditions: conditions,
actions: actions
}
end
def extract_conditions_and_actions(fragments)
pieces = Array(fragments).compact
return [ "", "" ] if pieces.empty?
combined = pieces.join(col_sep)
# If the CSV was split incorrectly because of unescaped quotes in the JSON
# payload, re-assemble the last two logical columns by splitting on the
# boundary between the two JSON arrays: ...]","[...
parts = combined.split(/(?<=\])"\s*,\s*"(?=\[)/, 2)
parts = [ pieces[0], pieces[1] ] if parts.length < 2
parts.map do |part|
next "" unless part
# Remove any stray leading/trailing quotes left from CSV parsing
part.to_s.strip.gsub(/\A"+|"+\z/, "")
end
end
def create_or_update_rule_from_row(row)
rule_name = row.name.to_s.strip.presence
resource_type = row.resource_type.to_s.strip
# Validate resource type
unless resource_type == "transaction"
errors.add(:base, :unsupported_resource_type, resource_type: resource_type)
raise ActiveRecord::RecordInvalid.new(self)
end
# Parse conditions and actions from JSON
begin
conditions_data = parse_json_safely(row.conditions, "conditions")
actions_data = parse_json_safely(row.actions, "actions")
rescue JSON::ParserError => e
errors.add(:base, :invalid_json, message: e.message)
raise ActiveRecord::RecordInvalid.new(self)
end
# Validate we have at least one action
if actions_data.empty?
errors.add(:base, :min_actions)
raise ActiveRecord::RecordInvalid.new(self)
end
# Find or create rule
rule = if rule_name.present?
family.rules.find_or_initialize_by(name: rule_name, resource_type: resource_type)
else
family.rules.build(resource_type: resource_type)
end
rule.active = row.active || false
rule.effective_date = parse_date(row.effective_date)
# Clear existing conditions and actions
rule.conditions.destroy_all
rule.actions.destroy_all
# Create conditions
conditions_data.each do |condition_data|
build_condition(rule, condition_data)
end
# Create actions
actions_data.each do |action_data|
build_action(rule, action_data)
end
rule.save!
end
def build_condition(rule, condition_data, parent: nil)
condition_type = condition_data["condition_type"]
operator = condition_data["operator"]
value = resolve_import_condition_value(condition_data)
condition = if parent
parent.sub_conditions.build(
condition_type: condition_type,
operator: operator,
value: value
)
else
rule.conditions.build(
condition_type: condition_type,
operator: operator,
value: value
)
end
# Handle compound conditions with sub_conditions
if condition_data["sub_conditions"].present?
condition_data["sub_conditions"].each do |sub_condition_data|
build_condition(rule, sub_condition_data, parent: condition)
end
end
condition
end
def build_action(rule, action_data)
action_type = action_data["action_type"]
value = resolve_import_action_value(action_data)
rule.actions.build(
action_type: action_type,
value: value
)
end
def resolve_import_condition_value(condition_data)
condition_type = condition_data["condition_type"]
value = condition_data["value"]
return value unless value.present?
# Map category names to UUIDs
if condition_type == "transaction_category"
category = family.categories.find_by(name: value)
unless category
category = family.categories.create!(
name: value,
color: Category::UNCATEGORIZED_COLOR,
lucide_icon: "shapes"
)
end
return category.id
end
# Map merchant names to UUIDs
if condition_type == "transaction_merchant"
merchant = family.merchants.find_by(name: value)
unless merchant
merchant = family.merchants.create!(name: value)
end
return merchant.id
end
value
end
def resolve_import_action_value(action_data)
action_type = action_data["action_type"]
value = action_data["value"]
return value unless value.present?
# Map category names to UUIDs
if action_type == "set_transaction_category"
category = family.categories.find_by(name: value)
# Create category if it doesn't exist
unless category
category = family.categories.create!(
name: value,
color: Category::UNCATEGORIZED_COLOR,
lucide_icon: "shapes"
)
end
return category.id
end
# Map merchant names to UUIDs
if action_type == "set_transaction_merchant"
merchant = family.merchants.find_by(name: value)
# Create merchant if it doesn't exist
unless merchant
merchant = family.merchants.create!(name: value)
end
return merchant.id
end
# Map tag names to UUIDs. `value` may be a comma-separated list of tag
# names for multi-tag actions (see Rule::Action#value=), so each name
# is resolved independently rather than treating the whole string as
# a single tag name.
if action_type == "set_transaction_tags"
return resolve_import_multi_tag_value(value)
end
value
end
def resolve_import_multi_tag_value(value)
names = Rule::Action.decode_multi_value_names(value).map(&:strip).reject(&:blank?)
return value if names.empty?
tags_by_name = family.tags.where(name: names).index_by(&:name)
tag_ids = names.map do |name|
tag = tags_by_name[name] ||= family.tags.create!(name: name)
tag.id
end
tag_ids.join(",")
end
def parse_boolean(value)
return true if value.to_s.downcase.in?(%w[true 1 yes y])
return false if value.to_s.downcase.in?(%w[false 0 no n])
false
end
def parse_date(value)
return nil if value.blank?
Date.parse(value.to_s)
rescue ArgumentError
nil
end
def parse_json_safely(json_string, field_name)
return [] if json_string.blank?
cleaned = json_string.to_s.strip
# Most API-created rows already store valid JSON. Parse them as-is before
# falling back to the legacy cleanup path for older malformed payloads.
parse_json_payload(cleaned, normalize_legacy_strings: false)
rescue JSON::ParserError
# Clean up the JSON string - remove extra escaping that might come from CSV parsing
# Remove surrounding quotes if present (both single and double)
cleaned = cleaned.gsub(/\A["']+|["']+\z/, "")
# Handle multiple levels of escaping iteratively
# Keep unescaping until no more changes occur
loop do
previous = cleaned.dup
# Unescape quotes - handle patterns like \" or \\\" or \\\\\" etc.
# Replace any number of backslashes followed by a quote with just a quote
cleaned = cleaned.gsub(/\\+"/, '"')
cleaned = cleaned.gsub(/\\+'/, "'")
# Unescape backslashes (\\\\ becomes \)
cleaned = cleaned.gsub(/\\\\/, "\\")
break if cleaned == previous
end
# Handle unicode escapes like \u003e (but only if not over-escaped)
# Try to find and decode unicode escapes
cleaned = cleaned.gsub(/\\u([0-9a-fA-F]{4})/i) do |match|
code_point = $1.to_i(16)
[ code_point ].pack("U")
rescue
match # If decoding fails, keep the original
end
# Try parsing
parse_json_payload(cleaned, normalize_legacy_strings: true)
rescue JSON::ParserError => e
raise JSON::ParserError.new("Invalid JSON in #{field_name}: #{e.message}. Raw value: #{json_string.inspect}")
end
def parse_json_payload(payload, normalize_legacy_strings:)
parsed = JSON.parse(payload)
parsed = JSON.parse(parsed) if wrapped_json_payload?(parsed)
normalize_json_values(parsed, normalize_legacy_strings:)
end
def wrapped_json_payload?(value)
return false unless value.is_a?(String)
stripped_value = value.strip
stripped_value.start_with?("[", "{")
end
def normalize_json_values(value, normalize_legacy_strings:)
case value
when Array
value.map { |item| normalize_json_values(item, normalize_legacy_strings:) }
when Hash
value.transform_values { |item| normalize_json_values(item, normalize_legacy_strings:) }
when String
normalized = value
.gsub(/\\u([0-9a-fA-F]{4})/i) { [ $1.to_i(16) ].pack("U") }
.gsub('\\"', '"')
if normalize_legacy_strings
normalized = normalized
.gsub("\\n", "\n")
.gsub("\\r", "\r")
.gsub("\\t", "\t")
end
normalized
else
value
end
end
end