mirror of
https://github.com/we-promise/sure.git
synced 2026-04-08 14:54:49 +00:00
* Add Recent Runs visibility for rule executions Adds a comprehensive tracking system for rule execution history with the following features: - Creates RuleRun model to track execution metadata: * Date/time of execution * Execution type (manual/scheduled) * Success/failure status * Rule reference * Transaction counts (processed and modified) * Error messages for failed runs - Updates RuleJob to automatically record execution results: * Captures transaction processing statistics * Handles success/failure states * Stores error details for debugging - Adds "Recent Runs" section to rules index page: * Paginated display (20 runs per page) * Columnar layout similar to LLM usage page * Visual status indicators (success/failed badges) * Error tooltips for failed runs * Responsive design with design system tokens - Includes i18n translations for all user-facing strings This provides users with visibility into rule execution history, making it easier to debug issues and monitor rule performance. * Update schema.rb with rule_runs table definition * Linter noise * Separate transaction counts into Queued, Processed, and Modified Previously, the code eagerly reported transactions as "processed" when they were only queued for processing. This commit separates the counts into three distinct metrics: - Transactions Queued: Count of transactions matching the rule's filter conditions before any processing begins - Transactions Processed: Count of transactions that were actually processed and modified by the rule actions - Transactions Modified: Count of transactions that had their values changed (currently same as Processed, but allows for future differentiation) Changes: - Add transactions_queued column to rule_runs table - Update RuleJob to track all three counts separately - Update action executors to return count of modified transactions - Update Rule#apply to aggregate modification counts from actions - Add transactions_queued label to locales - Update Recent Runs view to display new column - Add validation for transactions_queued in RuleRun model The tracking now correctly reports: 1. How many transactions matched the filter (queued) 2. How many were actually modified (processed/modified) 3. Distinguishes between matching and modifying transactions * Add Pending status to track async rule execution progress Introduced a new "pending" status for rule runs to properly track async AI operations. The system now: - Tracks pending async jobs with a counter that decrements as jobs complete - Updates transactions_modified incrementally as each job finishes - Only counts transactions that were actually modified (not just queued) - Displays pending status with yellow badge in the UI - Automatically transitions from pending to success when all jobs complete This provides better visibility into long-running AI categorization and merchant detection operations, showing real-time progress as Sidekiq processes the batches. * Fix migration version to 7.2 as per project standards * Consolidate rule_runs migrations into single migration file Merged three separate migrations (create, add_transactions_queued, add_pending_jobs_count) into a single CreateRuleRuns migration. This provides better clarity and maintains a clean migration history. Changes: - Updated CreateRuleRuns migration to include all columns upfront - Removed redundant add_column migrations - Updated schema version to 2025_11_24_000000 * Linter and test fixes * Space optimization * LLM l10n is better than no l10n * Fix implementation for tags/AI rules * Fix tests * Use batch_size * Consider jobs "unknown" status sometimes * Rabbit suggestion * Rescue block for RuleRun.create! --------- Co-authored-by: Claude <noreply@anthropic.com>
144 lines
4.5 KiB
Ruby
144 lines
4.5 KiB
Ruby
# Enrichable models can have 1+ of their fields enriched by various
|
|
# external sources (i.e. Plaid) or internal sources (i.e. Rules)
|
|
#
|
|
# This module defines how models should, lock, unlock, and edit attributes
|
|
# based on the source of the edit. User edits always take highest precedence.
|
|
#
|
|
# For example:
|
|
#
|
|
# If a Rule tells us to set the category to "Groceries", but the user later overrides
|
|
# a transaction with a category of "Food", we should not override the category again.
|
|
#
|
|
module Enrichable
|
|
extend ActiveSupport::Concern
|
|
|
|
InvalidAttributeError = Class.new(StandardError)
|
|
|
|
included do
|
|
scope :enrichable, ->(attrs) {
|
|
attrs = Array(attrs).map(&:to_s)
|
|
json_condition = attrs.each_with_object({}) { |attr, hash| hash[attr] = true }
|
|
where.not(Arel.sql("#{table_name}.locked_attributes ?| array[:keys]"), keys: attrs)
|
|
}
|
|
end
|
|
|
|
# Convenience method for a single attribute
|
|
def enrich_attribute(attr, value, source:, metadata: {})
|
|
enrich_attributes({ attr => value }, source:, metadata:)
|
|
end
|
|
|
|
# Enriches and logs all attributes that:
|
|
# - Are not locked
|
|
# - Are not ignored
|
|
# - Have changed value from the last saved value
|
|
# Returns true if any attributes were actually changed, false otherwise
|
|
def enrich_attributes(attrs, source:, metadata: {})
|
|
# Track current values before modification for virtual attributes (like tag_ids)
|
|
current_values = {}
|
|
enrichable_attrs = Array(attrs).reject do |attr_key, attr_value|
|
|
if locked?(attr_key) || ignored_enrichable_attributes.include?(attr_key)
|
|
true
|
|
else
|
|
# For virtual attributes (like tag_ids), use the getter method
|
|
# For regular attributes, use self[attr_key]
|
|
current_value = if respond_to?(attr_key.to_sym)
|
|
send(attr_key.to_sym)
|
|
else
|
|
self[attr_key.to_s]
|
|
end
|
|
|
|
# Normalize arrays for comparison (sort them)
|
|
if current_value.is_a?(Array) && attr_value.is_a?(Array)
|
|
current_values[attr_key] = current_value
|
|
current_value.sort == attr_value.sort
|
|
else
|
|
current_values[attr_key] = current_value
|
|
current_value == attr_value
|
|
end
|
|
end
|
|
end
|
|
|
|
return false if enrichable_attrs.empty?
|
|
|
|
was_modified = false
|
|
ActiveRecord::Base.transaction do
|
|
enrichable_attrs.each do |attr, value|
|
|
self.send("#{attr}=", value)
|
|
|
|
# If it's a new record, this isn't technically an "enrichment". No logging necessary.
|
|
unless self.new_record?
|
|
log_enrichment(attribute_name: attr, attribute_value: value, source: source, metadata: metadata)
|
|
end
|
|
end
|
|
|
|
save
|
|
|
|
# For virtual attributes (like tag_ids), previous_changes won't track them
|
|
# So we need to check if the value actually changed by comparing before/after
|
|
if previous_changes.any?
|
|
was_modified = true
|
|
else
|
|
# Check if any virtual attributes changed by comparing current value with what we set
|
|
enrichable_attrs.each do |attr, new_value|
|
|
# Get the current value after save (for virtual attributes, this reflects the change)
|
|
current_value = if respond_to?(attr.to_sym)
|
|
send(attr.to_sym)
|
|
else
|
|
self[attr.to_s]
|
|
end
|
|
|
|
old_value = current_values[attr]
|
|
if old_value.is_a?(Array) && new_value.is_a?(Array) && current_value.is_a?(Array)
|
|
was_modified = true if old_value.sort != current_value.sort
|
|
elsif old_value != current_value
|
|
was_modified = true
|
|
end
|
|
break if was_modified
|
|
end
|
|
end
|
|
end
|
|
|
|
# Return whether any attributes were actually saved
|
|
was_modified
|
|
end
|
|
|
|
def locked?(attr)
|
|
locked_attributes[attr.to_s].present?
|
|
end
|
|
|
|
def enrichable?(attr)
|
|
!locked?(attr)
|
|
end
|
|
|
|
def lock_attr!(attr)
|
|
update!(locked_attributes: locked_attributes.merge(attr.to_s => Time.current))
|
|
end
|
|
|
|
def unlock_attr!(attr)
|
|
update!(locked_attributes: locked_attributes.except(attr.to_s))
|
|
end
|
|
|
|
def lock_saved_attributes!
|
|
saved_changes.keys.reject { |attr| ignored_enrichable_attributes.include?(attr) }.each do |attr|
|
|
lock_attr!(attr)
|
|
end
|
|
end
|
|
|
|
private
|
|
def log_enrichment(attribute_name:, attribute_value:, source:, metadata: {})
|
|
de = DataEnrichment.find_or_create_by(
|
|
enrichable: self,
|
|
attribute_name: attribute_name,
|
|
source: source,
|
|
)
|
|
|
|
de.value = attribute_value
|
|
de.metadata = metadata
|
|
de.save
|
|
end
|
|
|
|
def ignored_enrichable_attributes
|
|
%w[id updated_at created_at]
|
|
end
|
|
end
|