Feat: /import endpoint & drag-n-drop imports (#501)

* Implement API v1 Imports controller

- Add Api::V1::ImportsController with index, show, and create actions
- Add Jbuilder views for index and show
- Add integration tests
- Implement row generation logic in create action
- Update routes

* Validate import account belongs to family

- Add validation to Import model to ensure account belongs to the same family
- Add regression test case in Api::V1::ImportsControllerTest

* updating docs to be more detailed

* Rescue StandardError instead of bare rescue in ImportsController

* Optimize Imports API and fix documentation

- Implement rows_count counter cache for Imports
- Preload rows in Api::V1::ImportsController#show
- Update documentation to show correct OAuth scopes

* Fix formatting in ImportsControllerTest

* Permit all import parameters and fix unknown attribute error

* Restore API routes for auth, chats, and messages

* removing pr summary

* Fix trailing whitespace and configured? test failure

- Update Import#configured? to use rows_count for performance and consistency
- Mock rows_count in TransactionImportTest
- Fix trailing whitespace in migration

* Harden security and fix mass assignment in ImportsController

- Handle type and account_id explicitly in create action
- Rename import_params to import_config_params for clarity
- Validate type against Import::TYPES

* Fix MintImport rows_count update and migration whitespace

- Update MintImport#generate_rows_from_csv to update rows_count counter cache
- Fix trailing whitespace and final newline in AddRowsCountToImports migration

* Implement full-screen Drag and Drop CSV import on Transactions page

- Add DragAndDropImport Stimulus controller listening on document
- Add full-screen overlay with icon and text to Transactions index
- Update ImportsController to handle direct file uploads via create action
- Add system test for drag and drop functionality

* Implement Drag and Drop CSV upload on Import Upload page

- Add drag-and-drop-import controller to import/uploads/show
- Add full-screen overlay to import/uploads/show
- Annotate upload form and input with drag-and-drop targets
- Add PR_SUMMARY.md

* removing pr summary

* Add file validation to ImportsController

- Validate file size (max 10MB) and MIME type in create action
- Prevent memory exhaustion and invalid file processing
- Defined MAX_CSV_SIZE and ALLOWED_MIME_TYPES in Import model

* Refactor dragLeave logic with counter pattern to prevent flickering

* Extract shared drag-and-drop overlay partial

- Create app/views/imports/_drag_drop_overlay.html.erb
- Update transactions/index and import/uploads/show to use the partial
- Reduce code duplication in views

* Update Brakeman and harden ImportsController security

- Update brakeman to 7.1.2
- Explicitly handle type assignment in ImportsController#create to avoid mass assignment
- Remove :type from permitted import parameters

* Fix trailing whitespace in DragAndDropImportTest

* Don't commit LLM comments as file

* FIX add api validation

---------

Co-authored-by: Carlos Adames <cj@Carloss-MacBook-Air.local>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: sokie <sokysrm@gmail.com>

app/controllers/api/v1/imports_controller.rb

@@ -0,0 +1,171 @@
+# frozen_string_literal: true
+
+class Api::V1::ImportsController < Api::V1::BaseController
+  include Pagy::Backend
+
+  # Ensure proper scope authorization
+  before_action :ensure_read_scope, only: [ :index, :show ]
+  before_action :ensure_write_scope, only: [ :create ]
+  before_action :set_import, only: [ :show ]
+
+  def index
+    family = current_resource_owner.family
+    imports_query = family.imports.ordered
+
+    # Apply filters
+    if params[:status].present?
+      imports_query = imports_query.where(status: params[:status])
+    end
+
+    if params[:type].present?
+      imports_query = imports_query.where(type: params[:type])
+    end
+
+    # Pagination
+    @pagy, @imports = pagy(
+      imports_query,
+      page: safe_page_param,
+      limit: safe_per_page_param
+    )
+
+    @per_page = safe_per_page_param
+
+    render :index
+
+  rescue StandardError => e
+    Rails.logger.error "ImportsController#index error: #{e.message}"
+    render json: { error: "internal_server_error", message: e.message }, status: :internal_server_error
+  end
+
+  def show
+    render :show
+  rescue StandardError => e
+    Rails.logger.error "ImportsController#show error: #{e.message}"
+    render json: { error: "internal_server_error", message: e.message }, status: :internal_server_error
+  end
+
+  def create
+    family = current_resource_owner.family
+
+    # 1. Determine type and validate
+    type = params[:type].to_s
+    type = "TransactionImport" unless Import::TYPES.include?(type)
+
+    # 2. Build the import object with permitted config attributes
+    @import = family.imports.build(import_config_params)
+    @import.type = type
+    @import.account_id = params[:account_id] if params[:account_id].present?
+
+    # 3. Attach the uploaded file if present (with validation)
+    if params[:file].present?
+      file = params[:file]
+
+      if file.size > Import::MAX_CSV_SIZE
+        return render json: {
+          error: "file_too_large",
+          message: "File is too large. Maximum size is #{Import::MAX_CSV_SIZE / 1.megabyte}MB."
+        }, status: :unprocessable_entity
+      end
+
+      unless Import::ALLOWED_MIME_TYPES.include?(file.content_type)
+        return render json: {
+          error: "invalid_file_type",
+          message: "Invalid file type. Please upload a CSV file."
+        }, status: :unprocessable_entity
+      end
+
+      @import.raw_file_str = file.read
+    elsif params[:raw_file_content].present?
+      if params[:raw_file_content].bytesize > Import::MAX_CSV_SIZE
+        return render json: {
+          error: "content_too_large",
+          message: "Content is too large. Maximum size is #{Import::MAX_CSV_SIZE / 1.megabyte}MB."
+        }, status: :unprocessable_entity
+      end
+
+      @import.raw_file_str = params[:raw_file_content]
+    end
+
+    # 4. Save and Process
+    if @import.save
+      # Generate rows if file content was provided
+      if @import.uploaded?
+        begin
+          @import.generate_rows_from_csv
+          @import.reload
+        rescue StandardError => e
+          Rails.logger.error "Row generation failed for import #{@import.id}: #{e.message}"
+        end
+      end
+
+      # If the import is configured (has rows), we can try to auto-publish or just leave it as pending
+      # For API simplicity, if enough info is provided, we might want to trigger processing
+
+      if @import.configured? && params[:publish] == "true"
+        @import.publish_later
+      end
+
+      render :show, status: :created
+    else
+      render json: {
+        error: "validation_failed",
+        message: "Import could not be created",
+        errors: @import.errors.full_messages
+      }, status: :unprocessable_entity
+    end
+
+  rescue StandardError => e
+    Rails.logger.error "ImportsController#create error: #{e.message}"
+    render json: { error: "internal_server_error", message: e.message }, status: :internal_server_error
+  end
+
+  private
+
+    def set_import
+      @import = current_resource_owner.family.imports.includes(:rows).find(params[:id])
+    rescue ActiveRecord::RecordNotFound
+      render json: { error: "not_found", message: "Import not found" }, status: :not_found
+    end
+
+    def ensure_read_scope
+      authorize_scope!(:read)
+    end
+
+    def ensure_write_scope
+      authorize_scope!(:write)
+    end
+
+    def import_config_params
+      params.permit(
+        :date_col_label,
+        :amount_col_label,
+        :name_col_label,
+        :category_col_label,
+        :tags_col_label,
+        :notes_col_label,
+        :account_col_label,
+        :qty_col_label,
+        :ticker_col_label,
+        :price_col_label,
+        :entity_type_col_label,
+        :currency_col_label,
+        :exchange_operating_mic_col_label,
+        :date_format,
+        :number_format,
+        :signage_convention,
+        :col_sep,
+        :amount_type_strategy,
+        :amount_type_inflow_value
+      )
+    end
+
+    def safe_page_param
+      page = params[:page].to_i
+      page > 0 ? page : 1
+    end
+
+    def safe_per_page_param
+      per_page = params[:per_page].to_i
+      (1..100).include?(per_page) ? per_page : 25
+    end
+end

app/controllers/imports_controller.rb

@@ -26,14 +26,38 @@ class ImportsController < ApplicationController
   end
 
   def create
+    type = params.dig(:import, :type).to_s
+    type = "TransactionImport" unless Import::TYPES.include?(type)
+
     account = Current.family.accounts.find_by(id: params.dig(:import, :account_id))
     import = Current.family.imports.create!(
-      type: import_params[:type],
+      type: type,
       account: account,
       date_format: Current.family.date_format,
     )
 
-    redirect_to import_upload_path(import)
+    if import_params[:csv_file].present?
+      file = import_params[:csv_file]
+
+      if file.size > Import::MAX_CSV_SIZE
+        import.destroy
+        redirect_to new_import_path, alert: "File is too large. Maximum size is #{Import::MAX_CSV_SIZE / 1.megabyte}MB."
+        return
+      end
+
+      unless Import::ALLOWED_MIME_TYPES.include?(file.content_type)
+        import.destroy
+        redirect_to new_import_path, alert: "Invalid file type. Please upload a CSV file."
+        return
+      end
+
+      # Stream reading is not fully applicable here as we store the raw string in the DB,
+      # but we have validated size beforehand to prevent memory exhaustion from massive files.
+      import.update!(raw_file_str: file.read)
+      redirect_to import_configuration_path(import), notice: "CSV uploaded successfully."
+    else
+      redirect_to import_upload_path(import)
+    end
   end
 
   def show
@@ -70,6 +94,6 @@ class ImportsController < ApplicationController
     end
 
     def import_params
-      params.require(:import).permit(:type)
+      params.require(:import).permit(:csv_file)
     end
 end