Commit Graph

3 Commits

Author SHA1 Message Date
Augusto Xavier
844cba2fd3 fix(imports): normalize CSV upload encoding when validations are skipped (#2299)
Import::UploadsController#update persists CSV uploads with
save!(validate: false), which skips the before_validation
:ensure_utf8_encoding callback. Non-UTF-8 files (e.g. ISO-8859-1 /
Windows-1252 exports from Brazilian banks) were written to the text
column as-is and rejected by Postgres with PG::CharacterNotInRepertoire,
surfacing as a 500 during upload.

Also register ensure_utf8_encoding on before_save so the existing
normalization (rchardet detection + Latin-1/Windows fallback) still runs
when validations are skipped. The callback is idempotent and no-ops on
valid UTF-8, so the validated path is unchanged.

Fixes #2294
2026-06-16 17:13:03 +02:00
soky srm
25ac822308 Reports print functionality (#622)
* Print initial impl

* Try to keep the bigger section together

* /* Tufte-inspired Print Report Styles */

* styling

* I8n

* Move print styling out.

* FIX unrelated test ordering

on line 53 - import.rows.first doesn't guarantee ordering. Without an explicit ORDER BY, the database may return rows in any order.

* Update print-report.css

* Update print.html.erb

* pass data to view

* Update index.html.erb

* Fix ERB helpers

* Update reports_helper.rb
2026-01-12 14:40:30 +01:00
Copilot
5b736bf691 Fix CSV import for non-UTF-8 encodings (Windows-1250, ISO-8859-2, etc.) (#617)
* Initial plan

* Add encoding detection for CSV imports to handle Windows-1250 and other non-UTF-8 encodings

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Improve encoding detection: prioritize Windows-1250 and increase confidence threshold

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Update Gemfile.lock with rchardet dependency

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Refactor: Extract common encodings to constant and deduplicate code

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Fix Rubocop style violations

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Fix linter violations and encoding detection logic

- Remove trailing whitespace from test file (9 lines)
- Fix ensure_utf8_encoding to handle binary strings properly by checking bytesize instead of blank?
- Add error handling for ArgumentError and Encoding::CompatibilityError
- Add invalid/undef replacement options to encode calls for robustness

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Fix encoding error in ensure_utf8_encoding method

- Use will_save_change_to_raw_file_str? instead of raw_file_str_changed? to avoid encoding errors when checking if attribute changed
- Wrap UTF-8 validation check in begin/rescue to handle ArgumentError from invalid encodings
- This fixes the test failure: "ArgumentError: invalid byte sequence in UTF-8"

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Fix test: add missing column labels and reload import before checking rows

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

* Fix test: ensure import is reloaded before checking rows_count and accessing rows

Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jjmata <187772+jjmata@users.noreply.github.com>
2026-01-12 10:17:55 +01:00