QT/sure
1
0
Fork 0
mirror of https://github.com/we-promise/sure.git synced 2026-06-06 11:19:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(imports): add SureImport session batches (#1785)

Browse Source 
* feat(imports): add SureImport session batches

Add first-class SureImport sessions for ordered multi-file NDJSON imports.

Persist source mappings across chunks, make session/chunk processing idempotent, expose progress readback, and keep existing single-file import behavior compatible.

Includes the devcontainer libvips runtime dependency needed by ActiveStorage variant tests.

Addresses #1610.

Related to #1458.

* fix(imports): avoid scanner-like API key test data

* test(imports): assert skipped balances are not persisted

* fix(imports): harden session publish retries

Validate expected import chunk sequences exactly before publish, and restore session state with error details when enqueueing the publish job fails.

* fix(imports): close session retry edge cases

Backfill expected chunk counts after client-session insert races and enqueue import-session jobs after the status transition commits. Persist a safe enqueue failure body so API readback does not expose raw queue errors.

* fix(imports): address session publish review gaps

Remove dead transaction external-id assignment, harden session publish retry/sync behavior, align session chunk status docs, and add regression coverage for partial retries and safe enqueue error readback.

* fix(imports): include sessions in family reset

Clear import sessions through the family reset job so chunk imports and source mappings do not survive a reset.

Expose import session and source mapping counts in the reset status response and regenerated OpenAPI schema so polling reflects the full reset surface.

* test(imports): cover split import mapping invariants

* test(imports): cover session verification invariants

* fix(imports): scope SureImport session reimports

* Tighten SureImport session batching

* fix(imports): export rule source ids for sessions

* test(imports): stabilize rule id export assertion

* test(imports): restore reset status session fixture
This commit is contained in:
ghost
2026-06-04 02:48:44 -07:00
committed by GitHub
parent 76d4c2a2fe
commit 6e04c6927d
23 changed files with 3542 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
test/controllers/api/v1/users_controller_test.rb
View File

@@ -116,6 +116,20 @@ class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
end
test "reset status returns family data counts" do
import_session = @user.family.import_sessions.create!(expected_chunks: 1)
import_session.imports.create!(
family: @user.family,
type: "SureImport",
sequence: 1,
checksum: "a" * 64
)
import_session.source_mappings.create!(
family: @user.family,
source_type: "Category",
source_id: "source-category-1",
target: @user.family.categories.first
)
get "/api/v1/users/reset/status", headers: api_headers(@read_only_api_key)
assert_response :ok
@@ -124,6 +138,8 @@ class Api::V1::UsersControllerTest < ActionDispatch::IntegrationTest
assert_includes %w[complete data_remaining], body["status"]
assert_equal body["counts"].values.sum.zero?, body["reset_complete"]
assert_equal expected_reset_count_keys.sort, body["counts"].keys.sort
assert_equal 1, body["counts"]["import_sessions"]
assert_equal 1, body["counts"]["import_source_mappings"]
end
test "reset status ignores the follow-up family sync after reset" do