Two structural changes from the second SQLAlchemy review. No behaviour
change; full activity-view suite (70 unit + 30 integration) still
green.
* Tidy 1 (Warning #2): switch _batch_chart_counts and
_batch_datasets_used_by_charts from positional row-unpack to
.mappings()-keyed access. Column-rename-safe — a future edit that
reorders columns in the SELECT can't silently misassign values
during the Python loop. Matches the existing convention in
_select_change_rows_for_kinds.
* Tidy 2 (Suggestion #1): docstring on _seed_activity_history
explaining why it intentionally commits without rollback (unlike
the T053 convention in activity_view_tests.py). The seed IS the
setup, not the unit under test — the endpoint reads a realistic
post-commit state of the shadow tables.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two opt-in tests (SUPERSET_PERF_VALIDATION=1 to run, skipped otherwise)
under the existing PerfValidationTests class:
* test_av_sc001_activity_endpoint_p95_under_1500ms — seeds dense
history on the birth_names dashboard (~12 chart edits, 5 dataset
edits, 1 dashboard edit), then hits /api/v1/dashboard/<uuid>/activity/
50 times after warmup and asserts p95 < 1500ms per SC-AV-001.
* test_av_sc003_save_path_p95_unaffected_by_activity_view — under the
same fixture, runs 50 dashboard saves and re-measures the SC-004
version-capture overhead (50ms p95 budget). Confirms the activity-
view branch hasn't added a new save-path coupling that regresses
sc-103156's save-path budget. PASSES on the current branch — read-
only feature, no new save-path code paths.
Seed fixture differs from the spec's "25 charts × 3 dataset windows
each" target — the birth_names fixture has ~12 charts on a single
dataset, so we approximate by editing many charts plus the dataset
multiple times. A bespoke multi-dataset fixture would let us measure
against the spec's exact scale; that's future work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a structured per-field change log alongside the foundational
shadow tables. Each save flush emits zero or more ``version_changes``
rows describing what changed relative to the previous version, with
shape ``[{kind, path, from_value, to_value, sequence}]`` keyed to
``version_transaction.id`` (FR-016 .. FR-021).
**Schema** — ``version_changes`` table, FK to ``version_transaction``
with ``ON DELETE CASCADE`` so retention drops dependent records
without explicit cleanup. Composite unique index on
``(transaction_id, entity_kind, entity_id, sequence)`` so the
listener can write monotonically and downstream readers see a
deterministic order.
**Diff engine** (``superset/versioning/diff.py``) — pure-function
diffing of pre-/post-state pairs:
- ``diff_scalar_fields`` for ordinary columns; emits one record per
changed field with JSON-safe ``from_value`` / ``to_value``.
- ``diff_json_field`` for ``json_metadata`` and ``params``, walking
the parsed structure and emitting per-sub-key records. Honours
an ``exclude_keys`` set
(``_DASHBOARD_JSON_METADATA_AUDIT_KEYS``: ``chart_configuration``,
``global_chart_configuration``, ``map_label_colors``,
``show_chart_timestamps``, ``color_namespace``;
``_CHART_PARAMS_AUDIT_KEYS``) so frontend-stamped sub-keys that
mutate on every save don't dominate the change log (FR-022).
- ``diff_dashboard_layout`` walks ``position_json`` structurally
and emits ``[verb, kind, id]`` records (verbs ``add``, ``remove``,
``move``, ``edit``; kinds from a ``CHART``/``ROW``/``COLUMN``/etc.
→ english map) so a UI can render "Added chart 'Foo'" without
re-parsing JSON. ``HEADER_ID`` is suppressed because it duplicates
the ``dashboard_title`` scalar record.
- ``fold_dashboard_layout_with_chart_changes`` deduplicates layout
records against M2M / chart-membership records by UUID so an
add-and-attach doesn't appear twice.
- ``_values_equivalent`` treats ``None`` and ``""`` as equal; this
matches the save path's habit of normalising nullable strings to
the empty string.
**Listener** — ``superset/versioning/changes.py`` registers a
``before_flush`` listener that captures pre-state for each dirty
entity and an ``after_flush`` listener that runs the diff engine
against the post-state and writes ``version_changes`` rows under
the resolved ``transaction_id``. Tracks processed transaction ids
on ``session.info`` so re-firings within a single transaction
(autoflush triggered by mid-commit queries) don't double-insert and
trip the unique constraint. Reads child rows via raw SELECT against
``table_columns`` / ``sql_metrics`` rather than ``dataset.columns``
because the live collection is stale during the restore path's raw
DELETE+INSERT cycle.
**Endpoint surface** — ``VersionDAO.list_change_records_batch``
batches the lookup across multiple transactions with a single
``WHERE transaction_id IN (...)`` query so the version-list
endpoint avoids N+1 round-trips. ``list_versions`` / ``get_version``
return entries with a populated ``changes`` array (empty for
``operation_type=0`` baseline rows).
**Tests** — ``test_diff.py`` covers the diff engine shape (39
unit cases across scalar, JSON, layout, child-collection, and
fold paths). ``change_records_tests.py`` exercises the listener
end-to-end with realistic save flows. ``perf_validation_tests.py``
is the T044 harness for SC-002/3/4 (list endpoint p95 < 1s,
restore < 3s, save overhead < 50ms).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
