Compare commits

...

174 Commits

Author SHA1 Message Date
Mike Bridge
1c8d414494 debug(activity-view): drop @has_access on debug shell route
The previous @has_access decorator on ActivityDebugView.show was
redirecting authenticated-but-not-authorized requests to the home
page (FAB's access-denied default), which is what the user was seeing
instead of the React app. Then @login_required was tried briefly but
crashed because flask-login's redirect endpoint isn't wired up under
this app's auth backend.

Simplest fix: no decorator on the shell route. The shell page exposes
no data of its own — the React component fires API calls to
/api/v1/{resource}/{uuid}/activity/ which gate access via
raise_for_ownership on the path entity. Anonymous users would see
the React UI with 401 errors inline, which is correct UX for a debug
tool.

Verified: GET /activity-debug/dashboard/<uuid>/ → 200 + SPA shell HTML.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
466cd5ffae debug(activity-view): register Flask route to serve React shell
The React route at /activity-debug/{resource}/{uuid} was returning
a JSON 404 on fresh page-load because Superset's SPA model doesn't
use a Flask catch-all — every React-router path needs a corresponding
Flask view that calls render_app_template(). Adding the missing
piece.

* superset/views/activity_debug.py — new ActivityDebugView (uses
  BaseSupersetView). Single @expose("/<resource>/<uuid>/") returning
  the React shell; the in-app router (superset-frontend/src/views/
  routes.tsx) handles the actual page rendering. @has_access keeps
  it gated by login.
* superset/initialization/__init__.py — appbuilder.add_view_no_menu
  registration alongside the other SPA shell views.

Throwaway by design; both edits tagged with "Throwaway: sc-107283"
comments for easy removal when the activity-view feature ships.

Verified: GET /activity-debug/dashboard/<uuid>/ returns 302 to
/login when unauthenticated (correct), 200 with the SPA shell when
logged in.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
69e782a601 debug(activity-view): throwaway React debug UI for activity timelines
Adds /activity-debug/{dashboard,chart,dataset}/{uuid} for visual
verification of the sc-107283 activity-view endpoints. Lets you eyeball
the response shape without building a curl/jq habit:

* Controls for include / page / page_size / since / until
* Per-record cards showing entity_kind chip, source badge, kind tag,
  tombstone affordance, issued_at, changed_by, summary, entity_uuid,
  version_uuid, path, from/to values, impact
* Deleted entities render with strikethrough + "deleted" tag
* Soft-deleted state surfaces as an orange tag (currently always null
  until sc-103157 lands deleted_at)

**Throwaway by design**: when the activity-view feature ships, delete
the lazy import + route entry from superset-frontend/src/views/routes.tsx
and remove the superset-frontend/src/pages/ActivityDebug/ directory.
Both edits are tagged with "Throwaway: sc-107283" comments so they're
easy to find.

The component uses @superset-ui/core/components (Card, Tag, Radio, Input,
Space, Typography, Loading, Empty) per CLAUDE.md conventions. TypeScript
clean, ruff/eslint/oxlint clean, no any types. Uses a native HTML
<select> for page_size — the @superset-ui Select wrapper's stricter prop
types weren't worth fighting for a debug tool.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
ec851849fb refactor(activity-view): JSON entity_kind uses user-facing labels
Breaking change to the (still-unmerged) activity-view JSON contract:
``entity_kind`` now emits ``"dashboard"`` / ``"chart"`` / ``"dataset"``
instead of the Python class names ``"Dashboard"`` / ``"Slice"`` /
``"SqlaTable"``. The class names are developer-facing artifacts that
leaked the model layer (e.g. ``"Slice"`` predates the UI rename to
"chart"; ``"SqlaTable"`` predates "dataset"). User-facing JSON should
speak user language.

Implementation: a new ``_USER_FACING_KIND`` map translates at JSON
serialization time only (in ``_decorate_records``). Internal code keeps
the Python class-name form (``model_cls.__name__``) for dispatch — the
existing ``_NAME_COLUMN``, ``_NOT_FOUND_EXC``, ``_API_KIND_LABEL``,
``_can_read``, ``_compute_impact``, etc. all key off class names and
are unchanged. The translation happens at the single ``record["entity_kind"]
= ...`` assignment.

Schema validator ``ACTIVITY_ENTITY_KINDS`` updated to the new tuple.
Integration tests' response-shape assertions renamed via bulk sed; unit
tests testing internal helpers are unchanged (they operate on internal
api_kind / class names).

UPDATING.md example payload updated. Spec updates (spec.md, data-model.md,
contracts/activity-view.yaml) committed separately to the spec repo.

Full suite: 66 unit + 35 integration + 1 xfailed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
36203af96d feat(activity-view): T037/T038 observability + T050 cross-coupling check
T037 — Instrument the five phases of get_activity with stats_logger
timing metrics under the `superset.activity_view.<kind>.<phase>_ms`
key convention (plan §D-17). Wrap each phase with a stats_timing
context manager:

* superset.activity_view.<kind>.relationship_resolution_ms (_resolve_scope)
* superset.activity_view.<kind>.fetch_ms (_fetch_change_records)
* superset.activity_view.<kind>.visibility_filter_ms
* superset.activity_view.<kind>.denormalize_ms
* superset.activity_view.<kind>.decorate_ms

`<kind>` is the lowercased path-entity model class name (dashboard /
slice / sqlatable), enabling per-endpoint-family Grafana panels.

T038 — Add request- and response-shape attributes via the existing
counter/gauge interface:

* superset.activity_view.<kind>.requests.include_<value>  (incr)
* superset.activity_view.<kind>.requests.has_since_filter_<true|false>  (incr)
* superset.activity_view.<kind>.page_size  (gauge)
* superset.activity_view.<kind>.record_count  (gauge — post-visibility-filter)
* superset.activity_view.<kind>.related_entity_count.charts  (gauge)
* superset.activity_view.<kind>.related_entity_count.datasets  (gauge)

Confirmed no PII: entity names, diff content, user identifiers — none
flow into the metric layer. Only counts and shape tags.

T050 — Cross-coupling sanity test (unit-scope): asserts
_METRIC_PREFIX == "superset.activity_view" so a code review catches
accidental drift from sc-103156's eventual "superset.versioning.*"
sibling namespace. Both endpoint families belong to one Grafana panel
under "superset.versioning.* OR superset.activity_view.*".

Full suite: 66 unit + 35 integration + 1 xfailed (T044 sc-103156
restore-kind dependency, unchanged).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
3a1667522a test(activity-view): four more polish tasks — T039/T042/T044/T051
* T042 test_activity_marks_hard_deleted_chart_with_tombstone (D-15)
  Edit a chart, hard-delete it via db.session.delete(chart) + commit,
  hit dashboard activity. Asserts tombstoned record has
  entity_deleted=True, entity_uuid=None, entity_name preserved from
  the last shadow row. The fixture's _cleanup tolerates missing slices
  (uses Slice.id.in_(slice_ids) which silently skips).

* T044 test_activity_surfaces_dashboard_restore_event (AV-015) — xfail
  AV-015 requires sc-103156's restore code to emit a synthetic
  kind="restore" change record with to_value carrying the source
  version_uuid + label. sc-103156's restore_version() currently does
  not — the diff capture surfaces the field changes as kind="field".
  The activity layer correctly passes through whatever kind sc-103156
  emits; the test will start passing once the upstream emission lands.
  Marked strict=True so the day sc-103156 emits, this fails-to-fail
  and we know to revisit.

* T051 test_activity_excludes_records_after_retention_prune (AV-010)
  Edit a chart (creates a new version_transaction), backdate that
  transaction's issued_at past the 30-day retention cutoff, run
  _prune_old_versions_impl(retention_days=30). Assert the prune
  removed ≥1 transaction AND the activity endpoint's filtered count
  decreased.

* T039 test_activity_pagination_is_deterministic_and_disjoint
  (SC-AV-002 pragmatic interpretation). The spec's stricter "no
  skip/duplicate under concurrent writes" is unprovable with offset
  pagination — new top-inserted records shift every later page by
  one. Cursor pagination would solve this (deferred per plan §D-10).
  Under offset, the testable guarantees are: (a) same request fired
  twice produces identical pages; (b) page N and page N+1 are
  disjoint under one request round. Both come from the
  (issued_at DESC, transaction_id DESC, sequence DESC) sort.

Full activity-view suite: 35 passed, 1 xfailed in 46s. The xfail is
T044 with the documented sc-103156 dependency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
fa73b00da6 test(activity-view): T040/T041 polish — ordering + page-size cap
Three of the Phase 6 polish tasks landed together:

* T040 test_activity_ordering_is_stable_by_issued_at_then_transaction_id
  — AV-006 deterministic-ordering contract. Iterates adjacent pairs in
  the response and asserts (issued_at, transaction_id) is monotonically
  non-increasing. Catches any sort-stability regression — random
  ordering would fail the pairwise check almost certainly.
* T041 test_activity_page_size_caps_returned_records_at_200 — pairs
  with the existing pagination_clamps test. The former confirms
  ?page_size=500 doesn't 400; this one confirms the response is
  bounded to ≤200 records as the OpenAPI contract guarantees.
* T053 verification: every fixture-mutating test in
  activity_view_tests.py was already following the
  try/finally + rollback convention (sweep verified 30 tests; zero
  non-conformant). No remediation needed; documenting the
  sweep result in the commit message.

Full activity-view suite: 32/32 integration + 65/65 unit tests green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
3a9f863194 refactor(activity-view): SQL review tidies — mappings + perf-seed docstring
Two structural changes from the second SQLAlchemy review. No behaviour
change; full activity-view suite (70 unit + 30 integration) still
green.

* Tidy 1 (Warning #2): switch _batch_chart_counts and
  _batch_datasets_used_by_charts from positional row-unpack to
  .mappings()-keyed access. Column-rename-safe — a future edit that
  reorders columns in the SELECT can't silently misassign values
  during the Python loop. Matches the existing convention in
  _select_change_rows_for_kinds.

* Tidy 2 (Suggestion #1): docstring on _seed_activity_history
  explaining why it intentionally commits without rollback (unlike
  the T053 convention in activity_view_tests.py). The seed IS the
  setup, not the unit under test — the endpoint reads a realistic
  post-commit state of the shadow tables.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
440e298c4f docs(activity-view): UPDATING.md note + T049 OpenAPI verification
Two doc-shaped tasks landed together because they're both about making
the activity-view endpoints discoverable by external consumers.

T047 — UPDATING.md: new section under the existing entity-version-history
entry documenting the three activity-view endpoints (dashboard / chart /
dataset), their query params (since / until / include / page / page_size),
the response shape with all DTO fields, the silent permission filter
(AV-008), tombstone behaviour (D-15), and the no-feature-flag / no-
new-tables impact statement. Mirrors the depth of the sc-103156
versioning section above it.

T048 satisfied by the UPDATING.md entry: the activity-view feature
adds no new config keys (no SUPERSET_* env vars, no feature flag), and
the per-endpoint API reference is auto-generated from the YAML
docstrings via FAB's OpenAPI integration. The `/swagger/v1` page picks
up the activity endpoints automatically — verified by the new tests
below. sc-103156 followed the same pattern (UPDATING.md only, no
standalone config doc).

T049 — Three new tests in TestActivityOpenApiSpec verify FAB's OpenAPI
generation includes the activity endpoints with the right shape:

* test_three_activity_paths_appear_in_openapi — the three
  /<uuid_str>/activity/ paths are surfaced in /api/v1/_openapi.
* test_activity_endpoints_document_query_params — since / until /
  include / page / page_size are all declared, and include's enum is
  exactly {"self", "related", "all"}.
* test_activity_endpoints_declare_200_response — 200 + 400/401/403/404
  are all declared response codes.

base_api_tests.py::TestOpenApiSpec::test_open_api_spec already
validates the full spec's structural correctness on every CI run, so
malformed YAML in the activity-view docstrings would have been caught
upstream. The new tests add activity-specific assertions about the
endpoints' presence and parameter shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
a2d5bd05d0 perf(activity-view): batch _datasets_used_by_chart for dashboard scope
Surfaced by T045's perf test (which failed at p95 4228ms vs 1500ms
budget) and an ad-hoc per-phase profile that pinpointed _resolve_scope
as accounting for ~1926ms of the wall clock.

The previous _resolve_dashboard_scope called _datasets_used_by_chart
once per slice on the dashboard. On a fixture with ~12 charts that's
12 sequential SQL round-trips for the same shape of query, each
returning a small result set. The new _batch_datasets_used_by_charts
takes the full set of slice_ids and returns
{slice_id: [(dataset_id, window), ...]} in one query; the singular
form _datasets_used_by_chart now forwards to the batch (used only by
_resolve_chart_scope which has a single slice).

Impact (measured on the same fixture):
* Wall-clock per request: 3955ms → 2257ms (43% faster)
* Total query count per request: 18,376 → 29 (driven by the
  identity-map populating during the warmup; the genuine count for
  the activity layer alone is ~13 SQL queries)
* T045 p95 over 50 runs: 4228ms → 2429ms

The remaining gap to the 1500ms budget is dominated by Python work
inside _fetch_change_records: post-filter and sort over ~26K
accumulated test-environment records. The spec's target scale is
"25 charts × 3 dataset windows" — the SQLite test environment has
accumulated far more state from prior test runs. T046 (index
migration) is deferred because the remaining bottleneck is Python-
side, not SQL-side; production data on Postgres with retention
pruning will not have this shape. Document and revisit when a
production-representative perf measurement is available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
52a3ab0d71 test(activity-view): add T045 perf-validation tests
Two opt-in tests (SUPERSET_PERF_VALIDATION=1 to run, skipped otherwise)
under the existing PerfValidationTests class:

* test_av_sc001_activity_endpoint_p95_under_1500ms — seeds dense
  history on the birth_names dashboard (~12 chart edits, 5 dataset
  edits, 1 dashboard edit), then hits /api/v1/dashboard/<uuid>/activity/
  50 times after warmup and asserts p95 < 1500ms per SC-AV-001.

* test_av_sc003_save_path_p95_unaffected_by_activity_view — under the
  same fixture, runs 50 dashboard saves and re-measures the SC-004
  version-capture overhead (50ms p95 budget). Confirms the activity-
  view branch hasn't added a new save-path coupling that regresses
  sc-103156's save-path budget. PASSES on the current branch — read-
  only feature, no new save-path code paths.

Seed fixture differs from the spec's "25 charts × 3 dataset windows
each" target — the birth_names fixture has ~12 charts on a single
dataset, so we approximate by editing many charts plus the dataset
multiple times. A bespoke multi-dataset fixture would let us measure
against the spec's exact scale; that's future work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
4ead3f405f perf(activity-view): batch impact-count query to remove N+1
Surfaced by the SQLAlchemy review (Warning #1) and required before
T045's p95 perf budget can be met on dashboards with many historical
dataset edits.

Before: _decorate_records fired one COUNT(DISTINCT slice_id) query
per related dataset record via _compute_impact → _count_dashboard_
charts_pointing_at_dataset_at_tx. For a dashboard activity stream
with N dataset-edit records, that's N round-trips to compute impacts.

After: _decorate_records collects the distinct (dataset_id, target_tx)
pairs once via _collect_impact_pairs, fires a single batched query
via _batch_chart_counts that pulls the (slice, dataset, validity-
window) state for every relevant slice, and filters by validity in
Python per pair. The result is a {(dataset_id, target_tx): count}
mapping consumed by the new pure helper _impact_for_record per row.

Round-trip count drops from O(N records) to O(1) for the impact
calculation. The SQL stays small and dialect-portable — same per-kind
IN-clause + Python validity filter pattern as _fetch_change_records.

Trade-off documented in _batch_chart_counts' docstring: the SELECT
pulls (slice_id, datasource_id, two validity-window pairs) for every
slice ever on the dashboard whose dataset matches one of the
requested dataset_ids. For a busy dashboard with 100 slices and 50
versions each, that's ~5000 rows into Python vs N small COUNT scans.
For N > ~5 (which is typical) the batch wins.

Removed:
* _compute_impact (replaced by _impact_for_record + _collect_impact_pairs)
* _count_dashboard_charts_pointing_at_dataset_at_tx (replaced by
  _batch_chart_counts)

Test changes: the three _compute_impact unit tests (no-impact paths)
become six _impact_for_record tests (positive count + four no-impact
paths + zero-count → None). Five new _collect_impact_pairs tests
cover dashboard/chart/dataset path branching plus dedupe and empty.

Full suite: 27/27 integration + 65/65 unit (was 57; +8 from the
restructure). No semantic regression on either side of the cut.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
b96cb01670 docs(activity-view): document per-kind query rationale in _select_change_rows_for_kinds
No-op docstring change. The function fires one SELECT per entity_kind
with entity_id IN (...) instead of one query with tuple_(entity_kind,
entity_id).in_(...). The latter is supported by Postgres but its SQL
emission across MySQL and SQLite is uneven (some dialects emit literal
ROW expressions, some emit a transformed multi-column form, some don't
support it at all in older versions). Three round-trips per request is
the correct trade-off given Superset's multi-dialect requirement.

Documenting now so a future reader doesn't reach for the tuple-IN
"optimisation" and silently break MySQL or SQLite. Surfaced by the
SQLAlchemy review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
9d8a318f34 feat(activity-view): Phase 5 US3 — dataset /activity/ endpoint (T033-T036)
Implements the dataset activity stream. Per AV-004 datasets have no
transitive layer in V2: dataset activity = dataset's own edits only,
regardless of whether charts use the dataset. The chart/dashboard
endpoints surface dataset edits in their *related* streams (already
shipped); the dataset endpoint stays narrowly self-only.

* T033 DatasetRestApi.activity — third application of the shared
  resolve_endpoint_path_entity + parse_activity_query_params pattern.
  Endpoint body is ~10 lines of real logic, same shape as T016 and
  T028 but with SqlaTable as the path entity. The activity orchestrator
  already short-circuits the related-entity resolution for datasets
  (_resolve_scope returns [] for related when path_kind="SqlaTable"),
  so the dataset endpoint inherits the V2 semantics for free.

* T034 TestDatasetActivityView class added to activity_view_tests.py.

* T035 test_dataset_activity_excludes_chart_edits — AS-1 of US3 /
  AV-004 verbatim. Edit a chart that uses the dataset; assert the
  chart edit does NOT appear in the dataset's activity stream
  (datasets are read-only upstream of charts in V2).

* T036 test_dataset_activity_includes_dataset_self_edits — confirms
  the positive path: editing the dataset's own description surfaces
  a SqlaTable/self record.

* New test_dataset_activity_related_only_returns_empty — AV-004 in
  contract form: ?include=related on a dataset returns an empty
  result list and count=0 because there are no related entities to
  draw from.

Plus the standard boundary tests (404 for unknown UUID, 400 for
malformed UUID / invalid include, 403 for non-owner, 200 envelope
shape).

Full activity-view suite: 27/27 integration tests + 57/57 unit tests
green. All three endpoint families live.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
6b2d75ca0b feat(activity-view): Phase 4 US2 — chart /activity/ endpoint (T028-T032)
Implements the chart cross-entity activity stream (US2). Builds on the
resolve_endpoint_path_entity helper from the prior tidy commit
(1d60513079), so the new endpoint is a small composition of: resolve
path entity → parse params → get_activity(Slice, ...) → serialize.

* T028 ChartRestApi.activity — @expose("/<uuid_str>/activity/") on the
  chart blueprint. Decorator stack mirrors list_versions; registered
  in include_route_methods. The MODEL_API_RW_METHOD_PERMISSION_MAP
  entry "activity": "write" added in Phase 3 already covers this
  endpoint family (the map is class-permission-agnostic).

* T029 TestChartActivityView class added to activity_view_tests.py,
  following the same patterns as TestDashboardActivityView.

* T030 test_chart_activity_includes_dataset_edit_as_related — Given
  a chart pointing at the birth_names dataset, When the dataset's
  description is edited, Then the chart activity stream surfaces a
  SqlaTable/related record (AS-1 of US2).

* T032 test_chart_activity_excludes_sibling_dashboards — Given the
  chart is on a dashboard, When the dashboard's title is edited,
  Then NO Dashboard records appear in the chart's activity. Per the
  spec's Relationship Traversal section: charts don't see sideways
  to dashboards they happen to be on.

* T031 (datasource-switch attribution) deferred — needs a second
  dataset fixture which the birth_names environment doesn't provide
  cleanly. Will land with a multi-dataset test harness in a follow-up.

Additional coverage: 404 for unknown UUID, 400 for malformed UUID,
400 for invalid include, 403 for non-owner, 200 envelope smoke,
chart-self-edit appears as source=self, include=self filter test.

Full suite: 19/19 integration tests + 57/57 unit tests green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
a412f1d9e6 refactor(activity-view): extract resolve_endpoint_path_entity helper
Make the change easy, then make the easy change. Phase 4 (T028 chart
/activity/ endpoint, T033 dataset endpoint) duplicates the UUID-parse
+ find_active_by_uuid + raise_for_ownership dance from the existing
DashboardRestApi.activity. Extracting now keeps the next two endpoints
single-line affairs.

* New PathEntityResponseError carries a pre-built error Response.
  Subclassing Exception (not ValueError) so the endpoint's catch is
  precise and the standard hierarchy stays unpolluted.
* New resolve_endpoint_path_entity(api, model_cls, uuid_str) runs the
  three-step preflight and raises PathEntityResponseError with the
  right 400/404/403 response carried inside.
* DashboardRestApi.activity refactored to use the helper. 12 lines of
  imports + try/except boilerplate collapse to a single try/except
  block that maps every preflight failure to its response.

No behaviour change: 10/10 dashboard activity integration tests still
green. Pure structural change, per Beck's tidy-first discipline (the
T028 endpoint code follows in the next commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:25 -06:00
Mike Bridge
f840ce9813 refactor(activity-view): Phase 3 tidies from clean-code review
Five structural changes — no behaviour change — applied as one commit
per the tidy-first discipline. All from the clean-code review of
e4070a4716.

* Tidy 1 — Fix _fetch_change_records sort key. Was calling .timestamp()
  on issued_at with an `else 0` fallback; the fallback was dead defense
  (column is non-null per sc-103156 schema) and .timestamp() introduces
  non-determinism on tz-naive datetimes. Sort on the datetimes directly.

* Tidy 2 — parse_activity_query_params now raises ActivityParamsError
  (subclass of ValueError) instead of returning (Optional[dict],
  Optional[str]). The tuple was forcing every caller into a defensive
  `if error or params is None: return self.response_400(message=error
  or "Invalid query parameters")`. The new shape — `try: params =
  parse(...); except ActivityParamsError as exc: return
  response_400(str(exc))` — is shorter, type-safe, and the contract
  is enforced at the boundary.

* Tidy 3 — Test helper now uses Flask client's query_string= parameter
  instead of f-string concatenation. Handles URL-encoding correctly
  for the day a test passes a value containing & / = / + / etc.

* Tidy 4 — get_activity pipeline collapses to a single rolling
  `records` variable instead of the mid-stream `raw / visible_raw /
  enriched / visible` naming. Each function call's name documents
  what the step does; no intermediate variable names needed.

* Tidy 5 — Extracted four per-parameter parsers: _parse_optional_iso,
  _parse_include, _parse_page, _parse_page_size. The "parse one
  parameter" concept now has a name. Cost: four small helpers (each
  10-15 lines, one job). Benefit: parse_activity_query_params is a
  10-line table-driven dispatcher.

Test changes: parser unit tests now use pytest.raises(ActivityParamsError)
instead of unpacking the (params, error) tuple. Added one test
confirming ActivityParamsError subclasses ValueError so the standard
library exception hierarchy still catches it. Total unit tests: 57.
Integration tests still 10/10 green.

Deferred per the review: the UUID-parse + entity-find + ownership-
check dance is duplicated between activity / list_versions / get_version
and will grow to T028 / T033. Refactor when T028 lands — three real
callers > one prospective one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
4360ad7101 feat(activity-view): Phase 3 US1 — dashboard /activity/ endpoint
Implements the dashboard cross-entity activity stream (US1 MVP).
Closes T015-T018, T023-T024, T026 from sc-107283 tasks.md. T019-T022
(complex fixture choreography) and T025 (RBAC fixture for restricted
chart access) deferred to a follow-up; T025's logical coverage is
provided by the new unit tests for _can_read kind dispatch.

* T015 Rename _dashboard_related_scope → _resolve_dashboard_scope and
  _chart_related_scope → _resolve_chart_scope, parallel to _resolve_scope
  / _resolve_path_entity. Aligns the codebase with the task spec names.
* T016 New activity(uuid_str) method on DashboardRestApi:
  @expose("/<uuid_str>/activity/"), @protect + @safe + @statsd_metrics
  + @event_logger. Same row-level ownership check pattern as
  /versions/. Registered in include_route_methods and mapped to "write"
  in MODEL_API_RW_METHOD_PERMISSION_MAP so the can_write Dashboard
  permission gates access (Alpha-non-owner gets 403 from
  raise_for_ownership, not 404 from missing permission).
* New parse_activity_query_params helper in activity.py — shared parser
  for since/until/include/page/page_size used by all three endpoint
  families. Tolerates the Z suffix Python <3.11 fromisoformat rejects.
  Silently clamps page_size to the contract max instead of rejecting.

Two correctness/scale bugs found and fixed under integration-test load:

* SQLite SQLITE_MAX_EXPR_DEPTH (1000) was tripping on dashboards with
  many slices × many historical attachment windows. _fetch_change_records
  used to emit one OR-clause per (entity_kind, entity_id, window) tuple;
  now it issues one SELECT per kind with entity_id IN (...) and filters
  by exact windows in Python via _row_within_any_window. SQL shape is
  proportional to the number of kinds (≤3); the per-entity window
  precision is preserved.

* _merge_entity_windows now unions overlapping/touching windows within
  each entity via _union_windows. Sequential fixture loads create many
  redundant Continuum shadow rows; without merging, the unfiltered
  windows still produce many OR branches downstream.

* Pipeline-ordering fix in get_activity: visibility filter runs BEFORE
  decoration. Decoration strips entity_id (not in API contract) and the
  filter needs it — and dropping invisible records early also avoids
  paying for name lookup + tombstone probes on records the requester
  can't see (AV-008's silent-filter contract).

Tests:

* tests/integration_tests/versioning/activity_view_tests.py — 10 tests
  for TestDashboardActivityView covering: 404 for unknown UUID, 400 for
  malformed UUID / invalid include / invalid since, 403 for non-owner,
  200 envelope smoke, chart-edit-appears-as-related, include=self
  filter, include=related filter, page_size clamping.
* tests/unit_tests/versioning/test_activity.py — grew from 30 to 56
  tests. New coverage: parse_activity_query_params (7 cases), _can_read
  per-kind dispatch (4 cases — covers T025 at unit scope), _union_windows
  (9 parametrized cases), _merge_entity_windows window-union case,
  _row_within_any_window (6 cases).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
9f9dee0bec refactor(activity-view): Phase 2 tidies + unit tests for pure helpers
Five structural changes — no behaviour change — applied as one commit
per the tidy-first discipline (separate from the prior feat commits
that landed the primitives and orchestrator):

* Add tests/unit_tests/versioning/test_activity.py — 30 tests for the
  pure-function helpers: _intersect_windows (11 boundary cases),
  _resolve_scope branching (per kind + per include mode), entity-
  window merging, AV-012 summary headlines, changed_by projection,
  kind-translation round-trip, _can_read fall-through, _compute_impact
  no-impact paths. Runs in <500ms, no DB, no Flask. The DB-touching
  helpers wait for the integration suite (Phase 3+).

* Hoist _datasets_used_by_chart(slice_id) out of the inner attachment-
  window loop in _dashboard_related_scope. Previously a chart with N
  attachment windows fired N queries for identical data; now once per
  slice. Removes a perf cliff at the historical-rich case the spec
  explicitly calls out (US1 AS-3).

* Delete the unused requesting_user parameter from
  _filter_records_by_visibility and from get_activity. It was never
  threaded through to the security manager calls (which read the user
  from Flask-Login implicitly) — speculative future-shape that the
  reader had to mentally trace through. If CLI/Celery bypass becomes
  necessary, add it then with a real call site.

* Delete the unused module-level logger. No observability lands here
  until T037/T038; reintroduce when those tasks add real logger calls.

* Fix the file docstring — it claimed _resolve_version_tables was
  reused; it isn't. Trimmed to (find_active_by_uuid,
  derive_version_uuid).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
fa54708276 feat(activity-view): Phase 2 orchestrator + get_activity (T011-T013)
Completes Phase 2 of sc-107283. The activity-view query layer now has
one public entry point — get_activity(model_cls, entity_uuid, ...) —
plus the supporting visibility filter and DTO decorator.

* T011 _filter_records_by_visibility — silent permission filter per
  AV-008. Batch-resolves live entities per kind, dispatches
  can_access_dashboard / can_access_chart / can_access_datasource.
  Tombstoned entities pass through (the decorator handles deleted
  messaging; no navigable entity_uuid is exposed). _can_read helper
  isolates the per-kind predicate dispatch.

* T012 _decorate_records — synthesizes the full ActivityRecord shape:
  entity_kind (API form), entity_uuid (live lookup; null for
  tombstones), entity_deleted, entity_deletion_state, source ("self"
  vs "related"), summary (AV-012 headline), impact (via T010),
  version_uuid (derive_version_uuid), changed_by (User projection).
  Strips internal-only columns (entity_id, sequence, raw user cols)
  from the final shape.

* T013 get_activity — top-level orchestrator. Branches by include
  ("self"/"related"/"all") via _resolve_scope; for related,
  _dashboard_related_scope walks charts-on-dashboard then datasets-
  used-by-chart and clips chart-on-dataset windows by attachment
  windows. _chart_related_scope skips the dashboard hop. Datasets
  return empty related scope per AV-004. _merge_entity_windows
  collapses repeated (kind, id) entries to keep the OR-clause in
  _fetch_change_records compact. Pagination is post-visibility-filter
  per AV-008; defaults page_size=25, clamps to 200.

Tasks T004-T014 in tasks.md updated to [X]. The next phase opens user
story implementation (T015 dashboard scope helper + T016 endpoint).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
5750cd4fef feat(activity-view): Phase 2 primitives (T004-T010, T014)
Adds the foundational query layer for the cross-entity activity view
in superset/versioning/activity.py. Pure helpers + single-purpose
shadow-table queries; no public entry point yet (T011-T013 follow).

* T004 _resolve_path_entity — UUID lookup via find_active_by_uuid;
  typed 404 dispatch by model class (DashboardNotFoundError /
  ChartNotFoundError / DatasetNotFoundError).
* T005 _charts_attached_to_dashboard — (slice_id, window) tuples from
  dashboard_slices_version, op=2 rows excluded.
* T006 _datasets_used_by_chart — (datasource_id, window) tuples from
  slices_version, filtered to datasource_type='table'.
* T007 _intersect_windows — pure half-open window intersection;
  end=None acts as positive infinity. Unit-verified.
* T008 _fetch_change_records — workhorse query joining version_changes
  + version_transaction + ab_user, OR-clause over (kind, id, [windows]),
  ordered by (issued_at DESC, transaction_id DESC, sequence DESC) per
  AV-006. No DB-level pagination here — AV-008's silent permission
  filter means count is computed after the visibility filter, so
  pagination happens in T013.
* T009 _denormalize_entity_names — batch shadow lookup keyed by
  (api_kind, entity_id, transaction_id); validity-strategy predicate.
  Helper _resolve_names_for_kind extracted to keep cyclomatic
  complexity under ruff's C901 threshold.
* T010 _compute_impact — dashboard+dataset path yields
  {"charts": N}; all other path/related shapes yield None per
  data-model.md §"impact computation".
* T014 _check_entity_tombstones — live-row existence + soft-delete
  state probe; tolerates the pre-sc-103157 absence of a deleted_at
  column via hasattr-style introspection on Table.c.

Kind translation isolated at the module boundary: version_changes
stores lowercase ("chart"/"dashboard"/"dataset"); the ActivityRecord
DTO returns class names ("Slice"/"Dashboard"/"SqlaTable"). The
_TABLE_KIND_TO_API / _API_KIND_TO_TABLE mappings translate at the
two crossings.

Phase 2 orchestration layer (T011-T013) lands in the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
47b804a7dd refactor(activity-view): enforce schema contracts (review polish)
Apply clean-code review polish on top of the Phase 1 scaffold before
Phase 2 builds on this surface:

* Replace magic-string field descriptions with ``validate.OneOf`` on
  ``entity_kind``, ``source``, ``entity_deletion_state``, and ``kind``.
  Canonical lists hoisted to module-level tuples (``ACTIVITY_*``).
* Type ``path`` as ``fields.List(fields.String())`` and ``impact`` as
  a new ``ActivityImpactSchema`` — replaces two ``fields.Raw`` ``Any``s
  whose shape was previously documented only in prose.
* Resolve the ``ActivityChangedBySchema`` docstring contradiction —
  the schema deliberately omits ``username`` so it is no longer
  honest to call it "identical shape to VersionChangedBySchema".
* Fix the ``activity.py`` module docstring: one polymorphic
  ``get_activity`` function, not three entry points.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
8f1ec17150 feat(activity-view): scaffold Phase 1 setup (T001/T002/T003)
* T001 — new ``superset/versioning/activity.py`` module with ASF
  header and docstring describing the cross-entity activity-view query
  layer (companion to ``queries.py``). Empty otherwise; Phase 2 fills
  it with the shared query helpers.
* T002 — empty test file
  ``tests/integration_tests/versioning/activity_view_tests.py`` with
  ASF header, module docstring naming the three test classes that will
  follow, and a ``SupersetTestCase`` import. Phase 3+ fills it with
  test classes.
* T003 — three Marshmallow schemas added to
  ``superset/versioning/schemas.py``:
  * ``ActivityChangedBySchema`` — User subset on each record.
  * ``ActivityRecordSchema`` — one change record (per data-model.md).
  * ``ActivityResponseSchema`` — envelope ``{result, count}``.
  Fields mirror data-model.md §"``ActivityRecord`` DTO" line-for-line;
  metadata descriptions are FAB/Swagger-ready.

No new endpoints, no listeners, no DB writes — pure scaffolding. The
three schemas are unused until Phase 3 (T016) wires the first endpoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:24 -06:00
Mike Bridge
d333510e83 refactor(versioning): consolidate three iterative migrations into one
Squashes the three migrations that captured the versioning schema as
the feature was developed:

* ``56cd24c07170 add_entity_version_history_tables`` — version_transaction
  + dashboards_version / slices_version / tables_version parent shadows.
* ``e1f3c5a7b9d0 add_version_changes_table`` — the field-level diff log.
* ``f7a2b3c4d5e6 spike_add_child_continuum_shadow_tables`` —
  table_columns_version / sql_metrics_version / dashboard_slices_version
  child shadows.

Combined into a single migration ``2026-05-28_19-50_56cd24c07170_
add_versioning_tables.py`` that creates the same 8 tables in dependency
order (sequence + version_transaction first, parent shadows, version_
changes, child shadows last) and drops them in reverse on downgrade.

Why:
* All three migrations represent one logical feature ("enable
  versioning") — applying any subset leaves a broken intermediate
  state.
* The third migration was literally named "spike_..."; the iterative
  shape was a development artifact, not a design intent.
* Downstream operators get one migration to apply / reverse and one
  file to review.
* The dev DB was rolled back through the three downgrades, the new
  migration was applied via ``superset db upgrade``, and column-count
  spot checks confirmed byte-identical schema (4/9/21/26/35/19/15/5).

The ``revision`` hash is reused from the original first migration
(``56cd24c07170``) for chain continuity. ``down_revision`` stays
``2bee73611e32`` (sc-105349 composite-PK tip), unchanged from the
original chain's head.

Net: -70 lines (605 deleted, 535 added), same schema, one Alembic step
instead of three.
2026-05-28 15:37:20 -06:00
Mike Bridge
b16ac7c21e refactor(versioning): leaf-level diff for JSON-blob fields (Shape B)
The diff engine now recurses into nested dicts inside JSON-blob fields
(``json_metadata``, ``Slice.params`` non-list sub-keys, layout node
``meta``) via a shared ``_recursive_leaf_diff`` helper, emitting one
``ChangeRecord`` per atomic leaf change rather than one record carrying
the whole top-level sub-tree on both sides.

Concretely, a dashboard header text change from "VERSION 2!" to
"HEADER!" used to emit a single record at path ``["edit", "header",
"HEADER-id-1"]`` with the entire layout node duplicated on both sides
of the diff. It now emits a single record at path ``["edit", "header",
"HEADER-id-1", "text"]`` with just the changed string as ``from_value``
/ ``to_value``. Multi-leaf edits produce one record per leaf.

Each call site passes its own depth cap via a named constant:

* ``_LAYOUT_META_DIFF_DEPTH = 3`` — layout meta is presentation state
  (text, sizes, colors), shallow by nature.
* ``_JSON_METADATA_DIFF_DEPTH = 6`` — native filter configuration can
  go five levels deep (``defaultDataMask.filterState.value``).
* ``_SLICE_PARAMS_DIFF_DEPTH = 6`` — adhoc filter sub-queries and pivot
  options can be similarly deep.

A cap is a usefulness bound (granularity that's meaningful in a
timeline), not a safety bound. Cap-on-dict-vs-dict emits a debug log so
production tuning can see when a cap is too tight for typical data.

Lists are treated as opaque leaves: positional paths break under
reorder. Lists with stable identity (adhoc filters, metrics, dataset
columns) already have natural-key walkers (``_diff_list_by_natural_key``)
that emit per-element records with the right identity; those are
unchanged.

Backward compatibility:
* Top-level scalar fields, child-collection records (column/metric),
  M2M slice membership, and layout add/remove/move records all keep
  their existing path/from/to shape. The change is scoped to JSON-blob
  recursion.
* All 56 existing diff unit tests pass without modification. 13 new
  tests cover the Shape B behaviour (helper directly, type mismatches,
  opaque list policy, depth cap, nested ``json_metadata``, layout
  edit, ``Slice.params`` deep unknown key, cross-node aggregation).
* Existing integration tests in ``change_records_tests.py`` assert on
  scalar-field changes only — unaffected.

Restoration is unaffected: the restore path reads from Continuum
shadow tables, not from ``version_changes``. This change is purely
about audit-log granularity.

Spec coverage: tasks.md T047d (added in spec repo). Rationale: plan.md
"Key Design Decision: Leaf-level recursive diff for JSON-blob fields
(Shape B)". Conventions: data-model.md "Leaf-level recursion".
2026-05-28 15:37:20 -06:00
Mike Bridge
863c73ec53 test(versioning): lock in flag_modified-suppresses-onupdate invariant (W2)
The ``_pin_audit_columns`` helper added in 1b0083bc03 relies on a
SA-version-dependent behavior: calling ``attributes.flag_modified(parent,
"changed_by_fk")`` causes the next UPDATE statement to use the
in-memory value rather than invoking the column's ``onupdate=callable``.
This is the mechanism that prevents a stale ``g.user.id`` from being
written into the parent's ``changed_by_fk`` when the synthetic
flag-flush triggers an UPDATE during an autoflush at a time when the
test user has already been deleted from ``ab_user`` — the cascade that
broke CI in sc-103156 T062.

Four unit tests, no app context required (uses a minimal in-memory
SQLite engine + declarative_base):

* ``test_flag_modified_suppresses_onupdate_callable``: the positive
  invariant — the in-memory value lands in the DB and ``onupdate`` is
  NOT called. If SA ever changes this semantic, the test fails.
* ``test_onupdate_does_fire_without_flag_modified``: negative sanity
  check — confirms the test setup is realistic. Without
  ``flag_modified``, ``onupdate`` fires as expected.
* ``test_pin_audit_columns_skips_missing_attribute``: the helper
  tolerates parents without audit columns (e.g., non-AuditMixin
  models). No raise.
* ``test_pin_audit_columns_tolerates_invalid_request_error``: SA
  raises ``InvalidRequestError`` from ``flag_modified`` when the
  attribute is unloaded in instance state (the freshly-constructed
  ``session.new`` case). The helper must catch and skip — this is the
  production-path safety net mentioned in the docstring.

Uses ``expire_on_commit=False`` for the positive invariant test so the
column stays loaded; the production-path case where the attribute is
expired is the ``InvalidRequestError`` branch covered separately.

Closes review item W2 from the sqlalchemy-review pass on
sc-103156-versioning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
9c9cb3a5f0 test(models): lock in UUIDMixin._coerce_uuid contract (S3)
Four unit tests pin the behavior the validator added in 46f138326b:

* Valid UUID-format strings get coerced to ``uuid.UUID`` instances —
  the primary contract that makes importers, test fixtures, and ad-hoc
  ``Model(uuid=...)`` construction see a consistent type.
* Already-``UUID`` values pass through unchanged (the validator must
  not re-wrap; ``is`` identity check pins this).
* Non-UUID-shaped strings pass through unchanged. This keeps test
  mocks that use placeholder strings (e.g.
  ``tests/unit_tests/mcp_service/dashboard/test_dashboard_schemas.py::test_dashboard_schema``
  with ``"dashboard-uuid-7"``) working. Without this contract, those
  tests would break under the validator and need to migrate to
  ``uuid.uuid4()``. If the contract is ever tightened to raise, this
  test catches the regression and signals the migration cost.
* ``None`` passes through unchanged.

Closes review item S3 from the sqlalchemy-review pass on
sc-103156-versioning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
1d78dd1ad0 refactor(versioning): thread pre-resolved entity through list/get_version (W3)
``ChartRestApi.list_versions`` (and the dashboard / dataset siblings)
resolves the entity via ``VersionDAO.find_active_by_uuid`` to run the
``raise_for_ownership`` check from T056, then calls
``VersionDAO.list_versions(model_cls, entity_uuid)`` — which internally
runs the SAME ``find_active_by_uuid`` query a second time. The lookup
filters on ``uuid`` (a unique non-PK column), so it isn't served from
SQLAlchemy's identity map — every call is a real SELECT.

Add an optional ``entity=`` keyword to ``list_versions``,
``get_version``, and ``resolve_version_uuid`` so callers that have
already resolved the path entity can skip the redundant lookup. The
public signature stays backward-compatible (kwarg-only).

Threading ``entity`` through ``get_version`` also avoids a third
redundant lookup inside the ``resolve_version_uuid`` call it makes.

Net: 6 fewer SELECT queries on the ``/versions/`` and
``/versions/<uuid>/`` hot paths (1 per call × 6 endpoints). All
``find_active_by_uuid`` callsites in the API layer were already
present from T056; this commit just plumbs them through.

Closes review item W3 from the sqlalchemy-review pass on
sc-103156-versioning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
fb78c8670d test(versioning): defensive setUp/tearDown for TestDatasetRestoreApi
Reset session state (``rollback`` + ``expire_all``) before and after each
test in this class. Defensive hygiene against the Postgres-only multi-
test cascade documented in spec task T062 — a previous test in the
full-suite ordering can leave Continuum's shadow-table session attributes
in a state where the restore command's ``@transaction`` boundary raises
mid-flush and the API returns 422 "Dataset could not be updated."

Note: this *doesn't* fix the cascade itself (the polluter is writing to
the DB, not just the session; reproducer remains 3 failures on Postgres
full-suite ordering). The root cause is queued as T062. This change just
guarantees that whatever state this class' tests inherit, they start
from a clean session view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
4a2ab1704d test(versioning): convert version-history test cleanup to try/finally
The version-history integration tests follow the codebase's dominant
pattern: a trailing ``# Cleanup`` block at the end of each test that
restores the fixture entity's mutated scalar fields (``description``,
``slice_name``, ``dashboard_title``). The pattern is brittle — when any
intermediate assertion fails, the cleanup is never reached and the
fixture stays polluted with the test's last edit, cascading state into
later tests in the same suite run.

This bit hard in CI's full-suite Postgres ordering: a failure in
``test_restore_applies_scalar_field`` left ``description="restore-test
v2"`` on the birth_names dataset, polluting downstream tests including
the RLS virtual-dataset cascade. The same shape is latent in the chart
and dashboard files.

Convert every mutating test in the three version-history files to wrap
the mutation block in ``try:`` and the restoration in ``finally:``,
with a ``db.session.rollback()`` at the top of the finally to clear any
half-flushed state left by a 422 response. Re-query the entity by id
inside the finally so the restore writes against a live object rather
than a stale handle the failing path may have expired or detached.

Pure hygiene improvement — doesn't fix the underlying Postgres failure
chain (still triggered by ``model_tests.py``'s interaction with later
tests in the full-suite ordering), but prevents this set of tests from
amplifying any future cascade.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
3314c2cd4b fix(versioning): align uuid-as-string assertions with UUIDMixin coercion
The ``@validates("uuid")`` coercion added in 46f138326b now ensures
``UUIDMixin.uuid`` always returns a ``uuid.UUID`` instance after
assignment, regardless of input type. This is the right semantic
(matches what ``UUIDType`` would emit on a fresh DB read) but it
broke three pre-existing tests that compared the attribute against
string literals — those assertions only passed before because the
post-INSERT refresh path either didn't run or ran inconsistently for
the model under test.

Update those three assertions to compare ``UUID`` to ``UUID`` so the
test contract matches the now-consistent attribute shape:

* ``test_import_database`` and ``test_import_database_no_creds`` —
  ``database.uuid`` now matches the coerced ``UUID`` literal.
* ``test_load_parquet_table_sets_uuid_on_new_table`` — ``tbl.uuid``
  same treatment.

Also harden the validator to pass non-UUID-shaped strings through
unchanged (instead of raising), so test mocks that use placeholder
strings like ``"dashboard-uuid-7"`` keep working — the SQL bind layer
will surface a clearer error if such a value is ever written to the
DB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
279d6a4484 fix(versioning): coerce string uuid values in UUIDMixin.uuid setter
``sqlalchemy_utils.UUIDType`` only coerces in ``process_bind_param`` /
``process_result_value`` — string-to-UUID conversion happens on SQL
write / read, not on Python attribute assignment. Importers and ad-hoc
construction (e.g., ``SqlMetric(uuid="00000000-...")`` in the
``test_import_dataset`` unit test) leave the in-memory attribute as a
``str`` until the next DB round-trip refreshes it.

For non-versioned mappers a post-INSERT attribute expiration usually
refreshes the value before the caller reads it. With SQLAlchemy-
Continuum versioning on a child mapper (``TableColumn``, ``SqlMetric``),
the expire-on-INSERT behaviour shifts enough that the assertion
``metric.uuid == uuid.UUID(...)`` fails with a string-vs-UUID inequality.
Bisection confirmed: removing ``__versioned__`` from either child
restores the implicit coercion path; adding it reproduces the failure.

Rather than chase the Continuum/SA flush interaction, add a defensive
``@validates("uuid")`` to ``UUIDMixin`` so the attribute is always a
``UUID`` at runtime regardless of where the value came from. This
aligns the in-memory shape with what ``UUIDType`` would emit on a
fresh DB read, and the coercion is the same one the SQL layer would
apply on the next bind anyway.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
2685b15d5c fix(versioning): pin audit columns when force-flagging parent dirty
``_force_parent_dirty_on_child_change`` calls ``flag_modified`` on a
versioned parent so Continuum records a parent shadow row whenever a
versioned child changes. The resulting UPDATE on ``tables`` /
``dashboards`` / ``slices`` inherits ``AuditMixin``'s
``onupdate=get_user_id`` on ``changed_by_fk`` and ``onupdate=datetime.now``
on ``changed_on`` — SQLAlchemy calls ``get_user_id()`` at flush time
and stamps whoever ``g.user`` resolves to.

When the flush is autoflush-triggered during a *previous* test's
teardown (or the fixture cleanup between two tests), ``g.user`` can
still point at a user that's already been deleted from ``ab_user``.
The parent UPDATE then fails the FK to ``ab_user``, surfacing as
``IntegrityError: FOREIGN KEY constraint failed`` /
``ForeignKeyViolation``. In CI's full-suite ordering this fires from
two distinct places — ``test_rls_filter_alters_no_role_user_birth_names_query``
teardown and ``TestDatasetRestoreApi::test_restore_applies_scalar_field``
mid-restore (the latter returning 422 "Dataset could not be updated.").

Fix: also ``flag_modified`` ``changed_by_fk`` and ``changed_on`` on
the parent. The columns now have dirty attribute history, so
SQLAlchemy uses the in-memory (previously-committed, valid) values
instead of invoking ``onupdate``. The synthetic parent UPDATE carries
the existing audit values; the FK violation goes away. The behaviour
on a real user-driven save is unchanged — those code paths go through
the normal write path with a live ``g.user`` and the audit columns
update as before.

Extracted ``_pin_audit_columns`` helper to keep the caller under
ruff's ``C901`` complexity ceiling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:20 -06:00
Mike Bridge
eba4363eeb chore(versioning): ruff-format fix on dataset restore-test target filter
CI's ruff-format would collapse the two-clause if expression onto a
single line — local ruff at v0.9.7 reformats the same way. Pure
formatting, no behavior change. Counterpart of ``6de8811873``.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
3138526816 test(versioning): skip DELETE rows when picking restore target in dataset test
Same fix as ``7caf9862be`` for the dashboard restore test. CI's
integration DB accumulates Continuum shadow rows across fixture teardown
cycles, and ``TestDatasetRestoreApi::test_restore_applies_scalar_field``
picks the first row whose ``description`` matches the original. In CI
that match can land on an ``operation_type=2`` (DELETE) row from a prior
test run; Continuum's ``revert()`` then restores the dataset to its
deleted state and the restore API returns 422 "Dataset could not be
updated."

Filter ``operation_type != 2`` so the test only ever targets INSERT or
UPDATE shadow rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
b7c6181e38 feat(versioning): row-level ownership on list_versions and get_version (T056)
The SIP's *Authorization layering* commitment promises that all nine
version endpoints enforce both model-level access (``@protect()``) AND
row-level access (``security_manager.raise_for_ownership(entity)``).
Restore endpoints already do this via ``BaseRestoreVersionCommand.validate``,
but ``list_versions`` and ``get_version`` (six endpoints across chart,
dashboard, and dataset) skipped the row-level check — a non-owning user
with ``can_write`` on the model could list or read the shadow rows of
any entity they could reach by UUID.

Each affected handler now looks up the entity via
``VersionDAO.find_active_by_uuid`` (which already runs inside the DAO),
returns 404 if it's gone, then calls ``raise_for_ownership`` and maps a
``SupersetSecurityException`` to 403. Admins bypass automatically via
the existing FAB mechanism. The redundant lookup hits the session
identity map on the second call, so the cost is one PK read.

Tests:

- Add ``test_list_versions_denies_non_owner`` (chart, dashboard, dataset)
  and ``test_restore_denies_non_owner`` (chart) — Alpha has ``can_write``
  on each model but isn't an owner of the admin-owned fixture, so the
  row-level branch is the only thing that can reject. The dataset row-
  level test is alongside the existing model-level Gamma test, which
  only exercises the ``@protect()`` denial path.
- Unskip the three placeholder tests on chart and dashboard that were
  blocked waiting for this gap to close (``@pytest.mark.skip(reason="...
  no built-in no-write user to exercise the 403 branch...")``).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
442c87f30f temp(versioning): update DashboardList snapshot for added `uuid` column
Pairs with ``25f1922b2d temp(versioning): demo version-history dropdowns``,
which adds ``uuid`` to the dashboard-list page's ``select_columns`` so the
per-row VersionHistoryDropdown can call ``/api/v1/dashboard/<uuid>/versions/``.
``DashboardList.test.tsx`` asserts the list-fetch URL via
``toMatchInlineSnapshot``, so the snapshot needs the new column. Revert
this entry along with the dropdown component (same lifecycle as the temp
parent commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
f730e968b9 test(versioning): skip DELETE rows when picking restore target in dashboard test
The integration SQLite DB accumulates Continuum shadow rows across
fixture teardown cycles. ``TestDashboardRestoreApi::test_restore_applies_scalar_field``
picks the first version row whose snapshot matches the original title,
and that match can land on an ``operation_type=2`` (DELETE) row left
over from a prior test run. Continuum's ``revert()`` then dutifully
restores the entity to its deleted state — the dashboard row is gone
after the restore returns 200, and the post-restore re-query fails
with ``NoResultFound``.

Filter ``operation_type != 2`` so the test only ever targets
INSERT or UPDATE shadow rows, which is also what a real user would
pick from the version-history dropdown.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
b449889e69 fix(versioning): narrow phantom-dirty filter to session.dirty only
Previous fix (9c2391d1b4) gated the force-parent-dirty hook on
is_modified(child) for ALL session collections (dirty/new/deleted).
That was over-restrictive: is_modified checks attribute history,
and deletion is a state transition with no attribute history —
so deleted children evaluated as not-modified and the parent
wasn't flagged. The change-records listener then didn't see the
deletion and no removal record was emitted.

Symptom: test_restore_emits_full_child_diff_in_one_transaction
failed expecting a column-removed change record after a restore
that removed the column; instead only the parent's scalar fields
appeared in observed paths.

Refine: apply the is_modified filter ONLY to persistent rows in
session.dirty. session.new (creation) and session.deleted
(removal) are always real content changes by virtue of their
session-collection membership — no is_modified check needed (and
in deletion's case, the check returns the wrong answer).
2026-05-28 15:37:19 -06:00
Mike Bridge
ed4c1792ad chore(versioning): import-sort autofix on daos/version.py (I001)
Pre-commit (previous) flagged I001 unsorted-imports on the
backward-compat façade. Two queries imports merged into one
block (the aliased ``derive_version_uuid as _derive_version_uuid``
moves inline rather than living in its own block), and the
restore-side names sorted: ``_RESTORE_RELATIONS``,
``_stamp_audit_fields_for_restore``, ``restore_version``.

Pure mechanical reformatting; no behaviour change.
2026-05-28 15:37:19 -06:00
Mike Bridge
2c080ab6a5 fix(versioning): force-parent-dirty only when child has real column changes
_force_parent_dirty_on_child_change was firing whenever ANY
TableColumn or SqlMetric of the parent appeared in
session.dirty / new / deleted — even when the child was there for
non-content reasons:

- Lazy-load side effects when a relationship is touched
- M2M relationship-cascade artifacts (e.g. RLS setUp doing
  rls_entry.tables.extend([dataset]) triggers cascade behavior
  that pulls children into the session)
- AuditMixin auto-bumps from earlier code paths
- Reverter side passes during restore

Force-touching the parent in those cases produced an incidental
UPDATE tables SET description=…, changed_on=…, changed_by_fk=…
whose changed_by_fk value or autoflush ordering tripped FK
integrity on some dialects. Symptoms:

- test_rls_filter_alters_no_role_user_birth_names_query → FK
  IntegrityError on autoflush during a query
- test_restore_applies_scalar_field → 422 "Dataset could not be
  updated" during restore

Fix: gate on Continuum's is_modified(child), which returns True
only when a non-excluded versioned column on the child has
SQLAlchemy attribute-history changes. New objects (session.new)
and genuinely-modified rows still flag the parent; phantom-dirty
rows do not.

The intended hook semantics — "child edit forces a parent shadow
row" — are preserved: a column-description edit through the
dataset API still triggers is_modified True, still flags the
parent. See test_dataset_column_edit_creates_parent_version.
2026-05-28 15:37:19 -06:00
Mike Bridge
9ff18b3e48 fix(versioning): flag `description instead of uuid` in force-parent-dirty
flag_modified(parent, "uuid") was producing FK integrity failures via
the column's BLOB/BINARY round-trip: SQLAlchemy logs the param as
``<memory at 0x…>`` and the UUID round-trip doesn't always match the
in-memory value byte-for-byte. Symptom: in scenarios where the parent
is already going to flush (Reverter applying historical state during
restore, RLS test triggering autoflush during a query), our added
``uuid`` UPDATE column tripped the FK check.

Pick ``description`` instead — plain Text column on all three
versioned parent classes (Dashboard, Slice, SqlaTable), no
TypeDecorator, no marshaling layer. Flagging it round-trips its
current value safely. Fallback chain ``description → uuid → col_keys[0]``
keeps the original deterministic-pick property for forks/subclasses
that excluded ``description``.

Should unblock test_restore_applies_scalar_field and the
test_rls_filter_alters_no_role_user_birth_names_query autoflush
error.
2026-05-28 15:37:19 -06:00
Mike Bridge
2731b099f1 chore(versioning): pre-commit autofixes (ruff 0.9.7) — round 2
- factory.py: TID251 banned ``import json``; switch to
  ``from superset.utils import json`` (project convention).
- factory.py: ruff format reflow on _matches_previous_version.
- version_restore.py: ruff format collapse on restore_version call.

CI was pinning a different ruff version than my local uvx default;
re-ran against ruff==0.9.7 (the version in requirements/development.txt)
which surfaced these.
2026-05-28 15:37:19 -06:00
Mike Bridge
771dd4bcd9 fix(versioning): replace session-state guard with InvalidRequestError catch
The previous attempt (d0520f6766) was too aggressive: skipping when
parent is in session.dirty/new/deleted bypassed the
persistent-and-clean case the hook EXISTS for. Some upstream code
paths put the dataset in session.dirty *before* this listener fires
(API controllers touching audit fields, etc.), so the
session-membership pre-check made us silently no-op on the very
scenario the hook needs to handle. CI symptom:
test_dataset_column_edit_creates_parent_version showed before=317,
after=317 (parent shadow not written).

Restore the unconditional flag_modified and catch the specific
InvalidRequestError that fires only for the session.new case
(uuid default callable hasn't populated state yet). Other states
fall through to the original behavior:
- persistent + clean → flag_modified succeeds, parent goes dirty,
  Continuum picks it up, SkipUnmodifiedPlugin keeps the row via
  _has_dirty_versioned_children. ✓
- persistent + dirty → flag_modified is harmless (already dirty).
- session.new → InvalidRequestError, skip (parent INSERTs anyway).
- session.deleted → flag_modified may or may not raise; if it does,
  we skip; if not, the delete dominates.

Should unblock test_dataset_column_edit_creates_parent_version,
test_get_version_returns_historical_snapshot_with_children, and
test_restore_with_column_edits_reverts_columns.
2026-05-28 15:37:19 -06:00
Mike Bridge
4af88e65c0 chore(versioning): apply pre-commit autofixes (ruff + auto-walrus)
- ruff: import sort + E501 reflow on the parent-state guard in
  baseline.py
- ruff format: function-signature collapse and join-chain reflow in
  queries.py
- auto-walrus: two ``entity_kind = …; if … is not None:`` patterns
  in queries.py converted to assignment-expressions
2026-05-28 15:37:19 -06:00
Mike Bridge
560567b64c fix(versioning): retention task FK violation on cross-entity transactions
When one ORM flush touches multiple versioned entities (dashboard +
slice + dataset all save at tx=X), each gets a shadow row sharing
that tx. If only the dashboard is later edited at tx=Y, the
dashboard row at tx=X is closed (end_tx=Y) while slice/dataset rows
stay live at tx=X. Retention then preserves tx=X (slice/dataset are
live there) and prunes tx=Y. The dashboard's closed row at tx=X
survives step 1, then its end_transaction_id=Y trips the FK when
step 2 deletes version_transaction row Y.

Fix: extend the shadow-row delete to also match end_transaction_id
IN tx_ids. Live rows have end_tx=NULL so they're never matched by
either predicate. Closed rows that touch a pruned tx at either
endpoint are pruned together — consistent with retention semantics
(any tx in the row's lifespan is gone, so the row's chain is broken
anyway).

Unblocks test_retention_prunes_old_rows on sqlite, mysql, postgres.
2026-05-28 15:37:19 -06:00
Mike Bridge
cbb4e8c788 fix(versioning): skip non-clean parents in force-parent-dirty hook
The force-parent-dirty listener was calling attributes.flag_modified
on every parent reachable from a dirty child — including parents
themselves in session.new (e.g. brand-new SqlaTable + brand-new
TableColumns from POST /api/v1/dataset/). flag_modified rejects
unloaded attributes, and a session.new SqlaTable's uuid (default=uuid4
fires at flush time) is unloaded until then. CI caught this with
InvalidRequestError cascading into 422s across dataset creation /
upload / Playwright dataset specs.

The hook is only needed for the persistent-and-clean case (child
edited, parent's own scalars untouched, dropdown otherwise empty).
Anything in session.new will flush anyway; anything in session.dirty
is already flagged; session.deleted shouldn't be touched. Short-
circuit before the flag_modified call.

Unblocks test-sqlite, test-mysql, test-postgres (previous), and
playwright dataset specs.
2026-05-28 15:37:19 -06:00
Mike Bridge
43abeb4447 refactor(versioning): apply cross-PR review feedback (#39977 H1/M3/M5)
Three small follow-ups surfaced by aminghadersohi's review of the
SoftDeleteMixin PR (#39977) that apply equally here:

- H1: cache _child_to_parent_registry() with functools.cache. Called
  twice per save flush; mapping depends only on import-time model
  classes, so unbounded cache is the right shape (no invalidation).
- M5: tighten _CHILD_BASELINE_HANDLERS type from dict[str, Any] to
  dict[str, Callable[[Session, Any, int], None]] via a named alias.
  Mypy now catches a future broken handler signature.
- M3/M4: explain the inline-import pattern once in the module
  docstrings of baseline.py and changes.py. Both modules use
  pylint disable=import-outside-toplevel uniformly because they
  load during init_versioning() before mappers are configured;
  the per-callsite "why" comments would just repeat the same
  reason. Module-level explanation + a hint to comment unusual
  cases is the cleaner shape.

M6 (listener placement) doesn't apply — init_versioning() already
runs inside init_app_in_ctx(). M8 (loose OpenAPI schema in
*/api.py docstrings) is real but its own change.
2026-05-28 15:37:19 -06:00
Mike Bridge
6e1b7896e7 docs(versioning): record why SkipUnmodifiedPlugin doesn't clean up orphan version_transaction rows inline
Extends the existing docstring note ("the orphan is swept by retention")
with the reasoning behind not cleaning it up in the same flush. The
inline-delete is appealing in principle but would couple this plugin
to the change-records listener's buffer state via the ON DELETE
CASCADE on ``version_changes.transaction_id``: both listeners would
have to agree that the flush produced nothing before the version_transaction
row could be dropped safely. The orphan's ~40-byte storage cost +
retention's correct-by-construction handling (orphans have no parent
shadow, so they're never in the "preserve" set) make the coordination
overhead not worth it.

Captures the design decision in the file where the next reader will
look for it.
2026-05-28 15:37:19 -06:00
Mike Bridge
18d134605d tidy(versioning): reading-order shuffle in baseline.py (newspaper-article order)
Pure file shuffle, zero behaviour change. Reorders ``baseline.py`` so it
reads top-down by level of abstraction (newspaper-article rule): the
public entry point at the top, supporting helpers descending below.

Before: 14 private helpers, then ``register_baseline_listener`` at the
bottom. A reader opening the file met the leaf builders first and had
to accumulate context before finding the call site.

After (top-down):

  - Entry point: ``register_baseline_listener`` + inner ``capture_baseline``
  - High-level helpers used by ``capture_baseline``:
      ``_force_parent_dirty_on_child_change``,
      ``_collect_parents_to_baseline``,
      ``_child_to_parent_registry``,
      ``_version_table_for``,
      ``_shadow_row_count``,
      ``_insert_baseline_and_children``
  - Mid-level builders:
      ``_insert_baseline_row``,
      ``_baseline_children_for_parent``
  - Per-entity child handlers + their dispatch table:
      ``_baseline_dataset_children``,
      ``_baseline_dashboard_children``,
      ``_CHILD_BASELINE_HANDLERS``
  - Leaf builders:
      ``_insert_child_baseline_rows``,
      ``_baseline_attached_slices``,
      ``_insert_synthetic_slice_baseline``

Three section-divider comments mark the abstraction levels. The
``_CHILD_BASELINE_HANDLERS`` dict literal stays after its referenced
handlers (module-level literals evaluate at import time and need names
already bound); a comment now flags this constraint.

Function bodies are byte-for-byte unchanged; ``git log -L`` on any
function shows only its relocation. 96 unit tests pass.
2026-05-28 15:37:19 -06:00
Mike Bridge
6fcd762cd3 tidy(versioning): extract read_row_outside_flush helper
baseline.py:_insert_baseline_row and changes.py:_read_pre_state both
issued the same "read a single row through ``session.connection()``
inside ``with session.no_autoflush:``" pattern. Same five-line block,
same intent ("read the pre-flush state without triggering the in-flight
edit's flush").

Promoted to ``superset.versioning.utils.read_row_outside_flush(session,
table, entity_id)``. Companion to ``single_flush_scope`` — they sit
next to each other in utils.py and frame the two directions of the
"don't autoflush mid-listener" pattern.

Returns ``dict[str, Any]`` (or ``None``) so callers can't accidentally
hold a cursor-bound ``RowMapping`` past the listener boundary. Both
call sites get shorter by ~5 lines.

Also picks up Decimal stringification in the changes.py docstring
update (was listed in the W4 commit but the docstring still said
"(datetime, UUID, bytes)" — now matches the implementation).

Behaviour unchanged. 96 unit tests pass.
2026-05-28 15:37:19 -06:00
Mike Bridge
6fa48f8bc2 tidy(versioning): extract shared helpers between list_versions and get_version
After the SRP split (8c9cf36) put both functions in the same module
~150 lines apart, their overlap became visible: same JOIN of
version_table → version_transaction → ab_user, same baseline-first
ordering, same user-row → ``changed_by`` projection, same lookup
``_ENTITY_KIND_BY_CLASS_NAME.get(model_cls.__name__)``. About 30 lines
of duplication.

Five small helpers extracted at the module top:

- ``_resolve_version_tables(model_cls)`` returns ``(ver_tbl, tx_tbl, user_tbl)``
- ``_version_with_tx_user_join(ver_tbl, tx_tbl, user_tbl)`` builds the join
- ``_baseline_first_ordering(ver_tbl)`` returns the order-by tuple
- ``_user_select_cols(user_tbl)`` returns the user-column list with
  ``user_id`` as the stable label (normalises the prior asymmetry
  where ``list_versions`` labelled it ``user_id`` and ``get_version``
  labelled it ``_user_id`` to dodge a column-name collision — the
  ``user_id`` label collides with neither)
- ``_changed_by_from_row(row)`` projects user columns onto the API shape
- ``_entity_kind_for(model_cls)`` resolves the change-records taxonomy lookup

Both call sites get shorter and read what they do (build query / project
user / build row) rather than how. Behavior unchanged; no test changes.

Also two small inline tidyings while in the file:

- Replace the ternary
  ``changes_by_tx = list_change_records_batch(...) if entity_kind else {}``
  with an explicit two-line if-statement in both functions. The ternary
  buries the decision; the if-statement reads as one thought.
- Inline the one-shot ``meta_cols`` set declaration in ``get_version``
  into the ``if col.name in {...}`` check that uses it three lines later.

Net: about 110 lines → about 80 lines across the two functions, plus
a small helper section at the top.
2026-05-28 15:37:19 -06:00
Mike Bridge
5d19a34b2d refactor(versioning): sqlalchemy-review follow-ups (W1–W8)
Cleanup pass from the SQLAlchemy + migration code review. Eight items,
all in the "warnings / suggestions" tier — no behaviour change visible
to the API, but each closes a real correctness, perf, or maintainability
concern surfaced in review.

baseline.py
- Delete unused ``_get_user_id`` (W1). The function wrapped a broad
  ``except Exception:  # noqa: S110`` swallow that hid bugs; grep
  confirmed no callers anywhere. The legitimate audit-field paths
  (``row.get("changed_by_fk")`` etc.) already drive the
  ``version_transaction.user_id`` write.
- Batch ``_baseline_attached_slices`` from O(N) round-trips to
  three queries (W2): one membership SELECT, one existing-shadow
  SELECT, one bulk live-row SELECT for the missing ids. The previous
  per-slice ``COUNT(*)`` + ``SELECT`` was a measurable first-save
  hotspot on dashboards with many charts. Drops the now-unused
  ``_slice_has_shadow`` helper.
- Pick a stable column name for ``flag_modified`` in
  ``_force_parent_dirty_on_child_change`` (W3). ``uuid`` is on all
  three versioned parent classes and excluded by none, so the
  flagged attribute is deterministic across SQLAlchemy versions /
  mapper-config orders instead of depending on
  ``versioned_column_properties(parent)[0]``. Falls back to the
  first available column for forks that exclude ``uuid``.

changes.py
- Add ``Decimal`` handling to ``_jsonable`` (W4) — ``json.dumps``
  rejects ``Decimal``, so any numeric column (e.g. ``SqlMetric.currency``
  contents, or fork/plugin Decimal columns) would crash the bulk
  insert. Stringify rather than ``float()`` to preserve precision;
  the diff engine compares ``from_value`` / ``to_value`` by string
  equality after this coercion so both sides round-trip identically.

queries.py
- Promote the inline ``{0: "baseline", 1: "update", 2: "delete"}``
  dict to module-level ``_OP_TYPE_LABELS`` (W7). The literal was
  duplicated across ``list_versions`` and ``get_version``; the third
  caller is one bug fix away.
- Comment on ``resolve_version_uuid``'s Python-side ``derive_version_uuid``
  loop (W8) — no portable SQL form for UUIDv5 across PostgreSQL /
  MySQL / SQLite, iteration count is bounded by the retention
  window. Flags the place to revisit if retention is ever disabled
  (``=0``) on a heavily-edited entity.

migrations/2026-05-01_23-36 (composite-PK)
- Belt-and-braces guard in ``_downgrade_mysql_table`` (W6): asserts
  ``t.name in AFFECTED_TABLES`` before interpolating into the
  backtick-quoted ALTER statements. The invariant was already
  structurally implied (callers iterate ``AFFECTED_TABLES``), but
  making it load-bearing means a future refactor can't slip an
  arbitrary table name through.

(W5 was verified-no-change: grepped ``tests/`` for ``metadata.create_all``
callers that exercise versioning tables; none. The cascade-FK
gap on ``version_changes.transaction_id`` is already documented
in ``tests/integration_tests/versioning/change_records_tests.py:27-32``.)

62 versioning unit tests pass.
2026-05-28 15:37:19 -06:00
Mike Bridge
00604b92d9 refactor(versioning): split VersionDAO into queries + restore modules
VersionDAO carried five distinct concerns under one class — UUID
derivation, version metadata queries, change-record loading,
single-version snapshot retrieval, and restore orchestration. Bob's
"and" test (the clean-code review flagged this as the next structural
fix after the dead-code purge) gives ~600 lines of "queries about
versioned state of one entity AND the workflow that mutates it."

Splits the read and write sides into purpose-built modules:

- ``superset/versioning/queries.py`` — UUID derivation
  (``VERSION_UUID_NAMESPACE``, ``derive_version_uuid``) + read-side
  helpers (``find_active_by_uuid``, ``current_version_number``,
  ``current_live_transaction_id``, ``current_live_version_uuid``,
  ``list_versions``, ``resolve_version_uuid``, ``get_version``,
  ``list_change_records_batch``). ~475 lines.

- ``superset/versioning/restore.py`` — write-side (``restore_version``,
  ``_stamp_audit_fields_for_restore``, ``_RESTORE_RELATIONS``).
  ~140 lines. Depends only on ``queries.find_active_by_uuid`` and
  ``utils.single_flush_scope``.

- ``superset/daos/version.py`` — collapsed to an ~85-line backward-compat
  façade that re-exports both modules under a single ``VersionDAO``
  class via ``staticmethod`` aliases. The module also re-exports
  ``VERSION_UUID_NAMESPACE`` and ``derive_version_uuid`` at module level
  so the ~10 existing callers (api.py handlers, command classes, the
  ETag emitter, integration tests) don't have to change their imports.
  New code is encouraged to import from the sub-modules directly.

The functions themselves are unchanged byte-for-byte aside from
internal call sites being rewritten from ``VersionDAO.foo`` to the bare
function name (since they now live as module-level functions, not
class methods).

One unit-test mock target moved: ``test_restore_version_returns_none_for_unknown_entity``
now patches ``superset.versioning.restore.find_active_by_uuid`` (the
actual call site) instead of ``VersionDAO.find_active_by_uuid`` (which
is now just an alias).

Each of the three modules now has one reason to change. When the
sc-103157 soft-delete pass adds the ``deleted_at IS NULL`` filter to
``find_active_by_uuid``, it touches only ``queries.py``. When a
per-entity-type restore Strategy replaces the string-keyed
``_RESTORE_RELATIONS`` dispatch, it touches only ``restore.py``.
2026-05-28 15:37:19 -06:00
Mike Bridge
cb13498f6a temp(versioning): strip URL params from dashboard restore navigation; regen lockfile
DashboardList demo dropdown previously instructed the user to "Reload
the page to see the change" after a restore. The URL the user
returns to may still carry ``?native_filters_key=…`` /
``permalink_key`` / ``form_data_key`` from a prior session — those
point at server-cached snapshots (in ``key_value`` and the
filter-state cache) captured before the restore. On rehydration the
cached state is merged on top of the restored ``json_metadata``,
masking the rollback (e.g. dashboard-level colour-scheme restore
appears not to take effect).

Replaces the alert + manual reload with a direct ``window.location.href``
navigation to ``/superset/dashboard/<uuid>/`` — drops all URL params,
forcing hydration from the freshly restored DB state.

Also regenerates ``package-lock.json`` to pick up the ``zod 4.4.1 →
4.4.3`` bump that master's ``package.json`` already reflects.

(``temp(versioning)`` prefix per the demo dropdown's status — this
file is not part of V1 scope per ADR-005; the V2 UI SIP owns the
actual restore UI surface.)
2026-05-28 15:37:19 -06:00
Mike Bridge
15abdbc678 refactor(versioning): rename find_active_by_uuid public + collapse restore commands onto BaseRestoreVersionCommand
Two coupled clean-code review fixes:

(1) Rename ``VersionDAO._find_active_entity_by_uuid`` →
``find_active_by_uuid``. The leading-underscore + three
``# pylint: disable=protected-access`` suppressions in the restore
commands were the smell of a wrongly-private API. The method is a
perfectly reasonable public DAO operation; dropping the underscore
removes the suppressions.

(2) Collapse ``RestoreChartVersionCommand``, ``RestoreDashboardVersionCommand``,
``RestoreDatasetVersionCommand`` onto a shared
``BaseRestoreVersionCommand`` (``superset/commands/version_restore.py``).
The three classes were textbook copy-paste — identical except for
the model class and three exception types. Each subclass now declares
``model_cls`` + ``not_found_exc`` + ``forbidden_exc`` and overrides
``run()`` with one ``@transaction(reraise=<failed_exc>)``-decorated
line delegating to ``self._do_restore()``. ~80 lines per file →
~45 lines per file; one shared workflow instead of three drift sources.

The api.py imports of ``RestoreChartVersionCommand`` /
``RestoreDashboardVersionCommand`` / ``RestoreDatasetVersionCommand`` are
unchanged — public class names preserved.
2026-05-28 15:37:19 -06:00
Mike Bridge
769d1670e2 refactor(versioning): purge dataset_snapshots dead code + fix get_version bug
The full-Continuum spike (ADR-004 revised) replaced the JSON-snapshot
restore path with Continuum's native Reverter and removed the
``dataset_snapshots`` / ``dashboard_snapshots`` tables from the
migration chain. Seven VersionDAO methods and two module-level
helpers that read/wrote those tables stayed in the code anyway and
went unused — dead code that looked live.

Worse, ``VersionDAO.get_version`` still read from
``dataset_snapshots`` in its SqlaTable branch. On any environment
where the snapshot tables don't exist (current production behavior),
``GET /api/v1/dataset/<uuid>/versions/<version_uuid>/`` raised
``OperationalError``. The branch is rewritten to read column and
metric state from Continuum's child shadow tables
(``table_columns_version`` / ``sql_metrics_version``) via the
existing ``_shadow_rows_valid_at`` helper.

Deleted:
- ``_deserialize_snapshot_value`` (module helper)
- ``_coerce_snapshot_list`` (module helper)
- ``RESTORE_EXCLUDE_FIELDS`` (constant — only referenced by deleted code
  and a docstring)
- ``VersionDAO._restore_dataset_children``
- ``VersionDAO._parse_slice_ids_json``
- ``VersionDAO._apply_dashboard_slices``
- ``VersionDAO._restore_dashboard_children``
- ``VersionDAO._apply_snapshot_children``

The corresponding ~17 unit tests in
``tests/unit_tests/daos/test_version_dao.py`` are removed alongside.

Stale docstring references in ``versioning/changes.py`` and
``versioning/diff.py`` that pointed at the retired snapshot tables are
also cleaned up.

Also strips an 8-line comment block in ``restore_version`` that
duplicated the docstring of ``_stamp_audit_fields_for_restore``.

Net: −290 lines from ``daos/version.py``; a production-shape bug
fixed; dead code that looked live is gone.
2026-05-28 15:37:19 -06:00
Mike Bridge
2c7ba5c3d5 refactor(versioning): single_flush_scope context manager + single-revert restore
VersionDAO.restore_version previously called Continuum's Reverter
once per relation in a split-revert loop with flush + expire between
calls. That closed an autoflush race in the Reverter when multiple
relations were reverted at once, but split one logical restore across
multiple Continuum transactions — and once the change-records listener
was wired up, the listener's tx-dedup guard skipped the second pass,
silently dropping child-addition records from version_changes. A
restore that re-added a calculated column would render as an empty
"Baseline" entry in the dropdown.

Replaces the split-revert with a single ``target_version.revert(relations=relations)``
call wrapped in a new ``single_flush_scope(db.session)`` context
manager (``superset/versioning/utils.py``). The context manager
suppresses autoflush inside the block and issues one trailing flush
on clean exit; on exception, the trailing flush is skipped so the
session's normal rollback path handles cleanup. Same autoflush window
closed, one Continuum transaction instead of N, the change-records
listener sees the complete shadow state in one after_flush pass.

The wrapper carries the full autoflush-race / cascade-add rationale
in its docstring so the restore_version call site can be a short
6-line block referencing it.

Integration coverage: ``test_restore_emits_full_child_diff_in_one_transaction``.
2026-05-28 15:37:19 -06:00
Mike Bridge
eb3bcf0d3d feat(versioning): force-parent-dirty on versioned-child change
SQLAlchemy doesn't mark a parent as dirty when only its children
(``TableColumn`` / ``SqlMetric`` on ``SqlaTable``) are modified.
Continuum's UnitOfWork only creates operations for entities in
``session.dirty``, so a column-only edit produces shadow rows in
``table_columns_version`` but no parent shadow row in
``tables_version``. ``VersionDAO.list_versions`` queries the parent
shadow, so the version dropdown is empty for child-only saves —
exactly the failure mode reported when "I edited a column description
but no version appeared."

Extends ``register_baseline_listener`` with a new before-flush hook
``_force_parent_dirty_on_child_change`` that walks the existing
``_child_to_parent_registry`` and ``attributes.flag_modified(parent,
<first non-excluded versioned column>)`` whenever a versioned child
is dirty / new / deleted but the parent's own scalars haven't been
touched. The flag puts the parent in ``session.dirty`` so Continuum's
UoW creates a parent UPDATE operation; the resulting shadow row's
scalar columns mirror the previous version (only the children
actually changed), and the row exists to anchor the transaction in
the parent's version chain.

``SkipUnmodifiedPlugin._is_no_op_update`` is updated in this commit's
predecessor to recognize the "scalars match but children dirty" case
via ``_has_dirty_versioned_children`` so the forced parent UPDATE
isn't skipped.

Integration coverage: ``test_dataset_column_edit_creates_parent_version``.
2026-05-28 15:37:19 -06:00
Mike Bridge
0850679edd feat(versioning): SkipUnmodifiedPlugin audit-key normalize for Dashboard.json_metadata
Continuum's no-op suppression compared post-flush column values
byte-for-byte against the previous live shadow row. For
``Dashboard.json_metadata`` that produced false-positive version rows
on saves where the user authored nothing — the frontend re-stamps
``map_label_colors`` (regenerated from the ``LabelsColorMap``
singleton) on every save, plus ``chart_configuration`` /
``global_chart_configuration`` / ``show_chart_timestamps`` /
``color_namespace`` (derived from the current chart set), so two
consecutive identical saves produce different bytes for the column.
The diff engine already excluded those keys via
``DASHBOARD_JSON_METADATA_AUDIT_KEYS`` when computing change records;
the skip-plugin diverged.

Adds a ``_COLUMN_NORMALIZERS`` registry keyed on
``(class_name, column_name)`` that maps to a per-column normalizer
applied to both pre- and post-image before equating. The first
entry parses ``Dashboard.json_metadata`` as JSON and drops the
audit-key set before comparing. The same registry is the extension
point for analogous transient fields on charts and datasets.

Promotes ``_DASHBOARD_JSON_METADATA_AUDIT_KEYS`` to a public name
(``DASHBOARD_JSON_METADATA_AUDIT_KEYS``) so the skip-plugin can import
it from ``superset.versioning.diff`` without reaching across a
leading-underscore boundary.

Integration coverage: ``test_map_label_colors_only_change_does_not_create_version``.
2026-05-28 15:37:19 -06:00
Mike Bridge
6cc0853952 fix(importer): use ORM relationship assignment for dashboard_slices
The v1 import pipeline previously wrote dashboard ↔ chart membership
via raw Core DML (``db.session.execute(delete(dashboard_slices)…)`` +
``db.session.execute(insert(dashboard_slices)…)``). With Continuum's
M2M tracker enabled by the versioning feature, those Core writes
emit malformed shadow INSERTs into ``dashboard_slices_version`` —
the tracker can't see the composite-PK columns through the Core
layer and produces rows with only ``(transaction_id, operation_type)``
populated, triggering a ``NOT NULL`` violation on
``(dashboard_id, slice_id)``.

Rewrites both import paths (``ImportAssetsCommand._import`` in
``commands/importers/v1/assets.py`` and ``ImportDashboardsCommand._import``
in ``commands/dashboard/importers/v1/__init__.py``) to use ORM-level
``dashboard.slices = [...]`` reassignment followed by an explicit
``db.session.flush()``. The explicit flush is necessary to land the
M2M rows before any subsequent autoflush fires an inner-flush event
handler that would reset the relationship change (cf. the SAWarning
``Attribute history events accumulated on N previously clean instances
within inner-flush event handlers have been reset``).

The unit tests previously called ``_import`` directly twice in the same
session — production wraps ``run()`` in ``@transaction`` so each invocation
gets its own DB+Continuum transaction. Added ``db.session.commit()`` between
calls in ``test_import_adds_dashboard_charts``,
``test_import_removes_dashboard_charts``, and
``test_dashboard_import_with_overwrite_replaces_charts`` so the tests
mirror production semantics; otherwise the second call's M2M shadow
inserts conflict with the first call's on
``UNIQUE (dashboard_id, slice_id, transaction_id)``.
2026-05-28 15:37:19 -06:00
Mike Bridge
96018dab71 temp(versioning): demo version-history dropdowns + French i18n
Adds debug-only ``VersionHistoryDropdown`` widgets to the chart,
dashboard, and dataset list pages so the version surface can be
exercised from the UI during the spike. Each row's actions column
gets a clock-icon dropdown that fetches ``/api/v1/{resource}/<uuid>/
versions/`` on click, lists the ten most recent versions with a
formatted change-log summary, and offers per-version restore via
``POST .../versions/<uuid>/restore``.

Strings are wrapped in ``t('...')`` with placeholder formatting
(e.g. ``t('Added %(kind)s "%(name)s"', { kind, name })``) so
translators can reorder verbs and nouns rather than concatenating
fragments. ``KIND_LABELS`` is a static map keying English layout
kinds (``chart``, ``row``, ``column``, ``tab``, ``markdown``, etc.)
to ``t(...)``-extractable labels. Empty change lists render as
"Baseline" rather than "No changes recorded" since the empty case
is overwhelmingly the ``operation_type=0`` baseline row.

Locale-aware date rendering: ``new Date(iso).toLocaleString(lang)``
where ``lang`` comes from ``document.documentElement.lang`` (set
by ``src/views/App.tsx`` from the bootstrap ``locale``), so dates
follow the user's chosen Superset locale rather than the browser's.

French translations for the new strings are appended to
``superset/translations/fr/LC_MESSAGES/messages.po`` (Ajouté,
Supprimé, Modifié, Version initiale, kind labels, …). Run
``npm run build-translation`` and ``pybabel compile -l fr`` to
regenerate the JSON / MO packs.

This commit is **demo-only** per ADR-005 (V1 is backend-only). It
is intentionally marked ``temp`` so it can be reverted before the
PR splits — the production V1 ships without UI.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
39cfda37e7 test(versioning): integration tests for SkipUnmodifiedPlugin (FR-026)
Locks in the no-op-suppression behavior implemented by
``SkipUnmodifiedPlugin`` (which lives in ``superset/versioning/factory.py``
shipping with the foundation commit). Five integration tests:

1. Owners-only edit doesn't mint a version row — exercises the
   case where every dirty column is an excluded relationship.
2. Re-save with identical scalar values doesn't mint a row —
   exercises the json_metadata re-serialise path where
   ``set_dash_metadata`` rewrites the column to a different byte
   sequence with identical parsed content; the plugin must compare
   post-flush values against the prior shadow row to detect this.
3. Real scalar change DOES mint a row — guards against the plugin
   over-suppressing.
4. Same assertion on a Slice (covers the ``String`` column path on
   a different entity type).
5. ``json_metadata`` sub-key edit DOES mint a row — covers the
   ``MediumText`` column path past the plugin's content-equality
   check.

Tests are designed so a column-type change in the parent entities
(e.g. flipping ``json_metadata`` from ``MediumText`` to ``JSON``)
will fail one of these if the plugin's Python ``!=`` comparison
breaks for the new type.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
47a1c9fa95 feat(versioning): ETag helper module + integration tests (T055)
Helper module that derives the strong-validator ``ETag`` value from
an entity's current live ``version_uuid`` and attaches it to a
Flask response. Two functions:

- ``set_version_etag(response, version_uuid)`` — direct path used by
  PUT handlers that already compute ``new_version_uuid`` (see the
  REST API commit two prior). Cheap; no extra query.
- ``set_version_etag_by_uuid(response, model_cls, entity_uuid)`` —
  used by version endpoints that operate on ``entity_uuid``; looks
  up ``entity_id`` then derives ``version_uuid`` via ``VersionDAO``.
  Costs one extra ``SELECT id WHERE uuid = ?``; documented in the
  docstring so callers prefer the cheap variant when they have the
  id already.

Integration tests cover all three entity types and four endpoint
shapes (entity GET, save PUT, version-list GET, single-version GET)
plus the entity-with-no-versions edge case (header is correctly
absent).

The ETag is wired into the API endpoints in the REST-API commit
(group 3) and the CORS ``expose_headers: ["ETag"]`` ships with the
retention commit (group 4) since both touch ``superset/config.py``.
Locking enforcement (``If-Match`` → 412) is explicitly NOT in this
change — deferred to the follow-up UI SIP per Open Question §7.
``ETag`` is informational in v1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
963f059df3 feat(versioning): time-based retention via Celery beat (FR-007)
Adds a scheduled Celery task that prunes version history older than
``SUPERSET_VERSION_HISTORY_RETENTION_DAYS`` (default 30; settable
via env var; ``0`` disables retention entirely).

**Task** — ``superset.tasks.version_history_retention.prune_old_versions``:

1. Computes ``cutoff = utcnow() - timedelta(days=N)``.
2. Selects ``version_transaction.id`` rows with ``issued_at <
   cutoff`` and filters out any tx whose parent shadow includes a
   live row (``end_transaction_id IS NULL``). The live row is the
   only preservation rule — closed historical rows including the
   baseline (``operation_type=0``) age out. Per-entity minimum-history
   floor is an open question tracked in ``future-work.md``.
3. Deletes rows owned by surviving txs in each parent shadow
   table (``dashboards_version`` / ``slices_version`` /
   ``tables_version``).
4. Deletes child-shadow rows for the same transactions
   (``table_columns_version`` / ``sql_metrics_version`` /
   ``dashboard_slices_version``).
5. Drops the surviving ``version_transaction`` rows. The
   ``version_changes`` rows cascade via the FK from the previous
   commit.

Idempotent and safely retried on partial failure.

**Schedule** — ``superset/config.py`` adds the task to the default
``CeleryConfig.beat_schedule`` (nightly at 03:00). Operators who
override ``CeleryConfig`` in their ``superset_config.py`` need to
merge this entry — see UPDATING.md.

Also adds ``"expose_headers": ["ETag"]`` to the default
``CORS_OPTIONS`` so cross-origin browser clients can read the
``ETag`` header introduced in the next commit. (Co-located here
because both touch ``superset/config.py``; the ETag mechanism
itself ships in the next commit.)

**Auto-discovery** — ``superset/tasks/celery_app.py`` adds
``version_history_retention`` to its late-imports so Celery's
auto-discovery picks up the task.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
24eef8c984 feat(versioning): REST API endpoints + restore commands
Exposes the version surface as three new endpoints per entity type
(chart, dashboard, dataset), each carrying the standard Superset
decorator stack (``@protect()``, ``@safe``, ``@statsd_metrics``,
``@event_logger.log_this_with_context``) so they appear in FAB's
``action_log`` alongside other audited operations.

| Method | Path | Purpose |
|---|---|---|
| GET  | ``/api/v1/{resource}/<uuid>/versions/`` | List version history (oldest-first; per entry: ``version_uuid``, ``version_number``, ``transaction_id``, ``operation_type``, ``issued_at``, ``changed_by``, ``changes`` array) |
| GET  | ``/api/v1/{resource}/<uuid>/versions/<version_uuid>/`` | Read-only snapshot of the entity at the requested version (scalar fields plus ``columns`` / ``metrics`` for datasets) |
| POST | ``/api/v1/{resource}/<uuid>/versions/<version_uuid>/restore`` | Replay the snapshot onto the live entity via Continuum's ``Reverter`` (non-destructive — produces a new version row stamping the restoring user via the standard save path) |

``<version_uuid>`` is a deterministic ``UUIDv5(entity_uuid,
transaction_id)`` so it's stable across replicas and retention
pruning. Authorisation reuses the resource's existing ``can_write``
permission; workspace admins can list / restore any entity.

**Restore commands** — ``superset/commands/{chart,dashboard,dataset}/
restore_version.py`` wrap ``VersionDAO.restore_version`` in the
standard ``@transaction()`` boundary. The command resolves the
``Reverter`` once per related collection (split-revert pattern, with
``flush + expire`` between calls) so a multi-relation restore
doesn't trip Continuum's autoflush race that would otherwise mark
half the collection as ``state.deleted=True`` mid-revert.

**Save responses** — ``PUT /api/v1/{resource}/<pk>`` is updated to
include ``old_version`` / ``new_version`` (0-based numbers),
``old_transaction_id`` / ``new_transaction_id`` (stable across
pruning), and ``old_version_uuid`` / ``new_version_uuid`` body
fields so callers can correlate a save with its resulting version
row. The ``ETag`` response header in the next commit is built on
top of this, but the body fields stay — they predate the header
and remain useful for clients that don't read response headers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
7ec3091d55 feat(versioning): change records + diff engine
Adds a structured per-field change log alongside the foundational
shadow tables. Each save flush emits zero or more ``version_changes``
rows describing what changed relative to the previous version, with
shape ``[{kind, path, from_value, to_value, sequence}]`` keyed to
``version_transaction.id`` (FR-016 .. FR-021).

**Schema** — ``version_changes`` table, FK to ``version_transaction``
with ``ON DELETE CASCADE`` so retention drops dependent records
without explicit cleanup. Composite unique index on
``(transaction_id, entity_kind, entity_id, sequence)`` so the
listener can write monotonically and downstream readers see a
deterministic order.

**Diff engine** (``superset/versioning/diff.py``) — pure-function
diffing of pre-/post-state pairs:

- ``diff_scalar_fields`` for ordinary columns; emits one record per
  changed field with JSON-safe ``from_value`` / ``to_value``.
- ``diff_json_field`` for ``json_metadata`` and ``params``, walking
  the parsed structure and emitting per-sub-key records. Honours
  an ``exclude_keys`` set
  (``_DASHBOARD_JSON_METADATA_AUDIT_KEYS``: ``chart_configuration``,
  ``global_chart_configuration``, ``map_label_colors``,
  ``show_chart_timestamps``, ``color_namespace``;
  ``_CHART_PARAMS_AUDIT_KEYS``) so frontend-stamped sub-keys that
  mutate on every save don't dominate the change log (FR-022).
- ``diff_dashboard_layout`` walks ``position_json`` structurally
  and emits ``[verb, kind, id]`` records (verbs ``add``, ``remove``,
  ``move``, ``edit``; kinds from a ``CHART``/``ROW``/``COLUMN``/etc.
  → english map) so a UI can render "Added chart 'Foo'" without
  re-parsing JSON. ``HEADER_ID`` is suppressed because it duplicates
  the ``dashboard_title`` scalar record.
- ``fold_dashboard_layout_with_chart_changes`` deduplicates layout
  records against M2M / chart-membership records by UUID so an
  add-and-attach doesn't appear twice.
- ``_values_equivalent`` treats ``None`` and ``""`` as equal; this
  matches the save path's habit of normalising nullable strings to
  the empty string.

**Listener** — ``superset/versioning/changes.py`` registers a
``before_flush`` listener that captures pre-state for each dirty
entity and an ``after_flush`` listener that runs the diff engine
against the post-state and writes ``version_changes`` rows under
the resolved ``transaction_id``. Tracks processed transaction ids
on ``session.info`` so re-firings within a single transaction
(autoflush triggered by mid-commit queries) don't double-insert and
trip the unique constraint. Reads child rows via raw SELECT against
``table_columns`` / ``sql_metrics`` rather than ``dataset.columns``
because the live collection is stale during the restore path's raw
DELETE+INSERT cycle.

**Endpoint surface** — ``VersionDAO.list_change_records_batch``
batches the lookup across multiple transactions with a single
``WHERE transaction_id IN (...)`` query so the version-list
endpoint avoids N+1 round-trips. ``list_versions`` / ``get_version``
return entries with a populated ``changes`` array (empty for
``operation_type=0`` baseline rows).

**Tests** — ``test_diff.py`` covers the diff engine shape (39
unit cases across scalar, JSON, layout, child-collection, and
fold paths). ``change_records_tests.py`` exercises the listener
end-to-end with realistic save flows. ``perf_validation_tests.py``
is the T044 harness for SC-002/3/4 (list endpoint p95 < 1s,
restore < 3s, save overhead < 50ms).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
b68fea17af feat(versioning): foundation — Continuum capture + parent/child shadow tables + VersionDAO
Adds SQLAlchemy-Continuum as a dependency and wires it as the
canonical capture mechanism for chart, dashboard, and dataset edits.

**Schema** — three Alembic migrations, leaving the chain at one
foundation revision plus one child-shadow revision:

- ``version_transaction`` (renamed from Continuum's default
  ``transaction``; SQL-reserved-word workaround) carries the per-save
  ``user_id`` / ``issued_at`` and is the join target for all shadow
  rows. Auto-incrementing PK; user_id has no FK so import / Celery /
  CLI saves can write rows without an active Flask user.
- Parent shadow tables for the three entity types:
  ``dashboards_version``, ``slices_version``, ``tables_version``.
- Child shadow tables for dataset children + dashboard M2M:
  ``table_columns_version``, ``sql_metrics_version``,
  ``dashboard_slices_version`` (composite PK on the M2M shadow,
  matching the live ``dashboard_slices`` reshape from
  sc-105349-composite-association-pks).

**Models** — ``Dashboard``, ``Slice``, ``SqlaTable`` (and dataset
children ``TableColumn`` / ``SqlMetric``) gain ``__versioned__``
class attributes. The exclude lists carry both M2M relationships
(``owners``, ``roles``, ``dashboards``) and the ``AuditMixin``
columns (``changed_on`` / ``created_on`` / ``changed_by_fk`` /
``created_by_fk`` plus ``last_saved_at`` / ``last_saved_by_fk``
on ``Slice``) so auto-bumped audit fields cannot trigger a
version row on their own (FR-025).

**Plugins** — ``superset/versioning/factory.py`` ships three
Continuum plugins:

- ``VersionTransactionFactory`` renames the transaction table and
  appends the unconditional ``user_id`` column.
- ``VersioningFlaskPlugin`` sources the acting user from Superset's
  ``g.user`` rather than ``flask_login.current_user`` (Superset's
  JWT auth populates ``g.user`` but leaves ``current_user``
  anonymous on API routes).
- ``SkipUnmodifiedPlugin`` filters Continuum's UPDATE operations,
  marking content-equivalent re-saves as ``processed=True`` so they
  don't mint no-op shadow rows (FR-026; see follow-up commits for
  the test). Lives in this commit because it shares the file with
  the other plugins.

**Save-path glue** — a ``before_flush`` baseline listener
(``superset/versioning/baseline.py``) inserts an ``operation_type=0``
shadow row the first time a pre-existing entity is saved, including
the slice-baseline-under-dashboard pattern that gives the dashboard
M2M shadow a row to join against. ``UpdateDashboardCommand`` wraps
its body in ``no_autoflush`` so ``process_tab_diff`` /
``process_native_filter_diff`` don't fire intermediate flushes that
would mint extra version rows. ``DatasetDAO.update_columns`` is
rewritten as a natural-key upsert keyed on ``column_name`` so child
edits flow through ORM events Continuum sees.

**DAO** — ``superset/daos/version.py`` exposes the read API used by
the version endpoints in the next commits:
``current_version_number`` (0-based index, unstable under retention
pruning), ``current_live_transaction_id`` (stable across pruning),
``current_live_version_uuid`` (deterministic UUIDv5), plus
``list_versions`` / ``get_version`` / ``restore_version`` and a
batch ``list_change_records_batch`` for N+1 avoidance.

**Initialization** — ``superset/initialization/__init__.py`` wires
``init_versioning()`` after ``make_versioned()`` runs and the
versioned mappers are configured. Registers the baseline listener
plus the change-record listener (the latter's body lives in the
next commit but the registration site is here because it shares
the init function).

**Tests** — version-capture and version-list integration tests for
each entity type, plus a ``VersionDAO`` unit test suite. Retention
test uses a backdated ``issued_at`` so it can drive
``_prune_old_versions_impl`` synchronously.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:19 -06:00
Mike Bridge
887d49cbac fix(migration): address aminghadersohi review feedback (sc-105349)
Three improvements from @aminghadersohi's review on
apache/superset#39859:

1. **`fk["name"]` unguarded in ``_downgrade_mysql_table`` re-add loop**

   The drop loop gates on ``if fk_name := fk.get("name"):`` but the
   re-add loop accessed ``fk["name"]`` unconditionally in an f-string.
   MySQL/InnoDB always assigns FK names, so this branch was defensive,
   but the asymmetry was confusing. Symmetrized via ``continue`` at the
   top of the re-add loop.

2. **``ondelete`` whitelist before raw-SQL interpolation**

   The value comes from MySQL's ``information_schema`` (not user
   input), but interpolating a reflected string into raw SQL without a
   guard left a "what if an unexpected value appears" footgun. Added
   ``_VALID_ONDELETE_ACTIONS`` (the four SQL-standard actions) and a
   ``RuntimeError`` when an unexpected value is reflected.

3. **Direct ALTER on PostgreSQL for tables with pre-existing UNIQUE**

   ``recreate="always"`` is dialect-agnostic — on PostgreSQL it
   triggers ``CREATE TABLE AS SELECT → DROP → RENAME`` holding
   ``ACCESS EXCLUSIVE`` for the full table-copy duration. For a
   multi-million-row ``dashboard_slices``, that lock window can be
   noticeable. The reflected UNIQUE constraint has a stable name on
   PostgreSQL (default ``<table>_<cols>_key`` convention), so dropping
   it directly and then running structural change as direct ALTER
   avoids the copy entirely.

   The reflected UNIQUE name is wrapped in a new
   ``_drop_redundant_unique_by_name()`` helper. Postgres takes the
   direct path; MySQL keeps ``recreate="always"`` because InnoDB binds
   FKs to the UNIQUE's underlying index for back-reference (``DROP
   CONSTRAINT`` on the UNIQUE there raises ``ERROR 1553``); SQLite
   keeps ``recreate="always"`` because unnamed UNIQUEs reflect with
   ``name=None`` and can't be dropped by name.

Verified end-to-end: downgrade-then-upgrade against MySQL with
~12M total junction rows seeded completes in ~1m 41s (within the
range of the prior measurements).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:37:02 -06:00
Mike Bridge
8f5a230c84 fix(migration): skip alter_column nullable=False on non-SQLite (sc-105349)
Justin Park (@justinpark) reported on apache/superset#39859:

  MySQLdb.OperationalError: (1832, "Cannot change column 'dashboard_id':
  used in a foreign key constraint 'fk_dashboard_roles_dashboard_id_dashboards'")

Root cause: ``batch_op.alter_column(fk1, nullable=False)`` for the six
non-UNIQUE association tables emits ``ALTER COLUMN`` on a column that
participates in an FK constraint. MySQL 8 rejects this with ERROR 1832
when the table has data — even when the change is just ``NULL`` →
``NOT NULL`` and the column is already part of a freshly-added
composite primary key (which InnoDB has just made implicitly NOT NULL
anyway). The error fires on populated tables only; CI's ``test-mysql``
shard runs against empty tables and so didn't catch this, while a
real production-shaped install does.

The ``alter_column`` was only ever needed for SQLite, where composite
``PRIMARY KEY`` does not promote constituent columns to ``NOT NULL``
(a long-standing SQLite quirk — only ``INTEGER PRIMARY KEY`` does).
PostgreSQL and MySQL implicitly promote PK columns to ``NOT NULL`` as
part of ``ADD PRIMARY KEY``, so the explicit step is unnecessary on
both — and on MySQL it's actively broken on populated tables.

Fix: extract the ``alter_column`` pair into a helper
``_enforce_not_null_for_sqlite()`` that no-ops on Postgres and MySQL.
Both branches of the per-table upgrade (the ``recreate="always"`` path
for the two UNIQUE-bearing tables, and the direct-ALTER path for the
other six) now call the helper instead of inlining the
``alter_column``.

Verified end-to-end: downgrade-then-upgrade against MySQL with
~12M total junction rows (10M dashboard_slices + 1M each
slice_user/dashboard_user + 100K dashboard_roles) completes in
1m 39s with no ERROR 1832. The 44 in-memory SQLite tests still pass.

Considered Justin's alternative (drop FKs on MySQL across all eight
tables, unifying the two branches) but rejected as more invasive —
it would require capturing FK metadata and explicitly re-creating
the FKs for the six non-recreate tables, since they don't go through
the ``copy_from`` path that re-creates FKs automatically. The
SQLite-only approach is more targeted: it removes the operation that
MySQL rejects rather than working around the rejection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:36:58 -06:00
Mike Bridge
aaa40761b1 feat(scripts): add --dirty-duplicates-pct to seed_junction_load.py
Extends the stress-test seed script with an optional duplicate-row
injection step, used to measure the empirical cost of the migration's
``_dedupe_by_min_id`` phase.

Usage: after running the normal seed at a given scale, add
``--dirty-duplicates-pct 5`` (or any non-zero value) to inject that
percentage of duplicate ``(fk1, fk2)`` rows into each non-UNIQUE
junction (slice_user, dashboard_user, dashboard_roles —
dashboard_slices is skipped because its UNIQUE constraint, present
both pre- and post-migration, rejects duplicates).

Pre-condition: requires the DB to be at the pre-migration revision
(33d7e0e21daa). The post-migration composite PK rejects duplicates,
so attempting to inject on the upgraded schema errors out.

Empirical result on MySQL @ 10M dashboard_slices + ~2.1M other
junction rows + 105K injected duplicates (5% on the 3 non-UNIQUE
tables):
  Upgrade time: 1m 36s vs clean baseline 1m 37s
  → dedupe cost is within measurement noise; the table-scan that
    the migration already performs dominates whether or not
    duplicates exist.

This empirically confirms what the cost-model predicted: the
``_dedupe_by_min_id`` GROUP BY scan is the dominant cost of that
phase, and the actual per-duplicate DELETE is negligible.

NULL-FK injection deliberately skipped — would require altering the
six non-UNIQUE FK columns from NOT NULL back to nullable (the
migration's downgrade keeps them NOT NULL by design), which adds
per-backend ALTER complexity for a code path that's structurally
identical in cost shape (DELETE WHERE col IS NULL is the same scan
shape as the dedupe scan).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
c2a761e44d build(scripts): add stress-test data generator for migration timing
Add ``scripts/seed_junction_load.py``, a backend-agnostic script that
bulk-inserts synthetic parent rows (dashboards, slices, users, roles,
tables, dbs) and many-to-many junction rows for the four largest
association tables targeted by the composite-PK migration:
``dashboard_slices``, ``slice_user``, ``dashboard_user``,
``dashboard_roles``.

Designed for measuring migration runtime at varying scales — run with
a series of size flags (100K / 1M / 5M / 10M for the target table)
and time the migration at each scale to verify the predicted
``O(N log N)`` extrapolation against real numbers.

Properties:
- **Reproducible**: deterministic cross-product walk through parent IDs
  produces a stable pair sequence; re-running is replayable.
- **Idempotent**: re-running with the same target is a no-op; with a
  higher target, only new rows are added.
- **Backend-agnostic**: connects via Superset's standard ``DATABASE_*``
  env vars (or ``SUPERSET__SQLALCHEMY_DATABASE_URI``). Branches on
  dialect for ``BINARY(16)`` vs ``UUID`` vs TEXT/BLOB UUID columns.
- **Batched**: bulk INSERT 10K rows per statement.
- **Per-phase timing**: logs elapsed wall time for the parents phase,
  the junctions phase as a whole, and per junction-table.
- **Avoidance set**: loads existing junction pairs into a Python set
  so re-runs on top of pre-existing data don't collide on the
  uniqueness constraint.

Usage (inside the Superset container):

    docker exec superset-superset-1 \\
        /app/.venv/bin/python /app/scripts/seed_junction_load.py \\
        --dashboard-slices 1000000

Defaults target a "large multi-team install" shape: 1M
``dashboard_slices``, 100K each ``slice_user`` / ``dashboard_user``,
10K ``dashboard_roles``. Override per-table via flags.

Tested locally on MySQL (the user's current eval stack):
- 200/100/100/50 row mini-run produced expected counts.
- Re-running at the same target is a no-op (idempotent).
- ``--dry-run`` plans without writing.

Junction tables not yet covered (``sqlatable_user``, ``rls_filter_*``,
``report_schedule_user``) are typically small in production and
require additional parent seeding (RLS filters, report schedules)
that wasn't worth the scope here. Adding them is straightforward by
extending ``JUNCTIONS`` and writing the corresponding parent seeder.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
acfdd22c21 fix(docker): MySQL examples DB + EXAMPLES_PORT override (sc-105349)
Fix two follow-on issues reported when starting the dev stack with
docker-compose-mysql.yml:

1. ``superset-init`` step 4 (load-examples) fails with
   ``MySQLdb.OperationalError: (2002, "Can't connect to server on 'db'")``
   because the analytics-examples DB connection inherits ``EXAMPLES_PORT=5432``
   (Postgres port) from ``docker/.env``. The override flipped
   ``DATABASE_DIALECT`` to ``mysql+mysqldb`` but left the EXAMPLES_*
   group on Postgres defaults, so the URI became
   ``mysql+mysqldb://examples:examples@db:5432/examples`` — MySQL
   container has no listener on 5432.

   Fix: add ``EXAMPLES_HOST/PORT/DB/USER/PASSWORD`` and a complete
   ``SUPERSET__SQLALCHEMY_EXAMPLES_URI`` to the ``mysql-env`` anchor.

2. The Postgres init scripts under
   ``docker/docker-entrypoint-initdb.d/`` (``cypress-init.sh``,
   ``examples-init.sh``) get mounted on the MySQL container too —
   compose merges volume lists. They invoke ``psql`` which doesn't
   exist in the MySQL image, abort with ``psql: command not found``,
   and prevent the ``examples`` DB from being created.

   Fix: add a MySQL-specific init script
   ``docker/mysql-init/examples-init.sql`` that creates the
   ``examples`` database and user, and mount it at
   ``/docker-entrypoint-initdb.d`` in the override. Compose's
   later-takes-precedence rule on duplicate volume targets displaces
   the Postgres init dir, so the MySQL container only sees the
   MySQL-compatible script.

   (Used a plain duplicate-target mount rather than the ``!override``
   tag because pre-commit's ``check-yaml`` doesn't recognize Compose's
   custom YAML tags.)

Recovery for an existing failed MySQL stack: ``docker compose -f
docker-compose.yml -f docker-compose-mysql.yml down``, then
``docker volume rm superset_db_home_mysql`` (so the new init script
runs on the next fresh boot), then ``up`` again.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
1fc7b17dfa build(docker): add MySQL compose override for dialect-swap evaluation
Adds ``docker-compose-mysql.yml``, a compose-override file that swaps
the default Postgres metadata DB for MySQL 8 with one extra ``-f``
flag:

  docker compose -f docker-compose.yml -f docker-compose-mysql.yml up

Useful for evaluating dialect-specific behaviour (e.g., the runtime
cost of DDL migrations on a deployment whose production metadata DB
is MySQL — the question raised by review feedback on this PR).

Mirrors the connection settings used by CI's ``test-mysql`` shard:
``mysql+mysqldb`` dialect, charset ``utf8mb4`` with binary_prefix.
Host port defaults to 13306 (configurable via ``DATABASE_PORT_MYSQL``)
to avoid colliding with a native MySQL install on 3306.

A separate volume (``db_home_mysql``) keeps MySQL data isolated from
the Postgres ``db_home`` volume, so switching between the two with
``-f`` flag toggles doesn't corrupt either side.

The Postgres-specific init scripts under
``docker/docker-entrypoint-initdb.d/`` are not mounted on the MySQL
service (they are postgres-only). Examples / cypress fixtures still
load via ``superset-init``'s post-startup steps, which run
``superset load-examples`` against whichever metadata DB is in use.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
08cfcf8fd4 docs(UPDATING): add MySQL-targeted maintenance-window queries (sc-105349)
Mirror of the PostgreSQL diagnostic queries added in 11148779ed,
adapted for MySQL/InnoDB. One important difference: InnoDB rebuilds
the clustered index on every PK change, so all eight tables undergo
a full table rebuild on MySQL — not just the two that go through
the explicit ``recreate="always"`` path. The lock-window estimate
query is updated to cover all eight rather than just two, and the
"migration_path" column makes the rebuild expectation explicit
("direct ALTER (still rebuilds InnoDB clustered index)").

Other notes:
- ``information_schema.TABLES.TABLE_ROWS`` is an InnoDB estimate,
  analogous to PostgreSQL's ``reltuples``; documented inline.
- ``KEY_COLUMN_USAGE`` carries both sides of the FK in a single
  row on MySQL, so the external-FK pre-flight check is simpler
  than the PostgreSQL version (no joins between three views).
- The aggregated dedupe query is portable standard SQL; included
  verbatim for copy-paste convenience.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
4eb42d7bdf docs(UPDATING): add Postgres-targeted maintenance-window queries (sc-105349)
Add a "Sizing the maintenance window on PostgreSQL" sub-section to the
operator runbook. The simple per-table COUNT/duplicate/NULL queries
that were already there are dialect-portable but only count rows;
operators on PostgreSQL with large deployments need to characterize
the migration's runtime cost before scheduling it.

Adds four diagnostic queries:

- Per-table size, row count (from pg_class.reltuples), and which
  migration path each table will take (recreate-rewrite vs direct
  ALTER). Sizes the work concretely.
- Aggregated duplicate-row roll-up: dup_groups + total rows_dropped
  per table. Replaces eight separate per-table queries with one
  consolidated result for audit/dump-before-apply decisions.
- External-FK pre-flight check (the same one the migration runs at
  upgrade time and aborts on). Lets operators surface any blocking
  external reference ahead of the maintenance window. Should be
  empty on a stock install.
- Lock-window estimate for the two full-rewrite tables, using
  pg_relation_size and a conservative 100 MB/s rewrite throughput
  assumption. The other six use direct ALTER and are dominated by
  composite-index build time (seconds for low-millions-of-rows
  tables).

Prompted by reviewer feedback on apache/superset#39859 from a large
deployment asking how to size the maintenance window. The original
pre-flight queries are kept for cross-dialect operators (MySQL,
SQLite) since the new queries use PostgreSQL-specific catalog views.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
9f03047913 fix(migration): rebase down_revision onto 33d7e0e21daa (sc-105349)
CI cypress + playwright shards were red with:

  ERROR [flask_migrate] Error: Multiple head revisions are present
  for given argument 'head'

The recent rebase onto master pulled in
``33d7e0e21daa_add_semantic_layers_and_views.py`` (from PR #37815,
"semantic layer extension"), which had been authored against
``ce6bd21901ab`` as its parent — the same parent our migration
referenced. After the rebase both migrations point at
``ce6bd21901ab``, producing two heads and breaking ``flask db
upgrade head`` for any downstream consumer (CI's Cypress / Playwright
shards spin up a real Superset instance via ``superset db upgrade``,
which is why those shards failed first; the integration shards run
against a precomputed schema and didn't surface this).

Fix: chain our migration after the semantic-layer migration by
pointing ``down_revision`` at ``33d7e0e21daa``. The chain is now
linear:

    ... → ce6bd21901ab → 33d7e0e21daa (semantic layers)
                          → 2bee73611e32 (composite PK, this PR)

Verified with ``superset db heads`` (returns single head
``2bee73611e32``) and the local migration test suite (44 passed,
1 skipped).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
7fe649ef68 fix(migration): explicit NOT NULL on FK columns for SQLite (sc-105349)
Found by running fresh-install + round-trip against a real SQLite DB:
6 of the 8 affected tables had FK columns that were originally
declared nullable. PostgreSQL and MySQL implicitly promote the
constituent columns of an ``ALTER TABLE ... ADD PRIMARY KEY`` to
``NOT NULL``; SQLite does not (it's a long-standing SQLite quirk —
only ``INTEGER PRIMARY KEY`` enforces NOT NULL on a composite-PK
column). Result: a fresh SQLite install would accept
``INSERT INTO dashboard_slices (NULL, 5)`` despite both columns
being part of the composite PK.

Our integration tests previously masked this: the test fixture seeds
columns with ``nullable=False``, so the post-upgrade NOT NULL
assertion passed regardless of whether the migration enforced it.

Fix: add explicit ``batch_op.alter_column(fk, nullable=False)`` for
both FK columns inside the per-table batch_alter_table block. On
PostgreSQL and MySQL this is a no-op (PK already implies NOT NULL);
on SQLite it adds the missing NOT NULL declaration so a fresh
install matches the data-model.md "After" contract.

Verified end-to-end:
- Postgres + MySQL: column shape unchanged (still NOT NULL)
- SQLite fresh install + round-trip: all 8 tables now have NOT NULL
  on FK columns, ``INSERT (NULL, 5)`` correctly rejected with
  IntegrityError on dashboard_slices, dashboard_user, sqlatable_user

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
e4e0b4b144 fix(migration): MySQL downgrade FK + AUTO_INCREMENT (sc-105349)
Two MySQL-only failures in the downgrade path, found by running the
full migration history against a fresh MySQL 8 container:

1. ``MySQLdb.OperationalError: (1553, "Cannot drop index 'PRIMARY':
   needed in a foreign key constraint")``. InnoDB uses the composite
   PK index to back the FK on the leftmost column. The downgrade
   tried to drop the composite PK before dropping the FKs, orphaning
   the FK's backing index. PostgreSQL and SQLite create separate
   indexes for FK columns and don't trip on this.

2. ``Field 'id' doesn't have a default value`` on subsequent INSERT.
   ``sa.Identity(always=False)`` only emits ``AUTO_INCREMENT`` on
   MySQL when the column is created with ``primary_key=True`` — our
   portable path adds the column first then creates the PK separately,
   so MySQL leaves the column without auto-generation. Existing rows
   would all collide on id=0; future inserts fail because no default.
   Postgres' ``GENERATED BY DEFAULT AS IDENTITY`` and SQLite's
   ``INTEGER PRIMARY KEY`` rowid alias don't have this gap.

Fix: extract ``_downgrade_mysql_table()`` that emits the canonical
MySQL idiom — drop FKs, then a single ALTER combining
``DROP PRIMARY KEY, ADD COLUMN id INT NOT NULL AUTO_INCREMENT,
ADD PRIMARY KEY (id)`` (which backfills existing rows with sequential
ids and preserves AUTO_INCREMENT), restore the redundant UNIQUE on
the 2 tables that originally had it, and re-add the FKs with their
original names. Postgres and SQLite keep the existing portable
``batch_alter_table`` path.

Raw SQL is unavoidable for the combined-ALTER form; per the
constitution it's allowed for dialect-specific DDL with no SQLA
equivalent, with triple-quoted strings for legibility.

Verified end-to-end: upgrade → downgrade → upgrade against a fresh
MySQL 8 container with INSERT-without-id sanity check showing the
restored ``id`` column auto-increments correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
bb996a578e fix(migration): drop FKs before recreate on MySQL (sc-105349)
CI test-mysql failed with:

  MySQLdb.OperationalError: (1826, "Duplicate foreign key constraint
  name 'fk_dashboard_slices_slice_id_slices'")

Root cause: MySQL scopes foreign-key constraint names per-database,
not per-table (PostgreSQL and SQLite scope per-table). The
``batch_alter_table(... recreate="always", copy_from=...)`` path
used for ``dashboard_slices`` and ``report_schedule_user`` builds
``_alembic_tmp_<table>`` carrying the original FK names from
``copy_from`` while the original table still holds those names — MySQL
rejects the temp-table creation with ERROR 1826.

Fix: on MySQL only, drop the original FK constraints by name before
the ``batch_alter_table`` runs. The ``copy_from`` re-creates them on
the rebuilt table with their original names, so the post-migration
shape is unchanged. On PostgreSQL and SQLite the original code path
still runs unchanged.

Local SQLite tests (44 passed, 1 skipped) still pass; CI will validate
on MySQL.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
828fd40eec refactor(migration): build pre-flight SQL via SQLAlchemy core (review)
Address Beto's review comments on apache/superset#39859: replace
``sa.text(f"...")`` SQL construction in the three pre-flight helpers
(``_delete_null_fk_rows``, ``_dedupe_by_min_id``, ``_assert_no_duplicates``)
with SQLAlchemy core constructs (``sa.delete``, ``sa.select``,
``sa.func``, ``.subquery()``, ``.notin_()``).

A small ``_table_clause()`` helper builds a lightweight ``TableClause``
exposing the columns the queries reference; the three helpers consume
it. Removes all ``# noqa: S608`` comments — they are no longer needed
because there is no string-interpolated SQL.

Verified the compiled SQL is identical on Postgres, MySQL, and SQLite,
including the MySQL ERROR 1093 workaround (the inner aggregation is
wrapped in a derived table via ``.subquery()``, producing
``... NOT IN (SELECT keep_id FROM (SELECT min(id) ...) AS keep_min)``).

Also drops the redundant ``f`` prefix on the two non-interpolating
lines of the ``_check_no_external_fks_to_id`` error message.

44 migration tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
229ed98a1b docs(migration): address SQLAlchemy review follow-ups
Four operator-experience improvements from the second review pass:

1. ``TABLES_WITH_NULLABLE_FKS`` is now explicitly documented as an
   informational set that is not consulted at runtime; the comment
   explains the previous ``dashboard_roles`` omission was the bug
   that motivated the always-run cleanup.
2. ``_delete_null_fk_rows`` docstring updated to match the
   "always run" semantics (was still claiming "called only on tables
   in TABLES_WITH_NULLABLE_FKS").
3. ``_check_no_external_fks_to_id`` now documents its scope
   limitation: ``Inspector.get_table_names()`` returns the default
   schema only, so cross-schema FKs in non-standard multi-schema
   PostgreSQL deployments would not be caught. The single-schema
   case (Superset's documented deployment) is fully covered.
4. ``_dedupe_by_min_id`` now logs a sample of up to 10 discarded
   ``(fk1, fk2, id)`` tuples at WARN before deletion, so operators
   can audit which rows the ``MIN(id)`` policy drops. The keep-
   original policy is correct in practice but discards later
   re-grants on ownership tables; the sample makes that visible.
5. ``UPDATING.md`` documents the upgrade/downgrade primary-key
   name divergence (``pk_<table>`` vs ``<table>_pkey``) so
   operators using schema-comparison tools don't mistake it for
   migration drift.

No schema or runtime-behaviour changes. All 44 migration tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
7b2c2d3b00 fix(migration): always run NULL-FK cleanup; correct RLS test parent name
Two cleanups from PR review:

1. ``dashboard_roles.dashboard_id`` was created nullable in revision
   e11ccdd12658 but was missing from ``TABLES_WITH_NULLABLE_FKS``. A
   production database with a stray NULL ``dashboard_id`` row would have
   failed the PK-add with a cryptic constraint violation. Fix by running
   the NULL-FK cleanup on every affected table — it is a no-op DELETE on
   tables whose FK columns are already NOT NULL, and it eliminates the
   risk of further drift in the hardcoded set. ``dashboard_roles`` is
   added to the documentation set; the runtime now does not consult it.

2. The unit-test parent-table name for ``rls_filter_roles`` and
   ``rls_filter_tables`` was ``rls_filter`` (does not exist) instead of
   the real parent ``row_level_security_filters``. Test passes either
   way (the in-memory FK is self-consistent), but the parameter is now
   accurate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Mike Bridge
f537472291 refactor(db): composite PK on M2M association tables (sc-105349)
Replace synthetic id INTEGER PRIMARY KEY with composite PRIMARY KEY (fk1, fk2)
on the eight pure-junction tables: dashboard_roles, dashboard_slices,
dashboard_user, report_schedule_user, rls_filter_roles, rls_filter_tables,
slice_user, sqlatable_user. The redundant UNIQUE(fk1, fk2) on dashboard_slices
and report_schedule_user is dropped (subsumed by the new PK).

Migration handles dialect quirks: copy_from for tables with pre-existing
UNIQUE (so SQLite's anonymous-constraint reflection doesn't matter), wrapped-
subquery dedupe for MySQL (ERROR 1093), sa.Identity(always=False) on downgrade
to backfill the restored id column without NOT NULL violations, and distinct
PK names per direction (pk_<table> on upgrade, <table>_pkey on downgrade) to
avoid round-trip index-name collisions on Postgres.

ORM Table() definitions updated to match. UPDATING.md entry added with
operator runbook (BI-tool impact, pre-flight inventory queries, dedupe-row-
loss notice, pg_dump workaround, FK-NOT-NULL downgrade asymmetry note).

Tests: 8 schema-shape assertions (post-upgrade), 8 duplicate-rejection unit
tests, 8 distinct-pair sanity tests, 1 round-trip + idempotency test
(in-memory SQLite via Alembic MigrationContext).

Continuum-restore verification against the new shape is out of scope for this
PR; it is the responsibility of the versioning epic (sc-103156).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 13:39:09 -06:00
Gabriel Torres Ruiz
e68251fa70 feat(mcp): support custom SQL metrics in generate_chart and update_chart (#40448) 2026-05-28 14:52:43 -03:00
Amin Ghadersohi
0dc58d1042 feat(mcp): browser hello page with working middleware and config-driven content (#40471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 11:47:38 -04:00
jesperct
c73106b7a2 fix(chart-echarts): drop white textBorder from Funnel segment labels (#40468) 2026-05-27 23:59:51 -07:00
dependabot[bot]
9441240e5c chore(deps): bump react-syntax-highlighter from 16.1.0 to 16.1.1 in /superset-frontend (#40436)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-27 15:28:56 -07:00
dependabot[bot]
08164e33bb chore(deps): bump d3-cloud from 1.2.8 to 1.2.9 in /superset-frontend (#40437)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-27 15:28:41 -07:00
dependabot[bot]
894058fe3d chore(deps): bump fs-extra from 11.3.2 to 11.3.5 in /superset-frontend (#40438)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-27 15:28:24 -07:00
dependabot[bot]
6bd1b46216 chore(deps): bump github/codeql-action from 4.35.5 to 4.36.0 (#40458)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-27 15:28:06 -07:00
Deva
ef4514f5ab fix(treemap): remove gaps between chart nodes (#40181) 2026-05-27 22:35:06 +03:00
Amin Ghadersohi
e041f25385 fix(mcp): return error when target_tab not found in add_chart_to_existing_dashboard (#40124) 2026-05-27 14:29:43 -04:00
Evan Rusackas
d744f5715c fix(dashboard-export): include and re-attach roles in import/export (#21000) (#40136)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: SBIN2010 <Sbin2010@mail.ru>
2026-05-27 10:49:07 -07:00
Amin Ghadersohi
fb60662353 chore(mcp): revert browser-friendly hello page for GET /mcp from browsers (#40467) 2026-05-27 11:43:01 -04:00
Onur Taşhan
207a7bf7f9 fix: preserve dashboard certification when saving layout changes (#40193)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 10:03:43 -04:00
Amin Ghadersohi
09a94fa26b feat(mcp): return browser-friendly hello page for GET /mcp from browsers (#40309)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 18:52:37 -07:00
Mike Bridge
7e088792b9 test(model): roll back uncommitted ds_col mutations in timestamp-expression tests (#40451)
Co-authored-by: Mike Bridge <michael.bridge@ext.preset.io>
2026-05-26 21:17:08 -03:00
Maxime Beauchemin
b6f545e61e feat(mcp): resolve call_tool proxy name and capture error_type in logging (#38915)
Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
2026-05-26 14:37:37 -04:00
Amin Ghadersohi
952a6f3a23 fix(mcp): prevent encoding error on tools/list when middleware raises (#40446) 2026-05-26 12:50:31 -04:00
dependabot[bot]
8b551d3f74 chore(deps-dev): bump duckdb from 1.4.2 to 1.5.2 (#40381)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 22:56:50 -07:00
dependabot[bot]
709ef9b615 chore(deps): bump d3-cloud from 1.2.8 to 1.2.9 in /superset-frontend (#40321)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 19:08:10 -07:00
dependabot[bot]
e9d46d843f chore(deps): bump react-map-gl from 8.1.0 to 8.1.1 in /superset-frontend (#40415)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 19:07:41 -07:00
dependabot[bot]
9cc2deb903 chore(deps): update zod requirement from ^4.4.1 to ^4.4.3 in /superset-frontend/plugins/plugin-chart-echarts (#40416)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 19:07:27 -07:00
dependabot[bot]
03d25277ba chore(deps): bump actions/upload-artifact from 7.0.0 to 7.0.1 (#40417)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 19:07:14 -07:00
dependabot[bot]
bbe2f207d2 chore(deps): bump fs-extra from 11.3.2 to 11.3.5 in /superset-frontend (#40418)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-25 19:07:00 -07:00
dependabot[bot]
c381677dfd chore(deps): bump click from 8.2.1 to 8.4.0 (#40312)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 18:55:38 -07:00
dependabot[bot]
09572cd5ef chore(deps): bump tabulate from 0.9.0 to 0.10.0 (#40315)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Evan <evan@preset.io>
2026-05-25 18:55:24 -07:00
Alexandru Soare
33585b0480 feat(mcp): make form_data_key optional in update_chart_preview (#39680) 2026-05-25 15:09:36 +03:00
Alexandru Soare
b64561f3a3 chore(mcp): Simplify chart preview response (#40020) 2026-05-25 13:16:27 +03:00
xavier-GitHub76
fe484f6bb2 fix(UserListModal): Success notification mentions user and not group (#40284) 2026-05-25 00:03:14 +03:00
dependabot[bot]
8caa74354f chore(deps): bump markdown from 3.8.1 to 3.10.2 (#40389)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 23:18:33 -07:00
Evan Rusackas
9c90a6854c ci(translations): hard-block translation regressions in CI (#39443)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-05-23 16:30:14 -07:00
Evan Rusackas
2fef4e41f2 feat(i18n): add Finnish (fi) translations (AI-generated, needs review) (#40390)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:29:44 -07:00
Jean Massucatto
965ec47296 fix(explore): hide value input for unary filter operators (#39924) 2026-05-23 11:43:49 -07:00
Evan Rusackas
b21450681d feat(i18n): add Thai (th) translations (AI-generated, needs review) (#40391)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 11:35:19 -07:00
Torsten Stöter
5003ee1499 docs: remove out-of-place phrase (#40226) 2026-05-23 13:42:52 +07:00
Abdul Rehman
816794b198 fix(frontend): handle null/undefined path in ensureAppRoot (#39940)
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-05-23 13:22:27 +07:00
dependabot[bot]
841871f1e7 chore(deps): bump qs from 6.14.2 to 6.15.2 in /superset-websocket/utils/client-ws-app (#40382)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-23 12:00:20 +07:00
dependabot[bot]
55203bbc74 chore(deps): bump qs from 6.14.2 to 6.15.2 in /docs (#40383)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-23 11:55:04 +07:00
Evan Rusackas
2fa3bbd91c chore(ci): limit /app/prefix matrix variant to master merges (#40385)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:11:21 -07:00
Evan Rusackas
168b49bf34 chore(cypress): remove dead _skip spec files and skipped inline tests (#40384)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:11:13 -07:00
Evan Rusackas
838ac8f553 fix(ci): stop cancelling Hold Label Check runs (#40380)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 21:10:23 -07:00
Evan Rusackas
42668cf634 ci(docker): pin QEMU binfmt image to stabilize arm64 builds (#40235)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-05-22 21:09:54 -07:00
Evan Rusackas
8d985d223b ci(e2e): run backend under gunicorn instead of flask dev server (#40234)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-05-22 21:09:14 -07:00
Evan Rusackas
e57387098b fix(bigquery): limit result set size to prevent browser memory crashes (#38588)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: ethan-l-geotab <ethanliong@geotab.com>
2026-05-22 21:07:45 -07:00
Evan Rusackas
af6ac4d09c feat(i18n): AI-assisted translation backfill tooling + Spanish translations (#39448)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: codeant-ai-for-open-source[bot] <244253245+codeant-ai-for-open-source[bot]@users.noreply.github.com>
Co-authored-by: Superset Dev <dev@superset.apache.org>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
Co-authored-by: Claude <claude@anthropic.com>
2026-05-22 21:07:27 -07:00
yousoph
f8e13770fc fix(dashboard): add top padding to "Create new chart" button in builder pane (#40033)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 17:37:04 -07:00
Miha Rejec
5cdd542ae5 fix(i18n): translate DateFilter tooltip for time range values (#40286)
Co-authored-by: Miha Rejec <mihar@comland.si>
2026-05-23 00:22:31 +03:00
dependabot[bot]
e40648dfcb chore(deps-dev): bump typescript from 5.9.3 to 6.0.3 in /superset-websocket (#39425)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:39:07 -07:00
dependabot[bot]
c728b4a11f chore(deps): bump sqlglot from 28.10.0 to 30.8.0 (#40186)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Claude <claude@anthropic.com>
2026-05-22 11:35:30 -07:00
dependabot[bot]
0febe32dc9 chore(deps): bump geostyler from 18.5.1 to 18.6.0 in /superset-frontend (#40323)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-05-22 11:34:30 -07:00
dependabot[bot]
2a0ebd7055 chore(deps-dev): bump ts-jest from 29.4.10 to 29.4.11 in /superset-websocket (#40363)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:34:16 -07:00
dependabot[bot]
6e23e4541d chore(deps): bump yeoman-generator from 8.1.2 to 8.2.2 in /superset-frontend (#40365)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:33:58 -07:00
dependabot[bot]
5af8fe77fa chore(deps): bump zod from 4.4.1 to 4.4.3 in /superset-frontend (#40367)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:33:44 -07:00
dependabot[bot]
3599c78a03 chore(deps): bump react-arborist from 3.6.1 to 3.7.0 in /superset-frontend (#40371)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:33:28 -07:00
dependabot[bot]
d97b5d6509 chore(deps): bump react-syntax-highlighter from 16.1.0 to 16.1.1 in /superset-frontend (#40372)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:33:12 -07:00
dependabot[bot]
869ab37f59 chore(deps): bump react-map-gl from 8.1.0 to 8.1.1 in /superset-frontend (#40366)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 11:28:14 -07:00
Joe Li
3b4892c48c fix(select): replace cached options with search results in AsyncSelect (#40039)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 11:25:03 -07:00
Đỗ Trọng Hải
91d96419fe feat(sec): delays version-bumping PR to avoid prematurely usage of compromised packages (#39783)
Signed-off-by: hainenber <dotronghai96@gmail.com>
Co-authored-by: Copilot <copilot@github.com>
2026-05-22 11:01:10 -07:00
Ali Gouta
42149f6a78 fix(chart): fix label and description translation on UI and enhance french translations (#40229)
Co-authored-by: Evan Rusackas <evan@preset.io>
2026-05-22 10:47:55 -07:00
Evan Rusackas
c945ef6763 chore(oxlint): enable import/newline-after-import + react/no-unstable-nested-components (#40319)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 10:11:38 -07:00
Beto Dealmeida
21059b54f0 feat(semantic layers): form for SL with a single SV (#40280) 2026-05-22 12:04:25 -04:00
Mehmet Salih Yavuz
8ab4695ba3 fix(mcp): use name URL param so AI-generated SQL Lab titles render (#40288) 2026-05-22 17:35:08 +03:00
dependabot[bot]
b0d26196fc chore(deps-dev): bump @swc/plugin-emotion from 14.9.0 to 14.10.0 in /superset-frontend (#40368)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 21:31:01 +07:00
dependabot[bot]
df8222ffcd chore(deps-dev): bump ts-jest from 29.4.10 to 29.4.11 in /superset-frontend (#40369)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 21:28:40 +07:00
Đỗ Trọng Hải
64f0e88de7 chore(backend/build): upgrade Gunicorn from v22 to v25 (#38788)
Signed-off-by: hainenber <dotronghai96@gmail.com>
2026-05-22 21:24:58 +07:00
Đỗ Trọng Hải
f4af6a2caf fix(docker): add missing service-worker.js into built container image (#39596)
Signed-off-by: hainenber <dotronghai96@gmail.com>
2026-05-22 21:23:21 +07:00
dependabot[bot]
31087177ab chore(deps-dev): bump webpack from 5.107.0 to 5.107.1 in /docs (#40364)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 21:21:22 +07:00
Shaitan
8e98ca6569 docs: expand out-of-scope vulnerability definitions (#40332) 2026-05-22 21:20:57 +07:00
dependabot[bot]
f7f6c29adf chore(deps-dev): bump webpack from 5.106.2 to 5.107.1 in /superset-frontend (#40370)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 21:15:44 +07:00
Miha Rejec
a94edfe418 fix(i18n): add Slovenian translation for 'Range type' in DateFilter (#40287)
Co-authored-by: Miha Rejec <mihar@comland.si>
2026-05-22 09:58:27 -04:00
David Kopelent
5549100601 feat(i18n): add missing Slovak translations (#40219) 2026-05-22 09:56:43 -04:00
Alexandru Soare
558ff4452b fix(preview): fix chart preview bugs (#40063) 2026-05-22 13:42:59 +03:00
Amin Ghadersohi
5966bb1c1e feat(mcp): add series_limit to generate_chart XY config (#40307)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 20:36:30 -04:00
chaselynisabella
ac035083d7 feat(path): support metric-based color scales & line width by metric (#39165) 2026-05-21 19:31:15 -04:00
Amin Ghadersohi
e25d708197 fix(mcp): hide write tools from users without write permissions (#40098)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 19:11:46 -04:00
dependabot[bot]
48cb3f5885 chore(deps-dev): bump baseline-browser-mapping from 2.10.29 to 2.10.31 in /superset-frontend (#40320)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-21 13:45:02 -07:00
dependabot[bot]
dcef6f8a41 chore(deps): bump react-map-gl from 8.1.0 to 8.1.1 in /superset-frontend (#40322)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-21 13:44:46 -07:00
dependabot[bot]
f09fd63495 chore(deps): bump @googleapis/sheets from 13.0.1 to 13.0.2 in /superset-frontend (#40324)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-21 13:44:05 -07:00
dependabot[bot]
bc26006a43 chore(deps-dev): update sqlalchemy-drill requirement from <2,>=1.1.4 to >=1.1.10,<2 (#40310)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-21 13:42:40 -07:00
dependabot[bot]
8b483f320e chore(deps): bump fs-extra from 11.3.2 to 11.3.5 in /superset-frontend (#40325)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-21 13:41:40 -07:00
Evan Rusackas
89c2a47433 fix(TableView): reset pagination when data reduces below current page (#34562)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-21 13:39:05 -07:00
JUST.in DO IT
b8b91574e0 fix(view query): Update style for code viewer container (#39635)
Co-authored-by: Copilot <copilot@github.com>
2026-05-21 13:37:56 -07:00
Jay Masiwal
5526464def fix(frontend): update safeStringify to surface [Circular] and DRY plugin code (#39156) 2026-05-21 13:37:05 -07:00
Evan Rusackas
73f66e4c14 fix(datasets): isolate filter state to fix concurrent /dataset race (#39685)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-05-21 11:12:32 -07:00
Elizabeth Thompson
f187a8e1c4 fix(reports): guard null dashboard height in Playwright screenshots (#40179) 2026-05-21 09:19:29 -07:00
Mehmet Salih Yavuz
4c3f65ef0b feat(mcp): make config optional in generate_explore_link (#39559) 2026-05-21 18:01:59 +03:00
Mehmet Salih Yavuz
53d8e5bdfa feat(mcp): include applied dashboard filters in get_chart_info (#39620)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 17:48:21 +03:00
Mehmet Salih Yavuz
2f95d288dd fix(mcp): eager-load dataset.metrics to prevent Excel export DetachedInstanceError (#39483)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:34:38 +03:00
Beto Dealmeida
2f5fcc21f9 fix(semantic layers): coerce filter types (#40222) 2026-05-21 09:25:27 -04:00
Mehmet Salih Yavuz
d1d07112aa feat(mcp): add find_users tool and owner filter columns for listings (#39679)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 15:59:09 +03:00
Alexandru Soare
e3711bec39 fix(recommandation): Fix chart recommandation (#39886) 2026-05-21 15:16:16 +03:00
Mehmet Salih Yavuz
ce9cab098f feat(mcp): chart formatting options across all supported chart types (#39887) 2026-05-21 15:00:32 +03:00
290 changed files with 100214 additions and 16054 deletions

16
.github/SECURITY.md vendored
View File

@@ -33,13 +33,21 @@ We kindly ask you to include the following information in your report to assist
- Expected vs. Actual Behavior: A clear description of the intended system behavior versus the observed vulnerability.
- Detailed Reproduction Steps: Clear, manual steps to reproduce the vulnerability.
**Vulnerability Definition**
Apache Superset considers a security vulnerability to be a demonstrable issue that has meaningful impact on confidentiality, integrity, or availability beyond the intended security model. Low-impact boundary variations or technical edge cases in existing access controls may be classified as hardening improvements rather than vulnerabilities, even if exploitable.
**Out of Scope Vulnerabilities**
To prioritize engineering efforts on genuine architectural risks, the following scenarios are explicitly out of scope and will not be issued a CVE:
- Attacks requiring Admin privileges: (e.g., CSS injection, template manipulation, dashboard ownership overrides, or modifying global system settings). Per the CVE vulnerability definition in CNA Operational Rules 4.1, a qualifying vulnerability must allow violation of a security policy. The Admin role is a fully trusted operational boundary defined by Apache Superset's security policy; actions within this boundary do not violate that policy and are therefore considered intended capabilities 'by design,' not vulnerabilities.
- Brute Force and Rate Limiting: Reports targeting a lack of resource exhaustion protections, generic rate-limiting, or volumetric Denial of Service (DoS) attempts.
- Theoretical attack vectors: Issues without a demonstrable, reproducible exploit path.
- Non-Exploitable Findings: Missing security headers, generic banner disclosures, or descriptive error messages that do not lead to a direct, documented exploit.
- **Attacks requiring Admin privileges**: (e.g., CSS injection, template manipulation, dashboard ownership overrides, or modifying global system settings). Per the CVE vulnerability definition in CNA Operational Rules 4.1, a qualifying vulnerability must allow violation of a security policy. The Admin role is a fully trusted operational boundary defined by Apache Superset's security policy; actions within this boundary do not violate that policy and are therefore considered intended capabilities 'by design,' not vulnerabilities.
- **Brute Force and Rate Limiting**: Reports targeting a lack of resource exhaustion protections, generic rate-limiting, or volumetric Denial of Service (DoS) attempts.
- **Theoretical attack vectors**: Issues without a demonstrable, reproducible exploit path.
- **Non-Exploitable Findings**: Missing security headers, generic banner disclosures, or descriptive error messages that do not lead to a direct, documented exploit.
- **User enumeration**: API responses, timing differences, or error messages that reveal whether user accounts, IDs, dashboards, or datasets exist.
- **Information disclosure (low impact)**: Software version disclosure, generic error messages, stack traces without sensitive data exposure, or system configuration details that don't enable further exploitation.
- **Resource exhaustion requiring authentication**: Denial of Service attacks that require valid user credentials and don't bypass rate limiting or resource controls.
- **Missing security headers**: Without demonstration of a concrete exploit scenario that leverages the missing header.
**Outcome of Reports**

View File

@@ -27,6 +27,15 @@ runs:
- name: Set up QEMU
if: ${{ inputs.build == 'true' }}
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
with:
# Pin the binfmt image to a specific QEMU release. The default
# (`tonistiigi/binfmt:latest`) is a moving target, and drift across
# QEMU's x86_64→aarch64 translator has been the proximate cause of
# intermittent `exit code: 132` (SIGILL) failures during the arm64
# leg of the multi-platform docker build — newer Node native modules
# emit instructions QEMU's user-mode emulation occasionally drops on
# the floor. Pinning a known-good release stabilises that path.
image: tonistiigi/binfmt:qemu-v8.1.5
- name: Set up Docker Buildx
if: ${{ inputs.build == 'true' }}

View File

@@ -1,7 +1,6 @@
version: 2
enable-beta-ecosystems: true
updates:
- package-ecosystem: "github-actions"
directory: "/"
ignore:
@@ -10,6 +9,8 @@ updates:
- dependency-name: anthropics/claude-code-action
schedule:
interval: "daily"
cooldown:
default-days: 5
- package-ecosystem: "npm"
ignore:
@@ -57,6 +58,8 @@ updates:
- dependabot
open-pull-requests-limit: 30
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "pip"
@@ -72,6 +75,8 @@ updates:
labels:
- pip
- dependabot
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: ".github/actions"
@@ -79,6 +84,8 @@ updates:
interval: "daily"
open-pull-requests-limit: 10
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/docs/"
@@ -102,6 +109,8 @@ updates:
interval: "daily"
open-pull-requests-limit: 10
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-websocket/"
@@ -111,6 +120,8 @@ updates:
- npm
- dependabot
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-websocket/utils/client-ws-app/"
@@ -121,6 +132,8 @@ updates:
- dependabot
open-pull-requests-limit: 10
versioning-strategy: increase
cooldown:
default-days: 5
# Now for all of our plugins and packages!
@@ -133,6 +146,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-partition/"
@@ -143,6 +158,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-world-map/"
@@ -153,6 +170,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-pivot-table/"
@@ -166,6 +185,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-chord/"
@@ -176,6 +197,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-horizon/"
@@ -186,6 +209,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-rose/"
@@ -196,6 +221,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-deckgl/"
@@ -206,6 +233,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-table/"
@@ -219,6 +248,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-country-map/"
@@ -229,6 +260,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-map-box/"
@@ -239,6 +272,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-nvd3/"
@@ -249,6 +284,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-word-cloud/"
@@ -259,6 +296,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-paired-t-test/"
@@ -269,6 +308,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-echarts/"
@@ -279,6 +320,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-ag-grid-table/"
@@ -289,6 +332,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-cartodiagram/"
@@ -299,6 +344,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-parallel-coordinates/"
@@ -309,6 +356,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-handlebars/"
@@ -323,6 +372,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/generator-superset/"
@@ -333,6 +384,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-chart-controls/"
@@ -343,6 +396,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-core/"
@@ -358,6 +413,8 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-switchboard/"
@@ -368,3 +425,5 @@ updates:
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
cooldown:
default-days: 5

View File

@@ -175,10 +175,13 @@ cypress-run-all() {
local APP_ROOT=$2
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"
# Start Flask and run it in background
# --no-debugger means disable the interactive debugger on the 500 page
# so errors can print to stderr.
local flasklog="${HOME}/flask.log"
# Start the Superset backend via gunicorn (not `flask run`). The Flask
# development server is single-threaded and has no crash-recovery, so
# heavy tests (dashboard import/export, SQL Lab) can knock it offline
# for the rest of the run — surfacing as `ECONNREFUSED` / `socket hang up`
# / `Missing CSRF token` cascades. Gunicorn gives us multiple workers,
# a request timeout, and worker-recycling under load.
local serverlog="${HOME}/superset-cypress.log"
local port=8081
CYPRESS_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
@@ -187,8 +190,58 @@ cypress-run-all() {
fi
export CYPRESS_BASE_URL
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
# Mirrors the args in docker/entrypoints/run-server.sh (1 worker × 20
# gthread threads) to keep parity with production. Multi-worker
# configurations expose timing-sensitive races in the SQL Lab → Explore
# navigation flow under E2E. We diverge from the entrypoint on:
# --timeout 120: heavy dashboard import/export specs exceed the 60s
# default
# --max-requests / --max-requests-jitter: recycle the worker under
# test load to avoid leaks accumulating across the run
# superset.app:create_app(): explicit factory so we don't depend on
# FLASK_APP being exported
nohup gunicorn \
--bind "127.0.0.1:$port" \
--workers 1 \
--worker-class gthread \
--threads 20 \
--timeout 120 \
--max-requests 500 \
--max-requests-jitter 50 \
--access-logfile - \
--error-logfile - \
"superset.app:create_app()" \
>"$serverlog" 2>&1 </dev/null &
local serverPid=$!
# Ensure the backend is cleaned up and its log is emitted even when the
# test runner fails under `set -e`.
trap '
echo "::group::gunicorn log for Cypress run"
cat "'"$serverlog"'" || true
echo "::endgroup::"
kill '"$serverPid"' 2>/dev/null || true
' EXIT
# Wait for the backend to be ready before launching Cypress; otherwise
# the first spec can race the server bind and see connection errors.
local timeout=60
say "Waiting for gunicorn server to start on port $port..."
while [ $timeout -gt 0 ]; do
if curl -f "http://localhost:${port}${APP_ROOT}/health" >/dev/null 2>&1; then
say "gunicorn server is ready"
break
fi
sleep 1
timeout=$((timeout - 1))
done
if [ $timeout -eq 0 ]; then
echo "::error::gunicorn server failed to start within 60 seconds"
echo "::group::Server startup log"
cat "$serverlog"
echo "::endgroup::"
return 1
fi
USE_DASHBOARD_FLAG=''
if [ "$USE_DASHBOARD" = "true" ]; then
@@ -200,13 +253,6 @@ cypress-run-all() {
# memoryMonitorPid=$!
python ../../scripts/cypress_run.py --parallelism $PARALLELISM --parallelism-id $PARALLEL_ID --group $PARALLEL_ID --retries 5 $USE_DASHBOARD_FLAG
# kill $memoryMonitorPid
# After job is done, print out Flask log for debugging
echo "::group::Flask log for default run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
}
playwright-install() {
@@ -224,9 +270,11 @@ playwright-run() {
local APP_ROOT=$1
local TEST_PATH=$2
# Start Flask from the project root (same as Cypress)
# Start the Superset backend via gunicorn from the project root.
# See cypress-run-all() above for the rationale — the Flask dev server
# cannot survive the dashboard import/export tests under load.
cd "$GITHUB_WORKSPACE"
local flasklog="${HOME}/flask-playwright.log"
local serverlog="${HOME}/superset-playwright.log"
local port=8081
PLAYWRIGHT_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
@@ -235,18 +283,37 @@ playwright-run() {
fi
export PLAYWRIGHT_BASE_URL
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
# See cypress-run-all() above for the args rationale (1 worker × 20
# gthread threads matching docker/entrypoints/run-server.sh, plus a
# 120s timeout and request-recycling for heavy E2E load).
nohup gunicorn \
--bind "127.0.0.1:$port" \
--workers 1 \
--worker-class gthread \
--threads 20 \
--timeout 120 \
--max-requests 500 \
--max-requests-jitter 50 \
--access-logfile - \
--error-logfile - \
"superset.app:create_app()" \
>"$serverlog" 2>&1 </dev/null &
local serverPid=$!
# Ensure cleanup on exit
trap "kill $flaskProcessId 2>/dev/null || true" EXIT
# Ensure cleanup on exit (and emit the server log on failure)
trap '
echo "::group::gunicorn log for Playwright run"
cat "'"$serverlog"'" || true
echo "::endgroup::"
kill '"$serverPid"' 2>/dev/null || true
' EXIT
# Wait for server to be ready with health check
local timeout=60
say "Waiting for Flask server to start on port $port..."
say "Waiting for gunicorn server to start on port $port..."
while [ $timeout -gt 0 ]; do
if curl -f ${PLAYWRIGHT_BASE_URL}/health >/dev/null 2>&1; then
say "Flask server is ready"
say "gunicorn server is ready"
break
fi
sleep 1
@@ -254,9 +321,9 @@ playwright-run() {
done
if [ $timeout -eq 0 ]; then
echo "::error::Flask server failed to start within 60 seconds"
echo "::group::Flask startup log"
cat "$flasklog"
echo "::error::gunicorn server failed to start within 60 seconds"
echo "::group::Server startup log"
cat "$serverlog"
echo "::endgroup::"
return 1
fi
@@ -271,7 +338,6 @@ playwright-run() {
if ! find "playwright/tests/${TEST_PATH}" -name "*.spec.ts" -type f 2>/dev/null | grep -q .; then
echo "No test files found in ${TEST_PATH} - skipping test run"
say "::endgroup::"
kill $flaskProcessId
return 0
fi
echo "Running tests: ${TEST_PATH}"
@@ -288,13 +354,6 @@ playwright-run() {
fi
say "::endgroup::"
# After job is done, print out Flask log for debugging
echo "::group::Flask log for Playwright run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
return $status
}

View File

@@ -41,7 +41,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4
uses: github/codeql-action/init@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -53,6 +53,6 @@ jobs:
- name: Perform CodeQL Analysis
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: github/codeql-action/analyze@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4
uses: github/codeql-action/analyze@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4
with:
category: "/language:${{matrix.language}}"

View File

@@ -7,10 +7,13 @@ on:
permissions:
pull-requests: read
# cancel previous workflow jobs for PRs
# Let each label event run to completion. Cancelling in-progress runs leaves
# CANCELLED entries in the PR's check-suite rollup, which poisons GitHub's
# `status:success` search filter even though all real CI passed. The job is
# a tiny no-op github-script call, so the wasted compute is negligible.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
cancel-in-progress: false
jobs:
check-hold-label:

View File

@@ -42,7 +42,7 @@ jobs:
matrix:
parallel_id: [0, 1, 2, 3, 4, 5]
browser: ["chrome"]
app_root: ["", "/app/prefix"]
app_root: ${{ github.event_name == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -161,7 +161,7 @@ jobs:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ["", "/app/prefix"]
app_root: ${{ github.event_name == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config

View File

@@ -0,0 +1,87 @@
name: Translation Regression Comment
on:
workflow_run:
workflows: ["Translations"]
types: [completed]
# This workflow posts a PR comment when the Translations workflow detects a
# regression. It uses the workflow_run trigger so that it always runs in the
# base-branch context and can safely be granted write permissions, even for
# PRs from forks.
#
# IMPORTANT: This workflow must NEVER check out code from the PR branch.
# All data comes from the artifact uploaded by the Translations workflow.
permissions:
pull-requests: write
actions: read
jobs:
post-comment:
runs-on: ubuntu-24.04
# Only act when the Translations workflow failed (which means a regression
# was detected — the workflow exits 1 on regression).
if: github.event.workflow_run.conclusion == 'failure'
steps:
- name: Download regression artifact
id: download
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: translation-regression
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
path: /tmp/translation-regression
- name: Post or update PR comment
if: steps.download.outcome == 'success'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const fs = require('fs');
const prNumberFile = '/tmp/translation-regression/pr-number.txt';
const reportFile = '/tmp/translation-regression/regression-report.md';
if (!fs.existsSync(prNumberFile) || !fs.existsSync(reportFile)) {
console.log('Artifact files not found, skipping comment.');
return;
}
const prNumber = parseInt(fs.readFileSync(prNumberFile, 'utf8').trim(), 10);
if (!prNumber) {
console.log('Could not parse PR number, skipping comment.');
return;
}
const report = fs.readFileSync(reportFile, 'utf8');
const marker = '<!-- translation-regression-bot -->';
const body = `${marker}\n${report}`;
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
});
const existing = comments.find(c => c.body && c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
console.log(`Updated existing comment ${existing.id} on PR #${prNumber}`);
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body,
});
console.log(`Created new comment on PR #${prNumber}`);
}

View File

@@ -20,6 +20,9 @@ concurrency:
jobs:
frontend-check-translations:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -51,12 +54,16 @@ jobs:
babel-extract:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
@@ -64,12 +71,85 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
uses: ./.github/actions/setup-backend/
- name: Install msgcat
run: sudo apt update && sudo apt install gettext
- name: Install gettext tools
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: sudo apt-get update && sudo apt-get install -y gettext
- name: Test babel extraction
if: steps.check.outputs.python
# Fetch the base ref so we can compare PR-introduced regressions
# against a fair baseline (also runs babel_update against the base
# source) — this isolates the PR's contribution from any pre-existing
# drift on the base branch.
- name: Fetch base ref and create comparison worktree
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: |
# For PRs use the base branch; for direct pushes compare against the previous commit.
BASE_REF="${{ github.event.pull_request.base.ref }}"
if [ -n "$BASE_REF" ]; then
git fetch --depth=1 origin "$BASE_REF"
else
git fetch --depth=2 origin "${{ github.ref }}"
fi
git worktree add /tmp/base-worktree FETCH_HEAD
# Run babel_update against BASE source + BASE translations. Any drift
# already present on the base branch (source strings that have changed
# without .po updates) shows up here as fuzzies — and will also show
# up in the PR run, so it cancels out in the comparison.
- name: Baseline — run babel_update against BASE source
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
working-directory: /tmp/base-worktree
run: ./scripts/translations/babel_update.sh
- name: Record baseline translation counts
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: |
python scripts/translations/check_translation_regression.py \
--count \
--translations-dir /tmp/base-worktree/superset/translations \
> /tmp/before.json
# Reset the PR worktree's translations to the pristine BASE state so
# both babel_update runs start from the same .po files. The only
# difference between the runs is the source code.
- name: Reset PR worktree translations to pristine BASE
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: git checkout FETCH_HEAD -- superset/translations/
- name: Run babel_update against PR source
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: ./scripts/translations/babel_update.sh
- name: Check for translation regression
id: regression
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
continue-on-error: true
run: |
python scripts/translations/check_translation_regression.py \
--compare /tmp/before.json \
--report /tmp/regression-report.md
# Save the PR number so the comment workflow can post the report without
# needing write permissions on this pull_request-triggered job.
- name: Save PR number for comment workflow
if: >-
github.event_name == 'pull_request' &&
steps.regression.outcome == 'failure'
run: echo "${{ github.event.pull_request.number }}" > /tmp/pr-number.txt
- name: Upload regression artifact
if: >-
github.event_name == 'pull_request' &&
steps.regression.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: translation-regression
path: |
/tmp/regression-report.md
/tmp/pr-number.txt
- name: Fail if regression detected
if: steps.regression.outcome == 'failure'
run: exit 1

2
.gitignore vendored
View File

@@ -115,6 +115,8 @@ release.json
superset/translations/**/messages.json
# these mo binary files are generated by `pybabel compile`
superset/translations/**/messages.mo
# cross-language index generated by scripts/translations/build_translation_index.py
superset/translations/translation_index.json
docker/requirements-local.txt

View File

@@ -50,7 +50,7 @@ repos:
hooks:
- id: check-docstring-first
- id: check-added-large-files
exclude: ^.*\.(geojson)$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$|^superset/examples/.*/data\.parquet$
exclude: ^.*\.(geojson)$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$|^superset/examples/.*/data\.parquet$|^superset/translations/.*\.po$
- id: check-yaml
exclude: ^helm/superset/templates/
- id: debug-statements

View File

@@ -53,7 +53,7 @@ extension-pkg-whitelist=pyarrow
[MESSAGES CONTROL]
disable=all
enable=json-import,disallowed-sql-import,consider-using-transaction
enable=disallowed-sql-import,consider-using-transaction
[REPORTS]

View File

@@ -202,6 +202,8 @@ RUN mkdir -p /app/data && chown -R superset:superset /app/data
# Copy compiled things from previous stages
COPY --from=superset-node /app/superset/static/assets superset/static/assets
# Copy service.worker.js optionall as it doesn't exist when DEV_MODE=true
COPY --from=superset-node /app/superset/static/service-worker.j[s] superset/static/service-worker.js
# TODO, when the next version comes out, use --exclude superset/translations
COPY superset superset

View File

@@ -24,6 +24,120 @@ assists people when migrating to a new version.
## Next
### Entity version history for charts, dashboards, and datasets
Saves of charts, dashboards, and datasets now automatically produce a version history — browsable and restorable via new API endpoints. No frontend UI in this release; the backend plumbing is the deliverable.
**New endpoints** (per entity type — same pattern for `chart`, `dashboard`, and `dataset`):
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/api/v1/{resource}/<uuid>/versions/` | List the entity's version history (0-based `version_number`, `version_uuid`, `issued_at`, `changed_by`) |
| `GET` | `/api/v1/{resource}/<uuid>/versions/<version_uuid>/` | Get a single version snapshot (scalar fields at that version; plus `columns` / `metrics` for datasets) |
| `POST` | `/api/v1/{resource}/<uuid>/versions/<version_uuid>/restore` | Restore the entity to the state captured by that version |
`<version_uuid>` is a deterministic `UUIDv5` derived from the entity's UUID and the Continuum transaction id — stable across replicas and retention pruning. Authorisation reuses the resource's existing `can_write` permission; workspace admins can list/restore any entity.
**Version response shape — `changes` array:**
Each entry returned by `GET /api/v1/{resource}/<uuid>/versions/` and `GET .../versions/<version_uuid>/` includes a `changes` array describing what changed relative to the previous version:
```json
"changes": [
{"kind": "field", "path": "slice_name", "from_value": "Old", "to_value": "New"}
]
```
The array is empty for baseline (`operation_type=0`) transactions. `kind` enumerates structured record types (`field`, layout-walker records for dashboards, dataset child diffs for `TableColumn` / `SqlMetric`); `path` is a dotted JSON-pointer-style locator; `from_value` / `to_value` are JSON-safe scalars or compact records.
**Save-response and ETag headers:**
- Save responses (`PUT /api/v1/{resource}/<pk>`) include `old_version_uuid` and `new_version_uuid` body fields so the client can correlate a save with its resulting version row.
- All entity GETs (`GET /api/v1/{chart,dashboard,dataset}/<pk>`), version-list GETs, single-version GETs, and save responses emit an `ETag: "<version_uuid>"` header reflecting the entity's current live version. The default `CORS_OPTIONS` now sets `expose_headers: ["ETag"]` so cross-origin browser clients can read the header. **No `If-Match` enforcement in v1**`ETag` is informational; concurrent-edit detection is deferred to a follow-up SIP.
- **Operators overriding `CORS_OPTIONS` in `superset_config.py` MUST include `"expose_headers": ["ETag"]`** (or merge with the default) for cross-origin clients to read the ETag. A bare `CORS_OPTIONS = {"origins": [...]}` will silently drop the expose-headers default.
**Behaviour changes on save:**
- Every save of a chart, dashboard, or dataset produces one new version row. Rows preserve the full post-save state (scalar fields for all three entity types; `TableColumn` / `SqlMetric` children for datasets; `dashboard_slices` chart membership for dashboards — children versioned via SQLAlchemy-Continuum shadow tables `table_columns_version`, `sql_metrics_version`, and `dashboard_slices_version`).
- First save after an entity already exists in the DB creates a retroactive baseline version so the UI can show "what this looked like before I edited it."
- Tags, owners, and roles are **not** versioned in v1 (ADR-005). A restore leaves those at their live values.
**New config key:**
| Key | Default | Purpose |
|---|---|---|
| `SUPERSET_VERSION_HISTORY_RETENTION_DAYS` | `30` | Versions older than this many days are pruned by a nightly Celery beat task (`superset.tasks.version_history_retention.prune_old_versions`). Each entity's live row (`end_transaction_id IS NULL`) is always preserved; closed historical rows including the baseline age out with the rest. Set to `0` to disable retention entirely. |
**Impact on external integrations:**
- New tables populated on every save — `dashboards_version`, `slices_version`, `tables_version` (parent shadow tables for the three entity types), `table_columns_version`, `sql_metrics_version`, `dashboard_slices_version` (child shadow tables), plus the shared `version_transaction` and `version_changes` tables. External tooling that queries Superset's DB directly will see writes to these tables proportional to save traffic.
- Existing entity endpoints (`GET`/`PUT /api/v1/{chart,dashboard,dataset}/<pk>`) gain an `ETag` response header and the save response gains `old_version_uuid` / `new_version_uuid` body fields. No existing fields are removed or repurposed.
- Version capture is always active — no feature flag.
### Cross-entity activity stream for charts, dashboards, and datasets
A read-only companion to the version-history endpoints (above). Each entity type gains an `/activity/` endpoint that returns a chronological stream of edits — the entity's own edits plus, for dashboards and charts, transitive edits to related entities during their association windows.
**New endpoints** (per entity type):
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/api/v1/dashboard/<uuid>/activity/` | Dashboard own edits + edits to charts attached during their dashboard window + edits to datasets those charts pointed at during their chart window |
| `GET` | `/api/v1/chart/<uuid>/activity/` | Chart own edits + edits to datasets the chart pointed at during association |
| `GET` | `/api/v1/dataset/<uuid>/activity/` | Dataset own edits only (no transitive layer in V2) |
**Query parameters** (all optional):
| Param | Type | Default | Purpose |
|---|---|---|---|
| `since` | ISO 8601 datetime | — | Lower bound on `issued_at` |
| `until` | ISO 8601 datetime | — | Upper bound on `issued_at` |
| `include` | `self` \| `related` \| `all` | `all` | Filter to only the entity's own edits, only related edits, or both |
| `page` | integer ≥ 0 | `0` | 0-based page index |
| `page_size` | integer in `[1, 200]` | `25` | Records per page (clamped silently to 200) |
**Response shape:**
```json
{
"result": [
{
"version_uuid": "...",
"entity_kind": "chart",
"entity_uuid": "...",
"entity_name": "Top 10 Girls",
"entity_deleted": false,
"entity_deletion_state": null,
"source": "related",
"transaction_id": 1234,
"issued_at": "2026-05-26T12:00:00",
"changed_by": {"id": 5, "first_name": "Mike", "last_name": "Bridge"},
"kind": "filter",
"path": ["params", "adhoc_filters", "country"],
"from_value": null,
"to_value": "US",
"summary": "Chart filter changed: Top 10 Girls",
"impact": null
}
],
"count": 47
}
```
`count` is the total record count *after* the silent permission filter (see below), not the raw query size.
**Authorisation:** reuses the resource's existing `can_write` permission. Workspace admins can read any entity's activity stream. The endpoint runs `raise_for_ownership` on the path entity — non-owners get `403`.
**Silent permission filter (AV-008):** records whose source entity the requesting user can't read are silently dropped — no placeholder, no count contribution. The frontend cannot distinguish "no activity" from "you can't see this activity."
**Tombstones (AV-009 / D-15):** when an activity record references a hard-deleted source entity, the record still appears with `entity_deleted: true`, `entity_uuid: null`, and `entity_name` recovered from the last shadow row.
**Impact on external integrations:**
- Pure read-only. No new tables, no new columns, no migrations. Reads sc-103156's shadow tables and the `version_changes` table.
- No new save-path code paths — perf-validation gate confirms the activity-view branch does not regress sc-103156's SC-004 50ms-overhead budget.
- No feature flag; the endpoints are always available once sc-103156's version-history feature is enabled.
### Granular Export Controls
A new feature flag `GRANULAR_EXPORT_CONTROLS` introduces three fine-grained permissions that replace the legacy `can_csv` permission:
@@ -310,6 +424,246 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
}
```
### Composite primary keys on many-to-many association tables
The eight M:N association tables listed below have been changed from a synthetic surrogate `id INTEGER PRIMARY KEY` to a composite `PRIMARY KEY (fk1, fk2)` on the two foreign-key columns. The `id` column is dropped, and the two tables that previously carried a redundant `UNIQUE (fk1, fk2)` constraint have that constraint removed (it is now subsumed by the composite primary key).
**Affected tables and their composite-PK column pairs:**
| Table | Composite PK |
|---|---|
| `dashboard_roles` | `(dashboard_id, role_id)` |
| `dashboard_slices` | `(dashboard_id, slice_id)` |
| `dashboard_user` | `(user_id, dashboard_id)` |
| `report_schedule_user` | `(user_id, report_schedule_id)` |
| `rls_filter_roles` | `(role_id, rls_filter_id)` |
| `rls_filter_tables` | `(table_id, rls_filter_id)` |
| `slice_user` | `(user_id, slice_id)` |
| `sqlatable_user` | `(user_id, table_id)` |
**Impact on external readers:** Any BI tool, custom report, backup script, or external integration that references these tables by their old surrogate `id` column (e.g., `SELECT id FROM dashboard_slices WHERE …`, `WHERE dashboard_slices.id IN (…)`) will break. Update such queries to project or filter on the FK pair (`dashboard_id, slice_id`) instead. The FK columns themselves are unchanged.
**Pre-flight inventory queries.** Before applying the upgrade, operators are encouraged to run the queries below against their database to assess what the migration will change. Two classes of pre-existing data are not preserved by the migration: duplicate `(fk1, fk2)` rows (the migration keeps `MIN(id)` and deletes the rest) and rows with `NULL` in either FK column (the migration deletes them, since FK columns are promoted to `NOT NULL` for the composite PK). Compliance- or audit-sensitive operators should also `\copy` (Postgres) or `SELECT … INTO OUTFILE` (MySQL) the affected rows for their own records before upgrading.
```sql
-- Duplicate (fk1, fk2) pairs (the migration will keep MIN(id) per group, delete the rest)
SELECT dashboard_id, role_id, COUNT(*) FROM dashboard_roles GROUP BY dashboard_id, role_id HAVING COUNT(*) > 1;
SELECT dashboard_id, slice_id, COUNT(*) FROM dashboard_slices GROUP BY dashboard_id, slice_id HAVING COUNT(*) > 1;
SELECT user_id, dashboard_id, COUNT(*) FROM dashboard_user GROUP BY user_id, dashboard_id HAVING COUNT(*) > 1;
SELECT user_id, report_schedule_id, COUNT(*) FROM report_schedule_user GROUP BY user_id, report_schedule_id HAVING COUNT(*) > 1;
SELECT role_id, rls_filter_id, COUNT(*) FROM rls_filter_roles GROUP BY role_id, rls_filter_id HAVING COUNT(*) > 1;
SELECT table_id, rls_filter_id, COUNT(*) FROM rls_filter_tables GROUP BY table_id, rls_filter_id HAVING COUNT(*) > 1;
SELECT user_id, slice_id, COUNT(*) FROM slice_user GROUP BY user_id, slice_id HAVING COUNT(*) > 1;
SELECT user_id, table_id, COUNT(*) FROM sqlatable_user GROUP BY user_id, table_id HAVING COUNT(*) > 1;
-- Rows with a NULL in either FK (the migration will delete these)
SELECT COUNT(*) FROM dashboard_roles WHERE dashboard_id IS NULL OR role_id IS NULL;
SELECT COUNT(*) FROM dashboard_slices WHERE dashboard_id IS NULL OR slice_id IS NULL;
SELECT COUNT(*) FROM dashboard_user WHERE user_id IS NULL OR dashboard_id IS NULL;
SELECT COUNT(*) FROM report_schedule_user WHERE user_id IS NULL OR report_schedule_id IS NULL;
SELECT COUNT(*) FROM rls_filter_roles WHERE role_id IS NULL OR rls_filter_id IS NULL;
SELECT COUNT(*) FROM rls_filter_tables WHERE table_id IS NULL OR rls_filter_id IS NULL;
SELECT COUNT(*) FROM slice_user WHERE user_id IS NULL OR slice_id IS NULL;
SELECT COUNT(*) FROM sqlatable_user WHERE user_id IS NULL OR table_id IS NULL;
```
**Sizing the maintenance window on PostgreSQL.** The queries above are dialect-portable but only count rows. Operators on PostgreSQL can run the diagnostic queries below to characterize the migration's runtime cost ahead of time: per-table row count and on-disk size, an aggregated duplicate roll-up, the external-FK pre-flight check (the migration runs the same check and aborts if it returns rows), and a rewrite-time estimate for the two tables that go through the slower full-table-rebuild path.
```sql
-- Per-table size, row count, and which migration path each will take.
-- Two tables ("dashboard_slices", "report_schedule_user") have a
-- redundant UNIQUE constraint that the migration drops via a full
-- table rewrite (op.batch_alter_table(recreate="always")). The other
-- six use direct ALTER TABLE, which is much cheaper.
WITH affected(name, has_unique) AS (
VALUES
('dashboard_roles', false),
('dashboard_slices', true),
('dashboard_user', false),
('report_schedule_user', true),
('rls_filter_roles', false),
('rls_filter_tables', false),
('slice_user', false),
('sqlatable_user', false)
)
SELECT
a.name AS table_name,
CASE WHEN a.has_unique THEN 'recreate (full rewrite)'
ELSE 'direct ALTER' END AS migration_path,
c.reltuples::bigint AS estimated_rows,
pg_size_pretty(pg_total_relation_size(c.oid)) AS total_size,
pg_size_pretty(pg_relation_size(c.oid)) AS heap_size,
pg_size_pretty(pg_indexes_size(c.oid)) AS index_size
FROM affected a
JOIN pg_class c ON c.relname = a.name AND c.relkind = 'r'
ORDER BY pg_total_relation_size(c.oid) DESC;
```
```sql
-- Aggregated duplicate-row roll-up.
-- "dup_groups" is the number of (fk1, fk2) pairs that appear more
-- than once; "rows_dropped" is the total number of rows the
-- migration will delete during the dedupe pass (it keeps MIN(id) per
-- group and discards the rest).
SELECT 'dashboard_roles' AS t, COUNT(*) AS dup_groups, SUM(c) - COUNT(*) AS rows_dropped
FROM (SELECT COUNT(*) c FROM dashboard_roles GROUP BY dashboard_id, role_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'dashboard_slices', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM dashboard_slices GROUP BY dashboard_id, slice_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'dashboard_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM dashboard_user GROUP BY user_id, dashboard_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'report_schedule_user',COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM report_schedule_user GROUP BY user_id, report_schedule_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'rls_filter_roles', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM rls_filter_roles GROUP BY role_id, rls_filter_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'rls_filter_tables', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM rls_filter_tables GROUP BY table_id, rls_filter_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'slice_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM slice_user GROUP BY user_id, slice_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'sqlatable_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM sqlatable_user GROUP BY user_id, table_id HAVING COUNT(*) > 1) g
ORDER BY rows_dropped DESC NULLS LAST;
```
```sql
-- External-FK pre-flight check.
-- The migration runs the equivalent check at upgrade time and aborts
-- if any external FK references one of the soon-to-be-removed `id`
-- columns. Running it ahead of time lets you discover (and migrate)
-- any such reference before the maintenance window. On a stock
-- Superset install this should return zero rows. (Default schema
-- only; multi-schema deployments need to broaden the lookup.)
SELECT
rc.constraint_name,
kcu.table_schema || '.' || kcu.table_name AS referencing_table,
kcu.column_name AS referencing_column,
ccu.table_name AS referenced_table,
ccu.column_name AS referenced_column
FROM information_schema.referential_constraints rc
JOIN information_schema.key_column_usage kcu
ON kcu.constraint_name = rc.constraint_name
AND kcu.constraint_schema = rc.constraint_schema
JOIN information_schema.constraint_column_usage ccu
ON ccu.constraint_name = rc.constraint_name
AND ccu.constraint_schema = rc.constraint_schema
WHERE ccu.table_name IN (
'dashboard_roles','dashboard_slices','dashboard_user',
'report_schedule_user','rls_filter_roles','rls_filter_tables',
'slice_user','sqlatable_user')
AND ccu.column_name = 'id';
```
```sql
-- Lock-window estimate for the two full-rewrite tables.
-- recreate="always" takes ACCESS EXCLUSIVE on the table for the full
-- rewrite. Use heap size combined with your hardware's effective
-- write throughput (~100-200 MB/s on commodity SSD; faster on NVMe)
-- to size the maintenance window. The other six tables use direct
-- ALTER and are dominated by composite-index build time, typically
-- seconds for tables in the low millions of rows.
SELECT
c.relname AS table_name,
pg_size_pretty(pg_relation_size(c.oid)) AS heap_size,
pg_relation_size(c.oid) / 1024 / 1024 AS heap_size_mb,
ROUND(pg_relation_size(c.oid) / 1024 / 1024 / 100.0, 1) AS est_rewrite_seconds_at_100mbs
FROM pg_class c
WHERE c.relname IN ('dashboard_slices', 'report_schedule_user');
```
**Sizing the maintenance window on MySQL.** Equivalent diagnostic queries for MySQL/InnoDB. One important difference from PostgreSQL: InnoDB rebuilds the clustered index on every PK change, so *all eight* tables undergo a full table rebuild on MySQL — not just the two that go through the explicit `recreate="always"` path. The lock-window estimate query below therefore covers all eight tables.
```sql
-- Per-table size, row count, and which migration path each will take.
-- TABLE_ROWS is an InnoDB estimate (analogous to PostgreSQL's reltuples);
-- run SELECT COUNT(*) per table for an exact count if needed.
SELECT
TABLE_NAME AS table_name,
CASE WHEN TABLE_NAME IN ('dashboard_slices', 'report_schedule_user')
THEN 'recreate (explicit, drops UNIQUE)'
ELSE 'direct ALTER (still rebuilds InnoDB clustered index)'
END AS migration_path,
TABLE_ROWS AS estimated_rows,
CONCAT(ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024, 1), ' MB') AS total_size,
CONCAT(ROUND(DATA_LENGTH / 1024 / 1024, 1), ' MB') AS heap_size,
CONCAT(ROUND(INDEX_LENGTH / 1024 / 1024, 1), ' MB') AS index_size
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME IN (
'dashboard_roles', 'dashboard_slices', 'dashboard_user',
'report_schedule_user', 'rls_filter_roles', 'rls_filter_tables',
'slice_user', 'sqlatable_user'
)
ORDER BY (DATA_LENGTH + INDEX_LENGTH) DESC;
```
```sql
-- Aggregated duplicate-row roll-up. Same SQL as the PostgreSQL version
-- (standard SQL); included here for copy-paste convenience.
SELECT 'dashboard_roles' AS t, COUNT(*) AS dup_groups, SUM(c) - COUNT(*) AS rows_dropped
FROM (SELECT COUNT(*) c FROM dashboard_roles GROUP BY dashboard_id, role_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'dashboard_slices', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM dashboard_slices GROUP BY dashboard_id, slice_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'dashboard_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM dashboard_user GROUP BY user_id, dashboard_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'report_schedule_user',COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM report_schedule_user GROUP BY user_id, report_schedule_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'rls_filter_roles', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM rls_filter_roles GROUP BY role_id, rls_filter_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'rls_filter_tables', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM rls_filter_tables GROUP BY table_id, rls_filter_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'slice_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM slice_user GROUP BY user_id, slice_id HAVING COUNT(*) > 1) g
UNION ALL SELECT 'sqlatable_user', COUNT(*), SUM(c) - COUNT(*)
FROM (SELECT COUNT(*) c FROM sqlatable_user GROUP BY user_id, table_id HAVING COUNT(*) > 1) g
ORDER BY rows_dropped DESC;
```
```sql
-- External-FK pre-flight check. KEY_COLUMN_USAGE on MySQL carries
-- both sides of the FK in a single row, so this is simpler than the
-- PostgreSQL version. Should return zero rows on a stock install.
SELECT
CONSTRAINT_NAME,
CONCAT(TABLE_SCHEMA, '.', TABLE_NAME) AS referencing_table,
COLUMN_NAME AS referencing_column,
REFERENCED_TABLE_NAME AS referenced_table,
REFERENCED_COLUMN_NAME AS referenced_column
FROM information_schema.KEY_COLUMN_USAGE
WHERE TABLE_SCHEMA = DATABASE()
AND REFERENCED_TABLE_NAME IN (
'dashboard_roles', 'dashboard_slices', 'dashboard_user',
'report_schedule_user', 'rls_filter_roles', 'rls_filter_tables',
'slice_user', 'sqlatable_user'
)
AND REFERENCED_COLUMN_NAME = 'id';
```
```sql
-- Lock-window estimate for ALL EIGHT tables (InnoDB rebuilds the
-- clustered index on PK change, so even "direct ALTER" is a rewrite).
-- ADD PRIMARY KEY is INPLACE but not LOCK=NONE — it allows concurrent
-- reads but blocks writes. Use heap size combined with your effective
-- rebuild throughput (~100-200 MB/s on commodity SSD; higher on NVMe).
SELECT
TABLE_NAME AS table_name,
CONCAT(ROUND(DATA_LENGTH / 1024 / 1024, 1), ' MB') AS heap_size,
ROUND(DATA_LENGTH / 1024 / 1024, 1) AS heap_size_mb,
ROUND(DATA_LENGTH / 1024 / 1024 / 100.0, 1) AS est_rewrite_seconds_at_100mbs
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME IN (
'dashboard_roles', 'dashboard_slices', 'dashboard_user',
'report_schedule_user', 'rls_filter_roles', 'rls_filter_tables',
'slice_user', 'sqlatable_user'
)
ORDER BY DATA_LENGTH DESC;
```
**Restoring an old `pg_dump` (or equivalent) against the new schema.** A dump taken before the migration includes `INSERT` statements that populate the now-removed `id` column. Restoring such a dump against the post-migration schema will fail. The supported workaround is to dump only the schema and reference data, then re-create the M:N associations from application data after restore — for example with `pg_dump --exclude-table-data` (or per-table `--exclude-table-data=dashboard_slices` etc.) for the eight junction tables, restore the rest, then run a one-shot script that re-INSERTs `(fk1, fk2)` pairs derived from your application export. Operators who need to restore an old dump verbatim should restore against a pre-migration Superset and then re-run the upgrade.
**Intentional downgrade asymmetry.** The migration's `downgrade()` restores the surrogate `id` column and (for `dashboard_slices` and `report_schedule_user`) the original `UNIQUE (fk1, fk2)` constraint, but it does **not** restore the original `NULL`-allowed state on the FK columns — they remain `NOT NULL`. This is intentional: under SQLAlchemy's `secondary=` semantics, a `NULL` in either FK column of a junction table is meaningless (it cannot participate in either side of the relationship). Operators downgrading are not expected to need this restored. The asymmetry is documented for completeness so that round-trip schema diffs are not mistaken for migration bugs.
**Constraint-name divergence between upgrade and downgrade.** The composite primary key created on upgrade is named `pk_<table>` (Alembic's default for `op.create_primary_key("pk_<table>", ...)`), while the surrogate `id` primary key restored on downgrade is named `<table>_pkey` (PostgreSQL's default convention for `PrimaryKeyConstraint("id")`). The two names alternate so that a round-trip (upgrade → downgrade → upgrade) does not collide on a pre-existing constraint name. Operators using schema-comparison tools (e.g. `pg_diff`, `migra`) against a downgraded database may see this as drift versus a fresh-install schema. It is cosmetic — no application code references either constraint name.
## 6.0.0
- [33055](https://github.com/apache/superset/pull/33055): Upgrades Flask-AppBuilder to 5.0.0. The AUTH_OID authentication type has been deprecated and is no longer available as an option in Flask-AppBuilder. OpenID (OID) is considered a deprecated authentication protocol - if you are using AUTH_OID, you will need to migrate to an alternative authentication method such as OAuth, LDAP, or database authentication before upgrading.
- [34871](https://github.com/apache/superset/pull/34871): Fixed Jest test hanging issue from Ant Design v5 upgrade. MessageChannel is now mocked in test environment to prevent rc-overflow from causing Jest to hang. Test environment only - no production impact.

117
docker-compose-mysql.yml Normal file
View File

@@ -0,0 +1,117 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# Compose override that swaps the default Postgres metadata DB for MySQL 8.
# Useful for evaluating dialect-specific behaviour (e.g., DDL-migration
# cost on a deployment whose production metadata DB is MySQL).
#
# Usage:
# docker compose -f docker-compose.yml -f docker-compose-mysql.yml up
# docker compose -f docker-compose.yml -f docker-compose-mysql.yml down
#
# To switch back to Postgres, just drop the second `-f` flag — the MySQL
# data lives in a separate volume (`db_home_mysql`) so neither side is
# corrupted by switching dialects.
#
# Notes:
# - Mirrors the connection settings used by CI's `test-mysql` shard:
# dialect ``mysql+mysqldb``, charset utf8mb4 with binary_prefix.
# - Host port 13306 (configurable via DATABASE_PORT_MYSQL) to avoid
# colliding with a native MySQL install on 3306.
# - The Postgres-specific init scripts under
# docker/docker-entrypoint-initdb.d/ are not mounted (they are
# postgres-only); examples / cypress fixtures still load via
# `superset-init`'s post-startup steps.
# Shared environment override applied to every Superset-side service that
# connects to the metadata DB. ``environment:`` takes precedence over the
# values inherited from the env_file in docker-compose.yml.
x-mysql-env: &mysql-env
DATABASE_DIALECT: mysql+mysqldb
DATABASE_HOST: db
DATABASE_PORT: "3306"
DATABASE_DB: superset
DATABASE_USER: superset
DATABASE_PASSWORD: superset
SQLALCHEMY_DATABASE_URI: "mysql+mysqldb://superset:superset@db:3306/superset?charset=utf8mb4&binary_prefix=true"
# Override the analytics-examples DB connection too. ``EXAMPLES_PORT``
# in docker/.env is hardcoded to 5432 (the Postgres port); without
# this override the examples connection would try MySQL on 5432 and
# fail. The examples user/DB are created by docker/mysql-init/
# examples-init.sql on first MySQL boot.
EXAMPLES_HOST: db
EXAMPLES_PORT: "3306"
EXAMPLES_DB: examples
EXAMPLES_USER: examples
EXAMPLES_PASSWORD: examples
SUPERSET__SQLALCHEMY_EXAMPLES_URI: "mysql+mysqldb://examples:examples@db:3306/examples?charset=utf8mb4&binary_prefix=true"
services:
db:
image: mysql:8.0
environment:
MYSQL_DATABASE: superset
MYSQL_USER: superset
MYSQL_PASSWORD: superset
MYSQL_ROOT_PASSWORD: root
# The original 5432 port mapping is harmless on a MySQL container
# (nothing listens on 5432 inside it) but we add 13306->3306 so the
# MySQL port is reachable from the host without colliding with a
# native MySQL on 3306. Compose merges port lists.
ports:
- "127.0.0.1:${DATABASE_PORT_MYSQL:-13306}:3306"
# Override the init-scripts mount by re-binding the same target path
# to a MySQL-compatible directory. Compose merges volume lists by
# target path; later definitions win on conflict, so this displaces
# the Postgres-specific ``./docker/docker-entrypoint-initdb.d`` mount
# from docker-compose.yml. Without this, MySQL would try to run
# ``cypress-init.sh`` (which invokes ``psql``, not in the MySQL
# image), abort the init phase, and never create the ``examples``
# database. Add the MySQL data volume separately.
volumes:
- db_home_mysql:/var/lib/mysql
- ./docker/mysql-init:/docker-entrypoint-initdb.d
command:
- --default-authentication-plugin=caching_sha2_password
- --character-set-server=utf8mb4
- --collation-server=utf8mb4_0900_ai_ci
healthcheck:
test: ["CMD-SHELL", "mysqladmin ping -h localhost -uroot -proot --silent"]
interval: 5s
timeout: 5s
retries: 20
superset:
environment: *mysql-env
superset-init:
environment: *mysql-env
superset-worker:
environment: *mysql-env
superset-worker-beat:
environment: *mysql-env
superset-node:
environment: *mysql-env
superset-tests-worker:
environment: *mysql-env
volumes:
db_home_mysql:

View File

@@ -0,0 +1,32 @@
-- Licensed to the Apache Software Foundation (ASF) under one
-- or more contributor license agreements. See the NOTICE file
-- distributed with this work for additional information
-- regarding copyright ownership. The ASF licenses this file
-- to you under the Apache License, Version 2.0 (the
-- "License"); you may not use this file except in compliance
-- with the License. You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing,
-- software distributed under the License is distributed on an
-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-- KIND, either express or implied. See the License for the
-- specific language governing permissions and limitations
-- under the License.
-- MySQL counterpart to docker/docker-entrypoint-initdb.d/examples-init.sh.
-- Creates the analytics-examples database and user that Superset's
-- ``load-examples`` command writes to. Mounted by docker-compose-mysql.yml
-- at /docker-entrypoint-initdb.d/ so the MySQL image's first-boot
-- entrypoint runs it automatically. (The Postgres init scripts under
-- docker/docker-entrypoint-initdb.d/ are NOT mounted on the MySQL
-- service — they invoke psql, which doesn't exist in the MySQL image.)
CREATE DATABASE IF NOT EXISTS examples
CHARACTER SET utf8mb4
COLLATE utf8mb4_0900_ai_ci;
CREATE USER IF NOT EXISTS 'examples'@'%' IDENTIFIED BY 'examples';
GRANT ALL PRIVILEGES ON examples.* TO 'examples'@'%';
FLUSH PRIVILEGES;

View File

@@ -88,7 +88,6 @@ using our `docker compose` constructs to support production-type use-cases. For
environments, we recommend using [minikube](https://minikube.sigs.k8s.io/docs/start/) along
our [installing on k8s](https://superset.apache.org/docs/installation/running-on-kubernetes)
documentation.
configured to be secure.
:::
### Supported environment variables

View File

@@ -335,6 +335,92 @@ npm run build-translation
pybabel compile -d superset/translations
```
### Backfilling missing translations with AI
For languages with many untranslated strings, the repo includes a script that
uses Claude AI to generate draft translations for any missing entries. All
AI-generated strings are marked `#, fuzzy` and tagged with an attribution
comment so that human reviewers know they need to be checked before merging.
#### Prerequisites
```bash
pip install -r superset/translations/requirements.txt
```
Claude Code must be installed and authenticated (`claude --version` should
work). The script calls `claude -p` internally — no separate API key is needed.
#### Step 1 — Build the translation index
The index captures every already-translated string in every language and
serves as cross-language context for the AI. Rebuild it whenever `.po` files
change significantly:
```bash
python scripts/translations/build_translation_index.py
# Writes: superset/translations/translation_index.json
```
#### Step 2 — Preview with a dry run
Check what would be translated without writing anything:
```bash
python scripts/translations/backfill_po.py --lang fr --limit 20 --dry-run
```
Output shows each string, its translation, and a context tag:
- No tag — 3+ reference languages available (high confidence)
- `[ctx:N]` — only N other languages have this string (lower confidence)
- `[ctx:0]` — no other language has this string yet; English alone used
#### Step 3 — Run the backfill
```bash
python scripts/translations/backfill_po.py --lang fr
```
Options:
| Flag | Default | Description |
|------|---------|-------------|
| `--lang LANG` | required | ISO language code (`fr`, `de`, `ja`, …) |
| `--batch-size N` | 50 | Strings per Claude request |
| `--limit N` | unlimited | Stop after N entries |
| `--min-context N` | 0 | Skip entries with fewer than N reference translations |
| `--model MODEL` | `claude-sonnet-4-6` | Claude model to use |
| `--dry-run` | off | Print without writing |
| `--no-fuzzy` | off | Don't mark entries as fuzzy |
Use `--min-context 2` to skip strings that have fewer than 2 reference
translations in other languages. Those strings are more likely to be ambiguous
(short labels, UI fragments) where the correct meaning can't be inferred
without additional context.
#### Step 4 — Review and commit
Open the target `.po` file and search for `fuzzy`. For each generated entry:
1. Verify the translation is correct for the UI context.
2. Remove the `# Machine-translated via backfill_po.py` comment and the
`#, fuzzy` flag line once you are satisfied.
3. If the translation is wrong, correct the `msgstr` before removing the flag.
4. Commit the `.po` file — do **not** commit `translation_index.json` (it is
gitignored and regenerated locally).
#### Running via npm
From `superset-frontend/`:
```bash
# Rebuild index
npm run translations:build-index
# Backfill (pass arguments after --)
npm run translations:backfill -- --lang fr --dry-run
```
## Linting
### Python

View File

@@ -110,7 +110,7 @@
"prettier": "^3.8.3",
"typescript": "~6.0.3",
"typescript-eslint": "^8.59.4",
"webpack": "^5.107.0"
"webpack": "^5.107.1"
},
"browserslist": {
"production": [

View File

@@ -12270,14 +12270,7 @@ pvutils@^1.1.3, pvutils@^1.1.5:
resolved "https://registry.yarnpkg.com/pvutils/-/pvutils-1.1.5.tgz#84b0dea4a5d670249aa9800511804ee0b7c2809c"
integrity sha512-KTqnxsgGiQ6ZAzZCVlJH5eOjSnvlyEgx1m8bkRJfOhmGRqfo5KLvmAlACQkrjEtOQ4B7wF9TdSLIs9O90MX9xA==
qs@^6.12.3:
version "6.14.2"
resolved "https://registry.yarnpkg.com/qs/-/qs-6.14.2.tgz#b5634cf9d9ad9898e31fba3504e866e8efb6798c"
integrity sha512-V/yCWTTF7VJ9hIh18Ugr2zhJMP01MY7c5kh4J870L7imm6/DIzBsNLTXzMwUA3yZ5b/KBqLx8Kp3uRvd7xSe3Q==
dependencies:
side-channel "^1.1.0"
qs@~6.15.1:
qs@^6.12.3, qs@~6.15.1:
version "6.15.2"
resolved "https://registry.yarnpkg.com/qs/-/qs-6.15.2.tgz#fd55426d710403ddccc45e0f9eab16db7727ece9"
integrity sha512-Rzq0KEyX/w/tEybncDgdkZrJgVUsUMk3xjh3t5bv3S1HTAtg+uOYt72+ZfwiQwKdysThkTBdL/rTi6HDmX9Ddw==
@@ -14964,10 +14957,10 @@ webpack-virtual-modules@^0.6.2:
resolved "https://registry.npmjs.org/webpack-virtual-modules/-/webpack-virtual-modules-0.6.2.tgz"
integrity sha512-66/V2i5hQanC51vBQKPH4aI8NMAcBW59FVBs+rC7eGHupMyfn34q7rZIE+ETlJ+XTevqfUhVVBgSUNSW2flEUQ==
webpack@^5.107.0, webpack@^5.88.1, webpack@^5.95.0:
version "5.107.0"
resolved "https://registry.yarnpkg.com/webpack/-/webpack-5.107.0.tgz#9e0d8d8baf24e76f058103f4f06ac6bb528b645a"
integrity sha512-PSxeHk/dmLYZlnTU+vL1Gej6Evg5RNtl3flhxBresfznFnzxinHMzHKloHnywM/3ouQv7/AlZCswWDIkNSggUA==
webpack@^5.107.1, webpack@^5.88.1, webpack@^5.95.0:
version "5.107.1"
resolved "https://registry.yarnpkg.com/webpack/-/webpack-5.107.1.tgz#01ad63131b7c413f607cc00a8136f467c1f10af0"
integrity sha512-mvdIWxj/H6QsfgDdH9djne3a5dYcmEmtsXGESkypaGN5jXjF/b+9KDlmTDQ2TKlFUeA2fI9Y65kihD30JOdB+Q==
dependencies:
"@types/estree" "^1.0.8"
"@types/json-schema" "^7.0.15"

View File

@@ -39,7 +39,7 @@ dependencies = [
"apache-superset-core",
"backoff>=1.8.0",
"celery>=5.3.6, <6.0.0",
"click>=8.0.3",
"click>=8.4.0",
"click-option-group",
"colorama",
"flask-cors>=6.0.0, <7.0",
@@ -58,7 +58,7 @@ dependencies = [
"flask-wtf>=1.1.0, <2.0",
"geopy",
"greenlet>=3.0.3, <=3.5.0",
"gunicorn>=22.0.0; sys_platform != 'win32'",
"gunicorn>=25.3.0, <26; sys_platform != 'win32'",
"hashids>=1.3.1, <2",
# holidays>=0.45 required for security fix
"holidays>=0.45, <1",
@@ -66,7 +66,7 @@ dependencies = [
"isodate",
"jsonpath-ng>=1.6.1, <2",
"Mako>=1.2.2",
"markdown>=3.0",
"markdown>=3.10.2",
# marshmallow>=4 has issues: https://github.com/apache/superset/issues/33162
"marshmallow>=3.0, <4",
"marshmallow-union>=0.1",
@@ -100,10 +100,11 @@ dependencies = [
"simplejson>=3.15.0",
"slack_sdk>=3.19.0, <4",
"sqlalchemy>=1.4, <2",
"sqlalchemy-continuum>=1.6.0, <2.0.0",
"sqlalchemy-utils>=0.38.0, <0.43", # expanding lowerbound to work with pydoris
"sqlglot>=28.10.0, <29",
"sqlglot>=30.8.0, <31",
# newer pandas needs 0.9+
"tabulate>=0.9.0, <1.0",
"tabulate>=0.10.0, <1.0",
"typing-extensions>=4, <5",
"waitress; sys_platform == 'win32'",
"watchdog>=6.0.0",
@@ -137,9 +138,9 @@ databricks = [
db2 = ["ibm-db-sa>0.3.8, <=0.4.4"]
denodo = ["denodo-sqlalchemy>=1.0.6,<2.1.0"]
dremio = ["sqlalchemy-dremio>=1.2.1, <4"]
drill = ["sqlalchemy-drill>=1.1.4, <2"]
drill = ["sqlalchemy-drill>=1.1.10, <2"]
druid = ["pydruid>=0.6.5,<0.7"]
duckdb = ["duckdb>=1.4.2,<2", "duckdb-engine>=0.17.0"]
duckdb = ["duckdb>=1.5.2,<2", "duckdb-engine>=0.17.0"]
dynamodb = ["pydynamodb>=0.4.2"]
solr = ["sqlalchemy-solr >= 0.2.0"]
elasticsearch = ["elasticsearch-dbapi>=0.2.13, <0.3.0"]
@@ -220,6 +221,7 @@ development = [
"openapi-spec-validator",
"parameterized",
"pip",
"polib", # used by scripts/translations/ and their unit tests
"pre-commit",
"progress>=1.5,<2",
"psutil",

View File

@@ -60,7 +60,7 @@ cffi==2.0.0
# pynacl
charset-normalizer==3.4.2
# via requests
click==8.2.1
click==8.4.1
# via
# apache-superset (pyproject.toml)
# celery
@@ -166,7 +166,7 @@ greenlet==3.1.1
# apache-superset (pyproject.toml)
# shillelagh
# sqlalchemy
gunicorn==23.0.0
gunicorn==25.3.0
# via apache-superset (pyproject.toml)
h11==0.16.0
# via wsproto
@@ -213,7 +213,7 @@ mako==1.3.11
# -r requirements/base.in
# apache-superset (pyproject.toml)
# alembic
markdown==3.8.1
markdown==3.10.2
# via apache-superset (pyproject.toml)
markdown-it-py==3.0.0
# via rich
@@ -409,19 +409,22 @@ sqlalchemy==1.4.54
# flask-sqlalchemy
# marshmallow-sqlalchemy
# shillelagh
# sqlalchemy-continuum
# sqlalchemy-utils
sqlalchemy-continuum==1.6.0
# via apache-superset (pyproject.toml)
sqlalchemy-utils==0.42.0
# via
# apache-superset (pyproject.toml)
# apache-superset-core
# flask-appbuilder
sqlglot==28.10.0
sqlglot==30.8.0
# via
# apache-superset (pyproject.toml)
# apache-superset-core
sshtunnel==0.4.0
# via apache-superset (pyproject.toml)
tabulate==0.9.0
tabulate==0.10.0
# via apache-superset (pyproject.toml)
trio==0.30.0
# via

View File

@@ -130,7 +130,7 @@ charset-normalizer==3.4.2
# via
# -c requirements/base-constraint.txt
# requests
click==8.2.1
click==8.4.1
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -219,7 +219,7 @@ docstring-parser==0.17.0
# via cyclopts
docutils==0.22.2
# via rich-rst
duckdb==1.4.2
duckdb==1.5.3
# via
# apache-superset
# duckdb-engine
@@ -388,7 +388,7 @@ grpcio==1.71.0
# grpcio-status
grpcio-status==1.60.1
# via google-api-core
gunicorn==23.0.0
gunicorn==25.3.0
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -511,7 +511,7 @@ mako==1.3.11
# -c requirements/base-constraint.txt
# alembic
# apache-superset
markdown==3.8.1
markdown==3.10.2
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -677,6 +677,8 @@ ply==3.11
# via
# -c requirements/base-constraint.txt
# jsonpath-ng
polib==1.2.0
# via apache-superset
polyline==2.0.2
# via
# -c requirements/base-constraint.txt
@@ -976,16 +978,21 @@ sqlalchemy==1.4.54
# marshmallow-sqlalchemy
# shillelagh
# sqlalchemy-bigquery
# sqlalchemy-continuum
# sqlalchemy-utils
sqlalchemy-bigquery==1.15.0
# via apache-superset
sqlalchemy-continuum==1.6.0
# via
# -c requirements/base-constraint.txt
# apache-superset
sqlalchemy-utils==0.42.0
# via
# -c requirements/base-constraint.txt
# apache-superset
# apache-superset-core
# flask-appbuilder
sqlglot==28.10.0
sqlglot==30.8.0
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -1004,7 +1011,7 @@ statsd==4.0.1
# via apache-superset
syntaqlite==0.1.0
# via apache-superset
tabulate==0.9.0
tabulate==0.10.0
# via
# -c requirements/base-constraint.txt
# apache-superset

View File

@@ -31,6 +31,7 @@ PATTERNS = {
r"^superset/",
r"^scripts/",
r"^setup\.py",
r"^pyproject\.toml$",
r"^requirements/.+\.txt",
r"^.pylintrc",
],

View File

@@ -0,0 +1,682 @@
#!/usr/bin/env python3
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# ----------------------------------------------------------------------
# Stress-test data generator for the composite-PK migration (sc-105349).
#
# Bulk-inserts synthetic parent rows and many-to-many junction rows for
# the eight association tables that the composite-PK migration touches.
# Useful for measuring migration runtime at varying scales — run this at
# 100K / 1M / 5M / 10M rows and time the migration at each scale to
# verify the O(N log N) extrapolation.
#
# Idempotent: rerunning with the same target is a no-op; rerunning with
# a higher target adds rows up to the new total. Batched bulk INSERTs
# (10K rows per statement) make it fast on Postgres, MySQL, and SQLite.
#
# Usage (inside the Superset container):
#
# docker exec superset-superset-1 \\
# /app/.venv/bin/python /app/scripts/seed_junction_load.py \\
# --dashboard-slices 1000000 \\
# --slice-user 100000 \\
# --dashboard-user 100000
#
# Run with no flags for the defaults shown below. Use ``--dry-run`` to
# print the planned inserts without writing anything.
#
# The script connects via Superset's standard ``DATABASE_*`` env vars
# (or ``SUPERSET__SQLALCHEMY_DATABASE_URI`` if set), so it works
# automatically inside the Superset container regardless of which
# metadata DB backend is in use.
from __future__ import annotations
import argparse
import logging
import os
import sys
import time
from contextlib import contextmanager
from typing import Iterator
from uuid import uuid4
import sqlalchemy as sa
from sqlalchemy.engine import Connection, Engine
logger = logging.getLogger("seed_junction_load")
# Bulk INSERT batch size. Larger values = fewer statements but more memory.
BATCH = 10_000
# Default per-junction-table target row counts. Tuned to mimic the shape
# of a large multi-team Superset install. Override via CLI flags.
DEFAULTS: dict[str, int] = {
"dashboard_slices": 1_000_000,
"slice_user": 100_000,
"dashboard_user": 100_000,
"dashboard_roles": 10_000,
}
# (junction_table, fk1_col, fk2_col, parent1_table, parent2_table)
# parents reference id columns; we generate (fk1, fk2) pairs by sampling
# from the parents' existing IDs.
JUNCTIONS: list[tuple[str, str, str, str, str]] = [
("dashboard_slices", "dashboard_id", "slice_id", "dashboards", "slices"),
("slice_user", "user_id", "slice_id", "ab_user", "slices"),
("dashboard_user", "user_id", "dashboard_id", "ab_user", "dashboards"),
("dashboard_roles", "dashboard_id", "role_id", "dashboards", "ab_role"),
]
# Junction tables that originally carried ``UNIQUE(fk1, fk2)`` and therefore
# cannot accept duplicate ``(fk1, fk2)`` pairs even on the pre-migration
# (downgrade) schema. The other JUNCTIONS allow duplicates pre-migration.
JUNCTIONS_WITH_UNIQUE: set[str] = {"dashboard_slices", "report_schedule_user"}
# ----------------------------------------------------------------------
# Connection setup
# ----------------------------------------------------------------------
def build_engine() -> Engine:
"""Build a SQLAlchemy engine from Superset env vars."""
if uri := os.environ.get("SUPERSET__SQLALCHEMY_DATABASE_URI"):
logger.info("Using SUPERSET__SQLALCHEMY_DATABASE_URI from env")
return sa.create_engine(uri)
try:
dialect = os.environ["DATABASE_DIALECT"]
user = os.environ["DATABASE_USER"]
password = os.environ["DATABASE_PASSWORD"]
host = os.environ["DATABASE_HOST"]
port = os.environ["DATABASE_PORT"]
db = os.environ["DATABASE_DB"]
except KeyError as exc:
sys.exit(
f"Missing env var {exc}; either set DATABASE_DIALECT/USER/PASSWORD/"
f"HOST/PORT/DB or SUPERSET__SQLALCHEMY_DATABASE_URI before running."
)
uri = f"{dialect}://{user}:{password}@{host}:{port}/{db}"
logger.info(
"Built URI from DATABASE_* env vars (dialect=%s, host=%s)", dialect, host
)
return sa.create_engine(uri)
# ----------------------------------------------------------------------
# Helpers
# ----------------------------------------------------------------------
def uuid_value(dialect_name: str) -> bytes | str:
"""Return a UUID in the form the active dialect expects.
MySQL stores UUIDs as ``BINARY(16)`` (16 raw bytes); Postgres has a
native ``UUID`` type that accepts strings; SQLite stores them as
BLOB/TEXT and accepts either. Branching here keeps the seed script
backend-agnostic without depending on Superset's custom column types.
"""
if dialect_name.startswith("mysql"):
return uuid4().bytes
return str(uuid4())
@contextmanager
def time_phase(name: str) -> Iterator[None]:
"""Log elapsed wall time for a named phase."""
start = time.monotonic()
logger.info("[%s] starting", name)
try:
yield
finally:
elapsed = time.monotonic() - start
logger.info("[%s] done in %.2fs", name, elapsed)
def count_rows(conn: Connection, table: str) -> int:
return conn.scalar(sa.text(f"SELECT COUNT(*) FROM {table}")) or 0 # noqa: S608
def existing_ids(conn: Connection, table: str, limit: int | None = None) -> list[int]:
sql = f"SELECT id FROM {table} ORDER BY id" # noqa: S608
if limit is not None:
sql += f" LIMIT {limit}"
return [row[0] for row in conn.execute(sa.text(sql))]
# ----------------------------------------------------------------------
# Parent seeders
#
# Each function ensures the named parent table has at least ``target``
# rows by inserting synthetic ones with minimal-but-valid columns.
# Returns nothing; subsequent code reads back IDs via ``existing_ids``.
# ----------------------------------------------------------------------
def seed_dashboards(conn: Connection, target: int, dry_run: bool) -> None:
current = count_rows(conn, "dashboards")
if current >= target:
logger.info(
"dashboards: %d rows (target %d) — no insert needed", current, target
)
return
needed = target - current
logger.info("dashboards: %d%d (+%d)", current, target, needed)
if dry_run:
return
dialect = conn.engine.dialect.name
sql = sa.text(
"INSERT INTO dashboards (uuid, dashboard_title, slug, published) "
"VALUES (:uuid, :title, :slug, :published)"
)
for batch_start in range(0, needed, BATCH):
rows = [
{
"uuid": uuid_value(dialect),
"title": f"seed_dashboard_{current + i}",
"slug": f"seed-dashboard-{current + i}-{uuid4().hex[:8]}",
"published": False,
}
for i in range(batch_start, min(batch_start + BATCH, needed))
]
conn.execute(sql, rows)
logger.info(" dashboards: inserted %d / %d", batch_start + len(rows), needed)
def seed_dbs(conn: Connection, dry_run: bool) -> int:
"""Ensure at least one row exists in ``dbs`` (parent of ``tables``).
Returns the id to use as ``database_id`` when seeding ``tables``."""
ids = existing_ids(conn, "dbs", limit=1)
if ids:
return ids[0]
if dry_run:
return -1 # placeholder
dialect = conn.engine.dialect.name
logger.info("dbs: inserting one synthetic database (no rows present)")
conn.execute(
sa.text(
"INSERT INTO dbs (uuid, database_name, sqlalchemy_uri, expose_in_sqllab) "
"VALUES (:uuid, :name, :uri, :expose)"
),
{
"uuid": uuid_value(dialect),
"name": f"seed_db_{uuid4().hex[:8]}",
"uri": "sqlite:///seed.db",
"expose": False,
},
)
return existing_ids(conn, "dbs", limit=1)[0]
def seed_tables(conn: Connection, target: int, dry_run: bool) -> None:
current = count_rows(conn, "tables")
if current >= target:
logger.info("tables: %d rows (target %d) — no insert needed", current, target)
return
needed = target - current
logger.info("tables: %d%d (+%d)", current, target, needed)
if dry_run:
return
database_id = seed_dbs(conn, dry_run=False)
dialect = conn.engine.dialect.name
sql = sa.text(
"INSERT INTO tables (uuid, table_name, database_id) "
"VALUES (:uuid, :name, :db_id)"
)
for batch_start in range(0, needed, BATCH):
rows = [
{
"uuid": uuid_value(dialect),
"name": f"seed_table_{current + i}",
"db_id": database_id,
}
for i in range(batch_start, min(batch_start + BATCH, needed))
]
conn.execute(sql, rows)
logger.info(" tables: inserted %d / %d", batch_start + len(rows), needed)
def seed_slices(conn: Connection, target: int, dry_run: bool) -> None:
current = count_rows(conn, "slices")
if current >= target:
logger.info("slices: %d rows (target %d) — no insert needed", current, target)
return
needed = target - current
logger.info("slices: %d%d (+%d)", current, target, needed)
if dry_run:
return
# Slices reference tables.id; ensure at least one ``tables`` row exists
# so the FK is satisfiable (datasource_id is nullable but we set it for
# realism). The migration test doesn't care, but a real Superset that
# re-renders these slices does.
seed_tables(conn, target=1, dry_run=False)
table_id = existing_ids(conn, "tables", limit=1)[0]
dialect = conn.engine.dialect.name
sql = sa.text(
"INSERT INTO slices "
"(uuid, slice_name, datasource_id, datasource_type, viz_type) "
"VALUES (:uuid, :name, :ds_id, :ds_type, :viz)"
)
for batch_start in range(0, needed, BATCH):
rows = [
{
"uuid": uuid_value(dialect),
"name": f"seed_slice_{current + i}",
"ds_id": table_id,
"ds_type": "table",
"viz": "table",
}
for i in range(batch_start, min(batch_start + BATCH, needed))
]
conn.execute(sql, rows)
logger.info(" slices: inserted %d / %d", batch_start + len(rows), needed)
def seed_users(conn: Connection, target: int, dry_run: bool) -> None:
current = count_rows(conn, "ab_user")
if current >= target:
logger.info("ab_user: %d rows (target %d) — no insert needed", current, target)
return
needed = target - current
logger.info("ab_user: %d%d (+%d)", current, target, needed)
if dry_run:
return
sql = sa.text(
"INSERT INTO ab_user (first_name, last_name, username, email, active) "
"VALUES (:first, :last, :username, :email, :active)"
)
for batch_start in range(0, needed, BATCH):
rows = [
{
"first": "seed",
"last": f"user_{current + i}",
"username": f"seed_user_{current + i}_{uuid4().hex[:8]}",
"email": f"seed_user_{current + i}_{uuid4().hex[:8]}@example.invalid",
"active": True,
}
for i in range(batch_start, min(batch_start + BATCH, needed))
]
conn.execute(sql, rows)
logger.info(" ab_user: inserted %d / %d", batch_start + len(rows), needed)
def seed_roles(conn: Connection, target: int, dry_run: bool) -> None:
current = count_rows(conn, "ab_role")
if current >= target:
logger.info("ab_role: %d rows (target %d) — no insert needed", current, target)
return
needed = target - current
logger.info("ab_role: %d%d (+%d)", current, target, needed)
if dry_run:
return
sql = sa.text("INSERT INTO ab_role (name) VALUES (:name)")
for batch_start in range(0, needed, BATCH):
rows = [
{"name": f"seed_role_{current + i}_{uuid4().hex[:8]}"}
for i in range(batch_start, min(batch_start + BATCH, needed))
]
conn.execute(sql, rows)
logger.info(" ab_role: inserted %d / %d", batch_start + len(rows), needed)
# ----------------------------------------------------------------------
# Junction seeder
# ----------------------------------------------------------------------
def _load_existing_pairs(
conn: Connection, junction: str, fk1_col: str, fk2_col: str
) -> set[tuple[int, int]]:
"""Load existing ``(fk1, fk2)`` pairs from a junction table into a set.
Used so the seeder can skip them when generating new pairs (junction
tables enforce uniqueness on the FK pair). Memory is ~32 bytes/tuple
on CPython, so 10M existing pairs is ~320MB — acceptable for a dev
machine. The junction / column names come from ``JUNCTIONS``, not
user input, so the f-string interpolation is safe.
"""
sql_text = f"SELECT {fk1_col}, {fk2_col} FROM {junction}" # noqa: S608
return {(row[0], row[1]) for row in conn.execute(sa.text(sql_text))}
def _generate_new_pairs(
p1_ids: list[int],
p2_ids: list[int],
existing_pairs: set[tuple[int, int]],
) -> Iterator[tuple[int, int]]:
"""Yield ``(fk1, fk2)`` pairs from the parent1 × parent2 cross-product
that are not already in ``existing_pairs``."""
for fk1 in p1_ids:
for fk2 in p2_ids:
if (fk1, fk2) not in existing_pairs:
yield (fk1, fk2)
def seed_junction(
conn: Connection,
junction: str,
fk1_col: str,
fk2_col: str,
parent1: str,
parent2: str,
target: int,
dry_run: bool,
) -> None:
"""Bulk-insert junction rows up to ``target`` rows total.
Generates ``(fk1, fk2)`` pairs by walking the cross-product of
parent1 IDs × parent2 IDs in row-major order, skipping pairs that
already exist. Walking the cross-product deterministically keeps
the script replayable: re-running with the same target is a no-op,
and re-running with a higher target appends new pairs in a stable
order regardless of how many runs preceded.
"""
current = count_rows(conn, junction)
if current >= target:
logger.info(
"%s: %d rows (target %d) — no insert needed", junction, current, target
)
return
needed = target - current
logger.info("%s: %d%d (+%d)", junction, current, target, needed)
if dry_run:
return
p1_ids = existing_ids(conn, parent1)
p2_ids = existing_ids(conn, parent2)
max_pairs = len(p1_ids) * len(p2_ids)
if max_pairs < target:
sys.exit(
f"Cannot reach {target} rows in {junction}: "
f"only {max_pairs} unique pairs available "
f"({len(p1_ids)} × {len(p2_ids)}). "
f"Increase parent targets and rerun."
)
existing_pairs: set[tuple[int, int]] = (
_load_existing_pairs(conn, junction, fk1_col, fk2_col) if current > 0 else set()
)
if existing_pairs:
logger.info(
" %s: loaded %d existing pairs into avoidance set",
junction,
len(existing_pairs),
)
insert_sql = sa.text(
f"INSERT INTO {junction} ({fk1_col}, {fk2_col}) " # noqa: S608
f"VALUES (:fk1, :fk2)"
)
inserted = 0
batch: list[dict[str, int]] = []
for fk1, fk2 in _generate_new_pairs(p1_ids, p2_ids, existing_pairs):
batch.append({"fk1": fk1, "fk2": fk2})
inserted += 1
if len(batch) == BATCH or inserted == needed:
conn.execute(insert_sql, batch)
logger.info(" %s: inserted %d / %d", junction, inserted, needed)
batch = []
if inserted == needed:
return
if inserted < needed:
sys.exit(
f"Ran out of unique pairs at {inserted}/{needed} for {junction} "
f"(parents have {len(p1_ids)} × {len(p2_ids)} = {max_pairs} pairs, "
f"{len(existing_pairs)} already present)"
)
# ----------------------------------------------------------------------
# Orchestration
# ----------------------------------------------------------------------
def required_parent_count(target_pairs: int, other_parent: int) -> int:
"""How many rows we need in this parent so that
(this_parent × other_parent) ≥ target_pairs."""
if other_parent == 0:
# Bootstrapping: assume we'll create at least 1
other_parent = 1
return -(-target_pairs // other_parent) # ceil(target_pairs / other_parent)
def _compute_parent_requirements(targets: dict[str, int]) -> dict[str, int]:
"""For each parent table, return the minimum row count needed so that
parent1 × parent2 ≥ target for every junction it participates in.
Allocates ceil(sqrt(target)) rows per parent, balanced across the two
parents of each junction. The actual junction seeder will then walk
the cross-product to produce the target number of unique pairs.
"""
parent_req: dict[str, int] = {}
for junction, _, _, p1, p2 in JUNCTIONS:
target = targets.get(junction, 0)
if target == 0:
continue
sqrt_n = int(target**0.5) + 1
parent_req[p1] = max(parent_req.get(p1, 0), sqrt_n)
parent_req[p2] = max(parent_req.get(p2, 0), sqrt_n)
return parent_req
def _seed_parents(conn: Connection, parent_req: dict[str, int], dry_run: bool) -> None:
"""Seed parent tables in dependency order:
independent parents (ab_user, ab_role) first, then dashboards / slices /
tables (which transitively depend on dbs, seeded inside seed_tables)."""
if "ab_user" in parent_req:
seed_users(conn, parent_req["ab_user"], dry_run)
if "ab_role" in parent_req:
seed_roles(conn, parent_req["ab_role"], dry_run)
if "dashboards" in parent_req:
seed_dashboards(conn, parent_req["dashboards"], dry_run)
if "slices" in parent_req:
seed_slices(conn, parent_req["slices"], dry_run)
if "tables" in parent_req:
seed_tables(conn, parent_req["tables"], dry_run)
def _seed_all_junctions(
conn: Connection, targets: dict[str, int], dry_run: bool
) -> None:
for junction, fk1, fk2, p1, p2 in JUNCTIONS:
target = targets.get(junction, 0)
if target == 0:
continue
with time_phase(f"junction:{junction}"):
seed_junction(conn, junction, fk1, fk2, p1, p2, target, dry_run)
def inject_duplicates(
conn: Connection,
junction: str,
fk1_col: str,
fk2_col: str,
pct: float,
dry_run: bool,
) -> None:
"""Insert duplicate ``(fk1, fk2)`` rows on a non-UNIQUE junction table.
Used to stress-test the migration's ``_dedupe_by_min_id`` phase, which
is otherwise a no-op on cleanly-seeded data. Computes ``count =
current_rows * pct / 100`` and inserts that many rows by re-sampling
existing ``(fk1, fk2)`` pairs in row-major order. The synthetic
duplicates land on top of distinct existing pairs (one duplicate per
distinct pair, then wraps), so the migration's dedupe finds and
deletes them.
**Pre-condition: the table must NOT have UNIQUE on (fk1, fk2)**, i.e.,
the schema must be the pre-migration shape (after running
``superset db downgrade``). On the post-migration schema the composite
PK rejects duplicates and this function will error.
"""
if pct == 0:
return
current = count_rows(conn, junction)
count = int(current * pct / 100)
if count == 0:
logger.info(
"%s: 0 duplicates to inject (current=%d, pct=%g)",
junction,
current,
pct,
)
return
logger.info(
"%s: injecting %d duplicate rows (%g%% of %d existing)",
junction,
count,
pct,
current,
)
if dry_run:
return
select_sql = sa.text(
f"SELECT {fk1_col}, {fk2_col} FROM {junction} ORDER BY id LIMIT :n" # noqa: S608
)
sample = conn.execute(select_sql, {"n": count}).fetchall()
if not sample:
logger.warning("%s: no rows to duplicate (table is empty)", junction)
return
insert_sql = sa.text(
f"INSERT INTO {junction} ({fk1_col}, {fk2_col}) " # noqa: S608
f"VALUES (:fk1, :fk2)"
)
inserted = 0
while inserted < count:
batch: list[dict[str, int]] = []
while len(batch) < BATCH and inserted < count:
row = sample[inserted % len(sample)]
batch.append({"fk1": row[0], "fk2": row[1]})
inserted += 1
conn.execute(insert_sql, batch)
logger.info(" %s: injected %d / %d duplicates", junction, inserted, count)
def _inject_dirty_data(conn: Connection, dirty_pct: float, dry_run: bool) -> None:
"""Inject duplicate rows on every non-UNIQUE seeded junction.
The two tables that originally carried ``UNIQUE(fk1, fk2)`` are
skipped because their composite-PK successor (and their pre-migration
UNIQUE constraint) both reject duplicate inserts.
"""
if dirty_pct == 0:
return
for junction, fk1, fk2, _, _ in JUNCTIONS:
if junction in JUNCTIONS_WITH_UNIQUE:
logger.info(
"%s: skipping duplicate injection (table has UNIQUE on FK pair)",
junction,
)
continue
with time_phase(f"dirty:{junction}"):
inject_duplicates(conn, junction, fk1, fk2, dirty_pct, dry_run)
def run(targets: dict[str, int], dry_run: bool, dirty_duplicates_pct: float) -> None:
engine = build_engine()
with engine.begin() as conn:
parent_req = _compute_parent_requirements(targets)
logger.info("Required parent row counts: %s", parent_req)
with time_phase("parents"):
_seed_parents(conn, parent_req, dry_run)
with time_phase("junctions"):
_seed_all_junctions(conn, targets, dry_run)
if dirty_duplicates_pct > 0:
with time_phase("dirty-duplicates"):
_inject_dirty_data(conn, dirty_duplicates_pct, dry_run)
# ----------------------------------------------------------------------
# CLI
# ----------------------------------------------------------------------
def main() -> None:
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
for table, default in DEFAULTS.items():
parser.add_argument(
f"--{table.replace('_', '-')}",
type=int,
default=default,
help=f"target row count for {table} (default: {default:,})",
)
parser.add_argument(
"--dry-run",
"-n",
action="store_true",
help="print planned inserts without writing to the DB",
)
parser.add_argument(
"--dirty-duplicates-pct",
type=float,
default=0,
help=(
"after seeding distinct pairs, inject this percentage of duplicate "
"rows on each non-UNIQUE junction (slice_user, dashboard_user, "
"dashboard_roles). Stress-tests the migration's _dedupe_by_min_id "
"phase. Requires the DB to be at the pre-migration revision "
"(33d7e0e21daa) — the post-migration composite PK rejects "
"duplicates and this will error. Default: 0 (no duplicates)."
),
)
parser.add_argument(
"--verbose",
"-v",
action="store_true",
help="increase log verbosity",
)
args = parser.parse_args()
logging.basicConfig(
level=logging.DEBUG if args.verbose else logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s",
datefmt="%H:%M:%S",
)
targets = {table: getattr(args, table) for table in DEFAULTS}
logger.info("Targets: %s", targets)
logger.info("Dry run: %s", args.dry_run)
logger.info("Dirty duplicates pct: %g", args.dirty_duplicates_pct)
with time_phase("total"):
run(
targets,
dry_run=args.dry_run,
dirty_duplicates_pct=args.dirty_duplicates_pct,
)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,653 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Backfill missing translations in a .po file using Claude AI.
For each untranslated (empty msgstr) entry in the target language, the script
sends the English source string along with all available translations in other
languages to Claude as context, then writes the AI-generated translation back
into the .po file marked as #, fuzzy for human review.
Usage:
# Build the translation index first (one-time or when .po files change)
python scripts/translations/build_translation_index.py
# Backfill French translations
python scripts/translations/backfill_po.py --lang fr
# Dry run (print what would be translated without writing)
python scripts/translations/backfill_po.py --lang de --dry-run
# Limit to 100 entries and use a specific model
python scripts/translations/backfill_po.py --lang es --limit 100 \
--model claude-opus-4-6
Options:
--lang LANG ISO language code to backfill (required)
--batch-size N Number of strings per Claude request (default: 50)
--limit N Stop after translating N entries (default: unlimited)
--min-context N Skip entries with fewer than N existing translations across
reference languages (default: 0 — translate everything)
--model MODEL Claude model ID (default: claude-sonnet-4-6)
--index PATH Path to translation_index.json (default: auto-detect)
--dry-run Print translations without writing to .po file
--no-fuzzy Do not mark generated translations as fuzzy (default: mark fuzzy)
"""
from __future__ import annotations
import argparse
import json
import re
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Any
try:
import polib # type: ignore[import-untyped]
except ImportError:
print("polib is required. Run: pip install polib", file=sys.stderr)
sys.exit(1)
TRANSLATIONS_DIR = Path(__file__).parent.parent.parent / "superset" / "translations"
DEFAULT_INDEX = TRANSLATIONS_DIR / "translation_index.json"
DEFAULT_MODEL = "claude-sonnet-4-6"
DEFAULT_BATCH_SIZE = 50
# Language names for the prompt, keyed by ISO code
LANGUAGE_NAMES: dict[str, str] = {
"ar": "Arabic",
"ca": "Catalan",
"de": "German",
"es": "Spanish",
"fa": "Persian (Farsi)",
"fr": "French",
"it": "Italian",
"ja": "Japanese",
"ko": "Korean",
"mi": "Māori",
"nl": "Dutch",
"pl": "Polish",
"pt": "Portuguese",
"pt_BR": "Brazilian Portuguese",
"ru": "Russian",
"sk": "Slovak",
"sl": "Slovenian",
"tr": "Turkish",
"uk": "Ukrainian",
"zh": "Chinese (Simplified)",
"zh_TW": "Chinese (Traditional)",
}
def _lang_name(code: str) -> str:
"""Return a human-readable language name for an ISO language code."""
return LANGUAGE_NAMES.get(code, code)
def _plural_key(msgid: str, msgid_plural: str) -> str:
"""Build the translation index key used for pluralized entries."""
return f"{msgid}\x00{msgid_plural}"
def _is_missing(entry: polib.POEntry) -> bool:
"""Return True for entries that need a translation."""
if entry.obsolete:
return False
if entry.msgid_plural:
return not any(v for v in entry.msgstr_plural.values())
return not entry.msgstr
def _context_langs(
item: dict[str, Any], index: dict[str, Any], target_lang: str
) -> list[str]:
"""Return sorted list of language codes that have translations for this entry."""
key = item["index_key"]
if key not in index:
return []
return sorted(
lang for lang, val in index[key].items() if lang != target_lang and val
)
def _context_count(
item: dict[str, Any], index: dict[str, Any], target_lang: str
) -> int:
"""Return the number of other-language translations available for this entry."""
return len(_context_langs(item, index, target_lang))
def _render_item(
i: int,
item: dict[str, Any],
index: dict[str, Any],
target_lang: str,
reference_langs_sorted: list[str],
) -> list[str]:
"""Render one batch entry as prompt lines."""
lines: list[str] = []
ctx = _context_count(item, index, target_lang)
if ctx == 0:
lines.append(
f"--- [{i}] (no reference translations — translate conservatively) ---"
)
else:
plural = "s" if ctx != 1 else ""
lines.append(f"--- [{i}] ({ctx} reference translation{plural}) ---")
lines.append(f"English: {json.dumps(item['msgid'], ensure_ascii=False)}")
if item.get("msgid_plural"):
plural_json = json.dumps(item["msgid_plural"], ensure_ascii=False)
lines.append(f"English plural: {plural_json}")
key = item["index_key"]
if key in index and reference_langs_sorted:
for lang in reference_langs_sorted:
val = index[key].get(lang)
if val is None:
continue
if isinstance(val, dict):
forms = "; ".join(
f"[{k}] {json.dumps(v, ensure_ascii=False)}" for k, v in val.items()
)
lines.append(f"{_lang_name(lang)}: {forms}")
else:
lines.append(
f"{_lang_name(lang)}: {json.dumps(val, ensure_ascii=False)}"
)
lines.append("")
return lines
def build_prompt(
target_lang: str,
batch: list[dict[str, Any]],
index: dict[str, Any],
) -> str:
"""Build the Claude prompt for a batch of entries."""
lang_name = _lang_name(target_lang)
# Collect which other languages actually have translations for this batch
reference_langs: set[str] = set()
for item in batch:
key = item["index_key"]
if key in index:
reference_langs.update(
lang for lang, val in index[key].items() if lang != target_lang and val
)
reference_langs_sorted = sorted(reference_langs)
lines: list[str] = [
"You are a professional translator specializing in software UI strings.",
f"Translate the following English strings into {lang_name} ({target_lang}).",
"",
"Rules:",
"- Preserve all format placeholders exactly: %(name)s, {name}, %s, %d, etc.",
"- Preserve HTML tags if present.",
"- Keep the same tone and register as the reference translations.",
"- For plural forms, provide translations for all plural forms"
" required by the language.",
"- Return ONLY a JSON object mapping each numeric index (as a string)"
" to its translation.",
"- Do not add any explanation, preamble, or markdown fences.",
"",
"Important: Many strings are short fragments or single words that are"
" ambiguous in English (e.g. 'Scale' could mean a measurement scale,"
" to scale an image, or fish scales). Use the translations in other"
" languages as your primary signal for which meaning is intended —"
" they collectively disambiguate the intended sense. When no"
" other-language translations are available for an entry, translate"
" conservatively based on the most common meaning in a data"
" visualization UI context.",
"",
]
if reference_langs_sorted:
lines.append(
f"Reference translations are provided per string where available "
f"({', '.join(_lang_name(lc) for lc in reference_langs_sorted)})."
)
lines.append("")
lines.append("Strings to translate:")
lines.append("")
for i, item in enumerate(batch):
lines.extend(_render_item(i, item, index, target_lang, reference_langs_sorted))
# Add guidance on plural form counts per language whenever ANY entry in
# the batch is plural — batches mix singular and plural in .po order, so
# gating on the first entry would silently drop the guidance whenever
# the plural entries happen to land after a singular one.
if any(item.get("msgid_plural") for item in batch):
lines.append(
"Note: provide ALL plural forms required by the target language "
"(e.g. French needs 2, Russian needs 3, Arabic needs 6)."
)
lines.append("")
lines.append(
'Expected output format: {"0": "<translation>", "1": "<translation>", ...}'
)
lines.append("(keys are the numeric indices of the strings above)")
return "\n".join(lines)
def parse_response(text: str, batch_size: int) -> dict[int, str]:
"""Parse the JSON object from Claude's response."""
# Strip any accidental markdown fences
text = re.sub(r"^```[^\n]*\n", "", text.strip())
text = re.sub(r"\n```$", "", text)
try:
raw = json.loads(text)
except json.JSONDecodeError as exc:
raise ValueError(
f"Could not parse response as JSON: {exc}\n\nResponse:\n{text}"
) from exc
# _process_batches only catches ValueError/RuntimeError, so a non-object
# response (list, scalar, null) must surface as ValueError rather than
# bubbling up an AttributeError from .items() and aborting the whole run.
if not isinstance(raw, dict):
raise ValueError(
f"Expected a JSON object mapping indices to translations, "
f"got {type(raw).__name__}.\n\nResponse:\n{text}"
)
# Preserve dict/list values as JSON strings so plural responses (where
# v is a dict of plural forms) can be re-parsed downstream by
# _apply_translation's json.loads. str(v) on a dict produces Python
# repr ({'0': 'x'}) which is not valid JSON.
return {
int(k): (
json.dumps(v, ensure_ascii=False) if isinstance(v, (dict, list)) else str(v)
)
for k, v in raw.items()
if str(k).isdigit()
}
def translate_batch(
model: str,
target_lang: str,
batch: list[dict[str, Any]],
index: dict[str, Any],
) -> dict[int, str]:
"""Send a batch of strings to Claude via `claude -p`.
Returns a dict mapping batch index to translated string.
"""
claude_bin = shutil.which("claude")
if not claude_bin:
raise RuntimeError(
"claude CLI not found. Install Claude Code or add it to PATH."
)
prompt = build_prompt(target_lang, batch, index)
# Pipe the prompt over stdin rather than passing it as argv: a single batch
# with many reference languages can grow into the tens of KB and approach
# ARG_MAX on some platforms.
# claude_bin is resolved via shutil.which — not user-controlled input
result = subprocess.run( # noqa: S603
[claude_bin, "--model", model, "-p"],
input=prompt,
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise RuntimeError(
f"claude exited with code {result.returncode}:\n{result.stderr}"
)
return parse_response(result.stdout.strip(), len(batch))
def _apply_plural_translation(entry: polib.POEntry, translation: str) -> None:
"""Distribute a model response across the entry's plural forms.
Model may return a JSON dict ({"0": "form0", "1": "form1"}), a JSON list
(["form0", "form1"], also valid since plural forms are ordered), a JSON
scalar (a single translation that fills every form), or a plain non-JSON
string (older models that ignore the JSON instruction).
"""
try:
plural_value = json.loads(translation)
except (json.JSONDecodeError, ValueError):
for k in entry.msgstr_plural:
entry.msgstr_plural[k] = translation
return
if isinstance(plural_value, dict):
entry.msgstr_plural = {int(k): str(v) for k, v in plural_value.items()}
return
if isinstance(plural_value, list) and plural_value:
# Distribute list items across plural form indices in order; if the
# model returned fewer forms than the language requires, repeat the
# last form rather than leaving slots blank.
forms = [str(v) for v in plural_value]
for k in sorted(entry.msgstr_plural):
entry.msgstr_plural[k] = forms[k] if k < len(forms) else forms[-1]
return
# Scalar (or empty list) — broadcast to every form.
fill = str(plural_value) if plural_value not in (None, []) else translation
for k in entry.msgstr_plural:
entry.msgstr_plural[k] = fill
def _apply_translation(
entry: polib.POEntry,
translation: str,
item: dict[str, Any],
model: str,
mark_fuzzy: bool,
) -> None:
"""Write a translation string into a POEntry and add attribution."""
if entry.msgid_plural:
_apply_plural_translation(entry, translation)
else:
entry.msgstr = translation
if mark_fuzzy and "fuzzy" not in entry.flags:
entry.flags.append("fuzzy")
refs = item["context_langs"]
refs_tag = f" [refs: {', '.join(refs)}]" if refs else " [no refs]"
attribution = f"Machine-translated via backfill_po.py ({model}){refs_tag}"
if entry.tcomment:
if attribution not in entry.tcomment:
entry.tcomment = f"{entry.tcomment}\n{attribution}"
else:
entry.tcomment = attribution
def _build_batch_items(
entries: list[polib.POEntry],
index: dict[str, Any],
lang: str,
) -> list[dict[str, Any]]:
"""Convert a list of POEntries into the dict format used by translate_batch."""
items: list[dict[str, Any]] = []
for entry in entries:
if entry.msgid_plural:
item: dict[str, Any] = {
"msgid": entry.msgid,
"msgid_plural": entry.msgid_plural,
"index_key": _plural_key(entry.msgid, entry.msgid_plural),
"is_plural": True,
}
else:
item = {
"msgid": entry.msgid,
"index_key": entry.msgid,
"is_plural": False,
}
item["context_langs"] = _context_langs(item, index, lang)
item["context_count"] = len(item["context_langs"])
items.append(item)
return items
def _process_batches(
missing: list[polib.POEntry],
index: dict[str, Any],
lang: str,
batch_size: int,
model: str,
dry_run: bool,
mark_fuzzy: bool,
cat: polib.POFile | None = None,
po_path: Path | None = None,
) -> tuple[int, int]:
"""Translate missing entries in batches. Returns (translated, failed) counts.
When ``cat`` and ``po_path`` are provided and ``dry_run`` is False, the
catalog is saved to disk after each batch that produced at least one
successful translation. This means a crash mid-run only loses the in-flight
batch rather than every batch translated so far.
"""
translated_count = 0
failed_count = 0
for batch_start in range(0, len(missing), batch_size):
batch_entries = missing[batch_start : batch_start + batch_size]
batch_items = _build_batch_items(batch_entries, index, lang)
end = min(batch_start + batch_size, len(missing))
print(
f" Translating entries {batch_start + 1}{end} of {len(missing)}",
file=sys.stderr,
)
try:
translations = translate_batch(model, lang, batch_items, index)
except (ValueError, RuntimeError) as exc:
print(f" ERROR in batch starting at {batch_start}: {exc}", file=sys.stderr)
failed_count += len(batch_entries)
continue
batch_applied = 0
for i, entry in enumerate(batch_entries):
translation = translations.get(i)
if translation is None:
print(
f" WARNING: no translation returned for index {i} "
f"(msgid: {entry.msgid[:60]!r})",
file=sys.stderr,
)
failed_count += 1
continue
if dry_run:
ctx = batch_items[i]["context_count"]
ctx_tag = f" [ctx:{ctx}]" if ctx < 3 else ""
print(
f" [{lang}]{ctx_tag} {entry.msgid[:60]!r}{translation[:60]!r}"
)
else:
_apply_translation(
entry, translation, batch_items[i], model, mark_fuzzy
)
batch_applied += 1
translated_count += 1
if (
not dry_run
and batch_applied > 0
and cat is not None
and po_path is not None
):
cat.save()
print(
f" Saved {po_path} ({batch_applied} entry(ies) in this batch).",
file=sys.stderr,
)
return translated_count, failed_count
def backfill(
lang: str,
*,
batch_size: int = DEFAULT_BATCH_SIZE,
limit: int | None = None,
min_context: int = 0,
model: str = DEFAULT_MODEL,
index_path: Path = DEFAULT_INDEX,
dry_run: bool = False,
mark_fuzzy: bool = True,
) -> None:
"""Backfill missing translations in the target language's .po file."""
# Defense against path traversal: ``lang`` lands in a filesystem path
# without further sanitization, so reject anything that isn't an
# ISO 639-1/639-2 code with an optional ISO 3166 region (e.g. ``pt_BR``).
if not re.fullmatch(r"[a-z]{2,3}(_[A-Z]{2})?", lang):
print(
f"Invalid language code: {lang!r} "
"(expected ISO 639 code, optionally with _<REGION>, e.g. 'fr' or 'pt_BR')",
file=sys.stderr,
)
sys.exit(1)
po_path = TRANSLATIONS_DIR / lang / "LC_MESSAGES" / "messages.po"
if not po_path.exists():
print(f"No .po file found for language '{lang}': {po_path}", file=sys.stderr)
sys.exit(1)
if not index_path.exists():
print(
f"Translation index not found at {index_path}.\n"
"Run: python scripts/translations/build_translation_index.py",
file=sys.stderr,
)
sys.exit(1)
print("Loading translation index …", file=sys.stderr)
with open(index_path, encoding="utf-8") as f:
index: dict[str, Any] = json.load(f)
print(f"Loading {po_path}", file=sys.stderr)
cat = polib.pofile(str(po_path))
missing: list[polib.POEntry] = [e for e in cat if e.msgid and _is_missing(e)]
print(f"Found {len(missing)} untranslated entries for '{lang}'.", file=sys.stderr)
if min_context > 0:
before = len(missing)
missing = [
e
for e in missing
if _context_count(
{
"index_key": (
_plural_key(e.msgid, e.msgid_plural)
if e.msgid_plural
else e.msgid
)
},
index,
lang,
)
>= min_context
]
skipped = before - len(missing)
print(
f"Skipping {skipped} entries with fewer than {min_context} reference "
f"translation(s) (use --min-context 0 to include them).",
file=sys.stderr,
)
if limit is not None:
missing = missing[:limit]
print(f"Limiting to {limit} entries.", file=sys.stderr)
if not missing:
print("Nothing to do.", file=sys.stderr)
return
translated_count, failed_count = _process_batches(
missing,
index,
lang,
batch_size,
model,
dry_run,
mark_fuzzy,
cat=cat,
po_path=po_path,
)
print(
f"\nDone. Translated: {translated_count}, Failed/skipped: {failed_count}.",
file=sys.stderr,
)
if not dry_run and translated_count > 0:
print(
f"Translations written to {po_path} (marked #, fuzzy for review).",
file=sys.stderr,
)
def main() -> None:
"""Parse CLI arguments and run translation backfill."""
parser = argparse.ArgumentParser(
description="Backfill missing .po translations using Claude AI",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument(
"--lang", required=True, help="ISO language code (e.g. fr, de, ja)"
)
parser.add_argument(
"--batch-size",
type=int,
default=DEFAULT_BATCH_SIZE,
help=f"Strings per Claude request (default: {DEFAULT_BATCH_SIZE})",
)
parser.add_argument(
"--limit",
type=int,
default=None,
help="Maximum number of entries to translate (default: unlimited)",
)
parser.add_argument(
"--model",
default=DEFAULT_MODEL,
help=f"Claude model ID (default: {DEFAULT_MODEL})",
)
parser.add_argument(
"--index",
type=Path,
default=DEFAULT_INDEX,
help=f"Path to translation_index.json (default: {DEFAULT_INDEX})",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Print translations without modifying the .po file",
)
parser.add_argument(
"--min-context",
type=int,
default=0,
metavar="N",
help=(
"Skip entries with fewer than N reference translations in other languages "
"(default: 0 = translate everything). Strings with low context are more "
"likely to be ambiguous single words or fragments — set to e.g. 2 to only "
"translate strings that have been confirmed in at least 2 other languages."
),
)
parser.add_argument(
"--no-fuzzy",
dest="mark_fuzzy",
action="store_false",
default=True,
help=(
"Do not mark generated translations as #, fuzzy. "
"WARNING: fuzzy entries are excluded from compiled .mo files. "
"Removing this flag causes AI-generated translations to be served "
"to end users without human review — only use after you have "
"manually verified the .po file."
),
)
args = parser.parse_args()
backfill(
lang=args.lang,
batch_size=args.batch_size,
limit=args.limit,
min_context=args.min_context,
model=args.model,
index_path=args.index,
dry_run=args.dry_run,
mark_fuzzy=args.mark_fuzzy,
)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,153 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Build a cross-language translation index from all .po files.
Outputs a JSON file structured as:
{
"<msgid>": {
"<lang>": "<translated string or null>",
...
},
...
}
For plural entries the key is "<msgid>\x00<msgid_plural>" and the value
is a dict mapping lang -> {0: "...", 1: "..."} (or null if untranslated).
Usage:
python scripts/translations/build_translation_index.py
python scripts/translations/build_translation_index.py \
--translations-dir superset/translations \
--output /tmp/translation_index.json
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from typing import Any
try:
import polib # type: ignore[import-untyped]
except ImportError:
print("polib is required. Install with: pip install polib", file=sys.stderr)
sys.exit(1)
TRANSLATIONS_DIR = Path(__file__).parent.parent.parent / "superset" / "translations"
DEFAULT_OUTPUT = (
Path(__file__).parent.parent.parent
/ "superset"
/ "translations"
/ "translation_index.json"
)
def _is_translated(entry: polib.POEntry) -> bool:
"""Return True if the entry has a non-empty, non-fuzzy translation."""
if "fuzzy" in entry.flags:
return False
if entry.msgid_plural:
return any(v for v in entry.msgstr_plural.values())
return bool(entry.msgstr)
def _plural_key(entry: polib.POEntry) -> str:
"""Build the combined key used for plural translation entries."""
return f"{entry.msgid}\x00{entry.msgid_plural}"
def build_index(translations_dir: Path) -> dict[str, Any]:
"""Read all .po files and build a combined translation index."""
index: dict[str, dict[str, Any]] = {}
langs = sorted(
d
for d in os.listdir(translations_dir)
if (translations_dir / d / "LC_MESSAGES" / "messages.po").exists()
and d != "en" # en has empty msgstr by convention (source = target)
)
for lang in langs:
po_path = translations_dir / lang / "LC_MESSAGES" / "messages.po"
cat = polib.pofile(str(po_path))
for entry in cat:
if not entry.msgid:
continue # skip header entry
if entry.msgid_plural:
key = _plural_key(entry)
if key not in index:
index[key] = {}
# Fuzzy entries are unreviewed (often machine-generated drafts),
# so excluding them prevents feeding unverified translations
# back into the AI backfill prompt as trusted context.
index[key][lang] = (
dict(entry.msgstr_plural) if _is_translated(entry) else None
)
else:
key = entry.msgid
if key not in index:
index[key] = {}
index[key][lang] = entry.msgstr if _is_translated(entry) else None
# Ensure every entry has a slot for every language (null if missing)
for key in index:
for lang in langs:
index[key].setdefault(lang, None)
return index
def main() -> None:
"""Parse arguments, build the translation index, and write it to disk."""
parser = argparse.ArgumentParser(
description="Build cross-language translation index"
)
parser.add_argument(
"--translations-dir",
type=Path,
default=TRANSLATIONS_DIR,
help="Path to the translations directory (default: superset/translations)",
)
parser.add_argument(
"--output",
"-o",
type=Path,
default=DEFAULT_OUTPUT,
help=(
"Output JSON file path"
" (default: superset/translations/translation_index.json)"
),
)
args = parser.parse_args()
print(f"Reading .po files from {args.translations_dir}", file=sys.stderr)
index = build_index(args.translations_dir)
print(f"Indexed {len(index)} message IDs.", file=sys.stderr)
args.output.parent.mkdir(parents=True, exist_ok=True)
with open(args.output, "w", encoding="utf-8") as f:
json.dump(index, f, ensure_ascii=False, indent=2)
print(f"Written to {args.output}", file=sys.stderr)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,250 @@
#!/usr/bin/env python3
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Check that source-code changes don't cause translation regressions.
Usage
-----
Count non-fuzzy translated entries in all .po files and write JSON to stdout:
python check_translation_regression.py --count
Compare the current .po state against a previously-recorded baseline and fail
if any language lost translations:
python check_translation_regression.py --compare /path/to/before.json
Optionally write a markdown report to a file (used by CI to post a PR comment):
python check_translation_regression.py --compare before.json --report report.md
Use a translations directory other than the repo default (used by CI to count
against a separate base-branch worktree):
python check_translation_regression.py --count \\
--translations-dir /tmp/base-worktree/superset/translations
Typical CI workflow
-------------------
1. Create a base-branch worktree alongside the PR worktree
2. Run babel_update.sh in the base worktree (extract from BASE source)
3. Record baseline: python ... --count --translations-dir BASE_TREE > before.json
4. Run babel_update.sh in the PR worktree (extract from PR source) starting
from the same pristine BASE translations
5. Compare: python ... --compare before.json [--report report.md]
Comparing two babel_update outputs that started from the same BASE .po files
isolates regressions caused by the PR's source diff from any pre-existing
drift on the base branch.
"""
import argparse
import json
import re
import subprocess
import sys
from pathlib import Path
from typing import Optional
DEFAULT_TRANSLATIONS_DIR = (
Path(__file__).resolve().parent.parent.parent / "superset" / "translations"
)
# English .po files use empty msgstr by convention (source language == target),
# so they always show 0 translated entries and should not be checked.
SKIP_LANGS = {"en"}
def count_translated(po_file: Path) -> int:
"""Return the number of non-fuzzy translated messages in a .po file.
Raises:
subprocess.CalledProcessError: if ``msgfmt`` fails (e.g. malformed
.po file). The regression check exists to surface translation
problems, so a silent zero would defeat its purpose — let the
caller see a malformed file as a hard failure.
"""
import shutil # noqa: PLC0415
msgfmt = shutil.which("msgfmt") or "msgfmt"
result = subprocess.run( # noqa: S603
[msgfmt, "--statistics", "-o", "/dev/null", str(po_file)],
capture_output=True,
text=True,
check=True,
)
# stderr: "123 translated messages, 4 fuzzy translations, 56 untranslated messages."
match = re.search(r"(\d+) translated message", result.stderr)
if not match:
raise RuntimeError(
f"Could not parse msgfmt --statistics output for {po_file}: "
f"{result.stderr!r}"
)
return int(match.group(1))
def get_counts(translations_dir: Path) -> dict[str, int]:
counts: dict[str, int] = {}
for po_file in sorted(translations_dir.glob("*/LC_MESSAGES/messages.po")):
lang = po_file.parent.parent.name
if lang in SKIP_LANGS:
continue
try:
counts[lang] = count_translated(po_file)
except (subprocess.CalledProcessError, RuntimeError) as exc:
# A malformed .po file (msgfmt non-zero exit, or stderr we
# can't parse) is a real problem worth seeing, but it shouldn't
# take the whole regression check down with it — that would
# hide every other language's status. Skip and warn instead;
# the missing lang will not appear in the comparison output.
print(
f"WARNING: skipping {lang}{po_file} could not be counted: {exc}",
file=sys.stderr,
)
return counts
def build_regression_report(regressions: list[tuple[str, int, int]]) -> str:
"""Build a markdown report for posting as a PR comment."""
rows = "\n".join(
f"| `{lang}` | {b} | {a} | -{b - a} |" for lang, b, a in regressions
)
affected = ", ".join(f"`{lang}`" for lang, _, _ in regressions)
return (
"## ⚠️ Translation Regression Detected\n\n"
f"This PR causes existing translations to become fuzzy or be removed "
f"in {affected}. Please fix the affected `.po` files before merging.\n\n"
"| Language | Before | After | Lost |\n"
"|----------|-------:|------:|-----:|\n"
f"{rows}\n\n"
"### How to fix\n\n"
"**1. Install dependencies** (if not already set up):\n\n"
"```bash\n"
"pip install -r superset/translations/requirements.txt\n"
"sudo apt-get install gettext # or: brew install gettext\n"
"```\n\n"
"**2. Re-extract strings and sync `.po` files:**\n\n"
"```bash\n"
"./scripts/translations/babel_update.sh\n"
"```\n\n"
"This rewrites `superset/translations/messages.pot` from the current "
"source files and merges the changes into every `.po` file. Strings "
"whose `msgid` changed will be marked `#, fuzzy`.\n\n"
f"**3. Resolve the fuzzy entries** in the affected language files "
f"({affected}):\n\n"
"```bash\n"
"grep -n '#, fuzzy' superset/translations/<lang>/LC_MESSAGES/messages.po\n"
"```\n\n"
"For each fuzzy entry, either rewrite the `msgstr` to match the new "
"string and remove the `#, fuzzy` line, or clear the `msgstr` to "
'`""` if you cannot provide a translation.\n\n'
"**4. Commit your changes to the `.po` files.**\n"
)
def cmd_count(translations_dir: Path) -> None:
counts = get_counts(translations_dir)
print(json.dumps(counts, indent=2))
def cmd_compare(
before_path: str,
translations_dir: Path,
report_path: Optional[str] = None,
) -> None:
with open(before_path) as f:
before: dict[str, int] = json.load(f)
after = get_counts(translations_dir)
regressions: list[tuple[str, int, int]] = []
for lang, before_count in sorted(before.items()):
after_count = after.get(lang, 0)
if after_count < before_count:
regressions.append((lang, before_count, after_count))
if regressions:
print("Translation regression detected!\n")
for lang, b, a in regressions:
lost = b - a
print(f" {lang}: {b} -> {a} (-{lost} string(s) became fuzzy or removed)")
print(
"\nStrings renamed or deleted by this PR invalidated existing translations."
)
print(
"Update the affected .po files to restore the lost entries before merging."
)
if report_path:
Path(report_path).write_text(
build_regression_report(regressions), encoding="utf-8"
)
sys.exit(1)
# All good — print a summary so it's easy to read in CI logs.
print("No translation regressions.\n")
for lang in sorted(after):
b = before.get(lang, 0)
a = after[lang]
if a > b:
delta = f"+{a - b}"
elif a == b:
delta = "no change"
else:
delta = f"-{b - a}"
print(f" {lang}: {b} -> {a} ({delta})")
def main() -> None:
parser = argparse.ArgumentParser(
description="Check for translation regressions in .po files."
)
action = parser.add_mutually_exclusive_group(required=True)
action.add_argument(
"--count",
action="store_true",
help="Output translation counts per language as JSON.",
)
action.add_argument(
"--compare",
metavar="BEFORE_JSON",
help="Compare current counts against a baseline JSON file.",
)
parser.add_argument(
"--report",
metavar="REPORT_MD",
help="When --compare detects regressions, write a markdown report here.",
)
parser.add_argument(
"--translations-dir",
type=Path,
default=DEFAULT_TRANSLATIONS_DIR,
help=(
"Path to the translations directory containing per-language "
"LC_MESSAGES/messages.po files (default: <repo>/superset/translations)."
),
)
args = parser.parse_args()
if args.count:
cmd_count(args.translations_dir)
else:
cmd_compare(args.compare, args.translations_dir, args.report)
if __name__ == "__main__":
main()

View File

@@ -48,7 +48,7 @@ dependencies = [
"pydantic>=2.8.0",
"sqlalchemy>=1.4.0,<2.0",
"sqlalchemy-utils>=0.38.0, <0.43", # expanding lowerbound to work with pydoris
"sqlglot>=28.10.0, <29",
"sqlglot>=30.8.0, <31",
"typing-extensions>=4.0.0",
]

View File

@@ -1,100 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
waitForChartLoad,
ChartSpec,
getChartAliasesBySpec,
} from 'cypress/utils';
import { WORLD_HEALTH_DASHBOARD } from 'cypress/utils/urls';
import { WORLD_HEALTH_CHARTS } from './utils';
import { isLegacyResponse } from '../../utils/vizPlugins';
describe('Dashboard top-level controls', () => {
beforeEach(() => {
cy.visit(WORLD_HEALTH_DASHBOARD);
});
// flaky test - query completes before assertion
it.skip('should allow chart level refresh', () => {
const mapSpec = WORLD_HEALTH_CHARTS.find(
({ viz }) => viz === 'world_map',
) as ChartSpec;
waitForChartLoad(mapSpec).then(gridComponent => {
const mapId = gridComponent.attr('data-test-chart-id');
cy.get('[data-test="grid-container"]').find('.world_map').should('exist');
cy.get(`#slice_${mapId}-controls`).click();
cy.get(`[data-test="slice_${mapId}-menu"]`)
.find('[data-test="refresh-chart-menu-item"]')
.click({ force: true });
// likely cause for flakiness:
// The query completes before this assertion happens.
// Solution: pause the network before clicking, assert, then unpause network.
cy.get('[data-test="refresh-chart-menu-item"]').should(
'have.class',
'ant-dropdown-menu-item-disabled',
);
waitForChartLoad(mapSpec);
cy.get('[data-test="refresh-chart-menu-item"]').should(
'not.have.class',
'ant-dropdown-menu-item-disabled',
);
});
});
it('should allow dashboard level force refresh', () => {
// when charts are not start loading, for example, under a secondary tab,
// should allow force refresh
WORLD_HEALTH_CHARTS.forEach(waitForChartLoad);
getChartAliasesBySpec(WORLD_HEALTH_CHARTS).then(aliases => {
cy.get('[aria-label="ellipsis"]').click();
cy.get('[data-test="refresh-dashboard-menu-item"]').should(
'not.have.class',
'ant-dropdown-menu-item-disabled',
);
cy.get('[data-test="refresh-dashboard-menu-item"]').click({
force: true,
});
cy.get('[data-test="refresh-dashboard-menu-item"]').should(
'have.class',
'ant-dropdown-menu-item-disabled',
);
// wait all charts force refreshed.
cy.wait(aliases).then(xhrs => {
xhrs.forEach(async ({ response, request }) => {
const responseBody = response?.body;
const isCached = isLegacyResponse(responseBody)
? responseBody.is_cached
: responseBody.result[0].is_cached;
// request url should indicate force-refresh operation
expect(request.url).to.have.string('force=true');
// is_cached in response should be false
expect(isCached).to.equal(false);
});
});
});
cy.get('[aria-label="ellipsis"]').click();
cy.get('[data-test="refresh-dashboard-menu-item"]').and(
'not.have.class',
'ant-dropdown-menu-item-disabled',
);
});
});

View File

@@ -1,292 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { nativeFilters } from 'cypress/support/directories';
import {
addCountryNameFilter,
applyNativeFilterValueWithIndex,
enterNativeFilterEditModal,
inputNativeFilterDefaultValue,
saveNativeFilterSettings,
validateFilterNameOnDashboard,
testItems,
interceptFilterState,
} from './utils';
import {
prepareDashboardFilters,
SAMPLE_CHART,
visitDashboard,
} from './shared_dashboard_functions';
function openMoreFilters(waitFilterState = true) {
interceptFilterState();
// Wait for the dropdown button to appear when filters are overflowed
// The button only appears when there are overflowed filters
cy.getBySel('dropdown-container-btn', { timeout: 10000 })
.should('exist')
.should('be.visible')
.click({ force: true });
if (waitFilterState) {
cy.wait('@postFilterState');
}
}
function openVerticalFilterBar() {
cy.getBySel('dashboard-filters-panel').should('exist');
cy.getBySel('filter-bar__expand-button').click();
}
function setFilterBarOrientation(orientation: 'vertical' | 'horizontal') {
cy.getBySel('filterbar-orientation-icon').click();
cy.wait(250);
cy.get('.filter-bar-orientation-submenu')
.contains('Orientation of filter bar')
.should('exist')
.trigger('mouseover');
if (orientation === 'vertical') {
cy.get('.ant-dropdown-menu-item-selected')
.contains('Horizontal (Top)')
.should('exist');
cy.get('.ant-dropdown-menu-item').contains('Vertical (Left)').click();
cy.getBySel('dashboard-filters-panel').should('exist');
} else {
cy.get('.ant-dropdown-menu-item-selected')
.contains('Vertical (Left)')
.should('exist');
cy.get('.ant-dropdown-menu-item').contains('Horizontal (Top)').click();
cy.getBySel('loading-indicator').should('exist');
cy.getBySel('filter-bar').should('exist');
cy.getBySel('dashboard-filters-panel').should('not.exist');
}
}
describe('Horizontal FilterBar', () => {
it('should go from vertical to horizontal and the opposite', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
setFilterBarOrientation('vertical');
});
it('should show all default actions in horizontal mode', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
cy.getBySel('horizontal-filterbar-empty')
.contains('No filters are currently added to this dashboard.')
.should('exist');
cy.get(nativeFilters.filtersPanel.filterGear).click({
force: true,
});
cy.get('.ant-dropdown-menu').should('be.visible');
cy.getBySel('filter-bar__create-filter').should('exist');
cy.getBySel('filterbar-action-buttons').should('exist');
});
it('should stay in horizontal mode when reloading', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
cy.reload();
cy.getBySel('dashboard-filters-panel').should('not.exist');
});
it('should show all filters in available space on load', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
{ name: 'test_3', column: 'region', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
cy.get('.filter-item-wrapper').should('have.length', 3);
});
it('should show "more filters" on window resizing up and down', () => {
// Use 4 filters with unique columns to ensure overflow testing while allowing all to fit at large viewport
prepareDashboardFilters([
{ name: 'Country', column: 'country_name', datasetId: 2 },
{ name: 'Code', column: 'country_code', datasetId: 2 },
{ name: 'Region', column: 'region', datasetId: 2 },
{ name: 'Year', column: 'year', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
// At full width, check how many filters are visible in main bar
cy.get('.filter-item-wrapper').then($items => {
cy.log(`Found ${$items.length} filter items at full width`);
});
// Resize to force overflow
cy.viewport(500, 1024);
cy.wait(500); // Allow layout to stabilize after viewport change
// Should have some filters visible and dropdown button present
cy.get('.filter-item-wrapper').should('have.length.lessThan', 4);
cy.getBySel('dropdown-container-btn').should('exist');
// Open more filters and verify all are accessible in the dropdown
openMoreFilters(false);
// Check that the dropdown content contains filters
cy.getBySel('dropdown-content').within(() => {
cy.getBySel('form-item-value').should('have.length.greaterThan', 0);
});
// Close the dropdown
cy.getBySel('filter-bar').click();
// Test with medium viewport
cy.viewport(800, 1024);
cy.wait(500); // Allow layout to stabilize after viewport change
// May or may not have overflow at this size - test adaptively
cy.get('body').then($body => {
if ($body.find('[data-test="dropdown-container-btn"]').length > 0) {
openMoreFilters(false);
cy.getBySel('dropdown-content').within(() => {
cy.getBySel('form-item-value').should('have.length.greaterThan', 0);
});
cy.getBySel('filter-bar').click(); // Close dropdown
}
});
// At large viewport, all filters should fit
cy.viewport(1300, 1024);
cy.wait(500); // Allow layout to stabilize after viewport change
cy.get('.filter-item-wrapper').then($items => {
cy.log(`Found ${$items.length} filter items at large width`);
// Just verify we have some filters, don't assert exact count
expect($items.length).to.be.greaterThan(0);
});
cy.getBySel('dropdown-container-btn').should('not.exist');
});
it('should show "more filters" and scroll', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
{ name: 'test_3', column: 'region', datasetId: 2 },
{ name: 'test_4', column: 'year', datasetId: 2 },
{ name: 'test_5', column: 'country_name', datasetId: 2 },
{ name: 'test_6', column: 'country_code', datasetId: 2 },
{ name: 'test_7', column: 'region', datasetId: 2 },
{ name: 'test_8', column: 'year', datasetId: 2 },
{ name: 'test_9', column: 'country_name', datasetId: 2 },
{ name: 'test_10', column: 'country_code', datasetId: 2 },
{ name: 'test_11', column: 'region', datasetId: 2 },
{ name: 'test_12', column: 'year', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
cy.get('.filter-item-wrapper').should('have.length', 4);
openMoreFilters();
cy.getBySel('form-item-value').should('have.length', 12);
cy.getBySel('filter-control-name').contains('test_3').should('be.visible');
cy.getBySel('filter-control-name')
.contains('test_12')
.should('not.be.visible');
cy.getBySel('filter-control-name').contains('test_12').scrollIntoView();
cy.getBySel('filter-control-name').contains('test_12').should('be.visible');
});
it('should display newly added filter', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
enterNativeFilterEditModal(false);
addCountryNameFilter();
saveNativeFilterSettings([]);
validateFilterNameOnDashboard(testItems.topTenChart.filterColumn);
});
it.skip('should spot changes in "more filters" and apply their values', () => {
cy.intercept(`**/api/v1/chart/data?form_data=**`).as('chart');
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
{ name: 'test_3', column: 'region', datasetId: 2 },
{ name: 'test_4', column: 'year', datasetId: 2 },
{ name: 'test_5', column: 'country_name', datasetId: 2 },
{ name: 'test_6', column: 'country_code', datasetId: 2 },
{ name: 'test_7', column: 'region', datasetId: 2 },
{ name: 'test_8', column: 'year', datasetId: 2 },
{ name: 'test_9', column: 'country_name', datasetId: 2 },
{ name: 'test_10', column: 'country_code', datasetId: 2 },
{ name: 'test_11', column: 'region', datasetId: 2 },
{ name: 'test_12', column: 'year', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
openMoreFilters();
applyNativeFilterValueWithIndex(8, testItems.filterDefaultValue);
cy.get(nativeFilters.applyFilter).click({ force: true });
cy.wait('@chart');
cy.get('.ant-scroll-number.ant-badge-count').should(
'have.attr',
'title',
'1',
);
});
it.skip('should focus filter and open "more filters" programmatically', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
{ name: 'test_3', column: 'region', datasetId: 2 },
{ name: 'test_4', column: 'year', datasetId: 2 },
{ name: 'test_5', column: 'country_name', datasetId: 2 },
{ name: 'test_6', column: 'country_code', datasetId: 2 },
{ name: 'test_7', column: 'region', datasetId: 2 },
{ name: 'test_8', column: 'year', datasetId: 2 },
{ name: 'test_9', column: 'country_name', datasetId: 2 },
{ name: 'test_10', column: 'country_code', datasetId: 2 },
{ name: 'test_11', column: 'region', datasetId: 2 },
{ name: 'test_12', column: 'year', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
openMoreFilters();
applyNativeFilterValueWithIndex(8, testItems.filterDefaultValue);
cy.get(nativeFilters.applyFilter).click({ force: true });
cy.getBySel('slice-header').within(() => {
cy.get('.filter-counts').trigger('mouseover');
});
cy.getBySel('filter-status-popover').contains('test_9').click();
cy.getBySel('dropdown-content').should('be.visible');
cy.get('.ant-select-focused').should('be.visible');
});
it.skip('should show tag count and one plain tag on focus and only count on blur in select ', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
]);
setFilterBarOrientation('horizontal');
enterNativeFilterEditModal();
inputNativeFilterDefaultValue('Albania');
cy.get('.ant-select-selection-search-input').clear({ force: true });
inputNativeFilterDefaultValue('Algeria', true);
saveNativeFilterSettings([SAMPLE_CHART]);
cy.getBySel('filter-bar').within(() => {
cy.get(nativeFilters.filterItem).contains('Albania').should('be.visible');
cy.get(nativeFilters.filterItem).contains('+ 1 ...').should('be.visible');
cy.get('.ant-select-selection-search-input').click();
cy.get(nativeFilters.filterItem).contains('+ 2 ...').should('be.visible');
});
});
});

View File

@@ -1,53 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import qs from 'querystringify';
import { waitForChartLoad } from 'cypress/utils';
import { WORLD_HEALTH_DASHBOARD } from 'cypress/utils/urls';
import { WORLD_HEALTH_CHARTS } from './utils';
interface QueryString {
native_filters_key: string;
}
describe('nativefilter url param key', () => {
// const urlParams = { param1: '123', param2: 'abc' };
let initialFilterKey: string;
it('should have cachekey in nativefilter param', () => {
// things in `before` will not retry and the `waitForChartLoad` check is
// especially flaky and may need more retries
cy.visit(WORLD_HEALTH_DASHBOARD);
WORLD_HEALTH_CHARTS.forEach(waitForChartLoad);
cy.wait(1000); // wait for key to be published (debounced)
cy.location().then(loc => {
const queryParams = qs.parse(loc.search) as QueryString;
expect(typeof queryParams.native_filters_key).eq('string');
});
});
it('should have different key when page reloads', () => {
cy.visit(WORLD_HEALTH_DASHBOARD);
WORLD_HEALTH_CHARTS.forEach(waitForChartLoad);
cy.wait(1000); // wait for key to be published (debounced)
cy.location().then(loc => {
const queryParams = qs.parse(loc.search) as QueryString;
expect(queryParams.native_filters_key).not.equal(initialFilterKey);
});
});
});

View File

@@ -1,51 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { WORLD_HEALTH_DASHBOARD } from 'cypress/utils/urls';
import { waitForChartLoad } from 'cypress/utils';
import { WORLD_HEALTH_CHARTS, interceptLog } from './utils';
describe('Dashboard load', () => {
it('should load dashboard', () => {
cy.visit(WORLD_HEALTH_DASHBOARD);
WORLD_HEALTH_CHARTS.forEach(waitForChartLoad);
});
it('should load in edit mode', () => {
cy.visit(`${WORLD_HEALTH_DASHBOARD}?edit=true&standalone=true`);
cy.getBySel('discard-changes-button').should('be.visible');
});
it('should load in standalone mode', () => {
cy.visit(`${WORLD_HEALTH_DASHBOARD}?edit=true&standalone=true`);
cy.get('#app-menu').should('not.exist');
});
it('should load in edit/standalone mode', () => {
cy.visit(`${WORLD_HEALTH_DASHBOARD}?edit=true&standalone=true`);
cy.getBySel('discard-changes-button').should('be.visible');
cy.get('#app-menu').should('not.exist');
});
// TODO flaky test. skipping to unblock CI
it.skip('should send log data', () => {
interceptLog();
cy.visit(WORLD_HEALTH_DASHBOARD);
cy.wait('@logs', { timeout: 15000 });
});
});

View File

@@ -1,385 +0,0 @@
/**
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import qs from 'querystring';
import {
dashboardView,
nativeFilters,
dataTestChartName,
} from 'cypress/support/directories';
import {
addCountryNameFilter,
applyAdvancedTimeRangeFilterOnDashboard,
applyNativeFilterValueWithIndex,
cancelNativeFilterSettings,
deleteNativeFilter,
enterNativeFilterEditModal,
fillNativeFilterForm,
inputNativeFilterDefaultValue,
saveNativeFilterSettings,
undoDeleteNativeFilter,
validateFilterContentOnDashboard,
validateFilterNameOnDashboard,
testItems,
WORLD_HEALTH_CHARTS,
} from './utils';
import {
prepareDashboardFilters,
SAMPLE_CHART,
visitDashboard,
} from './shared_dashboard_functions';
// function selectFilter(index: number) {
// cy.get("[data-test='filter-title-container'] [draggable='true']")
// .eq(index)
// .click();
// }
// function closeFilterModal() {
// cy.get('body').then($body => {
// if ($body.find('[data-test="native-filter-modal-cancel-button"]').length) {
// cy.getBySel('native-filter-modal-cancel-button').click();
// }
// });
// }
describe('Native filters', () => {
describe('Nativefilters initial state not required', () => {
it("User can check 'Filter has default value'", () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
enterNativeFilterEditModal();
inputNativeFilterDefaultValue(testItems.filterDefaultValue);
});
it('User can add a new native filter', () => {
prepareDashboardFilters([]);
let filterKey: string;
const removeFirstChar = (search: string) =>
search.split('').slice(1, search.length).join('');
cy.location().then(loc => {
cy.url().should('contain', 'native_filters_key');
const queryParams = qs.parse(removeFirstChar(loc.search));
filterKey = queryParams.native_filters_key as string;
expect(typeof filterKey).eq('string');
});
enterNativeFilterEditModal();
addCountryNameFilter();
saveNativeFilterSettings([SAMPLE_CHART]);
cy.location().then(loc => {
cy.url().should('contain', 'native_filters_key');
const queryParams = qs.parse(removeFirstChar(loc.search));
const newfilterKey = queryParams.native_filters_key;
expect(newfilterKey).eq(filterKey);
});
cy.get(nativeFilters.modal.container).should('not.exist');
});
it('User can restore a deleted native filter', () => {
prepareDashboardFilters([
{ name: 'country_code', column: 'country_code', datasetId: 2 },
]);
enterNativeFilterEditModal();
cy.get(nativeFilters.filtersList.removeIcon).first().click();
cy.get('[data-test="restore-filter-button"]')
.should('be.visible')
.click();
cy.get(nativeFilters.modal.container)
.find(nativeFilters.filtersPanel.filterName)
.should(
'have.attr',
'value',
testItems.topTenChart.filterColumnCountryCode,
);
});
it('User can create a time grain filter', () => {
prepareDashboardFilters([]);
enterNativeFilterEditModal();
fillNativeFilterForm(
testItems.filterType.timeGrain,
testItems.filterType.timeGrain,
testItems.datasetForNativeFilter,
);
saveNativeFilterSettings([SAMPLE_CHART]);
applyNativeFilterValueWithIndex(0, testItems.filterTimeGrain);
cy.get(nativeFilters.applyFilter).click();
cy.url().then(u => {
const ur = new URL(u);
expect(ur.search).to.include('native_filters');
});
validateFilterNameOnDashboard(testItems.filterType.timeGrain);
validateFilterContentOnDashboard(testItems.filterTimeGrain);
});
it.skip('User can create a time range filter', () => {
enterNativeFilterEditModal();
fillNativeFilterForm(
testItems.filterType.timeRange,
testItems.filterType.timeRange,
);
saveNativeFilterSettings(WORLD_HEALTH_CHARTS);
cy.get(dashboardView.salesDashboardSpecific.vehicleSalesFilterTimeRange)
.should('be.visible')
.click();
applyAdvancedTimeRangeFilterOnDashboard('2005-12-17', '2006-12-17');
cy.url().then(u => {
const ur = new URL(u);
expect(ur.search).to.include('native_filters');
});
validateFilterNameOnDashboard(testItems.filterType.timeRange);
cy.get(nativeFilters.filterFromDashboardView.timeRangeFilterContent)
.contains('2005-12-17')
.should('be.visible');
});
it.skip('User can create a time column filter', () => {
enterNativeFilterEditModal();
fillNativeFilterForm(
testItems.filterType.timeColumn,
testItems.filterType.timeColumn,
testItems.datasetForNativeFilter,
);
saveNativeFilterSettings(WORLD_HEALTH_CHARTS);
cy.intercept(`**/api/v1/chart/data?form_data=**`).as('chart');
cy.get(nativeFilters.modal.container).should('not.exist');
// assert that native filter is created
validateFilterNameOnDashboard(testItems.filterType.timeColumn);
applyNativeFilterValueWithIndex(
0,
testItems.topTenChart.filterColumnYear,
);
cy.get(nativeFilters.applyFilter).click({ force: true });
cy.wait('@chart');
validateFilterContentOnDashboard(testItems.topTenChart.filterColumnYear);
});
describe.only('Numerical Range Filter - Display Modes', () => {
beforeEach(() => {
visitDashboard();
});
const expandFilterConfiguration = () => {
cy.get('.ant-collapse-header')
.contains('Filter Configuration')
.should('be.visible')
.then($header => {
cy.wrap($header)
.closest('.ant-collapse-item')
.invoke('hasClass', 'ant-collapse-item-active')
.then(isExpanded => {
if (!isExpanded) cy.wrap($header).click();
});
});
cy.get('.ant-collapse-content-box').should('be.visible');
};
const selectRangeTypeOption = (label: string) => {
cy.contains('Range Type')
.should('be.visible')
.closest('.ant-form-item')
.within(() => {
cy.get('.ant-select-selector').click();
});
cy.get('.ant-select-dropdown:visible')
.contains('.ant-select-item-option', label)
.click();
};
const applyAndAssertInputs = (from: string, to: string) => {
// Set 'from' input
cy.get('[data-test="range-filter-from-input"]').clear();
cy.get('[data-test="range-filter-from-input"]').type(from);
cy.get('[data-test="range-filter-from-input"]').blur();
// Set 'to' input
cy.get('[data-test="range-filter-to-input"]').clear();
cy.get('[data-test="range-filter-to-input"]').type(to);
cy.get('[data-test="range-filter-to-input"]').blur();
// Assert values without chaining after .invoke()
cy.get('[data-test="range-filter-from-input"]')
.invoke('val')
.then(val => {
expect(val).to.equal(from);
});
cy.get('[data-test="range-filter-to-input"]')
.invoke('val')
.then(val => {
expect(val).to.equal(to);
});
};
it('User can create a numerical range filter with "Range Inputs" display mode', () => {
enterNativeFilterEditModal(false);
fillNativeFilterForm(
testItems.filterType.numerical,
testItems.filterNumericalColumn,
testItems.datasetForNativeFilter,
testItems.filterNumericalColumn,
);
expandFilterConfiguration();
selectRangeTypeOption('Range Inputs');
saveNativeFilterSettings([]);
cy.wait(500); // allow filter to mount
applyAndAssertInputs('40', '70');
});
it('User can change the display mode to "Slider"', () => {
enterNativeFilterEditModal(false);
fillNativeFilterForm(
testItems.filterType.numerical,
testItems.filterNumericalColumn,
testItems.datasetForNativeFilter,
testItems.filterNumericalColumn,
);
expandFilterConfiguration();
cy.contains('Range Type')
.should('be.visible')
.closest('.ant-form-item')
.within(() => {
cy.get('.ant-select-selector').click({ force: true });
});
cy.get('.ant-select-dropdown:visible .ant-select-item-option')
.contains(/^Slider$/)
.click({ force: true });
cy.get('.ant-select-selector').should('contain.text', 'Slider');
saveNativeFilterSettings([]);
cy.get('.ant-slider', { timeout: 10000 }).should('be.visible');
cy.get('[data-test="range-filter-from-input"]', {
timeout: 5000,
}).should('not.exist');
cy.get('[data-test="range-filter-to-input"]', { timeout: 5000 }).should(
'not.exist',
);
});
it('User can change the display mode to "Slider and range input"', () => {
enterNativeFilterEditModal(false);
// Re-create filter
fillNativeFilterForm(
testItems.filterType.numerical,
testItems.filterNumericalColumn,
testItems.datasetForNativeFilter,
testItems.filterNumericalColumn,
);
expandFilterConfiguration();
selectRangeTypeOption('Slider and range input');
saveNativeFilterSettings([]);
cy.wait(500);
applyAndAssertInputs('40', '70');
});
});
it('User can undo deleting a native filter', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
enterNativeFilterEditModal();
undoDeleteNativeFilter();
cy.get(nativeFilters.modal.container)
.find(nativeFilters.filtersPanel.filterName)
.should('have.attr', 'value', testItems.topTenChart.filterColumn);
});
it('User can cancel changes in native filter', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
enterNativeFilterEditModal();
cy.getBySel('filters-config-modal__name-input').type('|EDITED', {
force: true,
});
cancelNativeFilterSettings();
enterNativeFilterEditModal(false);
cy.get(nativeFilters.filtersList.removeIcon).first().click();
cy.contains('You have removed this filter.').should('be.visible');
});
it('User can create a value filter', () => {
visitDashboard();
enterNativeFilterEditModal(false);
addCountryNameFilter();
cy.get(nativeFilters.filtersPanel.filterTypeInput)
.find(nativeFilters.filtersPanel.filterTypeItem)
.should('have.text', testItems.filterType.value);
saveNativeFilterSettings([]);
validateFilterNameOnDashboard(testItems.topTenChart.filterColumn);
});
it('User can apply value filter with selected values', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
applyNativeFilterValueWithIndex(0, testItems.filterDefaultValue);
cy.get(nativeFilters.applyFilter).click();
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('not.exist');
});
});
it('User can stop filtering when filter is removed', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
enterNativeFilterEditModal();
inputNativeFilterDefaultValue(testItems.filterDefaultValue);
saveNativeFilterSettings([SAMPLE_CHART]);
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('not.exist');
});
cy.get(nativeFilters.filterItem)
.contains(testItems.filterDefaultValue)
.should('be.visible');
validateFilterNameOnDashboard(testItems.topTenChart.filterColumn);
enterNativeFilterEditModal(false);
deleteNativeFilter();
saveNativeFilterSettings([SAMPLE_CHART]);
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('be.visible');
});
});
});
});

View File

@@ -1,431 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
nativeFilters,
exploreView,
dataTestChartName,
} from 'cypress/support/directories';
import {
addParentFilterWithValue,
applyNativeFilterValueWithIndex,
cancelNativeFilterSettings,
checkNativeFilterTooltip,
clickOnAddFilterInModal,
collapseFilterOnLeftPanel,
enterNativeFilterEditModal,
expandFilterOnLeftPanel,
getNativeFilterPlaceholderWithIndex,
inputNativeFilterDefaultValue,
saveNativeFilterSettings,
nativeFilterTooltips,
validateFilterContentOnDashboard,
valueNativeFilterOptions,
validateFilterNameOnDashboard,
testItems,
} from './utils';
import {
prepareDashboardFilters,
SAMPLE_CHART,
visitDashboard,
} from './shared_dashboard_functions';
function selectFilter(index: number) {
cy.get("[data-test='filter-title-container'] [role='tab']").eq(index).click();
}
function closeFilterModal() {
cy.get('body').then($body => {
if ($body.find('[data-test="native-filter-modal-cancel-button"]').length) {
cy.getBySel('native-filter-modal-cancel-button').click();
}
});
}
describe('Native filters', () => {
describe('Nativefilters tests initial state required', () => {
beforeEach(() => {
cy.createSampleDashboards([0]);
});
it.skip('Verify that default value is respected after revisit', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name' },
]);
enterNativeFilterEditModal();
inputNativeFilterDefaultValue(testItems.filterDefaultValue);
saveNativeFilterSettings([SAMPLE_CHART]);
cy.get(nativeFilters.filterItem)
.contains(testItems.filterDefaultValue)
.should('be.visible');
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('not.exist');
});
// reload dashboard
cy.reload();
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('not.exist');
});
validateFilterContentOnDashboard(testItems.filterDefaultValue);
});
it('User can create parent filters using "Values are dependent on other filters"', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
]);
enterNativeFilterEditModal();
selectFilter(1);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
addParentFilterWithValue(0, testItems.topTenChart.filterColumnRegion);
saveNativeFilterSettings([SAMPLE_CHART]);
[
testItems.topTenChart.filterColumnRegion,
testItems.topTenChart.filterColumn,
].forEach(it => {
cy.get(nativeFilters.filterFromDashboardView.filterName)
.contains(it)
.should('be.visible');
});
getNativeFilterPlaceholderWithIndex(1)
.invoke('text')
.should('equal', '214 options', { timeout: 20000 });
// apply first filter value and validate 2nd filter is depden on 1st filter.
applyNativeFilterValueWithIndex(0, 'North America');
getNativeFilterPlaceholderWithIndex(0).should('have.text', '3 options', {
timeout: 20000,
});
});
it('user can delete dependent filter', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
]);
enterNativeFilterEditModal();
selectFilter(1);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
addParentFilterWithValue(0, testItems.topTenChart.filterColumnRegion);
// remove year native filter to cause it disappears from parent filter input in global sales
cy.get(nativeFilters.modal.tabsList.removeTab)
.should('be.visible')
.first()
.click();
// make sure you are seeing global sales filter which had parent filter
cy.get(nativeFilters.modal.tabsList.filterItemsContainer)
.children()
.last()
.click();
//
cy.wait(1000);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters').should(
'not.exist',
);
},
);
});
it('user cannot create bi-directional dependencies between filters', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
{ name: 'country_code', column: 'country_code' },
{ name: 'year', column: 'year' },
]);
enterNativeFilterEditModal();
// First, make country_name dependent on region
selectFilter(1);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
addParentFilterWithValue(0, testItems.topTenChart.filterColumnRegion);
// Second, make country_code dependent on country_name
selectFilter(2);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
addParentFilterWithValue(0, testItems.topTenChart.filterColumn);
// Now select region filter and try to add dependency
selectFilter(0);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
// Verify that only 'year' is available as dependency for region
// 'country_name' and 'country_code' should not be available (would create circular dependency)
cy.get('input[aria-label^="Limit type"]').click({ force: true });
cy.get('[role="listbox"]').should('be.visible');
cy.get('[role="listbox"]').should('contain', 'year');
cy.get('[role="listbox"]').should('not.contain', 'country_name');
cy.get('[role="listbox"]').should('not.contain', 'country_code');
cy.get('[role="listbox"]').contains('year').click();
},
);
});
it('Dependent filter selects first item based on parent filter selection', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
]);
enterNativeFilterEditModal();
selectFilter(0);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Select first filter value by default')
.should('be.visible')
.click();
},
);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Can select multiple values ')
.should('be.visible')
.click();
},
);
selectFilter(1);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Can select multiple values ')
.should('be.visible')
.click();
},
);
addParentFilterWithValue(0, testItems.topTenChart.filterColumnRegion);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Select first filter value by default')
.should('be.visible')
.click();
},
);
// cannot use saveNativeFilterSettings because there is a bug which
// sometimes does not allow charts to load when enabling the 'Select first filter value by default'
// to be saved when using dependent filters so,
// you reload the window.
cy.get(nativeFilters.modal.footer)
.contains('Save')
.should('be.visible')
.click({ force: true });
cy.get(nativeFilters.modal.container).should('not.exist');
cy.reload();
applyNativeFilterValueWithIndex(0, 'North America');
// Check that dependent filter auto-selects the first item
cy.get(nativeFilters.filterFromDashboardView.filterContent)
.eq(1)
.should('contain.text', 'Bermuda');
});
it('User can create filter depend on 2 other filters', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
{ name: 'country_code', column: 'country_code' },
]);
enterNativeFilterEditModal();
selectFilter(2);
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
cy.get(exploreView.controlPanel.addFieldValue).click();
},
);
// add value to the first input
addParentFilterWithValue(0, testItems.topTenChart.filterColumnRegion);
// add value to the second input
addParentFilterWithValue(1, testItems.topTenChart.filterColumn);
saveNativeFilterSettings([SAMPLE_CHART]);
// filters should be displayed in the left panel
[
testItems.topTenChart.filterColumnRegion,
testItems.topTenChart.filterColumn,
testItems.topTenChart.filterColumnCountryCode,
].forEach(it => {
validateFilterNameOnDashboard(it);
});
// initially first filter shows 39 options
getNativeFilterPlaceholderWithIndex(0).should('have.text', '7 options');
// initially second filter shows 409 options
getNativeFilterPlaceholderWithIndex(1).should('have.text', '214 options');
// verify third filter shows 409 options
getNativeFilterPlaceholderWithIndex(2).should('have.text', '214 options');
// apply first filter value
applyNativeFilterValueWithIndex(0, 'North America');
// verify second filter shows 409 options available still
getNativeFilterPlaceholderWithIndex(0).should('have.text', '214 options');
// verify second filter shows 69 options available still
getNativeFilterPlaceholderWithIndex(1).should('have.text', '3 options');
// apply second filter value
applyNativeFilterValueWithIndex(1, 'United States');
// verify number of available options for third filter - should be decreased to only one
getNativeFilterPlaceholderWithIndex(0).should('have.text', '1 option');
});
it('User can remove parent filters', () => {
prepareDashboardFilters([
{ name: 'region', column: 'region' },
{ name: 'country_name', column: 'country_name' },
]);
enterNativeFilterEditModal();
selectFilter(1);
// Select dependent option and auto use platform for genre
cy.get(nativeFilters.filterConfigurationSections.displayedSection).within(
() => {
cy.contains('Values are dependent on other filters')
.should('be.visible')
.click();
},
);
saveNativeFilterSettings([SAMPLE_CHART]);
enterNativeFilterEditModal(false);
cy.get(nativeFilters.modal.tabsList.removeTab)
.should('be.visible')
.first()
.click({
force: true,
});
saveNativeFilterSettings([SAMPLE_CHART]);
cy.get(dataTestChartName(testItems.topTenChart.name)).within(() => {
cy.contains(testItems.filterDefaultValue).should('be.visible');
cy.contains(testItems.filterOtherCountry).should('be.visible');
});
});
});
describe('Nativefilters basic interactions', () => {
before(() => {
visitDashboard();
});
beforeEach(() => {
cy.createSampleDashboards([0]);
closeFilterModal();
});
it('User can expand / retract native filter sidebar on a dashboard', () => {
expandFilterOnLeftPanel();
cy.get(nativeFilters.filtersPanel.filterGear).click({
force: true,
});
cy.get('.ant-dropdown-menu').should('be.visible');
cy.get(nativeFilters.filterFromDashboardView.createFilterButton).should(
'be.visible',
);
cy.get(nativeFilters.filterFromDashboardView.expand).should(
'not.be.visible',
);
collapseFilterOnLeftPanel();
});
it('User can enter filter edit pop-up by clicking on native filter edit icon', () => {
enterNativeFilterEditModal(false);
});
it('User can delete a native filter', () => {
enterNativeFilterEditModal(false);
cy.get(nativeFilters.filtersList.removeIcon).first().click();
cy.contains('Restore filter').should('not.exist', { timeout: 10000 });
});
it('User can cancel creating a new filter', () => {
enterNativeFilterEditModal(false);
cancelNativeFilterSettings();
});
it('Verify setting options and tooltips for value filter', () => {
enterNativeFilterEditModal(false);
cy.contains('Filter value is required').scrollIntoView();
cy.get('body').trigger('mousemove', { clientX: 0, clientY: 0 });
cy.wait(300);
cy.contains('Filter value is required').should('be.visible').click({
force: true,
});
checkNativeFilterTooltip(0, nativeFilterTooltips.preFilter);
checkNativeFilterTooltip(1, nativeFilterTooltips.defaultValue);
cy.get(nativeFilters.modal.container).should('be.visible');
valueNativeFilterOptions.forEach(el => {
cy.contains(el);
});
cy.contains('Values are dependent on other filters').should('not.exist');
cy.get(
nativeFilters.filterConfigurationSections.checkedCheckbox,
).contains('Can select multiple values');
checkNativeFilterTooltip(2, nativeFilterTooltips.required);
checkNativeFilterTooltip(3, nativeFilterTooltips.defaultToFirstItem);
checkNativeFilterTooltip(4, nativeFilterTooltips.searchAllFilterOptions);
checkNativeFilterTooltip(5, nativeFilterTooltips.inverseSelection);
clickOnAddFilterInModal();
cy.contains('Values are dependent on other filters').should('exist');
});
});
});

View File

@@ -1,194 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
parsePostForm,
waitForChartLoad,
getChartAliasBySpec,
} from 'cypress/utils';
import { TABBED_DASHBOARD } from 'cypress/utils/urls';
import { expandFilterOnLeftPanel } from './utils';
const TREEMAP = { name: 'Treemap', viz: 'treemap_v2' };
const LINE_CHART = { name: 'Growth Rate', viz: 'echarts_timeseries_line' };
const BOX_PLOT = { name: 'Box plot', viz: 'box_plot' };
const BIG_NUMBER = { name: 'Number of Girls', viz: 'big_number_total' };
const TABLE = { name: 'Names Sorted by Num in California', viz: 'table' };
function topLevelTabs() {
cy.getBySel('dashboard-component-tabs')
.first()
.find('[data-test="nav-list"] .ant-tabs-nav-list > .ant-tabs-tab')
.as('top-level-tabs');
}
function resetTabs() {
topLevelTabs();
cy.get('@top-level-tabs').first().click();
waitForChartLoad(TREEMAP);
waitForChartLoad(BIG_NUMBER);
waitForChartLoad(TABLE);
}
describe.skip('Dashboard tabs', () => {
before(() => {
cy.visit(TABBED_DASHBOARD);
});
beforeEach(() => {
resetTabs();
});
it('should switch tabs', () => {
topLevelTabs();
cy.get('@top-level-tabs').first().click();
cy.get('@top-level-tabs')
.first()
.should('have.class', 'ant-tabs-tab-active');
cy.get('@top-level-tabs')
.last()
.should('not.have.class', 'ant-tabs-tab-active');
cy.get('[data-test-chart-name="Box plot"]').should('not.exist');
cy.get('[data-test-chart-name="Trends"]').should('not.exist');
cy.get('@top-level-tabs').last().click();
cy.get('@top-level-tabs')
.last()
.should('have.class', 'ant-tabs-tab-active');
cy.get('@top-level-tabs')
.first()
.should('not.have.class', 'ant-tabs-tab-active');
waitForChartLoad(BOX_PLOT);
cy.get('[data-test-chart-name="Box plot"]').should('exist');
resetTabs();
// click row level tab, see 1 more chart
cy.getBySel('dashboard-component-tabs')
.eq(2)
.find('[data-test="nav-list"] .ant-tabs-nav-list > .ant-tabs-tab')
.as('row-level-tabs');
cy.get('@row-level-tabs').last().click();
waitForChartLoad(LINE_CHART);
cy.get('[data-test-chart-name="Trends"]').should('exist');
cy.get('@row-level-tabs').first().click();
});
it.skip('should send new queries when tab becomes visible', () => {
// landing in first tab
waitForChartLoad(TREEMAP);
getChartAliasBySpec(TREEMAP).then(treemapAlias => {
// apply filter
cy.get('.Select__control').first().should('be.visible').click();
cy.get('.Select__control input[type=text]').first().focus();
cy.focused().type('South');
cy.get('.Select__option').contains('South Asia').click();
cy.get('.filter button:not(:disabled)').contains('Apply').click();
// send new query from same tab
cy.wait(treemapAlias).then(({ request }) => {
const requestBody = parsePostForm(request.body);
const requestParams = JSON.parse(requestBody.form_data as string);
expect(requestParams.extra_filters[0]).deep.eq({
col: 'region',
op: 'IN',
val: ['South Asia'],
});
});
});
cy.intercept('**/superset/explore_json/?*').as('legacyChartData');
// click row level tab, send 1 more query
cy.get('.ant-tabs-tab').contains('row tab 2').click();
cy.wait('@legacyChartData').then(({ request }) => {
const requestBody = parsePostForm(request.body);
const requestParams = JSON.parse(requestBody.form_data as string);
expect(requestParams.extra_filters[0]).deep.eq({
col: 'region',
op: 'IN',
val: ['South Asia'],
});
expect(requestParams.viz_type).eq(LINE_CHART.viz);
});
cy.intercept('POST', '**/api/v1/chart/data?*').as('v1ChartData');
// click top level tab, send 1 more query
cy.get('.ant-tabs-tab').contains('Tab B').click();
cy.wait('@v1ChartData').then(({ request }) => {
expect(request.body.queries[0].filters[0]).deep.eq({
col: 'region',
op: 'IN',
val: ['South Asia'],
});
});
getChartAliasBySpec(BOX_PLOT).then(boxPlotAlias => {
// navigate to filter and clear filter
cy.get('.ant-tabs-tab').contains('Tab A').click();
cy.get('.ant-tabs-tab').contains('row tab 1').click();
cy.get('.Select__clear-indicator').click();
cy.get('.filter button:not(:disabled)').contains('Apply').click();
// trigger 1 new query
waitForChartLoad(TREEMAP);
// make sure query API not requested multiple times
cy.on('fail', err => {
expect(err.message).to.include('timed out waiting');
return false;
});
cy.wait(boxPlotAlias, { timeout: 1000 }).then(() => {
throw new Error('Unexpected API call.');
});
});
});
it('should update size when switch tab', () => {
cy.get('@top-level-tabs').last().click();
cy.get('@top-level-tabs')
.last()
.should('have.class', 'ant-tabs-tab-active');
expandFilterOnLeftPanel();
cy.wait(1000);
cy.get('@top-level-tabs').first().click();
cy.get('@top-level-tabs')
.first()
.should('have.class', 'ant-tabs-tab-active');
cy.wait(1000);
cy.get("[data-test-viz-type='treemap_v2'] .chart-container").then(
$chartContainer => {
expect($chartContainer.get(0).scrollWidth).eq(
$chartContainer.get(0).offsetWidth,
);
},
);
});
});

View File

@@ -1,45 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { parsePostForm, JsonObject, waitForChartLoad } from 'cypress/utils';
import { WORLD_HEALTH_DASHBOARD } from 'cypress/utils/urls';
import { WORLD_HEALTH_CHARTS } from './utils';
describe('Dashboard form data', () => {
const urlParams = { param1: '123', param2: 'abc' };
before(() => {
cy.visit(WORLD_HEALTH_DASHBOARD, { qs: urlParams });
});
it('should apply url params to slice requests', () => {
cy.intercept('**/api/v1/chart/data?*', request => {
// TODO: export url params to chart data API
request.body.queries.forEach((query: { url_params: JsonObject }) => {
expect(query.url_params).deep.eq(urlParams);
});
});
cy.intercept('**/superset/explore_json/*', request => {
const requestParams = JSON.parse(
parsePostForm(request.body).form_data as string,
);
expect(requestParams.url_params).deep.eq(urlParams);
});
WORLD_HEALTH_CHARTS.forEach(waitForChartLoad);
});
});

View File

@@ -1,109 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('AdhocFilters', () => {
beforeEach(() => {
cy.intercept('GET', '**/api/v1/datasource/table/*/column/name/values').as(
'filterValues',
);
cy.intercept('POST', '**/superset/explore_json/**').as('postJson');
cy.intercept('GET', '**/superset/explore_json/**').as('getJson');
cy.visitChartByName('Boys'); // a table chart
cy.verifySliceSuccess({ waitAlias: '@postJson' });
});
let numScripts = 0;
it('Should load AceEditor scripts when needed', () => {
cy.get('script').then(nodes => {
numScripts = nodes.length;
});
cy.get('[data-test=adhoc_filters]').within(() => {
cy.get('.Select__control').scrollIntoView();
cy.get('.Select__control').click();
cy.get('input[type=text]').focus();
cy.focused().type('name{enter}');
cy.get("div[role='button']").first().click();
});
// antd tabs do lazy loading, so we need to click on tab with ace editor
cy.get('#filter-edit-popover').within(() => {
cy.get('.ant-tabs-tab').contains('Custom SQL').click();
cy.get('.ant-tabs-tab').contains('Simple').click();
});
cy.get('script').then(nodes => {
// should load new script chunks for SQL editor
expect(nodes.length).to.greaterThan(numScripts);
});
});
it('Set simple adhoc filter', () => {
cy.get('[aria-label="Comparator option"] .Select__control').click();
cy.get('[data-test=adhoc-filter-simple-value] input[type=text]').focus();
cy.focused().type('Jack{enter}', { delay: 20 });
cy.get('[data-test="adhoc-filter-edit-popover-save-button"]').click();
cy.get(
'[data-test=adhoc_filters] .Select__control span.option-label',
).contains('name = Jack');
cy.get('button[data-test="run-query-button"]').click();
cy.verifySliceSuccess({
waitAlias: '@postJson',
chartSelector: 'svg',
});
});
it('Set custom adhoc filter', () => {
const filterType = 'name';
const filterContent = "'Amy' OR name = 'Donald'";
cy.get('[data-test=adhoc_filters] .Select__control').scrollIntoView();
cy.get('[data-test=adhoc_filters] .Select__control').click();
// remove previous input
cy.get('[data-test=adhoc_filters] input[type=text]').focus();
cy.focused().type('{backspace}');
cy.get('[data-test=adhoc_filters] input[type=text]').focus();
cy.focused().type(`${filterType}{enter}`);
cy.wait('@filterValues');
// selecting a new filter should auto-open the popup,
// so the tab should be visible by now
cy.get('#filter-edit-popover #adhoc-filter-edit-tabs-tab-SQL').click();
cy.get('#filter-edit-popover .ace_content').click();
cy.get('#filter-edit-popover .ace_text-input').type(filterContent);
cy.get('[data-test="adhoc-filter-edit-popover-save-button"]').click();
// check if the filter was saved correctly
cy.get(
'[data-test=adhoc_filters] .Select__control span.option-label',
).contains(`${filterType} = ${filterContent}`);
cy.get('button[data-test="run-query-button"]').click();
cy.verifySliceSuccess({
waitAlias: '@postJson',
chartSelector: 'svg',
});
});
});

View File

@@ -1,123 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptChart } from 'cypress/utils';
describe('AdhocMetrics', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
cy.visitChartByName('Num Births Trend');
cy.verifySliceSuccess({ waitAlias: '@chartData' });
});
it('Clear metric and set simple adhoc metric', () => {
const metric = 'sum(num_girls)';
const metricName = 'Sum Girls';
cy.get('[data-test=metrics]')
.find('[data-test="remove-control-button"]')
.click();
cy.get('[data-test=metrics]')
.contains('Drop columns/metrics here or click')
.click();
// Title edit for saved metrics is disabled - switch to Simple
cy.get('[id="adhoc-metric-edit-tabs-tab-SIMPLE"]').click();
cy.get('[data-test="AdhocMetricEditTitle#trigger"]').click();
cy.get('[data-test="AdhocMetricEditTitle#input"]').type(metricName);
cy.get('input[aria-label="Select column"]').click();
cy.get('input[aria-label="Select column"]').type('num_girls{enter}');
cy.get('input[aria-label="Select aggregate options"]').click();
cy.get('input[aria-label="Select aggregate options"]').type('sum{enter}');
cy.get('[data-test="AdhocMetricEdit#save"]').contains('Save').click();
cy.get('[data-test="control-label"]').contains(metricName);
cy.get('button[data-test="run-query-button"]').click();
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: `${metric} AS "${metricName}"`, // SQL statement
});
});
xit('Switch from simple to custom sql', () => {
cy.get('[data-test=metrics]')
.find('[data-test="metric-option"]')
.should('have.length', 1);
// select column "num"
cy.get('[data-test=metrics]').find('.Select__clear-indicator').click();
cy.get('[data-test=metrics]').find('.Select__control').click();
cy.get('[data-test=metrics]').find('.Select__control input').type('num');
cy.get('[data-test=metrics]')
.find('.option-label')
.first()
.should('have.text', 'num')
.click();
// add custom SQL
cy.get('#adhoc-metric-edit-tabs-tab-SQL').click();
cy.get('[data-test=metrics-edit-popover]').within(() => {
cy.get('.ace_content').click();
cy.get('.ace_text-input').type('/COUNT(DISTINCT name)', { force: true });
cy.get('[data-test="AdhocMetricEdit#save"]').contains('Save').click();
});
cy.get('button[data-test="run-query-button"]').click();
const metric = 'SUM(num)/COUNT(DISTINCT name)';
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: `${metric} AS "${metric}"`,
});
});
xit('Switch from custom sql tabs to simple', () => {
cy.get('[data-test=metrics]').within(() => {
cy.get('.Select__dropdown-indicator').click();
cy.get('input[type=text]').type('num_girls{enter}');
});
cy.get('[data-test=metrics]')
.find('[data-test="metric-option"]')
.should('have.length', 2);
cy.get('#metrics-edit-popover').within(() => {
cy.get('#adhoc-metric-edit-tabs-tab-SQL').click();
cy.get('.ace_identifier').contains('num_girls');
cy.get('.ace_content').click();
cy.get('.ace_text-input').type('{selectall}{backspace}SUM(num)');
cy.get('#adhoc-metric-edit-tabs-tab-SIMPLE').click();
cy.get('.Select__single-value').contains(/^num$/);
cy.get('button').contains('Save').click();
});
cy.get('button[data-test="run-query-button"]').click();
const metric = 'SUM(num)';
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: `${metric} AS "${metric}"`,
});
});
});

View File

@@ -1,65 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptV1ChartData } from './utils';
describe('Advanced analytics', () => {
beforeEach(() => {
interceptV1ChartData();
cy.intercept('PUT', '**/api/v1/explore/**').as('putExplore');
cy.intercept('GET', '**/explore/**').as('getExplore');
});
it('Create custom time compare', () => {
cy.visitChartByName('Num Births Trend');
cy.verifySliceSuccess({ waitAlias: '@v1Data' });
cy.get('.ant-collapse-header')
.contains('Advanced analytics')
.click({ force: true });
cy.get('[data-test=time_compare]').find('.ant-select').click();
cy.get('[data-test=time_compare]')
.find('input[type=search]')
.type('28 days{enter}');
cy.get('[data-test=time_compare]').find('input[type=search]').clear();
cy.get('[data-test=time_compare]')
.find('input[type=search]')
.type('1 year{enter}');
cy.get('button[data-test="run-query-button"]').click();
cy.wait('@v1Data');
cy.wait('@putExplore');
cy.reload();
cy.verifySliceSuccess({
waitAlias: '@v1Data',
});
cy.wait('@getExplore');
cy.get('.ant-collapse-header')
.contains('Advanced analytics')
.click({ force: true });
cy.get('[data-test=time_compare]')
.find('.ant-select-selector')
.contains('28 days');
cy.get('[data-test=time_compare]')
.find('.ant-select-selector')
.contains('1 year');
});
});

View File

@@ -1,48 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptChart } from 'cypress/utils';
describe('Annotations', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
});
it('Create formula annotation y-axis goal line', () => {
cy.visitChartByName('Num Births Trend');
cy.verifySliceSuccess({ waitAlias: '@chartData' });
const layerLabel = 'Goal line';
// get by text Annotations and Layers
cy.get('span').contains('Annotations and Layers').click();
cy.get('[data-test=annotation_layers]').click();
cy.get('[data-test="popover-content"]').within(() => {
cy.get('[aria-label=Name]').type(layerLabel);
cy.get('[aria-label=Formula]').type('y=1400000');
cy.get('button').contains('OK').click();
});
cy.get('button[data-test="run-query-button"]').click();
cy.get('[data-test=annotation_layers]').contains(layerLabel);
cy.verifySliceSuccess({ waitAlias: '@chartData' });
});
});

View File

@@ -1,192 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
// ***********************************************
// Tests for links in the explore UI
// ***********************************************
import rison from 'rison';
import { nanoid } from 'nanoid';
import { interceptChart } from 'cypress/utils';
import { HEALTH_POP_FORM_DATA_DEFAULTS } from './visualizations/shared.helper';
const apiURL = (endpoint: string, queryObject: Record<string, unknown>) =>
`${endpoint}?q=${rison.encode(queryObject)}`;
describe('Test explore links', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
});
it('Open and close view query modal', () => {
cy.visitChartByName('Growth Rate');
cy.verifySliceSuccess({ waitAlias: '@chartData' });
cy.get('[aria-label="Menu actions trigger"]').click();
cy.get('span').contains('View query').parent().click();
cy.wait('@chartData').then(() => {
cy.get('code');
});
cy.get('.ant-modal-content').within(() => {
cy.get('button.ant-modal-close').first().click({ force: true });
});
});
it('Test iframe link', () => {
cy.visitChartByName('Growth Rate');
cy.verifySliceSuccess({ waitAlias: '@chartData' });
cy.get('[aria-label="Menu actions trigger"]').click();
cy.get('div[role="menuitem"]').within(() => {
cy.contains('Share').parent().click();
});
cy.getBySel('embed-code-button').click();
cy.get('#embed-code-popover').within(() => {
cy.get('textarea[name=embedCode]').contains('iframe');
});
});
it('Test chart save as AND overwrite', () => {
interceptChart({ legacy: false }).as('tableChartData');
const formData = {
...HEALTH_POP_FORM_DATA_DEFAULTS,
viz_type: 'table',
metrics: ['sum__SP_POP_TOTL'],
groupby: ['country_name'],
};
const newChartName = `Test chart [${nanoid()}]`;
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@tableChartData' });
cy.url().then(() => {
cy.get('[data-test="query-save-button"]').click();
cy.get('[data-test="saveas-radio"]').check();
cy.get('[data-test="new-chart-name"]').type(newChartName, {
force: true,
});
cy.get('[data-test="btn-modal-save"]').click();
cy.verifySliceSuccess({ waitAlias: '@tableChartData' });
cy.visitChartByName(newChartName);
// Overwriting!
cy.get('[data-test="query-save-button"]').click();
cy.get('[data-test="save-overwrite-radio"]').check();
cy.get('[data-test="btn-modal-save"]').click();
cy.verifySliceSuccess({ waitAlias: '@tableChartData' });
const query = {
filters: [
{
col: 'slice_name',
opr: 'eq',
value: newChartName,
},
],
};
cy.request(apiURL('/api/v1/chart/', query)).then(response => {
expect(response.body.count).equals(1);
});
cy.deleteChartByName(newChartName, true);
});
});
it('Test chart save as and add to new dashboard', () => {
const chartName = 'Growth Rate';
const newChartName = `${chartName} [${nanoid()}]`;
const dashboardTitle = `Test dashboard [${nanoid()}]`;
cy.visitChartByName(chartName);
cy.verifySliceSuccess({ waitAlias: '@chartData' });
cy.get('[data-test="query-save-button"]').click();
cy.get('[data-test="saveas-radio"]').check();
cy.get('[data-test="new-chart-name"]').click();
cy.get('[data-test="new-chart-name"]').clear();
cy.get('[data-test="new-chart-name"]').type(newChartName);
// Add a new option using the "CreatableSelect" feature
cy.get('[data-test="save-chart-modal-select-dashboard-form"]')
.find('input[aria-label="Select a dashboard"]')
.type(`${dashboardTitle}`, { force: true });
cy.get(`.ant-select-item[title="${dashboardTitle}"]`).click({
force: true,
});
cy.get('[data-test="btn-modal-save"]').click();
cy.verifySliceSuccess({ waitAlias: '@chartData' });
let query = {
filters: [
{
col: 'dashboard_title',
opr: 'eq',
value: dashboardTitle,
},
],
};
cy.request(apiURL('/api/v1/dashboard/', query)).then(response => {
expect(response.body.count).equals(1);
});
cy.visitChartByName(newChartName);
cy.verifySliceSuccess({ waitAlias: '@chartData' });
cy.get('[data-test="query-save-button"]').click();
cy.get('[data-test="save-overwrite-radio"]').check();
cy.get('[data-test="new-chart-name"]').click();
cy.get('[data-test="new-chart-name"]').clear();
cy.get('[data-test="new-chart-name"]').type(newChartName);
// This time around, typing the same dashboard name
// will select the existing one
cy.get('[data-test="save-chart-modal-select-dashboard-form"]')
.find('input[aria-label^="Select a dashboard"]')
.type(`${dashboardTitle}{enter}`, { force: true });
cy.get(`.ant-select-item[title="${dashboardTitle}"]`).click({
force: true,
});
cy.get('[data-test="btn-modal-save"]').click();
cy.verifySliceSuccess({ waitAlias: '@chartData' });
query = {
filters: [
{
col: 'slice_name',
opr: 'eq',
value: chartName,
},
],
};
cy.request(apiURL('/api/v1/chart/', query)).then(response => {
expect(response.body.count).equals(1);
});
query = {
filters: [
{
col: 'dashboard_title',
opr: 'eq',
value: dashboardTitle,
},
],
};
cy.request(apiURL('/api/v1/dashboard/', query)).then(response => {
expect(response.body.count).equals(1);
});
cy.deleteDashboardByName(dashboardTitle, true);
});
});

View File

@@ -1,80 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptChart } from 'cypress/utils';
describe('Visualization > Big Number with Trendline', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
});
const BIG_NUMBER_FORM_DATA = {
datasource: '2__table',
viz_type: 'big_number',
slice_id: 42,
granularity_sqla: 'year',
time_grain_sqla: 'P1D',
time_range: '2000 : 2014-01-02',
metric: 'sum__SP_POP_TOTL',
adhoc_filters: [],
compare_lag: '10',
compare_suffix: 'over 10Y',
y_axis_format: '.3s',
show_trend_line: true,
start_y_axis_at_zero: true,
color_picker: {
r: 0,
g: 122,
b: 135,
a: 1,
},
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@chartData',
chartSelector: '.superset-legacy-chart-big-number',
});
}
it('should work', () => {
verify(BIG_NUMBER_FORM_DATA);
cy.get('.chart-container .header-line');
cy.get('.chart-container canvas');
});
it('should work without subheader', () => {
verify({
...BIG_NUMBER_FORM_DATA,
compare_lag: null,
});
cy.get('.chart-container .header-line');
cy.get('.chart-container .subtitle-line').should('not.exist');
cy.get('.chart-container canvas');
});
it('should not render trendline when hidden', () => {
verify({
...BIG_NUMBER_FORM_DATA,
show_trend_line: false,
});
cy.get('[data-test="chart-container"] .header-line');
cy.get('[data-test="chart-container"] canvas').should('not.exist');
});
});

View File

@@ -1,79 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptChart } from 'cypress/utils';
import { FORM_DATA_DEFAULTS, NUM_METRIC } from './shared.helper';
describe('Visualization > Big Number Total', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
});
const BIG_NUMBER_DEFAULTS = {
...FORM_DATA_DEFAULTS,
viz_type: 'big_number_total',
};
it('Test big number chart with adhoc metric', () => {
const formData = { ...BIG_NUMBER_DEFAULTS, metric: NUM_METRIC };
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: NUM_METRIC.label,
});
});
it('Test big number chart with simple filter', () => {
const filters = [
{
expressionType: 'SIMPLE',
subject: 'name',
operator: 'IN',
comparator: ['Aaron', 'Amy', 'Andrea'],
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_4y6teao56zs_ebjsvwy48c',
},
];
const formData = {
...BIG_NUMBER_DEFAULTS,
metric: 'count',
adhoc_filters: filters,
};
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@chartData' });
});
it('Test big number chart ignores groupby', () => {
const formData = {
...BIG_NUMBER_DEFAULTS,
metric: NUM_METRIC,
groupby: ['state'],
};
cy.visitChartByParams(formData);
cy.wait(['@chartData']).then(async ({ response }) => {
cy.verifySliceContainer();
const responseBody = response?.body;
expect(responseBody.result[0].query).not.contains(formData.groupby[0]);
});
});
});

View File

@@ -1,65 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { getDatasetId } from './shared.helper';
describe('Visualization > Box Plot', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data*').as('getJson');
});
const getBoxPlotFormData = datasetId => ({
datasource: `${datasetId}__table`,
viz_type: 'box_plot',
granularity_sqla: 'year',
time_grain_sqla: 'P1D',
time_range: '1960-01-01 : now',
metrics: ['sum__SP_POP_TOTL'],
adhoc_filters: [],
groupby: ['region'],
limit: '25',
color_scheme: 'bnbColors',
whisker_options: 'Min/max (no outliers)',
});
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson' });
}
it('should work', () => {
getDatasetId('wb_health_population').then(datasetId => {
verify(getBoxPlotFormData(datasetId));
cy.get('.chart-container .box_plot canvas').should('have.length', 1);
});
});
it('should allow type to search color schemes', () => {
getDatasetId('wb_health_population').then(datasetId => {
verify(getBoxPlotFormData(datasetId));
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('supersetColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="supersetColors"]',
).should('exist');
});
});
});

View File

@@ -1,108 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { getDatasetId } from './shared.helper';
describe('Visualization > Bubble', () => {
beforeEach(() => {
cy.intercept('POST', '**/superset/explore_json/**').as('getJson');
});
const getBubbleFormData = datasetId => ({
datasource: `${datasetId}__table`,
viz_type: 'bubble',
granularity_sqla: 'year',
time_grain_sqla: 'P1D',
time_range: '2011-01-01 : 2011-01-02',
series: 'region',
entity: 'country_name',
x: 'sum__SP_RUR_TOTL_ZS',
y: 'sum__SP_DYN_LE00_IN',
size: 'sum__SP_POP_TOTL',
max_bubble_size: '50',
limit: 0,
color_scheme: 'bnbColors',
show_legend: true,
x_axis_label: '',
left_margin: 'auto',
x_axis_format: '.3s',
x_ticks_layout: 'auto',
x_log_scale: false,
x_axis_showminmax: false,
y_axis_label: '',
bottom_margin: 'auto',
y_axis_format: '.3s',
y_log_scale: false,
y_axis_showminmax: false,
});
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson', chartSelector: 'svg' });
}
it('should work with filter', () => {
getDatasetId('wb_health_population').then(datasetId => {
verify({
...getBubbleFormData(datasetId),
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'region',
operator: '==',
comparator: 'South Asia',
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_b2tfg1rs8y_8kmrcyxvsqd',
},
],
});
cy.get('[data-test="chart-container"]').should('be.visible');
cy.get('[data-test="chart-container"]').within(() => {
cy.get('svg').find('.nv-point-clips circle').should('have.length', 8);
});
cy.get('[data-test="chart-container"]').then(nodeList => {
// Check that all circles have same color.
const color = nodeList[0].getAttribute('fill');
const circles = Array.prototype.slice.call(nodeList);
expect(circles.every(c => c.getAttribute('fill') === color)).to.equal(
true,
);
});
});
});
it('should allow type to search color schemes and apply the scheme', () => {
getDatasetId('wb_health_population').then(datasetId => {
cy.visitChartByParams(getBubbleFormData(datasetId));
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('supersetColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="supersetColors"]',
).should('exist');
cy.get('[data-test=run-query-button]').click();
cy.get('.bubble .nv-legend .nv-legend-symbol').should(
'have.css',
'fill',
'rgb(31, 168, 201)',
);
});
});
});

View File

@@ -1,100 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > Compare', () => {
beforeEach(() => {
cy.intercept('POST', '**/superset/explore_json/**').as('getJson');
});
const COMPARE_FORM_DATA = {
datasource: '3__table',
viz_type: 'compare',
slice_id: 60,
granularity_sqla: 'ds',
time_grain_sqla: 'P1D',
time_range: '100 years ago : now',
metrics: ['count'],
adhoc_filters: [],
groupby: [],
order_desc: true,
contribution: false,
row_limit: 50000,
color_scheme: 'bnbColors',
x_axis_label: 'Frequency',
bottom_margin: 'auto',
x_ticks_layout: 'auto',
x_axis_format: 'smart_date',
x_axis_showminmax: false,
y_axis_label: 'Num',
left_margin: 'auto',
y_axis_showminmax: false,
y_log_scale: false,
y_axis_format: '.3s',
rolling_type: 'None',
comparison_type: 'values',
annotation_layers: [],
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson', chartSelector: 'svg' });
}
it('should work without groupby', () => {
verify(COMPARE_FORM_DATA);
cy.get('.chart-container .nvd3 path.nv-line').should('have.length', 1);
});
it('should with group by', () => {
verify({
...COMPARE_FORM_DATA,
groupby: ['gender'],
});
cy.get('.chart-container .nvd3 path.nv-line').should('have.length', 2);
});
it('should work with filter', () => {
verify({
...COMPARE_FORM_DATA,
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'gender',
operator: '==',
comparator: 'boy',
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_tqx1en70hh_7nksse7nqic',
},
],
});
cy.get('.chart-container .nvd3 path.nv-line').should('have.length', 1);
});
it('should allow type to search color schemes and apply the scheme', () => {
verify(COMPARE_FORM_DATA);
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('supersetColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="supersetColors"]',
).should('exist');
});
});

View File

@@ -1,54 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { FORM_DATA_DEFAULTS, NUM_METRIC } from './shared.helper';
describe('Download Chart > Bar chart', () => {
const VIZ_DEFAULTS = {
...FORM_DATA_DEFAULTS,
viz_type: 'echarts_timeseries_bar',
};
beforeEach(() => {
cy.intercept('POST', '**/superset/explore_json/**').as('getJson');
});
it('download chart with image works', () => {
const formData = {
...VIZ_DEFAULTS,
metrics: NUM_METRIC,
groupby: ['state'],
};
cy.visitChartByParams(formData);
cy.get('.header-with-actions .ant-dropdown-trigger').click();
cy.get(':nth-child(3) > .ant-dropdown-menu-submenu-title').click();
cy.get(
'.ant-dropdown-menu-submenu > .ant-dropdown-menu li:nth-child(1) > .ant-dropdown-menu-submenu-title',
).click();
cy.get(
'.ant-dropdown-menu-submenu > .ant-dropdown-menu li:nth-child(3)',
).click();
cy.verifyDownload('.jpg', {
contains: true,
timeout: 25000,
interval: 600,
});
});
});

View File

@@ -1,75 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > Gauge', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data*').as('getJson');
});
const GAUGE_FORM_DATA = {
datasource: '3__table',
viz_type: 'gauge_chart',
metric: 'count',
adhoc_filters: [],
slice_id: 54,
row_limit: 10,
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson' });
}
it('should work', () => {
verify(GAUGE_FORM_DATA);
cy.get('.chart-container .gauge_chart canvas').should('have.length', 1);
});
it('should work with simple filter', () => {
verify({
...GAUGE_FORM_DATA,
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'country_code',
operator: '==',
comparator: 'USA',
clause: 'WHERE',
sqlExpression: null,
isExtra: false,
isNew: false,
filterOptionName: 'filter_jaemvkxd5h_ku22m3wyo',
},
],
});
cy.get('.chart-container .gauge_chart canvas').should('have.length', 1);
});
it('should allow type to search color schemes', () => {
verify(GAUGE_FORM_DATA);
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('bnbColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="bnbColors"]',
).should('exist');
});
});

View File

@@ -1,91 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
type adhocFilter = {
expressionType: string;
subject: string;
operator: string;
comparator: string;
clause: string;
sqlExpression: string | null;
filterOptionName: string;
};
describe('Visualization > Graph', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data*').as('getJson');
});
const GRAPH_FORM_DATA = {
datasource: '1__table',
viz_type: 'graph_chart',
slice_id: 55,
granularity_sqla: 'ds',
time_grain_sqla: 'P1D',
time_range: '100 years ago : now',
metric: 'sum__value',
adhoc_filters: [],
source: 'source',
target: 'target',
row_limit: 50000,
show_legend: true,
color_scheme: 'bnbColors',
};
function verify(formData: {
[name: string]: string | boolean | number | Array<adhocFilter>;
}): void {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson' });
}
it('should work with ad-hoc metric', () => {
verify(GRAPH_FORM_DATA);
cy.get('.chart-container .graph_chart canvas').should('have.length', 1);
});
it('should work with simple filter', () => {
verify({
...GRAPH_FORM_DATA,
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'source',
operator: '==',
comparator: 'Agriculture',
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_tqx1en70hh_7nksse7nqic',
},
],
});
cy.get('.chart-container .graph_chart canvas').should('have.length', 1);
});
it('should allow type to search color schemes', () => {
verify(GRAPH_FORM_DATA);
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('bnbColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="bnbColors"]',
).should('exist');
});
});

View File

@@ -1,82 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > Pie', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data*').as('getJson');
});
const PIE_FORM_DATA = {
datasource: '3__table',
viz_type: 'pie',
slice_id: 55,
granularity_sqla: 'ds',
time_grain_sqla: 'P1D',
time_range: '100 years ago : now',
metric: 'sum__num',
adhoc_filters: [],
groupby: ['gender'],
row_limit: 50000,
pie_label_type: 'key',
donut: false,
show_legend: true,
show_labels: true,
labels_outside: true,
color_scheme: 'bnbColors',
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson' });
}
it('should work with ad-hoc metric', () => {
verify(PIE_FORM_DATA);
cy.get('.chart-container .pie canvas').should('have.length', 1);
});
it('should work with simple filter', () => {
verify({
...PIE_FORM_DATA,
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'gender',
operator: '==',
comparator: 'boy',
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_tqx1en70hh_7nksse7nqic',
},
],
});
cy.get('.chart-container .pie canvas').should('have.length', 1);
});
it('should allow type to search color schemes', () => {
verify(PIE_FORM_DATA);
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('supersetColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="supersetColors"]',
).should('exist');
});
});

View File

@@ -1,106 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > Pivot Table', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data**').as('chartData');
});
const PIVOT_TABLE_FORM_DATA = {
datasource: '3__table',
viz_type: 'pivot_table_v2',
slice_id: 61,
granularity_sqla: 'ds',
time_grain_sqla: 'P1D',
time_range: '100 years ago : now',
metrics: ['sum__num'],
adhoc_filters: [],
groupbyRows: ['name'],
groupbyColumns: ['state'],
series_limit: 5000,
aggregateFunction: 'Sum',
rowTotals: true,
colTotals: true,
valueFormat: '.3s',
combineMetric: false,
};
const TEST_METRIC = {
expressionType: 'SIMPLE',
column: {
id: 338,
column_name: 'num_boys',
expression: '',
filterable: false,
groupby: false,
is_dttm: false,
type: 'BIGINT',
optionName: '_col_num_boys',
},
aggregate: 'SUM',
hasCustomLabel: false,
label: 'SUM(num_boys)',
optionName: 'metric_gvpdjt0v2qf_6hkf56o012',
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
}
it('should work with single groupby', () => {
verify(PIVOT_TABLE_FORM_DATA);
cy.get('.chart-container tr:eq(0) th:eq(2)').contains('sum__num');
cy.get('.chart-container tr:eq(1) th:eq(0)').contains('state');
cy.get('.chart-container tr:eq(2) th:eq(0)').contains('name');
});
it('should work with more than one groupby', () => {
verify({
...PIVOT_TABLE_FORM_DATA,
groupbyRows: ['name', 'gender'],
});
cy.get('.chart-container tr:eq(0) th:eq(2)').contains('sum__num');
cy.get('.chart-container tr:eq(1) th:eq(0)').contains('state');
cy.get('.chart-container tr:eq(2) th:eq(0)').contains('name');
cy.get('.chart-container tr:eq(2) th:eq(1)').contains('gender');
});
it('should work with multiple metrics', () => {
verify({
...PIVOT_TABLE_FORM_DATA,
metrics: ['sum__num', TEST_METRIC],
});
cy.get('.chart-container tr:eq(0) th:eq(2)').contains('sum__num');
cy.get('.chart-container tr:eq(0) th:eq(3)').contains('SUM(num_boys)');
cy.get('.chart-container tr:eq(1) th:eq(0)').contains('state');
cy.get('.chart-container tr:eq(2) th:eq(0)').contains('name');
});
it('should work with multiple groupby and multiple metrics', () => {
verify({
...PIVOT_TABLE_FORM_DATA,
groupbyRows: ['name', 'gender'],
metrics: ['sum__num', TEST_METRIC],
});
cy.get('.chart-container tr:eq(0) th:eq(2)').contains('sum__num');
cy.get('.chart-container tr:eq(0) th:eq(3)').contains('SUM(num_boys)');
cy.get('.chart-container tr:eq(2) th:eq(0)').contains('name');
cy.get('.chart-container tr:eq(2) th:eq(1)').contains('gender');
});
});

View File

@@ -1,97 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > Sunburst', () => {
beforeEach(() => {
cy.intercept('POST', '**/api/v1/chart/data**').as('chartData');
});
const SUNBURST_FORM_DATA = {
datasource: '2__table',
viz_type: 'sunburst_v2',
slice_id: 47,
granularity_sqla: 'year',
time_grain_sqla: 'P1D',
time_range: 'No filter',
columns: ['region'],
metric: 'sum__SP_POP_TOTL',
adhoc_filters: [],
row_limit: 50000,
color_scheme: 'bnbColors',
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@chartData' });
}
// requires the ability to render charts using SVG only for tests
it.skip('should work without secondary metric', () => {
verify(SUNBURST_FORM_DATA);
cy.get('.chart-container svg g path').should('have.length', 7);
});
// requires the ability to render charts using SVG only for tests
it.skip('should work with secondary metric', () => {
verify({
...SUNBURST_FORM_DATA,
secondary_metric: 'sum__SP_RUR_TOTL',
});
cy.get('.chart-container svg g path').should('have.length', 7);
});
// requires the ability to render charts using SVG only for tests
it.skip('should work with multiple columns', () => {
verify({
...SUNBURST_FORM_DATA,
columns: ['region', 'country_name'],
});
cy.get('.chart-container svg g path').should('have.length', 221);
});
// requires the ability to render charts using SVG only for tests
it.skip('should work with filter', () => {
verify({
...SUNBURST_FORM_DATA,
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'region',
operator: 'IN',
comparator: ['South Asia', 'North America'],
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_txje2ikiv6_wxmn0qwd1xo',
},
],
});
cy.get('.chart-container svg g path').should('have.length', 2);
});
it('should allow type to search color schemes', () => {
verify(SUNBURST_FORM_DATA);
cy.get('#controlSections-tab-CUSTOMIZE').click();
cy.get('.Control[data-test="color_scheme"]').scrollIntoView();
cy.get('.Control[data-test="color_scheme"] input[type="search"]').focus();
cy.focused().type('supersetColors{enter}');
cy.get(
'.Control[data-test="color_scheme"] .ant-select-selection-item [data-test="supersetColors"]',
).should('exist');
});
});

View File

@@ -1,474 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { interceptChart } from 'cypress/utils';
import {
FORM_DATA_DEFAULTS,
NUM_METRIC,
MAX_DS,
MAX_STATE,
SIMPLE_FILTER,
} from './shared.helper';
// Table
describe('Visualization > Table', () => {
beforeEach(() => {
interceptChart({ legacy: false }).as('chartData');
});
const VIZ_DEFAULTS = {
...FORM_DATA_DEFAULTS,
viz_type: 'table',
row_limit: 1000,
};
const PERCENT_METRIC = {
expressionType: 'SQL',
sqlExpression: 'CAST(SUM(num_girls) AS FLOAT)/SUM(num)',
column: null,
aggregate: null,
hasCustomLabel: true,
label: 'Girls',
optionName: 'metric_6qwzgc8bh2v_zox7hil1mzs',
};
it('Use default time column', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
granularity_sqla: undefined,
metrics: ['count'],
});
cy.get('[data-test=adhoc_filters]').contains('ds');
});
it('Format non-numeric metrics correctly', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
include_time: true,
granularity_sqla: 'ds',
time_grain_sqla: 'P3M',
metrics: [NUM_METRIC, MAX_DS, MAX_STATE],
});
// when format with smart_date, time column use format by granularity
cy.get('.chart-container td:nth-child(1)').contains('2008 Q1');
// other column with timestamp use adaptive formatting
cy.get('.chart-container td:nth-child(3)').contains('2008');
cy.get('.chart-container td:nth-child(4)').contains('TX');
});
it('Format with table_timestamp_format', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
include_time: true,
granularity_sqla: 'ds',
time_grain_sqla: 'P3M',
table_timestamp_format: '%Y-%m-%d %H:%M',
metrics: [NUM_METRIC, MAX_DS, MAX_STATE],
});
// time column and MAX(ds) metric column both use UTC time
cy.get('.chart-container td:nth-child(1)').contains('2008-01-01 00:00');
cy.get('.chart-container td:nth-child(3)').contains('2008-01-01 00:00');
cy.get('.chart-container td')
.contains('2008-01-01 08:00')
.should('not.exist');
// time column should not use time granularity when timestamp format is set
cy.get('.chart-container td').contains('2008 Q1').should('not.exist');
// other num numeric metric column should stay as string
cy.get('.chart-container td').contains('TX');
});
it('Test table with groupby', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: [NUM_METRIC, MAX_DS],
groupby: ['name'],
});
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: /GROUP BY.*name/i,
chartSelector: 'table',
});
});
it('Test table with groupby + time column', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
include_time: true,
granularity_sqla: 'ds',
time_grain_sqla: 'P3M',
metrics: [NUM_METRIC, MAX_DS],
groupby: ['name'],
});
cy.wait('@chartData').then(({ response }) => {
cy.verifySliceContainer('table');
const records = response?.body.result[0].data;
// should sort by first metric when no sort by metric is set
expect(records[0][NUM_METRIC.label]).greaterThan(
records[1][NUM_METRIC.label],
);
});
// should handle frontend sorting correctly
cy.get('.chart-container th').contains('name').click();
cy.get('.chart-container td:nth-child(2):eq(0)').contains('Adam');
cy.get('.chart-container th').contains('ds').click();
cy.get('.chart-container th').contains('ds').click();
cy.get('.chart-container td:nth-child(1):eq(0)').contains('2008');
});
it('Test table with percent metrics and groupby', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
percent_metrics: PERCENT_METRIC,
metrics: [],
groupby: ['name'],
});
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
});
it('Test table with groupby order desc', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: NUM_METRIC,
groupby: ['name'],
order_desc: true,
});
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
});
it('Test table with groupby + order by + no metric', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: [],
groupby: ['name'],
timeseries_limit_metric: NUM_METRIC,
order_desc: true,
});
// should contain only the group by column
cy.get('.chart-container th').its('length').should('eq', 1);
// should order correctly
cy.get('.chart-container td:eq(0)').contains('Michael');
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
});
it('Test table with groupby and limit', () => {
const limit = 10;
const formData = {
...VIZ_DEFAULTS,
metrics: NUM_METRIC,
groupby: ['name'],
row_limit: limit,
};
cy.visitChartByParams(formData);
cy.wait('@chartData').then(({ response }) => {
cy.verifySliceContainer('table');
expect(response?.body.result[0].data.length).to.eq(limit);
});
cy.get('[data-test="row-count-label"]').contains('10 rows');
});
it('Test table with columns and row limit', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
// should still work when query_mode is not-set/invalid
query_mode: undefined,
all_columns: ['state'],
metrics: [],
row_limit: 100,
});
// should display in raw records mode
cy.get(
'div[data-test="query_mode"] .ant-radio-button-wrapper-checked',
).contains('Raw records');
cy.get('div[data-test="all_columns"]').should('be.visible');
cy.get('div[data-test="groupby"]').should('not.exist');
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
cy.get('[data-test="row-count-label"]').contains('100 rows');
// should allow switch back to aggregate mode
cy.get('div[data-test="query_mode"] .ant-radio-button-wrapper')
.contains('Aggregate')
.click();
cy.get(
'div[data-test="query_mode"] .ant-radio-button-wrapper-checked',
).contains('Aggregate');
cy.get('div[data-test="all_columns"]').should('not.exist');
cy.get('div[data-test="groupby"]').should('be.visible');
});
it('Test table with columns, ordering, and row limit', () => {
const limit = 10;
const formData = {
...VIZ_DEFAULTS,
query_mode: 'raw',
all_columns: ['name', 'state', 'ds', 'num'],
metrics: [],
row_limit: limit,
order_by_cols: ['["num", false]'],
};
cy.visitChartByParams(formData);
cy.wait('@chartData').then(({ response }) => {
cy.verifySliceContainer('table');
const records = response?.body.result[0].data;
expect(records[0].num).greaterThan(records[records.length - 1].num);
});
});
it('Test table with simple filter', () => {
const metrics = ['count'];
const filters = [SIMPLE_FILTER];
const formData = { ...VIZ_DEFAULTS, metrics, adhoc_filters: filters };
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@chartData', chartSelector: 'table' });
});
it('Tests table number formatting with % in metric name', () => {
const formData = {
...VIZ_DEFAULTS,
percent_metrics: PERCENT_METRIC,
groupby: ['state'],
};
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@chartData',
querySubstring: /GROUP BY.*state/i,
chartSelector: 'table',
});
cy.get('td').contains(/\d*%/);
});
it('Test row limit with server pagination toggle', () => {
const serverPaginationSelector =
'[data-test="server_pagination-header"] div.pull-left [type="checkbox"]';
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: ['count'],
row_limit: 100,
});
// Enable server pagination
cy.get(serverPaginationSelector).click();
// Click row limit control and select high value (200k)
cy.get('div[aria-label="Row limit"]').click();
// Type 200000 and press enter to select the option
cy.get('div[aria-label="Row limit"]')
.find('.ant-select-selection-search-input:visible')
.type('200000{enter}');
// Verify that there is no error tooltip when server pagination is enabled
cy.get('[data-test="error-tooltip"]').should('not.exist');
// Disable server pagination
cy.get(serverPaginationSelector).click();
// Verify error tooltip appears
cy.get('[data-test="error-tooltip"]').should('be.visible');
// Trigger mouseover and verify tooltip text
cy.get('[data-test="error-tooltip"]').trigger('mouseover');
// Verify tooltip content
cy.get('.ant-tooltip-inner').should('be.visible');
cy.get('.ant-tooltip-inner').should(
'contain',
'Server pagination needs to be enabled for values over',
);
// Hide the tooltip by adding display:none style
cy.get('.ant-tooltip').invoke('attr', 'style', 'display: none');
// Enable server pagination again
cy.get(serverPaginationSelector).click();
cy.get('[data-test="error-tooltip"]').should('not.exist');
cy.get('div[aria-label="Row limit"]').click();
// Type 1000000
cy.get('div[aria-label="Row limit"]')
.find('.ant-select-selection-search-input:visible')
.type('1000000');
// Wait for 1 second
cy.wait(1000);
// Press enter
cy.get('div[aria-label="Row limit"]')
.find('.ant-select-selection-search-input:visible')
.type('{enter}');
// Wait for error tooltip to appear and verify its content
cy.get('[data-test="error-tooltip"]')
.should('be.visible')
.trigger('mouseover');
// Wait for tooltip content and verify
cy.get('.ant-tooltip-inner').should('exist');
cy.get('.ant-tooltip-inner').should('be.visible');
// Verify tooltip content separately
cy.get('.ant-tooltip-inner').should('contain', 'Value cannot exceed');
});
it('Test sorting with server pagination enabled', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: ['count'],
groupby: ['name'],
row_limit: 100000,
server_pagination: true, // Enable server pagination
});
// Wait for the initial data load
cy.wait('@chartData');
// Get the first column header (name)
cy.get('.chart-container th').contains('name').as('nameHeader');
// Click to sort ascending
cy.get('@nameHeader').click();
cy.wait('@chartData');
// Verify first row starts with 'A'
cy.get('.chart-container td:first').invoke('text').should('match', /^[Aa]/);
// Click again to sort descending
cy.get('@nameHeader').click();
cy.wait('@chartData');
// Verify first row starts with 'Z'
cy.get('.chart-container td:first').invoke('text').should('match', /^[Zz]/);
// Test numeric sorting
cy.get('.chart-container th').contains('COUNT').as('countHeader');
// Click to sort ascending by count
cy.get('@countHeader').click();
cy.wait('@chartData');
// Get first two count values and verify ascending order
cy.get('.chart-container td:nth-child(2)').then($cells => {
const first = parseFloat($cells[0].textContent || '0');
const second = parseFloat($cells[1].textContent || '0');
expect(first).to.be.at.most(second);
});
// Click again to sort descending
cy.get('@countHeader').click();
cy.wait('@chartData');
// Get first two count values and verify descending order
cy.get('.chart-container td:nth-child(2)').then($cells => {
const first = parseFloat($cells[0].textContent || '0');
const second = parseFloat($cells[1].textContent || '0');
expect(first).to.be.at.least(second);
});
});
it('Test search with server pagination enabled', () => {
cy.visitChartByParams({
...VIZ_DEFAULTS,
metrics: ['count'],
groupby: ['name', 'state'],
row_limit: 100000,
server_pagination: true,
include_search: true,
});
cy.wait('@chartData');
const searchInputSelector = '.dt-global-filter input';
// Basic search test
cy.get(searchInputSelector).should('be.visible');
cy.get(searchInputSelector).type('John');
cy.wait('@chartData');
cy.get('.chart-container tbody tr').each($row => {
cy.wrap($row).contains(/John/i);
});
// Clear and test case-insensitive search
cy.get(searchInputSelector).clear();
cy.wait('@chartData');
cy.get(searchInputSelector).type('mary');
cy.wait('@chartData');
cy.get('.chart-container tbody tr').each($row => {
cy.wrap($row).contains(/Mary/i);
});
// Test special characters
cy.get(searchInputSelector).clear();
cy.get(searchInputSelector).type('Nicole');
cy.wait('@chartData');
cy.get('.chart-container tbody tr').each($row => {
cy.wrap($row).contains(/Nicole/i);
});
// Test no results
cy.get(searchInputSelector).clear();
cy.get(searchInputSelector).type('XYZ123');
cy.wait('@chartData');
cy.get('.chart-container').contains('No records found');
// Test column-specific search
cy.get('.search-select').should('be.visible');
cy.get('.search-select').click();
cy.get('.ant-select-dropdown').should('be.visible');
cy.get('.ant-select-item-option').contains('state').should('be.visible');
cy.get('.ant-select-item-option').contains('state').click();
cy.get(searchInputSelector).clear();
cy.get(searchInputSelector).type('CA');
cy.wait('@chartData');
cy.wait(1000);
cy.get('td[aria-labelledby="header-state"]').should('be.visible');
cy.get('td[aria-labelledby="header-state"]')
.first()
.should('contain', 'CA');
});
});

View File

@@ -1,130 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { FORM_DATA_DEFAULTS, NUM_METRIC } from './shared.helper';
describe('Visualization > Time TableViz', () => {
beforeEach(() => {
cy.intercept('POST', '**/superset/explore_json/**').as('getJson');
});
const VIZ_DEFAULTS = { ...FORM_DATA_DEFAULTS, viz_type: 'time_table' };
it('Test time series table multiple metrics last year total', () => {
const formData = {
...VIZ_DEFAULTS,
metrics: [NUM_METRIC, 'count'],
column_collection: [
{
key: '9g4K-B-YL',
label: 'Last Year',
colType: 'time',
timeLag: '1',
comparisonType: 'value',
},
],
url: '',
};
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@getJson',
querySubstring: NUM_METRIC.label,
});
cy.get('[data-test="time-table"]').within(() => {
cy.get('span').contains('Sum(num)');
cy.get('span').contains('COUNT(*)');
});
});
it('Test time series table metric and group by last year total', () => {
const formData = {
...VIZ_DEFAULTS,
metrics: [NUM_METRIC],
groupby: ['gender'],
column_collection: [
{
key: '9g4K-B-YL',
label: 'Last Year',
colType: 'time',
timeLag: '1',
comparisonType: 'value',
},
],
url: '',
};
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@getJson',
querySubstring: NUM_METRIC.label,
});
cy.get('[data-test="time-table"]').within(() => {
cy.get('td').contains('boy');
cy.get('td').contains('girl');
});
});
it('Test time series various time columns', () => {
const formData = {
...VIZ_DEFAULTS,
metrics: [NUM_METRIC, 'count'],
column_collection: [
{ key: 'LHHNPhamU', label: 'Current', colType: 'time', timeLag: 0 },
{
key: '9g4K-B-YL',
label: 'Last Year',
colType: 'time',
timeLag: '1',
comparisonType: 'value',
},
{
key: 'JVZXtNu7_',
label: 'YoY',
colType: 'time',
timeLag: 1,
comparisonType: 'perc',
d3format: '%',
},
{ key: 'tN5Gba36u', label: 'Trend', colType: 'spark' },
],
url: '',
};
cy.visitChartByParams(formData);
cy.verifySliceSuccess({
waitAlias: '@getJson',
querySubstring: NUM_METRIC.label,
});
cy.get('[data-test="time-table"]').within(() => {
cy.get('th').contains('Current');
cy.get('th').contains('Last Year');
cy.get('th').contains('YoY');
cy.get('th').contains('Trend');
cy.get('span').contains('%');
cy.get('svg')
.first()
.then(charts => {
const firstChart = charts[0];
expect(firstChart.clientWidth).greaterThan(0);
expect(firstChart.clientHeight).greaterThan(0);
});
});
});
});

View File

@@ -1,95 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
describe('Visualization > World Map', () => {
beforeEach(() => {
cy.intercept('POST', '**/superset/explore_json/**').as('getJson');
});
const WORLD_MAP_FORM_DATA = {
datasource: '2__table',
viz_type: 'world_map',
slice_id: 45,
granularity_sqla: 'year',
time_grain_sqla: 'P1D',
time_range: '2014-01-01 : 2014-01-02',
entity: 'country_code',
country_fieldtype: 'cca3',
metric: 'sum__SP_RUR_TOTL_ZS',
adhoc_filters: [],
row_limit: 50000,
show_bubbles: true,
secondary_metric: 'sum__SP_POP_TOTL',
max_bubble_size: '25',
};
function verify(formData) {
cy.visitChartByParams(formData);
cy.verifySliceSuccess({ waitAlias: '@getJson', chartSelector: 'svg' });
}
it('should work with ad-hoc metric', () => {
verify(WORLD_MAP_FORM_DATA);
cy.get('.bubbles circle.datamaps-bubble').should('have.length', 206);
});
it('should work with simple filter', () => {
verify({
...WORLD_MAP_FORM_DATA,
metric: 'count',
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'region',
operator: '==',
comparator: 'South Asia',
clause: 'WHERE',
sqlExpression: null,
filterOptionName: 'filter_8aqxcf5co1a_x7lm2d1fq0l',
},
],
});
cy.get('.bubbles circle.datamaps-bubble').should('have.length', 8);
});
it('should hide bubbles when told so', () => {
verify({
...WORLD_MAP_FORM_DATA,
show_bubbles: false,
});
cy.get('.slice_container').then(containers => {
expect(
containers[0].querySelectorAll('.bubbles circle.datamaps-bubble')
.length,
).to.equal(0);
});
});
it('should allow type to search color schemes', () => {
verify(WORLD_MAP_FORM_DATA);
cy.get('.Control[data-test="linear_color_scheme"]').scrollIntoView();
cy.get(
'.Control[data-test="linear_color_scheme"] input[type="search"]',
).focus();
cy.focused().type('greens{enter}');
cy.get(
'.Control[data-test="linear_color_scheme"] .ant-select-selection-item [data-test="greens"]',
).should('exist');
});
});

View File

@@ -39,7 +39,7 @@
// oxlint versions (not actually enforced). Documented here for future
// maintainers — if/when oxlint adds them, re-enable in the relevant
// plugin section above.
// import: newline-after-import, no-extraneous-dependencies,
// import: no-extraneous-dependencies,
// no-import-module-exports, no-relative-packages,
// no-unresolved, no-useless-path-segments
// react: default-props-match-prop-types, destructuring-assignment,
@@ -47,7 +47,6 @@
// forbid-prop-types, function-component-definition,
// jsx-no-bind, jsx-uses-vars, no-access-state-in-setstate,
// no-deprecated, no-did-update-set-state, no-typos,
// no-unstable-nested-components,
// no-unused-class-component-methods, no-unused-prop-types,
// no-unused-state, prefer-stateless-function, prop-types,
// require-default-props, sort-comp, static-property-placement
@@ -137,6 +136,7 @@
"import/no-self-import": "error",
"import/no-cycle": "off",
"import/prefer-default-export": "off",
"import/newline-after-import": "error",
// === React plugin rules ===
"react/jsx-filename-extension": [
@@ -184,6 +184,10 @@
"error",
{ "button": true, "submit": true, "reset": false }
],
// TODO: Graduate to "error" after cleanup pass — ~150 violations
// across the codebase require hoisting nested component definitions
// out of their parent render functions.
"react/no-unstable-nested-components": "warn",
// === React Hooks rules ===
// TODO: Fix conditional hook usage and anonymous component issues
@@ -271,7 +275,10 @@
},
"overrides": [
{
"files": ["plugins/plugin-chart-table/src/TableChart.tsx", "plugins/plugin-chart-pivot-table/src/react-pivottable/TableRenderers.tsx"],
"files": [
"plugins/plugin-chart-table/src/TableChart.tsx",
"plugins/plugin-chart-pivot-table/src/react-pivottable/TableRenderers.tsx"
],
"rules": {
"jsx-a11y/no-redundant-roles": "off"
}

View File

@@ -31,7 +31,7 @@
"@fontsource/fira-code": "^5.2.7",
"@fontsource/ibm-plex-mono": "^5.2.7",
"@fontsource/inter": "^5.2.8",
"@googleapis/sheets": "^13.0.1",
"@googleapis/sheets": "^13.0.2",
"@great-expectations/jsonforms-antd-renderers": "^2.2.10",
"@jsonforms/core": "^3.7.0",
"@jsonforms/react": "^3.7.0",
@@ -96,7 +96,7 @@
"fs-extra": "^11.3.5",
"fuse.js": "^7.3.0",
"geolib": "^3.3.14",
"geostyler": "^18.5.1",
"geostyler": "^18.6.0",
"geostyler-data": "^1.1.0",
"geostyler-openlayers-parser": "^5.7.0",
"geostyler-style": "11.0.2",
@@ -121,7 +121,7 @@
"query-string": "9.3.1",
"re-resizable": "^6.11.2",
"react": "^18.2.0",
"react-arborist": "^3.6.1",
"react-arborist": "^3.7.0",
"react-checkbox-tree": "^1.8.0",
"react-diff-viewer-continued": "^4.2.2",
"react-dnd": "^11.1.3",
@@ -195,7 +195,7 @@
"@storybook/test-runner": "^0.17.0",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.33",
"@swc/plugin-emotion": "^14.9.0",
"@swc/plugin-emotion": "^14.10.0",
"@swc/plugin-transform-imports": "^12.5.0",
"@testing-library/dom": "^9.3.4",
"@testing-library/jest-dom": "^6.9.1",
@@ -229,7 +229,7 @@
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"babel-plugin-lodash": "^3.3.4",
"baseline-browser-mapping": "^2.10.29",
"baseline-browser-mapping": "^2.10.31",
"cheerio": "1.2.0",
"concurrently": "^9.2.1",
"copy-webpack-plugin": "^14.0.0",
@@ -284,14 +284,14 @@
"style-loader": "^4.0.0",
"swc-loader": "^0.2.7",
"terser-webpack-plugin": "^5.6.0",
"ts-jest": "^29.4.10",
"ts-jest": "^29.4.11",
"tscw-config": "^1.1.2",
"tsx": "^4.22.3",
"typescript": "5.4.5",
"unzipper": "^0.12.3",
"vm-browserify": "^1.1.2",
"wait-on": "^9.0.10",
"webpack": "^5.106.2",
"webpack": "^5.107.1",
"webpack-bundle-analyzer": "^5.3.0",
"webpack-cli": "^6.0.1",
"webpack-dev-server": "^5.2.4",
@@ -3958,9 +3958,9 @@
}
},
"node_modules/@googleapis/sheets": {
"version": "13.0.1",
"resolved": "https://registry.npmjs.org/@googleapis/sheets/-/sheets-13.0.1.tgz",
"integrity": "sha512-XTYObncN5Rqexc0uITZIN9OWTEyE/ZR2S6c7wAniqHe2oGXW9gcHR9f9hQwPMHFUTHjH7Jkj8SLdt0O0u37y2A==",
"version": "13.0.2",
"resolved": "https://registry.npmjs.org/@googleapis/sheets/-/sheets-13.0.2.tgz",
"integrity": "sha512-b1tBlMcfvNEziM4DZCikLOc9iqSlgCK1e5bMKtNQIADRXr1CQmbkHV3ZBVvTsFsjLErgihqO58Itn/kzCnSZ0A==",
"license": "Apache-2.0",
"dependencies": {
"googleapis-common": "^8.0.0"
@@ -12568,9 +12568,9 @@
}
},
"node_modules/@swc/plugin-emotion": {
"version": "14.9.0",
"resolved": "https://registry.npmjs.org/@swc/plugin-emotion/-/plugin-emotion-14.9.0.tgz",
"integrity": "sha512-h57mL/TsOrhimvHs6KQQLZO1T+D7FQyx+7WS17p9vV228qxmZatF0IgEXMyERWthm1QL7fAB6cEMBCtujSVbyw==",
"version": "14.10.0",
"resolved": "https://registry.npmjs.org/@swc/plugin-emotion/-/plugin-emotion-14.10.0.tgz",
"integrity": "sha512-uhPq0oJHk2/W2Hn6vLaNmbUUgNPPj0FINHISxfs9hqS2Hpv/TVzQFsnbxul1FJEa+YQe1Qebou2esDphwzIuKg==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
@@ -13164,22 +13164,13 @@
"integrity": "sha512-FXx2pKgId/WyYo2jXw63kk7/+TY7u7AziEJxJAnSFzHlqTAS3Ync6SvgYAN/k4/PQpnnVuzoMuVnByKK2qp0ag==",
"dev": true,
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@types/estree": "*",
"@types/json-schema": "*"
}
},
"node_modules/@types/eslint-scope": {
"version": "3.7.7",
"resolved": "https://registry.npmjs.org/@types/eslint-scope/-/eslint-scope-3.7.7.tgz",
"integrity": "sha512-MzMFlSLBqNF2gcHWO0G1vP/YQyfvrxZ0bF+u7mzUdZ1/xK4A4sru+nraZz5i3iEIk1l1uyicaDVTB4QbbEkAYg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/eslint": "*",
"@types/estree": "*"
}
},
"node_modules/@types/estree": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
@@ -17218,9 +17209,9 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.10.29",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.29.tgz",
"integrity": "sha512-Asa2krT+XTPZINCS+2QcyS8WTkObE77RwkydwF7h6DmnKqbvlalz93m/dnphUyCa6SWSP51VgtEUf2FN+gelFQ==",
"version": "2.10.31",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.31.tgz",
"integrity": "sha512-MujYO3eP72uvmSE0i4wltsodRfIpZATP3jvzRNRGGxgzId7aVocVJJV3nf01qnzzKFGxQVC9bpWxl5cjxTr/7Q==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -21827,14 +21818,14 @@
}
},
"node_modules/enhanced-resolve": {
"version": "5.20.0",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.20.0.tgz",
"integrity": "sha512-/ce7+jQ1PQ6rVXwe+jKEg5hW5ciicHwIQUagZkp6IufBoY3YDgdTTY1azVs0qoRgVmvsNB+rbjLJxDAeHHtwsQ==",
"version": "5.21.6",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.21.6.tgz",
"integrity": "sha512-aNnGCvbJ/RIyWo1IuhNdVjnNF+EjH9wpzpNHt+ci/m9He9LJvUN8wrCcXjp9cWsGNAuvSpVFTx/vraAFQ8qGjQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"graceful-fs": "^4.2.4",
"tapable": "^2.3.0"
"tapable": "^2.3.3"
},
"engines": {
"node": ">=10.13.0"
@@ -24541,9 +24532,9 @@
"license": "MIT"
},
"node_modules/geostyler": {
"version": "18.5.1",
"resolved": "https://registry.npmjs.org/geostyler/-/geostyler-18.5.1.tgz",
"integrity": "sha512-5+vLuDo1oR4QQTnrfkccIQSe3qEn0ytV9dLiFFhnxhPdziv/Wp3vKNhJZ37MUF5yIj2ISWZ+q/VmSNH6ifvWpg==",
"version": "18.6.0",
"resolved": "https://registry.npmjs.org/geostyler/-/geostyler-18.6.0.tgz",
"integrity": "sha512-q8x5V4yJlTFOIe5LSvhEHd62MrMJq1YXWJVTeAG2TUMgOudjrcglXDqKtFYtEdWHeORH6TXz7q+m6cg3RlZqAg==",
"license": "BSD-2-Clause",
"dependencies": {
"@ant-design/icons": "^5.5.1",
@@ -32537,9 +32528,9 @@
}
},
"node_modules/loader-runner": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/loader-runner/-/loader-runner-4.3.1.tgz",
"integrity": "sha512-IWqP2SCPhyVFTBtRcgMHdzlf9ul25NwaFx4wCEH/KjAXuuHY4yNjvPXsBokp8jCB936PyWRaPKUNh8NvylLp2Q==",
"version": "4.3.2",
"resolved": "https://registry.npmjs.org/loader-runner/-/loader-runner-4.3.2.tgz",
"integrity": "sha512-DFEqQ3ihfS9blba08cLfYf1NRAIEm+dDjic073DRDc3/JspI/8wYmtDsHwd3+4hwvdxSK7PGaElfTmm0awWJ4w==",
"dev": true,
"license": "MIT",
"engines": {
@@ -40280,9 +40271,9 @@
}
},
"node_modules/react-arborist": {
"version": "3.6.1",
"resolved": "https://registry.npmjs.org/react-arborist/-/react-arborist-3.6.1.tgz",
"integrity": "sha512-h2/sPz6PXL79h7mOWjCA6Y5WNUKmA0kL8Uh6RYZQbYk7UOFBd86Jeoga4RjHMBYpOWpBPYrOJOE3HbIPUETp8w==",
"version": "3.7.0",
"resolved": "https://registry.npmjs.org/react-arborist/-/react-arborist-3.7.0.tgz",
"integrity": "sha512-gh2SoO0eXQVSP6zxXMGqFeXF+l2uabDGBVn0+RKqy/s7mrG5xGnfM5mhyB67cMVobC3vWYLqe6HGh7ZEZadW/w==",
"license": "MIT",
"dependencies": {
"react-dnd": "^14.0.3",
@@ -44459,9 +44450,9 @@
}
},
"node_modules/tapable": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.0.tgz",
"integrity": "sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==",
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.3.tgz",
"integrity": "sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A==",
"dev": true,
"license": "MIT",
"engines": {
@@ -45223,9 +45214,9 @@
}
},
"node_modules/ts-jest": {
"version": "29.4.10",
"resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.10.tgz",
"integrity": "sha512-vMTlTTtvz5aKZgzOoc7DQ5TzAL2fCzl8JnG1+ZpwjQa/g0xLlwE44yQ+1Cao9ZP1xVv9y5g34IFXEiqGOGFBUA==",
"version": "29.4.11",
"resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.11.tgz",
"integrity": "sha512-IrFl7l9AuB/qrNw5quqvAv/hmKMb8dhWOH4jQOGo0Oq8tCeo1O86/iTFG1FaRimgUkF13l4PcepO8ATFT6Ns4g==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -47366,13 +47357,12 @@
}
},
"node_modules/webpack": {
"version": "5.106.2",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.106.2.tgz",
"integrity": "sha512-wGN3qcrBQIFmQ/c0AiOAQBvrZ5lmY8vbbMv4Mxfgzqd/B6+9pXtLo73WuS1dSGXM5QYY3hZnIbvx+K1xxe6FyA==",
"version": "5.107.1",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.107.1.tgz",
"integrity": "sha512-mvdIWxj/H6QsfgDdH9djne3a5dYcmEmtsXGESkypaGN5jXjF/b+9KDlmTDQ2TKlFUeA2fI9Y65kihD30JOdB+Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/eslint-scope": "^3.7.7",
"@types/estree": "^1.0.8",
"@types/json-schema": "^7.0.15",
"@webassemblyjs/ast": "^1.14.1",
@@ -47382,20 +47372,20 @@
"acorn-import-phases": "^1.0.3",
"browserslist": "^4.28.1",
"chrome-trace-event": "^1.0.2",
"enhanced-resolve": "^5.20.0",
"es-module-lexer": "^2.0.0",
"enhanced-resolve": "^5.21.4",
"es-module-lexer": "^2.1.0",
"eslint-scope": "5.1.1",
"events": "^3.2.0",
"glob-to-regexp": "^0.4.1",
"graceful-fs": "^4.2.11",
"loader-runner": "^4.3.1",
"loader-runner": "^4.3.2",
"mime-db": "^1.54.0",
"neo-async": "^2.6.2",
"schema-utils": "^4.3.3",
"tapable": "^2.3.0",
"terser-webpack-plugin": "^5.3.17",
"terser-webpack-plugin": "^5.5.0",
"watchpack": "^2.5.1",
"webpack-sources": "^3.3.4"
"webpack-sources": "^3.4.1"
},
"bin": {
"webpack": "bin/webpack.js"
@@ -48910,7 +48900,7 @@
"dependencies": {
"chalk": "^5.6.2",
"lodash-es": "^4.18.1",
"yeoman-generator": "^8.2.2",
"yeoman-generator": "^8.1.2",
"yosay": "^3.0.0"
},
"devDependencies": {
@@ -49983,7 +49973,7 @@
"acorn": "^8.16.0",
"d3-array": "^3.2.4",
"lodash": "^4.18.1",
"zod": "^4.4.1"
"zod": "^4.4.3"
},
"peerDependencies": {
"@apache-superset/core": "*",
@@ -50095,7 +50085,7 @@
"@math.gl/web-mercator": "^4.1.0",
"mapbox-gl": "^3.24.0",
"maplibre-gl": "^5.24.0",
"react-map-gl": "^8.1.1",
"react-map-gl": "^8.1.0",
"supercluster": "^8.0.1"
},
"peerDependencies": {
@@ -50184,7 +50174,7 @@
"license": "Apache-2.0",
"dependencies": {
"@types/d3-scale": "^4.0.9",
"d3-cloud": "^1.2.9",
"d3-cloud": "^1.2.8",
"d3-scale": "^4.0.2"
},
"devDependencies": {

View File

@@ -43,6 +43,8 @@
"build-instrumented": "cross-env NODE_ENV=production BABEL_ENV=instrumented webpack --mode=production --color",
"build-storybook": "storybook build",
"build-translation": "scripts/po2json.sh",
"translations:build-index": "python3 ../scripts/translations/build_translation_index.py",
"translations:backfill": "python3 ../scripts/translations/backfill_po.py",
"bundle-stats": "cross-env BUNDLE_ANALYZER=true npm run build && npx open-cli ../superset/static/stats/statistics.html",
"clear-npm": "mkdir -p /tmp/empty && rsync -a --delete /tmp/empty/ node_modules/ && rmdir node_modules /tmp/empty",
"core:cover": "cross-env NODE_ENV=test NODE_OPTIONS=\"--max-old-space-size=4096\" jest --coverage --coverageThreshold='{\"global\":{\"statements\":100,\"branches\":100,\"functions\":100,\"lines\":100}}' --collectCoverageFrom='[\"packages/**/src/**/*.{js,ts}\", \"!packages/superset-core/**/*\"]' packages",
@@ -112,7 +114,7 @@
"@fontsource/fira-code": "^5.2.7",
"@fontsource/ibm-plex-mono": "^5.2.7",
"@fontsource/inter": "^5.2.8",
"@googleapis/sheets": "^13.0.1",
"@googleapis/sheets": "^13.0.2",
"@great-expectations/jsonforms-antd-renderers": "^2.2.10",
"@jsonforms/core": "^3.7.0",
"@jsonforms/react": "^3.7.0",
@@ -177,7 +179,7 @@
"fs-extra": "^11.3.5",
"fuse.js": "^7.3.0",
"geolib": "^3.3.14",
"geostyler": "^18.5.1",
"geostyler": "^18.6.0",
"geostyler-data": "^1.1.0",
"geostyler-openlayers-parser": "^5.7.0",
"geostyler-style": "11.0.2",
@@ -202,7 +204,7 @@
"query-string": "9.3.1",
"re-resizable": "^6.11.2",
"react": "^18.2.0",
"react-arborist": "^3.6.1",
"react-arborist": "^3.7.0",
"react-checkbox-tree": "^1.8.0",
"react-diff-viewer-continued": "^4.2.2",
"react-dnd": "^11.1.3",
@@ -276,7 +278,7 @@
"@storybook/test-runner": "^0.17.0",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.33",
"@swc/plugin-emotion": "^14.9.0",
"@swc/plugin-emotion": "^14.10.0",
"@swc/plugin-transform-imports": "^12.5.0",
"@testing-library/dom": "^9.3.4",
"@testing-library/jest-dom": "^6.9.1",
@@ -310,7 +312,7 @@
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"babel-plugin-lodash": "^3.3.4",
"baseline-browser-mapping": "^2.10.29",
"baseline-browser-mapping": "^2.10.31",
"cheerio": "1.2.0",
"concurrently": "^9.2.1",
"copy-webpack-plugin": "^14.0.0",
@@ -365,14 +367,14 @@
"style-loader": "^4.0.0",
"swc-loader": "^0.2.7",
"terser-webpack-plugin": "^5.6.0",
"ts-jest": "^29.4.10",
"ts-jest": "^29.4.11",
"tscw-config": "^1.1.2",
"tsx": "^4.22.3",
"typescript": "5.4.5",
"unzipper": "^0.12.3",
"vm-browserify": "^1.1.2",
"wait-on": "^9.0.10",
"webpack": "^5.106.2",
"webpack": "^5.107.1",
"webpack-bundle-analyzer": "^5.3.0",
"webpack-cli": "^6.0.1",
"webpack-dev-server": "^5.2.4",

View File

@@ -30,7 +30,7 @@
"dependencies": {
"chalk": "^5.6.2",
"lodash-es": "^4.18.1",
"yeoman-generator": "^8.2.2",
"yeoman-generator": "^8.1.2",
"yosay": "^3.0.0"
},
"devDependencies": {

View File

@@ -48,6 +48,7 @@ import NoResultsComponent from './NoResultsComponent';
import { isMatrixifyEnabled } from '../types/matrixify';
import MatrixifyGridRenderer from './Matrixify/MatrixifyGridRenderer';
import { supersetTheme, SupersetTheme } from '@apache-superset/core/theme';
export type FallbackPropsWithDimension = FallbackProps & Partial<Dimension>;
export type WrapperProps = Dimension & {

View File

@@ -897,6 +897,476 @@ test('fires onChange when pasting a selection', async () => {
await waitFor(() => expect(onChange).toHaveBeenCalledTimes(1));
});
test('replaces cached options with search results instead of merging', async () => {
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const searchData = [{ label: 'Search Match', value: 100 }];
const loadOptions = jest.fn(async (search: string) => {
if (search === '') {
return { data: page0Data, totalCount: 100 };
}
return { data: searchData, totalCount: 1 };
});
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(1));
let options = await findAllSelectOptions();
expect(options).toHaveLength(10);
await type('search');
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(2));
options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Search Match');
});
test('shows all options when filterOption is false', async () => {
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Base ${i}`,
value: i,
}));
const searchData = Array.from({ length: 5 }, (_, i) => ({
label: `Server ${i}`,
value: 100 + i,
}));
const loadOptions = jest.fn(async (search: string) =>
search === ''
? { data: page0Data, totalCount: 100 }
: { data: searchData, totalCount: 5 },
);
render(
<AsyncSelect
{...defaultProps}
options={loadOptions}
filterOption={false}
/>,
);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(1));
await type('zzz_no_match');
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(2));
const options = await findAllSelectOptions();
expect(options).toHaveLength(5);
expect(options[0]).toHaveTextContent('Server 0');
});
test('preserves new option entry across search fetch when allowNewOptions is on', async () => {
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const loadOptions = jest.fn(async (search: string) => {
if (search === '') {
return { data: page0Data, totalCount: 100 };
}
return { data: [], totalCount: 0 };
});
render(
<AsyncSelect {...defaultProps} options={loadOptions} allowNewOptions />,
);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(1));
await type('newval');
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(2));
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('newval');
// Stale page-0 options must not bleed through.
expect(screen.queryByText('Option 0')).not.toBeInTheDocument();
});
test('restores base options when search is cleared', async () => {
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const searchData = [{ label: 'Search Match', value: 100 }];
const loadOptions = jest.fn(async (search: string) => {
if (search === '') {
return { data: page0Data, totalCount: 100 };
}
return { data: searchData, totalCount: 1 };
});
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(1));
await type('search');
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(2));
let options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Search Match');
// type() clears the input before typing, so passing '' clears the search.
await type('');
await waitFor(async () => {
options = await findAllSelectOptions();
expect(options).toHaveLength(10);
});
expect(options[0]).toHaveTextContent('Option 0');
expect(screen.queryByText('Search Match')).not.toBeInTheDocument();
});
test('replaces results when switching between two searches', async () => {
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const loadOptions = jest.fn(async (search: string) => {
if (search === '') {
return { data: page0Data, totalCount: 100 };
}
return {
data: [{ label: `Match-${search}`, value: `v-${search}` }],
totalCount: 1,
};
});
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledTimes(1));
await type('alpha');
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-alpha');
});
await type('beta');
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-beta');
});
expect(screen.queryByText('Match-alpha')).not.toBeInTheDocument();
});
test('refetches a dropped search response when the same search is repeated', async () => {
type OptionRow = { label: string; value: string | number };
type PageResponse = { data: OptionRow[]; totalCount: number };
// Resolves the in-flight loadOptions promise of the calling test.
let resolveAlpha: ((value: PageResponse) => void) | null = null;
const page0Data: OptionRow[] = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const alphaData: OptionRow[] = [{ label: 'Match-alpha', value: 'va' }];
const betaData: OptionRow[] = [{ label: 'Match-beta', value: 'vb' }];
const loadOptions = jest.fn((search: string) => {
if (search === '') {
return Promise.resolve<PageResponse>({
data: page0Data,
totalCount: 100,
});
}
if (search === 'alpha') {
// First call: hold the promise so it resolves only after beta returns.
// Second call (after beta): resolve immediately so the cache MUST allow
// a refetch.
if (!resolveAlpha) {
return new Promise<PageResponse>(resolve => {
resolveAlpha = resolve;
});
}
return Promise.resolve<PageResponse>({ data: alphaData, totalCount: 1 });
}
return Promise.resolve<PageResponse>({ data: betaData, totalCount: 1 });
});
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledWith('', 0, 10));
await type('alpha');
await waitFor(() => expect(loadOptions).toHaveBeenCalledWith('alpha', 0, 10));
// alpha's promise is held; switch to beta which resolves first.
await type('beta');
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-beta');
});
// Release the stale alpha response. It must be dropped — its key must not
// be cached, or returning to "alpha" later would short-circuit the fetch.
resolveAlpha!({ data: alphaData, totalCount: 1 });
await waitFor(async () => {
// Beta is still showing because alpha's response was dropped.
const options = await findAllSelectOptions();
expect(options[0]).toHaveTextContent('Match-beta');
});
// Returning to "alpha" must re-trigger the fetch (cache wasn't poisoned).
const callsBeforeAlphaReturn = loadOptions.mock.calls.filter(
args => args[0] === 'alpha',
).length;
await type('alpha');
await waitFor(() => {
const callsAfter = loadOptions.mock.calls.filter(
args => args[0] === 'alpha',
).length;
expect(callsAfter).toBeGreaterThan(callsBeforeAlphaReturn);
});
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options[0]).toHaveTextContent('Match-alpha');
});
});
test('keeps loading indicator while a newer request is in flight after a stale response is dropped', async () => {
// Regression for the P2 race: the `.finally` block that clears isLoading
// must not fire when a stale (dropped) response resolves while a newer
// request is still in flight. Otherwise the spinner disappears mid-search
// and the undebounced scroll-pagination handler can fire against stale
// totalCount before page 0 of the active search lands.
type OptionRow = { label: string; value: string | number };
type PageResponse = { data: OptionRow[]; totalCount: number };
// Initialized to no-op so the finally block can always call them, even if
// an assertion in the try throws before the corresponding mock ran.
let resolveAlpha: (value: PageResponse) => void = () => {};
let resolveBeta: (value: PageResponse) => void = () => {};
const page0Data: OptionRow[] = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const alphaData: OptionRow[] = [{ label: 'Match-alpha', value: 'va' }];
const betaData: OptionRow[] = [{ label: 'Match-beta', value: 'vb' }];
const loadOptions = jest.fn((search: string) => {
if (search === '') {
return Promise.resolve<PageResponse>({
data: page0Data,
totalCount: 100,
});
}
if (search === 'alpha') {
return new Promise<PageResponse>(resolve => {
resolveAlpha = resolve;
});
}
return new Promise<PageResponse>(resolve => {
resolveBeta = resolve;
});
});
const isSpinnerVisible = (): boolean =>
Boolean(document.querySelector('.ant-select-arrow .ant-spin'));
try {
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledWith('', 0, 10));
// Type 'alpha' — alpha fetch is held, loading should be true.
await type('alpha');
await waitFor(() =>
expect(loadOptions).toHaveBeenCalledWith('alpha', 0, 10),
);
await waitFor(() => expect(isSpinnerVisible()).toBe(true));
// Type 'beta' — beta fetch is also held; both are in flight.
await type('beta');
await waitFor(() =>
expect(loadOptions).toHaveBeenCalledWith('beta', 0, 10),
);
expect(isSpinnerVisible()).toBe(true);
// Release the stale alpha response. It is dropped at the early-return
// (search !== inputValueRef.current), but the in-flight counter is still
// non-zero because beta is pending — spinner must stay visible.
resolveAlpha({ data: alphaData, totalCount: 1 });
// Yield a microtask so alpha's .then/.finally runs, then re-assert.
await Promise.resolve();
expect(isSpinnerVisible()).toBe(true);
// Release beta. Now the in-flight counter drops to 0 and the spinner
// clears.
resolveBeta({ data: betaData, totalCount: 1 });
await waitFor(() => expect(isSpinnerVisible()).toBe(false));
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-beta');
} finally {
// Defensive: never leave a held promise that could hang a parallel worker
// if an assertion above threw. Promise resolve is idempotent.
resolveAlpha({ data: alphaData, totalCount: 1 });
resolveBeta({ data: betaData, totalCount: 1 });
}
});
test('re-shows search results when the same search term is repeated after a clear', async () => {
// Regression: a prior fix cached search responses' totalCount in
// fetchedQueries. After restore-on-clear had replaced selectOptions with
// the base list, re-typing a previously-resolved term would hit the cache
// short-circuit and leave selectOptions stale (empty / base-only).
const page0Data = Array.from({ length: 10 }, (_, i) => ({
label: `Option ${i}`,
value: i,
}));
const alphaData = [{ label: 'Match-alpha', value: 'va' }];
const loadOptions = jest.fn(async (search: string) => {
if (search === '') {
// totalCount > data.length so allValuesLoaded stays false and the
// search path is not bypassed by the "all loaded" short-circuit.
return { data: page0Data, totalCount: 100 };
}
return { data: alphaData, totalCount: 1 };
});
render(<AsyncSelect {...defaultProps} options={loadOptions} />);
await open();
await waitFor(() => expect(loadOptions).toHaveBeenCalledWith('', 0, 10));
await type('alpha');
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-alpha');
});
await type('');
await waitFor(() =>
expect(screen.queryByText('Match-alpha')).not.toBeInTheDocument(),
);
const callsBefore = loadOptions.mock.calls.filter(
args => args[0] === 'alpha',
).length;
await type('alpha');
await waitFor(() => {
const callsAfter = loadOptions.mock.calls.filter(
args => args[0] === 'alpha',
).length;
expect(callsAfter).toBeGreaterThan(callsBefore);
});
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Match-alpha');
});
});
test('appends page>1 results during an active search and discards them when search changes', async () => {
// Covers the production branch `else { mergeData(data) }` in fetchPage that
// fires when search is non-empty AND page > 0 — i.e. user scrolled within
// a multi-page search result. Switching to a new search must replace, not
// retain, the prior search's accumulated pages.
type OptionRow = { label: string; value: string | number };
const pageSize = 5;
const aliceData: OptionRow[] = Array.from({ length: 5 }, (_, i) => ({
label: `Alice-${i}`,
value: `a${i}`,
}));
const alicePage1: OptionRow[] = Array.from({ length: 3 }, (_, i) => ({
label: `Alice-${i + 5}`,
value: `a${i + 5}`,
}));
const bobData: OptionRow[] = [{ label: 'Bob-0', value: 'b0' }];
const loadOptions = jest.fn(
async (
search: string,
page: number,
): Promise<{
data: OptionRow[];
totalCount: number;
}> => {
if (search === '') {
return { data: [], totalCount: 100 };
}
if (search === 'alice') {
if (page === 0) return { data: aliceData, totalCount: 8 };
return { data: alicePage1, totalCount: 8 };
}
return { data: bobData, totalCount: 1 };
},
);
render(
<AsyncSelect {...defaultProps} pageSize={pageSize} options={loadOptions} />,
);
await open();
await type('alice');
await waitFor(() =>
expect(loadOptions).toHaveBeenCalledWith('alice', 0, pageSize),
);
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(5);
});
// Wait for loading to finish so handlePagination's `!isLoading` gate is
// open before we fire scroll.
await waitFor(() =>
expect(document.querySelector('.ant-select-arrow .ant-spin')).toBeNull(),
);
// Trigger pagination by dispatching a scroll event on the virtual-list
// scroll container. jsdom returns 0 for layout properties by default, so
// override the relevant ones before firing scroll. rc-virtual-list reads
// scrollTop via e.currentTarget in its onFallbackScroll handler, which
// then forwards to onPopupScroll (handlePagination here).
const holder = document.querySelector(
'.rc-virtual-list-holder',
) as HTMLElement | null;
if (!holder) throw new Error('virtual-list holder not rendered');
Object.defineProperty(holder, 'scrollHeight', {
configurable: true,
get: () => 1000,
});
Object.defineProperty(holder, 'offsetHeight', {
configurable: true,
get: () => 200,
});
Object.defineProperty(holder, 'clientHeight', {
configurable: true,
get: () => 200,
});
Object.defineProperty(holder, 'scrollTop', {
configurable: true,
get: () => 900,
set: () => {},
});
fireEvent.scroll(holder);
await waitFor(() =>
expect(loadOptions).toHaveBeenCalledWith('alice', 1, pageSize),
);
await waitFor(async () => {
const options = await findAllSelectOptions();
// Page 0 (5) + page 1 (3) merged
expect(options).toHaveLength(8);
});
// Switching to a new search must replace the accumulated pages, not retain
// them.
await type('bob');
await waitFor(() =>
expect(loadOptions).toHaveBeenCalledWith('bob', 0, pageSize),
);
await waitFor(async () => {
const options = await findAllSelectOptions();
expect(options).toHaveLength(1);
expect(options[0]).toHaveTextContent('Bob-0');
});
expect(screen.queryByText('Alice-0')).not.toBeInTheDocument();
expect(screen.queryByText('Alice-7')).not.toBeInTheDocument();
});
test('does not duplicate options when using numeric values', async () => {
render(
<AsyncSelect

View File

@@ -160,6 +160,13 @@ const AsyncSelect = forwardRef(
const [allValuesLoaded, setAllValuesLoaded] = useState(false);
const selectValueRef = useRef(selectValue);
const fetchedQueries = useRef(new Map<string, number>());
const initialOptionsRef = useRef<SelectOptionsType>(EMPTY_OPTIONS);
const inputValueRef = useRef('');
// Counts fetches whose `.finally` has not yet run. Loading is cleared only
// when this drops to 0, so a stale response (which returns early without
// updating selectOptions) cannot flip the spinner off while a newer
// request is still pending.
const inFlightFetchesRef = useRef(0);
const mappedMode = isSingleMode ? undefined : 'multiple';
const allowFetch = !fetchOnlyOnSearch || inputValue;
const [maxTagCount, setMaxTagCount] = useState(
@@ -183,6 +190,10 @@ const AsyncSelect = forwardRef(
selectValueRef.current = selectValue;
}, [selectValue]);
useEffect(() => {
inputValueRef.current = inputValue;
}, [inputValue]);
const sortSelectedFirst = useCallback(
(a: AntdLabeledValue, b: AntdLabeledValue) =>
sortSelectedFirstHelper(a, b, selectValueRef.current),
@@ -333,22 +344,78 @@ const AsyncSelect = forwardRef(
setIsLoading(true);
const fetchOptions = options as SelectOptionsPagePromise;
inFlightFetchesRef.current += 1;
fetchOptions(search, page, pageSize)
.then(({ data, totalCount }: SelectOptionsTypePage) => {
const mergedData = mergeData(data);
fetchedQueries.current.set(key, totalCount);
setTotalCount(totalCount);
if (
!fetchOnlyOnSearch &&
search === '' &&
mergedData.length >= totalCount
) {
setAllValuesLoaded(true);
// Drop responses whose search arg no longer matches the user's
// current input — otherwise a slow base fetch can land after a
// search fetch (or a stale debounced search after a clear) and
// re-pollute the dropdown via mergeData / search-replace. Search
// responses are never cached in fetchedQueries: the cache stores
// only totalCount, so a cache hit would short-circuit the fetch
// and leave selectOptions stale (e.g. after restore-on-clear).
// Re-issuing the search is cheap and correct.
const matchesCurrentSearch = inputValueRef.current === search;
if (search && !matchesCurrentSearch) {
return;
}
if (!search) {
// Accumulate base pages in a ref independent of selectOptions
// (during an active search, selectOptions holds search results
// and is not a safe accumulator). The accumulator is kept up
// to date even when this response landed during a search, so
// restore-on-clear has a complete snapshot. We don't sort here
// — restore-on-clear sorts a copy at consumption time, and the
// live selectOptions path below goes through mergeData which
// sorts there. Sorting here too would double the per-page sort
// cost on large cached option sets.
const dataValues = new Set(data.map(opt => opt.value));
const accumulated = initialOptionsRef.current
.filter(opt => !dataValues.has(opt.value))
.concat(data);
initialOptionsRef.current = accumulated;
if (!fetchOnlyOnSearch && accumulated.length >= totalCount) {
setAllValuesLoaded(true);
}
fetchedQueries.current.set(key, totalCount);
if (matchesCurrentSearch) {
// No active search — push to live selectOptions and update
// totalCount. When matchesCurrentSearch is false, the user
// is mid-search; leave the search's totalCount in place so
// pagination math stays correct.
mergeData(data);
setTotalCount(totalCount);
}
} else if (page === 0) {
// Replace cached options with server results; preserve
// optimistic isNewOption entries inserted by handleOnSearch
// so allowNewOptions users can still click the value they
// typed when the server returns no match.
setSelectOptions(prevOptions => {
const dataValues = new Set(data.map(opt => opt.value));
const preservedNew = prevOptions.filter(
opt => opt.isNewOption && !dataValues.has(opt.value),
);
return preservedNew
.concat(data)
.sort(sortComparatorForNoSearch);
});
setTotalCount(totalCount);
} else {
// page > 0 during an active search — append normally.
mergeData(data);
setTotalCount(totalCount);
}
})
.catch(internalOnError)
.finally(() => {
setIsLoading(false);
inFlightFetchesRef.current = Math.max(
0,
inFlightFetchesRef.current - 1,
);
if (inFlightFetchesRef.current === 0) {
setIsLoading(false);
}
});
},
[
@@ -358,6 +425,7 @@ const AsyncSelect = forwardRef(
internalOnError,
options,
pageSize,
sortComparatorForNoSearch,
],
);
@@ -500,6 +568,7 @@ const AsyncSelect = forwardRef(
fetchedQueries.current.clear();
setAllValuesLoaded(false);
setSelectOptions(EMPTY_OPTIONS);
initialOptionsRef.current = EMPTY_OPTIONS;
}, [options]);
useEffect(() => {
@@ -514,16 +583,36 @@ const AsyncSelect = forwardRef(
[debouncedFetchPage],
);
const previousInputValue = usePrevious(inputValue, '');
useEffect(() => {
if (loadingEnabled && allowFetch) {
// trigger fetch every time inputValue changes
if (inputValue) {
debouncedFetchPage(inputValue, 0);
} else {
// Cancel any pending debounced search fetch so it can't fire after
// we've already restored the base list.
debouncedFetchPage.cancel();
// On returning to empty input after a search, restore the cached
// base options so the dropdown shows the original page-0 list
// instead of the stale search results.
if (previousInputValue && initialOptionsRef.current.length > 0) {
setSelectOptions(
[...initialOptionsRef.current].sort(sortComparatorForNoSearch),
);
}
fetchPage('', 0);
}
}
}, [loadingEnabled, fetchPage, allowFetch, inputValue, debouncedFetchPage]);
}, [
loadingEnabled,
fetchPage,
allowFetch,
inputValue,
previousInputValue,
debouncedFetchPage,
sortComparatorForNoSearch,
]);
useEffect(() => {
if (loading !== undefined && loading !== isLoading) {
@@ -531,7 +620,11 @@ const AsyncSelect = forwardRef(
}
}, [isLoading, loading]);
const clearCache = () => fetchedQueries.current.clear();
const clearCache = () => {
fetchedQueries.current.clear();
initialOptionsRef.current = EMPTY_OPTIONS;
setAllValuesLoaded(false);
};
useImperativeHandle(
ref,

View File

@@ -211,6 +211,10 @@ export const handleFilterOptionHelper = (
return filterOption(search, option);
}
if (filterOption === false) {
return true;
}
if (filterOption) {
const searchValue = search.trim().toLowerCase();
if (optionFilterProps?.length) {

View File

@@ -371,3 +371,37 @@ test('should handle large datasets with pagination', () => {
expect(screen.getByRole('list')).toBeInTheDocument();
expect(screen.getByText('1-10 of 100')).toBeInTheDocument();
});
test('should reset to first page when data reduces below current page', async () => {
// Start with 30 items, 10 per page = 3 pages
const initialData = Array.from({ length: 30 }, (_, i) => ({
id: i,
age: 20 + i,
name: `Person ${i}`,
}));
const props = {
...mockedProps,
data: initialData,
pageSize: 10,
};
const { rerender } = render(<TableView {...props} />);
// Navigate to page 3 (last page)
const page3 = screen.getByRole('listitem', { name: '3' });
await userEvent.click(page3);
await waitFor(() => {
expect(screen.getByText('21-30 of 30')).toBeInTheDocument();
});
// Reduce data to only 5 items (fewer than current page would show)
const reducedData = initialData.slice(0, 5);
rerender(<TableView {...props} data={reducedData} />);
// Should reset to page 1 since page 3 no longer exists
await waitFor(() => {
expect(screen.getByText('1-5 of 5')).toBeInTheDocument();
});
});

View File

@@ -246,6 +246,21 @@ const RawTableView = ({
}
}, [initialSortBy, onServerPagination, serverPagination, sortBy]);
// Reset to first page when current page exceeds available pages
// (e.g., when filtering reduces the data below the current page)
const pageCount = Math.ceil(data.length / effectivePageSize);
useEffect(() => {
if (
withPagination &&
!serverPagination &&
!loading &&
pageIndex > pageCount - 1 &&
pageCount > 0
) {
setPageIndex(0);
}
}, [withPagination, serverPagination, loading, pageIndex, pageCount]);
return (
<TableViewStyles {...props} ref={tableRef}>
<TableCollection

View File

@@ -77,6 +77,7 @@ export interface ChartDataResponseResult {
// TODO(hainenber): define proper type for below attributes
rejected_filters?: any[];
applied_filters?: any[];
warning?: string | null;
/**
* Detected ISO 4217 currency code when AUTO mode is used.
* Returns the currency code if all filtered data contains a single currency,

View File

@@ -239,8 +239,6 @@ export default function transformProps(
formatter,
show: showLabels,
color: theme.colorText,
textBorderColor: theme.colorBgBase,
textBorderWidth: 1,
};
const legendData = keys.sort((a: string, b: string) => {
if (!legendSort) return 0;

View File

@@ -21,8 +21,8 @@ import { TreePathInfo } from '../types';
export const COLOR_SATURATION = [0.7, 0.4];
export const LABEL_FONTSIZE = 11;
export const BORDER_WIDTH = 2;
export const GAP_WIDTH = 2;
export const BORDER_WIDTH = 0;
export const GAP_WIDTH = 0;
export const extractTreePathInfo = (
treePathInfo: TreePathInfo[] | undefined,

View File

@@ -214,7 +214,8 @@ export default function transformProps(
colorAlpha: OpacityEnum.SemiTransparent,
color: theme.colorText,
borderColor: theme.colorBgBase,
borderWidth: 2,
borderWidth: BORDER_WIDTH,
gapWidth: GAP_WIDTH,
},
label: {
...labelProps,

View File

@@ -72,6 +72,15 @@ describe('Funnel transformProps', () => {
}),
);
});
test('does not apply a text border to segment labels', () => {
// A white textBorder washes out the dark text on light-colored segments.
const result = transformProps(chartProps as EchartsFunnelChartProps);
const { label } = (result.echartOptions.series as any)[0];
expect(label.color).toBe(supersetTheme.colorText);
expect(label.textBorderColor).toBeUndefined();
expect(label.textBorderWidth).toBeUndefined();
});
});
describe('formatFunnelLabel', () => {

View File

@@ -18,6 +18,7 @@
*/
import { ChartProps } from '@superset-ui/core';
import { supersetTheme } from '@apache-superset/core/theme';
import { OpacityEnum } from '../../src/constants';
import { EchartsTreemapChartProps } from '../../src/Treemap/types';
import transformProps from '../../src/Treemap/transformProps';
@@ -74,4 +75,44 @@ describe('Treemap transformProps', () => {
}),
);
});
test('should not render gaps between treemap nodes when filtered', () => {
const filteredChartProps = new ChartProps({
...chartProps,
filterState: { selectedValues: ['Sylvester,bar1'] },
});
expect(
transformProps(filteredChartProps as EchartsTreemapChartProps),
).toEqual(
expect.objectContaining({
echartOptions: expect.objectContaining({
series: [
expect.objectContaining({
data: expect.arrayContaining([
expect.objectContaining({
children: expect.arrayContaining([
expect.objectContaining({
name: 'Arnold',
children: expect.arrayContaining([
expect.objectContaining({
name: 'bar2',
itemStyle: expect.objectContaining({
borderWidth: 0,
gapWidth: 0,
colorAlpha: OpacityEnum.SemiTransparent,
}),
label: expect.objectContaining({}),
}),
]),
}),
]),
}),
]),
}),
],
}),
}),
);
});
});

View File

@@ -29,7 +29,7 @@
"@math.gl/web-mercator": "^4.1.0",
"mapbox-gl": "^3.24.0",
"maplibre-gl": "^5.24.0",
"react-map-gl": "^8.1.1",
"react-map-gl": "^8.1.0",
"supercluster": "^8.0.1"
},
"peerDependencies": {

View File

@@ -27,7 +27,7 @@ jest.mock('../../DeckGLContainer', () => ({
}));
jest.mock('../../factory', () => ({
createDeckGLComponent: jest.fn(() => () => null),
createCategoricalDeckGLComponent: jest.fn(() => () => null),
GetLayerType: {},
}));
@@ -53,6 +53,14 @@ const mockPayload = {
},
};
const mockLayerParams = {
onContextMenu: jest.fn(),
filterState: undefined,
setDataMask: jest.fn(),
setTooltip: jest.fn(),
emitCrossFilters: false,
};
test('getLayer uses line_width_unit from formData', () => {
const layer = getLayer({
formData: mockFormData,
@@ -117,3 +125,518 @@ test('getPoints extracts points from path data', () => {
expect(points[0]).toEqual([0, 0]);
expect(points[2]).toEqual([2, 2]);
});
test('Fixed width mode returns constant width for all paths', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 5,
},
{
path: [
[2, 2],
[3, 3],
],
width: 5,
},
{
path: [
[4, 4],
[5, 5],
],
width: 5,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
const widths = data.map(d => d.width);
widths.forEach(width => {
expect(width).toBe(widths[0]);
});
});
test('Fixed width mode applies multiplier correctly', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 5,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
line_width_multiplier: 3,
min_width: 1,
max_width: 100,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBe(15);
});
test('Fixed width mode enforces minimum width bound', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 0.1,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
min_width: 2,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBeGreaterThanOrEqual(2);
});
test('Fixed width mode enforces maximum width bound', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBeLessThanOrEqual(20);
});
test('Fixed width mode defaults width to 1 when no width is provided', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
line_width: undefined,
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBe(1);
});
test('Metric mode normalizes widths proportionally between min and max bounds', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
{
path: [
[2, 2],
[3, 3],
],
width: 200,
},
{
path: [
[4, 4],
[5, 5],
],
width: 300,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
line_width: { type: 'metric', value: 'some_metric' },
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
const widths = data.map((d: any) => d.width);
expect(widths[0]).toBeCloseTo(1);
expect(widths[1]).toBeCloseTo(10.5);
expect(widths[2]).toBeCloseTo(20);
});
test('Metric mode applies multiplier after normalization', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
{
path: [
[2, 2],
[3, 3],
],
width: 200,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
line_width: { type: 'metric', value: 'some_metric' },
min_width: 1,
max_width: 20,
line_width_multiplier: 2,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBeCloseTo(2);
expect(data[1].width).toBe(20);
});
test('Metric mode enforces bounds after multiplier', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
{
path: [
[2, 2],
[3, 3],
],
width: 500,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
min_width: 5,
max_width: 15,
line_width_multiplier: 10,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
data.forEach((d: any) => {
expect(d.width).toBeGreaterThanOrEqual(5);
expect(d.width).toBeLessThanOrEqual(15);
});
});
test('Metric mode handles equal width values.', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
{
path: [
[2, 2],
[3, 3],
],
width: 100,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].width).toBe(data[1].width);
});
test('Metric mode handles null width values', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
width: 100,
},
{
path: [
[2, 2],
[3, 3],
],
width: null,
},
{
path: [
[4, 4],
[5, 5],
],
width: 300,
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
line_width: { type: 'metric', value: 'some_metric' },
min_width: 1,
max_width: 20,
line_width_multiplier: 1,
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[1].width).toBe(1);
expect(data[0].width).toBeCloseTo(1);
expect(data[2].width).toBeCloseTo(20);
});
test('Fixed color mode returns same color for all paths', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
},
{
path: [
[2, 2],
[3, 3],
],
},
{
path: [
[4, 4],
[5, 5],
],
},
],
},
};
const layer = getLayer({
formData: {
...mockFormData,
color_picker: { r: 255, g: 100, b: 50, a: 1 },
},
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
const expectedColor = [255, 100, 50, 255];
data.forEach((d: any) => {
expect(d.color).toEqual(expectedColor);
});
});
test('Categorical mode preserves distinct colors for selected categories', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
color: [255, 0, 0, 255],
cat_color: 'A',
},
{
path: [
[2, 2],
[3, 3],
],
color: [0, 0, 255, 255],
cat_color: 'B',
},
{
path: [
[4, 4],
[5, 5],
],
color: [255, 0, 0, 255],
cat_color: 'A',
},
],
},
};
const layer = getLayer({
formData: mockFormData,
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].color).toEqual(data[2].color);
expect(data[0].color).not.toEqual(data[1].color);
});
test('Breakpoint mode preserves colors assigned by addColor based on metric ranges', () => {
const payload = {
data: {
features: [
{
path: [
[0, 0],
[1, 1],
],
color: [255, 0, 0, 255],
metric: 50,
},
{
path: [
[2, 2],
[3, 3],
],
color: [0, 0, 255, 255],
metric: 200,
},
{
path: [
[4, 4],
[5, 5],
],
color: [255, 0, 0, 255],
metric: 75,
},
],
},
};
const layer = getLayer({
formData: mockFormData,
payload,
...mockLayerParams,
});
const data = layer.props.data as any[];
expect(data[0].color).toEqual(data[2].color);
expect(data[0].color).not.toEqual(data[1].color);
});

View File

@@ -21,13 +21,14 @@ import { PathLayer } from '@deck.gl/layers';
import { JsonObject, QueryFormData } from '@superset-ui/core';
import { commonLayerProps } from '../common';
import sandboxedEval from '../../utils/sandbox';
import { GetLayerType, createDeckGLComponent } from '../../factory';
import { GetLayerType, createCategoricalDeckGLComponent } from '../../factory';
import { Point } from '../../types';
import {
createTooltipContent,
CommonTooltipRows,
} from '../../utilities/tooltipUtils';
import { HIGHLIGHT_COLOR_ARRAY } from '../../utils';
import { isMetricValue } from '../utils/metricUtils';
function setTooltipContent(formData: QueryFormData) {
const defaultTooltipGenerator = (o: JsonObject) => (
@@ -50,14 +51,69 @@ export const getLayer: GetLayerType<PathLayer> = function ({
emitCrossFilters,
}) {
const fd = formData;
const c = fd.color_picker;
const fixedColor = [c.r, c.g, c.b, 255 * c.a];
let data = payload.data.features.map((feature: JsonObject) => ({
...feature,
path: feature.path,
width: fd.line_width,
color: fixedColor,
}));
let data = payload.data.features.map((feature: JsonObject) => {
if (feature.color) {
return { ...feature };
}
const c = fd.color_picker || { r: 0, g: 0, b: 0, a: 1 };
const color = [c.r, c.g, c.b, 255 * c.a];
return {
...feature,
path: feature.path,
color,
};
});
// Variables for width scaling and normalization
const minWidth = Number(fd.min_width) || 1; // defaulted to 1
const maxWidth = Number(fd.max_width) || 20; // defaulted to 20
const multiplier = Number(fd.line_width_multiplier) || 1; // defaulted to 1
const widths = data.map((d: JsonObject) => d.width).filter(Number.isFinite);
// Metric or fixed value
const isMetricWidth = isMetricValue(fd.line_width);
if (isMetricWidth) {
// Get minimum and maximum widths in data set
const minVal = widths.length > 0 ? Math.min(...widths) : minWidth;
const maxVal = widths.length > 0 ? Math.max(...widths) : maxWidth;
data = data.map((d: JsonObject) => {
if (d.width == null) return { ...d, width: minWidth };
const normalized =
maxVal === minVal ? 0.5 : (d.width - minVal) / (maxVal - minVal);
// Map within range of min + max
let width = minWidth + normalized * (maxWidth - minWidth);
// Apply scaling multiplier
width *= multiplier;
// Enforce minimum and maximum width bounds
width = Math.max(minWidth, Math.min(maxWidth, width));
return { ...d, width };
});
} else {
// Fixed width mode
// Allows for use with legacy charts
const fixedWidth =
typeof fd.line_width === 'number'
? fd.line_width
: typeof fd.line_width === 'object' && fd.line_width?.type === 'fix'
? Number(fd.line_width.value)
: undefined;
data = data.map((d: JsonObject) => {
let width = (d.width ?? fixedWidth ?? 1) * multiplier;
width = Math.max(minWidth, Math.min(maxWidth, width));
return { ...d, width };
});
}
if (fd.js_data_mutator) {
const jsFnMutator = sandboxedEval(fd.js_data_mutator);
@@ -66,13 +122,15 @@ export const getLayer: GetLayerType<PathLayer> = function ({
return new PathLayer({
id: `path-layer-${fd.slice_id}` as const,
getColor: (d: any) => d.color,
getColor: (d: any) => d.color || [0, 0, 0, 255],
getPath: (d: any) => d.path,
getWidth: (d: any) => d.width,
data,
rounded: true,
widthScale: 1,
widthUnits: fd.line_width_unit,
widthMinPixels: Number(fd.min_width) || undefined,
widthMaxPixels: Number(fd.max_width) || undefined,
...commonLayerProps({
formData: fd,
setTooltip,
@@ -101,13 +159,23 @@ export const getHighlightLayer: GetLayerType<PathLayer> = function ({
filterState,
}) {
const fd = formData;
const minWidth = Number(fd.min_width) || 1;
const maxWidth = Number(fd.max_width) || 20;
const multiplier = Number(fd.line_width_multiplier) || 1;
const fixedColor = HIGHLIGHT_COLOR_ARRAY;
let data = payload.data.features.map((feature: JsonObject) => ({
...feature,
path: feature.path,
width: fd.line_width,
color: fixedColor,
}));
let data = payload.data.features.map((feature: JsonObject) => {
const baseWidth = Number.isFinite(feature.width) ? feature.width : 1;
let width = baseWidth * multiplier;
width = Math.max(minWidth, Math.min(maxWidth, width));
return {
...feature,
path: feature.path,
width,
color: fixedColor,
};
});
if (fd.js_data_mutator) {
const jsFnMutator = sandboxedEval(fd.js_data_mutator);
@@ -128,7 +196,13 @@ export const getHighlightLayer: GetLayerType<PathLayer> = function ({
rounded: true,
widthScale: 1,
widthUnits: fd.line_width_unit,
widthMinPixels: Number(fd.min_width) || undefined,
widthMaxPixels: Number(fd.max_width) || undefined,
});
};
export default createDeckGLComponent(getLayer, getPoints, getHighlightLayer);
export default createCategoricalDeckGLComponent(
getLayer,
getPoints,
getHighlightLayer,
);

View File

@@ -0,0 +1,355 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import buildQuery, { DeckPathFormData } from './buildQuery';
const baseFormData: DeckPathFormData = {
datasource: '1__table',
viz_type: 'deck_path',
line_column: 'path_json',
line_type: 'json',
row_limit: 100,
};
test('Path buildQuery should not include metric when line_width is fixed type', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'fix',
value: 5,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should handle numeric line_width value with fixed type', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'fix',
value: 5,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should handle missing line_width', () => {
const formData: DeckPathFormData = {
...baseFormData,
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should include metric when line_width is metric type', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: 'COUNT(*)',
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContain('COUNT(*)');
});
test('Path buildQuery should add line_column to groupby when using width metric', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: 'SUM(distance)',
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.groupby).toContain('path_json');
});
test('Path buildQuery should handle adhoc SQL metric for line_width', () => {
const adhocMetric = {
label: 'custom_width',
expressionType: 'SQL' as const,
sqlExpression: 'SUM(weight) / COUNT(*)',
};
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: adhocMetric,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContainEqual(adhocMetric);
});
test('Path buildQuery should handle adhoc SIMPLE metric for line_width', () => {
const adhocMetric = {
label: 'AVG(traffic)',
expressionType: 'SIMPLE' as const,
column: { column_name: 'traffic' },
aggregate: 'AVG' as const,
};
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: adhocMetric,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContainEqual(adhocMetric);
});
test('Path buildQuery should handle metric type with undefined value', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: undefined,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should not duplicate width metric if already in metrics', () => {
const formData: DeckPathFormData = {
...baseFormData,
metrics: ['AVG(weight)'],
line_width: {
type: 'metric',
value: 'AVG(weight)',
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toHaveLength(1);
});
test('Path buildQuery should preserve existing metrics when adding width metric', () => {
const formData: DeckPathFormData = {
...baseFormData,
metrics: ['COUNT(*)'],
line_width: {
type: 'metric',
value: 'AVG(weight)',
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContain('COUNT(*)');
expect(query.metrics).toContain('AVG(weight)');
expect(query.metrics).toHaveLength(2);
});
test('Path buildQuery should not modify existing metrics for fixed width', () => {
const formData: DeckPathFormData = {
...baseFormData,
metrics: ['COUNT(*)', 'SUM(value)'],
line_width: {
type: 'fix',
value: 5,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual(['COUNT(*)', 'SUM(value)']);
});
test('Path buildQuery should handle undefined value in metric type gracefully', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: undefined,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
// Should not add anything when value is undefined
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should handle line_width with undefined type', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: undefined,
value: 2,
},
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
// ─── Dimension (categorical color) ───
test('Path buildQuery should include dimension column when specified', () => {
const formData: DeckPathFormData = {
...baseFormData,
dimension: 'route_type',
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.columns).toContain('route_type');
});
test('Path buildQuery should include breakpoint_metric when specified', () => {
const formData: DeckPathFormData = {
...baseFormData,
breakpoint_metric: 'AVG(speed)',
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContain('AVG(speed)');
});
test('Path buildQuery should add line_column to groupby when using breakpoint metric', () => {
const formData: DeckPathFormData = {
...baseFormData,
breakpoint_metric: 'AVG(speed)',
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.groupby).toContain('path_json');
});
test('Path buildQuery should not duplicate breakpoint metric if already in metrics', () => {
const formData: DeckPathFormData = {
...baseFormData,
metrics: ['AVG(speed)'],
breakpoint_metric: 'AVG(speed)',
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toHaveLength(1);
expect(query.metrics).toContain('AVG(speed)');
});
test('Path buildQuery should handle breakpoint_metric and line_width metric together', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: 'SUM(distance)',
},
breakpoint_metric: 'AVG(speed)',
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContain('SUM(distance)');
expect(query.metrics).toContain('AVG(speed)');
});
test('Path buildQuery should handle adhoc breakpoint metric', () => {
const adhocMetric = {
label: 'avg_speed',
expressionType: 'SQL' as const,
sqlExpression: 'AVG(speed_mph)',
};
const formData: DeckPathFormData = {
...baseFormData,
breakpoint_metric: adhocMetric,
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContainEqual(adhocMetric);
});
test('Path buildQuery should handle missing breakpoint_metric', () => {
const formData: DeckPathFormData = {
...baseFormData,
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toEqual([]);
});
test('Path buildQuery should handle line_width and breakpoint_metrics together together', () => {
const formData: DeckPathFormData = {
...baseFormData,
line_width: {
type: 'metric',
value: 'SUM(distance)',
},
breakpoint_metric: 'AVG(speed)',
js_columns: ['color'],
tooltip_contents: ['name'],
row_limit: 500,
};
const queryContext = buildQuery(formData);
const [query] = queryContext.queries;
expect(query.metrics).toContain('SUM(distance)');
expect(query.metrics).toContain('AVG(speed)');
expect(query.columns).toContain('color');
expect(query.columns).toContain('name');
expect(query.row_limit).toBe(500);
});

View File

@@ -19,10 +19,13 @@
import {
buildQueryContext,
ensureIsArray,
getMetricLabel,
SqlaFormData,
QueryFormColumn,
QueryFormMetric,
} from '@superset-ui/core';
import { addNullFilters, addTooltipColumnsToQuery } from '../buildQueryUtils';
import { isMetricValue } from '../utils/metricUtils';
export interface DeckPathFormData extends SqlaFormData {
line_column?: string;
@@ -32,10 +35,26 @@ export interface DeckPathFormData extends SqlaFormData {
js_columns?: string[];
tooltip_contents?: unknown[];
tooltip_template?: string;
line_width?:
| string
| { type?: 'fix' | 'metric'; value?: QueryFormMetric | number };
line_width_multiplier?: number;
min_width?: number;
max_width?: number;
dimension?: string;
breakpoint_metric?: QueryFormMetric;
}
export default function buildQuery(formData: DeckPathFormData) {
const { line_column, metric, js_columns, tooltip_contents } = formData;
const {
line_column,
metric,
js_columns,
tooltip_contents,
line_width,
dimension,
breakpoint_metric,
} = formData;
if (!line_column) {
throw new Error('Line column is required for Path charts');
@@ -46,7 +65,7 @@ export default function buildQuery(formData: DeckPathFormData) {
const columns = ensureIsArray(
baseQueryObject.columns || [],
) as QueryFormColumn[];
const metrics = ensureIsArray(baseQueryObject.metrics || []);
let metrics = ensureIsArray(baseQueryObject.metrics || []);
const groupby = ensureIsArray(
baseQueryObject.groupby || [],
) as QueryFormColumn[];
@@ -63,6 +82,49 @@ export default function buildQuery(formData: DeckPathFormData) {
columns.push(line_column);
}
// Include dimension column for categorical color mode
if (dimension && !columns.includes(dimension)) {
columns.push(dimension);
}
// Add metric if line_width is a metric type
const isMetric = isMetricValue(line_width);
const rawWidthValue =
typeof line_width === 'string'
? line_width
: typeof line_width === 'number'
? undefined
: line_width?.value;
const widthMetric: QueryFormMetric | null =
isMetric &&
rawWidthValue !== undefined &&
typeof rawWidthValue !== 'number'
? (rawWidthValue as QueryFormMetric)
: null;
// ensure metric is not added to metric array twice
const existingLabels = new Set(metrics.map(m => getMetricLabel(m)));
if (widthMetric && !existingLabels.has(getMetricLabel(widthMetric))) {
metrics = [...metrics, widthMetric];
}
// ensure line_column is in groupby when aggregating by width metric
if (widthMetric && !groupby.includes(line_column)) {
groupby.push(line_column);
}
if (breakpoint_metric) {
const breakpointLabel = getMetricLabel(breakpoint_metric);
const currentLabels = new Set(metrics.map(m => getMetricLabel(m)));
if (!currentLabels.has(breakpointLabel)) {
metrics = [...metrics, breakpoint_metric];
}
// ensure line_column is in groupby when aggregating
if (!groupby.includes(line_column)) {
groupby.push(line_column);
}
}
jsColumns.forEach(col => {
if (!columns.includes(col) && !groupby.includes(col)) {
columns.push(col);

View File

@@ -0,0 +1,242 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import type {
ControlPanelSectionConfig,
ControlSetRow,
ControlSetItem,
} from '@superset-ui/chart-controls';
import controlPanel from './controlPanel';
test('controlPanel should have Path Size section', () => {
const pathSizeSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
expect(pathSizeSection).toBeDefined();
expect(pathSizeSection?.expanded).toBe(true);
});
test('controlPanel should include pathLineWidthFixedOrMetric control', () => {
const pathSizeSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
const control = pathSizeSection?.controlSetRows
.flat()
.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'line_width',
) as any;
expect(control).toBeDefined();
expect(control.config.type).toBe('FixedOrMetricControl');
expect(control.config.default).toEqual({ type: 'fix', value: 1 });
});
test('controlPanel should include line_width_unit control with pixels as default', () => {
const pathSizeSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
const lineWidthRow = pathSizeSection?.controlSetRows.find(
(row: ControlSetRow) =>
row.some(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'line_width_unit',
),
);
const lineWidthControl = lineWidthRow?.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'line_width_unit',
) as any;
expect(lineWidthControl).toBeDefined();
expect(lineWidthControl?.config?.default).toBe('pixels');
});
test('controlPanel should include min_width control with default of 1', () => {
const minWidthSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
const minWidthRow = minWidthSection?.controlSetRows.find(
(row: ControlSetRow) =>
row.some(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'min_width',
),
);
const minWidthControl = minWidthRow?.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'min_width',
) as any;
expect(minWidthControl).toBeDefined();
expect(minWidthControl?.config?.default).toBe(1);
});
test('controlPanel should include max_width control with default of 20', () => {
const maxWidthSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
const maxWidthRow = maxWidthSection?.controlSetRows.find(
(row: ControlSetRow) =>
row.some(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'max_width',
),
);
const maxWidthControl = maxWidthRow?.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'max_width',
) as any;
expect(maxWidthControl).toBeDefined();
expect(maxWidthControl?.config?.default).toBe(20);
});
test('controlPanel should include line_width_multiplier control with default of 1', () => {
const lineWidthMultiplierSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Size',
);
const lineWidthMultiplierRow =
lineWidthMultiplierSection?.controlSetRows.find((row: ControlSetRow) =>
row.some(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'line_width_multiplier',
),
);
const lineWidthMultiplierControl = lineWidthMultiplierRow?.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'line_width_multiplier',
) as any;
expect(lineWidthMultiplierControl).toBeDefined();
expect(lineWidthMultiplierControl?.config?.default).toBe(1);
});
test('controlPanel should have Path Color section', () => {
const pathColorSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Color',
);
expect(pathColorSection).toBeDefined();
expect(pathColorSection?.expanded).toBe(true);
});
test('controlPanel should have Path Color section with color scheme controls', () => {
const pathColorSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Color',
);
const controlNames = pathColorSection?.controlSetRows
.flat()
.filter(
(control: ControlSetItem) =>
control && typeof control === 'object' && 'name' in control,
)
.map((control: any) => control.name);
expect(controlNames).toContain('color_scheme_type');
expect(controlNames).toContain('color_picker');
expect(controlNames).toContain('dimension');
expect(controlNames).toContain('color_scheme');
expect(controlNames).toContain('breakpoint_metric');
expect(controlNames).toContain('default_breakpoint_color');
expect(controlNames).toContain('color_breakpoints');
});
test('color_scheme_type should default to fixed_color', () => {
const pathColorSection = controlPanel.controlPanelSections.find(
(
section: ControlPanelSectionConfig | null,
): section is ControlPanelSectionConfig =>
section != null && section.label === 'Path Color',
);
const schemeTypeControl = pathColorSection?.controlSetRows
.flat()
.find(
(control: ControlSetItem) =>
control &&
typeof control === 'object' &&
'name' in control &&
control.name === 'color_scheme_type',
) as any;
expect(schemeTypeControl).toBeDefined();
expect(schemeTypeControl?.config?.default).toBe('fixed_color');
});

View File

@@ -26,7 +26,6 @@ import {
jsTooltip,
jsOnclickHref,
viewport,
lineWidth,
lineType,
reverseLongLat,
mapboxStyle,
@@ -34,8 +33,12 @@ import {
mapProvider,
tooltipContents,
tooltipTemplate,
pathLineWidthFixedOrMetric,
generateDeckGLColorSchemeControls,
} from '../../utilities/Shared_DeckGL';
import { dndLineColumn } from '../../utilities/sharedDndControls';
import { validateNonEmpty } from '@superset-ui/core';
import { COLOR_SCHEME_TYPES } from '../../utilities/utils';
const config: ControlPanelConfig = {
controlPanelSections: [
@@ -71,25 +74,83 @@ const config: ControlPanelConfig = {
[mapboxStyle],
[maplibreStyle],
[viewport],
['color_picker'],
[lineWidth],
[reverseLongLat],
[autozoom],
],
},
{
label: t('Path Size'),
expanded: true,
controlSetRows: [
[pathLineWidthFixedOrMetric],
[
{
name: 'line_width_unit',
config: {
type: 'SelectControl',
label: t('Line width unit'),
default: 'meters',
default: 'pixels',
choices: [
['meters', t('meters')],
['pixels', t('pixels')],
],
renderTrigger: true,
},
},
],
[reverseLongLat],
[autozoom],
[
{
name: 'min_width',
config: {
type: 'TextControl',
label: t('Minimum Width'),
isFloat: true,
validators: [validateNonEmpty],
renderTrigger: true,
default: 1,
description: t(
'Minimum width size of the path, in pixels or meters.',
),
},
},
{
name: 'max_width',
config: {
type: 'TextControl',
label: t('Maximum Width'),
isFloat: true,
validators: [validateNonEmpty],
renderTrigger: true,
default: 20,
description: t(
'Maximum width size of the path, in pixels or meters.',
),
},
},
],
[
{
name: 'line_width_multiplier',
config: {
type: 'TextControl',
label: t('Width scale multiplier'),
renderTrigger: true,
isFloat: true,
default: 1,
description: t(
'Scale factor applied to metric-driven line widths',
),
},
},
],
],
},
{
label: t('Path Color'),
expanded: true,
controlSetRows: [
...generateDeckGLColorSchemeControls({
defaultSchemeType: COLOR_SCHEME_TYPES.fixed_color,
}),
],
},
{

View File

@@ -0,0 +1,364 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps, DatasourceType } from '@superset-ui/core';
import transformProps from './transformProps';
interface PathFeature {
path: [number, number][];
width?: number;
metric?: number;
cat_color?: string;
extraProps?: Record<string, unknown>;
[key: string]: unknown;
}
const samplePath1 = JSON.stringify([
[-122.4, 37.8],
[-122.3, 37.9],
]);
const samplePath2 = JSON.stringify([
[-122.5, 37.7],
[-122.4, 37.8],
]);
const samplePath3 = JSON.stringify([
[-122.6, 37.6],
[-122.5, 37.7],
]);
const mockChartProps: Partial<ChartProps> = {
rawFormData: {
line_column: 'path_json',
line_type: 'json',
viewport: {},
},
queriesData: [
{
data: [
{
path_json: samplePath1,
'AVG(weight)': 100,
'SUM(distance)': 500,
route_type: 'express',
},
{
path_json: samplePath2,
'AVG(weight)': 200,
'SUM(distance)': 1000,
route_type: 'local',
},
{
path_json: samplePath3,
'AVG(weight)': 50,
'SUM(distance)': 250,
route_type: 'express',
},
],
},
],
datasource: {
type: DatasourceType.Table,
id: 1,
name: 'test_datasource',
columns: [],
metrics: [],
},
height: 400,
width: 600,
hooks: {},
filterState: {},
emitCrossFilters: false,
};
test('Path transformProps should parse JSON paths correctly', () => {
const result = transformProps(mockChartProps as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features.length).toBe(3);
features.forEach(f => {
expect(f.path).toBeDefined();
expect(Array.isArray(f.path)).toBe(true);
expect(f.path.length).toBeGreaterThan(0);
});
});
test('Path transformProps should handle empty records', () => {
const props = {
...mockChartProps,
queriesData: [{ data: [] }],
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features).toHaveLength(0);
});
test('Path transformProps should handle missing line_column', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
line_column: undefined,
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features).toHaveLength(0);
});
test('Path transformProps should handle invalid JSON path data', () => {
const props = {
...mockChartProps,
queriesData: [
{
data: [{ path_json: 'not valid json' }, { path_json: '12345' }],
},
],
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features.length).toBe(2);
// Should not throw, paths should be empty arrays
features.forEach(f => {
expect(Array.isArray(f.path)).toBe(true);
});
});
test('Path transformProps should use fixed width value when line_width type is "fix"', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
line_width: {
type: 'fix',
value: 5,
},
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features.length).toBe(3);
features.forEach(f => {
expect(f.width).toBe(5);
});
});
test('Path transformProps should use fixed width with string value', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
line_width: {
type: 'fix',
value: '10',
},
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
features.forEach(f => {
expect(f.width).toBe(10);
});
});
test('Path transformProps should not set width when line_width is missing', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
line_width: undefined,
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
features.forEach(f => {
expect(f.width).toBeUndefined();
});
});
test('Path transformProps should use metric value for width when line_width type is "metric"', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
line_width: {
type: 'metric',
value: 'AVG(weight)',
},
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features).toHaveLength(3);
expect(features[0]?.width).toBe(50);
});
test('Path transformProps should include metric from breakpoint_metric', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
breakpoint_metric: 'AVG(weight)',
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
const metrics = features
.map(f => f.metric)
.filter((m): m is number => m !== undefined)
.sort((a, b) => a - b);
expect(metrics).toEqual([50, 100, 200]);
});
test('Path transformProps should fall back to base metric when breakpoint_metric is missing', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
metric: 'AVG(weight)',
breakpoint_metric: undefined,
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
const metrics = features
.map(f => f.metric)
.filter((m): m is number => m !== undefined)
.sort((a, b) => a - b);
expect(metrics).toEqual([50, 100, 200]);
});
test('Path transformProps should include both breakpoint_metric and width metrics if they are different', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
breakpoint_metric: 'AVG(weight)',
line_width: {
type: 'metric',
value: 'SUM(distance)',
},
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features).toHaveLength(3);
expect(result.payload.data.metricLabels).toEqual([
'AVG(weight)',
'SUM(distance)',
]);
});
test('Path transformProps should not include both breakpoint_metric and width metrics if they are the same', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
breakpoint_metric: 'SUM(distance)',
line_width: {
type: 'metric',
value: 'SUM(distance)',
},
},
};
const result = transformProps(props as ChartProps);
expect(result.payload.data.metricLabels).toEqual(['SUM(distance)']);
});
test('Path transformProps should set cat_color from dimension column', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
dimension: 'route_type',
},
};
const result = transformProps(props as ChartProps);
const features = result.payload.data.features as PathFeature[];
expect(features).toHaveLength(3);
expect(features[0]?.cat_color).toBe('express');
expect(features[1]?.cat_color).toBe('local');
expect(features[2]?.cat_color).toBe('express');
});
test('Path transformProps should include metric labels when breakpoint_metric is set', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
breakpoint_metric: 'AVG(weight)',
},
};
const result = transformProps(props as ChartProps);
expect(result.payload.data.metricLabels).toContain('AVG(weight)');
});
test('Path transformProps should include metric labels from base metric', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
metric: 'SUM(distance)',
},
};
const result = transformProps(props as ChartProps);
expect(result.payload.data.metricLabels).toContain('SUM(distance)');
});
test('Path transformProps should have empty metric labels when no metric is set', () => {
const props = {
...mockChartProps,
rawFormData: {
...mockChartProps.rawFormData,
metric: undefined,
breakpoint_metric: undefined,
},
};
const result = transformProps(props as ChartProps);
expect(result.payload.data.metricLabels).toEqual([]);
});

View File

@@ -16,7 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps, DTTM_ALIAS } from '@superset-ui/core';
import { ChartProps, DTTM_ALIAS, getMetricLabel } from '@superset-ui/core';
import { addJsColumnsToExtraProps, DataRecord } from '../spatialUtils';
import {
createBaseTransformResult,
@@ -26,6 +26,7 @@ import {
addPropertiesToFeature,
} from '../transformUtils';
import { DeckPathFormData } from './buildQuery';
import { isFixedValue, getFixedValue } from '../utils/metricUtils';
declare global {
interface Window {
@@ -48,6 +49,8 @@ interface PathFeature {
path: [number, number][];
metric?: number;
timestamp?: unknown;
width?: number;
cat_color?: string;
extraProps?: Record<string, unknown>;
[key: string]: unknown;
}
@@ -91,6 +94,9 @@ function processPathData(
reverseLongLat: boolean = false,
metricLabel?: string,
jsColumns?: string[],
widthMetricLabel?: string,
fixedWidthValue?: number | string | null,
categoryColumn?: string,
): PathFeature[] {
if (!records.length || !lineColumn) {
return [];
@@ -103,6 +109,8 @@ function processPathData(
'timestamp',
DTTM_ALIAS,
metricLabel,
widthMetricLabel,
categoryColumn,
...(jsColumns || []),
].filter(Boolean) as string[],
);
@@ -130,6 +138,24 @@ function processPathData(
feature.metric = metricValue;
}
}
// Set width from metric or fixed value
if (fixedWidthValue != null) {
// Use fixed width
const parsedFixedWidth = parseMetricValue(fixedWidthValue);
if (parsedFixedWidth !== undefined) {
feature.width = parsedFixedWidth;
}
} else if (widthMetricLabel && record[widthMetricLabel] != null) {
// Use metric value for width
const widthValue = parseMetricValue(record[widthMetricLabel]);
if (widthValue !== undefined) {
feature.width = widthValue;
}
}
if (categoryColumn && record[categoryColumn] != null) {
feature.cat_color = String(record[categoryColumn]);
}
feature = addJsColumnsToExtraProps(feature, record, jsColumns);
feature = addPropertiesToFeature(feature, record, excludeKeys);
@@ -143,11 +169,37 @@ export default function transformProps(chartProps: ChartProps) {
line_column,
line_type = 'json',
metric,
line_width,
dimension,
reverse_long_lat = false,
js_columns,
breakpoint_metric,
} = formData as DeckPathTransformPropsFormData;
const metricLabel = getMetricLabelFromFormData(metric);
// Check so legacy values still work
const fixedWidthValue =
typeof line_width === 'number'
? line_width
: isFixedValue(line_width)
? getFixedValue(line_width)
: undefined;
const widthMetricLabel = getMetricLabelFromFormData(line_width);
const breakpointMetricLabel = breakpoint_metric
? getMetricLabel(breakpoint_metric)
: undefined;
const baseMetricLabel = getMetricLabelFromFormData(metric);
const metricLabel = breakpointMetricLabel || baseMetricLabel;
// ensure all metric labels are included
const metricLabels = [
...(metricLabel ? [metricLabel] : []),
...(widthMetricLabel && widthMetricLabel !== metricLabel
? [widthMetricLabel]
: []),
];
const records = getRecordsFromQuery(chartProps.queriesData);
const features = processPathData(
records,
@@ -156,11 +208,10 @@ export default function transformProps(chartProps: ChartProps) {
reverse_long_lat,
metricLabel,
js_columns,
widthMetricLabel,
fixedWidthValue,
dimension,
).reverse();
return createBaseTransformResult(
chartProps,
features,
metricLabel ? [metricLabel] : [],
);
return createBaseTransformResult(chartProps, features, metricLabels);
}

View File

@@ -285,6 +285,22 @@ export const lineWidth = {
},
};
// created new const so as not to break lineWidth usages in other charts
export const pathLineWidthFixedOrMetric = {
name: 'line_width',
config: {
type: 'FixedOrMetricControl', // using existing type
label: t('Line width'),
default: { type: 'fix', value: 1 }, // kept same default as before
description: t(
'The width of the lines as either a fixed value or variable width based on a metric.',
),
mapStateToProps: (state: ControlPanelState) => ({
datasource: state.datasource,
}),
},
};
export const fillColorPicker: CustomControlItem = {
name: 'fill_color_picker',
config: {
@@ -673,6 +689,24 @@ export const deckGLColorBreakpointsSelect: CustomControlItem = {
},
};
export const deckGLBreakpointMetric: CustomControlItem = {
name: 'breakpoint_metric',
config: {
...sharedControls.metric,
label: t('Breakpoint Metric'),
default: null,
validators: [],
description: t(
'Select the metric used to determine which color breakpoint range each path falls into.',
),
// mapStateToProps: (state: ControlPanelState) => ({
// datasource: state.datasource,
// }),
visibility: ({ controls }: { controls: any }) =>
isColorSchemeTypeVisible(controls, COLOR_SCHEME_TYPES.color_breakpoints),
},
};
export const breakpointsDefaultColor: CustomControlItem = {
name: 'default_breakpoint_color',
config: {
@@ -725,6 +759,7 @@ export const generateDeckGLColorSchemeControls = ({
[deckGLFixedColor],
disableCategoricalColumn ? [] : [deckGLCategoricalColor],
[deckGLCategoricalColorSchemeSelect],
[deckGLBreakpointMetric],
[breakpointsDefaultColor],
[deckGLColorBreakpointsSelect],
];

View File

@@ -47,6 +47,13 @@ const Title = styled.h4`
font-weight: ${({ theme }) => theme.fontWeightStrong};
`;
const StyledTabs = styled(Tabs)`
margin-top: ${({ theme }) => theme.sizeUnit * -8}px;
.ant-tabs-nav {
margin-bottom: ${({ theme }) => theme.sizeUnit * 4}px;
}
`;
const shrinkSql = (sql: string, maxLines: number, maxWidth: number) => {
const ssql = sql || '';
let lines = ssql.split('\n');
@@ -94,7 +101,7 @@ function HighlightSqlModal({ rawSql, sql }: HighlightedSqlModalTypes) {
}
return (
<Tabs
<StyledTabs
defaultActiveKey="executed"
items={[
{

View File

@@ -0,0 +1,53 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import config from './controlPanel';
type ControlConfig = {
label?: unknown;
description?: unknown;
};
type ControlItem = {
config: ControlConfig;
} | null;
function collectFunctionProps(cfg: typeof config) {
const fns: Array<() => unknown> = [];
cfg.controlPanelSections.forEach(section => {
section?.controlSetRows.forEach(row => {
(row as ControlItem[]).forEach(item => {
if (item && typeof item === 'object' && 'config' in item) {
const { label, description } = item.config;
if (typeof label === 'function') fns.push(label as () => unknown);
if (typeof description === 'function')
fns.push(description as () => unknown);
}
});
});
});
return fns;
}
test('DynamicGroupBy controlPanel label and description functions return strings', () => {
const fns = collectFunctionProps(config);
expect(fns.length).toBeGreaterThan(0);
fns.forEach(fn => {
expect(typeof fn()).toBe('string');
});
});

View File

@@ -49,12 +49,12 @@ const config: ControlPanelConfig = {
name: 'canSelectMultiple',
config: {
type: 'CheckboxControl',
label: t('Can select multiple values'),
label: () => t('Can select multiple values'),
default: true,
renderTrigger: true,
resetConfig: true,
affectsDataMask: true,
description: t('Allow users to select multiple values'),
description: () => t('Allow users to select multiple values'),
},
},
],
@@ -63,12 +63,13 @@ const config: ControlPanelConfig = {
name: 'enableEmptyFilter',
config: {
type: 'CheckboxControl',
label: t('Chart customization value is required'),
label: () => t('Chart customization value is required'),
default: false,
renderTrigger: true,
description: t(
'User must select a value before applying the chart customization',
),
description: () =>
t(
'User must select a value before applying the chart customization',
),
},
},
],

View File

@@ -0,0 +1,53 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import config from './controlPanel';
type ControlConfig = {
label?: unknown;
description?: unknown;
};
type ControlItem = {
config: ControlConfig;
} | null;
function collectFunctionProps(cfg: typeof config) {
const fns: Array<() => unknown> = [];
cfg.controlPanelSections.forEach(section => {
section?.controlSetRows.forEach(row => {
(row as ControlItem[]).forEach(item => {
if (item && typeof item === 'object' && 'config' in item) {
const { label, description } = item.config;
if (typeof label === 'function') fns.push(label as () => unknown);
if (typeof description === 'function')
fns.push(description as () => unknown);
}
});
});
});
return fns;
}
test('TimeColumn controlPanel label and description functions return strings', () => {
const fns = collectFunctionProps(config);
expect(fns.length).toBeGreaterThan(0);
fns.forEach(fn => {
expect(typeof fn()).toBe('string');
});
});

View File

@@ -30,12 +30,11 @@ const config: ControlPanelConfig = {
name: 'enableEmptyFilter',
config: {
type: 'CheckboxControl',
label: t('Filter value is required'),
label: () => t('Filter value is required'),
default: false,
renderTrigger: true,
description: t(
'User must select a value before applying the filter',
),
description: () =>
t('User must select a value before applying the filter'),
},
},
],

View File

@@ -0,0 +1,53 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import config from './controlPanel';
type ControlConfig = {
label?: unknown;
description?: unknown;
};
type ControlItem = {
config: ControlConfig;
} | null;
function collectFunctionProps(cfg: typeof config) {
const fns: Array<() => unknown> = [];
cfg.controlPanelSections.forEach(section => {
section?.controlSetRows.forEach(row => {
(row as ControlItem[]).forEach(item => {
if (item && typeof item === 'object' && 'config' in item) {
const { label, description } = item.config;
if (typeof label === 'function') fns.push(label as () => unknown);
if (typeof description === 'function')
fns.push(description as () => unknown);
}
});
});
});
return fns;
}
test('TimeGrain controlPanel label and description functions return strings', () => {
const fns = collectFunctionProps(config);
expect(fns.length).toBeGreaterThan(0);
fns.forEach(fn => {
expect(typeof fn()).toBe('string');
});
});

View File

@@ -30,12 +30,11 @@ const config: ControlPanelConfig = {
name: 'enableEmptyFilter',
config: {
type: 'CheckboxControl',
label: t('Customization value is required'),
label: () => t('Customization value is required'),
default: false,
renderTrigger: true,
description: t(
'User must select a value before applying the customization',
),
description: () =>
t('User must select a value before applying the customization'),
},
},
],

View File

@@ -42,7 +42,10 @@ import {
getQuerySettings,
getChartDataUri,
} from 'src/explore/exploreUtils';
import { addDangerToast } from 'src/components/MessageToasts/actions';
import {
addDangerToast,
addWarningToast,
} from 'src/components/MessageToasts/actions';
import { logEvent } from 'src/logger/actions';
import { Logger, LOG_ACTIONS_LOAD_CHART } from 'src/logger/LogUtils';
import { allowCrossDomain as domainShardingEnabled } from 'src/utils/hostNamesConfig';
@@ -813,6 +816,11 @@ export function exploreJSON(
}),
),
);
(queriesResponse as QueryData[]).forEach(response => {
if (response.warning) {
dispatch(addWarningToast(response.warning, { noDuplicate: true }));
}
});
return dispatch(
chartUpdateSucceeded(queriesResponse as QueryData[], key as number),
);

View File

@@ -31,6 +31,7 @@ import {
AnnotationSourceType,
AnnotationStyle,
} from '@superset-ui/core';
import * as toastActions from 'src/components/MessageToasts/actions';
import { LOG_EVENT } from 'src/logger/actions';
import * as exploreUtils from 'src/explore/exploreUtils';
import * as actions from 'src/components/Chart/chartAction';
@@ -412,6 +413,56 @@ describe('chart actions', () => {
);
expect(result).toEqual([1, 2, 3]);
});
test('dispatches addWarningToast when a query response includes a warning', async () => {
const warningMessage =
'Results truncated to 1,000 rows due to memory constraints.';
fetchMock.removeRoute(MOCK_URL);
fetchMock.post(
`glob:*${MOCK_URL}*`,
{ result: [{ warning: warningMessage }] },
{ name: MOCK_URL },
);
const addWarningToastSpy = jest.spyOn(toastActions, 'addWarningToast');
const actionThunk = actions.postChartFormData(
{ viz_type: 'my_viz' } as QueryFormData,
false,
undefined,
undefined,
);
await actionThunk(
dispatch as unknown as actions.ChartThunkDispatch,
mockGetState as unknown as () => actions.RootState,
undefined,
);
expect(addWarningToastSpy).toHaveBeenCalledWith(warningMessage, {
noDuplicate: true,
});
addWarningToastSpy.mockRestore();
fetchMock.removeRoute(MOCK_URL);
setupDefaultFetchMock();
});
test('does not dispatch addWarningToast when no query response has a warning', async () => {
const addWarningToastSpy = jest.spyOn(toastActions, 'addWarningToast');
const actionThunk = actions.postChartFormData(
{ viz_type: 'my_viz' } as QueryFormData,
false,
undefined,
undefined,
);
await actionThunk(
dispatch as unknown as actions.ChartThunkDispatch,
mockGetState as unknown as () => actions.RootState,
undefined,
);
expect(addWarningToastSpy).not.toHaveBeenCalled();
addWarningToastSpy.mockRestore();
});
});
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks

View File

@@ -34,6 +34,7 @@ import { DropResult } from 'src/dashboard/components/dnd/dragDroppableConfig';
import { GetState, LayoutItem, RootState } from '../types';
import { updateLayoutComponents } from './dashboardFilters';
import { setUnsavedChanges } from './dashboardState';
type AppDispatch = ThunkDispatch<RootState, undefined, AnyAction>;
// Component CRUD -------------------------------------------------------------

View File

@@ -468,9 +468,12 @@ export function saveDashboardRequest(
);
const cleanedData: JsonObject = {
...data,
certified_by: certified_by || '',
certification_details:
certified_by && certification_details ? certification_details : '',
...(certified_by !== undefined && {
certified_by,
certification_details: certified_by
? (certification_details ?? '')
: '',
}),
css: css || '',
dashboard_title: dashboard_title || t('[ untitled dashboard ]'),
owners: ensureIsArray(owners as JsonObject[]).map((o: JsonObject) =>

View File

@@ -37,7 +37,7 @@ export const useDashboardMetadataBar = (dashboardInfo: DashboardInfo) => {
type: MetadataType.Owner as const,
createdBy: getOwnerName(dashboardInfo.created_by) || t('Not available'),
owners:
dashboardInfo.owners.length > 0
dashboardInfo.owners && dashboardInfo.owners.length > 0
? dashboardInfo.owners.map(getOwnerName)
: t('None'),
createdOn: dashboardInfo.created_on_delta_humanized,

View File

@@ -118,7 +118,7 @@ const NewChartButtonContainer = styled.div`
${({ theme }) => css`
display: flex;
justify-content: flex-end;
padding-right: ${theme.sizeUnit * 2}px;
padding: ${theme.sizeUnit * 3}px ${theme.sizeUnit * 2}px 0;
`}
`;

View File

@@ -234,6 +234,26 @@ describe('ControlPanelsContainer', () => {
);
});
test('renders section with function label and description', async () => {
getChartControlPanelRegistry().remove('table');
getChartControlPanelRegistry().registerValue('table', {
controlPanelSections: [
{
label: () => t('Dynamic Section Label'),
description: () => t('Dynamic section description'),
expanded: true,
controlSetRows: [['groupby']],
},
],
});
render(<ControlPanelsContainer {...getDefaultProps()} />, {
useRedux: true,
});
await waitFor(() => {
expect(screen.getByText('Dynamic Section Label')).toBeInTheDocument();
});
});
test('hidden state of controls is correctly applied', async () => {
getChartControlPanelRegistry().remove('table');
getChartControlPanelRegistry().registerValue('table', {

View File

@@ -630,10 +630,17 @@ export const ControlPanelsContainer = (props: ControlPanelsContainerProps) => {
line-height: 1.3;
`}
>
{label}
{typeof label === 'function' ? (label as () => ReactNode)() : label}
</span>{' '}
{description && (
<Tooltip id={sectionId} title={description}>
<Tooltip
id={sectionId}
title={
typeof description === 'function'
? (description as () => ReactNode)()
: description
}
>
<Icons.InfoCircleOutlined css={iconStyles} />
</Tooltip>
)}

View File

@@ -125,7 +125,7 @@ const getTooltipTitle = (
) =>
isLabelTruncated ? (
<div>
{label && <strong>{label}</strong>}
{label && <strong>{t(label)}</strong>}
{range && (
<div
css={(theme: SupersetTheme) => css`
@@ -160,7 +160,7 @@ export default function DateFilterLabel(props: DateFilterControlProps) {
const [timeRangeValue, setTimeRangeValue] = useState(value);
const [validTimeRange, setValidTimeRange] = useState<boolean>(false);
const [evalResponse, setEvalResponse] = useState<string>(value);
const [tooltipTitle, setTooltipTitle] = useState<ReactNode | null>(value);
const [tooltipTitle, setTooltipTitle] = useState<ReactNode | null>(t(value));
const theme = useTheme();
const [labelRef, labelIsTruncated] = useCSSTextTruncation<HTMLSpanElement>();
@@ -175,7 +175,7 @@ export default function DateFilterLabel(props: DateFilterControlProps) {
if (error) {
setEvalResponse(error || '');
setValidTimeRange(false);
setTooltipTitle(value || null);
setTooltipTitle(t(value) || null);
} else {
/*
HRT == human readable text

View File

@@ -486,6 +486,70 @@ test('sets comparator to undefined when operator is IS_NULL or IS_NOT_NULL', ()
});
});
test('hides the value input when operator is IS_NULL', () => {
setup({
adhocFilter: new AdhocFilter({
expressionType: ExpressionTypes.Simple,
subject: 'value',
operatorId: Operators.IsNull,
operator: OPERATOR_ENUM_TO_OPERATOR_TYPE[Operators.IsNull].operation,
comparator: undefined,
clause: Clauses.Where,
}),
});
expect(
screen.queryByPlaceholderText('Filter value (case sensitive)'),
).not.toBeInTheDocument();
});
test('hides the value input when operator is IS_NOT_NULL', () => {
setup({
adhocFilter: new AdhocFilter({
expressionType: ExpressionTypes.Simple,
subject: 'value',
operatorId: Operators.IsNotNull,
operator: OPERATOR_ENUM_TO_OPERATOR_TYPE[Operators.IsNotNull].operation,
comparator: undefined,
clause: Clauses.Where,
}),
});
expect(
screen.queryByPlaceholderText('Filter value (case sensitive)'),
).not.toBeInTheDocument();
});
test('hides the value input when operator is IS_TRUE', () => {
setup({
adhocFilter: new AdhocFilter({
expressionType: ExpressionTypes.Simple,
subject: 'value',
operatorId: Operators.IsTrue,
operator: OPERATOR_ENUM_TO_OPERATOR_TYPE[Operators.IsTrue].operation,
comparator: undefined,
clause: Clauses.Where,
}),
});
expect(
screen.queryByPlaceholderText('Filter value (case sensitive)'),
).not.toBeInTheDocument();
});
test('hides the value input when operator is IS_FALSE', () => {
setup({
adhocFilter: new AdhocFilter({
expressionType: ExpressionTypes.Simple,
subject: 'value',
operatorId: Operators.IsFalse,
operator: OPERATOR_ENUM_TO_OPERATOR_TYPE[Operators.IsFalse].operation,
comparator: undefined,
clause: Clauses.Where,
}),
});
expect(
screen.queryByPlaceholderText('Filter value (case sensitive)'),
).not.toBeInTheDocument();
});
test('should not call API when column has no advanced data type', async () => {
const props = getAdvancedDataTypeTestProps();

Some files were not shown because too many files have changed in this diff Show More