Compare commits

...
Author SHA1 Message Date
Evan RusackasandClaude Opus 4.8 7d355a254e fix(Modal): stop the draggable modal from hijacking text selection (#43498)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-27 03:30:23 -07:00
Evan RusackasandClaude Opus 4.8 ca63bb532d fix(plugin-chart-table): preserve comparison arrow when a column-specific formatter entry is missing (#43494)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-27 03:30:17 -07:00
b71293acde fix(sqllab): stop a database with no extra from breaking SET_DATABASES (#43216)
Co-authored-by: bikashJMV <bikash@jmv.co.in>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Evan Rusackas <evan@rusackas.com>
2026-08-26 20:51:08 -07:00
0e172a6ff3 feat(waterfall): add show/hide toggles for X and Y axes (#42371)
Co-authored-by: Greg Neighbors <gregneighbors@Gregs-MacBook-Air-2.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Greg Neighbors <gregneighbors@Gregs-Air-2.lan>
2026-08-26 20:29:26 -07:00
rlei 1c66b71e23 feat(plugin-chart-echarts): allow decal patterns in custom ECharts options (#43427) 2026-08-26 18:01:52 -07:00
Alejandro Solares 6b0aa8714d chore(deps): raise Pillow/PyJWT floors to match resolved pins (#43519) 2026-08-26 15:55:14 -07:00
Vitor Avila 371e5e25e8 fix(trino): honor the verify arg for the user impersonation flow (#43534) 2026-08-26 19:44:30 -03:00
Luiz OtavioandClaude Opus 5 a140e74f5f fix(excel): handle duplicate column labels in xlsx export (#43561)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 16:39:03 -03:00
Amin Ghadersohi b8308a44a3 fix(pivot-table): exclude rollup totals from conditional formatting scale (#43481) 2026-08-26 15:08:07 -04:00
Amin Ghadersohi 13927f27e2 fix(mcp): raise SDK floor (#43530) 2026-08-26 15:04:59 -04:00
Amin Ghadersohi 434511cb37 fix(mcp): reject unsafe dashboard layout replacements (#43476) 2026-08-26 14:28:26 -04:00
JUST.in DO ITandClaude Sonnet 5 70c9203aa9 fix(sqllab): release DB connection before fetching query results from backend (#43371)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-26 10:37:42 -07:00
Alexandru Soare 455e6603c1 feat(reports): gate retry functionality behind ALERT_REPORTS_RETRY flag (#43553) 2026-08-26 09:18:11 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 0597f36bb6 chore(deps-dev): bump oxlint from 1.78.0 to 1.79.0 in /superset-frontend (#43543)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 08:20:39 -07:00
Rafael BenitezandClaude Opus 4.8 e7ca8b8a57 fix(chart): render ECharts charts at full resolution in download-as-image (#43456)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-26 06:22:23 -04:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 686245a905 chore(deps-dev): bump oxfmt from 0.63.0 to 0.64.0 in /superset-websocket (#43537)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:17:00 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 5a4900abb1 chore(deps-dev): bump vitest from 4.1.10 to 4.1.11 in /superset-websocket (#43538)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:56 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 450e2edc2b chore(deps): bump @swc/core from 1.16.0 to 1.16.1 in /docs (#43539)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:53 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 2bc96a47a4 chore(deps): bump swagger-ui-react from 5.32.13 to 5.32.14 in /docs (#43540)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:49 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 622bc45c9e chore(deps-dev): bump @swc/core from 1.16.0 to 1.16.1 in /superset-frontend (#43541)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:45 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> e97db01482 chore(deps-dev): bump oxfmt from 0.63.0 to 0.64.0 in /superset-frontend (#43542)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:40 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 6194c852f1 chore(deps-dev): bump oxfmt from 0.63.0 to 0.64.0 in /docs (#43544)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:36 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 19eca6523f chore(deps): bump uuid from 14.0.1 to 14.0.2 in /superset-frontend (#43545)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-26 02:16:31 -07:00
Amin GhadersohiandClaude 1fd763bd29 fix(mcp): honor and validate chart filters (#43478)
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-25 20:54:09 -07:00
b4f7114a09 fix(semantic-layer): warn that deleting a layer cascade-deletes its dependent views (#42845)
Co-authored-by: Mike Bridge <michael.bridge@ext.preset.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 20:47:45 -07:00
Vitor Avila 2dfe8a5bd9 fix(OAuth2): Support creating OAuth2 connections via SQLAlchemy URI (#43489) 2026-08-26 00:07:19 -03:00
62d74be0af fix(metadb): apply SUPERSET_META_DB_LIMIT after join instead of per-table (#36304) (#42598)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Joe Li <joe@preset.io>
2026-08-25 19:31:11 -07:00
Shivam Goel 473c318335 chore(superset-core): add __init__.py to semantic_layers (#43528) 2026-08-25 18:07:28 -07:00
Evan RusackasandSuperset Dev 61ab0cdb5d fix(ocient): update GIS test fixtures for pyocient's relocated geo types (#43496)
Co-authored-by: Superset Dev <dev@superset.apache.org>
2026-08-25 17:18:02 -07:00
e6b9205821 fix(country-map): give Alborz its own ISO code instead of reusing Tehran's (#42429)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Joe Li <joe@preset.io>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-08-25 16:59:48 -07:00
0dcb2ca53a feat(maps): Add Italy regions and autonomous provinces country map (#42309)
Co-authored-by: lum4chi <francesco.lumachi@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Joe Li <joe@preset.io>
2026-08-25 15:45:31 -07:00
BexultanandBexultan Mustafin 649d6f1b41 fix(mcp): defer unknown numeric types to compile (#43131)
Co-authored-by: Bexultan Mustafin <bexultan.mustafin@ffins.kz>
2026-08-25 15:35:00 -07:00
BexultanandBexultan Mustafin 90dab7cf61 fix(mcp): accept common chart input variants (#43130)
Co-authored-by: Bexultan Mustafin <bexultan.mustafin@ffins.kz>
2026-08-25 15:22:52 -07:00
Sepuri Sai KrishnaandJoe Li 68386a53ee chore(database): remove dead extra validation exception classes (#42411)
Co-authored-by: Joe Li <joe@preset.io>
2026-08-25 15:14:16 -07:00
b89da3e9fc docs(versioning): fix post-flip doc and comment drift (#43493)
Co-authored-by: Mike Bridge <michael.bridge@ext.preset.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 13:00:25 -07:00
Mike BridgeandClaude Fable 5 88d2c2954e feat(deletion-retention): persist purge block reason codes on the audit log (#43485)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 12:46:55 -07:00
Amin Ghadersohi f903e02d91 chore(deps): restore permissive marshmallow lower bound (>=3.0, <5) (#43521) 2026-08-25 14:38:44 -04:00
Maxime Beaucheminandsadpandajoe fc4d7221ec fix(explore): skip re-fetch when navigating away from /explore (#39506)
Co-authored-by: sadpandajoe <jcli38@gmail.com>
2026-08-25 11:32:25 -07:00
Chen, Ting-AnandJoe Li 3585e8235a fix(i18n): review Traditional Chinese count labels (#43063)
Co-authored-by: Joe Li <joe@preset.io>
2026-08-25 10:28:12 -07:00
Evan RusackasandSuperset Dev 34ffa37aaa chore(ci): drop inert SQLALCHEMY_WARN_20 flag from unit-test CI (#43495)
Co-authored-by: Superset Dev <dev@superset.apache.org>
2026-08-25 10:19:38 -07:00
Russlan Ramdowar 02b3e43b66 fix(number-format): preserve custom smart formatter id (#43439) 2026-08-25 10:11:52 -07:00
DmitryandDmitry Kucher f9530f31ab fix(table): keep each metric's own aggregate in the summary row by default (#43421)
Co-authored-by: Dmitry Kucher <dima@virtuman.com>
2026-08-25 10:10:02 -07:00
dependabot[bot]dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>rusackasClaude Opus 4.8
fd3849cef1 chore(deps): bump antd from 6.6.0 to 6.6.1 in /superset-frontend (#43509)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: rusackas <evan@rusackas.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-25 05:56:18 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> ef0ad01aa5 chore(deps-dev): bump the storybook group across 1 directory with 5 updates (#43507)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 05:52:46 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> d480152735 chore(deps): bump the storybook group in /docs with 2 updates (#43505)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 02:46:30 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> cbfa94c4dd chore(deps): bump antd from 6.6.0 to 6.6.1 in /docs (#43506)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 02:46:26 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> d2aeb29223 chore(deps-dev): bump baseline-browser-mapping from 2.11.14 to 2.11.15 in /superset-frontend (#43508)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 02:46:22 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 7bad69f523 chore(deps): bump dayjs from 1.11.22 to 1.11.23 in /superset-frontend (#43510)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 02:46:17 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 9dd8c42f3d chore(deps): bump dompurify from 3.4.12 to 3.4.13 in /superset-frontend (#43511)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-25 02:46:12 -07:00
115 changed files with 7183 additions and 1828 deletions
@@ -53,12 +53,6 @@ jobs:
python-version: ${{ github.event_name == 'pull_request' && fromJSON('["current"]') || fromJSON('["current", "next"]') }}
env:
PYTHONPATH: ${{ github.workspace }}
# Promotes the SQLAlchemy 2.0 deprecation warnings already locked in as
# errors via pytest.ini's `filterwarnings` to actually run in CI, so a
# regression on those fails the build instead of relying on a
# contributor remembering to set this locally. See the migration
# battleplan: https://github.com/apache/superset/discussions/40273
SQLALCHEMY_WARN_20: "1"
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+18 -12
View File
@@ -58,6 +58,7 @@ the old counter to use the outcome-specific replacements.
- [42930](https://github.com/apache/superset/pull/42930): Dataset import data-URI fetches no longer honor an HTTP(S) proxy when `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS` is `False` (the default): the connection is now made directly to the destination so the peer-address check validates the real target instead of a proxy's. Deployments that require an egress proxy to reach legitimate external data URLs for dataset import should set `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS = True` or otherwise ensure those URLs resolve without one.
- [42935](https://github.com/apache/superset/pull/42935): The MCP service now refuses to start (`MCPAuthConfigError`) when `MCP_JWT_ISSUER` trusts more than one issuer and no `MCP_USER_RESOLVER` is configured, instead of only logging a warning. This was already a documented misconfiguration (the default resolver isn't issuer-scoped, so distinct trusted issuers minting the same username/email would resolve to the same Superset user); deployments trusting multiple issuers must configure an `MCP_USER_RESOLVER` that derives its identity from the token's `iss` claim before upgrading. Single-issuer deployments are unaffected.
- [42429](https://github.com/apache/superset/pull/42429): The Country Map chart's Iran GeoJSON now gives Alborz province its own ISO 3166-2 code, `IR-32`, instead of `IR-30`. `ISO` is the join key used to color/filter provinces on this chart, so any existing dataset keyed on `IR-30` for Alborz will silently stop matching after upgrading; re-key that data to `IR-32`.
- [43388](https://github.com/apache/superset/pull/43388): The MCP service now refuses to start (`MCPAuthConfigError`) if `MCP_DEV_USERNAME` and `MCP_AUTH_ENABLED = True` are both set, and separately if `MCP_AUTH_ENABLED = True` but no usable JWT key material is configured (RSA key/JWKS, or an explicit `MCP_JWT_SECRET` for HMAC) — both previously started with authentication silently weaker than configured. Deployments combining a dev-mode username with JWT auth enabled, or enabling JWT auth without key material, must pick one before upgrading: unset `MCP_DEV_USERNAME` for a real auth deployment, or unset `MCP_AUTH_ENABLED` (or configure the key material) for a dev-mode one. Response caching (`MCP_CACHE_CONFIG["enabled"] = True`) now also excludes every tool with a side effect by default, not only a partial list, so a previously-cached mutating tool call is no longer served from cache; no config change is needed to pick this up.
- [42393](https://github.com/apache/superset/pull/42393): Exported dataset YAML now carries a `uuid` for each metric and column so that custom folder assignments (which reference metrics/columns by UUID) survive an import into another workspace. This affects any export bundle that contains datasets, not just a dataset export: chart, dashboard, database and full-asset exports all embed the same dataset YAML, so a dashboard exported from this release also fails to import into an older one even though no dataset was exported directly. As with `folders` and `currency_code_column`, the affected `datasets/` files fail schema validation (`Unknown field: uuid`) when imported into Superset releases that predate this change; regenerate or hand-edit exports for older targets in mixed-version fleets.
- [42300](https://github.com/apache/superset/pull/42300): Timeseries charts (line/area/bar) with a Y-axis bound in effect — either an explicit `yAxisBounds` or one derived from `truncateYAxis` — now clamp out-of-range data points to that bound instead of letting ECharts drop the point (and the line segments around it) entirely. Any existing chart with a configured Y-axis bound and data outside it will look different after upgrading: a gap becomes a point pinned to the boundary. The clamp also rewrites the value ECharts reads for that point's tooltip and data label, so the displayed value is the bound rather than the true observation.
@@ -184,10 +185,12 @@ misrepresents the entity as unchanged.
- **Storage growth.** Capture writes shadow rows per save, so the metadata
database grows with edit volume. The `version_history.prune_old_versions`
beat task removes rows whose transaction is older than
`SUPERSET_VERSION_HISTORY_RETENTION_DAYS` (default 30). A deployment that
replaces `CELERY_CONFIG` rather than inheriting it must carry both the
`superset.tasks.version_history_retention` import and the beat entry; a
startup warning names whichever is absent.
`SUPERSET_VERSION_HISTORY_RETENTION_DAYS` (default 30).
- **Check a replaced `CELERY_CONFIG`.** Carry both the
`superset.tasks.version_history_retention` import and the
`version_history.prune_old_versions` beat entry; see
[Version-history retention (pruning)](#version-history-retention-pruning) for
the startup-warning behavior.
- **`PUT` responses change shape.** Entity updates now return populated
`old_version_uuid` / `new_version_uuid` fields and an `ETag` header, which
were null or absent while capture was off.
@@ -196,7 +199,10 @@ misrepresents the entity as unchanged.
kill-switch — not removed with the rollout toggles. Setting it to a falsy value
stops capture within a restart, without a revert-and-redeploy. Unlike the
soft-delete toggle, turning it off is a clean stop: existing version rows remain
readable and no entity state is altered.
readable and no entity state is altered. Restore is unavailable (404) while
capture is off. A full rollback also sets
`FEATURE_FLAGS = {"VERSION_HISTORY": False}` to hide the panel — capture off
with the panel left on shows an empty or stale history.
### Scheduled report execution now enforces one application deadline
@@ -658,9 +664,9 @@ ALTER TABLE tagged_object DROP CONSTRAINT <constraint_name>;
ALTER TABLE tagged_object DROP FOREIGN KEY <constraint_name>;
```
### Entity version-history infrastructure (gated off by default)
### Entity version-history infrastructure
Introduces the schema and SQLAlchemy-Continuum wiring that captures version history for charts, dashboards, and datasets, plus read-only `GET /api/v1/{chart,dashboard,dataset}/<uuid>/versions/` endpoints. This ships **inert**: a new config flag `ENABLE_VERSIONING_CAPTURE` defaults to `False`, so no save writes any version rows and the endpoints return empty. It is an operational kill-switch (a release toggle that becomes a permanent ops switch), not a feature flag — set it to `True` to enable capture once validated. The migration is additive; existing entity `PUT` responses gain `old_version_uuid` / `new_version_uuid` body fields and an `ETag` header (both null/absent when capture is off).
Introduces the schema and SQLAlchemy-Continuum wiring that captures version history for charts, dashboards, and datasets, plus read-only `GET /api/v1/{chart,dashboard,dataset}/<uuid>/versions/` endpoints. Capture is governed by the `ENABLE_VERSIONING_CAPTURE` config value — an operational kill-switch (a release toggle that became a permanent ops switch), not a feature flag; see "Version history is on by default" above for the shipped default. With capture off, no save writes version rows; the endpoints continue to serve already-captured rows read-only. The migration is additive; existing entity `PUT` responses gain `old_version_uuid` / `new_version_uuid` body fields and an `ETag` header (both null/absent when capture is off).
A few save- and import-path internals change **unconditionally** (independent of the flag), because the versioned mappers must behave correctly whether or not capture is enabled:
@@ -681,7 +687,7 @@ A read-only companion to the version-history endpoints: each entity type gains a
| `q` | string | — | Case-insensitive search over the full history, applied before pagination (so `count` reflects matches) |
| `page` / `page_size` | integer | `0` / `25` | Pagination (`page_size` clamped to 200) |
Authorization reuses the resource's `can_read` permission and per-object `raise_for_access`; related-entity rows are visibility-filtered to what the caller may see. The stream is empty unless version capture is on (`ENABLE_VERSIONING_CAPTURE`).
Authorization reuses the resource's `can_read` permission and per-object `raise_for_access`; related-entity rows are visibility-filtered to what the caller may see. The stream reflects captured history; with capture off it remains readable but stops accruing new entries.
### Version-history retention (pruning)
@@ -701,7 +707,7 @@ Purging is **live by default** (`SOFT_DELETE_PURGE_DRY_RUN=False`), so the reten
Deployments that replace the default `CELERY_CONFIG` must ensure workers register `superset.tasks.deletion_retention` and schedule the `deletion_retention.purge_soft_deleted` task themselves. The shipped Docker development config uses `imports` and includes both entries. While `SOFT_DELETE` is statically enabled, a missing beat entry logs a startup warning; when the override explicitly defines `imports`, a missing purge module is also reported.
Operators can immediately erase a specific entity for compliance (GDPR) via `superset deletion-retention force-purge --uuid <uuid>`; this applies legacy hard-delete semantics — a live chart referencing a force-purged dataset is left without a datasource until re-pointed (the chart is not modified), and it purges the named entity even when it was never soft-deleted. Every scheduled evaluation writes a provisional, content-free record to the new `purge_audit_log` table before the cascade starts. Meaningful retained outcomes survive the entity they name. Consecutive scheduled evaluations with the same blocked outcome suppress only the redundant current provisional record; completed outcomes, outcome transitions, and every force-purge attempt remain independent and immutable. The **scheduled** purge fails closed when its provisional record cannot be written, while **force-purge** proceeds even if the audit write fails — the operator is present and deletion outranks audit for a compliance erasure. Operators can monitor `deletion_retention.blocked_audit_suppressed` and `deletion_retention.blocked_audit_dedupe_fallback` to verify suppression and fail-safe fallback behavior without changing the existing blocked-workload gauge.
Operators can immediately erase a specific entity for compliance (GDPR) via `superset deletion-retention force-purge --uuid <uuid>`; this applies legacy hard-delete semantics — a live chart referencing a force-purged dataset is left without a datasource until re-pointed (the chart is not modified), and it purges the named entity even when it was never soft-deleted. Every scheduled evaluation writes a provisional, content-free record to the new `purge_audit_log` table before the cascade starts. Meaningful retained outcomes survive the entity they name. Blocked audit records carry a stable machine-readable `reason` code (`report_schedule`, `user_attribute`, or `cascade_integrity_failure` for an unexpected cascade failure caused by a database integrity constraint) so the audit table alone answers why an entity was not purged; records finalized before the column existed keep a NULL reason. Apply the migration before rolling out the new code: the audit model declares the column, so a worker on the new code with an un-migrated table fails its write-ahead write and the scheduled purge fails closed until the migration lands. During a rolling deploy, workers still on the old code write reason-less blocked rows and suppress on status alone; both effects are self-healing, since a NULL-reason record never matches a reason code and the next all-new-code run re-anchors the entity. Consecutive scheduled evaluations blocked with the same status **and reason** suppress only the redundant current provisional record — a reason change writes one new blocked record carrying the new code; completed outcomes, outcome transitions, and every force-purge attempt remain independent and immutable. Retained transition records are not automatically expired, so entities whose block reason changes repeatedly can accumulate multiple audit rows. The **scheduled** purge fails closed when its provisional record cannot be written, while **force-purge** proceeds even if the audit write fails — the operator is present and deletion outranks audit for a compliance erasure. Operators can monitor `deletion_retention.blocked_audit_suppressed` and `deletion_retention.blocked_audit_dedupe_fallback` to verify suppression and fail-safe fallback behavior without changing the existing blocked-workload gauge.
### Recently Archived view and permanent delete (purge) endpoints
@@ -897,7 +903,7 @@ The migration is transactional (all-or-nothing) and idempotent — it can be saf
### Soft delete and restore for datasets
**The soft-delete behavior in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/dataset/<id>` continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**The soft-delete behavior in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/dataset/<id>` uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If datasets are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live datasets in all lists, lookups, and relationship loads (including charts that reference them). The `POST /<uuid>/restore` endpoint and the `dataset_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
@@ -927,7 +933,7 @@ With the flag enabled: `DELETE /api/v1/dataset/<id>` no longer hard-deletes the
### Soft delete and restore for charts
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/chart/<id>` continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/chart/<id>` uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If charts are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live charts in all lists, lookups, and relationship loads (including dashboards that contained them). The `POST /<uuid>/restore` endpoint and the `chart_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
@@ -951,7 +957,7 @@ With the flag enabled: `DELETE /api/v1/chart/<id>` no longer hard-deletes the ch
### Soft delete and restore for dashboards
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/dashboard/<id>` continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/dashboard/<id>` uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If dashboards are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live dashboards in all lists and lookups (including slug lookups — if a soft-deleted dashboard's slug was reused while the flag was on, both rows become visible with the same slug). The `POST /<uuid>/restore` endpoint and the `dashboard_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
+1
View File
@@ -86,6 +86,7 @@
"Israel",
"Italy",
"Italy (regions)",
"Italy (regions and autonomous provinces)",
"Ivory Coast",
"Japan",
"Jordan",
+12 -12
View File
@@ -15,29 +15,29 @@ description of what changed — "Chart renamed to Q3 Revenue", "Added filter on
'Region'" — rather than a raw diff. You can search the history and filter it
down to changes on the entity itself or on the things it depends on.
## Enabling it
Two switches are involved, and both matter.
## Enabling and disabling it
| Setting | Type | Effect |
| --- | --- | --- |
| `VERSION_HISTORY` | Feature flag | Shows the version history UI |
| `ENABLE_VERSIONING_CAPTURE` | Config value | Records versions as entities are saved |
Both default to on. To turn the feature off:
```python
# superset_config.py
FEATURE_FLAGS = {"VERSION_HISTORY": True}
ENABLE_VERSIONING_CAPTURE = True
FEATURE_FLAGS = {"VERSION_HISTORY": False}
ENABLE_VERSIONING_CAPTURE = False
```
Both default to off. They are separate because capture is the expensive half:
an operator may want to start recording history before exposing the UI, so that
there is something to show when they do.
Restart Superset and its workers for the capture change to take effect. Existing
history remains readable while capture is off, but **Restore** is unavailable
(404).
Turning the UI on without capture gives a panel that reports "No history yet"
and never fills, so enable capture first — or at the same time. History only
accrues from the moment capture is switched on; earlier edits are not
reconstructed.
Disable them together: capture off with the UI left on gives a panel that
stops filling — an empty or stale history misrepresents the entity as
unchanged. History only accrues while capture is on; edits made while it was
off are not reconstructed.
## Viewing history
+6 -6
View File
@@ -58,10 +58,10 @@
"@fontsource/inter": "^5.3.0",
"@mdx-js/react": "^3.1.1",
"@saucelabs/theme-github-codeblock": "^0.3.0",
"@storybook/addon-docs": "^10.5.8",
"@storybook/addon-docs": "^10.5.9",
"@superset-ui/core": "^0.20.4",
"@swc/core": "^1.16.0",
"antd": "^6.6.0",
"@swc/core": "^1.16.1",
"antd": "^6.6.1",
"baseline-browser-mapping": "^2.11.15",
"caniuse-lite": "^1.0.30001809",
"docusaurus-plugin-openapi-docs": "^5.2.0",
@@ -77,8 +77,8 @@
"react-table": "^7.8.0",
"remark-import-partial": "^0.0.2",
"reselect": "^5.2.0",
"storybook": "^10.5.8",
"swagger-ui-react": "^5.32.13",
"storybook": "^10.5.9",
"swagger-ui-react": "^5.32.14",
"swc-loader": "^0.2.7",
"tinycolor2": "^1.4.2",
"unist-util-visit": "^5.1.0"
@@ -94,7 +94,7 @@
"eslint": "^9.39.2",
"eslint-plugin-react": "^7.37.5",
"globals": "^17.11.0",
"oxfmt": "^0.63.0",
"oxfmt": "^0.64.0",
"typescript": "~6.0.3",
"typescript-eslint": "^8.67.0",
"webpack": "^5.109.2"
+18 -12
View File
@@ -93,12 +93,6 @@
"lifecycle": "development",
"description": "Enable semantic layers and show semantic views alongside datasets"
},
{
"name": "SOFT_DELETE",
"default": true,
"lifecycle": "development",
"description": "Temporary rollout / kill-switch gate for soft delete (off = legacy hard delete). An emergency stop, not a clean rollback: flipping ON->OFF resurrects already-soft-deleted rows. Retained through this release as the move-back lever; removed (along with its two gate points \u2014 BaseDAO.delete routing and the do_orm_execute visibility listener) once post-flip confidence is established."
},
{
"name": "TABLE_V2_TIME_COMPARISON_ENABLED",
"default": false,
@@ -110,12 +104,6 @@
"default": false,
"lifecycle": "development",
"description": "Enables the tagging system for organizing assets"
},
{
"name": "VERSION_HISTORY",
"default": true,
"lifecycle": "development",
"description": "Enables the version history panel on Explore and Dashboard pages. History only accrues while ``ENABLE_VERSIONING_CAPTURE`` is also on; with capture off the panel renders but stays empty, so the two ship with matching defaults and should be changed together."
}
],
"testing": [
@@ -132,6 +120,12 @@
"lifecycle": "testing",
"description": "Enables filter functionality in Alerts and Reports"
},
{
"name": "ALERT_REPORTS_RETRY",
"default": false,
"lifecycle": "testing",
"description": "Enables automatic retry functionality for failed report executions"
},
{
"name": "ALERT_REPORT_SLACK_V2",
"default": true,
@@ -233,6 +227,12 @@
"lifecycle": "testing",
"description": "Apply RLS rules to SQL Lab queries. Requires query parsing/manipulation. May break queries or allow RLS bypass. Use with care!"
},
{
"name": "SOFT_DELETE",
"default": true,
"lifecycle": "testing",
"description": "Temporary rollout / kill-switch gate for soft delete (off = legacy hard delete). An emergency stop, not a clean rollback: flipping ON->OFF resurrects already-soft-deleted rows. Retained through this release as the move-back lever; removed (along with its two gate points \u2014 BaseDAO.delete routing and the do_orm_execute visibility listener) once post-flip confidence is established."
},
{
"name": "SSH_TUNNELING",
"default": false,
@@ -245,6 +245,12 @@
"default": false,
"lifecycle": "testing",
"description": "Use analogous colors in charts"
},
{
"name": "VERSION_HISTORY",
"default": true,
"lifecycle": "testing",
"description": "Enables the version history panel on Explore and Dashboard pages. History only accrues while ``ENABLE_VERSIONING_CAPTURE`` is also on; with capture off the panel renders empty or stale history, so the two ship with matching defaults and should be changed together."
}
],
"stable": [
+207 -207
View File
@@ -3175,100 +3175,100 @@
resolved "https://registry.yarnpkg.com/@oxc-resolver/binding-win32-x64-msvc/-/binding-win32-x64-msvc-11.23.0.tgz#8b66dbfa7b796139e719063fc0e44084e80a1c15"
integrity sha512-gUGJpr+Rn6zMxm5juApV0K3U845i8t47o8k+rbO0BHbi4PoJIfSPeQmrE2dgohQm2g5k6iviNFyXCGqvmaYUpw==
"@oxfmt/binding-android-arm-eabi@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-android-arm-eabi/-/binding-android-arm-eabi-0.63.0.tgz#136176dc94fdc41e21415cc770d86f5066282e0f"
integrity sha512-YmRth4ZPGgEXcgmkhvANbC9uD67dxmSobW7DQuyt5tOBOKvPnIpk5SVHBj88E+7wMNRI2FhqaDbOhQFBix+b8A==
"@oxfmt/binding-android-arm-eabi@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-android-arm-eabi/-/binding-android-arm-eabi-0.64.0.tgz#e14e25c032f6d8a6b025eb5ee7bb606c3cbdd10e"
integrity sha512-o6uzh/jTOQeAY5TdkAeXdqv7MBRcPxiRA08zrcBtkKj5cSu/FMu0Hl7Q6Fi1KCKyCWZ6lJVjBzdsJvsKltUsGQ==
"@oxfmt/binding-android-arm64@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-android-arm64/-/binding-android-arm64-0.63.0.tgz#10bc42457179210061c801122a64304619e3bdab"
integrity sha512-icbahX8X2X3sRamOMecvdYeZXWjPDazRDIfvWfy7Ca1nc/ZDT2Y9k5Nt7s46EqFd7NQPdgk+CM3/SgIT5LPCaQ==
"@oxfmt/binding-android-arm64@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-android-arm64/-/binding-android-arm64-0.64.0.tgz#294a15b8402eedde0e0a467748e3efadf61bf523"
integrity sha512-jRGSUeeP7p3Gynw2YaCVtjBIA6ZxY6bEB/ES5i54OhqmRTyuVg7ZgstEtzgq6GOAJd+2QZ5pvf+bFfmW5Mp9cw==
"@oxfmt/binding-darwin-arm64@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-darwin-arm64/-/binding-darwin-arm64-0.63.0.tgz#5f9084d9a760a1836387f8970a7f9d614ec3d909"
integrity sha512-WV+Ze5v5gI2qoj8jpAovt8KBTW8pjEz/AiMXXjeTQS+Bmf/MmZXTS40S8xNPDszX+W8WDv2Bbk6qKrMTtUGu1A==
"@oxfmt/binding-darwin-arm64@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-darwin-arm64/-/binding-darwin-arm64-0.64.0.tgz#d55b1a5d5d97d4ccde8e4be7b63e06e4e56f2d13"
integrity sha512-JINwtU2lW7nOFSqi+H2qplipNUqah9Gc1jgGmB82kTD4UnZrZIVxCJ9qEmFiKfjNq27gYLFhrUb0to86aCwMjw==
"@oxfmt/binding-darwin-x64@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-darwin-x64/-/binding-darwin-x64-0.63.0.tgz#badd4a02218a9a62319817d5c337b30159a54a21"
integrity sha512-CJGSBdDxXOWIpoFXHpverimCvz084KA7L483rqJ44c3jDtzv6d4qOSoR/V9ywSHfV+Ks1lwIj2P49BFhunLNAA==
"@oxfmt/binding-darwin-x64@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-darwin-x64/-/binding-darwin-x64-0.64.0.tgz#1c9673270ed597ba9456d40fa0607d50e81158ea"
integrity sha512-gCmuswrgrOSajV4HCRFkVCGIruPq8bjYuPYgSE2WQB3mD6XrdyZ3JMSRZCkQ8zCxOyGWriBo6QoZ5nmMHQ1BfA==
"@oxfmt/binding-freebsd-x64@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-freebsd-x64/-/binding-freebsd-x64-0.63.0.tgz#a17261e95c8ebef1f76d8aaac746a64fdb6ba51e"
integrity sha512-BDfKY+KhL2078cgswBBFQPAYuxCy93bS/iC5frdSeSbTLcGrR6VC2hsuPTanoJmg84+wSyWl0wWC1eR+uTnkRg==
"@oxfmt/binding-freebsd-x64@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-freebsd-x64/-/binding-freebsd-x64-0.64.0.tgz#9e8f8b3a5a558043c664d43d54e441756af30c56"
integrity sha512-Ab8g7a38pT0MMImjh7anRSTve6buWBIlcXIFBYa5xl4s6UxEgKSc2xOOhbGtLwvXnEi2PsEDGoJh3oUU7xkehQ==
"@oxfmt/binding-linux-arm-gnueabihf@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-0.63.0.tgz#baeee34bb08e0769af878623f442e83bc0aacd7a"
integrity sha512-Ov1cQEXT4mj7cojAokWSS1eoxkoyvbDfAbxNsGIKY2o36kvdAaFzPxRN6NxFRk9fD72B8oCoTTX/NuYTUWlpsg==
"@oxfmt/binding-linux-arm-gnueabihf@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-0.64.0.tgz#cfe552538c9e9402ca64d7b83b1ccf02457ef391"
integrity sha512-BgvS3CoQ+Xy2deoZqEN8JVKabcCZi2RxA3yant8G9OAv9KuPJ9TCjHkqigzdHUVwErZxEP5d2bzLIEyKYyBDLg==
"@oxfmt/binding-linux-arm-musleabihf@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-0.63.0.tgz#e70d5697ec4b6bb5f87a3f019e01b3f956b8e44b"
integrity sha512-0LE7ro3+6L79jcMANycAZfRaC7zxr9YZ2+vEL5uMD9QlEep+rS/r1kSJsnuLl991NXJZD60euh0PC1GHrR20vw==
"@oxfmt/binding-linux-arm-musleabihf@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-0.64.0.tgz#1944e367da59e8b1770c5ba96465d0c7e640053e"
integrity sha512-QXpNxwoMj0YvnceCNZadNSden3bIcnvjn/sDp/rwZhRoZoZYGpHvtPyhGsdJz9uvT9GkaMW7SsLddurU56dt8w==
"@oxfmt/binding-linux-arm64-gnu@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-0.63.0.tgz#638a8ed4f3d256c50aeb6d2c19cfc65792c902e1"
integrity sha512-izPk+2Z4gjuZK32Fqh5qXoMpT/2NXzLh++ob57HiEiVSQZ1iYXu8EKMzb+K5AvWyIEXhdDIt7ADjGGtFhkT9Bw==
"@oxfmt/binding-linux-arm64-gnu@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-0.64.0.tgz#510386113bf6a128cf3106d612471dbd1a13b0f4"
integrity sha512-BBgH3I1ppDsI5pZ4Pdhw0ceYxwVCfbU/bZEBCeZ6caRS9x0ZabErxubP7riGUn11PXZBhe8DYdjkDKP1FlVQ5w==
"@oxfmt/binding-linux-arm64-musl@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm64-musl/-/binding-linux-arm64-musl-0.63.0.tgz#af5a9b787f5233f27a3360ad56235fc1b011f760"
integrity sha512-alPmbOuWXFXiSo+lOtv6X71C7SYMEDW2WVvywOvf9BwKgEhSNGhMTLeFVSjKUMCamcjbbgVdsWF8GN1uy8xshg==
"@oxfmt/binding-linux-arm64-musl@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-arm64-musl/-/binding-linux-arm64-musl-0.64.0.tgz#7235405901cb0368b659eb42b362a817fc3330a3"
integrity sha512-v19HSjC/BGXdt26qEvKZtwAHgGmQ2Agcap2kQP+KIqoRZqivVzYth3ui2dJA1i+6/fjpjga85lIOaJJjQ/bOOw==
"@oxfmt/binding-linux-ppc64-gnu@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-0.63.0.tgz#c1a211206134a5577e355a495989e0d733218d60"
integrity sha512-BdzCPvolJc4AWZ+YMzgUDJcDzbQWrFjYuqBHoNHNqP1aCaluQRJNs4k3vNU5IG7vTpjf9zeD73D7MFM1TecZpg==
"@oxfmt/binding-linux-ppc64-gnu@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-0.64.0.tgz#1f0563c530dfa634682ffa32d16830404b95a8c6"
integrity sha512-PElLnOo4xFTBZrxPhgTIj0eHqZXwEBQoNWtb7facUV170T0B0FRET0iNbb3LUeLWTybkUW+vsdyv4ihOdyXGyw==
"@oxfmt/binding-linux-riscv64-gnu@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-0.63.0.tgz#4863f0311e5c1b88f75ef822959b3ca4fd938937"
integrity sha512-7sIgfLzqtNKSkMGsGVyRpHwpjNezRg2XONvUOheFZs95TSZpM0JAuPpA8KrQFsWc4wPU95roX2O69JgH8igOgw==
"@oxfmt/binding-linux-riscv64-gnu@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-0.64.0.tgz#36f55e955c5b38b587470f181146c9a11cf8bdb1"
integrity sha512-Qzsg15n4F5CH+MorcRW4MkAEMiLzXmeG+DiDSbP/bBTqCmWOH3K9DHryNrve+JHlV0txS+B6Z9P5Xz+cmWeL+g==
"@oxfmt/binding-linux-riscv64-musl@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-0.63.0.tgz#ad05a017d12553e2f544743c4940adb552aa1d1c"
integrity sha512-9Tcg0y0WcVa6Mm9AgcgFMseDS+VkFJZpKZ8We9SpDY4gg5jewSwln+0sO04QLcTS1BtfDl9MwR+NfID8L7PUTg==
"@oxfmt/binding-linux-riscv64-musl@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-0.64.0.tgz#bb9c6c3860c8832fe271623eb6131ea5f5e094cd"
integrity sha512-/GZ358wnQ/Ez4UVnCcZIi56JkY0sOdZ+B108pqXKqZz3jLS59F4KEAB1Qv3fRlObrFEk+3L2vUQ/xoPx+3vjXw==
"@oxfmt/binding-linux-s390x-gnu@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-0.63.0.tgz#2803f539db15bc66db115888fa8f84d6531ed2b9"
integrity sha512-qWKC1pEOpx1qYhXaugPhHUeXwSfqEOk2wJH2LqVXGPV5iQYfdAZdt+d2XDiX4DTSWA2QDMUcFB+wEORh3Xn/sA==
"@oxfmt/binding-linux-s390x-gnu@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-0.64.0.tgz#7d736d923f3c7f88743f26479a49903c6dbaf818"
integrity sha512-/C9We3DXegowfLXtVCYHeNiU9azwCDr5cQkEtCVlc74vyn+lLQSPApJ1CZmxAduqeq/Oi3gQ+IVptyhCaTMtkQ==
"@oxfmt/binding-linux-x64-gnu@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-x64-gnu/-/binding-linux-x64-gnu-0.63.0.tgz#c22a06a60ae2d6b3de522095e0c50a816040a033"
integrity sha512-S9wXYOiGSqYGS4Fx/TFsY+xDd/7dE5s+rUgbA4TsHiVF9e8J3ZcKmP7dsP/7iqLI9Wz7Ic7TzEr3mdthRCTdrA==
"@oxfmt/binding-linux-x64-gnu@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-x64-gnu/-/binding-linux-x64-gnu-0.64.0.tgz#34dfe2bde9ed124324b45aae078618456e850452"
integrity sha512-91KM2CeRWscIEHlj1NsW2WSnzGeq1Ehq+39bfDowTdkn+fcvK/x4Y1RcyqT7glyBjZio0ldkeCG6Usj3v7ASog==
"@oxfmt/binding-linux-x64-musl@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-x64-musl/-/binding-linux-x64-musl-0.63.0.tgz#48d3eeaf8e3757f638cf92de5ee4858befc9c0a3"
integrity sha512-5eGyTJuMZNwBSHCivXt8Yuta6GeTYksOPXRk2MIhajiyFGQx7bjaHIwY+ZusAoFHhT157A9x6sktLjYo9D5oMQ==
"@oxfmt/binding-linux-x64-musl@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-linux-x64-musl/-/binding-linux-x64-musl-0.64.0.tgz#b5edc644409aff9715279650767d34d2fb65d59a"
integrity sha512-gw7uEk9I+7zoT1EYLra1eWArIzNcz8e3jkv+Noo2+o2T7wPvsNSQbfoa4DSfZlvn1i6mJ05RiZ4/omaXPDNhQg==
"@oxfmt/binding-openharmony-arm64@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-openharmony-arm64/-/binding-openharmony-arm64-0.63.0.tgz#02be9e140ae35ba30f52bdce27612fece4a01ab3"
integrity sha512-Rz7hx+Dv3DoW/S6pwVAyjfFXp7/trdQ1zg+vNmsdsdDNlUccugp4XNqambSuEAeP0DaG9k72AtNyfDXCEg0AGw==
"@oxfmt/binding-openharmony-arm64@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-openharmony-arm64/-/binding-openharmony-arm64-0.64.0.tgz#6b1d9c662e08bf5fbc1e9ccdb45ed28c004b90c4"
integrity sha512-HYHFf616FHSPSO07c09mjmXBfQ73wIVM3m0txOiooa5XZkGoxFd6B14PVj0LB0DXIqJ6wAO/dDR/NX/5UUaqnw==
"@oxfmt/binding-win32-arm64-msvc@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-0.63.0.tgz#2226eaf52b6345a2cb926499216b2486cf0dbec2"
integrity sha512-T/IuizKN9mr4Xw6YYnptkXRNdLkyIlUZ7c8zfTOBpoytZyJ1BAsMUvsMDEx0X4YvSMpaivm+DR8112rQfzC25g==
"@oxfmt/binding-win32-arm64-msvc@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-0.64.0.tgz#bc5a005e159a8f9af4168eed2e61fe477f4029db"
integrity sha512-uQjFp081IZSWD6VAofX2iO2z01awAdHmfC+NrieWIPKrT2hZKQDyq/U18M7ifC0sm0Wz8aHY/p6+FDYIzs/CrQ==
"@oxfmt/binding-win32-ia32-msvc@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-0.63.0.tgz#58d263bb5ecd7330c02f9dcd8cda10f66e42e74b"
integrity sha512-XjrO5FJ5Wl9vsAxtCP1G/eaeT6y1K2s9CICUHGE42cEjou32/J6S+B1KnrOAboj6E7uhJnwPbRSvznWcxNdA0g==
"@oxfmt/binding-win32-ia32-msvc@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-0.64.0.tgz#88e90b96f7b39e4b6f75178c94c52d464fa58b53"
integrity sha512-lNM6byTAQ881jugzFu8juJTbNRgsUTlswMA6pJmwi1XDvmIqnnb49lcUAs5gz94fCJLrVN+/X3s3jOKqx23WIQ==
"@oxfmt/binding-win32-x64-msvc@0.63.0":
version "0.63.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-x64-msvc/-/binding-win32-x64-msvc-0.63.0.tgz#02a166c8a8049c55d0096d1ba9d8e73f3a4d26a7"
integrity sha512-sgsHCQy432OTQH4Ikk3tZptp3GqwnhwUDuY0loBH41zyHWfMZY9v8Dy78wsnSofHejvFozZGgJgBB1A0LQRwMQ==
"@oxfmt/binding-win32-x64-msvc@0.64.0":
version "0.64.0"
resolved "https://registry.yarnpkg.com/@oxfmt/binding-win32-x64-msvc/-/binding-win32-x64-msvc-0.64.0.tgz#788c7fe26f89e57269f79e8f8a34e9b1497bc674"
integrity sha512-BtmbtL/QjMtF1a6C3CqoDluH2IfB6fJt62E+B9RFfUPtFk4Iz9PFS6+y/SzzOvSxc7aUk2Kphwg7Dh8lMbwu6g==
"@parcel/watcher-android-arm64@2.5.6":
version "2.5.6"
@@ -3789,7 +3789,7 @@
"@rc-component/util" "^1.3.0"
clsx "^2.1.1"
"@rc-component/select@~1.10.0":
"@rc-component/select@~1.10.0", "@rc-component/select@~1.10.1":
version "1.10.1"
resolved "https://registry.yarnpkg.com/@rc-component/select/-/select-1.10.1.tgz#323b2f458a637e8e752f8341094783741c613c34"
integrity sha512-H+yQsl+qED9NilQ3g6zdpsMwUgwVjrcMTkNHAWRVU/MoNCYgTbDgU+MIMgZDK+rVdd2JUfI/MkysMcZZ0cyQKw==
@@ -3824,7 +3824,7 @@
"@rc-component/util" "^1.3.0"
clsx "^2.1.1"
"@rc-component/table@~1.11.0":
"@rc-component/table@~1.11.1":
version "1.11.1"
resolved "https://registry.yarnpkg.com/@rc-component/table/-/table-1.11.1.tgz#7b5c2a7c26fd37b6a403082029b5a72fcb330a4d"
integrity sha512-OWdS6DMmeWb7bJBGqPxYZpQbzBlBiXZUu2sqo6Ii7Sjs9GeK1IsrXrWk26SL2c6KEseabswdxrRj7WUm9LdECw==
@@ -3866,7 +3866,7 @@
"@rc-component/util" "^1.7.0"
clsx "^2.1.1"
"@rc-component/tree-select@~1.16.0":
"@rc-component/tree-select@~1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@rc-component/tree-select/-/tree-select-1.16.1.tgz#dcaea96e396e98108cb29cc051840d4fbdda38cc"
integrity sha512-a1Oi6EJhqAhdOxxupdJi6fP0RPHMKn5TcfkX2+llaQ4lF4nwfH7b6SCHcnsybaa2s+pk1yZYwVyeOYkDnEBRdg==
@@ -4122,23 +4122,23 @@
resolved "https://registry.yarnpkg.com/@standard-schema/utils/-/utils-0.3.0.tgz#3d5e608f16c2390c10528e98e59aef6bf73cae7b"
integrity sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==
"@storybook/addon-docs@^10.5.8":
version "10.5.8"
resolved "https://registry.yarnpkg.com/@storybook/addon-docs/-/addon-docs-10.5.8.tgz#767c10c7a4cc1b625b93f869b2a2b09fc8514f2e"
integrity sha512-NlHiMKW/UvW/uL8HXFDCEVwoH3qZeGYZ/qlWax4d7H471b/T54MBq2KcB4ZrdA785FfIH3numAJdBb5jwn00Mg==
"@storybook/addon-docs@^10.5.9":
version "10.5.9"
resolved "https://registry.yarnpkg.com/@storybook/addon-docs/-/addon-docs-10.5.9.tgz#6d871977f7ad833dc142d12ee43490105dec4d07"
integrity sha512-8sFsMkZYrrdqCLdV+hnwTwDF7RaBsBPRwl4wfc8ve9Q/7Yhi5REe/Xjvd8x1yn6fBPPw9tnID9dx6Agdnr81fw==
dependencies:
"@mdx-js/react" "^3.0.0"
"@storybook/csf-plugin" "10.5.8"
"@storybook/csf-plugin" "10.5.9"
"@storybook/icons" "^2.0.2"
"@storybook/react-dom-shim" "10.5.8"
"@storybook/react-dom-shim" "10.5.9"
react "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
react-dom "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
ts-dedent "^2.0.0"
"@storybook/csf-plugin@10.5.8":
version "10.5.8"
resolved "https://registry.yarnpkg.com/@storybook/csf-plugin/-/csf-plugin-10.5.8.tgz#c626c5bfe55d0e279b2457e5cf150a788d1fc637"
integrity sha512-/FHiMyOWWEXfwK/lM0WxmkP9GLzbSJJuzGtfeuNWSOVDnvAMbjavitxfHb5wSbWKIQo0XYC1EJ2Y7x91XNYP4w==
"@storybook/csf-plugin@10.5.9":
version "10.5.9"
resolved "https://registry.yarnpkg.com/@storybook/csf-plugin/-/csf-plugin-10.5.9.tgz#805e4c93a1704b220351d62bb0c74ce3b78c5e10"
integrity sha512-4H5QIHQVtQYCuL43GCRLGjNQhZpQg9gL03ja0DV80kO2Dn9LEt6ol87bSnSjn4VDgcAXtgTzXFvRLknfVgAAqg==
dependencies:
unplugin "^2.3.5"
@@ -4152,10 +4152,10 @@
resolved "https://registry.yarnpkg.com/@storybook/icons/-/icons-2.1.0.tgz#edfc2450a39c5e780f28c6cbc49acd7bff59b41a"
integrity sha512-Fxh9vYpX9bQqFeHRiY8h2ApeRGDzRSMLwJwNZ/AIRqnyOKHxRKL+yFe+ctEkVJmuptRE9u1Hrn8ZZNHyfDKKNg==
"@storybook/react-dom-shim@10.5.8":
version "10.5.8"
resolved "https://registry.yarnpkg.com/@storybook/react-dom-shim/-/react-dom-shim-10.5.8.tgz#40cc3e32af424baa2e4109a325dae2ede29999e2"
integrity sha512-N8D13/Xny+V3kfe1KBgsAHS0nKWXLLdgOOXS9poKdYzVwVCN+CGEGBxWX0zMMtdCptqa6/57em9coPlZMoO+bg==
"@storybook/react-dom-shim@10.5.9":
version "10.5.9"
resolved "https://registry.yarnpkg.com/@storybook/react-dom-shim/-/react-dom-shim-10.5.9.tgz#549793845bb8b966acd36002d33d7b54cc5c92d4"
integrity sha512-7qZD6CSa64p1m/zX9tG4ALcEHlEk0Bx+5++4RL3MFlRCgCFfAomXsCHTzF0RyF8cI4vl+hS7Y0ryjQchVs6UQA==
"@superset-ui/core@^0.20.4":
version "0.20.4"
@@ -4855,86 +4855,86 @@
dependencies:
apg-lite "^1.0.4"
"@swc/core-darwin-arm64@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-darwin-arm64/-/core-darwin-arm64-1.16.0.tgz#8c5a2af031c62ebcb6354aa6975bfb7eac895223"
integrity sha512-SJQPl+xG/zB8bNjC/gTg3WOmOvz7EzlQD+VShfCKFYPNr2qvb+vATUY11vYEjnMWCn6wV8H8eAtjQrVflYyX5A==
"@swc/core-darwin-arm64@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-darwin-arm64/-/core-darwin-arm64-1.16.1.tgz#f6f6983e2268888558cdbe043001d82449445def"
integrity sha512-zlJblJ8ncErD43lKdxjbUaUskJQf+LxiPXYcWXD8/8ZMV+7uuAT+CwjciLXpyZBd5Pq/S726bMpeeAwSeL1hhg==
"@swc/core-darwin-x64@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-darwin-x64/-/core-darwin-x64-1.16.0.tgz#0d2496c0429d7e8bc45b50348adf9d105bb56793"
integrity sha512-ql2JVch8V5t1i+HxiiuD4oVDI1dOku4/e3QiCkplONrm3SLitqNAP+nztHN51fSG2IgGuOwpAi3hgA+ukT5yQg==
"@swc/core-darwin-x64@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-darwin-x64/-/core-darwin-x64-1.16.1.tgz#98b61e8c7ffe9f6263a08677353ba5606f6992de"
integrity sha512-IN0BmPWb0YAh/17mmlWB/HDBtTw2MfuW4hulf/tQAgTQBRH17l+z499bNJLK6LizSjqs0P7V+jU38Zj+vJC1DA==
"@swc/core-linux-arm-gnueabihf@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.16.0.tgz#5f02a85842a04cd21f2ab9e8e67dc4b16f7024a4"
integrity sha512-PcdDBaRbe39y37h1rXVkhNy7mEU7f8b34KD761C68R23EsfMsj5oDPVddRzGdSRAvwwSfH0WSNEHgYmc/AJipg==
"@swc/core-linux-arm-gnueabihf@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.16.1.tgz#afc245521cd43a65a87cdd87fe99fb9e4f4eaa58"
integrity sha512-EYgrx2YOCQ2Twz2S793kqNjPkpvYVUPzzR95bIb7by+VQcyaai4lZZ2iz/tZvcFVKSNcN3/JTKwx+aBn2ZL52A==
"@swc/core-linux-arm64-gnu@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.16.0.tgz#6264498c88c51649511c6b4af532d330d3cf0631"
integrity sha512-t21IUztHQ/COucy7Kk9eIlehmq08H/hYq7aRA6fZox3S5ddi6TxWPK6e5S/+aTCf6+Od9qQ+LIpjHMiTy737vA==
"@swc/core-linux-arm64-gnu@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.16.1.tgz#c44ca749af555ef8127795de141094cd28da9714"
integrity sha512-moyKm0YZlHdHohzm1YwgAyesqnE853rO0REMfJLFAova51wF9BNi+3ZW2PeS7Vqvn6HeJuepLpAHbBdZctxpHA==
"@swc/core-linux-arm64-musl@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.16.0.tgz#7a451eba69aa9a80799b9b8b9af46bf6f49803bd"
integrity sha512-d9+iajbMB87b0umgbP+Gy3yBDSDgty4Q6H5pZ8fgTb/dOoKIwwynP4L4kvWCOFg2i49kxmAAUs1uJZh9s0E+RQ==
"@swc/core-linux-arm64-musl@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.16.1.tgz#a1a3d15d5fb074c474c9a60a14488ec16124253f"
integrity sha512-kKGBO9wdapiSzuf5ZzZ2fYtlu1BNSYtIIUxvH1ir/gcelTOREEHGDCLTDFx/2Knf878nU11A40z7LxwasEFxqA==
"@swc/core-linux-ppc64-gnu@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-ppc64-gnu/-/core-linux-ppc64-gnu-1.16.0.tgz#99a7ba46a56190a52c646506e940dffe554c5d10"
integrity sha512-QRpeKGOg+B0qmo3BFU+6rL/gpoKYYJ7OFSMf5DNMafohYZ/iq2qvAH9Gcrf8NxROj3iooKOVewJ+YgahH1nSLw==
"@swc/core-linux-ppc64-gnu@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-ppc64-gnu/-/core-linux-ppc64-gnu-1.16.1.tgz#7eb33976ece5e45e63f9c9c1ab0da9405df76f7f"
integrity sha512-nZ6qahtLxC3PM54cWOQZHxt4lTCF/3J4LIoWWzz6v7A+rLs8Dx54anYQf7mH3eIi8KlNpgKci/ie8ZSqFN8O7A==
"@swc/core-linux-s390x-gnu@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-s390x-gnu/-/core-linux-s390x-gnu-1.16.0.tgz#61473e056d1dd0d4690352a875c14f41bdd9f60a"
integrity sha512-q+Vr/hmHCcRXT/WFzOJC+T6GGEEtq2iaTtmyLfxO7yzu4ckgcqSNkg9m181wfNhuMwfNBoBhOfwQCnLsGZ5F4g==
"@swc/core-linux-s390x-gnu@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-s390x-gnu/-/core-linux-s390x-gnu-1.16.1.tgz#f02f2687d2ee1c8f59430ef638c63714862c9389"
integrity sha512-4ji5PNzhYq193Z4/4xUaSoNJza6iCkDJSzhetrbB6KOYxsr+kxtQr8ePWhMJUiMt6JUWtXaZ1PYT8FhtED+nGA==
"@swc/core-linux-x64-gnu@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.16.0.tgz#008fc149a9135bca92b1e1f63037e2612c4d0fb5"
integrity sha512-DWVBc3QnpsSgKoq8N4rmZeZa5r/XrHdLkITsExN/tvTdqPtAPDPt+Ysy33OfgBlyN8lNe4xwsXWe6DXlRkJeRQ==
"@swc/core-linux-x64-gnu@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.16.1.tgz#af4c571bbe07044ee0bec49ade1e53c1022d4979"
integrity sha512-VJQxqrisHV+B394IgrOu8YsIIXZgffnf5tO+yc9Z/hoUpuZEvuQTjWwlnpZdpyD+0nx6LTD1/3k646JYm43yJA==
"@swc/core-linux-x64-musl@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.16.0.tgz#4300ea0c63864dc3989ca0e956b4a5e4c666196c"
integrity sha512-6XCgDSc1HPf/5dpjvABhKHICiBcsuZyW3hQMkn8sxel0TqprkJGp+H4iaBYIUTPixhrBub2hBPtfjcZLE6yL3w==
"@swc/core-linux-x64-musl@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.16.1.tgz#113eb36a1d3bd21bbf4a48a22fad97dc1c7cc91c"
integrity sha512-r9oV1mwxxsIGcLV1IQ/tw76MW3doatKze1QFWuC+a7QqJUkhY/bKTSVk6NpKKUGm2LDsE33Va8VqSClfA7vSiQ==
"@swc/core-win32-arm64-msvc@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.16.0.tgz#1d4146b7c1aada2992692cdc72bb0b43a885136e"
integrity sha512-T/+9VVCZJ3AKEth9IP3U9AJ2YscQq+7LUqRTvfR4a2q36+Ri22oOwUizpAKOqQ42vb2Y/kOa4TOcJOfHoDIT/w==
"@swc/core-win32-arm64-msvc@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.16.1.tgz#7cc6cfde26ad7e15fe93de98033e7c1892bcf127"
integrity sha512-6huNRessoBLxWEqBm5zJXyCQ27TO7anvkdiuQ5MDO4CJni0nOXEqKtV9RllQ2TdyENKKsUMXVnIfW2hIXx/R5Q==
"@swc/core-win32-ia32-msvc@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.16.0.tgz#c5c2a60905ffa9e4647214bef75778f0c73ba0d4"
integrity sha512-Pr1lsR/PMs8ndL0UWMrW8nLZ7H7sspIxBRDdjL8f+YJ/FJNASgzfunbVVXAqj0csgIJYHPZy+OW9smjFmk1Rcg==
"@swc/core-win32-ia32-msvc@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.16.1.tgz#2330c734f4129c2064b8848fb956501788aab9a3"
integrity sha512-OVKJFUzphrGmsh+BGtcZDesx0YryV7/Yvy5XGgTqnrZfjnyfcr5uaqYQugCckdIlupc5Vs3XtDjRAj12z4ZPlw==
"@swc/core-win32-x64-msvc@1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.16.0.tgz#67dd85a90437e6fa9951cce7842f6cac3ec3f60d"
integrity sha512-ktdeYLgOQdaonvsj5tJijqgpb0wk7gfF80wCFVA0kucI1hhSUIyfcGbjo5+9sdqv38OhMnTdLoA6xbqgOgPQjw==
"@swc/core-win32-x64-msvc@1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.16.1.tgz#04825a3f9e6fbe390825ff02708a5ebdd3a9841b"
integrity sha512-Bt+VIhWYCGk4urklnkkteLUOeLv1VxigwTCeB/xC6rBZxY6IIKdDwCJf6on3E3SUGsIqmQS6QqtuJQc1VxF4Aw==
"@swc/core@^1.15.40", "@swc/core@^1.16.0":
version "1.16.0"
resolved "https://registry.yarnpkg.com/@swc/core/-/core-1.16.0.tgz#79cd13789725d3e3ad0df605dc88d9e255d7ebfd"
integrity sha512-zSdvEHxBg00WhUNtW/u58hhcdR33gjtMQvOBo8F7POWJDyjRCt/miKfhidT3hCc/118RUwNnlEAmxiihFMbK4Q==
"@swc/core@^1.15.40", "@swc/core@^1.16.1":
version "1.16.1"
resolved "https://registry.yarnpkg.com/@swc/core/-/core-1.16.1.tgz#5ea7ff32f3b352c871aa47195efd4932f709a569"
integrity sha512-nUaeu91O5QZKrQdaDCHd402ogUIoNOOjpkZNq0UomWK0G6gDaGmLhvddF1/3BXf5O8aLyo6ZPY/aMDWvaJQ/hg==
dependencies:
"@swc/counter" "^0.1.3"
"@swc/types" "^0.1.28"
optionalDependencies:
"@swc/core-darwin-arm64" "1.16.0"
"@swc/core-darwin-x64" "1.16.0"
"@swc/core-linux-arm-gnueabihf" "1.16.0"
"@swc/core-linux-arm64-gnu" "1.16.0"
"@swc/core-linux-arm64-musl" "1.16.0"
"@swc/core-linux-ppc64-gnu" "1.16.0"
"@swc/core-linux-s390x-gnu" "1.16.0"
"@swc/core-linux-x64-gnu" "1.16.0"
"@swc/core-linux-x64-musl" "1.16.0"
"@swc/core-win32-arm64-msvc" "1.16.0"
"@swc/core-win32-ia32-msvc" "1.16.0"
"@swc/core-win32-x64-msvc" "1.16.0"
"@swc/core-darwin-arm64" "1.16.1"
"@swc/core-darwin-x64" "1.16.1"
"@swc/core-linux-arm-gnueabihf" "1.16.1"
"@swc/core-linux-arm64-gnu" "1.16.1"
"@swc/core-linux-arm64-musl" "1.16.1"
"@swc/core-linux-ppc64-gnu" "1.16.1"
"@swc/core-linux-s390x-gnu" "1.16.1"
"@swc/core-linux-x64-gnu" "1.16.1"
"@swc/core-linux-x64-musl" "1.16.1"
"@swc/core-win32-arm64-msvc" "1.16.1"
"@swc/core-win32-ia32-msvc" "1.16.1"
"@swc/core-win32-x64-msvc" "1.16.1"
"@swc/counter@^0.1.3":
version "0.1.3"
@@ -6181,10 +6181,10 @@ ansis@^3.2.0:
resolved "https://registry.yarnpkg.com/ansis/-/ansis-3.17.0.tgz#fa8d9c2a93fe7d1177e0c17f9eeb562a58a832d7"
integrity sha512-0qWUglt9JEqLFr3w1I1pbrChn1grhaiAR2ocX1PP/flRmxgtwTzPFFFnfIlD6aMOLQZgSuCRlidD70lvx8yhzg==
antd@^6.6.0:
version "6.6.0"
resolved "https://registry.yarnpkg.com/antd/-/antd-6.6.0.tgz#8acb84c54b36594b5c1a9084c8acb6a03b79961b"
integrity sha512-UDwWIbpmrCHB9ZQ+bPh4vQfB6DTI2ulIyoQ0Tc9xxalFblttiNGHl3ySBD9SyV/8+gUjFzfSx1+iU1Fog2i46w==
antd@^6.6.1:
version "6.6.1"
resolved "https://registry.yarnpkg.com/antd/-/antd-6.6.1.tgz#3235d76413b525b1f3287b87bdaf6ba0e7148521"
integrity sha512-QHIHYoUk9N9nJy1T9fyxWKjY0qApdTEDd/6lzqYng8Uryv9FejNmbhKvYF7obGqB+TuLXQsPVF7fOVgyzM1KrQ==
dependencies:
"@ant-design/colors" "^8.0.1"
"@ant-design/cssinjs" "^2.1.2"
@@ -6217,16 +6217,16 @@ antd@^6.6.0:
"@rc-component/rate" "~1.0.1"
"@rc-component/resize-observer" "^1.1.2"
"@rc-component/segmented" "~1.3.0"
"@rc-component/select" "~1.10.0"
"@rc-component/select" "~1.10.1"
"@rc-component/slider" "~1.1.1"
"@rc-component/steps" "~1.2.2"
"@rc-component/switch" "~1.0.3"
"@rc-component/table" "~1.11.0"
"@rc-component/table" "~1.11.1"
"@rc-component/tabs" "~1.12.0"
"@rc-component/tooltip" "~1.5.0"
"@rc-component/tour" "~2.4.0"
"@rc-component/tree" "~1.4.0"
"@rc-component/tree-select" "~1.16.0"
"@rc-component/tree-select" "~1.16.1"
"@rc-component/trigger" "^3.10.1"
"@rc-component/upload" "~1.1.1"
"@rc-component/util" "^1.12.0"
@@ -11849,10 +11849,10 @@ neotraverse@0.6.15:
resolved "https://registry.yarnpkg.com/neotraverse/-/neotraverse-0.6.15.tgz#dc4abb64700c52440f13bc53635b559862420360"
integrity sha512-HZpdkco+JeXq0G+WWpMJ4NsX3pqb5O7eR9uGz3FfoFt+LYzU8iRWp49nJtud6hsDoywM8tIrDo3gjgmOqJA8LA==
neotraverse@=0.6.18:
version "0.6.18"
resolved "https://registry.yarnpkg.com/neotraverse/-/neotraverse-0.6.18.tgz#abcb33dda2e8e713cf6321b29405e822230cdb30"
integrity sha512-Z4SmBUweYa09+o6pG+eASabEpP6QkQ70yHj351pQoEXIs8uHbaU2DWVmzBANKgflPa47A50PtB2+NgRpQvr7vA==
neotraverse@=1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/neotraverse/-/neotraverse-1.0.1.tgz#7c89b43f6504ef85928c718f578c68621576d194"
integrity sha512-WmmLty1YWwJl9yZi77v2dVIV6X2kuYV8YYBI/G3LWGKdGHmHUvL1z7FW0iDvEvGAwNEoc5x1tOOOyDnf5jJw/w==
no-case@^3.0.4:
version "3.0.4"
@@ -12262,32 +12262,32 @@ oxc-resolver@^11.19.1:
"@oxc-resolver/binding-win32-arm64-msvc" "11.23.0"
"@oxc-resolver/binding-win32-x64-msvc" "11.23.0"
oxfmt@^0.63.0:
version "0.63.0"
resolved "https://registry.yarnpkg.com/oxfmt/-/oxfmt-0.63.0.tgz#c7338e6c43a68d5cf8dc61c08b617d77cb54e323"
integrity sha512-kgdDwv35wvVf6554U2Ab8Jnd0zTM+TsEQWwaB70RAjK3gICFAFGO+2Hd3Be27GMoXj3XRL9IKSNRVl7KBQL6iw==
oxfmt@^0.64.0:
version "0.64.0"
resolved "https://registry.yarnpkg.com/oxfmt/-/oxfmt-0.64.0.tgz#666a5148cdf7385007cd46e35e8ff8f94ecfd96b"
integrity sha512-XZ4GFBN/PLbXKq+0zrgpQfPKYuJlUuj+nzZJY7UpIbFMNyefNLCdN9EwViycNqnYcv0wrn0jXcQLlqJp8RCKBg==
dependencies:
tinypool "2.1.0"
optionalDependencies:
"@oxfmt/binding-android-arm-eabi" "0.63.0"
"@oxfmt/binding-android-arm64" "0.63.0"
"@oxfmt/binding-darwin-arm64" "0.63.0"
"@oxfmt/binding-darwin-x64" "0.63.0"
"@oxfmt/binding-freebsd-x64" "0.63.0"
"@oxfmt/binding-linux-arm-gnueabihf" "0.63.0"
"@oxfmt/binding-linux-arm-musleabihf" "0.63.0"
"@oxfmt/binding-linux-arm64-gnu" "0.63.0"
"@oxfmt/binding-linux-arm64-musl" "0.63.0"
"@oxfmt/binding-linux-ppc64-gnu" "0.63.0"
"@oxfmt/binding-linux-riscv64-gnu" "0.63.0"
"@oxfmt/binding-linux-riscv64-musl" "0.63.0"
"@oxfmt/binding-linux-s390x-gnu" "0.63.0"
"@oxfmt/binding-linux-x64-gnu" "0.63.0"
"@oxfmt/binding-linux-x64-musl" "0.63.0"
"@oxfmt/binding-openharmony-arm64" "0.63.0"
"@oxfmt/binding-win32-arm64-msvc" "0.63.0"
"@oxfmt/binding-win32-ia32-msvc" "0.63.0"
"@oxfmt/binding-win32-x64-msvc" "0.63.0"
"@oxfmt/binding-android-arm-eabi" "0.64.0"
"@oxfmt/binding-android-arm64" "0.64.0"
"@oxfmt/binding-darwin-arm64" "0.64.0"
"@oxfmt/binding-darwin-x64" "0.64.0"
"@oxfmt/binding-freebsd-x64" "0.64.0"
"@oxfmt/binding-linux-arm-gnueabihf" "0.64.0"
"@oxfmt/binding-linux-arm-musleabihf" "0.64.0"
"@oxfmt/binding-linux-arm64-gnu" "0.64.0"
"@oxfmt/binding-linux-arm64-musl" "0.64.0"
"@oxfmt/binding-linux-ppc64-gnu" "0.64.0"
"@oxfmt/binding-linux-riscv64-gnu" "0.64.0"
"@oxfmt/binding-linux-riscv64-musl" "0.64.0"
"@oxfmt/binding-linux-s390x-gnu" "0.64.0"
"@oxfmt/binding-linux-x64-gnu" "0.64.0"
"@oxfmt/binding-linux-x64-musl" "0.64.0"
"@oxfmt/binding-openharmony-arm64" "0.64.0"
"@oxfmt/binding-win32-arm64-msvc" "0.64.0"
"@oxfmt/binding-win32-ia32-msvc" "0.64.0"
"@oxfmt/binding-win32-x64-msvc" "0.64.0"
p-cancelable@^3.0.0:
version "3.0.0"
@@ -13583,7 +13583,7 @@ react-modal@^3.16.3:
react-lifecycles-compat "^3.0.0"
warning "^4.0.3"
react-redux@^9.2.0:
react-redux@^9.2.0, react-redux@^9.3.0:
version "9.3.0"
resolved "https://registry.yarnpkg.com/react-redux/-/react-redux-9.3.0.tgz#a30113bb6d95c0a715d54dda4308d450fca6ce09"
integrity sha512-KQopgqFo/p/fgmAs5qz6p5RWaNAzq40WAu7fJIXnQpYxFPbJYtsJPWvGeF2rOBaY/kEuV77AVsX8TsQzKm+A/g==
@@ -14783,10 +14783,10 @@ stop-iteration-iterator@^1.1.0:
es-errors "^1.3.0"
internal-slot "^1.1.0"
storybook@^10.5.8:
version "10.5.8"
resolved "https://registry.yarnpkg.com/storybook/-/storybook-10.5.8.tgz#d5f051983e6232c0a73ea02149a72c7bafb43275"
integrity sha512-rR4oFMSiWBSqI0lvsJPtcQUPj8+hzj3TkLu+Mw61Wo6YxPSb5FsLSHai0jZnuaIdKIlmu25KCfwlSQl4e1uvnA==
storybook@^10.5.9:
version "10.5.9"
resolved "https://registry.yarnpkg.com/storybook/-/storybook-10.5.9.tgz#61f476fd73785dcf09e9198ddf404b9b8c06964a"
integrity sha512-UfdMKSjEhIKr8LbqYyIE5r7vT/drL/PxN75YaouJ+UG0FssEy6cf49OdTF3kstAqVMHskc+zEqyRoiQHZXHwgA==
dependencies:
"@storybook/global" "^5.0.0"
"@storybook/icons" "^2.0.2"
@@ -15057,10 +15057,10 @@ svgo@^3.0.2, svgo@^3.2.0:
picocolors "^1.0.0"
sax "^1.5.0"
swagger-client@^3.37.8:
version "3.37.8"
resolved "https://registry.yarnpkg.com/swagger-client/-/swagger-client-3.37.8.tgz#26c24c89cbfda7459f6afb53bdfcb6d8dbe9ac82"
integrity sha512-uoKwfq+8DvWVDhoALDrEtex9f26Yi2VkvEFjsrMHd8Gl+TcApJkVXtNiE35p5JQjMsvwkvr1eLVlOFNF4GL1bQ==
swagger-client@^3.38.0:
version "3.38.0"
resolved "https://registry.yarnpkg.com/swagger-client/-/swagger-client-3.38.0.tgz#542431f02d809b49115272ff8b9e48d545b9f53c"
integrity sha512-n7aykm1BEdQ3fKePJJx63UGjYe8/5fuxFMi3qZP4OJGZvzljKvmhxNwIF/MB71sF/lop9NeWZReKvPib9CY+2g==
dependencies:
"@babel/runtime-corejs3" "^7.22.15"
"@scarf/scarf" "=1.4.0"
@@ -15074,7 +15074,7 @@ swagger-client@^3.37.8:
deepmerge "~4.3.0"
fast-json-patch "^3.0.0-1"
js-yaml "^4.2.0"
neotraverse "=0.6.18"
neotraverse "=1.0.1"
node-abort-controller "^3.1.1"
openapi-path-templating "^2.2.1"
openapi-server-url-templating "^1.3.0"
@@ -15103,10 +15103,10 @@ swagger-client@^3.37.8:
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-2" "^1.12.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.12.0"
swagger-ui-react@^5.32.13:
version "5.32.13"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.32.13.tgz#04c96140b0a2d4ea01ebec4d4cfc655d5ed9a500"
integrity sha512-XIDl+Ny6kE1N8wpSPiOFrjPfAevs4GR4XmV6BT6NLMikkMFIbIVocWbA8pnKYyYXQe8Rccfli5o2zDfySw0FnQ==
swagger-ui-react@^5.32.14:
version "5.32.14"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.32.14.tgz#31b69b0f6910e87dbcc81886208061ca1f72e034"
integrity sha512-6LAVBeC78DplbJ7kutm/YeBYo22nPzGOca4bIZAvQG4w2eSetnYDdazaUfY0qzQUlg/H90HnYZX3rg67EmENOw==
dependencies:
"@babel/runtime-corejs3" "^7.27.1"
"@scarf/scarf" "=1.4.0"
@@ -15129,7 +15129,7 @@ swagger-ui-react@^5.32.13:
react-immutable-proptypes "2.2.0"
react-immutable-pure-component "^2.2.0"
react-inspector "^6.0.1"
react-redux "^9.2.0"
react-redux "^9.3.0"
react-syntax-highlighter "^16.0.0"
redux "^5.0.1"
redux-immutable "^4.0.0"
@@ -15137,7 +15137,7 @@ swagger-ui-react@^5.32.13:
reselect "^5.1.1"
serialize-error "^8.1.0"
sha.js "^2.4.12"
swagger-client "^3.37.8"
swagger-client "^3.38.0"
url-parse "^1.5.10"
xml "=1.0.1"
xml-but-prettier "^1.0.1"
+11 -6
View File
@@ -80,7 +80,7 @@ dependencies = [
# marshmallow 4 compatibility: see superset/marshmallow_compatibility.py for a
# Flask-AppBuilder workaround. Tracking issue:
# https://github.com/apache/superset/issues/33162
"marshmallow>=4.3.1, <5",
"marshmallow>=3.0, <5",
"marshmallow-union>=0.1.15.post1",
"msgpack>=1.2.0, <1.3",
"nh3>=0.3.5, <0.4",
@@ -94,7 +94,7 @@ dependencies = [
"parsedatetime",
"paramiko>=3.4.0, <4.0", # 4.0 removed DSSKey, still referenced by sshtunnel
"pgsanity",
"Pillow>=11.0.0, <13",
"Pillow>=12.3.0, <13", # raise floor to match resolved pin; closes SCA false-positive on 11.x-range CVEs already fixed in 12.3.0
"polyline>=2.0.4, <3.0",
"pydantic>=2.8.0",
"pyparsing>=3.3.2, <4",
@@ -103,7 +103,7 @@ dependencies = [
"pygeohash",
"pyarrow>=25.0.1, <26", # before upgrading pyarrow, check that all db dependencies support this, see e.g. https://github.com/apache/superset/pull/34693
"pyyaml>=6.0.3, <7.0.0",
"PyJWT>=2.4.0, <3.0",
"PyJWT>=2.13.0, <3.0", # raise floor to match resolved pin; closes SCA false-positive on 2.4.x-range CVEs already fixed in 2.13.0
"redis>=5.0.0, <9.0",
"rison>=2.0.1, <3.0",
@@ -185,7 +185,8 @@ excel = ["xlrd>=2.0.2, <2.1"]
# installing this extra is only required to actually run exports.
excel-export = ["boto3"]
fastmcp = [
"fastmcp>=3.4.6,<4.0",
"fastmcp>=3.4.7,<4.0",
"mcp>=1.29.1,<2.0",
# tiktoken backs the response-size-guard token estimator. Without
# it, the middleware falls back to a coarser character-based
# heuristic that under-counts JSON-heavy MCP responses.
@@ -218,8 +219,12 @@ motherduck = ["apache-superset[duckdb]"]
mysql = ["mysqlclient>=2.2.8, <3"]
ocient = [
# Closed-source vendor package with no public changelog; permissive
# unpinned sqlalchemy>=1.4 declared, but SQLAlchemy 2.0 support is
# unverified. Lower confidence than the other bumps in this PR.
# unpinned sqlalchemy>=1.4 declared. Verified compatible with SQLAlchemy
# 2.0 against pyocient>=3.9.0 (discussion #40273): dialect construction,
# error extraction, and GIS-type sanitization all pass under 2.0.52. Note
# pyocient 3.9.0 relocated its geo-type classes from private top-level
# names (pyocient._STPoint) to public ones under pyocient.api
# (pyocient.api.STPoint), which is unrelated to the SQLAlchemy bump.
"sqlalchemy-ocient>=3.0.0, <4",
"pyocient>=3.9.0, <4",
"shapely",
+4 -2
View File
@@ -547,8 +547,10 @@ matplotlib==3.9.0
# via prophet
mccabe==0.7.0
# via pylint
mcp==1.24.0
# via fastmcp-slim
mcp==1.29.1
# via
# apache-superset
# fastmcp-slim
mdurl==0.1.2
# via
# -c requirements/base-constraint.txt
@@ -0,0 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Semantic layer contracts for extension authors."""
+302 -486
View File
File diff suppressed because it is too large Load Diff
+12 -12
View File
@@ -158,12 +158,12 @@
"@visx/xychart": "^4.0.0",
"ag-grid-community": "36.1.0",
"ag-grid-react": "36.1.0",
"antd": "^6.6.0",
"antd": "^6.6.1",
"chrono-node": "^2.10.1",
"classnames": "^2.2.5",
"content-disposition": "^2.0.1",
"d3-scale": "^4.0.2",
"dayjs": "^1.11.22",
"dayjs": "^1.11.23",
"dom-to-image-more": "^3.10.2",
"dom-to-pdf": "^0.3.2",
"echarts": "^6.1.0",
@@ -230,7 +230,7 @@
"use-event-callback": "^0.1.0",
"use-immer": "^0.11.0",
"use-query-params": "^2.2.2",
"uuid": "^14.0.1",
"uuid": "^14.0.2",
"xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz",
"yargs": "^18.1.0"
},
@@ -257,12 +257,12 @@
"@istanbuljs/nyc-config-typescript": "^1.0.1",
"@playwright/test": "^1.62.1",
"@pmmmwh/react-refresh-webpack-plugin": "^0.6.2",
"@storybook/addon-docs": "10.5.8",
"@storybook/addon-links": "10.5.8",
"@storybook/react-webpack5": "10.5.8",
"@storybook/addon-docs": "10.5.9",
"@storybook/addon-links": "10.5.9",
"@storybook/react-webpack5": "10.5.9",
"@storybook/test-runner": "0.24.4",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.16.0",
"@swc/core": "^1.16.1",
"@swc/plugin-emotion": "^15.0.0",
"@swc/plugin-transform-imports": "^13.0.0",
"@testing-library/dom": "^10.4.1",
@@ -295,7 +295,7 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.14",
"baseline-browser-mapping": "^2.11.15",
"cheerio": "1.2.0",
"concurrently": "^10.0.5",
"copy-webpack-plugin": "^14.0.0",
@@ -312,7 +312,7 @@
"eslint-plugin-no-only-tests": "^3.4.0",
"eslint-plugin-react-prefer-function-component": "^5.0.0",
"eslint-plugin-react-you-might-not-need-an-effect": "^1.0.1",
"eslint-plugin-storybook": "10.5.8",
"eslint-plugin-storybook": "10.5.9",
"eslint-plugin-testing-library": "^7.16.2",
"eslint-plugin-theme-colors": "file:eslint-rules/eslint-plugin-theme-colors",
"fetch-mock": "^12.6.0",
@@ -331,8 +331,8 @@
"mini-css-extract-plugin": "^2.10.2",
"minimizer-webpack-plugin": "^5.6.1",
"open-cli": "^9.0.0",
"oxfmt": "^0.63.0",
"oxlint": "^1.78.0",
"oxfmt": "^0.64.0",
"oxlint": "^1.79.0",
"po2json": "^0.4.5",
"postcss-styled-syntax": "^0.7.2",
"process": "^0.11.10",
@@ -343,7 +343,7 @@
"source-map": "^0.8.0",
"source-map-support": "^0.5.21",
"speed-measure-webpack-plugin": "^1.6.0",
"storybook": "10.5.8",
"storybook": "10.5.9",
"style-loader": "^4.0.0",
"stylelint": "^17.14.1",
"swc-loader": "^0.2.7",
@@ -17,7 +17,7 @@
* under the License.
*/
import { QueryFormMetric } from '@superset-ui/core';
import { getTotalsMetrics } from './getTotalsMetrics';
import { getTotalsMetrics, toTotalsAggregate } from './getTotalsMetrics';
const simpleMetric = (aggregate: string): QueryFormMetric =>
({
@@ -76,4 +76,31 @@ describe('getTotalsMetrics', () => {
test('returns an empty array when given no metrics', () => {
expect(getTotalsMetrics([], 'AVG')).toEqual([]);
});
test("ORIGINAL keeps each metric's own aggregate", () => {
const metrics = [
simpleMetric('COUNT_DISTINCT'),
sqlMetric(),
savedMetric(),
];
const result = getTotalsMetrics(metrics, 'ORIGINAL');
expect(result).toBe(metrics);
expect(result[0]).toEqual(
expect.objectContaining({ aggregate: 'COUNT_DISTINCT' }),
);
});
});
describe('toTotalsAggregate', () => {
test.each(['SUM', 'AVG'] as const)('passes %s through', value => {
expect(toTotalsAggregate(value)).toBe(value);
});
test.each([undefined, null, '', 'MEDIAN', 'sum'])(
'falls back to ORIGINAL for %p',
value => {
expect(toTotalsAggregate(value)).toBe('ORIGINAL');
},
);
});
@@ -18,26 +18,46 @@
*/
import { isAdhocMetricSimple, QueryFormMetric } from '@superset-ui/core';
export type TotalsAggregate = 'SUM' | 'AVG';
/**
* How the "Show summary" totals row aggregates each metric.
*
* ``ORIGINAL`` keeps every metric's own aggregation. It is the default because
* overriding is not universally valid: ``SUM`` over a ``COUNT_DISTINCT`` of a
* non-numeric column (a uuid, say) is rejected outright by the database, and
* over a numeric id column it silently produces a meaningless number.
*/
export type TotalsAggregate = 'ORIGINAL' | 'SUM' | 'AVG';
/**
* Build the metrics for a chart's "Show summary" totals query, overriding
* each Simple (adhoc) metric's aggregate function with the user-chosen
* totals aggregate. The totals query has no GROUP BY, so the database
* evaluates each metric fresh over all rows -- swapping the aggregate here
* is a correct, independent computation, not a re-aggregation of
* already-aggregated per-row values.
* Build the metrics for a chart's "Show summary" totals query.
*
* Custom-SQL metrics and saved (string) metrics pass through unchanged:
* there is no safe way to rewrite an arbitrary SQL expression's aggregate
* function without parsing it, so the totals row keeps their own native
* aggregate for those.
* With SUM or AVG, each Simple (adhoc) metric is cloned with its aggregate
* replaced. The totals query has no GROUP BY, so the database evaluates each
* metric fresh over all rows -- that swap is an independent computation, not a
* re-aggregation of already-aggregated per-row values.
*
* Custom-SQL and saved (string) metrics always pass through unchanged: there is
* no safe way to rewrite an arbitrary SQL expression's aggregate without
* parsing it, so the totals row keeps their own native aggregate.
*/
export function getTotalsMetrics(
metrics: QueryFormMetric[],
aggregate: TotalsAggregate,
): QueryFormMetric[] {
if (aggregate === 'ORIGINAL') {
return metrics;
}
return metrics.map(metric =>
isAdhocMetricSimple(metric) ? { ...metric, aggregate } : metric,
);
}
/**
* Narrow a raw ``totals_aggregate`` form-data value to a TotalsAggregate.
*
* Anything other than an explicit SUM/AVG including charts saved before the
* control existed keeps each metric's own aggregation.
*/
export function toTotalsAggregate(value: unknown): TotalsAggregate {
return value === 'SUM' || value === 'AVG' ? value : 'ORIGINAL';
}
@@ -67,7 +67,7 @@
"d3-scale": "^4.0.2",
"d3-time": "^3.1.0",
"d3-time-format": "^4.1.0",
"dayjs": "^1.11.22",
"dayjs": "^1.11.23",
"dompurify": "^3.4.13",
"fetch-retry": "^6.0.0",
"handlebars": "^4.7.9",
@@ -0,0 +1,153 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { useState } from 'react';
import { fireEvent, render, screen } from '@superset-ui/core/spec';
import { Input } from '../Input';
import { Modal } from './Modal';
const drag = (
target: Element,
from: [number, number],
to: [number, number],
) => {
fireEvent.mouseDown(target, { clientX: from[0], clientY: from[1] });
fireEvent.mouseMove(document, { clientX: to[0], clientY: to[1] });
fireEvent.mouseUp(document);
};
const isDragged = () => !!document.querySelector('.react-draggable-dragged');
describe('Modal draggable', () => {
test('dragging from the title bar moves the modal', () => {
render(
<Modal show onHide={() => {}} title="Edit Dataset" draggable name="test">
<Input data-test="field" defaultValue="value" />
</Modal>,
);
const trigger = document.querySelector('.draggable-trigger') as HTMLElement;
drag(trigger, [100, 50], [150, 90]);
expect(isDragged()).toBe(true);
});
test('dragging inside modal content does not move the modal', () => {
render(
<Modal show onHide={() => {}} title="Edit Dataset" draggable name="test">
<Input data-test="field" defaultValue="first_view_event" />
</Modal>,
);
const input = screen.getByTestId('field');
drag(input, [200, 400], [260, 430]);
expect(isDragged()).toBe(false);
});
test('dragging inside modal content does not move the modal, even after an unrelated re-render while the title was hovered', () => {
// Regression test: the title bar used to gate dragging with a
// hover-tracked boolean (mouseover/mouseout on `.draggable-trigger`)
// instead of react-draggable's own `handle` prop. Because the title
// element was defined as an inline component recreated on every
// render, any unrelated state change while the cursor was over the
// title (e.g. typing in any field) force-remounted it without a real
// mouseout ever firing, leaving dragging permanently enabled -- so
// selecting text anywhere in the modal dragged the whole modal
// instead.
function Harness() {
const [tick, setTick] = useState(0);
return (
<Modal
show
onHide={() => {}}
title="Edit Dataset"
draggable
name="test"
>
<button
type="button"
data-test="rerender"
onClick={() => setTick(tick + 1)}
>
rerender
</button>
<Input data-test="field" defaultValue="first_view_event" />
</Modal>
);
}
render(<Harness />);
const trigger = document.querySelector('.draggable-trigger') as HTMLElement;
fireEvent.mouseOver(trigger);
fireEvent.click(screen.getByTestId('rerender'));
const input = screen.getByTestId('field');
drag(input, [200, 400], [260, 430]);
expect(isDragged()).toBe(false);
});
test('dragging is disabled entirely when draggable is not set', () => {
render(
<Modal show onHide={() => {}} title="Edit Dataset" name="test">
<Input data-test="field" defaultValue="value" />
</Modal>,
);
expect(document.querySelector('.draggable-trigger')).toBeNull();
});
test('draggableConfig cannot re-enable dragging on a non-draggable modal', () => {
render(
<Modal
show
onHide={() => {}}
title="Edit Dataset"
name="test"
draggableConfig={{ disabled: false }}
>
<Input data-test="field" defaultValue="value" />
</Modal>,
);
expect(document.querySelector('.draggable-trigger')).toBeNull();
});
test('draggableConfig can still opt a draggable modal out of dragging', () => {
render(
<Modal
show
onHide={() => {}}
title="Edit Dataset"
draggable
name="test"
draggableConfig={{ disabled: true }}
>
<Input data-test="field" defaultValue="value" />
</Modal>,
);
const trigger = document.querySelector('.draggable-trigger') as HTMLElement;
drag(trigger, [100, 50], [150, 90]);
expect(isDragged()).toBe(false);
});
});
@@ -269,7 +269,6 @@ const CustomModal = ({
);
const draggableRef = useRef<HTMLDivElement>(null);
const [bounds, setBounds] = useState<DraggableBounds>({});
const [dragDisabled, setDragDisabled] = useState<boolean>(true);
const theme = useTheme();
const handleOnHide = () => {
@@ -339,19 +338,7 @@ const CustomModal = ({
}, [hideFooter, resizableConfig]);
const ModalTitle = () =>
draggable ? (
<div
className="draggable-trigger"
onMouseOver={() => dragDisabled && setDragDisabled(false)}
onMouseOut={() => !dragDisabled && setDragDisabled(true)}
onFocus={() => dragDisabled && setDragDisabled(false)}
onBlur={() => !dragDisabled && setDragDisabled(true)}
>
{title}
</div>
) : (
<>{title}</>
);
draggable ? <div className="draggable-trigger">{title}</div> : <>{title}</>;
return (
<StyledModal
@@ -378,13 +365,19 @@ const CustomModal = ({
modalRender={modal =>
resizable || draggable ? (
<Draggable
disabled={!draggable || dragDisabled}
bounds={bounds ?? false}
onStart={(event, uiData) => onDragStart(event, uiData)}
{...draggableConfig}
// `disabled` and `handle` are applied after the spread so callers
// can't use `draggableConfig` to re-enable dragging on a
// non-draggable modal or move the drag handle off the title bar.
// A caller opting a draggable modal out via
// `draggableConfig.disabled` is still honored.
disabled={!draggable || !!draggableConfig?.disabled}
handle={draggable ? '.draggable-trigger' : undefined}
// Pass nodeRef so react-draggable does not fall back to
// ReactDOM.findDOMNode (deprecated in React 18+ Strict Mode).
nodeRef={draggableRef}
{...draggableConfig}
>
{resizable ? (
<Resizable className="resizable" {...getResizableConfig}>
@@ -64,9 +64,8 @@ export default function createSmartNumberFormatter(
description,
formatFunc: value => `${getSign(value)}${formatValue(value)}`,
id:
id || signed
? NumberFormats.SMART_NUMBER_SIGNED
: NumberFormats.SMART_NUMBER,
id ??
(signed ? NumberFormats.SMART_NUMBER_SIGNED : NumberFormats.SMART_NUMBER),
label: label ?? 'Adaptive formatter',
});
}
@@ -28,6 +28,7 @@ export enum FeatureFlag {
AlertReportSlackV2 = 'ALERT_REPORT_SLACK_V2',
AlertReportWebhook = 'ALERT_REPORT_WEBHOOK',
AlertReportsFilter = 'ALERT_REPORTS_FILTER',
AlertReportsRetry = 'ALERT_REPORTS_RETRY',
AllowFullCsvExport = 'ALLOW_FULL_CSV_EXPORT',
ChartPluginsExperimental = 'CHART_PLUGINS_EXPERIMENTAL',
ConfirmDashboardDiff = 'CONFIRM_DASHBOARD_DIFF',
@@ -24,6 +24,12 @@ describe('createSmartNumberFormatter(options)', () => {
const formatter = createSmartNumberFormatter();
expect(formatter).toBeInstanceOf(NumberFormatter);
});
test('uses the supplied formatter id regardless of signed option', () => {
expect(createSmartNumberFormatter({ id: 'custom' }).id).toBe('custom');
expect(
createSmartNumberFormatter({ id: 'custom-signed', signed: true }).id,
).toBe('custom-signed');
});
describe('using default options', () => {
const formatter = createSmartNumberFormatter();
test('formats 0 correctly', () => {
@@ -38,7 +38,7 @@ import {
getTotalsMetrics,
isTimeComparison,
timeCompareOperator,
TotalsAggregate,
toTotalsAggregate,
} from '@superset-ui/chart-controls';
import { isEmpty } from 'lodash-es';
import { TableChartFormData } from './types';
@@ -696,13 +696,16 @@ export const buildQueryUncached: BuildQuery<TableChartFormData> = (
formData.show_totals &&
queryMode === QueryMode.Aggregate,
);
const totalsAggregate: TotalsAggregate =
formData.totals_aggregate === 'AVG' ? 'AVG' : 'SUM';
const totalsAggregate = toTotalsAggregate(formData.totals_aggregate);
// Raw-mode summary columns have no metric of their own to preserve, so
// ORIGINAL has nothing to fall back to; sum them as before.
const rawSummaryAggregate =
totalsAggregate === 'ORIGINAL' ? 'SUM' : totalsAggregate;
const totalsMetrics =
rawSummaryColumns.length > 0
? rawSummaryColumns.map(columnName => ({
expressionType: 'SIMPLE' as const,
aggregate: totalsAggregate,
aggregate: rawSummaryAggregate,
column: { column_name: columnName },
label: columnName,
}))
@@ -503,14 +503,18 @@ const config: ControlPanelConfig = {
label: t('Summary aggregation'),
renderTrigger: true,
description: t(
'Aggregation used for the summary row, independent of each ' +
"metric's own aggregation. Only applies to simple metrics " +
'(a metric built from custom SQL keeps its own aggregation ' +
'in the summary row).',
'Aggregation used for the summary row. By default each metric ' +
'keeps its own aggregation; Sum and Average override it for ' +
'the summary row only. The override applies to simple ' +
'metrics (a metric built from custom SQL always keeps its ' +
'own aggregation). Overriding a count or a distinct count ' +
'sums the counted column instead, which fails outright on a ' +
'non-numeric column.',
),
default: 'SUM',
default: 'ORIGINAL',
clearable: false,
choices: [
['ORIGINAL', t("Each metric's own")],
['SUM', t('Sum')],
['AVG', t('Average')],
],
@@ -1561,7 +1561,7 @@ describe('plugin-chart-ag-grid-table', () => {
expect(queries[1].metrics).toEqual(['count']);
});
test('defaults aggregate-mode totals to SUM for a simple metric', () => {
test("defaults aggregate-mode totals to the metric's own aggregate", () => {
const simpleMetric = {
expressionType: 'SIMPLE' as const,
column: { column_name: 'sales' },
@@ -1580,9 +1580,29 @@ describe('plugin-chart-ag-grid-table', () => {
{ ownState: {} },
);
expect(queries[1].metrics).toEqual([
{ ...simpleMetric, aggregate: 'SUM' },
]);
expect(queries[1].metrics).toEqual([simpleMetric]);
});
test('keeps COUNT_DISTINCT in aggregate-mode totals by default', () => {
const countDistinctMetric = {
expressionType: 'SIMPLE' as const,
column: { column_name: 'contract_id' },
aggregate: 'COUNT_DISTINCT' as const,
label: 'contracts',
};
const { queries } = buildQuery(
{
viz_type: VizType.Table,
datasource: '11__table',
query_mode: QueryMode.Aggregate,
groupby: ['state'],
metrics: [countDistinctMetric],
show_totals: true,
},
{ ownState: {} },
);
expect(queries[1].metrics).toEqual([countDistinctMetric]);
});
test('overrides aggregate-mode totals to AVG for a simple metric when totals_aggregate is set', () => {
@@ -105,7 +105,7 @@
"source": [
"## Download Data\n",
"\n",
"Download datasets (_Admin 0 - Countries_ in [1:10](https://www.naturalearthdata.com/downloads/10m-cultural-vectors/), and _Admin 1 States, Provinces_ in 1:10 and [1:50](https://www.naturalearthdata.com/downloads/50m-cultural-vectors/)) from Natural Earch Data:"
"Download datasets (_Admin 0 - Countries_ in [1:10](https://www.naturalearthdata.com/downloads/10m-cultural-vectors/), and _Admin 1 \u2013 States, Provinces_ in 1:10 and [1:50](https://www.naturalearthdata.com/downloads/50m-cultural-vectors/)) from Natural Earch Data:"
]
},
{
@@ -584,7 +584,7 @@
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>9 rows × 121 columns</p>\n",
"<p>9 rows \u00d7 121 columns</p>\n",
"</div>"
],
"text/plain": [
@@ -926,33 +926,33 @@
" <td>11.0</td>\n",
" <td>11.0</td>\n",
" <td>Q34617</td>\n",
" <td>سان بيير وميكلون</td>\n",
" <td>সাঁ পিয়ের ও মিকলোঁ</td>\n",
" <td>\u0633\u0627\u0646 \u0628\u064a\u064a\u0631 \u0648\u0645\u064a\u0643\u0644\u0648\u0646</td>\n",
" <td>\u09b8\u09be\u0981 \u09aa\u09bf\u09af\u09bc\u09c7\u09b0 \u0993 \u09ae\u09bf\u0995\u09b2\u09cb\u0981</td>\n",
" <td>Saint-Pierre und Miquelon</td>\n",
" <td>Saint Pierre and Miquelon</td>\n",
" <td>San Pedro y Miquelón</td>\n",
" <td>San Pedro y Miquel\u00f3n</td>\n",
" <td>Saint-Pierre-et-Miquelon</td>\n",
" <td>Σαιν-Πιερ και Μικελόν</td>\n",
" <td>सन्त पियर और मिकलान</td>\n",
" <td>Saint-Pierre és Miquelon</td>\n",
" <td>\u03a3\u03b1\u03b9\u03bd-\u03a0\u03b9\u03b5\u03c1 \u03ba\u03b1\u03b9 \u039c\u03b9\u03ba\u03b5\u03bb\u03cc\u03bd</td>\n",
" <td>\u0938\u0928\u094d\u0924 \u092a\u093f\u092f\u0930 \u0914\u0930 \u092e\u093f\u0915\u0932\u093e\u0928</td>\n",
" <td>Saint-Pierre \u00e9s Miquelon</td>\n",
" <td>Saint Pierre dan Miquelon</td>\n",
" <td>Saint-Pierre e Miquelon</td>\n",
" <td>サンピエール島・ミクロン島</td>\n",
" <td>생피에르 미클롱</td>\n",
" <td>\u30b5\u30f3\u30d4\u30a8\u30fc\u30eb\u5cf6\u30fb\u30df\u30af\u30ed\u30f3\u5cf6</td>\n",
" <td>\uc0dd\ud53c\uc5d0\ub974 \ubbf8\ud074\ub871</td>\n",
" <td>Saint-Pierre en Miquelon</td>\n",
" <td>Saint-Pierre i Miquelon</td>\n",
" <td>Saint-Pierre e Miquelon</td>\n",
" <td>Сен-Пьер и Микелон</td>\n",
" <td>\u0421\u0435\u043d-\u041f\u044c\u0435\u0440 \u0438 \u041c\u0438\u043a\u0435\u043b\u043e\u043d</td>\n",
" <td>Saint-Pierre och Miquelon</td>\n",
" <td>Saint Pierre ve Miquelon</td>\n",
" <td>Saint-Pierre và Miquelon</td>\n",
" <td>圣皮埃尔和密克隆</td>\n",
" <td>Saint-Pierre v\u00e0 Miquelon</td>\n",
" <td>\u5723\u76ae\u57c3\u5c14\u548c\u5bc6\u514b\u9686</td>\n",
" <td>1159315673</td>\n",
" <td>סן-פייר ומיקלון</td>\n",
" <td>Сен-П'єр і Мікелон</td>\n",
" <td>سینٹ پیئر و میکیلون</td>\n",
" <td>سن پیر و میکلن</td>\n",
" <td>聖皮埃與密克隆群島</td>\n",
" <td>\u05e1\u05df-\u05e4\u05d9\u05d9\u05e8 \u05d5\u05de\u05d9\u05e7\u05dc\u05d5\u05df</td>\n",
" <td>\u0421\u0435\u043d-\u041f'\u0454\u0440 \u0456 \u041c\u0456\u043a\u0435\u043b\u043e\u043d</td>\n",
" <td>\u0633\u06cc\u0646\u0679 \u067e\u06cc\u0626\u0631 \u0648 \u0645\u06cc\u06a9\u06cc\u0644\u0648\u0646</td>\n",
" <td>\u0633\u0646 \u067e\u06cc\u0631 \u0648 \u0645\u06cc\u06a9\u0644\u0646</td>\n",
" <td>\u8056\u76ae\u57c3\u8207\u5bc6\u514b\u9686\u7fa4\u5cf6</td>\n",
" <td>None</td>\n",
" <td>None</td>\n",
" <td>None</td>\n",
@@ -1051,33 +1051,33 @@
" <td>11.0</td>\n",
" <td>11.0</td>\n",
" <td>None</td>\n",
" <td>ميكلون ولانغليد</td>\n",
" <td>মিকুইলন-ল্যাংলেড</td>\n",
" <td>\u0645\u064a\u0643\u0644\u0648\u0646 \u0648\u0644\u0627\u0646\u063a\u0644\u064a\u062f</td>\n",
" <td>\u09ae\u09bf\u0995\u09c1\u0987\u09b2\u09a8-\u09b2\u09cd\u09af\u09be\u0982\u09b2\u09c7\u09a1</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelón-Langlade</td>\n",
" <td>Miquel\u00f3n-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Μικελόν-Λαγκλέιντ</td>\n",
" <td>मिकेलॉन-लैंगलेड</td>\n",
" <td>\u039c\u03b9\u03ba\u03b5\u03bb\u03cc\u03bd-\u039b\u03b1\u03b3\u03ba\u03bb\u03ad\u03b9\u03bd\u03c4</td>\n",
" <td>\u092e\u093f\u0915\u0947\u0932\u0949\u0928-\u0932\u0948\u0902\u0917\u0932\u0947\u0921</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>ミクロン=ラングラード</td>\n",
" <td>미클롱-랭글레이드</td>\n",
" <td>\u30df\u30af\u30ed\u30f3\uff1d\u30e9\u30f3\u30b0\u30e9\u30fc\u30c9</td>\n",
" <td>\ubbf8\ud074\ub871-\ub7ad\uae00\ub808\uc774\ub4dc</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelão-Langlade</td>\n",
" <td>Микелон-Ланглад</td>\n",
" <td>Miquel\u00e3o-Langlade</td>\n",
" <td>\u041c\u0438\u043a\u0435\u043b\u043e\u043d-\u041b\u0430\u043d\u0433\u043b\u0430\u0434</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>Miquelon-Langlade</td>\n",
" <td>密克隆-朗格拉德</td>\n",
" <td>\u5bc6\u514b\u9686-\u6717\u683c\u62c9\u5fb7</td>\n",
" <td>1159315961</td>\n",
" <td>מירה</td>\n",
" <td>Міквелон-Лангладе</td>\n",
" <td>میکیولون لینگلاڈے</td>\n",
" <td>میکوئلون-لانگلید</td>\n",
" <td>密克隆-朗格拉德</td>\n",
" <td>\u05de\u05d9\u05e8\u05d4</td>\n",
" <td>\u041c\u0456\u043a\u0432\u0435\u043b\u043e\u043d-\u041b\u0430\u043d\u0433\u043b\u0430\u0434\u0435</td>\n",
" <td>\u0645\u06cc\u06a9\u06cc\u0648\u0644\u0648\u0646 \u0644\u06cc\u0646\u06af\u0644\u0627\u0688\u06d2</td>\n",
" <td>\u0645\u06cc\u06a9\u0648\u0626\u0644\u0648\u0646-\u0644\u0627\u0646\u06af\u0644\u06cc\u062f</td>\n",
" <td>\u5bc6\u514b\u9686-\u6717\u683c\u62c9\u5fb7</td>\n",
" <td>None</td>\n",
" <td>None</td>\n",
" <td>None</td>\n",
@@ -1167,48 +1167,48 @@
"2177 PM.97501 None None 1.0 fra SB00 None \n",
"\n",
" min_label max_label min_zoom wikidataid name_ar \\\n",
"2176 11.0 11.0 11.0 Q34617 سان بيير وميكلون \n",
"2177 11.0 11.0 11.0 None ميكلون ولانغليد \n",
"2176 11.0 11.0 11.0 Q34617 \u0633\u0627\u0646 \u0628\u064a\u064a\u0631 \u0648\u0645\u064a\u0643\u0644\u0648\u0646 \n",
"2177 11.0 11.0 11.0 None \u0645\u064a\u0643\u0644\u0648\u0646 \u0648\u0644\u0627\u0646\u063a\u0644\u064a\u062f \n",
"\n",
" name_bn name_de \\\n",
"2176 সাঁ পিয়ের ও মিকলোঁ Saint-Pierre und Miquelon \n",
"2177 মিকুইলন-ল্যাংলেড Miquelon-Langlade \n",
"2176 \u09b8\u09be\u0981 \u09aa\u09bf\u09af\u09bc\u09c7\u09b0 \u0993 \u09ae\u09bf\u0995\u09b2\u09cb\u0981 Saint-Pierre und Miquelon \n",
"2177 \u09ae\u09bf\u0995\u09c1\u0987\u09b2\u09a8-\u09b2\u09cd\u09af\u09be\u0982\u09b2\u09c7\u09a1 Miquelon-Langlade \n",
"\n",
" name_en name_es \\\n",
"2176 Saint Pierre and Miquelon San Pedro y Miquelón \n",
"2177 Miquelon-Langlade Miquelón-Langlade \n",
"2176 Saint Pierre and Miquelon San Pedro y Miquel\u00f3n \n",
"2177 Miquelon-Langlade Miquel\u00f3n-Langlade \n",
"\n",
" name_fr name_el name_hi \\\n",
"2176 Saint-Pierre-et-Miquelon Σαιν-Πιερ και Μικελόν सन्त पियर और मिकलान \n",
"2177 Miquelon-Langlade Μικελόν-Λαγκλέιντ मिकेलॉन-लैंगलेड \n",
"2176 Saint-Pierre-et-Miquelon \u03a3\u03b1\u03b9\u03bd-\u03a0\u03b9\u03b5\u03c1 \u03ba\u03b1\u03b9 \u039c\u03b9\u03ba\u03b5\u03bb\u03cc\u03bd \u0938\u0928\u094d\u0924 \u092a\u093f\u092f\u0930 \u0914\u0930 \u092e\u093f\u0915\u0932\u093e\u0928 \n",
"2177 Miquelon-Langlade \u039c\u03b9\u03ba\u03b5\u03bb\u03cc\u03bd-\u039b\u03b1\u03b3\u03ba\u03bb\u03ad\u03b9\u03bd\u03c4 \u092e\u093f\u0915\u0947\u0932\u0949\u0928-\u0932\u0948\u0902\u0917\u0932\u0947\u0921 \n",
"\n",
" name_hu name_id \\\n",
"2176 Saint-Pierre és Miquelon Saint Pierre dan Miquelon \n",
"2176 Saint-Pierre \u00e9s Miquelon Saint Pierre dan Miquelon \n",
"2177 Miquelon-Langlade Miquelon-Langlade \n",
"\n",
" name_it name_ja name_ko \\\n",
"2176 Saint-Pierre e Miquelon サンピエール島・ミクロン島 생피에르 미클롱 \n",
"2177 Miquelon-Langlade ミクロン=ラングラード 미클롱-랭글레이드 \n",
"2176 Saint-Pierre e Miquelon \u30b5\u30f3\u30d4\u30a8\u30fc\u30eb\u5cf6\u30fb\u30df\u30af\u30ed\u30f3\u5cf6 \uc0dd\ud53c\uc5d0\ub974 \ubbf8\ud074\ub871 \n",
"2177 Miquelon-Langlade \u30df\u30af\u30ed\u30f3\uff1d\u30e9\u30f3\u30b0\u30e9\u30fc\u30c9 \ubbf8\ud074\ub871-\ub7ad\uae00\ub808\uc774\ub4dc \n",
"\n",
" name_nl name_pl \\\n",
"2176 Saint-Pierre en Miquelon Saint-Pierre i Miquelon \n",
"2177 Miquelon-Langlade Miquelon-Langlade \n",
"\n",
" name_pt name_ru name_sv \\\n",
"2176 Saint-Pierre e Miquelon Сен-Пьер и Микелон Saint-Pierre och Miquelon \n",
"2177 Miquelão-Langlade Микелон-Ланглад Miquelon-Langlade \n",
"2176 Saint-Pierre e Miquelon \u0421\u0435\u043d-\u041f\u044c\u0435\u0440 \u0438 \u041c\u0438\u043a\u0435\u043b\u043e\u043d Saint-Pierre och Miquelon \n",
"2177 Miquel\u00e3o-Langlade \u041c\u0438\u043a\u0435\u043b\u043e\u043d-\u041b\u0430\u043d\u0433\u043b\u0430\u0434 Miquelon-Langlade \n",
"\n",
" name_tr name_vi name_zh \\\n",
"2176 Saint Pierre ve Miquelon Saint-Pierre và Miquelon 圣皮埃尔和密克隆 \n",
"2177 Miquelon-Langlade Miquelon-Langlade 密克隆-朗格拉德 \n",
"2176 Saint Pierre ve Miquelon Saint-Pierre v\u00e0 Miquelon \u5723\u76ae\u57c3\u5c14\u548c\u5bc6\u514b\u9686 \n",
"2177 Miquelon-Langlade Miquelon-Langlade \u5bc6\u514b\u9686-\u6717\u683c\u62c9\u5fb7 \n",
"\n",
" ne_id name_he name_uk name_ur \\\n",
"2176 1159315673 סן-פייר ומיקלון Сен-П'єр і Мікелон سینٹ پیئر و میکیلون \n",
"2177 1159315961 מירה Міквелон-Лангладе میکیولون لینگلاڈے \n",
"2176 1159315673 \u05e1\u05df-\u05e4\u05d9\u05d9\u05e8 \u05d5\u05de\u05d9\u05e7\u05dc\u05d5\u05df \u0421\u0435\u043d-\u041f'\u0454\u0440 \u0456 \u041c\u0456\u043a\u0435\u043b\u043e\u043d \u0633\u06cc\u0646\u0679 \u067e\u06cc\u0626\u0631 \u0648 \u0645\u06cc\u06a9\u06cc\u0644\u0648\u0646 \n",
"2177 1159315961 \u05de\u05d9\u05e8\u05d4 \u041c\u0456\u043a\u0432\u0435\u043b\u043e\u043d-\u041b\u0430\u043d\u0433\u043b\u0430\u0434\u0435 \u0645\u06cc\u06a9\u06cc\u0648\u0644\u0648\u0646 \u0644\u06cc\u0646\u06af\u0644\u0627\u0688\u06d2 \n",
"\n",
" name_fa name_zht FCLASS_ISO FCLASS_US FCLASS_FR FCLASS_RU \\\n",
"2176 سن پیر و میکلن 聖皮埃與密克隆群島 None None None None \n",
"2177 میکوئلون-لانگلید 密克隆-朗格拉德 None None None None \n",
"2176 \u0633\u0646 \u067e\u06cc\u0631 \u0648 \u0645\u06cc\u06a9\u0644\u0646 \u8056\u76ae\u57c3\u8207\u5bc6\u514b\u9686\u7fa4\u5cf6 None None None None \n",
"2177 \u0645\u06cc\u06a9\u0648\u0626\u0644\u0648\u0646-\u0644\u0627\u0646\u06af\u0644\u06cc\u062f \u5bc6\u514b\u9686-\u6717\u683c\u62c9\u5fb7 None None None None \n",
"\n",
" FCLASS_ES FCLASS_CN FCLASS_TW FCLASS_IN FCLASS_NP FCLASS_PK FCLASS_DE \\\n",
"2176 None None None None None None None \n",
@@ -1330,7 +1330,7 @@
" 'costa rica',\n",
" 'croatia',\n",
" 'cuba',\n",
" 'curaçao',\n",
" 'cura\u00e7ao',\n",
" 'cyprus',\n",
" 'czech republic',\n",
" 'denmark',\n",
@@ -1343,7 +1343,7 @@
" 'equatorial guinea',\n",
" 'eritrea',\n",
" 'estonia',\n",
" # 'eswatini', # not sure why this doesn't work Swaziland isn't available to alias, either.\n",
" # 'eswatini', # not sure why this doesn't work \u2014 Swaziland isn't available to alias, either.\n",
" 'ethiopia',\n",
" 'falkland islands',\n",
" 'faroe islands',\n",
@@ -1443,7 +1443,7 @@
" 'portugal',\n",
" 'puerto rico',\n",
" 'qatar',\n",
" # 'réunion', # part of France, in Natural Earth data\n",
" # 'r\u00e9union', # part of France, in Natural Earth data\n",
" 'republic of serbia',\n",
" 'romania',\n",
" 'russia',\n",
@@ -1911,34 +1911,34 @@
" <td>9.0</td>\n",
" <td>1159320473</td>\n",
" <td>Q8646</td>\n",
" <td>هونغ كونغ</td>\n",
" <td>হংকং</td>\n",
" <td>\u0647\u0648\u0646\u063a \u0643\u0648\u0646\u063a</td>\n",
" <td>\u09b9\u0982\u0995\u0982</td>\n",
" <td>Hongkong</td>\n",
" <td>Hong Kong</td>\n",
" <td>Hong Kong</td>\n",
" <td>هنگ کنگ</td>\n",
" <td>\u0647\u0646\u06af \u06a9\u0646\u06af</td>\n",
" <td>Hong Kong</td>\n",
" <td>Χονγκ Κονγκ</td>\n",
" <td>הונג קונג</td>\n",
" <td>हांगकांग</td>\n",
" <td>\u03a7\u03bf\u03bd\u03b3\u03ba \u039a\u03bf\u03bd\u03b3\u03ba</td>\n",
" <td>\u05d4\u05d5\u05e0\u05d2 \u05e7\u05d5\u05e0\u05d2</td>\n",
" <td>\u0939\u093e\u0902\u0917\u0915\u093e\u0902\u0917</td>\n",
" <td>Hongkong</td>\n",
" <td>Hong Kong</td>\n",
" <td>Hong Kong</td>\n",
" <td>香港</td>\n",
" <td>홍콩</td>\n",
" <td>\u9999\u6e2f</td>\n",
" <td>\ud64d\ucf69</td>\n",
" <td>Hongkong</td>\n",
" <td>Hongkong</td>\n",
" <td>Hong Kong</td>\n",
" <td>Гонконг</td>\n",
" <td>\u0413\u043e\u043d\u043a\u043e\u043d\u0433</td>\n",
" <td>Hongkong</td>\n",
" <td>Hong Kong</td>\n",
" <td>Гонконг</td>\n",
" <td>ہانگ کانگ</td>\n",
" <td>Hồng Kông</td>\n",
" <td>香港</td>\n",
" <td>香港</td>\n",
" <td>\u0413\u043e\u043d\u043a\u043e\u043d\u0433</td>\n",
" <td>\u06c1\u0627\u0646\u06af \u06a9\u0627\u0646\u06af</td>\n",
" <td>H\u1ed3ng K\u00f4ng</td>\n",
" <td>\u9999\u6e2f</td>\n",
" <td>\u9999\u6e2f</td>\n",
" <td>MULTIPOLYGON (((114.22983 22.55581, 114.23471 ...</td>\n",
" <td>香港特别行政区</td>\n",
" <td>\u9999\u6e2f\u7279\u522b\u884c\u653f\u533a</td>\n",
" <td>CN-91</td>\n",
" </tr>\n",
" <tr>\n",
@@ -1965,34 +1965,34 @@
" <td>8.0</td>\n",
" <td>1159321335</td>\n",
" <td>Q865</td>\n",
" <td>تايوان</td>\n",
" <td>তাইওয়ান</td>\n",
" <td>\u062a\u0627\u064a\u0648\u0627\u0646</td>\n",
" <td>\u09a4\u09be\u0987\u0993\u09af\u09bc\u09be\u09a8</td>\n",
" <td>Republik China</td>\n",
" <td>Taiwan</td>\n",
" <td>República de China</td>\n",
" <td>تایوان</td>\n",
" <td>Taïwan</td>\n",
" <td>Δημοκρατία της Κίνας</td>\n",
" <td>טאיוואן</td>\n",
" <td>चीनी गणराज्य</td>\n",
" <td>Kínai Köztársaság</td>\n",
" <td>Rep\u00fablica de China</td>\n",
" <td>\u062a\u0627\u06cc\u0648\u0627\u0646</td>\n",
" <td>Ta\u00efwan</td>\n",
" <td>\u0394\u03b7\u03bc\u03bf\u03ba\u03c1\u03b1\u03c4\u03af\u03b1 \u03c4\u03b7\u03c2 \u039a\u03af\u03bd\u03b1\u03c2</td>\n",
" <td>\u05d8\u05d0\u05d9\u05d5\u05d5\u05d0\u05df</td>\n",
" <td>\u091a\u0940\u0928\u0940 \u0917\u0923\u0930\u093e\u091c\u094d\u092f</td>\n",
" <td>K\u00ednai K\u00f6zt\u00e1rsas\u00e1g</td>\n",
" <td>Taiwan</td>\n",
" <td>Taiwan</td>\n",
" <td>中華民国</td>\n",
" <td>중화민국</td>\n",
" <td>\u4e2d\u83ef\u6c11\u56fd</td>\n",
" <td>\uc911\ud654\ubbfc\uad6d</td>\n",
" <td>Taiwan</td>\n",
" <td>Republika Chińska</td>\n",
" <td>Republika Chi\u0144ska</td>\n",
" <td>Taiwan</td>\n",
" <td>Тайвань</td>\n",
" <td>\u0422\u0430\u0439\u0432\u0430\u043d\u044c</td>\n",
" <td>Taiwan</td>\n",
" <td>Çin Cumhuriyeti</td>\n",
" <td>Республіка Китай</td>\n",
" <td>تائیوان</td>\n",
" <td>Đài Loan</td>\n",
" <td>中华民国</td>\n",
" <td>中華民國</td>\n",
" <td>\u00c7in Cumhuriyeti</td>\n",
" <td>\u0420\u0435\u0441\u043f\u0443\u0431\u043b\u0456\u043a\u0430 \u041a\u0438\u0442\u0430\u0439</td>\n",
" <td>\u062a\u0627\u0626\u06cc\u0648\u0627\u0646</td>\n",
" <td>\u0110\u00e0i Loan</td>\n",
" <td>\u4e2d\u534e\u6c11\u56fd</td>\n",
" <td>\u4e2d\u83ef\u6c11\u570b</td>\n",
" <td>MULTIPOLYGON (((121.90577 24.9501, 121.83473 2...</td>\n",
" <td>中国台湾</td>\n",
" <td>\u4e2d\u56fd\u53f0\u6e7e</td>\n",
" <td>CN-71</td>\n",
" </tr>\n",
" <tr>\n",
@@ -2019,34 +2019,34 @@
" <td>9.0</td>\n",
" <td>1159320475</td>\n",
" <td>Q14773</td>\n",
" <td>ماكاو</td>\n",
" <td>মাকাও</td>\n",
" <td>\u0645\u0627\u0643\u0627\u0648</td>\n",
" <td>\u09ae\u09be\u0995\u09be\u0993</td>\n",
" <td>Macau</td>\n",
" <td>Macau</td>\n",
" <td>Macao</td>\n",
" <td>ماکائو</td>\n",
" <td>\u0645\u0627\u06a9\u0627\u0626\u0648</td>\n",
" <td>Macao</td>\n",
" <td>Μακάου</td>\n",
" <td>מקאו</td>\n",
" <td>मकाउ</td>\n",
" <td>Makaó</td>\n",
" <td>\u039c\u03b1\u03ba\u03ac\u03bf\u03c5</td>\n",
" <td>\u05de\u05e7\u05d0\u05d5</td>\n",
" <td>\u092e\u0915\u093e\u0909</td>\n",
" <td>Maka\u00f3</td>\n",
" <td>Makau</td>\n",
" <td>Macao</td>\n",
" <td>マカオ</td>\n",
" <td>마카오</td>\n",
" <td>\u30de\u30ab\u30aa</td>\n",
" <td>\ub9c8\uce74\uc624</td>\n",
" <td>Macau</td>\n",
" <td>Makau</td>\n",
" <td>Macau</td>\n",
" <td>Макао</td>\n",
" <td>\u041c\u0430\u043a\u0430\u043e</td>\n",
" <td>Macao</td>\n",
" <td>Makao</td>\n",
" <td>Аоминь</td>\n",
" <td>مکاؤ</td>\n",
" <td>\u0410\u043e\u043c\u0438\u043d\u044c</td>\n",
" <td>\u0645\u06a9\u0627\u0624</td>\n",
" <td>Ma Cao</td>\n",
" <td>澳门</td>\n",
" <td>澳門</td>\n",
" <td>\u6fb3\u95e8</td>\n",
" <td>\u6fb3\u9580</td>\n",
" <td>MULTIPOLYGON (((113.5586 22.16303, 113.56943 2...</td>\n",
" <td>澳门特别行政区</td>\n",
" <td>\u6fb3\u95e8\u7279\u522b\u884c\u653f\u533a</td>\n",
" <td>CN-92</td>\n",
" </tr>\n",
" </tbody>\n",
@@ -2070,34 +2070,34 @@
"2 4 3 MO 20070017 5 0.0 4.0 \n",
"\n",
" max_label ne_id wikidataid name_ar name_bn name_de \\\n",
"0 9.0 1159320473 Q8646 هونغ كونغ হংকং Hongkong \n",
"1 8.0 1159321335 Q865 تايوان তাইওয়ান Republik China \n",
"2 9.0 1159320475 Q14773 ماكاو মাকাও Macau \n",
"0 9.0 1159320473 Q8646 \u0647\u0648\u0646\u063a \u0643\u0648\u0646\u063a \u09b9\u0982\u0995\u0982 Hongkong \n",
"1 8.0 1159321335 Q865 \u062a\u0627\u064a\u0648\u0627\u0646 \u09a4\u09be\u0987\u0993\u09af\u09bc\u09be\u09a8 Republik China \n",
"2 9.0 1159320475 Q14773 \u0645\u0627\u0643\u0627\u0648 \u09ae\u09be\u0995\u09be\u0993 Macau \n",
"\n",
" name_en name_es name_fa name_fr name_el \\\n",
"0 Hong Kong Hong Kong هنگ کنگ Hong Kong Χονγκ Κονγκ \n",
"1 Taiwan República de China تایوان Taïwan Δημοκρατία της Κίνας \n",
"2 Macau Macao ماکائو Macao Μακάου \n",
"0 Hong Kong Hong Kong \u0647\u0646\u06af \u06a9\u0646\u06af Hong Kong \u03a7\u03bf\u03bd\u03b3\u03ba \u039a\u03bf\u03bd\u03b3\u03ba \n",
"1 Taiwan Rep\u00fablica de China \u062a\u0627\u06cc\u0648\u0627\u0646 Ta\u00efwan \u0394\u03b7\u03bc\u03bf\u03ba\u03c1\u03b1\u03c4\u03af\u03b1 \u03c4\u03b7\u03c2 \u039a\u03af\u03bd\u03b1\u03c2 \n",
"2 Macau Macao \u0645\u0627\u06a9\u0627\u0626\u0648 Macao \u039c\u03b1\u03ba\u03ac\u03bf\u03c5 \n",
"\n",
" name_he name_hi name_hu name_id name_it name_ja \\\n",
"0 הונג קונג हांगकांग Hongkong Hong Kong Hong Kong 香港 \n",
"1 טאיוואן चीनी गणराज्य Kínai Köztársaság Taiwan Taiwan 中華民国 \n",
"2 מקאו मकाउ Makaó Makau Macao マカオ \n",
"0 \u05d4\u05d5\u05e0\u05d2 \u05e7\u05d5\u05e0\u05d2 \u0939\u093e\u0902\u0917\u0915\u093e\u0902\u0917 Hongkong Hong Kong Hong Kong \u9999\u6e2f \n",
"1 \u05d8\u05d0\u05d9\u05d5\u05d5\u05d0\u05df \u091a\u0940\u0928\u0940 \u0917\u0923\u0930\u093e\u091c\u094d\u092f K\u00ednai K\u00f6zt\u00e1rsas\u00e1g Taiwan Taiwan \u4e2d\u83ef\u6c11\u56fd \n",
"2 \u05de\u05e7\u05d0\u05d5 \u092e\u0915\u093e\u0909 Maka\u00f3 Makau Macao \u30de\u30ab\u30aa \n",
"\n",
" name_ko name_nl name_pl name_pt name_ru name_sv \\\n",
"0 홍콩 Hongkong Hongkong Hong Kong Гонконг Hongkong \n",
"1 중화민국 Taiwan Republika Chińska Taiwan Тайвань Taiwan \n",
"2 마카오 Macau Makau Macau Макао Macao \n",
"0 \ud64d\ucf69 Hongkong Hongkong Hong Kong \u0413\u043e\u043d\u043a\u043e\u043d\u0433 Hongkong \n",
"1 \uc911\ud654\ubbfc\uad6d Taiwan Republika Chi\u0144ska Taiwan \u0422\u0430\u0439\u0432\u0430\u043d\u044c Taiwan \n",
"2 \ub9c8\uce74\uc624 Macau Makau Macau \u041c\u0430\u043a\u0430\u043e Macao \n",
"\n",
" name_tr name_uk name_ur name_vi name_zh_x name_zht \\\n",
"0 Hong Kong Гонконг ہانگ کانگ Hồng Kông 香港 香港 \n",
"1 Çin Cumhuriyeti Республіка Китай تائیوان Đài Loan 中华民国 中華民國 \n",
"2 Makao Аоминь مکاؤ Ma Cao 澳门 澳門 \n",
"0 Hong Kong \u0413\u043e\u043d\u043a\u043e\u043d\u0433 \u06c1\u0627\u0646\u06af \u06a9\u0627\u0646\u06af H\u1ed3ng K\u00f4ng \u9999\u6e2f \u9999\u6e2f \n",
"1 \u00c7in Cumhuriyeti \u0420\u0435\u0441\u043f\u0443\u0431\u043b\u0456\u043a\u0430 \u041a\u0438\u0442\u0430\u0439 \u062a\u0627\u0626\u06cc\u0648\u0627\u0646 \u0110\u00e0i Loan \u4e2d\u534e\u6c11\u56fd \u4e2d\u83ef\u6c11\u570b \n",
"2 Makao \u0410\u043e\u043c\u0438\u043d\u044c \u0645\u06a9\u0627\u0624 Ma Cao \u6fb3\u95e8 \u6fb3\u9580 \n",
"\n",
" geometry name_zh_y iso_3166_2 \n",
"0 MULTIPOLYGON (((114.22983 22.55581, 114.23471 ... 香港特别行政区 CN-91 \n",
"1 MULTIPOLYGON (((121.90577 24.9501, 121.83473 2... 中国台湾 CN-71 \n",
"2 MULTIPOLYGON (((113.5586 22.16303, 113.56943 2... 澳门特别行政区 CN-92 "
"0 MULTIPOLYGON (((114.22983 22.55581, 114.23471 ... \u9999\u6e2f\u7279\u522b\u884c\u653f\u533a CN-91 \n",
"1 MULTIPOLYGON (((121.90577 24.9501, 121.83473 2... \u4e2d\u56fd\u53f0\u6e7e CN-71 \n",
"2 MULTIPOLYGON (((113.5586 22.16303, 113.56943 2... \u6fb3\u95e8\u7279\u522b\u884c\u653f\u533a CN-92 "
]
},
"execution_count": 14,
@@ -2114,7 +2114,7 @@
"china_sars = china_sars.merge(pd.DataFrame(\n",
" data={\n",
" \"name_en\": [\"Taiwan\", \"Hong Kong\", \"Macau\"],\n",
" \"name_zh\": [\"中国台湾\", \"香港特别行政区\", \"澳门特别行政区\"],\n",
" \"name_zh\": [\"\u4e2d\u56fd\u53f0\u6e7e\", \"\u9999\u6e2f\u7279\u522b\u884c\u653f\u533a\", \"\u6fb3\u95e8\u7279\u522b\u884c\u653f\u533a\"],\n",
" \"iso_3166_2\": [\"CN-71\", \"CN-91\", \"CN-92\"],\n",
" },\n",
"), on=\"name_en\", how=\"left\")\n",
@@ -2252,7 +2252,7 @@
" }\n",
")[[\"geometry\", \"iso_3166_2\", \"name\"]].copy()\n",
"\n",
"# Convert MA01 MA-01\n",
"# Convert MA01 \u2192 MA-01\n",
"morocco_copy[\"iso_3166_2\"] = morocco_copy[\n",
" \"iso_3166_2\"\n",
"].str.replace(\n",
@@ -2290,7 +2290,7 @@
"source": [
"#### Finland\n",
"\n",
"- The Åland Islands (ISO country code AX) is an autonomous region of Finland, and carries the ISO-3166 code FI-01."
"- The \u00c5land Islands (ISO country code AX) is an autonomous region of Finland, and carries the ISO-3166 code FI-01."
]
},
{
@@ -2312,12 +2312,12 @@
"outputs": [],
"source": [
"finland_aland = df_admin0_10m.loc[\n",
" df_admin0_10m.name_en.isin(['Åland']),\n",
" df_admin0_10m.name_en.isin(['\u00c5land']),\n",
" [x for x in df_admin0_10m.columns if x in df.columns]\n",
"]\n",
"finland_aland = finland_aland.merge(pd.DataFrame(\n",
" data={\n",
" \"name_en\": [\"Åland\"],\n",
" \"name_en\": [\"\u00c5land\"],\n",
" \"name_fi\": [\"Ahvenanmaan maakunta\"],\n",
" \"iso_3166_2\": [\"FI-01\"],\n",
" },\n",
@@ -3197,34 +3197,34 @@
"\n",
"# Turkey city name corrections\n",
"# Fix completely wrong spellings\n",
"replace_column('name', turkey, 'Kinkkale', 'Kırıkkale')\n",
"replace_column('name', turkey, 'Kinkkale', 'K\u0131r\u0131kkale')\n",
"replace_column('name', turkey, 'Zinguldak', 'Zonguldak')\n",
"replace_column('name', turkey, 'K. Maras', 'Kahramanmaraş')\n",
"replace_column('name', turkey, 'K. Maras', 'Kahramanmara\u015f')\n",
"\n",
"# Fix missing Turkish characters\n",
"replace_column('name', turkey, 'Adiyaman', 'Adıyaman')\n",
"replace_column('name', turkey, 'Agri', 'Ağrı')\n",
"replace_column('name', turkey, 'Aydin', 'Aydın')\n",
"replace_column('name', turkey, 'Balikesir', 'Balıkesir')\n",
"replace_column('name', turkey, 'Çankiri', 'Çankırı')\n",
"replace_column('name', turkey, 'Diyarbakir', 'Diyarbakır')\n",
"replace_column('name', turkey, 'Elazig', 'Elâzığ')\n",
"replace_column('name', turkey, 'Eskisehir', 'Eskişehir')\n",
"replace_column('name', turkey, 'Gümüshane', 'Gümüşhane')\n",
"replace_column('name', turkey, 'Hakkari', 'Hakkâri')\n",
"replace_column('name', turkey, 'Istanbul', 'İstanbul')\n",
"replace_column('name', turkey, 'Izmir', 'İzmir')\n",
"replace_column('name', turkey, 'Iğdir', 'Iğdır')\n",
"replace_column('name', turkey, 'Kirklareli', 'Kırklareli')\n",
"replace_column('name', turkey, 'Kirsehir', 'Kıehir')\n",
"replace_column('name', turkey, 'Mugla', 'Muğla')\n",
"replace_column('name', turkey, 'Mus', 'Muş')\n",
"replace_column('name', turkey, 'Nevsehir', 'Nevşehir')\n",
"replace_column('name', turkey, 'Nigde', 'Niğde')\n",
"replace_column('name', turkey, 'Sanliurfa', 'Şanlıurfa')\n",
"replace_column('name', turkey, 'Sirnak', 'Şırnak')\n",
"replace_column('name', turkey, 'Tekirdag', 'Tekirdağ')\n",
"replace_column('name', turkey, 'Usak', 'Uşak')\n",
"replace_column('name', turkey, 'Adiyaman', 'Ad\u0131yaman')\n",
"replace_column('name', turkey, 'Agri', 'A\u011fr\u0131')\n",
"replace_column('name', turkey, 'Aydin', 'Ayd\u0131n')\n",
"replace_column('name', turkey, 'Balikesir', 'Bal\u0131kesir')\n",
"replace_column('name', turkey, '\u00c7ankiri', '\u00c7ank\u0131r\u0131')\n",
"replace_column('name', turkey, 'Diyarbakir', 'Diyarbak\u0131r')\n",
"replace_column('name', turkey, 'Elazig', 'El\u00e2z\u0131\u011f')\n",
"replace_column('name', turkey, 'Eskisehir', 'Eski\u015fehir')\n",
"replace_column('name', turkey, 'G\u00fcm\u00fcshane', 'G\u00fcm\u00fc\u015fhane')\n",
"replace_column('name', turkey, 'Hakkari', 'Hakk\u00e2ri')\n",
"replace_column('name', turkey, 'Istanbul', '\u0130stanbul')\n",
"replace_column('name', turkey, 'Izmir', '\u0130zmir')\n",
"replace_column('name', turkey, 'I\u011fdir', 'I\u011fd\u0131r')\n",
"replace_column('name', turkey, 'Kirklareli', 'K\u0131rklareli')\n",
"replace_column('name', turkey, 'Kirsehir', 'K\u0131r\u015fehir')\n",
"replace_column('name', turkey, 'Mugla', 'Mu\u011fla')\n",
"replace_column('name', turkey, 'Mus', 'Mu\u015f')\n",
"replace_column('name', turkey, 'Nevsehir', 'Nev\u015fehir')\n",
"replace_column('name', turkey, 'Nigde', 'Ni\u011fde')\n",
"replace_column('name', turkey, 'Sanliurfa', '\u015eanl\u0131urfa')\n",
"replace_column('name', turkey, 'Sirnak', '\u015e\u0131rnak')\n",
"replace_column('name', turkey, 'Tekirdag', 'Tekirda\u011f')\n",
"replace_column('name', turkey, 'Usak', 'U\u015fak')\n",
"turkey_copy = turkey.copy()"
]
},
@@ -3263,18 +3263,18 @@
"\n",
"# Region names corresponding to NUTS-1\n",
"\n",
"region_name_dict = {'TR1':'İstanbul',\n",
" 'TR2':'Batı Marmara',\n",
"region_name_dict = {'TR1':'\u0130stanbul',\n",
" 'TR2':'Bat\u0131 Marmara',\n",
" 'TR3':'Ege',\n",
" 'TR4':'Doğu Marmara',\n",
" 'TR5':'Batı Anadolu',\n",
" 'TR4':'Do\u011fu Marmara',\n",
" 'TR5':'Bat\u0131 Anadolu',\n",
" 'TR6':'Akdeniz',\n",
" 'TR7':'Orta Anadolu',\n",
" 'TR8':'Batı Karadeniz',\n",
" 'TR9':'Doğu Karadeniz',\n",
" 'TRA':'Kuzeydoğu Anadolu',\n",
" 'TRC':'Güneydoğu Anadolu',\n",
" 'TRB':'Ortadoğu Anadolu'\n",
" 'TR8':'Bat\u0131 Karadeniz',\n",
" 'TR9':'Do\u011fu Karadeniz',\n",
" 'TRA':'Kuzeydo\u011fu Anadolu',\n",
" 'TRC':'G\u00fcneydo\u011fu Anadolu',\n",
" 'TRB':'Ortado\u011fu Anadolu'\n",
" }\n",
"\n",
"\n",
@@ -3517,8 +3517,8 @@
"france_copy = france.copy()\n",
"reposition(france_copy, france.name=='Guadeloupe', 57.4, 25.4, 1.5, 1.5)\n",
"reposition(france_copy, france.name=='Martinique', 58.4, 27.1, 1.5, 1.5)\n",
"reposition(france_copy, france.name=='Guyane française', 52, 37.7, 0.35, 0.35)\n",
"reposition(france_copy, france.name=='La Réunion', -55, 62.8, 1.5, 1.5)\n",
"reposition(france_copy, france.name=='Guyane fran\u00e7aise', 52, 37.7, 0.35, 0.35)\n",
"reposition(france_copy, france.name=='La R\u00e9union', -55, 62.8, 1.5, 1.5)\n",
"reposition(france_copy, france.name=='Mayotte', -43, 54.3, 1.5, 1.5)\n",
"\n",
"not speed_run and france_copy.plot(figsize=(8, 8), **plot_styles)"
@@ -3669,8 +3669,8 @@
"france_overseas = france.copy()\n",
"reposition(france_overseas, france.name=='Guadeloupe', 53.2, 29, 1.5, 1.5)\n",
"reposition(france_overseas, france.name=='Martinique', 52.8, 27.5, 1.5, 1.5)\n",
"reposition(france_overseas, france.name=='Guyane française', 45, 35.5, 0.3, 0.3)\n",
"reposition(france_overseas, france.name=='La Réunion', -58.2, 60.5, 1.5, 1.5)\n",
"reposition(france_overseas, france.name=='Guyane fran\u00e7aise', 45, 35.5, 0.3, 0.3)\n",
"reposition(france_overseas, france.name=='La R\u00e9union', -58.2, 60.5, 1.5, 1.5)\n",
"reposition(france_overseas, france.name=='Mayotte', -50.5, 52.2, 2, 2)\n",
"\n",
"# Tahiti\n",
@@ -3713,7 +3713,7 @@
"france_overseas = pd.concat([france_overseas, saint_martin_data], ignore_index=True)\n",
"reposition(france_overseas, france_overseas.admin=='Saint Martin', 54.8, 30.3, 5, 5)\n",
"\n",
"# Saint Barthélémy\n",
"# Saint Barth\u00e9l\u00e9my\n",
"saint_barthelemy_data = df[(df['admin'] == 'Saint Barthelemy')]\n",
"france_overseas = pd.concat([france_overseas, saint_barthelemy_data], ignore_index=True)\n",
"reposition(france_overseas, france_overseas.admin=='Saint Barthelemy', 54.5, 30, 8, 8)\n",
@@ -3729,13 +3729,13 @@
"france_overseas = pd.concat([france_overseas, paris_and_littlecrowndpts_copy], ignore_index=True)\n",
"\n",
"# Update metadata properly\n",
"france_overseas.loc[france_overseas['name'] == 'Windward Islands', ['name', 'iso_3166_2']] = ['Polynésie française', 'FR-PF']\n",
"france_overseas.loc[france_overseas['name'] == 'Archipel des Kerguelen', ['name', 'iso_3166_2']] = ['Terres australes et antarctiques françaises', 'FR-TF']\n",
"france_overseas.loc[france_overseas['name'] == 'Windward Islands', ['name', 'iso_3166_2']] = ['Polyn\u00e9sie fran\u00e7aise', 'FR-PF']\n",
"france_overseas.loc[france_overseas['name'] == 'Archipel des Kerguelen', ['name', 'iso_3166_2']] = ['Terres australes et antarctiques fran\u00e7aises', 'FR-TF']\n",
"france_overseas.loc[france_overseas['admin'] == 'Wallis and Futuna', ['name', 'iso_3166_2']] = ['Wallis et Futuna', 'FR-WF']\n",
"france_overseas.loc[france_overseas['admin'] == 'New Caledonia', ['name', 'iso_3166_2']] = ['Nouvelle-Calédonie', 'FR-NC']\n",
"france_overseas.loc[france_overseas['admin'] == 'New Caledonia', ['name', 'iso_3166_2']] = ['Nouvelle-Cal\u00e9donie', 'FR-NC']\n",
"france_overseas.loc[france_overseas['admin'] == 'Saint Pierre and Miquelon', ['name', 'iso_3166_2']] = ['Saint-Pierre-et-Miquelon', 'FR-PM']\n",
"france_overseas.loc[france_overseas['admin'] == 'Saint Martin', ['name', 'iso_3166_2']] = ['Saint-Martin', 'FR-MF']\n",
"france_overseas.loc[france_overseas['admin'] == 'Saint Barthelemy', ['name', 'iso_3166_2']] = ['Saint-Barthélémy', 'FR-BL']\n",
"france_overseas.loc[france_overseas['admin'] == 'Saint Barthelemy', ['name', 'iso_3166_2']] = ['Saint-Barth\u00e9l\u00e9my', 'FR-BL']\n",
"\n",
"# Plot data\n",
"france_overseas = france_overseas.rename(columns={'NAME_1': 'name','ISO': 'iso_3166_2'})\n",
@@ -3821,6 +3821,51 @@
"not speed_run and italy_regions.plot(figsize=(10, 7), **plot_styles)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "65aIalqEt1LR"
},
"source": [
"#### Italy Regions and Autonomous Provinces"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"execution": {
"iopub.execute_input": "2026-07-27T19:54:28.892892Z",
"iopub.status.busy": "2026-07-27T19:54:28.892454Z",
"iopub.status.idle": "2026-07-27T19:54:31.123499Z",
"shell.execute_reply": "2026-07-27T19:54:31.122932Z"
}
},
"outputs": [],
"source": [
"trento_and_bozen = df[(df.admin == 'Italy') & (df.iso_3166_2.isin(['IT-TN', 'IT-BZ']))][['geometry','iso_3166_2','name']]\n",
"\n",
"italy_regions_and_autonomous_provinces = pd.concat([italy_regions, trento_and_bozen])\n",
"\n",
"italy_regions_and_autonomous_provinces = italy_regions_and_autonomous_provinces[italy_regions_and_autonomous_provinces['iso_3166_2'] != 'IT-32']"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"execution": {
"iopub.execute_input": "2026-07-27T19:54:28.892892Z",
"iopub.status.busy": "2026-07-27T19:54:28.892454Z",
"iopub.status.idle": "2026-07-27T19:54:31.123499Z",
"shell.execute_reply": "2026-07-27T19:54:31.122932Z"
}
},
"outputs": [],
"source": [
"not speed_run and italy_regions_and_autonomous_provinces.plot(figsize=(10, 7), **plot_styles)"
]
},
{
"cell_type": "markdown",
"metadata": {
@@ -4331,86 +4376,86 @@
"output_type": "stream",
"text": [
"Kon Tum\n",
"Đắk Nông\n",
"Đắk Lắk\n",
"\u0110\u1eafk N\u00f4ng\n",
"\u0110\u1eafk L\u1eafk\n",
"Gia Lai\n",
"Bình Phước\n",
"Tây Ninh\n",
"B\u00ecnh Ph\u01b0\u1edbc\n",
"T\u00e2y Ninh\n",
"Long An\n",
"Đồng Tháp\n",
"\u0110\u1ed3ng Th\u00e1p\n",
"An Giang\n",
"Kiên Giang\n",
"Điện Biên\n",
"Sơn La\n",
"Thanh Hóa\n",
"Ngh An\n",
"Hà Tĩnh\n",
"Quảng Bình\n",
"Quảng Trị\n",
"Thừa Thiên - Huế\n",
"Qung Nam\n",
"Hà Giang\n",
"Cao Bng\n",
"Lào Cai\n",
"Lai Châu\n",
"Lạng Sơn\n",
"Qung Ninh\n",
"Sóc Trăng\n",
"Tin Giang\n",
"Bà Rịa - Vũng Tàu\n",
"Thành phố Hồ Chí Minh\n",
"Khánh Hòa\n",
"Cà Mau\n",
"Bạc Liêu\n",
"Hu Giang\n",
"Vĩnh Long\n",
"Trà Vinh\n",
"Bến Tre\n",
"Đồng Nai\n",
"Bình Thuận\n",
"Ninh Thun\n",
"Phú Yên\n",
"Bình Định\n",
"Quảng Ngãi\n",
"Đà Nẵng\n",
"Ninh Bình\n",
"Nam Định\n",
"Thái Bình\n",
"Hải Phòng\n",
"Hòa Bình\n",
"Tuyên Quang\n",
"Yên Bái\n",
"Vĩnh Phúc\n",
"Phú Thọ\n",
"Hà Nội\n",
"Bắc Kạn\n",
"Hưng Yên\n",
"Bc Ninh\n",
"Bc Giang\n",
"Thái Nguyên\n",
"Hải Dương\n",
"Hà Nam\n",
"Bình Dương\n",
"Lâm Đồng\n",
"Cần Thơ\n"
"Ki\u00ean Giang\n",
"\u0110i\u1ec7n Bi\u00ean\n",
"S\u01a1n La\n",
"Thanh H\u00f3a\n",
"Ngh\u1ec7 An\n",
"H\u00e0 T\u0129nh\n",
"Qu\u1ea3ng B\u00ecnh\n",
"Qu\u1ea3ng Tr\u1ecb\n",
"Th\u1eeba Thi\u00ean - Hu\u1ebf\n",
"Qu\u1ea3ng Nam\n",
"H\u00e0 Giang\n",
"Cao B\u1eb1ng\n",
"L\u00e0o Cai\n",
"Lai Ch\u00e2u\n",
"L\u1ea1ng S\u01a1n\n",
"Qu\u1ea3ng Ninh\n",
"S\u00f3c Tr\u0103ng\n",
"Ti\u1ec1n Giang\n",
"B\u00e0 R\u1ecba - V\u0169ng T\u00e0u\n",
"Th\u00e0nh ph\u1ed1 H\u1ed3 Ch\u00ed Minh\n",
"Kh\u00e1nh H\u00f2a\n",
"C\u00e0 Mau\n",
"B\u1ea1c Li\u00eau\n",
"H\u1eadu Giang\n",
"V\u0129nh Long\n",
"Tr\u00e0 Vinh\n",
"B\u1ebfn Tre\n",
"\u0110\u1ed3ng Nai\n",
"B\u00ecnh Thu\u1eadn\n",
"Ninh Thu\u1eadn\n",
"Ph\u00fa Y\u00ean\n",
"B\u00ecnh \u0110\u1ecbnh\n",
"Qu\u1ea3ng Ng\u00e3i\n",
"\u0110\u00e0 N\u1eb5ng\n",
"Ninh B\u00ecnh\n",
"Nam \u0110\u1ecbnh\n",
"Th\u00e1i B\u00ecnh\n",
"H\u1ea3i Ph\u00f2ng\n",
"H\u00f2a B\u00ecnh\n",
"Tuy\u00ean Quang\n",
"Y\u00ean B\u00e1i\n",
"V\u0129nh Ph\u00fac\n",
"Ph\u00fa Th\u1ecd\n",
"H\u00e0 N\u1ed9i\n",
"B\u1eafc K\u1ea1n\n",
"H\u01b0ng Y\u00ean\n",
"B\u1eafc Ninh\n",
"B\u1eafc Giang\n",
"Th\u00e1i Nguy\u00ean\n",
"H\u1ea3i D\u01b0\u01a1ng\n",
"H\u00e0 Nam\n",
"B\u00ecnh D\u01b0\u01a1ng\n",
"L\u00e2m \u0110\u1ed3ng\n",
"C\u1ea7n Th\u01a1\n"
]
}
],
"source": [
"vietnam = df[df.admin == 'Vietnam']\n",
"vietnam_copy = vietnam.copy()\n",
"replace_column('name', vietnam_copy, 'Ðong Tháp', 'Đồng Tháp')\n",
"replace_column('name', vietnam_copy, 'Son La', 'Sơn La')\n",
"replace_column('name', vietnam_copy, 'Ha Tinh', 'Hà Tĩnh')\n",
"replace_column('name', vietnam_copy, 'Quàng Nam', 'Qung Nam')\n",
"replace_column('name', vietnam_copy, 'Lai Chau', 'Lai Châu')\n",
"replace_column('name', vietnam_copy, 'Hồ Chí Minh city', 'Thành phố Hồ Chí Minh')\n",
"replace_column('name', vietnam_copy, 'Hau Giang', 'Hu Giang')\n",
"replace_column('name', vietnam_copy, 'Ha Noi', 'Hà Nội')\n",
"replace_column('name', vietnam_copy, 'Can Tho', 'Cần Thơ')\n",
"replace_column('name', vietnam_copy, 'Đông Nam Bộ', 'Đồng Nai')\n",
"replace_column('name', vietnam_copy, 'Đông Bắc', 'Bắc Kạn')\n",
"replace_column('name', vietnam_copy, 'Đồng Bằng Sông Hồng', 'Hưng Yên')\n",
"replace_column('name', vietnam_copy, '\u00d0ong Th\u00e1p', '\u0110\u1ed3ng Th\u00e1p')\n",
"replace_column('name', vietnam_copy, 'Son La', 'S\u01a1n La')\n",
"replace_column('name', vietnam_copy, 'Ha Tinh', 'H\u00e0 T\u0129nh')\n",
"replace_column('name', vietnam_copy, 'Qu\u00e0ng Nam', 'Qu\u1ea3ng Nam')\n",
"replace_column('name', vietnam_copy, 'Lai Chau', 'Lai Ch\u00e2u')\n",
"replace_column('name', vietnam_copy, 'H\u1ed3 Ch\u00ed Minh city', 'Th\u00e0nh ph\u1ed1 H\u1ed3 Ch\u00ed Minh')\n",
"replace_column('name', vietnam_copy, 'Hau Giang', 'H\u1eadu Giang')\n",
"replace_column('name', vietnam_copy, 'Ha Noi', 'H\u00e0 N\u1ed9i')\n",
"replace_column('name', vietnam_copy, 'Can Tho', 'C\u1ea7n Th\u01a1')\n",
"replace_column('name', vietnam_copy, '\u0110\u00f4ng Nam B\u1ed9', '\u0110\u1ed3ng Nai')\n",
"replace_column('name', vietnam_copy, '\u0110\u00f4ng B\u1eafc', 'B\u1eafc K\u1ea1n')\n",
"replace_column('name', vietnam_copy, '\u0110\u1ed3ng B\u1eb1ng S\u00f4ng H\u1ed3ng', 'H\u01b0ng Y\u00ean')\n",
"for i in vietnam_copy['name']:\n",
" print(i)"
]
@@ -4454,6 +4499,7 @@
" \"turkey\": turkey_copy,\n",
" \"turkey_regions\": turkey_regions,\n",
" \"italy_regions\": italy_regions,\n",
" \"italy_regions_and_autonomous_provinces\": italy_regions_and_autonomous_provinces,\n",
" \"philippines_regions\": philippines_regions,\n",
" \"latvia\": latvia_copy,\n",
" \"netherlands\": netherlands_copy,\n",
@@ -4492,7 +4538,7 @@
"aruba has only one subdivision - removing from countries array\n",
"british indian ocean territory has only one subdivision - removing from countries array\n",
"cayman islands has only one subdivision - removing from countries array\n",
"curaçao has only one subdivision - removing from countries array\n",
"cura\u00e7ao has only one subdivision - removing from countries array\n",
"falkland islands has only one subdivision - removing from countries array\n",
"faroe islands has only one subdivision - removing from countries array\n",
"gibraltar has only one subdivision - removing from countries array\n",
@@ -4528,7 +4574,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"curaçao has only one subdivision - removing from countries array\n",
"cura\u00e7ao has only one subdivision - removing from countries array\n",
"falkland islands has only one subdivision - removing from countries array\n",
"faroe islands has only one subdivision - removing from countries array\n"
]
@@ -103,6 +103,7 @@ import iran from './countries/iran.geojson';
import israel from './countries/israel.geojson';
import italy from './countries/italy.geojson';
import italy_regions from './countries/italy_regions.geojson';
import italy_regions_and_autonomous_provinces from './countries/italy_regions_and_autonomous_provinces.geojson';
import ivory_coast from './countries/ivory_coast.geojson';
import japan from './countries/japan.geojson';
import jordan from './countries/jordan.geojson';
@@ -306,6 +307,7 @@ export const countries = {
israel,
italy,
italy_regions,
italy_regions_and_autonomous_provinces,
ivory_coast,
japan,
jordan,
@@ -430,6 +432,9 @@ export const countryOptions = Object.keys(countries).map(x => {
if (x === 'italy_regions') {
return [x, 'Italy (regions)'];
}
if (x === 'italy_regions_and_autonomous_provinces') {
return [x, 'Italy (regions and autonomous provinces)'];
}
if (x === 'france_regions') {
return [x, 'France (regions)'];
}
@@ -33,6 +33,6 @@
{ "type": "Feature", "properties": { "ISO": "IR-25", "NAME_1": "Yazd" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 53.650710076708663, 32.61286754000497 ], [ 54.70904341032508, 32.920083929729572 ], [ 54.814670038291581, 32.970700994954939 ], [ 54.908204380227914, 33.088161526533213 ], [ 54.986339146034709, 33.329955553206219 ], [ 55.040082635105477, 33.385688585459832 ], [ 55.139404737638529, 33.430362861356969 ], [ 55.217229445082808, 33.437261664194409 ], [ 55.274590285313366, 33.470644639738339 ], [ 55.359856397954388, 33.594642238948154 ], [ 55.386418085026548, 33.665464788968791 ], [ 55.375255974983247, 34.344363918859756 ], [ 55.484706658785228, 34.365163682857656 ], [ 55.741125116131172, 34.360331935770205 ], [ 55.850885858295669, 34.407254137268581 ], [ 56.210450474010145, 34.906060898691521 ], [ 56.619211053447316, 34.993678289721288 ], [ 57.016602818165495, 35.148294175835531 ], [ 57.268370395577278, 35.201314195393763 ], [ 57.37957807766611, 35.177336330707078 ], [ 57.671239861630283, 34.993936672139682 ], [ 57.705553013161023, 34.930994777786736 ], [ 57.692013788105783, 34.86686432502853 ], [ 57.595172154271495, 34.788807075386217 ], [ 57.198917271115135, 34.557735908285508 ], [ 57.13080773289056, 34.456295071360444 ], [ 57.004820591397163, 34.142567449728006 ], [ 56.9966557148893, 33.973482164370353 ], [ 57.05887413883039, 33.685256863513416 ], [ 57.105796340328823, 33.623581041032196 ], [ 57.304440545394812, 33.603737291442826 ], [ 57.559928826753946, 33.653243313528094 ], [ 57.602716913155007, 33.607716376009932 ], [ 57.642301060445561, 33.54405101214445 ], [ 57.71702518066752, 33.121699530808655 ], [ 57.782240838144162, 32.996797594033694 ], [ 58.04062300025123, 32.871068834059429 ], [ 58.147696568142067, 32.748595689139734 ], [ 58.152554151852598, 32.671882025734874 ], [ 58.10036095639299, 32.568141588163769 ], [ 58.224384393125206, 32.352547512457704 ], [ 58.222213982789071, 32.297563788138291 ], [ 58.173948195053129, 32.146048489246368 ], [ 58.040726353038735, 31.994533189455126 ], [ 58.003932732809346, 31.907200019265474 ], [ 57.901199985590836, 31.771549384496495 ], [ 57.834744093764868, 31.637268175286067 ], [ 56.761631300743886, 32.03008657533519 ], [ 56.634093865639386, 32.049181016990303 ], [ 56.566397738564774, 31.978926906851257 ], [ 56.358451776328366, 31.879036363537352 ], [ 56.287758416617578, 31.815086777932436 ], [ 55.756834750623227, 31.576264146373262 ], [ 55.712289666834636, 31.495183823874356 ], [ 55.684487746312413, 31.110917873960716 ], [ 55.514989048006157, 31.046787421202509 ], [ 55.32637006962301, 31.024773261276948 ], [ 55.116460401726158, 31.043247586207144 ], [ 54.554427525110157, 30.957774767091792 ], [ 54.466474237296211, 30.873361314273211 ], [ 54.420275506209634, 30.797913722740077 ], [ 54.398054640709063, 30.725075792014195 ], [ 54.400638462195275, 30.675466417141422 ], [ 54.515566848231458, 30.450441393055826 ], [ 54.539441359231319, 30.350550848842602 ], [ 54.603830194407919, 30.297349962131022 ], [ 54.591221144440226, 29.973106187400617 ], [ 54.616749301838809, 29.847971707528245 ], [ 54.430197381004291, 29.793220527205563 ], [ 54.227212355265976, 29.882879137362295 ], [ 54.071769646851692, 29.984268297443975 ], [ 54.043140904029485, 30.044161282317305 ], [ 54.007587518149421, 30.263191840231229 ], [ 53.964075962235825, 30.329983628841376 ], [ 53.80460249241105, 30.499068915098348 ], [ 53.638411086002804, 30.755151476559377 ], [ 53.404936964569231, 31.261502996865772 ], [ 53.276469354377184, 31.395086574985442 ], [ 53.125160760160952, 31.51394236874529 ], [ 52.870706007576189, 31.597451483998782 ], [ 52.827091098875087, 31.744419257542688 ], [ 52.824093866238911, 31.813846544482431 ], [ 52.904915806319366, 32.164600328542292 ], [ 52.883728468693846, 32.505199692911503 ], [ 53.060358513834331, 32.576694037399875 ], [ 53.164641554664001, 32.640333563742956 ], [ 53.261069777348325, 32.672011216944099 ], [ 53.334036900182753, 32.67412995043685 ], [ 53.650710076708663, 32.61286754000497 ] ] ] } },
{ "type": "Feature", "properties": { "ISO": "IR-26", "NAME_1": "Qom" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 51.788911574109818, 34.54197459605075 ], [ 51.451671176683419, 34.469730942728802 ], [ 51.158355747219957, 34.452522691469028 ], [ 51.063064406097453, 34.418235379259329 ], [ 51.013765089587196, 34.357903143914939 ], [ 50.983689405941277, 34.161610216338374 ], [ 51.004670037991843, 34.11838288036563 ], [ 51.032988723350854, 34.105799668819657 ], [ 50.802201776190998, 34.157889513290399 ], [ 50.699262323397477, 34.20488922825524 ], [ 50.441190219652867, 34.224784653788731 ], [ 50.32874230321471, 34.317879747052643 ], [ 50.30445438016551, 34.367463284403016 ], [ 50.30869184805033, 34.408520209140306 ], [ 50.264146763362419, 34.466423651730111 ], [ 50.158313429820907, 34.492106838759582 ], [ 50.060024855162908, 34.577062893038089 ], [ 50.069429966020095, 34.628739325459492 ], [ 50.162757603280738, 34.671992498954637 ], [ 50.173402947587931, 34.692042955018337 ], [ 50.152629022011752, 34.716124172492528 ], [ 50.153972609148582, 34.781494858700739 ], [ 50.21112674290481, 34.819373684547884 ], [ 50.301147088267498, 34.809374295387386 ], [ 50.388170200094635, 34.829915676067571 ], [ 50.447598097873879, 34.862058417262119 ], [ 50.57151818181859, 34.878129787859393 ], [ 50.693681268375826, 34.915956935963777 ], [ 50.723033481609889, 35.107883206244935 ], [ 50.784735141613453, 35.218419093866089 ], [ 51.072159457692806, 35.213251450893722 ], [ 51.31235151570985, 35.153952745222966 ], [ 51.882342564157966, 34.875494290429117 ], [ 51.893194614040169, 34.754002997440352 ], [ 51.866116163929803, 34.66646312077637 ], [ 51.788911574109818, 34.54197459605075 ] ] ] } },
{ "type": "Feature", "properties": { "ISO": "IR-07", "NAME_1": "Tehran" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 50.784735141613453, 35.218419093866089 ], [ 50.8712414886038, 35.4471906603207 ], [ 50.870828078353099, 35.517031358410463 ], [ 50.798687778717976, 35.604777939750079 ], [ 50.825095940725078, 35.654454591106457 ], [ 50.889640414373616, 35.686726828380415 ], [ 50.942082799381637, 35.739169213388493 ], [ 50.970321006209417, 35.799679657490174 ], [ 51.059069657138934, 35.803713687037032 ], [ 51.075205776225573, 35.916666515247471 ], [ 51.135716220327254, 35.97314292980235 ], [ 51.216396812163111, 35.997347107083328 ], [ 51.297077403998969, 35.989279047989669 ], [ 51.353553818553848, 36.017517255716768 ], [ 51.356896600397761, 36.116271267305137 ], [ 51.465003697062969, 36.064698188570503 ], [ 51.618275995141062, 36.054052843363991 ], [ 51.754391717004125, 36.010980536122815 ], [ 51.856504348396925, 35.921554469962814 ], [ 51.950245395908269, 35.799804796354238 ], [ 52.029206984015048, 35.77091767021426 ], [ 52.108168573021203, 35.767171128744565 ], [ 52.177001580758315, 35.789960435925366 ], [ 52.306812778986512, 35.917394518242418 ], [ 52.398486768949226, 35.976202298397311 ], [ 52.625863072322886, 35.931347154447622 ], [ 52.740274692622961, 35.881014309162993 ], [ 52.816342400881069, 35.86825023046373 ], [ 52.901401807947025, 35.889695950507701 ], [ 52.944810011972436, 35.881556912421559 ], [ 53.0347270036483, 35.831094875927761 ], [ 53.066249628117816, 35.718905341008679 ], [ 53.079375441123659, 35.618136298551292 ], [ 53.047232699929111, 35.528374334707735 ], [ 52.888275994941182, 35.410190335415621 ], [ 52.674955682358814, 35.336189683806822 ], [ 52.594133742278359, 35.338721829348913 ], [ 52.21968631437187, 35.414221095926791 ], [ 51.982078077840981, 35.54431651499516 ], [ 51.922443475386046, 35.54684866053725 ], [ 51.870146926239613, 35.569818833971965 ], [ 51.853403762073924, 35.555452785717478 ], [ 51.821054315304366, 35.403420721988709 ], [ 51.822501255228701, 35.315260727700377 ], [ 51.980631137916646, 35.125246487135655 ], [ 51.968022087948953, 35.063725694285324 ], [ 51.916862420364396, 34.998587551174523 ], [ 51.882342564157966, 34.875494290429117 ], [ 51.31235151570985, 35.153952745222966 ], [ 51.072159457692806, 35.213251450893722 ], [ 50.784735141613453, 35.218419093866089 ] ] ] } },
{ "type": "Feature", "properties": { "ISO": "IR-30", "NAME_1": "Alborz" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 50.696471795436992, 35.550543525163562 ], [ 50.689753858853464, 35.639246120911707 ], [ 50.650996534762271, 35.676478989813518 ], [ 50.594772576992796, 35.677047431493747 ], [ 50.581956822349468, 35.650072333271567 ], [ 50.620714146440662, 35.60635407178296 ], [ 50.610378858697345, 35.574831448212763 ], [ 50.527696568441854, 35.62911754054204 ], [ 50.285540805663629, 35.666014513558935 ], [ 50.238618605064573, 35.741022853721688 ], [ 50.2302470229817, 35.771925361466231 ], [ 50.242752720161889, 35.81083771428905 ], [ 50.297736443581982, 35.853961697474347 ], [ 50.500204706282148, 35.934266872717956 ], [ 50.534517856913567, 35.960802721368452 ], [ 50.625881789413029, 36.164769599037754 ], [ 50.47188602092308, 36.239648748890602 ], [ 50.420829706126028, 36.296389472396186 ], [ 50.449665155422565, 36.331684474958536 ], [ 50.563146599735774, 36.339177557897983 ], [ 50.966222772263109, 36.292203681354806 ], [ 51.029888137027854, 36.27179149008515 ], [ 51.086008742009824, 36.222259630477538 ], [ 51.127970005211523, 36.20745433175199 ], [ 51.291991001283634, 36.178102118517927 ], [ 51.356896600397761, 36.116271267305137 ], [ 51.353553818553848, 36.017517255716768 ], [ 51.297077403998969, 35.989279047989669 ], [ 51.216396812163111, 35.997347107083328 ], [ 51.135716220327254, 35.97314292980235 ], [ 51.075205776225573, 35.916666515247471 ], [ 51.059069657138934, 35.803713687037032 ], [ 50.970321006209417, 35.799679657490174 ], [ 50.942082799381637, 35.739169213388493 ], [ 50.889640414373616, 35.686726828380415 ], [ 50.825095940725078, 35.654454591106457 ], [ 50.798687778717976, 35.604777939750079 ], [ 50.75269575410573, 35.601341458441539 ], [ 50.724893832684188, 35.555168564877363 ], [ 50.696471795436992, 35.550543525163562 ] ] ] } }
{ "type": "Feature", "properties": { "ISO": "IR-32", "NAME_1": "Alborz" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 50.696471795436992, 35.550543525163562 ], [ 50.689753858853464, 35.639246120911707 ], [ 50.650996534762271, 35.676478989813518 ], [ 50.594772576992796, 35.677047431493747 ], [ 50.581956822349468, 35.650072333271567 ], [ 50.620714146440662, 35.60635407178296 ], [ 50.610378858697345, 35.574831448212763 ], [ 50.527696568441854, 35.62911754054204 ], [ 50.285540805663629, 35.666014513558935 ], [ 50.238618605064573, 35.741022853721688 ], [ 50.2302470229817, 35.771925361466231 ], [ 50.242752720161889, 35.81083771428905 ], [ 50.297736443581982, 35.853961697474347 ], [ 50.500204706282148, 35.934266872717956 ], [ 50.534517856913567, 35.960802721368452 ], [ 50.625881789413029, 36.164769599037754 ], [ 50.47188602092308, 36.239648748890602 ], [ 50.420829706126028, 36.296389472396186 ], [ 50.449665155422565, 36.331684474958536 ], [ 50.563146599735774, 36.339177557897983 ], [ 50.966222772263109, 36.292203681354806 ], [ 51.029888137027854, 36.27179149008515 ], [ 51.086008742009824, 36.222259630477538 ], [ 51.127970005211523, 36.20745433175199 ], [ 51.291991001283634, 36.178102118517927 ], [ 51.356896600397761, 36.116271267305137 ], [ 51.353553818553848, 36.017517255716768 ], [ 51.297077403998969, 35.989279047989669 ], [ 51.216396812163111, 35.997347107083328 ], [ 51.135716220327254, 35.97314292980235 ], [ 51.075205776225573, 35.916666515247471 ], [ 51.059069657138934, 35.803713687037032 ], [ 50.970321006209417, 35.799679657490174 ], [ 50.942082799381637, 35.739169213388493 ], [ 50.889640414373616, 35.686726828380415 ], [ 50.825095940725078, 35.654454591106457 ], [ 50.798687778717976, 35.604777939750079 ], [ 50.75269575410573, 35.601341458441539 ], [ 50.724893832684188, 35.555168564877363 ], [ 50.696471795436992, 35.550543525163562 ] ] ] } }
]
}
@@ -0,0 +1,58 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import fs from 'fs';
import path from 'path';
import { countryOptions } from '../src/countries';
type ItalyFeature = { properties: { ISO: string; NAME_1: string } };
test('countryOptions includes labeled entries for the Italy region variants', () => {
expect(countryOptions).toContainEqual(['italy_regions', 'Italy (regions)']);
expect(countryOptions).toContainEqual([
'italy_regions_and_autonomous_provinces',
'Italy (regions and autonomous provinces)',
]);
});
test('italy_regions_and_autonomous_provinces geojson has the expected shape', () => {
// jest maps `.geojson` imports to an empty object mock, so the file is
// read from disk directly to verify its actual shape.
const geojsonPath = path.join(
__dirname,
'../src/countries/italy_regions_and_autonomous_provinces.geojson',
);
const geojson = JSON.parse(fs.readFileSync(geojsonPath, 'utf-8'));
const features: ItalyFeature[] = geojson.features;
expect(features).toHaveLength(21);
features.forEach(feature => {
expect(feature.properties).toEqual(
expect.objectContaining({
ISO: expect.any(String),
NAME_1: expect.any(String),
}),
);
});
const isoCodes = features.map(feature => feature.properties.ISO);
expect(new Set(isoCodes).size).toBe(isoCodes.length);
expect(isoCodes).toContain('IT-BZ');
expect(isoCodes).toContain('IT-TN');
expect(isoCodes).not.toContain('IT-32');
});
@@ -0,0 +1,69 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import fs from 'fs';
import path from 'path';
type Feature = {
properties: {
ISO: string;
NAME_1: string;
};
};
// `.geojson` imports are mocked out to an empty object by the Jest module
// mapper (see jest.config.js), so the file is read from disk directly to
// exercise the real, committed data.
function loadIranGeoJson(): { features: Feature[] } {
const filePath = path.join(__dirname, '../src/countries/iran.geojson');
return JSON.parse(fs.readFileSync(filePath, 'utf-8'));
}
test('every Iranian province has its own distinct ISO 3166-2 code', () => {
const { features } = loadIranGeoJson();
// Pin the feature count too, so dropping a province other than
// Tehran/Alborz (which would still leave every remaining ISO code
// distinct) doesn't slip past the checks below.
expect(features.length).toBe(31);
// Sanity check: every province name in this file is unique, so a
// duplicate ISO code below can only mean two different provinces were
// mistakenly assigned the same code (as opposed to one province being
// split across multiple polygon features).
const names = features.map(feature => feature.properties.NAME_1);
expect(new Set(names).size).toBe(names.length);
const isoByName = new Map(
features.map(feature => [
feature.properties.NAME_1,
feature.properties.ISO,
]),
);
const isoCodes = features.map(feature => feature.properties.ISO);
expect(new Set(isoCodes).size).toBe(isoCodes.length);
// Tehran and Alborz were split into separate provinces in 2010, but the
// GeoJSON still assigned both the same ISO code (IR-07), which used to
// make it impossible to distinguish them on the Country Map chart. Alborz
// now uses its pre-2020 ISO 3166-2 code, IR-32.
expect(isoByName.get('Tehran')).toBe('IR-07');
expect(isoByName.get('Alborz')).toBe('IR-32');
});
@@ -175,6 +175,18 @@ const config: ControlPanelConfig = {
label: t('X Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'show_x_axis',
config: {
type: 'CheckboxControl',
label: t('Show X axis'),
renderTrigger: true,
default: true,
description: t('Show or hide the X axis line, ticks, and labels'),
},
},
],
[
{
name: 'x_axis_label',
@@ -183,6 +195,8 @@ const config: ControlPanelConfig = {
label: t('X Axis Label'),
renderTrigger: true,
default: '',
visibility: ({ controls }) =>
controls?.show_x_axis?.value !== false,
},
},
],
@@ -193,6 +207,8 @@ const config: ControlPanelConfig = {
...sharedControls.x_axis_time_format,
default: DEFAULT_TIME_FORMAT,
description: `${D3_TIME_FORMAT_DOCS}.`,
visibility: ({ controls }) =>
controls?.show_x_axis?.value !== false,
},
},
],
@@ -213,6 +229,8 @@ const config: ControlPanelConfig = {
clearable: false,
renderTrigger: true,
description: t('The way the ticks are laid out on the X-axis'),
visibility: ({ controls }) =>
controls?.show_x_axis?.value !== false,
},
},
],
@@ -222,6 +240,20 @@ const config: ControlPanelConfig = {
label: t('Y Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'show_y_axis',
config: {
type: 'CheckboxControl',
label: t('Show Y axis'),
renderTrigger: true,
default: true,
description: t(
'Show or hide the Y axis line, ticks, gridlines, and labels',
),
},
},
],
[
{
name: 'y_axis_label',
@@ -230,6 +262,8 @@ const config: ControlPanelConfig = {
label: t('Y Axis Label'),
renderTrigger: true,
default: '',
visibility: ({ controls }) =>
controls?.show_y_axis?.value !== false,
},
},
],
@@ -239,6 +273,10 @@ const config: ControlPanelConfig = {
},
],
controlOverrides: {
// Note: y_axis_format and currency_format are intentionally NOT gated on
// show_y_axis. They drive `defaultFormatter`, which formats the bar labels
// and tooltips as well as the axis, so they must stay configurable even
// when the Y axis itself is hidden.
groupby: {
label: t('Breakdowns'),
description:
@@ -185,6 +185,8 @@ export default function transformProps(
xTicksLayout,
xAxisTimeFormat,
showLegend,
showXAxis = true,
showYAxis = true,
yAxisLabel,
xAxisLabel,
yAxisFormat,
@@ -433,8 +435,10 @@ export default function transformProps(
grid: {
...defaultGrid,
top: theme.sizeUnit * 7,
bottom: theme.sizeUnit * 7,
left: theme.sizeUnit * 5,
// Reclaim the axis-oriented padding when an axis is hidden so an
// axis-free chart gets a clean, tight layout instead of empty margins.
bottom: theme.sizeUnit * (showXAxis ? 7 : 3),
left: theme.sizeUnit * (showYAxis ? 5 : 2),
right: theme.sizeUnit * 7,
},
legend: {
@@ -443,6 +447,7 @@ export default function transformProps(
data: [legendNames.INCREASE, legendNames.DECREASE, legendNames.TOTAL],
},
xAxis: {
show: showXAxis,
data: xAxisData,
type: 'category',
name: xAxisLabel,
@@ -454,6 +459,7 @@ export default function transformProps(
},
yAxis: {
...defaultYAxis,
show: showYAxis,
type: 'value',
nameTextStyle: {
padding: [0, 0, theme.sizeUnit * 5, 0],
@@ -55,8 +55,10 @@ export type EchartsWaterfallFormData = QueryFormData &
xAxisLabel: string;
xAxisTimeFormat?: string;
xTicksLayout?: WaterfallFormXTicksLayout;
showXAxis: boolean;
yAxisLabel: string;
yAxisFormat: string;
showYAxis: boolean;
increaseLabel?: string;
decreaseLabel?: string;
totalLabel?: string;
@@ -65,6 +67,8 @@ export type EchartsWaterfallFormData = QueryFormData &
export const DEFAULT_FORM_DATA: Partial<EchartsWaterfallFormData> = {
showLegend: true,
showXAxis: true,
showYAxis: true,
};
export interface EchartsWaterfallChartProps extends ChartProps {
@@ -114,6 +114,23 @@ export const textStyleSchema = z.object({
// Style Schemas
// =============================================================================
/** Repeating tile pattern painted over a fill, e.g. hatching */
export const decalSchema = z.object({
symbol: z.union([symbolTypeSchema, z.array(symbolTypeSchema)]).optional(),
symbolSize: z.number().optional(),
symbolKeepAspect: z.boolean().optional(),
color: colorSchema.optional(),
backgroundColor: colorSchema.optional(),
dashArrayX: z
.union([z.number(), z.array(z.union([z.number(), z.array(z.number())]))])
.optional(),
dashArrayY: z.union([z.number(), z.array(z.number())]).optional(),
/** Radians, not degrees. */
rotation: z.number().optional(),
maxTileWidth: z.number().optional(),
maxTileHeight: z.number().optional(),
});
export const lineStyleSchema = z.object({
color: colorSchema.optional(),
width: z.number().optional(),
@@ -150,6 +167,7 @@ export const itemStyleSchema = z.object({
shadowOffsetX: z.number().optional(),
shadowOffsetY: z.number().optional(),
opacity: z.number().min(0).max(1).optional(),
decal: decalSchema.optional(),
});
// =============================================================================
@@ -818,6 +836,7 @@ export type TextStyleOption = z.infer<typeof textStyleSchema>;
export type LineStyleOption = z.infer<typeof lineStyleSchema>;
export type AreaStyleOption = z.infer<typeof areaStyleSchema>;
export type ItemStyleOption = z.infer<typeof itemStyleSchema>;
export type DecalOption = z.infer<typeof decalSchema>;
export type LabelOption = z.infer<typeof labelSchema>;
export type TitleOption = z.infer<typeof titleSchema>;
export type LegendOption = z.infer<typeof legendSchema>;
@@ -593,3 +593,94 @@ test('strips tooltip extraCssText instead of passing raw CSS through', () => {
expect(result.success).toBe(true);
expect(result.data).toEqual({ tooltip: { show: true } });
});
test('accepts a decal pattern on itemStyle', () => {
const result = parseEChartOptions(
`{ series: { itemStyle: { decal: {
symbol: 'rect',
dashArrayX: [1, 0],
dashArrayY: [2, 4],
rotation: -0.7853981633974483,
color: 'rgba(0, 0, 0, 0.2)',
} } } }`,
);
expect(result.success).toBe(true);
expect(result.data).toEqual({
series: {
itemStyle: {
decal: {
symbol: 'rect',
dashArrayX: [1, 0],
dashArrayY: [2, 4],
rotation: -0.7853981633974483,
color: 'rgba(0, 0, 0, 0.2)',
},
},
},
});
});
test('accepts the full decal shape, including per-row dash arrays', () => {
// `dashArrayX` nests one level to offset rows from each other; `dashArrayY`
// has no equivalent.
const result = parseEChartOptions(
`{ series: { itemStyle: { decal: {
symbol: ['rect', 'circle'],
symbolSize: 0.8,
symbolKeepAspect: false,
color: '#383838',
backgroundColor: 'transparent',
dashArrayX: [[1, 0], [0, 1]],
dashArrayY: 5,
rotation: 0,
maxTileWidth: 512,
maxTileHeight: 512,
} } } }`,
);
expect(result.success).toBe(true);
expect(
(result.data as { series: { itemStyle: { decal: unknown } } }).series
.itemStyle.decal,
).toEqual({
symbol: ['rect', 'circle'],
symbolSize: 0.8,
symbolKeepAspect: false,
color: '#383838',
backgroundColor: 'transparent',
dashArrayX: [
[1, 0],
[0, 1],
],
dashArrayY: 5,
rotation: 0,
maxTileWidth: 512,
maxTileHeight: 512,
});
});
test('strips unknown keys from a decal rather than passing them through', () => {
const result = parseEChartOptions(
`{ series: { itemStyle: { decal: { symbol: 'rect', onclick: 'alert(1)' } } } }`,
);
expect(result.success).toBe(true);
expect(result.data).toEqual({
series: { itemStyle: { decal: { symbol: 'rect' } } },
});
});
test('rejects a decal whose values are of the wrong type', () => {
// Unknown keys are stripped; a known key with a bad type is an error.
const input = `{ series: { itemStyle: { decal: { rotation: 'sideways' } } } }`;
expect(() => parseEChartOptions(input)).toThrow(EChartOptionsParseError);
try {
parseEChartOptions(input);
} catch (error) {
expect((error as EChartOptionsParseError).errorType).toBe(
'validation_error',
);
}
});
@@ -166,3 +166,56 @@ test('hide totals', () => {
['-', '-'],
]);
});
const buildAxes = (extraFormData: Record<string, unknown>) => {
const chartProps = new ChartProps({
formData: { ...formData, ...extraFormData },
width: 800,
height: 600,
queriesData: [{ data }],
theme: supersetTheme,
});
const transformedProps = transformProps(
chartProps as unknown as EchartsWaterfallChartProps,
);
return {
xAxis: transformedProps.echartOptions.xAxis as any,
yAxis: transformedProps.echartOptions.yAxis as any,
grid: transformedProps.echartOptions.grid as any,
};
};
test('shows both axes by default', () => {
const { xAxis, yAxis } = buildAxes({});
expect(xAxis.show).not.toBe(false);
expect(yAxis.show).not.toBe(false);
});
test('hides the whole X axis when showXAxis is false', () => {
const { xAxis } = buildAxes({ showXAxis: false });
// echarts hides the axis line, ticks, labels, name, and gridlines when
// `show` is false — a single flag rather than a set of sub-flags.
expect(xAxis.show).toBe(false);
});
test('hides the whole Y axis when showYAxis is false', () => {
const { yAxis } = buildAxes({ showYAxis: false });
expect(yAxis.show).toBe(false);
});
test('reclaims the bottom grid margin when the X axis is hidden', () => {
const { grid: shown } = buildAxes({ showXAxis: true });
const { grid: hidden } = buildAxes({ showXAxis: false });
// The bottom margin reserves room for the X-axis labels and name; with the
// axis hidden that space should be reclaimed for a clean, axis-free layout.
expect(hidden.bottom).toBeLessThan(shown.bottom);
// Hiding the X axis must not shrink the Y-axis (left) margin.
expect(hidden.left).toBe(shown.left);
});
test('reclaims the left grid margin when the Y axis is hidden', () => {
const { grid: shown } = buildAxes({ showYAxis: true });
const { grid: hidden } = buildAxes({ showYAxis: false });
expect(hidden.left).toBeLessThan(shown.left);
expect(hidden.bottom).toBe(shown.bottom);
});
@@ -107,6 +107,11 @@ export default function transformProps(chartProps: ChartProps<QueryFormData>) {
);
const metricsArr = ensureIsArray(formData.metrics);
let data: QueryData[];
// The rows that conditional formatting derives its color scale from. Only the
// leaf (detail) cells belong in that domain: the totals are aggregates of the
// very cells being shaded, so letting them in makes the grand total the max
// and leaves every detail cell nearly unshaded.
let colorScaleRows: DataRecord[];
if (allMetricsAdditive(metricsArr)) {
// Additive fast-path: a single full-detail query was issued; synthesize
// each rollup level by reducing the leaf rows on the client (see SIP.md).
@@ -128,6 +133,11 @@ export default function transformProps(chartProps: ChartProps<QueryFormData>) {
data: synthesized[i] as DataRecord[],
groupby: combination,
}));
// The query returned the leaf rows and nothing else, so no totals can leak
// into the domain. Use those raw rows rather than the synthesized leaf
// level, whose reduction coerces values through `Number` and drops
// non-numeric ones -- that would shift the domain for additive metrics.
colorScaleRows = leafRows;
} else {
// Non-additive: a single GROUPING SETS query returned all rollup levels
// tagged with GROUPING() markers; split the combined result back into one
@@ -145,6 +155,13 @@ export default function transformProps(chartProps: ChartProps<QueryFormData>) {
data: splitRows[i] as DataRecord[],
groupby: combination,
}));
// This result *does* carry the rollup levels, so pick out the leaf level --
// the one grouping every dimension, same definition the splitter uses.
const leafIndex = levelLabels.findIndex(labels => {
const grouped = new Set(labels);
return allGroupbyLabels.every(label => grouped.has(label));
});
colorScaleRows = (splitRows[leafIndex] as DataRecord[]) ?? [];
}
// The full-granularity query has the most colnames -- use it for column/type
// metadata and formatters.
@@ -221,7 +238,7 @@ export default function transformProps(chartProps: ChartProps<QueryFormData>) {
);
const metricColorFormatters = getColorFormatters(
pivotConditionalFormatting,
mainQuery.data,
colorScaleRows,
theme,
);
@@ -452,3 +452,152 @@ test('additive metrics: synthesizes rollup levels from a single leaf query', ()
{ region: 'EU', v: 5 },
]);
});
test('conditional formatting scales over leaf cells only, not rollup totals', () => {
const gm = (col: string) => `${col}__superset_grouping`;
// `metrics` below is a plain string, i.e. a saved-metric reference.
// `isAdditiveMetric` treats every string as non-additive no matter what it is
// named, because form data does not reveal the aggregate behind a saved
// metric -- so a saved metric labelled "SUM(sales)" (as in the report this
// regression comes from) takes the non-additive path despite the name. That
// path issues a single GROUPING SETS query whose result carries the rollup
// levels alongside the leaf rows, so with both totals toggles on the grand
// total (100) is part of that result.
const row = (
productLine: string | null,
dealSize: string | null,
sales: number,
) => ({
product_line: productLine,
deal_size: dealSize,
'SUM(sales)': sales,
[gm('product_line')]: productLine === null ? 1 : 0,
[gm('deal_size')]: dealSize === null ? 1 : 0,
});
const totalsChartProps = new ChartProps<QueryFormData>({
formData: {
...formData,
combineMetric: false,
transposePivot: false,
metricsLayout: MetricsLayoutEnum.ROWS,
groupbyRows: ['product_line'],
groupbyColumns: ['deal_size'],
metrics: ['SUM(sales)'],
colTotals: true,
rowTotals: true,
conditionalFormatting: [
{
colorScheme: '#ACE1C4',
column: 'SUM(sales)',
operator: '>',
targetValue: 0,
},
],
},
width: 800,
height: 600,
queriesData: [
{
data: [
// leaf cells
row('Classic Cars', 'Small', 10),
row('Classic Cars', 'Large', 20),
row('Motorcycles', 'Small', 30),
row('Motorcycles', 'Large', 40),
// row totals
row('Classic Cars', null, 30),
row('Motorcycles', null, 70),
// column totals
row(null, 'Small', 40),
row(null, 'Large', 60),
// grand total
row(null, null, 100),
],
colnames: [
'product_line',
'deal_size',
'SUM(sales)',
gm('product_line'),
gm('deal_size'),
],
coltypes: [1, 1, 0, 0, 0],
},
],
hooks: { setDataMask },
filterState: { selectedFilters: {} },
datasource: { verboseMap: {}, columnFormats: {} },
theme: supersetTheme,
});
const { getColorFromValue } =
transformProps(totalsChartProps).metricColorFormatters[0];
// The largest leaf cell must be fully saturated. Including the grand total
// in the domain would stretch it to 100 and leave this cell washed out.
expect(getColorFromValue(40)).toEqual('#ACE1C4FF');
});
test('conditional formatting on the additive path uses the raw leaf query rows', () => {
// Counterpart to the test above for the additive fast path. Its query returns
// the leaf rows only, so the domain is those rows verbatim -- deliberately not
// the synthesized leaf level, whose reduction would coerce values through
// `Number` and drop non-numeric ones.
const additiveChartProps = new ChartProps<QueryFormData>({
formData: {
...formData,
combineMetric: false,
transposePivot: false,
metricsLayout: MetricsLayoutEnum.ROWS,
groupbyRows: ['product_line'],
groupbyColumns: ['deal_size'],
metrics: [
{
expressionType: 'SIMPLE',
aggregate: 'SUM',
column: { column_name: 'sales' },
label: 'SUM(sales)',
},
],
colTotals: true,
rowTotals: true,
conditionalFormatting: [
{
colorScheme: '#ACE1C4',
column: 'SUM(sales)',
operator: '>',
targetValue: 0,
},
],
},
width: 800,
height: 600,
queriesData: [
{
data: [
{
product_line: 'Classic Cars',
deal_size: 'Small',
'SUM(sales)': 10,
},
{
product_line: 'Classic Cars',
deal_size: 'Large',
'SUM(sales)': 20,
},
{ product_line: 'Motorcycles', deal_size: 'Small', 'SUM(sales)': 30 },
{ product_line: 'Motorcycles', deal_size: 'Large', 'SUM(sales)': 40 },
],
colnames: ['product_line', 'deal_size', 'SUM(sales)'],
coltypes: [1, 1, 0],
},
],
hooks: { setDataMask },
filterState: { selectedFilters: {} },
datasource: { verboseMap: {}, columnFormats: {} },
theme: supersetTheme,
});
const { getColorFromValue } =
transformProps(additiveChartProps).metricColorFormatters[0];
// Scale spans the leaf cells (max 40), never the client-side grand total 100.
expect(getColorFromValue(40)).toEqual('#ACE1C4FF');
});
@@ -1138,7 +1138,8 @@ export default function TableChart<D extends DataRecord = DataRecord>(
?.backgroundColor || backgroundColor;
arrow =
column.label === comparisonLabels[0]
? basicColorColumnFormatters[row.index]?.[column.key]?.mainArrow
? (basicColorColumnFormatters[row.index]?.[column.key]
?.mainArrow ?? arrow)
: '';
}
const rowSurfaceColor =
@@ -1194,30 +1195,36 @@ export default function TableChart<D extends DataRecord = DataRecord>(
}
`;
let arrowStyles = css`
color: ${
// Plain inline style (rather than the `css` prop) so the arrow's
// color is guaranteed to apply regardless of whether the consuming
// app's build wires up the emotion JSX pragma for the `css` prop --
// notably, this codebase's own Jest/Babel config does not, which
// silently no-ops any `css` prop on a plain DOM element.
let arrowStyles: CSSProperties = {
color:
basicColorFormatters &&
basicColorFormatters[row.index]?.[originKey]?.arrowColor ===
ColorSchemeEnum.Green
? theme.colorSuccess
: theme.colorError
};
margin-right: ${theme.sizeUnit}px;
`;
: theme.colorError,
marginRight: theme.sizeUnit,
};
if (
basicColorColumnFormatters &&
basicColorColumnFormatters?.length > 0
) {
arrowStyles = css`
color: ${
basicColorColumnFormatters[row.index]?.[column.key]
?.arrowColor === ColorSchemeEnum.Green
? theme.colorSuccess
: theme.colorError
const columnArrowColor =
basicColorColumnFormatters[row.index]?.[column.key]?.arrowColor;
if (columnArrowColor) {
arrowStyles = {
color:
columnArrowColor === ColorSchemeEnum.Green
? theme.colorSuccess
: theme.colorError,
marginRight: theme.sizeUnit,
};
margin-right: ${theme.sizeUnit}px;
`;
}
}
const cellProps = {
@@ -1302,12 +1309,12 @@ export default function TableChart<D extends DataRecord = DataRecord>(
className="dt-truncate-cell"
style={columnWidth ? { width: columnWidth } : undefined}
>
{arrow && <span css={arrowStyles}>{arrow}</span>}
{arrow && <span style={arrowStyles}>{arrow}</span>}
{text}
</div>
) : (
<>
{arrow && <span css={arrowStyles}>{arrow}</span>}
{arrow && <span style={arrowStyles}>{arrow}</span>}
{text}
</>
)}
@@ -34,7 +34,7 @@ import {
getTotalsMetrics,
isTimeComparison,
timeCompareOperator,
TotalsAggregate,
toTotalsAggregate,
} from '@superset-ui/chart-controls';
import { isEmpty } from 'lodash-es';
import { TableChartFormData } from './types';
@@ -349,8 +349,7 @@ export const buildQuery: BuildQuery<TableChartFormData> = (
formData.show_totals &&
queryMode === QueryMode.Aggregate
) {
const totalsAggregate: TotalsAggregate =
formData.totals_aggregate === 'AVG' ? 'AVG' : 'SUM';
const totalsAggregate = toTotalsAggregate(formData.totals_aggregate);
extraQueries.push({
...queryObject,
columns: [],
@@ -475,14 +475,18 @@ const config: ControlPanelConfig = {
type: 'SelectControl',
label: t('Summary aggregation'),
description: t(
'Aggregation used for the summary row, independent of each ' +
"metric's own aggregation. Only applies to simple metrics " +
'(a metric built from custom SQL keeps its own aggregation ' +
'in the summary row).',
'Aggregation used for the summary row. By default each metric ' +
'keeps its own aggregation; Sum and Average override it for ' +
'the summary row only. The override applies to simple ' +
'metrics (a metric built from custom SQL always keeps its ' +
'own aggregation). Overriding a count or a distinct count ' +
'sums the counted column instead, which fails outright on a ' +
'non-numeric column.',
),
default: 'SUM',
default: 'ORIGINAL',
clearable: false,
choices: [
['ORIGINAL', t("Each metric's own")],
['SUM', t('Sum')],
['AVG', t('Average')],
],
@@ -2111,7 +2111,14 @@ describe('plugin-chart-table', () => {
expect(() =>
render(
<TableChart {...propsWithMissingFormatterEntry} sticky={false} />,
ProviderWrapper({
children: (
<TableChart
{...propsWithMissingFormatterEntry}
sticky={false}
/>
),
}),
),
).not.toThrow();
@@ -2125,8 +2132,24 @@ describe('plugin-chart-table', () => {
'rgba(0, 150, 0, 0.2)',
);
// the row missing a formatter entry still renders its raw value
expect(screen.getAllByTitle('110').length).toBeGreaterThan(0);
// the row missing a formatter entry falls back to the row-level
// comparison arrow instead of losing it: before the fix, this row's
// arrow was silently cleared (and its color, computed the same way,
// would have flipped to the "decrease" color) whenever the
// column-specific lookup for this row was undefined.
const arrowCell = screen
.getAllByTitle('110')
.find(cell => cell.querySelector('span'));
expect(arrowCell).toHaveTextContent('↑110');
expect(getComputedStyle(arrowCell!).background).toContain(
'rgba(0, 150, 0, 0.2)',
);
// the fallback arrow itself must also keep the "increase" color --
// asserting only the cell background would still pass if the arrow's
// own color had regressed to the "decrease" color.
expect(arrowCell!.querySelector('span')).toHaveStyle({
color: supersetTheme.colorSuccess,
});
});
test('preserves client-side search text across temporal table rerenders', async () => {
@@ -340,7 +340,7 @@ describe('plugin-chart-table', () => {
label: 'sum_sales',
};
test('defaults the totals query metric aggregate to SUM', () => {
test("defaults to each metric's own aggregate", () => {
const { queries } = buildQueryCached({
...basicFormData,
query_mode: QueryMode.Aggregate,
@@ -350,9 +350,28 @@ describe('plugin-chart-table', () => {
});
expect(queries).toHaveLength(2);
expect(queries[1].metrics).toEqual([
{ ...simpleMetric, aggregate: 'SUM' },
]);
expect(queries[1].metrics).toEqual([simpleMetric]);
});
test('keeps COUNT_DISTINCT in the summary row by default', () => {
// Overriding this to SUM sums the counted column instead of counting
// it, which is meaningless on a numeric id and is rejected outright by
// the database on a non-numeric one (e.g. a uuid).
const countDistinctMetric = {
expressionType: 'SIMPLE' as const,
column: { column_name: 'contract_id' },
aggregate: 'COUNT_DISTINCT' as const,
label: 'contracts',
};
const { queries } = buildQueryCached({
...basicFormData,
query_mode: QueryMode.Aggregate,
metrics: [countDistinctMetric],
groupby: ['category'],
show_totals: true,
});
expect(queries[1].metrics).toEqual([countDistinctMetric]);
});
test('overrides simple metric aggregate with totals_aggregate for the summary query only', () => {
@@ -61,6 +61,53 @@ describe('sqlLabReducer', () => {
});
});
test('should default extra_json to an empty object when extra is unset', () => {
// `extra` is nullable in the metadata database, and JSON.parse(extra || '')
// is guaranteed to throw because '' is never valid JSON, so one such row
// took down the whole reducer.
const incomingDb = {
...databases.result[0],
extra: null,
};
const incomingDbId = Number(incomingDb.id);
const action = actions.setDatabases([incomingDb] as any);
const newState = sqlLabReducer(initialState, action);
expect(newState.databases[incomingDbId]).toEqual({
...incomingDb,
extra_json: {},
});
});
test('defaults extra_json when a database has malformed extra', () => {
const incomingDb = { ...databases.result[0], extra: '{not json' };
const newState = sqlLabReducer(
initialState,
actions.setDatabases([incomingDb] as any),
);
expect(newState.databases[Number(incomingDb.id)].extra_json).toEqual({});
});
test('keeps a valid extra payload', () => {
const incomingDb = {
...databases.result[0],
extra: '{"engine_params": {"pool_size": 5}}',
};
const newState = sqlLabReducer(
initialState,
actions.setDatabases([incomingDb] as any),
);
expect(newState.databases[Number(incomingDb.id)].extra_json).toEqual({
engine_params: { pool_size: 5 },
});
});
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
describe('Query editors actions', () => {
let newState: SqlLabState;
@@ -36,6 +36,27 @@ import {
type SqlLabState = SqlLabRootState['sqlLab'];
/**
* A database's `extra` column is free-form and frequently empty: it is nullable
* in the metadata database and the API returns it verbatim. `JSON.parse` cannot
* represent that, and `JSON.parse(extra || '')` is guaranteed to throw, since
* the empty string is never valid JSON so a single database row with no
* `extra` took down the whole SET_DATABASES reducer and with it SQL Lab.
* Malformed JSON is treated the same way: one bad row must not cost the user
* every other database.
*/
function parseDatabaseExtra(extra: unknown): Record<string, unknown> {
if (typeof extra !== 'string' || extra.trim() === '') {
return {};
}
try {
const parsed = JSON.parse(extra);
return parsed && typeof parsed === 'object' ? parsed : {};
} catch {
return {};
}
}
function alterUnsavedQueryEditorState(
state: SqlLabState,
updatedState: Partial<QueryEditor>,
@@ -727,7 +748,7 @@ export default function sqlLabReducer(
(action.databases as any[])!.forEach((db: any) => {
databases[db.id] = {
...db,
extra_json: JSON.parse(db.extra || ''),
extra_json: parseDatabaseExtra(db.extra),
};
});
return {
@@ -723,11 +723,13 @@ const AlertReportModal: FunctionComponent<AlertReportModalProps> = ({
force_screenshot: false,
include_cta: true,
grace_period: undefined,
retry_on_failure: false,
retry_max_attempts: 3,
send_failed_reports: false,
retry_notify_owners: true,
retry_notify_recipients: false,
...(isFeatureEnabled(FeatureFlag.AlertReportsRetry) && {
retry_on_failure: false,
retry_max_attempts: 3,
send_failed_reports: false,
retry_notify_owners: true,
retry_notify_recipients: false,
}),
};
const fetchDashboardFilterValues = async (
@@ -2762,7 +2764,7 @@ const AlertReportModal: FunctionComponent<AlertReportModalProps> = ({
</>
),
},
...(isReport
...(isReport && isFeatureEnabled(FeatureFlag.AlertReportsRetry)
? [
{
key: 'error-handling',
@@ -669,11 +669,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information/i,
name: /General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings/i,
name: /File settings/i,
});
await userEvent.click(fileSettingsTab);
await waitFor(() => {
@@ -689,11 +689,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information/i,
name: /General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings/i,
name: /File settings/i,
});
await userEvent.click(fileSettingsTab);
await waitFor(() => {
@@ -709,11 +709,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information/i,
name: /General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings/i,
name: /File settings/i,
});
await userEvent.click(fileSettingsTab);
await waitFor(() => {
@@ -452,7 +452,23 @@ test('submit failure dispatches danger toast and keeps modal open', async () =>
// Error Handling section tests
// ---------------------------------------------------------------------------
test('Error Handling section is visible and Enable Retries checkbox is unchecked by default', () => {
const enableRetryFlag = () => {
mockedIsFeatureEnabled.mockImplementation(
(featureFlag: string) =>
featureFlag === FeatureFlag.AlertReports ||
featureFlag === FeatureFlag.AlertReportsRetry,
);
};
test('Error Handling section is hidden when ALERT_REPORTS_RETRY flag is off', () => {
const store = createStore({}, reducerIndex);
render(<ReportModal {...defaultProps} />, { useRedux: true, store });
expect(screen.queryByText('Error Handling')).not.toBeInTheDocument();
});
test('Error Handling section is visible when ALERT_REPORTS_RETRY flag is on', () => {
enableRetryFlag();
const store = createStore({}, reducerIndex);
render(<ReportModal {...defaultProps} />, { useRedux: true, store });
@@ -464,6 +480,7 @@ test('Error Handling section is visible and Enable Retries checkbox is unchecked
});
test('conditional retry fields are hidden when Enable Retries is unchecked', () => {
enableRetryFlag();
const store = createStore({}, reducerIndex);
render(<ReportModal {...defaultProps} />, { useRedux: true, store });
@@ -473,6 +490,7 @@ test('conditional retry fields are hidden when Enable Retries is unchecked', ()
});
test('conditional retry fields appear when Enable Retries is checked', async () => {
enableRetryFlag();
const store = createStore({}, reducerIndex);
render(<ReportModal {...defaultProps} />, { useRedux: true, store });
@@ -489,6 +507,7 @@ test('conditional retry fields appear when Enable Retries is checked', async ()
});
test('retry fields are included in the POST body when Enable Retries is enabled', async () => {
enableRetryFlag();
fetchMock.post(REPORT_ENDPOINT, { result: {} }, { name: 'post-retry' });
const store = createStore({}, reducerIndex);
render(
@@ -26,7 +26,12 @@ import {
} from 'react';
import { t } from '@apache-superset/core/translation';
import { getClientErrorObject, VizType } from '@superset-ui/core';
import {
getClientErrorObject,
isFeatureEnabled,
FeatureFlag,
VizType,
} from '@superset-ui/core';
import { Alert } from '@apache-superset/core/components';
import { SupersetTheme } from '@apache-superset/core/theme';
import { useDispatch, useSelector } from 'react-redux';
@@ -196,11 +201,13 @@ function ReportModal({
crontab: currentReport.crontab,
report_format: currentReport.report_format || defaultNotificationFormat,
timezone: currentReport.timezone,
retry_on_failure: currentReport.retry_on_failure ?? false,
retry_max_attempts: currentReport.retry_max_attempts ?? 3,
send_failed_reports: currentReport.send_failed_reports ?? false,
retry_notify_owners: currentReport.retry_notify_owners ?? true,
retry_notify_recipients: currentReport.retry_notify_recipients ?? false,
...(isFeatureEnabled(FeatureFlag.AlertReportsRetry) && {
retry_on_failure: currentReport.retry_on_failure ?? false,
retry_max_attempts: currentReport.retry_max_attempts ?? 3,
send_failed_reports: currentReport.send_failed_reports ?? false,
retry_notify_owners: currentReport.retry_notify_owners ?? true,
retry_notify_recipients: currentReport.retry_notify_recipients ?? false,
}),
};
setCurrentReport({ isSubmitting: true, error: undefined });
@@ -478,7 +485,8 @@ function ReportModal({
/>
{isChart && renderMessageContentSection}
{(!isChart || !isTextBasedChart) && renderCustomWidthSection}
{renderErrorHandlingSection}
{isFeatureEnabled(FeatureFlag.AlertReportsRetry) &&
renderErrorHandlingSection}
</StyledBottomSection>
{currentReport.error && (
<Alert
@@ -223,7 +223,7 @@ describe('ChartPage', () => {
window.history.pushState(
{},
'',
`/?${URL_PARAMS.dashboardPageId.name}=${dashboardPageId}`,
`/explore/?${URL_PARAMS.dashboardPageId.name}=${dashboardPageId}`,
);
const { getByTestId } = render(<ChartPage />, {
useRouter: true,
@@ -261,13 +261,13 @@ describe('ChartPage', () => {
window.history.pushState(
{},
'',
`/?${URL_PARAMS.dashboardPageId.name}=${dashboardPageId}`,
`/explore/?${URL_PARAMS.dashboardPageId.name}=${dashboardPageId}`,
);
const { getByTestId } = render(
<>
<Link
to={{
pathname: '/',
pathname: '/explore/',
search: `?${URL_PARAMS.dashboardPageId.name}=${dashboardPageId}`,
state: { saveAction: 'overwrite' },
}}
@@ -324,7 +324,7 @@ describe('ChartPage', () => {
});
render(
<>
<Link to="/?slice_id=99">Navigate away</Link>
<Link to="/explore/?slice_id=99">Navigate away</Link>
<ChartPage />
</>,
{
@@ -382,7 +382,7 @@ describe('ChartPage', () => {
<>
<Link
to={{
pathname: '/',
pathname: '/explore/',
search: `?${URL_PARAMS.sliceId.name}=${formData.slice_id}`,
state: toChartStateHistoryState({
...formData,
@@ -392,7 +392,7 @@ describe('ChartPage', () => {
>
Change the chart
</Link>
<Link to="/?slice_id=99">Navigate away</Link>
<Link to="/explore/?slice_id=99">Navigate away</Link>
<ChartPage />
</>,
{ useRouter: true, useRedux: true, useDnd: true },
@@ -433,14 +433,14 @@ describe('ChartPage', () => {
<>
<Link
to={{
pathname: '/',
pathname: '/explore/',
search: `?${URL_PARAMS.sliceId.name}=99`,
state: toChartStateHistoryState({ ...formData, slice_id: 99 }),
}}
>
Another chart
</Link>
<Link to="/?slice_id=100">Navigate away</Link>
<Link to="/explore/?slice_id=100">Navigate away</Link>
<ChartPage />
</>,
{ useRouter: true, useRedux: true, useDnd: true },
@@ -477,14 +477,14 @@ describe('ChartPage', () => {
<>
<Link
to={{
pathname: '/',
pathname: '/explore/',
search: `?${URL_PARAMS.sliceId.name}=${formData.slice_id}`,
state: toChartStateHistoryState(formData),
}}
>
Change the chart
</Link>
<Link to="/?slice_id=99">Navigate away</Link>
<Link to="/explore/?slice_id=99">Navigate away</Link>
<ChartPage />
</>,
{ useRouter: true, useRedux: true, useDnd: true, store },
@@ -507,6 +507,32 @@ describe('ChartPage', () => {
window.history.back();
await waitFor(() => expect(loads()).toBe(1));
});
test('does not re-fetch explore data when navigating to a dashboard', async () => {
const exploreApiRoute = 'glob:*/api/v1/explore/*';
const exploreFormData = getExploreFormData({
viz_type: VizType.Table,
show_cell_bars: true,
});
fetchMock.get(exploreApiRoute, {
result: { dataset: { id: 1 }, form_data: exploreFormData },
});
render(
<>
<Link to="/dashboard/5/">Go to dashboard</Link>
<ChartPage />
</>,
{ useRouter: true, useRedux: true, useDnd: true },
);
await waitFor(() =>
expect(fetchMock.callHistory.calls(exploreApiRoute).length).toBe(1),
);
fireEvent.click(screen.getByText('Go to dashboard'));
await new Promise(resolve => setTimeout(resolve, 0));
expect(fetchMock.callHistory.calls(exploreApiRoute).length).toBe(1);
});
});
test('does not show error toast when request is aborted on unmount', async () => {
@@ -559,7 +585,7 @@ describe('ChartPage', () => {
render(
<>
<Link to="/?slice_id=99">Navigate</Link>
<Link to="/explore/?slice_id=99">Navigate</Link>
<ChartPage />
</>,
{
@@ -56,6 +56,11 @@ const isValidResult = (rv: JsonObject): boolean =>
const hasDatasetId = (rv: JsonObject): boolean =>
isDefined(rv?.result?.dataset?.id);
const EXPLORE_ROUTE_PREFIX = '/explore/';
const isExploreRoute = (pathname: string): boolean =>
pathname.startsWith(EXPLORE_ROUTE_PREFIX);
const fetchExploreData = async (
exploreUrlParams: URLSearchParams,
signal?: AbortSignal,
@@ -312,6 +317,8 @@ export default function ExplorePage() {
// Other REPLACE: ignored (URL sync from updateHistory).
// Entries holding a chart state of the loaded chart are skipped: Explore
// pushed them itself, and ExploreViewContainer restores a popped one in place.
// Navigations that leave Explore must not trigger a re-fetch while the page
// is unmounting, as the destination's URL params are not chart params.
useEffect(() => {
const unlisten = history.listen((loc: Location, action: Action) => {
const saveAction = (loc.state as Record<string, unknown>)?.saveAction as
@@ -326,6 +333,9 @@ export default function ExplorePage() {
return;
}
}
if (!isExploreRoute(loc.pathname)) {
return;
}
if (action === 'PUSH' || action === 'POP') {
setIsLoaded(false);
loadExploreData(loc, saveAction);
@@ -0,0 +1,425 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import fetchMock from 'fetch-mock';
import rison from 'rison';
import { configureStore } from '@reduxjs/toolkit';
import {
act,
fireEvent,
render,
screen,
waitFor,
within,
} from 'spec/helpers/testing-library';
import userEvent from '@testing-library/user-event';
import DatabaseList from 'src/pages/DatabaseList';
/**
* Deleting a semantic layer cascade-deletes its semantic views (SC-108418).
* These tests pin the delete confirmation's cascade warning: the dependent
* views are counted and named before the user confirms, and a failed lookup
* still opens the modal with an uncounted warning rather than blocking.
*/
const SL_UUID = '6a000000-0000-4000-8000-000000000001';
const SL_UUID_B = '6b000000-0000-4000-8000-000000000002';
const semanticLayerRow = {
source_type: 'semantic_layer',
uuid: SL_UUID,
database_name: 'Demo Semantic Layer',
backend: 'Demo',
sl_type: 'demo',
description: null,
allow_run_async: null,
allow_dml: null,
allow_file_upload: null,
expose_in_sqllab: null,
changed_on_delta_humanized: 'a day ago',
changed_by: null,
};
const semanticLayerRowB = {
...semanticLayerRow,
uuid: SL_UUID_B,
database_name: 'Second Semantic Layer',
};
const CONNECTIONS_ROUTE = 'glob:*/api/v1/semantic_layer/connections/*';
const DATASOURCE_ROUTE = 'glob:*/api/v1/datasource/?*';
const DELETE_ROUTE = `glob:*/api/v1/semantic_layer/${SL_UUID}`;
const mockUser = {
userId: 1,
firstName: 'Admin',
lastName: 'User',
roles: { Admin: [['can_write', 'Database']] },
permissions: {},
isActive: true,
email: 'admin@example.com',
createdOn: '2026-01-01T00:00:00',
};
const dependentView = (id: number, name: string) => ({
id,
table_name: name,
kind: 'semantic_view',
source_type: 'semantic_layer',
});
const setupMocks = ({
dependents,
dependentsError = false,
rows = [semanticLayerRow],
}: {
dependents: { id: number; table_name: string }[];
dependentsError?: boolean;
rows?: (typeof semanticLayerRow)[];
}) => {
fetchMock.clearHistory().removeRoutes();
fetchMock.get('glob:*/api/v1/database/_info*', {
permissions: ['can_read', 'can_write', 'can_export'],
});
fetchMock.get('glob:*/api/v1/database/?q=*', { result: [], count: 0 });
fetchMock.get('glob:*/api/v1/database/related/*', { result: [], count: 0 });
fetchMock.get(CONNECTIONS_ROUTE, {
result: rows,
count: rows.length,
});
if (dependentsError) {
fetchMock.get(DATASOURCE_ROUTE, 500, { name: DATASOURCE_ROUTE });
} else {
fetchMock.get(
DATASOURCE_ROUTE,
{ result: dependents, count: dependents.length },
{ name: DATASOURCE_ROUTE },
);
}
fetchMock.delete(DELETE_ROUTE, {});
};
const renderDatabaseList = () => {
const store = configureStore({
reducer: {
user: (state = mockUser) => state,
common: (
state = {
conf: {
CSV_EXTENSIONS: ['csv'],
EXCEL_EXTENSIONS: ['xls'],
COLUMNAR_EXTENSIONS: ['parquet'],
ALLOWED_EXTENSIONS: ['csv', 'xls', 'parquet'],
SYNC_DB_PERMISSIONS_IN_ASYNC_MODE: false,
},
},
) => state,
},
middleware: getDefaultMiddleware =>
getDefaultMiddleware({ serializableCheck: false, immutableCheck: false }),
});
return render(<DatabaseList user={mockUser} />, {
store,
useQueryParams: true,
useRouter: true,
});
};
const openDeleteModal = async () => {
const deleteButton = await screen.findByTestId('Delete');
await userEvent.click(deleteButton);
return screen.findByRole('dialog');
};
beforeEach(() => {
window.featureFlags = { SEMANTIC_LAYERS: true } as never;
});
afterEach(() => {
window.featureFlags = {} as never;
fetchMock.clearHistory();
fetchMock.removeRoutes();
});
test('delete confirmation warns about cascade-deleting dependent views by count and name', async () => {
setupMocks({
dependents: [
dependentView(1, 'marketing'),
dependentView(2, 'sales'),
dependentView(3, 'orders'),
],
});
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'This will also permanently delete its 3 semantic views. Charts built on those views will stop working.',
),
).toBeInTheDocument();
expect(
within(dialog).getByText('Affected semantic views'),
).toBeInTheDocument();
expect(within(dialog).getByText('marketing')).toBeInTheDocument();
expect(within(dialog).getByText('sales')).toBeInTheDocument();
expect(within(dialog).getByText('orders')).toBeInTheDocument();
// The dependent lookup must target this layer's views.
const lookupCalls = fetchMock.callHistory.calls(DATASOURCE_ROUTE);
expect(lookupCalls).toHaveLength(1);
const q = new URL(lookupCalls[0].url).searchParams.get('q') as string;
expect(rison.decode(q)).toMatchObject({
filters: [{ col: 'semantic_layer_uuid', opr: 'eq', value: SL_UUID }],
});
});
test('a single dependent view is announced in the singular', async () => {
setupMocks({ dependents: [dependentView(1, 'marketing')] });
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'This will also permanently delete its 1 semantic view. Charts built on that view will stop working.',
),
).toBeInTheDocument();
});
test('a genuinely empty layer says so instead of warning about nonexistent views', async () => {
// A successful count === 0 is always genuinely empty, never
// access-filtering: a layer is only reachable when its perm is granted,
// and the dependent-view count ORs on that same perm.
setupMocks({ dependents: [] });
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'This semantic layer has no dependent semantic views.',
),
).toBeInTheDocument();
expect(
within(dialog).queryByText(
/charts built on those views will stop working/i,
),
).not.toBeInTheDocument();
expect(
within(dialog).queryByText('Affected semantic views'),
).not.toBeInTheDocument();
});
test('a counted response with an empty name page keeps the count but omits the list', async () => {
setupMocks({ dependents: [] });
fetchMock.removeRoutes({ names: [DATASOURCE_ROUTE] });
fetchMock.get(
DATASOURCE_ROUTE,
{ result: [], count: 3 },
{ name: DATASOURCE_ROUTE },
);
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'This will also permanently delete its 3 semantic views. Charts built on those views will stop working.',
),
).toBeInTheDocument();
expect(
within(dialog).queryByText('Affected semantic views'),
).not.toBeInTheDocument();
});
test('a failed dependent lookup still opens the modal with an uncounted warning', async () => {
setupMocks({ dependents: [], dependentsError: true });
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'Deleting this semantic layer also permanently deletes any semantic views it contains, and charts built on those views will stop working. The affected views could not be listed.',
),
).toBeInTheDocument();
});
test('the overflow footer reports dependent views beyond the listed page', async () => {
// The lookup pages at 10 names; the count is the full total.
setupMocks({ dependents: [] });
fetchMock.removeRoutes({ names: [DATASOURCE_ROUTE] });
fetchMock.get(
DATASOURCE_ROUTE,
{
result: Array.from({ length: 10 }, (_, i) =>
dependentView(i + 1, `view_${i + 1}`),
),
count: 12,
},
{ name: DATASOURCE_ROUTE },
);
renderDatabaseList();
const dialog = await openDeleteModal();
expect(
within(dialog).getByText(
'This will also permanently delete its 12 semantic views. Charts built on those views will stop working.',
),
).toBeInTheDocument();
expect(within(dialog).getByText('view_10')).toBeInTheDocument();
expect(within(dialog).getByText('... and 2 others')).toBeInTheDocument();
});
test('the overflow footer uses the singular for one unlisted view', async () => {
setupMocks({ dependents: [] });
fetchMock.removeRoutes({ names: [DATASOURCE_ROUTE] });
fetchMock.get(
DATASOURCE_ROUTE,
{
result: Array.from({ length: 10 }, (_, i) =>
dependentView(i + 1, `view_${i + 1}`),
),
count: 11,
},
{ name: DATASOURCE_ROUTE },
);
renderDatabaseList();
const dialog = await openDeleteModal();
expect(within(dialog).getByText('... and 1 other')).toBeInTheDocument();
});
test('a pending lookup disables repeated delete requests and shows progress', async () => {
setupMocks({ dependents: [dependentView(1, 'marketing')] });
fetchMock.removeRoutes({ names: [DATASOURCE_ROUTE] });
let releaseLookup: () => void = () => {};
const lookupGate = new Promise<void>(resolve => {
releaseLookup = resolve;
});
fetchMock.get(
DATASOURCE_ROUTE,
async () => {
await lookupGate;
return { result: [dependentView(1, 'marketing')], count: 1 };
},
{ name: DATASOURCE_ROUTE },
);
renderDatabaseList();
const deleteButton = await screen.findByTestId('Delete');
await userEvent.click(deleteButton);
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
await waitFor(() => {
expect(screen.getByTestId('Delete')).toHaveAttribute(
'aria-disabled',
'true',
);
});
expect(screen.getByTestId('Delete')).toHaveAccessibleName(
'Loading dependent semantic views',
);
await userEvent.click(screen.getByTestId('Delete'));
expect(fetchMock.callHistory.calls(DATASOURCE_ROUTE)).toHaveLength(1);
releaseLookup();
expect(await screen.findByRole('dialog')).toBeInTheDocument();
});
test("a stale lookup resolving late cannot replace a newer row's modal", async () => {
// Click Delete on layer A (its lookup hangs), then on layer B (resolves
// immediately). When A's lookup finally resolves, the generation guard must
// drop it: the modal keeps showing B's preview.
setupMocks({
dependents: [],
rows: [semanticLayerRow, semanticLayerRowB],
});
fetchMock.removeRoutes({ names: [DATASOURCE_ROUTE] });
let releaseFirstLookup: () => void = () => {};
const firstLookupGate = new Promise<void>(resolve => {
releaseFirstLookup = resolve;
});
fetchMock.get(
DATASOURCE_ROUTE,
async ({ url }) => {
// The layer uuid rides in the rison-encoded `q` filter and its
// characters survive URL encoding, so a substring check is enough to
// tell the two lookups apart.
if (url.includes(SL_UUID)) {
await firstLookupGate;
return { result: [dependentView(1, 'stale_view')], count: 1 };
}
return { result: [dependentView(2, 'fresh_view')], count: 1 };
},
{ name: DATASOURCE_ROUTE },
);
renderDatabaseList();
const deleteButtons = await screen.findAllByTestId('Delete');
expect(deleteButtons).toHaveLength(2);
// fireEvent, not userEvent: userEvent's hover step re-renders the row
// (tooltip) and detaches the pressed node mid-sequence when the table has
// multiple rows, so its click never reaches the handler.
fireEvent.click(deleteButtons[0]);
await waitFor(() => {
expect(screen.getAllByTestId('Delete')[0]).toHaveAttribute(
'aria-disabled',
'true',
);
});
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
fireEvent.click(screen.getAllByTestId('Delete')[1]);
const dialog = await screen.findByRole('dialog');
expect(within(dialog).getByText('fresh_view')).toBeInTheDocument();
await act(async () => {
releaseFirstLookup();
await new Promise(resolve => {
setTimeout(resolve, 0);
});
});
expect(
within(screen.getByRole('dialog')).getByText('fresh_view'),
).toBeInTheDocument();
expect(screen.queryByText('stale_view')).not.toBeInTheDocument();
});
test('confirming the modal deletes the semantic layer', async () => {
setupMocks({ dependents: [dependentView(1, 'marketing')] });
renderDatabaseList();
const dialog = await openDeleteModal();
await userEvent.type(
within(dialog).getByTestId('delete-modal-input'),
'DELETE',
);
await userEvent.click(within(dialog).getByRole('button', { name: 'Delete' }));
await waitFor(() => {
expect(fetchMock.callHistory.calls(DELETE_ROUTE)).toHaveLength(1);
});
});
@@ -16,7 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { t, tn } from '@apache-superset/core/translation';
import {
getExtensionsRegistry,
SupersetClient,
@@ -24,7 +24,7 @@ import {
FeatureFlag,
} from '@superset-ui/core';
import { css, useTheme } from '@apache-superset/core/theme';
import { useState, useMemo, useEffect, useCallback } from 'react';
import { useState, useMemo, useEffect, useCallback, useRef } from 'react';
import type { CellProps } from 'react-table';
import rison from 'rison';
import { useSelector } from 'react-redux';
@@ -101,6 +101,91 @@ interface DatabaseDeleteObject extends DatabaseObject {
dashboards: any;
sqllab_tab_count: number;
}
/** How many dependent semantic views the delete confirmation lists by name. */
const MAX_DEPENDENT_VIEWS_LISTED = 10;
type SemanticLayerDeletePreview =
| { status: 'loading'; item: ConnectionItem }
| {
status: 'loaded';
item: ConnectionItem;
dependentViewCount: number;
dependentViewNames: string[];
}
| { status: 'failed'; item: ConnectionItem };
type ResolvedSemanticLayerDeletePreview = Exclude<
SemanticLayerDeletePreview,
{ status: 'loading' }
>;
function SemanticLayerCascadeWarning({
preview,
}: {
preview: ResolvedSemanticLayerDeletePreview;
}) {
if (preview.status === 'failed') {
return (
<p>
{t(
'Deleting this semantic layer also permanently deletes any semantic views it contains, and charts built on those views will stop working. The affected views could not be listed.',
)}
</p>
);
}
// A reachable layer always has all of its views counted (the layer's perm
// and its views' perms travel together), so zero means genuinely empty —
// never access-filtered. An honest empty message keeps the destructive
// warning credible for the layers where it matters.
if (preview.dependentViewCount === 0) {
return <p>{t('This semantic layer has no dependent semantic views.')}</p>;
}
const listedViewCount = preview.dependentViewNames.length;
const overflowViewCount = preview.dependentViewCount - listedViewCount;
return (
<>
<p>
{tn(
'This will also permanently delete its %s semantic view. Charts built on that view will stop working.',
'This will also permanently delete its %s semantic views. Charts built on those views will stop working.',
preview.dependentViewCount,
preview.dependentViewCount,
)}
</p>
{listedViewCount > 0 && (
<>
<h4>{t('Affected semantic views')}</h4>
<List
split={false}
size="small"
dataSource={preview.dependentViewNames}
renderItem={(name: string, index: number) => (
<List.Item key={`${index}-${name}`} compact>
<List.Item.Meta avatar={<span></span>} title={name} />
</List.Item>
)}
footer={
overflowViewCount > 0 && (
<div>
{tn(
'... and %s other',
'... and %s others',
overflowViewCount,
overflowViewCount,
)}
</div>
)
}
/>
</>
)}
</>
);
}
interface DatabaseListProps {
addDangerToast: (msg: string) => void;
addSuccessToast: (msg: string) => void;
@@ -257,8 +342,8 @@ function DatabaseList({
const [slCurrentlyEditing, setSlCurrentlyEditing] = useState<string | null>(
null,
);
const [slCurrentlyDeleting, setSlCurrentlyDeleting] =
useState<ConnectionItem | null>(null);
const [slDeletePreview, setSlDeletePreview] =
useState<SemanticLayerDeletePreview | null>(null);
const [allowUploads, setAllowUploads] = useState<boolean>(false);
const isAdmin = isUserAdmin(fullUser);
@@ -304,6 +389,43 @@ function DatabaseList({
[],
);
// Deleting a semantic layer cascade-deletes its semantic views, so the
// confirmation must say what else is about to be destroyed. If the lookup
// fails the modal still opens, with an uncounted warning: the count is an
// aid, not a gate on deleting. The generation counter drops stale
// resolutions -- without it a slow lookup could reopen a modal the user
// already dismissed, or replace a newer row's modal with an older one.
const slDeleteLookupRef = useRef(0);
const openSemanticLayerDeleteModal = useCallback((item: ConnectionItem) => {
slDeleteLookupRef.current += 1;
const lookupId = slDeleteLookupRef.current;
setSlDeletePreview({ status: 'loading', item });
return SupersetClient.get({
endpoint: `/api/v1/datasource/?q=${rison.encode_uri({
filters: [{ col: 'semantic_layer_uuid', opr: 'eq', value: item.uuid }],
order_column: 'table_name',
order_direction: 'asc',
page: 0,
page_size: MAX_DEPENDENT_VIEWS_LISTED,
})}`,
})
.then(({ json = {} }) => {
if (slDeleteLookupRef.current !== lookupId) return;
setSlDeletePreview({
status: 'loaded',
item,
dependentViewCount: json.count ?? 0,
dependentViewNames: (json.result ?? []).map(
(view: { table_name: string }) => view.table_name,
),
});
})
.catch(() => {
if (slDeleteLookupRef.current !== lookupId) return;
setSlDeletePreview({ status: 'failed', item });
});
}, []);
function handleDatabaseDelete(database: DatabaseObject) {
const { id, database_name: dbName } = database;
SupersetClient.delete({
@@ -566,7 +688,7 @@ function DatabaseList({
() => {
refreshData();
addSuccessToast(t('Deleted: %s', item.database_name));
setSlCurrentlyDeleting(null);
setSlDeletePreview(null);
},
createErrorHandler(errMsg =>
addDangerToast(
@@ -677,15 +799,29 @@ function DatabaseList({
if (isSemanticLayer) {
if (!canEdit && !canDelete) return null;
const isLoadingDependents =
slDeletePreview?.status === 'loading' &&
slDeletePreview.item.uuid === original.uuid;
return (
<div className="actions">
{canDelete && (
<ActionButton
label={t('Delete')}
tooltip={t('Delete')}
tooltip={
isLoadingDependents
? t('Loading dependent semantic views')
: t('Delete')
}
placement="bottom"
icon={<Icons.DeleteOutlined iconSize="l" />}
onClick={() => setSlCurrentlyDeleting(original)}
icon={
isLoadingDependents ? (
<Icons.LoadingOutlined iconSize="l" spin />
) : (
<Icons.DeleteOutlined iconSize="l" />
)
}
disabled={isLoadingDependents}
onClick={() => openSemanticLayerDeleteModal(original)}
/>
)}
{canEdit && (
@@ -781,6 +917,8 @@ function DatabaseList({
handleDatabaseExport,
handleDatabasePermSync,
openDatabaseDeleteModal,
openSemanticLayerDeleteModal,
slDeletePreview,
],
);
@@ -932,20 +1070,21 @@ function DatabaseList({
addSuccessToast={addSuccessToast}
semanticLayerUuid={slCurrentlyEditing ?? undefined}
/>
{slCurrentlyDeleting && (
{slDeletePreview && slDeletePreview.status !== 'loading' && (
<DeleteModal
description={
<p>
{t('Are you sure you want to delete')}{' '}
<b>{slCurrentlyDeleting.database_name}</b>?
</p>
<>
<p>
{t('Are you sure you want to delete')}{' '}
<b>{slDeletePreview.item.database_name}</b>?
</p>
<SemanticLayerCascadeWarning preview={slDeletePreview} />
</>
}
onConfirm={() => {
if (slCurrentlyDeleting) {
handleSemanticLayerDelete(slCurrentlyDeleting);
}
handleSemanticLayerDelete(slDeletePreview.item);
}}
onHide={() => setSlCurrentlyDeleting(null)}
onHide={() => setSlDeletePreview(null)}
open
title={
<ModalTitleWithIcon
@@ -182,7 +182,7 @@ test('With sql role - renders all panels on the page on page load', async () =>
test('With sql role - renders distinct recent activities', async () => {
await renderWelcome();
const recentPanel = screen.getByRole('button', { name: 'collapsed Recents' });
const recentPanel = screen.getByRole('button', { name: 'Recents' });
userEvent.click(recentPanel);
await waitFor(() =>
expect(
@@ -17,6 +17,7 @@
* under the License.
*/
import domToImage from 'dom-to-image-more';
import { getInstanceByDom } from 'echarts/core';
import { addWarningToast } from 'src/components/MessageToasts/actions';
import downloadAsImageOptimized, {
waitForStableScrollHeight,
@@ -27,6 +28,11 @@ jest.mock('dom-to-image-more', () => ({
default: { toJpeg: jest.fn(), toPng: jest.fn() },
}));
jest.mock('echarts/core', () => ({
__esModule: true,
getInstanceByDom: jest.fn(),
}));
jest.mock('src/components/MessageToasts/actions', () => ({
addWarningToast: jest.fn(),
}));
@@ -38,6 +44,7 @@ jest.mock('@apache-superset/core/translation', () => ({
const mockToJpeg = domToImage.toJpeg as jest.Mock;
const mockToPng = domToImage.toPng as jest.Mock;
const mockAddWarningToast = addWarningToast as jest.Mock;
const mockGetInstanceByDom = getInstanceByDom as jest.Mock;
// document.fonts.ready is not implemented in jsdom; provide a resolved promise
Object.defineProperty(document, 'fonts', {
@@ -81,6 +88,9 @@ function attachMockApi(
beforeEach(() => {
jest.clearAllMocks();
// clearAllMocks does not clear a mockReturnValue, so reset the instance lookup explicitly to
// stop a return value leaking into any clone-path test added after the ECharts ones below.
mockGetInstanceByDom.mockReset();
mockToJpeg.mockResolvedValue('data:image/jpeg;base64,test');
mockToPng.mockResolvedValue('data:image/png;base64,test');
});
@@ -734,3 +744,227 @@ test('clone path falls back to white background when theme is absent', async ()
document.body.removeChild(container);
});
// jsdom does not implement HTMLCanvasElement.getContext, so stub a minimal 2d context.
function stubCanvasContext() {
const drawImage = jest.fn();
const spy = jest
.spyOn(HTMLCanvasElement.prototype, 'getContext')
.mockReturnValue({ drawImage } as unknown as CanvasRenderingContext2D);
return { drawImage, restore: () => spy.mockRestore() };
}
test('re-renders an ECharts canvas at PNG_SCALE pixel ratio so the export is crisp', async () => {
const { restore } = stubCanvasContext();
const container = document.createElement('div');
const host = document.createElement('div');
host.className = 'echarts-host';
const canvas = document.createElement('canvas');
// on-screen backing store: CSS 400×300 at devicePixelRatio 1
canvas.width = 400;
canvas.height = 300;
host.appendChild(canvas);
container.appendChild(host);
document.body.appendChild(container);
// Fake ECharts instance whose renderToCanvas returns a 2× (high-res) canvas
const hiRes = document.createElement('canvas');
hiRes.width = 800;
hiRes.height = 600;
let renderOpts: Record<string, unknown> | undefined;
const renderToCanvas = jest.fn((opts?: Record<string, unknown>) => {
renderOpts = opts;
return hiRes;
});
mockGetInstanceByDom.mockReturnValue({ renderToCanvas });
// Capture the cloned canvas backing store handed to dom-to-image
let clonedCanvasWidth: number | undefined;
let clonedCanvasHeight: number | undefined;
mockToPng.mockImplementation((cloneRoot: HTMLElement) => {
const c = cloneRoot.querySelector('canvas');
clonedCanvasWidth = c?.width;
clonedCanvasHeight = c?.height;
return Promise.resolve('data:image/png;base64,test');
});
const handler = downloadAsImageOptimized(
'div',
'Sunburst',
false,
undefined,
{ format: 'png' },
);
await handler(syntheticEventFor(container));
// Instance recovered from the canvas's echarts-host ancestor...
expect(mockGetInstanceByDom).toHaveBeenCalledWith(host);
// ...and re-rendered at PNG_SCALE (2). No backgroundColor is forced, so the chart keeps its
// own configured background (matching the on-screen canvas).
expect(renderToCanvas).toHaveBeenCalled();
expect(renderOpts).toEqual(expect.objectContaining({ pixelRatio: 2 }));
expect(renderOpts).not.toHaveProperty('backgroundColor');
// The cloned canvas dom-to-image serializes is the 2× high-res source
expect(clonedCanvasWidth).toBe(800);
expect(clonedCanvasHeight).toBe(600);
expect(mockToPng).toHaveBeenCalled();
restore();
document.body.removeChild(container);
});
test('preserves a non-ECharts canvas at its on-screen resolution (no re-render)', async () => {
const { drawImage, restore } = stubCanvasContext();
const container = document.createElement('div');
const canvas = document.createElement('canvas');
canvas.width = 400;
canvas.height = 300;
container.appendChild(canvas);
document.body.appendChild(container);
let clonedCanvasWidth: number | undefined;
mockToPng.mockImplementation((cloneRoot: HTMLElement) => {
clonedCanvasWidth = cloneRoot.querySelector('canvas')?.width;
return Promise.resolve('data:image/png;base64,test');
});
const handler = downloadAsImageOptimized(
'div',
'Deck Chart',
false,
undefined,
{ format: 'png' },
);
await handler(syntheticEventFor(container));
// No echarts-host ancestor → echarts is never imported/consulted and the on-screen bitmap is
// copied 1:1.
expect(mockGetInstanceByDom).not.toHaveBeenCalled();
expect(clonedCanvasWidth).toBe(400);
expect(drawImage).toHaveBeenCalled();
restore();
document.body.removeChild(container);
});
test('falls back to a 1:1 copy when the ECharts instance is gone (getInstanceByDom returns undefined)', async () => {
const { drawImage, restore } = stubCanvasContext();
// Disposed / not-yet-initialised chart: the host is in the DOM but has no live instance.
mockGetInstanceByDom.mockReturnValue(undefined);
const container = document.createElement('div');
const host = document.createElement('div');
host.className = 'echarts-host';
const canvas = document.createElement('canvas');
canvas.width = 400;
canvas.height = 300;
host.appendChild(canvas);
container.appendChild(host);
document.body.appendChild(container);
let clonedCanvasWidth: number | undefined;
mockToPng.mockImplementation((cloneRoot: HTMLElement) => {
clonedCanvasWidth = cloneRoot.querySelector('canvas')?.width;
return Promise.resolve('data:image/png;base64,test');
});
const handler = downloadAsImageOptimized(
'div',
'Sunburst',
false,
undefined,
{ format: 'png' },
);
await handler(syntheticEventFor(container));
expect(mockGetInstanceByDom).toHaveBeenCalledWith(host);
// No instance → the on-screen bitmap is copied 1:1 and the export still completes
expect(clonedCanvasWidth).toBe(400);
expect(drawImage).toHaveBeenCalled();
expect(mockToPng).toHaveBeenCalled();
expect(mockAddWarningToast).not.toHaveBeenCalled();
restore();
document.body.removeChild(container);
});
test('falls back to a 1:1 copy (and still exports) when renderToCanvas throws', async () => {
const { drawImage, restore } = stubCanvasContext();
// A valid but unhealthy instance (mid-dispose, errored chart) whose re-render throws.
const renderToCanvas = jest.fn(() => {
throw new Error('chart is disposing');
});
mockGetInstanceByDom.mockReturnValue({ renderToCanvas });
const container = document.createElement('div');
const host = document.createElement('div');
host.className = 'echarts-host';
const canvas = document.createElement('canvas');
canvas.width = 400;
canvas.height = 300;
host.appendChild(canvas);
container.appendChild(host);
document.body.appendChild(container);
let clonedCanvasWidth: number | undefined;
mockToPng.mockImplementation((cloneRoot: HTMLElement) => {
clonedCanvasWidth = cloneRoot.querySelector('canvas')?.width;
return Promise.resolve('data:image/png;base64,test');
});
const handler = downloadAsImageOptimized(
'div',
'Sunburst',
false,
undefined,
{ format: 'png' },
);
await handler(syntheticEventFor(container));
// The throw is swallowed per-canvas: the export completes via the on-screen 1:1 copy rather
// than aborting the whole capture.
expect(renderToCanvas).toHaveBeenCalled();
expect(clonedCanvasWidth).toBe(400);
expect(drawImage).toHaveBeenCalled();
expect(mockToPng).toHaveBeenCalled();
expect(mockAddWarningToast).not.toHaveBeenCalled();
restore();
document.body.removeChild(container);
});
test('re-renders an ECharts host only once when it owns multiple canvas layers', async () => {
const { restore } = stubCanvasContext();
const container = document.createElement('div');
const host = document.createElement('div');
host.className = 'echarts-host';
// ECharts may add a second <canvas> for a hover/progressive layer
host.appendChild(document.createElement('canvas'));
host.appendChild(document.createElement('canvas'));
container.appendChild(host);
document.body.appendChild(container);
const hiRes = document.createElement('canvas');
hiRes.width = 800;
hiRes.height = 600;
const renderToCanvas = jest.fn(() => hiRes);
mockGetInstanceByDom.mockReturnValue({ renderToCanvas });
const handler = downloadAsImageOptimized(
'div',
'Sunburst',
false,
undefined,
{ format: 'png' },
);
await handler(syntheticEventFor(container));
// Both canvases resolve to the same instance; the flattened render happens once
expect(renderToCanvas).toHaveBeenCalledTimes(1);
restore();
document.body.removeChild(container);
});
@@ -27,10 +27,23 @@ import { forceLoadAllCharts, restoreVirtualization } from './downloadUtils';
const IMAGE_DOWNLOAD_QUALITY = 0.95;
const PNG_SCALE = 2; // Higher quality for PNG
// ECharts canvas charts (e.g. sunburst) bake their pixel detail into the on-screen backing
// store (CSS size × devicePixelRatio). Copying that 1:1 and then letting the PNG path upscale
// it via transform: scale(PNG_SCALE) only stretches the bitmap, producing a blurry export. To
// keep the export crisp, these charts are re-rendered at this pixel ratio at capture time; it is
// tied to PNG_SCALE so the re-render matches the scaled output box.
const EXPORT_CANVAS_PIXEL_RATIO = PNG_SCALE;
// The div passed to ECharts `init()` carries this class (source of truth:
// plugins/plugin-chart-echarts/src/components/Echart.tsx `ECHARTS_HOST_CLASS`). It lets the
// exporter recover the live ECharts instance for a canvas via `getInstanceByDom`.
const ECHARTS_HOST_CLASS = 'echarts-host';
export type BackgroundType = 'transparent' | 'solid';
const TRANSPARENT_RGBA = 'transparent';
const POLL_INTERVAL_MS = 100;
// Resolved lazily via a dynamic import so echarts stays out of the core bundle.
type EChartsGetInstanceByDom = typeof import('echarts/core').getInstanceByDom;
// Tracks original cell styles to restore after capture
type CellFixup = { el: HTMLElement; minHeight: string; overflow: string };
@@ -223,30 +236,73 @@ const processCloneForVisibility = (clone: HTMLElement) => {
});
};
const preserveCanvasContent = (original: Element, clone: Element) => {
const preserveCanvasContent = (
original: Element,
clone: Element,
getInstanceByDom?: EChartsGetInstanceByDom,
) => {
const originalCanvases = original.querySelectorAll('canvas');
const clonedCanvases = clone.querySelectorAll('canvas');
// `renderToCanvas` flattens all of an ECharts instance's zrender layers into a single canvas,
// so once a host is re-rendered its other <canvas> layers (e.g. a hover layer) are skipped;
// if the re-render throws, the host is marked 'failed' so each layer falls back to a 1:1 copy.
const hostRenderState = new Map<Element, 'rendered' | 'failed'>();
originalCanvases.forEach((originalCanvas, i) => {
if (originalCanvases[i] && clonedCanvases[i]) {
const clonedCanvas = clonedCanvases[i] as HTMLCanvasElement;
const ctx = clonedCanvas.getContext('2d');
if (ctx) {
clonedCanvas.width = originalCanvas.width;
clonedCanvas.height = originalCanvas.height;
ctx.drawImage(originalCanvas, 0, 0);
const clonedCanvas = clonedCanvases[i] as HTMLCanvasElement | undefined;
if (!clonedCanvas) return;
const ctx = clonedCanvas.getContext('2d');
if (!ctx) return;
// For ECharts (canvas renderer) charts such as sunburst, re-render the chart at a higher
// pixel ratio instead of copying the on-screen bitmap, so the PNG path upscales a matching
// high-resolution source rather than stretching a low-resolution one. `getInstanceByDom` is
// only supplied on the PNG path, so JPEG (and non-ECharts canvases) keep the 1:1 copy below.
const host = originalCanvas.closest(`.${ECHARTS_HOST_CLASS}`);
const hostState = host ? hostRenderState.get(host) : undefined;
// Sibling layer of a host already re-rendered: the flattened render covers it.
if (hostState === 'rendered') return;
const instance =
host && hostState !== 'failed'
? getInstanceByDom?.(host as HTMLElement)
: undefined;
if (host && instance) {
try {
// No `backgroundColor` is passed, so renderToCanvas inherits the chart's own configured
// background (transparent when unset) — matching the on-screen canvas. The overall export
// background is applied separately via the dom-to-image `bgcolor` option.
const hiResCanvas = instance.renderToCanvas({
pixelRatio: EXPORT_CANVAS_PIXEL_RATIO,
});
clonedCanvas.width = hiResCanvas.width;
clonedCanvas.height = hiResCanvas.height;
ctx.drawImage(hiResCanvas, 0, 0);
hostRenderState.set(host, 'rendered');
return;
} catch {
// A valid but unhealthy instance (mid-dispose, errored chart) can throw. Mark the host
// 'failed' and fall back to the on-screen 1:1 copy below so a single bad chart can never
// abort the whole export (e.g. a 20-chart dashboard capture).
hostRenderState.set(host, 'failed');
}
}
// Non-ECharts canvases (deck.gl/WebGL and friends), and the fallback when a re-render is
// unavailable or threw: preserve the on-screen bitmap as-is.
clonedCanvas.width = originalCanvas.width;
clonedCanvas.height = originalCanvas.height;
ctx.drawImage(originalCanvas, 0, 0);
});
};
const createEnhancedClone = (
originalElement: Element,
theme?: SupersetTheme,
getInstanceByDom?: EChartsGetInstanceByDom,
): { clone: HTMLElement; cleanup: () => void } => {
const clone = originalElement.cloneNode(true) as HTMLElement;
copyAllComputedStyles(originalElement, clone, theme);
preserveCanvasContent(originalElement, clone);
preserveCanvasContent(originalElement, clone, getInstanceByDom);
const tempContainer = document.createElement('div');
tempContainer.style.cssText = `
@@ -504,10 +560,26 @@ export default function downloadAsImageOptimized(
// All other chart types: use the clone-based approach
let cleanup: (() => void) | null = null;
// Only the PNG path upscales the layout (transform: scale(PNG_SCALE)), so only there does a
// higher-resolution canvas help; JPEG keeps the throw-free 1:1 copy. Re-render ECharts
// (canvas renderer) charts, e.g. sunburst, at a higher pixel ratio so the upscale samples a
// matching source instead of stretching the on-screen bitmap. echarts is pulled in lazily —
// only when a chart is actually present — so it stays out of the core bundle; if the import
// fails the canvases fall back to a 1:1 copy.
let getInstanceByDom: EChartsGetInstanceByDom | undefined;
if (isPng && elementToPrint.querySelector(`.${ECHARTS_HOST_CLASS}`)) {
try {
({ getInstanceByDom } = await import('echarts/core'));
} catch {
// echarts not available in this context; canvases keep their on-screen resolution.
}
}
try {
const { clone, cleanup: cleanupFn } = createEnhancedClone(
elementToPrint,
theme,
getInstanceByDom,
);
cleanup = cleanupFn;
+282 -344
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -37,11 +37,11 @@
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.11.0",
"oxfmt": "^0.63.0",
"oxfmt": "^0.64.0",
"tscw-config": "^1.1.2",
"typescript": "^6.0.3",
"typescript-eslint": "^8.67.0",
"vitest": "^4.1.10"
"vitest": "^4.1.11"
},
"engines": {
"node": "^24.16.0",
-36
View File
@@ -52,42 +52,6 @@ class DatabaseRequiredFieldValidationError(ValidationError):
)
class DatabaseExtraJSONValidationError(ValidationError):
"""
Marshmallow validation error for database encrypted extra must be a valid JSON
"""
def __init__(self, json_error: str = "") -> None:
super().__init__(
[
_(
"Field cannot be decoded by JSON. %(json_error)s",
json_error=json_error,
)
],
field_name="extra",
)
class DatabaseExtraValidationError(ValidationError):
"""
Marshmallow validation error for database encrypted extra must be a valid JSON
"""
def __init__(self, key: str = "") -> None:
super().__init__(
[
_(
"The metadata_params in Extra field "
"is not configured correctly. The key "
"%{key}s is invalid.",
key=key,
)
],
field_name="extra",
)
class DatabaseConnectionSyncPermissionsError(CommandException):
status = 500
message = _("Unable to sync permissions for this database connection.")
+99 -23
View File
@@ -24,8 +24,10 @@ purge rolls back. The record is written ``pending`` *before* the purge and
flipped to ``confirmed`` *after* it commits, so a crash leaves at most a
``pending`` row, never a missing one. ``pending`` rows are reconciled on the
next run. Completed records are immutable. Consecutive scheduled evaluations
that remain blocked may discard only their current redundant provisional row;
force-purge and other meaningful outcomes are retained independently.
that remain blocked for the same reason may discard only their current
redundant provisional row a reason change retains one new row carrying the
new code; force-purge and other meaningful outcomes are retained
independently.
The dedicated ``purge_audit_log`` table is content-free (no name or PII; only
action, actor, UTC time, entity type, UUID, and affected referrers) and is never
@@ -87,6 +89,10 @@ class _AuditRecoverySnapshot:
entity_type: str
entity_uuid: str | None
created_on: datetime
# Sourced from the finalization call's reason argument, never from the
# row: pending rows are reason-less by design, so reading the row here
# would silently record NULL.
reason: str | None
def _utc_now() -> datetime:
@@ -136,8 +142,18 @@ def write_ahead(
session.close()
def finalize(record_id: UUID | None, status: str, **details: Any) -> None:
"""Finalize a pending attempt on the dedicated audit session."""
def finalize(
record_id: UUID | None,
status: str,
*,
reason: str | None = None,
**details: Any,
) -> None:
"""Finalize a pending attempt on the dedicated audit session.
``reason`` is persisted only for blocked outcomes; the audit records a
cause for a purge that did not happen, never for one that did.
"""
if record_id is None:
return
session = _dedicated_session()
@@ -148,6 +164,8 @@ def finalize(record_id: UUID | None, status: str, **details: Any) -> None:
referrers = details.get("affected_referrers")
if referrers:
values["affected_referrers"] = ",".join(referrers)
if reason is not None and status == STATUS_BLOCKED:
values["reason"] = reason
removed_dashboard_slices = details.get("removed_dashboard_slices")
if removed_dashboard_slices is not None:
values["removed_dashboard_slices"] = removed_dashboard_slices
@@ -192,12 +210,21 @@ def fail(record_id: UUID | None) -> None:
finalize(record_id, STATUS_FAILED)
def block(record_id: UUID | None) -> None:
"""Mark an attempt blocked by ordinary deletion policy."""
finalize(record_id, STATUS_BLOCKED)
def block(record_id: UUID | None, reason: str | None) -> None:
"""Mark an attempt blocked by ordinary deletion policy.
``reason`` is a stable machine code from the closed ``REASON_*``
vocabulary in :mod:`superset.commands.deletion_retention.purge_policy`.
It is required by signature (every blocked outcome has a classified
cause); ``None`` is tolerated defensively so a threading gap can never
block a purge, and leaves the persisted reason NULL.
"""
finalize(record_id, STATUS_BLOCKED, reason=reason)
def _capture_recovery_snapshot(record: PurgeAuditLog) -> _AuditRecoverySnapshot:
def _capture_recovery_snapshot(
record: PurgeAuditLog, reason: str | None
) -> _AuditRecoverySnapshot:
"""Capture the content-free fields needed for fail-safe recovery."""
return _AuditRecoverySnapshot(
id=cast(UUID, record.id),
@@ -205,26 +232,37 @@ def _capture_recovery_snapshot(record: PurgeAuditLog) -> _AuditRecoverySnapshot:
entity_type=str(record.entity_type),
entity_uuid=record.entity_uuid,
created_on=cast(datetime, record.created_on),
reason=reason,
)
def _retention_predecessor(
session: Session, current: PurgeAuditLog
) -> PurgeAuditLog | None:
"""Return the latest row that could unambiguously precede ``current``."""
"""Return the latest row that could unambiguously precede ``current``.
The latest same-entity retention row by ``created_on`` deliberately not
bounded by ``current.created_on``. If another visible row has a later
timestamp, it surfaces here so the caller's strictly-older check retains
the current row. Timestamps provide database ordering for this predicate,
not causal ordering across workers.
"""
predecessor: PurgeAuditLog | None = session.execute(
sa.select(PurgeAuditLog)
.where(PurgeAuditLog.entity_uuid == current.entity_uuid)
.where(PurgeAuditLog.entity_type == current.entity_type)
.where(PurgeAuditLog.trigger == TRIGGER_RETENTION)
.where(PurgeAuditLog.created_on <= current.created_on)
.where(PurgeAuditLog.id != current.id)
.order_by(PurgeAuditLog.created_on.desc())
.limit(1)
).scalar_one_or_none()
if predecessor is None:
return predecessor
tied_mixed_status_exists: bool = session.execute(
# A timestamp-tied row differing in status OR reason makes the
# predecessor ambiguous. ``is_distinct_from`` keeps the reason
# comparison NULL-safe on all supported dialects (reason is nullable;
# status is not).
tied_mixed_exists: bool = session.execute(
sa.select(
sa.exists().where(
PurgeAuditLog.entity_uuid == current.entity_uuid,
@@ -232,23 +270,43 @@ def _retention_predecessor(
PurgeAuditLog.trigger == TRIGGER_RETENTION,
PurgeAuditLog.created_on == predecessor.created_on,
PurgeAuditLog.id != current.id,
PurgeAuditLog.status != predecessor.status,
sa.or_(
PurgeAuditLog.status != predecessor.status,
PurgeAuditLog.reason.is_distinct_from(predecessor.reason),
),
)
)
).scalar_one()
if tied_mixed_status_exists:
if tied_mixed_exists:
return None
return predecessor
def _suppress_redundant_block(
session: Session, current: PurgeAuditLog, predecessor: PurgeAuditLog | None
session: Session,
current: PurgeAuditLog,
predecessor: PurgeAuditLog | None,
reason: str | None,
) -> bool:
"""Delete only a pending row with a strictly older blocked predecessor."""
"""Delete a pending row only against a strictly older same-reason block."""
if not reason:
# Fail safe on a threading gap: a missing current code must retain
# the row (and be visible), never silently revive the status-only
# predicate.
logger.warning(
"deletion_retention: blocked audit row %s has no reason code; "
"refusing suppression",
current.id,
)
return False
if (
predecessor is None
or predecessor.created_on >= current.created_on
or predecessor.status != STATUS_BLOCKED
# A reason-less (pre-feature) predecessor never matches: the first
# post-upgrade block of a long-blocked entity is retained once and
# becomes the new suppression anchor.
or predecessor.reason != reason
):
return False
deleted_rows: int | None = session.execute(
@@ -264,7 +322,7 @@ def _suppress_redundant_block(
return deleted_rows == 1
def _retain_blocked(session: Session, record_id: UUID) -> None:
def _retain_blocked(session: Session, record_id: UUID, reason: str | None) -> None:
"""Conditionally retain the current provisional row as blocked."""
session.execute(
sa.update(PurgeAuditLog.__table__)
@@ -272,7 +330,7 @@ def _retain_blocked(session: Session, record_id: UUID) -> None:
PurgeAuditLog.__table__.c.id == record_id,
PurgeAuditLog.__table__.c.status == STATUS_PENDING,
)
.values(status=STATUS_BLOCKED, removed_dashboard_slices=0)
.values(status=STATUS_BLOCKED, removed_dashboard_slices=0, reason=reason)
)
@@ -285,7 +343,11 @@ def _recover_retention_blocked(
current: PurgeAuditLog | None = recovery_session.get(PurgeAuditLog, record_id)
if current is not None:
if current.status == STATUS_PENDING:
_retain_blocked(recovery_session, record_id)
_retain_blocked(
recovery_session,
record_id,
snapshot.reason if snapshot else None,
)
recovery_session.commit()
return "fallback"
if snapshot is None:
@@ -300,6 +362,7 @@ def _recover_retention_blocked(
entity_uuid=snapshot.entity_uuid,
removed_dashboard_slices=0,
created_on=snapshot.created_on,
reason=snapshot.reason,
)
)
recovery_session.commit()
@@ -319,9 +382,14 @@ def _recover_retention_blocked(
def finalize_retention_blocked(
record_id: UUID | None,
record_id: UUID | None, reason: str | None
) -> RetentionBlockedDisposition:
"""Finalize a scheduled blocker, suppressing only proven redundant evidence."""
"""Finalize a scheduled blocker, suppressing only proven redundant evidence.
``reason`` is the stable machine code for the block (see
:func:`block`); it is persisted on retained rows and captured in the
snapshot used by the crash-recovery path.
"""
if record_id is None:
return "fallback"
session: Session = _dedicated_session()
@@ -330,15 +398,17 @@ def finalize_retention_blocked(
current: PurgeAuditLog | None = session.get(PurgeAuditLog, record_id)
if current is None:
return "fallback"
snapshot = _capture_recovery_snapshot(current)
snapshot = _capture_recovery_snapshot(current, reason)
if current.status != STATUS_PENDING or current.trigger != TRIGGER_RETENTION:
return "retained"
predecessor: PurgeAuditLog | None = None
if current.entity_uuid is not None:
predecessor = _retention_predecessor(session, current)
suppressed: bool = _suppress_redundant_block(session, current, predecessor)
suppressed: bool = _suppress_redundant_block(
session, current, predecessor, reason
)
if not suppressed:
_retain_blocked(session, record_id)
_retain_blocked(session, record_id, reason)
session.commit()
return "suppressed" if suppressed else "retained"
except SQLAlchemyError:
@@ -383,6 +453,12 @@ def reconcile_pending(stale_before: datetime | None = None) -> dict[str, int]:
``confirmed``. A surviving or unresolvable entity means the attempt did
not durably purge it and is finalized as failed; normal selection may
retry.
An attempt that had already decided ``blocked`` when its worker died is
indistinguishable here from any other stalled attempt, so it reconciles
as failed with no reason: pending rows carry no reason by design, and
inventing one would assert evidence this process never witnessed. The
next scheduled attempt re-anchors the entity with its real code.
"""
cutoff = stale_before or _utc_now() - _PENDING_STALE_AFTER
reconciled = absent = failed = 0
@@ -183,7 +183,7 @@ class ForcePurgeCommand:
removed_dashboard_slices=result.removed_dashboard_slices,
)
elif result.blocked_reason is not None:
audit.block(record_id)
audit.block(record_id, result.blocker.code if result.blocker else None)
else:
audit.fail(record_id)
if result.purged:
@@ -53,9 +53,11 @@ from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from superset.commands.deletion_retention.purge_policy import (
BlockerReason,
get_purge_policy,
PurgeBlockedError,
PurgeEntityPolicy,
REASON_CASCADE_INTEGRITY_FAILURE,
)
logger: logging.Logger = logging.getLogger(__name__)
@@ -144,7 +146,17 @@ class CascadeResult:
dangling_chart_uuids: list[str] = field(default_factory=list)
removed_dashboard_slices: int = 0
version_rows_removed: int = 0
blocked_reason: str | None = None
blocker: BlockerReason | None = None
@property
def blocked_reason(self) -> str | None:
"""Return the operator-facing blocker phrase, if the purge was blocked."""
return self.blocker.phrase if self.blocker else None
@property
def blocked_reason_code(self) -> str | None:
"""Return the stable audit code, if the purge was blocked."""
return self.blocker.code if self.blocker else None
class PurgeRaceLostError(Exception):
@@ -267,20 +279,21 @@ def cascade_hard_delete(
purged=False,
entity_type=entity_type,
entity_uuid=uuid,
blocked_reason=str(ex),
blocker=ex.reason,
)
except IntegrityError as ex:
# Not a policy decision: a restrictive FK the cascade did not handle.
# Not a policy decision: a database integrity constraint failed.
# Two audiences, two messages. The curated reason goes to the caller
# (and from there into a user toast), because raw driver text carries
# the failing SQL and bind parameters. The constraint detail goes to
# the log at WARNING, because an entity permanently unpurgeable via an
# unknown FK is a cascade-coverage bug someone has to be able to
# diagnose -- reported at INFO as a policy block, it read as intended
# behaviour.
# the log at WARNING, because an entity permanently unpurgeable after
# an integrity failure represents a cascade defect someone has to be
# able to diagnose. The stable audit code identifies this as an unexpected
# cascade failure rather than intended policy behavior without
# claiming which kind of constraint the database reported.
logger.warning(
"deletion_retention: %s id=%s purge failed on a restrictive "
"foreign key the cascade does not handle: %s",
"deletion_retention: %s id=%s purge failed on a database "
"integrity constraint: %s",
entity_type,
entity_id,
ex,
@@ -289,7 +302,10 @@ def cascade_hard_delete(
purged=False,
entity_type=entity_type,
entity_uuid=uuid,
blocked_reason="blocked by database references",
blocker=BlockerReason(
REASON_CASCADE_INTEGRITY_FAILURE,
"cascade blocked by a database integrity constraint",
),
)
return CascadeResult(
@@ -24,7 +24,7 @@ from dataclasses import dataclass
from enum import Enum
from functools import lru_cache
from types import MappingProxyType
from typing import Any, cast
from typing import Any, cast, NamedTuple
import sqlalchemy as sa
from sqlalchemy.orm import Mapper, Session
@@ -36,10 +36,44 @@ from superset.utils.sqlalchemy_events import (
logger: logging.Logger = logging.getLogger(__name__)
# Stable machine-readable reason codes persisted on purge audit records.
# The values are frozen identifiers pinned by a golden-set test: they equal
# the related-table names at introduction by coincidence, never by derivation,
# so a physical table rename changes only the blocker mapping's key and
# leaves the persisted code untouched — audit history and the suppression
# predicate compare these literals.
REASON_REPORT_SCHEDULE: str = "report_schedule"
REASON_USER_ATTRIBUTE: str = "user_attribute"
REASON_CASCADE_INTEGRITY_FAILURE: str = "cascade_integrity_failure"
ALL_REASON_CODES: frozenset[str] = frozenset(
{
REASON_REPORT_SCHEDULE,
REASON_USER_ATTRIBUTE,
REASON_CASCADE_INTEGRITY_FAILURE,
}
)
class BlockerReason(NamedTuple):
"""One blocker's persisted audit code paired with its operator phrase."""
code: str
phrase: str
class PurgeBlockedError(Exception):
"""Raised when ordinary deletion policy forbids purging an entity."""
def __init__(self, reason: BlockerReason) -> None:
super().__init__(reason.phrase)
self.reason: BlockerReason = reason
@property
def reason_code(self) -> str:
"""Return the stable machine-readable blocker code."""
return self.reason.code
class DependencyClassification(str, Enum):
"""Describe how purge treats a persistence dependency."""
@@ -106,11 +140,21 @@ class DependencyPolicy:
key: DependencyKey
classification: DependencyClassification
phase: ExecutionPhase | None = None
blocked_reason: str | None = None
blocker: BlockerReason | None = None
optional_listener: bool = False
listener_action: ListenerAction | None = None
version_column: str | None = None
@property
def blocked_reason(self) -> str | None:
"""Return the operator-facing blocker phrase, if this policy blocks."""
return self.blocker.phrase if self.blocker else None
@property
def blocked_reason_code(self) -> str | None:
"""Return the stable audit code, if this policy blocks."""
return self.blocker.code if self.blocker else None
@dataclass(frozen=True)
class PurgeEntityPolicy:
@@ -416,7 +460,7 @@ def purge_policy_registry() -> Mapping[type[Any], PurgeEntityPolicy]:
keys: tuple[DependencyKey, ...],
classifications: tuple[DependencyClassification, ...],
synthetic: tuple[DependencyPolicy, ...],
blocked_reasons: Mapping[str, str] = MappingProxyType({}),
blocked_reasons: Mapping[str, BlockerReason] = MappingProxyType({}),
version_columns: Mapping[str, str] = MappingProxyType({}),
) -> tuple[DependencyPolicy, ...]:
phases: dict[DependencyClassification, ExecutionPhase | None] = {
@@ -428,15 +472,22 @@ def purge_policy_registry() -> Mapping[type[Any], PurgeEntityPolicy]:
}
if len(keys) != len(classifications):
raise ValueError("Every dependency key requires one classification")
def declare(
key: DependencyKey, classification: DependencyClassification
) -> DependencyPolicy:
blocker: BlockerReason | None = blocked_reasons.get(key.related_table)
return DependencyPolicy(
key,
classification,
phases[classification],
blocker=blocker,
version_column=version_columns.get(key.related_table),
)
return (
tuple(
DependencyPolicy(
key,
classification,
phases[classification],
blocked_reason=blocked_reasons.get(key.related_table),
version_column=version_columns.get(key.related_table),
)
declare(key, classification)
for key, classification in zip(keys, classifications, strict=True)
)
+ synthetic
@@ -539,7 +590,12 @@ def purge_policy_registry() -> Mapping[type[Any], PurgeEntityPolicy]:
DependencyClassification.PRESERVE,
),
(tag_cleanup, chart_membership_versions),
{"report_schedule": "associated alerts or reports exist"},
# Keyed by related table; the audit code is declared, not derived.
{
"report_schedule": BlockerReason(
REASON_REPORT_SCHEDULE, "associated alerts or reports exist"
)
},
{"slices_version": "id"},
),
validate=validate_deletion_allowed,
@@ -687,10 +743,16 @@ def purge_policy_registry() -> Mapping[type[Any], PurgeEntityPolicy]:
DependencyClassification.PRESERVE,
),
(tag_cleanup, dashboard_membership_versions),
# Keyed by related table; the audit code is declared, not derived.
# Declaration order is part of the audit contract: the first
# matching blocker's code is the one recorded.
{
"report_schedule": "associated alerts or reports exist",
"user_attribute": (
"a user has this dashboard set as their welcome page"
"report_schedule": BlockerReason(
REASON_REPORT_SCHEDULE, "associated alerts or reports exist"
),
"user_attribute": BlockerReason(
REASON_USER_ATTRIBUTE,
"a user has this dashboard set as their welcome page",
),
},
{"dashboards_version": "id"},
@@ -893,9 +955,9 @@ def validate_deletion_allowed(
if session.execute(
sa.select(sa.literal(1)).select_from(table).where(*predicates).limit(1)
).first():
if dependency.blocked_reason is None:
if dependency.blocker is None:
raise RuntimeError(f"Missing blocker reason for {key.describe()}")
raise PurgeBlockedError(dependency.blocked_reason)
raise PurgeBlockedError(dependency.blocker)
def count_dashboard_slices(
+9 -2
View File
@@ -1620,6 +1620,9 @@ class BaseReportState:
(caller should ``return`` without re-raising), or False if the caller
should fall through to its own error handling path.
"""
if not feature_flag_manager.is_feature_enabled("ALERT_REPORTS_RETRY"):
return False
retry_on_failure: bool = self._report_schedule.retry_on_failure
if not retry_on_failure:
return False
@@ -1735,7 +1738,8 @@ class ReportNotTriggeredErrorState(BaseReportState):
# retry delay, consider the retry chain dead and let the new window
# proceed (e.g., apply_async failed after committing RETRYING).
if (
self._report_schedule.last_state == ReportState.RETRYING
feature_flag_manager.is_feature_enabled("ALERT_REPORTS_RETRY")
and self._report_schedule.last_state == ReportState.RETRYING
and self._is_retry_window_stale()
):
max_delay: int = app.config.get(
@@ -1767,6 +1771,8 @@ class ReportNotTriggeredErrorState(BaseReportState):
return
self.send()
# Clear any retry state from previous failed attempts in this window.
# Always reset on success regardless of feature flag — prevents
# stale counters from being reused if the flag is later re-enabled.
self._reset_retry_counter()
warning_message = (
";".join(self._execution_warnings) if self._execution_warnings else None
@@ -2056,7 +2062,8 @@ class ReportSuccessState(BaseReportState):
raise
# send() succeeded — clear retry state and log success. Any execution
# warnings are incorporated by create_log().
# warnings are incorporated by create_log(). Always reset regardless
# of feature flag to prevent stale counters.
self._reset_retry_counter()
self.update_report_schedule_and_log(ReportState.SUCCESS, error_message=None)
+18 -17
View File
@@ -22,7 +22,7 @@ from flask_appbuilder.models.sqla import Model
from flask_babel import gettext as _
from marshmallow import ValidationError
from superset import security_manager
from superset import is_feature_enabled, security_manager
from superset.commands.base import UpdateMixin
from superset.commands.report.base import BaseReportScheduleCommand
from superset.commands.report.exceptions import (
@@ -193,23 +193,24 @@ class UpdateReportScheduleCommand(UpdateMixin, BaseReportScheduleCommand):
include_viewers=False,
)
# Validate retry config: send_failed_reports requires retry_on_failure.
# Fall back to the existing DB value for fields not in the payload.
send_failed = self._properties.get(
"send_failed_reports", self._model.send_failed_reports
)
retry_enabled = self._properties.get(
"retry_on_failure", self._model.retry_on_failure
)
if send_failed and not retry_enabled:
msg = _("send_failed_reports requires retry_on_failure to be enabled")
exceptions.append(ValidationError({"send_failed_reports": [msg]}))
# Validate retry config when the feature is enabled.
if is_feature_enabled("ALERT_REPORTS_RETRY"):
# Fall back to the existing DB value for fields not in the payload.
send_failed = self._properties.get(
"send_failed_reports", self._model.send_failed_reports
)
retry_enabled = self._properties.get(
"retry_on_failure", self._model.retry_on_failure
)
if send_failed and not retry_enabled:
msg = _("send_failed_reports requires retry_on_failure to be enabled")
exceptions.append(ValidationError({"send_failed_reports": [msg]}))
# Retries are only supported for reports, not alerts.
report_type = self._properties.get("type", self._model.type)
if report_type == ReportScheduleType.ALERT and retry_enabled:
msg = _("Retries are not supported for alerts")
exceptions.append(ValidationError({"retry_on_failure": [msg]}))
# Retries are only supported for reports, not alerts.
report_type = self._properties.get("type", self._model.type)
if report_type == ReportScheduleType.ALERT and retry_enabled:
msg = _("Retries are not supported for alerts")
exceptions.append(ValidationError({"retry_on_failure": [msg]}))
if exceptions:
raise ReportScheduleInvalidError(exceptions=exceptions)
+12
View File
@@ -34,6 +34,7 @@ from superset.exceptions import (
from superset.models.sql_lab import Query
from superset.sqllab.utils import apply_display_max_row_configuration_if_require
from superset.utils import core as utils
from superset.utils.database import warm_and_release_connection
from superset.utils.dates import now_as_float
from superset.views.utils import _deserialize_results_payload
@@ -109,6 +110,17 @@ class SqlExecutionResultsCommand(BaseCommand):
status=400,
) from ex
# Release the DB connection back to the pool before the S3 fetch and the
# CPU-bound decompress/deserialize/expand work below: none of that needs
# the DB, and holding a connection for their duration (which can run well
# past this endpoint's client-side timeout for large results) is what
# exhausts the small per-worker SQLAlchemy pool when several large-result
# downloads land concurrently on the same gunicorn worker. `database` is
# warmed first since `_deserialize_results_payload` needs
# `self._query.database.db_engine_spec` later in `run()`, after the
# connection has been released.
warm_and_release_connection(self._query, "database")
# Now fetch results from backend (query exists, so this is a valid request)
read_from_results_backend_start = now_as_float()
self._blob = results_backend.get(self._key)
+2 -2
View File
@@ -138,8 +138,8 @@ class BaseRestoreVersionCommand(BaseCommand):
# With capture off, Continuum's write listeners are detached: a
# revert would mutate the live entity with NO new version row —
# a destructive, untracked write. The whole restore surface is
# therefore inert under the kill-switch, matching the read-side
# convention (404, indistinguishable from "no such version").
# therefore inert under the kill-switch (404, indistinguishable from
# "no such version"). Existing history remains readable.
if not capture_enabled():
raise self.not_found_exc()
entity = find_active_by_uuid(self.model_cls, self._uuid)
+10 -15
View File
@@ -708,7 +708,7 @@ DEFAULT_FEATURE_FLAGS: dict[str, bool] = {
# the move-back lever; removed (along with its two gate points —
# BaseDAO.delete routing and the do_orm_execute visibility listener) once
# post-flip confidence is established.
# @lifecycle: development
# @lifecycle: testing
"SOFT_DELETE": True,
# Enable semantic layers and show semantic views alongside datasets
# @lifecycle: development
@@ -742,9 +742,9 @@ DEFAULT_FEATURE_FLAGS: dict[str, bool] = {
"TAGGING_SYSTEM": False,
# Enables the version history panel on Explore and Dashboard pages.
# History only accrues while ``ENABLE_VERSIONING_CAPTURE`` is also on;
# with capture off the panel renders but stays empty, so the two ship
# with matching defaults and should be changed together.
# @lifecycle: development
# with capture off the panel renders empty or stale history, so the two
# ship with matching defaults and should be changed together.
# @lifecycle: testing
"VERSION_HISTORY": True,
# =================================================================
# IN TESTING
@@ -759,6 +759,9 @@ DEFAULT_FEATURE_FLAGS: dict[str, bool] = {
# @lifecycle: testing
# @docs: https://superset.apache.org/docs/configuration/alerts-reports
"ALERT_REPORTS": False,
# Enables automatic retry functionality for failed report executions
# @lifecycle: testing
"ALERT_REPORTS_RETRY": False,
# Enables Slack V2 integration for Alerts and Reports.
# Defaults to True; the legacy Slack v1 path is deprecated and will be removed
# in the next major release. Operators must grant the Slack bot both the
@@ -1694,17 +1697,9 @@ DATETIME_FORMAT_DETECTION_SAMPLE_SIZE = 1000
# The limit for the Superset Meta DB when the feature flag ENABLE_SUPERSET_META_DB is on
SUPERSET_META_DB_LIMIT: int | None = 1000
# Master switch for entity-version-history capture. Capture is enabled by
# default, so saves write shadow rows and a ``version_transaction`` /
# ``version_changes`` record. Set this to a falsy value in
# ``superset_config.py`` (or via the environment variable of the same name) to
# disable the before-flush listeners while keeping the /versions/ endpoints
# available read-only.
# Capture ships on. It is an operational escape hatch — set the environment
# variable to a falsy value when a versioning-induced regression needs a
# 30-second recovery instead of revert-and-redeploy — not a feature flag,
# and it remains permanently as the kill-switch rather than being removed
# with the rollout toggles.
# Master switch for entity-version-history capture. A falsy value disables
# version writes while keeping existing history available read-only through the
# ``/versions/`` endpoints; Restore is unavailable while capture is disabled.
ENABLE_VERSIONING_CAPTURE: bool = utils.parse_boolean_string(
os.environ.get("ENABLE_VERSIONING_CAPTURE", "true")
)
+4
View File
@@ -1316,6 +1316,10 @@ class DatabaseRestApi(BaseSupersetModelRestApi):
try:
TestConnectionDatabaseCommand(item).run()
return self.response(200, message="OK")
except OAuth2RedirectError:
# OAuth2 connections pass, so they can be saved. A user later
# can then store an OAuth2 token.
return self.response(200, message="OK")
except (
SSHTunnelingNotEnabledError,
SSHTunnelDatabasePortError,
+3
View File
@@ -290,6 +290,9 @@ class TrinoEngineSpec(PrestoBaseEngineSpec):
if user_token is not None:
http_session = requests.Session()
http_session.headers.update({"Authorization": f"Bearer {user_token}"})
# Persists `verify` to the new `http_session`
if "verify" in connect_args:
http_session.verify = connect_args["verify"]
connect_args["http_session"] = http_session
return url, engine_kwargs
+84 -2
View File
@@ -37,6 +37,7 @@ joins and unions are done in memory, using the SQLite engine.
from __future__ import annotations
import contextvars
import datetime
import decimal
import operator
@@ -68,7 +69,34 @@ from sqlalchemy.exc import NoSuchTableError
from sqlalchemy.sql import Select, select
from superset import db, feature_flag_manager, security_manager
from superset.sql.parse import Table
from superset.sql.parse import count_referenced_tables, Table
def _count_referenced_tables(statement: str) -> int:
"""
Count the distinct `superset://` virtual tables a statement references,
so ``get_data`` can tell whether it's being asked for a standalone table
or for one side of a multi-table statement (see ``get_data`` for why this
matters). Shillelagh calls `SupersetShillelaghAdapter.get_data` once per
underlying table, independently of any other table referenced by the
same statement, so it has no way on its own to tell the two cases apart.
Uses the real SQL parser rather than pattern-matching on quoted
identifiers, since a naive `"db.table"`-shaped regex also matches
dotted, double-quoted column aliases (e.g. `AS "metric.value"`) that
have nothing to do with table references, and would misclassify a
single-table statement as multi-table.
"""
return count_referenced_tables(statement, "sqlite")
# `SupersetAPSWDialect.on_connect` populates `_executing_multi_table_query` for
# the duration of a statement so that `get_data` can tell whether it's being
# asked for a standalone table or for one side of a multi-table query (see
# `get_data` for why this matters).
_executing_multi_table_query: contextvars.ContextVar[bool] = contextvars.ContextVar(
"_executing_multi_table_query", default=False
)
# pylint: disable=abstract-method
@@ -119,6 +147,49 @@ class SupersetAPSWDialect(APSWDialect):
},
)
def on_connect(self) -> Callable[[Any], None]:
"""
Wrap cursor creation on every new DBAPI connection so ``execute`` tracks
whether the statement it's about to run references more than one
`superset://` virtual table, no matter how that statement reaches the
cursor.
SQLAlchemy's `do_execute*` hooks only fire for statements executed
through a SQLAlchemy `Connection` (the ORM/Core path). SQL Lab, the
primary way users query these tables, instead pulls a raw DBAPI cursor
via `engine.raw_connection()` and calls `cursor.execute()` on it
directly, bypassing those hooks entirely -- which would leave
`_executing_multi_table_query` permanently `False` for that path, and
`get_data` back to silently truncating one side of a join (see
`get_data` and #36304). Patching the cursor factory here, at the point
a new physical connection is established, catches every path, since
each one ultimately calls `execute()` on a cursor obtained from this
same connection.
"""
def setup(dbapi_connection: Any) -> None:
original_cursor = dbapi_connection.cursor
def cursor(*args: Any, **kwargs: Any) -> Any:
raw_cursor = original_cursor(*args, **kwargs)
original_execute = raw_cursor.execute
def execute(operation: str, parameters: Any = None) -> Any:
token = _executing_multi_table_query.set(
_count_referenced_tables(operation) > 1
)
try:
return original_execute(operation, parameters)
finally:
_executing_multi_table_query.reset(token)
raw_cursor.execute = execute
return raw_cursor
dbapi_connection.cursor = cursor
return setup
F = TypeVar("F", bound=Callable[..., Any])
@@ -409,7 +480,18 @@ class SupersetShillelaghAdapter(Adapter):
"""
app_limit: int | None = current_app.config["SUPERSET_META_DB_LIMIT"]
if limit is None:
limit = app_limit
# Shillelagh calls `get_data` once per table, independently of any
# other table referenced by the same statement, so a value of `None`
# here doesn't necessarily mean this table is the whole query -- it
# can equally mean this table is one side of a join (or other
# multi-table statement). Applying the app-wide default in that case
# would silently truncate this table before the in-memory join runs,
# dropping rows that have a genuine match on the other side with no
# error (see #36304). Only fall back to the default for statements
# that reference a single table, where truncating it can't hide
# otherwise-valid matches.
if app_limit is not None and not _executing_multi_table_query.get():
limit = app_limit
elif app_limit is not None:
limit = min(limit, app_limit)
+3 -10
View File
@@ -791,16 +791,9 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
Must be called after all versioned model classes have been imported so
that VERSIONED_MODELS can be populated and configure_mappers() has run.
``ENABLE_VERSIONING_CAPTURE`` (ships default ``False``) gates the two
before-flush listener registrations. The flag is operational, not
feature: with it off the infrastructure is inert (no save writes
shadow rows); flipping it on activates capture. The switch also lets
an operator who observes a versioning-induced regression (e.g. a
save-path slowdown attributable to the change-record listener)
disable capture in ``superset_config.py`` and restart workers a
30-second recovery instead of revert-and-redeploy. Shadow tables
already created by the migration stay; they just stop accumulating
new rows.
``ENABLE_VERSIONING_CAPTURE`` gates the baseline and change-record
listener registrations. When disabled, initialization also detaches
SQLAlchemy-Continuum's write listeners.
The fallback here is ``False`` so that any app-factory path that
does not load ``superset.config`` (some test factories, embedded
@@ -199,6 +199,8 @@ def apply_form_data_filters_to_query(
query["where"] = where
if having := form_data.get("having"):
query["having"] = having
if extras := form_data.get("extras"):
query["extras"] = {**(query.get("extras") or {}), **extras}
def _join_sql_clause(existing_clause: str, additional_clause: str) -> str:
@@ -256,6 +258,9 @@ def merge_form_data_filters_into_query(
else:
query[clause] = additional_clause
if extras := form_data.get("extras"):
query["extras"] = {**(query.get("extras") or {}), **extras}
def merge_extra_form_data_filters_into_query(
query: dict[str, Any],
@@ -665,6 +665,8 @@ def add_legend_config(form_data: Dict[str, Any], config: XYChartConfig) -> None:
# Canonical form_data key is camelCase; the echarts plugins read
# `legendOrientation` directly off form_data.
form_data["legendOrientation"] = config.legend.position
if config.legend_orientation:
form_data["legendOrientation"] = config.legend_orientation
def add_color_scheme(form_data: Dict[str, Any], color_scheme: str | None) -> None:
@@ -1415,6 +1417,8 @@ def map_filter_operator(op: str) -> str:
"NOT LIKE": "NOT LIKE",
"IN": "IN",
"NOT IN": "NOT IN",
"IS NULL": "IS NULL",
"IS NOT NULL": "IS NOT NULL",
}
return operator_map.get(op, op)
@@ -19,7 +19,6 @@
from __future__ import annotations
import re
from collections.abc import Mapping
from typing import Any, ClassVar
@@ -29,7 +28,10 @@ from superset.mcp_service.chart.chart_utils import (
)
from superset.mcp_service.chart.plugin import BaseChartPlugin
from superset.mcp_service.chart.schemas import ColumnRef, HistogramChartConfig
from superset.mcp_service.chart.validation.dataset_validator import DatasetValidator
from superset.mcp_service.chart.validation.dataset_validator import (
DatasetValidator,
is_numeric_column,
)
from superset.mcp_service.common.error_schemas import ChartGenerationError
@@ -116,27 +118,13 @@ class HistogramChartPlugin(BaseChartPlugin):
# Column existence is validated separately; don't double-report.
return None
def _is_numeric(col: dict[str, Any]) -> bool:
if col.get("is_numeric", False):
return True
# Backends report many spellings (BIGINT, SMALLINT, REAL, NUMBER,
# DOUBLE PRECISION); match numeric tokens at word boundaries so
# INTERVAL/POINT (which merely contain "INT") stay non-numeric.
type_upper = str(col.get("type", "")).upper()
return bool(
re.search(
r"\b(?:TINY|SMALL|MEDIUM|BIG)?INT(?:EGER)?\b"
r"|\bFLOAT\b|\bDOUBLE\b|\bDECIMAL\b"
r"|\bNUMERIC\b|\bREAL\b|\bNUMBER\b",
type_upper,
)
)
if _is_numeric(col_info):
if is_numeric_column(col_info):
return None
numeric_columns = sorted(
col["name"] for col in dataset_context.available_columns if _is_numeric(col)
col["name"]
for col in dataset_context.available_columns
if is_numeric_column(col)
)
return ChartGenerationError(
error_type="non_numeric_histogram_column",
+33 -5
View File
@@ -706,7 +706,7 @@ class ColumnRef(UnknownFieldCheckMixin):
None,
min_length=1,
max_length=255,
validation_alias=AliasChoices("name", "column_name"),
validation_alias=AliasChoices("name", "column_name", "column"),
)
label: str | None = Field(None, max_length=500)
dtype: str | None = None
@@ -884,17 +884,32 @@ class FilterConfig(UnknownFieldCheckMixin):
"NOT LIKE",
"IN",
"NOT IN",
"IS NULL",
"IS NOT NULL",
] = Field(
...,
description="LIKE/ILIKE use % wildcards. IN/NOT IN take a list.",
description=(
"LIKE/ILIKE use % wildcards. IN/NOT IN take a list. "
"IS NULL/IS NOT NULL omit value."
),
validation_alias=AliasChoices("op", "operator", "opr"),
)
value: str | int | float | bool | list[str | int | float | bool] = Field(
...,
description="For IN/NOT IN, provide a list.",
value: str | int | float | bool | list[str | int | float | bool] | None = Field(
None,
description="For IN/NOT IN, provide a list. Omit for null operators.",
validation_alias=AliasChoices("value", "val"),
)
@model_validator(mode="after")
def validate_value(self) -> "FilterConfig":
"""Null checks have no comparator; every other operator requires one."""
if self.op in {"IS NULL", "IS NOT NULL"}:
if self.value is not None:
raise ValueError(f"Filter operator {self.op!r} must not have 'value'.")
elif self.value is None:
raise ValueError(f"Filter operator {self.op!r} requires 'value'.")
return self
@field_validator("column")
@classmethod
def sanitize_column(cls, v: str) -> str:
@@ -1662,6 +1677,10 @@ class XYChartConfig(BaseChartConfig):
None,
validation_alias=AliasChoices("legend", "show_legend"),
)
legend_orientation: LEGEND_POSITION_LITERAL | None = Field(
None,
description="Legend placement around the chart",
)
x_axis_time_format: str | None = Field(
None,
description=(
@@ -2810,6 +2829,15 @@ class GetChartSqlRequest(BaseModel):
"Can be used alone (without identifier) for unsaved charts."
),
)
extra_form_data: dict[str, Any] | None = Field(
default=None,
description=(
"Extra form data to merge into the chart query before rendering SQL, "
"typically from dashboard native filters. Same format accepted by "
"get_chart_data. Format: "
'{"filters": [{"col": "country", "op": "IN", "val": ["US"]}]}'
),
)
@model_validator(mode="after")
def validate_identifier_or_form_data_key(self) -> "GetChartSqlRequest":
@@ -62,6 +62,42 @@ from superset.utils.core import GenericDataType
logger = logging.getLogger(__name__)
def _requested_filter_columns(extra_form_data: dict[str, Any] | None) -> set[str]:
"""Return simple column names explicitly requested through extra form data."""
if not extra_form_data:
return set()
columns: set[str] = set()
for filter_ in extra_form_data.get("filters", []):
if isinstance(filter_, dict) and isinstance(column := filter_.get("col"), str):
columns.add(column)
for filter_ in extra_form_data.get("adhoc_filters", []):
if (
isinstance(filter_, dict)
and filter_.get("expressionType") == "SIMPLE"
and isinstance(column := filter_.get("subject"), str)
):
columns.add(column)
return columns
def _rejected_requested_filter_columns(
result: Any, extra_form_data: dict[str, Any] | None
) -> list[str]:
"""Find request filters rejected by datasource query construction."""
if not isinstance(result, dict):
return []
requested = _requested_filter_columns(extra_form_data)
rejected = {
column
for query in result.get("queries", [])
for column in query.get("rejected_filter_columns", [])
if isinstance(column, str)
}
return sorted(requested & rejected)
_GENERIC_TYPE_MAP: dict[int, str] = {
GenericDataType.NUMERIC: "numeric",
GenericDataType.STRING: "string",
@@ -685,6 +721,19 @@ async def get_chart_data( # noqa: C901
command.validate()
result = command.run()
if rejected := _rejected_requested_filter_columns(
result, request.extra_form_data
):
rejected_columns = ", ".join(rejected)
await ctx.warning(
"Requested filters reference unknown dataset columns: %s"
% rejected_columns
)
return ChartError(
error=f"Unknown dataset column(s) in filters: {rejected_columns}",
error_type="ValidationError",
)
# Handle empty query results for certain chart types
if not result or ("queries" not in result) or len(result["queries"]) == 0:
await ctx.warning(
@@ -979,7 +1028,7 @@ async def get_chart_data( # noqa: C901
)
async def _query_from_form_data(
async def _query_from_form_data( # noqa: C901
form_data: Dict[str, Any],
request: GetChartDataRequest,
ctx: Context,
@@ -1034,6 +1083,19 @@ async def _query_from_form_data(
command.validate()
result = command.run()
if rejected := _rejected_requested_filter_columns(
result, request.extra_form_data
):
rejected_columns = ", ".join(rejected)
await ctx.warning(
"Requested filters reference unknown dataset columns: %s"
% rejected_columns
)
return ChartError(
error=f"Unknown dataset column(s) in filters: {rejected_columns}",
error_type="ValidationError",
)
if not result or "queries" not in result or len(result["queries"]) == 0:
logger.warning(
"get_chart_data: empty query results for unsaved chart "
@@ -23,11 +23,13 @@ import logging
from typing import Any, TYPE_CHECKING
from fastmcp import Context
from marshmallow import ValidationError as MarshmallowValidationError
from superset_core.mcp.decorators import tool, ToolAnnotations
if TYPE_CHECKING:
from superset.models.slice import Slice
from superset.charts.data.form_data import set_query_context_form_data
from superset.commands.exceptions import CommandException
from superset.commands.explore.form_data.parameters import CommandParameters
from superset.exceptions import SupersetException, SupersetSecurityException
@@ -35,6 +37,8 @@ from superset.extensions import event_logger
from superset.mcp_service.chart.chart_helpers import (
build_query_context_from_form_data,
extract_x_axis_col,
merge_extra_form_data_filters_into_query,
resolve_form_data_datasource,
resolve_groupby,
resolve_metrics,
resolve_metrics_and_groupby,
@@ -90,6 +94,7 @@ def _extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
def _build_query_context_from_form_data(
form_data: dict[str, Any],
chart: "Slice | None" = None,
extra_form_data: dict[str, Any] | None = None,
) -> Any:
"""Build a QueryContext from form_data with result_type=QUERY.
@@ -101,6 +106,7 @@ def _build_query_context_from_form_data(
return build_query_context_from_form_data(
form_data,
chart=chart,
extra_form_data=extra_form_data,
result_type=ChartDataResultType.QUERY,
force=False,
)
@@ -150,6 +156,7 @@ def _resolve_effective_form_data(
def _sql_from_saved_query_context(
chart: "Slice",
extra_form_data: dict[str, Any] | None = None,
) -> ChartSql | ChartError | None:
"""Try to extract SQL from a chart's saved query_context.
@@ -168,8 +175,63 @@ def _sql_from_saved_query_context(
qc_json["result_type"] = ChartDataResultType.QUERY
qc_json["force"] = False
query_context = ChartDataQueryContextSchema().load(qc_json)
if extra_form_data:
# Resolve the pieces of the saved context the merge depends on first.
# Failures here mean the context itself is stale, not that the
# request's filters are bad, so the caller should rebuild it from
# form_data rather than surfacing a validation error.
try:
datasource_id = qc_json["datasource"]["id"]
datasource_type = qc_json["datasource"]["type"]
queries = qc_json.get("queries", [])
if not isinstance(queries, list):
raise TypeError("queries must be a list")
except (AttributeError, KeyError, TypeError) as ex:
logger.warning(
"Saved query context is unusable for chart %s; "
"falling back to form_data: %s",
chart.id,
ex,
)
return None
try:
for query in queries:
merge_extra_form_data_filters_into_query(
query,
extra_form_data,
datasource_id,
datasource_type,
)
except (AttributeError, KeyError, TypeError) as ex:
return ChartError(
error=f"Invalid extra_form_data filter: {ex}",
error_type="ValidationError",
)
try:
query_context = ChartDataQueryContextSchema().load(qc_json)
except MarshmallowValidationError as ex:
# A saved query context can become stale as schemas evolve. Let the
# caller rebuild it from form_data; malformed request filters will
# still produce a ValidationError from that fallback path.
logger.warning(
"Saved query context validation failed for chart %s; "
"falling back to form_data: %s",
chart.id,
ex,
)
return None
query_context.result_type = ChartDataResultType.QUERY
# ChartDataDatasourceSchema only requires "id", so fall back to the
# chart's own datasource rather than raising on a context that the
# schema itself considers valid.
datasource_json = qc_json.get("datasource") or {}
set_query_context_form_data(
query_context,
datasource_json.get("id", chart.datasource_id),
datasource_json.get("type", chart.datasource_type),
)
command = ChartDataCommand(query_context)
command.validate()
@@ -235,11 +297,26 @@ def _resolve_datasource_name(
def _sql_from_form_data(
form_data: dict[str, Any],
chart: "Slice | None",
extra_form_data: dict[str, Any] | None = None,
) -> ChartSql | ChartError:
"""Build SQL from form_data (fallback path)."""
from superset.commands.chart.data.get_data_command import ChartDataCommand
query_context = _build_query_context_from_form_data(form_data, chart)
try:
_, datasource_type = resolve_form_data_datasource(form_data, chart)
query_context = _build_query_context_from_form_data(
form_data, chart, extra_form_data=extra_form_data
)
except (AttributeError, KeyError, TypeError, MarshmallowValidationError) as ex:
return ChartError(
error=f"Invalid chart query data: {ex}",
error_type="ValidationError",
)
set_query_context_form_data(
query_context,
query_context.datasource.id,
datasource_type,
)
command = ChartDataCommand(query_context)
command.validate()
result = command.run()
@@ -335,6 +412,8 @@ async def get_chart_sql(
Supports:
- Numeric ID or UUID lookup
- form_data_key: get SQL for unsaved chart state from Explore view
- extra_form_data: preview SQL with dashboard-filter-style predicates merged
in, same format accepted by get_chart_data
Example usage:
```json
@@ -382,7 +461,9 @@ async def _handle_chart_sql_request(
# Handle unsaved chart (form_data_key only, no identifier)
if not request.identifier and request.form_data_key:
return await _handle_unsaved_chart_sql(request.form_data_key, ctx)
return await _handle_unsaved_chart_sql(
request.form_data_key, ctx, request.extra_form_data
)
# Find the chart by identifier
if request.identifier is None:
@@ -425,7 +506,7 @@ async def _handle_chart_sql_request(
# Try saved query_context first (faster, more accurate)
with event_logger.log_context(action="mcp.get_chart_sql.build_query"):
if not using_unsaved_state:
saved_result = _sql_from_saved_query_context(chart)
saved_result = _sql_from_saved_query_context(chart, request.extra_form_data)
if saved_result is not None:
return saved_result
await ctx.warning(
@@ -435,7 +516,9 @@ async def _handle_chart_sql_request(
# Fallback: build query context from form_data
try:
return _sql_from_form_data(effective_form_data, chart)
return _sql_from_form_data(
effective_form_data, chart, request.extra_form_data
)
except (SupersetException, CommandException, ValueError) as e:
await ctx.warning("Failed to build SQL from form_data: %s" % str(e))
return ChartError(
@@ -445,7 +528,9 @@ async def _handle_chart_sql_request(
async def _handle_unsaved_chart_sql(
form_data_key: str, ctx: Context
form_data_key: str,
ctx: Context,
extra_form_data: dict[str, Any] | None = None,
) -> ChartSql | ChartError:
"""Handle SQL retrieval for unsaved charts (form_data_key only)."""
from superset.utils import json as utils_json
@@ -476,7 +561,9 @@ async def _handle_unsaved_chart_sql(
)
try:
return _sql_from_form_data(form_data, chart=None)
return _sql_from_form_data(
form_data, chart=None, extra_form_data=extra_form_data
)
except (SupersetException, CommandException, ValueError) as e:
await ctx.warning("Failed to generate SQL from form_data: %s" % str(e))
return ChartError(
@@ -22,6 +22,8 @@ Validates that referenced columns exist in the dataset schema.
import difflib
import logging
import re
from collections.abc import Mapping
from typing import Any, Dict, List, Tuple, TypeVar
from superset.mcp_service.chart.schemas import (
@@ -38,6 +40,18 @@ _C = TypeVar("_C", bound=ChartConfig)
logger = logging.getLogger(__name__)
_NUMERIC_TYPE_PATTERN = re.compile(
r"\b(?:(?:TINY|SMALL|MEDIUM|BIG)?INT(?:EGER)?|INT[248]|FLOAT[48]?|"
r"DOUBLE(?:\s+PRECISION)?|DECIMAL|NUMERIC|REAL|NUMBER|(?:SMALL)?MONEY)\b"
)
def is_numeric_column(column: Mapping[str, Any]) -> bool:
"""Return whether dataset metadata identifies a numeric SQL column."""
if column.get("is_numeric", False):
return True
return bool(_NUMERIC_TYPE_PATTERN.search(str(column.get("type") or "").upper()))
def is_dataset_column_temporal(
column: Any, column_name: str, db_engine_spec: Any
@@ -702,11 +716,11 @@ class DatasetValidator:
"STDDEV",
"VAR",
]
type_name = str(col_info.get("type") or "").strip().upper()
if (
col_ref.aggregate in numeric_aggs
and not col_info.get("is_numeric", False)
and col_info.get("type", "").upper()
not in ["INTEGER", "FLOAT", "DOUBLE", "DECIMAL", "NUMERIC"]
and type_name not in {"", "UNKNOWN"}
and not is_numeric_column(col_info)
):
from superset.mcp_service.utils.error_builder import ( # noqa: E501
ChartErrorBuilder,
@@ -0,0 +1,312 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Validation for dashboard layouts supplied through MCP tools."""
from __future__ import annotations
from collections.abc import Collection
from typing import Any
_ROOT_ID = "ROOT_ID"
_GRID_ID = "GRID_ID"
_HEADER_ID = "HEADER_ID"
_VERSION_KEY = "DASHBOARD_VERSION_KEY"
_CHART_TYPE = "CHART"
# Chart IDs are database integers; a decimal string longer than a 64-bit value
# is malformed input rather than an ID that could ever resolve.
_MAX_CHART_ID_DIGITS = 19
# Keep in sync with the frontend's parent/child contract in
# superset-frontend/src/dashboard/util/isValidChild.ts, which admits a child
# only when both its type and its parent's nesting depth are allowed. The
# values below are that file's ``parentMaxDepthLookup``; traversal is iterative
# so deeply nested input cannot overflow Python's call stack.
_ROOT_DEPTH = 0
_DEPTH_ONE = _ROOT_DEPTH + 1
_DEPTH_THREE = _ROOT_DEPTH + 3
_DEPTH_FOUR = _ROOT_DEPTH + 4
_DEPTH_FIVE = _ROOT_DEPTH + 5
_PARENT_MAX_DEPTH: dict[str, dict[str, int]] = {
"ROOT": {"GRID": _ROOT_DEPTH, "TABS": _ROOT_DEPTH},
"GRID": {
"CHART": _DEPTH_ONE,
"COLUMN": _DEPTH_ONE,
"DIVIDER": _DEPTH_ONE,
"DYNAMIC": _DEPTH_ONE,
"HEADER": _DEPTH_ONE,
"MARKDOWN": _DEPTH_ONE,
"ROW": _DEPTH_ONE,
"TABS": _DEPTH_ONE,
},
"ROW": {
"CHART": _DEPTH_FOUR,
"COLUMN": _DEPTH_FOUR,
"DYNAMIC": _DEPTH_FOUR,
"MARKDOWN": _DEPTH_FOUR,
},
"TABS": {"TAB": _DEPTH_THREE},
"TAB": {
"CHART": _DEPTH_FIVE,
"COLUMN": _DEPTH_THREE,
"DIVIDER": _DEPTH_FIVE,
"DYNAMIC": _DEPTH_FIVE,
"HEADER": _DEPTH_FIVE,
"MARKDOWN": _DEPTH_FIVE,
"ROW": _DEPTH_THREE,
"TABS": _DEPTH_THREE,
},
"COLUMN": {
"CHART": _DEPTH_FIVE,
"DIVIDER": _DEPTH_THREE,
"HEADER": _DEPTH_FIVE,
"MARKDOWN": _DEPTH_FIVE,
"ROW": _DEPTH_THREE,
"TABS": _DEPTH_THREE,
},
"CHART": {},
"DIVIDER": {},
"DYNAMIC": {},
"HEADER": {},
"MARKDOWN": {},
}
_ALLOWED_CHILD_TYPES: dict[str, frozenset[str]] = {
parent_type: frozenset(child_depths)
for parent_type, child_depths in _PARENT_MAX_DEPTH.items()
}
# TABS and TAB deliberately render their children at their own depth; every
# other container increments it. See the worked examples in isValidChild.ts.
_DEPTH_TRANSPARENT_TYPES = frozenset({"TABS", "TAB"})
_CONTAINER_TYPES = frozenset(
component_type
for component_type, child_types in _ALLOWED_CHILD_TYPES.items()
if child_types
)
_META_REQUIRED_TYPES = frozenset(_ALLOWED_CHILD_TYPES) - {"ROOT", "GRID"}
def normalize_chart_id(value: Any) -> int | None:
"""Normalize an integer or canonical decimal-string chart ID.
Only canonical decimal strings are accepted. Leading-zero forms such as
``"007"`` are rejected so that layout lookups and the ``json_metadata``
cleanup in ``remove_chart_from_dashboard`` which keys off
``str(chart_id)`` cannot disagree about whether a reference matches and
leave stale references behind. The digit bound keeps ``int()`` away from
CPython's integer string conversion limit, which would otherwise raise
``ValueError`` out of the validator instead of returning a structured
error; no real chart ID approaches it.
"""
if isinstance(value, bool):
return None
if isinstance(value, int):
return value if value > 0 else None
if (
isinstance(value, str)
and value.isascii()
and value.isdecimal()
and len(value) <= _MAX_CHART_ID_DIGITS
and not value.startswith("0")
):
return int(value)
return None
def _validate_component_shapes( # noqa: C901
layout: dict[str, Any],
) -> tuple[dict[str, dict[str, Any]], str | None]:
"""Validate and return every component object in a raw layout mapping."""
if layout.get(_VERSION_KEY) != "v2":
return {}, f"{_VERSION_KEY} must be the string 'v2'."
components: dict[str, dict[str, Any]] = {}
for component_id, component in layout.items():
if component_id == _VERSION_KEY:
continue
if not isinstance(component, dict):
return {}, f"Layout value {component_id} must be a component object."
if component.get("id") != component_id:
return {}, f"Layout component {component_id} must have the same id value."
component_type = component.get("type")
if not isinstance(component_type, str) or component_type not in (
_ALLOWED_CHILD_TYPES
):
return {}, f"Layout component {component_id} has unsupported type."
if component_type == "DYNAMIC":
return {}, (
f"Layout component {component_id} uses DYNAMIC, which cannot be "
"safely validated by the server."
)
children = component.get("children")
if component_type in _CONTAINER_TYPES and children is None:
return {}, f"Layout component {component_id} must define children."
if children is not None and (
not isinstance(children, list)
or not all(isinstance(child_id, str) for child_id in children)
):
return {}, f"Layout component {component_id}.children must be a list."
if component_type not in _CONTAINER_TYPES and children not in (None, []):
return {}, f"Layout component {component_id} cannot have children."
if component_type == "TABS" and not children:
return {}, f"Tabs component {component_id} must contain at least one tab."
if component_type in _META_REQUIRED_TYPES and not isinstance(
component.get("meta"), dict
):
return {}, f"Layout component {component_id}.meta must be an object."
components[component_id] = component
return components, None
def _validate_edges(
components: dict[str, dict[str, Any]],
) -> tuple[dict[str, str], str | None]:
"""Validate graph edges and return each component's actual parent."""
parent_by_child: dict[str, str] = {}
for parent_id, parent in components.items():
parent_type = parent["type"]
for child_id in parent.get("children") or []:
child = components.get(child_id)
if child is None:
return {}, f"Layout references missing component {child_id}."
if child["type"] not in _ALLOWED_CHILD_TYPES[parent_type]:
return {}, (
f"Layout component {child_id} cannot be a child of {parent_id}."
)
if child_id in parent_by_child:
return {}, f"Layout component {child_id} has more than one parent."
parent_by_child[child_id] = parent_id
if _ROOT_ID in parent_by_child:
return {}, "ROOT_ID must not have a parent."
return parent_by_child, None
def _find_cycle(components: dict[str, dict[str, Any]]) -> str | None:
"""Return a component ID in a cycle using an iterative depth-first walk."""
state: dict[str, int] = {}
for start_id in components:
if state.get(start_id) == 2:
continue
stack: list[tuple[str, bool]] = [(start_id, False)]
while stack:
component_id, exiting = stack.pop()
if exiting:
state[component_id] = 2
continue
if state.get(component_id) == 1:
return component_id
if state.get(component_id) == 2:
continue
state[component_id] = 1
stack.append((component_id, True))
for child_id in reversed(components[component_id].get("children") or []):
stack.append((child_id, False))
return None
def validate_dashboard_layout( # noqa: C901
layout: dict[str, Any], expected_chart_ids: Collection[int]
) -> str | None:
"""Return an error when an MCP layout replacement is unsafe to persist.
Superset renders only components reachable from ``ROOT_ID`` but indexes all
chart nodes during hydration. This validates renderer-required component
shape and graph topology, then requires the reachable charts to match the
dashboard's associated charts before allowing a full replacement.
``HEADER_ID`` is dashboard metadata rather than a rendered tree child.
Superset also retains an empty, detached ``GRID_ID`` when top-level tabs are
used; both are allowed as explicit reserved-node exceptions.
"""
components, error = _validate_component_shapes(layout)
if error:
return error
root = components.get(_ROOT_ID)
if root is None or root.get("type") != "ROOT":
return "Layout must contain a ROOT_ID component with type ROOT."
root_children = root.get("children") or []
if len(root_children) != 1:
return "ROOT_ID must contain exactly one GRID or TABS component."
parent_by_child, error = _validate_edges(components)
if error:
return error
if cycle_id := _find_cycle(components):
return f"Layout contains a cycle at {cycle_id}."
visited: set[str] = set()
reachable_chart_ids: set[int] = set()
# The frontend treats ``parents`` as derived metadata and recomputes it
# during hydration. Saved layouts can therefore omit it or retain stale
# values after drag-and-drop; the validated child edges are authoritative.
# Depth is likewise derived here rather than trusted, and is only defined
# for reachable nodes, so the nesting limits are checked on this walk.
stack: list[tuple[str, int]] = [(_ROOT_ID, _ROOT_DEPTH)]
while stack:
component_id, depth = stack.pop()
component = components[component_id]
component_type = component["type"]
visited.add(component_id)
if component_type == _CHART_TYPE:
chart_id = normalize_chart_id(component["meta"].get("chartId"))
if chart_id is None:
return (
f"Chart component {component_id} must have a positive integer "
"or decimal-string chartId."
)
reachable_chart_ids.add(chart_id)
child_depth = depth if component_type in _DEPTH_TRANSPARENT_TYPES else depth + 1
for child_id in reversed(component.get("children") or []):
# ``_validate_edges`` already accepted this parent/child type pair,
# so a missing entry here is impossible.
if depth > _PARENT_MAX_DEPTH[component_type][components[child_id]["type"]]:
return (
f"Layout component {child_id} is nested too deeply under "
f"{component_id}."
)
stack.append((child_id, child_depth))
top_level_type = components[root_children[0]]["type"]
for component_id, component in components.items():
if component_id in visited:
continue
if component_id == _HEADER_ID and component["type"] == "HEADER":
continue
if (
component_id == _GRID_ID
and top_level_type == "TABS"
and component["type"] == "GRID"
and component.get("children") == []
):
continue
return f"Layout component {component_id} is unreachable from ROOT_ID."
expected = set(expected_chart_ids)
if missing := sorted(expected - reachable_chart_ids):
return f"Layout would hide dashboard charts: {missing}."
if unknown := sorted(reachable_chart_ids - expected):
return f"Layout references charts not associated with the dashboard: {unknown}."
return None
+5 -3
View File
@@ -762,9 +762,11 @@ class UpdateDashboardRequest(BaseModel):
None,
description=(
"Optional replacement layout (Superset's position_json dict). "
"When set, fully replaces the existing layout. Get the current "
"layout via ``get_dashboard_info`` first if you want to make "
"incremental changes."
"When set, fully replaces the existing layout and must keep every "
"dashboard chart reachable from ROOT_ID, with consistent children "
"and parents. Do not use this field for incremental edits: MCP does "
"not currently expose the complete raw layout tree needed to safely "
"round-trip a replacement. Prefer purpose-built dashboard tools."
),
)
json_metadata_overrides: Dict[str, Any] | None = Field(
@@ -35,6 +35,7 @@ from superset_core.mcp.decorators import tool, ToolAnnotations
from superset.commands.exceptions import CommandException, ForbiddenError
from superset.extensions import event_logger
from superset.mcp_service.dashboard.layout_validation import normalize_chart_id
from superset.mcp_service.dashboard.schemas import (
DashboardInfo,
RemoveChartFromDashboardRequest,
@@ -59,14 +60,12 @@ def _find_chart_keys(layout: Dict[str, Any], chart_id: int) -> list[str]:
A chart can legitimately appear more than once in a layout (e.g. under
multiple tabs), so all occurrences are returned.
"""
# Accept both int and string chartId — position_json is user/frontend-authored
# and imported or hand-edited layouts may store chartId as a string.
return [
key
for key, node in layout.items()
if isinstance(node, dict)
and node.get("type") == "CHART"
and (node.get("meta") or {}).get("chartId") in (chart_id, str(chart_id))
and normalize_chart_id((node.get("meta") or {}).get("chartId")) == chart_id
]
@@ -33,6 +33,7 @@ from superset_core.mcp.decorators import tool, ToolAnnotations
from superset.commands.dashboard.exceptions import DashboardNotFoundError
from superset.exceptions import SupersetSecurityException
from superset.extensions import db, event_logger
from superset.mcp_service.dashboard.layout_validation import validate_dashboard_layout
from superset.mcp_service.dashboard.schemas import (
dashboard_serializer,
DashboardError,
@@ -237,6 +238,14 @@ def _validate_update_request(
from superset.dashboards.schemas import validate_css
from superset.tags.models import ObjectType
if request.position_json is not None:
chart_ids = [chart.id for chart in dashboard.slices]
if error := validate_dashboard_layout(request.position_json, chart_ids):
return DashboardError(
error=f"Dashboard layout is invalid: {error}",
error_type="InvalidDashboardLayout",
)
# Empty string clears CSS (no validation needed); only validate real content.
if request.css:
try:
@@ -289,9 +298,10 @@ async def update_dashboard(
) -> UpdateDashboardResponse | DashboardError:
"""Patch an existing dashboard's layout, theme, styling, or metadata.
Companion to ``generate_dashboard`` for incremental edits. An LLM can:
Companion to ``generate_dashboard`` for incremental metadata and styling
edits. An LLM can:
- Set or replace ``position_json`` after auto-generation
- Replace ``position_json`` only when it already has the complete raw tree
- Apply brand ``label_colors`` and ``color_scheme`` via
``json_metadata_overrides``
- Inject ``css`` to hide chrome on print-ready dashboards
@@ -0,0 +1,60 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Add a reason to purge_audit_log.
Adds a nullable ``reason`` column to ``purge_audit_log`` holding a stable
machine code identifying which policy rule blocked a purge (or the
cascade-integrity failure class). Written at finalization for
blocked outcomes only; NULL for confirmed, failed, non-blocked, and
pre-existing rows. No backfill: the information was never captured for
historical records, and readers treat the column as optional.
Apply this migration before deploying the code that depends on it. The
audit model declares the column, so a worker running the new code against
the un-migrated table cannot write its write-ahead record; the scheduled
purge then fails closed (nothing is purged unaudited) and logs a
write-ahead warning every run until the migration lands.
The downgrade discards every recorded block reason -- the rows survive and
revert to reason-less, exactly like pre-feature history.
Revision ID: 39097d124752
Revises: 1072de5ed955
Create Date: 2026-08-24 12:00:00.000000
"""
import sqlalchemy as sa
from superset.migrations.shared.utils import add_columns, drop_columns
# revision identifiers, used by Alembic.
revision: str = "39097d124752"
down_revision: str = "1072de5ed955"
def upgrade() -> None:
"""Add the nullable ``reason`` column to ``purge_audit_log``."""
add_columns(
"purge_audit_log",
sa.Column("reason", sa.String(64), nullable=True),
)
def downgrade() -> None:
"""Drop the ``reason`` column from ``purge_audit_log``."""
drop_columns("purge_audit_log", "reason")
+6
View File
@@ -72,6 +72,12 @@ class PurgeAuditLog(Model):
# Comma-joined UUIDs of charts left dangling / dashboards that lost a join
# row (force-purge visibility). Free text, content-free.
affected_referrers = Column(Text, nullable=True)
# Stable machine code identifying which rule blocked the purge (or the
# cascade-integrity failure class). Written at finalization for
# blocked outcomes only; NULL for confirmed, failed, non-blocked, and
# pre-feature rows. Vocabulary: REASON_* constants in
# superset.commands.deletion_retention.purge_policy.
reason: Column[str] = Column(String(64), nullable=True)
removed_dashboard_slices = Column(Integer, nullable=False, default=0)
created_on = Column(
DateTime()
+41 -3
View File
@@ -20,10 +20,19 @@ from typing import Any, Optional, Union
from croniter import croniter
from flask import current_app
from flask_babel import gettext as _
from marshmallow import EXCLUDE, fields, Schema, validate, validates, validates_schema
from marshmallow import (
EXCLUDE,
fields,
pre_load,
Schema,
validate,
validates,
validates_schema,
)
from marshmallow.validate import Length, Range, ValidationError
from pytz import all_timezones
from superset import is_feature_enabled
from superset.reports.models import (
ReportCreationMethod,
ReportDataFormat,
@@ -180,7 +189,34 @@ class ReportRecipientSchema(Schema):
validate_addresses("bccTarget", config.get("bccTarget"), required=False)
class ReportSchedulePostSchema(Schema):
_RETRY_FIELD_KEYS = (
"retry_on_failure",
"retry_max_attempts",
"send_failed_reports",
"retry_notify_owners",
"retry_notify_recipients",
)
class RetryFieldStripMixin:
"""Strip retry fields from the raw payload before validation when the
feature is off. Using ``@pre_load`` ensures that field-level validators
(e.g. ``Range`` on ``retry_max_attempts``) are never reached for values
that will be discarded anyway."""
@pre_load
def strip_retry_fields_if_disabled(
self,
data: dict[str, Any],
**kwargs: Any,
) -> dict[str, Any]:
if not is_feature_enabled("ALERT_REPORTS_RETRY"):
for key in _RETRY_FIELD_KEYS:
data.pop(key, None)
return data
class ReportSchedulePostSchema(RetryFieldStripMixin, Schema):
type = fields.String(
metadata={"description": type_description},
allow_none=False,
@@ -355,6 +391,8 @@ class ReportSchedulePostSchema(Schema):
data: dict[str, Any],
**kwargs: Any,
) -> None:
if not is_feature_enabled("ALERT_REPORTS_RETRY"):
return
if data.get("send_failed_reports") and not data.get("retry_on_failure"):
raise ValidationError(
{
@@ -396,7 +434,7 @@ class ReportScheduleSubscribeSchema(ReportSchedulePostSchema):
unknown = EXCLUDE
class ReportSchedulePutSchema(Schema):
class ReportSchedulePutSchema(RetryFieldStripMixin, Schema):
type = fields.String(
metadata={"description": type_description},
required=False,
+133 -45
View File
@@ -22,7 +22,6 @@ import enum
import logging
import re
import urllib.parse
from collections.abc import Iterable
from dataclasses import dataclass
from typing import Any, Generic, Optional, TYPE_CHECKING, TypeVar
@@ -2152,12 +2151,47 @@ class SQLScript:
return len(self.statements) == 1 and self.statements[0].is_select()
def extract_tables_from_statement(
def _find_show_statement_tables(statement: exp.Show) -> set[Table]:
"""
Build the table references for a ``SHOW`` statement.
Structured metadata statements (`SHOW CREATE TABLE foo.bar`,
`SHOW COLUMNS FROM foo`, ...) reference their target via dedicated
args rather than query sources, so build the table references
explicitly. Statements with no extractable target (e.g.
`SHOW TABLES FROM some_schema`) yield an empty set and are treated
as unparseable for authorization purposes (see
`SQLScript.has_unparseable_statement`).
``SHOW`` statements reference a single metadata target, never a join, so
(unlike ``_find_table_sources``) there is no distinct occurrence-counting
variant of this helper: the deduplicated set is always the right count.
"""
show_tables = {
Table(
source.name,
source.db if source.db != "" else None,
source.catalog if source.catalog != "" else None,
)
for source in statement.find_all(exp.Table)
}
if target := statement.args.get("target"):
db = statement.args.get("db")
show_tables.add(
Table(
target.name if isinstance(target, exp.Expression) else str(target),
db.name if isinstance(db, exp.Expression) else db,
)
)
return show_tables
def _find_table_sources(
statement: exp.Expression,
dialect: Dialects | None,
) -> set[Table]:
) -> list[exp.Table]:
"""
Extract all table references in a single statement.
Find every table reference (occurrence, not deduplicated) in a statement.
Please note that this is not trivial; consider the following queries:
@@ -2165,60 +2199,45 @@ def extract_tables_from_statement(
SHOW PARTITIONS FROM some_table;
WITH masked_name AS (SELECT * FROM some_table) SELECT * FROM masked_name;
See the unit tests for other tricky cases.
See the unit tests for other tricky cases. Note that `exp.Show` statements
are not handled here: see `_find_show_statement_tables`.
"""
sources: Iterable[exp.Table]
if isinstance(statement, exp.Describe):
# A `DESCRIBE` query has no sources in sqlglot, so we need to explicitly
# query for all tables.
sources = statement.find_all(exp.Table)
elif isinstance(statement, exp.Command):
return list(statement.find_all(exp.Table))
if isinstance(statement, exp.Command):
# Commands, like `SHOW COLUMNS FROM foo`, have to be converted into a
# `SELECT` statetement in order to extract tables.
literal = statement.find(exp.Literal)
if not literal:
return set()
return []
pseudo_sql = f"SELECT {literal.this}"
try:
_check_script_length(pseudo_sql, None)
pseudo_query = sqlglot.parse_one(pseudo_sql, dialect=dialect)
except (ParseError, SupersetParseError):
return set()
sources = pseudo_query.find_all(exp.Table)
elif isinstance(statement, exp.Show):
# Structured metadata statements (`SHOW CREATE TABLE foo.bar`,
# `SHOW COLUMNS FROM foo`, ...) reference their target via dedicated
# args rather than query sources, so build the table references
# explicitly. Statements with no extractable target (e.g.
# `SHOW TABLES FROM some_schema`) yield an empty set and are treated
# as unparseable for authorization purposes (see
# `SQLScript.has_unparseable_statement`).
show_tables = {
Table(
source.name,
source.db if source.db != "" else None,
source.catalog if source.catalog != "" else None,
)
for source in statement.find_all(exp.Table)
}
if target := statement.args.get("target"):
db = statement.args.get("db")
show_tables.add(
Table(
target.name if isinstance(target, exp.Expression) else str(target),
db.name if isinstance(db, exp.Expression) else db,
)
)
return show_tables
else:
sources = [
source
for scope in traverse_scope(statement)
for source in scope.sources.values()
if isinstance(source, exp.Table) and not is_cte(source, scope)
]
return []
return list(pseudo_query.find_all(exp.Table))
return [
source
for scope in traverse_scope(statement)
for source in scope.sources.values()
if isinstance(source, exp.Table) and not is_cte(source, scope)
]
def extract_tables_from_statement(
statement: exp.Expression,
dialect: Dialects | None,
) -> set[Table]:
"""
Extract all distinct table references in a single statement.
"""
if isinstance(statement, exp.Show):
return _find_show_statement_tables(statement)
return {
Table(
@@ -2226,10 +2245,79 @@ def extract_tables_from_statement(
source.db if source.db != "" else None,
source.catalog if source.catalog != "" else None,
)
for source in sources
for source in _find_table_sources(statement, dialect)
}
def count_referenced_tables(statement: str, dialect: Dialects | str | None) -> int:
"""
Count the table references in a raw SQL string.
This counts occurrences, not distinct tables, so a self-join referencing
the same physical table twice (via two aliases) is still counted as 2 -
callers use this count to decide whether a statement is a join, and a
self-join needs the same treatment as a join across different tables.
A CTE that's referenced more than once (e.g. self-joined) is weighted the
same way: each reference to it counts its own underlying tables again,
since a CTE is inlined at every place it's used (see
``_count_weighted_table_references``).
Falls back to a conservative count of 1 (i.e. "not multi-table") if the
statement can't be parsed, since callers gating multi-table-only behavior
on this count should default to treating an unparseable statement as a
single table.
"""
try:
_check_script_length(statement, str(dialect) if dialect else None)
parsed = sqlglot.parse_one(statement, dialect=dialect)
if isinstance(parsed, exp.Show):
return len(_find_show_statement_tables(parsed))
if isinstance(parsed, (exp.Describe, exp.Command)):
# Neither has join semantics for a per-table row cap to interact
# with, so the plain (unweighted) extraction already used for
# permissioning is fine here too.
return len(_find_table_sources(parsed, dialect))
return _count_weighted_table_references(parsed)
except Exception: # pylint: disable=broad-except
return 1
def _count_weighted_table_references(statement: exp.Expression) -> int:
"""
Count table references the way callers gating multi-table-only behavior
need: weighting each CTE by how many times it's actually referenced,
not by how many distinct tables its own definition reads.
``_find_table_sources`` (used for permissioning) intentionally counts a
CTE's underlying tables exactly once regardless of how many times the
CTE is referenced downstream, since permission checks only care about
the *set* of tables read. But a CTE that wraps a single virtual table
and is then self-joined N ways is inlined at each of those N places, so
it triggers N separate reads of that table -- one per join side -- and
must count as N here too. Otherwise a per-table row cap (see
``SUPERSET_META_DB_LIMIT`` and #36304) looks safe to apply and silently
truncates one side of the self-join away before the join runs.
"""
def resolve(scope: Scope, seen: frozenset[int]) -> list[exp.Table]:
if id(scope) in seen:
return [] # guards a WITH RECURSIVE self-reference from looping forever
seen = seen | {id(scope)}
tables: list[exp.Table] = []
for _, source in scope.selected_sources.values():
if isinstance(source, exp.Table) and not is_cte(source, scope):
tables.append(source)
elif isinstance(source, Scope) and source.scope_type == ScopeType.CTE:
tables.extend(resolve(source, seen))
return tables
return sum(
len(resolve(scope, frozenset()))
for scope in traverse_scope(statement)
if scope.scope_type != ScopeType.CTE
)
def is_cte(source: exp.Table, scope: Scope) -> bool:
"""
Does this reference resolve to a CTE rather than to a real table?
+3 -7
View File
@@ -67,6 +67,7 @@ from superset.utils.core import (
QuerySource,
zlib_compress,
)
from superset.utils.database import warm_and_release_connection
from superset.utils.dates import now_as_float
from superset.utils.decorators import stats_timing
from superset.utils.rls import apply_rls
@@ -287,13 +288,8 @@ def execute_query( # pylint: disable=too-many-statements, too-many-locals # no
# that stays idle for the query duration; if the query runs longer
# than the DB's idle_in_transaction_session_timeout the connection
# is killed, leaving the query stuck in "running" state forever.
db.session.expire_on_commit = False
try:
db.session.refresh(query)
_ = query.database
db.session.commit()
finally:
db.session.expire_on_commit = True
db.session.refresh(query)
warm_and_release_connection(query, "database")
with event_logger.log_context(
action="execute_sql",
database=database,
+17 -12
View File
@@ -32,6 +32,7 @@ import logging
from collections.abc import Iterator
from datetime import datetime, timedelta
from typing import Any, cast
from uuid import UUID
import sqlalchemy as sa
from flask import current_app
@@ -45,6 +46,7 @@ from superset.commands.deletion_retention.purge_cascade import (
entity_uuid,
suppress_purge_association_versions,
)
from superset.commands.deletion_retention.purge_policy import BlockerReason
from superset.commands.deletion_retention.window import resolve_retention_window
from superset.extensions import celery_app, feature_flag_manager, stats_logger_manager
from superset.models.helpers import (
@@ -205,6 +207,19 @@ def _purge_model(
return purged, would, failures, blocked
def _finalize_blocked(record_id: UUID | None, blocker: BlockerReason) -> None:
"""Finalize a blocked retention outcome and count suppression metrics."""
disposition: audit.RetentionBlockedDisposition = audit.finalize_retention_blocked(
record_id, blocker.code
)
if disposition == "suppressed":
stats_logger_manager.instance.incr(f"{_METRIC_PREFIX}.blocked_audit_suppressed")
elif disposition == "fallback":
stats_logger_manager.instance.incr(
f"{_METRIC_PREFIX}.blocked_audit_dedupe_fallback"
)
def _purge_one(
model: type[SoftDeleteMixin], entity_id: int, cutoff: datetime
) -> CascadeResult | None:
@@ -279,18 +294,8 @@ def _purge_one(
affected_referrers=result.dangling_chart_uuids,
removed_dashboard_slices=result.removed_dashboard_slices,
)
elif result.blocked_reason is not None:
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
)
if disposition == "suppressed":
stats_logger_manager.instance.incr(
f"{_METRIC_PREFIX}.blocked_audit_suppressed"
)
elif disposition == "fallback":
stats_logger_manager.instance.incr(
f"{_METRIC_PREFIX}.blocked_audit_dedupe_fallback"
)
elif result.blocker is not None:
_finalize_blocked(record_id, result.blocker)
else:
audit.fail(record_id)
return result
@@ -379,37 +379,37 @@ msgstr "虛擬"
msgid "%s aggregates(s)"
msgstr "%s 聚合"
#, fuzzy, python-format
#, python-format
msgid "%s column"
msgid_plural "%s columns"
msgstr[0] "%s "
msgstr[0] "%s 個欄位"
#, python-format
msgid "%s column(s)"
msgstr "%s "
msgstr "%s 個欄位"
#, fuzzy, python-format
#, python-format
msgid "%s day ago"
msgid_plural "%s days ago"
msgstr[0] "1前"
msgstr[0] "%s 天前"
#, fuzzy, python-format
#, python-format
msgid "%s hr ago"
msgid_plural "%s hr ago"
msgstr[0] "%s "
msgstr[0] "%s 小時前"
#, fuzzy, python-format
#, python-format
msgid "%s imported"
msgstr "數據集已導入"
msgstr "已匯入 %s"
#, fuzzy, python-format
#, python-format
msgid "%s item"
msgid_plural "%s items"
msgstr[0] "%s 個項"
msgstr[0] "%s 個項"
#, fuzzy, python-format
#, python-format
msgid "%s item(s)"
msgstr "%s 個項"
msgstr "%s 個項"
# Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ar, cs, de,
# es, fa, fr, ja, lv, mi, nl, pl, pt_BR, ru, sk, sl, sr, sr_Latn, tr, uk]
@@ -419,15 +419,15 @@ msgid ""
"all selected objects."
msgstr "%s 個項目無法標記,因為您對所有選取的物件沒有編輯權限。"
#, fuzzy, python-format
#, python-format
msgid "%s metric"
msgid_plural "%s metrics"
msgstr[0] "排序指標"
msgstr[0] "%s 個指標"
#, fuzzy, python-format
#, python-format
msgid "%s min ago"
msgid_plural "%s min ago"
msgstr[0] ""
msgstr[0] "%s 分鐘前"
#, python-format
msgid ""
@@ -451,47 +451,47 @@ msgstr[0] "%s 個選項"
msgid "%s option(s)"
msgstr "%s 個選項"
#, fuzzy, python-format
#, python-format
msgid "%s out of %s column"
msgid_plural "%s out of %s columns"
msgstr[0] "自定義列"
msgstr[0] "已選取 %s%s 個欄位"
#, fuzzy, python-format
#, python-format
msgid "%s out of %s metric"
msgid_plural "%s out of %s metrics"
msgstr[0] "排序指標"
msgstr[0] "已選取 %s%s 個指標"
#, fuzzy, python-format
#, python-format
msgid "%s out of %s selected"
msgstr "%s 已選定"
msgstr "已選取 %s%s 個項目"
#, fuzzy, python-format
#, python-format
msgid "%s recipients"
msgstr "%s 最近"
msgstr "%s 收件者"
#, fuzzy, python-format
#, python-format
msgid "%s record..."
msgid_plural "%s records..."
msgstr[0] "%s 異常"
msgstr[0] "%s 筆記錄..."
#, fuzzy, python-format
#, python-format
msgid "%s row"
msgid_plural "%s rows"
msgstr[0] "%s "
msgstr[0] "%s "
#, fuzzy, python-format
#, python-format
msgid "%s s ago"
msgid_plural "%s s ago"
msgstr[0] "30 天之前"
msgstr[0] "%s 秒前"
#, python-format
msgid "%s saved metric(s)"
msgstr "%s 保存的指標"
#, fuzzy, python-format
#, python-format
msgid "%s second"
msgid_plural "%s seconds"
msgstr[0] "5 秒"
msgstr[0] "%s 秒"
# Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: de, es, sr,
# sr_Latn]
@@ -511,13 +511,13 @@ msgstr "%s 個語意檢視新增失敗"
msgid "%s semantic view(s) failed to add: %s"
msgstr "%s 個語意檢視新增失敗:%s"
#, fuzzy, python-format
#, python-format
msgid "%s tab selected"
msgstr "%s 已選定"
msgstr "已選取「%s」分頁"
#, fuzzy, python-format
#, python-format
msgid "%s updated"
msgstr "上次更新 %s"
msgstr "更新 %s"
#, python-format
msgid "%s%s"
@@ -677,13 +677,11 @@ msgstr "每年年初的頻率"
msgid "10 minute"
msgstr "10 分鐘"
#, fuzzy
msgid "10 seconds"
msgstr "30 秒"
msgstr "10 秒"
#, fuzzy
msgid "10/90 percentiles"
msgstr "9/91 百分位"
msgstr "10/90 百分位"
#. do-not-translate
msgid "10000"
@@ -696,9 +694,8 @@ msgstr "週"
msgid "104 weeks ago"
msgstr "104 週之前"
#, fuzzy
msgid "12 hours"
msgstr "1 小時"
msgstr "12 小時"
msgid "15 minute"
msgstr "15 分鐘"
@@ -761,9 +758,8 @@ msgstr "2/98 百分位"
msgid "22"
msgstr "22"
#, fuzzy
msgid "24 hours"
msgstr "6 小時"
msgstr "24 小時"
#, fuzzy
msgid "28 days"
@@ -831,9 +827,8 @@ msgstr "5 秒"
msgid "5 seconds"
msgstr "5 秒"
#, fuzzy
msgid "5/95 percentiles"
msgstr "9/91 百分位"
msgstr "5/95 百分位"
#, fuzzy
msgid "52 weeks"
+36
View File
@@ -87,6 +87,42 @@ def remove_database(database: Database) -> None:
db.session.flush()
def warm_and_release_connection(instance: Any, *relationships: str) -> None:
"""
Eagerly load the named relationships on ``instance``, then release the
current session's DB connection back to the pool without detaching any
object in the session.
Prefer this over ``db.session.close()`` before slow, non-DB work (a
long-running cursor execution, a results-backend fetch, CPU-bound
decompress/deserialize work) that still needs attributes already
loaded on session objects: ``close()`` detaches every object in the
session -- including ``g.user``, not just ``instance`` -- so a later
attribute access anywhere in the request can raise on a detached
instance or silently open a fresh connection. Committing with
``expire_on_commit`` disabled instead releases the connection while
keeping objects attached and their already-loaded attributes valid.
"""
# pylint: disable=import-outside-toplevel
from superset import db
for relationship in relationships:
getattr(instance, relationship)
# ``db.session`` is a ``scoped_session`` proxy: it only forwards a fixed
# allowlist of attributes to the real ``Session`` (bind, dirty, deleted,
# new, identity_map, is_active, autoflush, no_autoflush, info).
# ``expire_on_commit`` isn't on that list, so setting it on ``db.session``
# directly would silently no-op -- it has to be set on the real Session
# returned by calling the proxy.
session = db.session()
session.expire_on_commit = False
try:
session.commit() # pylint: disable=consider-using-transaction
finally:
session.expire_on_commit = True
def apply_mariadb_ddl_fix() -> None:
"""
Fix MariaDB "NO CYCLE" syntax issue - MariaDB uses "NOCYCLE" (no space).
+33 -8
View File
@@ -60,8 +60,22 @@ def quote_formulas(df: pd.DataFrame) -> pd.DataFrame:
"""
Make sure to quote any formulas for security reasons.
"""
for col in df.select_dtypes(include="object").columns:
df[col] = df[col].apply(_quote_formula)
# Columns are addressed by position rather than by label: a dataframe can
# carry duplicate column labels (the verbose_map rename in
# QueryContextProcessor.get_data can collapse two columns onto the same
# name), and ``df[label]`` then yields a DataFrame instead of a Series.
# ``DataFrame.apply`` would hand whole columns to the mapper rather than
# individual cells, silently leaving formulas unquoted.
for idx in range(len(df.columns)):
series = df.iloc[:, idx]
# ``is_string_dtype`` rather than an ``object`` comparison: pandas 3
# gives string columns a dedicated ``str`` dtype, which an object-only
# check (as the ``select_dtypes(include="object")`` this replaced) would
# skip, silently leaving formulas unquoted.
if pd.api.types.is_object_dtype(series.dtype) or pd.api.types.is_string_dtype(
series.dtype
):
df.isetitem(idx, series.map(_quote_formula))
# Column headers and index labels are written to the sheet as well, and
# pivot exports promote data values into both (a hostile warehouse string
@@ -104,20 +118,31 @@ def apply_column_types(
:param column_types: The types of the columns
:return: The dataframe with the column types applied
"""
for column, column_type in zip(df.columns, column_types, strict=False):
# Columns are addressed by position for the same reason as in
# ``quote_formulas``: duplicate column labels make ``df[label]`` return a
# DataFrame, and ``DataFrame`` has no ``dtype``. Slicing column_types keeps
# the lenient pairing the previous ``zip(..., strict=False)`` provided.
for idx, column_type in enumerate(column_types[: len(df.columns)]):
series = df.iloc[:, idx]
if column_type == GenericDataType.NUMERIC:
try:
df[column] = pd.to_numeric(df[column])
series = pd.to_numeric(series)
# if the number is too large, convert it to a string
# Excel does not support numbers larger than 10^15
df[column] = df[column].apply(
series = series.apply(
lambda x: (
str(x) if isinstance(x, (int, float)) and abs(x) > 10**15 else x
)
)
except ValueError:
df[column] = df[column].astype(str)
elif isinstance(df[column].dtype, pd.DatetimeTZDtype):
series = series.astype(str)
elif isinstance(series.dtype, pd.DatetimeTZDtype):
# timezones are not supported
df[column] = df[column].astype(str)
series = series.astype(str)
else:
continue
# ``isetitem`` replaces the column at that position, which is both
# unambiguous under duplicate labels and free of the in-place dtype
# casting that ``iloc`` assignment attempts.
df.isetitem(idx, series)
return df
@@ -2403,6 +2403,54 @@ class TestDatabaseApi(SupersetTestCase):
assert rv.status_code == 200
assert rv.headers["Content-Type"] == "application/json; charset=utf-8"
@with_config({"PREVENT_UNSAFE_DB_CONNECTIONS": False})
def test_test_connection_oauth2(self):
"""
Database API: Test test connection flow with a connection authenticated via
OAuth2.
The test would always raise ``OAuth2RedirectError``, and we can't start the
OAuth2 dance before the connection is saved, so it should return a 200 status.
"""
self.login(ADMIN_USERNAME)
example_db = get_example_database()
masked_encrypted_extra = json.dumps(
{
"oauth2_client_info": {
"id": "client_id",
"secret": "client_secret",
"scope": "some-scope",
"authorization_request_uri": "https://example.org/authorize",
"token_request_uri": "https://example.org/token",
}
}
)
data = {
"database_name": "examples",
"masked_encrypted_extra": masked_encrypted_extra,
"impersonate_user": True,
"sqlalchemy_uri": example_db.safe_sqlalchemy_uri(),
"server_cert": None,
}
url = "api/v1/database/test_connection/"
with (
mock.patch(
"superset.commands.database.test_connection.ping",
side_effect=Exception("Unauthorized"),
),
mock.patch.object(
example_db.db_engine_spec,
"needs_oauth2",
return_value=True,
),
):
rv = self.post_assert_metric(url, data, "test_connection")
assert rv.status_code == 200
assert rv.headers["Content-Type"] == "application/json; charset=utf-8"
assert json.loads(rv.data.decode("utf-8")) == {"message": "OK"}
def test_test_connection_failed(self):
"""
Database API: Test test connection failed
@@ -29,6 +29,10 @@ from sqlalchemy.orm import Session
from superset import db
from superset.commands.deletion_retention import audit
from superset.commands.deletion_retention.audit import PurgeAuditLog
from superset.commands.deletion_retention.purge_policy import (
REASON_CASCADE_INTEGRITY_FAILURE,
REASON_REPORT_SCHEDULE,
)
from superset.models.slice import Slice
from superset.tasks.deletion_retention import _purge_impl
@@ -75,6 +79,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
assert row.trigger == audit.TRIGGER_RETENTION
assert row.actor == audit.ACTOR_SYSTEM
assert row.confirmed_on is not None
assert row.reason is None
assert isinstance(row.id, UUID)
def test_known_failure_finalizes_audit_row(self) -> None:
@@ -97,6 +102,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
row = db.session.query(PurgeAuditLog).filter_by(entity_uuid=chart_uuid).one()
assert row.status == audit.STATUS_FAILED
assert row.confirmed_on is None
assert row.reason is None
def test_reconcile_confirms_pending_after_entity_commit(self) -> None:
"""A crash after entity commit is reconciled to confirmed."""
@@ -172,6 +178,9 @@ class TestPurgeAudit(DeletionRetentionTestBase):
row = db.session.get(PurgeAuditLog, record_id)
assert row.status == audit.STATUS_TARGET_ABSENT
assert row.removed_dashboard_slices == 0
# The reconcile crash window is the documented reason-losing path:
# finalized rows here never carry a fabricated code.
assert row.reason is None
def test_blocked_attempt_does_not_keep_the_intended_removal_count(self) -> None:
"""The write-ahead row records what the purge INTENDED to remove;
@@ -184,7 +193,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
entity_uuid="00000000-0000-0000-0000-00000000cafe",
removed_dashboard_slices=7,
)
audit.block(record_id)
audit.block(record_id, REASON_REPORT_SCHEDULE)
row = db.session.query(PurgeAuditLog).filter_by(id=record_id).one()
assert row.status == audit.STATUS_BLOCKED
@@ -194,20 +203,56 @@ class TestPurgeAudit(DeletionRetentionTestBase):
record_id: UUID = self._write_retention_record(entity_uuid="first-block")
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(record_id)
assert disposition == "retained"
assert record.status == audit.STATUS_BLOCKED
assert record.reason == REASON_REPORT_SCHEDULE
def test_reason_is_persisted_only_for_blocked_outcomes(self) -> None:
"""A reason offered for a non-blocked outcome is refused, not stored.
The audit records a cause for a purge that did not happen; a
confirmed or failed row asserting a blocker would misreport its own
outcome.
"""
for status in (
audit.STATUS_CONFIRMED,
audit.STATUS_FAILED,
audit.STATUS_TARGET_ABSENT,
):
record_id: UUID = self._write_retention_record(
entity_uuid=f"non-blocked-{status}"
)
audit.finalize(record_id, status, reason=REASON_REPORT_SCHEDULE)
record: PurgeAuditLog = self._get_audit_record(record_id)
db.session.refresh(record)
assert record.status == status
assert record.reason is None
def test_finalized_reason_is_immutable(self) -> None:
"""A second finalization attempt never rewrites the recorded reason."""
record_id: UUID = self._write_retention_record(entity_uuid="reason-immutable")
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
audit.block(record_id, "some_other_code")
audit.finalize_retention_blocked(record_id, "some_other_code")
record: PurgeAuditLog = self._get_audit_record(record_id)
db.session.refresh(record)
assert record.status == audit.STATUS_BLOCKED
assert record.reason == REASON_REPORT_SCHEDULE
def test_repeated_retention_block_suppresses_current_provisional(self) -> None:
first_id: UUID = self._write_retention_record(entity_uuid="repeat-block")
audit.finalize_retention_blocked(first_id)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
second_id: UUID = self._write_retention_record(entity_uuid="repeat-block")
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(second_id)
audit.finalize_retention_blocked(second_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "suppressed"
@@ -215,18 +260,160 @@ class TestPurgeAudit(DeletionRetentionTestBase):
assert first.status == audit.STATUS_BLOCKED
assert db.session.get(PurgeAuditLog, second_id) is None
def test_reason_change_breaks_suppression_exactly_once(self) -> None:
"""A reason change writes one new blocked row, then re-suppresses.
The suppression predicate keys on status AND reason: same-reason
nights suppress; the night the reason changes is retained with the
new code and becomes the new anchor.
"""
entity: str = "reason-change"
first_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
== "retained"
)
second_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(second_id, REASON_REPORT_SCHEDULE)
== "suppressed"
)
changed_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(
changed_id, REASON_CASCADE_INTEGRITY_FAILURE
)
== "retained"
)
repeat_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(
repeat_id, REASON_CASCADE_INTEGRITY_FAILURE
)
== "suppressed"
)
rows: list[PurgeAuditLog] = (
db.session.query(PurgeAuditLog).filter_by(entity_uuid=entity).all()
)
assert {row.reason for row in rows} == {
REASON_REPORT_SCHEDULE,
REASON_CASCADE_INTEGRITY_FAILURE,
}
assert len(rows) == 2
assert all(row.status == audit.STATUS_BLOCKED for row in rows)
def test_mixed_reason_timestamp_tie_is_ambiguous_and_retains(self) -> None:
"""Tied predecessors differing only in reason refuse suppression."""
timestamp: datetime = datetime.utcnow()
first_id: UUID = self._write_retention_record(
entity_uuid="mixed-reason-tie", created_on=timestamp
)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
second_id: UUID = self._write_retention_record(
entity_uuid="mixed-reason-tie", created_on=timestamp
)
audit.finalize_retention_blocked(second_id, REASON_CASCADE_INTEGRITY_FAILURE)
current_id: UUID = self._write_retention_record(
entity_uuid="mixed-reason-tie", created_on=timestamp + timedelta(seconds=1)
)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "retained"
current: PurgeAuditLog = self._get_audit_record(current_id)
assert current.status == audit.STATUS_BLOCKED
def test_null_reason_historical_predecessor_never_suppresses(self) -> None:
"""The first post-upgrade block of a long-blocked entity is retained.
Pre-feature blocked rows carry NULL; NULL never matches a current
code, so the entity anchors once with its code and same-code nights
suppress against the new anchor.
"""
entity: str = "null-historical"
prior_id: UUID = self._write_retention_record(entity_uuid=entity)
audit.finalize(prior_id, audit.STATUS_BLOCKED)
prior: PurgeAuditLog = self._get_audit_record(prior_id)
assert prior.reason is None
current_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
== "retained"
)
current: PurgeAuditLog = self._get_audit_record(current_id)
assert current.reason == REASON_REPORT_SCHEDULE
repeat_id: UUID = self._write_retention_record(entity_uuid=entity)
assert (
audit.finalize_retention_blocked(repeat_id, REASON_REPORT_SCHEDULE)
== "suppressed"
)
def test_none_current_code_never_suppresses_and_warns(self) -> None:
"""A missing current code fails safe: retained, with a warning."""
entity: str = "none-current-code"
first_id: UUID = self._write_retention_record(entity_uuid=entity)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
current_id: UUID = self._write_retention_record(entity_uuid=entity)
with patch(
"superset.commands.deletion_retention.audit.logger.warning"
) as warning:
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id, None)
)
assert disposition == "retained"
assert warning.called
current: PurgeAuditLog = self._get_audit_record(current_id)
assert current.status == audit.STATUS_BLOCKED
assert current.reason is None
def test_predecessor_is_the_latest_row_overall(self) -> None:
"""A newer same-entity row forbids suppressing against an older one.
With rows timestamped both before and after the current attempt, the
later-timestamped row is selected, fails the strictly-older check, and
causes retention. This verifies timestamp ordering, not causal order
across workers.
"""
timestamp: datetime = datetime.utcnow()
older_id: UUID = self._write_retention_record(
entity_uuid="latest-overall", created_on=timestamp - timedelta(seconds=1)
)
audit.finalize_retention_blocked(older_id, REASON_REPORT_SCHEDULE)
newer_id: UUID = self._write_retention_record(
entity_uuid="latest-overall", created_on=timestamp + timedelta(seconds=1)
)
audit.finalize(newer_id, audit.STATUS_BLOCKED, reason=REASON_REPORT_SCHEDULE)
current_id: UUID = self._write_retention_record(
entity_uuid="latest-overall", created_on=timestamp
)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "retained"
current: PurgeAuditLog = self._get_audit_record(current_id)
assert current.status == audit.STATUS_BLOCKED
def test_equal_timestamp_is_ambiguous_and_retains_current(self) -> None:
timestamp: datetime = datetime.utcnow()
first_id: UUID = self._write_retention_record(
entity_uuid="equal-time", created_on=timestamp
)
audit.finalize_retention_blocked(first_id)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
second_id: UUID = self._write_retention_record(
entity_uuid="equal-time", created_on=timestamp
)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(second_id)
audit.finalize_retention_blocked(second_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "retained"
@@ -257,7 +444,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
session.close()
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
assert predecessor is None
@@ -273,10 +460,10 @@ class TestPurgeAudit(DeletionRetentionTestBase):
newer_id: UUID = self._write_retention_record(
entity_uuid="overlap", created_on=current_time + timedelta(seconds=1)
)
audit.finalize_retention_blocked(newer_id)
audit.finalize_retention_blocked(newer_id, REASON_REPORT_SCHEDULE)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "retained"
@@ -291,7 +478,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
assert disposition == "retained"
@@ -300,7 +487,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
null_id: UUID = self._write_retention_record(entity_uuid=None)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(null_id)
audit.finalize_retention_blocked(null_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(null_id)
@@ -312,13 +499,13 @@ class TestPurgeAudit(DeletionRetentionTestBase):
chart_id: UUID = self._write_retention_record(
entity_uuid="shared-type", entity_type="slices"
)
audit.finalize_retention_blocked(chart_id)
audit.finalize_retention_blocked(chart_id, REASON_REPORT_SCHEDULE)
dashboard_id: UUID = self._write_retention_record(
entity_uuid="shared-type", entity_type="dashboards"
)
dashboard_disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(dashboard_id)
audit.finalize_retention_blocked(dashboard_id, REASON_REPORT_SCHEDULE)
)
assert dashboard_disposition == "retained"
@@ -328,7 +515,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
audit.fail(record_id)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(record_id)
@@ -343,7 +530,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
side_effect=audit.SQLAlchemyError("lookup failed"),
):
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(record_id)
@@ -352,7 +539,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
def test_suppression_delete_failure_recovers_blocked_evidence(self) -> None:
first_id: UUID = self._write_retention_record(entity_uuid="delete-failure")
audit.finalize_retention_blocked(first_id)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
current_id: UUID = self._write_retention_record(entity_uuid="delete-failure")
with patch(
@@ -360,7 +547,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
side_effect=audit.SQLAlchemyError("delete failed"),
):
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(current_id)
@@ -383,16 +570,18 @@ class TestPurgeAudit(DeletionRetentionTestBase):
),
):
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(record_id)
assert disposition == "fallback"
assert record.status == audit.STATUS_BLOCKED
# The recovery retain branch carries the argument-sourced snapshot reason.
assert record.reason == REASON_REPORT_SCHEDULE
def test_uncertain_suppression_commit_recreates_absent_evidence(self) -> None:
first_id: UUID = self._write_retention_record(entity_uuid="absent-current")
audit.finalize_retention_blocked(first_id)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
current_id: UUID = self._write_retention_record(entity_uuid="absent-current")
primary_session: Session = audit._dedicated_session()
recovery_session: Session = audit._dedicated_session()
@@ -410,12 +599,16 @@ class TestPurgeAudit(DeletionRetentionTestBase):
),
):
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(current_id)
assert disposition == "fallback"
assert record.status == audit.STATUS_BLOCKED
# The recovery re-insert branch sources the reason from the snapshot
# (populated from the call argument, never from the reason-less
# pending row).
assert record.reason == REASON_REPORT_SCHEDULE
def test_failed_fallback_leaves_pending_evidence_for_reconciliation(self) -> None:
record_id: UUID = self._write_retention_record(entity_uuid="fallback-failure")
@@ -438,7 +631,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
),
):
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(record_id)
audit.finalize_retention_blocked(record_id, REASON_REPORT_SCHEDULE)
)
record: PurgeAuditLog = self._get_audit_record(record_id)
@@ -464,31 +657,34 @@ class TestPurgeAudit(DeletionRetentionTestBase):
entity_type="slices",
entity_uuid="indeterminate-rowcount",
created_on=timestamp - timedelta(seconds=1),
reason=REASON_REPORT_SCHEDULE,
)
result: MagicMock = MagicMock(rowcount=-1)
session: MagicMock = MagicMock()
session.execute.return_value = result
with pytest.raises(audit.SQLAlchemyError, match="indeterminate"):
audit._suppress_redundant_block(session, current, predecessor)
audit._suppress_redundant_block(
session, current, predecessor, REASON_REPORT_SCHEDULE
)
def test_overlap_duplicates_do_not_cause_unbounded_sequential_growth(self) -> None:
timestamp: datetime = datetime.utcnow()
first_id: UUID = self._write_retention_record(
entity_uuid="bounded-overlap", created_on=timestamp
)
audit.finalize_retention_blocked(first_id)
audit.finalize_retention_blocked(first_id, REASON_REPORT_SCHEDULE)
overlap_id: UUID = self._write_retention_record(
entity_uuid="bounded-overlap", created_on=timestamp
)
audit.finalize_retention_blocked(overlap_id)
audit.finalize_retention_blocked(overlap_id, REASON_REPORT_SCHEDULE)
later_id: UUID = self._write_retention_record(
entity_uuid="bounded-overlap",
created_on=timestamp + timedelta(seconds=1),
)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(later_id)
audit.finalize_retention_blocked(later_id, REASON_REPORT_SCHEDULE)
)
retained_count: int = (
@@ -516,7 +712,7 @@ class TestPurgeAudit(DeletionRetentionTestBase):
current_id: UUID = self._write_retention_record(entity_uuid=entity_uuid)
disposition: audit.RetentionBlockedDisposition = (
audit.finalize_retention_blocked(current_id)
audit.finalize_retention_blocked(current_id, REASON_REPORT_SCHEDULE)
)
current: PurgeAuditLog = self._get_audit_record(current_id)
@@ -140,6 +140,7 @@ class TestForcePurge(DeletionRetentionTestBase):
assert self.exists(Slice, chart_id)
row = db.session.query(PurgeAuditLog).filter_by(entity_uuid=chart_uuid).one()
assert row.status == "blocked"
assert row.reason == "report_schedule"
log_info.assert_called_once_with(
"force_purge: blocked %s uuid=%s reason=%s",
"chart",
@@ -24,6 +24,7 @@ guarantee under FK enforcement OFF, and the version-tables-absent no-op.
from __future__ import annotations
from collections.abc import Callable
from dataclasses import replace
from datetime import datetime, timedelta
from typing import Any
@@ -45,6 +46,9 @@ from superset.commands.deletion_retention.purge_cascade import (
from superset.commands.deletion_retention.purge_policy import (
get_purge_policy,
PurgeEntityPolicy,
REASON_CASCADE_INTEGRITY_FAILURE,
REASON_REPORT_SCHEDULE,
REASON_USER_ATTRIBUTE,
)
from superset.connectors.sqla.models import (
RLSFilterTables,
@@ -286,6 +290,7 @@ class TestSoftDeletePurge(DeletionRetentionTestBase):
.one()
)
assert row.status == audit.STATUS_BLOCKED
assert row.reason == REASON_REPORT_SCHEDULE
def test_repeated_report_blocker_preserves_counts_and_suppresses_noise(
self,
@@ -768,6 +773,7 @@ class TestExplicitBlockerGuards(DeletionRetentionTestBase):
assert result.purged is False
assert result.blocked_reason is not None
assert "welcome page" in result.blocked_reason
assert result.blocked_reason_code == REASON_USER_ATTRIBUTE
assert self.exists(Dashboard, dashboard_id)
finally:
self._restore_welcome(attribute, created, previous)
@@ -812,10 +818,120 @@ class TestExplicitBlockerGuards(DeletionRetentionTestBase):
db.session.commit()
assert result.purged is False
assert result.blocked_reason == "blocked by database references"
assert (
result.blocked_reason
== "cascade blocked by a database integrity constraint"
)
assert "SQL:" not in result.blocked_reason
assert result.blocked_reason_code == REASON_CASCADE_INTEGRITY_FAILURE
assert self.exists(Slice, chart_id)
def test_three_way_distinction_is_readable_from_the_audit_alone(self) -> None:
"""Report, welcome, and database-integrity blocks write distinct codes.
The audit table is the durable record: each of the three
non-completing outcomes must be identifiable from its row alone,
with no SQL fragments and the integrity case keeping blocked status.
"""
chart: Slice = self.make_chart("threeway_report")
report: ReportSchedule = ReportSchedule(
type="Report",
name="retention_it_threeway",
crontab="0 0 * * *",
chart=chart,
)
db.session.add(report)
db.session.commit()
chart_uuid: str = str(chart.uuid)
self.soft_delete(chart, days_ago=90)
dashboard: Dashboard = self.make_dashboard("threeway_welcome")
dashboard_uuid: str = str(dashboard.uuid)
self.soft_delete(dashboard, days_ago=90)
attribute: UserAttribute
created: bool
previous: int | None
attribute, created, previous = self._set_welcome(dashboard.id)
fk_chart: Slice = self.make_chart("threeway_fk")
fk_uuid: str = str(fk_chart.uuid)
self.soft_delete(fk_chart, days_ago=90)
real_get_policy: Callable[[type[Any]], PurgeEntityPolicy] = get_purge_policy
def fail_fk_chart_cleanup(
session: Session, policy: PurgeEntityPolicy, entity_id: int
) -> None:
if entity_id == fk_chart.id:
raise IntegrityError("FOREIGN KEY constraint failed", None, Exception())
real_get_policy(Slice).delete_associations(session, policy, entity_id)
def patched_policy(model: type[Any]) -> PurgeEntityPolicy:
policy: PurgeEntityPolicy = real_get_policy(model)
if model is Slice:
return replace(policy, delete_associations=fail_fk_chart_cleanup)
return policy
try:
with patch(
"superset.commands.deletion_retention.purge_cascade.get_purge_policy",
side_effect=patched_policy,
):
_purge(window=30)
rows: dict[str, audit.PurgeAuditLog] = {
uuid: db.session.query(audit.PurgeAuditLog)
.filter_by(entity_uuid=uuid)
.one()
for uuid in (chart_uuid, dashboard_uuid, fk_uuid)
}
assert rows[chart_uuid].reason == REASON_REPORT_SCHEDULE
assert rows[dashboard_uuid].reason == REASON_USER_ATTRIBUTE
assert rows[fk_uuid].reason == REASON_CASCADE_INTEGRITY_FAILURE
assert len({row.reason for row in rows.values()}) == 3
for row in rows.values():
assert row.status == audit.STATUS_BLOCKED
assert "SQL" not in row.reason
assert "?" not in row.reason
finally:
self._restore_welcome(attribute, created, previous)
def test_first_declared_blocker_wins_in_the_audit_record(self) -> None:
"""A dashboard blocked by both rules records the first-declared code.
Declaration order is part of the audit contract: report_schedule is
declared before user_attribute, so a dashboard that is both
report-referenced and someone's welcome page records
REASON_REPORT_SCHEDULE.
"""
dashboard: Dashboard = self.make_dashboard("firstmatch")
report: ReportSchedule = ReportSchedule(
type="Report",
name="retention_it_firstmatch",
crontab="0 0 * * *",
dashboard=dashboard,
)
db.session.add(report)
db.session.commit()
dashboard_uuid: str = str(dashboard.uuid)
self.soft_delete(dashboard, days_ago=90)
attribute: UserAttribute
created: bool
previous: int | None
attribute, created, previous = self._set_welcome(dashboard.id)
try:
_purge(window=30)
row: audit.PurgeAuditLog = (
db.session.query(audit.PurgeAuditLog)
.filter_by(entity_uuid=dashboard_uuid)
.one()
)
assert row.status == audit.STATUS_BLOCKED
assert row.reason == REASON_REPORT_SCHEDULE
finally:
self._restore_welcome(attribute, created, previous)
def test_policy_action_failure_rolls_back_prior_phases(self) -> None:
"""A later policy-action failure restores earlier association cleanup."""
chart: Slice = self.make_chart("action_rollback")
@@ -3147,6 +3147,7 @@ def test__send_with_server_errors(notification_mock, logger_mock):
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.reports.notifications.email.send_email_smtp")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -3191,6 +3192,7 @@ def test_retry_on_failure_schedules_retry(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.commands.report.execute.BaseReportState.send_retry_notification")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -3247,6 +3249,7 @@ def test_retry_exhausted_transitions_to_error(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.commands.report.execute.BaseReportState.send_final_failure_report")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -3291,6 +3294,7 @@ def test_send_failed_reports_sends_to_recipients(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
def test_retrying_state_schedules_another_retry(
@@ -3336,6 +3340,7 @@ def test_retrying_state_schedules_another_retry(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.reports.notifications.email.send_email_smtp")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -3372,6 +3377,7 @@ def test_retry_disabled_preserves_default_error_path(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.commands.report.execute.BaseReportState.send_retry_notification")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -3414,6 +3420,7 @@ def test_retry_notify_owners_sends_notification(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
def test_new_crontab_window_skipped_while_retrying(
@@ -3462,6 +3469,7 @@ def test_new_crontab_window_skipped_while_retrying(
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
@with_feature_flags(ALERT_REPORTS_RETRY=True)
@patch("superset.commands.report.execute.ReportNotTriggeredErrorState._schedule_retry")
@patch("superset.reports.notifications.email.send_email_smtp")
@patch("superset.utils.screenshots.ChartScreenshot.get_screenshot")
@@ -14,6 +14,7 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from typing import Any
from unittest import mock
from unittest.mock import Mock, patch
@@ -22,10 +23,13 @@ import pytest
from flask import current_app
from flask_babel import gettext as __
from jinja2.exceptions import TemplateError, TemplateSyntaxError
from sqlalchemy import inspect as sa_inspect
from sqlalchemy.orm import object_session
from superset import db, sql_lab
from superset.commands.sql_lab import estimate, export, results
from superset.common.db_query_status import QueryStatus
from superset.db_engine_specs.base import BaseEngineSpec
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import (
SerializationError,
@@ -35,6 +39,7 @@ from superset.exceptions import (
)
from superset.models.core import Database # noqa: F401
from superset.models.sql_lab import Query
from superset.result_set import SupersetResultSet
from superset.sqllab.limiting_factor import LimitingFactor
from superset.sqllab.schemas import EstimateQueryCostSchema
from superset.utils import core as utils
@@ -429,6 +434,93 @@ class TestSqlExecutionResultsCommand(SupersetTestCase):
)
assert ex_info.value.status == 400
@pytest.mark.usefixtures("create_database_and_query")
@patch("superset.commands.sql_lab.results.results_backend_use_msgpack", False)
def test_validation_releases_db_connection_before_fetching_from_results_backend(
self,
) -> None:
# The DB connection must be released back to the pool (via
# warm_and_release_connection(), not a full session close -- see
# ``test_validation_warms_database_relationship_before_releasing_connection``
# for why) before the (potentially slow) results-backend fetch, so a
# large download doesn't hold a connection out of the pool for its
# duration.
#
# This spies on ``warm_and_release_connection`` itself rather than on
# ``db.session.commit`` -- the latter is a ``scoped_session`` proxy
# method, and patching it wouldn't be observed by the real
# ``Session`` object that ``warm_and_release_connection`` commits
# (see the fix for the analogous ``expire_on_commit`` proxy pitfall).
call_order: list[str] = []
original_warm_and_release_connection = results.warm_and_release_connection
def tracked_warm_and_release_connection(
instance: Any, *relationships: str
) -> None:
call_order.append("connection_released")
original_warm_and_release_connection(instance, *relationships)
def tracked_get(key: str) -> None:
call_order.append("results_backend_get")
return None
results.results_backend = mock.Mock()
results.results_backend.get.side_effect = tracked_get
command = results.SqlExecutionResultsCommand("abc_query", 1000)
admin = self.get_user("admin")
with current_app.test_request_context():
with override_user(admin):
with mock.patch(
"superset.commands.sql_lab.results.warm_and_release_connection",
side_effect=tracked_warm_and_release_connection,
):
with pytest.raises(SupersetErrorException):
# ``get`` returns ``None`` above, so validation goes
# on to raise the "results missing" (410) error --
# irrelevant here, we only care about the call order
# leading up to it.
command.validate()
assert call_order == ["connection_released", "results_backend_get"]
@pytest.mark.usefixtures("create_database_and_query")
@patch("superset.commands.sql_lab.results.results_backend_use_msgpack", False)
def test_validation_warms_database_relationship_before_releasing_connection(
self,
) -> None:
# ``run`` needs ``self._query.database.db_engine_spec`` after the
# connection has been released by ``validate``. The relationship
# must therefore already be loaded by then, and the query must stay
# attached to the session (unlike a full ``db.session.close()``,
# which would detach every object in the session -- including
# ``g.user`` -- not just the query), or accessing it later would
# either raise (detached instance with an unloaded attribute) or
# silently open a fresh, unwanted connection.
data = [{"col_0": i} for i in range(104)]
payload = {
"status": QueryStatus.SUCCESS,
"query": {"rows": 104},
"data": data,
}
serialized_payload = sql_lab._serialize_payload(payload, False)
compressed = utils.zlib_compress(serialized_payload)
results.results_backend = mock.Mock()
results.results_backend.get.return_value = compressed
command = results.SqlExecutionResultsCommand("abc_query", 1000)
admin = self.get_user("admin")
with current_app.test_request_context():
with override_user(admin):
command.validate()
assert object_session(command._query) is not None
assert "database" not in sa_inspect(command._query).unloaded
assert command._query.database is not None
@pytest.mark.usefixtures("create_database_and_query")
@patch("superset.commands.sql_lab.results.results_backend_use_msgpack", False)
def test_run_succeeds(self) -> None:
@@ -452,4 +544,51 @@ class TestSqlExecutionResultsCommand(SupersetTestCase):
assert result.get("status") == "success"
assert result["query"].get("rows") == 104
assert result.get("data") == data
@pytest.mark.usefixtures("create_database_and_query")
@patch("superset.commands.sql_lab.results.results_backend_use_msgpack", True)
def test_run_succeeds_with_msgpack(self) -> None:
# ``query.database.db_engine_spec`` is only touched in the
# ``use_msgpack=True`` branch of ``_deserialize_results_payload`` --
# which is the production default. All the other tests here run
# with msgpack off, so this exercises the full ``run()`` path with
# msgpack on, to catch a regression that leaves ``query.database``
# unloaded or detached after ``validate()`` releases the connection.
cursor_descr = (
("a", "string", None, None, None, None, True),
("b", "int", None, None, None, None, True),
("c", "float", None, None, None, None, True),
)
result_set = SupersetResultSet(
[("a", 4, 4.0)],
cursor_descr,
BaseEngineSpec,
)
(
serialized_data,
selected_columns,
all_columns,
expanded_columns,
) = sql_lab._serialize_and_expand_data(result_set, BaseEngineSpec(), True)
payload = {
"status": QueryStatus.SUCCESS,
"query": {"rows": 1},
"data": serialized_data,
"columns": all_columns,
"selected_columns": selected_columns,
"expanded_columns": expanded_columns,
}
serialized_payload = sql_lab._serialize_payload(payload, True)
compressed = utils.zlib_compress(serialized_payload)
results.results_backend = mock.Mock()
results.results_backend.get.return_value = compressed
admin = self.get_user("admin")
with current_app.test_request_context():
with override_user(admin):
command = results.SqlExecutionResultsCommand("abc_query", 1000)
result = command.run()
assert result.get("status") == "success"
assert result["data"]
@@ -0,0 +1,182 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Golden-set tests pinning the purge-audit reason-code vocabulary."""
from __future__ import annotations
from dataclasses import replace
from unittest.mock import MagicMock
import pytest
from superset.commands.deletion_retention.purge_policy import (
ALL_REASON_CODES,
DependencyClassification,
DependencyPolicy,
get_purge_policy,
purge_policy_registry,
PurgeBlockedError,
PurgeEntityPolicy,
REASON_CASCADE_INTEGRITY_FAILURE,
REASON_REPORT_SCHEDULE,
REASON_USER_ATTRIBUTE,
validate_deletion_allowed,
)
def test_reason_code_literals_are_frozen() -> None:
"""The persisted code values are frozen identifiers.
Audit history and the suppression predicate compare these exact strings;
a physical table rename or constant refactor must not re-mint them. If
this test fails, the fix is to restore the literal, never to update the
expectation.
"""
assert REASON_REPORT_SCHEDULE == "report_schedule"
assert REASON_USER_ATTRIBUTE == "user_attribute"
assert REASON_CASCADE_INTEGRITY_FAILURE == "cascade_integrity_failure"
assert ALL_REASON_CODES == {
"report_schedule",
"user_attribute",
"cascade_integrity_failure",
}
def test_reason_codes_are_distinct_and_column_sized() -> None:
"""Codes are mutually distinct and fit the String(64) audit column."""
codes: list[str] = [
REASON_REPORT_SCHEDULE,
REASON_USER_ATTRIBUTE,
REASON_CASCADE_INTEGRITY_FAILURE,
]
assert len(set(codes)) == len(codes)
assert all(0 < len(code) <= 64 for code in ALL_REASON_CODES)
def test_every_declared_blocker_code_is_in_the_closed_set() -> None:
"""Each blocker declared in the registry carries a code from ALL_REASON_CODES."""
blocker_codes: set[str] = set()
for policy in purge_policy_registry().values():
for dependency in policy.dependencies:
if dependency.classification is DependencyClassification.BLOCK:
assert dependency.blocker is not None, (
f"blocker {dependency.key.describe()} has no reason code"
)
blocker_codes.add(dependency.blocker.code)
assert blocker_codes <= ALL_REASON_CODES
assert blocker_codes == {REASON_REPORT_SCHEDULE, REASON_USER_ATTRIBUTE}
def test_cascade_integrity_failure_code_is_reserved_for_the_cascade() -> None:
"""No declared policy blocker may claim the cascade-failure code."""
for policy in purge_policy_registry().values():
for dependency in policy.dependencies:
assert (
dependency.blocker is None
or dependency.blocker.code != REASON_CASCADE_INTEGRITY_FAILURE
)
def _session_matching_blockers(*matches: bool) -> MagicMock:
"""A mock session whose Nth blocker query reports a match iff matches[N].
Deliberately positional: which blocker matches first is the audit
contract under test, so these cases are coupled to the order (and the
count) of the queries ``validate_deletion_allowed`` issues.
"""
session: MagicMock = MagicMock()
session.execute.side_effect = [
MagicMock(first=MagicMock(return_value=(1,) if match else None))
for match in matches
]
return session
def test_report_block_raises_with_the_report_schedule_code() -> None:
"""A chart blocked by a report reference carries REASON_REPORT_SCHEDULE."""
# avoid app-init regression: superset.models.* evaluates
# encrypted_field_factory at class-definition time, which fails
# in a partial-collection unit run with no Flask app active.
from superset.models.slice import Slice
info: pytest.ExceptionInfo[PurgeBlockedError]
with pytest.raises(PurgeBlockedError) as info:
validate_deletion_allowed(
_session_matching_blockers(True), get_purge_policy(Slice), 1
)
assert info.value.reason_code == REASON_REPORT_SCHEDULE
assert str(info.value) == "associated alerts or reports exist"
def test_welcome_dashboard_block_raises_with_the_user_attribute_code() -> None:
"""A welcome-page block carries a code distinct from the report code."""
# avoid app-init regression: superset.models.* evaluates
# encrypted_field_factory at class-definition time, which fails
# in a partial-collection unit run with no Flask app active.
from superset.models.dashboard import Dashboard
info: pytest.ExceptionInfo[PurgeBlockedError]
with pytest.raises(PurgeBlockedError) as info:
validate_deletion_allowed(
_session_matching_blockers(False, True), get_purge_policy(Dashboard), 1
)
assert info.value.reason_code == REASON_USER_ATTRIBUTE
def test_reason_code_survives_a_related_table_rename() -> None:
"""A renamed table keeps the blocker's declared code.
The code is declared on the blocker, never derived from the physical
table name, so a schema rename changes only which table the blocker
looks at persisted audit history and the suppression predicate keep
comparing the same literal.
"""
# avoid app-init regression: superset.models.* evaluates
# encrypted_field_factory at class-definition time, which fails
# in a partial-collection unit run with no Flask app active.
from superset.models.slice import Slice
policy: PurgeEntityPolicy = get_purge_policy(Slice)
renamed: tuple[DependencyPolicy, ...] = tuple(
replace(dependency, key=replace(dependency.key, related_table="reports_v2"))
if dependency.classification is DependencyClassification.BLOCK
else dependency
for dependency in policy.dependencies
)
blocker: DependencyPolicy = next(
dependency
for dependency in renamed
if dependency.classification is DependencyClassification.BLOCK
)
assert blocker.key.related_table == "reports_v2"
assert blocker.blocker is not None
assert blocker.blocker.code == REASON_REPORT_SCHEDULE
def test_first_declared_blocker_wins_when_several_match() -> None:
"""A dashboard matching both blockers records the first-declared code."""
# avoid app-init regression: superset.models.* evaluates
# encrypted_field_factory at class-definition time, which fails
# in a partial-collection unit run with no Flask app active.
from superset.models.dashboard import Dashboard
info: pytest.ExceptionInfo[PurgeBlockedError]
with pytest.raises(PurgeBlockedError) as info:
validate_deletion_allowed(
_session_matching_blockers(True, True), get_purge_policy(Dashboard), 1
)
assert info.value.reason_code == REASON_REPORT_SCHEDULE
@@ -513,10 +513,14 @@ def test_alert_with_nonexistent_database_rejected(mocker: MockerFixture) -> None
# --- Retry config validation on update ---
_PATCH_RETRY_FLAG = "superset.commands.report.update.is_feature_enabled"
def test_update_rejects_retry_on_alert(mocker: MockerFixture) -> None:
"""Enabling retries on an alert schedule is rejected."""
model = _make_model(mocker, model_type=ReportScheduleType.ALERT, database_id=5)
_setup_mocks(mocker, model)
mocker.patch(_PATCH_RETRY_FLAG, return_value=True)
cmd = UpdateReportScheduleCommand(model_id=1, data={"retry_on_failure": True})
with pytest.raises(ReportScheduleInvalidError) as exc_info:
@@ -529,6 +533,7 @@ def test_update_rejects_send_failed_without_retry(mocker: MockerFixture) -> None
"""send_failed_reports=True requires retry_on_failure=True."""
model = _make_model(mocker, model_type=ReportScheduleType.REPORT, database_id=None)
_setup_mocks(mocker, model)
mocker.patch(_PATCH_RETRY_FLAG, return_value=True)
cmd = UpdateReportScheduleCommand(model_id=1, data={"send_failed_reports": True})
with pytest.raises(ReportScheduleInvalidError) as exc_info:
@@ -541,6 +546,7 @@ def test_update_accepts_retry_on_report(mocker: MockerFixture) -> None:
"""Enabling retries on a report schedule is accepted."""
model = _make_model(mocker, model_type=ReportScheduleType.REPORT, database_id=None)
_setup_mocks(mocker, model)
mocker.patch(_PATCH_RETRY_FLAG, return_value=True)
cmd = UpdateReportScheduleCommand(
model_id=1, data={"retry_on_failure": True, "retry_max_attempts": 5}
@@ -68,11 +68,10 @@ def test_dashboard_import_with_overwrite_replaces_charts(
}
ImportDashboardsCommand._import(initial_configs, overwrite=True)
# Commit between imports, as production does: ``run()`` carries
# ``@transaction()``, so two imports are two transactions. Calling the
# private ``_import`` twice without committing puts both in one Continuum
# transaction, where adding and removing the same association collides on
# ``dashboard_slices_version``'s (dashboard_id, slice_id, transaction_id)
# key — an artifact of the test's shortcut, not a reachable state.
# ``@transaction()``, so two imports are two transactions. Without this the
# add and the remove of one association share a Continuum transaction and
# collide on ``dashboard_slices_version``'s composite key — an artifact of
# the test's shortcut, not a reachable production state.
db.session.commit()
# Verify initial state: 2 charts associated with the dashboard

Some files were not shown because too many files have changed in this diff Show More