Compare commits

...
Author SHA1 Message Date
sadpandajoeandJoe Li c7360d1455 fix(mysql): restore pymysql mypy ignore and satisfy ruff-format
- Re-add `# type: ignore[import-untyped]` on the pymysql fallback
  import: mypy's pre-commit hook doesn't have types-PyMySQL installed,
  so a prior commit removing this as "obsolete" reintroduced the
  Library-stubs-not-installed error it was suppressing.
- Reformat the DATE_FORMAT truncation assertion in test_mysql.py to
  match ruff-format 0.9.7 output.
2026-08-12 04:59:21 +00:00
sadpandajoeandJoe Li ba4c2423d4 fix(mysql): remove obsolete PyMySQL type ignore 2026-08-11 19:41:40 +00:00
Joe LiandGitHub 33c24ba990 Merge branch 'master' into fix-mysql-wire-protocol-column-types-42203 2026-08-11 12:37:22 -07:00
sadpandajoeandJoe Li be478bed03 fix(mysql): satisfy ruff in MySQL type fallback tests 2026-08-11 19:09:40 +00:00
Joe LiandGitHub a3d58d1211 fix(ci): use current actions for Docker refresh (#42482) 2026-08-11 12:04:09 -07:00
7b32b870e1 chore: Fix/remove hardcode of admin role (#27779)
Co-authored-by: Evan <evan@preset.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Joe Li <joe@preset.io>
Co-authored-by: rusackas <evan@rusackas.com>
2026-08-11 10:26:50 -07:00
3cd4150539 chore(deps): bump sqlglot from 30.15.0 to 30.16.0 (#43029)
Co-authored-by: Superset Dev <dev@superset.apache.org>
2026-08-11 10:16:44 -07:00
2feb6c2bb5 refactor(deletion-retention): add declarative purge policies (#42888)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 09:55:36 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5767c3981d chore(deps): bump the storybook group across 1 directory with 2 updates (#43037)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 09:48:45 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ec6b2d2347 chore(deps-dev): bump oxfmt from 0.61.0 to 0.62.0 in /superset-websocket (#43033)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 09:48:42 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
aade32cad8 chore(deps-dev): bump the typescript-eslint group across 1 directory with 3 updates (#43039)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 09:48:38 -07:00
d6771e96d2 fix(core): let UnsavedChangesModal use Ant Design's automatic z-index stacking (#42548)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-11 09:42:49 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
305638e98c chore(deps-dev): bump baseline-browser-mapping from 2.11.11 to 2.11.12 in /superset-frontend (#43048)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 09:42:16 -07:00
5099b6ffa9 fix(export): derive chart_hashes and zip from a single chart list (#42946)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-11 09:35:59 -07:00
bc436bacad chore(security): warn when legacy encryption engine is configured (#42937)
Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
2026-08-11 09:35:46 -07:00
e31a894be6 fix(common): remove unsupported cross join mode from left_join_df (#43016)
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-11 09:35:16 -07:00
553808daa3 fix(reports): make webhook notification request timeout configurable (#43018)
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-11 09:35:01 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
544756a631 chore(deps): bump github/codeql-action/analyze from 4.37.5 to 4.37.6 (#43040)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 09:34:49 -07:00
c4e8a9da90 test(explore): add end-to-end regression test for DatasourcePanel through the real AutoSizer + react-window List pipeline (#43023)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-08-11 09:34:06 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
47adbe31d5 chore(deps-dev): bump typescript-eslint from 8.65.0 to 8.66.0 in /superset-websocket (#43034)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:43:16 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
64165fbdae chore(deps-dev): bump @typescript-eslint/parser from 8.65.0 to 8.66.0 in /superset-websocket (#43035)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:43:12 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
581fb876e4 chore(deps-dev): bump oxfmt from 0.61.0 to 0.62.0 in /docs (#43042)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:43:09 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
29e8e396ee chore(deps-dev): bump the storybook group in /superset-frontend with 5 updates (#43045)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:43:05 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
8bbf0cfc20 chore(deps-dev): bump oxlint from 1.76.0 to 1.77.0 in /superset-frontend (#43049)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:43:01 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
497c73e837 chore(deps): bump github/codeql-action/init from 4.37.5 to 4.37.6 (#43044)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:53 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a5a6f5a5b1 chore(deps): bump ws from 8.21.1 to 8.21.2 in /superset-websocket (#43036)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:50 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
156a59a190 chore(deps): bump swagger-ui-react from 5.32.11 to 5.32.12 in /docs (#43041)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:46 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
590003cbee chore(deps): bump baseline-browser-mapping from 2.11.11 to 2.11.12 in /docs (#43043)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:43 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
579011dea9 chore(deps): bump nanoid from 6.0.0 to 6.0.1 in /superset-frontend (#43050)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:39 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
65d808a5fe chore(deps): bump markdown-to-jsx from 9.10.1 to 9.10.2 in /superset-frontend (#43052)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-11 06:42:34 -07:00
5ee279363c fix(mcp): reject list values for ne on relationship filters (#42944)
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-11 00:42:13 -07:00
sadpandajoeandJoe Li 688e6abf07 fix(mysql): address wire-protocol type review feedback 2026-08-11 04:47:47 +00:00
34b2d3a533 fix(rls): match null-schema datasets in predicate lookup (#41426)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-10 19:58:51 -07:00
7157b752ee fix(pie): apply percentage number format to labels (#42880)
Co-authored-by: Joe Li <joe@preset.io>
2026-08-10 19:36:03 -07:00
d84a00f3c6 feat(dashboard): handle empty chart query context in Excel export (#42284)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2026-08-10 21:12:08 -04:00
bfa1e775c4 fix(theming): guard ensureStaticPrefix against null/undefined input (#43019)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 17:41:33 -07:00
265269018e fix(mcp): require issuer-aware resolver for multi-issuer JWT, fix sanitizer input ordering (#42935)
Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:56:46 -07:00
5f204a9af2 fix(frontend): remove any type and duplicate route in routes.tsx (#43014)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:51:58 -07:00
0c09cf087d fix(reports): translate execute-now success message (#43012)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:51:05 -07:00
651f75b1e8 docs(dashboard): document native filters on semantic views (#43011)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:50:34 -07:00
e48512568d docs(dashboard): document the expand_all_slices metadata flag (#43010)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:50:03 -07:00
4d0eeb7d86 docs: document FAB_API_SWAGGER_UI_SUPERSET_APP_ROOT config flag (#43009)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 15:49:47 -07:00
Joe LiandGitHub 9487291990 Merge branch 'master' into fix-mysql-wire-protocol-column-types-42203 2026-08-07 09:00:29 -07:00
1c1c613302 fix(mysql): address PR review feedback on wire-protocol type mapping
- Match plain DDL type TEXT alongside BLOB in the wire-protocol regex
  so physical dataset columns resolve correctly too
- Only import MySQLdb/pymysql in get_datatype when type_code is an
  int, avoiding an unnecessary import and keeping tests independent
  of those optional DBAPI modules
- Only copy fetch_data results to a list when the driver already
  returns an immutable sequence, avoiding an unneeded O(n) copy

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-06 17:55:03 +00:00
sadpandajoeandJoe Li a985cf8d06 Merge remote-tracking branch 'origin/master' into fix/mysql-wire-protocol-column-types
# Conflicts:
#	superset/db_engine_specs/mysql.py
2026-08-06 01:02:22 +00:00
Jean Massucatto cb76e5aa69 fix(mysql): resolve wire-protocol column types and mutate rows from immutable results 2026-07-18 12:52:35 -03:00
74 changed files with 6820 additions and 1327 deletions
+2 -2
View File
@@ -64,7 +64,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@d1ba80a13dd99fba24a470575428917156a28b43 # v4.37.5
uses: github/codeql-action/init@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -75,6 +75,6 @@ jobs:
# queries: security-extended,security-and-quality
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@d1ba80a13dd99fba24a470575428917156a28b43 # v4.37.5
uses: github/codeql-action/analyze@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
category: "/language:${{matrix.language}}"
@@ -108,8 +108,18 @@ jobs:
fetch-depth: 0
persist-credentials: false
# Keep workflow tooling on the triggering revision. Release tags can
# contain action pins that no longer satisfy the repository allowlist.
- name: Checkout workflow actions
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.sha }}
path: workflow-source
persist-credentials: false
sparse-checkout: .github/actions
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker
uses: ./workflow-source/.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -122,7 +132,7 @@ jobs:
node-version: 20
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
uses: ./workflow-source/.github/actions/setup-supersetbot/
- name: Rebuild and push
env:
@@ -171,7 +181,7 @@ jobs:
--repo "$REPOSITORY" \
--title "Scheduled Docker image refresh failed for ${LATEST_RELEASE}" \
--label "infra:container" \
--label "bug" \
--label "#bug" \
--body "The weekly Docker base-image refresh failed for release \`${LATEST_RELEASE}\`. Published images may be missing upstream base-layer security patches until this is resolved.
Failed run: ${RUN_URL}"
+24 -2
View File
@@ -24,6 +24,7 @@ assists people when migrating to a new version.
## Next
- [42935](https://github.com/apache/superset/pull/42935): The MCP service now refuses to start (`MCPAuthConfigError`) when `MCP_JWT_ISSUER` trusts more than one issuer and no `MCP_USER_RESOLVER` is configured, instead of only logging a warning. This was already a documented misconfiguration (the default resolver isn't issuer-scoped, so distinct trusted issuers minting the same username/email would resolve to the same Superset user); deployments trusting multiple issuers must configure an `MCP_USER_RESOLVER` that derives its identity from the token's `iss` claim before upgrading. Single-issuer deployments are unaffected.
- [42393](https://github.com/apache/superset/pull/42393): Exported dataset YAML now carries a `uuid` for each metric and column so that custom folder assignments (which reference metrics/columns by UUID) survive an import into another workspace. This affects any export bundle that contains datasets, not just a dataset export: chart, dashboard, database and full-asset exports all embed the same dataset YAML, so a dashboard exported from this release also fails to import into an older one even though no dataset was exported directly. As with `folders` and `currency_code_column`, the affected `datasets/` files fail schema validation (`Unknown field: uuid`) when imported into Superset releases that predate this change; regenerate or hand-edit exports for older targets in mixed-version fleets.
### Soft delete is on by default, and purging is live
@@ -122,6 +123,15 @@ their `extra` is still reduced to the fields the client needs. Responses to
every non-guest principal are unchanged, and the full error is still logged
server-side.
### `UnsavedChangesModal` no longer accepts a `zIndex` prop
`@superset-ui/core`'s `UnsavedChangesModal` dropped its `zIndex` prop (and the
hardcoded default it fed) in favor of letting Ant Design's own stacking
handle placement. Callers passing `zIndex` to override the modal's layering
will now get a TypeScript error and must remove the prop; keeping a manual
override was exactly the footgun this change removes (see #42510). No
callers in the Superset frontend codebase itself passed this prop.
### Principal listing APIs now honour related-field filters
Two authorization-related listing behaviors changed for API clients. Neither
@@ -170,12 +180,24 @@ A new dashboard action exports every chart's data to a single multi-sheet
requires a running Celery worker and a configured SMTP transport, since the task
emails the requesting user a pre-signed download link. New config keys:
`EXCEL_EXPORT_S3_BUCKET`, `EXCEL_EXPORT_S3_KEY_PREFIX`,
`EXCEL_EXPORT_LINK_TTL_SECONDS`, `EXCEL_EXPORT_S3_CLIENT_KWARGS`, and
`EXCEL_EXPORT_TABLE_VIZ_TYPES`.
`EXCEL_EXPORT_LINK_TTL_SECONDS`, `EXCEL_EXPORT_S3_CLIENT_KWARGS`,
`EXCEL_EXPORT_TABLE_VIZ_TYPES`, and `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`.
The feature depends on `boto3`, which is **not** installed by default; install it
with `pip install apache-superset[excel-export]`.
Charts store their `query_context` only once they have been (re-)saved in
Explore, so older charts may have none. For a fixed, conservative set of viz
types (`table`, `big_number_total`, `big_number`, `pie`) the export rebuilds a
query context from the chart's saved form data so those charts still export.
The rebuild is a single-query mapping and does **not** reproduce plugin
post-processing (pivot, rolling, forecast) or multi-query charts, so any chart of
another type without a saved query context is skipped and listed in the email for
the user to re-save. To cover those types, set `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`
to a callable that receives the chart's form data and returns a query-context
payload (or `None` to fall back to the built-in rebuild) — for example one backed
by a service that runs the chart's real frontend `buildQuery`.
A second mode, **Export Images to Excel**, embeds non-table charts as rendered
images (which viz types stay tabular is controlled by
`EXCEL_EXPORT_TABLE_VIZ_TYPES`). It renders through the headless webdriver, so the
@@ -175,6 +175,16 @@ ALERT_REPORTS_WEBHOOK_HTTPS_ONLY = True
When enabled, Superset rejects webhook configurations that use `http://` URLs.
#### Request Timeout
Webhook deliveries use a socket timeout so a request can't hang forever if the webhook target is unreachable, which would otherwise leave the report schedule stuck in a `WORKING` state. Configure it with:
```python
ALERT_REPORTS_WEBHOOK_TIMEOUT = 60 # seconds
```
Set to `None` to disable the timeout (not recommended).
#### Retry Behavior
Superset automatically retries webhook deliveries on `429 Too Many Requests` and `5xx` server errors using exponential backoff. Retries are bounded to roughly 120 seconds of cumulative wall-clock time (worst case ~210 seconds, because the bound is checked against the time elapsed before each attempt, so the final request can begin just under the limit and still run its full request timeout), after which the delivery is abandoned.
@@ -307,6 +307,22 @@ to simplify the process of setting up a non-default root path across the service
In `docker/.env-local` set `SUPERSET_APP_ROOT` to the desired prefix and then bring the
services up with `docker compose up --detach`.
### Swagger UI
By default, Superset's Swagger UI and OpenAPI spec (enabled via `FAB_API_SWAGGER_UI`) are
served by Flask-AppBuilder and don't account for a non-root `APPLICATION_ROOT` prefix. If
you're running Superset behind a URL prefix and want the Swagger UI and OpenAPI spec to
resolve correctly, set:
```python
FAB_API_SWAGGER_UI_SUPERSET_APP_ROOT = True
```
in your `superset_config.py` file. This serves an `APPLICATION_ROOT`-aware Swagger UI and
OpenAPI spec at `/swagger/<version>` and `/api/<version>/_openapi` respectively, resolved
through the configured prefix. This flag only takes effect when `FAB_API_SWAGGER_UI` is
also enabled, and defaults to `False`.
## Custom OAuth2 Configuration
Superset is built on Flask-AppBuilder (FAB), which supports many providers out of the box
+16
View File
@@ -277,6 +277,22 @@ second etc). Example:
}
```
## How do I expand all chart descriptions on a dashboard by default?
Charts can have a markdown description, set in the chart's **Edit chart properties** dialog, that's
hidden by default and toggled on a per-chart basis from the chart's context menu on a dashboard. If
you'd rather have every chart's description expanded by default when the dashboard loads, add the
`expand_all_slices` key to the dashboard JSON Metadata field:
```json
{
"expand_all_slices": true
}
```
Charts that have already been manually expanded or collapsed on the dashboard keep that per-chart
override (tracked in the `expanded_slices` key) regardless of the `expand_all_slices` setting.
## Does Superset work with [insert database engine here]?
The [Connecting to Databases section](/user-docs/databases/) provides the best
@@ -165,6 +165,31 @@ You can also certify metrics if you'd like for your team in this view.
- [Blog: Unlocking the Power of Virtual Datasets](https://preset.io/blog/unlocking-the-power-of-virtual-datasets-in-apache-superset/)
:::
### Native filters on semantic views
When the `SEMANTIC_LAYERS` feature flag is enabled, Superset can connect to external semantic layers
(such as dbt Semantic Layer or Cube) and expose their semantic views as data sources alongside your
regular Datasets. Semantic views can be used as filter targets when adding a native (dashboard) filter,
the same way a Dataset can.
To add a filter on a semantic view:
1. Open the dashboard, click the **⋮** (more options) menu, and select **Edit dashboard**.
2. Open the Filter Bar and click **+ Add/Edit Filters**.
3. Add a new filter and, in the datasource dropdown, select a semantic view. Semantic views are listed
alongside datasets and can be identified by their type.
4. Select one of the semantic view's dimensions in the **Column** field, the same way you'd select a
column on a dataset.
5. Configure the remaining filter options (filter type, default value, scope, etc.) and click **Save**.
Any chart on the dashboard that's powered by the same semantic view is filtered by the selected
dimension when the filter is applied.
:::note
Semantic views and native filter support for them are part of the experimental Semantic Layers
feature and require the `SEMANTIC_LAYERS` feature flag to be enabled.
:::
### Creating charts in Explore view
Superset has 2 main interfaces for exploring data:
@@ -32,8 +32,13 @@ Notes on the generated workbook:
Excel's 31-character limit; the chart id keeps names unique).
- Charts nested in tabs are included.
- Data reflects the dashboard's active filter state at the time of export.
- A chart with no saved query context is skipped and listed in the email; open
the chart in Explore and re-save it to include it next time.
- A chart with no saved query context (charts only store one once they've been
re-saved in Explore) still exports when it is a `table`, `big_number`,
`big_number_total` or `pie`, by rebuilding the query from the chart's saved
form data. Charts of other types — and charts relying on post-processing the
rebuild can't reproduce — are skipped and listed in the email; open the chart
in Explore and re-save it to include it next time, or configure
`EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`.
- Row counts per sheet are capped the same way as the chart-level CSV/Excel
export (`ROW_LIMIT`, bounded by `SQL_MAX_ROW`), and never exceed Excel's
per-sheet maximum.
@@ -74,6 +79,7 @@ will not register.
| `EXCEL_EXPORT_LINK_TTL_SECONDS` | `86400` | Lifetime of the pre-signed download URL (24h). |
| `EXCEL_EXPORT_S3_CLIENT_KWARGS` | `{}` | Extra kwargs for `boto3.client("s3", ...)` — e.g. `region_name`, or `endpoint_url` for MinIO/LocalStack. |
| `EXCEL_EXPORT_TABLE_VIZ_TYPES` | `None` | Viz types kept tabular in **Export Images to Excel** mode; every other type is embedded as an image. `None` uses the built-in default (`table`, `pivot_table`, `pivot_table_v2`). |
| `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER` | `None` | Optional `Callable[[form_data_dict], dict \| None]` to build a query context for a chart missing a saved one, tried before the built-in form-data rebuild. Point it at a service that runs the chart's real frontend `buildQuery` to faithfully export viz types the built-in rebuild can't handle. Must return `None` when it can't build faithfully, so the export falls back. |
Credentials and region resolve through the standard boto3 chain (environment
variables, shared config, or instance role) unless overridden via
+8 -8
View File
@@ -58,11 +58,11 @@
"@fontsource/inter": "^5.3.0",
"@mdx-js/react": "^3.1.1",
"@saucelabs/theme-github-codeblock": "^0.3.0",
"@storybook/addon-docs": "^10.5.5",
"@storybook/addon-docs": "^10.5.6",
"@superset-ui/core": "^0.20.4",
"@swc/core": "^1.15.47",
"antd": "^6.5.3",
"baseline-browser-mapping": "^2.11.11",
"baseline-browser-mapping": "^2.11.12",
"caniuse-lite": "^1.0.30001806",
"docusaurus-plugin-openapi-docs": "^5.1.3",
"docusaurus-theme-openapi-docs": "^5.1.3",
@@ -77,8 +77,8 @@
"react-table": "^7.8.0",
"remark-import-partial": "^0.0.2",
"reselect": "^5.2.0",
"storybook": "^10.5.5",
"swagger-ui-react": "^5.32.11",
"storybook": "^10.5.6",
"swagger-ui-react": "^5.32.12",
"swc-loader": "^0.2.7",
"tinycolor2": "^1.4.2",
"unist-util-visit": "^5.1.0"
@@ -89,14 +89,14 @@
"@eslint/js": "^9.39.2",
"@types/js-yaml": "^4.0.9",
"@types/react": "^19.1.8",
"@typescript-eslint/eslint-plugin": "^8.65.0",
"@typescript-eslint/parser": "^8.65.0",
"@typescript-eslint/eslint-plugin": "^8.66.0",
"@typescript-eslint/parser": "^8.66.0",
"eslint": "^9.39.2",
"eslint-plugin-react": "^7.37.5",
"globals": "^17.9.0",
"oxfmt": "^0.61.0",
"oxfmt": "^0.62.0",
"typescript": "~6.0.3",
"typescript-eslint": "^8.65.0",
"typescript-eslint": "^8.66.0",
"webpack": "^5.109.2"
},
"browserslist": {
+574 -509
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -118,7 +118,7 @@ dependencies = [
"sqlalchemy>=1.4.43, <2", # 1.4.43 adds the python-oracledb (oracle+oracledb) dialect
"sqlalchemy-continuum>=1.6.0, <2.0.0",
"sqlalchemy-utils>=0.42.1, <0.43", # expanding lowerbound to work with pydoris
"sqlglot>=30.14.0, <31",
"sqlglot>=30.16.0, <31", # 30.16.0 adds Trino inline UDF IF/CASE routine statement parsing
# newer pandas needs 0.9+
"tabulate>=0.10.0, <1.0",
"typing-extensions>=4.16.0, <5",
+1 -1
View File
@@ -418,7 +418,7 @@ sqlalchemy-utils==0.42.1
# apache-superset (pyproject.toml)
# apache-superset-core
# flask-appbuilder
sqlglot==30.15.0
sqlglot==30.16.0
# via
# apache-superset (pyproject.toml)
# apache-superset-core
+1 -1
View File
@@ -1002,7 +1002,7 @@ sqlalchemy-utils==0.42.1
# apache-superset
# apache-superset-core
# flask-appbuilder
sqlglot==30.15.0
sqlglot==30.16.0
# via
# -c requirements/base-constraint.txt
# apache-superset
+1
View File
@@ -42,6 +42,7 @@ RETRYABLE_STATUS_CODES: frozenset[int] = frozenset({429})
PATTERNS = {
"python": [
r"^\.github/workflows/.*python",
r"^\.github/workflows/scheduled-docker-image-refresh\.yml$",
r"^docker-compose-image-tag\.yml$",
r"^tests/",
r"^superset/",
+181 -166
View File
@@ -109,12 +109,12 @@
"lodash": "^4.18.1",
"lodash-es": "^4.18.1",
"mapbox-gl": "^3.27.0",
"markdown-to-jsx": "^9.10.1",
"markdown-to-jsx": "^9.10.2",
"match-sorter": "^8.3.0",
"memoize-one": "^6.0.0",
"mousetrap": "^1.6.5",
"mustache": "^4.2.0",
"nanoid": "^6.0.0",
"nanoid": "^6.0.1",
"ol": "^10.10.0",
"query-string": "9.4.1",
"re-resizable": "^6.11.2",
@@ -180,9 +180,9 @@
"@istanbuljs/nyc-config-typescript": "^1.0.1",
"@playwright/test": "^1.62.1",
"@pmmmwh/react-refresh-webpack-plugin": "^0.6.2",
"@storybook/addon-docs": "10.5.5",
"@storybook/addon-links": "10.5.5",
"@storybook/react-webpack5": "10.5.5",
"@storybook/addon-docs": "10.5.6",
"@storybook/addon-links": "10.5.6",
"@storybook/react-webpack5": "10.5.6",
"@storybook/test-runner": "0.24.4",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.47",
@@ -218,7 +218,7 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.11",
"baseline-browser-mapping": "^2.11.12",
"cheerio": "1.2.0",
"concurrently": "^10.0.4",
"copy-webpack-plugin": "^14.0.0",
@@ -235,7 +235,7 @@
"eslint-plugin-no-only-tests": "^3.4.0",
"eslint-plugin-react-prefer-function-component": "^5.0.0",
"eslint-plugin-react-you-might-not-need-an-effect": "^1.0.1",
"eslint-plugin-storybook": "10.5.5",
"eslint-plugin-storybook": "10.5.6",
"eslint-plugin-testing-library": "^7.16.2",
"eslint-plugin-theme-colors": "file:eslint-rules/eslint-plugin-theme-colors",
"fetch-mock": "^12.6.0",
@@ -255,7 +255,7 @@
"minimizer-webpack-plugin": "^5.6.1",
"open-cli": "^9.0.0",
"oxfmt": "^0.61.0",
"oxlint": "^1.76.0",
"oxlint": "^1.77.0",
"po2json": "^0.4.5",
"postcss-styled-syntax": "^0.7.2",
"process": "^0.11.10",
@@ -266,7 +266,7 @@
"source-map": "^0.8.0",
"source-map-support": "^0.5.21",
"speed-measure-webpack-plugin": "^1.6.0",
"storybook": "10.5.5",
"storybook": "10.5.6",
"style-loader": "^4.0.0",
"stylelint": "^17.14.1",
"swc-loader": "^0.2.7",
@@ -8764,9 +8764,9 @@
}
},
"node_modules/@oxlint/binding-android-arm-eabi": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm-eabi/-/binding-android-arm-eabi-1.76.0.tgz",
"integrity": "sha512-ZHIE5Zt9AsPDcY4nOlofXt0YfneEeo+QrKMPcPzLf2Z6Q8VtV2W73d7SFJ920WUwyik783u/doKCs3KXdwG+7w==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm-eabi/-/binding-android-arm-eabi-1.77.0.tgz",
"integrity": "sha512-E06sKWS6PiI6HRxS1wyQg22HvApt01hI7fV+T3wUk3OSbaaP4a3hYGY/MIQDmASqCiRjBdpRQYkgMkqH82cWmQ==",
"cpu": [
"arm"
],
@@ -8781,9 +8781,9 @@
}
},
"node_modules/@oxlint/binding-android-arm64": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm64/-/binding-android-arm64-1.76.0.tgz",
"integrity": "sha512-shm/ngQilHK6bs+ElJWa4oHfNj5vL1Gl/iVEJldTQjpr0/67oSgr0KUpbmcnLig5Fo0v/l6j2567A7TOL89ONA==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm64/-/binding-android-arm64-1.77.0.tgz",
"integrity": "sha512-NvsKz0KZxTp9cYWPLf+FXaSZwB3oO3peAjtukpOMBgse2vhQSoIIVqeO1yR0lEo/UcdZIDL18uq+kL0LzQ0ytA==",
"cpu": [
"arm64"
],
@@ -8798,9 +8798,9 @@
}
},
"node_modules/@oxlint/binding-darwin-arm64": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-arm64/-/binding-darwin-arm64-1.76.0.tgz",
"integrity": "sha512-rvJmrAPKSQ9aWJ6wIS6CK2tJjwzfW0ApQH9qokq6sfDvmHwoyIHxHFMq7z7i7GiV6fdE6s8qvBqWKPTu8RmT6Q==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-arm64/-/binding-darwin-arm64-1.77.0.tgz",
"integrity": "sha512-bgjTn6nW4bQCFBvSvuHCpDD+sONvmpo4lGI4PxzMt1quBA+xYxhczk6RiCn3GZ9gY8uhaBbwhj9MdKGfu6T9DA==",
"cpu": [
"arm64"
],
@@ -8815,9 +8815,9 @@
}
},
"node_modules/@oxlint/binding-darwin-x64": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-x64/-/binding-darwin-x64-1.76.0.tgz",
"integrity": "sha512-U/zYdb7VYKGY6pA9Vd2rYl9O/HlCylcOlb5PGPvVLtg+oLGsk6H3XGKEMHKyqD3nmmtmlmwb/8SwU2vfSAtvMw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-x64/-/binding-darwin-x64-1.77.0.tgz",
"integrity": "sha512-aotaIttH1R6j1Rwhx0M0htgeZyGtVQqYNTVEYMN/UcgHPquGA6kmk9OyuDc3a2GKUQBC+3C3GVQCcrRPMYqAFA==",
"cpu": [
"x64"
],
@@ -8832,9 +8832,9 @@
}
},
"node_modules/@oxlint/binding-freebsd-x64": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-freebsd-x64/-/binding-freebsd-x64-1.76.0.tgz",
"integrity": "sha512-WvKG9CAriuo0XNiFzpXjDngUZcRGFNpaK2kLyMUsnJlShxkT96u+BpJQ3KqdQwGOrvI14L6V8bAwXwAYNNY6Jg==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-freebsd-x64/-/binding-freebsd-x64-1.77.0.tgz",
"integrity": "sha512-nNx/wta7ksRAdYvq+l4AWjXkLxEXHALhENxjj2cYbQAIR4ybaA5L+hCbE63HOmft5czQ6ks+hb8vmEAnn7YGPg==",
"cpu": [
"x64"
],
@@ -8849,9 +8849,9 @@
}
},
"node_modules/@oxlint/binding-linux-arm-gnueabihf": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.76.0.tgz",
"integrity": "sha512-qJ5+RH99TqFRq3UCDxkW0zJJu9c+OAHFY72vGlxZLEpuO+MpKo3POgqb8sYipL9KYm8XY6ofb0HsOuvY6hQNqQ==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.77.0.tgz",
"integrity": "sha512-tMLLjM7xXtzXisVCzkOTXNCy9bZVId2wteNwjohlFDR/jY6WagpEDA1c1wu4xRc20Hojaxj+V6DSR7gbKxijWA==",
"cpu": [
"arm"
],
@@ -8866,9 +8866,9 @@
}
},
"node_modules/@oxlint/binding-linux-arm-musleabihf": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-1.76.0.tgz",
"integrity": "sha512-PvPCVptkgVARsucgIqFQQcSmJ6xc6GtnVB5bRBekRahTc9eObMtjHfMjy5M+C2tHt5UCMttWM9RuSk/H9NqYeg==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-1.77.0.tgz",
"integrity": "sha512-MiAFDFaqR0tmHTAyo0YDcZ5hyLREdYw/RQhc2R3cbT+8O3tB+zqPM2th9TTQ+Uo3jn/embS+DO+HyX9ztCPkOQ==",
"cpu": [
"arm"
],
@@ -8883,13 +8883,16 @@
}
},
"node_modules/@oxlint/binding-linux-arm64-gnu": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.76.0.tgz",
"integrity": "sha512-3KeFDx8Bu4HPAXbuHZOr/oHvN+QT+JQhMw/NYPz7Z071xLSsG27Jfh9PIQVEY7hk1I+jr43ExqRIeJ6VKk2yLw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.77.0.tgz",
"integrity": "sha512-/xqQ3B16i1T4cyt/9Mn+4CpzhUXoBXp7kVpIwzOXNFLj5JmK1bIjsbSnX296Gg8A/o7oDtKWikFgBx0SLwztkw==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8900,13 +8903,16 @@
}
},
"node_modules/@oxlint/binding-linux-arm64-musl": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.76.0.tgz",
"integrity": "sha512-oPFkkKTgl0K/EIg9fQ8oA3IGcI05/Mq1en04iFa41mmNPT+6KEiByVazTOZZJiHMBBrbsns1YJ2e1Scqwzesjw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.77.0.tgz",
"integrity": "sha512-LSbwuRKiNCenPDcbARqAZ5RfBy7gmj7vOvfJRLeCDU3gFtSxWbhv/+VTlaUqzUhNj1gFLHB8h7ALnxa/Az6z6g==",
"cpu": [
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8917,13 +8923,16 @@
}
},
"node_modules/@oxlint/binding-linux-ppc64-gnu": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.76.0.tgz",
"integrity": "sha512-gN7yZ0eqflA5Fhf1wvHxGUltIV3FsvmB1zhNMDEK9vSHhc7E6qg9CuPeBgPZab66Tjzq6w6kHAtNEvnTHf4cyw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.77.0.tgz",
"integrity": "sha512-QWdcH31mXEUe5Nq1s0CfCpceaKjIo9uZtwDjAuL681g1axf+5x8xrg/eXWaw//4NCxYZ4V4e5Hu5tvdR+pTBlg==",
"cpu": [
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8934,13 +8943,16 @@
}
},
"node_modules/@oxlint/binding-linux-riscv64-gnu": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-1.76.0.tgz",
"integrity": "sha512-S/HqMbn22mQrjtErUxEoS/a55u8kIeXvreIxiJu5G7Le3UecEd6SQZxrDIpuhtgaFnsY/nVra3ytP+pRljDilA==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-1.77.0.tgz",
"integrity": "sha512-GnOfYgJxbcElOiPZaDFDl406ONddwvOWk2jvAAAEjwAl4GofNoHF+/HHUIBYa6bFCArlcGPi0XjC4cU1pkgF/Q==",
"cpu": [
"riscv64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8951,13 +8963,16 @@
}
},
"node_modules/@oxlint/binding-linux-riscv64-musl": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-1.76.0.tgz",
"integrity": "sha512-ZIga3097VJZolGZk6SrIAUokIGfRkxRlhiHDUznZptGBfwrhD7pNfD1rzEzsCwvk/1DX0A1bLz+liuNh5QKIVQ==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-1.77.0.tgz",
"integrity": "sha512-AyEMTUCf0xY+hHF+IxqXFQIX0yQOIR8ykpY0lJNOw9xYqOzUX8dyZfRvlG0RfXwuQn2eonf/8NrMmDSZJjdqsA==",
"cpu": [
"riscv64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8968,13 +8983,16 @@
}
},
"node_modules/@oxlint/binding-linux-s390x-gnu": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.76.0.tgz",
"integrity": "sha512-ZGiiA7pFzMJSyMWYZTVlPgbTsx+Vl8ihLGMIujPwaslUF7kIPPWAbVmAlTc+9lWDV+DCiB8Ikixu+lSHeOIIWQ==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.77.0.tgz",
"integrity": "sha512-sPLzEcNvxd/oyVQ5oZo92CiHkFkpBeRop13E/P3TPY+hZfXHKCOWKI70TE2RYwMKFJDc20EMjH16L7NZICtKTw==",
"cpu": [
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -8985,13 +9003,16 @@
}
},
"node_modules/@oxlint/binding-linux-x64-gnu": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.76.0.tgz",
"integrity": "sha512-JLiy5WuvEBFTT6ErIFV35SLzi0R7Iri6MKU6dZbTxfIx8pndbbPs3Mj780nMipBFcPkti+okAPOJ9POKkHFEgg==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.77.0.tgz",
"integrity": "sha512-1Oh2ssH2L7lwyvkdSqaMUfsGfwU2Wfvew+obBUYjRVqhpBcUpwnsPSEr1IzVi9XqkuY10geiLsNKecqaZC34Dw==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -9002,13 +9023,16 @@
}
},
"node_modules/@oxlint/binding-linux-x64-musl": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-musl/-/binding-linux-x64-musl-1.76.0.tgz",
"integrity": "sha512-z7lgKQtbo/I1NIe8G5NHLesxJDv0tRSUWTpXKb9Pm3E9nKFKfO4IOSDtFroKgXtOYb0jQbcdH+0wzTyMXVes+A==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-musl/-/binding-linux-x64-musl-1.77.0.tgz",
"integrity": "sha512-0j/2wRgNGO+Qj/M1uu/p57h/hFTTWWcfie0ufkbabeus2s5+/QqkCflnMOwLLN5m2GsNeWp4xdl4cPa4n7QCOQ==",
"cpu": [
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -9019,9 +9043,9 @@
}
},
"node_modules/@oxlint/binding-openharmony-arm64": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-openharmony-arm64/-/binding-openharmony-arm64-1.76.0.tgz",
"integrity": "sha512-JOjKymIpb9QcYfEhZsN6h4V9Ivd474W38cNIBRv6bg2TbIvogbMTH0Mg6YWW9TiRDqfcX+/Hyfsbo5vcSE5guQ==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-openharmony-arm64/-/binding-openharmony-arm64-1.77.0.tgz",
"integrity": "sha512-BJ/j54qS0usEnyDkLYURMj2iiD9h5Cyy+ppzeMSXBGRXaGRNWnj1Mw14NqWMR5E/PzdgB30OOCCzLzbRoduafw==",
"cpu": [
"arm64"
],
@@ -9036,9 +9060,9 @@
}
},
"node_modules/@oxlint/binding-win32-arm64-msvc": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.76.0.tgz",
"integrity": "sha512-pqDWZiwcmByWUEm1NFUBNiT6aentCcaoMWJv0HbXEmuYermJ4sg8ppVrshubYP2MZ6SHccJJcpr6x469PuDFIw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.77.0.tgz",
"integrity": "sha512-Yh8w+g2Lpx7StrvtYkoz9JJvXjB9wxgFChFNb85nrXm/wj/XTwGWS1hve9+900HL7llrntYB3YP+y32E3tRqzA==",
"cpu": [
"arm64"
],
@@ -9053,9 +9077,9 @@
}
},
"node_modules/@oxlint/binding-win32-ia32-msvc": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-1.76.0.tgz",
"integrity": "sha512-Ba0O659kgMv6pwO3z9PdO+K3aMxQRaw9HnG+e6AtOfgwcKFvYilciQYBoUBmxfQvOCKZe1SwjMkuB542NkuDMQ==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-1.77.0.tgz",
"integrity": "sha512-zja5b7+6a7UsRFgAQSrnax5vrzliEyNPLCjfXONu/vTWswaIVZGFajJZptaeRvPE4LghtFdAzVFlexTm7MVTGA==",
"cpu": [
"ia32"
],
@@ -9070,9 +9094,9 @@
}
},
"node_modules/@oxlint/binding-win32-x64-msvc": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.76.0.tgz",
"integrity": "sha512-5qcirPHO8nKfkoowEVWtpAoVTcYDy6g0UT0NGic450Qv8J2NrOqg4uQ8QppRP4MDTC7Xx47lbZnmadTH03CGGA==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/@oxlint/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.77.0.tgz",
"integrity": "sha512-+teyvPDZ2RjUvo+SuCqS/UhaJl1QtdW5fWT5NJTV61V5MIuIS90Db9LixmtEGvXixyttiK62P96MSu3UlpviBw==",
"cpu": [
"x64"
],
@@ -10693,16 +10717,16 @@
"license": "MIT"
},
"node_modules/@storybook/addon-docs": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/addon-docs/-/addon-docs-10.5.5.tgz",
"integrity": "sha512-0YpKlimS4XE0kQ8Maa5coeefQxdyDrBHg1wOP3WTPuBe4FolFSCDveR0ge2+vuUBk+fZfn2+l+3Q2jmAWaRGDg==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/addon-docs/-/addon-docs-10.5.6.tgz",
"integrity": "sha512-zyUJBrrpC9NTrmsREaVFNr+9WW6pikJtmRvo7GgZGqthEyhjQKSarHrW0aNWkwae2ep3jp1CZi8vIUVG1Dnp0w==",
"dev": true,
"license": "MIT",
"dependencies": {
"@mdx-js/react": "^3.0.0",
"@storybook/csf-plugin": "10.5.5",
"@storybook/csf-plugin": "10.5.6",
"@storybook/icons": "^2.0.2",
"@storybook/react-dom-shim": "10.5.5",
"@storybook/react-dom-shim": "10.5.6",
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"ts-dedent": "^2.0.0"
@@ -10713,7 +10737,7 @@
},
"peerDependencies": {
"@types/react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"@types/react": {
@@ -10722,9 +10746,9 @@
}
},
"node_modules/@storybook/addon-docs/node_modules/@storybook/csf-plugin": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/csf-plugin/-/csf-plugin-10.5.5.tgz",
"integrity": "sha512-/euibhRFqklYCZqUseokojmfYcQpXshVY2QmA1qCuxMz9SzVFD3iSTw+aFLTxpsJGGdcZJk8fnm/rEthLzZ9jA==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/csf-plugin/-/csf-plugin-10.5.6.tgz",
"integrity": "sha512-PJLyOmcKe1OZDBw7RaGX/gjuiJuVfS5pVgc4W2RnHYOFpU6F5Bv9+9MqQwp0i7tWZBWc4fsCJudgVqwgjuTROA==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -10737,7 +10761,7 @@
"peerDependencies": {
"esbuild": "*",
"rollup": "*",
"storybook": "10.5.5",
"storybook": "10.5.6",
"vite": "*",
"webpack": "*"
},
@@ -10757,9 +10781,9 @@
}
},
"node_modules/@storybook/addon-docs/node_modules/@storybook/react-dom-shim": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/react-dom-shim/-/react-dom-shim-10.5.5.tgz",
"integrity": "sha512-PIk7N3LLrZIxfNxmkvmQN1d5UQ70XEedT8n0GhBiXnM6XL09xPGB8n8TZXeJBRYluKhDQcAyQeT0/OZmcDVQJg==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/react-dom-shim/-/react-dom-shim-10.5.6.tgz",
"integrity": "sha512-dV3oOHc5ImggxEqeIiUj4vvnQO5SKScFtqAkxgIWLju1wiSZSIqQ5Q4Mp12Rhs9hQrjF039DueH7f2xJJZfvSw==",
"dev": true,
"license": "MIT",
"funding": {
@@ -10771,7 +10795,7 @@
"@types/react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"@types/react": {
@@ -10783,9 +10807,9 @@
}
},
"node_modules/@storybook/addon-links": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/addon-links/-/addon-links-10.5.5.tgz",
"integrity": "sha512-rqb8EYosKG7eZ6XmFiQ5HwpZIke11QYS0OhuK+1oBLHVfMrlbEgDDLo/wtbiUKKAzj8D1hlwgC6mzSyyW4oQTw==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/addon-links/-/addon-links-10.5.6.tgz",
"integrity": "sha512-pw+OS/wUZ4ijdVGOsE5QOt59+C2i4fwtFBs2ircB7KMlwEE7gslovZEjnrz8bbaznvmoLoYGWYZxXcLi+bYmzg==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -10798,7 +10822,7 @@
"peerDependencies": {
"@types/react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"@types/react": {
@@ -10892,15 +10916,15 @@
}
},
"node_modules/@storybook/react-webpack5": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/react-webpack5/-/react-webpack5-10.5.5.tgz",
"integrity": "sha512-Se16ayzr9Gz9+uRirf3HPHioDwJthoi3N4WII+BR8bFIpuvnjxyTn3vl5EwdZiXW+bVXyIBr4mkLXt/7u8KUPg==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/react-webpack5/-/react-webpack5-10.5.6.tgz",
"integrity": "sha512-UdsC+IrZHBAtEvvDkfCPhg5sy5jnJAHT4RS3I8wNHVJ+93gaUrZLElSIV+w5UB1u/yghoqmQm7/LfpPpZrjPZQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@storybook/builder-webpack5": "10.5.5",
"@storybook/preset-react-webpack": "10.5.5",
"@storybook/react": "10.5.5"
"@storybook/builder-webpack5": "10.5.6",
"@storybook/preset-react-webpack": "10.5.6",
"@storybook/react": "10.5.6"
},
"funding": {
"type": "opencollective",
@@ -10909,7 +10933,7 @@
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5",
"storybook": "10.5.6",
"typescript": ">= 4.9.x"
},
"peerDependenciesMeta": {
@@ -10919,13 +10943,13 @@
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/builder-webpack5": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/builder-webpack5/-/builder-webpack5-10.5.5.tgz",
"integrity": "sha512-hLjPH/A3NP5mqXsDbIdHb6+wvbPyvwjFvc+YxbEFprZPbjREIa0WP3933jf4Yg5xAjxpHdAhe7SES3k0fpR1/w==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/builder-webpack5/-/builder-webpack5-10.5.6.tgz",
"integrity": "sha512-uWo/MzNC6HXMEpy8QQfbeYh1j6aOC6Ly0sAR6RE0LPvpyGEWC0VaVOBERIJJBjeuPuonDBVKfEjh1iUvZhJopg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@storybook/core-webpack": "10.5.5",
"@storybook/core-webpack": "10.5.6",
"case-sensitive-paths-webpack-plugin": "^2.4.0",
"cjs-module-lexer": "^1.2.3",
"css-loader": "^7.1.2",
@@ -10947,7 +10971,7 @@
"url": "https://opencollective.com/storybook"
},
"peerDependencies": {
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"typescript": {
@@ -10956,9 +10980,9 @@
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/builder-webpack5/node_modules/@storybook/core-webpack": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/core-webpack/-/core-webpack-10.5.5.tgz",
"integrity": "sha512-VkKmJ+hjV/6ztdjm83aO4gHOHcpoT/PysnV1ZDe5xuC2fc3KM4A6ArWU48nrZ48X4eBdIAmhDJKcEABCq4+4Jw==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/core-webpack/-/core-webpack-10.5.6.tgz",
"integrity": "sha512-o5PP3K+NcJAitZF7Ywweow0d8dJrEA1jxV5T1LMGMiWHUrnpoaPTiK1HcYw39pOQkdKL88mMSfDDUipGurZthw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -10969,17 +10993,17 @@
"url": "https://opencollective.com/storybook"
},
"peerDependencies": {
"storybook": "10.5.5"
"storybook": "10.5.6"
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/preset-react-webpack": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/preset-react-webpack/-/preset-react-webpack-10.5.5.tgz",
"integrity": "sha512-mR63cbAdWfAqrFtoqo9xdwFr7U8T4pO1N5rd1jZfXDj/fk9rUxAPDL/uBvgMNfMsZHaCu9ZaYydhn9Vpmio7lw==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/preset-react-webpack/-/preset-react-webpack-10.5.6.tgz",
"integrity": "sha512-QPUl2t+0VIp1Wy7JfqvV8cI1NrULUt+XFMKdIaNp39TuyMn3El4txvmxQWKhcYvnSOEzQ2SGNDqSWcJISwmeAA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@storybook/core-webpack": "10.5.5",
"@storybook/core-webpack": "10.5.6",
"@storybook/react-docgen-typescript-plugin": "1.0.6--canary.9.0c3f3b7.0",
"@types/semver": "^7.7.1",
"magic-string": "^0.30.5",
@@ -10996,7 +11020,7 @@
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"typescript": {
@@ -11005,9 +11029,9 @@
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/preset-react-webpack/node_modules/@storybook/core-webpack": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/core-webpack/-/core-webpack-10.5.5.tgz",
"integrity": "sha512-VkKmJ+hjV/6ztdjm83aO4gHOHcpoT/PysnV1ZDe5xuC2fc3KM4A6ArWU48nrZ48X4eBdIAmhDJKcEABCq4+4Jw==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/core-webpack/-/core-webpack-10.5.6.tgz",
"integrity": "sha512-o5PP3K+NcJAitZF7Ywweow0d8dJrEA1jxV5T1LMGMiWHUrnpoaPTiK1HcYw39pOQkdKL88mMSfDDUipGurZthw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -11018,18 +11042,18 @@
"url": "https://opencollective.com/storybook"
},
"peerDependencies": {
"storybook": "10.5.5"
"storybook": "10.5.6"
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/react": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/react/-/react-10.5.5.tgz",
"integrity": "sha512-T2Xj0ey7a9RHU6coYLC0L5lhjcdyhLCs9wNv15FvHvgmrRobkynEV72kq5vGW8tFkahNWI1X9+GZPQ6r8Nm38w==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/react/-/react-10.5.6.tgz",
"integrity": "sha512-dXSdNoc9yAvpa4hiegQhmZPXOKunAxkPX94DxvRw/kM6+wujVFAGlZjYygKrWw357KOjPRK7SO1LRTc70mgrhQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@storybook/global": "^5.0.0",
"@storybook/react-dom-shim": "10.5.5",
"@storybook/react-dom-shim": "10.5.6",
"react-docgen": "^8.0.2",
"react-docgen-typescript": "^2.2.2"
},
@@ -11042,7 +11066,7 @@
"@types/react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5",
"storybook": "10.5.6",
"typescript": ">= 4.9.x"
},
"peerDependenciesMeta": {
@@ -11058,9 +11082,9 @@
}
},
"node_modules/@storybook/react-webpack5/node_modules/@storybook/react/node_modules/@storybook/react-dom-shim": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/@storybook/react-dom-shim/-/react-dom-shim-10.5.5.tgz",
"integrity": "sha512-PIk7N3LLrZIxfNxmkvmQN1d5UQ70XEedT8n0GhBiXnM6XL09xPGB8n8TZXeJBRYluKhDQcAyQeT0/OZmcDVQJg==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/@storybook/react-dom-shim/-/react-dom-shim-10.5.6.tgz",
"integrity": "sha512-dV3oOHc5ImggxEqeIiUj4vvnQO5SKScFtqAkxgIWLju1wiSZSIqQ5Q4Mp12Rhs9hQrjF039DueH7f2xJJZfvSw==",
"dev": true,
"license": "MIT",
"funding": {
@@ -11072,7 +11096,7 @@
"@types/react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"storybook": "10.5.5"
"storybook": "10.5.6"
},
"peerDependenciesMeta": {
"@types/react": {
@@ -15618,9 +15642,9 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.11.11",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.11.tgz",
"integrity": "sha512-/yImnXwyTvgMkhgekLHok/Rx5vO6E0BmStWlSqKWMVm2a2ITuZ1Tn+9bgLS+gZRdZmWtd8nxuhHpdmCUOWsTQQ==",
"version": "2.11.12",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.12.tgz",
"integrity": "sha512-r7WnVImvVCeFpf2DOXfy41aPWzeNg3H/A2X4dKmy1QL0MSyyk/e7z8ihJ3N6Nn2PsdhkVlqnEfnUE4a05P2aTA==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -20137,9 +20161,9 @@
}
},
"node_modules/eslint-plugin-storybook": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/eslint-plugin-storybook/-/eslint-plugin-storybook-10.5.5.tgz",
"integrity": "sha512-xGCrddoZ8pMmwc7M5mp+habWbvqBjRuyGVTJ6RpavpoAP6NBUfGSRXHuBs7cHRDpDSuErYeDiEo4XHTonRp68g==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/eslint-plugin-storybook/-/eslint-plugin-storybook-10.5.6.tgz",
"integrity": "sha512-uOXhNkIH+iTdyViSmWnCrwtapasL57M3nq5yfST1H7y9djRLyuAIfNcf9cPBedc2G1oqI8jn3up/VHdN3y3Btw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -20148,7 +20172,7 @@
},
"peerDependencies": {
"eslint": ">=8",
"storybook": "10.5.5"
"storybook": "10.5.6"
}
},
"node_modules/eslint-plugin-testing-library": {
@@ -28464,9 +28488,9 @@
}
},
"node_modules/markdown-to-jsx": {
"version": "9.10.1",
"resolved": "https://registry.npmjs.org/markdown-to-jsx/-/markdown-to-jsx-9.10.1.tgz",
"integrity": "sha512-/fVJ8tKefGDQP40IONtD8O39V0sP0KkJmgkkFH1RBmPWKgnS6ljcOyUZ/ahEnQ3a3GsNZynM8KTR6b1n/6gFaQ==",
"version": "9.10.2",
"resolved": "https://registry.npmjs.org/markdown-to-jsx/-/markdown-to-jsx-9.10.2.tgz",
"integrity": "sha512-iR9GadlIox0q1uXnpqdxpF02Vb1WDmZ/QIXWjBR5htzjUEEhyIWbM65LuVKavdwJaVo4q95C/F2OOyNjWe91ig==",
"license": "MIT",
"engines": {
"node": ">= 18"
@@ -30112,9 +30136,9 @@
}
},
"node_modules/nanoid": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-6.0.0.tgz",
"integrity": "sha512-mkUH+rPkwU2qPadJ0oJZOjeZ5Mxn8Q1UhevwkTRWNuUZzyia3h4rhzK39hxaHTk0o2OxB8W2SQ6A8k23ZDi1pQ==",
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-6.0.1.tgz",
"integrity": "sha512-3wVS3i51pE2pi1k5FFL/95BGfVS0kSsvDVuGXHOtxox/TywUmtgq+3qiTOTbs9J7KfHaXPiN171k/A6dBnaXFw==",
"funding": [
{
"type": "github",
@@ -31818,9 +31842,9 @@
}
},
"node_modules/oxlint": {
"version": "1.76.0",
"resolved": "https://registry.npmjs.org/oxlint/-/oxlint-1.76.0.tgz",
"integrity": "sha512-6QoFioEU4fNdiUx/2Eo6TRd6NG7H7njnRCz8rhB66cZmMHDTqcm1Rjvl8Wry+ZTQMBAmyb4Mlf62Mk5X+eHSOw==",
"version": "1.77.0",
"resolved": "https://registry.npmjs.org/oxlint/-/oxlint-1.77.0.tgz",
"integrity": "sha512-qnGh8XJHaQ0dprrDXNQZgS0FgjI6v+V3+X8DwmaV++5Aamy6jGKfDdQ1TUvhUxtmKFAbEf4/WeO5QZX+5WSngg==",
"dev": true,
"license": "MIT",
"bin": {
@@ -31833,25 +31857,25 @@
"url": "https://github.com/sponsors/Boshen"
},
"optionalDependencies": {
"@oxlint/binding-android-arm-eabi": "1.76.0",
"@oxlint/binding-android-arm64": "1.76.0",
"@oxlint/binding-darwin-arm64": "1.76.0",
"@oxlint/binding-darwin-x64": "1.76.0",
"@oxlint/binding-freebsd-x64": "1.76.0",
"@oxlint/binding-linux-arm-gnueabihf": "1.76.0",
"@oxlint/binding-linux-arm-musleabihf": "1.76.0",
"@oxlint/binding-linux-arm64-gnu": "1.76.0",
"@oxlint/binding-linux-arm64-musl": "1.76.0",
"@oxlint/binding-linux-ppc64-gnu": "1.76.0",
"@oxlint/binding-linux-riscv64-gnu": "1.76.0",
"@oxlint/binding-linux-riscv64-musl": "1.76.0",
"@oxlint/binding-linux-s390x-gnu": "1.76.0",
"@oxlint/binding-linux-x64-gnu": "1.76.0",
"@oxlint/binding-linux-x64-musl": "1.76.0",
"@oxlint/binding-openharmony-arm64": "1.76.0",
"@oxlint/binding-win32-arm64-msvc": "1.76.0",
"@oxlint/binding-win32-ia32-msvc": "1.76.0",
"@oxlint/binding-win32-x64-msvc": "1.76.0"
"@oxlint/binding-android-arm-eabi": "1.77.0",
"@oxlint/binding-android-arm64": "1.77.0",
"@oxlint/binding-darwin-arm64": "1.77.0",
"@oxlint/binding-darwin-x64": "1.77.0",
"@oxlint/binding-freebsd-x64": "1.77.0",
"@oxlint/binding-linux-arm-gnueabihf": "1.77.0",
"@oxlint/binding-linux-arm-musleabihf": "1.77.0",
"@oxlint/binding-linux-arm64-gnu": "1.77.0",
"@oxlint/binding-linux-arm64-musl": "1.77.0",
"@oxlint/binding-linux-ppc64-gnu": "1.77.0",
"@oxlint/binding-linux-riscv64-gnu": "1.77.0",
"@oxlint/binding-linux-riscv64-musl": "1.77.0",
"@oxlint/binding-linux-s390x-gnu": "1.77.0",
"@oxlint/binding-linux-x64-gnu": "1.77.0",
"@oxlint/binding-linux-x64-musl": "1.77.0",
"@oxlint/binding-openharmony-arm64": "1.77.0",
"@oxlint/binding-win32-arm64-msvc": "1.77.0",
"@oxlint/binding-win32-ia32-msvc": "1.77.0",
"@oxlint/binding-win32-x64-msvc": "1.77.0"
},
"peerDependencies": {
"oxlint-tsgolint": ">=7.0.2001",
@@ -37607,16 +37631,16 @@
}
},
"node_modules/storybook": {
"version": "10.5.5",
"resolved": "https://registry.npmjs.org/storybook/-/storybook-10.5.5.tgz",
"integrity": "sha512-UscBIBJDloUeqntukHOhP1a5W/vouePDJbzPSxj466WK801FZtzQiMffMtkjzJiWSuj20wfaYlB2QQKh9aOYAg==",
"version": "10.5.6",
"resolved": "https://registry.npmjs.org/storybook/-/storybook-10.5.6.tgz",
"integrity": "sha512-VhYwqxPySa24CVXKoWD6gCZXx9//DTmo43YpusGuAoHDYj5Osjt8wuBRQVeGoaLUWnHiPWv8S+GYHrJEaBM6Rg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@storybook/global": "^5.0.0",
"@storybook/icons": "^2.0.2",
"@testing-library/dom": "^10.4.1",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/jest-dom": "6.9.1",
"@testing-library/user-event": "^14.6.1",
"@vitest/expect": "3.2.4",
"@vitest/spy": "3.2.4",
@@ -42969,15 +42993,6 @@
"node": ">=12"
}
},
"packages/superset-ui-core/node_modules/dompurify": {
"version": "3.4.13",
"resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.4.13.tgz",
"integrity": "sha512-2vmYIoqjze2d+kakP8S/nS5shfsl587kzwEjcGlTdiksUVgFHnFCsLYDVj/JNqJVOQZGSYBTmuycv0PodwmnMQ==",
"license": "(MPL-2.0 OR Apache-2.0)",
"optionalDependencies": {
"@types/trusted-types": "^2.0.7"
}
},
"packages/superset-ui-core/node_modules/react-ace": {
"version": "14.0.1",
"resolved": "https://registry.npmjs.org/react-ace/-/react-ace-14.0.1.tgz",
+9 -9
View File
@@ -186,12 +186,12 @@
"lodash": "^4.18.1",
"lodash-es": "^4.18.1",
"mapbox-gl": "^3.27.0",
"markdown-to-jsx": "^9.10.1",
"markdown-to-jsx": "^9.10.2",
"match-sorter": "^8.3.0",
"memoize-one": "^6.0.0",
"mousetrap": "^1.6.5",
"mustache": "^4.2.0",
"nanoid": "^6.0.0",
"nanoid": "^6.0.1",
"ol": "^10.10.0",
"query-string": "9.4.1",
"re-resizable": "^6.11.2",
@@ -257,9 +257,9 @@
"@istanbuljs/nyc-config-typescript": "^1.0.1",
"@playwright/test": "^1.62.1",
"@pmmmwh/react-refresh-webpack-plugin": "^0.6.2",
"@storybook/addon-docs": "10.5.5",
"@storybook/addon-links": "10.5.5",
"@storybook/react-webpack5": "10.5.5",
"@storybook/addon-docs": "10.5.6",
"@storybook/addon-links": "10.5.6",
"@storybook/react-webpack5": "10.5.6",
"@storybook/test-runner": "0.24.4",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.47",
@@ -295,7 +295,7 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.11",
"baseline-browser-mapping": "^2.11.12",
"cheerio": "1.2.0",
"concurrently": "^10.0.4",
"copy-webpack-plugin": "^14.0.0",
@@ -312,7 +312,7 @@
"eslint-plugin-no-only-tests": "^3.4.0",
"eslint-plugin-react-prefer-function-component": "^5.0.0",
"eslint-plugin-react-you-might-not-need-an-effect": "^1.0.1",
"eslint-plugin-storybook": "10.5.5",
"eslint-plugin-storybook": "10.5.6",
"eslint-plugin-testing-library": "^7.16.2",
"eslint-plugin-theme-colors": "file:eslint-rules/eslint-plugin-theme-colors",
"fetch-mock": "^12.6.0",
@@ -332,7 +332,7 @@
"minimizer-webpack-plugin": "^5.6.1",
"open-cli": "^9.0.0",
"oxfmt": "^0.61.0",
"oxlint": "^1.76.0",
"oxlint": "^1.77.0",
"po2json": "^0.4.5",
"postcss-styled-syntax": "^0.7.2",
"process": "^0.11.10",
@@ -343,7 +343,7 @@
"source-map": "^0.8.0",
"source-map-support": "^0.5.21",
"speed-measure-webpack-plugin": "^1.6.0",
"storybook": "10.5.5",
"storybook": "10.5.6",
"style-loader": "^4.0.0",
"stylelint": "^17.14.1",
"swc-loader": "^0.2.7",
@@ -17,6 +17,7 @@
* under the License.
*/
import { useState } from 'react';
import { Button } from '../Button';
import { Modal } from './Modal';
import type { ModalProps, ModalFuncProps } from './types';
@@ -179,3 +180,74 @@ ModalFunctions.args = {
maskClosable: true,
mask: true,
};
/**
* Two top-level Modals that are React siblings, not nested inside one
* another (e.g. a "View query" modal and a confirmation dialog it can
* trigger, like `UnsavedChangesModal`). Ant Design only assigns an
* automatically-incremented z-index when a Modal is nested inside another
* *currently open* Modal's React tree, so two siblings always fall back to
* the same static z-index and are tie-broken by DOM order: whichever
* `.ant-modal-wrap` was inserted later paints on top.
*
* With `destroyOnHidden={false}` (Ant Design's default), a Modal's wrap
* node is created once, lazily, on first open, and is never removed or
* recreated afterward. So the modal that happens to have been opened
* *first ever*, not most recently, keeps winning the DOM-order tiebreak
* even after being closed and reopened. Toggle "Reproduce stale DOM order"
* off to see the fix: with `destroyOnHidden`, every open recreates the wrap
* node at the end of the document, so DOM order (and stacking) always
* matches true open-recency and no manual z-index is ever needed.
*
* To see the bug: click "Open A", close it, then "Open B", then "Open A"
* again -- with the toggle on, A renders behind B despite being the modal
* that was opened most recently.
*/
export const SiblingModalStacking = ({
reproduceStaleDomOrder,
}: {
reproduceStaleDomOrder: boolean;
}) => {
const [showA, setShowA] = useState(false);
const [showB, setShowB] = useState(false);
return (
<div>
<Button onClick={() => setShowA(true)} buttonStyle="secondary">
Open A
</Button>
<Button onClick={() => setShowB(true)} buttonStyle="secondary">
Open B
</Button>
<Modal
name="modal-a"
title="Modal A"
show={showA}
onHide={() => setShowA(false)}
destroyOnHidden={!reproduceStaleDomOrder}
>
Modal A content
</Modal>
<Modal
name="modal-b"
title="Modal B"
show={showB}
onHide={() => setShowB(false)}
destroyOnHidden={!reproduceStaleDomOrder}
>
Modal B content
</Modal>
</div>
);
};
SiblingModalStacking.args = {
reproduceStaleDomOrder: true,
};
SiblingModalStacking.argTypes = {
reproduceStaleDomOrder: {
control: 'boolean',
description:
'On: Ant Design default behavior, a modal opened once keeps its DOM position forever (the bug from #42510). Off: destroyOnHidden, DOM order always matches true open-recency (the fix).',
},
};
@@ -16,7 +16,15 @@
* specific language governing permissions and limitations
* under the License.
*/
import { render, screen, userEvent } from '@superset-ui/core/spec';
import { useState } from 'react';
import {
render,
screen,
userEvent,
waitFor,
within,
} from '@superset-ui/core/spec';
import { Modal, RawAntdTooltip } from '@superset-ui/core/components';
import { UnsavedChangesModal } from '.';
test('should render nothing if showModal is false', () => {
@@ -94,3 +102,173 @@ test('should only call handleSave when clicking the Save button', async () => {
expect(mockOnHide).not.toHaveBeenCalled();
expect(mockOnConfirmNavigation).not.toHaveBeenCalled();
});
// Regression coverage for the underlying bug (#42510): this modal could
// render BEHIND another already-open modal (e.g. a draggable "View query"
// modal). Two plain top-level Modal siblings (neither nested inside the
// other's React tree) fall back to the same static z-index, tie-broken by
// DOM order: whichever `.ant-modal-wrap` comes later in the document paints
// on top -- `destroyOnHidden` is what makes every open recreate this
// modal's wrap fresh at the end of the document, so it wins that tie. But
// the real #42510 repro isn't actually a tie: "View query" renders as a
// dropdown menu item's label, and Ant Design's Menu.Item wraps every item's
// content in a Tooltip (even one that never opens), which hands its
// children a real elevated z-index via React context. That's why this
// modal also sets an explicit `zIndex` -- comfortably above what that
// inherited context can produce -- rather than relying on DOM order alone.
function dialogWrap(titleText: string) {
const dialogs = screen.queryAllByRole('dialog');
// rc-util's `useId` hook always returns the same mocked id ("test-id") in
// test environments, so with two dialogs open at once their
// `aria-labelledby` ids collide and `getByRole('dialog', { name })` can't
// tell them apart. Find each by its title text instead.
const dialog = dialogs.find(d => within(d).queryByText(titleText));
return dialog?.closest<HTMLElement>('.ant-modal-wrap') ?? null;
}
test('renders above an already-open modal that also has no elevated z-index', async () => {
render(
<>
<Modal show title="Other open modal" onHide={() => {}}>
<div>Other modal content</div>
</Modal>
<UnsavedChangesModal
showModal
onHide={() => {}}
handleSave={() => {}}
onConfirmNavigation={() => {}}
/>
</>,
);
const otherWrap = await waitFor(() => {
const wrap = dialogWrap('Other open modal');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
const unsavedChangesWrap = await waitFor(() => {
const wrap = dialogWrap('Unsaved Changes');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
// eslint-disable-next-line no-bitwise
expect(
otherWrap.compareDocumentPosition(unsavedChangesWrap) &
Node.DOCUMENT_POSITION_FOLLOWING,
).toBeTruthy();
});
// This is the actual #42510 repro, not just a tied-sibling stand-in: "View
// query" is rendered as a dropdown menu item's label, so Ant Design's
// Menu.Item silently wraps it in a Tooltip (title/open both stay falsy, it
// never visibly opens) purely for its own ellipsis-title behavior. That
// Tooltip still supplies a real, elevated z-index to its children via
// context, so the modal nested inside it doesn't tie with a plain top-level
// modal the way the previous test's "Other open modal" does -- DOM order
// can't be the tie-breaker for two z-indexes that were never equal.
test('renders above a modal nested in a menu item Tooltip wrapper, which gets a real elevated z-index', async () => {
render(
<>
<RawAntdTooltip title={null} open={false}>
<Modal show title="View query" onHide={() => {}}>
<div>query body</div>
</Modal>
</RawAntdTooltip>
<UnsavedChangesModal
showModal
onHide={() => {}}
handleSave={() => {}}
onConfirmNavigation={() => {}}
/>
</>,
);
const viewQueryWrap = await waitFor(() => {
const wrap = dialogWrap('View query');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
const unsavedChangesWrap = await waitFor(() => {
const wrap = dialogWrap('Unsaved Changes');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
// The Tooltip wrapper does give "View query" a real inline z-index above
// the base -- confirming this test actually exercises an elevated,
// non-tied sibling rather than accidentally falling back to the tied
// case the previous test already covers.
expect(Number(viewQueryWrap.style.zIndex)).toBeGreaterThan(0);
expect(Number(unsavedChangesWrap.style.zIndex)).toBeGreaterThan(
Number(viewQueryWrap.style.zIndex),
);
});
test('still renders on top after being opened, closed, and reopened once the other modal is already open', async () => {
function Harness() {
const [showOther, setShowOther] = useState(false);
const [showUnsaved, setShowUnsaved] = useState(false);
return (
<>
<button type="button" onClick={() => setShowOther(true)}>
open other
</button>
<button type="button" onClick={() => setShowUnsaved(true)}>
open unsaved
</button>
<Modal
show={showOther}
title="Other open modal"
onHide={() => setShowOther(false)}
>
<div>Other modal content</div>
</Modal>
<UnsavedChangesModal
showModal={showUnsaved}
onHide={() => setShowUnsaved(false)}
handleSave={() => {}}
// Mirrors real callers: confirming navigation is what dismisses
// this modal, not `onHide` directly (see the Discard-button test
// above -- clicking Discard never calls `onHide` on its own).
onConfirmNavigation={() => setShowUnsaved(false)}
/>
</>
);
}
render(<Harness />);
// Open this modal once -- e.g. some other in-app action tripped it --
// before the modal it's supposed to interrupt has ever been opened. Its
// wrap node gets created now, first in the document.
userEvent.click(screen.getByText('open unsaved'));
await waitFor(() => expect(dialogWrap('Unsaved Changes')).not.toBeNull());
userEvent.click(await screen.findByRole('button', { name: /discard/i }));
await waitFor(() => expect(dialogWrap('Unsaved Changes')).toBeNull());
// Now open the modal it's meant to interrupt for the first time.
userEvent.click(screen.getByText('open other'));
const otherWrap = await waitFor(() => {
const wrap = dialogWrap('Other open modal');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
// Reopen this modal -- the real scenario the bug report describes. If its
// wrap node were still the one created on the first open above, it would
// be stuck earlier in the document than `otherWrap` and render behind it
// again.
userEvent.click(screen.getByText('open unsaved'));
const unsavedChangesWrap = await waitFor(() => {
const wrap = dialogWrap('Unsaved Changes');
expect(wrap).not.toBeNull();
return wrap as HTMLElement;
});
// eslint-disable-next-line no-bitwise
expect(
otherWrap.compareDocumentPosition(unsavedChangesWrap) &
Node.DOCUMENT_POSITION_FOLLOWING,
).toBeTruthy();
});
@@ -17,13 +17,10 @@
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { useTheme } from '@apache-superset/core/theme';
import { Icons, Modal, Typography, Button } from '@superset-ui/core/components';
import type { FC, ReactElement } from 'react';
// Ant Design's default modal zIndex is 1000. Using a higher value ensures
// this dialog always renders above other open modals (e.g. a draggable View SQL modal).
const UNSAVED_CHANGES_MODAL_Z_INDEX = 1300;
export type UnsavedChangesModalProps = {
showModal: boolean;
onHide: () => void;
@@ -31,7 +28,6 @@ export type UnsavedChangesModalProps = {
onConfirmNavigation: () => void;
title?: string;
body?: string;
zIndex?: number;
};
export const UnsavedChangesModal: FC<UnsavedChangesModalProps> = ({
@@ -41,32 +37,61 @@ export const UnsavedChangesModal: FC<UnsavedChangesModalProps> = ({
onConfirmNavigation,
title = 'Unsaved Changes',
body = "If you don't save, changes will be lost.",
zIndex = UNSAVED_CHANGES_MODAL_Z_INDEX,
}: UnsavedChangesModalProps): ReactElement => (
<Modal
centered
responsive
onHide={onHide}
show={showModal}
width="444px"
zIndex={zIndex}
title={
<>
<Icons.WarningOutlined iconSize="m" style={{ marginRight: 8 }} />
{title}
</>
}
footer={
<>
<Button buttonStyle="secondary" onClick={onConfirmNavigation}>
{t('Discard')}
</Button>
<Button buttonStyle="primary" onClick={handleSave}>
{t('Save')}
</Button>
</>
}
>
<Typography.Text>{body}</Typography.Text>
</Modal>
);
}: UnsavedChangesModalProps): ReactElement => {
const theme = useTheme();
return (
<Modal
centered
responsive
onHide={onHide}
show={showModal}
width="444px"
// This modal always interrupts something already on screen (a
// draggable "View query" modal, an in-progress form, etc). Ant
// Design only assigns a higher z-index automatically when a Modal is
// nested inside another *currently open Modal's* React tree. This
// one is always a top-level sibling of whatever it interrupts, so on
// its own it would fall back to the same static base z-index -- BUT
// the modal it's interrupting isn't always a plain top-level sibling
// itself: "View query" is rendered as a dropdown menu item's label,
// and Ant Design's Menu.Item silently wraps every item's content in
// a Tooltip (even when that tooltip never opens), which supplies a
// real ZIndexContext to its children. That gives the nested "View
// query" Modal a genuinely higher, non-tied z-index (theme's popup
// base plus ~200) than this modal's plain base value, so DOM order
// alone (destroyOnHidden below) can't win the tie -- there isn't
// one. An explicit zIndex, comfortably above any such context-fed
// value, guarantees this modal isn't shadowed by a sibling that
// happens to inherit an elevated stacking context.
zIndex={theme.zIndexPopupBase + 1000}
// Without destroyOnHidden, a Modal's portal node is created once
// (lazily, on first open) and then left in place forever, so if this
// dialog is ever opened once before whatever it's interrupting is
// opened, a later reopen would go right back to that stale,
// now-too-early DOM position. destroyOnHidden tears the portal down
// on every close so every open recreates it fresh at the end of the
// DOM, keeping DOM order (the tie-breaker for any modals that
// genuinely do share this one's base z-index) tracking true
// open-recency.
destroyOnHidden
title={
<>
<Icons.WarningOutlined iconSize="m" style={{ marginRight: 8 }} />
{title}
</>
}
footer={
<>
<Button buttonStyle="secondary" onClick={onConfirmNavigation}>
{t('Discard')}
</Button>
<Button buttonStyle="primary" onClick={handleSave}>
{t('Save')}
</Button>
</>
}
>
<Typography.Text>{body}</Typography.Text>
</Modal>
);
};
@@ -55,15 +55,19 @@ import { getDefaultTooltip } from '../utils/tooltip';
import { Refs } from '../types';
import { getContributionLabel } from './utils';
const percentFormatter = getNumberFormatter(NumberFormats.PERCENT_2_POINT);
const defaultPercentFormatter = getNumberFormatter(
NumberFormats.PERCENT_2_POINT,
);
export function parseParams({
params,
numberFormatter,
percentFormatter = defaultPercentFormatter,
sanitizeName = false,
}: {
params: Pick<CallbackDataParams, 'name' | 'value' | 'percent'>;
numberFormatter: ValueFormatter;
percentFormatter?: ValueFormatter;
sanitizeName?: boolean;
}): string[] {
const { name: rawName = '', value, percent } = params;
@@ -296,6 +300,9 @@ export default function transformProps(
currencyCodeColumn,
detectedCurrency,
);
const percentFormatter = numberFormat?.endsWith('%')
? getNumberFormatter(numberFormat)
: defaultPercentFormatter;
let data = rawData;
const otherRows: DataRecord[] = [];
@@ -446,6 +453,7 @@ export default function transformProps(
const [name, formattedValue, formattedPercent] = parseParams({
params,
numberFormatter,
percentFormatter,
});
switch (labelType) {
case EchartsPieLabelType.Key:
@@ -566,6 +574,7 @@ export default function transformProps(
const [name, formattedValue, formattedPercent] = parseParams({
params,
numberFormatter,
percentFormatter,
sanitizeName: true,
});
if (params?.data?.isOther) {
@@ -311,6 +311,26 @@ describe('Pie label string template', () => {
).toEqual('Tablet:123,456\n55.50%');
});
test('should format percentages using a percentage number format', () => {
expect(
format({
label_type: 'template',
label_template: '{name}:{value}\n{percent}',
number_format: '.1%',
}),
).toEqual('Tablet:12345600.0%\n55.5%');
});
test('should use the default percentage format when number format is absent', () => {
expect(
format({
label_type: 'template',
label_template: '{name}:{value}\n{percent}',
number_format: undefined,
}),
).toEqual('Tablet:123k\n55.50%');
});
test('should be compatible with ECharts raw variable syntax', () => {
expect(
format({
@@ -261,6 +261,60 @@ describe('isUserEditorOrAdmin', () => {
});
});
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
describe('isUserAdmin with a custom AUTH_ROLE_ADMIN', () => {
// The file-level `jest.mock('src/utils/getBootstrapData', ...)` above
// permanently stubs out getBootstrapData with fixed data, which would
// shadow the DOM-driven bootstrap data these tests set up. Unmock it so
// the re-imported permissionUtils picks up the real implementation
// (reading document.getElementById('app')), then restore the mock
// afterward so later tests in this file keep their expected stub.
beforeEach(() => {
jest.unmock('src/utils/getBootstrapData');
});
afterEach(() => {
document.body.innerHTML = '';
jest.resetModules();
jest.mock('src/utils/getBootstrapData', () => ({
__esModule: true,
default: jest.fn(() => ({
common: {
user_subjects: [10],
},
})),
}));
});
test('recognizes a user in the configured custom admin role', async () => {
document.body.innerHTML =
'<div id="app" data-bootstrap=\'{"common":{"conf":{"AUTH_ROLE_ADMIN":"SuperAdmin"}}}\'></div>';
jest.resetModules();
const { isUserAdmin: isUserAdminWithCustomRole } =
await import('./permissionUtils');
expect(
isUserAdminWithCustomRole({
username: 'super-admin',
permissions: {},
roles: { SuperAdmin: [['can_write', 'Dashboard']] },
}),
).toEqual(true);
});
test('does not throw and falls back to the default role when bootstrap data has no conf', async () => {
document.body.innerHTML =
'<div id="app" data-bootstrap=\'{"common":{}}\'></div>';
jest.resetModules();
const { isUserAdmin: isUserAdminWithoutConf } =
await import('./permissionUtils');
expect(isUserAdminWithoutConf(adminUser)).toEqual(true);
});
});
test('userHasPermission always returns true for admin user', () => {
arbitraryPermissions.forEach(permissionView => {
expect(
@@ -26,9 +26,8 @@ import Subject from 'src/types/Subject';
import { findPermission } from 'src/utils/findPermission';
import getBootstrapData from 'src/utils/getBootstrapData';
// this should really be a config value,
// but is hardcoded in backend logic already, so...
const ADMIN_ROLE_NAME = 'admin';
const bootstrapData = getBootstrapData();
const ADMIN_ROLE_NAME = bootstrapData.common?.conf?.AUTH_ROLE_ADMIN || 'Admin';
const getUserSubjects = (): number[] =>
getBootstrapData()?.common?.user_subjects ?? [];
@@ -64,7 +63,7 @@ export const isUserAdmin = (
) =>
isUserWithPermissionsAndRoles(user) &&
Object.keys(user.roles || {}).some(
role => role.toLowerCase() === ADMIN_ROLE_NAME,
role => role.toLowerCase() === ADMIN_ROLE_NAME.toLowerCase(),
);
export const isUserEditorOrAdmin = (
@@ -0,0 +1,128 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { render, screen, waitFor } from 'spec/helpers/testing-library';
import { DatasourceType } from '@superset-ui/core';
import DatasourcePanel, {
IDatasource,
} from 'src/explore/components/DatasourcePanel';
import {
columns,
metrics,
} from 'src/explore/components/DatasourcePanel/fixtures';
import ExploreContainer from 'src/explore/components/ExploreContainer';
import { DndMetricSelect } from 'src/explore/components/controls/DndColumnSelectControl';
import DatasourceControl from 'src/explore/components/controls/DatasourceControl';
/**
* DatasourcePanel.test.tsx mocks react-virtualized-auto-sizer to a fixed
* height, which bypasses react-window's own size self-measurement path
* entirely. This test instead drives a real ResizeObserver callback so the
* panel renders through the same AutoSizer -> react-window List pipeline
* the browser uses, to guard against regressions like
* https://github.com/apache/superset/issues/43008.
*/
class FakeResizeObserver {
callback: ResizeObserverCallback;
constructor(callback: ResizeObserverCallback) {
this.callback = callback;
}
observe(target: Element) {
Object.defineProperty(target, 'getBoundingClientRect', {
configurable: true,
value: () => ({
width: 300,
height: 600,
top: 0,
left: 0,
bottom: 600,
right: 300,
x: 0,
y: 0,
toJSON: () => {},
}),
});
setTimeout(() => {
this.callback(
[
{
target,
contentRect: { width: 300, height: 600 },
} as ResizeObserverEntry,
],
this,
);
}, 0);
}
unobserve() {}
disconnect() {}
}
const originalResizeObserver = window.ResizeObserver;
beforeEach(() => {
window.ResizeObserver = FakeResizeObserver;
});
afterEach(() => {
window.ResizeObserver = originalResizeObserver;
});
const datasource: IDatasource = {
id: 1,
type: DatasourceType.Table,
columns,
metrics,
database: { id: 1 },
datasource_name: 'table1',
};
test('renders metrics and columns through the real (unmocked) AutoSizer + react-window List pipeline', async () => {
render(
<ExploreContainer>
<DatasourcePanel
datasource={datasource}
controls={{
datasource: {
validationErrors: null,
mapStateToProps: () => ({ value: undefined }),
type: DatasourceControl,
label: 'Datasource',
datasource,
},
}}
actions={{ setControlValue: jest.fn() }}
width={300}
/>
<DndMetricSelect savedMetrics={[]} columns={[]} onChange={jest.fn()} />
</ExploreContainer>,
{ useDnd: true, useRedux: true },
);
await waitFor(
() => {
expect(screen.getByText(metrics[0].metric_name)).toBeInTheDocument();
},
{ timeout: 3000 },
);
expect(screen.getByText(columns[0].column_name)).toBeInTheDocument();
});
@@ -84,4 +84,14 @@ describe('ensureStaticPrefix should be idempotent', () => {
staticAssetsPrefixMock.mockReturnValue('');
expect(ensureStaticPrefix('/static/x.png')).toBe('/static/x.png');
});
test('returns undefined instead of throwing when passed undefined', () => {
staticAssetsPrefixMock.mockReturnValue('/superset');
expect(ensureStaticPrefix(undefined)).toBeUndefined();
});
test('returns undefined instead of throwing when passed null', () => {
staticAssetsPrefixMock.mockReturnValue('/superset');
expect(ensureStaticPrefix(null)).toBeUndefined();
});
});
+9 -1
View File
@@ -35,9 +35,17 @@ export function assetUrl(path: string): string {
* segment boundary is returned unchanged, mirroring the dedupe pattern used by
* `ensureAppRoot` in pathUtils.ts and `SupersetClient.getUrl`.
*
* If `url_or_path` is null or undefined, `undefined` is returned so callers
* (e.g. a partial theme reaching the frontend from a source other than
* `superset_config.py`, such as a DB-stored theme created via the API) don't
* crash on the unconditional `.startsWith()` call below.
*
* @param url_or_path A url or relative path to a resource
*/
export function ensureStaticPrefix(url_or_path: string): string {
export function ensureStaticPrefix(
url_or_path: string | null | undefined,
): string | undefined {
if (url_or_path == null) return undefined;
if (!url_or_path.startsWith('/')) return url_or_path;
const prefix = staticAssetsPrefix();
if (
+2 -3
View File
@@ -195,8 +195,8 @@ const RedirectWarning = lazy(
type Routes = {
path: string;
Component: ComponentType<any>;
Fallback?: ComponentType<any>;
Component: ComponentType;
Fallback?: ComponentType;
props?: ComponentProps<any>;
}[];
@@ -241,7 +241,6 @@ export const routes: Routes = [
{ path: RoutePaths.SQLLAB, Component: SqlLab },
{ path: RoutePaths.USER_INFO, Component: UserInfo },
{ path: RoutePaths.ACTION_LOG, Component: ActionLogList },
{ path: RoutePaths.REGISTRATIONS, Component: UserRegistrations },
];
if (isFeatureEnabled(FeatureFlag.TaggingSystem)) {
+147 -147
View File
@@ -15,7 +15,7 @@
"jsonwebtoken": "^9.0.3",
"lodash-es": "^4.18.1",
"winston": "^3.19.0",
"ws": "^8.21.1"
"ws": "^8.21.2"
},
"devDependencies": {
"@eslint/js": "^9.25.1",
@@ -25,14 +25,14 @@
"@types/node": "^26.1.2",
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.65.0",
"@typescript-eslint/parser": "^8.65.0",
"@typescript-eslint/parser": "^8.66.0",
"eslint": "^10.8.0",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.9.0",
"oxfmt": "^0.61.0",
"oxfmt": "^0.62.0",
"tscw-config": "^1.1.2",
"typescript": "^6.0.3",
"typescript-eslint": "^8.65.0",
"typescript-eslint": "^8.66.0",
"vitest": "^4.1.10"
},
"engines": {
@@ -310,9 +310,9 @@
}
},
"node_modules/@oxfmt/binding-android-arm-eabi": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-android-arm-eabi/-/binding-android-arm-eabi-0.61.0.tgz",
"integrity": "sha512-BaS+1OVvg9sr+Xav0+KdWedQRcAzrdoEcwMZeqoc2F6ieC1s/t5eM35YQoRPQ7vAqkZ+p3tbQb1r9I9mrV5oGA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-android-arm-eabi/-/binding-android-arm-eabi-0.62.0.tgz",
"integrity": "sha512-pdsv0C4gPjJ8H1+sd8u0BDx+yLACTL+rgeMIOL1ln4ihSnhw8CWXtYWgvcSkyTfgGBIzFKab+d8rx9Xl4en/Kw==",
"cpu": [
"arm"
],
@@ -327,9 +327,9 @@
}
},
"node_modules/@oxfmt/binding-android-arm64": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-android-arm64/-/binding-android-arm64-0.61.0.tgz",
"integrity": "sha512-of8atAV0M1egGcVOMbgZCvc10sFOP3ayQBNQV5h5G3fNq8gACdEswfFk9bzGrdbM23rtg0Coxi7np7oPLcueNw==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-android-arm64/-/binding-android-arm64-0.62.0.tgz",
"integrity": "sha512-WC3YQ7uS/KtDrjmqwBviwFKe9qeoi+eXx8aX1z/ffG23Md75myjrJaQqTuJvdOLPoa4EYTjDWH0dHXfwulCVog==",
"cpu": [
"arm64"
],
@@ -344,9 +344,9 @@
}
},
"node_modules/@oxfmt/binding-darwin-arm64": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-darwin-arm64/-/binding-darwin-arm64-0.61.0.tgz",
"integrity": "sha512-7l8+5ov4BGwtAcmpzvEik/TG3bciwyw/S3e6j5GKH7pcQqcgCVxD3AuJeP6upto+SOTBKQ4wrrdbMt0gq8fHSQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-darwin-arm64/-/binding-darwin-arm64-0.62.0.tgz",
"integrity": "sha512-GM8Yf3LjjaR1I8PD0SfeoIlwhsh9GvSF+cQ8sf624Yxnjsyumn95aFzYfKJVefblfDIiOAnZ7QVm2sa21Er/0Q==",
"cpu": [
"arm64"
],
@@ -361,9 +361,9 @@
}
},
"node_modules/@oxfmt/binding-darwin-x64": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-darwin-x64/-/binding-darwin-x64-0.61.0.tgz",
"integrity": "sha512-Fnz4dDDXBb7udk+DmwelNjxbD6yptyxwCqwCH2ebo4RVLxVsRfFsn/AHJC49KIltPrVokamGv4SSOsiV50DTxQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-darwin-x64/-/binding-darwin-x64-0.62.0.tgz",
"integrity": "sha512-d5THp7F8bCxLqNogEXDORRsQD6dosf3EyFtnXfBer6v+8tGdcWIjoDX9WaXrrF/26zOmL8qHpPTKCEvpBDmZkQ==",
"cpu": [
"x64"
],
@@ -378,9 +378,9 @@
}
},
"node_modules/@oxfmt/binding-freebsd-x64": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-freebsd-x64/-/binding-freebsd-x64-0.61.0.tgz",
"integrity": "sha512-mddOebKNCP+AucmzfNsk3jgbr681qAUvgMqi865GW5gWLJ/AnzXbvjQRrny0e++NAN8aphav/aRSrfFxNsNjpA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-freebsd-x64/-/binding-freebsd-x64-0.62.0.tgz",
"integrity": "sha512-1DnrtXGZooOZ0fHgAXZUaDQzBVh1CM2MNW4oBXyQ2aWKvCHjyljvT9fgBkOM0fEOb96X5eqtcfJ0YUVt9jj66g==",
"cpu": [
"x64"
],
@@ -395,9 +395,9 @@
}
},
"node_modules/@oxfmt/binding-linux-arm-gnueabihf": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-0.61.0.tgz",
"integrity": "sha512-svx59iYL+DbaZGZUIoice4W0CjRXGExnbz7Re+awIb60rVxBS2KrU7Hnlx+nZYanLGLpjneUEgo/VFEKkSZAyQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-0.62.0.tgz",
"integrity": "sha512-4pQDHOYRH+Huqe0StIaWyvk2CVl/aTaqSrbZpA3/pLS2xH24ME7lBgYprhQF2fRkHBzhGGGKliwxFsDdHwx59g==",
"cpu": [
"arm"
],
@@ -412,9 +412,9 @@
}
},
"node_modules/@oxfmt/binding-linux-arm-musleabihf": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-0.61.0.tgz",
"integrity": "sha512-BYK9MPJPCf6d+fLKMTruThmEyCtHzQ1zLcsrTlUVkmnoXIaHAbfpeLYQwX1tkjs7W11dyzoi6HFvKcdnvX1zNg==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-0.62.0.tgz",
"integrity": "sha512-X0jAaZJFMCVKhB6YyWVTQ/wN2DLsBcZKSMqTS76bF6riT+XZdtg2FPEdjDvdVbunO9cG+tWiVaEs4Zs38lxYog==",
"cpu": [
"arm"
],
@@ -429,9 +429,9 @@
}
},
"node_modules/@oxfmt/binding-linux-arm64-gnu": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-0.61.0.tgz",
"integrity": "sha512-QUaCNLq2/EC6G5ljOuFanl9Lgw6ZWp4co7rs4+KOMUzbGfA4Lq58FHRjjF9sVIG+93XSbo343MxFATrOU1qctA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-0.62.0.tgz",
"integrity": "sha512-682Z8T5s8T5ATArYtsejKvbIfd8LEAXyyDkKkoZVq8HND7Vx8TYLlrDjDSeYfodMeVwHOgkj13lJYR8cj6vUSg==",
"cpu": [
"arm64"
],
@@ -449,9 +449,9 @@
}
},
"node_modules/@oxfmt/binding-linux-arm64-musl": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm64-musl/-/binding-linux-arm64-musl-0.61.0.tgz",
"integrity": "sha512-S6uvJ6MXnRXl+zTs0CARNDvkE+cymj0EVWEKKsyKnlLlqTyQJBjw5s4D2pSIOZc+S46cy4STefzcr/sm0VzVPA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-arm64-musl/-/binding-linux-arm64-musl-0.62.0.tgz",
"integrity": "sha512-lk25fAl7KWaLWVJcW0CHEXB7QlQZtx5eDkjpaGMK0hzXTjUe0Wmlu8IKuFHoviSOcEJedRTs4VE/506VqGxGew==",
"cpu": [
"arm64"
],
@@ -469,9 +469,9 @@
}
},
"node_modules/@oxfmt/binding-linux-ppc64-gnu": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-0.61.0.tgz",
"integrity": "sha512-6VDlRcytvZG6UlSIdAFKDLbppo9tvPxrWzle6vHldYFMeuDPQEfMKrkwezp7FaBq1wik9ra554ZZeRPsyIkFpg==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-0.62.0.tgz",
"integrity": "sha512-SFyNqHQLwySceWNLhiSldx7wPXRAzP0L0WcW9GegP3uWrpZGJiZlQO85NbHAFPEfxR9PhZ9qSnZryEh7+v+4Gw==",
"cpu": [
"ppc64"
],
@@ -489,9 +489,9 @@
}
},
"node_modules/@oxfmt/binding-linux-riscv64-gnu": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-0.61.0.tgz",
"integrity": "sha512-KkBTYbzExpbmn15XjKPLu2fRV2PVlq+KWt+brad5rwIa03vdYoaDRWiS7raHII/dCTR6Ro4UpYUCH4t6lif4WQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-0.62.0.tgz",
"integrity": "sha512-KYj55C1ywJfHo6+aKDuEmUtVEdJALsC5GwayDGsI6FGz2GxFqNr/mA8nxVsNbJzm7sE5MRqTQ9ziImSzhYXysA==",
"cpu": [
"riscv64"
],
@@ -509,9 +509,9 @@
}
},
"node_modules/@oxfmt/binding-linux-riscv64-musl": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-0.61.0.tgz",
"integrity": "sha512-69tzIq7sJLVB9dxYYtvMzcSSsnZHSO+U2U19O2RqDqgj6+Q4O7HjSXdaszbcgqzhsUwzSH7z5kWvk8nmf6BHTg==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-0.62.0.tgz",
"integrity": "sha512-BhZDNo5GOU5nC378RhD0/XpvaEBHsH3HLgJp8YZX3A0InC7oivzA63HsRmiXFLtLSHAstEVrDf6fbC7Rs8Jh/A==",
"cpu": [
"riscv64"
],
@@ -529,9 +529,9 @@
}
},
"node_modules/@oxfmt/binding-linux-s390x-gnu": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-0.61.0.tgz",
"integrity": "sha512-Oqi/N0OvtOVXsPKAOOhKgGH3msRYF8BLJaNBbWiupRiKoKVyc8JRKPCfarkQJC+RgP9U8raUKLe+bNwd0HUMiA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-0.62.0.tgz",
"integrity": "sha512-UyAFmyHkgSgUJ/wOM4p3U8AC2yAFvRH5PNBs7TnK0fObTT/XSWcdr/lAzPSWaekHaZFaMeFZyk9n93Joq3J93A==",
"cpu": [
"s390x"
],
@@ -549,9 +549,9 @@
}
},
"node_modules/@oxfmt/binding-linux-x64-gnu": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-x64-gnu/-/binding-linux-x64-gnu-0.61.0.tgz",
"integrity": "sha512-3TKwv/ed4uwJSemAA8P9XcoqETpjQI4waquF9UilhA9Mn/dhr1PdUEXWlL74mtc6ZNfmKPA9+NEJm01nRF8CVA==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-x64-gnu/-/binding-linux-x64-gnu-0.62.0.tgz",
"integrity": "sha512-1iYMP0leytWazFubD/WnINJuIrzRPuoL1aWEJdlGezEzDbTxcd29R4r8IUzP2oWeKst5V02uMJgR2NILlPlG6w==",
"cpu": [
"x64"
],
@@ -569,9 +569,9 @@
}
},
"node_modules/@oxfmt/binding-linux-x64-musl": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-x64-musl/-/binding-linux-x64-musl-0.61.0.tgz",
"integrity": "sha512-uFso4u4nLkVSlMCpgjyvWV60Gt7GvDQHnk1mmRxHIkZTMB0ljpUKwCD9FYGgN9H97x2wYl0UwEjgRZaPIuhEhw==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-linux-x64-musl/-/binding-linux-x64-musl-0.62.0.tgz",
"integrity": "sha512-4rA/URtJSTVNVAQz6Q8wf7SaRvOXVy+TizriT9hs/Y1XhLR/R+92uWKRQG8yFWRAIEBbFHJ6WevQcl/G9SXEfw==",
"cpu": [
"x64"
],
@@ -589,9 +589,9 @@
}
},
"node_modules/@oxfmt/binding-openharmony-arm64": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-openharmony-arm64/-/binding-openharmony-arm64-0.61.0.tgz",
"integrity": "sha512-keGLkzeOvkMpNmPp4hffXWpfoSsY6e1K8++KXD4mSSfxdvM8q9QUDsYY689TB1k6Co832DZn1MnaaVx6cIBMWQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-openharmony-arm64/-/binding-openharmony-arm64-0.62.0.tgz",
"integrity": "sha512-mSZuFHU2ar1KLUjXpI2QBQcJ1VsOB3mOCgQXuXCpKs19dgh4u+OaovNfrWDfiJb+ihJ2+f7YFcaO9bS2dlTCXA==",
"cpu": [
"arm64"
],
@@ -606,9 +606,9 @@
}
},
"node_modules/@oxfmt/binding-win32-arm64-msvc": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-0.61.0.tgz",
"integrity": "sha512-VzsAISkFxmNhJ5LBDEL9VuH6tJsVJMtqYit2LyIUf/HLnsCe4Pg9SMOjjVQzGWt0bnpyfJ94CrqTqcpNZzK+ug==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-0.62.0.tgz",
"integrity": "sha512-OfwuhkcjDlqC4EgDojtiV9mzpLqeB9KqTOWPOjLEYBVdDCVSxqW3qzp/xcIxsbtI0UgGCnKvAqYKyY25kf5JZw==",
"cpu": [
"arm64"
],
@@ -623,9 +623,9 @@
}
},
"node_modules/@oxfmt/binding-win32-ia32-msvc": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-0.61.0.tgz",
"integrity": "sha512-xv4t7yzwJoYaLB6Zv28B3W+j7brEjsyv50rLTAQgmxJzddce9fAMCxed8dSAkbWES0zz2J29nYK5FaTuD2YBHg==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-0.62.0.tgz",
"integrity": "sha512-P9uDDNFRzghO3X8QAzhkjKhK7JvtABsVn8UYtFX7uor12IAnwNt8nNIctvfWj1JkQU/kE+fmLRPiw7XlrIHsZw==",
"cpu": [
"ia32"
],
@@ -640,9 +640,9 @@
}
},
"node_modules/@oxfmt/binding-win32-x64-msvc": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-x64-msvc/-/binding-win32-x64-msvc-0.61.0.tgz",
"integrity": "sha512-6EZXFkqOwxdDYjIn3TSNnPk3ST5E5GiYd4FiM6UF/mCL/LZSfr6D6UygTfW3R1PCQP2quCKpCEGRlij8E3VYbg==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/@oxfmt/binding-win32-x64-msvc/-/binding-win32-x64-msvc-0.62.0.tgz",
"integrity": "sha512-dlI5SY7XYQCiCBafntWagCR6HcAJB/NpsLtdlPx8x08+Osz8Ok1HHz1GZuusegCe/VoJ6pAnF5a4pd5OZAq7qQ==",
"cpu": [
"x64"
],
@@ -1070,17 +1070,17 @@
}
},
"node_modules/@typescript-eslint/eslint-plugin": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.65.0.tgz",
"integrity": "sha512-IEgob78X12rHpUmtcwFsXhZdVGJtwTVP8FiCLZkR6GlYVrl2PcuB+KhCE5BlVC/eQpQnu8WXRtkHZuPar+gCRA==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.66.0.tgz",
"integrity": "sha512-p088eaGrzYz1s+7cov0aMOCkNGTJlVxF4jgubf28c8L0Cv9Rloj8YBHnv4hXLq6IIEE1AsjNWavO+k+8kP2Y0A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/regexpp": "^4.12.2",
"@typescript-eslint/scope-manager": "8.65.0",
"@typescript-eslint/type-utils": "8.65.0",
"@typescript-eslint/utils": "8.65.0",
"@typescript-eslint/visitor-keys": "8.65.0",
"@typescript-eslint/scope-manager": "8.66.0",
"@typescript-eslint/type-utils": "8.66.0",
"@typescript-eslint/utils": "8.66.0",
"@typescript-eslint/visitor-keys": "8.66.0",
"ignore": "^7.0.5",
"natural-compare": "^1.4.0",
"ts-api-utils": "^2.5.0"
@@ -1093,22 +1093,22 @@
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"@typescript-eslint/parser": "^8.65.0",
"@typescript-eslint/parser": "^8.66.0",
"eslint": "^8.57.0 || ^9.0.0 || ^10.0.0",
"typescript": ">=4.8.4 <6.1.0"
}
},
"node_modules/@typescript-eslint/parser": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.65.0.tgz",
"integrity": "sha512-CZ4nMxWwgu1HEEFNkeaCptra9QCtkmKdgf3sWh1rl1trIhmxLilgTV4cwcbQ4wemnT4sWQN8CaKOmdYx+g2gMA==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.66.0.tgz",
"integrity": "sha512-X6ypGChaWYk6PBtUg2BwuTZEFFcHJAtGTVJ9/lCTOufhZ4i9fNolQNnktq+kkMCwMj7V8Svsq7+TxSDslmhE0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/scope-manager": "8.65.0",
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/typescript-estree": "8.65.0",
"@typescript-eslint/visitor-keys": "8.65.0",
"@typescript-eslint/scope-manager": "8.66.0",
"@typescript-eslint/types": "8.66.0",
"@typescript-eslint/typescript-estree": "8.66.0",
"@typescript-eslint/visitor-keys": "8.66.0",
"debug": "^4.4.3"
},
"engines": {
@@ -1124,14 +1124,14 @@
}
},
"node_modules/@typescript-eslint/project-service": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.65.0.tgz",
"integrity": "sha512-SxnPhbTsGahizDgbu7oqFH/xVtzIqMd/s+WtnSxNxJZJpLbdT5IPdzg8EZxO3+PoKahXmwJLeNQOpKJb3/bi7Q==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.66.0.tgz",
"integrity": "sha512-7MthGPTt4BP69lSryqpqq8HQqxuzynssckL/jyDyk3+TNMQ3y2jFWkptCrktWvBrP+EH787Nl5N5Qpw7WZg+5g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/tsconfig-utils": "^8.65.0",
"@typescript-eslint/types": "^8.65.0",
"@typescript-eslint/tsconfig-utils": "^8.66.0",
"@typescript-eslint/types": "^8.66.0",
"debug": "^4.4.3"
},
"engines": {
@@ -1146,14 +1146,14 @@
}
},
"node_modules/@typescript-eslint/scope-manager": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.65.0.tgz",
"integrity": "sha512-Esbl8OSYiVxBokYgWPf7VVWg/BE798wXhimnn9ML9Pt5qoDf8bfQlgjlKXR/k98+AcNzlLKYrpCcrcuZ9DZLgg==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.66.0.tgz",
"integrity": "sha512-8TGcH25j9zqJ/IULB/ppyhRvxA8QYfFEZ7nfbg6/BN9spDgb8fPWQXlE5l8TWBL50EtUx007uZ1o9VOwrq2/9g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/visitor-keys": "8.65.0"
"@typescript-eslint/types": "8.66.0",
"@typescript-eslint/visitor-keys": "8.66.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -1164,9 +1164,9 @@
}
},
"node_modules/@typescript-eslint/tsconfig-utils": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.65.0.tgz",
"integrity": "sha512-j6GzGqCiRdA7Qhur2VVmKZAkBLfnHFQfx4TaJGL9RMveZqCo48jSHHO0DTgizEnGhtWnqmbtCUSrqSkdiY/0Hg==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.66.0.tgz",
"integrity": "sha512-9D5gLYZG4rOjcoag8MQ/fWI8WqA9wcPDyOGyWtWFhvM1lHRbliqUSPIY5J3zqCU1tvSwzXxnnjhQhz5Ne7mJ4g==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1181,15 +1181,15 @@
}
},
"node_modules/@typescript-eslint/type-utils": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.65.0.tgz",
"integrity": "sha512-YjaZ7PRI5qY7ax2L3PbvX0rRyGtipAReCWs0mhhDBHjH/vl0g0BonaGXrKdKpMbIIsMIwDgbk/xzkBTyAltS5g==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.66.0.tgz",
"integrity": "sha512-LG2dWfjZQQp0ADtAu/EWJVayefGL2UEZ3CDeI44D9v3rXB/WYUqE/jpO28KrEKul5AySrmI+Zh1v6v+xW2U9+g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/typescript-estree": "8.65.0",
"@typescript-eslint/utils": "8.65.0",
"@typescript-eslint/types": "8.66.0",
"@typescript-eslint/typescript-estree": "8.66.0",
"@typescript-eslint/utils": "8.66.0",
"debug": "^4.4.3",
"ts-api-utils": "^2.5.0"
},
@@ -1206,9 +1206,9 @@
}
},
"node_modules/@typescript-eslint/types": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.65.0.tgz",
"integrity": "sha512-JSSwWNy+H0E/01jJEM+hrX6N0OFDzFzeIhHFSAS01tlVaevpG8cFyYRPhS5yjGOvBUx3sqQHVMjCL1CAZZMxBg==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.66.0.tgz",
"integrity": "sha512-H6gcYaSDOyvL3AD/jHUtUFo2jqGgn/F6nuyuZSu0QTesxL+cP4dQoIMrODRofuJC09g64+WgZ6tE19Y1N2YIFQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1220,16 +1220,16 @@
}
},
"node_modules/@typescript-eslint/typescript-estree": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.65.0.tgz",
"integrity": "sha512-JboAE2swaYt4tb1fHhHTABE2K+OLy09XfcTbhnk4Pw96f9dd2e9iYsJ28gBggHlo5z5x1rkyWvcPoTuNTd4oGg==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.66.0.tgz",
"integrity": "sha512-8/x4INiiQb10jGgXYD7116/zQ+OL84ZIFn0za68wwFHCanT/VLbBEroWht8RV8fn0/ZCAoazHLQgwUC0UQcDfg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/project-service": "8.65.0",
"@typescript-eslint/tsconfig-utils": "8.65.0",
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/visitor-keys": "8.65.0",
"@typescript-eslint/project-service": "8.66.0",
"@typescript-eslint/tsconfig-utils": "8.66.0",
"@typescript-eslint/types": "8.66.0",
"@typescript-eslint/visitor-keys": "8.66.0",
"debug": "^4.4.3",
"minimatch": "^10.2.2",
"semver": "^7.7.3",
@@ -1248,16 +1248,16 @@
}
},
"node_modules/@typescript-eslint/utils": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.65.0.tgz",
"integrity": "sha512-gXiwIHsYreboxeJucHKPvgwl7dXt50mF8s1/c00cP/WoVTyWKFdtfhRWwZiXYFU5H2O8vVoSLNrexFZjYS/SGA==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.66.0.tgz",
"integrity": "sha512-jasearZPolBw5NJNYGMwxzHMF83niVWmMU1VdHzG1CyfI2VS7f7nZltnKtHcg20hW+7Uo5GfK4MeDPoU3qI8EA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/eslint-utils": "^4.9.1",
"@typescript-eslint/scope-manager": "8.65.0",
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/typescript-estree": "8.65.0"
"@typescript-eslint/scope-manager": "8.66.0",
"@typescript-eslint/types": "8.66.0",
"@typescript-eslint/typescript-estree": "8.66.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -1272,13 +1272,13 @@
}
},
"node_modules/@typescript-eslint/visitor-keys": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.65.0.tgz",
"integrity": "sha512-8C71BQkGjiMmXtop7pHVJu1l2NNShFdkCyD6a2ezzs5vU/L3LRtb69EtcteFwz0mYMPzIgOw0n6OV4VBUWZd7A==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.66.0.tgz",
"integrity": "sha512-dkKR8q+lKciskj1Y3vthHktl+3cMLWGyVUP23bRiPZ5O9BRT++4EqDDV+TVeIKBL1VXVEqrJlz8MYbcnvJcAlg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.65.0",
"@typescript-eslint/types": "8.66.0",
"eslint-visitor-keys": "^5.0.0"
},
"engines": {
@@ -2709,9 +2709,9 @@
}
},
"node_modules/oxfmt": {
"version": "0.61.0",
"resolved": "https://registry.npmjs.org/oxfmt/-/oxfmt-0.61.0.tgz",
"integrity": "sha512-DxdHBEMYpcEnHoUHjjOigUqV2TYKsvxLwUPXnVYBjgFdqrcQ/91OtwubtZ2PUodCs3sStI8R5Qw3fKNGK4e8wQ==",
"version": "0.62.0",
"resolved": "https://registry.npmjs.org/oxfmt/-/oxfmt-0.62.0.tgz",
"integrity": "sha512-vxgGHTmnDU9j4CX7dDBLzxgmHxfda/yPcgJkGCMUSCwRmz+euo/V08xXLNgXTeqAB9Fhf3Pe2nO1RNKLCVgphQ==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -2727,25 +2727,25 @@
"url": "https://github.com/sponsors/Boshen"
},
"optionalDependencies": {
"@oxfmt/binding-android-arm-eabi": "0.61.0",
"@oxfmt/binding-android-arm64": "0.61.0",
"@oxfmt/binding-darwin-arm64": "0.61.0",
"@oxfmt/binding-darwin-x64": "0.61.0",
"@oxfmt/binding-freebsd-x64": "0.61.0",
"@oxfmt/binding-linux-arm-gnueabihf": "0.61.0",
"@oxfmt/binding-linux-arm-musleabihf": "0.61.0",
"@oxfmt/binding-linux-arm64-gnu": "0.61.0",
"@oxfmt/binding-linux-arm64-musl": "0.61.0",
"@oxfmt/binding-linux-ppc64-gnu": "0.61.0",
"@oxfmt/binding-linux-riscv64-gnu": "0.61.0",
"@oxfmt/binding-linux-riscv64-musl": "0.61.0",
"@oxfmt/binding-linux-s390x-gnu": "0.61.0",
"@oxfmt/binding-linux-x64-gnu": "0.61.0",
"@oxfmt/binding-linux-x64-musl": "0.61.0",
"@oxfmt/binding-openharmony-arm64": "0.61.0",
"@oxfmt/binding-win32-arm64-msvc": "0.61.0",
"@oxfmt/binding-win32-ia32-msvc": "0.61.0",
"@oxfmt/binding-win32-x64-msvc": "0.61.0"
"@oxfmt/binding-android-arm-eabi": "0.62.0",
"@oxfmt/binding-android-arm64": "0.62.0",
"@oxfmt/binding-darwin-arm64": "0.62.0",
"@oxfmt/binding-darwin-x64": "0.62.0",
"@oxfmt/binding-freebsd-x64": "0.62.0",
"@oxfmt/binding-linux-arm-gnueabihf": "0.62.0",
"@oxfmt/binding-linux-arm-musleabihf": "0.62.0",
"@oxfmt/binding-linux-arm64-gnu": "0.62.0",
"@oxfmt/binding-linux-arm64-musl": "0.62.0",
"@oxfmt/binding-linux-ppc64-gnu": "0.62.0",
"@oxfmt/binding-linux-riscv64-gnu": "0.62.0",
"@oxfmt/binding-linux-riscv64-musl": "0.62.0",
"@oxfmt/binding-linux-s390x-gnu": "0.62.0",
"@oxfmt/binding-linux-x64-gnu": "0.62.0",
"@oxfmt/binding-linux-x64-musl": "0.62.0",
"@oxfmt/binding-openharmony-arm64": "0.62.0",
"@oxfmt/binding-win32-arm64-msvc": "0.62.0",
"@oxfmt/binding-win32-ia32-msvc": "0.62.0",
"@oxfmt/binding-win32-x64-msvc": "0.62.0"
},
"peerDependencies": {
"svelte": "^5.0.0",
@@ -3211,16 +3211,16 @@
}
},
"node_modules/typescript-eslint": {
"version": "8.65.0",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.65.0.tgz",
"integrity": "sha512-/ggrHAwyjENDusvyxbuqxAC2dTnZg/Z8F+fgQtYIz+L6n/9HfSlEZcFGV/NsMNa6CkGk0xUjUAFwC0vHOflvIA==",
"version": "8.66.0",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.66.0.tgz",
"integrity": "sha512-QlEbBPz/RuJ1XUHj29nm3t0F/O/cSlEnntozqPOYHnnTGAXFamnMBu5i9Vn6vhUPHGAjR+Vl+5J8vPN/BMUrJw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/eslint-plugin": "8.65.0",
"@typescript-eslint/parser": "8.65.0",
"@typescript-eslint/typescript-estree": "8.65.0",
"@typescript-eslint/utils": "8.65.0"
"@typescript-eslint/eslint-plugin": "8.66.0",
"@typescript-eslint/parser": "8.66.0",
"@typescript-eslint/typescript-estree": "8.66.0",
"@typescript-eslint/utils": "8.66.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -3520,9 +3520,9 @@
}
},
"node_modules/ws": {
"version": "8.21.1",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.1.tgz",
"integrity": "sha512-+0NTnW77fFN/DjQi6k/Sq/Yvk4Sgajw7urW8V+asjXnRgDs9gyGkdb7EzgfhA4goXsRIZKE28fzIXBHEzhuiWw==",
"version": "8.21.2",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.2.tgz",
"integrity": "sha512-54dMVAo4WIe6SKy3vBgN+9bJZqqQ8IMRevAkOLQALhi49qkkQDQfWdAZ8KQlXiEabw88ARXXdUrlvtbKQX+aKw==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
+4 -4
View File
@@ -23,7 +23,7 @@
"jsonwebtoken": "^9.0.3",
"lodash-es": "^4.18.1",
"winston": "^3.19.0",
"ws": "^8.21.1"
"ws": "^8.21.2"
},
"devDependencies": {
"@eslint/js": "^9.25.1",
@@ -33,14 +33,14 @@
"@types/node": "^26.1.2",
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.65.0",
"@typescript-eslint/parser": "^8.65.0",
"@typescript-eslint/parser": "^8.66.0",
"eslint": "^10.8.0",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.9.0",
"oxfmt": "^0.61.0",
"oxfmt": "^0.62.0",
"tscw-config": "^1.1.2",
"typescript": "^6.0.3",
"typescript-eslint": "^8.65.0",
"typescript-eslint": "^8.66.0",
"vitest": "^4.1.10"
},
"engines": {
+9 -4
View File
@@ -19,7 +19,7 @@
import logging
import uuid as uuid_module
from typing import Any, Optional, Callable
from collections.abc import Iterator
from collections.abc import Collection, Iterator
import yaml
@@ -62,8 +62,13 @@ def get_default_position(title: str) -> dict[str, Any]:
}
def append_charts(position: dict[str, Any], charts: set[Slice]) -> dict[str, Any]:
chart_hashes = [f"CHART-{str(chart.uuid)}" for chart in charts]
def append_charts(
position: dict[str, Any], charts: Collection[Slice]
) -> dict[str, Any]:
# Materialize the collection into a list once so chart_hashes and the zip
# below iterate the exact same ordering and can never desynchronize.
chart_list = list(charts)
chart_hashes = [f"CHART-{str(chart.uuid)}" for chart in chart_list]
# if we have ROOT_ID/GRID_ID, append orphan charts to a new row inside the grid
row_hash = None
@@ -78,7 +83,7 @@ def append_charts(position: dict[str, Any], charts: set[Slice]) -> dict[str, Any
"parents": ["ROOT_ID", "GRID_ID"],
}
for chart_hash, chart in zip(chart_hashes, charts, strict=False):
for chart_hash, chart in zip(chart_hashes, chart_list, strict=False):
position[chart_hash] = {
"children": [],
"id": chart_hash,
@@ -52,6 +52,12 @@ import sqlalchemy as sa
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from superset.commands.deletion_retention.purge_policy import (
get_purge_policy,
PurgeBlockedError,
PurgeEntityPolicy,
)
logger: logging.Logger = logging.getLogger(__name__)
@@ -141,10 +147,6 @@ class CascadeResult:
blocked_reason: str | None = None
class PurgeBlockedError(Exception):
"""Raised when ordinary deletion policy forbids purging an entity."""
class PurgeRaceLostError(Exception):
"""Raised to roll back dependent cleanup when the entity delete loses."""
@@ -196,10 +198,6 @@ def cascade_hard_delete(
window) requires only that it is still soft-deleted, so a restore
committed after the caller resolved the entity cannot be destroyed.
"""
# pylint: disable=import-outside-toplevel
from superset.connectors.sqla.models import SqlaTable
from superset.models.slice import Slice
if enforce_window and cutoff is None:
raise ValueError("cutoff is required when enforce_window=True")
@@ -207,12 +205,13 @@ def cascade_hard_delete(
table = model.__table__
entity_id = entity.id
uuid = entity_uuid(entity)
entity_type = _USER_FACING_TYPE.get(table.name, table.name)
policy: PurgeEntityPolicy = get_purge_policy(model)
entity_type: str = policy.entity_type
dangling_chart_uuids: list[str] = []
removed_dashboard_slices = 0
version_rows = 0
permission_name = _dataset_permission_name(entity) if model is SqlaTable else None
permission_name: str | None = None
try:
with session.begin_nested():
@@ -230,22 +229,19 @@ def cascade_hard_delete(
if session.execute(claim.with_for_update()).scalar_one_or_none() is None:
raise PurgeRaceLostError
_validate_deletion_allowed(session, model, entity_id)
removed_dashboard_slices = _count_dashboard_slices(
session, model, entity_id
policy.validate(session, policy, entity_id)
# Captured under the lock: the row is claimed, so the identity
# the permission name is built from can no longer change.
permission_name = policy.capture_permission_name(session, policy, entity_id)
removed_dashboard_slices = policy.count_dashboard_slices(
session, policy, entity_id
)
dangling_chart_uuids = policy.collect_dangling_chart_uuids(
session, policy, entity_id
)
if model is SqlaTable:
dangling_chart_uuids = [
str(chart_uuid)
for (chart_uuid,) in session.execute(
sa.select(Slice.uuid)
.where(Slice.datasource_id == entity_id)
.where(Slice.datasource_type == "table")
)
]
_delete_m2m_joins(session, model, entity_id)
_delete_owned_children(session, model, entity_id)
policy.delete_associations(session, policy, entity_id)
policy.delete_owned_children(session, policy, entity_id)
version_rows = _delete_version_history(session, entity, entity_id)
delete_entity = sa.delete(table).where(*identity).where(*eligibility)
@@ -253,7 +249,7 @@ def cascade_hard_delete(
raise PurgeRaceLostError
if permission_name is not None:
_cleanup_dataset_permission(session, permission_name, entity_id)
policy.cleanup_permission(session, policy, permission_name, entity_id)
except PurgeRaceLostError:
logger.info(
"deletion_retention: %s id=%s not purged (restored or already gone)",
@@ -306,207 +302,10 @@ def cascade_hard_delete(
)
_USER_FACING_TYPE: dict[str, str] = {
"slices": "chart",
"dashboards": "dashboard",
"tables": "dataset",
}
def _validate_deletion_allowed(
session: Session, model: type[Any], entity_id: int
) -> None:
"""Apply the dependency guards used by ordinary delete commands."""
# pylint: disable=import-outside-toplevel
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
from superset.models.user_attributes import UserAttribute
from superset.reports.models import ReportSchedule
column: Any | None = None
if model is Slice:
column = ReportSchedule.chart_id
elif model is Dashboard:
column = ReportSchedule.dashboard_id
if (
column is not None
and session.execute(
sa.select(ReportSchedule.id).where(column == entity_id).limit(1)
).first()
):
raise PurgeBlockedError("associated alerts or reports exist")
# The welcome-dashboard reference must be an explicit guard, not a hope
# that the database enforces it: user_attributes.welcome_dashboard_id has
# no ondelete, so on FK-enforcing backends the delete fails with an
# IntegrityError misreported as a policy block -- while on SQLite with
# FKs off the dashboard purges "successfully", strands a dangling pointer
# (a broken user homepage), and the audit row says confirmed. One check,
# both dialect families, and a reason the blocked entity can be named by.
if (
model is Dashboard
and session.execute(
sa.select(UserAttribute.id)
.where(UserAttribute.welcome_dashboard_id == entity_id)
.limit(1)
).first()
):
raise PurgeBlockedError("a user has this dashboard set as their welcome page")
def _count_dashboard_slices(session: Session, model: type[Any], entity_id: int) -> int:
"""Snapshot relationship counts before DB cascades can remove rows."""
# pylint: disable=import-outside-toplevel
from superset.models.dashboard import Dashboard, dashboard_slices
from superset.models.slice import Slice
predicate: Any | None = None
if model is Dashboard:
predicate = dashboard_slices.c.dashboard_id == entity_id
elif model is Slice:
predicate = dashboard_slices.c.slice_id == entity_id
if predicate is None:
return 0
return int(
session.execute(
sa.select(sa.func.count()).select_from(dashboard_slices).where(predicate)
).scalar_one()
)
def dashboard_slice_count(session: Session, entity: Any) -> int:
"""Return the current dashboard relationship count for audit write-ahead."""
return _count_dashboard_slices(session, type(entity), entity.id)
def _delete_m2m_joins(session: Session, model: type[Any], entity_id: int) -> None:
"""Hard-delete every M:N join / association row the entity owns.
Relationship counts are captured before this function runs so database
cascades cannot make the reported values dialect-dependent.
"""
# pylint: disable=import-outside-toplevel
from superset.connectors.sqla.models import SqlaTable
from superset.models.dashboard import Dashboard, dashboard_slices
from superset.models.slice import Slice
from superset.subjects.models import (
chart_editors,
chart_viewers,
dashboard_editors,
dashboard_viewers,
sqlatable_editors,
)
from superset.tags.models import ObjectType, TaggedObject
if model is Dashboard:
session.execute(
sa.delete(dashboard_slices).where(
dashboard_slices.c.dashboard_id == entity_id
)
)
for association in (dashboard_editors, dashboard_viewers):
session.execute(
sa.delete(association).where(association.c.dashboard_id == entity_id)
)
_delete_tags(session, TaggedObject, ObjectType.dashboard, entity_id)
elif model is Slice:
# Every dashboard_slices row pointing at this chart, including those
# owned by live dashboards (the live dashboard survives, minus this
# chart from its layout).
session.execute(
sa.delete(dashboard_slices).where(dashboard_slices.c.slice_id == entity_id)
)
for association in (chart_editors, chart_viewers):
session.execute(
sa.delete(association).where(association.c.chart_id == entity_id)
)
_delete_tags(session, TaggedObject, ObjectType.chart, entity_id)
elif model is SqlaTable:
from superset.connectors.sqla.models import RLSFilterTables
session.execute(
sa.delete(sqlatable_editors).where(
sqlatable_editors.c.table_id == entity_id
)
)
session.execute(
sa.delete(RLSFilterTables).where(RLSFilterTables.c.table_id == entity_id)
)
_delete_tags(session, TaggedObject, ObjectType.dataset, entity_id)
def _delete_tags(
session: Session, tagged_object: type[Any], object_type: Any, entity_id: int
) -> None:
"""Remove ``tagged_object`` rows skipped by the Core bulk delete."""
session.execute(
sa.delete(tagged_object.__table__).where(
tagged_object.object_id == entity_id,
tagged_object.object_type == object_type,
)
)
def _delete_owned_children(session: Session, model: type[Any], entity_id: int) -> None:
"""Hard-delete the entity's owned children — rows with no independent
existence: a dataset's columns and metrics, a dashboard's embedded
configs. Charts have no such owned child tables today.
"""
# pylint: disable=import-outside-toplevel
from superset.connectors.sqla.models import SqlaTable, SqlMetric, TableColumn
from superset.models.dashboard import Dashboard
from superset.models.embedded_dashboard import EmbeddedDashboard
if model is SqlaTable:
session.execute(
sa.delete(TableColumn.__table__).where(
TableColumn.__table__.c.table_id == entity_id
)
)
session.execute(
sa.delete(SqlMetric.__table__).where(
SqlMetric.__table__.c.table_id == entity_id
)
)
elif model is Dashboard:
# Embedded configs (delete-orphan children carrying the public
# embed UUID and allowed_domains) — the ORM cascade does not fire
# for Core deletes and the DB cascade is a backstop only.
session.execute(
sa.delete(EmbeddedDashboard.__table__).where(
EmbeddedDashboard.__table__.c.dashboard_id == entity_id
)
)
def _dataset_permission_name(entity: Any) -> str:
"""Capture the permission identifier while dataset attributes are readable."""
# pylint: disable=import-outside-toplevel
from superset import security_manager
return str(
security_manager.get_dataset_perm(
entity.id, entity.table_name, entity.database.database_name
)
)
def _cleanup_dataset_permission(
session: Session, permission_name: str, entity_id: int
) -> None:
"""Replicate ``SqlaTable.after_delete`` permission cleanup.
Core ``sa.delete`` does not fire the ORM ``after_delete`` listener that
normally removes the dataset's ``datasource access`` view-menu /
permission-view, so it is done explicitly here or the PVM is orphaned.
"""
# pylint: disable=import-outside-toplevel
from superset import security_manager
security_manager._delete_pvm_on_sqla_event( # pylint: disable=protected-access
None, session.connection(), "datasource_access", permission_name
)
logger.debug("deletion_retention: removed dataset permission for id=%s", entity_id)
policy: PurgeEntityPolicy = get_purge_policy(type(entity))
return policy.count_dashboard_slices(session, policy, entity.id)
def _entity_version_targets(
@@ -520,24 +319,19 @@ def _entity_version_targets(
the dashboard/chart M2M shadow (``dashboard_slices_version``) and a
dataset's child shadows (``table_columns_version`` / ``sql_metrics_version``
keyed by ``table_id``). It never touches another entity's rows."""
# pylint: disable=import-outside-toplevel
from superset.connectors.sqla.models import SqlaTable
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
targets: list[tuple[sa.Table, Any]] = [
(parent_shadow, parent_shadow.c.id == entity_id)
]
m2m = metadata.tables.get("dashboard_slices_version")
if m2m is not None and model is Dashboard:
targets.append((m2m, m2m.c.dashboard_id == entity_id))
elif m2m is not None and model is Slice:
targets.append((m2m, m2m.c.slice_id == entity_id))
elif model is SqlaTable:
for child_name in ("table_columns_version", "sql_metrics_version"):
child = metadata.tables.get(child_name)
if child is not None and "table_id" in child.c:
targets.append((child, child.c.table_id == entity_id))
targets: list[tuple[sa.Table, Any]] = []
for table_name, column_name in get_purge_policy(model).version_shadow_names:
shadow: sa.Table | None = (
parent_shadow
if table_name == parent_shadow.name
else metadata.tables.get(table_name)
)
if shadow is None or column_name not in shadow.c:
raise RuntimeError(
f"Invalid version shadow declaration for {model.__name__}: "
f"{table_name}.{column_name}"
)
targets.append((shadow, shadow.c[column_name] == entity_id))
return targets
File diff suppressed because it is too large Load Diff
+356
View File
@@ -0,0 +1,356 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Synthesize a query context from a chart's saved form data (``params``).
A chart's ``query_context`` is normally generated client-side by each viz
plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
Explore. Charts that predate that behavior keep their ``params`` (form data) but
carry no ``query_context``, so server-side consumers that need to run the query
(e.g. the dashboard Excel export) have nothing to execute.
This module rebuilds a best-effort query context from the form data columns,
metrics, filters (including free-form SQL and the time range), ordering and time
grain mirroring the shared parts of the viz plugins' ``buildQuery``. Beyond
that shared core it reproduces exactly one piece of plugin post-processing, Pie's
unconditional ``contribution`` operator (see
:func:`_pie_contribution_post_processing`). It does **not** reproduce any other
post-processing (pivot, percent-metric transforms, rolling/forecast) or
multi-query fan-out, so callers must restrict it to viz types whose data maps
faithfully to a single query.
The mirrored logic lives on the frontend in
``superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts`` (query mode,
ordering), ``superset-frontend/packages/superset-ui-core/src/query/`` (field
extraction, ``processFilters``). There is no automated tripwire tying the two
across the language boundary; the per-helper pointers below must be kept in sync
when that frontend logic changes.
"""
from __future__ import annotations
from typing import Any
from superset.utils import json
from superset.utils.core import as_list, get_metric_name
# Suffix Pie's contribution operator appends when renaming the metric column,
# mirroring ``CONTRIBUTION_SUFFIX`` in
# ``superset-frontend/plugins/plugin-chart-echarts/src/Pie/constants.ts``.
PIE_CONTRIBUTION_SUFFIX = "__contribution"
def adhoc_filters_to_query_filters(
adhoc_filters: list[dict[str, Any]],
where_only: bool = False,
) -> list[dict[str, Any]]:
"""
Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
Adhoc filters use ``{subject, operator, comparator}`` while a query object
expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
val}`` equivalent and are handled separately (see
:func:`freeform_where_having`).
By default all ``SIMPLE`` filters are converted (the behavior the MCP
compile/preview path relies on). Pass ``where_only=True`` to convert only
``WHERE``-clause filters, matching the frontend's ``processFilters``
(``superset-ui-core/src/query/processFilters.ts``) the dashboard export uses
this so it applies the same rows the chart shows and does not additionally
filter on ``SIMPLE`` ``HAVING`` clauses.
"""
result: list[dict[str, Any]] = []
for flt in adhoc_filters or []:
if flt.get("expressionType") != "SIMPLE":
continue
if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
continue
result.append(
{
"col": flt.get("subject"),
"op": flt.get("operator"),
"val": flt.get("comparator"),
}
)
return result
def _sanitize_clause(clause: str) -> str:
"""
Parenthesize a free-form SQL clause, terminating a trailing line comment.
Mirrors ``sanitizeClause`` (``superset-ui-core/src/query/processFilters.ts``):
a clause containing ``--`` gets a newline appended *inside* the parentheses,
so a predicate ending in a comment (``sales > 0 -- note``) does not comment
out the closing paren and everything joined after it.
"""
if "--" in clause:
clause = f"{clause}\n"
return f"({clause})"
def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
"""
Collect free-form SQL predicates into a query ``extras`` mapping.
Mirrors ``processFilters`` on the frontend
(``superset-ui-core/src/query/processFilters.ts``): ``SQL`` adhoc filters (and
a legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
clause, so a chart restricted by a custom SQL predicate exports the same rows
it displays instead of the full, unrestricted result.
"""
where: list[str] = []
having: list[str] = []
if form_data.get("where"):
where.append(form_data["where"])
for flt in form_data.get("adhoc_filters") or []:
if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
clause = (flt.get("clause") or "WHERE").upper()
(having if clause == "HAVING" else where).append(flt["sqlExpression"])
extras: dict[str, str] = {}
if where:
extras["where"] = " AND ".join(_sanitize_clause(clause) for clause in where)
if having:
extras["having"] = " AND ".join(_sanitize_clause(clause) for clause in having)
return extras
def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
"""
Derive the query's grouping/raw columns from form data.
Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
order.
"""
if form_data.get("query_mode") == "raw" and (
form_data.get("all_columns") or form_data.get("columns")
):
return list(form_data.get("all_columns") or form_data.get("columns") or [])
groupby_columns: list[Any] = form_data.get("groupby") or []
raw_columns: list[Any] = form_data.get("columns") or []
# Prefer explicit raw columns only when they are actually present; a stale
# empty ``columns: []`` key must not shadow the group-by dimensions (which
# would silently drop the grouping and change the aggregation).
columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
x_axis = form_data.get("x_axis")
if isinstance(x_axis, str) and x_axis and x_axis not in columns:
columns.insert(0, x_axis)
elif isinstance(x_axis, dict):
col_name = x_axis.get("column_name")
if col_name and col_name not in columns:
columns.insert(0, col_name)
return columns
def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
"""
Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
``getQueryMode`` (``plugin-chart-table/src/buildQuery.ts``): an explicit
``query_mode`` wins, otherwise the presence of ``all_columns`` implies raw mode.
"""
if mode := form_data.get("query_mode"):
return mode == "raw"
return bool(form_data.get("all_columns"))
def orderby_from_form_data(
form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
) -> list[list[Any]]:
"""
Derive ordering so a ``row_limit`` returns the chart's top-N, not an
arbitrary N.
Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
pairs). Aggregate charts order by the configured sort metric
(``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
set), otherwise fall back to the first metric descending matching the
table/pie ``buildQuery`` defaults.
``order_by_cols`` is a raw-mode-only control (``resetOnHide: false`` in the
plugin control panels), so an aggregate chart can carry a stale value from a
previous raw-mode configuration. Aggregate mode must ignore it, mirroring the
frontend, where ``plugin-chart-table/src/buildQuery.ts:136-145`` overrides
``orderby`` with the sort metric (``order_by_cols`` reaches ``orderby`` only
via the alias in ``extractQueryFields.ts``, then gets overwritten in aggregate
mode).
"""
if is_raw_query_mode(form_data):
parsed: list[list[Any]] = []
for col in form_data.get("order_by_cols") or []:
if isinstance(col, str):
try:
col = json.loads(col)
except (TypeError, ValueError):
continue
# Anything that isn't a ``[column, ascending]`` pair (a stray null, a
# bare column, an over-long tuple) would append junk to ``orderby``
# and fail the query; drop it like an unparseable entry.
if isinstance(col, (list, tuple)) and len(col) == 2:
parsed.append(list(col))
return parsed
if not metrics:
return []
# The drag-and-drop "sort by" control persists a list; the frontend unwraps it
# with ``ensureIsArray(...)[0]`` (``plugin-chart-table/src/buildQuery.ts:67``).
# Read raw, a list would nest inside ``orderby`` and fail the query.
raw_sort_metric = form_data.get("timeseries_limit_metric")
sort_metric = (
next(iter(as_list(raw_sort_metric)), None) if raw_sort_metric else None
) or (metrics[0] if form_data.get("sort_by_metric") else None)
if sort_metric is not None:
# The Table plugin defaults ``order_desc`` to False (ascending); Pie and
# others sort by metric descending. Match that so a row limit keeps the
# chart's top/bottom-N rather than flipping it.
default_desc = viz_type != "table"
order_desc = form_data.get("order_desc", default_desc)
return [[sort_metric, not order_desc]]
# No explicit sort metric: default to the first metric, descending.
return [[metrics[0], False]]
def _columns_and_metrics(
form_data: dict[str, Any], viz_type: str | None
) -> tuple[list[Any], list[Any]]:
"""
Resolve the query's ``(columns, metrics)`` from form data, honoring raw vs.
aggregate mode and the Big Number trendline promotion.
"""
if is_raw_query_mode(form_data):
# Raw mode returns individual rows: use only the selected columns and
# ignore ``metrics``/``groupby``, which stay in form data as stale values
# (the controls aren't reset when hidden) but are ignored by the chart.
columns = list(form_data.get("all_columns") or form_data.get("columns") or [])
return columns, []
metrics = list(form_data.get("metrics") or [])
# Single-metric charts (e.g. Big Number) store ``metric`` rather than
# ``metrics``.
if not metrics and form_data.get("metric"):
metrics = [form_data["metric"]]
columns = columns_from_form_data(form_data)
# Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
# time column; ``big_number_total`` is a single aggregate and must not be
# grouped, or it would return one row per timestamp instead of a total.
if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
return [form_data["granularity_sqla"]], metrics
return columns, metrics
def _pie_contribution_post_processing(metrics: list[Any]) -> list[dict[str, Any]]:
"""
Pie's ``contribution`` post-processing operator, or ``[]`` when it can't apply.
``plugins/plugin-chart-echarts/src/Pie/buildQuery.ts`` attaches this operator
unconditionally it is not gated on ``percent_metrics`` or a contribution
mode and ``Pie/transformProps.ts`` reads the renamed column. Rebuilding a
pie without it drops the percentage column that a saved-context pie carries,
so two pies on one dashboard would export different columns based only on
whether they had been re-saved in Explore.
"""
if not metrics:
return []
try:
label = get_metric_name(metrics[0])
except ValueError:
# A metric this malformed will fail the query anyway; leave the operator
# off rather than turning a rebuild into an error before it runs.
return []
return [
{
"operation": "contribution",
"options": {
"columns": [label],
"rename_columns": [f"{label}{PIE_CONTRIBUTION_SUFFIX}"],
},
}
]
def build_query_context_from_form_data(
form_data: dict[str, Any],
datasource: dict[str, Any],
viz_type: str | None = None,
) -> dict[str, Any]:
"""
Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
loads) from a chart's form data and datasource reference.
:param form_data: The chart's saved ``params`` parsed to a dict.
:param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
:param viz_type: The chart's viz type, used for viz-specific handling.
:returns: A single-query query-context dict.
"""
columns, metrics = _columns_and_metrics(form_data, viz_type)
# SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
# free-form SQL predicates go into ``extras``. Only ``WHERE``-clause SIMPLE
# filters are applied (matching the chart), so the export never filters on a
# ``HAVING`` clause the chart itself ignores.
filters = adhoc_filters_to_query_filters(
form_data.get("adhoc_filters", []), where_only=True
)
for flt in form_data.get("filters") or []:
if isinstance(flt, dict) and flt.get("col") is not None:
filters.append(flt)
extras = freeform_where_having(form_data)
if form_data.get("time_grain_sqla"):
extras["time_grain_sqla"] = form_data["time_grain_sqla"]
# Prefer the modern ``time_range``; fall back to the legacy ``since``/``until``
# pair (older charts store the range that way) before defaulting to no filter.
time_range = form_data.get("time_range")
if not time_range and (form_data.get("since") or form_data.get("until")):
time_range = f"{form_data.get('since') or ''} : {form_data.get('until') or ''}"
time_range = time_range or "No filter"
query: dict[str, Any] = {
"columns": columns,
"metrics": metrics,
"orderby": orderby_from_form_data(form_data, metrics, viz_type),
"filters": filters,
"time_range": time_range,
}
if extras:
query["extras"] = extras
if viz_type == "pie" and (
post_processing := _pie_contribution_post_processing(metrics)
):
query["post_processing"] = post_processing
# ``granularity`` does two jobs downstream: it names the temporal column the
# time range filters on, and it is the column ``time_grain_sqla`` buckets
# (``models/helpers.py`` swaps a selected column for its timestamp expression
# when that column equals ``granularity``). Only the first job depends on
# there being an active range, so set it whenever form data carries one —
# matching ``extractExtras.ts``, which sets it unconditionally. Gating it on
# ``time_range`` dropped the bucketing, so an ordinary "all-time totals by
# month" chart exported one row per raw timestamp instead of one per month.
if granularity := form_data.get("granularity") or form_data.get("granularity_sqla"):
query["granularity"] = granularity
if form_data.get("row_limit"):
query["row_limit"] = form_data["row_limit"]
return {
"datasource": datasource,
"queries": [query],
"form_data": form_data,
}
+5 -2
View File
@@ -32,12 +32,15 @@ def left_join_df(
join_keys: list[str],
lsuffix: str = "",
rsuffix: str = "",
how: Literal["left", "right", "inner", "outer", "cross"] = "left",
how: Literal["left", "right", "inner", "outer"] = "left",
) -> pd.DataFrame:
# `how` defaults to "left" so callers that only want the left frame's rows are
# unaffected. Passing how="outer" keeps right-only rows, which is used by the
# time-comparison "full range" option so historical series are not truncated to
# the main series' time range.
# the main series' time range. "cross" is intentionally excluded: the join is
# implemented via `Index.join`, which doesn't support cross joins the way
# `pd.merge` does, so passing "cross" here would silently drop the join keys
# instead of producing a real cross join.
df = left_df.set_index(join_keys).join(
right_df.set_index(join_keys), how=how, lsuffix=lsuffix, rsuffix=rsuffix
)
+26 -2
View File
@@ -350,6 +350,9 @@ SQLALCHEMY_ENCRYPTED_FIELD_TYPE_ADAPTER = ( # pylint: disable=invalid-name
# (database passwords, SSH tunnel credentials, OAuth tokens, ...) will make
# those values undecryptable unless they are re-encrypted first. See the
# authenticated-encryption SIP/migration before switching an existing install.
# Leaving this at "aes" logs a startup warning
# (SupersetAppInitializer.check_encryption_engine) pointing at the
# `superset re-encrypt-secrets --engine aes-gcm` migration path.
SQLALCHEMY_ENCRYPTED_FIELD_ENGINE: Literal["aes", "aes-gcm"] = "aes"
# Extends the default SQLGlot dialects with additional dialects
@@ -485,8 +488,7 @@ FAB_API_SWAGGER_UI_SUPERSET_APP_ROOT = False
# AUTH_REMOTE_USER : Is for using REMOTE_USER from web server
AUTH_TYPE = AUTH_DB
# Uncomment to setup Full admin role name
# AUTH_ROLE_ADMIN = 'Admin'
# AUTH_ROLE_ADMIN = "Admin"
# Uncomment to setup Public role name, no authentication needed
# AUTH_ROLE_PUBLIC = 'Public'
@@ -1551,6 +1553,22 @@ EXCEL_EXPORT_S3_CLIENT_KWARGS: dict[str, Any] = {}
# a rendered image. Set to None to fall back to the built-in default.
EXCEL_EXPORT_TABLE_VIZ_TYPES: set[str] | None = None
# Optional hook to build a query context for a chart that has no saved
# ``query_context``, called before the built-in form-data rebuild. Receives the
# chart's form data (its ``params`` with ``viz_type`` and the
# ``datasource="{id}__{type}"`` string injected — i.e. ``Slice.form_data``) and
# returns a query-context payload dict (the shape ``ChartDataQueryContextSchema``
# loads) or ``None``. A deployment can point this at a service that runs the
# chart's real frontend ``buildQuery`` (faithful post-processing / multi-query)
# for viz types the built-in rebuild can't handle. Must return ``None`` — not a
# partial/stub context — whenever it cannot build the chart faithfully, so the
# export falls through to the built-in rebuild. The export deep-copies whatever
# it returns before applying dashboard filters, so a builder is free to memoize
# or share its payloads. Defaults to ``None`` (built-in behavior only).
EXCEL_EXPORT_QUERY_CONTEXT_BUILDER: (
Callable[[dict[str, Any]], dict[str, Any] | None] | None
) = None
# ---------------------------------------------------
# Time grain configurations
# ---------------------------------------------------
@@ -2532,6 +2550,12 @@ REPORT_MINIMUM_INTERVAL = int(timedelta(minutes=0).total_seconds())
# Enforce HTTPS for webhook alerts/reports
ALERT_REPORTS_WEBHOOK_HTTPS_ONLY = True
# Socket timeout (in seconds) for the HTTP request that dispatches webhook
# alerts/reports. Without a timeout the request blocks indefinitely if the
# webhook target is unreachable, which leaves the report schedule stuck in
# the WORKING state. Set to None to disable (not recommended).
ALERT_REPORTS_WEBHOOK_TIMEOUT = 60
# When True, webhook alert/report dispatch is permitted to call private/internal
# IP addresses (RFC-1918, loopback, link-local). Intended for deployments where
# the webhook target is on an internal network (a chatops bridge, an internal
+13 -1
View File
@@ -120,6 +120,11 @@ from superset.superset_typing import (
)
from superset.utils import core as utils, json
from superset.utils.backports import StrEnum
from superset.utils.sqlalchemy_events import (
DeleteListenerDeclaration,
DeleteListenerEffect,
register_delete_listener,
)
config = current_app.config # Backward compatibility for tests
metadata = Model.metadata # pylint: disable=no-member
@@ -2392,7 +2397,14 @@ class SqlaTable(
sa.event.listen(SqlaTable, "before_update", SqlaTable.before_update)
sa.event.listen(SqlaTable, "after_insert", SqlaTable.after_insert)
sa.event.listen(SqlaTable, "after_delete", SqlaTable.after_delete)
register_delete_listener(
DeleteListenerDeclaration(
SqlaTable,
"datasource_permission_cleanup",
DeleteListenerEffect.PERMISSION_ARTIFACT,
SqlaTable.after_delete,
)
)
RLSFilterSubjects = DBTable(
"rls_filter_subjects",
+15 -4
View File
@@ -670,10 +670,21 @@ class BaseDAO(CoreBaseDAO[T], Generic[T]):
f"found {len(pk_cols)} columns."
)
related_pk = pk_cols[0]
if operator_enum == ColumnOperatorEnum.eq:
return query.filter(column.any(related_pk == value))
if operator_enum == ColumnOperatorEnum.ne:
# "no related row has id == value"
if operator_enum in (ColumnOperatorEnum.eq, ColumnOperatorEnum.ne):
# `value` must be scalar for both eq and ne: a list/tuple would
# silently compile to `related_pk == [...]` (or `!= [...]`),
# which behaves unpredictably across backends instead of
# failing fast. Use `in`/`nin` to match multiple related ids.
if isinstance(value, (list, tuple)):
counterpart = "in" if operator_enum == ColumnOperatorEnum.eq else "nin"
raise ValueError(
f"Operator '{operator_enum.value}' on relationship "
f"column '{col_name}' requires a scalar value, got "
f"{type(value).__name__}. Use '{counterpart}' to match "
f"multiple related ids."
)
if operator_enum == ColumnOperatorEnum.eq:
return query.filter(column.any(related_pk == value))
return query.filter(~column.any(related_pk == value))
if operator_enum == ColumnOperatorEnum.in_:
values = value if isinstance(value, (list, tuple)) else [value]
+8 -8
View File
@@ -1326,12 +1326,12 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
return cursor.fetchmany(limit)
data = cursor.fetchall()
description = cursor.description or []
# Create a mapping between column name and a mutator function to normalize
# values with. The first two items in the description row are
# the column name and type.
# Create a mapping between column index and a mutator function to normalize
# values with. The first two items in the description row are the column
# name and type.
column_mutators = {
row[0]: func
for row in description
index: func
for index, row in enumerate(description)
if (
func := cls.column_type_mutators.get(
type(cls.get_sqla_column_type(cls.get_datatype(row[1])))
@@ -1339,11 +1339,11 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
)
}
if column_mutators:
indexes = {row[0]: idx for idx, row in enumerate(description)}
if not isinstance(data, list):
data = list(data)
for row_idx, row in enumerate(data):
new_row = list(row)
for col, func in column_mutators.items():
col_idx = indexes[col]
for col_idx, func in column_mutators.items():
new_row[col_idx] = func(row[col_idx])
data[row_idx] = tuple(new_row)
+56 -14
View File
@@ -245,6 +245,43 @@ class MySQLEngineSpec(BasicParametersMixin, BaseEngineSpec):
types.VARCHAR(),
GenericDataType.STRING,
),
# wire-protocol FIELD_TYPE names emitted by `get_datatype`, seen on
# SQL Lab and virtual dataset columns instead of DDL type names
(
re.compile(r"^newdecimal", re.IGNORECASE),
DECIMAL(),
GenericDataType.NUMERIC,
),
(
re.compile(r"^tiny$", re.IGNORECASE),
TINYINT(),
GenericDataType.NUMERIC,
),
(
re.compile(r"^short$", re.IGNORECASE),
types.SmallInteger(),
GenericDataType.NUMERIC,
),
(
re.compile(r"^(blob|text)$", re.IGNORECASE),
types.String(),
GenericDataType.STRING,
),
(
re.compile(r"^year$", re.IGNORECASE),
types.Integer(),
GenericDataType.NUMERIC,
),
(
re.compile(r"^enum\b", re.IGNORECASE),
types.String(),
GenericDataType.STRING,
),
(
re.compile(r"^set\b", re.IGNORECASE),
types.String(),
GenericDataType.STRING,
),
)
column_type_mutators: dict[types.TypeEngine, Callable[[Any], Any]] = {
DECIMAL: lambda val: Decimal(val) if isinstance(val, str) else val
@@ -403,22 +440,27 @@ class MySQLEngineSpec(BasicParametersMixin, BaseEngineSpec):
@classmethod
def get_datatype(cls, type_code: Any) -> Optional[str]:
if not cls.type_code_map:
# only import and store if needed at least once
# pylint: disable=import-outside-toplevel
try:
import MySQLdb
mysql_module = MySQLdb
except ImportError:
mysql_module = __import__("pymysql")
ft = mysql_module.constants.FIELD_TYPE
cls.type_code_map = {
getattr(ft, k): k for k in dir(ft) if not k.startswith("_")
}
datatype = type_code
if isinstance(type_code, int):
if not cls.type_code_map:
# only import and store if needed at least once
# pylint: disable=import-outside-toplevel
try:
import MySQLdb
ft = MySQLdb.constants.FIELD_TYPE
except ImportError:
try:
import pymysql # type: ignore[import-untyped]
ft = pymysql.constants.FIELD_TYPE
except ImportError:
from mysql.connector.constants import FieldType
ft = FieldType
cls.type_code_map = {
getattr(ft, k): k for k in dir(ft) if not k.startswith("_")
}
datatype = cls.type_code_map.get(type_code)
if datatype and isinstance(datatype, str) and datatype:
return datatype
+59
View File
@@ -1147,6 +1147,64 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
)
sys.exit(1)
def check_encryption_engine(self) -> None:
"""Warn when app-encrypted fields use the legacy AES-CBC engine.
``SQLALCHEMY_ENCRYPTED_FIELD_ENGINE`` defaults to ``"aes"`` for backward
compatibility: every secret an existing install has ever written through
this mechanism (database passwords, SSH tunnel credentials, OAuth2
tokens, and similar) is stored in that engine's ciphertext format, and
there is no per-value marker recording which engine produced it the
engine is a single, global setting shared by every encrypted column.
Unlike ``check_secret_key`` and its siblings, this never refuses to
start. ``"aes"`` is a working, still-supported configuration, not a
known-bad placeholder value: blocking startup on it would turn an
opt-in hardening step into a forced-migration outage for every
deployment that has not yet run the engine migration. It only warns,
on every boot, so operators have a documented path to the
authenticated ``"aes-gcm"`` engine (see ``superset re-encrypt-secrets``
and ``docs/sip/authenticated-encryption-at-rest.md``).
"""
# pylint: disable=import-outside-toplevel
from sqlalchemy_utils.types.encrypted.encrypted_type import AesEngine
from superset.utils.encrypt import (
DEFAULT_ENCRYPTION_ENGINE_NAME,
resolve_encryption_engine,
)
engine_name = self.config.get(
"SQLALCHEMY_ENCRYPTED_FIELD_ENGINE", DEFAULT_ENCRYPTION_ENGINE_NAME
)
try:
engine_cls = resolve_encryption_engine(engine_name)
except ValueError:
# An unrecognized value already fails closed at field construction
# (see ``resolve_encryption_engine``); nothing more to warn about.
return
if engine_cls is not AesEngine:
return
self._log_config_warning(
"SQLALCHEMY_ENCRYPTED_FIELD_ENGINE is set to the legacy 'aes' "
"engine (AES-CBC, unauthenticated). App-encrypted fields — "
"database passwords, SSH tunnel credentials, OAuth2 tokens, and "
"similar — would benefit from the authenticated 'aes-gcm' engine "
"instead.\n"
"Switching engines on a populated database requires "
"re-encrypting existing values first, since the two ciphertext "
"formats are not interchangeable:\n"
" 1. Back up the metadata database.\n"
" 2. superset re-encrypt-secrets --engine aes-gcm\n"
" 3. Set SQLALCHEMY_ENCRYPTED_FIELD_ENGINE = 'aes-gcm' in "
"superset_config.py.\n"
" 4. Restart Superset, then re-run the command above once more "
"to sweep up any values written during the cutover.\n"
"See UPDATING.md and "
"docs/sip/authenticated-encryption-at-rest.md for the full "
"runbook."
)
def configure_session(self) -> None:
if self.config["SESSION_SERVER_SIDE"]:
Session(self.superset_app)
@@ -1269,6 +1327,7 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
self.configure_feature_flags()
self.check_guest_token_secret()
self.check_async_query_secret()
self.check_encryption_engine()
self.configure_db_encrypt()
self.setup_db()
+7
View File
@@ -82,6 +82,13 @@ MCP_REQUIRED_SCOPES = ["superset:read"]
MCP_DEV_USERNAME = None
```
**Trusting Multiple Issuers**: `MCP_JWT_ISSUER` can be set to a list of
issuers instead of a single string. Because the default user resolver maps
token claims to Superset users by username/email without binding the
token's `iss` claim, trusting more than one issuer requires configuring a
custom, issuer-aware `MCP_USER_RESOLVER` (e.g. deriving a compound
`iss`+`sub` identity). Without one, the service refuses to start.
**JWT Issuer Setup Examples**:
**Auth0**:
+7
View File
@@ -74,6 +74,13 @@ MCP_JWT_ALGORITHM = "HS256"
MCP_JWT_SECRET = "your-shared-secret-key"
```
**Multiple Trusted Issuers**: `MCP_JWT_ISSUER` may also be set to a list of
issuers. The default user resolver maps token claims to Superset users by
username/email without binding the token's `iss` claim, so trusting more
than one issuer requires a custom, issuer-aware `MCP_USER_RESOLVER` (e.g.
one that derives a compound `iss`+`sub` identity). The service refuses to
start in this configuration until such a resolver is provided.
**JWT Token Structure**:
```json
+10 -17
View File
@@ -60,6 +60,7 @@ from superset.mcp_service.guest_token_verifier import GUEST_TOKEN_CLAIM
from superset.mcp_service.mcp_config import (
default_user_resolver,
get_mcp_api_key_enabled,
validate_multi_issuer_user_resolver,
)
from superset.mcp_service.utils.error_sanitization import (
sanitize_for_log as _sanitize_for_log,
@@ -526,6 +527,10 @@ def _resolve_user_from_jwt_context(app: Any) -> MCPUser | None: # noqa: C901
Raises:
ValueError: If JWT resolves a username that doesn't exist in the DB
(fail closed do NOT fall through to weaker auth sources).
MCPAuthConfigError: If more than one JWT issuer is trusted
(``MCP_JWT_ISSUER`` is a list/tuple/set) and no issuer-aware
``MCP_USER_RESOLVER`` is configured (fail closed see
``validate_multi_issuer_user_resolver``).
"""
try:
from fastmcp.server.dependencies import get_access_token
@@ -591,23 +596,11 @@ def _resolve_user_from_jwt_context(app: Any) -> MCPUser | None: # noqa: C901
# Single-issuer deployments (the common case) are safe — the issuer is
# already pinned by the verifier, so the username space is unambiguous and
# we keep the existing lookup key to avoid breaking them. For multi-issuer
# configs we warn: operators should provide an issuer-aware MCP_USER_RESOLVER
# that derives a compound (iss + sub) identity. This is the least-breaking
# correct option (warn, don't change the key out from under existing
# single-issuer deployments).
configured_issuer = app.config.get("MCP_JWT_ISSUER")
if isinstance(configured_issuer, (list, tuple, set)) and len(configured_issuer) > 1:
if not app.config.get("MCP_USER_RESOLVER"):
token_iss = claims.get("iss") if isinstance(claims, dict) else None
logger.warning(
"Multiple JWT issuers are trusted (MCP_JWT_ISSUER is a list) but "
"the default user resolver maps token claims to Superset users by "
"username/email without binding the issuer (iss=%s). Distinct "
"issuers minting the same username/email will collide. Configure an "
"issuer-aware MCP_USER_RESOLVER to derive a compound (iss+sub) "
"identity.",
_sanitize_for_log(token_iss),
)
# configs without an issuer-aware MCP_USER_RESOLVER, fail closed rather
# than resolving an identity that isn't actually scoped to the trusted
# issuer (mirrors the startup-time config checks in
# create_default_mcp_auth_factory / superset.initialization).
validate_multi_issuer_user_resolver(app)
# Use configurable resolver or default
+7 -11
View File
@@ -461,18 +461,14 @@ def adhoc_filters_to_query_filters(
Adhoc filters use ``{subject, operator, comparator}`` keys while
``QueryContextFactory`` expects ``{col, op, val}`` (QueryObjectFilterClause).
Delegates to the shared builder so the MCP and dashboard-export paths stay in
sync (single source of truth).
"""
result: list[Dict[str, Any]] = []
for f in adhoc_filters:
if f.get("expressionType") == "SIMPLE":
result.append(
{
"col": f.get("subject"),
"op": f.get("operator"),
"val": f.get("comparator"),
}
)
return result
from superset.common.form_data_query_context import (
adhoc_filters_to_query_filters as _shared,
)
return _shared(adhoc_filters)
def map_table_config(config: TableChartConfig) -> Dict[str, Any]:
+6 -18
View File
@@ -39,26 +39,14 @@ SUPPORTED_FORM_DATA_PREVIEW_FORMATS = frozenset({"ascii", "table", "vega_lite"})
def _build_query_columns(form_data: Dict[str, Any]) -> list[str]:
"""Build query columns list from form_data, including both x_axis and groupby."""
# Table charts in raw mode use all_columns or columns
all_columns = form_data.get("all_columns", [])
raw_columns_field = form_data.get("columns", [])
if form_data.get("query_mode") == "raw" and (all_columns or raw_columns_field):
return list(all_columns or raw_columns_field)
"""Build query columns list from form_data, including both x_axis and groupby.
x_axis_config = form_data.get("x_axis")
groupby_columns: list[str] = form_data.get("groupby") or []
raw_columns: list[str] = form_data.get("columns") or []
Delegates to the shared builder so the MCP and dashboard-export paths stay in
sync (single source of truth).
"""
from superset.common.form_data_query_context import columns_from_form_data
columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
if x_axis_config and isinstance(x_axis_config, str):
if x_axis_config not in columns:
columns.insert(0, x_axis_config)
elif x_axis_config and isinstance(x_axis_config, dict):
col_name = x_axis_config.get("column_name")
if col_name and col_name not in columns:
columns.insert(0, col_name)
return columns
return columns_from_form_data(form_data)
def generate_preview_from_form_data(
+49
View File
@@ -510,6 +510,8 @@ def create_default_mcp_auth_factory(app: Flask) -> Optional[Any]:
jwt_verifier: Any | None = None
if auth_enabled:
validate_multi_issuer_user_resolver(app)
jwks_uri = app.config.get("MCP_JWKS_URI")
public_key = app.config.get("MCP_JWT_PUBLIC_KEY")
secret = app.config.get("MCP_JWT_SECRET")
@@ -568,6 +570,53 @@ def _is_mcp_guest_auth_enabled(app: Flask) -> bool:
return True
def validate_multi_issuer_user_resolver(app: Flask) -> None:
"""Reject a multi-issuer JWT trust config that has no issuer-aware resolver.
``default_user_resolver`` maps token claims to Superset users by
username/email without binding the token's ``iss`` claim. When more than
one issuer is trusted (``MCP_JWT_ISSUER`` configured as a list/tuple/set),
that lookup is not issuer-scoped: distinct issuers minting the same
username or email claim would resolve to the identical Superset user.
Single-issuer deployments are unaffected the issuer is already pinned
by the verifier, so the username space is unambiguous.
Operators trusting more than one issuer must supply an ``MCP_USER_RESOLVER``
that derives its identity from the token's ``iss`` claim (e.g. a compound
iss+sub identity), not merely one that returns a username or email, before
the service will consider that configuration usable. This function can only
confirm that a resolver is configured -- it cannot verify an arbitrary
operator-supplied callable actually binds the issuer; enforcing that is the
operator's responsibility.
"""
configured_issuer = app.config.get("MCP_JWT_ISSUER")
if (
isinstance(configured_issuer, (list, tuple, set))
# str()-normalize before deduplicating: a plain set() would raise
# TypeError on unhashable entries (e.g. an accidental nested list),
# and that TypeError is not MCPAuthConfigError, so the caller's
# except MCPAuthConfigError / except Exception split would swallow
# it and fail OPEN (start unauthenticated) instead of fail closed.
and len({str(issuer) for issuer in configured_issuer}) > 1
and not app.config.get("MCP_USER_RESOLVER")
):
# MCPAuthConfigError specifically: callers re-raise this type to
# refuse startup / fail closed rather than silently proceeding with
# an identity lookup that is not scoped to the trusted issuer.
raise MCPAuthConfigError(
"MCP_JWT_ISSUER trusts multiple issuers but no MCP_USER_RESOLVER "
"is configured. The default user resolver maps token claims to "
"Superset users by username/email without binding the issuer, so "
"distinct trusted issuers minting the same username/email would "
"resolve to the same Superset user. This check only confirms a "
"resolver is configured, not that it binds the issuer -- the "
"configured MCP_USER_RESOLVER MUST derive its identity from the "
"token's iss claim (e.g. a compound iss+sub identity), not just "
"username/email, or the same collision risk persists under a "
"custom resolver that happens to be username/email-only too."
)
def _validate_guest_config(app: Flask) -> None:
"""Hard-fail on the default GUEST_TOKEN_JWT_SECRET; warn on an unset audience."""
if app.config.get("GUEST_TOKEN_JWT_SECRET") == CHANGE_ME_GUEST_TOKEN_JWT_SECRET:
+50 -10
View File
@@ -385,8 +385,31 @@ def sanitize_user_input(
f"Maximum allowed length is {max_length} characters."
)
# Strip all HTML tags using nh3
# Remove dangerous Unicode characters BEFORE any check so zero-widths
# smuggled inside a denylisted keyword can't slip past the pattern
# checks below (mirrors sanitize_sql_expression's ordering).
value = _remove_dangerous_unicode(value)
# Canonicalization above can reduce a truthy input (e.g. a lone
# zero-width character) to "" — recheck emptiness so that case still
# honors the documented non-empty / allow_empty contract instead of
# silently returning "".
if not value:
if allow_empty:
return None
raise ValueError(f"{field_name} cannot be empty")
# Strip all HTML tags using nh3. This decodes HTML entities internally
# (see _strip_html_tags), which can turn an entity-encoded zero-width
# character (e.g. "&#8203;") into the raw character -- re-run the
# Unicode strip below so an entity-encoded smuggling attempt is caught
# too, not just a raw one. Deliberately no emptiness recheck here: nh3
# legitimately reduces tag-heavy input (e.g. "<script>...</script>") to
# an empty string as the correct sanitized result, not an error case --
# unlike the recheck above, which guards the value *before* any content
# has been intentionally stripped away.
value = _strip_html_tags(value)
value = _remove_dangerous_unicode(value)
# Check for dangerous patterns (URL schemes, event handlers)
_check_dangerous_patterns(value, field_name)
@@ -395,9 +418,6 @@ def sanitize_user_input(
if check_sql_keywords:
_check_sql_patterns(value, field_name)
# Remove dangerous Unicode characters
value = _remove_dangerous_unicode(value)
return value
@@ -433,8 +453,18 @@ def sanitize_filter_value(
f"Maximum allowed length is {max_length} characters."
)
# Strip all HTML tags using nh3
# Remove dangerous Unicode characters BEFORE any check so zero-widths
# smuggled inside a denylisted pattern can't slip past the pattern
# checks below (mirrors sanitize_sql_expression's ordering).
value = _remove_dangerous_unicode(value)
# Strip all HTML tags using nh3. This decodes HTML entities internally
# (see _strip_html_tags), which can turn an entity-encoded zero-width
# character (e.g. "&#8203;") into the raw character -- re-run the
# Unicode strip below so an entity-encoded smuggling attempt is caught
# too, not just a raw one.
value = _strip_html_tags(value)
value = _remove_dangerous_unicode(value)
# Check for dangerous patterns
_check_dangerous_patterns(value, "Filter value")
@@ -466,9 +496,6 @@ def sanitize_filter_value(
if re.search(r"\\x[0-9a-fA-F]{2}", value):
raise ValueError("Filter value contains hex encoding which is not allowed.")
# Remove dangerous Unicode characters
value = _remove_dangerous_unicode(value)
return value
@@ -518,8 +545,11 @@ def sanitize_sql_expression( # noqa: C901
f"Maximum allowed length is {max_length} characters."
)
# Strip + decode entities BEFORE any check so zero-widths and entity
# encoding can't smuggle past the tag-pattern / keyword scans.
# Strip zero-widths, then decode entities, then strip again: decoding
# can turn an entity-encoded zero-width character (e.g. "&#8203;") into
# the raw character, so a single strip-then-decode order would let an
# entity-encoded smuggling attempt through the tag-pattern / keyword
# scans below.
value = _remove_dangerous_unicode(value)
prev: str | None = None
iterations = 0
@@ -527,6 +557,16 @@ def sanitize_sql_expression( # noqa: C901
prev = value
value = html.unescape(value)
iterations += 1
value = _remove_dangerous_unicode(value)
# Canonicalization above can reduce a truthy input (e.g. a lone
# zero-width character) to "" — recheck emptiness so that case still
# honors the documented non-empty / allow_empty contract instead of
# silently returning "".
if not value:
if allow_empty:
return None
raise ValueError(f"{field_name} cannot be empty")
if _HTML_TAG_LIKE_RE.search(value):
raise ValueError(
@@ -29,7 +29,6 @@ down_revision = "c617da68de7d"
from datetime import datetime # noqa: E402
from alembic import op # noqa: E402
from flask_appbuilder.models.mixins import AuditMixin # noqa: E402
from sqlalchemy import Column, DateTime, Enum, ForeignKey, Integer, String # noqa: E402
from sqlalchemy.orm import declarative_base, declared_attr # noqa: E402
@@ -39,10 +38,15 @@ from superset.utils.core import get_user_id # noqa: E402
Base = declarative_base()
class AuditMixinNullable(AuditMixin):
"""Altering the AuditMixin to use nullable fields
class AuditMixinNullable:
"""Nullable audit columns, without FAB's ``AuditMixin`` relationships.
Allows creating objects programmatically outside of CRUD
This module only needs the audit *columns* for ``__table__.create``.
Inheriting FAB's ``AuditMixin`` would also declare ``created_by`` /
``changed_by`` relationships on these throwaway mapped classes, and once
alembic imports this script the resulting mapper cannot be configured
breaking ``sqlalchemy.orm.configure_mappers()`` process-wide for any
later caller.
"""
created_on = Column(DateTime, default=datetime.now, nullable=True)
+1 -1
View File
@@ -2705,7 +2705,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
df: pd.DataFrame,
offset_df: pd.DataFrame,
actual_join_keys: list[str],
how: Literal["left", "right", "inner", "outer", "cross"] = "left",
how: Literal["left", "right", "inner", "outer"] = "left",
) -> pd.DataFrame:
"""Perform the appropriate join operation."""
if actual_join_keys:
+4 -2
View File
@@ -27,7 +27,7 @@ from flask_appbuilder.api import (
)
from flask_appbuilder.hooks import before_request
from flask_appbuilder.models.sqla.interface import SQLAInterface
from flask_babel import ngettext
from flask_babel import gettext, ngettext
from marshmallow import ValidationError
from superset import is_feature_enabled
@@ -773,7 +773,9 @@ class ReportScheduleRestApi(BaseSupersetModelRestApi):
**response_schema.dump(
{
"execution_id": execution_id,
"message": "Report schedule execution started successfully",
"message": gettext(
"Report schedule execution started successfully"
),
}
),
)
+10 -8
View File
@@ -153,12 +153,13 @@ class WebhookNotification(BaseNotification):
# start of each attempt and checks it against max_time only after that
# attempt fails -- so the giveup decision uses the time measured before
# the attempt ran, ignoring the attempt's own duration. With each
# request carrying timeout=60, a third attempt can begin past the 120s
# mark (its start gated by the prior check, which still saw ~60-70s) and
# then run its full 60s before the check trips. The loop therefore makes
# 3 attempts: total wall-clock is ~180-210s (180s of requests + up to
# ~30s of jitter sleeps: <=10s then <=20s), not 120s. factor is kept at
# 10 so legitimately-transient 5xx targets are not abandoned early.
# request carrying the ALERT_REPORTS_WEBHOOK_TIMEOUT (default 60s), a
# third attempt can begin past the 120s mark (its start gated by the
# prior check, which still saw ~60-70s) and then run its full request
# timeout before the check trips. The loop therefore makes 3 attempts:
# total wall-clock can exceed 120s by up to one request timeout plus
# jitter sleeps (<=10s then <=20s). factor is kept at 10 so
# legitimately-transient 5xx targets are not abandoned early.
max_time=120,
)
@statsd_gauge("reports.webhook.send")
@@ -172,6 +173,7 @@ class WebhookNotification(BaseNotification):
self._validate_webhook_url(wh_url)
payload = self._get_req_payload()
files = self._get_files()
timeout = current_app.config["ALERT_REPORTS_WEBHOOK_TIMEOUT"]
try:
if files:
@@ -186,12 +188,12 @@ class WebhookNotification(BaseNotification):
wh_url,
data=data,
files=files,
timeout=60,
timeout=timeout,
allow_redirects=False,
)
else:
response = requests.post(
wh_url, json=payload, timeout=60, allow_redirects=False
wh_url, json=payload, timeout=timeout, allow_redirects=False
)
logger.info(
+50 -6
View File
@@ -16,6 +16,42 @@
# under the License.
# pylint: disable=import-outside-toplevel
from superset.utils.sqlalchemy_events import (
DeleteListenerDeclaration,
DeleteListenerEffect,
register_delete_listener,
remove_delete_listener,
)
def _tag_delete_listener_declarations() -> tuple[DeleteListenerDeclaration, ...]:
"""Build tag cleanup declarations without introducing model import cycles."""
from superset.connectors.sqla.models import SqlaTable
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
from superset.tags.models import ChartUpdater, DashboardUpdater, DatasetUpdater
return (
DeleteListenerDeclaration(
SqlaTable,
"tagged_object_cleanup",
DeleteListenerEffect.PERSISTENT_RECORD,
DatasetUpdater.after_delete,
),
DeleteListenerDeclaration(
Slice,
"tagged_object_cleanup",
DeleteListenerEffect.PERSISTENT_RECORD,
ChartUpdater.after_delete,
),
DeleteListenerDeclaration(
Dashboard,
"tagged_object_cleanup",
DeleteListenerEffect.PERSISTENT_RECORD,
DashboardUpdater.after_delete,
),
)
def register_sqla_event_listeners() -> None:
import sqlalchemy as sqla
@@ -33,17 +69,21 @@ def register_sqla_event_listeners() -> None:
QueryUpdater,
)
declarations: tuple[DeleteListenerDeclaration, ...] = (
_tag_delete_listener_declarations()
)
sqla.event.listen(SqlaTable, "after_insert", DatasetUpdater.after_insert)
sqla.event.listen(SqlaTable, "after_update", DatasetUpdater.after_update)
sqla.event.listen(SqlaTable, "after_delete", DatasetUpdater.after_delete)
register_delete_listener(declarations[0])
sqla.event.listen(Slice, "after_insert", ChartUpdater.after_insert)
sqla.event.listen(Slice, "after_update", ChartUpdater.after_update)
sqla.event.listen(Slice, "after_delete", ChartUpdater.after_delete)
register_delete_listener(declarations[1])
sqla.event.listen(Dashboard, "after_insert", DashboardUpdater.after_insert)
sqla.event.listen(Dashboard, "after_update", DashboardUpdater.after_update)
sqla.event.listen(Dashboard, "after_delete", DashboardUpdater.after_delete)
register_delete_listener(declarations[2])
sqla.event.listen(FavStar, "after_insert", FavStarUpdater.after_insert)
sqla.event.listen(FavStar, "after_delete", FavStarUpdater.after_delete)
@@ -69,17 +109,21 @@ def clear_sqla_event_listeners() -> None:
QueryUpdater,
)
declarations: tuple[DeleteListenerDeclaration, ...] = (
_tag_delete_listener_declarations()
)
sqla.event.remove(SqlaTable, "after_insert", DatasetUpdater.after_insert)
sqla.event.remove(SqlaTable, "after_update", DatasetUpdater.after_update)
sqla.event.remove(SqlaTable, "after_delete", DatasetUpdater.after_delete)
remove_delete_listener(declarations[0])
sqla.event.remove(Slice, "after_insert", ChartUpdater.after_insert)
sqla.event.remove(Slice, "after_update", ChartUpdater.after_update)
sqla.event.remove(Slice, "after_delete", ChartUpdater.after_delete)
remove_delete_listener(declarations[1])
sqla.event.remove(Dashboard, "after_insert", DashboardUpdater.after_insert)
sqla.event.remove(Dashboard, "after_update", DashboardUpdater.after_update)
sqla.event.remove(Dashboard, "after_delete", DashboardUpdater.after_delete)
remove_delete_listener(declarations[2])
sqla.event.remove(FavStar, "after_insert", FavStarUpdater.after_insert)
sqla.event.remove(FavStar, "after_delete", FavStarUpdater.after_delete)
+171 -12
View File
@@ -29,6 +29,7 @@ filters) and embedded, while table-like charts stay tabular.
from __future__ import annotations
import copy
import logging
import os
import tempfile
@@ -47,6 +48,10 @@ from superset.charts.schemas import ChartDataQueryContextSchema
from superset.commands.chart.data.get_data_command import ChartDataCommand
from superset.commands.distributed_lock.release import ReleaseDistributedLock
from superset.common.chart_data import ChartDataResultFormat, ChartDataResultType
from superset.common.form_data_query_context import (
build_query_context_from_form_data,
is_raw_query_mode,
)
from superset.dashboards.excel_export import email
from superset.dashboards.excel_export.layout import get_charts_in_layout_order
from superset.dashboards.excel_export.screenshot import render_chart_image
@@ -67,6 +72,12 @@ EXPORT_MODE_IMAGES = "images"
# image. Operators can override the set via ``EXCEL_EXPORT_TABLE_VIZ_TYPES``.
TABLE_VIZ_TYPES = {"table", "pivot_table_v2", "pivot_table"}
# Viz types whose missing query context may be rebuilt from saved form data.
# Conservative: only charts whose data maps faithfully to a single plain query
# (no post-processing, no multi-query fan-out). Every other viz type without a
# saved query context is skipped and listed for the user to re-save in Explore.
REBUILD_VIZ_TYPES = {"table", "big_number_total", "big_number", "pie"}
EXPORT_SOFT_TIME_LIMIT = 600
EXPORT_HARD_TIME_LIMIT = 660
@@ -96,6 +107,139 @@ def _chart_label(chart: Any) -> str:
return f"{chart.id} - {chart.slice_name or ''}".strip()
def _usable_query_context(value: Any) -> dict[str, Any] | None:
"""
``value`` when it is a usable query-context payload, else ``None``.
A payload is usable only if it is a dict with a non-empty ``queries`` list; a
blank, query-less, mistyped, or non-object value (e.g. ``{}``,
``{"queries": []}``, ``{"queries": "oops"}``, ``None``) is treated the same as
a missing context. Shared by the saved-context path and the builder hook so
both apply the same validity rule and so a malformed builder return falls
through to the built-in rebuild instead of failing later in the general
error bucket.
"""
if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
return None
return value if value["queries"] else None
def _saved_query_context(raw: Any) -> dict[str, Any] | None:
"""
The chart's saved query context parsed to a dict, or ``None`` when it is
missing or unusable.
Returns ``None`` for a blank value, a string that does not parse as JSON, and
any value that is not a dict with a non-empty ``queries`` list.
"""
if not raw:
return None
try:
parsed = json.loads(raw)
except (TypeError, ValueError):
return None
return _usable_query_context(parsed)
# Form-data keys whose behavior needs plugin post-processing or extra queries
# (contribution/time comparison, rolling window, resampling, raw big-number
# aggregation) that the single-query rebuild cannot reproduce. A chart using any
# of these is skipped rather than exported with values that differ from the chart.
_UNSUPPORTED_PROCESSING_KEYS = ("time_compare", "rolling_type", "resample_rule")
def _needs_unsupported_processing(form_data: dict[str, Any]) -> bool:
"""Whether the form data relies on processing the rebuild can't reproduce."""
# ``percent_metrics`` are "% of total" columns produced by contribution
# post-processing the rebuild can't apply; skip so the export doesn't silently
# omit columns the user sees.
if form_data.get("percent_metrics"):
return True
# ``show_totals`` adds a totals row via a *second* query
# (``plugin-chart-table/src/buildQuery.ts``, gated on aggregate mode); the
# single-query rebuild would silently drop that row. The mode check mirrors
# the frontend so a raw-mode table carrying a stale value still exports.
if form_data.get("show_totals") and not is_raw_query_mode(form_data):
return True
for key in _UNSUPPORTED_PROCESSING_KEYS:
value = form_data.get(key)
# ``rolling_type`` is often the literal string ``"None"`` when unset.
if value and value != "None":
return True
return form_data.get("aggregation") == "raw"
def _resolve_query_context(chart: Any) -> dict[str, Any] | None:
"""
The query-context payload to run for a chart's data export, or ``None`` when
none can be obtained.
Resolution order:
1. the chart's saved ``query_context``;
2. an optional ``EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`` hook, letting a deployment
supply a faithful context (e.g. from a service running the chart's real
frontend ``buildQuery``) for viz types the built-in rebuild can't handle;
3. the built-in form-data rebuild, restricted to viz types whose data maps
faithfully to a single plain query (``REBUILD_VIZ_TYPES``) without
post-processing or extra queries.
Returns ``None`` when none apply, so the caller lists the chart for re-saving
rather than exporting inaccurate data.
"""
if saved := _saved_query_context(chart.query_context):
return saved
# The hook receives the chart's form data and must return ``None`` — not a
# partial/stub context — whenever it can't build the chart faithfully, so we
# fall through to the built-in rebuild (which handles the allowlisted viz types
# well). A hook failure falls through too, preserving "builder problem →
# rebuild, don't fail the export" — the one exception being a task-level
# timeout, which has to abort the whole export rather than this chart.
if builder := current_app.config.get("EXCEL_EXPORT_QUERY_CONTEXT_BUILDER"):
try:
built = builder(chart.form_data)
except SoftTimeLimitExceeded:
# A soft timeout is a task-level signal, not a builder failure: let it
# propagate to _build_workbook so the export aborts cleanly instead of
# continuing on to rebuild this chart and start the next one.
raise
except Exception: # pylint: disable=broad-except
logger.warning(
"EXCEL_EXPORT_QUERY_CONTEXT_BUILDER failed for chart %s; "
"falling back to the built-in rebuild",
chart.id,
exc_info=True,
)
built = None
if (from_builder := _usable_query_context(built)) is not None:
# Copy: the payload's ``queries`` are mutated in place downstream (by
# ``apply_dashboard_filter_context``), and a builder is free to
# memoize or otherwise share its return value — which would then
# accumulate filters across charts and across exports.
return copy.deepcopy(from_builder)
# The allowlist and ``_needs_unsupported_processing`` bound only the built-in
# rebuild below; the builder hook above is intentionally not gated by them (a
# faithful builder includes the post-processing the built-in rebuild lacks).
if chart.viz_type not in REBUILD_VIZ_TYPES or chart.datasource_id is None:
return None
try:
form_data = json.loads(chart.params) if chart.params else {}
except (TypeError, ValueError):
return None
if not isinstance(form_data, dict) or not form_data:
return None
if _needs_unsupported_processing(form_data):
return None
return build_query_context_from_form_data(
form_data,
{"id": chart.datasource_id, "type": chart.datasource_type or "table"},
chart.viz_type,
)
def _record_to_row(record: dict[str, Any], colnames: list[str]) -> list[Any]:
return [record.get(col) for col in colnames]
@@ -131,17 +275,25 @@ def _write_chart_image_sheet(
def _write_chart_sheets(
writer: StreamingXlsxWriter,
chart: Any,
json_body: dict[str, Any],
dashboard_id: int,
active_data_mask: dict[str, Any],
) -> None:
"""
Run a single chart's query and stream its result(s) into the workbook.
Charts may yield more than one query (e.g. mixed-series charts); each becomes
its own sheet. Raises if the chart cannot be exported, so the caller can skip
it and note it in the email.
``json_body`` is the resolved query-context payload (the chart's saved
context or one synthesized from its form data). Charts may yield more than
one query (e.g. mixed-series charts); each becomes its own sheet. Raises if
the chart cannot be exported, so the caller can skip it and note it in the
email.
"""
json_body = json.loads(chart.query_context)
# Shallow-copy before setting our own top-level keys so the caller's payload
# keeps its original result_format/result_type. (The nested ``queries`` are
# mutated in place by apply_dashboard_filter_context below, which is safe
# because every payload ``_resolve_query_context`` returns is this chart's
# alone: freshly parsed, freshly built, or deep-copied from the builder hook.)
json_body = dict(json_body)
# Override any stale saved values: we always want full JSON results.
json_body["result_format"] = ChartDataResultFormat.JSON
json_body["result_type"] = ChartDataResultType.FULL
@@ -196,19 +348,26 @@ def _build_workbook(
try:
for chart in get_charts_in_layout_order(dashboard):
label = _chart_label(chart)
as_image = _renders_as_image(chart, mode)
# Image charts render from their saved params and don't need a query
# context; data (and table) charts still do.
if not as_image and not chart.query_context:
errored.setdefault(email.ERROR_NO_QUERY_CONTEXT, []).append(label)
continue
try:
if as_image:
if _renders_as_image(chart, mode):
# Image charts render from their saved params via the
# webdriver and don't need a query context.
_write_chart_image_sheet(
writer, chart, dashboard.id, active_data_mask, user
)
else:
_write_chart_sheets(writer, chart, dashboard.id, active_data_mask)
# Data charts need a query context: use the saved one, or
# rebuild it from form data for eligible viz types. Skip
# cleanly when none is available rather than failing.
json_body = _resolve_query_context(chart)
if json_body is None:
errored.setdefault(email.ERROR_NO_QUERY_CONTEXT, []).append(
label
)
continue
_write_chart_sheets(
writer, chart, json_body, dashboard.id, active_data_mask
)
except SoftTimeLimitExceeded:
# A soft timeout is a task-level signal, not a per-chart failure:
# let it propagate so the outer handler emails a failure and runs
+26 -3
View File
@@ -52,6 +52,7 @@ def apply_rls(
method = database.db_engine_spec.get_rls_method()
# collect all RLS predicates for all tables in the query
default_catalog = database.get_default_catalog()
predicates: dict[Table, list[Any]] = {}
for table in parsed_statement.tables:
table = table.qualify(catalog=catalog, schema=schema)
@@ -60,7 +61,7 @@ def apply_rls(
for predicate in get_predicates_for_table(
table,
database,
database.get_default_catalog(),
default_catalog,
exclude_dataset_id=exclude_dataset_id,
)
if predicate
@@ -98,7 +99,6 @@ def get_predicates_for_table(
filters = [
SqlaTable.database_id == database.id,
catalog_predicate,
SqlaTable.schema == table.schema,
SqlaTable.table_name == table.table,
]
# When applying RLS to a virtual dataset's inner SQL, skip a match against
@@ -109,7 +109,30 @@ def get_predicates_for_table(
if exclude_dataset_id is not None:
filters.append(SqlaTable.id != exclude_dataset_id)
dataset = db.session.query(SqlaTable).filter(and_(*filters)).one_or_none()
dataset = (
db.session.query(SqlaTable)
.filter(and_(*filters, SqlaTable.schema == table.schema))
.one_or_none()
)
if not dataset and table.schema:
# A dataset stored without a schema is scoped to the database's default
# schema, so a query resolving to that same schema must still pick up its
# predicates. This mirrors the null-catalog fallback above.
#
# This is a second query rather than an ``OR`` on the first so that an exact
# schema match always wins and neither query can match more than one dataset.
# Resolving the default schema probes the analytic database, so it is deferred
# until a null-schema dataset is known to exist.
null_schema_dataset = (
db.session.query(SqlaTable)
.filter(and_(*filters, SqlaTable.schema.is_(None)))
.one_or_none()
)
if null_schema_dataset and table.schema == database.get_default_schema(
table.catalog
):
dataset = null_schema_dataset
if not dataset:
return []
+103
View File
@@ -0,0 +1,103 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Typed declarations for SQLAlchemy deletion-listener effects."""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from enum import Enum
from threading import RLock
from typing import Any
import sqlalchemy as sa
class DeleteListenerEffect(str, Enum):
"""Classify the durable effect of an ORM deletion listener."""
PERSISTENT_RECORD = "persistent_record"
PERMISSION_ARTIFACT = "permission_artifact"
OBSERVATIONAL = "observational"
@dataclass(frozen=True)
class DeleteListenerDeclaration:
"""Describe one supported-root ``after_delete`` listener."""
target: type[Any]
responsibility: str
effect: DeleteListenerEffect
listener: Callable[..., None]
@property
def key(self) -> tuple[type[Any], str]:
"""Return the stable catalog key for this declaration."""
return self.target, self.responsibility
_DELETE_LISTENERS: dict[tuple[type[Any], str], DeleteListenerDeclaration] = {}
_DELETE_LISTENER_LOCK: RLock = RLock()
def register_delete_listener(declaration: DeleteListenerDeclaration) -> None:
"""Register a declared listener idempotently."""
with _DELETE_LISTENER_LOCK:
existing: DeleteListenerDeclaration | None = _DELETE_LISTENERS.get(
declaration.key
)
if existing is not None and existing != declaration:
raise ValueError(
"Conflicting delete-listener declaration: "
f"{declaration.target.__name__}.{declaration.responsibility}"
)
_DELETE_LISTENERS[declaration.key] = declaration
if not sa.event.contains(
declaration.target, "after_delete", declaration.listener
):
sa.event.listen(declaration.target, "after_delete", declaration.listener)
def remove_delete_listener(declaration: DeleteListenerDeclaration) -> None:
"""Remove a declared listener while preserving catalog identity."""
with _DELETE_LISTENER_LOCK:
existing: DeleteListenerDeclaration | None = _DELETE_LISTENERS.get(
declaration.key
)
if existing is not None and existing != declaration:
raise ValueError(
"Conflicting delete-listener declaration: "
f"{declaration.target.__name__}.{declaration.responsibility}"
)
if sa.event.contains(declaration.target, "after_delete", declaration.listener):
sa.event.remove(declaration.target, "after_delete", declaration.listener)
_DELETE_LISTENERS.pop(declaration.key, None)
def declared_delete_listeners() -> tuple[DeleteListenerDeclaration, ...]:
"""Return declared listeners in deterministic order."""
with _DELETE_LISTENER_LOCK:
return tuple(
sorted(
_DELETE_LISTENERS.values(),
key=lambda declaration: (
declaration.target.__module__,
declaration.target.__qualname__,
declaration.responsibility,
),
)
)
+1
View File
@@ -77,6 +77,7 @@ from superset.views.error_handling import json_error_response
from .utils import bootstrap_user_data, get_config_value
FRONTEND_CONF_KEYS = (
"AUTH_ROLE_ADMIN",
"SUPERSET_WEBSERVER_TIMEOUT",
"SUPERSET_DASHBOARD_POSITION_DATA_LIMIT",
"SUPERSET_DASHBOARD_PERIODICAL_REFRESH_LIMIT",
@@ -0,0 +1,259 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Deterministic query-count guard for representative purge graphs."""
from __future__ import annotations
import os
from datetime import datetime, timedelta
from statistics import median
from time import perf_counter
from typing import Any
import pytest
import sqlalchemy as sa
from sqlalchemy.engine import Connection
from superset import db
from superset.commands.deletion_retention.purge_cascade import (
cascade_hard_delete,
CascadeResult,
suppress_purge_association_versions,
)
from superset.connectors.sqla.models import (
RowLevelSecurityFilter,
SqlaTable,
SqlMetric,
TableColumn,
)
from superset.models.core import Database
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
from ._base import DeletionRetentionTestBase
CHART_PURGE_BASELINE_STATEMENTS: int = 35
DASHBOARD_PURGE_BASELINE_STATEMENTS: int = 36
DATASET_PURGE_BASELINE_STATEMENTS: int = 43
MAX_STATEMENT_REGRESSION: float = 0.10
WARMUP_RUNS: int = 5
MEASURED_RUNS: int = 20
def statement_budget(baseline: int) -> int:
"""Return the inclusive integer budget for a measured SQL baseline."""
return int(baseline * (1 + MAX_STATEMENT_REGRESSION) + 0.9999)
class TestPurgeQueryCount(DeletionRetentionTestBase):
"""Guard against accidental graph loading or query-count explosions."""
def test_representative_chart_purge_query_count(self) -> None:
"""A chart with five dashboard memberships stays within its SQL budget."""
chart: Slice = self.make_chart("perf_chart")
for index in range(5):
self.make_dashboard(f"perf_dashboard_{index}", slices=[chart])
statement_count: int = self._purge_statement_count(chart)
if os.environ.get("SUPERSET_PURGE_BENCHMARK") == "1":
print(f"chart purge statements: {statement_count}")
assert statement_count <= statement_budget(CHART_PURGE_BASELINE_STATEMENTS)
def test_representative_dashboard_purge_query_count(self) -> None:
"""A dashboard with five chart memberships stays within its SQL budget."""
charts: list[Slice] = [
self.make_chart(f"perf_dashboard_chart_{index}") for index in range(5)
]
dashboard: Dashboard = self.make_dashboard("perf_dashboard", slices=charts)
statement_count: int = self._purge_statement_count(dashboard)
if os.environ.get("SUPERSET_PURGE_BENCHMARK") == "1":
print(f"dashboard purge statements: {statement_count}")
assert statement_count <= statement_budget(DASHBOARD_PURGE_BASELINE_STATEMENTS)
def test_representative_dataset_purge_query_count(self) -> None:
"""A dataset's fixed owned and association graph stays bounded."""
dataset: SqlaTable = self.make_dataset("perf_dataset")
for index in range(10):
db.session.add(
TableColumn(column_name=f"retention_it_column_{index}", table=dataset)
)
for index in range(5):
db.session.add(
SqlMetric(
metric_name=f"retention_it_metric_{index}",
expression="count(*)",
table=dataset,
)
)
db.session.add(
RowLevelSecurityFilter(
name=f"retention_it_rls_{index}",
clause="1=1",
filter_type="Regular",
tables=[dataset],
)
)
db.session.commit()
statement_count: int = self._purge_statement_count(dataset)
if os.environ.get("SUPERSET_PURGE_BENCHMARK") == "1":
print(f"dataset purge statements: {statement_count}")
assert statement_count <= statement_budget(DATASET_PURGE_BASELINE_STATEMENTS)
@pytest.mark.skipif(
os.environ.get("SUPERSET_PURGE_BENCHMARK") != "1",
reason="manual fixed-cardinality timing protocol",
)
@pytest.mark.parametrize(
("entity_type", "baseline_environment_variable"),
[
("chart", "SUPERSET_PURGE_BASELINE_CHART_SECONDS"),
("dashboard", "SUPERSET_PURGE_BASELINE_DASHBOARD_SECONDS"),
("dataset", "SUPERSET_PURGE_BASELINE_DATASET_SECONDS"),
],
)
def test_representative_purge_median(
self,
entity_type: str,
baseline_environment_variable: str,
) -> None:
"""Report fixed-cardinality medians and enforce supplied baselines."""
durations: list[float] = []
for iteration in range(WARMUP_RUNS + MEASURED_RUNS):
if iteration:
self._reset_benchmark_fixture()
entity: Slice | Dashboard | SqlaTable = self._make_benchmark_entity(
entity_type, iteration
)
self.soft_delete(entity, days_ago=90)
started_at: float = perf_counter()
with suppress_purge_association_versions(db.session):
result: CascadeResult = cascade_hard_delete(
db.session,
entity,
enforce_window=True,
cutoff=datetime.now() - timedelta(days=30),
)
db.session.commit()
elapsed: float = perf_counter() - started_at
assert result.purged
if iteration >= WARMUP_RUNS:
durations.append(elapsed)
measured_median: float = median(durations)
baseline_value: str | None = os.environ.get(baseline_environment_variable)
print(f"{entity_type} purge median: {measured_median:.6f}s")
if baseline_value is not None:
baseline_median: float = float(baseline_value)
regression: float = (measured_median - baseline_median) / baseline_median
print(f"{entity_type} purge elapsed-time delta: {regression:+.2%}")
assert regression <= MAX_STATEMENT_REGRESSION
assert len(durations) == MEASURED_RUNS
def _make_benchmark_entity(
self, entity_type: str, iteration: int
) -> Slice | Dashboard | SqlaTable:
"""Create one fixed-cardinality root for the manual timing protocol."""
if entity_type == "chart":
chart: Slice = self.make_chart(f"benchmark_chart_{iteration}")
for dashboard_index in range(5):
self.make_dashboard(
f"benchmark_dashboard_{iteration}_{dashboard_index}",
slices=[chart],
)
return chart
if entity_type == "dashboard":
charts: list[Slice] = [
self.make_chart(f"benchmark_chart_{iteration}_{index}")
for index in range(5)
]
return self.make_dashboard(
f"benchmark_dashboard_{iteration}", slices=charts
)
if entity_type == "dataset":
dataset: SqlaTable = self.make_dataset(f"benchmark_dataset_{iteration}")
for index in range(10):
db.session.add(
TableColumn(
column_name=f"benchmark_column_{iteration}_{index}",
table=dataset,
)
)
for index in range(5):
db.session.add(
SqlMetric(
metric_name=f"benchmark_metric_{iteration}_{index}",
expression="count(*)",
table=dataset,
)
)
db.session.add(
RowLevelSecurityFilter(
name=f"benchmark_rls_{iteration}_{index}",
clause="1=1",
filter_type="Regular",
tables=[dataset],
)
)
db.session.commit()
return dataset
raise ValueError(f"Unsupported benchmark entity type: {entity_type}")
def _reset_benchmark_fixture(self) -> None:
"""Rebuild the fixed fixture between manual timing samples."""
self._cleanup()
self.database: Database = Database(
database_name="retention_it_db", sqlalchemy_uri="sqlite://"
)
db.session.add(self.database)
db.session.commit()
self.dataset: SqlaTable = self.make_dataset("ds")
def _purge_statement_count(self, entity: Any) -> int:
"""Purge one root and return SQL statements within the measured region."""
self.soft_delete(entity, days_ago=90)
statements: list[str] = []
def count_statement(
_connection: Connection,
_cursor: object,
statement: str,
_parameters: object,
_context: object,
_executemany: bool,
) -> None:
statements.append(statement)
sa.event.listen(db.engine, "before_cursor_execute", count_statement)
try:
with suppress_purge_association_versions(db.session):
result: CascadeResult = cascade_hard_delete(
db.session,
entity,
enforce_window=True,
cutoff=datetime.now() - timedelta(days=30),
)
db.session.commit()
finally:
sa.event.remove(db.engine, "before_cursor_execute", count_statement)
assert result.purged
return len(statements)
@@ -24,25 +24,35 @@ guarantee under FK enforcement OFF, and the version-tables-absent no-op.
from __future__ import annotations
from dataclasses import replace
from datetime import datetime, timedelta
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
import sqlalchemy as sa
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from sqlalchemy.orm.attributes import set_committed_value
from sqlalchemy.sql.dml import Delete
from superset import db, security_manager
from superset.commands.deletion_retention import audit
from superset.commands.deletion_retention.purge_cascade import cascade_hard_delete
from superset.commands.deletion_retention.purge_cascade import (
cascade_hard_delete,
suppress_purge_association_versions,
)
from superset.commands.deletion_retention.purge_policy import (
get_purge_policy,
PurgeEntityPolicy,
)
from superset.connectors.sqla.models import (
RLSFilterTables,
RowLevelSecurityFilter,
SqlaTable,
)
from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES
from superset.models.dashboard import Dashboard
from superset.models.dashboard import Dashboard, dashboard_slices
from superset.models.slice import Slice
from superset.models.user_attributes import UserAttribute
from superset.reports.models import ReportSchedule
@@ -779,9 +789,19 @@ class TestExplicitBlockerGuards(DeletionRetentionTestBase):
"(sqlite3.IntegrityError) FOREIGN KEY constraint failed "
"[SQL: DELETE FROM slices WHERE slices.id = ?] [parameters: (1,)]"
)
def fail_association_cleanup(
_session: Session, _policy: PurgeEntityPolicy, _entity_id: int
) -> None:
raise IntegrityError(driver_text, None, Exception("fk"))
policy: PurgeEntityPolicy = replace(
get_purge_policy(Slice),
delete_associations=fail_association_cleanup,
)
with patch(
"superset.commands.deletion_retention.purge_cascade._delete_m2m_joins",
side_effect=IntegrityError(driver_text, None, Exception("fk")),
"superset.commands.deletion_retention.purge_cascade.get_purge_policy",
return_value=policy,
):
result = cascade_hard_delete(
db.session,
@@ -796,6 +816,88 @@ class TestExplicitBlockerGuards(DeletionRetentionTestBase):
assert "SQL:" not in result.blocked_reason
assert self.exists(Slice, chart_id)
def test_policy_action_failure_rolls_back_prior_phases(self) -> None:
"""A later policy-action failure restores earlier association cleanup."""
chart: Slice = self.make_chart("action_rollback")
chart_id: int = chart.id
dashboard: Dashboard = self.make_dashboard("action_rollback", slices=[chart])
dashboard_id: int = dashboard.id
self.soft_delete(chart, days_ago=90)
def fail_owned_cleanup(
_session: Session, _policy: PurgeEntityPolicy, _entity_id: int
) -> None:
raise RuntimeError("injected owned cleanup failure")
policy: PurgeEntityPolicy = replace(
get_purge_policy(Slice),
delete_owned_children=fail_owned_cleanup,
)
with (
patch(
"superset.commands.deletion_retention.purge_cascade.get_purge_policy",
return_value=policy,
),
pytest.raises(RuntimeError, match="injected owned cleanup failure"),
):
with suppress_purge_association_versions(db.session):
cascade_hard_delete(
db.session,
chart,
enforce_window=True,
cutoff=datetime.now() - timedelta(days=30),
)
membership_count: int = int(
db.session.execute(
sa.select(sa.func.count())
.select_from(dashboard_slices)
.where(
dashboard_slices.c.dashboard_id == dashboard_id,
dashboard_slices.c.slice_id == chart_id,
)
).scalar_one()
)
assert membership_count == 1
assert self.exists(Slice, chart_id)
def test_history_cleanup_failure_rolls_back_prior_phases(self) -> None:
"""A history-phase failure restores association cleanup and the root."""
chart: Slice = self.make_chart("history_rollback")
chart_id: int = chart.id
dashboard: Dashboard = self.make_dashboard("history_rollback", slices=[chart])
dashboard_id: int = dashboard.id
self.soft_delete(chart, days_ago=90)
with (
patch(
"superset.commands.deletion_retention.purge_cascade."
"_delete_version_history",
side_effect=RuntimeError("injected history cleanup failure"),
),
pytest.raises(RuntimeError, match="injected history cleanup failure"),
):
with suppress_purge_association_versions(db.session):
cascade_hard_delete(
db.session,
chart,
enforce_window=True,
cutoff=datetime.now() - timedelta(days=30),
)
membership_count: int = int(
db.session.execute(
sa.select(sa.func.count())
.select_from(dashboard_slices)
.where(
dashboard_slices.c.dashboard_id == dashboard_id,
dashboard_slices.c.slice_id == chart_id,
)
).scalar_one()
)
assert membership_count == 1
assert self.exists(Slice, chart_id)
class TestFailClosedAudienceDefault(DeletionRetentionTestBase):
"""A soft-delete model without editors must not enumerate to everyone."""
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
@@ -0,0 +1,570 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Contract tests for declarative hard-purge policies."""
from __future__ import annotations
from dataclasses import replace
from typing import Any
from unittest.mock import MagicMock
import pytest
import sqlalchemy as sa
from sqlalchemy.engine import Dialect
from sqlalchemy.orm import configure_mappers, registry
from sqlalchemy.sql import Select
from superset.commands.deletion_retention.purge_policy import (
_dependency_owner_depth,
_dependency_predicates,
_fk_key,
compare_policy,
delete_associations,
delete_owned_children,
DependencyClassification,
DependencyKey,
DependencyPolicy,
discover_dependencies,
get_purge_policy,
listener_responsibilities,
PolicyCoverage,
purge_policy_registry,
PurgeEntityPolicy,
validate_deletion_allowed,
validate_unique_root_policies,
)
from superset.connectors.sqla.models import SqlaTable
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
from superset.tasks.deletion_retention import _soft_delete_models
from superset.utils.sqlalchemy_events import (
declared_delete_listeners,
DeleteListenerDeclaration,
DeleteListenerEffect,
register_delete_listener,
remove_delete_listener,
)
def test_compare_policy_reports_missing_duplicate_and_stale_dependencies() -> None:
"""Coverage diagnostics identify every kind of registry drift."""
discovered: set[DependencyKey] = {
DependencyKey("foreign_key", "root", "owned"),
DependencyKey("relationship", "root", "preserved", relationship="item"),
}
duplicate: DependencyPolicy = DependencyPolicy(
DependencyKey("foreign_key", "root", "owned"),
DependencyClassification.OWNED,
)
stale: DependencyPolicy = DependencyPolicy(
DependencyKey("foreign_key", "root", "stale"),
DependencyClassification.PRESERVE,
)
coverage: PolicyCoverage = compare_policy(discovered, (duplicate, duplicate, stale))
assert len(coverage.missing) == 1
assert coverage.duplicates == (duplicate.key,)
assert coverage.stale == (stale.key,)
def test_adding_a_complete_policy_restores_coverage() -> None:
"""A new dependency passes after its policy is supplied."""
key: DependencyKey = DependencyKey("foreign_key", "root", "child")
assert not compare_policy({key}, ()).complete
assert compare_policy(
{key},
(DependencyPolicy(key, DependencyClassification.OWNED),),
).complete
def test_omitted_inbound_fk_and_listener_are_reported() -> None:
"""Inbound metadata edges and persistent listeners are obligations."""
inbound: DependencyKey = DependencyKey(
"foreign_key",
"root",
"referrer",
("id",),
("root_id",),
"inbound",
)
coverage: PolicyCoverage = compare_policy(
{inbound},
(),
discovered_listeners={"persistent_cleanup"},
)
assert coverage.missing == (inbound,)
assert coverage.missing_listeners == ("persistent_cleanup",)
@pytest.mark.parametrize(
"classification",
[DependencyClassification.PRESERVE, DependencyClassification.BLOCK],
)
def test_terminal_dependency_classifications_are_complete(
classification: DependencyClassification,
) -> None:
"""Preserve and block are explicit terminal treatments, not omissions."""
key: DependencyKey = DependencyKey("foreign_key", "root", "terminal")
assert compare_policy({key}, (DependencyPolicy(key, classification),)).complete
def test_composite_foreign_key_is_one_atomic_dependency() -> None:
"""Composite constraints retain ordered local and remote column tuples."""
metadata: sa.MetaData = sa.MetaData()
root: sa.Table = sa.Table(
"root",
metadata,
sa.Column("tenant_id", sa.Integer, primary_key=True),
sa.Column("id", sa.Integer, primary_key=True),
)
child: sa.Table = sa.Table(
"child",
metadata,
sa.Column("tenant_id", sa.Integer),
sa.Column("root_id", sa.Integer),
sa.ForeignKeyConstraint(
("tenant_id", "root_id"), ("root.tenant_id", "root.id")
),
)
constraint: sa.ForeignKeyConstraint = next(iter(child.foreign_key_constraints))
key: DependencyKey = _fk_key(root, constraint, "inbound")
assert key.local_columns == ("tenant_id", "id")
assert key.remote_columns == ("tenant_id", "root_id")
def test_duplicate_root_policies_are_rejected() -> None:
"""Registry construction cannot silently replace a root declaration."""
policy: PurgeEntityPolicy = get_purge_policy(Slice)
with pytest.raises(ValueError, match="Duplicate purge policy for Slice"):
validate_unique_root_policies((policy, policy))
def test_every_soft_delete_root_has_a_purge_policy() -> None:
"""Every built-in retention root has exactly one purge policy."""
production_roots: set[type[Any]] = {
model
for model in _soft_delete_models()
if model.__module__.startswith("superset.")
}
assert set(purge_policy_registry()) == production_roots
def test_listener_coverage_reports_stale_and_optional_declarations() -> None:
"""Required stale listeners fail while disabled optional listeners pass."""
stale: PolicyCoverage = compare_policy(
(), (), declared_listeners={"removed_cleanup"}
)
optional: PolicyCoverage = compare_policy(
(),
(),
declared_listeners={"optional_cleanup"},
optional_declared_listeners={"optional_cleanup"},
)
assert stale.stale_listeners == ("removed_cleanup",)
assert optional.complete
@pytest.mark.parametrize("model", [Slice, Dashboard, SqlaTable])
def test_real_mapper_graph_has_complete_policy(model: type[Any]) -> None:
"""Every supported root mapper dependency has one policy."""
configure_mappers()
metadata_tables: set[str] = set(sa.inspect(Slice).local_table.metadata.tables)
assert {
"embedded_dashboards",
"report_schedule",
"rls_filter_tables",
"tagged_object",
"user_attribute",
} <= metadata_tables
policy: PurgeEntityPolicy = get_purge_policy(model)
recursive_tables: frozenset[str] = frozenset(
dependency.key.related_table
for dependency in policy.dependencies
if dependency.classification
in {
DependencyClassification.OWNED,
DependencyClassification.ASSOCIATION,
}
)
coverage: PolicyCoverage = compare_policy(
discover_dependencies(sa.inspect(model), recursive_tables=recursive_tables),
policy.dependencies,
discovered_listeners=listener_responsibilities(model),
declared_listeners=policy.listener_responsibilities,
optional_declared_listeners=policy.optional_listener_responsibilities,
)
assert coverage.complete, coverage
@pytest.mark.parametrize("model", [Slice, Dashboard, SqlaTable])
def test_non_preserve_dependencies_carry_phases(model: type[Any]) -> None:
"""Every executable classification declares its execution phase."""
policy: PurgeEntityPolicy = get_purge_policy(model)
assert all(
dependency.phase is not None
for dependency in policy.dependencies
if dependency.classification is not DependencyClassification.PRESERVE
)
def test_recursive_discovery_stops_at_owned_cycles() -> None:
"""Owned-child backrefs terminate instead of walking the graph forever."""
dependencies: frozenset[DependencyKey] = discover_dependencies(
sa.inspect(Dashboard),
recursive_tables=frozenset({"dashboards", "embedded_dashboards"}),
)
assert dependencies
assert len(dependencies) == len(set(dependencies))
assert any(
dependency.owner_table == "embedded_dashboards"
and dependency.related_table == "dashboards"
for dependency in dependencies
)
def test_dependency_owner_depth_handles_deep_paths_without_recursion() -> None:
"""Ownership ordering and predicates are independent of recursion limits."""
path_length: int = 1_100
metadata: sa.MetaData = sa.MetaData()
root_table: sa.Table = sa.Table(
"synthetic_root",
metadata,
sa.Column("id", sa.Integer, primary_key=True),
)
mapper_registry: registry = registry()
class SyntheticRoot:
"""Temporary mapped root for deep ownership-path construction."""
mapper_registry.map_imperatively(SyntheticRoot, root_table)
for index in range(path_length):
sa.Table(
f"owned_{index}",
metadata,
sa.Column("id", sa.Integer, primary_key=True),
sa.Column("owner_id", sa.Integer),
)
sa.Table(
"leaf",
metadata,
sa.Column("id", sa.Integer, primary_key=True),
sa.Column("owner_id", sa.Integer),
)
dependencies: tuple[DependencyPolicy, ...] = tuple(
DependencyPolicy(
DependencyKey(
"foreign_key",
"synthetic_root" if index == 0 else f"owned_{index - 1}",
f"owned_{index}",
("id",),
("owner_id",),
"inbound",
),
DependencyClassification.OWNED,
)
for index in range(path_length)
)
policy: PurgeEntityPolicy = replace(
get_purge_policy(Slice), model=SyntheticRoot, dependencies=dependencies
)
leaf: DependencyKey = DependencyKey(
"foreign_key",
f"owned_{path_length - 1}",
"leaf",
("id",),
("owner_id",),
"inbound",
)
assert _dependency_owner_depth(policy, leaf) == path_length
predicates: tuple[Any, ...] = _dependency_predicates(
policy, leaf, 1, metadata.tables["leaf"]
)
assert len(predicates) == 1
mapper_registry.dispose()
def test_dependency_owner_depth_rejects_cycles() -> None:
"""Malformed ownership declarations fail clearly instead of looping."""
first: DependencyPolicy = DependencyPolicy(
DependencyKey("foreign_key", "owned_b", "owned_a", direction="inbound"),
DependencyClassification.OWNED,
)
second: DependencyPolicy = DependencyPolicy(
DependencyKey("foreign_key", "owned_a", "owned_b", direction="inbound"),
DependencyClassification.OWNED,
)
policy: PurgeEntityPolicy = replace(
get_purge_policy(Slice), dependencies=(first, second)
)
leaf: DependencyKey = DependencyKey("foreign_key", "owned_a", "leaf")
with pytest.raises(RuntimeError, match="Cyclic ownership path"):
_dependency_owner_depth(policy, leaf)
@pytest.mark.parametrize(
("model", "expected_targets"),
[
(
Slice,
(
("slices_version", "id"),
("dashboard_slices_version", "slice_id"),
),
),
(
Dashboard,
(
("dashboards_version", "id"),
("dashboard_slices_version", "dashboard_id"),
),
),
(
SqlaTable,
(
("tables_version", "id"),
("sql_metrics_version", "table_id"),
("table_columns_version", "table_id"),
),
),
],
)
def test_version_targets_are_policy_owned(
model: type[Any], expected_targets: tuple[tuple[str, str], ...]
) -> None:
"""Root, association, and owned-child shadows come from the policy."""
policy: PurgeEntityPolicy = get_purge_policy(model)
assert policy.version_shadow_names == expected_targets
def test_version_target_resolution_rejects_invalid_declarations(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A shadow-table typo cannot silently leave version history behind."""
from superset.commands.deletion_retention import purge_cascade
metadata: sa.MetaData = sa.MetaData()
parent_shadow: sa.Table = sa.Table(
"slices_version", metadata, sa.Column("id", sa.Integer)
)
slice_policy: PurgeEntityPolicy = purge_policy_registry()[Slice]
dependencies: tuple[DependencyPolicy, ...] = tuple(
replace(
dependency,
key=replace(dependency.key, related_table="missing_version"),
)
if dependency.key.related_table == "slices_version"
else dependency
for dependency in slice_policy.dependencies
)
policy: PurgeEntityPolicy = replace(slice_policy, dependencies=dependencies)
def fake_policy(_model: type[Any]) -> PurgeEntityPolicy:
return policy
monkeypatch.setattr(purge_cascade, "get_purge_policy", fake_policy)
with pytest.raises(RuntimeError, match="missing_version.id"):
purge_cascade._entity_version_targets(
Slice, metadata, parent_shadow, entity_id=1
)
def test_listener_registration_is_idempotent_and_symmetric() -> None:
"""Declared listener registration can safely repeat and clear."""
def observe(*_args: Any) -> None:
return None
declaration: DeleteListenerDeclaration = DeleteListenerDeclaration(
Slice,
"test_observer",
DeleteListenerEffect.OBSERVATIONAL,
observe,
)
try:
register_delete_listener(declaration)
register_delete_listener(declaration)
assert declaration in declared_delete_listeners()
finally:
remove_delete_listener(declaration)
assert declaration not in declared_delete_listeners()
def test_listener_removal_rejects_a_conflicting_declaration() -> None:
"""Removal cannot erase a different declaration with the same key."""
def first(*_args: Any) -> None:
return None
def conflicting(*_args: Any) -> None:
return None
declaration: DeleteListenerDeclaration = DeleteListenerDeclaration(
Slice,
"test_conflict",
DeleteListenerEffect.OBSERVATIONAL,
first,
)
conflicting_declaration: DeleteListenerDeclaration = DeleteListenerDeclaration(
Slice,
"test_conflict",
DeleteListenerEffect.OBSERVATIONAL,
conflicting,
)
try:
register_delete_listener(declaration)
with pytest.raises(ValueError, match="Conflicting delete-listener"):
remove_delete_listener(conflicting_declaration)
assert declaration in declared_delete_listeners()
finally:
remove_delete_listener(declaration)
@pytest.mark.parametrize("model", [Slice, Dashboard, SqlaTable])
def test_supported_root_after_delete_listeners_are_declared_and_installed(
model: type[Any],
) -> None:
"""Every runtime root listener is represented in the listener catalog."""
declarations: tuple[DeleteListenerDeclaration, ...] = tuple(
declaration
for declaration in declared_delete_listeners()
if declaration.target is model
and declaration.effect is not DeleteListenerEffect.OBSERVATIONAL
)
policy: PurgeEntityPolicy = get_purge_policy(model)
assert {declaration.responsibility for declaration in declarations} <= set(
policy.listener_responsibilities
)
assert all(
sa.event.contains(model, "after_delete", declaration.listener)
for declaration in declarations
)
installed_listeners: set[Any] = {
cell.cell_contents
for wrapper in model.__mapper__.dispatch.after_delete
for cell in (wrapper.__closure__ or ())
if callable(cell.cell_contents)
and not getattr(cell.cell_contents, "__module__", "").startswith(
"sqlalchemy_continuum"
)
}
assert installed_listeners == {declaration.listener for declaration in declarations}
def test_delete_associations_rejects_unknown_entity_type() -> None:
"""An unsupported root cannot fall through to dataset cleanup."""
from superset.commands.deletion_retention.purge_policy import delete_associations
slice_policy: PurgeEntityPolicy = get_purge_policy(Slice)
listener_dependencies: tuple[DependencyPolicy, ...] = tuple(
dependency
for dependency in slice_policy.dependencies
if dependency.classification is DependencyClassification.LISTENER_EFFECT
)
policy: PurgeEntityPolicy = replace(
slice_policy,
entity_type="unsupported",
dependencies=listener_dependencies,
)
with pytest.raises(ValueError, match="Unsupported purge entity type"):
delete_associations(MagicMock(), policy, 1)
def test_association_owned_children_are_deleted_before_their_owner() -> None:
"""Association traversal deletes nested rows before their owning rows."""
association: DependencyPolicy = DependencyPolicy(
DependencyKey(
"foreign_key",
"slices",
"dashboard_slices",
("id",),
("slice_id",),
"inbound",
),
DependencyClassification.ASSOCIATION,
)
association_child: DependencyPolicy = DependencyPolicy(
DependencyKey(
"foreign_key",
"dashboard_slices",
"dashboard_slices_version",
("dashboard_id", "slice_id"),
("dashboard_id", "slice_id"),
"inbound",
),
DependencyClassification.ASSOCIATION,
)
policy: PurgeEntityPolicy = replace(
get_purge_policy(Slice),
dependencies=(association, association_child),
)
session: MagicMock = MagicMock()
delete_associations(session, policy, 7)
statements: list[Any] = [call.args[0] for call in session.execute.call_args_list]
assert [statement.table.name for statement in statements] == [
"dashboard_slices_version",
"dashboard_slices",
]
assert "dashboard_slices.slice_id" in str(statements[0])
assert "slices.id" in str(statements[0])
@pytest.mark.parametrize("dialect", ["sqlite", "postgresql", "mysql"])
def test_core_delete_actions_compile_for_supported_dialects(dialect: str) -> None:
"""Statements emitted by policy callbacks compile for supported dialects."""
from sqlalchemy.dialects import mysql, postgresql, sqlite
dialects: dict[str, Dialect] = {
"sqlite": sqlite.dialect(),
"postgresql": postgresql.dialect(),
"mysql": mysql.dialect(),
}
compiled: list[str] = []
for model in (Slice, Dashboard, SqlaTable):
policy: PurgeEntityPolicy = get_purge_policy(model)
session: MagicMock = MagicMock()
session.execute.return_value.first.return_value = None
validate_deletion_allowed(session, policy, 1)
delete_associations(session, policy, 1)
delete_owned_children(session, policy, 1)
calls: list[Any] = list(session.execute.call_args_list)
for call in calls:
statement: Any = call.args[0]
compiled.append(str(statement.compile(dialect=dialects[dialect])))
claim: Select = sa.select(Slice.id).where(Slice.id == 1).with_for_update()
compiled.append(str(claim.compile(dialect=dialects[dialect])))
assert compiled
assert all(statement.startswith(("SELECT", "DELETE")) for statement in compiled)
@@ -0,0 +1,570 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from superset.common.form_data_query_context import (
adhoc_filters_to_query_filters,
build_query_context_from_form_data,
columns_from_form_data,
)
DATASOURCE = {"id": 7, "type": "table"}
def test_adhoc_filters_converts_simple_and_drops_custom_sql() -> None:
adhoc = [
{
"expressionType": "SIMPLE",
"subject": "country",
"operator": "==",
"comparator": "US",
},
{"expressionType": "SQL", "sqlExpression": "1 = 1"},
]
assert adhoc_filters_to_query_filters(adhoc) == [
{"col": "country", "op": "==", "val": "US"}
]
assert adhoc_filters_to_query_filters([]) == []
def test_columns_prefers_groupby_and_x_axis() -> None:
form_data = {"groupby": ["region"], "x_axis": "ds"}
assert columns_from_form_data(form_data) == ["ds", "region"]
def test_columns_raw_mode_uses_all_columns() -> None:
form_data = {"query_mode": "raw", "all_columns": ["a", "b"]}
assert columns_from_form_data(form_data) == ["a", "b"]
def test_columns_x_axis_as_adhoc_dict() -> None:
# An x_axis stored as an adhoc column dict contributes its column_name,
# prepended ahead of the groupby dimensions.
form_data = {"groupby": ["region"], "x_axis": {"column_name": "ds"}}
assert columns_from_form_data(form_data) == ["ds", "region"]
def test_columns_x_axis_dict_without_column_name_is_ignored() -> None:
form_data = {
"groupby": ["region"],
"x_axis": {"label": "custom", "sqlExpression": "a+b"},
}
assert columns_from_form_data(form_data) == ["region"]
def test_columns_empty_columns_key_does_not_shadow_groupby() -> None:
# A stale, explicitly-present-but-empty ``columns`` key must not drop the
# group-by dimensions (which would silently change the aggregation).
form_data = {"groupby": ["country"], "columns": []}
assert columns_from_form_data(form_data) == ["country"]
def test_build_context_maps_groupby_metrics_and_filters() -> None:
form_data = {
"groupby": ["country"],
"metrics": ["count"],
"adhoc_filters": [
{
"expressionType": "SIMPLE",
"subject": "year",
"operator": ">",
"comparator": 2000,
},
],
"time_range": "Last year",
"row_limit": 500,
}
ctx = build_query_context_from_form_data(form_data, DATASOURCE)
assert ctx["datasource"] == DATASOURCE
assert ctx["form_data"] == form_data
assert len(ctx["queries"]) == 1
query = ctx["queries"][0]
assert query["columns"] == ["country"]
assert query["metrics"] == ["count"]
assert query["filters"] == [{"col": "year", "op": ">", "val": 2000}]
assert query["time_range"] == "Last year"
assert query["row_limit"] == 500
def test_build_context_big_number_singular_metric_and_default_time_range() -> None:
form_data = {"metric": "sum__sales"}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["metrics"] == ["sum__sales"]
assert query["time_range"] == "No filter"
# No row_limit in form data → not forced into the query.
assert "row_limit" not in query
def test_build_context_merges_legacy_and_adhoc_filters() -> None:
# Legacy charts store simple filters directly under ``filters`` (already in
# QueryObject shape); they are honored alongside adhoc_filters, and malformed
# entries are dropped.
form_data = {
"groupby": ["country"],
"adhoc_filters": [
{
"expressionType": "SIMPLE",
"subject": "year",
"operator": ">",
"comparator": 2000,
},
],
"filters": [
{"col": "region", "op": "==", "val": "EMEA"},
{"not_a_filter": True},
],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["filters"] == [
{"col": "year", "op": ">", "val": 2000},
{"col": "region", "op": "==", "val": "EMEA"},
]
def test_big_number_trendline_promotes_granularity_sqla_column() -> None:
# A Big Number *with a trendline* (viz_type "big_number") has no
# groupby/columns; its time column (granularity_sqla) becomes the sole column.
form_data = {"metric": "count", "granularity_sqla": "order_date"}
query = build_query_context_from_form_data(
form_data, DATASOURCE, viz_type="big_number"
)["queries"][0]
assert query["columns"] == ["order_date"]
assert query["metrics"] == ["count"]
def test_big_number_total_does_not_promote_granularity_sqla_column() -> None:
# big_number_total is a single aggregate; promoting granularity_sqla to a
# column would turn one total into one row per timestamp.
form_data = {"metric": "count", "granularity_sqla": "order_date"}
query = build_query_context_from_form_data(
form_data, DATASOURCE, viz_type="big_number_total"
)["queries"][0]
assert query["columns"] == []
def test_build_context_sets_granularity_for_time_filtering() -> None:
# Without a `granularity`, the `time_range` is inert downstream, so a legacy
# chart with granularity_sqla + time_range would export its full history.
form_data = {
"metrics": ["count"],
"granularity_sqla": "ds",
"time_range": "Last quarter",
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["granularity"] == "ds"
assert query["time_range"] == "Last quarter"
def test_build_context_prefers_explicit_granularity_over_sqla() -> None:
form_data = {
"metrics": ["count"],
"granularity": "event_time",
"granularity_sqla": "ds",
"time_range": "Last week",
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["granularity"] == "event_time"
def test_build_context_sets_granularity_without_active_time_range() -> None:
# `granularity` also drives time-grain bucketing of a selected column, not just
# the time filter, so it is set whenever form data carries one — matching
# extractExtras.ts, which sets it unconditionally.
form_data = {"metrics": ["count"], "granularity_sqla": "ds"}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["granularity"] == "ds"
def test_orderby_defaults_to_first_metric_descending() -> None:
# With a row_limit, ordering must be deterministic so the export returns the
# chart's top-N, not an arbitrary N.
form_data = {"metrics": ["count"], "groupby": ["c"], "row_limit": 10}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["orderby"] == [["count", False]]
def test_orderby_uses_timeseries_limit_metric_and_order_desc() -> None:
form_data = {
"metrics": ["count"],
"groupby": ["c"],
"timeseries_limit_metric": "revenue",
"order_desc": False,
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["orderby"] == [["revenue", True]]
def test_orderby_pie_sort_by_metric() -> None:
form_data = {"metric": "count", "groupby": ["c"], "sort_by_metric": True}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="pie")[
"queries"
][0]
assert query["orderby"] == [["count", False]]
def test_orderby_raw_mode_parses_order_by_cols() -> None:
form_data = {
"query_mode": "raw",
"all_columns": ["a"],
# A malformed entry is skipped rather than raising.
"order_by_cols": ['["a", true]', "not json", ["b", False]],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["orderby"] == [["a", True], ["b", False]]
def test_aggregate_mode_ignores_stale_order_by_cols() -> None:
# order_by_cols is a raw-mode-only control (resetOnHide: false), so it isn't
# reset when switching to aggregate mode. The rebuild must ignore a stale value
# and order by the metric like the chart does, or a row_limit would return a
# different top-N than the chart shows.
form_data = {
"metrics": ["count"],
"groupby": ["c"],
"order_by_cols": ['["a", true]'],
"row_limit": 10,
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["orderby"] == [["count", False]]
def test_sql_filters_and_legacy_where_go_into_extras() -> None:
form_data = {
"groupby": ["c"],
"where": "region = 'EMEA'",
"adhoc_filters": [
{"expressionType": "SQL", "clause": "WHERE", "sqlExpression": "sales > 0"},
{
"expressionType": "SQL",
"clause": "HAVING",
"sqlExpression": "SUM(x) > 5",
},
],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["extras"]["where"] == "(region = 'EMEA') AND (sales > 0)"
assert query["extras"]["having"] == "(SUM(x) > 5)"
def test_table_carries_time_grain() -> None:
# ``time_grain_sqla`` is passed through in ``extras`` so a temporal dimension
# is bucketed as the chart does. (Charts with percent_metrics are skipped
# upstream in the export, not rebuilt — see the export task tests.)
form_data = {
"groupby": ["c"],
"metrics": ["count"],
"time_grain_sqla": "P1M",
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["metrics"] == ["count"]
assert query["extras"]["time_grain_sqla"] == "P1M"
def test_table_groupby_time_column_without_time_range_is_bucketed() -> None:
# Verified against a live export (dashboard Excel export, PR #42284): a table
# chart grouped by its own time column, with a time grain but no active
# time_range ("all-time totals by month" — a very ordinary configuration),
# must still bucket that column by its time grain. Confirmed live: the same
# chart with an explicit time_range instead of "No filter" correctly returns
# one row per year; with "No filter" it instead returns one row per *raw*
# timestamp (e.g. one per individual order date) — i.e. completely
# unaggregated data, not merely "the full history" the granularity comment
# in build_query_context_from_form_data anticipates. Gating ``granularity``
# on ``time_range != "No filter"`` conflates "should we apply a WHERE time
# filter" (legitimately time_range-dependent) with "should this selected
# column be truncated to its time grain" (not time_range-dependent at all —
# the real frontend's extractExtras.ts sets `granularity` unconditionally
# whenever granularity_sqla/granularity is present).
form_data = {
"groupby": ["order_date"],
"metrics": ["count"],
"granularity_sqla": "order_date",
"time_grain_sqla": "P1Y",
"time_range": "No filter",
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["columns"] == ["order_date"]
assert query["extras"]["time_grain_sqla"] == "P1Y"
assert query["granularity"] == "order_date"
def test_simple_having_filter_converted_by_default_but_not_where_only() -> None:
# Default: all SIMPLE filters convert (the behavior MCP relies on).
# where_only=True: SIMPLE HAVING is dropped (matching the chart), which the
# export uses so it doesn't filter on a clause the chart ignores.
adhoc = [
{
"expressionType": "SIMPLE",
"clause": "HAVING",
"subject": "count",
"operator": ">",
"comparator": 5,
}
]
assert adhoc_filters_to_query_filters(adhoc) == [
{"col": "count", "op": ">", "val": 5}
]
assert adhoc_filters_to_query_filters(adhoc, where_only=True) == []
def test_build_context_ignores_simple_having_filter() -> None:
# The export must not apply a SIMPLE HAVING filter the chart itself ignores.
form_data = {
"groupby": ["c"],
"metrics": ["count"],
"adhoc_filters": [
{
"expressionType": "SIMPLE",
"clause": "HAVING",
"subject": "count",
"operator": ">",
"comparator": 5,
},
{
"expressionType": "SIMPLE",
"subject": "region",
"operator": "==",
"comparator": "EMEA",
},
],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["filters"] == [{"col": "region", "op": "==", "val": "EMEA"}]
def test_big_number_trendline_sets_granularity_without_time_range() -> None:
# A Big Number trendline groups by its time column; granularity must be set so
# time_grain_sqla buckets it even when there's no active time range.
form_data = {
"metric": "count",
"granularity_sqla": "ds",
"time_grain_sqla": "P1M",
}
query = build_query_context_from_form_data(
form_data, DATASOURCE, viz_type="big_number"
)["queries"][0]
assert query["columns"] == ["ds"]
assert query["granularity"] == "ds"
assert query["extras"]["time_grain_sqla"] == "P1M"
def test_time_range_falls_back_to_since_until() -> None:
# Older charts store the range as separate since/until rather than time_range.
form_data = {"metrics": ["count"], "since": "2020-01-01", "until": "2020-12-31"}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["time_range"] == "2020-01-01 : 2020-12-31"
def test_raw_mode_ignores_stale_metrics_and_groupby() -> None:
# Raw-mode form data can carry stale metrics/groupby (the controls aren't
# reset when hidden); the rebuild must ignore them like the chart does, or it
# would aggregate/group and re-order by a stale metric.
form_data = {
"query_mode": "raw",
"all_columns": ["name", "sales"],
"metrics": ["count"],
"groupby": ["genre"],
"timeseries_limit_metric": "count",
"row_limit": 10,
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["columns"] == ["name", "sales"]
assert query["metrics"] == []
# No order_by_cols and no metrics → no metric-based ordering.
assert query["orderby"] == []
def test_raw_mode_inferred_from_all_columns_without_query_mode() -> None:
# No explicit query_mode, but all_columns present → raw (mirrors getQueryMode).
form_data = {"all_columns": ["a", "b"], "groupby": ["c"], "metrics": ["m"]}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["columns"] == ["a", "b"]
assert query["metrics"] == []
def test_orderby_table_sort_metric_defaults_ascending() -> None:
# Table defaults order_desc to False → ascending (matching the chart), so a row
# limit keeps the chart's bottom-N rather than flipping it to top-N.
form_data = {
"metrics": ["count"],
"groupby": ["c"],
"timeseries_limit_metric": "revenue",
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["orderby"] == [["revenue", True]]
def test_orderby_unwraps_list_valued_sort_metric() -> None:
# The drag-and-drop "sort by" control persists timeseries_limit_metric as a
# list; the frontend unwraps it with ensureIsArray(...)[0]. Read raw, the
# nested list produces an orderby the query runner rejects, so the chart
# lands in the general error bucket instead of exporting.
form_data = {
"metrics": ["count"],
"groupby": ["c"],
"timeseries_limit_metric": ["revenue"],
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["orderby"] == [["revenue", True]]
def test_orderby_empty_list_sort_metric_falls_back_to_first_metric() -> None:
# An emptied sort-by control leaves `[]` behind; treat it as unset.
form_data = {"metrics": ["count"], "groupby": ["c"], "timeseries_limit_metric": []}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["orderby"] == [["count", False]]
def test_orderby_adhoc_sort_metric_is_not_unwrapped() -> None:
# An adhoc metric is a dict, not a list: it must pass through whole rather
# than being reduced to one of its keys.
adhoc_metric = {
"expressionType": "SIMPLE",
"column": {"column_name": "sales"},
"aggregate": "SUM",
"label": "SUM(sales)",
}
form_data = {
"metrics": ["count"],
"groupby": ["c"],
"timeseries_limit_metric": adhoc_metric,
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert query["orderby"] == [[adhoc_metric, True]]
def test_raw_mode_order_by_cols_drops_non_pair_entries() -> None:
# order_by_cols entries that parse but aren't [col, asc] pairs (a stray null,
# a bare column, an over-long tuple) would append junk to orderby and fail the
# query; only well-formed pairs survive.
form_data = {
"query_mode": "raw",
"all_columns": ["a"],
"order_by_cols": ["null", '["a"]', '["b", true, 1]', '["c", false]', 5],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["orderby"] == [["c", False]]
def test_freeform_where_clause_with_sql_comment_is_newline_terminated() -> None:
# A free-form SQL filter ending in a `--` comment would otherwise comment out
# the closing paren and every predicate joined after it, so the export fails
# on a chart that renders fine. Mirrors sanitizeClause in processFilters.ts.
form_data = {
"groupby": ["c"],
"adhoc_filters": [
{
"expressionType": "SQL",
"clause": "WHERE",
"sqlExpression": "sales > 0 -- note",
},
{"expressionType": "SQL", "clause": "WHERE", "sqlExpression": "qty > 1"},
],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["extras"]["where"] == "(sales > 0 -- note\n) AND (qty > 1)"
def test_freeform_having_clause_with_sql_comment_is_newline_terminated() -> None:
form_data = {
"groupby": ["c"],
"adhoc_filters": [
{
"expressionType": "SQL",
"clause": "HAVING",
"sqlExpression": "SUM(x) > 5 -- note",
},
],
}
query = build_query_context_from_form_data(form_data, DATASOURCE)["queries"][0]
assert query["extras"]["having"] == "(SUM(x) > 5 -- note\n)"
def test_pie_carries_contribution_post_processing() -> None:
# Pie's buildQuery attaches the contribution operator unconditionally and its
# transformProps reads the renamed column, so a rebuilt pie sheet must carry
# the same percentage column a saved-context pie sheet has.
form_data = {"metric": "count", "groupby": ["c"]}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="pie")[
"queries"
][0]
assert query["post_processing"] == [
{
"operation": "contribution",
"options": {
"columns": ["count"],
"rename_columns": ["count__contribution"],
},
}
]
def test_pie_contribution_uses_adhoc_metric_label() -> None:
# getMetricLabel resolves an adhoc metric to its label; the renamed column
# must match what the chart produces for the same metric.
form_data = {
"metric": {
"expressionType": "SIMPLE",
"column": {"column_name": "sales"},
"aggregate": "SUM",
"label": "Total sales",
},
"groupby": ["c"],
}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="pie")[
"queries"
][0]
assert query["post_processing"][0]["options"] == {
"columns": ["Total sales"],
"rename_columns": ["Total sales__contribution"],
}
def test_non_pie_carries_no_post_processing() -> None:
form_data = {"metrics": ["count"], "groupby": ["c"]}
query = build_query_context_from_form_data(form_data, DATASOURCE, viz_type="table")[
"queries"
][0]
assert "post_processing" not in query
@@ -104,8 +104,6 @@ class TestApplyColumnOperatorsRelationship:
@pytest.mark.parametrize(
"operator",
[
ColumnOperatorEnum.eq,
ColumnOperatorEnum.ne,
ColumnOperatorEnum.in_,
ColumnOperatorEnum.nin,
ColumnOperatorEnum.is_null,
@@ -113,7 +111,9 @@ class TestApplyColumnOperatorsRelationship:
],
)
def test_supported_relationship_operators_dispatch(self, operator):
"""eq/ne/in/nin/is_null/is_not_null all dispatch to .any() variants."""
"""in/nin/is_null/is_not_null all dispatch to .any() variants and
accept a list value. `eq`/`ne` are scalar-only and covered by their
own dedicated tests."""
mock_query = MagicMock()
mock_query.filter.return_value = mock_query
@@ -123,6 +123,49 @@ class TestApplyColumnOperatorsRelationship:
)
assert mock_query.filter.call_count == 1
def test_eq_on_relationship_rejects_list_value(self):
"""`eq` on a relationship column requires a scalar value, same as
`ne`. Passing a list (e.g. value=[1, 2]) would silently compile to
`related_pk == [1, 2]`, which behaves unpredictably across
backends, so it must raise a clear ValueError instead."""
mock_query = MagicMock()
with pytest.raises(ValueError, match="requires a scalar value"):
_SliceDAO.apply_column_operators(
mock_query,
[
ColumnOperator(
col="dashboards", opr=ColumnOperatorEnum.eq, value=[1, 2]
)
],
)
def test_ne_on_relationship_dispatches_to_any(self):
"""`ne` with a scalar value dispatches to the negated .any() variant."""
mock_query = MagicMock()
mock_query.filter.return_value = mock_query
_SliceDAO.apply_column_operators(
mock_query,
[ColumnOperator(col="dashboards", opr=ColumnOperatorEnum.ne, value=42)],
)
assert mock_query.filter.call_count == 1
def test_ne_on_relationship_rejects_list_value(self):
"""`ne` on a relationship column requires a scalar value. Passing a
list (e.g. value=[1, 2]) would silently compile to
`related_pk == [1, 2]`, which behaves unpredictably across
backends, so it must raise a clear ValueError instead."""
mock_query = MagicMock()
with pytest.raises(ValueError, match="requires a scalar value"):
_SliceDAO.apply_column_operators(
mock_query,
[
ColumnOperator(
col="dashboards", opr=ColumnOperatorEnum.ne, value=[1, 2]
)
],
)
@pytest.mark.parametrize(
"operator",
[
+86 -5
View File
@@ -73,6 +73,16 @@ from tests.unit_tests.fixtures.common import dttm # noqa: F401
("DATETIME", types.DateTime, None, GenericDataType.TEMPORAL, True),
("TIMESTAMP", types.TIMESTAMP, None, GenericDataType.TEMPORAL, True),
("TIME", types.Time, None, GenericDataType.TEMPORAL, True),
# Wire-protocol names
("VAR_STRING", types.VARCHAR, None, GenericDataType.STRING, False),
("NEWDECIMAL", DECIMAL, None, GenericDataType.NUMERIC, False),
("TINY", TINYINT, None, GenericDataType.NUMERIC, False),
("SHORT", types.SmallInteger, None, GenericDataType.NUMERIC, False),
("BLOB", types.String, None, GenericDataType.STRING, False),
("TEXT", types.String, None, GenericDataType.STRING, False),
("YEAR", types.Integer, None, GenericDataType.NUMERIC, False),
("ENUM", types.String, None, GenericDataType.STRING, False),
("SET", types.String, None, GenericDataType.STRING, False),
],
)
def test_get_column_spec(
@@ -87,6 +97,50 @@ def test_get_column_spec(
assert_column_spec(spec, native_type, sqla_type, attrs, generic_type, is_dttm)
def test_fetch_data_mutates_decimal_rows_in_tuple_results() -> None:
from superset.db_engine_specs.mysql import MySQLEngineSpec as spec # noqa: N813
newdecimal, var_string = 246, 253
cursor = Mock()
cursor.description = [("amount", newdecimal), ("label", var_string)]
cursor.fetchall.return_value = (("10.50", "Ships"), ("22.30", "Planes"))
# Stub the type_code_map so this test doesn't depend on MySQLdb or
# pymysql being importable in the test environment.
original_type_code_map = spec.type_code_map
spec.type_code_map = {newdecimal: "NEWDECIMAL", var_string: "VAR_STRING"}
try:
data = spec.fetch_data(cursor)
finally:
spec.type_code_map = original_type_code_map
assert data == [(Decimal("10.50"), "Ships"), (Decimal("22.30"), "Planes")]
def test_fetch_data_mutates_duplicate_decimal_column_names() -> None:
from superset.db_engine_specs.mysql import MySQLEngineSpec as spec # noqa: N813
newdecimal, var_string = 246, 253
cursor = Mock()
cursor.description = [
("amount", newdecimal),
("amount", var_string),
("amount", newdecimal),
]
cursor.fetchall.return_value = [("10.50", "not a decimal", "22.30")]
original_type_code_map = spec.type_code_map
spec.type_code_map = {newdecimal: "NEWDECIMAL", var_string: "VAR_STRING"}
try:
data = spec.fetch_data(cursor)
finally:
spec.type_code_map = original_type_code_map
assert data == [(Decimal("10.50"), "not a decimal", Decimal("22.30"))]
@pytest.mark.parametrize(
"target_type,expected_result",
[
@@ -269,7 +323,7 @@ def test_column_type_mutator(
assert spec.fetch_data(mock_cursor) == expected_result
def test_get_datatype_pymysql_fallback():
def test_get_datatype_pymysql_fallback() -> None:
"""get_datatype() falls back to pymysql when MySQLdb is not installed."""
from superset.db_engine_specs.mysql import MySQLEngineSpec
@@ -280,14 +334,14 @@ def test_get_datatype_pymysql_fallback():
try:
# Build a fake pymysql module with constants.FIELD_TYPE
fake_field_type = ModuleType("pymysql.constants.FIELD_TYPE")
fake_field_type.TINY = 1
fake_field_type.VARCHAR = 15
setattr(fake_field_type, "TINY", 1) # noqa: B010
setattr(fake_field_type, "VARCHAR", 15) # noqa: B010
fake_constants = ModuleType("pymysql.constants")
fake_constants.FIELD_TYPE = fake_field_type
setattr(fake_constants, "FIELD_TYPE", fake_field_type) # noqa: B010
fake_pymysql = ModuleType("pymysql")
fake_pymysql.constants = fake_constants
setattr(fake_pymysql, "constants", fake_constants) # noqa: B010
original_import = builtins.__import__
@@ -308,6 +362,33 @@ def test_get_datatype_pymysql_fallback():
MySQLEngineSpec.type_code_map = original_type_code_map
def test_get_datatype_mysqlconnector_fallback() -> None:
"""get_datatype() supports mysql-connector-python without PyMySQL."""
from superset.db_engine_specs.mysql import MySQLEngineSpec
original_type_code_map = MySQLEngineSpec.type_code_map
MySQLEngineSpec.type_code_map = {}
try:
fake_field_type = ModuleType("mysql.connector.constants.FieldType")
setattr(fake_field_type, "NEWDECIMAL", 246) # noqa: B010
fake_constants = ModuleType("mysql.connector.constants")
setattr(fake_constants, "FieldType", fake_field_type) # noqa: B010
original_import = builtins.__import__
def mock_import(name: str, *args: Any, **kwargs: Any) -> Any:
if name in {"MySQLdb", "pymysql"}:
raise ImportError(f"No module named '{name}'")
if name == "mysql.connector.constants":
return fake_constants
return original_import(name, *args, **kwargs)
with patch("builtins.__import__", side_effect=mock_import):
assert MySQLEngineSpec.get_datatype(246) == "NEWDECIMAL"
finally:
MySQLEngineSpec.type_code_map = original_type_code_map
@pytest.mark.parametrize(
("grain", "expected_expression"),
[
@@ -0,0 +1,88 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Unit tests for the legacy-encryption-engine startup warning."""
from typing import Any
from unittest.mock import patch
from superset.initialization import SupersetAppInitializer
def _make_initializer(config: dict[str, Any]) -> SupersetAppInitializer:
"""Build a bare initializer with just the attributes the check needs."""
initializer = SupersetAppInitializer.__new__(SupersetAppInitializer)
initializer.config = config
return initializer
def test_warns_when_engine_unset_defaults_to_legacy_aes() -> None:
"""An absent config value resolves to the legacy engine and warns."""
initializer = _make_initializer({})
with patch.object(initializer, "_log_config_warning") as log_warning:
initializer.check_encryption_engine()
log_warning.assert_called_once()
message = log_warning.call_args.args[0]
assert "aes-gcm" in message
assert "re-encrypt-secrets" in message
def test_warns_when_engine_explicitly_set_to_aes() -> None:
"""An explicit 'aes' value warns the same as the implicit default."""
initializer = _make_initializer({"SQLALCHEMY_ENCRYPTED_FIELD_ENGINE": "aes"})
with patch.object(initializer, "_log_config_warning") as log_warning:
initializer.check_encryption_engine()
log_warning.assert_called_once()
def test_silent_when_engine_is_gcm() -> None:
"""An operator who has already opted into 'aes-gcm' gets no warning."""
initializer = _make_initializer({"SQLALCHEMY_ENCRYPTED_FIELD_ENGINE": "aes-gcm"})
with patch.object(initializer, "_log_config_warning") as log_warning:
initializer.check_encryption_engine()
log_warning.assert_not_called()
def test_silent_when_engine_value_is_unrecognized() -> None:
"""An unrecognized value already fails closed at field construction
(``resolve_encryption_engine``); this check does not pile on a second,
redundant warning for the same misconfiguration.
"""
initializer = _make_initializer({"SQLALCHEMY_ENCRYPTED_FIELD_ENGINE": "bogus"})
with patch.object(initializer, "_log_config_warning") as log_warning:
initializer.check_encryption_engine()
log_warning.assert_not_called()
def test_never_raises_system_exit() -> None:
"""Unlike check_secret_key/check_guest_token_secret/check_async_query_secret,
this check must never refuse to start: the legacy engine is a supported
configuration, not a known-bad placeholder, so blocking startup on it
would turn an opt-in hardening step into a forced-migration outage.
"""
initializer = _make_initializer({})
with patch.object(initializer, "_log_config_warning"):
# Should not raise SystemExit.
initializer.check_encryption_engine()
@@ -197,3 +197,15 @@ class TestPreviewUtilsColumnBuilding:
columns.insert(0, x_axis_config)
assert columns == ["territory", "year"]
def test_build_query_columns_empty_columns_key_keeps_groupby():
"""MCP path: an explicitly empty ``columns`` list no longer shadows ``groupby``.
``_build_query_columns`` delegates to the shared
``superset.common.form_data_query_context.columns_from_form_data``; this pins
the (intentional) behavior change so the export and MCP paths stay in sync.
"""
assert preview_utils._build_query_columns(
{"groupby": ["country"], "columns": []}
) == ["country"]
@@ -608,19 +608,35 @@ def test_setup_user_context_allows_active_user(app) -> None:
# -- Multi-issuer binding guard --
def test_multi_issuer_warns_without_custom_resolver(app, caplog) -> None:
def test_multi_issuer_fails_closed_without_custom_resolver(app) -> None:
"""When multiple issuers are trusted and no issuer-aware resolver is set,
a WARNING is emitted about unbound (non-issuer-scoped) user resolution."""
import logging
resolution fails closed (raises) instead of returning a user via an
unbound (non-issuer-scoped) lookup."""
from superset.mcp_service.mcp_config import MCPAuthConfigError
mock_user = _make_mock_user("alice")
token = _make_access_token(claims={"sub": "alice", "iss": "issuer-a"})
with app.app_context():
app.config["MCP_JWT_ISSUER"] = ["issuer-a", "issuer-b"]
try:
with patch(
"fastmcp.server.dependencies.get_access_token", return_value=token
):
with pytest.raises(MCPAuthConfigError):
_resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
def test_single_issuer_does_not_fail_closed(app) -> None:
"""A single configured issuer is safe and resolves normally."""
mock_user = _make_mock_user("alice")
token = _make_access_token(claims={"sub": "alice", "iss": "issuer-a"})
with app.app_context():
app.config["MCP_JWT_ISSUER"] = "issuer-a"
try:
with (
caplog.at_level(logging.WARNING),
patch(
"fastmcp.server.dependencies.get_access_token", return_value=token
),
@@ -633,44 +649,12 @@ def test_multi_issuer_warns_without_custom_resolver(app, caplog) -> None:
finally:
app.config.pop("MCP_JWT_ISSUER", None)
assert result is not None
warnings = [r.message for r in caplog.records if r.levelno == logging.WARNING]
assert any("Multiple JWT issuers are trusted" in m for m in warnings)
assert result is mock_user
def test_single_issuer_does_not_warn(app, caplog) -> None:
"""A single configured issuer is safe and emits no multi-issuer warning."""
import logging
mock_user = _make_mock_user("alice")
token = _make_access_token(claims={"sub": "alice", "iss": "issuer-a"})
with app.app_context():
app.config["MCP_JWT_ISSUER"] = "issuer-a"
try:
with (
caplog.at_level(logging.WARNING),
patch(
"fastmcp.server.dependencies.get_access_token", return_value=token
),
patch(
"superset.mcp_service.auth.load_user_with_relationships",
return_value=mock_user,
),
):
_resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
warnings = [r.message for r in caplog.records if r.levelno == logging.WARNING]
assert not any("Multiple JWT issuers are trusted" in m for m in warnings)
def test_multi_issuer_no_warn_with_custom_resolver(app, caplog) -> None:
"""A custom MCP_USER_RESOLVER (assumed issuer-aware) suppresses the
multi-issuer warning."""
import logging
def test_multi_issuer_does_not_fail_closed_with_custom_resolver(app) -> None:
"""A custom MCP_USER_RESOLVER (assumed issuer-aware) is exempt from the
multi-issuer fail-closed guard and resolves normally."""
mock_user = _make_mock_user("alice")
token = _make_access_token(claims={"sub": "alice", "iss": "issuer-a"})
@@ -679,7 +663,6 @@ def test_multi_issuer_no_warn_with_custom_resolver(app, caplog) -> None:
app.config["MCP_USER_RESOLVER"] = MagicMock(return_value="alice")
try:
with (
caplog.at_level(logging.WARNING),
patch(
"fastmcp.server.dependencies.get_access_token", return_value=token
),
@@ -688,10 +671,9 @@ def test_multi_issuer_no_warn_with_custom_resolver(app, caplog) -> None:
return_value=mock_user,
),
):
_resolve_user_from_jwt_context(app)
result = _resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
app.config.pop("MCP_USER_RESOLVER", None)
warnings = [r.message for r in caplog.records if r.levelno == logging.WARNING]
assert not any("Multiple JWT issuers are trusted" in m for m in warnings)
assert result is mock_user
@@ -0,0 +1,201 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Identity-binding behavior of ``_resolve_user_from_jwt_context`` when more
than one JWT issuer is trusted (``MCP_JWT_ISSUER`` configured as a list).
The default resolver (``default_user_resolver``) derives a Superset username
from token claims (``preferred_username`` / ``username`` / ``email`` / ``sub``)
without folding the token's ``iss`` claim into the lookup key. Without an
issuer-aware ``MCP_USER_RESOLVER``, two tokens minted by different trusted
issuers but sharing a username claim would otherwise resolve to the
identical Superset user, since the DB lookup would be performed on username
alone.
To prevent that, ``_resolve_user_from_jwt_context`` fails closed (raises
``MCPAuthConfigError``) when more than one issuer is trusted and no custom
``MCP_USER_RESOLVER`` is configured, instead of proceeding with a lookup that
isn't scoped to the trusted issuer. An operator-supplied, issuer-aware
resolver (e.g. one deriving a compound iss+sub identity) is unaffected and
continues to resolve normally.
"""
from unittest.mock import MagicMock, patch
import pytest
from superset.mcp_service.auth import _resolve_user_from_jwt_context
from superset.mcp_service.mcp_config import (
MCPAuthConfigError,
validate_multi_issuer_user_resolver,
)
def _make_mock_user(username: str = "alice") -> MagicMock:
user = MagicMock()
user.username = username
user.roles = []
user.groups = []
return user
def _make_access_token(claims: dict[str, str]) -> MagicMock:
token = MagicMock()
token.claims = claims
token.client_id = ""
token.scopes = []
for attr in ("subject", "payload"):
delattr(token, attr)
return token
def test_multi_issuer_without_custom_resolver_fails_closed(app) -> None:
"""Multiple trusted issuers with no issuer-aware ``MCP_USER_RESOLVER``
refuses to resolve an identity rather than performing a username-only
lookup that isn't scoped to the trusted issuer."""
token = _make_access_token(
claims={"sub": "alice", "iss": "https://issuer-a.example.com"}
)
with app.app_context():
app.config["MCP_JWT_ISSUER"] = [
"https://issuer-a.example.com",
"https://issuer-b.example.com",
]
app.config.pop("MCP_USER_RESOLVER", None)
try:
with patch(
"fastmcp.server.dependencies.get_access_token",
return_value=token,
):
with pytest.raises(MCPAuthConfigError):
_resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
def test_duplicate_issuer_entries_do_not_fail_closed(app) -> None:
"""A list/tuple naming the same issuer more than once is one logical
issuer, not a multi-issuer trust configuration -- deduplicate before
counting so ``MCP_JWT_ISSUER = ["a", "a"]`` doesn't trigger the guard
the way ``["a", "b"]`` correctly does."""
token = _make_access_token(
claims={"sub": "alice", "iss": "https://issuer-a.example.com"}
)
shared_user = _make_mock_user("alice")
with app.app_context():
app.config["MCP_JWT_ISSUER"] = [
"https://issuer-a.example.com",
"https://issuer-a.example.com",
]
app.config.pop("MCP_USER_RESOLVER", None)
try:
with (
patch(
"fastmcp.server.dependencies.get_access_token",
return_value=token,
),
patch(
"superset.mcp_service.auth.load_user_with_relationships",
return_value=shared_user,
),
):
result = _resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
assert result is shared_user
def test_unhashable_issuer_entries_fail_closed_not_typeerror(app) -> None:
"""A malformed ``MCP_JWT_ISSUER`` containing unhashable entries (e.g. a
nested list from a bad config template) must still fail closed with
``MCPAuthConfigError``, not raise a bare ``TypeError`` from ``set()``.
A plain ``TypeError`` isn't caught by the ``except MCPAuthConfigError``
re-raise in ``_create_auth_provider``, so it would be swallowed by the
trailing ``except Exception`` there and start the server unauthenticated."""
with app.app_context():
app.config["MCP_JWT_ISSUER"] = [["issuer-a"], ["issuer-b"]]
app.config.pop("MCP_USER_RESOLVER", None)
try:
with pytest.raises(MCPAuthConfigError):
validate_multi_issuer_user_resolver(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
def test_multi_issuer_with_custom_resolver_resolves_normally(app) -> None:
"""An operator-supplied, issuer-aware ``MCP_USER_RESOLVER`` is unaffected
by the multi-issuer guard: legitimate multi-issuer deployments that
provide their own resolver continue to resolve users normally."""
shared_user = _make_mock_user("alice")
token_from_issuer_a = _make_access_token(
claims={"sub": "alice", "iss": "https://issuer-a.example.com"}
)
token_from_issuer_b = _make_access_token(
claims={"sub": "alice", "iss": "https://issuer-b.example.com"}
)
# A stand-in issuer-aware resolver: derives a compound iss+sub identity
# instead of the default username/email-only lookup.
def _issuer_aware_resolver(_app: object, access_token: MagicMock) -> str:
claims = access_token.claims
return f"{claims['iss']}:{claims['sub']}"
with app.app_context():
app.config["MCP_JWT_ISSUER"] = [
"https://issuer-a.example.com",
"https://issuer-b.example.com",
]
app.config["MCP_USER_RESOLVER"] = _issuer_aware_resolver
try:
with (
patch(
"fastmcp.server.dependencies.get_access_token",
return_value=token_from_issuer_a,
),
patch(
"superset.mcp_service.auth.load_user_with_relationships",
return_value=shared_user,
) as mock_load_a,
):
result_a = _resolve_user_from_jwt_context(app)
with (
patch(
"fastmcp.server.dependencies.get_access_token",
return_value=token_from_issuer_b,
),
patch(
"superset.mcp_service.auth.load_user_with_relationships",
return_value=shared_user,
) as mock_load_b,
):
result_b = _resolve_user_from_jwt_context(app)
finally:
app.config.pop("MCP_JWT_ISSUER", None)
app.config.pop("MCP_USER_RESOLVER", None)
assert result_a is shared_user
assert result_b is shared_user
# The custom resolver's compound iss+sub key is what reaches the DB
# lookup, not a bare username — confirming the guard doesn't interfere
# with a resolver that already binds identity to the issuer.
mock_load_a.assert_called_once_with("https://issuer-a.example.com:alice")
mock_load_b.assert_called_once_with("https://issuer-b.example.com:alice")
@@ -0,0 +1,192 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Ordering of the zero-width-character removal step relative to the
keyword/pattern denylist checks and HTML-entity decoding in
``sanitize_user_input``, ``sanitize_filter_value``, and
``sanitize_sql_expression``.
All three functions now run ``_remove_dangerous_unicode`` *both* before and
after HTML-entity decoding happens (entity decoding is internal to
``_strip_html_tags`` for the first two functions, and an explicit loop in
the third). Zero-width characters (e.g. U+200B ZERO WIDTH SPACE) can sit in
the middle of a denylisted keyword and break a ``\\b(KEYWORD)\\b`` style
match, whether they appear as a raw character or as an HTML entity
(``&#8203;`` / ``&#x200b;``) that only becomes the raw character after
decoding. Stripping only before decoding catches the raw form but lets an
entity-encoded one survive decoding and reach the denylist checks intact;
stripping again after decoding closes that gap.
"""
import pytest
from superset.mcp_service.utils.sanitization import (
sanitize_filter_value,
sanitize_sql_expression,
sanitize_user_input,
)
ZERO_WIDTH_CHARS = [
"", # ZERO WIDTH SPACE
"", # ZERO WIDTH NON-JOINER
"", # ZERO WIDTH JOINER
"", # ZERO WIDTH NO-BREAK SPACE / BOM
]
# Decimal and hex HTML entity encodings of ZERO WIDTH SPACE (U+200B) --
# distinct textual forms that both decode to the same raw character.
ZERO_WIDTH_SPACE_ENTITIES = [
"&#8203;",
"&#x200b;",
"&#x200B;",
]
# --- sanitize_user_input(check_sql_keywords=True) ---
def test_sanitize_user_input_rejects_unsplit_sql_keyword():
"""Baseline: an un-obfuscated denylisted keyword is rejected as documented."""
with pytest.raises(ValueError, match="unsafe SQL keywords"):
sanitize_user_input("DROP TABLE users", "Column name", check_sql_keywords=True)
@pytest.mark.parametrize("zwc", ZERO_WIDTH_CHARS)
def test_sanitize_user_input_rejects_split_keyword_payload(zwc):
"""
A zero-width character placed inside the denylisted keyword ``DROP``
no longer defeats the ``\\b(DROP|...)\\b`` regex: unicode canonicalization
now runs before the keyword check, so the reconstructed keyword is what
the denylist check sees and a ``ValueError`` is raised, same as the
unobfuscated baseline.
"""
obfuscated = f"DR{zwc}OP TABLE users"
with pytest.raises(ValueError, match="unsafe SQL keywords"):
sanitize_user_input(obfuscated, "Column name", check_sql_keywords=True)
@pytest.mark.parametrize("entity", ZERO_WIDTH_SPACE_ENTITIES)
def test_sanitize_user_input_rejects_entity_encoded_split_keyword_payload(entity):
"""
An HTML-entity-encoded zero-width character only becomes the raw
character once ``_strip_html_tags`` decodes it -- if nothing re-checks
for dangerous Unicode after that decoding step, the reconstructed
keyword reaches the denylist check without ever having been
canonicalized, and the entity-encoded payload slips through where the
raw-character form (above) is caught. Stripping again after decoding
closes that gap.
"""
obfuscated = f"DR{entity}OP TABLE users"
with pytest.raises(ValueError, match="unsafe SQL keywords"):
sanitize_user_input(obfuscated, "Column name", check_sql_keywords=True)
def test_sanitize_user_input_still_removes_a_raw_dangerous_unicode_char():
"""
Regression check: the second Unicode-strip pass must not stop the
function from silently removing a dangerous character that arrives
HTML-entity-encoded and isn't part of a denylisted keyword -- e.g.
U+2028 LINE SEPARATOR, a statement terminator on some SQL drivers per
this module's own docstring. This should sanitize to plain text, not
raise, matching the documented "removes dangerous Unicode" contract.
"""
assert sanitize_user_input("a&#8232;b", "Column name") == "ab"
def test_sanitize_user_input_tag_only_payload_reduces_to_empty_string():
"""
Regression check: a tag-heavy input that nh3 legitimately strips down
to "" (e.g. "<script>...</script>") must sanitize to an empty string,
not raise -- there is deliberately no emptiness recheck immediately
after ``_strip_html_tags`` (only before it), since reducing to "" here
is the correct sanitized result of intentionally-stripped content, not
an error case like an all-zero-width input would be.
"""
assert (
sanitize_user_input(
"<script>alert(1)</script>", "Column name", allow_empty=True
)
== ""
)
# --- sanitize_filter_value ---
def test_sanitize_filter_value_rejects_unsplit_union_select():
"""Baseline: an un-obfuscated ``UNION SELECT`` pattern is rejected as documented."""
with pytest.raises(ValueError, match="malicious SQL patterns"):
sanitize_filter_value("UNION SELECT password FROM users")
@pytest.mark.parametrize("zwc", ZERO_WIDTH_CHARS)
def test_sanitize_filter_value_rejects_split_union_select_payload(zwc):
"""
Same ordering fix as ``sanitize_user_input``: splitting ``UNION`` with a
zero-width character no longer defeats the ``UNION\\s+SELECT`` pattern
check, since canonicalization now runs first and the pattern check sees
the reconstructed value.
"""
obfuscated = f"UNI{zwc}ON SELECT password FROM users" # noqa: S608
with pytest.raises(ValueError, match="malicious SQL patterns"):
sanitize_filter_value(obfuscated)
@pytest.mark.parametrize("entity", ZERO_WIDTH_SPACE_ENTITIES)
def test_sanitize_filter_value_rejects_entity_encoded_split_union_select_payload(
entity,
):
"""Entity-encoded counterpart of the raw zero-width case above."""
obfuscated = f"UNI{entity}ON SELECT password FROM users" # noqa: S608
with pytest.raises(ValueError, match="malicious SQL patterns"):
sanitize_filter_value(obfuscated)
# --- sanitize_sql_expression ---
def test_sanitize_sql_expression_rejects_same_split_keyword_payload():
"""
``sanitize_sql_expression`` strips dangerous Unicode both before and
after decoding HTML entities, so the same zero-width obfuscation
technique used against the two functions above does not defeat it
either: the value is canonicalized, decoded, then canonicalized again,
and the reconstructed ``DROP`` keyword is caught by the denylist check
as normal.
"""
obfuscated = f"DR{ZERO_WIDTH_CHARS[0]}OP TABLE users"
with pytest.raises(ValueError, match="disallowed SQL keyword"):
sanitize_sql_expression(obfuscated, "SQL expression")
@pytest.mark.parametrize("entity", ZERO_WIDTH_SPACE_ENTITIES)
def test_sanitize_sql_expression_rejects_entity_encoded_split_keyword_payload(entity):
"""
Entity-encoded counterpart of the raw zero-width case above: decoding
happens between the two Unicode-strip passes, so a zero-width character
that only exists as an HTML entity until decoded is still caught.
"""
obfuscated = f"DR{entity}OP TABLE users"
with pytest.raises(ValueError, match="disallowed SQL keyword"):
sanitize_sql_expression(obfuscated, "SQL expression")
@@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import importlib
import sqlalchemy.orm as orm
def test_add_implicit_tags_script_leaves_mappers_configurable() -> None:
"""Importing the migration script must not poison the mapper registry.
Alembic imports every version script when it walks revision history (the
unit-test app fixture triggers this via its pending-migration check), so a
script whose throwaway declarative models cannot be configured breaks
``configure_mappers()`` process-wide for whichever test happens to trigger
mapper configuration next.
"""
importlib.import_module(
"superset.migrations.versions.2018-07-26_11-10_c82ee8a39623_add_implicit_tags"
)
orm.configure_mappers()
@@ -282,6 +282,7 @@ def test_send_treats_redirect_as_failure(monkeypatch, mock_header_data) -> None:
config = {
"ALERT_REPORTS_WEBHOOK_HTTPS_ONLY": True,
"ALERT_REPORTS_WEBHOOK_ALLOW_INTERNAL_HOSTS": True,
"ALERT_REPORTS_WEBHOOK_TIMEOUT": 60,
}
class MockResponse:
@@ -304,6 +305,58 @@ def test_send_treats_redirect_as_failure(monkeypatch, mock_header_data) -> None:
webhook_notification.send()
def test_send_forwards_configured_timeout(monkeypatch, mock_header_data) -> None:
"""
send() forwards ALERT_REPORTS_WEBHOOK_TIMEOUT to requests.post so the
call can't hang forever if the webhook target is unreachable.
"""
from superset.reports.models import ReportRecipients, ReportRecipientType
from superset.reports.notifications.base import NotificationContent
content = NotificationContent(
name="test alert", header_data=mock_header_data, description="Test description"
)
webhook_notification = WebhookNotification(
recipient=ReportRecipients(
type=ReportRecipientType.WEBHOOK,
recipient_config_json='{"target": "https://example.com/webhook"}',
),
content=content,
)
class MockCurrentApp:
config = {
"ALERT_REPORTS_WEBHOOK_HTTPS_ONLY": True,
"ALERT_REPORTS_WEBHOOK_ALLOW_INTERNAL_HOSTS": True,
"ALERT_REPORTS_WEBHOOK_TIMEOUT": 45,
}
class MockResponse:
status_code = 200
text = ""
captured_kwargs: dict[str, object] = {}
def fake_post(*args, **kwargs):
captured_kwargs.update(kwargs)
return MockResponse()
monkeypatch.setattr(
"superset.reports.notifications.webhook.current_app", MockCurrentApp
)
monkeypatch.setattr(
"superset.reports.notifications.webhook.feature_flag_manager.is_feature_enabled",
lambda flag: True,
)
monkeypatch.setattr(
"superset.reports.notifications.webhook.requests.post", fake_post
)
webhook_notification.send()
assert captured_kwargs["timeout"] == 45
def _make_webhook(mock_header_data) -> WebhookNotification:
from superset.reports.models import ReportRecipients, ReportRecipientType
from superset.reports.notifications.base import NotificationContent
@@ -330,6 +383,7 @@ def _allow_internal_app() -> type:
config = {
"ALERT_REPORTS_WEBHOOK_HTTPS_ONLY": True,
"ALERT_REPORTS_WEBHOOK_ALLOW_INTERNAL_HOSTS": True,
"ALERT_REPORTS_WEBHOOK_TIMEOUT": 60,
}
return MockCurrentApp
@@ -0,0 +1,65 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from pathlib import Path
from typing import Any
import yaml
from scripts import change_detector
WORKFLOW_PATH = (
Path(__file__).resolve().parents[2]
/ ".github/workflows/scheduled-docker-image-refresh.yml"
)
def load_workflow() -> dict[str, Any]:
return yaml.safe_load(WORKFLOW_PATH.read_text())
def test_scheduled_refresh_uses_current_workflow_actions() -> None:
workflow = load_workflow()
steps = {step["name"]: step for step in workflow["jobs"]["docker-rebuild"]["steps"]}
action_checkout = steps["Checkout workflow actions"]
assert action_checkout["with"]["ref"] == "${{ github.sha }}"
assert action_checkout["with"]["path"] == "workflow-source"
assert (
steps["Setup Docker Environment"]["uses"]
== "./workflow-source/.github/actions/setup-docker"
)
assert (
steps["Setup supersetbot"]["uses"]
== "./workflow-source/.github/actions/setup-supersetbot/"
)
def test_scheduled_refresh_notifier_uses_existing_labels() -> None:
workflow = load_workflow()
notify_step = workflow["jobs"]["notify-on-failure"]["steps"][0]
assert '--label "infra:container"' in notify_step["run"]
assert '--label "#bug"' in notify_step["run"]
assert '--label "bug"' not in notify_step["run"]
def test_scheduled_refresh_changes_trigger_python_tests() -> None:
assert change_detector.detect_changes(
[".github/workflows/scheduled-docker-image-refresh.yml"],
change_detector.PATTERNS["python"],
)
+79 -1
View File
@@ -17,13 +17,16 @@
# pylint: disable=import-outside-toplevel, invalid-name, unused-argument, too-many-locals
import json # noqa: TID251
from unittest.mock import MagicMock
from typing import Any
from unittest.mock import MagicMock, patch
from urllib.parse import parse_qs, urlparse
from uuid import UUID
import pytest
from freezegun import freeze_time
from pytest_mock import MockerFixture
from sqlalchemy import text
from sqlalchemy.orm import Session
from superset.app import SupersetApp
from superset.common.db_query_status import QueryStatus
@@ -608,6 +611,81 @@ def test_get_predicates_for_table(mocker: MockerFixture) -> None:
)
def test_get_predicates_for_table_null_schema_dataset(session: Session) -> None:
"""
A dataset stored with a NULL schema is scoped to the database's default
schema, mirroring the existing null-catalog fallback.
A query resolving to that default schema must find the dataset, so its RLS
predicates are applied instead of being silently dropped. A query against a
different schema must not, since the null-schema dataset doesn't describe it.
"""
from superset.connectors.sqla.models import SqlaTable
SqlaTable.metadata.create_all(session.get_bind())
database = Database(database_name="rls_db", sqlalchemy_uri="sqlite://")
# registered without an explicit schema, e.g. via the dataset API
dataset = SqlaTable(table_name="t1", schema=None, catalog=None, database=database)
session.add_all([database, dataset])
session.flush()
with (
patch.object(
SqlaTable, "get_sqla_row_level_filters", return_value=[text("c1 = 1")]
),
patch.object(Database, "get_default_schema", return_value="public"),
):
assert get_predicates_for_table(
Table("t1", "public", None), database, None
) == ["c1 = 1"]
assert (
get_predicates_for_table(Table("t1", "sales", None), database, None) == []
)
def test_get_predicates_for_table_prefers_exact_schema_match(session: Session) -> None:
"""
A dataset stored without a schema and one stored with the default schema can
coexist for the same table. The exact match must win, and the lookup must stay
unambiguous rather than treating both rows as candidates for a single dataset.
"""
from superset.connectors.sqla.models import SqlaTable
SqlaTable.metadata.create_all(session.get_bind())
database = Database(database_name="rls_db_exact", sqlalchemy_uri="sqlite://")
session.add_all(
[
database,
SqlaTable(table_name="t1", schema=None, catalog=None, database=database),
SqlaTable(
table_name="t1", schema="public", catalog=None, database=database
),
]
)
session.flush()
def row_level_filters(
self: Any, include_global_guest_rls: bool = True
) -> list[Any]:
return [text(f"c1 = '{self.schema}'")]
with (
patch.object(
SqlaTable,
"get_sqla_row_level_filters",
autospec=True,
side_effect=row_level_filters,
),
patch.object(Database, "get_default_schema", return_value="public"),
):
assert get_predicates_for_table(
Table("t1", "public", None), database, None
) == ["c1 = 'public'"]
def test_get_predicates_for_table_excludes_self(mocker: MockerFixture) -> None:
"""
When ``exclude_dataset_id`` is supplied, the lookup query must add an
@@ -20,7 +20,7 @@ import glob
import os
import tempfile
from collections.abc import Iterator
from contextlib import ExitStack
from contextlib import contextmanager, ExitStack
from typing import Any
from unittest import mock
@@ -182,6 +182,446 @@ def test_chart_without_query_context_is_skipped(mocks: dict[str, Any]) -> None:
}
@pytest.mark.parametrize(
"raw_context",
[
"", # blank
"null", # parses to None
"{}", # object with no queries
'{"queries": []}', # object with an empty queries list
'{"queries": "oops"}', # queries present but not a list
'{"queries": {"a": 1}}', # queries present but not a list
"not valid json", # unparseable
],
)
def test_chart_with_empty_query_context_is_skipped(
mocks: dict[str, Any], raw_context: str
) -> None:
# A present-but-empty/unusable query context is treated the same as a
# missing one: the chart is listed under "no query context" and the export
# continues, rather than raising mid-export and landing in the general bucket.
good = _chart(10, "Good")
empty = _chart(20, "Empty")
empty.query_context = raw_context
mocks["get_charts_in_layout_order"].return_value = [good, empty]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
_, kwargs = mocks["email"].build_success_email.call_args
assert kwargs["errored"] == {mocks["email"].ERROR_NO_QUERY_CONTEXT: ["20 - Empty"]}
# The empty chart is skipped before any query runs; only the good one runs.
mocks["ChartDataCommand"].return_value.run.assert_called_once()
def test_empty_query_context_rebuilt_from_form_data_for_eligible_viz(
mocks: dict[str, Any],
) -> None:
# An eligible viz type (table) with no saved query context is rebuilt from
# its form data and exported instead of being skipped.
good = _chart(10, "Good")
rebuilt = _chart(20, "Rebuilt", viz_type="table")
rebuilt.query_context = None
rebuilt.params = json.dumps({"groupby": ["country"], "metrics": ["count"]})
rebuilt.datasource_id = 5
rebuilt.datasource_type = "table"
mocks["get_charts_in_layout_order"].return_value = [good, rebuilt]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
_, kwargs = mocks["email"].build_success_email.call_args
assert kwargs["errored"] == {}
# Both charts ran a query (the saved one and the rebuilt one).
assert mocks["ChartDataCommand"].return_value.run.call_count == 2
def test_empty_query_context_ineligible_viz_is_skipped(
mocks: dict[str, Any],
) -> None:
# A viz type outside the rebuild allowlist (mixed_timeseries — a multi-query
# chart the generic rebuild can't reproduce) is skipped, not exported wrong.
good = _chart(10, "Good")
ineligible = _chart(20, "Ineligible", viz_type="mixed_timeseries")
ineligible.query_context = None
ineligible.params = json.dumps({"groupby": ["x"], "metrics": ["count"]})
ineligible.datasource_id = 5
mocks["get_charts_in_layout_order"].return_value = [good, ineligible]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
_, kwargs = mocks["email"].build_success_email.call_args
assert kwargs["errored"] == {
mocks["email"].ERROR_NO_QUERY_CONTEXT: ["20 - Ineligible"]
}
mocks["ChartDataCommand"].return_value.run.assert_called_once()
def _rebuildable_chart(
viz_type: str = "table",
form_data: dict[str, Any] | None = None,
) -> mock.MagicMock:
"""A chart with no saved query context whose form data can be rebuilt."""
fd = form_data or {"groupby": ["country"], "metrics": ["count"]}
chart = _chart(20, "Rebuilt", has_context=False, viz_type=viz_type)
chart.params = json.dumps(fd)
chart.datasource_id = 5
chart.datasource_type = "table"
# Mirror Slice.form_data, which injects viz_type + the "id__type" datasource.
chart.form_data = {
**fd,
"slice_id": 20,
"viz_type": viz_type,
"datasource": "5__table",
}
return chart
@contextmanager
def _builder_hook(builder: Any) -> Iterator[None]:
"""Patch current_app so EXCEL_EXPORT_QUERY_CONTEXT_BUILDER resolves to builder."""
from superset.tasks import export_dashboard_excel as module
fake_app = mock.MagicMock()
fake_app.config.get.side_effect = lambda key, default=None: (
builder if key == "EXCEL_EXPORT_QUERY_CONTEXT_BUILDER" else default
)
# Real values for the keys the task subscripts directly, so a full export can
# run under the hook (a MagicMock ttl would blow up building the link expiry).
fake_app.config.__getitem__.side_effect = {
"EXCEL_EXPORT_S3_BUCKET": "bucket",
"EXCEL_EXPORT_S3_KEY_PREFIX": "dashboard-exports/",
"EXCEL_EXPORT_LINK_TTL_SECONDS": 3600,
}.__getitem__
with mock.patch.object(module, "current_app", fake_app):
yield
def test_builder_hook_context_is_used_for_any_viz_type() -> None:
# A configured builder can supply a context for a viz type outside the
# built-in allowlist (pivot_table_v2), and is called with the chart's form data.
from superset.tasks import export_dashboard_excel as module
ctx = {
"datasource": {"id": 5, "type": "table"},
"queries": [{"metrics": ["count"]}],
}
builder = mock.MagicMock(return_value=ctx)
chart = _rebuildable_chart(viz_type="pivot_table_v2")
with _builder_hook(builder):
result = module._resolve_query_context(chart)
assert result == ctx
builder.assert_called_once_with(chart.form_data)
def test_builder_hook_none_falls_through_to_builtin_rebuild() -> None:
# When the builder returns None (can't build faithfully) the export falls
# through to the built-in rebuild, so an allowlisted table is unaffected.
from superset.tasks import export_dashboard_excel as module
builder = mock.MagicMock(return_value=None)
chart = _rebuildable_chart(viz_type="table")
with _builder_hook(builder):
result = module._resolve_query_context(chart)
builder.assert_called_once_with(chart.form_data)
assert result is not None
assert result["queries"][0]["metrics"] == ["count"]
@pytest.mark.parametrize(
"built",
[
{},
{"queries": []},
"not-a-dict",
42,
# ``queries`` present but not a list: truthy, yet unusable downstream.
{"queries": "oops"},
{"queries": {"a": 1}},
],
)
def test_builder_hook_malformed_result_falls_through(built: Any) -> None:
# A stub / empty / malformed builder result is treated as "not built" and
# falls through to the built-in rebuild rather than shipping an empty context.
from superset.tasks import export_dashboard_excel as module
builder = mock.MagicMock(return_value=built)
chart = _rebuildable_chart(viz_type="table")
with _builder_hook(builder):
result = module._resolve_query_context(chart)
builder.assert_called_once_with(chart.form_data)
assert result is not None
assert result["queries"][0]["metrics"] == ["count"]
def test_builder_hook_exception_falls_through() -> None:
# A raising builder (e.g. sidecar down) must not fail the chart; the export
# falls through to the built-in rebuild and no exception escapes.
from superset.tasks import export_dashboard_excel as module
builder = mock.MagicMock(side_effect=RuntimeError("sidecar down"))
chart = _rebuildable_chart(viz_type="table")
with _builder_hook(builder):
result = module._resolve_query_context(chart)
builder.assert_called_once_with(chart.form_data)
assert result is not None
assert result["queries"][0]["metrics"] == ["count"]
def test_builder_hook_soft_time_limit_propagates() -> None:
# A soft timeout raised while the builder is in flight is a task-level signal,
# not a builder failure: it must escape _resolve_query_context so the export
# aborts cleanly, rather than being swallowed by the broad fall-through guard.
from superset.tasks import export_dashboard_excel as module
builder = mock.MagicMock(side_effect=SoftTimeLimitExceeded())
chart = _rebuildable_chart(viz_type="table")
with _builder_hook(builder), pytest.raises(SoftTimeLimitExceeded):
module._resolve_query_context(chart)
builder.assert_called_once_with(chart.form_data)
def test_no_builder_hook_leaves_builtin_behavior_unchanged() -> None:
# With no builder configured, an allowlisted chart is rebuilt and an
# ineligible one is skipped — identical to the pre-hook behavior.
from superset.tasks import export_dashboard_excel as module
with _builder_hook(None):
table = module._resolve_query_context(_rebuildable_chart(viz_type="table"))
ineligible = module._resolve_query_context(
_rebuildable_chart(viz_type="mixed_timeseries")
)
assert table is not None
assert table["queries"][0]["metrics"] == ["count"]
assert ineligible is None
def test_saved_context_short_circuits_builder_hook() -> None:
# A saved query context wins over the builder hook, which is never called.
from superset.tasks import export_dashboard_excel as module
builder = mock.MagicMock(return_value={"queries": [{"from": "hook"}]})
chart = _rebuildable_chart(viz_type="table")
chart.query_context = json.dumps({"queries": [{"from": "saved"}]})
with _builder_hook(builder):
result = module._resolve_query_context(chart)
assert result == {"queries": [{"from": "saved"}]}
builder.assert_not_called()
def test_builder_hook_result_is_not_mutated_across_charts(
mocks: dict[str, Any],
) -> None:
# Dashboard filters are applied by mutating queries[*] in place. A builder
# that memoizes or caches its return value would otherwise accumulate isExtra
# filters chart after chart (and export after export), progressively
# over-filtering, so the hook's payload must be copied before use.
shared_context: dict[str, Any] = {
"datasource": {"id": 5, "type": "table"},
"queries": [{"metrics": ["count"], "filters": []}],
}
builder = mock.MagicMock(return_value=shared_context)
first = _rebuildable_chart(viz_type="pivot_table_v2")
first.id = 10
second = _rebuildable_chart(viz_type="pivot_table_v2")
second.id = 20
mocks["get_charts_in_layout_order"].return_value = [first, second]
def _fresh_filter_context(**kwargs: Any) -> mock.MagicMock:
# Resolved per chart in production, and consumed destructively by
# apply_dashboard_filter_context, so it must not be shared here.
context = mock.MagicMock()
context.extra_form_data = {
"filters": [{"col": "country", "op": "IN", "val": ["US"]}]
}
return context
mocks["get_dashboard_filter_context"].side_effect = _fresh_filter_context
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
with _builder_hook(builder):
_run()
loads = mocks["ChartDataQueryContextSchema"].return_value.load.call_args_list
assert len(loads) == 2
for call in loads:
assert call.args[0]["queries"][0]["filters"] == [
{"col": "country", "op": "IN", "val": ["US"], "isExtra": True}
]
# The builder's own object is untouched, so a cached return stays reusable.
assert shared_context["queries"][0]["filters"] == []
@pytest.mark.parametrize(
("params", "datasource_id"),
[
("not valid json", 5), # params don't parse → cannot rebuild
("null", 5), # params parse to a non-object → cannot rebuild
('{"groupby": ["x"]}', None), # no datasource to point the query at
],
)
def test_eligible_viz_skipped_when_form_data_unusable(
mocks: dict[str, Any], params: str, datasource_id: int | None
) -> None:
# Even for an allowlisted viz type, a rebuild is only attempted when the form
# data is a usable object and a datasource is known; otherwise the chart is
# skipped rather than raising.
good = _chart(10, "Good")
bad = _chart(20, "Bad", viz_type="table")
bad.query_context = None
bad.params = params
bad.datasource_id = datasource_id
bad.datasource_type = "table"
mocks["get_charts_in_layout_order"].return_value = [good, bad]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
_, kwargs = mocks["email"].build_success_email.call_args
assert kwargs["errored"] == {mocks["email"].ERROR_NO_QUERY_CONTEXT: ["20 - Bad"]}
# Only the good chart ran a query; the unusable one never reached execution.
mocks["ChartDataCommand"].return_value.run.assert_called_once()
@pytest.mark.parametrize(
"form_data",
[
{"groupby": ["c"], "metrics": ["m"], "time_compare": ["1 year ago"]},
{"groupby": ["c"], "metrics": ["m"], "rolling_type": "mean"},
{"groupby": ["c"], "metrics": ["m"], "resample_rule": "1D"},
{"groupby": ["c"], "metrics": ["m"], "percent_metrics": ["pct"]},
{"groupby": ["c"], "metrics": ["m"], "show_totals": True},
{"columns": ["a", "b"], "aggregation": "raw"},
],
)
def test_eligible_viz_skipped_when_form_data_needs_post_processing(
mocks: dict[str, Any], form_data: dict[str, Any]
) -> None:
# An allowlisted viz type whose form data relies on post-processing or extra
# queries the single-query rebuild can't reproduce (time comparison, rolling,
# resample, raw aggregation, a totals row) is skipped rather than exported
# with data that differs from the chart.
good = _chart(10, "Good")
fancy = _chart(20, "Fancy", viz_type="table")
fancy.query_context = None
fancy.params = json.dumps(form_data)
fancy.datasource_id = 5
fancy.datasource_type = "table"
mocks["get_charts_in_layout_order"].return_value = [good, fancy]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
_, kwargs = mocks["email"].build_success_email.call_args
assert kwargs["errored"] == {mocks["email"].ERROR_NO_QUERY_CONTEXT: ["20 - Fancy"]}
mocks["ChartDataCommand"].return_value.run.assert_called_once()
def test_rebuilt_query_context_payload_carries_query_shape(
mocks: dict[str, Any],
) -> None:
# Assert the actual payload handed to ChartDataQueryContextSchema().load for a
# rebuilt chart: columns, filters, ordering and granularity must survive so the
# exported data matches the chart (not just that a query ran).
chart = _chart(10, "Rebuilt", viz_type="table")
chart.query_context = None
chart.datasource_id = 5
chart.datasource_type = "table"
chart.params = json.dumps(
{
"groupby": ["country"],
"metrics": ["count"],
"granularity_sqla": "ds",
"time_range": "Last quarter",
"row_limit": 25,
"adhoc_filters": [
{
"expressionType": "SIMPLE",
"subject": "year",
"operator": ">",
"comparator": 2000,
}
],
}
)
mocks["get_charts_in_layout_order"].return_value = [chart]
mocks["ChartDataCommand"].return_value.run.return_value = {
"queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
}
_run()
load_args = mocks["ChartDataQueryContextSchema"].return_value.load.call_args
payload = load_args.args[0]
query = payload["queries"][0]
assert query["columns"] == ["country"]
assert query["metrics"] == ["count"]
assert query["filters"] == [{"col": "year", "op": ">", "val": 2000}]
assert query["orderby"] == [["count", False]]
assert query["granularity"] == "ds"
assert query["time_range"] == "Last quarter"
assert query["row_limit"] == 25
def test_raw_mode_table_ignores_stale_show_totals() -> None:
# show_totals only produces a second query in aggregate mode (the frontend
# gates the totals query on queryMode === Aggregate), and the control isn't
# reset when hidden. A raw-mode table carrying a stale value must still
# rebuild rather than be needlessly skipped.
from superset.tasks import export_dashboard_excel as module
chart = _rebuildable_chart(
viz_type="table",
form_data={"query_mode": "raw", "all_columns": ["a"], "show_totals": True},
)
with _builder_hook(None):
result = module._resolve_query_context(chart)
assert result is not None
assert result["queries"][0]["columns"] == ["a"]
def test_rebuild_viz_types_is_the_conservative_default() -> None:
# The rebuild allow-list is a fixed fallback (no config override): only viz
# types whose data maps faithfully to a single plain query.
from superset.tasks import export_dashboard_excel as module
assert module.REBUILD_VIZ_TYPES == {
"table",
"big_number_total",
"big_number",
"pie",
}
def test_chart_query_error_grouped_as_general_export_continues(
mocks: dict[str, Any],
) -> None:
@@ -0,0 +1,80 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Characterization tests for the IV behavior of Superset's default
app-encryption engine (``sqlalchemy_utils`` ``AesEngine``, AES-CBC).
``AesEngine._initialize_engine`` (see
``sqlalchemy_utils.types.encrypted.encrypted_type``) derives its IV as the
first 16 bytes of the SHA-256 digest of the configured key: ``self.iv =
self.secret_key[:16]``. That derivation is a pure function of the key alone,
so every encryption performed under the same key reuses the same IV, unlike
``AesGcmEngine``, which samples a fresh random IV (``os.urandom``) on every
call and embeds it in the output. These tests pin that observable difference:
same-plaintext encryptions are identical under the default engine and distinct
under the GCM engine.
"""
from sqlalchemy import String
from sqlalchemy.engine import make_url
from superset.utils.encrypt import (
DEFAULT_ENCRYPTION_ENGINE_NAME,
EncryptedType,
resolve_encryption_engine,
)
DIALECT = make_url("sqlite://").get_dialect()
SECRET_KEY = "k" * 32
def _field(engine: type) -> EncryptedType:
return EncryptedType(String(1024), key=lambda: SECRET_KEY, engine=engine)
def test_default_encryption_engine_name_resolves_to_cbc() -> None:
"""The engine name the codebase falls back to when config is unset resolves
to the unauthenticated AES-CBC engine, not the authenticated AES-GCM one.
"""
from sqlalchemy_utils.types.encrypted.encrypted_type import AesEngine
assert resolve_encryption_engine(DEFAULT_ENCRYPTION_ENGINE_NAME) is AesEngine
def test_default_engine_repeats_ciphertext_for_repeated_plaintext() -> None:
"""Encrypting the same plaintext twice under one key produces identical
ciphertext with the default engine, because its IV is a deterministic
function of the key rather than freshly sampled per call.
"""
field = _field(resolve_encryption_engine(DEFAULT_ENCRYPTION_ENGINE_NAME))
first = field.process_bind_param("hunter2", DIALECT)
second = field.process_bind_param("hunter2", DIALECT)
assert first == second
def test_gcm_engine_varies_ciphertext_for_repeated_plaintext() -> None:
"""Contrast case: the authenticated engine samples a fresh random IV each
call, so encrypting the same plaintext twice under one key produces
different ciphertext.
"""
field = _field(resolve_encryption_engine("aes-gcm"))
first = field.process_bind_param("hunter2", DIALECT)
second = field.process_bind_param("hunter2", DIALECT)
assert first != second