zizmor's self-repository audit flags uses: ./.github/actions/setup-backend/
in the test-postgres job in favor of GitHub's dedicated uses: $/... syntax.
This mirrors the mechanical fix already applied to the analogous
setup-backend references in this same workflow (test-sqlite, #44018) and in
superset-python-presto-hive.yml (#43975), now that the ASF allowlist check
has been bumped to recognize $/ refs (#44014).
Resolves code-scanning alert #2656.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Li <joe@preset.io>
--title "Scheduled Docker image refresh failed for ${LATEST_RELEASE}" \
--label "infra:container" \
--label "bug" \
--label "#bug" \
--body "The weekly Docker base-image refresh failed for release \`${LATEST_RELEASE}\`. Published images may be missing upstream base-layer security patches until this is resolved.
# cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses:./.github/actions/cached-dependencies# zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
run:docker compose -f scripts/databases/hive/docker-compose.yml up -d
- name:Setup Python
uses:./.github/actions/setup-backend/
uses:$/.github/actions/setup-backend/
- name:Setup Postgres
uses:./.github/actions/cached-dependencies
with:
run:setup-postgres
- name:Start Celery worker
uses:./.github/actions/cached-dependencies
# cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses:./.github/actions/cached-dependencies# zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
# zizmor: ignore[dangerous-triggers] - runs in base-branch context and only consumes artifacts uploaded by Python-Unit; never checks out PR code (see note below)
workflow_run:
workflows:["Python-Unit"]
types:[completed]
# This workflow publishes a check run annotating failing Python unit tests
# inline on the PR diff, using JUnit XML uploaded by the Python-Unit workflow.
# It uses the workflow_run trigger so that it always runs in the base-branch
# context and can safely be granted write permissions, even for PRs from
# forks or Dependabot.
#
# IMPORTANT: This workflow must NEVER check out code from the PR branch. All
# data comes from artifacts uploaded by the Python-Unit workflow.
# cached-dependencies is a git submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's gitlink. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses:./.github/actions/cached-dependencies# zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
@@ -84,6 +84,16 @@ The `sql_lab` role is *additive*: it grants the SQL Lab permission set on top of
Deployments may grant or revoke individual view-menu permissions, which shifts the boundary for that deployment but does not redefine the model. Any custom role created by an operator inherits the same principle: its capabilities are whatever the operator has explicitly granted it. The Public principal follows the same rule: operators may grant the Public role read access to specific datasets or dashboards (typically for anonymous reporting use cases), which shifts the boundary for that deployment without redefining the model.
### Async Execution and Realtime Notifications
Asynchronous execution paths do not create a separate data-access capability. A background task is a continuation of an already-authorized action, such as reading chart data or executing SQL through SQL Lab. The initiating route, command, or scheduler must enforce the same route-level and object-level checks the synchronous path would enforce before it creates the task, and the worker must execute under the initiating principal's effective identity when row-level security, impersonation, embedded guest-token scope, or similar controls affect the result.
Task metadata is itself a request-scoped resource. Non-admin users and embedded guests may read or cancel only tasks they are subscribed to or that otherwise represent work they are entitled to observe; Admin may observe and manage tasks as part of the trusted operational boundary. A bug that lets a principal create, read, join, cancel, or receive task state for work outside the role and capability matrix is in scope.
Realtime transports, including WebSocket delivery backed by Redis or Valkey Pub/Sub, are notification mechanisms rather than authorization sources. WebSocket notification access is controlled by `can_read` on the `Realtime` resource. The broadcast scope is authenticated-global, not public: it reaches every authenticated realtime socket, and an anonymous request receives no realtime principal, no JWT cookie, and therefore no socket, so it never receives these messages (true anonymous/Public-role realtime is not offered and would require a separate, restricted model). Broadcast Pub/Sub messages, such as list-view entity-change events, must be context-free nudges; sensitive or authoritative state must not be published on the broadcast scope. Targeted Pub/Sub messages may carry task state only when the producer names routing keys derived from an authorized identity, such as a task subscriber's principal channel (or a per-tab channel derived from it); the producer validates every routing key against the task's own subscriber principals before publishing, and the websocket server forwards the payload only to sockets bound to those keys. Full data and result payloads must still be fetched through the normal protected REST API or cache-read path. Redis Streams used for task completion, dependency, and lock-release signalling are likewise coordination signals; the metastore or cache entry they wake a consumer to read remains the source of truth.
The realtime notification permission is distinct from the permission to read the underlying object. It controls whether a principal receives push notifications, not whether they may read the object once they call the protected REST API. Existing websocket connections are authorized by the JWT accepted at upgrade time; permission revocation after token minting is bounded by `WEBSOCKET_JWT_EXPIRATION_SECONDS` plus the websocket server's socket-check interval. Redis Streams are internal server-to-server coordination primitives and should not be directly exposed as an end-user subscription surface.
### Vulnerability Scope
The test for whether a finding is in scope is a single question:
`GET /api/v1/dataset/<uuid>/purge-impact` returns the charts and distinct
dashboards affected by permanently deleting an archived dataset, together with
an opaque `impact_token`. The dataset purge endpoint now requires that token in
the JSON body as `confirmed_impact_token`. API clients that call
`POST /api/v1/dataset/<uuid>/purge` must fetch and display the impact first;
requests with a missing or malformed token are rejected with 400.
The server rechecks the dependency identities immediately before mutation. If
they changed, purge performs no deletion and returns 409 with a refreshed impact
payload. Clients must display the new impact and obtain renewed confirmation
before retrying. Preview or recheck failures fail closed rather than treating
unknown impact as zero. Chart and dashboard purge endpoints are unchanged.
- The dashboard datasource-based visibility fallback now fails closed: a dashboard whose member charts’ datasources cannot be resolved (deleted datasource rows, missing `datasource_id`, or unsupported datasource types) is no longer accessible to users without explicit editor/viewer rights, and a dashboard composed of semantic-view charts now requires `datasource_access` on (at least one of) its semantic views or their parent semantic layer — previously any authenticated user could open such a dashboard’s shell. Because the fallback now considers every member chart rather than only table-backed ones, a user holding `datasource_access` on any single member datasource — including a semantic view or its parent layer — can open a mixed dashboard that previously denied them. Dashboards with no charts remain accessible, and dashboards with explicit viewers are unaffected. Conversely, holders of `all_datasource_access` now see every published no-viewer dashboard in the dashboard list — including chart-less ones previously hidden by the inner joins — matching what the object-level gate already allowed them to open.
- `SAMPLES_ROW_LIMIT` is now the default for `/datasource/samples` requests without a valid explicit `per_page`, rather than a hard per-request ceiling; explicit limits are honored up to the existing global row-limit ceiling, matching `/chart/data` SAMPLES requests.
- The `cockroachdb` extra (`pip install apache-superset[cockroachdb]`) now installs `sqlalchemy-cockroachdb` instead of the abandoned `cockroachdb` package, whose SQLAlchemy dialect could not be imported under SQLAlchemy 2.0. Existing environments with the old package installed should `pip uninstall cockroachdb && pip install sqlalchemy-cockroachdb` (or simply reinstall the extra) to restore CockroachDB connectivity.
### Native Value filter "Select all" always targets the whole column
The native "Value" filter's bulk "Select all" / "Clear" controls now operate on the entire loaded set of column values regardless of any text typed into the filter's search box. Previously the "Select all (N)" count briefly flickered to the search-scoped count before settling on the full-column count, and clicking "Select all" while searching could select only the currently matching subset. Search-scoped bulk selection was never a supported feature; the count is now stable and always matches what "Select all" selects (the full column). No configuration change is required.
Structured MCP tool results no longer add `<UNTRUSTED-CONTENT>` wrappers or
rewrite delimiter-looking text inside string fields. Tool-result content remains
user-controlled data, but clients must convey that trust boundary outside domain
values instead of recognizing or removing marker strings.
Clients that handled the former delimiter convention should stop stripping marker
text: the same text can be legitimate stored content. Response models and content
types are unchanged, and no metadata-database migration is required. Automated
read-modify-write workflows should be paused or pinned away from older instances
until every serving instance is upgraded; a mixed-version response has no reliable
signal that tells a client whether its text is decorated. Redis-backed MCP response
caches use a new internal namespace after the upgrade, so upgraded instances do not
reuse older cached results.
Values that a client already wrote back with presentation wrappers cannot be
distinguished safely from intentional content. Operators should review possible
`<UNTRUSTED-CONTENT>` / `</UNTRUSTED-CONTENT>` wrappers and
`[ESCAPED-UNTRUSTED-CONTENT-OPEN]` /
`[ESCAPED-UNTRUSTED-CONTENT-CLOSE]` substitutions rather than applying an automatic
marker-removal migration.
### OAuth2 database callback metrics include their outcome
The unqualified `DatabaseRestApi.oauth2` StatsD counter has been replaced with
`DatabaseRestApi.oauth2.success`, `DatabaseRestApi.oauth2.warning`, and
`DatabaseRestApi.oauth2.error`. Update monitoring rules and dashboards that consume
the old counter to use the outcome-specific replacements.
- [42930](https://github.com/apache/superset/pull/42930): Dataset import data-URI fetches no longer honor an HTTP(S) proxy when `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS` is `False` (the default): the connection is now made directly to the destination so the peer-address check validates the real target instead of a proxy's. Deployments that require an egress proxy to reach legitimate external data URLs for dataset import should set `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS = True` or otherwise ensure those URLs resolve without one.
- [42935](https://github.com/apache/superset/pull/42935): The MCP service now refuses to start (`MCPAuthConfigError`) when `MCP_JWT_ISSUER` trusts more than one issuer and no `MCP_USER_RESOLVER` is configured, instead of only logging a warning. This was already a documented misconfiguration (the default resolver isn't issuer-scoped, so distinct trusted issuers minting the same username/email would resolve to the same Superset user); deployments trusting multiple issuers must configure an `MCP_USER_RESOLVER` that derives its identity from the token's `iss` claim before upgrading. Single-issuer deployments are unaffected.
- [42429](https://github.com/apache/superset/pull/42429): The Country Map chart's Iran GeoJSON now gives Alborz province its own ISO 3166-2 code, `IR-32`, instead of `IR-30`. `ISO` is the join key used to color/filter provinces on this chart, so any existing dataset keyed on `IR-30` for Alborz will silently stop matching after upgrading; re-key that data to `IR-32`.
- [43388](https://github.com/apache/superset/pull/43388): The MCP service now refuses to start (`MCPAuthConfigError`) if `MCP_DEV_USERNAME` and `MCP_AUTH_ENABLED = True` are both set, and separately if `MCP_AUTH_ENABLED = True` but no usable JWT key material is configured (RSA key/JWKS, or an explicit `MCP_JWT_SECRET` for HMAC) — both previously started with authentication silently weaker than configured. Deployments combining a dev-mode username with JWT auth enabled, or enabling JWT auth without key material, must pick one before upgrading: unset `MCP_DEV_USERNAME` for a real auth deployment, or unset `MCP_AUTH_ENABLED` (or configure the key material) for a dev-mode one. Response caching (`MCP_CACHE_CONFIG["enabled"] = True`) now also excludes every tool with a side effect by default, not only a partial list, so a previously-cached mutating tool call is no longer served from cache; no config change is needed to pick this up.
- [42393](https://github.com/apache/superset/pull/42393): Exported dataset YAML now carries a `uuid` for each metric and column so that custom folder assignments (which reference metrics/columns by UUID) survive an import into another workspace. This affects any export bundle that contains datasets, not just a dataset export: chart, dashboard, database and full-asset exports all embed the same dataset YAML, so a dashboard exported from this release also fails to import into an older one even though no dataset was exported directly. As with `folders` and `currency_code_column`, the affected `datasets/` files fail schema validation (`Unknown field: uuid`) when imported into Superset releases that predate this change; regenerate or hand-edit exports for older targets in mixed-version fleets.
- [42300](https://github.com/apache/superset/pull/42300): Timeseries charts (line/area/bar) with a Y-axis bound in effect — either an explicit `yAxisBounds` or one derived from `truncateYAxis` — now clamp out-of-range data points to that bound instead of letting ECharts drop the point (and the line segments around it) entirely. Any existing chart with a configured Y-axis bound and data outside it will look different after upgrading: a gap becomes a point pinned to the boundary. The clamp also rewrites the value ECharts reads for that point's tooltip and data label, so the displayed value is the bound rather than the true observation.
- [42087](https://github.com/apache/superset/pull/42087): Stored calculated-column and metric expressions are validated when a query is built, under the same sub-query policy already applied to adhoc expressions. Previously only the dataset update path checked them on save, so expressions written by v1 import, by dataset duplication, or before that check existed were never validated. Since `ALLOW_ADHOC_SUBQUERY` defaults to `False` (see [19242](https://github.com/apache/superset/pull/19242)), a dataset whose stored expression contains a sub-query works before upgrading and afterwards fails at chart render with `Custom SQL fields cannot contain sub-queries.` There is no migration step, and the error does not name the offending dataset column, so audit stored expressions before upgrading: either rewrite them without the sub-query, or set `ALLOW_ADHOC_SUBQUERY = True` to keep the previous behaviour for both stored and adhoc expressions.
### Selenium support removed — Playwright is now required for screenshots
Selenium support has been removed. **Playwright is now required** for all
report and thumbnail screenshot generation. Install it with:
`EXCEL_EXPORT_TABLE_VIZ_TYPES`, and `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`.
The feature depends on `boto3`, which is **not** installed by default; install it
with `pip install apache-superset[excel-export]`.
Charts store their `query_context` only once they have been (re-)saved in
Explore, so older charts may have none. For a fixed, conservative set of viz
types (`table`, `big_number_total`, `big_number`, `pie`) the export rebuilds a
query context from the chart's saved form data so those charts still export.
The rebuild is a single-query mapping and does **not** reproduce plugin
post-processing (pivot, rolling, forecast) or multi-query charts, so any chart of
another type without a saved query context is skipped and listed in the email for
the user to re-save. To cover those types, set `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`
to a callable that receives the chart's form data and returns a query-context
payload (or `None` to fall back to the built-in rebuild) — for example one backed
by a service that runs the chart's real frontend `buildQuery`.
A second mode, **Export Images to Excel**, embeds non-table charts as rendered
images (which viz types stay tabular is controlled by
`EXCEL_EXPORT_TABLE_VIZ_TYPES`). It renders through the headless webdriver, so the
@@ -233,6 +718,10 @@ are the intended model going forward; deprecating and removing implicit viewersh
in a later major version.
- [41044](https://github.com/apache/superset/issues/41044): Removes the deprecated `AVOID_COLORS_COLLISION` feature flag (it defaulted to `True`). Color-collision avoidance is now permanently enabled; any config override setting it to `False` is ignored.
- [41714](https://github.com/apache/superset/pull/41714): **Breaking — the legacy `explore_json` chart-data pipeline is removed** at its long-declared `5.0.0` EOL. The `/superset/explore_json/` and `/superset/explore_json/data/<cache_key>` endpoints, `superset/viz.py`, the `Slice.viz` property, the `get_viz` factory, the `load_explore_json_into_cache` celery task and the `viz=` overload of `security_manager.raise_for_access` are gone. Anything importing `superset.viz` must migrate to the QueryContext / `pandas_postprocessing` pipeline behind `/api/v1/chart/data`. All 15 remaining legacy charts were migrated first: most keep their `viz_type` and renderer (no action needed for saved charts), while saved nvd3 Bubble charts are auto-migrated to the ECharts Bubble Chart (`bubble_v2`) and saved "Time-series Percent Change" (`compare`) charts to the ECharts Line Chart, which restores the nvd3 renderer's interactive percent re-basing via a draggable baseline. The deck.gl Multiple Layers chart now fetches its layers entirely client-side, refitting the viewport as each layer's data arrives, and caps the number of sub-slices fanned out per chart at `DECK_MULTI_MAX_SLICES` (default 50, configurable); dashboard filter badges no longer aggregate child-layer filter metadata.
- [41714](https://github.com/apache/superset/pull/41714): Charts migrated in place keep a `NULL` saved query context until they are next opened in Explore (which regenerates it automatically) or re-saved. Until then, cache warm-up and annotation layers referencing such a chart report an actionable error rather than warming/rendering; opening the chart once resolves it.
- [41714](https://github.com/apache/superset/pull/41714): **Breaking for third-party viz plugins** — the `useLegacyApi` field of `ChartMetadata` in `@superset-ui/core` is removed. Plugins that set it must provide a `buildQuery` and consume `/api/v1/chart/data`. The migrated first-party packages also drop their `legacy-` prefix: `@superset-ui/legacy-plugin-chart-{calendar,chord,country-map,horizon,paired-t-test,parallel-coordinates,partition,rose,world-map}` → `@superset-ui/plugin-chart-*`, and `@superset-ui/legacy-preset-chart-nvd3` → `@superset-ui/preset-chart-nvd3`. The `can_explore_json` permission is no longer created or granted; custom roles referencing it should switch to the `can_read` permissions on `Chart`.
- [41813](https://github.com/apache/superset/pull/41813): `redis` (the Python client, `redis-py`) is bumped from 5.3.1 to 8.0.1. redis-py 8 changes several connection defaults; Superset's own Redis-backed features (`GLOBAL_ASYNC_QUERIES_CACHE_BACKEND`, `DISTRIBUTED_COORDINATION_CONFIG`, and the MCP Redis store) explicitly pin the pre-upgrade behavior so this bump is a no-op for them: the wire protocol stays RESP2 (not the new RESP3 default, which requires Redis/Sentinel 6+ to speak `HELLO`) and there is still no socket timeout by default (redis-py 8 defaults to 5s, which could otherwise newly time out large cached payloads or slow networks). The no-timeout default can now be overridden via two new config keys, `CACHE_REDIS_SOCKET_TIMEOUT` / `CACHE_REDIS_SOCKET_CONNECT_TIMEOUT`, on any `CacheConfig` dict using `CACHE_TYPE: RedisCache` or `RedisSentinelCache`. Separately, redis-py 6+ changed the default for `ssl_check_hostname` from `False` to `True` for SSL connections using `ssl_cert_reqs="required"` (the default) — this is a security improvement, so it has **not** been reverted; deployments with `CACHE_REDIS_SSL=True` whose certificates lack a hostname matching the connection address should set `CACHE_REDIS_SSL_CERT_REQS="none"` (disables cert verification entirely, matching hostname-check bypass) or replace the certificate. General-purpose cache/results backends configured via `CACHE_CONFIG` / `DATA_CACHE_CONFIG` / `RESULTS_BACKEND` with `CACHE_TYPE: RedisCache` go through `flask-caching`'s own Redis backend (outside Superset's code) and are subject to the same new defaults; pass `socket_timeout` / `protocol` via `CACHE_OPTIONS` there if needed. Celery broker and result-backend connections (built by `kombu`, also outside Superset's code) keep their no-socket-timeout behavior (`kombu` passes `socket_timeout=None` explicitly) but do **not** pin the wire protocol, so they follow redis-py's RESP3 default — which requires a Redis server new enough to speak `HELLO` (Redis 6+). Deployments using a pre-6.0 Redis server (EOL) as a Celery broker should upgrade the server before taking this bump.
@@ -252,7 +741,7 @@ in a later major version.
- **`SqlaTable.sql_url` query-string format.** `SqlaTable.sql_url` now URL-encodes `table_name` and joins it as a query parameter rather than concatenating a second `?`. Previously, with `Database.sql_url` returning `/sqllab/?dbid=<id>`, the concatenation produced `/sqllab/?dbid=<id>?table_name=<raw>` — a malformed second `?` that broke the query parser. External code that parsed the legacy `<base>?table_name=<raw>` shape now sees properly percent-encoded values (e.g. `/` → `%2F`, ` ` → `+` or `%20`); decode with `urllib.parse.parse_qsl`.
- **New config flag `EMBEDDED_DISABLE_PERMALINK_ORIGIN_REWRITE` (default `False`).** Share/permalink URLs now substitute `window.location.origin` for the backend-supplied origin so a proxied or subdirectory-deployed Superset never hands the user an unreachable internal hostname. Operators whose reverse proxy correctly forwards `X-Forwarded-Host`*and* who want permalinks to carry the backend's literal origin can opt out by setting `EMBEDDED_DISABLE_PERMALINK_ORIGIN_REWRITE = True` in `superset_config.py`. Default `False` (rewrite is on); flipping the default would regress the dominant proxied/subdir deployment to an unreachable host.
- **New config flag `EMBEDDED_DISABLE_PERMALINK_ORIGIN_REWRITE` (default `False`).** Share/permalink URLs now substitute `window.location.origin` for the backend-supplied origin so a proxied or subdirectory-deployed Superset never hands the user an unreachable internal hostname. Operators whose reverse proxy correctly forwards `X-Forwarded-Host`_and_ who want permalinks to carry the backend's literal origin can opt out by setting `EMBEDDED_DISABLE_PERMALINK_ORIGIN_REWRITE = True` in `superset_config.py`. Default `False` (rewrite is on); flipping the default would regress the dominant proxied/subdir deployment to an unreachable host.
- [41651](https://github.com/apache/superset/pull/41651): **New do-not-translate standard for translation catalogs.** Strings that must stay identical to the source — icon names (e.g. `bolt`), enum/option values (`step-after`), SQL keywords, API field names (`error_message`), code constants, and example placeholders — are now marked with a `#. do-not-translate` extracted comment. The list lives in the `superset/translations/do-not-translate.txt` registry; `scripts/translations/apply_do_not_translate.py` stamps the marker onto `messages.pot` during `babel_update.sh`, and `pybabel update` propagates it to every `.po`, so the status is consistent across all languages. The AI backfill (`backfill_po.py`) and translators leave these entries untranslated (source fallback). The legacy per-catalog convention (a `# Не переводить` translator comment in the `ru` catalog) is still honored for back-compat but is superseded by this standard; contributors adding new machine-read strings should add the msgid to the registry rather than annotating individual catalogs.
@@ -289,7 +778,7 @@ Theme tokens are unaffected — antd 6 removed none of the tokens Superset expos
### Guest-token RLS rules reject unknown fields
The `rls` rules passed to `POST /api/v1/security/guest_token/` are now validated strictly: a rule may only contain `dataset` and `clause`. Previously unknown fields were silently dropped, so a mistyped or legacy scope key (most commonly `datasource` instead of `dataset`) produced a rule with no `dataset`, which is treated as a *global* rule applied to every dataset the embedded resource can reach. Such a request now returns HTTP 400 identifying the offending field instead of issuing a token with an unintended global rule. Integrators that were sending extra fields in RLS rules must remove them; valid dataset-scoped (`{"dataset": 41, "clause": "..."}`) and global (`{"clause": "..."}`) rules are unaffected.
The `rls` rules passed to `POST /api/v1/security/guest_token/` are now validated strictly: a rule may only contain `dataset` and `clause`. Previously unknown fields were silently dropped, so a mistyped or legacy scope key (most commonly `datasource` instead of `dataset`) produced a rule with no `dataset`, which is treated as a _global_ rule applied to every dataset the embedded resource can reach. Such a request now returns HTTP 400 identifying the offending field instead of issuing a token with an unintended global rule. Integrators that were sending extra fields in RLS rules must remove them; valid dataset-scoped (`{"dataset": 41, "clause": "..."}`) and global (`{"clause": "..."}`) rules are unaffected.
### MCP service requires `MCP_JWT_AUDIENCE` when JWT auth is enabled
@@ -340,9 +829,23 @@ As a result the per-table **"Aggregation function"** control (which let you pick
how totals were aggregated client-side, e.g. Sum/Average/Count) has been
removed: totals now always reflect the metric's own definition evaluated at the
total's granularity. For additive metrics (`SUM`/`COUNT`/`MIN`/`MAX`) the result
is unchanged. Saved charts that set `aggregateFunction` will ignore it; no
migration is required. If you previously relied on a plain sum-of-cells total
for a non-additive metric, that specific behavior is no longer available.
is unchanged. If you previously relied on a plain sum-of-cells total for a
non-additive metric, that specific behavior is no longer available.
The "Sum as Fraction of Total/Rows/Columns" display options are back as a
new, standalone **"Show values as"** control (below "Combine metrics" in the
Options panel), since those were mathematically correct even before this
change and are unrelated to the totals-correctness fix. A DB migration
derives the new field from any still-present `aggregateFunction` Sum-fraction
value, so a chart that had one of those options configured picks the
equivalent "Show values as" setting back up automatically. The "Count as
Fraction of ..." variants are **not** migrated: they divided a record count,
while the new control divides the metric's own value, so translating them
automatically would silently change what the chart displays rather than
restore it; those charts need to be manually reconfigured if the value-based
percentage is what's wanted. Charts that used any other non-fraction
`aggregateFunction` value (Sum, Average, Count, ...) are unaffected, since
that specific behavior remains unavailable per the above.
### `thumbnail_url` removed from dashboard list API response
@@ -373,9 +876,10 @@ ALTER TABLE tagged_object DROP CONSTRAINT <constraint_name>;
-- MySQL: find names via `SHOW CREATE TABLE tagged_object;`
ALTER TABLE tagged_object DROP FOREIGN KEY <constraint_name>;
```
### Entity version-history infrastructure (gated off by default)
Introduces the schema and SQLAlchemy-Continuum wiring that captures version history for charts, dashboards, and datasets, plus read-only `GET /api/v1/{chart,dashboard,dataset}/<uuid>/versions/` endpoints. This ships **inert**: a new config flag `ENABLE_VERSIONING_CAPTURE` defaults to `False`, so no save writes any version rows and the endpoints return empty. It is an operational kill-switch (a release toggle that becomes a permanent ops switch), not a feature flag — set it to `True` to enable capture once validated. The migration is additive; existing entity `PUT` responses gain `old_version_uuid` / `new_version_uuid` body fields and an `ETag` header (both null/absent when capture is off).
### Entity version-history infrastructure
Introduces the schema and SQLAlchemy-Continuum wiring that captures version history for charts, dashboards, and datasets, plus read-only `GET /api/v1/{chart,dashboard,dataset}/<uuid>/versions/` endpoints. Capture is governed by the `ENABLE_VERSIONING_CAPTURE` config value — an operational kill-switch (a release toggle that became a permanent ops switch), not a feature flag; see "Version history is on by default" above for the shipped default. With capture off, no save writes version rows; the endpoints continue to serve already-captured rows read-only. The migration is additive; existing entity `PUT` responses gain `old_version_uuid` / `new_version_uuid` body fields and an `ETag` header (both null/absent when capture is off).
A few save- and import-path internals change **unconditionally** (independent of the flag), because the versioned mappers must behave correctly whether or not capture is enabled:
@@ -389,14 +893,14 @@ These are behavior changes that take effect on upgrade regardless of `ENABLE_VER
A read-only companion to the version-history endpoints: each entity type gains a `GET /api/v1/{chart,dashboard,dataset}/<uuid>/activity/` endpoint returning a chronological, access-filtered stream of edits — the entity's own edits plus, for charts and dashboards, transitive edits to related entities during their association windows. Datasets have no related layer in V2, so `include=related` returns an empty stream for a dataset and `include=all` reduces to the dataset's own edits.
Authorization reuses the resource's `can_read` permission and per-object `raise_for_access`; related-entity rows are visibility-filtered to what the caller may see. The stream is empty unless version capture is on (`ENABLE_VERSIONING_CAPTURE`).
Authorization reuses the resource's `can_read` permission and per-object `raise_for_access`; related-entity rows are visibility-filtered to what the caller may see. The stream reflects captured history; with capture off it remains readable but stops accruing new entries.
### Version-history retention (pruning)
@@ -406,17 +910,30 @@ Entity version history (the `version_transaction` / `*_version` shadow tables th
|---|---|---|
| `SUPERSET_VERSION_HISTORY_RETENTION_DAYS` | `30` | Version rows whose owning `version_transaction.issued_at` is older than this many days are pruned. Each entity's live row (`end_transaction_id IS NULL`) is always preserved, as are the live rows of its children and associations; closed historical rows (including the baseline) age out. Set to `0` or a negative value to disable pruning. |
The task ships in the default `CeleryConfig.beat_schedule`; a deployment that overrides `CELERY_CONFIG` without inheriting the default will log a startup warning that the prune task is absent (so it never silently stops running). Retention only prunes whatever history exists — capture itself is gated separately by `ENABLE_VERSIONING_CAPTURE`(ships off).
The task ships in the default `CeleryConfig` (both the `superset.tasks.version_history_retention` import and the beat entry). A deployment that overrides `CELERY_CONFIG` without the beat entry logs a startup warning. When the override explicitly defines `imports`, a missing retention module is also reported; an absent `imports` setting is not diagnosed because Celery may register tasks through `include`, autodiscovery, or worker startup imports. Retention only prunes whatever history exists — capture itself is gated separately by `ENABLE_VERSIONING_CAPTURE`, which now ships on.
### Deletion retention (soft-deleted entities are eventually purged)
Soft-deleted dashboards, charts, and datasets are now permanently removed after a retention window (default 30 days; `SOFT_DELETE_RETENTION_DAYS`, `0` disables; settable per workspace at runtime via the `deletion-retention set-window` CLI, which takes precedence). The `deletion_retention.purge_soft_deleted` Celery beat task runs daily and removes each aged-out entity together with its M:N join rows, owned children, datasource permission, and version-history shadow rows. After purge an entity is **unrecoverable** — its detail and `/restore` endpoints return 404 and its version history is gone.
The introducing release **defaults to dry-run** (`SOFT_DELETE_PURGE_DRY_RUN=True`): the task logs`would_purge` counts but deletes nothing, so operators can validate against production before activating real purging by setting it to `False`. Note `would_purge` is an **upper bound** — it counts every entity past the retention window without evaluating deletion blockers, so a real run may purge fewer (entities referenced by report schedules or set as a user's welcome dashboard are blocked and reported separately). The task only acts while the temporary `SOFT_DELETE` rollout flag is on.
Purging is **live by default** (`SOFT_DELETE_PURGE_DRY_RUN=False`), so the retention promise above is real on a stock deployment. Set it to `True` to have the task log `would_purge` counts and delete nothing — the lever is retained, so an operator can return to dry-run at any time. Note `would_purge` is an **upper bound** — it counts every entity past the retention window without evaluating deletion blockers, so a real run may purge fewer (entities referenced by report schedules or set as a user's welcome dashboard are blocked and reported separately). The task only acts while the `SOFT_DELETE` rollout flag is on; it now ships on by default.
Deployments that replace the default `CELERY_CONFIG` must add`superset.tasks.deletion_retention` to the Celery `imports` and schedule the `deletion_retention.purge_soft_deleted` task themselves. The shipped Docker development config includes both entries.
Deployments that replace the default `CELERY_CONFIG` must ensure workers register`superset.tasks.deletion_retention` and schedule the `deletion_retention.purge_soft_deleted` task themselves. The shipped Docker development config uses `imports` and includes both entries. While `SOFT_DELETE` is statically enabled, a missing beat entry logs a startup warning; when the override explicitly defines `imports`, a missing purge module is also reported.
Operators can immediately erase a specific entity for compliance (GDPR) via `superset deletion-retention force-purge --uuid <uuid>`; this applies legacy hard-delete semantics — a live chart referencing a force-purged dataset is left without a datasource until re-pointed (the chart is not modified), and it purges the named entity even when it was never soft-deleted. Every scheduled evaluation writes a provisional, content-free record to the new `purge_audit_log` table before the cascade starts. Meaningful retained outcomes survive the entity they name. Blocked audit records carry a stable machine-readable `reason` code (`report_schedule`, `user_attribute`, or `cascade_integrity_failure` for an unexpected cascade failure caused by a database integrity constraint) so the audit table alone answers why an entity was not purged; records finalized before the column existed keep a NULL reason. Apply the migration before rolling out the new code: the audit model declares the column, so a worker on the new code with an un-migrated table fails its write-ahead write and the scheduled purge fails closed until the migration lands. During a rolling deploy, workers still on the old code write reason-less blocked rows and suppress on status alone; both effects are self-healing, since a NULL-reason record never matches a reason code and the next all-new-code run re-anchors the entity. Consecutive scheduled evaluations blocked with the same status **and reason** suppress only the redundant current provisional record — a reason change writes one new blocked record carrying the new code; completed outcomes, outcome transitions, and every force-purge attempt remain independent and immutable. Retained transition records are not automatically expired, so entities whose block reason changes repeatedly can accumulate multiple audit rows. The **scheduled** purge fails closed when its provisional record cannot be written, while **force-purge** proceeds even if the audit write fails — the operator is present and deletion outranks audit for a compliance erasure. Operators can monitor `deletion_retention.blocked_audit_suppressed` and `deletion_retention.blocked_audit_dedupe_fallback` to verify suppression and fail-safe fallback behavior without changing the existing blocked-workload gauge.
### Recently Archived view and permanent delete (purge) endpoints
Behind the same `SOFT_DELETE` flag, a **Recently Archived** page (Settings menu, `/archived/`) lists soft-deleted charts, dashboards, and datasets with their archive time and archiving user, and offers **Recover** and **Delete permanently** row actions. The page admits any viewer holding `can_read` on **any** of the three types and offers each viewer only the types they can read; a viewer with none of the three sees an explanatory empty state, and unauthenticated requests are redirected to login.
**New endpoints** — `POST /api/v1/{chart,dashboard,dataset}/<uuid>/purge` permanently delete a single **soft-deleted** row, running the same cascade as the retention task. Irreversible. Requires `can_write` on the entity plus editorship of the row (or admin), mirroring `/restore`. The endpoints answer 404 while `SOFT_DELETE` is off (restore deliberately stays live so rows archived before a flag-off remain recoverable), 404 for rows that are not soft-deleted, and 422 with a reason when the purge is blocked (an alert/report references the entity, a user has the dashboard as their welcome page, or a restrictive foreign key intervenes). A purge that cannot write its audit record is refused with 422 rather than executed unrecorded — unlike the operator CLI, an end user's purge never outranks the audit.
With the flag on, delete confirmations across the chart/dashboard/dataset list pages change shape: a recoverable archive is confirmed with a primary **Archive** button and no type-DELETE friction, and the copy states the retention window when one is configured. A bulk dataset selection that includes semantic views keeps the full danger treatment, because semantic views have no soft-delete — they are deleted permanently and the confirmation says so.
This also resolves the limitation noted under *Soft delete and restore for datasets*: a database blocked by soft-deleted datasets can now be freed by purging those datasets (per-entity endpoint, retention task, or `force-purge` CLI) instead of hard-deleting `tables` rows out-of-band.
The `purge_audit_log` table is **never pruned by design** — the audit must survive the entities it names; operators who need to age it out should prune manually.
Operators can immediately erase a specific entity for compliance (GDPR) via `superset deletion-retention force-purge --uuid <uuid>`; this applies legacy hard-delete semantics — a live chart referencing a force-purged dataset is left without a datasource until re-pointed (the chart is not modified), and it purges the named entity even when it was never soft-deleted. Every purge writes an immutable, content-free audit record to the new `purge_audit_log` table that survives the entity it names: the **scheduled** purge fails closed (an entity whose audit row cannot be written is skipped and retried next run), while **force-purge** proceeds even if the audit write fails — the operator is present and deletion outranks audit for a compliance erasure.
### Webhook alerts/reports block private/internal hosts by default
@@ -465,6 +982,7 @@ Operators can tune or disable the policy via config:
### Data uploads bounded by UPLOAD_MAX_FILE_SIZE_BYTES
Single data-file uploads (CSV, Excel, columnar) are now bounded by the `UPLOAD_MAX_FILE_SIZE_BYTES` config option, which defaults to `100 * 1024 * 1024` (100 MB). Files larger than this are rejected with a `413` before their contents are buffered into memory. Set `UPLOAD_MAX_FILE_SIZE_BYTES = None` to disable the check and restore unbounded uploads.
### Currency symbol position follows the locale when unset
When a chart's currency control leaves the **Prefix or suffix** field empty, the currency symbol position is now derived from the deployment locale's own convention via `Intl.NumberFormat` instead of always defaulting to a suffix. For example, under the default `en-US` locale `USD`, `GBP`, and `EUR` render as a prefix (`$ 1,000`), while eurozone locales such as `fr-FR` render `EUR` as a suffix (`1 000 €`). An explicit Prefix/Suffix selection is always honored and is unaffected.
A live instance keeps writing *new* secrets as AES-CBC during the window between step 2 and the restart in step 4; this second pass sweeps those up (it is idempotent, so already-migrated values are skipped).
A live instance keeps writing _new_ secrets as AES-CBC during the window between step 2 and the restart in step 4; this second pass sweeps those up (it is idempotent, so already-migrated values are skipped).
Schedule the cutover in a quiet window. Runtime reads use only the single configured engine, so in a multi-worker deployment there is an unavoidable brief decrypt-outage between the migration commit and the last worker restarting with the new config — each migrator run is transactional, but the fleet-wide cutover is not zero-downtime.
@@ -598,7 +1116,7 @@ The migration is transactional (all-or-nothing) and idempotent — it can be saf
### Soft delete and restore for datasets
**The soft-delete behavior in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/dataset/<id>`continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**The soft-delete behavior in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/dataset/<id>`uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If datasets are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live datasets in all lists, lookups, and relationship loads (including charts that reference them). The `POST /<uuid>/restore` endpoint and the `dataset_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
@@ -618,17 +1136,17 @@ With the flag enabled: `DELETE /api/v1/dataset/<id>` no longer hard-deletes the
**Schema migration:** the migration adds a nullable `deleted_at` column and an index on it (`ix_tables_deleted_at`) to the `tables` table. The column add is instant; the index build runs inline (no `CONCURRENTLY`) and may briefly block writes on the `tables` table (INSERT/UPDATE/DELETE are queued while the index builds; reads are unaffected) on large Postgres deployments. MySQL InnoDB builds the index online (no blocking). Production deployments with many thousands of datasets should run this migration during a maintenance window.
**Rollback note:** if the application code is rolled back after datasets have been soft-deleted, the older code path's visibility filter no longer applies and previously hidden rows become visible to the older code. Pair the rollback with a data decision (restore the rows, hard-delete them, or also downgrade the migration) rather than assuming the old hard-delete semantics still hold. **Downgrading the migration destroys the deletion markers**: `downgrade()` drops the `deleted_at` column, so any not-yet-restored soft-deleted datasets silently become live, active datasets with no record they were ever deleted. Reconcile the trash (restore or hard-delete each row) *before* downgrading, and disable the `SOFT_DELETE` flag first so no new soft deletes land mid-rollback.
**Rollback note:** if the application code is rolled back after datasets have been soft-deleted, the older code path's visibility filter no longer applies and previously hidden rows become visible to the older code. Pair the rollback with a data decision (restore the rows, hard-delete them, or also downgrade the migration) rather than assuming the old hard-delete semantics still hold. **Downgrading the migration destroys the deletion markers**: `downgrade()` drops the `deleted_at` column, so any not-yet-restored soft-deleted datasets silently become live, active datasets with no record they were ever deleted. Reconcile the trash (restore or hard-delete each row) _before_ downgrading, and disable the `SOFT_DELETE` flag first so no new soft deletes land mid-rollback.
**SQL Lab / dataset-creation flows:** creating a dataset over a table whose dataset sits in the trash is refused. The SQL Lab "save as dataset" flow (`get_or_create_dataset`) and file uploads return a **422 naming the hidden twin and the restore endpoint**; the plain create, update, and duplicate paths currently fail with the generic "already exists" 422. In all cases the remediation is the same: restore the hidden dataset (or use a different table name). Perm-string maintenance also covers hidden rows: renaming a database rewrites `perm`/`schema_perm`/`catalog_perm` on soft-deleted datasets and their charts, so a later restore does not resurrect stale permission strings.
**Importer behavior:** importing a dataset YAML whose UUID matches an existing **soft-deleted** dataset is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active dataset imported without `overwrite=true` is returned unchanged, but a soft-deleted UUID match is restored *and* has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted dataset's exact UUID is an explicit request to bring it back. The restore preserves the original PK, the chart back-reference, `table_columns`, and `sql_metrics`. Non-editors get `ImportFailedError`. Callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
**Importer behavior:** importing a dataset YAML whose UUID matches an existing **soft-deleted** dataset is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active dataset imported without `overwrite=true` is returned unchanged, but a soft-deleted UUID match is restored _and_ has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted dataset's exact UUID is an explicit request to bring it back. The restore preserves the original PK, the chart back-reference, `table_columns`, and `sql_metrics`. Non-editors get `ImportFailedError`. Callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
**Uniqueness-validation changes that apply regardless of the feature flag:** two dataset uniqueness checks were tightened alongside this work and are active even with `SOFT_DELETE` off. (1) Create/update uniqueness treats a dataset whose `catalog` is `NULL` as belonging to the database's default catalog, so a legacy twin pair (`catalog=NULL` vs. `catalog=<default>`, same database/schema/name) that older versions allowed now fails validation with "already exists" when either row is edited — resolve by renaming or removing one of the twins. (2) Duplicating a dataset now checks name collisions scoped to the target (database, catalog, schema) instead of globally by name alone: duplicates into other databases that were previously blocked are now allowed.
### Soft delete and restore for charts
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/chart/<id>`continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/chart/<id>`uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If charts are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live charts in all lists, lookups, and relationship loads (including dashboards that contained them). The `POST /<uuid>/restore` endpoint and the `chart_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
@@ -642,15 +1160,17 @@ With the flag enabled: `DELETE /api/v1/chart/<id>` no longer hard-deletes the ch
**Schema migration:** the migration adds a nullable `deleted_at` column and an index on it (`ix_slices_deleted_at`) to the `slices` table. The column add is instant; the index build runs inline (no `CONCURRENTLY`) and may briefly block writes on the `slices` table (INSERT/UPDATE/DELETE are queued while the index builds; reads are unaffected) on large Postgres deployments. MySQL InnoDB builds the index online (no blocking).
**Rollback note:** if the application code is rolled back after charts have been soft-deleted, the older code path's visibility filter no longer applies and previously hidden rows become visible to the older code. Pair the rollback with a data decision (restore the rows, hard-delete them, or also downgrade the migration) rather than assuming the old hard-delete semantics still hold. **Downgrading the migration destroys the deletion markers**: `downgrade()` drops the `deleted_at` column, so any not-yet-restored soft-deleted charts silently become live, active charts with no record they were ever deleted. Reconcile the trash (restore or hard-delete each row) *before* downgrading, and disable the `SOFT_DELETE` flag first so no new soft deletes land mid-rollback.
**Rollback note:** if the application code is rolled back after charts have been soft-deleted, the older code path's visibility filter no longer applies and previously hidden rows become visible to the older code. Pair the rollback with a data decision (restore the rows, hard-delete them, or also downgrade the migration) rather than assuming the old hard-delete semantics still hold. **Downgrading the migration destroys the deletion markers**: `downgrade()` drops the `deleted_at` column, so any not-yet-restored soft-deleted charts silently become live, active charts with no record they were ever deleted. Reconcile the trash (restore or hard-delete each row) _before_ downgrading, and disable the `SOFT_DELETE` flag first so no new soft deletes land mid-rollback.
**Importer behavior:** importing a chart YAML whose UUID matches an existing **soft-deleted** chart is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active chart imported without `overwrite=true` is returned unchanged, but a soft-deleted UUID match is restored *and* has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted chart's exact UUID is an explicit request to bring it back. The restore preserves the original PK and all out-of-archive references (`dashboard_slices` junctions, `report.chart_id`, tag rows). The operation is permission-gated: non-editors get `ImportFailedError`, and callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
**Importer behavior:** importing a chart YAML whose UUID matches an existing **soft-deleted** chart is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active chart imported without `overwrite=true` is returned unchanged, but a soft-deleted UUID match is restored _and_ has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted chart's exact UUID is an explicit request to bring it back. The restore preserves the original PK and all out-of-archive references (`dashboard_slices` junctions, `report.chart_id`, tag rows). The operation is permission-gated: non-editors get `ImportFailedError`, and callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
- [39914](https://github.com/apache/superset/pull/39914) `ALERT_REPORT_SLACK_V2` now defaults to `True` and the legacy Slack v1 integration (`Slack` recipient type, `files.upload` API) is deprecated for removal in the next major. Slack blocked new apps from `files.upload` in May 2024 and fully retired the method for all apps on November 12, 2025; because the v1 path sends files through `files.upload`, v1 file-bearing sends now fail at the API level — only text-only `chat_postMessage` still works via the legacy path. Grant your Slack bot the `channels:read` and `groups:read` scopes so existing `Slack` recipients can be auto-upgraded to `SlackV2` on next send. Operators who explicitly override the flag to `False`, or whose Slack bot is missing those scopes, will see deprecation warnings while text-only sends continue through the legacy path.
- [42089](https://github.com/apache/superset/pull/42089) automatically upgrades resolvable Slack v1 recipients, preserves text-only v1 delivery with execution warnings when migration cannot finish, and rejects retired v1 file uploads with actionable scope guidance. Slack delivery uses at-most-once terminal writes and a schedule-wide retry budget configured by `SLACK_SEND_RETRY_MAX_TIME`, clamped to the report's remaining working timeout. Deployments using `SupersetMetastoreCache` for the Slack channel cache must schedule the `slack.cache_channels` Celery task to repopulate misses outside report transactions; see [Alerts and Reports](https://superset.apache.org/admin-docs/configuration/alerts-reports#slack-delivery-timeouts-and-retries).
### Soft delete and restore for dashboards
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/dashboard/<id>`continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below.
**Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `True`** (`@lifecycle: testing`), so on a default deployment `DELETE /api/v1/dashboard/<id>`uses the recoverable soft-delete behavior described below. Setting `SOFT_DELETE` to `False` restores legacy permanent hard-delete behavior for subsequent deletes.
**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If dashboards are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live dashboards in all lists and lookups (including slug lookups — if a soft-deleted dashboard's slug was reused while the flag was on, both rows become visible with the same slug). The `POST /<uuid>/restore` endpoint and the `dashboard_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag.
@@ -668,7 +1188,7 @@ The partial-index replacement is dialect-dependent: PostgreSQL uses a native `WH
**Slug semantics:** on PostgreSQL and MySQL 8.0.13+, the slug of a soft-deleted dashboard is **free for reuse**. A new active dashboard can claim it immediately. Restoring a soft-deleted dashboard whose slug has since been claimed returns **422 with a clean error** (`DashboardSlugConflictError`) — rename one of the dashboards and retry; the restore is not silently rejected by a database-level constraint violation.
**Importer behavior:** importing a dashboard YAML whose UUID matches an existing **soft-deleted** dashboard is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active dashboard imported without `overwrite=true` is returned unchanged (the import never mutates it), but a soft-deleted UUID match is restored *and* has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted dashboard's exact UUID is an explicit request to bring it back. The restore preserves the original PK and all pre-deletion relationship rows (`dashboard_slices` junctions, editor/viewer subjects, tags). Callers whose imports must never mutate existing state should treat bundles that may contain previously deleted UUIDs accordingly. The operation is permission-gated: it requires `can_write` and editorship of the deleted row (or admin) — non-editors get `ImportFailedError`, and callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
**Importer behavior:** importing a dashboard YAML whose UUID matches an existing **soft-deleted** dashboard is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active dashboard imported without `overwrite=true` is returned unchanged (the import never mutates it), but a soft-deleted UUID match is restored _and_ has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted dashboard's exact UUID is an explicit request to bring it back. The restore preserves the original PK and all pre-deletion relationship rows (`dashboard_slices` junctions, editor/viewer subjects, tags). Callers whose imports must never mutate existing state should treat bundles that may contain previously deleted UUIDs accordingly. The operation is permission-gated: it requires `can_write` and editorship of the deleted row (or admin) — non-editors get `ImportFailedError`, and callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row.
When the feature flag is enabled, these permissions are enforced on both the frontend (disabled buttons with tooltips) and backend (403 responses from API endpoints). When disabled, legacy `can_csv` behavior is preserved.
@@ -727,14 +1247,17 @@ The Kenya country map has been updated to reflect the 47 counties established un
MCP (Model Context Protocol) tools now include enhanced observability instrumentation for monitoring and debugging:
**Two-layer instrumentation:**
1. **Middleware layer** (`LoggingMiddleware`): Automatically logs all MCP tool calls with `duration_ms` and `success` status in the audit log (Action Log UI, logs table)
2. **Sub-operation tracking**: All 19 MCP tools include granular `event_logger.log_context()` blocks for tracking individual operations like validation, database writes, and query execution
SELECT action, COUNT(*) as calls, AVG(duration_ms) as avg_ms
@@ -769,6 +1292,7 @@ A new `DISTRIBUTED_COORDINATION_CONFIG` configuration provides a unified Redis-b
The distributed coordination is used by the Global Task Framework (GTF) for abort notifications and task completion signaling, and will eventually replace `GLOBAL_ASYNC_QUERIES_CACHE_BACKEND` as the standard signaling backend. Configuring this is recommended for Redis enabled production deployments.
Example configuration in `superset_config.py`:
```python
DISTRIBUTED_COORDINATION_CONFIG = {
"CACHE_TYPE": "RedisCache",
@@ -783,9 +1307,11 @@ See `superset/config.py` for complete configuration options.
### WebSocket config for GAQ with Docker
[35896](https://github.com/apache/superset/pull/35896) and [37624](https://github.com/apache/superset/pull/37624) updated documentation on how to run and configure Superset with Docker. Specifically for the WebSocket configuration, a new `docker/superset-websocket/config.example.json` was added to the repo, so that users could copy it to create a `docker/superset-websocket/config.json` file. The existing `docker/superset-websocket/config.json` was removed and git-ignored, so if you're using GAQ / WebSocket make sure to:
- Stash/backup your existing `config.json` file, to re-apply it after (will get git-ignored going forward)
- Update the `volumes` configuration for the `superset-websocket` service in your `docker-compose.override.yml` file, to include the `docker/superset-websocket/config.json` file. For example:
``` yaml
```yaml
services:
superset-websocket:
volumes:
@@ -798,7 +1324,9 @@ services:
### Example Data Loading Improvements
#### New Directory Structure
Examples are now organized by name with data and configs co-located:
```
superset/examples/
├── _shared/ # Shared database & metadata configs
@@ -811,12 +1339,14 @@ superset/examples/
```
#### Simplified Parquet-based Loading
- Auto-discovery: create `superset/examples/my_dataset/data.parquet` to add a new example
- Parquet is an Apache project format: compressed (~27% smaller), self-describing schema
- YAML configs define datasets, charts, and dashboards declaratively
- Removed Python-based data generation from individual example files
#### Test Data Reorganization
- Moved `big_data.py` to `superset/cli/test_loaders.py` - better reflects its purpose as a test utility
- Fixed inverted logic for `--load-test-data` flag (now correctly includes .test.yaml files when flag is set)
- Clarified CLI flags:
@@ -826,6 +1356,7 @@ superset/examples/
- `--load-big-data` / `-b`: Generate synthetic stress-test data
#### Bug Fixes
- Fixed numpy array serialization for PostgreSQL (converts complex types to JSON strings)
- Fixed KeyError for `allow_csv_upload` field in database configs (now optional with default)
- Fixed test data loading logic that was incorrectly filtering files
@@ -835,6 +1366,7 @@ superset/examples/
The MCP (Model Context Protocol) service enables AI assistants and automation tools to interact programmatically with Superset.
#### New Features
- MCP service infrastructure with FastMCP framework
- Tools for dashboards, charts, datasets, SQL Lab, and instance metadata
- Optional dependency: install with `pip install apache-superset[fastmcp]`
@@ -844,6 +1376,7 @@ The MCP (Model Context Protocol) service enables AI assistants and automation to
#### New Configuration Options
**Development** (single-user, local testing):
```python
# superset_config.py
MCP_DEV_USERNAME = "admin" # User for MCP authentication
@@ -852,6 +1385,7 @@ MCP_SERVICE_PORT = 5008
```
**Production** (JWT-based, multi-user):
```python
# superset_config.py
MCP_AUTH_ENABLED = True
@@ -897,12 +1431,14 @@ superset mcp run --port 5008 --use-factory-config
The MCP service runs as a **separate process** from the Superset web server.
**Important**:
- Requires same Python environment and configuration as Superset
- Shares database connections with main Superset app
- See `superset/mcp_service/SECURITY.md` for details
@@ -935,8 +1472,10 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
- [35062](https://github.com/apache/superset/pull/35062): Changed the function signature of `setupExtensions` to `setupCodeOverrides` with options as arguments.
### Breaking Changes
- [37370](https://github.com/apache/superset/pull/37370): The `APP_NAME` configuration variable no longer controls the browser window/tab title or other frontend branding. Application names should now be configured using the theme system with the `brandAppName` token. The `APP_NAME` config is still used for backend contexts (MCP service, logs, etc.) and serves as a fallback if `brandAppName` is not set.
- **Migration:**
```python
# Before (Superset 5.x)
APP_NAME = "My Custom App"
@@ -980,16 +1519,16 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
Eight M:N association tables move from a synthetic `id INTEGER PRIMARY KEY` to a composite `PRIMARY KEY (fk1, fk2)` on their two foreign-key columns. The surrogate `id` is dropped, and the redundant `UNIQUE (fk1, fk2)` on the two tables that carried one is removed (now subsumed by the PK).
@@ -1000,6 +1539,7 @@ Eight M:N association tables move from a synthetic `id INTEGER PRIMARY KEY` to a
For large `dashboard_slices` / `report_schedule_user` tables, see the operator runbook in [#39859](https://github.com/apache/superset/pull/39859) — pre-flight inventory queries, per-dialect lock-window sizing, and the duplicate / NULL-FK roll-up — to plan the maintenance window.
## 6.0.0
- [33055](https://github.com/apache/superset/pull/33055): Upgrades Flask-AppBuilder to 5.0.0. The AUTH_OID authentication type has been deprecated and is no longer available as an option in Flask-AppBuilder. OpenID (OID) is considered a deprecated authentication protocol - if you are using AUTH_OID, you will need to migrate to an alternative authentication method such as OAuth, LDAP, or database authentication before upgrading.
- [34871](https://github.com/apache/superset/pull/34871): Fixed Jest test hanging issue from Ant Design v5 upgrade. MessageChannel is now mocked in test environment to prevent rc-overflow from causing Jest to hang. Test environment only - no production impact.
- [34782](https://github.com/apache/superset/pull/34782): Dataset exports now include the dataset ID in their file name (similar to charts and dashboards). If managing assets as code, make sure to rename existing dataset YAMLs to include the ID (and avoid duplicated files).
@@ -1008,8 +1548,8 @@ For large `dashboard_slices` / `report_schedule_user` tables, see the operator r
- Change any hex color values to one of: `"success"`, `"processing"`, `"error"`, `"warning"`, `"default"`
- Custom colors are no longer supported to maintain consistency with Ant Design components
- [34561](https://github.com/apache/superset/pull/34561) Added tiled screenshot functionality for Playwright-based reports to handle large dashboards more efficiently. When enabled (default: `SCREENSHOT_TILED_ENABLED = True`), dashboards with 20+ charts or height exceeding 5000px will be captured using multiple viewport-sized tiles and combined into a single image. This improves report generation performance and reliability for large dashboards.
Note: Pillow is now a required dependency (previously optional) to support image processing for tiled screenshots.
`thumbnails` optional dependency is now deprecated and will be removed in the next major release (7.0).
Note: Pillow is now a required dependency (previously optional) to support image processing for tiled screenshots.
`thumbnails` optional dependency is now deprecated and will be removed in the next major release (7.0).
- [33084](https://github.com/apache/superset/pull/33084) The DISALLOWED_SQL_FUNCTIONS configuration now includes additional potentially sensitive database functions across PostgreSQL, MySQL, SQLite, MS SQL Server, and ClickHouse. Existing queries using these functions may now be blocked. Review your SQL Lab queries and dashboards if you encounter "disallowed function" errors after upgrading
- [34235](https://github.com/apache/superset/pull/34235) CSV exports now use `utf-8-sig` encoding by default to include a UTF-8 BOM, improving compatibility with Excel.
- [34258](https://github.com/apache/superset/pull/34258) changing the default in Dockerfile to INCLUDE_CHROMIUM="false" (from "true") in the past. This ensures the `lean` layer is lean by default, and people can opt-in to the `chromium` layer by setting the build arg `INCLUDE_CHROMIUM=true`. This is a breaking change for anyone using the `lean` layer, as it will no longer include Chromium by default.
WEBDRIVER_BASEURL=f"http://superset_app{os.environ.get('SUPERSET_APP_ROOT','/')}/"# When using docker compose baseurl should be http://superset_nginx{ENV{BASEPATH}}/ # noqa: E501
# The Docker Compose app service is named "superset" and listens on 8088. Report
# paths are root-relative, so urljoin drops the base path; only the scheme, host,
# and port must be correct here. SUPERSET_APP_ROOT is kept for consumers that
# concatenate paths directly (e.g. cache warm-up). For screenshots in the dev
# stack (unbuilt static assets) point this at the nginx service instead:
@@ -9,11 +9,13 @@ When working on the Storybook-to-MDX documentation system:
## Why This Matters
The generator (`scripts/generate-superset-components.mjs`) should be lightweight - it extracts data from stories and passes it through. When you add special cases to the generator:
- It becomes harder to maintain
- Stories diverge from their docs representation
- Future stories need to know about generator quirks
When you fix stories to match the expected patterns:
- Stories work identically in Storybook and Docs
- The generator stays simple and predictable
- Patterns are consistent and learnable
@@ -21,6 +23,7 @@ When you fix stories to match the expected patterns:
## Story Patterns for Docs Generation
### Required Structure
```tsx
// Use inline export default (NOT const meta = ...; export default meta)
export default {
@@ -45,6 +48,7 @@ export const InteractiveMyComponent: Story = {
```
### For Components with Variants (size × style grids)
@@ -596,8 +614,10 @@ Ensure you have Python 3.9+ and Node.js 16+ installed before proceeding.
git clone https://github.com/apache/superset.git
cd superset
```
````
2. **Install Python dependencies**
```bash
pip install -e .
```
@@ -611,7 +631,8 @@ Ensure you have Python 3.9+ and Node.js 16+ installed before proceeding.
:::tip Success Check
Navigate to http://localhost:8088 and login with admin/admin
:::
```
````
### Documenting API Endpoints
@@ -630,9 +651,10 @@ Returns a list of charts.
```bash
curl -X GET "http://localhost:8088/api/v1/chart/" \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN"
```
````
**Example Response:**
```json
{
"count": 42,
@@ -645,8 +667,10 @@ curl -X GET "http://localhost:8088/api/v1/chart/" \
]
}
```
```
---
**Note**: This documentation site serves as the primary resource for Superset users, administrators, and contributors. Always prioritize clarity, accuracy, and completeness when creating or updating documentation.
1. Refreshes auto-generated content via `generate:smart` (database pages, API reference, component pages).
2. Calls `yarn docusaurus docs:version` (or the per-section equivalent) to snapshot the section.
3. Freezes any data-file imports (`@site/static/*.json`, `../../data/*.json`) into a snapshot-local `_versioned_data/` dir so the historical version doesn't silently mutate when the source files change.
@@ -73,6 +74,7 @@ What the script does:
5. Updates `versions-config.json` and `<section>_versions.json`.
**Do NOT use** the native Docusaurus commands directly (`yarn docusaurus docs:version`), as they will:
- ❌ Create version files but NOT update `versions-config.json`
- ❌ Skip auto-gen refresh, freezing whatever was on disk
- ❌ Skip data-import freezing, leaving the snapshot pointed at live data
@@ -82,9 +84,11 @@ What the script does:
### Managing Versions
#### With Automated Scripts
The automated scripts handle all configuration updates automatically. No manual editing required!
#### Manual Configuration
If creating versions manually, you'll need to:
1. **Update `versions-config.json`** (or `docusaurus.config.ts` if not using dynamic config):
@@ -109,6 +113,7 @@ If creating versions manually, you'll need to:
Users can configure automated alerts and reports to send dashboards or charts to an email recipient or Slack channel.
- *Alerts* are sent when a SQL condition is reached
- *Reports* are sent on a schedule
- _Alerts_ are sent when a SQL condition is reached
- _Reports_ are sent on a schedule
Alerts and reports are disabled by default. To turn them on, you'll need to change configuration settings and install a suitable headless browser in your environment.
## Content Format Options
When scheduling an alert or report, you can choose the format used to deliver the dashboard or chart:
- **PDF** – a full-page screenshot rendered as a PDF attachment. Available for both dashboards and charts.
- **PNG** – a screenshot delivered as an attachment. Emails embed the image inline in the message body; Slack and webhook recipients receive it as an uploaded file. Available for both dashboards and charts.
- **CSV** – chart data attached as a `.csv` file. Available for charts only.
- **XLSX (Excel)** – chart data attached as a `.xlsx` file. Available for charts only. If the chart's data spans multiple server-paginated files, email delivery detects the bundle and renames the attachment to `.zip`; Slack and webhook deliveries always name the file with an `.xlsx` extension even when the contents are a multi-file ZIP archive.
- **Text** – chart data embedded directly in the email or Slack message body. Available only for charts using a text-based visualization type (e.g. Table, Pivot Table, Paired t-test).
Dashboard reports and alerts are limited to the PDF and PNG formats; the CSV, XLSX, and Text options are only available when scheduling a report or alert for an individual chart.
For alerts (not scheduled reports), PNG/PDF screenshots and chart CSV/XLSX data are only generated when the `ALERTS_ATTACH_REPORTS` [feature flag](/admin-docs/configuration/configuring-superset#feature-flags) is enabled (the default); with it disabled, an alert notification is still sent, but without the attachment.
## Requirements
### Commons
@@ -26,16 +40,18 @@ Alerts and reports are disabled by default. To turn them on, you'll need to chan
- emails: `SMTP_*` settings
- Slack messages: `SLACK_API_TOKEN`
- Users can customize the email subject by including date code placeholders, which will automatically be replaced with the corresponding UTC date when the email is sent. To enable this functionality, activate the `"DATE_FORMAT_IN_EMAIL_SUBJECT"` [feature flag](/admin-docs/configuration/configuring-superset#feature-flags). This enables date formatting in email subjects, preventing all reporting emails from being grouped into the same thread (optional for the reporting feature).
- Use date codes from [strftime.org](https://strftime.org/) to create the email subject.
- If no date code is provided, the original string will be used as the email subject.
- Use date codes from [strftime.org](https://strftime.org/) to create the email subject.
- If no date code is provided, the original string will be used as the email subject.
- Each alert/report has an "Include a link back to Superset" option (enabled by default) controlling whether the call-to-action link is included in email and Slack notifications. The link text in emails is configurable via `EMAIL_REPORTS_CTA`; the Slack message always uses "Explore in Superset". Uncheck the option when recipients should not receive a link to your Superset host, e.g. for external audiences.
##### Disable dry-run mode
Screenshots will be taken but no messages actually sent as long as `ALERT_REPORTS_NOTIFICATION_DRY_RUN = True`, its default value in `docker/pythonpath_dev/superset_config.py`. To disable dry-run mode and start receiving email/Slack notifications, set `ALERT_REPORTS_NOTIFICATION_DRY_RUN` to `False` in [superset config](https://github.com/apache/superset/blob/master/docker/pythonpath_dev/superset_config.py).
Screenshots will be taken but no messages actually sent as long as `ALERT_REPORTS_NOTIFICATION_DRY_RUN = True`, its default value in `docker/pythonpath_dev/superset_config.py`. To disable dry-run mode and start receiving email/Slack notifications, set `ALERT_REPORTS_NOTIFICATION_DRY_RUN` to `False` in [superset config](https://github.com/apache/superset/blob/master/docker/pythonpath_dev/superset_config.py).
#### In your `Dockerfile`
You'll need to extend the Superset image to include a headless browser. Your options include:
- Use Playwright with Chromium: this is the recommended approach as of version 4.1.x or greater. Playwright always uses Chromium — the `WEBDRIVER_TYPE` config setting has no effect when Playwright is active. A working example of a Dockerfile that installs these tools is provided under "Building your own production Docker image" on the [Docker Builds](/admin-docs/installation/docker-builds#building-your-own-production-docker-image) page. Enable the `PLAYWRIGHT_REPORTS_AND_THUMBNAILS` feature flag in your config to activate it.
- Use Firefox (Selenium): you'll need to install geckodriver and Firefox. Set `WEBDRIVER_TYPE` to `"firefox"` in your `superset_config.py`.
- Use Chrome (Selenium): you'll need to install Chrome. Set `WEBDRIVER_TYPE` to `"chrome"` in your `superset_config.py`.
When enabled, Superset rejects webhook configurations that use `http://` URLs.
#### Request Timeout
Webhook deliveries use a socket timeout so a request can't hang forever if the webhook target is unreachable, which would otherwise leave the report schedule stuck in a `WORKING` state. Configure it with:
```python
ALERT_REPORTS_WEBHOOK_TIMEOUT = 60 # seconds
```
Set to `None` to disable the timeout (not recommended).
#### Retry Behavior
Superset automatically retries webhook deliveries on `429 Too Many Requests` and `5xx` server errors using exponential backoff. Retries are bounded to roughly 120 seconds of cumulative wall-clock time (worst case ~210 seconds, because the bound is checked against the time elapsed before each attempt, so the final request can begin just under the limit and still run its full request timeout), after which the delivery is abandoned.
@@ -204,7 +284,7 @@ You need to replace default values with your custom Redis, Slack and/or SMTP con
Superset uses Celery beat and Celery worker(s) to send alerts and reports.
- The beat is the scheduler that tells the worker when to perform its tasks. This schedule is defined when you create the alert or report.
- The worker will process the tasks that need to be performed when an alert or report is fired.
- The worker will process the tasks that need to be performed when an alert or report is fired.
In the `CeleryConfig`, only the `beat_schedule` is relevant to this feature, the rest of the `CeleryConfig` can be changed for your needs.
@@ -244,6 +324,53 @@ class CeleryConfig:
}
CELERY_CONFIG = CeleryConfig
# Scheduled reports share one deadline across browser readiness, capture/PDF
# generation, delivery, and terminal-state persistence. The effective budget
# for a schedule is min(this value, the schedule's working_timeout), so the
# per-schedule field keeps its meaning as a user-facing cap. The default (one
# hour) matches the historical working_timeout default, so upgrading changes
# no default behavior; lower it to enforce a tighter report SLA.
ALERT_REPORTS_EXECUTION_BUDGET_SECONDS = 3600
# These reserves are part of (not additions to) the total budget and their sum
# must be less than it. Readiness polling stops in time to leave capacity for
Alternatively, you can assign a function to `ALERT_MINIMUM_INTERVAL` and/or `REPORT_MINIMUM_INTERVAL`. This is useful to dynamically retrieve a value as needed:
For security, Superset rewrites external links in alert/report email HTML so
they go through a warning page before the user is navigated to the external
site. Internal links (matching your configured base URL) are not affected.
site. Internal links (matching your configured base URL) are not affected.
```python
# Disable external link redirection entirely (default: True)
@@ -347,17 +474,50 @@ to determine which hosts are internal.
## Troubleshooting
There are many reasons that reports might not be working. Try these steps to check for specific issues.
There are many reasons that reports might not be working. Try these steps to check for specific issues.
### Confirm feature flag is enabled and you have sufficient permissions
If you don't see "Alerts & Reports" under the *Manage* section of the Settings dropdown in the Superset UI, you need to enable the `ALERT_REPORTS` feature flag (see above). Enable another feature flag and check to see that it took effect, to verify that your config file is getting loaded.
If you don't see "Alerts & Reports" under the _Manage_ section of the Settings dropdown in the Superset UI, you need to enable the `ALERT_REPORTS` feature flag (see above). Enable another feature flag and check to see that it took effect, to verify that your config file is getting loaded.
Log in as an admin user to ensure you have adequate permissions.
### Check the logs of your Celery worker
This is the best source of information about the problem. In a docker compose deployment, you can do this with a command like `docker logs superset_worker --since 1h`.
This is the best source of information about the problem. In a docker compose deployment, you can do this with a command like `docker logs superset_worker --since 1h`.
### CSV and Excel chart-data request failures
The worker uses the saved query context to POST to the chart-data export endpoint,
falling back to the legacy GET export when a query context cannot be generated.
`ALERT_REPORTS_CSV_REQUEST_TIMEOUT` (60 seconds by default) limits socket operations;
the report execution budget and its delivery/cleanup reserves also cap the request.
Connection and read timeouts are reported as CSV/Excel generation timeouts.
These attachment timeouts are logged at error level and explicitly mark the report
task as failed, while the report execution retains its ERROR state and separate
error-notification history. Other HTTP 408 exception handling is unchanged.
To tolerate short-lived transport failures, operators can opt in with
`ALERT_REPORTS_CSV_REQUEST_RETRY = True` (default: `False`). This permits **one** retry
for transient connection/read failures and HTTP 429, 500, 502, 503, or 504. Other
HTTP statuses are not retried. Backoff is 0.5 seconds, extended to at most 2 seconds
for a numeric `Retry-After`; longer, invalid, or date-based delays are not retried
inline. Both attempts and backoff share the initial request timeout allowance and
respect the remaining execution budget. Unbounded requests are not retried.
A request that consumes its entire timeout does **not** get another full timeout.
Socket timeouts are not wall-clock cancellation: existing report task limits still
interrupt in-flight work. A timed-out server query can continue running, so enabling
retries can increase database load. Leave retries disabled unless appropriate for
your deployment; disable the setting to roll back retry behavior.
Worker diagnostics include schedule/chart identifiers, a fixed endpoint path (no
For HTTP errors, at most 4097 response bytes are read to enforce a 4096-byte limit.
Only recognized Superset error types from up to four JSON errors are retained;
free-form messages, extra fields, and non-JSON or oversized bodies are redacted or
omitted. Cookies, authentication headers, URLs, SQL, and query payloads are not
included in these transport diagnostics. HTTP 400 therefore remains a failure to
investigate, not a reason to repeat the same request.
### Check web browser and webdriver installation
@@ -369,7 +529,7 @@ If you are handling the installation of the headless browser on your own, do you
One symptom of an invalid connection to an email server is receiving an error of `[Errno 110] Connection timed out` in your logs when the report tries to send.
Confirm via testing that your outbound email configuration is correct. Here is the simplest test, for an un-authenticated email SMTP email service running on port 25. If you are sending over SSL, for instance, study how [Superset's codebase sends emails](https://github.com/apache/superset/blob/master/superset/utils/core.py#L818) and then test with those commands and arguments.
Confirm via testing that your outbound email configuration is correct. Here is the simplest test, for an un-authenticated email SMTP email service running on port 25. If you are sending over SSL, for instance, study how [Superset's codebase sends emails](https://github.com/apache/superset/blob/master/superset/utils/core.py#L818) and then test with those commands and arguments.
Start Python in your worker environment, replace all example values, and run:
@@ -395,16 +555,16 @@ This should send an email.
Possible fixes:
- Some cloud hosts disable outgoing unauthenticated SMTP email to prevent spam. For instance, [Azure blocks port 25 by default on some machines](https://learn.microsoft.com/en-us/azure/virtual-network/troubleshoot-outbound-smtp-connectivity). Enable that port or use another sending method.
- Some cloud hosts disable outgoing unauthenticated SMTP email to prevent spam. For instance, [Azure blocks port 25 by default on some machines](https://learn.microsoft.com/en-us/azure/virtual-network/troubleshoot-outbound-smtp-connectivity). Enable that port or use another sending method.
- Use another set of SMTP credentials that you verify works in this setup.
### Browse to your report from the worker
The worker may be unable to reach the report. It will use the value of `WEBDRIVER_BASEURL` to browse to the report. If that route is invalid, or presents an authentication challenge that the worker can't pass, the report screenshot will fail.
The worker may be unable to reach the report. It will use the value of `WEBDRIVER_BASEURL` to browse to the report. If that route is invalid, or presents an authentication challenge that the worker can't pass, the report screenshot will fail.
Check this by attempting to `curl` the URL of a report that you see in the error logs of your worker. For instance, from the worker environment, run `curl http://superset_app:8088/superset/dashboard/1/`. You may get different responses depending on whether the dashboard exists - for example, you may need to change the `1` in that URL. If there's a URL in your logs from a failed report screenshot, that's a good place to start. The goal is to determine a valid value for `WEBDRIVER_BASEURL` and determine if an issue like HTTPS or authentication is redirecting your worker.
Check this by attempting to `curl` the URL of a report that you see in the error logs of your worker. For instance, from the worker environment, run `curl http://superset:8088/dashboard/1/`. You may get different responses depending on whether the dashboard exists - for example, you may need to change the `1` in that URL. If there's a URL in your logs from a failed report screenshot, that's a good place to start. The goal is to determine a valid value for `WEBDRIVER_BASEURL` and determine if an issue like HTTPS or authentication is redirecting your worker.
In a deployment with authentication measures enabled like HTTPS and Single Sign-On, it may make sense to have the worker navigate directly to the Superset application running in the same location, avoiding the need to sign in. For instance, you could use `WEBDRIVER_BASEURL="http://superset_app:8088"` for a docker compose deployment, and set `"force_https": False,` in your `TALISMAN_CONFIG`.
In a deployment with authentication measures enabled like HTTPS and Single Sign-On, it may make sense to have the worker navigate directly to the Superset application running in the same location, avoiding the need to sign in. For instance, you could use `WEBDRIVER_BASEURL="http://superset:8088"` for a docker compose deployment, and set `"force_https": False,` in your `TALISMAN_CONFIG` — appropriate only for an isolated development stack or when HTTPS is enforced by a trusted proxy in front of Superset.
### Duplicate report deliveries
@@ -512,6 +672,7 @@ schedule the queries that have `schedule_info` in their JSON metadata. For sched
Airflow, additional fields can be easily added to the configuration file above.
:::resources
- [Tutorial: Automated Alerts and Reporting via Slack/Email in Superset](https://dev.to/ngtduc693/apache-superset-topic-5-automated-alerts-and-reporting-via-slackemail-in-superset-2gbe)
- [Blog: Integrating Slack alerts and Apache Superset for better data observability](https://medium.com/affinityanswers-tech/integrating-slack-alerts-and-apache-superset-for-better-data-observability-fd2f9a12c350)
- [Blog: How to Set Up Global Async Queries (GAQ) in Apache Superset](https://medium.com/@ngigilevis/how-to-set-up-global-async-queries-gaq-in-apache-superset-a-complete-guide-9d2f4a047559)
@@ -13,7 +13,7 @@ Cross-account IAM role assumption via STS `AssumeRole` is supported, allowing a
## Prerequisites
- Enable the `AWS_DATABASE_IAM_AUTH` feature flag in `superset_config.py`. IAM authentication is gated behind this flag; if it is disabled, connections using `aws_iam` fail with *"AWS IAM database authentication is not enabled."*
- Enable the `AWS_DATABASE_IAM_AUTH` feature flag in `superset_config.py`. IAM authentication is gated behind this flag; if it is disabled, connections using `aws_iam` fail with _"AWS IAM database authentication is not enabled."_
```python
FEATURE_FLAGS = {
"AWS_DATABASE_IAM_AUTH": True,
@@ -48,14 +48,14 @@ IAM authentication is configured via the **encrypted_extra** field of the databa
}
```
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | Yes | Set to `true` to activate IAM auth |
| `role_arn` | No | ARN of the cross-account IAM role to assume via STS. Omit for same-account auth |
| `external_id` | No | External ID for the STS `AssumeRole` call, if required by the target role's trust policy |
| `region` | Yes | AWS region of the database cluster |
| `db_username` | Yes | The database username associated with the IAM identity |
| `session_duration` | No | STS session duration in seconds (default: `3600`) |
SimpleCache (in-memory), MinIO/S3, or the local filesystem.
[Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported.
Caching can be configured by providing dictionaries in
`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
@@ -47,6 +46,7 @@ In order to use dedicated cache stores, additional python libraries must be inst
- For Redis: we recommend the [redis](https://pypi.python.org/pypi/redis) Python package
- Memcached: we recommend using [pylibmc](https://pypi.org/project/pylibmc/) client library as
`python-memcached` does not handle storing binary data correctly.
- MinIO (S3): we recommend using the [minio-flask-cache](https://github.com/greggailly/minio-flask-cache) package
These libraries can be installed using pip.
@@ -76,6 +76,58 @@ value defined in `DATA_CACHE_CONFIG`.
Note, that by setting the cache timeout to `-1`, caching for charting data can be disabled, either
per chart, dataset or database, or by default if set in `DATA_CACHE_CONFIG`.
Native filter option queries (the dropdown values for native filters) go through this same
chart-data cache, but their freshness needs often differ from regular chart queries, especially for
datasets whose visible values change frequently, including RLS-constrained datasets. Set
`NATIVE_FILTER_OPTIONS_CACHE_TIMEOUT` in `superset_config.py` to give these queries a dedicated
timeout, checked before the chart/dataset/database chain and the `DATA_CACHE_CONFIG` default above:
This will cache the top 5 most popular dashboards every hour. For other
strategies, check the `superset/tasks/cache.py` file.
### Warming Up Native Filter Options
Native filter Value-type dropdown option queries (e.g. `SELECT DISTINCT column FROM table`) are
cached the same way as chart data, via `DATA_CACHE_CONFIG`. However, the strategies above only warm
up chart render queries, so the first user to open a dashboard's filter dropdown after a cache entry
expires still triggers a fresh database query.
The `native_filter_options` strategy pre-populates the cache for these dropdown queries. It reads
each dashboard's `native_filter_configuration`, builds the same `filter_select` chart-data query the
frontend would send, and executes it as the configured `SUPERSET_CACHE_WARMUP_USER`:
```python
class CustomCeleryConfig(CeleryConfig):
beat_schedule = {
**CeleryConfig.beat_schedule,
'cache-warmup-native-filters': {
'task': 'cache-warmup',
'schedule': crontab(minute=0, hour=3), # daily at 03:00
'kwargs': {
'strategy_name': 'native_filter_options',
'dashboard_ids': [1, 2, 3],
},
},
}
```
Requirements and limitations:
- `SUPERSET_CACHE_WARMUP_USER` must be set to a user with access to the dashboards and datasets
referenced by the native filters.
- `DATA_CACHE_CONFIG` must use a backend that actually persists entries (Redis recommended); the
default `NullCache` discards writes, so warming has nothing to warm. The effective timeout also
needs to be positive — `NATIVE_FILTER_OPTIONS_CACHE_TIMEOUT = -1` disables cache writes for these
queries entirely, even with a working backend.
- Schedule the warm-up at least as often as the effective native filter cache timeout (whichever of
`NATIVE_FILTER_OPTIONS_CACHE_TIMEOUT`, the chart/dataset/database timeout, or `DATA_CACHE_CONFIG`'s
default applies). A looser schedule still leaves a window of cold, unwarmed queries between expiry
and the next run — the daily example above assumes a TTL of a day or more.
- Cache entries are warmed under the warm-up user's own cache partition, the same entry that user
would create by opening the filter dropdown manually. Users with a different role set or row-level
security context may still see a cache miss on first load.
- Cascading/dependent native filters and search-term variants of filter option queries are not
warmed by this strategy.
## Caching Thumbnails
This is an optional feature that can be turned on by activating its [feature flag](/admin-docs/configuration/configuring-superset#feature-flags) on config:
@@ -251,14 +347,25 @@ high-performance distributed operations. This configuration enables:
- **Distributed locking**: Moves lock operations from the metadata database to Redis, improving
performance and reducing metastore load
- **Real-time event notifications**: Enables instant pub/sub messaging for task abort signals and
completion notifications instead of polling-based approaches
- **Event-driven notifications**: Task completion and abort signals are delivered over Redis
**Streams**, so waiters (sync join-and-wait, task-dependency DAGs, abort listeners) wake when a
signal lands instead of polling the metadata database. Because stream entries are persisted, a
waiter that reads slightly late, reconnects, or fails over still receives the signal. Without this
backend, these operations poll the metadata database instead.
:::note
This requires Redis or Valkey specifically—it uses Redis-specific features (pub/sub, `SET NX EX`)
that are not available in general Flask-Caching backends.
This requires Redis or Valkey specifically—it uses Redis-specific features (Streams, pub/sub,
`SET NX EX`) that are not available in general Flask-Caching backends.
:::
Each signal stream keeps only its latest entry and is given a TTL, so signal streams for tasks that
are never awaited do not accumulate in Redis/Valkey. Set the retention window with
These apply to `RedisSentinelCache` connections as well, covering both the sentinel-node
connections and the resolved master connection.
### Distributed Lock TTL
You can configure the default lock TTL (time-to-live) in seconds. Locks automatically expire after
@@ -323,6 +447,7 @@ While database-backed operations work reliably, the Redis backend is recommended
deployments where low latency and reduced database load are important.
:::resources
- [Blog: The Data Engineer's Guide to Lightning-Fast Superset Dashboards](https://preset.io/blog/the-data-engineers-guide-to-lightning-fast-apache-superset-dashboards/)
- [Blog: Accelerating Dashboards with Materialized Views](https://preset.io/blog/accelerating-apache-superset-dashboards-with-materialized-views/)
In case you want to assign the `Admin` role on new user registration, it can be assigned as follows:
```python
AUTH_USER_REGISTRATION_ROLE = "Admin"
```
If you encounter the [issue](https://github.com/apache/superset/issues/13243) of not being able to list users from the Superset main page settings, although a newly registered user has an `Admin` role, please re-run `superset init` to sync the required permissions. Below is the command to re-run `superset init` using docker compose.
Alternatively, split a very large dashboard into several smaller ones. Note that this check is enforced when saving layout edits in the UI; a dashboard imported from a ZIP with an oversized layout will load and render, but cannot be edited and re-saved until the limit is raised.
:::resources
- [Blog: Feature Flags in Apache Superset](https://preset.io/blog/feature-flags-in-apache-superset-and-preset/)
1. Generate a new GeoJSON file for your country following the guide in [this Jupyter notebook](https://github.com/apache/superset/blob/master/superset-frontend/plugins/legacy-plugin-chart-country-map/scripts/Country%20Map%20GeoJSON%20Generator.ipynb).
2. Edit the countries list in [legacy-plugin-chart-country-map/src/countries.ts](https://github.com/apache/superset/blob/master/superset-frontend/plugins/legacy-plugin-chart-country-map/src/countries.ts).
1. Generate a new GeoJSON file for your country following the guide in [this Jupyter notebook](https://github.com/apache/superset/blob/master/superset-frontend/plugins/plugin-chart-country-map/scripts/Country%20Map%20GeoJSON%20Generator.ipynb).
2. Edit the countries list in [plugin-chart-country-map/src/countries.ts](https://github.com/apache/superset/blob/master/superset-frontend/plugins/plugin-chart-country-map/src/countries.ts).
[statsd](https://pypi.org/project/statsd/) in version ~3.3.0 must be installed.
Note that it’s also possible to implement your own logger by deriving
`superset.stats_logger.BaseStatsLogger`.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
