mirror of
https://github.com/apache/superset.git
synced 2026-08-25 01:21:18 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f638a55b57 | ||
|
|
ae1838814f | ||
|
|
ca3d7670b7 | ||
|
|
94855e9626 | ||
|
|
7dbf71a379 | ||
|
|
24b95f9ca7 | ||
|
|
c635d0754f | ||
|
|
31f06c0ee6 | ||
|
|
27ec80c07b | ||
|
|
a8a8b51afb | ||
|
|
6ab21b381a | ||
|
|
b0962ba5ed | ||
|
|
bf5f3a9e6b | ||
|
|
c980b3a361 | ||
|
|
dc436c76f7 | ||
|
|
6eced8e919 | ||
|
|
1bde62f997 | ||
|
|
f1f6347885 | ||
|
|
8591f52ced | ||
|
|
ceb75b9350 | ||
|
|
db539288ac | ||
|
|
a392e8b102 | ||
|
|
e450acf1c7 | ||
|
|
2c02965f2b | ||
|
|
f3142e7b15 | ||
|
|
649c062825 |
@@ -105,6 +105,7 @@ jobs:
|
||||
tool: customSmallerIsBetter
|
||||
output-file-path: bundle-size-summary.json
|
||||
external-data-json-path: bundle-size-history.json
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
fail-on-alert: false
|
||||
summary-always: true
|
||||
|
||||
|
||||
+25
@@ -26,6 +26,30 @@ assists people when migrating to a new version.
|
||||
|
||||
- `SAMPLES_ROW_LIMIT` is now the default for `/datasource/samples` requests without a valid explicit `per_page`, rather than a hard per-request ceiling; explicit limits are honored up to the existing global row-limit ceiling, matching `/chart/data` SAMPLES requests.
|
||||
|
||||
### Superset no longer auto-generates `type:`/`editor:`/`favorited_by:` tags
|
||||
|
||||
With `TAGGING_SYSTEM` enabled, Superset used to auto-tag every chart,
|
||||
dashboard, saved query, and dataset with implicit tags derived from
|
||||
metadata (object type, editors, and who favorited it), and generate a
|
||||
`favorited_by:<user id>` tag on every favorite/unfavorite. Nothing in the
|
||||
UI ever surfaced these tags to users — every tags list and filter in the
|
||||
frontend explicitly excluded them — so the generation added continuous
|
||||
write overhead (13 SQLAlchemy event listeners across 5 models) with no
|
||||
user-visible benefit. That generation is removed.
|
||||
|
||||
Manually-created (custom) tags are unaffected: creating, editing,
|
||||
listing, and filtering tags still works exactly as before, including the
|
||||
`custom_tag` API filter used to distinguish custom from implicit tags.
|
||||
|
||||
Deployments already running with `TAGGING_SYSTEM` enabled keep any
|
||||
`type:`/`editor:`/`favorited_by:` tag rows created before upgrading — they
|
||||
remain queryable via the API and MCP's `list_tags`/`get_tag_info` tools,
|
||||
and are still exempt from bulk tag deletion — but no new ones are created,
|
||||
and the `superset sync_tags` CLI command that backfilled them has been
|
||||
removed. The `DASHBOARD_LIST_CUSTOM_TAGS_ONLY` config flag and the
|
||||
dashboard-list optimization it enabled are also removed, since every tag
|
||||
returned is now a custom tag by default.
|
||||
|
||||
### MCP tool results preserve stored string values
|
||||
|
||||
Structured MCP tool results no longer add `<UNTRUSTED-CONTENT>` wrappers or
|
||||
@@ -58,6 +82,7 @@ the old counter to use the outcome-specific replacements.
|
||||
|
||||
- [42930](https://github.com/apache/superset/pull/42930): Dataset import data-URI fetches no longer honor an HTTP(S) proxy when `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS` is `False` (the default): the connection is now made directly to the destination so the peer-address check validates the real target instead of a proxy's. Deployments that require an egress proxy to reach legitimate external data URLs for dataset import should set `DATASET_IMPORT_ALLOW_INTERNAL_DATA_URLS = True` or otherwise ensure those URLs resolve without one.
|
||||
- [42935](https://github.com/apache/superset/pull/42935): The MCP service now refuses to start (`MCPAuthConfigError`) when `MCP_JWT_ISSUER` trusts more than one issuer and no `MCP_USER_RESOLVER` is configured, instead of only logging a warning. This was already a documented misconfiguration (the default resolver isn't issuer-scoped, so distinct trusted issuers minting the same username/email would resolve to the same Superset user); deployments trusting multiple issuers must configure an `MCP_USER_RESOLVER` that derives its identity from the token's `iss` claim before upgrading. Single-issuer deployments are unaffected.
|
||||
- [43388](https://github.com/apache/superset/pull/43388): The MCP service now refuses to start (`MCPAuthConfigError`) if `MCP_DEV_USERNAME` and `MCP_AUTH_ENABLED = True` are both set, and separately if `MCP_AUTH_ENABLED = True` but no usable JWT key material is configured (RSA key/JWKS, or an explicit `MCP_JWT_SECRET` for HMAC) — both previously started with authentication silently weaker than configured. Deployments combining a dev-mode username with JWT auth enabled, or enabling JWT auth without key material, must pick one before upgrading: unset `MCP_DEV_USERNAME` for a real auth deployment, or unset `MCP_AUTH_ENABLED` (or configure the key material) for a dev-mode one. Response caching (`MCP_CACHE_CONFIG["enabled"] = True`) now also excludes every tool with a side effect by default, not only a partial list, so a previously-cached mutating tool call is no longer served from cache; no config change is needed to pick this up.
|
||||
- [42393](https://github.com/apache/superset/pull/42393): Exported dataset YAML now carries a `uuid` for each metric and column so that custom folder assignments (which reference metrics/columns by UUID) survive an import into another workspace. This affects any export bundle that contains datasets, not just a dataset export: chart, dashboard, database and full-asset exports all embed the same dataset YAML, so a dashboard exported from this release also fails to import into an older one even though no dataset was exported directly. As with `folders` and `currency_code_column`, the affected `datasets/` files fail schema validation (`Unknown field: uuid`) when imported into Superset releases that predate this change; regenerate or hand-edit exports for older targets in mixed-version fleets.
|
||||
- [42300](https://github.com/apache/superset/pull/42300): Timeseries charts (line/area/bar) with a Y-axis bound in effect — either an explicit `yAxisBounds` or one derived from `truncateYAxis` — now clamp out-of-range data points to that bound instead of letting ECharts drop the point (and the line segments around it) entirely. Any existing chart with a configured Y-axis bound and data outside it will look different after upgrading: a gap becomes a point pinned to the boundary. The clamp also rewrites the value ECharts reads for that point's tooltip and data label, so the displayed value is the bound rather than the true observation.
|
||||
- [42087](https://github.com/apache/superset/pull/42087): Stored calculated-column and metric expressions are validated when a query is built, under the same sub-query policy already applied to adhoc expressions. Previously only the dataset update path checked them on save, so expressions written by v1 import, by dataset duplication, or before that check existed were never validated. Since `ALLOW_ADHOC_SUBQUERY` defaults to `False` (see [19242](https://github.com/apache/superset/pull/19242)), a dataset whose stored expression contains a sub-query works before upgrading and afterwards fails at chart render with `Custom SQL fields cannot contain sub-queries.` There is no migration step, and the error does not name the offending dataset column, so audit stored expressions before upgrading: either rewrite them without the sub-query, or set `ALLOW_ADHOC_SUBQUERY = True` to keep the previous behaviour for both stored and adhoc expressions.
|
||||
|
||||
@@ -782,13 +782,18 @@ Enable response caching for read-heavy workloads (dashboards/datasets that don't
|
||||
```python
|
||||
MCP_CACHE_CONFIG = {
|
||||
"enabled": True,
|
||||
# Cache keys don't include the requesting principal and hits are served
|
||||
# ahead of auth/RBAC, so a shared cache can return one caller's response
|
||||
# to another. Required for caching to actually start -- only appropriate
|
||||
# when every request is guaranteed to come from the same principal.
|
||||
"dangerously_share_cache_across_principals": True,
|
||||
"CACHE_KEY_PREFIX": "mcp_cache_",
|
||||
"call_tool_ttl": 3600,
|
||||
}
|
||||
MCP_STORE_CONFIG = {"enabled": True, "CACHE_REDIS_URL": "redis://redis:6379/0"}
|
||||
```
|
||||
|
||||
Mutating tools (`generate_chart`, `update_chart`, `execute_sql`, `generate_dashboard`) are always excluded from caching regardless of this setting.
|
||||
Every tool with a side effect (create/update/delete/execute) is always excluded from caching regardless of this setting -- see the `excluded_tools` default in `superset/mcp_service/mcp_config.py` for the current list.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+2
-2
@@ -60,9 +60,9 @@
|
||||
"@saucelabs/theme-github-codeblock": "^0.3.0",
|
||||
"@storybook/addon-docs": "^10.5.8",
|
||||
"@superset-ui/core": "^0.20.4",
|
||||
"@swc/core": "^1.15.47",
|
||||
"@swc/core": "^1.16.0",
|
||||
"antd": "^6.6.0",
|
||||
"baseline-browser-mapping": "^2.11.13",
|
||||
"baseline-browser-mapping": "^2.11.15",
|
||||
"caniuse-lite": "^1.0.30001809",
|
||||
"docusaurus-plugin-openapi-docs": "^5.2.0",
|
||||
"docusaurus-theme-openapi-docs": "^5.2.0",
|
||||
|
||||
Vendored
-7
@@ -3257,12 +3257,6 @@
|
||||
"description": "Override CSS for the dashboard.",
|
||||
"type": "string"
|
||||
},
|
||||
"custom_tags": {
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/Tag1"
|
||||
},
|
||||
"type": "array"
|
||||
},
|
||||
"dashboard_title": {
|
||||
"description": "A title for the dashboard.",
|
||||
"type": "string"
|
||||
@@ -17207,7 +17201,6 @@
|
||||
"charts": [],
|
||||
"created_on_delta_humanized": "string",
|
||||
"css": "string",
|
||||
"custom_tags": [],
|
||||
"dashboard_title": "string",
|
||||
"id": 1,
|
||||
"is_managed_externally": true,
|
||||
|
||||
+73
-73
@@ -4855,86 +4855,86 @@
|
||||
dependencies:
|
||||
apg-lite "^1.0.4"
|
||||
|
||||
"@swc/core-darwin-arm64@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-darwin-arm64/-/core-darwin-arm64-1.15.47.tgz#345ce6a1bf4033da189c2e3eff1244190195d15b"
|
||||
integrity sha512-GsoMtan3ojGGMGFbl31mmRu5ctZ56re8grGE8mO/OHJ8O+JRkzod02fe7X6ZQ8JvamA3imkEkx/h3u+vsOgPgA==
|
||||
"@swc/core-darwin-arm64@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-darwin-arm64/-/core-darwin-arm64-1.16.0.tgz#8c5a2af031c62ebcb6354aa6975bfb7eac895223"
|
||||
integrity sha512-SJQPl+xG/zB8bNjC/gTg3WOmOvz7EzlQD+VShfCKFYPNr2qvb+vATUY11vYEjnMWCn6wV8H8eAtjQrVflYyX5A==
|
||||
|
||||
"@swc/core-darwin-x64@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-darwin-x64/-/core-darwin-x64-1.15.47.tgz#f3debf50b5c1602bf392acb412bd33fd6d7e4f98"
|
||||
integrity sha512-leTi7Rx3KF4zcC637iqWgk9SoV8VXAD8ppQYXsep63px5A/UftOcxLN1pmr8Z1si/YvX90ompP/rHgpYkgwXWg==
|
||||
"@swc/core-darwin-x64@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-darwin-x64/-/core-darwin-x64-1.16.0.tgz#0d2496c0429d7e8bc45b50348adf9d105bb56793"
|
||||
integrity sha512-ql2JVch8V5t1i+HxiiuD4oVDI1dOku4/e3QiCkplONrm3SLitqNAP+nztHN51fSG2IgGuOwpAi3hgA+ukT5yQg==
|
||||
|
||||
"@swc/core-linux-arm-gnueabihf@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.15.47.tgz#14a247a12c6d3de1ee63fa4fdbf5a4302936b5d6"
|
||||
integrity sha512-hBqHuoWKKIsKmDBn9qVeWqj5GWZhtlcczVaqQmNRXsDfq+voR5CxKRfamA367QjJXtceYuliLFfEL8QsskRM2g==
|
||||
"@swc/core-linux-arm-gnueabihf@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.16.0.tgz#5f02a85842a04cd21f2ab9e8e67dc4b16f7024a4"
|
||||
integrity sha512-PcdDBaRbe39y37h1rXVkhNy7mEU7f8b34KD761C68R23EsfMsj5oDPVddRzGdSRAvwwSfH0WSNEHgYmc/AJipg==
|
||||
|
||||
"@swc/core-linux-arm64-gnu@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.15.47.tgz#3b8d09c481ae51c7b72d98fb6ce98f7b90065a1a"
|
||||
integrity sha512-TBxvRz+B4K205TWHHZxWVxkC2RFNP/Mz3PNcECBos5PsKwxjg3QSJzdoebr0VCf0Bfh8HOPldKxAP/8XkFe9gA==
|
||||
"@swc/core-linux-arm64-gnu@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.16.0.tgz#6264498c88c51649511c6b4af532d330d3cf0631"
|
||||
integrity sha512-t21IUztHQ/COucy7Kk9eIlehmq08H/hYq7aRA6fZox3S5ddi6TxWPK6e5S/+aTCf6+Od9qQ+LIpjHMiTy737vA==
|
||||
|
||||
"@swc/core-linux-arm64-musl@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.15.47.tgz#7ff2baa16e67b29017fdf7c6b69e40de7920ce1a"
|
||||
integrity sha512-3Yu3Uq/VgytqsPjTMbkPU1ExADytbdWbruJYhA584E9jrpE2Ki+R6VVPoZCeAVk1Cb7QxcRTgblw6bSa6a/R+w==
|
||||
"@swc/core-linux-arm64-musl@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.16.0.tgz#7a451eba69aa9a80799b9b8b9af46bf6f49803bd"
|
||||
integrity sha512-d9+iajbMB87b0umgbP+Gy3yBDSDgty4Q6H5pZ8fgTb/dOoKIwwynP4L4kvWCOFg2i49kxmAAUs1uJZh9s0E+RQ==
|
||||
|
||||
"@swc/core-linux-ppc64-gnu@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-ppc64-gnu/-/core-linux-ppc64-gnu-1.15.47.tgz#a3841982fe2eb2d889648c8e212b6d821db316d6"
|
||||
integrity sha512-wfdMi5IaOaNtmh2/6geRoxIdNfqylUZFdtzTKS655y1axWfIWyx7As74vv0wVdjeCIZ3WmCI9odDd4rUttXOSQ==
|
||||
"@swc/core-linux-ppc64-gnu@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-ppc64-gnu/-/core-linux-ppc64-gnu-1.16.0.tgz#99a7ba46a56190a52c646506e940dffe554c5d10"
|
||||
integrity sha512-QRpeKGOg+B0qmo3BFU+6rL/gpoKYYJ7OFSMf5DNMafohYZ/iq2qvAH9Gcrf8NxROj3iooKOVewJ+YgahH1nSLw==
|
||||
|
||||
"@swc/core-linux-s390x-gnu@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-s390x-gnu/-/core-linux-s390x-gnu-1.15.47.tgz#edbfd705d6285f7dce48915871478bc9603904c3"
|
||||
integrity sha512-3hHYBY0yx8Ez7GMRrkhXHQzMdR5IZA6Wq5Ee4svlgwvSECLpnAJ9+0AimEGUFDvuLwE7nV/2+PYe8+Nm4rvNcQ==
|
||||
"@swc/core-linux-s390x-gnu@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-s390x-gnu/-/core-linux-s390x-gnu-1.16.0.tgz#61473e056d1dd0d4690352a875c14f41bdd9f60a"
|
||||
integrity sha512-q+Vr/hmHCcRXT/WFzOJC+T6GGEEtq2iaTtmyLfxO7yzu4ckgcqSNkg9m181wfNhuMwfNBoBhOfwQCnLsGZ5F4g==
|
||||
|
||||
"@swc/core-linux-x64-gnu@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.15.47.tgz#e7f61a7771d6a9b5b274521ba61809b3d7644325"
|
||||
integrity sha512-TjfhjgP/jGCfFHYC3JQPhJA1HwErbIJ9JfREDc1KNkvY6P0LodCgKVIlQ5deeTbkG7ih3bF5PHJLuLpaZjdRyQ==
|
||||
"@swc/core-linux-x64-gnu@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.16.0.tgz#008fc149a9135bca92b1e1f63037e2612c4d0fb5"
|
||||
integrity sha512-DWVBc3QnpsSgKoq8N4rmZeZa5r/XrHdLkITsExN/tvTdqPtAPDPt+Ysy33OfgBlyN8lNe4xwsXWe6DXlRkJeRQ==
|
||||
|
||||
"@swc/core-linux-x64-musl@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.15.47.tgz#7c1ef8305444bcc7894de177fe225f2d8f3be609"
|
||||
integrity sha512-CQpS8Ge/avfjZd0UEwG/sds83Uu32deQXcV1Jo3jD0mmvQQqtYAjpsDZXugmheeAwmt+YIuoVtVHro8LMYHqsQ==
|
||||
"@swc/core-linux-x64-musl@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.16.0.tgz#4300ea0c63864dc3989ca0e956b4a5e4c666196c"
|
||||
integrity sha512-6XCgDSc1HPf/5dpjvABhKHICiBcsuZyW3hQMkn8sxel0TqprkJGp+H4iaBYIUTPixhrBub2hBPtfjcZLE6yL3w==
|
||||
|
||||
"@swc/core-win32-arm64-msvc@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.15.47.tgz#953856d26b28956d1a18ef10e5f221202b2cb8f1"
|
||||
integrity sha512-0W8IKHsUTYiT7G2RqtOoVWk+89yzZikIiDUb/sCK6BmQDBhN91hQSfyUtW12jhEWLzYgcfmisfsZrmZE+84U1A==
|
||||
"@swc/core-win32-arm64-msvc@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.16.0.tgz#1d4146b7c1aada2992692cdc72bb0b43a885136e"
|
||||
integrity sha512-T/+9VVCZJ3AKEth9IP3U9AJ2YscQq+7LUqRTvfR4a2q36+Ri22oOwUizpAKOqQ42vb2Y/kOa4TOcJOfHoDIT/w==
|
||||
|
||||
"@swc/core-win32-ia32-msvc@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.15.47.tgz#2743a5bccc49f252c23bad3135193640cbdcef3a"
|
||||
integrity sha512-ZIp49d2Z4/ka2jO9otOg4hDvTdPmp86kVOgS2M5FCPI7eKKZ1W0boxWn+8XeZrfERtFGW0AlMRm4JhlJa7l3NA==
|
||||
"@swc/core-win32-ia32-msvc@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.16.0.tgz#c5c2a60905ffa9e4647214bef75778f0c73ba0d4"
|
||||
integrity sha512-Pr1lsR/PMs8ndL0UWMrW8nLZ7H7sspIxBRDdjL8f+YJ/FJNASgzfunbVVXAqj0csgIJYHPZy+OW9smjFmk1Rcg==
|
||||
|
||||
"@swc/core-win32-x64-msvc@1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.15.47.tgz#9674ad0c9187b7cbe5cc3080b31b960d3ee688b9"
|
||||
integrity sha512-2h8Iek95vnixkBRCo+H8p09+Q5ll2NgSMFrWTy0iKt7+/t+8/T5mBpiT6c0ZxSS7wcWjwZ9sGZkK70tTSYHdDw==
|
||||
"@swc/core-win32-x64-msvc@1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.16.0.tgz#67dd85a90437e6fa9951cce7842f6cac3ec3f60d"
|
||||
integrity sha512-ktdeYLgOQdaonvsj5tJijqgpb0wk7gfF80wCFVA0kucI1hhSUIyfcGbjo5+9sdqv38OhMnTdLoA6xbqgOgPQjw==
|
||||
|
||||
"@swc/core@^1.15.40", "@swc/core@^1.15.47":
|
||||
version "1.15.47"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core/-/core-1.15.47.tgz#6226e842160e247eb79a9aeac1095ebddb56639f"
|
||||
integrity sha512-FbsO5JcfOjfH38W/rohBRBweJeERsAuIP4f377lmkmxTcq9exjtx4SkRuZY5CdfhR2CBVwDIJegBpJDffwNsOg==
|
||||
"@swc/core@^1.15.40", "@swc/core@^1.16.0":
|
||||
version "1.16.0"
|
||||
resolved "https://registry.yarnpkg.com/@swc/core/-/core-1.16.0.tgz#79cd13789725d3e3ad0df605dc88d9e255d7ebfd"
|
||||
integrity sha512-zSdvEHxBg00WhUNtW/u58hhcdR33gjtMQvOBo8F7POWJDyjRCt/miKfhidT3hCc/118RUwNnlEAmxiihFMbK4Q==
|
||||
dependencies:
|
||||
"@swc/counter" "^0.1.3"
|
||||
"@swc/types" "^0.1.27"
|
||||
"@swc/types" "^0.1.28"
|
||||
optionalDependencies:
|
||||
"@swc/core-darwin-arm64" "1.15.47"
|
||||
"@swc/core-darwin-x64" "1.15.47"
|
||||
"@swc/core-linux-arm-gnueabihf" "1.15.47"
|
||||
"@swc/core-linux-arm64-gnu" "1.15.47"
|
||||
"@swc/core-linux-arm64-musl" "1.15.47"
|
||||
"@swc/core-linux-ppc64-gnu" "1.15.47"
|
||||
"@swc/core-linux-s390x-gnu" "1.15.47"
|
||||
"@swc/core-linux-x64-gnu" "1.15.47"
|
||||
"@swc/core-linux-x64-musl" "1.15.47"
|
||||
"@swc/core-win32-arm64-msvc" "1.15.47"
|
||||
"@swc/core-win32-ia32-msvc" "1.15.47"
|
||||
"@swc/core-win32-x64-msvc" "1.15.47"
|
||||
"@swc/core-darwin-arm64" "1.16.0"
|
||||
"@swc/core-darwin-x64" "1.16.0"
|
||||
"@swc/core-linux-arm-gnueabihf" "1.16.0"
|
||||
"@swc/core-linux-arm64-gnu" "1.16.0"
|
||||
"@swc/core-linux-arm64-musl" "1.16.0"
|
||||
"@swc/core-linux-ppc64-gnu" "1.16.0"
|
||||
"@swc/core-linux-s390x-gnu" "1.16.0"
|
||||
"@swc/core-linux-x64-gnu" "1.16.0"
|
||||
"@swc/core-linux-x64-musl" "1.16.0"
|
||||
"@swc/core-win32-arm64-msvc" "1.16.0"
|
||||
"@swc/core-win32-ia32-msvc" "1.16.0"
|
||||
"@swc/core-win32-x64-msvc" "1.16.0"
|
||||
|
||||
"@swc/counter@^0.1.3":
|
||||
version "0.1.3"
|
||||
@@ -5021,10 +5021,10 @@
|
||||
"@swc/html-win32-ia32-msvc" "1.15.43"
|
||||
"@swc/html-win32-x64-msvc" "1.15.43"
|
||||
|
||||
"@swc/types@^0.1.27":
|
||||
version "0.1.27"
|
||||
resolved "https://registry.yarnpkg.com/@swc/types/-/types-0.1.27.tgz#12080b0c426dea450634f202d9a3c82ac396e793"
|
||||
integrity sha512-K6h3iUlqeM946U4sXFYeahefR1YBbXJvko+hv8WS8/0BNJ4OHiHRywMnQUJCqkR7Y9+hqQ1TvEpiKqUhz7NEFg==
|
||||
"@swc/types@^0.1.28":
|
||||
version "0.1.28"
|
||||
resolved "https://registry.yarnpkg.com/@swc/types/-/types-0.1.28.tgz#e3cd892383fba3b8904c40518bbe1265a50753f2"
|
||||
integrity sha512-V6Mnml8v09QALx6K0elJ7o9K/MkVDtW3t6L+7Ou/JcWtb3xwId2AH4FeOceySd2JaO87IMw4+6vSZxLm34LPbw==
|
||||
dependencies:
|
||||
"@swc/counter" "^0.1.3"
|
||||
|
||||
@@ -6522,10 +6522,10 @@ base64-js@^1.3.1, base64-js@^1.5.1:
|
||||
resolved "https://registry.yarnpkg.com/base64-js/-/base64-js-1.5.1.tgz#1b1b440160a5bf7ad40b650f095963481903930a"
|
||||
integrity sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==
|
||||
|
||||
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.13, baseline-browser-mapping@^2.9.19:
|
||||
version "2.11.13"
|
||||
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.13.tgz#660073103c1bee93e54df55f117b7528adf6af19"
|
||||
integrity sha512-k9HNuUVMlqVjQ9UHzfPjIqiDbWw7WqT1AoT7GL8VwvF3r0ZfArtgiSPAlmupyNquNgOJHTuH4CKYf8ttMTWBTQ==
|
||||
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.15, baseline-browser-mapping@^2.9.19:
|
||||
version "2.11.15"
|
||||
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.15.tgz#9c0cac93d7d304f3d61bb41088a102cd62e68676"
|
||||
integrity sha512-FwMjJJ7HnyZpWe+oWxegG0fezZyBZUagI5LZEoO3GCbtbKNwRfMH9Ue5d5v01PNePBy1QSfPSDTTeVL0Hb9EzA==
|
||||
|
||||
batch@0.6.1:
|
||||
version "0.6.1"
|
||||
|
||||
+7
-7
@@ -80,7 +80,7 @@ dependencies = [
|
||||
# marshmallow 4 compatibility: see superset/marshmallow_compatibility.py for a
|
||||
# Flask-AppBuilder workaround. Tracking issue:
|
||||
# https://github.com/apache/superset/issues/33162
|
||||
"marshmallow>=3.0, <5",
|
||||
"marshmallow>=4.3.1, <5",
|
||||
"marshmallow-union>=0.1.15.post1",
|
||||
"msgpack>=1.2.0, <1.3",
|
||||
"nh3>=0.3.5, <0.4",
|
||||
@@ -101,7 +101,7 @@ dependencies = [
|
||||
"python-dateutil",
|
||||
"python-dotenv", # optional dependencies for Flask but required for Superset, see https://flask.palletsprojects.com/en/stable/installation/#optional-dependencies
|
||||
"pygeohash",
|
||||
"pyarrow>=24.0.0, <26", # before upgrading pyarrow, check that all db dependencies support this, see e.g. https://github.com/apache/superset/pull/34693
|
||||
"pyarrow>=25.0.1, <26", # before upgrading pyarrow, check that all db dependencies support this, see e.g. https://github.com/apache/superset/pull/34693
|
||||
"pyyaml>=6.0.3, <7.0.0",
|
||||
"PyJWT>=2.4.0, <3.0",
|
||||
"redis>=5.0.0, <9.0",
|
||||
@@ -111,10 +111,10 @@ dependencies = [
|
||||
"sshtunnel>=0.4.0, <0.5",
|
||||
"simplejson>=4.1.1",
|
||||
"slack_sdk>=3.43.0, <4",
|
||||
"sqlalchemy>=2.0.0, <2.1",
|
||||
"sqlalchemy>=2.0.52, <2.1",
|
||||
"sqlalchemy-continuum>=1.6.0, <2.0.0",
|
||||
"sqlalchemy-utils>=0.42.1, <0.43", # expanding lowerbound to work with pydoris
|
||||
"sqlglot>=30.16.0, <31", # 30.16.0 adds Trino inline UDF IF/CASE routine statement parsing
|
||||
"sqlglot>=30.17.0, <31", # 30.16.0 adds Trino inline UDF IF/CASE routine statement parsing
|
||||
# newer pandas needs 0.9+
|
||||
"tabulate>=0.10.0, <1.0",
|
||||
"typing-extensions>=4.16.0, <5",
|
||||
@@ -197,7 +197,7 @@ fastmcp = [
|
||||
# landed (discussion #40273).
|
||||
firebird = ["sqlalchemy-firebird>=2.2.0"]
|
||||
firebolt = ["firebolt-sqlalchemy>=1.1.2, <2"]
|
||||
gevent = ["gevent>=26.7.0"]
|
||||
gevent = ["gevent>=26.8.0"]
|
||||
gsheets = ["shillelagh[gsheetsapi]>=1.4.5, <2"]
|
||||
hana = ["hdbcli==2.29.25", "sqlalchemy_hana==3.0.3"]
|
||||
hive = [
|
||||
@@ -232,7 +232,7 @@ playwright = ["playwright>=1.62.0, <2"]
|
||||
postgres = ["psycopg2-binary==2.9.12"]
|
||||
presto = ["pyhive[presto]>=0.6.5"]
|
||||
trino = ["trino>=0.338.0"]
|
||||
prophet = ["prophet>=1.3.0, <2"]
|
||||
prophet = ["prophet>=1.4.0, <2"]
|
||||
# sqlalchemy-redshift cuts hard from SQLAlchemy 1.4-only (0.8.x) to 2.0-only
|
||||
# (>=1.0.0) with no dual-compat release. Bumped now that Superset's own
|
||||
# SQLAlchemy 2.0 core bump has landed (discussion #40273).
|
||||
@@ -255,7 +255,7 @@ tdengine = [
|
||||
"taospy>=2.8.10",
|
||||
"taos-ws-py>=0.7.0"
|
||||
]
|
||||
teradata = ["teradatasql>=20.0.0.64"]
|
||||
teradata = ["teradatasql>=20.0.0.65"]
|
||||
thumbnails = [] # deprecated, will be removed in 7.0
|
||||
vertica = ["sqlalchemy-vertica-python>= 0.6.3, < 0.7"]
|
||||
netezza = ["nzalchemy>= 11.1.2, < 11.2"]
|
||||
|
||||
@@ -30,7 +30,7 @@ cryptography>=50.0.0,<51.0.0
|
||||
# Security: Snyk - XSS vulnerability in Mako templates
|
||||
mako>=1.4.1,<2.0.0
|
||||
# Security: CVE-2024-52338 (CRITICAL) - Deserialization of untrusted data in IPC/Parquet readers
|
||||
pyarrow>=24.0.0,<26.0.0
|
||||
pyarrow>=25.0.1,<26.0.0
|
||||
# Security: CVE-2026-27459 - pyopenssl certificate validation
|
||||
pyopenssl>=26.0.0,<27.0.0
|
||||
# Security: CVE-2026-25645 (MEDIUM) - Insecure Temporary File
|
||||
|
||||
@@ -222,7 +222,7 @@ markupsafe==3.0.2
|
||||
# mako
|
||||
# werkzeug
|
||||
# wtforms
|
||||
marshmallow==4.3.0
|
||||
marshmallow==4.3.1
|
||||
# via
|
||||
# apache-superset (pyproject.toml)
|
||||
# flask-appbuilder
|
||||
@@ -287,7 +287,7 @@ prison==0.2.1
|
||||
# via flask-appbuilder
|
||||
prompt-toolkit==3.0.51
|
||||
# via click-repl
|
||||
pyarrow==25.0.0
|
||||
pyarrow==25.0.1
|
||||
# via
|
||||
# -r requirements/base.in
|
||||
# apache-superset (pyproject.toml)
|
||||
@@ -381,7 +381,7 @@ six==1.17.0
|
||||
# wtforms-json
|
||||
slack-sdk==3.43.0
|
||||
# via apache-superset (pyproject.toml)
|
||||
sqlalchemy==2.0.51
|
||||
sqlalchemy==2.0.52
|
||||
# via
|
||||
# apache-superset (pyproject.toml)
|
||||
# alembic
|
||||
@@ -399,7 +399,7 @@ sqlalchemy-utils==0.42.1
|
||||
# apache-superset (pyproject.toml)
|
||||
# apache-superset-core
|
||||
# flask-appbuilder
|
||||
sqlglot==30.16.0
|
||||
sqlglot==30.17.0
|
||||
# via
|
||||
# apache-superset (pyproject.toml)
|
||||
# apache-superset-core
|
||||
|
||||
@@ -337,7 +337,7 @@ geopy==2.4.1
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# apache-superset
|
||||
gevent==26.7.0
|
||||
gevent==26.8.0
|
||||
# via apache-superset
|
||||
google-api-core==2.33.0
|
||||
# via
|
||||
@@ -434,8 +434,6 @@ importlib-metadata==8.7.0
|
||||
# via
|
||||
# keyring
|
||||
# opentelemetry-api
|
||||
importlib-resources==6.5.2
|
||||
# via prophet
|
||||
iniconfig==2.0.0
|
||||
# via pytest
|
||||
isodate==0.7.2
|
||||
@@ -530,7 +528,7 @@ markupsafe==3.0.2
|
||||
# mako
|
||||
# werkzeug
|
||||
# wtforms
|
||||
marshmallow==4.3.0
|
||||
marshmallow==4.3.1
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# apache-superset
|
||||
@@ -693,7 +691,7 @@ prompt-toolkit==3.0.51
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# click-repl
|
||||
prophet==1.3.0
|
||||
prophet==1.4.0
|
||||
# via apache-superset
|
||||
proto-plus==1.25.0
|
||||
# via google-api-core
|
||||
@@ -711,7 +709,7 @@ psycopg2-binary==2.9.12
|
||||
# via apache-superset
|
||||
py-key-value-aio==0.4.4
|
||||
# via fastmcp-slim
|
||||
pyarrow==25.0.0
|
||||
pyarrow==25.0.1
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# apache-superset
|
||||
@@ -950,7 +948,7 @@ slack-sdk==3.43.0
|
||||
# apache-superset
|
||||
sniffio==1.3.1
|
||||
# via anyio
|
||||
sqlalchemy==2.0.51
|
||||
sqlalchemy==2.0.52
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# alembic
|
||||
@@ -976,7 +974,7 @@ sqlalchemy-utils==0.42.1
|
||||
# apache-superset
|
||||
# apache-superset-core
|
||||
# flask-appbuilder
|
||||
sqlglot==30.16.0
|
||||
sqlglot==30.17.0
|
||||
# via
|
||||
# -c requirements/base-constraint.txt
|
||||
# apache-superset
|
||||
|
||||
@@ -42,6 +42,7 @@ RETRYABLE_STATUS_CODES: frozenset[int] = frozenset({429})
|
||||
PATTERNS = {
|
||||
"python": [
|
||||
r"^\.github/workflows/.*python",
|
||||
r"^\.github/workflows/frontend-bundle-size-nightly\.yml$",
|
||||
r"^\.github/workflows/scheduled-docker-image-refresh\.yml$",
|
||||
r"^docker-compose-image-tag\.yml$",
|
||||
r"^tests/",
|
||||
|
||||
@@ -122,6 +122,12 @@ export const timeComparisonControls: ({
|
||||
}
|
||||
return newState;
|
||||
},
|
||||
// Re-run this control's validation whenever `time_compare` changes so
|
||||
// the "date required" error clears once a non-custom shift is picked.
|
||||
// Without it the stale error survives in Redux (see the
|
||||
// dependantControls path in exploreReducer's SET_FIELD_VALUE handler)
|
||||
// and blocks further chart updates until a page refresh.
|
||||
validationDependencies: ['time_compare'],
|
||||
},
|
||||
},
|
||||
],
|
||||
|
||||
@@ -71,6 +71,9 @@ export type AntdExposedProps = Pick<
|
||||
| 'virtual'
|
||||
| 'getPopupContainer'
|
||||
| 'menuItemSelectedIcon'
|
||||
// lets a caller with long option labels stop the popup inheriting the
|
||||
// trigger's width, which otherwise truncates every option
|
||||
| 'popupMatchSelectWidth'
|
||||
>;
|
||||
|
||||
export type SelectOptionsType = Exclude<AntdProps['options'], undefined>;
|
||||
|
||||
@@ -96,16 +96,57 @@ export class Menu {
|
||||
itemText: string,
|
||||
options?: { timeout?: number },
|
||||
): Promise<void> {
|
||||
const popup = await this.openSubmenu(submenuText, {
|
||||
timeout: options?.timeout,
|
||||
itemText,
|
||||
});
|
||||
|
||||
// Use dispatchEvent instead of click to bypass viewport and pointer interception
|
||||
// issues. Ant Design renders submenu popups in a portal that can be positioned
|
||||
// outside the viewport or behind chart content (e.g., large tables with z-index).
|
||||
await popup.getByText(itemText, { exact: true }).dispatchEvent('click');
|
||||
}
|
||||
|
||||
/**
|
||||
* Opens a submenu and returns its popup locator, without selecting an item.
|
||||
* Useful when the caller needs to read the popup's contents (e.g. the set of
|
||||
* offered items) rather than clicking a known item.
|
||||
*
|
||||
* Uses hover as primary approach, falls back to keyboard then dispatchEvent -
|
||||
* same fallback chain as {@link selectSubmenuItem}.
|
||||
*
|
||||
* @param submenuText - The text of the submenu to open (e.g., "Download")
|
||||
* @param options - Optional timeout, an `itemText` to scope the popup lookup
|
||||
* to (useful when multiple submenu popups could otherwise match), and a
|
||||
* `popupSelector` override for submenus that render with an additional,
|
||||
* more specific class than the generic Ant Design popup class.
|
||||
*/
|
||||
async openSubmenu(
|
||||
submenuText: string,
|
||||
options?: { timeout?: number; itemText?: string; popupSelector?: string },
|
||||
): Promise<Locator> {
|
||||
const timeout = options?.timeout ?? TIMEOUT.FORM_LOAD;
|
||||
const matchPopup = (): Locator => {
|
||||
const base = this.page.locator(
|
||||
options?.popupSelector ?? Menu.SELECTORS.SUBMENU_POPUP,
|
||||
);
|
||||
return options?.itemText
|
||||
? base.filter({ hasText: options.itemText })
|
||||
: base;
|
||||
};
|
||||
|
||||
// Try hover first (most natural user interaction)
|
||||
let popup = await this.openSubmenuWithHover(submenuText, itemText, timeout);
|
||||
let popup = await this.openSubmenuWithHover(
|
||||
submenuText,
|
||||
matchPopup,
|
||||
timeout,
|
||||
);
|
||||
|
||||
// Fallback to keyboard navigation
|
||||
if (!popup) {
|
||||
popup = await this.openSubmenuWithKeyboard(
|
||||
submenuText,
|
||||
itemText,
|
||||
matchPopup,
|
||||
timeout,
|
||||
);
|
||||
}
|
||||
@@ -114,7 +155,7 @@ export class Menu {
|
||||
if (!popup) {
|
||||
popup = await this.openSubmenuWithDispatchEvent(
|
||||
submenuText,
|
||||
itemText,
|
||||
matchPopup,
|
||||
timeout,
|
||||
);
|
||||
}
|
||||
@@ -125,10 +166,7 @@ export class Menu {
|
||||
);
|
||||
}
|
||||
|
||||
// Use dispatchEvent instead of click to bypass viewport and pointer interception
|
||||
// issues. Ant Design renders submenu popups in a portal that can be positioned
|
||||
// outside the viewport or behind chart content (e.g., large tables with z-index).
|
||||
await popup.getByText(itemText, { exact: true }).dispatchEvent('click');
|
||||
return popup;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -137,17 +175,14 @@ export class Menu {
|
||||
*/
|
||||
private async openSubmenuWithHover(
|
||||
submenuText: string,
|
||||
itemText: string,
|
||||
matchPopup: () => Locator,
|
||||
timeout: number,
|
||||
): Promise<Locator | null> {
|
||||
try {
|
||||
const submenuTitle = this.getSubmenuTitle(submenuText);
|
||||
await submenuTitle.hover();
|
||||
|
||||
// Find the popup that contains the expected item (scopes to correct popup)
|
||||
const popup = this.page
|
||||
.locator(Menu.SELECTORS.SUBMENU_POPUP)
|
||||
.filter({ hasText: itemText });
|
||||
const popup = matchPopup();
|
||||
await popup.waitFor({ state: 'visible', timeout });
|
||||
|
||||
// Allow Ant Design's slide-in animation to complete before clicking.
|
||||
@@ -166,7 +201,7 @@ export class Menu {
|
||||
*/
|
||||
private async openSubmenuWithKeyboard(
|
||||
submenuText: string,
|
||||
itemText: string,
|
||||
matchPopup: () => Locator,
|
||||
timeout: number,
|
||||
): Promise<Locator | null> {
|
||||
try {
|
||||
@@ -174,9 +209,7 @@ export class Menu {
|
||||
await submenuTitle.focus();
|
||||
await this.page.keyboard.press('ArrowRight');
|
||||
|
||||
const popup = this.page
|
||||
.locator(Menu.SELECTORS.SUBMENU_POPUP)
|
||||
.filter({ hasText: itemText });
|
||||
const popup = matchPopup();
|
||||
await popup.waitFor({ state: 'visible', timeout });
|
||||
|
||||
return popup;
|
||||
@@ -191,7 +224,7 @@ export class Menu {
|
||||
*/
|
||||
private async openSubmenuWithDispatchEvent(
|
||||
submenuText: string,
|
||||
itemText: string,
|
||||
matchPopup: () => Locator,
|
||||
timeout: number,
|
||||
): Promise<Locator | null> {
|
||||
try {
|
||||
@@ -214,9 +247,7 @@ export class Menu {
|
||||
);
|
||||
});
|
||||
|
||||
const popup = this.page
|
||||
.locator(Menu.SELECTORS.SUBMENU_POPUP)
|
||||
.filter({ hasText: itemText });
|
||||
const popup = matchPopup();
|
||||
await popup.waitFor({ state: 'visible', timeout });
|
||||
|
||||
return popup;
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
|
||||
import { Locator, Page } from '@playwright/test';
|
||||
import { Modal } from '../core';
|
||||
|
||||
/**
|
||||
* The "Drill to detail" modal (`DrillDetailModal.tsx`), opened from a chart's
|
||||
* "More Options" menu or its right-click context menu. Renders the chart's
|
||||
* underlying sample rows, optionally scoped to a drilled-by value, via the
|
||||
* `/datasource/samples` API.
|
||||
*/
|
||||
export class DrillDetailModal extends Modal {
|
||||
private static readonly SELECTORS = {
|
||||
CLOSE_BUTTON: '[data-test="close-drilltodetail-modal"]',
|
||||
ROW_COUNT_LABEL: '[data-test="row-count-label"]',
|
||||
METADATA_BAR: '[data-test="metadata-bar"]',
|
||||
FILTER_COLUMN: '[data-test="filter-col"]',
|
||||
FILTER_VALUE: '[data-test="filter-val"]',
|
||||
PAGE_ITEM: '.ant-pagination-item',
|
||||
ACTIVE_PAGE_ITEM: '.ant-pagination-item-active',
|
||||
GRID_CELL: '.virtual-table-cell',
|
||||
} as const;
|
||||
|
||||
private readonly specificLocator: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
super(page);
|
||||
// Matched by accessible name rather than a data-test: the antd Modal's own
|
||||
// data-test (`${name}-modal`) is derived from this same i18n'd `name`
|
||||
// prop, so it isn't a locale-independent alternative. No data-test exists
|
||||
// on the dialog root itself.
|
||||
this.specificLocator = page.getByRole('dialog', {
|
||||
name: /^Drill to detail:/,
|
||||
});
|
||||
}
|
||||
|
||||
override get element(): Locator {
|
||||
return this.specificLocator;
|
||||
}
|
||||
|
||||
/**
|
||||
* The applied-filter value tags (`<col>=<val>`). Empty when the drill was
|
||||
* whole-chart (no row/point-level filter applied).
|
||||
*/
|
||||
get filterValues(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.FILTER_VALUE);
|
||||
}
|
||||
|
||||
/** The applied-filter chip(s); each is closable via its own "Close" icon. */
|
||||
get filterColumns(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.FILTER_COLUMN);
|
||||
}
|
||||
|
||||
/** Row-count label above the results grid, e.g. "1-50 of 500 rows". */
|
||||
get rowCountLabel(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.ROW_COUNT_LABEL);
|
||||
}
|
||||
|
||||
/** The metadata bar (column/row summary) shown once samples have loaded. */
|
||||
get metadataBar(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.METADATA_BAR);
|
||||
}
|
||||
|
||||
/** Pagination page-number items below the results grid. */
|
||||
get pageItems(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.PAGE_ITEM);
|
||||
}
|
||||
|
||||
/** The currently active pagination page-number item. */
|
||||
get activePageItem(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.ACTIVE_PAGE_ITEM);
|
||||
}
|
||||
|
||||
/** Cells of the virtualized results grid. */
|
||||
get gridCells(): Locator {
|
||||
return this.element.locator(DrillDetailModal.SELECTORS.GRID_CELL);
|
||||
}
|
||||
|
||||
/**
|
||||
* Removes the first applied filter by clicking its chip's Close icon,
|
||||
* re-fetching the unfiltered samples.
|
||||
*/
|
||||
async clearFirstFilter(): Promise<void> {
|
||||
await this.filterColumns.first().getByLabel('Close').click();
|
||||
}
|
||||
|
||||
/** Navigates to the given 1-indexed pagination page. */
|
||||
async goToPage(pageNumber: number): Promise<void> {
|
||||
await this.pageItems.nth(pageNumber - 1).click();
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-fetches the current samples query, resetting pagination to page 1.
|
||||
*
|
||||
* Matched by accessible name: the Reload icon carries an i18n'd
|
||||
* `aria-label` (`t('Reload')`) and no data-test, so this breaks in
|
||||
* non-English locales the same way `DrillDetailModal.tsx`'s dialog `name`
|
||||
* does above; the predecessor Cypress test used the same English string.
|
||||
*/
|
||||
async reload(): Promise<void> {
|
||||
await this.element.getByRole('button', { name: 'Reload' }).click();
|
||||
}
|
||||
|
||||
/**
|
||||
* Closes the modal via its footer Close button.
|
||||
*
|
||||
* Targets the button by data-test rather than Modal.clickFooterButton,
|
||||
* which finds buttons by their visible text. The button label is i18n'd
|
||||
* ("Close" / "Fermer" / …), so name-based lookups break in non-English
|
||||
* locales; see DeleteConfirmationModal.clickDelete for the same rationale.
|
||||
*/
|
||||
async close(): Promise<void> {
|
||||
await this.element.locator(DrillDetailModal.SELECTORS.CLOSE_BUTTON).click();
|
||||
await this.waitForHidden();
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,7 @@
|
||||
export { ChartPropertiesModal } from './ChartPropertiesModal';
|
||||
export { ConfirmDialog } from './ConfirmDialog';
|
||||
export { DeleteConfirmationModal } from './DeleteConfirmationModal';
|
||||
export { DrillDetailModal } from './DrillDetailModal';
|
||||
export { DuplicateDatasetModal } from './DuplicateDatasetModal';
|
||||
export { EditDatasetModal } from './EditDatasetModal';
|
||||
export { ImportDatasetModal } from './ImportDatasetModal';
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
import { Page, Download, Locator, expect } from '@playwright/test';
|
||||
import { Button, Input, Menu, Tabs } from '../components/core';
|
||||
import { DashboardFilterBar } from '../components/dashboard';
|
||||
import { DrillDetailModal } from '../components/modals';
|
||||
import { gotoWithRetry } from '../helpers/navigation';
|
||||
import { html5DragAndDrop } from '../helpers/dnd';
|
||||
import { TIMEOUT } from '../utils/constants';
|
||||
@@ -454,4 +455,124 @@ export class DashboardPage {
|
||||
|
||||
return { heightBefore: boxBefore.height, heightAfter: boxAfter.height };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Drill to detail
|
||||
//
|
||||
// Charts that implement the DRILL_TO_DETAIL behavior expose two entry points:
|
||||
// the chart's "More Options" header menu, and a right-click context menu on
|
||||
// the chart body (a cell, the big-number value, or a canvas data point). Both
|
||||
// open the same DrillDetailModal, which renders the underlying sample rows for
|
||||
// the (optionally filtered) chart by calling the `/datasource/samples` API.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Open the "Drill to detail" item from a chart's "More Options" header menu.
|
||||
* This is the whole-chart entry point (no row-level filters applied).
|
||||
*/
|
||||
async openDrillToDetailFromMenu(chartId: number): Promise<void> {
|
||||
const moreOptions = new Button(
|
||||
this.page,
|
||||
this.getChart(chartId).getByLabel('More Options', { exact: true }),
|
||||
);
|
||||
await moreOptions.click();
|
||||
await this.page
|
||||
.getByRole('menuitem', { name: 'Drill to detail', exact: true })
|
||||
.click();
|
||||
}
|
||||
|
||||
/**
|
||||
* The DrillDetailModal dialog (titled "Drill to detail: <chart name>").
|
||||
*/
|
||||
drillModal(): DrillDetailModal {
|
||||
return new DrillDetailModal(this.page);
|
||||
}
|
||||
|
||||
/**
|
||||
* Click the plain "Drill to detail" item in an open chart context menu
|
||||
* (whole chart, no row-level filter).
|
||||
*/
|
||||
async contextMenuDrillToDetail(): Promise<void> {
|
||||
await this.page
|
||||
.getByRole('menuitem', { name: 'Drill to detail', exact: true })
|
||||
.click();
|
||||
}
|
||||
|
||||
/**
|
||||
* The "Drill to detail by" submenu parent (title) in an open context menu.
|
||||
* Targeted by its submenu-title element rather than role+name because antd
|
||||
* appends the arrow-icon name ("right") to the accessible name, and the leaf
|
||||
* items ("Drill to detail by boy") would otherwise match a role+name lookup.
|
||||
*/
|
||||
drillBySubmenuTitle(): Locator {
|
||||
return this.page.locator('.ant-dropdown-menu-submenu-title', {
|
||||
hasText: 'Drill to detail by',
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The chart context menu's Menu component, scoped to the open context
|
||||
* menu's root. Used to open the "Drill to detail by" submenu robustly:
|
||||
* plain hover is not reliably picked up by Ant Design's submenu trigger in
|
||||
* headless Chromium, so this falls back to keyboard and dispatchEvent - see
|
||||
* {@link Menu.openSubmenu}.
|
||||
*/
|
||||
private contextMenu(): Menu {
|
||||
return new Menu(this.page, '[data-test="chart-context-menu"]');
|
||||
}
|
||||
|
||||
/**
|
||||
* Opens the "Drill to detail by" submenu and returns its popup, containing
|
||||
* the leaf value items (e.g. "Drill to detail by boy").
|
||||
*/
|
||||
private openDrillBySubmenu(): Promise<Locator> {
|
||||
return this.contextMenu().openSubmenu('Drill to detail by', {
|
||||
popupSelector: '.chart-context-submenu',
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* From an open chart context menu, open the "Drill to detail by" submenu and
|
||||
* click the entry for a specific value (e.g. "boy", "1965", "all").
|
||||
*/
|
||||
async contextMenuDrillToDetailBy(value: string): Promise<void> {
|
||||
const popup = await this.openDrillBySubmenu();
|
||||
// Use dispatchEvent instead of click to bypass viewport and pointer
|
||||
// interception issues - see Menu.selectSubmenuItem.
|
||||
await popup
|
||||
.getByRole('menuitem', {
|
||||
name: `Drill to detail by ${value}`,
|
||||
exact: true,
|
||||
})
|
||||
.dispatchEvent('click');
|
||||
}
|
||||
|
||||
/**
|
||||
* From an open chart context menu, open "Drill to detail by" and return the
|
||||
* concrete values offered by the submenu (e.g. ["1965", "boy"]), skipping the
|
||||
* aggregate "all" entry. Used by canvas charts where the value under the
|
||||
* cursor is data-dependent: the test drills by whatever the menu actually
|
||||
* offers and asserts that same value round-trips into the modal, which keeps
|
||||
* the assertion independent of exact pixel/slice geometry.
|
||||
*
|
||||
* Reads rendered (HTML-stripped) menu text rather than the item's
|
||||
* `aria-label`, which carries the raw, unstripped formatted value
|
||||
* (`useDrillDetailMenuItems`). The two only diverge for formatted values
|
||||
* that contain HTML markup; callers pass the returned value both to
|
||||
* `contextMenuDrillToDetailBy` (accessible-name lookup) and to a
|
||||
* displayed-text assertion on the modal's filter chip, so a value straddling
|
||||
* both uses only works when it's markup-free. Every value currently offered
|
||||
* by this dashboard's charts is a plain string, so this hasn't been
|
||||
* reachable in practice; revisit if a test starts exercising HTML-formatted
|
||||
* dimension values.
|
||||
*/
|
||||
async drillByOfferedValues(): Promise<string[]> {
|
||||
const popup = await this.openDrillBySubmenu();
|
||||
const items = popup.locator('[role="menuitem"]');
|
||||
await items.first().waitFor();
|
||||
const labels = await items.allInnerTexts();
|
||||
return labels
|
||||
.map(l => l.replace(/^Drill to detail by\s*/i, '').trim())
|
||||
.filter(v => v.length > 0 && v.toLowerCase() !== 'all');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,747 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* E2E migration of the Cypress "Drill to detail modal" suite
|
||||
* (dashboard/drilltodetail.test.ts).
|
||||
*
|
||||
* Drill to detail lets a viewer open a modal of the underlying sample rows for a
|
||||
* chart — optionally filtered to a single data point — by either the chart's
|
||||
* "More Options" header menu or a right-click context menu on the chart body.
|
||||
* The modal calls the real `/datasource/samples` API, so this is genuinely
|
||||
* end-to-end: each test API-builds a hermetic dashboard from the `birth_names`
|
||||
* dataset, renders it in the browser, drives the real menus, and asserts the
|
||||
* resulting backend round-trip (the samples POST and the filter the modal
|
||||
* applies).
|
||||
*
|
||||
* Why the original suite was fully `describe.skip`:
|
||||
* "it has issues with autoscrolling and the locked title flakes intricately
|
||||
* when the rightClick is obstructed by the title."
|
||||
* That failure mode is Cypress-specific — Cypress auto-scrolls the target under
|
||||
* the sticky chart header before every action. Playwright scrolls once and the
|
||||
* target stays put, so the entry points are portable here.
|
||||
*
|
||||
* What is migrated, and how it is kept deterministic:
|
||||
* - Modal mechanics (open from header menu, pagination, reload-resets-page)
|
||||
* and the no-filter big-number drill use stable DOM elements.
|
||||
* - Table and Pivot drills right-click real DOM cells (no canvas pixels).
|
||||
* - Canvas (echarts) charts — Pie, Line, Scatter, generic/smooth/step
|
||||
* time-series, Mixed, Box plot, Funnel, Gauge, Treemap — DID rely on
|
||||
* hard-coded pixel coordinates in Cypress to land on a specific slice/point.
|
||||
* Instead of reproducing those brittle pixels, these tests scan a stable
|
||||
* region of the canvas (see `rightClickCanvasDatum`), read whichever value
|
||||
* the drill submenu actually offers for the point under the cursor, drill by
|
||||
* that value, and assert the SAME value round-trips into the modal filter.
|
||||
* This exercises the full canvas → contextmenu → datum → samples pipeline
|
||||
* while staying independent of exact geometry. `Big Number with Trendline`
|
||||
* drills the whole chart (no datum filter), like `Big Number`.
|
||||
*
|
||||
* Excluded (kept out, matching the original's own `describe.skip`s): Bar, Area,
|
||||
* World Map, Radar — skipped upstream for chart-specific reasons.
|
||||
*/
|
||||
import {
|
||||
testWithAssets,
|
||||
expect,
|
||||
type TestAssets,
|
||||
} from '../../helpers/fixtures';
|
||||
import type { Page, TestInfo } from '@playwright/test';
|
||||
import { TIMEOUT } from '../../utils/constants';
|
||||
import { DashboardPage } from '../../pages/DashboardPage';
|
||||
import { createDashboardWithCharts } from './dashboard-test-helpers';
|
||||
|
||||
const DATASET_NAME = 'birth_names';
|
||||
|
||||
/**
|
||||
* Parse a RowCountLabel value ("75.7k rows", "1,234 rows") into a number so
|
||||
* tests can assert the *invariant* (filtered < unfiltered) without hard-coding
|
||||
* the dataset-specific totals the original Cypress suite baked in.
|
||||
*/
|
||||
function parseRowCount(text: string): number {
|
||||
const m = text.match(/([\d.,]+)\s*([kKmM]?)/);
|
||||
if (!m) return NaN;
|
||||
let n = parseFloat(m[1].replace(/,/g, ''));
|
||||
const suffix = m[2].toLowerCase();
|
||||
if (suffix === 'k') n *= 1e3;
|
||||
if (suffix === 'm') n *= 1e6;
|
||||
return n;
|
||||
}
|
||||
|
||||
interface ChartSpec {
|
||||
vizType: string;
|
||||
chartNamePrefix: string;
|
||||
params: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* API-build a hermetic single-chart dashboard from birth_names and return its
|
||||
* dashboard and chart ids. Thin single-chart wrapper around
|
||||
* `createDashboardWithCharts`, the build helper shared by the other migrated
|
||||
* dashboard specs — reused here rather than hand-rolling position-json and id
|
||||
* extraction again.
|
||||
*/
|
||||
async function buildSingleChartDashboard(
|
||||
page: Page,
|
||||
testAssets: TestAssets,
|
||||
testInfo: TestInfo,
|
||||
spec: ChartSpec,
|
||||
): Promise<{ dashboardId: number; chartId: number }> {
|
||||
const { dashboardId, charts } = await createDashboardWithCharts(
|
||||
page,
|
||||
testAssets,
|
||||
testInfo,
|
||||
{
|
||||
datasetName: DATASET_NAME,
|
||||
chartNamePrefix: spec.chartNamePrefix,
|
||||
dashboardTitlePrefix: spec.chartNamePrefix,
|
||||
chartSpecs: [{ viz_type: spec.vizType, params: spec.params }],
|
||||
},
|
||||
);
|
||||
return { dashboardId, chartId: charts[0].id };
|
||||
}
|
||||
|
||||
/**
|
||||
* Right-click an echarts canvas until a data point is hit — i.e. until the
|
||||
* context menu offers an *enabled* "Drill to detail by" submenu (a miss renders
|
||||
* that item disabled, as a plain menu item rather than a submenu title).
|
||||
*
|
||||
* echarts renders to a single canvas, so there is no per-datum DOM element to
|
||||
* target and the exact pixel of a mark depends on chart geometry (donut hole,
|
||||
* legend size, axis padding). Rather than hard-code Cypress's brittle pixel
|
||||
* coordinates, this scans a small set of candidate points — a radial ring for
|
||||
* pie/radial charts, a grid for cartesian charts — and stops at the first that
|
||||
* lands on a mark. The drill value is then whatever that mark represents, so the
|
||||
* caller asserts a value round-trip rather than a specific geometry.
|
||||
*/
|
||||
async function rightClickCanvasDatum(
|
||||
page: Page,
|
||||
dashboard: DashboardPage,
|
||||
canvas: ReturnType<Page['locator']>,
|
||||
pattern: 'ring' | 'grid' | 'dense',
|
||||
): Promise<void> {
|
||||
const box = await canvas.boundingBox();
|
||||
if (!box) throw new Error('canvas has no bounding box');
|
||||
|
||||
const ringPoints = (): Array<{ x: number; y: number }> => {
|
||||
const pts: Array<{ x: number; y: number }> = [];
|
||||
const cx = box.width / 2;
|
||||
const cy = box.height / 2;
|
||||
const minSide = Math.min(box.width, box.height);
|
||||
for (const rf of [0.3, 0.22, 0.38]) {
|
||||
for (let a = 0; a < 360; a += 45) {
|
||||
const rad = (a * Math.PI) / 180;
|
||||
pts.push({
|
||||
x: cx + Math.cos(rad) * minSide * rf,
|
||||
y: cy + Math.sin(rad) * minSide * rf,
|
||||
});
|
||||
}
|
||||
}
|
||||
return pts;
|
||||
};
|
||||
const gridPoints = (): Array<{ x: number; y: number }> => {
|
||||
const pts: Array<{ x: number; y: number }> = [];
|
||||
for (const yf of [0.5, 0.4, 0.6, 0.3, 0.7]) {
|
||||
for (const xf of [0.3, 0.45, 0.6, 0.2, 0.75]) {
|
||||
pts.push({ x: box.width * xf, y: box.height * yf });
|
||||
}
|
||||
}
|
||||
return pts;
|
||||
};
|
||||
|
||||
// 'dense' merges both scans for radial/stacked shapes (gauge, funnel, box
|
||||
// plot) whose drillable marks don't fall neatly on a single ring or grid.
|
||||
let candidates: Array<{ x: number; y: number }>;
|
||||
if (pattern === 'ring') candidates = ringPoints();
|
||||
else if (pattern === 'grid') candidates = gridPoints();
|
||||
else candidates = [...gridPoints(), ...ringPoints()];
|
||||
|
||||
// The submenu *title* element only exists when "Drill to detail by" is an
|
||||
// enabled submenu (a real datum was hit); a miss renders a disabled item.
|
||||
const enabledDrillBy = dashboard.drillBySubmenuTitle();
|
||||
const contextMenu = page.locator('[data-test="chart-context-menu"]');
|
||||
|
||||
for (const pt of candidates) {
|
||||
await canvas.click({ button: 'right', position: pt });
|
||||
const hit = await enabledDrillBy
|
||||
.waitFor({ state: 'visible', timeout: 400 })
|
||||
.then(() => true)
|
||||
.catch(() => false);
|
||||
if (hit) return;
|
||||
await page.keyboard.press('Escape');
|
||||
// Wait for the portal to actually close before the next right-click;
|
||||
// otherwise a still-open (or mid-close-animation) menu can make the
|
||||
// next click/locator behave nondeterministically on slower/contended CI.
|
||||
await contextMenu
|
||||
.waitFor({ state: 'hidden', timeout: 400 })
|
||||
.catch(() => {});
|
||||
}
|
||||
throw new Error(
|
||||
`no drillable datum found on canvas after scanning ${candidates.length} points`,
|
||||
);
|
||||
}
|
||||
|
||||
/** A samples POST fired (proves the modal hit the real backend). */
|
||||
function expectSamplesPost(page: Page) {
|
||||
return page.waitForResponse(
|
||||
r =>
|
||||
r.url().includes('/datasource/samples') &&
|
||||
r.request().method() === 'POST',
|
||||
{ timeout: TIMEOUT.API_RESPONSE },
|
||||
);
|
||||
}
|
||||
|
||||
async function loadDashboardWithChart(
|
||||
dashboard: DashboardPage,
|
||||
dashboardId: number,
|
||||
chartId: number,
|
||||
): Promise<void> {
|
||||
await dashboard.gotoById(dashboardId);
|
||||
await dashboard.waitForLoad();
|
||||
await dashboard
|
||||
.getChart(chartId)
|
||||
.locator('[data-test="chart-container"]')
|
||||
.first()
|
||||
.waitFor({ state: 'visible', timeout: TIMEOUT.QUERY_EXECUTION });
|
||||
await dashboard.waitForChartsToLoad();
|
||||
}
|
||||
|
||||
/**
|
||||
* From an already-open "Drill to detail by" submenu, drill by the first
|
||||
* offered value and assert that same value lands in the modal filter. The
|
||||
* shared tail of every "drill by whatever value is under the cursor" test —
|
||||
* canvas charts and the pivot table alike, which differ only in how they open
|
||||
* the submenu in the first place.
|
||||
*/
|
||||
async function drillByFirstOfferedValueAndAssert(
|
||||
page: Page,
|
||||
dashboard: DashboardPage,
|
||||
): Promise<void> {
|
||||
const offered = await dashboard.drillByOfferedValues();
|
||||
expect(offered.length).toBeGreaterThan(0);
|
||||
const [value] = offered;
|
||||
const samples = expectSamplesPost(page);
|
||||
await dashboard.contextMenuDrillToDetailBy(value);
|
||||
await samples;
|
||||
|
||||
await expect(dashboard.drillModal().element).toBeVisible();
|
||||
await expect(dashboard.drillModal().filterValues.first()).toContainText(
|
||||
value,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Full canvas-drill round-trip for an echarts (canvas-rendered) chart: build a
|
||||
* hermetic single-chart dashboard, render it, right-click a real datum, drill by
|
||||
* whatever value the submenu offers under the cursor, and assert that same value
|
||||
* lands in the modal filter. Geometry-independent — see rightClickCanvasDatum.
|
||||
* Reused across every canvas viz type so each migrated chart is a thin caller.
|
||||
*/
|
||||
async function expectCanvasDrillByValueRoundTrips(
|
||||
page: Page,
|
||||
testAssets: TestAssets,
|
||||
testInfo: TestInfo,
|
||||
spec: ChartSpec,
|
||||
pattern: 'ring' | 'grid' | 'dense',
|
||||
): Promise<void> {
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testInfo,
|
||||
spec,
|
||||
);
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
const canvas = dashboard.getChart(chartId).locator('canvas').first();
|
||||
await expect(canvas).toBeVisible();
|
||||
await rightClickCanvasDatum(page, dashboard, canvas, pattern);
|
||||
|
||||
await drillByFirstOfferedValueAndAssert(page, dashboard);
|
||||
}
|
||||
|
||||
/**
|
||||
* Right-click a big-number chart's rendered value to open its context menu,
|
||||
* drill the whole chart (no row/point filter), and assert the modal opened
|
||||
* with no filter tags and a real row count. Shared by Big Number and Big
|
||||
* Number with Trendline, which differ only in their chart params.
|
||||
*/
|
||||
async function expectWholeChartDrillFromContextMenu(
|
||||
page: Page,
|
||||
dashboard: DashboardPage,
|
||||
chartId: number,
|
||||
): Promise<void> {
|
||||
const samples = expectSamplesPost(page);
|
||||
await dashboard
|
||||
.getChart(chartId)
|
||||
.locator('.header-line')
|
||||
.click({ button: 'right' });
|
||||
await dashboard.contextMenuDrillToDetail();
|
||||
await samples;
|
||||
|
||||
await expect(dashboard.drillModal().element).toBeVisible();
|
||||
// Whole-chart drill: no per-value filter tag.
|
||||
await expect(dashboard.drillModal().filterValues).toHaveCount(0);
|
||||
await expect(dashboard.drillModal().rowCountLabel).toContainText('rows');
|
||||
}
|
||||
|
||||
// Shared form-data fragment for the echarts time-series family (line/scatter/
|
||||
// generic/smooth/step): one temporal axis, one metric, split by gender series.
|
||||
const TIMESERIES_PARAMS = {
|
||||
x_axis: 'ds',
|
||||
time_grain_sqla: 'P1Y',
|
||||
metrics: ['count'],
|
||||
groupby: ['gender'],
|
||||
row_limit: 1000,
|
||||
};
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: opens from the header menu, paginates, and reload resets to page 1',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'big_number_total',
|
||||
chartNamePrefix: 'drill_bignum',
|
||||
params: { metric: 'count', adhoc_filters: [] },
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
// Open the modal from the chart's "More Options" header menu.
|
||||
const samplesOnOpen = expectSamplesPost(page);
|
||||
await dashboard.openDrillToDetailFromMenu(chartId);
|
||||
await samplesOnOpen;
|
||||
|
||||
const modal = dashboard.drillModal();
|
||||
await expect(modal.element).toBeVisible();
|
||||
await expect(modal.element).toContainText('Drill to detail:');
|
||||
// The metadata bar and a real row count prove the modal loaded backend data.
|
||||
await expect(modal.metadataBar).toBeVisible();
|
||||
await expect(modal.rowCountLabel).toContainText('rows');
|
||||
// No drill filter was applied (whole-chart drill).
|
||||
await expect(modal.filterValues).toHaveCount(0);
|
||||
|
||||
// The full dataset spans multiple pages, and the grid has rendered rows.
|
||||
expect(await modal.pageItems.count()).toBeGreaterThan(1);
|
||||
await expect(modal.gridCells.first()).toBeVisible();
|
||||
await expect(modal.activePageItem).toContainText('1');
|
||||
|
||||
// Paginate forward: clicking page 2 fires a real samples fetch and moves the
|
||||
// active page to 2.
|
||||
const samplesOnPage2 = expectSamplesPost(page);
|
||||
await modal.goToPage(2);
|
||||
await samplesOnPage2;
|
||||
await expect(modal.activePageItem).toContainText('2');
|
||||
|
||||
// Reload re-fetches and resets back to the first page.
|
||||
const samplesOnReload = expectSamplesPost(page);
|
||||
await modal.reload();
|
||||
await samplesOnReload;
|
||||
await expect(modal.activePageItem).toContainText('1');
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: big number value right-click drills the whole chart (no filter)',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'big_number_total',
|
||||
chartNamePrefix: 'drill_bignum_rc',
|
||||
params: { metric: 'count', adhoc_filters: [] },
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
await expectWholeChartDrillFromContextMenu(page, dashboard, chartId);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: table cell right-click drills by that value and clearing the filter restores the full set',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'table',
|
||||
chartNamePrefix: 'drill_table',
|
||||
params: {
|
||||
query_mode: 'aggregate',
|
||||
groupby: ['gender'],
|
||||
metrics: ['count'],
|
||||
row_limit: 100,
|
||||
server_pagination: false,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
// Right-click the "boy" dimension cell and drill by it.
|
||||
const samplesOnDrill = expectSamplesPost(page);
|
||||
await dashboard
|
||||
.getChart(chartId)
|
||||
.getByText('boy', { exact: true })
|
||||
.first()
|
||||
.click({ button: 'right' });
|
||||
await dashboard.contextMenuDrillToDetailBy('boy');
|
||||
await samplesOnDrill;
|
||||
|
||||
const modal = dashboard.drillModal();
|
||||
await expect(modal.element).toBeVisible();
|
||||
await expect(modal.filterValues.first()).toContainText('boy');
|
||||
|
||||
const filteredCount = parseRowCount(await modal.rowCountLabel.innerText());
|
||||
expect(filteredCount).toBeGreaterThan(0);
|
||||
|
||||
// Clearing the filter reloads the samples and restores the larger, unfiltered total.
|
||||
const samplesOnClear = expectSamplesPost(page);
|
||||
await modal.clearFirstFilter();
|
||||
await samplesOnClear;
|
||||
await expect(modal.filterValues).toHaveCount(0);
|
||||
await expect
|
||||
.poll(async () => parseRowCount(await modal.rowCountLabel.innerText()))
|
||||
.toBeGreaterThan(filteredCount);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: pivot table cell right-click drills by the cell value',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'pivot_table_v2',
|
||||
chartNamePrefix: 'drill_pivot',
|
||||
params: {
|
||||
groupbyRows: ['gender'],
|
||||
groupbyColumns: [],
|
||||
metrics: ['count'],
|
||||
aggregateFunction: 'Sum',
|
||||
rowTotals: false,
|
||||
colTotals: false,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
await dashboard
|
||||
.getChart(chartId)
|
||||
.locator('[role="gridcell"]')
|
||||
.first()
|
||||
.click({ button: 'right' });
|
||||
|
||||
// The cell's row dimension determines the offered value; drill by it and
|
||||
// assert the same value lands in the modal filter.
|
||||
await drillByFirstOfferedValueAndAssert(page, dashboard);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: pie slice right-click (canvas) drills by the slice value',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
// Pie is a donut by default (center is a hole), so scan the ring for a slice.
|
||||
await expectCanvasDrillByValueRoundTrips(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'pie',
|
||||
chartNamePrefix: 'drill_pie',
|
||||
params: { groupby: ['gender'], metric: 'count' },
|
||||
},
|
||||
'ring',
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: line chart point right-click (canvas) drills by the point value',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
// Scan the plot grid for a point on one of the series lines.
|
||||
await expectCanvasDrillByValueRoundTrips(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'echarts_timeseries_line',
|
||||
chartNamePrefix: 'drill_line',
|
||||
params: TIMESERIES_PARAMS,
|
||||
},
|
||||
'grid',
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: big number with trendline right-click drills the whole chart (no filter)',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'big_number',
|
||||
chartNamePrefix: 'drill_bignum_trend',
|
||||
params: {
|
||||
metric: 'count',
|
||||
x_axis: 'ds',
|
||||
time_grain_sqla: 'P1Y',
|
||||
adhoc_filters: [],
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
await expectWholeChartDrillFromContextMenu(page, dashboard, chartId);
|
||||
},
|
||||
);
|
||||
|
||||
interface CanvasDrillCase {
|
||||
title: string;
|
||||
spec: ChartSpec;
|
||||
pattern: 'ring' | 'grid' | 'dense';
|
||||
}
|
||||
|
||||
// Every remaining canvas (echarts) chart is a thin caller of
|
||||
// expectCanvasDrillByValueRoundTrips, differing only in viz type, chart
|
||||
// params, and which point-scan pattern finds a drillable mark.
|
||||
const CANVAS_DRILL_CASES: CanvasDrillCase[] = [
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: scatter chart point right-click (canvas) drills by the point value',
|
||||
spec: {
|
||||
vizType: 'echarts_timeseries_scatter',
|
||||
chartNamePrefix: 'drill_scatter',
|
||||
// Enlarge the markers so a region scan reliably lands on a point;
|
||||
// scatter's default dots are a few pixels wide and a sparse grid misses
|
||||
// them.
|
||||
params: { ...TIMESERIES_PARAMS, markerSize: 20 },
|
||||
},
|
||||
pattern: 'dense',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: generic time-series point right-click (canvas) drills by the point value',
|
||||
spec: {
|
||||
vizType: 'echarts_timeseries',
|
||||
chartNamePrefix: 'drill_generic',
|
||||
params: TIMESERIES_PARAMS,
|
||||
},
|
||||
pattern: 'grid',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: smooth line point right-click (canvas) drills by the point value',
|
||||
spec: {
|
||||
vizType: 'echarts_timeseries_smooth',
|
||||
chartNamePrefix: 'drill_smooth',
|
||||
params: TIMESERIES_PARAMS,
|
||||
},
|
||||
pattern: 'grid',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: step line point right-click (canvas) drills by the point value',
|
||||
spec: {
|
||||
vizType: 'echarts_timeseries_step',
|
||||
chartNamePrefix: 'drill_step',
|
||||
params: TIMESERIES_PARAMS,
|
||||
},
|
||||
pattern: 'grid',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: mixed time-series point right-click (canvas) drills by the point value',
|
||||
spec: {
|
||||
vizType: 'mixed_timeseries',
|
||||
chartNamePrefix: 'drill_mixed',
|
||||
params: {
|
||||
x_axis: 'ds',
|
||||
time_grain_sqla: 'P1Y',
|
||||
metrics: ['count'],
|
||||
groupby: ['gender'],
|
||||
metrics_b: ['count'],
|
||||
groupby_b: ['gender'],
|
||||
row_limit: 1000,
|
||||
},
|
||||
},
|
||||
pattern: 'grid',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: box plot right-click (canvas) drills by the box value',
|
||||
spec: {
|
||||
vizType: 'box_plot',
|
||||
chartNamePrefix: 'drill_boxplot',
|
||||
params: {
|
||||
groupby: ['gender'],
|
||||
metrics: ['count'],
|
||||
columns: ['ds'],
|
||||
},
|
||||
},
|
||||
pattern: 'dense',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: funnel segment right-click (canvas) drills by the segment value',
|
||||
spec: {
|
||||
vizType: 'funnel',
|
||||
chartNamePrefix: 'drill_funnel',
|
||||
params: { groupby: ['gender'], metric: 'count' },
|
||||
},
|
||||
pattern: 'dense',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: gauge right-click (canvas) drills by the gauge value',
|
||||
spec: {
|
||||
vizType: 'gauge_chart',
|
||||
chartNamePrefix: 'drill_gauge',
|
||||
params: { groupby: ['gender'], metric: 'count' },
|
||||
},
|
||||
pattern: 'dense',
|
||||
},
|
||||
{
|
||||
title:
|
||||
'drill-to-detail modal: treemap tile right-click (canvas) drills by the tile value',
|
||||
spec: {
|
||||
vizType: 'treemap_v2',
|
||||
chartNamePrefix: 'drill_treemap',
|
||||
params: { metric: 'count', groupby: ['gender'] },
|
||||
},
|
||||
pattern: 'dense',
|
||||
},
|
||||
];
|
||||
|
||||
for (const { title, spec, pattern } of CANVAS_DRILL_CASES) {
|
||||
testWithAssets(title, async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
await expectCanvasDrillByValueRoundTrips(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
spec,
|
||||
pattern,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: drilling a time-series point "by all" applies every dimension of that point',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'echarts_timeseries_line',
|
||||
chartNamePrefix: 'drill_all',
|
||||
// Two groupby dimensions so each point genuinely carries more than one
|
||||
// drillable value — the whole point of "Drill to detail by all".
|
||||
params: { ...TIMESERIES_PARAMS, groupby: ['gender', 'state'] },
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
const canvas = dashboard.getChart(chartId).locator('canvas').first();
|
||||
await expect(canvas).toBeVisible();
|
||||
await rightClickCanvasDatum(page, dashboard, canvas, 'grid');
|
||||
|
||||
// A line point carries two dimensions (the temporal value and the gender
|
||||
// series), so "Drill to detail by all" must apply both as filters.
|
||||
const offered = await dashboard.drillByOfferedValues();
|
||||
expect(offered.length).toBeGreaterThanOrEqual(2);
|
||||
const samples = expectSamplesPost(page);
|
||||
await dashboard.contextMenuDrillToDetailBy('all');
|
||||
await samples;
|
||||
|
||||
await expect(dashboard.drillModal().element).toBeVisible();
|
||||
expect(
|
||||
await dashboard.drillModal().filterValues.count(),
|
||||
).toBeGreaterThanOrEqual(2);
|
||||
},
|
||||
);
|
||||
|
||||
testWithAssets(
|
||||
'drill-to-detail modal: table drills correctly by each of multiple dimension values',
|
||||
async ({ page, testAssets }) => {
|
||||
testWithAssets.setTimeout(TIMEOUT.SLOW_TEST);
|
||||
const dashboard = new DashboardPage(page);
|
||||
const { dashboardId, chartId } = await buildSingleChartDashboard(
|
||||
page,
|
||||
testAssets,
|
||||
testWithAssets.info(),
|
||||
{
|
||||
vizType: 'table',
|
||||
chartNamePrefix: 'drill_table_multi',
|
||||
params: {
|
||||
query_mode: 'aggregate',
|
||||
groupby: ['gender'],
|
||||
metrics: ['count'],
|
||||
row_limit: 100,
|
||||
server_pagination: false,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
await loadDashboardWithChart(dashboard, dashboardId, chartId);
|
||||
|
||||
for (const value of ['boy', 'girl']) {
|
||||
const samples = expectSamplesPost(page);
|
||||
await dashboard
|
||||
.getChart(chartId)
|
||||
.getByText(value, { exact: true })
|
||||
.first()
|
||||
.click({ button: 'right' });
|
||||
await dashboard.contextMenuDrillToDetailBy(value);
|
||||
await samples;
|
||||
|
||||
const modal = dashboard.drillModal();
|
||||
await expect(modal.element).toBeVisible();
|
||||
await expect(modal.filterValues.first()).toContainText(value);
|
||||
await modal.close();
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -38,7 +38,7 @@ export default function transformProps(chartProps: ChartProps) {
|
||||
includeSeries,
|
||||
isDarkMode: isThemeDark(theme),
|
||||
linearColorScheme,
|
||||
metrics: metrics.map((m: { label?: string } | string) =>
|
||||
metrics: (metrics ?? []).map((m: { label?: string } | string) =>
|
||||
typeof m === 'string' ? m : m.label || m,
|
||||
),
|
||||
colorMetric: secondaryMetric?.label || secondaryMetric,
|
||||
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
|
||||
import { ChartProps } from '@superset-ui/core';
|
||||
import transformProps from '../src/transformProps';
|
||||
|
||||
const createProps = () =>
|
||||
({
|
||||
width: 800,
|
||||
height: 600,
|
||||
formData: {
|
||||
includeSeries: false,
|
||||
linearColorScheme: 'superset_seq_1',
|
||||
metrics: undefined,
|
||||
secondaryMetric: 'sum__SP_POP_TOTL',
|
||||
series: 'country_name',
|
||||
showDatatable: false,
|
||||
},
|
||||
queriesData: [{ data: [{ country_id: 'FRA', metric: 10 }] }],
|
||||
theme: {},
|
||||
}) as unknown as ChartProps;
|
||||
|
||||
test('do not crash on undefined metrics', () => {
|
||||
expect(() => transformProps(createProps())).not.toThrow();
|
||||
});
|
||||
@@ -1069,10 +1069,10 @@ export default function TableChart<D extends DataRecord = DataRecord>(
|
||||
const originKey = column.key.substring(column.label.length).trim();
|
||||
if (!hasColumnColorFormatters && hasBasicColorFormatters) {
|
||||
backgroundColor =
|
||||
basicColorFormatters[row.index][originKey]?.backgroundColor;
|
||||
basicColorFormatters[row.index]?.[originKey]?.backgroundColor;
|
||||
arrow =
|
||||
column.label === comparisonLabels[0]
|
||||
? basicColorFormatters[row.index][originKey]?.mainArrow
|
||||
? basicColorFormatters[row.index]?.[originKey]?.mainArrow
|
||||
: '';
|
||||
}
|
||||
|
||||
@@ -1134,11 +1134,11 @@ export default function TableChart<D extends DataRecord = DataRecord>(
|
||||
basicColorColumnFormatters?.length > 0
|
||||
) {
|
||||
backgroundColor =
|
||||
basicColorColumnFormatters[row.index][column.key]
|
||||
basicColorColumnFormatters[row.index]?.[column.key]
|
||||
?.backgroundColor || backgroundColor;
|
||||
arrow =
|
||||
column.label === comparisonLabels[0]
|
||||
? basicColorColumnFormatters[row.index][column.key]?.mainArrow
|
||||
? basicColorColumnFormatters[row.index]?.[column.key]?.mainArrow
|
||||
: '';
|
||||
}
|
||||
const rowSurfaceColor =
|
||||
@@ -1197,7 +1197,7 @@ export default function TableChart<D extends DataRecord = DataRecord>(
|
||||
let arrowStyles = css`
|
||||
color: ${
|
||||
basicColorFormatters &&
|
||||
basicColorFormatters[row.index][originKey]?.arrowColor ===
|
||||
basicColorFormatters[row.index]?.[originKey]?.arrowColor ===
|
||||
ColorSchemeEnum.Green
|
||||
? theme.colorSuccess
|
||||
: theme.colorError
|
||||
@@ -1211,7 +1211,7 @@ export default function TableChart<D extends DataRecord = DataRecord>(
|
||||
) {
|
||||
arrowStyles = css`
|
||||
color: ${
|
||||
basicColorColumnFormatters[row.index][column.key]
|
||||
basicColorColumnFormatters[row.index]?.[column.key]
|
||||
?.arrowColor === ColorSchemeEnum.Green
|
||||
? theme.colorSuccess
|
||||
: theme.colorError
|
||||
|
||||
@@ -20,6 +20,7 @@ import '@testing-library/jest-dom';
|
||||
import {
|
||||
getTextColorForBackground,
|
||||
ObjectFormattingEnum,
|
||||
ColorSchemeEnum,
|
||||
} from '@superset-ui/chart-controls';
|
||||
import { supersetTheme } from '@apache-superset/core/theme';
|
||||
import {
|
||||
@@ -2075,6 +2076,59 @@ describe('plugin-chart-table', () => {
|
||||
});
|
||||
});
|
||||
|
||||
test('does not crash when a comparison-color-formatter array has no entry for a rendered row', () => {
|
||||
// Regression test: the per-cell comparison-color lookups in the Cell
|
||||
// renderer (`basicColorFormatters`/`basicColorColumnFormatters`,
|
||||
// indexed by `row.index`) must stay safe even if those arrays ever
|
||||
// end up with fewer entries than the number of rendered rows -- e.g.
|
||||
// when "Show summary" is combined with time comparison and a
|
||||
// comparison-based conditional color scheme ("Green for increase,
|
||||
// red for decrease") applied to a Time Comparison column. Without
|
||||
// the `?.` guard on the array-index lookup, this throws
|
||||
// `TypeError: Cannot read properties of undefined (reading 'Main
|
||||
// metric_1')`.
|
||||
const propsInput = {
|
||||
...testData.comparison,
|
||||
rawFormData: {
|
||||
...testData.comparison.rawFormData,
|
||||
conditional_formatting: [
|
||||
{ column: 'Main metric_1', colorScheme: ColorSchemeEnum.Green },
|
||||
],
|
||||
},
|
||||
};
|
||||
const transformedProps = transformProps(propsInput);
|
||||
expect(transformedProps.data).toHaveLength(2);
|
||||
expect(transformedProps.basicColorColumnFormatters).toHaveLength(2);
|
||||
|
||||
// Simulate the row-count mismatch: the formatter array has an entry
|
||||
// for only the first row, matching the shape of the bug (an entry
|
||||
// missing for one of the rendered rows).
|
||||
const propsWithMissingFormatterEntry = {
|
||||
...transformedProps,
|
||||
basicColorColumnFormatters:
|
||||
transformedProps.basicColorColumnFormatters!.slice(0, 1),
|
||||
};
|
||||
|
||||
expect(() =>
|
||||
render(
|
||||
<TableChart {...propsWithMissingFormatterEntry} sticky={false} />,
|
||||
),
|
||||
).not.toThrow();
|
||||
|
||||
// the row that still has a formatter entry keeps its comparison
|
||||
// background color and arrow: the "Main metric_1" cell for the
|
||||
// first row (value 100) renders before the derived "△ metric_1"
|
||||
// cell that happens to share the same value and aria label.
|
||||
const [styledCell] = screen.getAllByTitle('100');
|
||||
expect(styledCell).toHaveTextContent('↑100');
|
||||
expect(getComputedStyle(styledCell).background).toContain(
|
||||
'rgba(0, 150, 0, 0.2)',
|
||||
);
|
||||
|
||||
// the row missing a formatter entry still renders its raw value
|
||||
expect(screen.getAllByTitle('110').length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('preserves client-side search text across temporal table rerenders', async () => {
|
||||
const formDataWithSearch = {
|
||||
...testData.basic.formData,
|
||||
|
||||
+49
-19
@@ -196,6 +196,54 @@ interface DatasourceObject {
|
||||
folders?: DatasourceFolder[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Lift the certification and warning fields a metric keeps inside its `extra`
|
||||
* JSON blob onto the metric itself, which is the shape the editor's fields bind
|
||||
* to.
|
||||
*
|
||||
* Two entry points feed the editor two different metric shapes: the dataset
|
||||
* list hands over the API payload, where `extra` is still a JSON string, while
|
||||
* Explore hands over its bootstrap payload, where `SqlMetric.data` has already
|
||||
* flattened `extra` into `warning_markdown` and dropped the raw string. The
|
||||
* parsed blob is therefore only authoritative when `extra` is actually present;
|
||||
* otherwise the already-flattened value stands, instead of being reset to an
|
||||
* empty field.
|
||||
*
|
||||
* A malformed `extra` string is treated the same as an absent one (falls
|
||||
* through to the already-flattened value) rather than throwing, mirroring
|
||||
* the backend's own tolerance for bad `extra` JSON in
|
||||
* `CertificationMixin.get_extra_dict()`.
|
||||
*/
|
||||
export function hydrateMetricExtra(metric: Metric): Metric {
|
||||
const {
|
||||
certified_by: certifiedByMetric,
|
||||
certification_details: certificationDetails,
|
||||
} = metric;
|
||||
let parsedExtra;
|
||||
if (metric.extra) {
|
||||
try {
|
||||
parsedExtra = JSON.parse(metric.extra) || {};
|
||||
} catch {
|
||||
parsedExtra = undefined;
|
||||
}
|
||||
}
|
||||
const {
|
||||
certification: {
|
||||
details = undefined,
|
||||
certified_by: certifiedBy = undefined,
|
||||
} = {},
|
||||
} = parsedExtra || {};
|
||||
const warningMarkdown = parsedExtra
|
||||
? parsedExtra.warning_markdown
|
||||
: metric.warning_markdown;
|
||||
return {
|
||||
...metric,
|
||||
certification_details: certificationDetails || details,
|
||||
warning_markdown: warningMarkdown || '',
|
||||
certified_by: certifiedBy || certifiedByMetric,
|
||||
};
|
||||
}
|
||||
|
||||
interface DatasourceEditorOwnProps {
|
||||
datasource: DatasourceObject;
|
||||
onChange?: (datasource: DatasourceObject, errors: string[]) => void;
|
||||
@@ -852,25 +900,7 @@ function DatasourceEditor({
|
||||
const [datasource, setDatasource] = useState<DatasourceObject>(() => ({
|
||||
...propsDatasource,
|
||||
editors: normalizeSubjectsToPickerValues(propsDatasource.editors || []),
|
||||
metrics: propsDatasource.metrics?.map(metric => {
|
||||
const {
|
||||
certified_by: certifiedByMetric,
|
||||
certification_details: certificationDetails,
|
||||
} = metric;
|
||||
const {
|
||||
certification: {
|
||||
details = undefined,
|
||||
certified_by: certifiedBy = undefined,
|
||||
} = {},
|
||||
warning_markdown: warningMarkdown,
|
||||
} = JSON.parse(metric.extra || '{}') || {};
|
||||
return {
|
||||
...metric,
|
||||
certification_details: certificationDetails || details,
|
||||
warning_markdown: warningMarkdown || metric.warning_markdown || '',
|
||||
certified_by: certifiedBy || certifiedByMetric,
|
||||
};
|
||||
}),
|
||||
metrics: propsDatasource.metrics?.map(hydrateMetricExtra),
|
||||
}));
|
||||
|
||||
const [errors, setErrors] = useState<string[]>([]);
|
||||
|
||||
+96
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
import { hydrateMetricExtra } from '../DatasourceEditor';
|
||||
|
||||
const metric = { metric_name: 'sum__num', expression: 'SUM(num)' };
|
||||
|
||||
test('lifts the warning and certification out of the extra JSON string', () => {
|
||||
expect(
|
||||
hydrateMetricExtra({
|
||||
...metric,
|
||||
extra: JSON.stringify({
|
||||
warning_markdown: 'Handle with care',
|
||||
certification: { certified_by: 'Data team', details: 'Reviewed' },
|
||||
}),
|
||||
}),
|
||||
).toMatchObject({
|
||||
warning_markdown: 'Handle with care',
|
||||
certified_by: 'Data team',
|
||||
certification_details: 'Reviewed',
|
||||
});
|
||||
});
|
||||
|
||||
test('keeps an already-flattened warning when the metric carries no extra (#42704)', () => {
|
||||
// Explore's bootstrap payload flattens `extra` into `warning_markdown` and
|
||||
// drops the raw string, so the flattened value is all there is to go on.
|
||||
expect(
|
||||
hydrateMetricExtra({ ...metric, warning_markdown: 'Handle with care' })
|
||||
.warning_markdown,
|
||||
).toBe('Handle with care');
|
||||
});
|
||||
|
||||
test('lets an empty warning in extra clear the flattened value', () => {
|
||||
expect(
|
||||
hydrateMetricExtra({
|
||||
...metric,
|
||||
warning_markdown: 'stale',
|
||||
extra: '{}',
|
||||
}).warning_markdown,
|
||||
).toBe('');
|
||||
});
|
||||
|
||||
test('normalizes a missing warning to an empty string', () => {
|
||||
expect(hydrateMetricExtra(metric).warning_markdown).toBe('');
|
||||
});
|
||||
|
||||
test('resolves certification conflicts between the metric and its extra blob', () => {
|
||||
expect(
|
||||
hydrateMetricExtra({
|
||||
...metric,
|
||||
certified_by: 'Analytics',
|
||||
certification_details: 'Owned by Analytics',
|
||||
extra: JSON.stringify({
|
||||
certification: { certified_by: 'Data team', details: 'Reviewed' },
|
||||
}),
|
||||
}),
|
||||
).toMatchObject({
|
||||
// extra wins for the certifier, while the metric's own details field wins
|
||||
// for the description — the certification form writes both back into extra
|
||||
// on save, so the two settle on the same source afterwards
|
||||
certified_by: 'Data team',
|
||||
certification_details: 'Owned by Analytics',
|
||||
});
|
||||
});
|
||||
|
||||
test('does not throw on malformed extra, falling back like an absent extra', () => {
|
||||
expect(() =>
|
||||
hydrateMetricExtra({
|
||||
...metric,
|
||||
warning_markdown: 'Handle with care',
|
||||
extra: '{not valid json',
|
||||
}),
|
||||
).not.toThrow();
|
||||
expect(
|
||||
hydrateMetricExtra({
|
||||
...metric,
|
||||
warning_markdown: 'Handle with care',
|
||||
extra: '{not valid json',
|
||||
}).warning_markdown,
|
||||
).toBe('Handle with care');
|
||||
});
|
||||
+1
-1
@@ -60,7 +60,7 @@ export default function DndAdhocFilterOption({
|
||||
<OptionWrapper
|
||||
key={index}
|
||||
index={index}
|
||||
label={actualTimeRange ?? adhocFilter.getDefaultLabel()}
|
||||
label={actualTimeRange ?? adhocFilter.getDefaultLabel(options)}
|
||||
tooltipTitle={title ?? adhocFilter.getTooltipTitle()}
|
||||
clickClose={onClickClose}
|
||||
onShiftOptions={onShiftOptions}
|
||||
|
||||
+30
-1
@@ -43,7 +43,7 @@ import {
|
||||
DndFilterSelectProps,
|
||||
} from 'src/explore/components/controls/DndColumnSelectControl/DndFilterSelect';
|
||||
import { PLACEHOLDER_DATASOURCE } from 'src/dashboard/constants';
|
||||
import { ExpressionTypes } from '../FilterControl/types';
|
||||
import { Clauses, ExpressionTypes } from '../FilterControl/types';
|
||||
import { DndItemType } from '../../DndItemType';
|
||||
import { Datasource } from '../../../types';
|
||||
import {
|
||||
@@ -137,6 +137,35 @@ test('renders with value', async () => {
|
||||
expect(await screen.findByText('COUNT(*)')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test('renders the pill using the column verbose_name when one is set', async () => {
|
||||
const value = new AdhocFilter({
|
||||
expressionType: ExpressionTypes.Simple,
|
||||
subject: 'num',
|
||||
operator: '>',
|
||||
comparator: '500',
|
||||
clause: Clauses.Where,
|
||||
});
|
||||
render(
|
||||
setup({
|
||||
value,
|
||||
columns: [
|
||||
{
|
||||
id: 1,
|
||||
type: 'BIGINT',
|
||||
type_generic: GenericDataType.Numeric,
|
||||
column_name: 'num',
|
||||
verbose_name: 'total_count',
|
||||
},
|
||||
],
|
||||
}),
|
||||
{
|
||||
useDndKit: true,
|
||||
store,
|
||||
},
|
||||
);
|
||||
expect(await screen.findByText('total_count > 500')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test('renders options with saved metric', async () => {
|
||||
render(
|
||||
setup({
|
||||
|
||||
+28
@@ -370,4 +370,32 @@ describe('AdhocFilter', () => {
|
||||
});
|
||||
expect(adhocFilter.getDefaultLabel()).toBe('');
|
||||
});
|
||||
test('uses the column verbose_name in the label when one is given', () => {
|
||||
const adhocFilter = new AdhocFilter({
|
||||
expressionType: ExpressionTypes.Simple,
|
||||
subject: 'num',
|
||||
operator: '>',
|
||||
comparator: '500',
|
||||
clause: Clauses.Where,
|
||||
});
|
||||
expect(
|
||||
adhocFilter.getDefaultLabel([
|
||||
{ column_name: 'num', verbose_name: 'total_count' },
|
||||
]),
|
||||
).toBe('total_count > 500');
|
||||
});
|
||||
test('falls back to the column_name when no verbose_name is set', () => {
|
||||
const adhocFilter = new AdhocFilter({
|
||||
expressionType: ExpressionTypes.Simple,
|
||||
subject: 'num',
|
||||
operator: '>',
|
||||
comparator: '500',
|
||||
clause: Clauses.Where,
|
||||
});
|
||||
expect(
|
||||
adhocFilter.getDefaultLabel([{ column_name: 'num', verbose_name: '' }]),
|
||||
).toBe('num > 500');
|
||||
expect(adhocFilter.getDefaultLabel([])).toBe('num > 500');
|
||||
expect(adhocFilter.getDefaultLabel()).toBe('num > 500');
|
||||
});
|
||||
});
|
||||
|
||||
+5
-5
@@ -23,7 +23,7 @@ import {
|
||||
OPERATOR_ENUM_TO_OPERATOR_TYPE,
|
||||
Operators,
|
||||
} from 'src/explore/constants';
|
||||
import { translateToSql } from '../utils/translateToSQL';
|
||||
import { translateToSql, VerboseColumn } from '../utils/translateToSQL';
|
||||
import { Clauses, ExpressionTypes } from '../types';
|
||||
|
||||
const CUSTOM_OPERATIONS = [...CUSTOM_OPERATORS].map(
|
||||
@@ -193,8 +193,8 @@ export default class AdhocFilter {
|
||||
);
|
||||
}
|
||||
|
||||
getDefaultLabel(): string {
|
||||
const label = this.translateToSql();
|
||||
getDefaultLabel(columns?: VerboseColumn[]): string {
|
||||
const label = this.translateToSql({ columns });
|
||||
return label.length < 43 ? label : `${label.substring(0, 40)}...`;
|
||||
}
|
||||
|
||||
@@ -202,8 +202,8 @@ export default class AdhocFilter {
|
||||
return this.translateToSql();
|
||||
}
|
||||
|
||||
translateToSql(): string {
|
||||
return translateToSql(this as unknown as CoreAdhocFilter);
|
||||
translateToSql(params: { columns?: VerboseColumn[] } = {}): string {
|
||||
return translateToSql(this as unknown as CoreAdhocFilter, params);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+37
@@ -23,6 +23,7 @@ import {
|
||||
screen,
|
||||
userEvent,
|
||||
waitFor,
|
||||
within,
|
||||
} from 'spec/helpers/testing-library';
|
||||
import thunk from 'redux-thunk';
|
||||
import configureStore from 'redux-mock-store';
|
||||
@@ -914,3 +915,39 @@ test('dropdown should remain open when clicked after filter is configured', asyn
|
||||
|
||||
expect(operatorDropdown).toHaveAttribute('aria-expanded', 'true');
|
||||
});
|
||||
|
||||
test('filters the subject select by column verbose_name as well as column_name', async () => {
|
||||
setup({
|
||||
options: [
|
||||
{
|
||||
type: 'BIGINT',
|
||||
column_name: 'num',
|
||||
verbose_name: 'total_count',
|
||||
id: 1,
|
||||
},
|
||||
{
|
||||
type: 'VARCHAR(255)',
|
||||
column_name: 'name',
|
||||
verbose_name: 'Full Name',
|
||||
id: 2,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const combobox = screen.getByRole('combobox', { name: 'Select subject' });
|
||||
userEvent.click(combobox);
|
||||
|
||||
await userEvent.type(combobox, 'total');
|
||||
|
||||
const dropdown = document.querySelector(
|
||||
'.ant-select-dropdown-list',
|
||||
) as HTMLElement;
|
||||
expect(within(dropdown).getByText('total_count')).toBeInTheDocument();
|
||||
expect(within(dropdown).queryByText('Full Name')).not.toBeInTheDocument();
|
||||
|
||||
await userEvent.clear(combobox);
|
||||
await userEvent.type(combobox, 'num');
|
||||
|
||||
expect(within(dropdown).getByText('total_count')).toBeInTheDocument();
|
||||
expect(within(dropdown).queryByText('Full Name')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
+4
@@ -639,7 +639,11 @@ const AdhocFilterEditPopoverSimpleTabContent: FC<Props> = props => {
|
||||
('optionName' in column && column.optionName) ||
|
||||
undefined,
|
||||
label: renderSubjectOptionLabel(column),
|
||||
column_name: 'column_name' in column ? column.column_name : undefined,
|
||||
verbose_name:
|
||||
'verbose_name' in column ? column.verbose_name : undefined,
|
||||
}))}
|
||||
optionFilterProps={['column_name', 'verbose_name']}
|
||||
{...subjectSelectProps}
|
||||
/>
|
||||
);
|
||||
|
||||
+18
@@ -71,6 +71,24 @@ test('should render the control label', async () => {
|
||||
expect(await screen.findByText('value > 10')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test('should render the control label using the column verbose_name when one is set', async () => {
|
||||
render(
|
||||
setup({
|
||||
...mockedProps,
|
||||
options: [
|
||||
{
|
||||
type: 'DOUBLE',
|
||||
column_name: 'value',
|
||||
verbose_name: 'total_count',
|
||||
id: 3,
|
||||
},
|
||||
],
|
||||
}),
|
||||
{ useDnd: true, useRedux: true },
|
||||
);
|
||||
expect(await screen.findByText('total_count > 10')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test('should render the remove button', async () => {
|
||||
render(setup(mockedProps), { useDnd: true, useRedux: true });
|
||||
const removeBtn = await screen.findByTestId('remove-control-button');
|
||||
|
||||
+1
-1
@@ -65,7 +65,7 @@ export default function AdhocFilterOption({
|
||||
partitionColumn={partitionColumn ?? undefined}
|
||||
>
|
||||
<OptionControlLabel
|
||||
label={actualTimeRange ?? adhocFilter.getDefaultLabel()}
|
||||
label={actualTimeRange ?? adhocFilter.getDefaultLabel(options)}
|
||||
tooltipTitle={title ?? adhocFilter.getTooltipTitle()}
|
||||
onRemove={() =>
|
||||
onRemoveFilter({
|
||||
|
||||
+32
-2
@@ -63,9 +63,35 @@ export const OPERATORS_TO_SQL = {
|
||||
`= '{{ presto.latest_partition('${datasource.schema}.${datasource.datasource_name}') }}'`,
|
||||
};
|
||||
|
||||
export interface VerboseColumn {
|
||||
column_name?: string;
|
||||
verbose_name?: string | null;
|
||||
}
|
||||
|
||||
// Resolves the display label for a filter's subject: the verbose_name of the
|
||||
// matching column when one is supplied, falling back to the technical
|
||||
// subject used for SQL generation.
|
||||
const getDisplaySubject = (
|
||||
subject: string | { column_name?: string } | null | undefined,
|
||||
columns?: VerboseColumn[],
|
||||
) => {
|
||||
if (!columns) {
|
||||
return subject ?? undefined;
|
||||
}
|
||||
const columnName =
|
||||
typeof subject === 'object' ? subject?.column_name : subject;
|
||||
const verboseName = columns.find(
|
||||
column => column.column_name === columnName,
|
||||
)?.verbose_name;
|
||||
return verboseName || (subject ?? undefined);
|
||||
};
|
||||
|
||||
export const translateToSql = (
|
||||
adhocFilter: AdhocFilter,
|
||||
{ useSimple }: { useSimple: boolean } = { useSimple: false },
|
||||
{
|
||||
useSimple,
|
||||
columns,
|
||||
}: { useSimple?: boolean; columns?: VerboseColumn[] } = {},
|
||||
) => {
|
||||
if (isSimpleAdhocFilter(adhocFilter) || useSimple) {
|
||||
const { subject, operator } = adhocFilter as SimpleAdhocFilter;
|
||||
@@ -81,7 +107,11 @@ export const translateToSql = (
|
||||
OPERATORS_TO_SQL[operator](adhocFilter)
|
||||
: // @ts-expect-error TODO: fix missing operator type `NOT LIKE` and `TEMPORAL RANGE`.
|
||||
OPERATORS_TO_SQL[operator];
|
||||
return getSimpleSQLExpression(subject, op, comparator);
|
||||
return getSimpleSQLExpression(
|
||||
getDisplaySubject(subject, columns),
|
||||
op,
|
||||
comparator,
|
||||
);
|
||||
}
|
||||
if (isFreeFormAdhocFilter(adhocFilter)) {
|
||||
return adhocFilter.sqlExpression;
|
||||
|
||||
+5
-4
@@ -22,10 +22,11 @@ import FixedOrMetricControl from '.';
|
||||
jest.mock(
|
||||
'@superset-ui/core/components/Icons/AsyncIcon',
|
||||
() =>
|
||||
({ fileName }: { fileName: string }) => (
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
({ fileName }: { fileName: string }) =>
|
||||
(
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
);
|
||||
|
||||
const createProps = () => ({
|
||||
|
||||
@@ -17,9 +17,11 @@
|
||||
* under the License.
|
||||
*/
|
||||
|
||||
import exploreReducer, { ExploreState } from './exploreReducer';
|
||||
import { setStashFormData } from '../actions/exploreActions';
|
||||
import { QueryFormData } from '@superset-ui/core';
|
||||
import { sections, CustomControlItem } from '@superset-ui/chart-controls';
|
||||
import { getControlStateFromControlConfig } from 'src/explore/controlUtils';
|
||||
import exploreReducer, { ExploreState } from './exploreReducer';
|
||||
import { setControlValue, setStashFormData } from '../actions/exploreActions';
|
||||
|
||||
test('reset hiddenFormData on SET_STASH_FORM_DATA', () => {
|
||||
const initialState: ExploreState = {
|
||||
@@ -52,3 +54,72 @@ test('skips updates when the field is already updated on SET_STASH_FORM_DATA', (
|
||||
const newState = exploreReducer(initialState, restoreAction);
|
||||
expect(newState).toBe(initialState);
|
||||
});
|
||||
|
||||
// Regression guard for the shared Time Comparison section (used by the Table
|
||||
// chart, among others): selecting "Custom date" for Time shift and then
|
||||
// clearing "Shift start date" raises a required-date validation error. When the
|
||||
// user then switches Time shift to a non-custom preset the error must clear.
|
||||
// Because `start_date_offset` did not declare `validationDependencies` on
|
||||
// `time_compare`, SET_FIELD_VALUE never re-ran its mapStateToProps and the stale
|
||||
// error survived in Redux, blocking further chart updates until a page refresh.
|
||||
test('SET_FIELD_VALUE clears the custom-shift date error when time_compare leaves "custom"', () => {
|
||||
const REQUIRED_DATE_ERROR = 'A date is required when using custom date shift';
|
||||
const timeComparisonSection = sections.timeComparisonControls({
|
||||
multi: false,
|
||||
showCalculationType: false,
|
||||
showFullChoices: false,
|
||||
});
|
||||
const timeCompareConfig = (
|
||||
timeComparisonSection.controlSetRows[0][0] as CustomControlItem
|
||||
).config;
|
||||
const startDateOffsetConfig = (
|
||||
timeComparisonSection.controlSetRows[1][0] as CustomControlItem
|
||||
).config;
|
||||
|
||||
const form_data = {
|
||||
time_compare: 'custom',
|
||||
start_date_offset: '2021-01-01',
|
||||
} as unknown as QueryFormData;
|
||||
|
||||
// Build the control states the way the explore store does so they carry the
|
||||
// real mapStateToProps / validationDependencies from the control config.
|
||||
const controlPanelState = { controls: {}, form_data };
|
||||
const initialState: ExploreState = {
|
||||
form_data,
|
||||
controls: {
|
||||
time_compare: getControlStateFromControlConfig(
|
||||
timeCompareConfig,
|
||||
controlPanelState,
|
||||
'custom',
|
||||
)!,
|
||||
start_date_offset: getControlStateFromControlConfig(
|
||||
startDateOffsetConfig,
|
||||
controlPanelState,
|
||||
'2021-01-01',
|
||||
)!,
|
||||
},
|
||||
};
|
||||
|
||||
// A valid custom date starts without a validation error.
|
||||
expect(initialState.controls.start_date_offset.validationErrors).toEqual([]);
|
||||
|
||||
// 1) Clearing "Shift start date" raises the required-date error (expected).
|
||||
const afterClear = exploreReducer(
|
||||
initialState,
|
||||
setControlValue('start_date_offset', '') as Parameters<
|
||||
typeof exploreReducer
|
||||
>[1],
|
||||
);
|
||||
expect(afterClear.controls.start_date_offset.validationErrors).toEqual([
|
||||
REQUIRED_DATE_ERROR,
|
||||
]);
|
||||
|
||||
// 2) Switching Time shift to a non-custom preset must clear the stale error.
|
||||
const afterSwitch = exploreReducer(
|
||||
afterClear,
|
||||
setControlValue('time_compare', '1 week ago') as Parameters<
|
||||
typeof exploreReducer
|
||||
>[1],
|
||||
);
|
||||
expect(afterSwitch.controls.start_date_offset.validationErrors).toEqual([]);
|
||||
});
|
||||
|
||||
+5
-4
@@ -38,10 +38,11 @@ import {
|
||||
jest.mock(
|
||||
'@superset-ui/core/components/Icons/AsyncIcon',
|
||||
() =>
|
||||
({ fileName }: { fileName: string }) => (
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
({ fileName }: { fileName: string }) =>
|
||||
(
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
);
|
||||
|
||||
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
|
||||
|
||||
+5
-4
@@ -29,10 +29,11 @@ import DatasetPanelWrapper from 'src/features/datasets/AddDataset/DatasetPanel';
|
||||
jest.mock(
|
||||
'@superset-ui/core/components/Icons/AsyncIcon',
|
||||
() =>
|
||||
({ fileName }: { fileName: string }) => (
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
({ fileName }: { fileName: string }) =>
|
||||
(
|
||||
// eslint-disable-next-line jsx-a11y/prefer-tag-over-role -- mirrors AsyncIcon's real span+role="img" shape
|
||||
<span role="img" aria-label={fileName.replace('_', '-')} />
|
||||
),
|
||||
);
|
||||
|
||||
const errorMessageRegistry = getErrorMessageComponentRegistry();
|
||||
|
||||
@@ -72,6 +72,12 @@ export const PermissionsField = ({
|
||||
.replace(/_/g, ' ')
|
||||
.includes(input.toLowerCase().replace(/_/g, ' '))
|
||||
}
|
||||
// Permission labels are long ("all datasource access on all_datasource_access",
|
||||
// "can write on DashboardFilterStateRestApi"), and the dropdown otherwise
|
||||
// inherits the trigger's width inside the modal, so every option was truncated
|
||||
// to the point of being indistinguishable. Let the popup size to its content
|
||||
// instead. See #40430.
|
||||
popupMatchSelectWidth={false}
|
||||
getPopupContainer={trigger => trigger.closest('.ant-modal-container')}
|
||||
data-test="permissions-select"
|
||||
/>
|
||||
|
||||
@@ -43,6 +43,21 @@ function findEndpoint(spy: jest.SpyInstance, substring: string): string {
|
||||
return (match[0] as Record<string, string>).endpoint;
|
||||
}
|
||||
|
||||
function deferredJsonResponse() {
|
||||
let resolveResponse: ((value: JsonResponse) => void) | undefined;
|
||||
let rejectResponse: ((reason?: unknown) => void) | undefined;
|
||||
const promise = new Promise<JsonResponse>((resolve, reject) => {
|
||||
resolveResponse = resolve;
|
||||
rejectResponse = reject;
|
||||
});
|
||||
|
||||
if (!resolveResponse || !rejectResponse) {
|
||||
throw new Error('Deferred response handlers were not initialized');
|
||||
}
|
||||
|
||||
return { promise, resolve: resolveResponse, reject: rejectResponse };
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
jest.restoreAllMocks();
|
||||
});
|
||||
@@ -282,6 +297,147 @@ test('useListViewResource: fetchData sets loading to true then false', async ()
|
||||
});
|
||||
});
|
||||
|
||||
test('useListViewResource: ignores an older response that resolves last', async () => {
|
||||
const older = deferredJsonResponse();
|
||||
const newer = deferredJsonResponse();
|
||||
const toISOString = jest
|
||||
.spyOn(Date.prototype, 'toISOString')
|
||||
.mockReturnValueOnce('newer-response-time')
|
||||
.mockReturnValueOnce('older-response-time');
|
||||
jest
|
||||
.spyOn(SupersetClient, 'get')
|
||||
.mockReturnValueOnce(older.promise)
|
||||
.mockReturnValueOnce(newer.promise);
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useListViewResource('chart', 'Charts', jest.fn(), false),
|
||||
);
|
||||
|
||||
act(() => {
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [],
|
||||
});
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [{ id: 'name', operator: 'ct', value: 'newer' }],
|
||||
});
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
newer.resolve({
|
||||
json: { result: [], count: 0 },
|
||||
} as unknown as JsonResponse);
|
||||
});
|
||||
expect(result.current.state.resourceCollection).toEqual([]);
|
||||
expect(result.current.state.resourceCount).toBe(0);
|
||||
expect(result.current.state.lastFetched).toBe('newer-response-time');
|
||||
|
||||
await act(async () => {
|
||||
older.resolve({
|
||||
json: { result: [{ id: 1 }, { id: 2 }], count: 2 },
|
||||
} as unknown as JsonResponse);
|
||||
});
|
||||
|
||||
expect(result.current.state.resourceCollection).toEqual([]);
|
||||
expect(result.current.state.resourceCount).toBe(0);
|
||||
expect(result.current.state.lastFetched).toBe('newer-response-time');
|
||||
expect(toISOString).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
test('useListViewResource: stale completion keeps the latest request loading', async () => {
|
||||
const older = deferredJsonResponse();
|
||||
const newer = deferredJsonResponse();
|
||||
jest
|
||||
.spyOn(SupersetClient, 'get')
|
||||
.mockReturnValueOnce(older.promise)
|
||||
.mockReturnValueOnce(newer.promise);
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useListViewResource('chart', 'Charts', jest.fn(), false),
|
||||
);
|
||||
|
||||
act(() => {
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [],
|
||||
});
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [{ id: 'name', operator: 'ct', value: 'newer' }],
|
||||
});
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
older.resolve({
|
||||
json: { result: [{ id: 1 }], count: 1 },
|
||||
} as unknown as JsonResponse);
|
||||
});
|
||||
|
||||
expect(result.current.state.resourceCollection).toEqual([]);
|
||||
expect(result.current.state.loading).toBe(true);
|
||||
|
||||
await act(async () => {
|
||||
newer.resolve({
|
||||
json: { result: [{ id: 2 }], count: 1 },
|
||||
} as unknown as JsonResponse);
|
||||
});
|
||||
|
||||
expect(result.current.state.resourceCollection).toEqual([{ id: 2 }]);
|
||||
expect(result.current.state.loading).toBe(false);
|
||||
});
|
||||
|
||||
test('useListViewResource: only the latest request reports an error', async () => {
|
||||
const older = deferredJsonResponse();
|
||||
const newer = deferredJsonResponse();
|
||||
const handleErrorMsg = jest.fn();
|
||||
jest
|
||||
.spyOn(SupersetClient, 'get')
|
||||
.mockReturnValueOnce(older.promise)
|
||||
.mockReturnValueOnce(newer.promise);
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useListViewResource('chart', 'Charts', handleErrorMsg, false),
|
||||
);
|
||||
|
||||
act(() => {
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [],
|
||||
});
|
||||
result.current.fetchData({
|
||||
pageIndex: 0,
|
||||
pageSize: 25,
|
||||
sortBy: [{ id: 'name' }],
|
||||
filters: [{ id: 'name', operator: 'ct', value: 'newer' }],
|
||||
});
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
older.reject('older request failed');
|
||||
});
|
||||
|
||||
expect(handleErrorMsg).not.toHaveBeenCalled();
|
||||
expect(result.current.state.loading).toBe(true);
|
||||
|
||||
await act(async () => {
|
||||
newer.reject('newer request failed');
|
||||
});
|
||||
|
||||
expect(handleErrorMsg).toHaveBeenCalledTimes(1);
|
||||
expect(result.current.state.loading).toBe(false);
|
||||
});
|
||||
|
||||
test('useListViewResource: refreshData re-fetches with last config', async () => {
|
||||
const getSpy = jest.spyOn(SupersetClient, 'get').mockResolvedValue({
|
||||
json: { result: [], count: 0 },
|
||||
|
||||
@@ -148,6 +148,7 @@ export function useListViewResource<D extends object = any>(
|
||||
);
|
||||
|
||||
const lastFetchDataConfigRef = useRef<FetchDataConfig | null>(null);
|
||||
const latestRequestIdRef = useRef(0);
|
||||
|
||||
const fetchData = useCallback(
|
||||
({
|
||||
@@ -156,6 +157,9 @@ export function useListViewResource<D extends object = any>(
|
||||
sortBy,
|
||||
filters: filterValues,
|
||||
}: FetchDataConfig) => {
|
||||
const requestId = latestRequestIdRef.current + 1;
|
||||
latestRequestIdRef.current = requestId;
|
||||
const isLatest = () => latestRequestIdRef.current === requestId;
|
||||
const config: FetchDataConfig = {
|
||||
filters: filterValues,
|
||||
pageIndex,
|
||||
@@ -196,24 +200,31 @@ export function useListViewResource<D extends object = any>(
|
||||
})
|
||||
.then(
|
||||
({ json = {} }) => {
|
||||
if (!isLatest()) {
|
||||
return;
|
||||
}
|
||||
updateState({
|
||||
collection: json.result,
|
||||
count: json.count,
|
||||
lastFetched: new Date().toISOString(),
|
||||
});
|
||||
},
|
||||
createErrorHandler(errMsg =>
|
||||
handleErrorMsg(
|
||||
t(
|
||||
'An error occurred while fetching %ss: %s',
|
||||
resourceLabel,
|
||||
errMsg,
|
||||
),
|
||||
),
|
||||
),
|
||||
createErrorHandler(errMsg => {
|
||||
if (isLatest()) {
|
||||
handleErrorMsg(
|
||||
t(
|
||||
'An error occurred while fetching %ss: %s',
|
||||
resourceLabel,
|
||||
errMsg,
|
||||
),
|
||||
);
|
||||
}
|
||||
}),
|
||||
)
|
||||
.finally(() => {
|
||||
updateState({ loading: false });
|
||||
if (isLatest()) {
|
||||
updateState({ loading: false });
|
||||
}
|
||||
});
|
||||
},
|
||||
[
|
||||
|
||||
+2
-1
@@ -194,7 +194,8 @@ class SupersetApp(Flask):
|
||||
|
||||
logger.info("Syncing configuration to database...")
|
||||
|
||||
# Register SQLA event listeners for tagging system
|
||||
# Register the tagged_object cleanup listeners for the tagging system
|
||||
# (deletion only; see superset.tags.core.register_sqla_event_listeners)
|
||||
if feature_flag_manager.is_feature_enabled("TAGGING_SYSTEM"):
|
||||
from superset.tags.core import register_sqla_event_listeners
|
||||
|
||||
|
||||
@@ -25,7 +25,6 @@ from apispec import APISpec
|
||||
from apispec.ext.marshmallow import MarshmallowPlugin
|
||||
from flask import current_app
|
||||
from flask.cli import with_appcontext
|
||||
from flask_appbuilder import Model
|
||||
from flask_appbuilder.api import BaseApi
|
||||
from flask_appbuilder.api.manager import resolver
|
||||
|
||||
@@ -53,22 +52,6 @@ def set_database_uri(database_name: str, uri: str, skip_create: bool) -> None:
|
||||
database_utils.get_or_create_db(database_name, uri, not skip_create)
|
||||
|
||||
|
||||
@click.command()
|
||||
@with_appcontext
|
||||
@transaction()
|
||||
def sync_tags() -> None:
|
||||
"""Rebuilds special tags (owner, type, favorited by)."""
|
||||
# pylint: disable=no-member
|
||||
metadata = Model.metadata
|
||||
|
||||
# pylint: disable=import-outside-toplevel
|
||||
from superset.common.tags import add_favorites, add_owners, add_types
|
||||
|
||||
add_types(metadata)
|
||||
add_owners(metadata)
|
||||
add_favorites(metadata)
|
||||
|
||||
|
||||
@click.command()
|
||||
@with_appcontext
|
||||
def update_api_docs() -> None:
|
||||
|
||||
@@ -196,13 +196,26 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
|
||||
# 1. 'metric_name' - name of predefined metric
|
||||
# 2. { label: 'label_name' } - legacy format for a predefined metric
|
||||
# 3. { expressionType: 'SIMPLE' | 'SQL', ... } - adhoc metric
|
||||
def is_str_or_adhoc(metric: Metric) -> bool:
|
||||
return isinstance(metric, str) or is_adhoc_metric(metric)
|
||||
# Keys that only ever appear on an ad-hoc metric definition. A dict
|
||||
# carrying one of these but missing `expressionType` is a malformed
|
||||
# ad-hoc metric, not a legacy predefined-metric reference, and must
|
||||
# not be silently collapsed to its label, which would later be
|
||||
# misread as a request for a saved metric of that name.
|
||||
adhoc_metric_keys = {"sqlExpression", "aggregate", "column"}
|
||||
|
||||
self.metrics = metrics and [
|
||||
x if is_str_or_adhoc(x) else x["label"] # type: ignore
|
||||
for x in metrics
|
||||
]
|
||||
def normalize_metric(metric: Metric) -> Metric:
|
||||
if isinstance(metric, str) or is_adhoc_metric(metric):
|
||||
return metric
|
||||
if adhoc_metric_keys & metric.keys():
|
||||
raise QueryObjectValidationError(
|
||||
_(
|
||||
"Invalid ad-hoc metric %(label)s: `expressionType` is missing",
|
||||
label=metric.get("label"),
|
||||
)
|
||||
)
|
||||
return metric["label"] # type: ignore
|
||||
|
||||
self.metrics = metrics and [normalize_metric(x) for x in metrics]
|
||||
|
||||
def _set_post_processing(
|
||||
self, post_processing: list[dict[str, Any] | None] | None
|
||||
|
||||
@@ -1,496 +0,0 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
import contextlib
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import MetaData
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.sql import and_, func, join, literal, select
|
||||
|
||||
from superset.extensions import db
|
||||
from superset.tags.models import ObjectType, TagType
|
||||
|
||||
|
||||
def add_types_to_charts(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
slices = metadata.tables["slices"]
|
||||
|
||||
charts = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
slices.c.id.label("object_id"),
|
||||
literal(ObjectType.chart.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(slices, tag, tag.c.name == "type:chart"),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == slices.c.id,
|
||||
tagged_object.c.object_type == "chart",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, charts)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_types_to_dashboards(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
dashboard_table = metadata.tables["dashboards"]
|
||||
|
||||
dashboards = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
dashboard_table.c.id.label("object_id"),
|
||||
literal(ObjectType.dashboard.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(dashboard_table, tag, tag.c.name == "type:dashboard"),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == dashboard_table.c.id,
|
||||
tagged_object.c.object_type == "dashboard",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, dashboards)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_types_to_saved_queries(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
saved_query = metadata.tables["saved_query"]
|
||||
|
||||
saved_queries = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
saved_query.c.id.label("object_id"),
|
||||
literal(ObjectType.query.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(saved_query, tag, tag.c.name == "type:query"),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == saved_query.c.id,
|
||||
tagged_object.c.object_type == "query",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, saved_queries)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_types_to_datasets(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
tables = metadata.tables["tables"]
|
||||
|
||||
datasets = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
tables.c.id.label("object_id"),
|
||||
literal(ObjectType.dataset.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(tables, tag, tag.c.name == "type:dataset"),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == tables.c.id,
|
||||
tagged_object.c.object_type == "dataset",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, datasets)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_types(metadata: MetaData) -> None:
|
||||
"""
|
||||
Tag every object according to its type:
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
slices.id AS object_id,
|
||||
'chart' AS object_type
|
||||
FROM slices
|
||||
JOIN tag
|
||||
ON tag.name = 'type:chart'
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = slices.id
|
||||
AND tagged_object.object_type = 'chart'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
dashboards.id AS object_id,
|
||||
'dashboard' AS object_type
|
||||
FROM dashboards
|
||||
JOIN tag
|
||||
ON tag.name = 'type:dashboard'
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = dashboards.id
|
||||
AND tagged_object.object_type = 'dashboard'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
saved_query.id AS object_id,
|
||||
'query' AS object_type
|
||||
FROM saved_query
|
||||
JOIN tag
|
||||
ON tag.name = 'type:query';
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = saved_query.id
|
||||
AND tagged_object.object_type = 'query'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
tables.id AS object_id,
|
||||
'dataset' AS object_type
|
||||
FROM tables
|
||||
JOIN tag
|
||||
ON tag.name = 'type:dataset'
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = tables.id
|
||||
AND tagged_object.object_type = 'dataset'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
"""
|
||||
|
||||
tag = metadata.tables["tag"]
|
||||
tagged_object = metadata.tables["tagged_object"]
|
||||
columns = ["tag_id", "object_id", "object_type"]
|
||||
|
||||
# add a tag for each object type
|
||||
insert = tag.insert()
|
||||
for type_ in ObjectType.__members__:
|
||||
with contextlib.suppress(IntegrityError): # already exists
|
||||
db.session.execute(insert, name=f"type:{type_}", type=TagType.type)
|
||||
|
||||
add_types_to_charts(metadata, tag, tagged_object, columns)
|
||||
add_types_to_dashboards(metadata, tag, tagged_object, columns)
|
||||
add_types_to_saved_queries(metadata, tag, tagged_object, columns)
|
||||
add_types_to_datasets(metadata, tag, tagged_object, columns)
|
||||
|
||||
|
||||
def add_owners_to_charts(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
slices = metadata.tables["slices"]
|
||||
|
||||
charts = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
slices.c.id.label("object_id"),
|
||||
literal(ObjectType.chart.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(
|
||||
slices,
|
||||
tag,
|
||||
tag.c.name == "editor:" + slices.c.created_by_fk,
|
||||
),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == slices.c.id,
|
||||
tagged_object.c.object_type == "chart",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, charts)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_owners_to_dashboards(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
dashboard_table = metadata.tables["dashboards"]
|
||||
|
||||
dashboards = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
dashboard_table.c.id.label("object_id"),
|
||||
literal(ObjectType.dashboard.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(
|
||||
dashboard_table,
|
||||
tag,
|
||||
tag.c.name == "editor:" + dashboard_table.c.created_by_fk,
|
||||
),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == dashboard_table.c.id,
|
||||
tagged_object.c.object_type == "dashboard",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, dashboards)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_owners_to_saved_queries(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
saved_query = metadata.tables["saved_query"]
|
||||
|
||||
saved_queries = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
saved_query.c.id.label("object_id"),
|
||||
literal(ObjectType.query.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(
|
||||
saved_query,
|
||||
tag,
|
||||
tag.c.name == "editor:" + saved_query.c.created_by_fk,
|
||||
),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == saved_query.c.id,
|
||||
tagged_object.c.object_type == "query",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, saved_queries)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_owners_to_datasets(
|
||||
metadata: MetaData, tag: Any, tagged_object: Any, columns: list[str]
|
||||
) -> None:
|
||||
tables = metadata.tables["tables"]
|
||||
|
||||
datasets = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
tables.c.id.label("object_id"),
|
||||
literal(ObjectType.dataset.name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(
|
||||
tables,
|
||||
tag,
|
||||
tag.c.name == "editor:" + tables.c.created_by_fk,
|
||||
),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == tables.c.id,
|
||||
tagged_object.c.object_type == "dataset",
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, datasets)
|
||||
db.session.execute(query)
|
||||
|
||||
|
||||
def add_owners(metadata: MetaData) -> None:
|
||||
"""
|
||||
Tag every object according to its editor:
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
slices.id AS object_id,
|
||||
'chart' AS object_type
|
||||
FROM slices
|
||||
JOIN tag
|
||||
ON tag.name = CONCAT('editor:', slices.created_by_fk)
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = slices.id
|
||||
AND tagged_object.object_type = 'chart'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
dashboards.id AS object_id,
|
||||
'dashboard' AS object_type
|
||||
FROM dashboards
|
||||
JOIN tag
|
||||
ON tag.name = CONCAT('editor:', dashboards.created_by_fk)
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = dashboards.id
|
||||
AND tagged_object.object_type = 'dashboard'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
saved_query.id AS object_id,
|
||||
'query' AS object_type
|
||||
FROM saved_query
|
||||
JOIN tag
|
||||
ON tag.name = CONCAT('editor:', saved_query.created_by_fk)
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = saved_query.id
|
||||
AND tagged_object.object_type = 'query'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
tables.id AS object_id,
|
||||
'dataset' AS object_type
|
||||
FROM tables
|
||||
JOIN tag
|
||||
ON tag.name = CONCAT('editor:', tables.created_by_fk)
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = tables.id
|
||||
AND tagged_object.object_type = 'dataset'
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
"""
|
||||
|
||||
tag = metadata.tables["tag"]
|
||||
tagged_object = metadata.tables["tagged_object"]
|
||||
users = metadata.tables["ab_user"]
|
||||
columns = ["tag_id", "object_id", "object_type"]
|
||||
|
||||
# create a custom tag for each user
|
||||
ids = select(users.c.id)
|
||||
insert = tag.insert()
|
||||
for (id_,) in db.session.execute(ids):
|
||||
with contextlib.suppress(IntegrityError): # already exists
|
||||
db.session.execute(insert, name=f"editor:{id_}", type=TagType.editor)
|
||||
add_owners_to_charts(metadata, tag, tagged_object, columns)
|
||||
add_owners_to_dashboards(metadata, tag, tagged_object, columns)
|
||||
add_owners_to_saved_queries(metadata, tag, tagged_object, columns)
|
||||
add_owners_to_datasets(metadata, tag, tagged_object, columns)
|
||||
|
||||
|
||||
def add_favorites(metadata: MetaData) -> None:
|
||||
"""
|
||||
Tag every object that was favorited:
|
||||
|
||||
INSERT INTO tagged_object (tag_id, object_id, object_type)
|
||||
SELECT
|
||||
tag.id AS tag_id,
|
||||
favstar.obj_id AS object_id,
|
||||
LOWER(favstar.class_name) AS object_type
|
||||
FROM favstar
|
||||
JOIN tag
|
||||
ON tag.name = CONCAT('favorited_by:', favstar.user_id)
|
||||
LEFT OUTER JOIN tagged_object
|
||||
ON tagged_object.tag_id = tag.id
|
||||
AND tagged_object.object_id = favstar.obj_id
|
||||
AND tagged_object.object_type = LOWER(favstar.class_name)
|
||||
WHERE tagged_object.tag_id IS NULL;
|
||||
|
||||
"""
|
||||
|
||||
tag = metadata.tables["tag"]
|
||||
tagged_object = metadata.tables["tagged_object"]
|
||||
users = metadata.tables["ab_user"]
|
||||
favstar = metadata.tables["favstar"]
|
||||
columns = ["tag_id", "object_id", "object_type"]
|
||||
|
||||
# create a custom tag for each user
|
||||
ids = select(users.c.id)
|
||||
insert = tag.insert()
|
||||
for (id_,) in db.session.execute(ids):
|
||||
with contextlib.suppress(IntegrityError): # already exists
|
||||
db.session.execute(insert, name=f"favorited_by:{id_}", type=TagType.type)
|
||||
favstars = (
|
||||
select(
|
||||
tag.c.id.label("tag_id"),
|
||||
favstar.c.obj_id.label("object_id"),
|
||||
func.lower(favstar.c.class_name).label("object_type"),
|
||||
)
|
||||
.select_from(
|
||||
join(
|
||||
join(
|
||||
favstar,
|
||||
tag,
|
||||
tag.c.name == "favorited_by:" + favstar.c.user_id,
|
||||
),
|
||||
tagged_object,
|
||||
and_(
|
||||
tagged_object.c.tag_id == tag.c.id,
|
||||
tagged_object.c.object_id == favstar.c.obj_id,
|
||||
tagged_object.c.object_type == func.lower(favstar.c.class_name),
|
||||
),
|
||||
isouter=True,
|
||||
full=False,
|
||||
)
|
||||
)
|
||||
.where(tagged_object.c.tag_id.is_(None))
|
||||
)
|
||||
query = tagged_object.insert().from_select(columns, favstars)
|
||||
db.session.execute(query)
|
||||
@@ -1782,11 +1782,6 @@ DASHBOARD_AUTO_REFRESH_INTERVALS = [
|
||||
[86400, "24 hours"],
|
||||
]
|
||||
|
||||
# Performance optimization: Return only custom tags in dashboard list API
|
||||
# When enabled, filters out implicit tags (owner, type, favorited_by) at SQL JOIN level
|
||||
# Reduces response payload and query time for dashboards with many editors
|
||||
DASHBOARD_LIST_CUSTOM_TAGS_ONLY: bool = False
|
||||
|
||||
# This is used as a workaround for the alerts & reports scheduler task to get the time
|
||||
# celery beat triggered it, see https://github.com/celery/celery/issues/6974 for details
|
||||
CELERY_BEAT_SCHEDULER_EXPIRES = timedelta(weeks=1)
|
||||
|
||||
@@ -99,6 +99,7 @@ from superset.models.helpers import (
|
||||
AuditMixinNullable,
|
||||
CertificationMixin,
|
||||
ExploreMixin,
|
||||
get_effective_hours_offset,
|
||||
ImportExportMixin,
|
||||
QueryResult,
|
||||
SoftDeleteMixin,
|
||||
@@ -1247,6 +1248,8 @@ class TableColumn(AuditMixinNullable, ImportExportMixin, CertificationMixin, Mod
|
||||
time_grain: str | None,
|
||||
label: str | None = None,
|
||||
template_processor: BaseTemplateProcessor | None = None,
|
||||
apply_dataset_offset: bool = False,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> TimestampExpression | Label:
|
||||
"""
|
||||
Return a SQLAlchemy Core element representation of self to be used in a query.
|
||||
@@ -1254,6 +1257,8 @@ class TableColumn(AuditMixinNullable, ImportExportMixin, CertificationMixin, Mod
|
||||
:param time_grain: Optional time grain, e.g. P1Y
|
||||
:param label: alias/label that column is expected to have
|
||||
:param template_processor: template processor
|
||||
:param apply_dataset_offset: shift the selected axis before truncation
|
||||
:param sql_shifted_temporal_labels: labels shifted before truncation
|
||||
:return: A TimeExpression object wrapped in a Label if supported by db
|
||||
"""
|
||||
label = label or utils.DTTM_ALIAS
|
||||
@@ -1291,6 +1296,27 @@ class TableColumn(AuditMixinNullable, ImportExportMixin, CertificationMixin, Mod
|
||||
col = literal_column(expression, type_=type_)
|
||||
else:
|
||||
col = column(self.column_name, type_=type_)
|
||||
if (
|
||||
apply_dataset_offset
|
||||
and time_grain
|
||||
and self.table
|
||||
and self.db_engine_spec.supports_temporal_column_shift
|
||||
and (offset_hours := self.table.offset or 0)
|
||||
and not self.table.get_dataset_timezone()
|
||||
):
|
||||
effective_offset_hours = get_effective_hours_offset(
|
||||
self.db_engine_spec,
|
||||
self.type,
|
||||
offset_hours,
|
||||
db_extra=self.db_extra,
|
||||
)
|
||||
if effective_offset_hours:
|
||||
col = self.db_engine_spec.get_temporal_column_shift_expr(
|
||||
col,
|
||||
effective_offset_hours,
|
||||
)
|
||||
if sql_shifted_temporal_labels is not None:
|
||||
sql_shifted_temporal_labels.add(label)
|
||||
time_expr = self.db_engine_spec.get_timestamp_expr(col, pdf, time_grain)
|
||||
return self.database.make_sqla_column_compatible(time_expr, label)
|
||||
|
||||
@@ -1975,11 +2001,26 @@ class SqlaTable(
|
||||
)
|
||||
) from ex
|
||||
|
||||
def _shift_temporal_column_if_needed(
|
||||
self,
|
||||
sqla_column: ColumnClause,
|
||||
effective_offset_hours: int,
|
||||
) -> ColumnClause:
|
||||
"""Apply a nonzero effective dataset offset to a temporal expression."""
|
||||
if not effective_offset_hours:
|
||||
return sqla_column
|
||||
return self.db_engine_spec.get_temporal_column_shift_expr(
|
||||
sqla_column,
|
||||
effective_offset_hours,
|
||||
)
|
||||
|
||||
def adhoc_column_to_sqla( # pylint: disable=too-many-locals
|
||||
self,
|
||||
col: AdhocColumn,
|
||||
force_type_check: bool = False,
|
||||
template_processor: BaseTemplateProcessor | None = None,
|
||||
apply_dataset_offset: bool = False,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> tuple[ColumnElement, utils.GenericDataType | None]:
|
||||
"""
|
||||
Turn an adhoc column into a sqlalchemy column.
|
||||
@@ -1989,6 +2030,8 @@ class SqlaTable(
|
||||
This is needed to validate if a filter with an adhoc column
|
||||
is applicable.
|
||||
:param template_processor: template_processor instance
|
||||
:param apply_dataset_offset: shift the selected axis before truncation
|
||||
:param sql_shifted_temporal_labels: labels shifted before truncation
|
||||
:returns: A tuple of (SQLAlchemy column, generic column type). The
|
||||
generic type is populated when the column type is resolved
|
||||
(either because the adhoc column matches a physical column, or
|
||||
@@ -2005,6 +2048,7 @@ class SqlaTable(
|
||||
pdf = None
|
||||
is_column_reference = col.get("isColumnReference", False)
|
||||
generic_type: utils.GenericDataType | None = None
|
||||
native_type: str | None = None
|
||||
|
||||
metadata_lookup_key = self._render_adhoc_expression_for_metadata_lookup(
|
||||
sql_expression, template_processor
|
||||
@@ -2019,6 +2063,7 @@ class SqlaTable(
|
||||
is_dttm = col_in_metadata.is_temporal
|
||||
pdf = col_in_metadata.python_date_format
|
||||
generic_type = col_in_metadata.type_generic
|
||||
native_type = col_in_metadata.type
|
||||
else:
|
||||
# Column doesn't exist in metadata or is not a reference - treat as ad-hoc
|
||||
# expression Note: If isColumnReference=true but column not found, we still
|
||||
@@ -2081,8 +2126,28 @@ class SqlaTable(
|
||||
# stay unquoted for numeric adhoc expressions like
|
||||
# CAST(... AS BIGINT)).
|
||||
generic_type = col_desc[0].get("type_generic")
|
||||
probed_type = col_desc[0].get("type")
|
||||
native_type = str(probed_type) if probed_type is not None else None
|
||||
|
||||
if is_dttm and has_timegrain:
|
||||
if (
|
||||
apply_dataset_offset
|
||||
and self.db_engine_spec.supports_temporal_column_shift
|
||||
and (offset_hours := self.offset or 0)
|
||||
and not self.get_dataset_timezone()
|
||||
):
|
||||
effective_offset_hours = get_effective_hours_offset(
|
||||
self.db_engine_spec,
|
||||
native_type,
|
||||
offset_hours,
|
||||
db_extra=self.db_extra,
|
||||
)
|
||||
sqla_column = self._shift_temporal_column_if_needed(
|
||||
sqla_column,
|
||||
effective_offset_hours,
|
||||
)
|
||||
if sql_shifted_temporal_labels is not None:
|
||||
sql_shifted_temporal_labels.add(label)
|
||||
sqla_column = self.db_engine_spec.get_timestamp_expr(
|
||||
col=sqla_column,
|
||||
pdf=pdf,
|
||||
|
||||
@@ -21,11 +21,7 @@ from flask import g
|
||||
from sqlalchemy.exc import NoResultFound
|
||||
|
||||
from superset.commands.tag.exceptions import TagNotFoundError
|
||||
from superset.commands.tag.utils import (
|
||||
current_user_can_modify_object,
|
||||
to_object_model,
|
||||
to_object_type,
|
||||
)
|
||||
from superset.commands.tag.utils import to_object_model, to_object_type
|
||||
from superset.daos.base import BaseDAO
|
||||
from superset.daos.chart import ChartDAO
|
||||
from superset.daos.dashboard import DashboardDAO
|
||||
@@ -349,6 +345,10 @@ class TagDAO(BaseDAO[Tag]):
|
||||
Returns:
|
||||
None.
|
||||
"""
|
||||
# Imported lazily: superset.commands.utils itself imports TagDAO from
|
||||
# this module, so a top-level import here would be circular.
|
||||
from superset.commands.utils import current_user_can_modify_object
|
||||
|
||||
tagged_objects = []
|
||||
if not tag:
|
||||
raise TagNotFoundError()
|
||||
|
||||
@@ -188,7 +188,6 @@ from superset.views.base_api import (
|
||||
statsd_metrics,
|
||||
validate_feature_flags,
|
||||
)
|
||||
from superset.views.custom_tags_api_mixin import CustomTagsOptimizationMixin
|
||||
from superset.views.error_handling import handle_api_exception
|
||||
from superset.views.filters import (
|
||||
BaseFilterRelatedUsers,
|
||||
@@ -259,20 +258,12 @@ BASE_LIST_COLUMNS = [
|
||||
"uuid",
|
||||
]
|
||||
|
||||
# Full tags (current behavior - includes all tag types)
|
||||
FULL_TAG_LIST_COLUMNS = BASE_LIST_COLUMNS + [
|
||||
TAG_LIST_COLUMNS = BASE_LIST_COLUMNS + [
|
||||
"tags.id",
|
||||
"tags.name",
|
||||
"tags.type",
|
||||
]
|
||||
|
||||
# Custom tags only
|
||||
CUSTOM_TAG_LIST_COLUMNS = BASE_LIST_COLUMNS + [
|
||||
"custom_tags.id",
|
||||
"custom_tags.name",
|
||||
"custom_tags.type",
|
||||
]
|
||||
|
||||
# Fields dropped from a dashboard member dataset when the caller cannot access
|
||||
# that datasource on its own: everything describing the dataset's schema,
|
||||
# connection, and query construction. The identifying fields the dashboard
|
||||
@@ -300,9 +291,7 @@ DASHBOARD_DATASET_INACCESSIBLE_FIELDS = (
|
||||
|
||||
|
||||
# pylint: disable=too-many-public-methods
|
||||
class DashboardRestApi(
|
||||
SoftDeleteApiMixin, CustomTagsOptimizationMixin, BaseSupersetModelRestApi
|
||||
):
|
||||
class DashboardRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
|
||||
datamodel = SQLAInterface(Dashboard)
|
||||
|
||||
include_route_methods = RouteMethod.REST_MODEL_VIEW_CRUD_SET | {
|
||||
@@ -357,17 +346,7 @@ class DashboardRestApi(
|
||||
"purge": "write",
|
||||
}
|
||||
|
||||
# Default list_columns (used if config not set)
|
||||
list_columns = FULL_TAG_LIST_COLUMNS
|
||||
|
||||
def __init__(self) -> None:
|
||||
# Configure custom tags optimization (mixin handles the logic)
|
||||
self._setup_custom_tags_optimization(
|
||||
config_key="DASHBOARD_LIST_CUSTOM_TAGS_ONLY",
|
||||
full_columns=FULL_TAG_LIST_COLUMNS,
|
||||
custom_columns=CUSTOM_TAG_LIST_COLUMNS,
|
||||
)
|
||||
super().__init__()
|
||||
list_columns = TAG_LIST_COLUMNS
|
||||
|
||||
@expose("/", methods=("GET",))
|
||||
@protect()
|
||||
|
||||
@@ -292,7 +292,6 @@ class DashboardGetResponseSchema(Schema):
|
||||
editors = fields.List(fields.Nested(SubjectResponseSchema))
|
||||
viewers = fields.List(fields.Nested(SubjectResponseSchema))
|
||||
tags = fields.Nested(TagSchema, many=True)
|
||||
custom_tags = fields.Nested(TagSchema, many=True)
|
||||
changed_on_humanized = fields.String(data_key="changed_on_delta_humanized")
|
||||
created_on_humanized = fields.String(data_key="created_on_delta_humanized")
|
||||
is_managed_externally = fields.Boolean(allow_none=True, dump_default=False)
|
||||
@@ -302,12 +301,6 @@ class DashboardGetResponseSchema(Schema):
|
||||
# pylint: disable=unused-argument
|
||||
@post_dump()
|
||||
def post_dump(self, serialized: dict[str, Any], **kwargs: Any) -> dict[str, Any]:
|
||||
# Handle custom_tags → tags renaming when flag is enabled
|
||||
# When DASHBOARD_LIST_CUSTOM_TAGS_ONLY=True, FAB populates custom_tags
|
||||
# Rename it to tags for frontend compatibility
|
||||
if "custom_tags" in serialized:
|
||||
serialized["tags"] = serialized.pop("custom_tags")
|
||||
|
||||
if security_manager.is_guest_user():
|
||||
del serialized["changed_by_name"]
|
||||
del serialized["changed_by"]
|
||||
|
||||
@@ -539,6 +539,7 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
# the ``array_*`` capability methods below must be implemented. Defaults to
|
||||
# False so engines that have not opted in keep treating arrays as strings.
|
||||
supports_multivalue_columns = False
|
||||
supports_temporal_column_shift: bool = False
|
||||
allows_joins = True
|
||||
allows_subqueries = True
|
||||
allows_alias_in_select = True
|
||||
@@ -1242,6 +1243,19 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
|
||||
return TimestampExpression(time_expr, col, type_=col.type)
|
||||
|
||||
@classmethod
|
||||
def get_temporal_column_shift_expr(
|
||||
cls,
|
||||
col: ColumnClause,
|
||||
offset_hours: int,
|
||||
) -> TimestampExpression:
|
||||
"""Shift a temporal SQL expression by a bounded number of hours."""
|
||||
return TimestampExpression(
|
||||
f"{{col}} + INTERVAL '{offset_hours}' HOUR",
|
||||
col,
|
||||
type_=col.type,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _apply_year_to_dttm(cls, time_expr: str) -> str:
|
||||
"""
|
||||
@@ -1378,12 +1392,12 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
return cursor.fetchmany(limit)
|
||||
data = cursor.fetchall()
|
||||
description = cursor.description or []
|
||||
# Create a mapping between column name and a mutator function to normalize
|
||||
# values with. The first two items in the description row are
|
||||
# the column name and type.
|
||||
# Create a mapping between column index and a mutator function to normalize
|
||||
# values with. The first two items in the description row are the column
|
||||
# name and type.
|
||||
column_mutators = {
|
||||
row[0]: func
|
||||
for row in description
|
||||
index: func
|
||||
for index, row in enumerate(description)
|
||||
if (
|
||||
func := cls.column_type_mutators.get(
|
||||
type(cls.get_sqla_column_type(cls.get_datatype(row[1])))
|
||||
@@ -1391,11 +1405,11 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
)
|
||||
}
|
||||
if column_mutators:
|
||||
indexes = {row[0]: idx for idx, row in enumerate(description)}
|
||||
if not isinstance(data, list):
|
||||
data = list(data)
|
||||
for row_idx, row in enumerate(data):
|
||||
new_row = list(row)
|
||||
for col, func in column_mutators.items():
|
||||
col_idx = indexes[col]
|
||||
for col_idx, func in column_mutators.items():
|
||||
new_row[col_idx] = func(row[col_idx])
|
||||
data[row_idx] = tuple(new_row)
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from datetime import datetime
|
||||
from re import Pattern
|
||||
from typing import Any, TYPE_CHECKING, TypedDict
|
||||
|
||||
@@ -32,7 +33,7 @@ from marshmallow.exceptions import ValidationError
|
||||
from requests import Session
|
||||
from shillelagh.adapters.api.gsheets.lib import SCOPES
|
||||
from shillelagh.exceptions import UnauthenticatedError
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy import text, types
|
||||
from sqlalchemy.engine import create_engine
|
||||
from sqlalchemy.engine.reflection import Inspector
|
||||
from sqlalchemy.engine.url import URL
|
||||
@@ -155,6 +156,27 @@ class GSheetsEngineSpec(ShillelaghEngineSpec):
|
||||
oauth2_token_request_uri = "https://oauth2.googleapis.com/token" # noqa: S105
|
||||
oauth2_exception = (UnauthenticatedError, OAuth2TokenRefreshError)
|
||||
|
||||
@classmethod
|
||||
def convert_dttm(
|
||||
cls, target_type: str, dttm: datetime, db_extra: dict[str, Any] | None = None
|
||||
) -> str | None:
|
||||
"""
|
||||
Convert a datetime to a SQL literal understood by shillelagh's GSheets
|
||||
adapter.
|
||||
|
||||
``SqliteEngineSpec.convert_dttm`` (inherited via ``ShillelaghEngineSpec``)
|
||||
has no case for ``types.Date`` and returns ``None``, which makes Superset
|
||||
fall back to a literal that still carries a time-of-day component. The
|
||||
GSheets adapter's virtual table layer parses that literal with
|
||||
``datetime.date.fromisoformat``, which rejects the trailing time and
|
||||
silently drops the filter value, producing an invalid query against the
|
||||
Google Sheets API. A bare ``YYYY-MM-DD`` literal is required instead.
|
||||
"""
|
||||
sqla_type = cls.get_sqla_column_type(target_type)
|
||||
if isinstance(sqla_type, types.Date):
|
||||
return f"'{dttm.date().isoformat()}'"
|
||||
return super().convert_dttm(target_type, dttm, db_extra=db_extra)
|
||||
|
||||
@classmethod
|
||||
def get_oauth2_authorization_uri(
|
||||
cls,
|
||||
|
||||
@@ -267,6 +267,43 @@ class MySQLEngineSpec(BasicParametersMixin, BaseEngineSpec):
|
||||
types.VARCHAR(),
|
||||
GenericDataType.STRING,
|
||||
),
|
||||
# wire-protocol FIELD_TYPE names emitted by `get_datatype`, seen on
|
||||
# SQL Lab and virtual dataset columns instead of DDL type names
|
||||
(
|
||||
re.compile(r"^newdecimal", re.IGNORECASE),
|
||||
DECIMAL(),
|
||||
GenericDataType.NUMERIC,
|
||||
),
|
||||
(
|
||||
re.compile(r"^tiny$", re.IGNORECASE),
|
||||
TINYINT(),
|
||||
GenericDataType.NUMERIC,
|
||||
),
|
||||
(
|
||||
re.compile(r"^short$", re.IGNORECASE),
|
||||
types.SmallInteger(),
|
||||
GenericDataType.NUMERIC,
|
||||
),
|
||||
(
|
||||
re.compile(r"^(blob|text)$", re.IGNORECASE),
|
||||
types.String(),
|
||||
GenericDataType.STRING,
|
||||
),
|
||||
(
|
||||
re.compile(r"^year$", re.IGNORECASE),
|
||||
types.Integer(),
|
||||
GenericDataType.NUMERIC,
|
||||
),
|
||||
(
|
||||
re.compile(r"^enum\b", re.IGNORECASE),
|
||||
types.String(),
|
||||
GenericDataType.STRING,
|
||||
),
|
||||
(
|
||||
re.compile(r"^set\b", re.IGNORECASE),
|
||||
types.String(),
|
||||
GenericDataType.STRING,
|
||||
),
|
||||
)
|
||||
column_type_mutators: dict[types.TypeEngine, Callable[[Any], Any]] = {
|
||||
DECIMAL: lambda val: Decimal(val) if isinstance(val, str) else val
|
||||
@@ -425,22 +462,27 @@ class MySQLEngineSpec(BasicParametersMixin, BaseEngineSpec):
|
||||
|
||||
@classmethod
|
||||
def get_datatype(cls, type_code: Any) -> Optional[str]:
|
||||
if not cls.type_code_map:
|
||||
# only import and store if needed at least once
|
||||
# pylint: disable=import-outside-toplevel
|
||||
try:
|
||||
import MySQLdb
|
||||
|
||||
mysql_module = MySQLdb
|
||||
except ImportError:
|
||||
mysql_module = __import__("pymysql")
|
||||
|
||||
ft = mysql_module.constants.FIELD_TYPE
|
||||
cls.type_code_map = {
|
||||
getattr(ft, k): k for k in dir(ft) if not k.startswith("_")
|
||||
}
|
||||
datatype = type_code
|
||||
if isinstance(type_code, int):
|
||||
if not cls.type_code_map:
|
||||
# only import and store if needed at least once
|
||||
# pylint: disable=import-outside-toplevel
|
||||
try:
|
||||
import MySQLdb
|
||||
|
||||
ft = MySQLdb.constants.FIELD_TYPE
|
||||
except ImportError:
|
||||
try:
|
||||
import pymysql # type: ignore[import-untyped]
|
||||
|
||||
ft = pymysql.constants.FIELD_TYPE
|
||||
except ImportError:
|
||||
from mysql.connector.constants import FieldType
|
||||
|
||||
ft = FieldType
|
||||
cls.type_code_map = {
|
||||
getattr(ft, k): k for k in dir(ft) if not k.startswith("_")
|
||||
}
|
||||
datatype = cls.type_code_map.get(type_code)
|
||||
if datatype and isinstance(datatype, str) and datatype:
|
||||
return datatype
|
||||
|
||||
@@ -360,6 +360,7 @@ class PostgresEngineSpec(BasicParametersMixin, PostgresBaseEngineSpec):
|
||||
supports_catalog = True
|
||||
supports_dynamic_catalog = True
|
||||
supports_grouping_sets = True
|
||||
supports_temporal_column_shift = True
|
||||
|
||||
default_driver = "psycopg2"
|
||||
parameters_schema = PostgresParametersSchema()
|
||||
|
||||
@@ -25,9 +25,14 @@ from typing import Any, TYPE_CHECKING
|
||||
from flask_babel import gettext as __
|
||||
from sqlalchemy import types
|
||||
from sqlalchemy.engine.reflection import Inspector
|
||||
from sqlalchemy.sql.elements import ColumnClause
|
||||
|
||||
from superset.constants import TimeGrain
|
||||
from superset.db_engine_specs.base import BaseEngineSpec, DatabaseCategory
|
||||
from superset.db_engine_specs.base import (
|
||||
BaseEngineSpec,
|
||||
DatabaseCategory,
|
||||
TimestampExpression,
|
||||
)
|
||||
from superset.errors import SupersetErrorType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -43,6 +48,7 @@ class SqliteEngineSpec(BaseEngineSpec):
|
||||
|
||||
disable_ssh_tunneling = True
|
||||
supports_multivalues_insert = True
|
||||
supports_temporal_column_shift = True
|
||||
|
||||
metadata = {
|
||||
"description": "SQLite is a self-contained, serverless SQL database engine.",
|
||||
@@ -140,6 +146,20 @@ class SqliteEngineSpec(BaseEngineSpec):
|
||||
"ELSE printf('%04d-01-01', CAST({col} AS INTEGER)) END)"
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def get_temporal_column_shift_expr(
|
||||
cls,
|
||||
col: ColumnClause,
|
||||
offset_hours: int,
|
||||
) -> TimestampExpression:
|
||||
"""Shift a temporal expression with SQLite's datetime modifier syntax."""
|
||||
modifier = f"{offset_hours:+d} hours"
|
||||
return TimestampExpression(
|
||||
f"DATETIME({{col}}, '{modifier}')",
|
||||
col,
|
||||
type_=col.type,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def convert_dttm(
|
||||
cls, target_type: str, dttm: datetime, db_extra: dict[str, Any] | None = None
|
||||
|
||||
@@ -30,7 +30,6 @@ from superset.commands.temporary_cache.exceptions import (
|
||||
TemporaryCacheResourceNotFoundError,
|
||||
)
|
||||
from superset.constants import MODEL_API_RW_METHOD_PERMISSION_MAP
|
||||
from superset.exceptions import SupersetTemplateException
|
||||
from superset.explore.form_data.schemas import FormDataPostSchema, FormDataPutSchema
|
||||
from superset.extensions import event_logger
|
||||
from superset.views.base_api import BaseSupersetApi, requires_json, statsd_metrics
|
||||
@@ -111,8 +110,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
|
||||
return self.response(403, message=str(ex))
|
||||
except TemporaryCacheResourceNotFoundError as ex:
|
||||
return self.response(404, message=str(ex))
|
||||
except SupersetTemplateException as ex:
|
||||
return self.response(ex.status, message=str(ex))
|
||||
|
||||
@expose("/form_data/<string:key>", methods=("PUT",))
|
||||
@protect()
|
||||
@@ -186,8 +183,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
|
||||
return self.response(403, message=str(ex))
|
||||
except TemporaryCacheResourceNotFoundError as ex:
|
||||
return self.response(404, message=str(ex))
|
||||
except SupersetTemplateException as ex:
|
||||
return self.response(ex.status, message=str(ex))
|
||||
|
||||
@expose("/form_data/<string:key>", methods=("GET",))
|
||||
@protect()
|
||||
@@ -239,8 +234,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
|
||||
return self.response(403, message=str(ex))
|
||||
except TemporaryCacheResourceNotFoundError as ex:
|
||||
return self.response(404, message=str(ex))
|
||||
except SupersetTemplateException as ex:
|
||||
return self.response(ex.status, message=str(ex))
|
||||
|
||||
@expose("/form_data/<string:key>", methods=("DELETE",))
|
||||
@protect()
|
||||
@@ -293,5 +286,3 @@ class ExploreFormDataRestApi(BaseSupersetApi):
|
||||
return self.response(403, message=str(ex))
|
||||
except TemporaryCacheResourceNotFoundError as ex:
|
||||
return self.response(404, message=str(ex))
|
||||
except SupersetTemplateException as ex:
|
||||
return self.response(ex.status, message=str(ex))
|
||||
|
||||
@@ -16,8 +16,6 @@
|
||||
# under the License.
|
||||
from typing import Optional
|
||||
|
||||
from jinja2.exceptions import TemplateError
|
||||
|
||||
from superset import security_manager
|
||||
from superset.commands.chart.exceptions import (
|
||||
ChartAccessDeniedError,
|
||||
@@ -35,7 +33,6 @@ from superset.commands.exceptions import (
|
||||
from superset.daos.chart import ChartDAO
|
||||
from superset.daos.dataset import DatasetDAO
|
||||
from superset.daos.query import QueryDAO
|
||||
from superset.exceptions import SupersetTemplateException
|
||||
from superset.utils.core import DatasourceType
|
||||
|
||||
|
||||
@@ -56,13 +53,7 @@ def check_query_access(query_id: int) -> Optional[bool]:
|
||||
# Access checks below, no need to validate them twice as they can be expensive.
|
||||
query = QueryDAO.find_by_id(query_id, skip_base_filter=True)
|
||||
if query:
|
||||
try:
|
||||
security_manager.raise_for_access(query=query)
|
||||
except TemplateError as ex:
|
||||
# raise_for_access() Jinja-renders the query's SQL to resolve
|
||||
# the tables it touches; a malformed template surfaces here as
|
||||
# a raw jinja2 exception rather than a Superset one.
|
||||
raise SupersetTemplateException(str(ex)) from ex
|
||||
security_manager.raise_for_access(query=query)
|
||||
return True
|
||||
raise QueryNotFoundValidationError()
|
||||
|
||||
|
||||
@@ -672,21 +672,34 @@ kubectl get ingress -n superset
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `MCP_DEV_USERNAME` | Superset username for MCP authentication | `admin` |
|
||||
| `MCP_AUTH_ENABLED` | Enable/disable authentication | `true` |
|
||||
| `MCP_DEV_USERNAME` | Superset username for MCP authentication in dev mode. Mutually exclusive with `MCP_AUTH_ENABLED = True`: the server refuses to start if both are set. | - |
|
||||
| `MCP_AUTH_ENABLED` | Enable/disable authentication | `false` |
|
||||
| `MCP_JWT_PUBLIC_KEY` | JWT public key for token validation | - |
|
||||
| `SUPERSET_WEBSERVER_ADDRESS` | Internal Superset URL | `http://localhost:8088` |
|
||||
| `WEBDRIVER_BASEURL` | URL for screenshot generation | Same as webserver |
|
||||
|
||||
#### superset_config.py Options
|
||||
|
||||
Dev mode (`MCP_DEV_USERNAME`) and JWT authentication (`MCP_AUTH_ENABLED`) are
|
||||
mutually exclusive -- the server raises at startup if both are set, since a
|
||||
fixed dev-mode identity would defeat the point of requiring real auth. Pick one:
|
||||
|
||||
```python
|
||||
# MCP Service Configuration
|
||||
# MCP Service Configuration -- development/testing (no auth)
|
||||
MCP_DEV_USERNAME = 'admin' # Username for development/testing
|
||||
|
||||
# WebDriver for chart screenshots
|
||||
WEBDRIVER_BASEURL = 'http://superset:8088/'
|
||||
WEBDRIVER_TYPE = 'chrome'
|
||||
WEBDRIVER_OPTION_ARGS = ['--headless', '--no-sandbox']
|
||||
```
|
||||
|
||||
```python
|
||||
# MCP Service Configuration -- production with JWT authentication
|
||||
MCP_AUTH_ENABLED = True # Enable authentication
|
||||
MCP_JWT_PUBLIC_KEY = 'your-public-key' # For JWT token validation
|
||||
|
||||
# For production with JWT authentication
|
||||
# Or, for a fully custom auth setup instead of the built-in JWT verifier:
|
||||
MCP_AUTH_FACTORY = 'your.custom.auth_factory'
|
||||
MCP_USER_RESOLVER = 'your.custom.user_resolver'
|
||||
|
||||
|
||||
@@ -58,6 +58,7 @@ if os.environ.get("FASTMCP_TRANSPORT", "stdio") == "stdio":
|
||||
click.secho = secho_to_stderr
|
||||
|
||||
from superset.mcp_service.app import init_fastmcp_server, mcp
|
||||
from superset.mcp_service.caching import create_response_caching_middleware
|
||||
from superset.mcp_service.middleware import create_response_size_guard_middleware
|
||||
from superset.mcp_service.server import build_middleware_list
|
||||
|
||||
@@ -67,8 +68,9 @@ def _add_default_middlewares() -> None:
|
||||
|
||||
Delegates to ``server.build_middleware_list()`` for the core stack so
|
||||
the stdio entry point stays in sync with the HTTP server without
|
||||
duplicating middleware ordering. The optional response size guard is
|
||||
appended separately (innermost position, same as in run_server()).
|
||||
duplicating middleware ordering. The optional response size guard and
|
||||
response caching middleware are appended separately (innermost
|
||||
position, same order as in run_server()).
|
||||
|
||||
FastMCP wraps handlers so that the FIRST-added middleware is outermost.
|
||||
``build_middleware_list()`` already returns middlewares in the correct
|
||||
@@ -77,12 +79,16 @@ def _add_default_middlewares() -> None:
|
||||
for middleware in build_middleware_list():
|
||||
mcp.add_middleware(middleware)
|
||||
|
||||
# Response size guard is innermost (added last)
|
||||
# Response size guard is innermost (added last), then response caching.
|
||||
if size_guard := create_response_size_guard_middleware():
|
||||
mcp.add_middleware(size_guard)
|
||||
limit = size_guard.token_limit
|
||||
sys.stderr.write(f"[MCP] Response size guard enabled (token_limit={limit})\n")
|
||||
|
||||
if caching_middleware := create_response_caching_middleware():
|
||||
mcp.add_middleware(caching_middleware)
|
||||
sys.stderr.write("[MCP] Response caching enabled\n")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""
|
||||
@@ -157,8 +163,19 @@ def main() -> None:
|
||||
sys.stderr.write(f"[MCP] Client disconnected: {e}\n")
|
||||
sys.exit(0)
|
||||
else:
|
||||
# For other transports, use normal initialization
|
||||
init_fastmcp_server()
|
||||
# For other transports (network listeners), install the same auth
|
||||
# provider as the supported entry point (`superset mcp run` ->
|
||||
# server.run_server()) instead of starting with no verifier at all.
|
||||
# _create_auth_provider fails closed (raises MCPAuthConfigError) when
|
||||
# auth is configured but a verifier could not be built, so letting
|
||||
# that propagate here refuses to start rather than silently running
|
||||
# this transport unauthenticated.
|
||||
from superset.mcp_service.flask_singleton import get_flask_app
|
||||
from superset.mcp_service.server import _create_auth_provider
|
||||
|
||||
flask_app = get_flask_app()
|
||||
auth_provider = _create_auth_provider(flask_app)
|
||||
init_fastmcp_server(auth=auth_provider)
|
||||
_add_default_middlewares()
|
||||
|
||||
# Run with specified transport
|
||||
|
||||
@@ -575,7 +575,8 @@ def _resolve_user_from_jwt_context(app: Any) -> MCPUser | None: # noqa: C901
|
||||
the corresponding ``GuestUser`` built from the token's resources/RLS.
|
||||
|
||||
Raises:
|
||||
ValueError: If JWT resolves a username that doesn't exist in the DB
|
||||
ValueError: If JWT resolves a username that doesn't exist in the DB,
|
||||
or a guest-marked token is presented while guest auth is disabled
|
||||
(fail closed — do NOT fall through to weaker auth sources).
|
||||
MCPAuthConfigError: If more than one JWT issuer is trusted
|
||||
(``MCP_JWT_ISSUER`` is a list/tuple/set) and no issuer-aware
|
||||
@@ -618,7 +619,14 @@ def _resolve_user_from_jwt_context(app: Any) -> MCPUser | None: # noqa: C901
|
||||
"Guest-marked token presented but embedded guest auth is not "
|
||||
"enabled; rejecting"
|
||||
)
|
||||
return None
|
||||
# Fail closed, matching the sibling failure branches below: a
|
||||
# guest-marked token is an explicit (rejected) authentication
|
||||
# attempt, not an absent one. Returning None here would let the
|
||||
# request degrade to weaker auth sources (API key,
|
||||
# MCP_DEV_USERNAME, or a middleware-set g.user).
|
||||
raise ValueError(
|
||||
"Guest-marked token presented but embedded guest auth is not enabled"
|
||||
)
|
||||
logger.debug("Resolving MCP request as embedded guest user")
|
||||
# Drop the internal marker so it does not leak into GuestUser.guest_token.
|
||||
guest_claims: dict[str, Any] = {
|
||||
|
||||
@@ -123,6 +123,27 @@ def create_response_caching_middleware() -> Any | None:
|
||||
logger.debug("MCP response caching disabled")
|
||||
return None
|
||||
|
||||
# ResponseCachingMiddleware keys cache entries on the method/tool
|
||||
# name + arguments only and runs ahead of the per-request auth/RBAC
|
||||
# checks, so a cache hit returns a response computed for a different
|
||||
# caller. Only appropriate when every request is guaranteed to come
|
||||
# from the same principal.
|
||||
# that sends byte-identical arguments within the TTL, skipping every
|
||||
# authorization check. Fail closed unless the operator explicitly
|
||||
# accepts a cache shared across principals -- only safe when every
|
||||
# request is guaranteed to come from the same principal (e.g. a
|
||||
# single-user development deployment).
|
||||
if not cache_config.get("dangerously_share_cache_across_principals", False):
|
||||
logger.warning(
|
||||
"MCP_CACHE_CONFIG['enabled'] is set, but response caching "
|
||||
"stays disabled: cache keys do not include the requesting "
|
||||
"principal, so cached responses would be served across users "
|
||||
"without any authorization checks. Set "
|
||||
"'dangerously_share_cache_across_principals': True only when "
|
||||
"all requests share a single principal."
|
||||
)
|
||||
return None
|
||||
|
||||
try:
|
||||
from fastmcp.server.middleware.caching import ResponseCachingMiddleware
|
||||
except ImportError:
|
||||
|
||||
@@ -116,6 +116,53 @@ async def restore_chart(
|
||||
return RestoreChartResponse(success=False, error=msg, error_type="NotFound")
|
||||
|
||||
chart_id = chart.id
|
||||
|
||||
# The lookup above deliberately bypasses the RBAC base filter (see
|
||||
# _find_chart_for_restore), so enforce the restore audience *before*
|
||||
# composing any response that embeds the chart's name: without this gate,
|
||||
# iterating identifiers would disclose the existence and exact title of
|
||||
# charts the caller cannot see (the web API answers 404 for those).
|
||||
from superset import security_manager
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
try:
|
||||
try:
|
||||
security_manager.raise_for_editorship(chart)
|
||||
except SupersetSecurityException:
|
||||
from superset.daos.chart import ChartDAO
|
||||
|
||||
# Distinguish "visible but not an editor" from "outside the
|
||||
# caller's RBAC scope": the latter must be indistinguishable
|
||||
# from a chart that does not exist.
|
||||
visible = ChartDAO.find_by_id_or_uuid(
|
||||
str(request.identifier), skip_visibility_filter=True
|
||||
)
|
||||
if visible is None:
|
||||
display_id = str(request.identifier)[:200]
|
||||
return RestoreChartResponse(
|
||||
success=False,
|
||||
error=f"No chart found with identifier: {display_id}.",
|
||||
error_type="NotFound",
|
||||
)
|
||||
await ctx.warning("Permission denied restoring chart id=%s" % (chart_id,))
|
||||
return RestoreChartResponse(
|
||||
success=False,
|
||||
permission_denied=True,
|
||||
error=(
|
||||
f"You do not have permission to restore chart id={chart_id}. "
|
||||
"Ask the user to restore it or grant access; do not retry."
|
||||
),
|
||||
error_type="Forbidden",
|
||||
)
|
||||
except SQLAlchemyError:
|
||||
_rollback()
|
||||
logger.exception("Editorship check failed during restore_chart")
|
||||
return RestoreChartResponse(
|
||||
success=False,
|
||||
error="Chart lookup failed due to a database error.",
|
||||
error_type="LookupFailed",
|
||||
)
|
||||
|
||||
# Chart names are user-controlled and must remain exact in response text.
|
||||
chart_name = chart.slice_name
|
||||
|
||||
|
||||
@@ -118,6 +118,56 @@ async def restore_dashboard(
|
||||
return RestoreDashboardResponse(success=False, error=msg, error_type="NotFound")
|
||||
|
||||
dashboard_id = dashboard.id
|
||||
|
||||
# The lookup above deliberately bypasses the RBAC base filter (see
|
||||
# _find_dashboard_for_restore), so enforce the restore audience *before*
|
||||
# composing any response that embeds the dashboard's title: without this
|
||||
# gate, iterating identifiers would disclose the existence and exact title
|
||||
# of dashboards the caller cannot see (the web API answers 404 for those).
|
||||
from superset import security_manager
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
try:
|
||||
try:
|
||||
security_manager.raise_for_editorship(dashboard)
|
||||
except SupersetSecurityException:
|
||||
from superset.daos.dashboard import DashboardDAO
|
||||
|
||||
# Distinguish "visible but not an editor" from "outside the
|
||||
# caller's RBAC scope": the latter must be indistinguishable
|
||||
# from a dashboard that does not exist.
|
||||
visible = DashboardDAO.find_by_id_or_uuid(
|
||||
str(request.identifier), skip_visibility_filter=True
|
||||
)
|
||||
if visible is None:
|
||||
display_id = str(request.identifier)[:200]
|
||||
return RestoreDashboardResponse(
|
||||
success=False,
|
||||
error=f"No dashboard found with identifier: {display_id}.",
|
||||
error_type="NotFound",
|
||||
)
|
||||
await ctx.warning(
|
||||
"Permission denied restoring dashboard id=%s" % (dashboard_id,)
|
||||
)
|
||||
return RestoreDashboardResponse(
|
||||
success=False,
|
||||
permission_denied=True,
|
||||
error=(
|
||||
f"You do not have permission to restore dashboard "
|
||||
f"id={dashboard_id}. Ask the user to restore it or grant "
|
||||
"access; do not retry."
|
||||
),
|
||||
error_type="Forbidden",
|
||||
)
|
||||
except SQLAlchemyError:
|
||||
_rollback()
|
||||
logger.exception("Editorship check failed during restore_dashboard")
|
||||
return RestoreDashboardResponse(
|
||||
success=False,
|
||||
error="Dashboard lookup failed due to a database error.",
|
||||
error_type="LookupFailed",
|
||||
)
|
||||
|
||||
# Dashboard titles are user-controlled and must remain exact in response text.
|
||||
dashboard_name = dashboard.dashboard_title
|
||||
|
||||
|
||||
@@ -259,7 +259,9 @@ MCP_FACTORY_CONFIG = {
|
||||
#
|
||||
# Configuration Flow:
|
||||
# -------------------
|
||||
# - MCP_CACHE_CONFIG controls whether caching is enabled and its TTL settings
|
||||
# - MCP_CACHE_CONFIG controls whether caching is enabled and its TTL settings.
|
||||
# Note "enabled" alone is not sufficient -- see
|
||||
# "dangerously_share_cache_across_principals" below.
|
||||
# - MCP_STORE_CONFIG controls the Redis store (optional)
|
||||
#
|
||||
# Scenarios:
|
||||
@@ -270,11 +272,13 @@ MCP_FACTORY_CONFIG = {
|
||||
#
|
||||
# 2. Caching with in-memory store:
|
||||
# MCP_CACHE_CONFIG["enabled"] = True
|
||||
# MCP_CACHE_CONFIG["dangerously_share_cache_across_principals"] = True
|
||||
# MCP_STORE_CONFIG["enabled"] = False (or not configured)
|
||||
# → Caching uses FastMCP's default in-memory store, no Prefix wrapper used
|
||||
#
|
||||
# 3. Caching with Redis store:
|
||||
# MCP_CACHE_CONFIG["enabled"] = True
|
||||
# MCP_CACHE_CONFIG["dangerously_share_cache_across_principals"] = True
|
||||
# MCP_STORE_CONFIG["enabled"] = True
|
||||
# MCP_STORE_CONFIG["CACHE_REDIS_URL"] = "redis://..."
|
||||
# → Caching uses Redis with PrefixKeysWrapper
|
||||
@@ -322,6 +326,13 @@ MCP_STORE_CONFIG: dict[str, Any] = {
|
||||
# When enabled with MCP_STORE_CONFIG, uses Redis store.
|
||||
MCP_CACHE_CONFIG: dict[str, Any] = {
|
||||
"enabled": False, # Disabled by default
|
||||
# Cache keys are method/tool + arguments only and cache hits are served
|
||||
# ahead of per-request auth/RBAC, so a shared cache can return one
|
||||
# caller's response to another. Response caching refuses to start
|
||||
# unless this is explicitly set -- only appropriate when every request
|
||||
# is guaranteed to come from the same principal (e.g. a single-user
|
||||
# development deployment).
|
||||
"dangerously_share_cache_across_principals": False,
|
||||
# Base prefix for the shared store. Superset appends an internal response-
|
||||
# contract namespace so incompatible cached values are not reused.
|
||||
"CACHE_KEY_PREFIX": None, # Only needed when using the store
|
||||
@@ -332,12 +343,38 @@ MCP_CACHE_CONFIG: dict[str, Any] = {
|
||||
"get_prompt_ttl": 60 * 60, # 1 hour
|
||||
"call_tool_ttl": 60 * 60, # 1 hour
|
||||
"max_item_size": 1024 * 1024, # 1MB
|
||||
"excluded_tools": [ # Tools that should never be cached (side effects, dynamic)
|
||||
"execute_sql",
|
||||
"generate_dashboard",
|
||||
# Every tool whose ToolAnnotations set readOnlyHint=False, i.e. every tool
|
||||
# with a side effect. A cache hit is served ahead of per-request
|
||||
# auth/RBAC, so caching a mutating tool can replay a stale create/update/
|
||||
# delete result -- including to a caller who repeats an identical call
|
||||
# expecting it to run again. This list is enforced complete by
|
||||
# test_mcp_caching.py::test_excluded_tools_covers_every_mutating_tool,
|
||||
# which fails with the specific missing tool name(s) if a new
|
||||
# non-read-only tool is added without also being added here.
|
||||
"excluded_tools": [
|
||||
"add_chart_to_existing_dashboard",
|
||||
"create_dataset",
|
||||
"create_theme",
|
||||
"create_virtual_dataset",
|
||||
"delete_chart",
|
||||
"delete_dashboard",
|
||||
"duplicate_dashboard",
|
||||
"execute_sql",
|
||||
"generate_chart",
|
||||
"generate_dashboard",
|
||||
"generate_explore_link",
|
||||
"manage_dashboard_certification",
|
||||
"manage_dashboard_owners",
|
||||
"manage_dashboard_roles",
|
||||
"manage_native_filters",
|
||||
"remove_chart_from_dashboard",
|
||||
"restore_chart",
|
||||
"restore_dashboard",
|
||||
"save_sql_query",
|
||||
"update_chart",
|
||||
"update_chart_preview",
|
||||
"update_dashboard",
|
||||
"update_dataset_metric",
|
||||
],
|
||||
}
|
||||
|
||||
@@ -495,6 +532,16 @@ def create_default_mcp_auth_factory(app: Flask) -> Optional[Any]:
|
||||
if not (auth_enabled or api_key_enabled or guest_enabled):
|
||||
return None
|
||||
|
||||
# MCP_DEV_USERNAME makes user resolution fall back to a fixed user for
|
||||
# requests that carry no resolvable identity, which defeats the point of
|
||||
# having transport auth enabled. Refuse the combination outright.
|
||||
if auth_enabled and app.config.get("MCP_DEV_USERNAME"):
|
||||
raise MCPAuthConfigError(
|
||||
"MCP_DEV_USERNAME must not be set when MCP_AUTH_ENABLED is True: "
|
||||
"it would execute callers without a resolvable identity as that "
|
||||
"user. Unset MCP_DEV_USERNAME (a development-only convenience)."
|
||||
)
|
||||
|
||||
# When JWT auth is enabled, an audience must be configured so issued tokens
|
||||
# are bound to this service. Without it the verifier accepts any otherwise
|
||||
# valid same-issuer token, regardless of which service it was minted for.
|
||||
@@ -519,22 +566,40 @@ def create_default_mcp_auth_factory(app: Flask) -> Optional[Any]:
|
||||
secret = app.config.get("MCP_JWT_SECRET")
|
||||
|
||||
if not (jwks_uri or public_key or secret):
|
||||
logger.warning("MCP_AUTH_ENABLED is True but no JWT keys/secret configured")
|
||||
if not (api_key_enabled or guest_enabled):
|
||||
return None
|
||||
else:
|
||||
try:
|
||||
jwt_verifier = _build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=jwks_uri,
|
||||
public_key=public_key,
|
||||
secret=secret,
|
||||
)
|
||||
except Exception:
|
||||
# Do not log the exception — it may contain secrets (e.g., key material)
|
||||
logger.error("Failed to create MCP JWT verifier")
|
||||
if not (api_key_enabled or guest_enabled):
|
||||
return None
|
||||
# Fail closed regardless of API-key/guest fallbacks: JWT auth was
|
||||
# explicitly enabled, so silently starting without it would leave
|
||||
# the operator's chosen JWT mode disabled without warning them
|
||||
# via anything louder than a log line.
|
||||
raise MCPAuthConfigError(
|
||||
"MCP_AUTH_ENABLED is True but no JWT verification key is "
|
||||
"configured; refusing to start an unauthenticated MCP "
|
||||
"server. Set MCP_JWKS_URI, MCP_JWT_PUBLIC_KEY, or "
|
||||
"MCP_JWT_SECRET (with MCP_JWT_ALGORITHM='HS256')."
|
||||
)
|
||||
|
||||
try:
|
||||
jwt_verifier = _build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=jwks_uri,
|
||||
public_key=public_key,
|
||||
secret=secret,
|
||||
)
|
||||
except MCPAuthConfigError:
|
||||
raise
|
||||
except Exception:
|
||||
# Do not log or chain the exception — it may contain secrets
|
||||
# (e.g., key material)
|
||||
logger.error("Failed to create MCP JWT verifier")
|
||||
# Fail closed regardless of API-key/guest fallbacks: JWT auth
|
||||
# was explicitly enabled, so silently starting without it is
|
||||
# a permissive state the operator did not choose.
|
||||
raise MCPAuthConfigError(
|
||||
"Failed to construct the MCP JWT verifier from the "
|
||||
"configured key material; refusing to start with JWT "
|
||||
"auth silently disabled. Verify MCP_JWT_ALGORITHM "
|
||||
"matches the configured key (HS256 for MCP_JWT_SECRET; "
|
||||
"RS256 needs MCP_JWKS_URI or MCP_JWT_PUBLIC_KEY)."
|
||||
) from None
|
||||
|
||||
# A composite verifier is needed whenever API-key OR guest auth is on, so
|
||||
# those token types are recognized before (or instead of) the JWT verifier.
|
||||
@@ -708,15 +773,49 @@ def _build_jwt_verifier(
|
||||
"required_scopes": app.config.get("MCP_REQUIRED_SCOPES", []),
|
||||
}
|
||||
|
||||
# For HS256 (symmetric), use the secret as the public_key parameter
|
||||
if app.config.get("MCP_JWT_ALGORITHM") == "HS256" and secret:
|
||||
algorithm = app.config.get("MCP_JWT_ALGORITHM", "RS256")
|
||||
|
||||
if algorithm in ("HS256", "HS384", "HS512"):
|
||||
# HMAC algorithms are symmetric: verification MUST be keyed on an
|
||||
# explicit shared secret, never on public-key material (PEM or
|
||||
# JWKS), which isn't confidential. Refuse the contradictory
|
||||
# configuration outright instead of honoring it.
|
||||
if not secret:
|
||||
raise MCPAuthConfigError(
|
||||
f"MCP_JWT_ALGORITHM is '{algorithm}' but MCP_JWT_SECRET is "
|
||||
"not set. Refusing to build an HMAC verifier keyed on "
|
||||
"public-key material. Set MCP_JWT_SECRET, or switch to an "
|
||||
"asymmetric algorithm (e.g. RS256) with MCP_JWT_PUBLIC_KEY "
|
||||
"or MCP_JWKS_URI."
|
||||
)
|
||||
if public_key or jwks_uri:
|
||||
raise MCPAuthConfigError(
|
||||
"MCP_JWT_PUBLIC_KEY/MCP_JWKS_URI are configured alongside "
|
||||
f"MCP_JWT_ALGORITHM='{algorithm}'. This usually indicates "
|
||||
"leftover asymmetric-key configuration; remove the public "
|
||||
"key/JWKS settings, or switch back to an asymmetric "
|
||||
"algorithm."
|
||||
)
|
||||
# For HMAC (symmetric), use the secret as the public_key parameter
|
||||
common_kwargs["public_key"] = secret
|
||||
common_kwargs["algorithm"] = "HS256"
|
||||
common_kwargs["algorithm"] = algorithm
|
||||
else:
|
||||
# For RS256 (asymmetric), use public key or JWKS
|
||||
if not (jwks_uri or public_key):
|
||||
# Only a secret is configured but the algorithm is asymmetric: a
|
||||
# keyless verifier cannot validate anything. Name the fix rather
|
||||
# than letting the verifier constructor raise opaquely (it would
|
||||
# still fail closed via the caller's fail-closed exception
|
||||
# handling, but with a less actionable message).
|
||||
raise MCPAuthConfigError(
|
||||
"MCP_JWT_SECRET is set but MCP_JWT_ALGORITHM is not 'HS256' "
|
||||
"and no MCP_JWKS_URI/MCP_JWT_PUBLIC_KEY is configured. Set "
|
||||
"MCP_JWT_ALGORITHM='HS256' to use the secret, or configure "
|
||||
"an asymmetric key."
|
||||
)
|
||||
common_kwargs["jwks_uri"] = jwks_uri
|
||||
common_kwargs["public_key"] = public_key
|
||||
common_kwargs["algorithm"] = app.config.get("MCP_JWT_ALGORITHM", "RS256")
|
||||
common_kwargs["algorithm"] = algorithm
|
||||
|
||||
if debug_errors:
|
||||
# DetailedJWTVerifier: detailed server-side logging of JWT
|
||||
|
||||
@@ -218,18 +218,30 @@ _SENSITIVE_PARAM_KEYS = frozenset(
|
||||
)
|
||||
|
||||
|
||||
def _sanitize_value(value: Any) -> Any:
|
||||
"""Apply ``_sanitize_params`` recursively to any dict/list container."""
|
||||
if isinstance(value, dict):
|
||||
return _sanitize_params(value)
|
||||
if isinstance(value, list):
|
||||
return [_sanitize_value(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def _sanitize_params(params: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Remove sensitive fields from params before logging."""
|
||||
"""Remove sensitive fields from params before logging.
|
||||
|
||||
Recurses into nested containers, including lists of lists, so sensitive
|
||||
keys are redacted no matter which wrapper they arrive under
|
||||
(``arguments``, ``request``, etc.).
|
||||
"""
|
||||
if not isinstance(params, dict):
|
||||
return params
|
||||
result: dict[str, Any] = {}
|
||||
for k, v in params.items():
|
||||
if k.lower() in _SENSITIVE_PARAM_KEYS:
|
||||
result[k] = "[REDACTED]"
|
||||
elif k == "arguments" and isinstance(v, dict):
|
||||
result[k] = _sanitize_params(v)
|
||||
else:
|
||||
result[k] = v
|
||||
result[k] = _sanitize_value(v)
|
||||
return result
|
||||
|
||||
|
||||
@@ -1005,7 +1017,9 @@ class GlobalErrorHandlerMiddleware(Middleware):
|
||||
) from error
|
||||
elif isinstance(error, HTTPException):
|
||||
# HTTP errors from screenshot endpoints or API calls
|
||||
raise ToolError(f"Service error in {tool_name}: {error.detail}") from error
|
||||
raise ToolError(
|
||||
f"Service error in {tool_name}: {_sanitize_error_for_logging(error)}"
|
||||
) from error
|
||||
elif isinstance(error, MCPPermissionDeniedError):
|
||||
# MCP RBAC permission denied — convert to structured ToolError.
|
||||
# Must come before the generic PermissionError branch because
|
||||
@@ -1020,7 +1034,8 @@ class GlobalErrorHandlerMiddleware(Middleware):
|
||||
elif isinstance(error, ValueError):
|
||||
# Value/parameter errors from tool code
|
||||
raise ToolError(
|
||||
f"Invalid parameter in {tool_name}: {str(error)}"
|
||||
f"Invalid parameter in {tool_name}: "
|
||||
f"{_sanitize_error_for_logging(error)}"
|
||||
) from error
|
||||
elif isinstance(error, (ObjectNotFoundError, CommandInvalidError)):
|
||||
# Superset command: not found (404) or validation (422)
|
||||
|
||||
@@ -808,11 +808,19 @@ def _create_auth_provider(flask_app: Any) -> Any | None:
|
||||
when either ``MCP_AUTH_ENABLED`` (JWT auth), ``MCP_API_KEY_ENABLED``, or
|
||||
``FAB_API_KEY_ENABLED`` (API key auth) is True. The default factory builds a
|
||||
``CompositeTokenVerifier`` that handles either or both auth modes.
|
||||
|
||||
Fail-closed: when auth has been explicitly configured, any error while
|
||||
building the provider (or a configured factory yielding no provider)
|
||||
raises ``MCPAuthConfigError`` so the service refuses to start rather
|
||||
than coming up as an unauthenticated server.
|
||||
"""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
auth_provider = None
|
||||
if auth_factory := flask_app.config.get("MCP_AUTH_FACTORY"):
|
||||
from superset.mcp_service.mcp_config import MCPAuthConfigError
|
||||
|
||||
try:
|
||||
auth_provider = auth_factory(flask_app)
|
||||
logger.info(
|
||||
@@ -838,17 +846,18 @@ def _create_auth_provider(flask_app: Any) -> Any | None:
|
||||
"refusing to start the MCP service without authentication. "
|
||||
"Fix the factory or unset MCP_AUTH_FACTORY."
|
||||
) from None
|
||||
if auth_provider is None:
|
||||
raise MCPAuthConfigError(
|
||||
"MCP_AUTH_FACTORY returned no auth provider; refusing to "
|
||||
"start an unauthenticated MCP server. Return a token "
|
||||
"verifier or unset MCP_AUTH_FACTORY."
|
||||
)
|
||||
elif (
|
||||
flask_app.config.get("MCP_AUTH_ENABLED", False)
|
||||
or flask_app.config.get("MCP_API_KEY_ENABLED", False)
|
||||
or flask_app.config.get("FAB_API_KEY_ENABLED", False)
|
||||
or flask_app.config.get("MCP_EMBEDDED_GUEST_AUTH_ENABLED", False)
|
||||
):
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
try:
|
||||
auth_provider = create_default_mcp_auth_factory(flask_app)
|
||||
logger.info(
|
||||
@@ -862,8 +871,19 @@ def _create_auth_provider(flask_app: Any) -> Any | None:
|
||||
# no secret material.
|
||||
raise
|
||||
except Exception:
|
||||
# Do not log the exception — it may contain secrets
|
||||
# Do not log or chain the exception — it may contain secrets.
|
||||
# Auth was explicitly enabled, so a provider that cannot be built
|
||||
# must also fail closed instead of starting unauthenticated.
|
||||
logger.error("Failed to create auth provider from default factory")
|
||||
raise MCPAuthConfigError(
|
||||
"Failed to build the MCP auth provider from the configured "
|
||||
"auth settings; refusing to start an unauthenticated MCP "
|
||||
"server. Check the MCP auth configuration."
|
||||
) from None
|
||||
# ``None`` here is deliberate only when the factory itself resolved
|
||||
# every auth mode to disabled (e.g. MCP_API_KEY_ENABLED=False
|
||||
# explicitly overriding FAB_API_KEY_ENABLED); misconfigurations of an
|
||||
# enabled mode raise MCPAuthConfigError inside the factory instead.
|
||||
return auth_provider
|
||||
|
||||
|
||||
|
||||
@@ -65,7 +65,12 @@ async def _validate_non_destructive_sql(
|
||||
with event_logger.log_context(action="mcp.execute_sql.ddl_check"):
|
||||
try:
|
||||
sql_to_check: str = request.sql
|
||||
if request.template_params:
|
||||
# Render whenever template_params is not None, mirroring the
|
||||
# executor (SQLExecutor._render_sql_template), which also renders
|
||||
# for an empty dict. A truthiness check would let destructive SQL
|
||||
# that only appears after rendering slip past the guard when
|
||||
# template_params={}.
|
||||
if request.template_params is not None:
|
||||
from superset.jinja_context import get_template_processor
|
||||
|
||||
tp = get_template_processor(database=database)
|
||||
|
||||
@@ -24,6 +24,7 @@ system-level info.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import datetime
|
||||
from typing import Annotated, Any, List
|
||||
|
||||
@@ -32,6 +33,12 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
from superset.mcp_service.constants import DEFAULT_PAGE_SIZE, MAX_PAGE_SIZE
|
||||
from superset.subjects.types import SubjectType
|
||||
|
||||
# Shape-only check, not RFC validation: just enough to catch "local@domain.tld"
|
||||
# so an email-shaped query can be rejected before it reaches the username
|
||||
# column, since usernames are frequently email addresses under OAuth
|
||||
# provisioning.
|
||||
_EMAIL_SHAPE_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
|
||||
|
||||
|
||||
class HealthCheckResponse(BaseModel):
|
||||
"""Response model for health check.
|
||||
@@ -176,7 +183,7 @@ def serialize_user_object(user: Any) -> UserInfo | None:
|
||||
class FindUsersRequest(BaseModel):
|
||||
"""Request schema for find_users tool.
|
||||
|
||||
Resolves a person's name (or partial name, username, or email) to user IDs
|
||||
Resolves a person's name (or partial name or username) to user IDs
|
||||
so they can be passed to listing tools as filter values for created_by_fk
|
||||
or changed_by_fk. This is the only sanctioned path for "show me what
|
||||
<person> is working on" queries.
|
||||
@@ -191,8 +198,9 @@ class FindUsersRequest(BaseModel):
|
||||
max_length=200,
|
||||
description=(
|
||||
"Substring to match (case-insensitive) against username, "
|
||||
"first_name, last_name, and email. Required and non-empty: "
|
||||
"this tool does not enumerate the full user directory."
|
||||
"first_name, and last_name (never email; email-shaped "
|
||||
"queries are rejected). Required and non-empty: this tool "
|
||||
"does not enumerate the full user directory."
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -217,6 +225,20 @@ class FindUsersRequest(BaseModel):
|
||||
raise ValueError("query must contain at least one non-whitespace character")
|
||||
return stripped
|
||||
|
||||
@field_validator("query")
|
||||
@classmethod
|
||||
def _reject_email_shaped_query(cls, value: str) -> str:
|
||||
# Email isn't a searchable column here, but usernames are commonly
|
||||
# email addresses under OAuth provisioning, so an email-shaped query
|
||||
# would still confirm an account's existence via the username column.
|
||||
# Reject the shape outright rather than relying on the column
|
||||
# exclusion alone.
|
||||
if _EMAIL_SHAPE_RE.match(value):
|
||||
raise ValueError(
|
||||
"query must not be an email address; search by name or username instead"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
class UserMatch(BaseModel):
|
||||
"""Minimal user projection returned by find_users.
|
||||
|
||||
@@ -50,9 +50,14 @@ async def find_users(request: FindUsersRequest, ctx: Context) -> FindUsersRespon
|
||||
the value for a created_by_fk or changed_by_fk filter on list_dashboards,
|
||||
list_charts, or list_datasets.
|
||||
|
||||
Matches case-insensitively against username, first_name, last_name, and
|
||||
email. The query is required and non-empty; this tool does not enumerate
|
||||
the full user directory.
|
||||
Matches case-insensitively against username, first_name, and last_name.
|
||||
Email is deliberately not matched, and an email-shaped query is rejected
|
||||
outright rather than falling through to the username column: since
|
||||
usernames are frequently email addresses under OAuth provisioning, a
|
||||
plain column exclusion would still let an email lookup confirm whether
|
||||
an address has an account (and resolve it to a person), a directory
|
||||
capability the web API reserves for admins. The query is required and
|
||||
non-empty; this tool does not enumerate the full user directory.
|
||||
|
||||
Privacy: returning a user's identity here is sanctioned only for resolving
|
||||
filter values. Do not use the response to answer "who owns X", "who can
|
||||
@@ -75,7 +80,6 @@ async def find_users(request: FindUsersRequest, ctx: Context) -> FindUsersRespon
|
||||
user_model.username.ilike(needle, escape="\\"),
|
||||
user_model.first_name.ilike(needle, escape="\\"),
|
||||
user_model.last_name.ilike(needle, escape="\\"),
|
||||
user_model.email.ilike(needle, escape="\\"),
|
||||
)
|
||||
)
|
||||
.order_by(user_model.username.asc())
|
||||
|
||||
@@ -54,8 +54,10 @@ async def get_tag_info(request: GetTagInfoRequest, ctx: Context) -> TagInfo | Ta
|
||||
|
||||
Returns tag details including name, type, and description.
|
||||
|
||||
Tag types: custom (user-created), type (implicit by object type),
|
||||
editor (implicit by editorship), favorited_by (implicit by favorites).
|
||||
Tag types: custom (user-created), plus legacy implicit types no longer
|
||||
generated for new objects -- type (by object type), editor (by
|
||||
editorship), favorited_by (by favorites) -- kept for tags created before
|
||||
Superset stopped auto-generating them.
|
||||
|
||||
To find a tag ID, use the list_tags tool first.
|
||||
|
||||
|
||||
@@ -73,8 +73,10 @@ async def list_tags(
|
||||
|
||||
Returns tag metadata including name, type, and description.
|
||||
|
||||
Tag types: custom (user-created), type (implicit by object type),
|
||||
editor (implicit by editorship), favorited_by (implicit by favorites).
|
||||
Tag types: custom (user-created), plus legacy implicit types no longer
|
||||
generated for new objects -- type (by object type), editor (by
|
||||
editorship), favorited_by (by favorites) -- kept for tags created before
|
||||
Superset stopped auto-generating them.
|
||||
|
||||
Sortable columns for order_column: id, name, changed_on, created_on
|
||||
"""
|
||||
|
||||
@@ -191,16 +191,6 @@ class Dashboard(CoreDashboard, SoftDeleteMixin, AuditMixinNullable, ImportExport
|
||||
secondaryjoin="TaggedObject.tag_id == Tag.id",
|
||||
viewonly=True, # cascading deletion already handled by superset.tags.models.ObjectUpdater.after_delete # noqa: E501
|
||||
)
|
||||
custom_tags = relationship(
|
||||
"Tag",
|
||||
overlaps="objects,tag,tags,custom_tags",
|
||||
secondary="tagged_object",
|
||||
primaryjoin="and_(Dashboard.id == TaggedObject.object_id, "
|
||||
"TaggedObject.object_type == 'dashboard')",
|
||||
secondaryjoin="and_(TaggedObject.tag_id == Tag.id, "
|
||||
"cast(Tag.type, String) == 'custom')", # Filtering at JOIN level
|
||||
viewonly=True,
|
||||
)
|
||||
theme = relationship("Theme", foreign_keys=[theme_id])
|
||||
published = Column(Boolean, default=False)
|
||||
is_managed_externally = Column(Boolean, nullable=False, default=False)
|
||||
|
||||
@@ -179,6 +179,21 @@ SERIES_LIMIT_SUBQ_ALIAS = "series_limit"
|
||||
# Offset join column suffix used for joining offset results
|
||||
OFFSET_JOIN_COLUMN_SUFFIX = "__offset_join_column_"
|
||||
|
||||
|
||||
def get_effective_hours_offset(
|
||||
db_engine_spec: type["BaseEngineSpec"],
|
||||
column_type: str | None,
|
||||
offset_hours: int,
|
||||
db_extra: dict[str, Any] | None = None,
|
||||
) -> int:
|
||||
"""Return the dataset offset representable by a temporal column's type."""
|
||||
sqla_type = db_engine_spec.get_sqla_column_type(column_type, db_extra=db_extra)
|
||||
if isinstance(sqla_type, sa.Date):
|
||||
# int() deliberately truncates toward zero; // would turn -1h into -24h.
|
||||
return int(offset_hours / 24) * 24
|
||||
return offset_hours
|
||||
|
||||
|
||||
# Right suffix used for joining offset results
|
||||
R_SUFFIX = "__right_suffix"
|
||||
|
||||
@@ -1543,6 +1558,7 @@ class QueryResult: # pylint: disable=too-few-public-methods
|
||||
errors: Optional[list[dict[str, Any]]] = None,
|
||||
from_dttm: Optional[datetime] = None,
|
||||
to_dttm: Optional[datetime] = None,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> None:
|
||||
self.df = df
|
||||
self.query = query
|
||||
@@ -1555,6 +1571,7 @@ class QueryResult: # pylint: disable=too-few-public-methods
|
||||
self.errors = errors or []
|
||||
self.from_dttm = from_dttm
|
||||
self.to_dttm = to_dttm
|
||||
self.sql_shifted_temporal_labels = sql_shifted_temporal_labels or set()
|
||||
self.sql_rowcount = len(self.df.index) if not self.df.empty else 0
|
||||
|
||||
|
||||
@@ -1654,6 +1671,7 @@ class QueryStringExtended(NamedTuple):
|
||||
labels_expected: list[str]
|
||||
prequeries: list[str]
|
||||
sql: str
|
||||
sql_shifted_temporal_labels: set[str]
|
||||
|
||||
|
||||
class SqlaQuery(NamedTuple):
|
||||
@@ -1665,6 +1683,7 @@ class SqlaQuery(NamedTuple):
|
||||
labels_expected: list[str]
|
||||
prequeries: list[str]
|
||||
sqla_query: Select
|
||||
sql_shifted_temporal_labels: set[str]
|
||||
|
||||
|
||||
class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
@@ -2019,6 +2038,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
labels_expected=sqlaq.labels_expected,
|
||||
prequeries=sqlaq.prequeries,
|
||||
sql=sql,
|
||||
sql_shifted_temporal_labels=sqlaq.sql_shifted_temporal_labels,
|
||||
)
|
||||
|
||||
def _normalize_prequery_result_type(
|
||||
@@ -2223,6 +2243,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
query=sql,
|
||||
errors=errors,
|
||||
error_message=error_message,
|
||||
sql_shifted_temporal_labels=query_str_ext.sql_shifted_temporal_labels,
|
||||
)
|
||||
|
||||
def exc_query(self, qry: Any) -> QueryResult:
|
||||
@@ -2292,6 +2313,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
df: pd.DataFrame,
|
||||
query_object: QueryObject,
|
||||
already_collected: set[str],
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> list[DateColumn]:
|
||||
"""``DateColumn`` entries that only need the dataset HOURS OFFSET (and any
|
||||
time shift) applied, for temporal columns the database already returns as
|
||||
@@ -2311,6 +2333,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
):
|
||||
return []
|
||||
|
||||
sql_shifted_temporal_labels = sql_shifted_temporal_labels or set()
|
||||
extra: list[DateColumn] = []
|
||||
for label in df.columns:
|
||||
if label in already_collected or label == DTTM_ALIAS:
|
||||
@@ -2329,7 +2352,9 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
extra.append(
|
||||
DateColumn(
|
||||
timestamp_format=None,
|
||||
offset=self.offset,
|
||||
offset=(
|
||||
0 if label in sql_shifted_temporal_labels else self.offset
|
||||
),
|
||||
time_shift=query_object.time_shift,
|
||||
col_label=label,
|
||||
)
|
||||
@@ -2337,15 +2362,22 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
already_collected.add(label)
|
||||
return extra
|
||||
|
||||
def normalize_df(self, df: pd.DataFrame, query_object: QueryObject) -> pd.DataFrame:
|
||||
def normalize_df(
|
||||
self,
|
||||
df: pd.DataFrame,
|
||||
query_object: QueryObject,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> pd.DataFrame:
|
||||
"""
|
||||
Normalize the dataframe by converting datetime columns and ensuring
|
||||
numerical metrics.
|
||||
|
||||
:param df: The dataframe to normalize
|
||||
:param query_object: The query object with metadata about columns
|
||||
:param sql_shifted_temporal_labels: labels already shifted in generated SQL
|
||||
:return: Normalized dataframe
|
||||
"""
|
||||
sql_shifted_temporal_labels = sql_shifted_temporal_labels or set()
|
||||
labels = self._collect_dttm_labels(query_object)
|
||||
|
||||
# ``get_dataset_timezone`` lives on ``ExploreMixin``; datasource doubles
|
||||
@@ -2357,7 +2389,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
dttm_cols = [
|
||||
DateColumn(
|
||||
timestamp_format=fmt,
|
||||
offset=self.offset,
|
||||
offset=0 if label in sql_shifted_temporal_labels else self.offset,
|
||||
time_shift=query_object.time_shift,
|
||||
timezone=dataset_timezone,
|
||||
col_label=label,
|
||||
@@ -2369,7 +2401,9 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
dttm_cols.append(
|
||||
DateColumn.get_legacy_time_column(
|
||||
timestamp_format=self._python_date_format(query_object.granularity),
|
||||
offset=self.offset,
|
||||
offset=(
|
||||
0 if DTTM_ALIAS in sql_shifted_temporal_labels else self.offset
|
||||
),
|
||||
time_shift=query_object.time_shift,
|
||||
timezone=dataset_timezone,
|
||||
)
|
||||
@@ -2377,7 +2411,10 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
|
||||
dttm_cols.extend(
|
||||
self._offset_only_dttm_cols(
|
||||
df, query_object, {col.col_label for col in dttm_cols}
|
||||
df,
|
||||
query_object,
|
||||
{col.col_label for col in dttm_cols},
|
||||
sql_shifted_temporal_labels,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -2423,7 +2460,11 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
df = result.df
|
||||
if not df.empty:
|
||||
# Normalize datetime columns and metrics
|
||||
df = self.normalize_df(df, query_object)
|
||||
df = self.normalize_df(
|
||||
df,
|
||||
query_object,
|
||||
result.sql_shifted_temporal_labels,
|
||||
)
|
||||
|
||||
# Process time offsets if requested
|
||||
if query_object.time_offsets:
|
||||
@@ -2733,7 +2774,9 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
else:
|
||||
# 1. normalize df, set dttm column
|
||||
offset_metrics_df = self.normalize_df(
|
||||
offset_metrics_df, query_object_clone
|
||||
offset_metrics_df,
|
||||
query_object_clone,
|
||||
result.sql_shifted_temporal_labels,
|
||||
)
|
||||
|
||||
# 2. rename extra query columns
|
||||
@@ -3745,6 +3788,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
col: AdhocColumn,
|
||||
force_type_check: bool = False,
|
||||
template_processor: Optional[BaseTemplateProcessor] = None,
|
||||
apply_dataset_offset: bool = False,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> tuple[ColumnElement, Optional[GenericDataType]]:
|
||||
raise NotImplementedError()
|
||||
|
||||
@@ -3929,6 +3974,12 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
dataset_timezone = None
|
||||
|
||||
if not dataset_timezone and (offset_hours := getattr(self, "offset", 0) or 0):
|
||||
offset_hours = get_effective_hours_offset(
|
||||
self.db_engine_spec,
|
||||
time_col.type,
|
||||
offset_hours,
|
||||
db_extra=self.db_extra,
|
||||
)
|
||||
if start_dttm is not None:
|
||||
start_dttm = start_dttm - timedelta(hours=offset_hours)
|
||||
if end_dttm is not None:
|
||||
@@ -4298,6 +4349,7 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
template_kwargs["applied_filters"] = applied_template_filters
|
||||
template_processor = self.get_template_processor(**template_kwargs)
|
||||
prequeries: list[str] = []
|
||||
sql_shifted_temporal_labels: set[str] = set()
|
||||
orderby = orderby or []
|
||||
need_groupby = bool(metrics is not None or groupby)
|
||||
metrics = metrics or []
|
||||
@@ -4406,6 +4458,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
col, _unused = self.adhoc_column_to_sqla(
|
||||
col=adhoc_columns_by_label[col],
|
||||
template_processor=template_processor,
|
||||
apply_dataset_offset=True,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
elif col in metrics_by_name:
|
||||
col = metrics_by_name[col].get_sqla_col(
|
||||
@@ -4445,6 +4499,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
time_grain=time_grain,
|
||||
label=selected,
|
||||
template_processor=template_processor,
|
||||
apply_dataset_offset=True,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
# if groupby field equals a selected column
|
||||
elif selected in columns_by_name:
|
||||
@@ -4466,6 +4522,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
outer, _unused = self.adhoc_column_to_sqla(
|
||||
col=selected,
|
||||
template_processor=template_processor,
|
||||
apply_dataset_offset=True,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
groupby_all_columns[outer.name] = outer
|
||||
if (
|
||||
@@ -4506,6 +4564,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
outer, _unused = self.adhoc_column_to_sqla(
|
||||
col=selected,
|
||||
template_processor=template_processor,
|
||||
apply_dataset_offset=True,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
select_exprs.append(outer)
|
||||
continue
|
||||
@@ -4541,7 +4601,10 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
|
||||
if is_timeseries:
|
||||
timestamp = dttm_col.get_timestamp_expression(
|
||||
time_grain=time_grain, template_processor=template_processor
|
||||
time_grain=time_grain,
|
||||
template_processor=template_processor,
|
||||
apply_dataset_offset=True,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
# always put timestamp as the first column
|
||||
select_exprs.insert(0, timestamp)
|
||||
@@ -5350,4 +5413,5 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
|
||||
labels_expected=labels_expected,
|
||||
sqla_query=qry,
|
||||
prequeries=prequeries,
|
||||
sql_shifted_temporal_labels=sql_shifted_temporal_labels,
|
||||
)
|
||||
|
||||
@@ -437,6 +437,8 @@ class Query(
|
||||
col: "AdhocColumn", # type: ignore # noqa: F821
|
||||
force_type_check: bool = False,
|
||||
template_processor: Optional[BaseTemplateProcessor] = None,
|
||||
apply_dataset_offset: bool = False,
|
||||
sql_shifted_temporal_labels: set[str] | None = None,
|
||||
) -> tuple[ColumnElement, Optional[GenericDataType]]:
|
||||
"""
|
||||
Turn an adhoc column into a sqlalchemy column.
|
||||
|
||||
+22
-62
@@ -54,80 +54,40 @@ def _tag_delete_listener_declarations() -> tuple[DeleteListenerDeclaration, ...]
|
||||
|
||||
|
||||
def register_sqla_event_listeners() -> None:
|
||||
"""Register cleanup of ``tagged_object`` rows on object deletion.
|
||||
|
||||
Only deletion is handled here: Superset no longer auto-generates
|
||||
``type:``/``editor:``/``favorited_by:`` tags (see ``TagType``'s docstring),
|
||||
so there's nothing left to do on insert/update. Deletion cleanup stays,
|
||||
since it applies to every tag on the object, custom tags included, and
|
||||
``tagged_object.object_id`` has no foreign key to cascade on its own.
|
||||
"""
|
||||
import sqlalchemy as sqla
|
||||
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import FavStar
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.models.slice import Slice
|
||||
from superset.models.sql_lab import SavedQuery
|
||||
from superset.tags.models import (
|
||||
ChartUpdater,
|
||||
DashboardUpdater,
|
||||
DatasetUpdater,
|
||||
FavStarUpdater,
|
||||
QueryUpdater,
|
||||
)
|
||||
from superset.tags.models import QueryUpdater
|
||||
|
||||
declarations: tuple[DeleteListenerDeclaration, ...] = (
|
||||
_tag_delete_listener_declarations()
|
||||
)
|
||||
declarations = _tag_delete_listener_declarations()
|
||||
|
||||
sqla.event.listen(SqlaTable, "after_insert", DatasetUpdater.after_insert)
|
||||
sqla.event.listen(SqlaTable, "after_update", DatasetUpdater.after_update)
|
||||
register_delete_listener(declarations[0])
|
||||
register_delete_listener(declarations[0]) # dataset
|
||||
register_delete_listener(declarations[1]) # chart
|
||||
register_delete_listener(declarations[2]) # dashboard
|
||||
|
||||
sqla.event.listen(Slice, "after_insert", ChartUpdater.after_insert)
|
||||
sqla.event.listen(Slice, "after_update", ChartUpdater.after_update)
|
||||
register_delete_listener(declarations[1])
|
||||
|
||||
sqla.event.listen(Dashboard, "after_insert", DashboardUpdater.after_insert)
|
||||
sqla.event.listen(Dashboard, "after_update", DashboardUpdater.after_update)
|
||||
register_delete_listener(declarations[2])
|
||||
|
||||
sqla.event.listen(FavStar, "after_insert", FavStarUpdater.after_insert)
|
||||
sqla.event.listen(FavStar, "after_delete", FavStarUpdater.after_delete)
|
||||
|
||||
sqla.event.listen(SavedQuery, "after_insert", QueryUpdater.after_insert)
|
||||
sqla.event.listen(SavedQuery, "after_update", QueryUpdater.after_update)
|
||||
sqla.event.listen(SavedQuery, "after_delete", QueryUpdater.after_delete)
|
||||
if not sqla.event.contains(SavedQuery, "after_delete", QueryUpdater.after_delete):
|
||||
sqla.event.listen(SavedQuery, "after_delete", QueryUpdater.after_delete)
|
||||
|
||||
|
||||
def clear_sqla_event_listeners() -> None:
|
||||
import sqlalchemy as sqla
|
||||
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import FavStar
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.models.slice import Slice
|
||||
from superset.models.sql_lab import SavedQuery
|
||||
from superset.tags.models import (
|
||||
ChartUpdater,
|
||||
DashboardUpdater,
|
||||
DatasetUpdater,
|
||||
FavStarUpdater,
|
||||
QueryUpdater,
|
||||
)
|
||||
from superset.tags.models import QueryUpdater
|
||||
|
||||
declarations: tuple[DeleteListenerDeclaration, ...] = (
|
||||
_tag_delete_listener_declarations()
|
||||
)
|
||||
declarations = _tag_delete_listener_declarations()
|
||||
|
||||
sqla.event.remove(SqlaTable, "after_insert", DatasetUpdater.after_insert)
|
||||
sqla.event.remove(SqlaTable, "after_update", DatasetUpdater.after_update)
|
||||
remove_delete_listener(declarations[0])
|
||||
remove_delete_listener(declarations[0]) # dataset
|
||||
remove_delete_listener(declarations[1]) # chart
|
||||
remove_delete_listener(declarations[2]) # dashboard
|
||||
|
||||
sqla.event.remove(Slice, "after_insert", ChartUpdater.after_insert)
|
||||
sqla.event.remove(Slice, "after_update", ChartUpdater.after_update)
|
||||
remove_delete_listener(declarations[1])
|
||||
|
||||
sqla.event.remove(Dashboard, "after_insert", DashboardUpdater.after_insert)
|
||||
sqla.event.remove(Dashboard, "after_update", DashboardUpdater.after_update)
|
||||
remove_delete_listener(declarations[2])
|
||||
|
||||
sqla.event.remove(FavStar, "after_insert", FavStarUpdater.after_insert)
|
||||
sqla.event.remove(FavStar, "after_delete", FavStarUpdater.after_delete)
|
||||
|
||||
sqla.event.remove(SavedQuery, "after_insert", QueryUpdater.after_insert)
|
||||
sqla.event.remove(SavedQuery, "after_update", QueryUpdater.after_update)
|
||||
sqla.event.remove(SavedQuery, "after_delete", QueryUpdater.after_delete)
|
||||
if sqla.event.contains(SavedQuery, "after_delete", QueryUpdater.after_delete):
|
||||
sqla.event.remove(SavedQuery, "after_delete", QueryUpdater.after_delete)
|
||||
|
||||
+21
-208
@@ -21,17 +21,7 @@ import enum
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from flask_appbuilder import Model
|
||||
from sqlalchemy import (
|
||||
Column,
|
||||
Enum,
|
||||
exists,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
orm,
|
||||
String,
|
||||
Table,
|
||||
Text,
|
||||
)
|
||||
from sqlalchemy import Column, Enum, ForeignKey, Integer, orm, String, Table, Text
|
||||
from sqlalchemy.engine.base import Connection
|
||||
from sqlalchemy.orm import relationship, sessionmaker
|
||||
from sqlalchemy.orm.mapper import Mapper
|
||||
@@ -40,11 +30,9 @@ from superset_core.common.models import Tag as CoreTag
|
||||
|
||||
from superset import security_manager
|
||||
from superset.models.helpers import AuditMixinNullable
|
||||
from superset.subjects.types import SubjectType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import FavStar
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.models.slice import Slice
|
||||
from superset.models.sql_lab import Query
|
||||
@@ -63,16 +51,21 @@ class TagType(enum.Enum):
|
||||
"""
|
||||
Types for tags.
|
||||
|
||||
Objects (queries, charts, dashboards, and datasets) will have with implicit tags based
|
||||
on metadata: types, editors and who favorited them. This way, user "alice"
|
||||
can find all their objects by querying for the tag `editor:alice`.
|
||||
""" # noqa: E501
|
||||
``type``, ``editor``, and ``favorited_by`` are no longer generated: Superset
|
||||
used to auto-tag every query, chart, dashboard, and dataset with implicit
|
||||
tags based on metadata (object type, editors, and who favorited them), but
|
||||
nothing ever surfaced them to the user, so the generation was removed. The
|
||||
values are kept, and rows of these types are still recognized (e.g. exempt
|
||||
from bulk deletion) and filterable via the API, so upgraded deployments that
|
||||
already have such tags, or MCP tooling that queries by tag type, keep
|
||||
working.
|
||||
"""
|
||||
|
||||
# pylint: disable=invalid-name
|
||||
# explicit tags, added manually by the owner
|
||||
custom = 1
|
||||
|
||||
# implicit tags, generated automatically
|
||||
# legacy implicit tag types; no longer generated (see docstring above)
|
||||
type = 2
|
||||
editor = 3
|
||||
favorited_by = 4
|
||||
@@ -155,145 +148,26 @@ def get_tag(
|
||||
return tag
|
||||
|
||||
|
||||
def get_object_type(class_name: str) -> ObjectType:
|
||||
mapping = {
|
||||
"slice": ObjectType.chart,
|
||||
"dashboard": ObjectType.dashboard,
|
||||
"query": ObjectType.query,
|
||||
"dataset": ObjectType.dataset,
|
||||
}
|
||||
try:
|
||||
return mapping[class_name.lower()]
|
||||
except KeyError as ex:
|
||||
raise Exception( # pylint: disable=broad-exception-raised
|
||||
f"No mapping found for {class_name}"
|
||||
) from ex
|
||||
|
||||
|
||||
class ObjectUpdater:
|
||||
"""Cleans up ``tagged_object`` rows when a tagged object is deleted.
|
||||
|
||||
``TaggedObject.object_id`` is a polymorphic reference with no foreign key
|
||||
(see the comment on that column), so nothing at the database level removes
|
||||
a tag association when the dashboard/chart/query/dataset it points at is
|
||||
deleted. This listener does that cleanup for every tag on the object
|
||||
(custom tags included), independent of how the tag was created.
|
||||
"""
|
||||
|
||||
object_type: str = "default"
|
||||
|
||||
@classmethod
|
||||
def get_editor_user_ids(
|
||||
cls, target: Dashboard | FavStar | Slice | Query | SqlaTable
|
||||
) -> list[int]:
|
||||
raise NotImplementedError("Subclass should implement `get_editor_user_ids`")
|
||||
|
||||
@classmethod
|
||||
def get_editor_tag_ids(
|
||||
cls,
|
||||
session: orm.Session, # pylint: disable=disallowed-name
|
||||
target: Dashboard | FavStar | Slice | Query | SqlaTable,
|
||||
) -> set[int]:
|
||||
tag_ids = set()
|
||||
for user_id in cls.get_editor_user_ids(target):
|
||||
name = f"editor:{user_id}"
|
||||
tag = get_tag(name, session, TagType.editor)
|
||||
tag_ids.add(tag.id)
|
||||
return tag_ids
|
||||
|
||||
@classmethod
|
||||
def _add_editors(
|
||||
cls,
|
||||
session: orm.Session, # pylint: disable=disallowed-name
|
||||
target: Dashboard | FavStar | Slice | Query | SqlaTable,
|
||||
) -> None:
|
||||
for user_id in cls.get_editor_user_ids(target):
|
||||
name: str = f"editor:{user_id}"
|
||||
tag = get_tag(name, session, TagType.editor)
|
||||
cls.add_tag_object_if_not_tagged(
|
||||
session, tag_id=tag.id, object_id=target.id, object_type=cls.object_type
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def add_tag_object_if_not_tagged(
|
||||
cls,
|
||||
session: orm.Session, # pylint: disable=disallowed-name
|
||||
tag_id: int,
|
||||
object_id: int,
|
||||
object_type: str,
|
||||
) -> None:
|
||||
# Check if the object is already tagged
|
||||
exists_query = exists().where(
|
||||
TaggedObject.tag_id == tag_id,
|
||||
TaggedObject.object_id == object_id,
|
||||
TaggedObject.object_type == object_type,
|
||||
)
|
||||
already_tagged = session.query(exists_query).scalar()
|
||||
|
||||
# Add TaggedObject to the session if it isn't already tagged
|
||||
if not already_tagged:
|
||||
tagged_object = TaggedObject(
|
||||
tag_id=tag_id, object_id=object_id, object_type=object_type
|
||||
)
|
||||
session.add(tagged_object)
|
||||
|
||||
@classmethod
|
||||
def after_insert(
|
||||
cls,
|
||||
_mapper: Mapper,
|
||||
connection: Connection,
|
||||
target: Dashboard | FavStar | Slice | Query | SqlaTable,
|
||||
) -> None:
|
||||
with Session(bind=connection) as session: # pylint: disable=disallowed-name
|
||||
# add `editor:` tags
|
||||
cls._add_editors(session, target)
|
||||
|
||||
# add `type:` tags
|
||||
tag = get_tag(f"type:{cls.object_type}", session, TagType.type)
|
||||
cls.add_tag_object_if_not_tagged(
|
||||
session, tag_id=tag.id, object_id=target.id, object_type=cls.object_type
|
||||
)
|
||||
session.commit()
|
||||
|
||||
@classmethod
|
||||
def after_update(
|
||||
cls,
|
||||
_mapper: Mapper,
|
||||
connection: Connection,
|
||||
target: Dashboard | FavStar | Slice | Query | SqlaTable,
|
||||
) -> None:
|
||||
with Session(bind=connection) as session: # pylint: disable=disallowed-name
|
||||
# Fetch current editor tags
|
||||
existing_tags = (
|
||||
session.query(TaggedObject)
|
||||
.join(Tag)
|
||||
.filter(
|
||||
TaggedObject.object_type == cls.object_type,
|
||||
TaggedObject.object_id == target.id,
|
||||
Tag.type == TagType.editor,
|
||||
)
|
||||
.all()
|
||||
)
|
||||
existing_editor_tag_ids = {tag.tag_id for tag in existing_tags}
|
||||
|
||||
# Determine new editor IDs
|
||||
new_editor_tag_ids = cls.get_editor_tag_ids(session, target)
|
||||
|
||||
# Add missing tags
|
||||
for editor_tag_id in new_editor_tag_ids - existing_editor_tag_ids:
|
||||
tagged_object = TaggedObject(
|
||||
tag_id=editor_tag_id,
|
||||
object_id=target.id,
|
||||
object_type=cls.object_type,
|
||||
)
|
||||
session.add(tagged_object)
|
||||
|
||||
# Remove unnecessary tags
|
||||
for tag in existing_tags:
|
||||
if tag.tag_id not in new_editor_tag_ids:
|
||||
session.delete(tag)
|
||||
session.commit()
|
||||
|
||||
@classmethod
|
||||
def after_delete(
|
||||
cls,
|
||||
_mapper: Mapper,
|
||||
connection: Connection,
|
||||
target: Dashboard | FavStar | Slice | Query | SqlaTable,
|
||||
target: Dashboard | Slice | Query | SqlaTable,
|
||||
) -> None:
|
||||
with Session(bind=connection) as session: # pylint: disable=disallowed-name
|
||||
# delete row from `tagged_objects`
|
||||
session.query(TaggedObject).filter(
|
||||
TaggedObject.object_type == cls.object_type,
|
||||
TaggedObject.object_id == target.id,
|
||||
@@ -305,75 +179,14 @@ class ObjectUpdater:
|
||||
class ChartUpdater(ObjectUpdater):
|
||||
object_type = "chart"
|
||||
|
||||
@classmethod
|
||||
def get_editor_user_ids(cls, target: Slice) -> list[int]:
|
||||
return [
|
||||
s.user.id for s in target.editors if s.type == SubjectType.USER and s.user
|
||||
]
|
||||
|
||||
|
||||
class DashboardUpdater(ObjectUpdater):
|
||||
object_type = "dashboard"
|
||||
|
||||
@classmethod
|
||||
def get_editor_user_ids(cls, target: Dashboard) -> list[int]:
|
||||
return [
|
||||
s.user.id for s in target.editors if s.type == SubjectType.USER and s.user
|
||||
]
|
||||
|
||||
|
||||
class QueryUpdater(ObjectUpdater):
|
||||
object_type = "query"
|
||||
|
||||
@classmethod
|
||||
def get_editor_user_ids(cls, target: Query) -> list[int]:
|
||||
return [target.user_id]
|
||||
|
||||
|
||||
class DatasetUpdater(ObjectUpdater):
|
||||
object_type = "dataset"
|
||||
|
||||
@classmethod
|
||||
def get_editor_user_ids(cls, target: SqlaTable) -> list[int]:
|
||||
return [
|
||||
s.user.id for s in target.editors if s.type == SubjectType.USER and s.user
|
||||
]
|
||||
|
||||
|
||||
class FavStarUpdater:
|
||||
@classmethod
|
||||
def after_insert(
|
||||
cls, _mapper: Mapper, connection: Connection, target: FavStar
|
||||
) -> None:
|
||||
with Session(bind=connection) as session: # pylint: disable=disallowed-name
|
||||
name = f"favorited_by:{target.user_id}"
|
||||
tag = get_tag(name, session, TagType.favorited_by)
|
||||
tagged_object = TaggedObject(
|
||||
tag_id=tag.id,
|
||||
object_id=target.obj_id,
|
||||
object_type=get_object_type(target.class_name),
|
||||
)
|
||||
session.add(tagged_object)
|
||||
session.commit()
|
||||
|
||||
@classmethod
|
||||
def after_delete(
|
||||
cls, _mapper: Mapper, connection: Connection, target: FavStar
|
||||
) -> None:
|
||||
with Session(bind=connection) as session: # pylint: disable=disallowed-name
|
||||
name = f"favorited_by:{target.user_id}"
|
||||
query = (
|
||||
session.query(TaggedObject.id)
|
||||
.join(Tag)
|
||||
.filter(
|
||||
TaggedObject.object_id == target.obj_id,
|
||||
Tag.type == TagType.favorited_by,
|
||||
Tag.name == name,
|
||||
)
|
||||
)
|
||||
ids = [row[0] for row in query]
|
||||
session.query(TaggedObject).filter(TaggedObject.id.in_(ids)).delete(
|
||||
synchronize_session=False
|
||||
)
|
||||
|
||||
session.commit()
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
"""Mixin for APIs that need custom_tags optimization with frontend compatibility."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from flask import current_app, request, Response
|
||||
from werkzeug.datastructures import ImmutableMultiDict
|
||||
|
||||
|
||||
class CustomTagsOptimizationMixin:
|
||||
"""Reusable mixin for APIs that optimize tag queries via custom_tags relationship.
|
||||
|
||||
When enabled via config, this mixin:
|
||||
1. Configures list_columns to use custom_tags (filtered relationship)
|
||||
2. Exposes custom_tags as tags in the response schema
|
||||
3. Rewrites frontend requests from 'tags.*' to 'custom_tags.*'
|
||||
4. Transforms responses to rename 'custom_tags' back to 'tags'
|
||||
|
||||
This provides SQL query optimization (97% reduction) while maintaining
|
||||
frontend compatibility.
|
||||
|
||||
Usage:
|
||||
class MyRestApi(CustomTagsOptimizationMixin, BaseSupersetModelRestApi):
|
||||
def __init__(self):
|
||||
self._setup_custom_tags_optimization(
|
||||
config_key="MY_API_CUSTOM_TAGS_ONLY",
|
||||
full_columns=FULL_TAG_COLUMNS,
|
||||
custom_columns=CUSTOM_TAG_COLUMNS,
|
||||
)
|
||||
super().__init__()
|
||||
"""
|
||||
|
||||
_custom_tags_only: bool
|
||||
|
||||
def _setup_custom_tags_optimization(
|
||||
self,
|
||||
config_key: str,
|
||||
full_columns: list[str],
|
||||
custom_columns: list[str],
|
||||
) -> None:
|
||||
"""Configure custom tags optimization based on config.
|
||||
|
||||
Args:
|
||||
config_key: Config key to check (e.g., "DASHBOARD_LIST_CUSTOM_TAGS_ONLY")
|
||||
full_columns: list_columns when optimization disabled (includes all tags)
|
||||
custom_columns: list_columns when optimization enabled (only custom_tags)
|
||||
"""
|
||||
self._custom_tags_only = current_app.config.get(config_key, False)
|
||||
self.list_columns = custom_columns if self._custom_tags_only else full_columns
|
||||
|
||||
def _init_model_schemas(self) -> None:
|
||||
"""Keep the optimized relationship's public schema name stable."""
|
||||
super()._init_model_schemas() # type: ignore[misc]
|
||||
|
||||
list_model_schema = getattr(self, "list_model_schema", None)
|
||||
if (
|
||||
self._custom_tags_only
|
||||
and list_model_schema
|
||||
and "custom_tags" in list_model_schema.fields
|
||||
):
|
||||
list_model_schema.fields["custom_tags"].data_key = "tags"
|
||||
|
||||
def get_list(self, **kwargs: Any) -> Response:
|
||||
"""Override to rewrite request parameters for custom_tags optimization.
|
||||
|
||||
When config is enabled, rewrites 'tags.*' → 'custom_tags.*' in request
|
||||
so FAB can find the columns in list_columns.
|
||||
"""
|
||||
if self._custom_tags_only:
|
||||
# Parse and rewrite query parameter
|
||||
query_str = request.args.get("q", "")
|
||||
if query_str and "tags." in query_str:
|
||||
# Replace 'tags.' with 'custom_tags.' in select_columns
|
||||
modified_query = query_str.replace("tags.id", "custom_tags.id")
|
||||
modified_query = modified_query.replace("tags.name", "custom_tags.name")
|
||||
modified_query = modified_query.replace("tags.type", "custom_tags.type")
|
||||
|
||||
# Temporarily patch request.args
|
||||
modified_args = request.args.copy()
|
||||
modified_args["q"] = modified_query
|
||||
original_args = request.args
|
||||
request.args = ImmutableMultiDict(modified_args)
|
||||
|
||||
try:
|
||||
return super().get_list(**kwargs) # type: ignore
|
||||
finally:
|
||||
# Restore original args
|
||||
request.args = original_args
|
||||
|
||||
return super().get_list(**kwargs) # type: ignore
|
||||
|
||||
def pre_get_list(self, data: dict[str, Any]) -> None:
|
||||
"""Rename custom_tags → tags in response for frontend compatibility.
|
||||
|
||||
Called by FAB before sending the list response. This ensures the frontend
|
||||
always receives 'tags' regardless of backend optimization config.
|
||||
"""
|
||||
if self._custom_tags_only and "result" in data:
|
||||
for item in data["result"]:
|
||||
if "custom_tags" in item:
|
||||
item["tags"] = item.pop("custom_tags")
|
||||
|
||||
super().pre_get_list(data) # type: ignore
|
||||
@@ -4623,107 +4623,3 @@ class TestDashboardApi(ApiEditorsTestCaseMixin, InsertChartMixin, SupersetTestCa
|
||||
# Cleanup
|
||||
db.session.delete(dashboard)
|
||||
db.session.commit()
|
||||
|
||||
|
||||
class TestDashboardCustomTagsFiltering(SupersetTestCase):
|
||||
"""Test dashboard list API tags field behavior.
|
||||
|
||||
Note: DASHBOARD_LIST_CUSTOM_TAGS_ONLY config is checked at app startup in
|
||||
DashboardRestApi.__init__(), so these tests verify the current runtime behavior.
|
||||
"""
|
||||
|
||||
def setUp(self) -> None:
|
||||
"""Set up test fixtures."""
|
||||
self.login(username="admin")
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_dashboard_custom_tags_relationship_filters_correctly(self):
|
||||
"""Verify custom_tags filtering at model and API level.
|
||||
|
||||
With DASHBOARD_LIST_CUSTOM_TAGS_ONLY=True in superset_test_config.py:
|
||||
1. dashboard.tags returns ALL tags (custom + editor + type)
|
||||
2. dashboard.custom_tags returns ONLY custom tags
|
||||
3. API response returns ONLY custom tags in the "tags" property
|
||||
"""
|
||||
dashboard = Dashboard(
|
||||
dashboard_title="test-custom-only",
|
||||
slug="test-slug-custom",
|
||||
editors=subjects_from_users([self.get_user("admin")]),
|
||||
)
|
||||
db.session.add(dashboard)
|
||||
db.session.flush()
|
||||
|
||||
custom_tag = Tag(name="critical", type=TagType.custom)
|
||||
db.session.add(custom_tag)
|
||||
db.session.flush()
|
||||
|
||||
tagged_obj = TaggedObject(
|
||||
tag_id=custom_tag.id,
|
||||
object_id=dashboard.id,
|
||||
object_type="dashboard",
|
||||
)
|
||||
db.session.add(tagged_obj)
|
||||
db.session.commit()
|
||||
|
||||
try:
|
||||
# 1. MODEL: dashboard.tags returns ALL tags
|
||||
all_tags = dashboard.tags
|
||||
all_tag_names = [t.name for t in all_tags]
|
||||
assert "critical" in all_tag_names, "Should include custom tag"
|
||||
assert any(t.name.startswith("editor:") for t in all_tags), (
|
||||
"Should include editor tags"
|
||||
)
|
||||
assert any(t.name.startswith("type:") for t in all_tags), (
|
||||
"Should include type tags"
|
||||
)
|
||||
|
||||
# 2. MODEL: dashboard.custom_tags returns ONLY custom tags
|
||||
custom_only = dashboard.custom_tags
|
||||
custom_tag_names = [t.name for t in custom_only]
|
||||
assert "critical" in custom_tag_names, "Should include custom tag"
|
||||
assert not any(t.name.startswith("editor:") for t in custom_only), (
|
||||
f"custom_tags should NOT include editor tags, got: {custom_tag_names}"
|
||||
)
|
||||
assert not any(t.name.startswith("type:") for t in custom_only), (
|
||||
f"custom_tags should NOT include type tags, got: {custom_tag_names}"
|
||||
)
|
||||
assert len(custom_only) < len(all_tags), "Should filter out implicit tags"
|
||||
|
||||
# Verify all tags in custom_tags have type=custom
|
||||
for tag in custom_only:
|
||||
assert tag.type == TagType.custom, (
|
||||
f"Tag {tag.name} has type {tag.type}, expected TagType.custom"
|
||||
)
|
||||
|
||||
# 3. API: With config=True, API returns ONLY custom tags
|
||||
rv = self.client.get("api/v1/dashboard/")
|
||||
data = json.loads(rv.data.decode("utf-8"))
|
||||
|
||||
assert rv.status_code == 200
|
||||
test_dash = next(
|
||||
(d for d in data["result"] if d["id"] == dashboard.id), None
|
||||
)
|
||||
assert test_dash is not None
|
||||
# API returns "tags" (get_list override renames custom_tags→tags)
|
||||
assert "tags" in test_dash, (
|
||||
f"Response should have tags, got: {test_dash.keys()}"
|
||||
)
|
||||
|
||||
# API should return ONLY custom tags
|
||||
api_tag_names = [t["name"] for t in test_dash["tags"]]
|
||||
assert "critical" in api_tag_names, "API should include custom tag"
|
||||
assert not any(
|
||||
t["name"].startswith("editor:") for t in test_dash["tags"]
|
||||
), f"API should NOT include editor tags, got: {api_tag_names}"
|
||||
assert not any(t["name"].startswith("type:") for t in test_dash["tags"]), (
|
||||
f"API should NOT include type tags, got: {api_tag_names}"
|
||||
)
|
||||
assert len(test_dash["tags"]) == 1, (
|
||||
f"API should return only 1 custom tag, "
|
||||
f"got {len(test_dash['tags'])}: {api_tag_names}"
|
||||
)
|
||||
finally:
|
||||
db.session.delete(dashboard)
|
||||
db.session.commit()
|
||||
db.session.delete(custom_tag)
|
||||
db.session.commit()
|
||||
|
||||
@@ -183,7 +183,4 @@ CUSTOM_TEMPLATE_PROCESSORS = {
|
||||
|
||||
PRESERVE_CONTEXT_ON_EXCEPTION = False
|
||||
|
||||
# Dashboard API: Return only custom tags (performance optimization)
|
||||
DASHBOARD_LIST_CUSTOM_TAGS_ONLY = True
|
||||
|
||||
print("Loaded TEST config for INTEGRATION tests")
|
||||
|
||||
@@ -1,294 +0,0 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
|
||||
import pytest
|
||||
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.extensions import db
|
||||
from superset.models.core import FavStar
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.models.slice import Slice
|
||||
from superset.models.sql_lab import SavedQuery
|
||||
from superset.tags.models import TaggedObject
|
||||
from superset.utils.core import DatasourceType
|
||||
from superset.utils.database import get_main_database
|
||||
from tests.integration_tests.base_tests import SupersetTestCase
|
||||
from tests.integration_tests.conftest import with_feature_flags
|
||||
from tests.integration_tests.fixtures.tags import (
|
||||
with_tagging_system_feature, # noqa: F401
|
||||
)
|
||||
|
||||
|
||||
class TestTagging(SupersetTestCase):
|
||||
def query_tagged_object_table(self):
|
||||
query = db.session.query(TaggedObject).all()
|
||||
return query
|
||||
|
||||
def clear_tagged_object_table(self):
|
||||
db.session.query(TaggedObject).delete()
|
||||
db.session.commit()
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_dataset_tagging(self):
|
||||
"""
|
||||
Test to make sure that when a new dataset is created,
|
||||
a corresponding tag in the tagged_objects table
|
||||
is created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a dataset and add it to the db
|
||||
test_dataset = SqlaTable(
|
||||
table_name="foo",
|
||||
schema=None,
|
||||
editors=[],
|
||||
database=get_main_database(),
|
||||
sql=None,
|
||||
extra='{"certification": 1}',
|
||||
)
|
||||
db.session.add(test_dataset)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that a dataset tag was added to the tagged_object table
|
||||
tags = self.query_tagged_object_table()
|
||||
assert 1 == len(tags)
|
||||
assert "ObjectType.dataset" == str(tags[0].object_type)
|
||||
assert test_dataset.id == tags[0].object_id
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_dataset)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure the tag is deleted when the associated object is deleted
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_chart_tagging(self):
|
||||
"""
|
||||
Test to make sure that when a new chart is created,
|
||||
a corresponding tag in the tagged_objects table
|
||||
is created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a chart and add it to the db
|
||||
test_chart = Slice(
|
||||
slice_name="test_chart",
|
||||
datasource_type=DatasourceType.TABLE,
|
||||
viz_type="bubble",
|
||||
datasource_id=1,
|
||||
)
|
||||
db.session.add(test_chart)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that a chart tag was added to the tagged_object table
|
||||
tags = self.query_tagged_object_table()
|
||||
assert 1 == len(tags)
|
||||
assert "ObjectType.chart" == str(tags[0].object_type)
|
||||
assert test_chart.id == tags[0].object_id
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_chart)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure the tag is deleted when the associated object is deleted
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_dashboard_tagging(self):
|
||||
"""
|
||||
Test to make sure that when a new dashboard is created,
|
||||
a corresponding tag in the tagged_objects table
|
||||
is created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a dashboard and add it to the db
|
||||
test_dashboard = Dashboard()
|
||||
test_dashboard.dashboard_title = "test_dashboard"
|
||||
test_dashboard.slug = "test_slug"
|
||||
test_dashboard.published = True
|
||||
|
||||
db.session.add(test_dashboard)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that a dashboard tag was added to the tagged_object table
|
||||
tags = self.query_tagged_object_table()
|
||||
assert 1 == len(tags)
|
||||
assert "ObjectType.dashboard" == str(tags[0].object_type)
|
||||
assert test_dashboard.id == tags[0].object_id
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_dashboard)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure the tag is deleted when the associated object is deleted
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_saved_query_tagging(self):
|
||||
"""
|
||||
Test to make sure that when a new saved query is
|
||||
created, a corresponding tag in the tagged_objects
|
||||
table is created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a saved query and add it to the db
|
||||
test_saved_query = SavedQuery(label="test saved query")
|
||||
db.session.add(test_saved_query)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that a saved query tag was added to the tagged_object table
|
||||
tags = self.query_tagged_object_table()
|
||||
|
||||
assert 2 == len(tags)
|
||||
|
||||
assert "ObjectType.query" == str(tags[0].object_type)
|
||||
assert "editor:None" == str(tags[0].tag.name)
|
||||
assert "TagType.editor" == str(tags[0].tag.type)
|
||||
assert test_saved_query.id == tags[0].object_id
|
||||
|
||||
assert "ObjectType.query" == str(tags[1].object_type)
|
||||
assert "type:query" == str(tags[1].tag.name)
|
||||
assert "TagType.type" == str(tags[1].tag.type)
|
||||
assert test_saved_query.id == tags[1].object_id
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_saved_query)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure the tag is deleted when the associated object is deleted
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_favorite_tagging(self):
|
||||
"""
|
||||
Test to make sure that when a new favorite object is
|
||||
created, a corresponding tag in the tagged_objects
|
||||
table is created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a favorited object and add it to the db
|
||||
test_saved_query = FavStar(user_id=1, class_name="slice", obj_id=1)
|
||||
db.session.add(test_saved_query)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that a favorited object tag was added to the tagged_object table # noqa: E501
|
||||
tags = self.query_tagged_object_table()
|
||||
assert 1 == len(tags)
|
||||
assert "ObjectType.chart" == str(tags[0].object_type)
|
||||
assert test_saved_query.obj_id == tags[0].object_id
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_saved_query)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure the tag is deleted when the associated object is deleted
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
@with_feature_flags(TAGGING_SYSTEM=False)
|
||||
def test_tagging_system(self):
|
||||
"""
|
||||
Test to make sure that when the TAGGING_SYSTEM
|
||||
feature flag is false, that no tags are created
|
||||
"""
|
||||
|
||||
# Remove all existing rows in the tagged_object table
|
||||
self.clear_tagged_object_table()
|
||||
|
||||
# Test to make sure nothing is in the tagged_object table
|
||||
assert [] == self.query_tagged_object_table()
|
||||
|
||||
# Create a dataset and add it to the db
|
||||
test_dataset = SqlaTable(
|
||||
table_name="foo",
|
||||
schema=None,
|
||||
editors=[],
|
||||
database=get_main_database(),
|
||||
sql=None,
|
||||
extra='{"certification": 1}',
|
||||
)
|
||||
|
||||
# Create a chart and add it to the db
|
||||
test_chart = Slice(
|
||||
slice_name="test_chart",
|
||||
datasource_type=DatasourceType.TABLE,
|
||||
viz_type="bubble",
|
||||
datasource_id=1,
|
||||
)
|
||||
|
||||
# Create a dashboard and add it to the db
|
||||
test_dashboard = Dashboard()
|
||||
test_dashboard.dashboard_title = "test_dashboard"
|
||||
test_dashboard.slug = "test_slug"
|
||||
test_dashboard.published = True
|
||||
|
||||
# Create a saved query and add it to the db
|
||||
test_saved_query = SavedQuery(label="test saved query")
|
||||
|
||||
# Create a favorited object and add it to the db
|
||||
test_favorited_object = FavStar(user_id=1, class_name="slice", obj_id=1)
|
||||
|
||||
db.session.add(test_dataset)
|
||||
db.session.add(test_chart)
|
||||
db.session.add(test_dashboard)
|
||||
db.session.add(test_saved_query)
|
||||
db.session.add(test_favorited_object)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure that no tags were added to the tagged_object table
|
||||
tags = self.query_tagged_object_table()
|
||||
assert 0 == len(tags)
|
||||
|
||||
# Cleanup the db
|
||||
db.session.delete(test_dataset)
|
||||
db.session.delete(test_chart)
|
||||
db.session.delete(test_dashboard)
|
||||
db.session.delete(test_saved_query)
|
||||
db.session.delete(test_favorited_object)
|
||||
db.session.commit()
|
||||
|
||||
# Test to make sure all the tags are deleted when the associated objects are deleted # noqa: E501
|
||||
assert [] == self.query_tagged_object_table()
|
||||
@@ -22,7 +22,7 @@ from superset import db
|
||||
from superset.daos.tag import TagDAO
|
||||
from superset.models.dashboard import Dashboard
|
||||
from superset.models.slice import Slice
|
||||
from superset.tags.models import ObjectType, Tag, TaggedObject
|
||||
from superset.tags.models import ObjectType, Tag, TaggedObject, TagType
|
||||
from tests.integration_tests.base_tests import SupersetTestCase
|
||||
from tests.integration_tests.constants import ADMIN_USERNAME
|
||||
from tests.integration_tests.fixtures.tags import (
|
||||
@@ -336,3 +336,47 @@ class TestTagsDAO(SupersetTestCase):
|
||||
.first()
|
||||
)
|
||||
assert tagged_object is None
|
||||
|
||||
@pytest.mark.usefixtures("with_tagging_system_feature")
|
||||
def test_tagged_object_cleanup_on_dashboard_delete(self):
|
||||
"""Deleting a tagged dashboard cleans up its tagged_object rows.
|
||||
|
||||
Regression guard for ObjectUpdater.after_delete, the one part of the
|
||||
old auto-tagging event-listener machinery still registered: unlike
|
||||
editor:/type:/favorited_by: generation (removed, see TagType's
|
||||
docstring), this cleanup applies to every tag on the object -- custom
|
||||
tags included -- and nothing else removes these rows, since
|
||||
TaggedObject.object_id carries no foreign key (see its column
|
||||
comment).
|
||||
"""
|
||||
dashboard = Dashboard(
|
||||
dashboard_title="tag cleanup test", slug="tag-cleanup-test"
|
||||
)
|
||||
db.session.add(dashboard)
|
||||
db.session.commit()
|
||||
dashboard_id = dashboard.id
|
||||
|
||||
tag = self.insert_tag(name="cleanup_test_tag", tag_type=TagType.custom)
|
||||
self.insert_tagged_object(
|
||||
tag_id=tag.id, object_id=dashboard_id, object_type=ObjectType.dashboard
|
||||
)
|
||||
|
||||
def tagged_object_count() -> int:
|
||||
return (
|
||||
db.session.query(TaggedObject)
|
||||
.filter(
|
||||
TaggedObject.object_type == ObjectType.dashboard.name,
|
||||
TaggedObject.object_id == dashboard_id,
|
||||
)
|
||||
.count()
|
||||
)
|
||||
|
||||
assert tagged_object_count() == 1
|
||||
|
||||
db.session.delete(dashboard)
|
||||
db.session.commit()
|
||||
|
||||
assert tagged_object_count() == 0
|
||||
|
||||
db.session.delete(tag)
|
||||
db.session.commit()
|
||||
|
||||
@@ -36,9 +36,6 @@ from tests.integration_tests.fixtures.birth_names_dashboard import (
|
||||
load_birth_names_data, # noqa: F401
|
||||
)
|
||||
from tests.integration_tests.fixtures.query_context import get_query_context
|
||||
from tests.integration_tests.fixtures.tags import (
|
||||
with_tagging_system_feature, # noqa: F401
|
||||
)
|
||||
from tests.integration_tests.test_app import app
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
# pylint: disable=import-outside-toplevel
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from typing import Any, TYPE_CHECKING
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from pytest_mock import MockerFixture
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm.session import Session
|
||||
from sqlalchemy.pool import StaticPool
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from superset.models.core import Database
|
||||
|
||||
# A Custom SQL ad-hoc metric exactly as Explore serializes it into a
|
||||
# ``/api/v1/chart/data`` payload. Its auto-derived ``label`` is the SQL text
|
||||
# itself, which is what makes the downstream failure mode so confusing.
|
||||
CUSTOM_SQL_METRIC: dict[str, Any] = {
|
||||
"expressionType": "SQL",
|
||||
"sqlExpression": "count(DISTINCT product_line)",
|
||||
"label": "count(DISTINCT product_line)",
|
||||
"hasCustomLabel": False,
|
||||
"optionName": "metric_abc123",
|
||||
}
|
||||
|
||||
# The same metric with ``expressionType`` absent. Every other key still marks it
|
||||
# unambiguously as an ad-hoc definition rather than a reference to a metric
|
||||
# saved on the dataset.
|
||||
MALFORMED_ADHOC_METRIC: dict[str, Any] = {
|
||||
key: value for key, value in CUSTOM_SQL_METRIC.items() if key != "expressionType"
|
||||
}
|
||||
|
||||
|
||||
def _chart_data_payload(metric: Any) -> dict[str, Any]:
|
||||
return {
|
||||
"datasource": {"id": 1, "type": "table"},
|
||||
"queries": [
|
||||
{
|
||||
"columns": ["source", "target"],
|
||||
"metrics": [metric],
|
||||
"row_limit": 100,
|
||||
}
|
||||
],
|
||||
"result_format": "json",
|
||||
"result_type": "full",
|
||||
}
|
||||
|
||||
|
||||
def _load_metrics(payload: dict[str, Any]) -> Any:
|
||||
"""Deserialize a chart data payload the way ``/api/v1/chart/data`` does."""
|
||||
from superset.charts.schemas import ChartDataQueryContextSchema
|
||||
|
||||
with patch(
|
||||
"superset.common.query_context_factory.DatasourceDAO.get_datasource",
|
||||
return_value=MagicMock(),
|
||||
):
|
||||
query_context = ChartDataQueryContextSchema().load(payload)
|
||||
return query_context.queries[0].metrics
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def database(mocker: MockerFixture, session: Session) -> Database:
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.models.core import Database
|
||||
|
||||
SqlaTable.metadata.create_all(session.get_bind())
|
||||
|
||||
engine = create_engine(
|
||||
"sqlite://",
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=StaticPool,
|
||||
future=True,
|
||||
)
|
||||
database = Database(database_name="db", sqlalchemy_uri="sqlite://")
|
||||
|
||||
connection = engine.raw_connection()
|
||||
connection.execute("CREATE TABLE t (product_line TEXT, source TEXT, target TEXT)")
|
||||
connection.commit()
|
||||
|
||||
# since we're using an in-memory SQLite database, make sure we always
|
||||
# return the same engine where the table was created
|
||||
@contextmanager
|
||||
def mock_get_sqla_engine(catalog=None, schema=None, **kwargs):
|
||||
yield engine
|
||||
|
||||
mocker.patch.object(database, "get_sqla_engine", new=mock_get_sqla_engine)
|
||||
|
||||
return database
|
||||
|
||||
|
||||
def _table(database: Database) -> Any:
|
||||
from superset.connectors.sqla.models import SqlaTable, TableColumn
|
||||
|
||||
return SqlaTable(
|
||||
database=database,
|
||||
schema=None,
|
||||
table_name="t",
|
||||
columns=[
|
||||
TableColumn(column_name="product_line"),
|
||||
TableColumn(column_name="source"),
|
||||
TableColumn(column_name="target"),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def test_adhoc_metric_without_expression_type_is_not_read_as_a_saved_metric(
|
||||
app_context: Any,
|
||||
) -> None:
|
||||
"""
|
||||
An ad-hoc metric that is missing ``expressionType`` must not be silently
|
||||
reinterpreted as a reference to a metric saved on the dataset.
|
||||
|
||||
``QueryObject._set_metrics`` used to rewrite any metric ``dict`` that was
|
||||
not recognized as ad-hoc down to its ``label``, to support the legacy
|
||||
``{"label": "saved_metric_name"}`` reference format. ``is_adhoc_metric``
|
||||
recognizes a metric solely by the presence of ``expressionType``, so an
|
||||
ad-hoc definition that lacks that one key used to be collapsed into a bare
|
||||
string. For a Custom SQL metric the label is the SQL text, so the request
|
||||
was then resolved as if the user had asked for a saved metric literally
|
||||
named ``count(DISTINCT product_line)``.
|
||||
|
||||
``ChartDataAdhocMetricSchema`` declares ``expressionType`` as required, but
|
||||
``ChartDataQueryObjectSchema.metrics`` is a list of ``fields.Raw``, so that
|
||||
contract is never enforced at the API boundary. ``_set_metrics`` is the
|
||||
last point that can tell an ad-hoc-shaped dict apart from a legacy
|
||||
reference, so it must reject the malformed shape outright rather than
|
||||
guess.
|
||||
"""
|
||||
from superset.exceptions import QueryObjectValidationError
|
||||
|
||||
with pytest.raises(
|
||||
QueryObjectValidationError,
|
||||
match=r"Invalid ad-hoc metric count\(DISTINCT product_line\): "
|
||||
r"`expressionType` is missing",
|
||||
):
|
||||
_load_metrics(_chart_data_payload(MALFORMED_ADHOC_METRIC))
|
||||
|
||||
|
||||
def test_malformed_adhoc_metric_surfaces_as_a_missing_saved_metric(
|
||||
database: Database,
|
||||
) -> None:
|
||||
"""
|
||||
Downstream symptom of the coercion above: once the ad-hoc definition has
|
||||
been reduced to its label, metric resolution looks the label up among the
|
||||
dataset's saved metrics, fails, and reports the SQL text as a metric name.
|
||||
"""
|
||||
from superset.exceptions import QueryObjectValidationError
|
||||
|
||||
with pytest.raises(
|
||||
QueryObjectValidationError,
|
||||
match=r"Metric 'count\(DISTINCT product_line\)' does not exist",
|
||||
):
|
||||
_table(database).get_sqla_query(
|
||||
columns=["source", "target"],
|
||||
metrics=["count(DISTINCT product_line)"],
|
||||
extras={},
|
||||
filter=[],
|
||||
granularity=None,
|
||||
is_timeseries=False,
|
||||
)
|
||||
|
||||
|
||||
def test_legacy_label_only_metric_still_resolves_to_a_saved_metric_name(
|
||||
app_context: Any,
|
||||
) -> None:
|
||||
"""
|
||||
Guards the fix from over-correcting: a ``dict`` carrying only ``label`` is
|
||||
the documented legacy way to reference a metric saved on the dataset, and
|
||||
must keep collapsing to that name.
|
||||
"""
|
||||
metrics = _load_metrics(_chart_data_payload({"label": "sum__num"}))
|
||||
|
||||
assert metrics == ["sum__num"]
|
||||
|
||||
|
||||
def test_well_formed_custom_sql_metric_is_preserved(
|
||||
app_context: Any,
|
||||
database: Database,
|
||||
) -> None:
|
||||
"""
|
||||
Guards the fix from over-correcting: with ``expressionType`` present the
|
||||
metric stays an ad-hoc definition and builds SQL without consulting the
|
||||
dataset's saved metrics.
|
||||
"""
|
||||
metrics = _load_metrics(_chart_data_payload(CUSTOM_SQL_METRIC))
|
||||
|
||||
assert metrics == [CUSTOM_SQL_METRIC]
|
||||
assert (
|
||||
_table(database).get_sqla_query(
|
||||
columns=["source", "target"],
|
||||
metrics=metrics,
|
||||
extras={},
|
||||
filter=[],
|
||||
granularity=None,
|
||||
is_timeseries=False,
|
||||
)
|
||||
is not None
|
||||
)
|
||||
@@ -1162,7 +1162,7 @@ def test_processing_time_offsets_quarter_offset_shifts_query_window(
|
||||
|
||||
datasource.query = fake_query
|
||||
datasource.normalize_df = MagicMock(
|
||||
side_effect=lambda offset_df, _query_object: offset_df
|
||||
side_effect=lambda offset_df, _query_object, _labels=None: offset_df
|
||||
)
|
||||
|
||||
with (
|
||||
@@ -1256,7 +1256,7 @@ def test_processing_time_offsets_accepts_zero_shift_offset(
|
||||
|
||||
datasource.query = fake_query
|
||||
datasource.normalize_df = MagicMock(
|
||||
side_effect=lambda offset_df, _query_object: offset_df
|
||||
side_effect=lambda offset_df, _query_object, _labels=None: offset_df
|
||||
)
|
||||
|
||||
with (
|
||||
@@ -2364,7 +2364,7 @@ def test_relative_offset_preserves_inner_bounds(
|
||||
|
||||
datasource.query = fake_query
|
||||
datasource.normalize_df = MagicMock(
|
||||
side_effect=lambda offset_df, _query_object: offset_df
|
||||
side_effect=lambda offset_df, _query_object, _labels=None: offset_df
|
||||
)
|
||||
|
||||
with (
|
||||
|
||||
@@ -52,7 +52,6 @@ def mock_dashboard() -> MagicMock:
|
||||
dash.editors = []
|
||||
dash.viewers = []
|
||||
dash.tags = []
|
||||
dash.custom_tags = []
|
||||
dash.is_managed_externally = False
|
||||
dash.uuid = None
|
||||
return dash
|
||||
|
||||
@@ -17,6 +17,9 @@
|
||||
|
||||
# pylint: disable=import-outside-toplevel, invalid-name, line-too-long
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, TYPE_CHECKING
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
@@ -33,6 +36,8 @@ from superset.sql.parse import Table
|
||||
from superset.superset_typing import OAuth2ClientConfig
|
||||
from superset.utils import json
|
||||
from superset.utils.oauth2 import decode_oauth2_state
|
||||
from tests.unit_tests.db_engine_specs.utils import assert_convert_dttm
|
||||
from tests.unit_tests.fixtures.common import dttm # noqa: F401
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from superset.db_engine_specs.base import OAuth2State
|
||||
@@ -1068,3 +1073,33 @@ def test_validate_parameters_skips_oauth2_connections_with_masked_encrypted_extr
|
||||
|
||||
assert errors == []
|
||||
conn.execute.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"target_type,expected_result",
|
||||
[
|
||||
("Date", "'2019-01-02'"),
|
||||
("DateTime", "'2019-01-02 03:04:05'"),
|
||||
("UnknownType", None),
|
||||
],
|
||||
)
|
||||
def test_convert_dttm(
|
||||
target_type: str,
|
||||
expected_result: str | None,
|
||||
dttm: datetime, # noqa: F811
|
||||
) -> None:
|
||||
"""
|
||||
A Date-typed column must produce a plain ISO date literal ('YYYY-MM-DD').
|
||||
|
||||
Without this, ``SqliteEngineSpec.convert_dttm`` (inherited via
|
||||
``ShillelaghEngineSpec``) returns ``None`` for ``types.Date``, and Superset falls
|
||||
back to a full ``'YYYY-MM-DD HH:MM:SS.ffffff'`` literal. shillelagh's virtual
|
||||
table layer parses that bound value with ``datetime.date.fromisoformat``, which
|
||||
rejects the trailing time-of-day and silently coerces the constraint to ``None``,
|
||||
which the GSheets adapter renders as the SQL literal ``null`` -- an unquoted
|
||||
bareword that Google's Chart API parses as a missing column reference, raising
|
||||
"Invalid query: NO_COLUMN: null".
|
||||
"""
|
||||
from superset.db_engine_specs.gsheets import GSheetsEngineSpec
|
||||
|
||||
assert_convert_dttm(GSheetsEngineSpec, target_type, expected_result, dttm)
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
import builtins
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from types import ModuleType
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, Optional
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
@@ -73,6 +73,16 @@ from tests.unit_tests.fixtures.common import dttm # noqa: F401
|
||||
("DATETIME", types.DateTime, None, GenericDataType.TEMPORAL, True),
|
||||
("TIMESTAMP", types.TIMESTAMP, None, GenericDataType.TEMPORAL, True),
|
||||
("TIME", types.Time, None, GenericDataType.TEMPORAL, True),
|
||||
# Wire-protocol names
|
||||
("VAR_STRING", types.VARCHAR, None, GenericDataType.STRING, False),
|
||||
("NEWDECIMAL", DECIMAL, None, GenericDataType.NUMERIC, False),
|
||||
("TINY", TINYINT, None, GenericDataType.NUMERIC, False),
|
||||
("SHORT", types.SmallInteger, None, GenericDataType.NUMERIC, False),
|
||||
("BLOB", types.String, None, GenericDataType.STRING, False),
|
||||
("TEXT", types.String, None, GenericDataType.STRING, False),
|
||||
("YEAR", types.Integer, None, GenericDataType.NUMERIC, False),
|
||||
("ENUM", types.String, None, GenericDataType.STRING, False),
|
||||
("SET", types.String, None, GenericDataType.STRING, False),
|
||||
],
|
||||
)
|
||||
def test_get_column_spec(
|
||||
@@ -87,6 +97,50 @@ def test_get_column_spec(
|
||||
assert_column_spec(spec, native_type, sqla_type, attrs, generic_type, is_dttm)
|
||||
|
||||
|
||||
def test_fetch_data_mutates_decimal_rows_in_tuple_results() -> None:
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec as spec # noqa: N813
|
||||
|
||||
newdecimal, var_string = 246, 253
|
||||
cursor = Mock()
|
||||
cursor.description = [("amount", newdecimal), ("label", var_string)]
|
||||
cursor.fetchall.return_value = (("10.50", "Ships"), ("22.30", "Planes"))
|
||||
|
||||
# Stub the type_code_map so this test doesn't depend on MySQLdb or
|
||||
# pymysql being importable in the test environment.
|
||||
original_type_code_map = spec.type_code_map
|
||||
spec.type_code_map = {newdecimal: "NEWDECIMAL", var_string: "VAR_STRING"}
|
||||
|
||||
try:
|
||||
data = spec.fetch_data(cursor)
|
||||
finally:
|
||||
spec.type_code_map = original_type_code_map
|
||||
|
||||
assert data == [(Decimal("10.50"), "Ships"), (Decimal("22.30"), "Planes")]
|
||||
|
||||
|
||||
def test_fetch_data_mutates_duplicate_decimal_column_names() -> None:
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec as spec # noqa: N813
|
||||
|
||||
newdecimal, var_string = 246, 253
|
||||
cursor = Mock()
|
||||
cursor.description = [
|
||||
("amount", newdecimal),
|
||||
("amount", var_string),
|
||||
("amount", newdecimal),
|
||||
]
|
||||
cursor.fetchall.return_value = [("10.50", "not a decimal", "22.30")]
|
||||
|
||||
original_type_code_map = spec.type_code_map
|
||||
spec.type_code_map = {newdecimal: "NEWDECIMAL", var_string: "VAR_STRING"}
|
||||
|
||||
try:
|
||||
data = spec.fetch_data(cursor)
|
||||
finally:
|
||||
spec.type_code_map = original_type_code_map
|
||||
|
||||
assert data == [(Decimal("10.50"), "not a decimal", Decimal("22.30"))]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"target_type,expected_result",
|
||||
[
|
||||
@@ -269,7 +323,7 @@ def test_column_type_mutator(
|
||||
assert spec.fetch_data(mock_cursor) == expected_result
|
||||
|
||||
|
||||
def test_get_datatype_pymysql_fallback():
|
||||
def test_get_datatype_pymysql_fallback() -> None:
|
||||
"""get_datatype() falls back to pymysql when MySQLdb is not installed."""
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec
|
||||
|
||||
@@ -279,15 +333,9 @@ def test_get_datatype_pymysql_fallback():
|
||||
|
||||
try:
|
||||
# Build a fake pymysql module with constants.FIELD_TYPE
|
||||
fake_field_type = ModuleType("pymysql.constants.FIELD_TYPE")
|
||||
fake_field_type.TINY = 1
|
||||
fake_field_type.VARCHAR = 15
|
||||
|
||||
fake_constants = ModuleType("pymysql.constants")
|
||||
fake_constants.FIELD_TYPE = fake_field_type
|
||||
|
||||
fake_pymysql = ModuleType("pymysql")
|
||||
fake_pymysql.constants = fake_constants
|
||||
fake_field_type = SimpleNamespace(TINY=1, VARCHAR=15)
|
||||
fake_constants = SimpleNamespace(FIELD_TYPE=fake_field_type)
|
||||
fake_pymysql = SimpleNamespace(constants=fake_constants)
|
||||
|
||||
original_import = builtins.__import__
|
||||
|
||||
@@ -308,6 +356,31 @@ def test_get_datatype_pymysql_fallback():
|
||||
MySQLEngineSpec.type_code_map = original_type_code_map
|
||||
|
||||
|
||||
def test_get_datatype_mysqlconnector_fallback() -> None:
|
||||
"""get_datatype() supports mysql-connector-python without PyMySQL."""
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec
|
||||
|
||||
original_type_code_map = MySQLEngineSpec.type_code_map
|
||||
MySQLEngineSpec.type_code_map = {}
|
||||
|
||||
try:
|
||||
fake_field_type = SimpleNamespace(NEWDECIMAL=246)
|
||||
fake_constants = SimpleNamespace(FieldType=fake_field_type)
|
||||
original_import = builtins.__import__
|
||||
|
||||
def mock_import(name: str, *args: Any, **kwargs: Any) -> Any:
|
||||
if name in {"MySQLdb", "pymysql"}:
|
||||
raise ImportError(f"No module named '{name}'")
|
||||
if name == "mysql.connector.constants":
|
||||
return fake_constants
|
||||
return original_import(name, *args, **kwargs)
|
||||
|
||||
with patch("builtins.__import__", side_effect=mock_import):
|
||||
assert MySQLEngineSpec.get_datatype(246) == "NEWDECIMAL"
|
||||
finally:
|
||||
MySQLEngineSpec.type_code_map = original_type_code_map
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("grain", "expected_expression"),
|
||||
[
|
||||
|
||||
@@ -15,7 +15,6 @@
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from flask_appbuilder.security.sqla.models import User
|
||||
from jinja2.exceptions import TemplateSyntaxError
|
||||
from pytest import raises # noqa: PT013
|
||||
from pytest_mock import MockerFixture
|
||||
|
||||
@@ -31,7 +30,7 @@ from superset.commands.exceptions import (
|
||||
DatasourceNotFoundValidationError,
|
||||
QueryNotFoundValidationError,
|
||||
)
|
||||
from superset.exceptions import SupersetSecurityException, SupersetTemplateException
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
from superset.utils.core import DatasourceType, override_user
|
||||
|
||||
dataset_find_by_id = "superset.daos.dataset.DatasetDAO.find_by_id"
|
||||
@@ -341,28 +340,6 @@ def test_query_has_access(mocker: MockerFixture) -> None:
|
||||
)
|
||||
|
||||
|
||||
def test_query_malformed_jinja_template(mocker: MockerFixture) -> None:
|
||||
"""
|
||||
``raise_for_access(query=...)`` Jinja-renders the query's SQL to resolve
|
||||
the tables it touches. A malformed template must surface as a
|
||||
``SupersetTemplateException``, not the raw ``jinja2`` exception.
|
||||
"""
|
||||
from superset.explore.utils import check_datasource_access
|
||||
from superset.models.sql_lab import Query
|
||||
|
||||
mocker.patch(query_find_by_id, return_value=Query())
|
||||
mocker.patch(
|
||||
raise_for_access,
|
||||
side_effect=TemplateSyntaxError("unexpected end of template", lineno=1),
|
||||
)
|
||||
|
||||
with raises(SupersetTemplateException): # noqa: PT012
|
||||
check_datasource_access(
|
||||
datasource_id=1,
|
||||
datasource_type=DatasourceType.QUERY,
|
||||
)
|
||||
|
||||
|
||||
def test_query_no_access(mocker: MockerFixture, client) -> None:
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.explore.utils import check_datasource_access
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts import change_detector
|
||||
|
||||
WORKFLOW_PATH = (
|
||||
Path(__file__).resolve().parents[2]
|
||||
/ ".github/workflows/frontend-bundle-size-nightly.yml"
|
||||
)
|
||||
|
||||
|
||||
def load_workflow() -> dict[str, Any]:
|
||||
return yaml.safe_load(WORKFLOW_PATH.read_text())
|
||||
|
||||
|
||||
def test_scheduled_bundle_size_action_uses_read_only_token() -> None:
|
||||
workflow = load_workflow()
|
||||
job = workflow["jobs"]["refresh-baseline"]
|
||||
steps = {step["name"]: step for step in job["steps"]}
|
||||
|
||||
benchmark_step = steps["Update bundle size baseline"]
|
||||
assert benchmark_step["with"]["github-token"] == "${{ secrets.GITHUB_TOKEN }}"
|
||||
assert workflow["permissions"]["contents"] == "read"
|
||||
effective_permissions = job.get("permissions", workflow["permissions"])
|
||||
assert effective_permissions.get("contents") in {None, "read"}
|
||||
|
||||
|
||||
def test_scheduled_bundle_size_changes_trigger_python_tests() -> None:
|
||||
assert change_detector.detect_changes(
|
||||
[".github/workflows/frontend-bundle-size-nightly.yml"],
|
||||
change_detector.PATTERNS["python"],
|
||||
)
|
||||
@@ -23,6 +23,7 @@ autouse mock_auth fixture, matching the other chart tool test files.
|
||||
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from unittest.mock import Mock, patch
|
||||
from uuid import UUID
|
||||
|
||||
@@ -45,11 +46,15 @@ def mcp_server() -> object:
|
||||
@pytest.fixture(autouse=True)
|
||||
def mock_auth() -> Iterator[Mock]:
|
||||
with patch("superset.mcp_service.auth.get_user_from_request") as mock_get_user:
|
||||
mock_user = Mock()
|
||||
mock_user.id = 1
|
||||
mock_user.username = "admin"
|
||||
mock_get_user.return_value = mock_user
|
||||
yield mock_get_user
|
||||
# The tool's editorship gate calls the real security manager; default
|
||||
# it to a no-op (caller is an editor) so unrelated tests keep passing.
|
||||
# The disclosure regression tests below re-patch it to raise.
|
||||
with patch("superset.security_manager.raise_for_editorship"):
|
||||
mock_user = Mock()
|
||||
mock_user.id = 1
|
||||
mock_user.username = "admin"
|
||||
mock_get_user.return_value = mock_user
|
||||
yield mock_get_user
|
||||
|
||||
|
||||
def _mock_chart(
|
||||
@@ -232,6 +237,33 @@ async def test_restore_chart_lookup_db_error_is_structured(
|
||||
assert "down" not in (content["error"] or "")
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_chart_editorship_check_db_error_is_structured(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""DB failures during the editorship check (not just the initial lookup)
|
||||
must return the structured LookupFailed response instead of escaping the
|
||||
tool as an unhandled error."""
|
||||
from sqlalchemy.exc import OperationalError
|
||||
|
||||
mock_find.return_value = _mock_chart(chart_id=10, slice_name="Sales")
|
||||
|
||||
with patch(
|
||||
"superset.security_manager.raise_for_editorship",
|
||||
side_effect=OperationalError("SELECT ...", {}, Exception("down")),
|
||||
):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_chart", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["error_type"] == "LookupFailed"
|
||||
assert "down" not in (content["error"] or "")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_chart_rejects_boolean_identifier(mcp_server: object) -> None:
|
||||
"""bool subclasses int; identifier=true must not coerce to chart ID 1."""
|
||||
@@ -240,3 +272,81 @@ async def test_restore_chart_rejects_boolean_identifier(mcp_server: object) -> N
|
||||
async with Client(mcp_server) as client:
|
||||
with pytest.raises(ToolError):
|
||||
await client.call_tool("restore_chart", {"request": {"identifier": True}})
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_chart_inaccessible_chart_reads_as_not_found(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""A chart outside the caller's RBAC scope must not leak its existence or
|
||||
title: the unfiltered restore lookup finds it, the base-filtered re-lookup
|
||||
does not, so the tool must answer exactly as if it does not exist.
|
||||
|
||||
The mock's return value is keyed on the actual ``skip_base_filter`` kwarg
|
||||
of each call rather than call order, so a regression that accidentally
|
||||
keeps ``skip_base_filter=True`` on the re-lookup (turning the intended
|
||||
NotFound response into a disclosure) is caught by a call-order-based
|
||||
mock returning the chart on both calls, not masked by it."""
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
def _find_side_effect(*args: Any, **kwargs: Any) -> Any | None:
|
||||
# Mirrors the real DAO contract: only an explicit skip_base_filter=True
|
||||
# (the initial unfiltered restore lookup) sees the chart; the
|
||||
# re-lookup omits it, so it defaults to False and must see nothing.
|
||||
if kwargs.get("skip_base_filter"):
|
||||
return _mock_chart(chart_id=10, slice_name="Secret KPI")
|
||||
return None
|
||||
|
||||
mock_find.side_effect = _find_side_effect
|
||||
forbidden = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="forbidden",
|
||||
error_type=SupersetErrorType.MISSING_OWNERSHIP_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
with patch("superset.security_manager.raise_for_editorship", side_effect=forbidden):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_chart", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["error_type"] == "NotFound"
|
||||
assert "Secret KPI" not in (content["error"] or "")
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_chart_visible_non_editor_gets_nameless_forbidden(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""A caller who can see the chart but cannot edit it gets a permission
|
||||
error naming the id only, never the title."""
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
chart = _mock_chart(chart_id=10, slice_name="Secret KPI")
|
||||
mock_find.side_effect = [chart, chart]
|
||||
forbidden = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="forbidden",
|
||||
error_type=SupersetErrorType.MISSING_OWNERSHIP_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
with patch("superset.security_manager.raise_for_editorship", side_effect=forbidden):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_chart", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["permission_denied"] is True
|
||||
assert content["error_type"] == "Forbidden"
|
||||
assert "Secret KPI" not in (content["error"] or "")
|
||||
assert "10" in (content["error"] or "")
|
||||
|
||||
@@ -23,6 +23,7 @@ autouse mock_auth fixture, matching the other dashboard tool test files.
|
||||
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from unittest.mock import Mock, patch
|
||||
from uuid import UUID
|
||||
|
||||
@@ -45,11 +46,15 @@ def mcp_server() -> object:
|
||||
@pytest.fixture(autouse=True)
|
||||
def mock_auth() -> Iterator[Mock]:
|
||||
with patch("superset.mcp_service.auth.get_user_from_request") as mock_get_user:
|
||||
mock_user = Mock()
|
||||
mock_user.id = 1
|
||||
mock_user.username = "admin"
|
||||
mock_get_user.return_value = mock_user
|
||||
yield mock_get_user
|
||||
# The tool's editorship gate calls the real security manager; default
|
||||
# it to a no-op (caller is an editor) so unrelated tests keep passing.
|
||||
# The disclosure regression tests below re-patch it to raise.
|
||||
with patch("superset.security_manager.raise_for_editorship"):
|
||||
mock_user = Mock()
|
||||
mock_user.id = 1
|
||||
mock_user.username = "admin"
|
||||
mock_get_user.return_value = mock_user
|
||||
yield mock_get_user
|
||||
|
||||
|
||||
def _mock_dashboard(
|
||||
@@ -236,6 +241,33 @@ async def test_restore_dashboard_slug_conflict(
|
||||
assert "slug" in (content["error"] or "").lower()
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_dashboard_editorship_check_db_error_is_structured(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""DB failures during the editorship check (not just the initial lookup)
|
||||
must return the structured LookupFailed response instead of escaping the
|
||||
tool as an unhandled error."""
|
||||
from sqlalchemy.exc import OperationalError
|
||||
|
||||
mock_find.return_value = _mock_dashboard(10, "Sales Dashboard")
|
||||
|
||||
with patch(
|
||||
"superset.security_manager.raise_for_editorship",
|
||||
side_effect=OperationalError("SELECT ...", {}, Exception("down")),
|
||||
):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_dashboard", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["error_type"] == "LookupFailed"
|
||||
assert "down" not in (content["error"] or "")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_dashboard_rejects_boolean_identifier(
|
||||
mcp_server: object,
|
||||
@@ -248,3 +280,82 @@ async def test_restore_dashboard_rejects_boolean_identifier(
|
||||
await client.call_tool(
|
||||
"restore_dashboard", {"request": {"identifier": True}}
|
||||
)
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_dashboard_inaccessible_dashboard_reads_as_not_found(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""A dashboard outside the caller's RBAC scope must not leak its
|
||||
existence or title: the unfiltered restore lookup finds it, the
|
||||
base-filtered re-lookup does not, so the tool must answer exactly as if it
|
||||
does not exist.
|
||||
|
||||
The mock's return value is keyed on the actual ``skip_base_filter`` kwarg
|
||||
of each call rather than call order, so a regression that accidentally
|
||||
keeps ``skip_base_filter=True`` on the re-lookup (turning the intended
|
||||
NotFound response into a disclosure) is caught by a call-order-based
|
||||
mock returning the dashboard on both calls, not masked by it."""
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
def _find_side_effect(*args: Any, **kwargs: Any) -> Any | None:
|
||||
# Mirrors the real DAO contract: only an explicit skip_base_filter=True
|
||||
# (the initial unfiltered restore lookup) sees the dashboard; the
|
||||
# re-lookup omits it, so it defaults to False and must see nothing.
|
||||
if kwargs.get("skip_base_filter"):
|
||||
return _mock_dashboard(dashboard_id=10, title="Secret Board")
|
||||
return None
|
||||
|
||||
mock_find.side_effect = _find_side_effect
|
||||
forbidden = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="forbidden",
|
||||
error_type=SupersetErrorType.MISSING_OWNERSHIP_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
with patch("superset.security_manager.raise_for_editorship", side_effect=forbidden):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_dashboard", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["error_type"] == "NotFound"
|
||||
assert "Secret Board" not in (content["error"] or "")
|
||||
|
||||
|
||||
@patch(_FIND)
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_dashboard_visible_non_editor_gets_nameless_forbidden(
|
||||
mock_find: Mock, mcp_server: object
|
||||
) -> None:
|
||||
"""A caller who can see the dashboard but cannot edit it gets a
|
||||
permission error naming the id only, never the title."""
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
dashboard = _mock_dashboard(dashboard_id=10, title="Secret Board")
|
||||
mock_find.side_effect = [dashboard, dashboard]
|
||||
forbidden = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="forbidden",
|
||||
error_type=SupersetErrorType.MISSING_OWNERSHIP_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
with patch("superset.security_manager.raise_for_editorship", side_effect=forbidden):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"restore_dashboard", {"request": {"identifier": 10}}
|
||||
)
|
||||
|
||||
content = result.structured_content
|
||||
assert content["success"] is False
|
||||
assert content["permission_denied"] is True
|
||||
assert content["error_type"] == "Forbidden"
|
||||
assert "Secret Board" not in (content["error"] or "")
|
||||
assert "10" in (content["error"] or "")
|
||||
|
||||
@@ -1399,6 +1399,52 @@ class TestDestructiveDDLBlocking:
|
||||
assert "Destructive DDL" in data["error"]
|
||||
ddl_mocks.execute.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_template_params_rendered_before_ddl_check(
|
||||
self, ddl_mocks, mcp_server
|
||||
):
|
||||
"""template_params={} must not skip Jinja rendering in the DDL guard.
|
||||
|
||||
The executor renders templates whenever template_params is not None
|
||||
(including {}), so the guard must parse the same rendered SQL. With a
|
||||
truthiness check, SQL whose destructive statement is hidden inside a
|
||||
Jinja expression in a comment passes the guard unrendered and then
|
||||
renders and executes.
|
||||
"""
|
||||
ddl_mocks.execute.return_value = _create_select_result(
|
||||
rows=[{"x": 1}], columns=["x"], original_sql="SELECT 1"
|
||||
)
|
||||
mock_tp = MagicMock()
|
||||
# The raw (unrendered) SQL below parses as a single, non-destructive
|
||||
# SELECT -- the Jinja expression sits inside a `--` comment. Only
|
||||
# after rendering does it become a second, destructive statement.
|
||||
mock_tp.process_template.return_value = (
|
||||
"SELECT 1;\n-- \nDROP TABLE important_table;"
|
||||
)
|
||||
|
||||
with patch(
|
||||
"superset.jinja_context.get_template_processor",
|
||||
return_value=mock_tp,
|
||||
):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"execute_sql",
|
||||
{
|
||||
"request": {
|
||||
"database_id": 1,
|
||||
"sql": (
|
||||
'SELECT 1 -- {{ "\\nDROP TABLE important_table; --" }}'
|
||||
),
|
||||
"template_params": {},
|
||||
}
|
||||
},
|
||||
)
|
||||
data = result.structured_content
|
||||
assert data["success"] is False
|
||||
assert "Destructive DDL" in data["error"]
|
||||
mock_tp.process_template.assert_called_once()
|
||||
ddl_mocks.execute.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_select_allowed(self, ddl_mocks, mcp_server):
|
||||
"""SELECT queries pass through the DDL check."""
|
||||
|
||||
@@ -141,9 +141,9 @@ async def test_find_users_returns_matches(mcp_server):
|
||||
# required for filter resolution. Catch regressions on the response shape.
|
||||
for forbidden in ("email", "active", "roles"):
|
||||
assert forbidden not in data["users"][0]
|
||||
# or_ should have been built across the four matched columns
|
||||
# or_ should have been built across the three matched columns (no email)
|
||||
assert mock_or.called
|
||||
assert len(mock_or.call_args.args) == 4
|
||||
assert len(mock_or.call_args.args) == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -201,6 +201,19 @@ def test_find_users_request_strips_query_whitespace():
|
||||
assert request.query == "maxime"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"email",
|
||||
["victim@example.com", "Victim@Example.COM", "a.b+tag@sub.example.co"],
|
||||
)
|
||||
def test_find_users_request_rejects_email_shaped_query(email):
|
||||
# Usernames are frequently email addresses under OAuth provisioning, so
|
||||
# an email-shaped query must be rejected outright rather than relying on
|
||||
# the email-column exclusion alone -- otherwise it still confirms an
|
||||
# account's existence via the username column.
|
||||
with pytest.raises(ValidationError):
|
||||
FindUsersRequest(query=email)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filter contract: created_by_fk / changed_by_fk filtering on list tools
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -336,3 +349,51 @@ async def test_find_users_escapes_literal_backslash(mcp_server):
|
||||
assert ilike_call is not None
|
||||
assert ilike_call.args[0] == "%\\\\%"
|
||||
assert ilike_call.kwargs.get("escape") == "\\"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_users_does_not_match_on_email(mcp_server):
|
||||
"""Email must not be a searchable column: substring or exact email
|
||||
matching would let any MCP credential confirm which addresses have
|
||||
accounts (an email-disclosure oracle the web API reserves for admins).
|
||||
Uses a non-email-shaped query so this exercises the OR-clause contract
|
||||
itself, independent of the email-shape rejection covered below."""
|
||||
session, _ = _patch_user_query([])
|
||||
|
||||
with (
|
||||
patch.object(find_users_module, "db") as mock_db,
|
||||
patch.object(find_users_module, "security_manager") as mock_sm,
|
||||
patch.object(find_users_module, "or_") as mock_or,
|
||||
):
|
||||
mock_db.session = session
|
||||
user_model = MagicMock()
|
||||
mock_sm.user_model = user_model
|
||||
mock_or.return_value = MagicMock()
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
await client.call_tool("find_users", {"request": {"query": "victim"}})
|
||||
|
||||
assert user_model.username.ilike.called
|
||||
assert user_model.first_name.ilike.called
|
||||
assert user_model.last_name.ilike.called
|
||||
assert not user_model.email.ilike.called
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_users_rejects_email_shaped_query_via_client(mcp_server):
|
||||
"""An email-shaped query is rejected before it ever reaches the DB
|
||||
filter: since usernames are frequently email addresses under OAuth
|
||||
provisioning, letting it fall through to the username column would
|
||||
still let an email lookup confirm whether an address has an account,
|
||||
defeating the documented non-enumeration guarantee."""
|
||||
with (
|
||||
patch.object(find_users_module, "db") as mock_db,
|
||||
patch.object(find_users_module, "security_manager"),
|
||||
):
|
||||
async with Client(mcp_server) as client:
|
||||
with pytest.raises(ToolError):
|
||||
await client.call_tool(
|
||||
"find_users", {"request": {"query": "victim@example.com"}}
|
||||
)
|
||||
|
||||
mock_db.session.query.assert_not_called()
|
||||
|
||||
@@ -233,6 +233,36 @@ def test_raises_when_no_auth_source(app) -> None:
|
||||
get_user_from_request()
|
||||
|
||||
|
||||
def test_rejected_guest_token_does_not_fall_through_to_dev_username(app) -> None:
|
||||
"""A guest-marked token rejected for disabled guest auth must not degrade
|
||||
to a weaker auth source (MCP_DEV_USERNAME here) via get_user_from_request.
|
||||
|
||||
Before the fix, _resolve_user_from_jwt_context returned None for this
|
||||
case, which get_user_from_request treats identically to "no token
|
||||
present" and falls through to the next priority source -- silently
|
||||
executing the caller as MCP_DEV_USERNAME in a JWT-only deployment with a
|
||||
dev username configured.
|
||||
"""
|
||||
from superset.mcp_service.guest_token_verifier import GUEST_TOKEN_CLAIM
|
||||
|
||||
token = MagicMock()
|
||||
token.claims = {GUEST_TOKEN_CLAIM: True, "sub": "attacker"}
|
||||
token.client_id = "guest"
|
||||
|
||||
with app.app_context():
|
||||
app.config["MCP_DEV_USERNAME"] = "dev_admin"
|
||||
app.config["MCP_EMBEDDED_GUEST_AUTH_ENABLED"] = False
|
||||
try:
|
||||
with patch(
|
||||
"fastmcp.server.dependencies.get_access_token", return_value=token
|
||||
):
|
||||
with pytest.raises(ValueError, match="Guest-marked token"):
|
||||
get_user_from_request()
|
||||
finally:
|
||||
app.config.pop("MCP_DEV_USERNAME", None)
|
||||
app.config.pop("MCP_EMBEDDED_GUEST_AUTH_ENABLED", None)
|
||||
|
||||
|
||||
def test_no_auth_source_error_message_has_no_config_details(app) -> None:
|
||||
"""Client-facing auth error must be generic — no server config disclosed.
|
||||
|
||||
|
||||
@@ -575,7 +575,13 @@ def test_resolve_rejects_guest_marker_when_guest_auth_disabled(
|
||||
app: SupersetApp,
|
||||
) -> None:
|
||||
"""Even with the marker + client_id, a token is not treated as a guest when
|
||||
embedded guest auth is disabled (defense against marker forgery)."""
|
||||
embedded guest auth is disabled (defense against marker forgery).
|
||||
|
||||
A guest-marked token is an explicit, rejected authentication attempt, not
|
||||
an absent one, so it must fail closed (raise) rather than return None and
|
||||
let the caller fall through to a weaker auth source (API key,
|
||||
MCP_DEV_USERNAME, or a middleware-set g.user).
|
||||
"""
|
||||
token = MagicMock()
|
||||
token.claims = {GUEST_TOKEN_CLAIM: True, **_parsed_guest_claims()}
|
||||
token.client_id = "guest"
|
||||
@@ -586,15 +592,15 @@ def test_resolve_rejects_guest_marker_when_guest_auth_disabled(
|
||||
patch("fastmcp.server.dependencies.get_access_token", return_value=token),
|
||||
patch("superset.mcp_service.auth.is_feature_enabled", return_value=True),
|
||||
):
|
||||
result = _resolve_user_from_jwt_context(app)
|
||||
|
||||
assert result is None
|
||||
with pytest.raises(ValueError, match="Guest-marked token"):
|
||||
_resolve_user_from_jwt_context(app)
|
||||
|
||||
|
||||
def test_resolve_rejects_guest_marker_when_embedded_flag_off(app: SupersetApp) -> None:
|
||||
"""The other half of the gate: with the marker + client_id + the MCP guest
|
||||
flag on, a token is still not treated as a guest when the EMBEDDED_SUPERSET
|
||||
feature flag is off (both gates are required)."""
|
||||
feature flag is off (both gates are required). Same fail-closed
|
||||
requirement as the sibling case above."""
|
||||
token = MagicMock()
|
||||
token.claims = {GUEST_TOKEN_CLAIM: True, **_parsed_guest_claims()}
|
||||
token.client_id = "guest"
|
||||
@@ -605,9 +611,8 @@ def test_resolve_rejects_guest_marker_when_embedded_flag_off(app: SupersetApp) -
|
||||
patch("fastmcp.server.dependencies.get_access_token", return_value=token),
|
||||
patch("superset.mcp_service.auth.is_feature_enabled", return_value=False),
|
||||
):
|
||||
result = _resolve_user_from_jwt_context(app)
|
||||
|
||||
assert result is None
|
||||
with pytest.raises(ValueError, match="Guest-marked token"):
|
||||
_resolve_user_from_jwt_context(app)
|
||||
|
||||
|
||||
def test_resolve_ignores_guest_marker_without_guest_client_id(app: SupersetApp) -> None:
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
"""Tests for the ``python -m superset.mcp_service`` entrypoint's non-stdio path.
|
||||
|
||||
Network transports (streamable-http, sse, ...) must install the same auth
|
||||
provider as the supported CLI path (``superset mcp run`` -> server.run_server()
|
||||
-> _create_auth_provider()) instead of starting with no verifier at all.
|
||||
"""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_main_installs_auth_provider_for_network_transport(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The non-stdio branch must call _create_auth_provider and pass its
|
||||
result into init_fastmcp_server, mirroring run_server()."""
|
||||
monkeypatch.setenv("FASTMCP_TRANSPORT", "sse")
|
||||
|
||||
from superset.mcp_service import __main__ as main_module
|
||||
|
||||
flask_app = MagicMock()
|
||||
auth_provider = object()
|
||||
mcp_instance = MagicMock()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"superset.mcp_service.flask_singleton.get_flask_app",
|
||||
return_value=flask_app,
|
||||
),
|
||||
patch(
|
||||
"superset.mcp_service.server._create_auth_provider",
|
||||
return_value=auth_provider,
|
||||
) as mock_create_auth_provider,
|
||||
patch.object(
|
||||
main_module, "init_fastmcp_server", return_value=mcp_instance
|
||||
) as mock_init,
|
||||
patch.object(main_module, "_add_default_middlewares"),
|
||||
patch.object(main_module.mcp, "run") as mock_run,
|
||||
):
|
||||
main_module.main()
|
||||
|
||||
mock_create_auth_provider.assert_called_once_with(flask_app)
|
||||
mock_init.assert_called_once_with(auth=auth_provider)
|
||||
mock_run.assert_called_once_with(transport="sse")
|
||||
|
||||
|
||||
def test_main_propagates_auth_config_error_for_network_transport(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A verifier-construction failure must abort startup, not fall through to
|
||||
an unauthenticated server.
|
||||
|
||||
Before the fix, the non-stdio branch never called _create_auth_provider
|
||||
at all, so MCP_AUTH_ENABLED was silently discarded by this entrypoint.
|
||||
"""
|
||||
monkeypatch.setenv("FASTMCP_TRANSPORT", "streamable-http")
|
||||
|
||||
from superset.mcp_service import __main__ as main_module
|
||||
from superset.mcp_service.mcp_config import MCPAuthConfigError
|
||||
|
||||
flask_app = MagicMock()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"superset.mcp_service.flask_singleton.get_flask_app",
|
||||
return_value=flask_app,
|
||||
),
|
||||
patch(
|
||||
"superset.mcp_service.server._create_auth_provider",
|
||||
side_effect=MCPAuthConfigError("bad auth config"),
|
||||
),
|
||||
patch.object(main_module, "init_fastmcp_server") as mock_init,
|
||||
patch.object(main_module, "_add_default_middlewares"),
|
||||
patch.object(main_module.mcp, "run") as mock_run,
|
||||
):
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
main_module.main()
|
||||
|
||||
mock_init.assert_not_called()
|
||||
mock_run.assert_not_called()
|
||||
|
||||
|
||||
def test_add_default_middlewares_installs_response_caching() -> None:
|
||||
"""``_add_default_middlewares`` must install response caching when
|
||||
configured, matching ``run_server()``'s default (non-factory) path --
|
||||
otherwise MCP_CACHE_CONFIG has no effect for this entrypoint's transports.
|
||||
"""
|
||||
from superset.mcp_service import __main__ as main_module
|
||||
|
||||
caching_middleware = object()
|
||||
|
||||
with (
|
||||
patch.object(main_module, "build_middleware_list", return_value=[]),
|
||||
patch.object(
|
||||
main_module, "create_response_size_guard_middleware", return_value=None
|
||||
),
|
||||
patch.object(
|
||||
main_module,
|
||||
"create_response_caching_middleware",
|
||||
return_value=caching_middleware,
|
||||
),
|
||||
patch.object(main_module.mcp, "add_middleware") as mock_add_middleware,
|
||||
):
|
||||
main_module._add_default_middlewares()
|
||||
|
||||
mock_add_middleware.assert_called_once_with(caching_middleware)
|
||||
@@ -19,6 +19,8 @@
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from superset.mcp_service.caching import (
|
||||
_build_caching_settings,
|
||||
_version_cache_prefix,
|
||||
@@ -106,7 +108,11 @@ def test_create_response_caching_middleware_falls_back_to_memory_when_no_prefix(
|
||||
"""Caching middleware uses in-memory store when CACHE_KEY_PREFIX is not set."""
|
||||
mock_flask_app = MagicMock()
|
||||
mock_configs = {
|
||||
"MCP_CACHE_CONFIG": {"enabled": True, "list_tools_ttl": 300},
|
||||
"MCP_CACHE_CONFIG": {
|
||||
"enabled": True,
|
||||
"dangerously_share_cache_across_principals": True,
|
||||
"list_tools_ttl": 300,
|
||||
},
|
||||
"MCP_STORE_CONFIG": {"enabled": True}, # Store enabled but no CACHE_KEY_PREFIX
|
||||
}
|
||||
mock_flask_app.config.get.side_effect = lambda key, default=None: mock_configs.get(
|
||||
@@ -141,7 +147,11 @@ def test_create_response_caching_middleware_uses_memory_store_when_store_disable
|
||||
"""Caching middleware uses in-memory store when MCP_STORE_CONFIG is disabled."""
|
||||
mock_flask_app = MagicMock()
|
||||
mock_configs = {
|
||||
"MCP_CACHE_CONFIG": {"enabled": True, "list_tools_ttl": 300},
|
||||
"MCP_CACHE_CONFIG": {
|
||||
"enabled": True,
|
||||
"dangerously_share_cache_across_principals": True,
|
||||
"list_tools_ttl": 300,
|
||||
},
|
||||
"MCP_STORE_CONFIG": {"enabled": False},
|
||||
}
|
||||
mock_flask_app.config.get.side_effect = lambda key, default=None: mock_configs.get(
|
||||
@@ -177,6 +187,7 @@ def test_create_response_caching_middleware_creates_middleware():
|
||||
mock_flask_app = MagicMock()
|
||||
mock_flask_app.config.get.return_value = {
|
||||
"enabled": True,
|
||||
"dangerously_share_cache_across_principals": True,
|
||||
"CACHE_KEY_PREFIX": "mcp_cache_v1_",
|
||||
"list_tools_ttl": 300,
|
||||
}
|
||||
@@ -211,3 +222,62 @@ def test_create_response_caching_middleware_creates_middleware():
|
||||
call_kwargs = mock_middleware_class.call_args[1]
|
||||
assert call_kwargs["cache_storage"] is mock_store
|
||||
assert call_kwargs["list_tools_settings"] == {"ttl": 300}
|
||||
|
||||
|
||||
def test_create_response_caching_middleware_fails_closed_without_principal_optin():
|
||||
"""Enabling the cache without the explicit cross-principal opt-in is refused.
|
||||
|
||||
The cache key contains no principal and hits are served before any
|
||||
authorization runs, so a shared cache would replay one principal's
|
||||
responses to another (see create_response_caching_middleware).
|
||||
"""
|
||||
mock_flask_app = MagicMock()
|
||||
mock_configs = {
|
||||
"MCP_CACHE_CONFIG": {"enabled": True, "call_tool_ttl": 3600},
|
||||
"MCP_STORE_CONFIG": {"enabled": False},
|
||||
}
|
||||
mock_flask_app.config.get.side_effect = lambda key, default=None: mock_configs.get(
|
||||
key, default
|
||||
)
|
||||
|
||||
with patch(
|
||||
"superset.mcp_service.flask_singleton.get_flask_app",
|
||||
return_value=mock_flask_app,
|
||||
):
|
||||
with patch("flask.has_app_context", return_value=True):
|
||||
from superset.mcp_service.caching import (
|
||||
create_response_caching_middleware,
|
||||
)
|
||||
|
||||
assert create_response_caching_middleware() is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_excluded_tools_covers_every_mutating_tool():
|
||||
"""Every registered tool without readOnlyHint=True must be listed in
|
||||
MCP_CACHE_CONFIG["excluded_tools"].
|
||||
|
||||
Caching is keyed on tool name + arguments and served ahead of
|
||||
per-request auth/RBAC, so a mutating tool left off this list can
|
||||
silently replay a stale create/update/delete result to a caller who
|
||||
repeats an identical call expecting it to actually run again. This
|
||||
list is maintained by hand (FastMCP's caching middleware only accepts
|
||||
a static exclusion list); this test is what keeps it complete as new
|
||||
tools are added, by failing with the specific tool name(s) missing.
|
||||
"""
|
||||
from superset.mcp_service.app import mcp
|
||||
from superset.mcp_service.mcp_config import MCP_CACHE_CONFIG
|
||||
|
||||
tools = await mcp.list_tools()
|
||||
mutating_tool_names = {
|
||||
tool.name
|
||||
for tool in tools
|
||||
if tool.annotations is None or tool.annotations.readOnlyHint is not True
|
||||
}
|
||||
|
||||
excluded = set(MCP_CACHE_CONFIG["excluded_tools"])
|
||||
missing = mutating_tool_names - excluded
|
||||
assert not missing, (
|
||||
f"These mutating tools are cacheable because they're missing from "
|
||||
f"MCP_CACHE_CONFIG['excluded_tools']: {sorted(missing)}"
|
||||
)
|
||||
|
||||
@@ -421,9 +421,12 @@ def test_create_default_mcp_auth_factory_jwt_with_keys():
|
||||
mock_build.assert_called_once()
|
||||
|
||||
|
||||
def test_create_default_mcp_auth_factory_jwt_enabled_without_keys_returns_none():
|
||||
"""MCP_AUTH_ENABLED=True with no keys/secret and no API key auth returns None."""
|
||||
from superset.mcp_service.mcp_config import create_default_mcp_auth_factory
|
||||
def test_create_default_mcp_auth_factory_jwt_enabled_without_keys_fails_closed():
|
||||
"""MCP_AUTH_ENABLED=True with no keys/secret and no fallback must abort."""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
mock_app = MagicMock()
|
||||
mock_app.config.get.side_effect = lambda key, default=None: {
|
||||
@@ -433,16 +436,40 @@ def test_create_default_mcp_auth_factory_jwt_enabled_without_keys_returns_none()
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}.get(key, default)
|
||||
|
||||
with patch("superset.mcp_service.mcp_config.logger") as mock_logger:
|
||||
result = create_default_mcp_auth_factory(mock_app)
|
||||
|
||||
assert result is None
|
||||
mock_logger.warning.assert_called_once()
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
create_default_mcp_auth_factory(mock_app)
|
||||
|
||||
|
||||
def test_create_default_mcp_auth_factory_jwt_build_failure_returns_none():
|
||||
"""A JWT verifier build failure with no API key fallback returns None."""
|
||||
from superset.mcp_service.mcp_config import create_default_mcp_auth_factory
|
||||
def test_create_default_mcp_auth_factory_jwt_missing_keys_fails_closed_with_api_key():
|
||||
"""A missing JWT key must abort startup even when API-key auth is also
|
||||
enabled: silently starting without the operator's requested JWT mode
|
||||
would leave JWT clients unable to authenticate with only a log line to
|
||||
show for it.
|
||||
"""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
mock_app = MagicMock()
|
||||
mock_app.config.get.side_effect = lambda key, default=None: {
|
||||
"MCP_AUTH_ENABLED": True,
|
||||
"MCP_API_KEY_ENABLED": True,
|
||||
"FAB_API_KEY_ENABLED": False,
|
||||
"FAB_API_KEY_PREFIXES": ["sst_"],
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}.get(key, default)
|
||||
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
create_default_mcp_auth_factory(mock_app)
|
||||
|
||||
|
||||
def test_create_default_mcp_auth_factory_jwt_build_failure_fails_closed():
|
||||
"""A JWT verifier build failure must abort startup, not disable auth."""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
mock_app = MagicMock()
|
||||
mock_app.config.get.side_effect = lambda key, default=None: {
|
||||
@@ -460,12 +487,34 @@ def test_create_default_mcp_auth_factory_jwt_build_failure_returns_none():
|
||||
),
|
||||
patch("superset.mcp_service.mcp_config.logger") as mock_logger,
|
||||
):
|
||||
result = create_default_mcp_auth_factory(mock_app)
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
create_default_mcp_auth_factory(mock_app)
|
||||
|
||||
assert result is None
|
||||
mock_logger.error.assert_called_once()
|
||||
|
||||
|
||||
def test_create_default_mcp_auth_factory_refuses_dev_username_with_auth():
|
||||
"""MCP_DEV_USERNAME + MCP_AUTH_ENABLED is a standing auth bypass."""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
mock_app = MagicMock()
|
||||
mock_app.config.get.side_effect = lambda key, default=None: {
|
||||
"MCP_AUTH_ENABLED": True,
|
||||
"MCP_API_KEY_ENABLED": False,
|
||||
"FAB_API_KEY_ENABLED": False,
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
"MCP_JWT_SECRET": "shhh",
|
||||
"MCP_JWT_ALGORITHM": "HS256",
|
||||
"MCP_DEV_USERNAME": "admin",
|
||||
}.get(key, default)
|
||||
|
||||
with pytest.raises(MCPAuthConfigError, match="MCP_DEV_USERNAME"):
|
||||
create_default_mcp_auth_factory(mock_app)
|
||||
|
||||
|
||||
def test_create_default_mcp_auth_factory_requires_audience_when_jwt_enabled():
|
||||
"""MCP_AUTH_ENABLED=True without MCP_JWT_AUDIENCE fails closed.
|
||||
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
"""HS256 key-confusion guard: never key an HMAC verifier on public material."""
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
PEM_PUBLIC_KEY = "-----BEGIN PUBLIC KEY-----\nMFkw...\n-----END PUBLIC KEY-----"
|
||||
|
||||
|
||||
def _mock_app(config: dict[str, Any]) -> MagicMock:
|
||||
mock_app = MagicMock()
|
||||
mock_app.config.get.side_effect = lambda key, default=None: config.get(key, default)
|
||||
return mock_app
|
||||
|
||||
|
||||
def test_build_jwt_verifier_refuses_hs256_without_secret():
|
||||
"""HS256 pinned but only public-key material configured must hard-error.
|
||||
|
||||
Before the fix this silently built an HS256 verifier keyed on the PEM
|
||||
public key, letting anyone holding the (public) key forge admin tokens.
|
||||
"""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
_build_jwt_verifier,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
app = _mock_app(
|
||||
{
|
||||
"MCP_JWT_ALGORITHM": "HS256",
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}
|
||||
)
|
||||
with pytest.raises(MCPAuthConfigError, match="MCP_JWT_SECRET"):
|
||||
_build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=None,
|
||||
public_key=PEM_PUBLIC_KEY,
|
||||
secret=None,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("algorithm", ["HS256", "HS384", "HS512"])
|
||||
def test_build_jwt_verifier_refuses_hmac_alongside_public_key_material(
|
||||
algorithm: str,
|
||||
):
|
||||
"""HMAC algorithm plus leftover public key / JWKS config is refused."""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
_build_jwt_verifier,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
app = _mock_app(
|
||||
{
|
||||
"MCP_JWT_ALGORITHM": algorithm,
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}
|
||||
)
|
||||
with pytest.raises(MCPAuthConfigError, match="MCP_JWT_PUBLIC_KEY"):
|
||||
_build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=None,
|
||||
public_key=PEM_PUBLIC_KEY,
|
||||
secret="shhh", # noqa: S106
|
||||
)
|
||||
|
||||
|
||||
def test_build_jwt_verifier_hs256_with_explicit_secret_still_works():
|
||||
"""A correct HS256 config (secret only) builds an HS256 verifier."""
|
||||
from superset.mcp_service import mcp_config
|
||||
|
||||
app = _mock_app(
|
||||
{
|
||||
"MCP_JWT_ALGORITHM": "HS256",
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}
|
||||
)
|
||||
with patch.object(mcp_config, "MCPJWTVerifier") as mock_verifier:
|
||||
mcp_config._build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=None,
|
||||
public_key=None,
|
||||
secret="shhh", # noqa: S106
|
||||
)
|
||||
kwargs = mock_verifier.call_args.kwargs
|
||||
assert kwargs["algorithm"] == "HS256"
|
||||
assert kwargs["public_key"] == "shhh"
|
||||
|
||||
|
||||
def test_build_jwt_verifier_refuses_rs256_secret_only():
|
||||
"""RS256 (asymmetric) pinned but only a secret configured must hard-error.
|
||||
|
||||
A keyless RS256 verifier cannot validate anything -- name the fix rather
|
||||
than letting the underlying verifier constructor raise opaquely.
|
||||
"""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
_build_jwt_verifier,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
app = _mock_app(
|
||||
{
|
||||
"MCP_JWT_ALGORITHM": "RS256",
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
}
|
||||
)
|
||||
with pytest.raises(MCPAuthConfigError, match="MCP_JWT_ALGORITHM"):
|
||||
_build_jwt_verifier(
|
||||
app=app,
|
||||
jwks_uri=None,
|
||||
public_key=None,
|
||||
secret="shhh", # noqa: S106
|
||||
)
|
||||
|
||||
|
||||
def test_auth_factory_propagates_hs256_config_error():
|
||||
"""The factory must not swallow the misconfiguration into a None provider."""
|
||||
from superset.mcp_service.mcp_config import (
|
||||
create_default_mcp_auth_factory,
|
||||
MCPAuthConfigError,
|
||||
)
|
||||
|
||||
app = _mock_app(
|
||||
{
|
||||
"MCP_AUTH_ENABLED": True,
|
||||
"MCP_API_KEY_ENABLED": False,
|
||||
"FAB_API_KEY_ENABLED": False,
|
||||
"MCP_JWT_AUDIENCE": "superset-mcp",
|
||||
"MCP_JWT_ALGORITHM": "HS256",
|
||||
"MCP_JWT_PUBLIC_KEY": PEM_PUBLIC_KEY,
|
||||
}
|
||||
)
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
create_default_mcp_auth_factory(app)
|
||||
@@ -383,6 +383,55 @@ def test_create_auth_provider_fails_closed_on_insecure_guest_secret() -> None:
|
||||
_create_auth_provider(flask_app)
|
||||
|
||||
|
||||
def test_create_auth_provider_fails_closed_on_default_factory_error() -> None:
|
||||
"""A generic error while building the enabled auth provider must abort.
|
||||
|
||||
Verifier-construction failures (bad key material, config typos) used to be
|
||||
swallowed, silently starting an unauthenticated server.
|
||||
"""
|
||||
from superset.mcp_service.mcp_config import MCPAuthConfigError
|
||||
from superset.mcp_service.server import _create_auth_provider
|
||||
|
||||
flask_app = MagicMock()
|
||||
flask_app.config.get.side_effect = lambda key, default=None: {
|
||||
"MCP_AUTH_FACTORY": None,
|
||||
"MCP_AUTH_ENABLED": True,
|
||||
"MCP_API_KEY_ENABLED": False,
|
||||
"FAB_API_KEY_ENABLED": False,
|
||||
}.get(key, default)
|
||||
|
||||
with patch(
|
||||
"superset.mcp_service.mcp_config.create_default_mcp_auth_factory",
|
||||
side_effect=ValueError("bad PEM"),
|
||||
):
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
_create_auth_provider(flask_app)
|
||||
|
||||
|
||||
def test_create_auth_provider_fails_closed_on_custom_factory_error() -> None:
|
||||
"""MCP_AUTH_FACTORY raising (or yielding None) must abort startup."""
|
||||
from superset.mcp_service.mcp_config import MCPAuthConfigError
|
||||
from superset.mcp_service.server import _create_auth_provider
|
||||
|
||||
def broken_factory(app: Any) -> Any:
|
||||
raise ValueError("bad key material")
|
||||
|
||||
flask_app = MagicMock()
|
||||
flask_app.config.get.side_effect = lambda key, default=None: {
|
||||
"MCP_AUTH_FACTORY": broken_factory,
|
||||
}.get(key, default)
|
||||
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
_create_auth_provider(flask_app)
|
||||
|
||||
flask_app.config.get.side_effect = lambda key, default=None: {
|
||||
"MCP_AUTH_FACTORY": lambda app: None,
|
||||
}.get(key, default)
|
||||
|
||||
with pytest.raises(MCPAuthConfigError):
|
||||
_create_auth_provider(flask_app)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _run_server_dependencies(
|
||||
flask_config: dict[str, Any],
|
||||
|
||||
@@ -39,6 +39,7 @@ from superset.mcp_service.constants import DEFAULT_MAX_LIST_ITEMS
|
||||
from superset.mcp_service.mcp_config import MCP_RESPONSE_SIZE_CONFIG
|
||||
from superset.mcp_service.middleware import (
|
||||
_is_user_error,
|
||||
_sanitize_params,
|
||||
create_response_size_guard_middleware,
|
||||
GlobalErrorHandlerMiddleware,
|
||||
RBACToolVisibilityMiddleware,
|
||||
@@ -1381,6 +1382,106 @@ class TestGlobalErrorHandlerLogLevels:
|
||||
# Should log at ERROR (both the classification log and the error_id log)
|
||||
assert mock_logger.error.call_count >= 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_value_error_message_is_sanitized(self) -> None:
|
||||
"""A ValueError's text reaches the client through _sanitize_error_for_logging.
|
||||
|
||||
ValueError is deliberately not in that sanitizer's generic-message
|
||||
list (so LLM callers still get parameter feedback), but a connection
|
||||
string embedded in the message must still be redacted, not returned
|
||||
verbatim.
|
||||
"""
|
||||
middleware = GlobalErrorHandlerMiddleware()
|
||||
|
||||
context = MagicMock()
|
||||
context.message.name = "execute_sql"
|
||||
context.method = "tools/call"
|
||||
|
||||
call_next = AsyncMock(
|
||||
side_effect=ValueError(
|
||||
"Invalid config: postgresql://admin:hunter2@db.internal/prod"
|
||||
)
|
||||
)
|
||||
|
||||
with (
|
||||
patch("superset.mcp_service.middleware.get_user_id", return_value=1),
|
||||
patch("superset.mcp_service.middleware.event_logger"),
|
||||
patch("superset.mcp_service.middleware.logger"),
|
||||
):
|
||||
with pytest.raises(ToolError) as exc_info:
|
||||
await middleware.on_message(context, call_next)
|
||||
|
||||
message = str(exc_info.value)
|
||||
assert "hunter2" not in message
|
||||
assert "db.internal" not in message
|
||||
assert "Invalid config" in message
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_http_exception_detail_is_sanitized(self) -> None:
|
||||
"""HTTPException.detail reaches the client through
|
||||
_sanitize_error_for_logging instead of being interpolated raw."""
|
||||
from starlette.exceptions import HTTPException
|
||||
|
||||
middleware = GlobalErrorHandlerMiddleware()
|
||||
|
||||
context = MagicMock()
|
||||
context.message.name = "get_chart_preview"
|
||||
context.method = "tools/call"
|
||||
|
||||
call_next = AsyncMock(
|
||||
side_effect=HTTPException(
|
||||
status_code=502,
|
||||
detail="Upstream failed: postgresql://admin:hunter2@db.internal/prod",
|
||||
)
|
||||
)
|
||||
|
||||
with (
|
||||
patch("superset.mcp_service.middleware.get_user_id", return_value=1),
|
||||
patch("superset.mcp_service.middleware.event_logger"),
|
||||
patch("superset.mcp_service.middleware.logger"),
|
||||
):
|
||||
with pytest.raises(ToolError) as exc_info:
|
||||
await middleware.on_message(context, call_next)
|
||||
|
||||
message = str(exc_info.value)
|
||||
assert "hunter2" not in message
|
||||
assert "db.internal" not in message
|
||||
assert "Upstream failed" in message
|
||||
|
||||
|
||||
class TestSanitizeParams:
|
||||
"""Tests for _sanitize_params's recursion into nested containers."""
|
||||
|
||||
def test_redacts_top_level_sensitive_key(self) -> None:
|
||||
result = _sanitize_params({"password": "hunter2", "name": "alice"})
|
||||
assert result["password"] == "[REDACTED]" # noqa: S105
|
||||
assert result["name"] == "alice"
|
||||
|
||||
def test_redacts_sensitive_key_nested_under_arguments(self) -> None:
|
||||
result = _sanitize_params({"arguments": {"password": "hunter2"}})
|
||||
assert result["arguments"]["password"] == "[REDACTED]" # noqa: S105
|
||||
|
||||
def test_redacts_sensitive_key_nested_under_request(self) -> None:
|
||||
"""Any nested dict wrapper is redacted, not just the literal
|
||||
'arguments' key -- Pydantic-request tools wrap params under 'request'."""
|
||||
result = _sanitize_params({"request": {"password": "hunter2"}})
|
||||
assert result["request"]["password"] == "[REDACTED]" # noqa: S105
|
||||
|
||||
def test_redacts_sensitive_key_inside_list_of_dicts(self) -> None:
|
||||
result = _sanitize_params({"items": [{"token": "abc123"}, {"name": "x"}]})
|
||||
assert result["items"][0]["token"] == "[REDACTED]" # noqa: S105
|
||||
assert result["items"][1]["name"] == "x"
|
||||
|
||||
def test_redacts_sensitive_key_inside_nested_list_of_lists(self) -> None:
|
||||
"""A list nested inside another list must still be recursed into,
|
||||
not copied unchanged -- otherwise a sensitive key inside it would
|
||||
reach the audit log unredacted."""
|
||||
result = _sanitize_params({"items": [[{"password": "hunter2"}]]})
|
||||
assert result["items"][0][0]["password"] == "[REDACTED]" # noqa: S105
|
||||
|
||||
def test_non_dict_passthrough(self) -> None:
|
||||
assert _sanitize_params("not-a-dict") == "not-a-dict" # type: ignore[arg-type]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_event_logger_includes_severity(self) -> None:
|
||||
"""Event logger payload should include severity field."""
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user