Compare commits

..
Author SHA1 Message Date
rusackasandClaude Sonnet 5 3c3c92e96b fix(security): use self-repository syntax for setup-backend in postgres job
zizmor's self-repository audit flags uses: ./.github/actions/setup-backend/
in the test-postgres job in favor of GitHub's dedicated uses: $/... syntax.
This mirrors the mechanical fix already applied to the analogous
setup-backend references in this same workflow (test-sqlite, #44018) and in
superset-python-presto-hive.yml (#43975), now that the ASF allowlist check
has been bumped to recognize $/ refs (#44014).

Resolves code-scanning alert #2656.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 18:25:50 -07:00
986b648f1c fix(versioning): resolve dashboard M2M membership by attach/detach windows, not end_transaction_id (#44010)
Co-authored-by: Mike Bridge <michael.bridge@ext.preset.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-08 17:59:04 -07:00
dependabot[bot]andClaude Code acb29937ae chore(deps): bump the security group across 1 directory with 3 updates (#44030)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-08 17:52:32 -07:00
dependabot[bot]andClaude Code 2875c76d55 chore(deps): bump react-ace from 14.0.1 to 15.0.0 in /superset-frontend (#42833)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-08 17:52:09 -07:00
d0f139c210 feat(deps): bump flask-caching from 2.4.1 to 2.5.0 + bump flask from v2 to v3 (#43875)
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: hainenber <dotronghai96@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: rusackas <evan@rusackas.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: hainenber <dotronghai96@gmail.com>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-09-08 17:20:53 -07:00
dependabot[bot] 9de50b85e5 chore(deps): bump the security group across 1 directory with 3 updates (#44029)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 17:04:39 -07:00
Joe LiandVitor Avila c510946df5 fix: BigQuery pivot-table Custom SQL row label with a space drops from GROUPING SETS (#43720)
Co-authored-by: Vitor Avila <vitor.avila@preset.io>
2026-09-08 16:58:49 -07:00
Joe LiandClaude Sonnet 5 91f4d01ca2 fix(chart-creation): stop antd v6 Steps from doubling the icon-content gap (#43730)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:57:55 -07:00
Evan Rusackas 3828dcb922 fix(security): suppress zizmor self-repository false positive for cached-dependencies in presto-hive workflow (#43976) 2026-09-08 16:53:30 -07:00
Evan Rusackas 362d3509cb fix(security): suppress zizmor self-repository false positive for cached-dependencies in presto-hive workflow (#43984) 2026-09-08 16:53:23 -07:00
Evan RusackasandClaude Sonnet 5 fa31e9c1c3 fix(security): suppress zizmor self-repository false positive for cached-dependencies (#43972)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:52:42 -07:00
Evan RusackasandClaude Sonnet 5 335a2a78ea fix(dashboard): expand clipped scrollable content before screenshot/PDF export (#43979)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:46:34 -07:00
Evan RusackasandClaude Sonnet 5 3564baabf6 fix(security): suppress zizmor self-repository false positive for setup-backend step (#44018)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:41:46 -07:00
36 changed files with 1222 additions and 363 deletions
@@ -160,7 +160,7 @@ jobs:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: ./.github/actions/setup-backend/
uses: $/.github/actions/setup-backend/
with:
python-version: ${{ matrix.python-version }}
- name: Setup Postgres
@@ -169,13 +169,7 @@ jobs:
run: |
setup-postgres
- name: Start Celery worker
# cached-dependencies is a git submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's gitlink. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python integration tests (PostgreSQL)
@@ -216,7 +210,7 @@ jobs:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: ./.github/actions/setup-backend/
uses: $/.github/actions/setup-backend/
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
@@ -82,7 +82,13 @@ jobs:
- name: Setup Python
uses: $/.github/actions/setup-backend/
- name: Setup Postgres
uses: ./.github/actions/cached-dependencies
# cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with:
run: setup-postgres
- name: Start Celery worker
@@ -147,7 +153,13 @@ jobs:
with:
run: setup-postgres
- name: Start Celery worker
uses: ./.github/actions/cached-dependencies
# cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with:
run: celery-worker
- name: Python unit tests (PostgreSQL)
+1 -1
View File
@@ -51,7 +51,7 @@ jobs:
# checkout, so it can't see into a submodule's gitlink. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with:
run: npm-install
- name: lint
+9 -9
View File
@@ -6924,9 +6924,9 @@ color-name@~1.1.4:
integrity sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==
colord@^2.9.3:
version "2.9.3"
resolved "https://registry.yarnpkg.com/colord/-/colord-2.9.3.tgz#4f8ce919de456f1d5c1c368c307fe20f3e59fb43"
integrity sha512-jeC1axXpnb0/2nn/Y1LPuLdgXBLH7aDcHu4KEKfqw3CUhX7ZpfBSlPKyqXE6btIgEzfWtrX3/tyBCaCvXvMkOw==
version "2.10.0"
resolved "https://registry.yarnpkg.com/colord/-/colord-2.10.0.tgz#56c9050e6b06b4b6c62ddec366a48d65ef57e860"
integrity sha512-AidJptpBJmjTclAp9BkLwJi0T93fo5epJnbaZslpg6QVzpHjAiveF55mE9AcUJiGMqRHgMDY8soMsQtuNYMHfw==
colorette@^1.2.0:
version "1.4.0"
@@ -9646,9 +9646,9 @@ jiti@^1.20.0:
integrity sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A==
joi@^17.9.2:
version "17.13.4"
resolved "https://registry.yarnpkg.com/joi/-/joi-17.13.4.tgz#ad6153d97ce558eb3a3b593e0d43eab51df1c474"
integrity sha512-1RuuER6kmt8K8I3nIWvPZKi5RQCb568ZPyY4Pwjlua+yo+63ZTmIwxLZH0heBmiKN4uxjvCiarDrjaeH84xicQ==
version "17.13.7"
resolved "https://registry.yarnpkg.com/joi/-/joi-17.13.7.tgz#92e212c50dbbbcb1a1592424f84083eb265cc778"
integrity sha512-MF80Dm5Y2veNy8QWVx9Bj3ui4mo7+VPSPsR1M+oaHXV0Gx6zGX9a2F+OZG3Blby9tOlzU9Rs5FUimlEhbKtfnQ==
dependencies:
"@hapi/hoek" "^9.3.0"
"@hapi/topo" "^5.1.0"
@@ -14184,9 +14184,9 @@ svg-parser@^2.0.4:
integrity sha512-e4hG1hRwoOdRb37cIMSgzNsxyzKfayW6VOflrwvR+/bzrkyxY/31WkbgnQpgtrNp1SdpJvpUAGTa/ZoiPNDuRQ==
svgo@^3.0.2, svgo@^3.2.0:
version "3.3.4"
resolved "https://registry.yarnpkg.com/svgo/-/svgo-3.3.4.tgz#fd2aa10ff585b3bd2b83ce3602f5582bc0718bb5"
integrity sha512-GsNRis4e8jxn2Y9ENz/8lbJ93CstG8svtMnuRaHbiF2LTJ5tK0/q3t/URPq9Zc7zVWBJnNnJMIp6bevK7bSmNg==
version "3.3.5"
resolved "https://registry.yarnpkg.com/svgo/-/svgo-3.3.5.tgz#8a3d9557ab2f386eca7e24760385849554985a1c"
integrity sha512-8SQMzdrvWaD8deUmrnYB+ASyxBVgWUOilg+A75nE/76WdLpj6LopCwiAVvkzkcqy/9b7t2Mg7faFLjg0ZRcZ3w==
dependencies:
commander "^7.2.0"
css-select "^5.1.0"
+1 -1
View File
@@ -54,7 +54,7 @@ dependencies = [
"deprecation>=2.1.0, <2.2.0",
"flask>=2.2.5, <4.0.0",
"flask-appbuilder>=5.2.2, <6.0.0",
"flask-caching>=2.4.1, <3",
"flask-caching>=2.5.0, <3",
"flask-compress>=1.13, <2.0",
"flask-talisman>=1.0.0, <2.0",
"flask-login>=0.6.0, < 1.0",
+5 -4
View File
@@ -40,7 +40,7 @@ brotli==1.2.0
# via
# -r requirements/base.in
# flask-compress
cachelib==0.13.0
cachelib==0.17.0
# via
# flask-caching
# flask-session
@@ -105,7 +105,7 @@ et-xmlfile==2.0.0
# via openpyxl
filelock==3.20.3
# via -r requirements/base.in
flask==2.3.3
flask==3.1.3
# via
# apache-superset (pyproject.toml)
# flask-appbuilder
@@ -124,9 +124,9 @@ flask-appbuilder==5.2.2
# via
# apache-superset (pyproject.toml)
# apache-superset-core
flask-babel==3.1.0
flask-babel==4.0.0
# via flask-appbuilder
flask-caching==2.4.1
flask-caching==2.5.0
# via apache-superset (pyproject.toml)
flask-compress==1.24
# via apache-superset (pyproject.toml)
@@ -218,6 +218,7 @@ markdown-it-py==3.0.0
# via rich
markupsafe==3.0.2
# via
# flask
# jinja2
# mako
# werkzeug
+5 -4
View File
@@ -94,7 +94,7 @@ brotli==1.2.0
# via
# -c requirements/base-constraint.txt
# flask-compress
cachelib==0.13.0
cachelib==0.17.0
# via
# -c requirements/base-constraint.txt
# flask-caching
@@ -247,7 +247,7 @@ filelock==3.20.3
# via
# -c requirements/base-constraint.txt
# virtualenv
flask==2.3.3
flask==3.1.3
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -269,11 +269,11 @@ flask-appbuilder==5.2.2
# -c requirements/base-constraint.txt
# apache-superset
# apache-superset-core
flask-babel==3.1.0
flask-babel==4.0.0
# via
# -c requirements/base-constraint.txt
# flask-appbuilder
flask-caching==2.4.1
flask-caching==2.5.0
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -524,6 +524,7 @@ markdown-it-py==3.0.0
markupsafe==3.0.2
# via
# -c requirements/base-constraint.txt
# flask
# jinja2
# mako
# werkzeug
+89 -114
View File
@@ -8007,11 +8007,12 @@
}
},
"node_modules/@mapbox/jsonlint-lines-primitives": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/@mapbox/jsonlint-lines-primitives/-/jsonlint-lines-primitives-2.0.2.tgz",
"integrity": "sha512-rY0o9A5ECsTQRVhv7tL/OyDpGAoUB4tTvLiW1DSzQGq4bvTPhNw1VpSNjDJc5GFZ2XuyOtSWSVN05qOtcD71qQ==",
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/@mapbox/jsonlint-lines-primitives/-/jsonlint-lines-primitives-2.0.3.tgz",
"integrity": "sha512-0SElaV0uMxEnxzBhhX9WTuPyUeMsAN/SS0i16tjuba4/mio63MG9khjC1a0JAiPGXAwvwm4UfHJURCN7nyudQg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
"node": ">= 22"
}
},
"node_modules/@mapbox/martini": {
@@ -8027,9 +8028,9 @@
"license": "ISC"
},
"node_modules/@mapbox/tiny-sdf": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/@mapbox/tiny-sdf/-/tiny-sdf-2.1.0.tgz",
"integrity": "sha512-uFJhNh36BR4OCuWIEiWaEix9CA2WzT6CAIcqVjWYpnx8+QDtS+oC4QehRrx5cX4mgWs37MmKnwUejeHxVymzNg==",
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/@mapbox/tiny-sdf/-/tiny-sdf-2.2.0.tgz",
"integrity": "sha512-LVL4wgI9YAum5V+LNVQO6QgFBPw7/MIIY4XJPNsPDMrjEwcE+JfKk1LuIl8GnF197ejVdC9QdPaxrx5gfgdGXg==",
"license": "BSD-2-Clause"
},
"node_modules/@mapbox/unitbezier": {
@@ -8047,29 +8048,23 @@
"@mapbox/point-geometry": "~0.1.0"
}
},
"node_modules/@mapbox/whoots-js": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/@mapbox/whoots-js/-/whoots-js-3.1.0.tgz",
"integrity": "sha512-Es6WcD0nO5l+2BOQS4uLfNPYQaNDfbot3X1XUoloz+x0mPDS3eeORZJl06HXjwBG1fOGwCRnzK88LMdxKRrd6Q==",
"license": "ISC",
"engines": {
"node": ">=6.0.0"
}
},
"node_modules/@maplibre/geojson-vt": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-5.0.4.tgz",
"integrity": "sha512-KGg9sma45S+stfH9vPCJk1J0lSDLWZgCT9Y8u8qWZJyjFlP8MNP1WGTxIMYJZjDvVT3PDn05kN1C95Sut1HpgQ==",
"license": "ISC"
},
"node_modules/@maplibre/maplibre-gl-style-spec": {
"version": "24.8.5",
"resolved": "https://registry.npmjs.org/@maplibre/maplibre-gl-style-spec/-/maplibre-gl-style-spec-24.8.5.tgz",
"integrity": "sha512-EzEJmMt6thioRH7GI9LWS7ahXTcAhAPGWCe6oTP2Ps4YnsXOOAfeqx854lZaiDnwURfHmcCKV1mr6oo0i23x6w==",
"version": "6.1.1",
"resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-6.1.1.tgz",
"integrity": "sha512-FVMOcmSP/yqol45t7StApEyTL5/vmqBCuFhH9n+fFuINenhaX+YgHHIt1yJ86S8kln3uJLcMvmEU2cfn6E2eCQ==",
"license": "ISC",
"dependencies": {
"@mapbox/jsonlint-lines-primitives": "~2.0.2",
"@mapbox/unitbezier": "^0.0.1",
"kdbush": "^4.1.0"
}
},
"node_modules/@maplibre/maplibre-gl-style-spec": {
"version": "26.4.2",
"resolved": "https://registry.npmjs.org/@maplibre/maplibre-gl-style-spec/-/maplibre-gl-style-spec-26.4.2.tgz",
"integrity": "sha512-6J0vZqMZvRAJKtdWJdDGHEh1YJ2ZHG08/GOur8gCArYhO8ZkM//OXqtI2AAEz4jc2G+Wq4MfcePg8qNK5TZ4Kg==",
"license": "ISC",
"dependencies": {
"@mapbox/jsonlint-lines-primitives": "^2.0.3",
"@mapbox/unitbezier": "^1.0.0",
"json-stringify-pretty-compact": "^4.0.0",
"minimist": "^1.2.8",
"quickselect": "^3.0.0",
@@ -8081,10 +8076,16 @@
"gl-style-validate": "dist/gl-style-validate.mjs"
}
},
"node_modules/@maplibre/maplibre-gl-style-spec/node_modules/@mapbox/unitbezier": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/unitbezier/-/unitbezier-1.0.0.tgz",
"integrity": "sha512-fqd515fjBmANKGGsQ286E2Wvj/XvDFpGzwJxq4CI6jMQue6Oy04uCKp+JWKF00xRTmk6cEu1jPJ9p3xqH8YWqQ==",
"license": "BSD-2-Clause"
},
"node_modules/@maplibre/mlt": {
"version": "1.1.8",
"resolved": "https://registry.npmjs.org/@maplibre/mlt/-/mlt-1.1.8.tgz",
"integrity": "sha512-8vtfYGidr1rNkv5IwIoU2lfe3Oy+Wa8HluzQYcQi9cveU9K3pweAal/poQj4GJ0K/EW4bTQp2wVAs09g2yDRZg==",
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@maplibre/mlt/-/mlt-1.2.1.tgz",
"integrity": "sha512-5n5dgolE2EYxwCKgx8vlwURCB8A+kyfJJyThlYimjlGgOcOe2Bhw9VxxnnHC91OC9PgHg+nVdgVPTYPkIo62vg==",
"license": "(MIT OR Apache-2.0)",
"dependencies": {
"@mapbox/point-geometry": "^1.1.0"
@@ -8097,18 +8098,14 @@
"license": "ISC"
},
"node_modules/@maplibre/vt-pbf": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/@maplibre/vt-pbf/-/vt-pbf-4.3.0.tgz",
"integrity": "sha512-jIvp8F5hQCcreqOOpEt42TJMUlsrEcpf/kI1T2v85YrQRV6PPXUcEXUg5karKtH6oh47XJZ4kHu56pUkOuqA7w==",
"version": "4.3.2",
"resolved": "https://registry.npmjs.org/@maplibre/vt-pbf/-/vt-pbf-4.3.2.tgz",
"integrity": "sha512-j6p0AdjvAR19Z3XaCysle7A4ZSo08tYOzxD0Y9NQylwPAkwJJeYub5b2eVucdeDh7erhv69DahoLOevDRERRUw==",
"license": "MIT",
"dependencies": {
"@mapbox/point-geometry": "^1.1.0",
"@mapbox/vector-tile": "^2.0.4",
"@maplibre/geojson-vt": "^5.0.4",
"@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3",
"pbf": "^4.0.1",
"supercluster": "^8.0.1"
"pbf": "^5.1.0"
}
},
"node_modules/@maplibre/vt-pbf/node_modules/@mapbox/point-geometry": {
@@ -8117,21 +8114,10 @@
"integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==",
"license": "ISC"
},
"node_modules/@maplibre/vt-pbf/node_modules/@mapbox/vector-tile": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-2.0.5.tgz",
"integrity": "sha512-pXj8m7KTsqZt+1jsE0xIpGvqTSbblfkuEJL/NJmNePMtEwxO8V3XMDo9WMSfDeqHvCtBI9Lmt4mGcGR10zecmw==",
"license": "BSD-3-Clause",
"dependencies": {
"@mapbox/point-geometry": "~1.1.0",
"@types/geojson": "^7946.0.16",
"pbf": "^4.0.2"
}
},
"node_modules/@maplibre/vt-pbf/node_modules/pbf": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-4.0.2.tgz",
"integrity": "sha512-J0ajxARhZfpUEebxYs1vhMGMuLSXtBe1e+fFPDrf2uA2hgo+UshKfNUWOz92HJNz6/NFEXseQPddnHkTreWRqg==",
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-5.1.2.tgz",
"integrity": "sha512-mnvGdvOrIvJOBGUEdGkrVXjN8E/VkIJCkf2eS1DH2yv82ORUlLttmDt0rWY38yYZmVwciZwBUvHM20qxBZf40w==",
"license": "BSD-3-Clause",
"dependencies": {
"resolve-protobuf-schema": "^2.1.0"
@@ -28781,9 +28767,9 @@
}
},
"node_modules/jest-process-manager/node_modules/joi": {
"version": "17.13.4",
"resolved": "https://registry.npmjs.org/joi/-/joi-17.13.4.tgz",
"integrity": "sha512-1RuuER6kmt8K8I3nIWvPZKi5RQCb568ZPyY4Pwjlua+yo+63ZTmIwxLZH0heBmiKN4uxjvCiarDrjaeH84xicQ==",
"version": "17.13.7",
"resolved": "https://registry.npmjs.org/joi/-/joi-17.13.7.tgz",
"integrity": "sha512-MF80Dm5Y2veNy8QWVx9Bj3ui4mo7+VPSPsR1M+oaHXV0Gx6zGX9a2F+OZG3Blby9tOlzU9Rs5FUimlEhbKtfnQ==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
@@ -30783,9 +30769,9 @@
}
},
"node_modules/joi": {
"version": "18.2.3",
"resolved": "https://registry.npmjs.org/joi/-/joi-18.2.3.tgz",
"integrity": "sha512-N5A3KTWQpPWT4ExxxPlUx7WmykGXRzhNidWhV41d6Abu9YfI2NyWCJuxdPnslJCPWtbRpSVOWSnSS6GakLM/Rg==",
"version": "18.2.8",
"resolved": "https://registry.npmjs.org/joi/-/joi-18.2.8.tgz",
"integrity": "sha512-G2TX62h58ZHuwqetJgP2F4ualakqAmZtBYe3jWen7gxQRw5xApX6crnFtuB91WC0c3ESBnva+kGSnb3+6pIQDQ==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
@@ -32288,7 +32274,8 @@
"version": "4.4.2",
"resolved": "https://registry.npmjs.org/lodash.get/-/lodash.get-4.4.2.tgz",
"integrity": "sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==",
"license": "MIT"
"license": "MIT",
"peer": true
},
"node_modules/lodash.isempty": {
"version": "4.4.0",
@@ -32300,7 +32287,8 @@
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/lodash.isequal/-/lodash.isequal-4.5.0.tgz",
"integrity": "sha512-pDo3lu8Jhfjqls6GkMgpahsF9kCyayhgykjyLMNFTKWrpVdAQtYyB4muAMWozBB4ig/dtWAmsMxLEI8wuz+DYQ==",
"license": "MIT"
"license": "MIT",
"peer": true
},
"node_modules/lodash.isplainobject": {
"version": "4.0.6",
@@ -32575,27 +32563,25 @@
]
},
"node_modules/maplibre-gl": {
"version": "5.24.0",
"resolved": "https://registry.npmjs.org/maplibre-gl/-/maplibre-gl-5.24.0.tgz",
"integrity": "sha512-ALyFxgtd5R+65UqZ/++lOqwWcC0SNho9c27fYSyLmG7AfnAul2o46F05aDJGPbFU57wos9dgcIySHs0Xe6ia3A==",
"version": "6.8.0",
"resolved": "https://registry.npmjs.org/maplibre-gl/-/maplibre-gl-6.8.0.tgz",
"integrity": "sha512-+ZkjKTodsVLY0ewQThvXRxXQsclcsSgOm5LlnrBM3G8AloMJKt6Haw8mY4xrOl8T0WMqH6DTYjktZ9zGQXZd4w==",
"license": "BSD-3-Clause",
"dependencies": {
"@mapbox/jsonlint-lines-primitives": "^2.0.2",
"@mapbox/point-geometry": "^1.1.0",
"@mapbox/tiny-sdf": "^2.1.0",
"@mapbox/unitbezier": "^0.0.1",
"@mapbox/vector-tile": "^2.0.4",
"@mapbox/whoots-js": "^3.1.0",
"@maplibre/geojson-vt": "^6.1.0",
"@maplibre/maplibre-gl-style-spec": "^24.8.1",
"@maplibre/mlt": "^1.1.8",
"@maplibre/vt-pbf": "^4.3.0",
"@mapbox/tiny-sdf": "^2.2.0",
"@mapbox/unitbezier": "^1.0.0",
"@mapbox/vector-tile": "^3.0.0",
"@maplibre/geojson-vt": "^6.1.1",
"@maplibre/maplibre-gl-style-spec": "^26.4.1",
"@maplibre/mlt": "^1.2.1",
"@maplibre/vt-pbf": "^4.3.2",
"@types/geojson": "^7946.0.16",
"earcut": "^3.0.2",
"earcut": "^3.2.3",
"gl-matrix": "^3.4.4",
"kdbush": "^4.0.2",
"kdbush": "^4.1.0",
"murmurhash-js": "^1.0.0",
"pbf": "^4.0.1",
"pbf": "^5.1.2",
"potpack": "^2.1.0",
"quickselect": "^3.0.0",
"tinyqueue": "^3.0.0"
@@ -32614,36 +32600,33 @@
"integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==",
"license": "ISC"
},
"node_modules/maplibre-gl/node_modules/@mapbox/unitbezier": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/unitbezier/-/unitbezier-1.0.0.tgz",
"integrity": "sha512-fqd515fjBmANKGGsQ286E2Wvj/XvDFpGzwJxq4CI6jMQue6Oy04uCKp+JWKF00xRTmk6cEu1jPJ9p3xqH8YWqQ==",
"license": "BSD-2-Clause"
},
"node_modules/maplibre-gl/node_modules/@mapbox/vector-tile": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-2.0.5.tgz",
"integrity": "sha512-pXj8m7KTsqZt+1jsE0xIpGvqTSbblfkuEJL/NJmNePMtEwxO8V3XMDo9WMSfDeqHvCtBI9Lmt4mGcGR10zecmw==",
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-3.0.0.tgz",
"integrity": "sha512-Qf10S1uIHMk20ri/IVBnpS+esUEkVaR5Hftmz88jTInrpmWgPGJfPe3LVjjlE77trLx8tH6qjTG7uWH9hIq/0Q==",
"license": "BSD-3-Clause",
"dependencies": {
"@mapbox/point-geometry": "~1.1.0",
"@types/geojson": "^7946.0.16",
"pbf": "^4.0.2"
}
},
"node_modules/maplibre-gl/node_modules/@maplibre/geojson-vt": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-6.1.0.tgz",
"integrity": "sha512-2eIY4gZxeKIVOZVNkAMb+5NgXhgsMQpOveTQAvnp53LYqHGJZDidk7Ew0Tged9PThidpbS+NFTh0g4zivhPDzQ==",
"license": "ISC",
"dependencies": {
"kdbush": "^4.0.2"
"pbf": "^5.0.0"
}
},
"node_modules/maplibre-gl/node_modules/earcut": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/earcut/-/earcut-3.0.2.tgz",
"integrity": "sha512-X7hshQbLyMJ/3RPhyObLARM2sNxxmRALLKx1+NVFFnQ9gKzmCrxm9+uLIAdBcvc8FNLpctqlQ2V6AE92Ol9UDQ==",
"version": "3.2.3",
"resolved": "https://registry.npmjs.org/earcut/-/earcut-3.2.3.tgz",
"integrity": "sha512-vnS4AVwp1KHAF13i1vp1/2D5evWy3k5u/iW/B81QVsUZtV8cv2tU0b2VNFlqvh4kYwrFMDdjPCfAmfyJW9y14Q==",
"license": "ISC"
},
"node_modules/maplibre-gl/node_modules/pbf": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-4.0.2.tgz",
"integrity": "sha512-J0ajxARhZfpUEebxYs1vhMGMuLSXtBe1e+fFPDrf2uA2hgo+UshKfNUWOz92HJNz6/NFEXseQPddnHkTreWRqg==",
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-5.1.2.tgz",
"integrity": "sha512-mnvGdvOrIvJOBGUEdGkrVXjN8E/VkIJCkf2eS1DH2yv82ORUlLttmDt0rWY38yYZmVwciZwBUvHM20qxBZf40w==",
"license": "BSD-3-Clause",
"dependencies": {
"resolve-protobuf-schema": "^2.1.0"
@@ -42614,15 +42597,6 @@
"integrity": "sha512-Orov6g6BB1sDfYgzWfTHDOxamtX1bE/zo104Dh9e6fqJ3PooipYyfJ0pUmrZO2wAvO8YbEyeFrkV91XTsGMSrw==",
"license": "MIT"
},
"node_modules/supercluster": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/supercluster/-/supercluster-8.0.1.tgz",
"integrity": "sha512-IiOea5kJ9iqzD2t7QJq/cREyLHTtSmUT6gQsweojg9WH2sYJqZK9SswTu6jrscO6D1G5v5vYZ9ru/eq85lXeZQ==",
"license": "ISC",
"dependencies": {
"kdbush": "^4.0.2"
}
},
"node_modules/supports-color": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
@@ -42715,9 +42689,9 @@
"dev": true
},
"node_modules/svgo": {
"version": "3.3.4",
"resolved": "https://registry.npmjs.org/svgo/-/svgo-3.3.4.tgz",
"integrity": "sha512-GsNRis4e8jxn2Y9ENz/8lbJ93CstG8svtMnuRaHbiF2LTJ5tK0/q3t/URPq9Zc7zVWBJnNnJMIp6bevK7bSmNg==",
"version": "3.3.5",
"resolved": "https://registry.npmjs.org/svgo/-/svgo-3.3.5.tgz",
"integrity": "sha512-8SQMzdrvWaD8deUmrnYB+ASyxBVgWUOilg+A75nE/76WdLpj6LopCwiAVvkzkcqy/9b7t2Mg7faFLjg0ZRcZ3w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -47102,7 +47076,7 @@
"math-expression-evaluator": "^2.0.7",
"parse-ms": "^4.0.0",
"re-resizable": "^6.11.2",
"react-ace": "^14.0.1",
"react-ace": "^15.0.0",
"react-draggable": "^4.7.1",
"react-error-boundary": "^6.1.4",
"react-js-cron": "^6.0.2",
@@ -47203,15 +47177,14 @@
}
},
"packages/superset-ui-core/node_modules/react-ace": {
"version": "14.0.1",
"resolved": "https://registry.npmjs.org/react-ace/-/react-ace-14.0.1.tgz",
"integrity": "sha512-z6YAZ20PNf/FqmYEic//G/UK6uw0rn21g58ASgHJHl9rfE4nITQLqthr9rHMVQK4ezwohJbp2dGrZpkq979PYQ==",
"version": "15.0.0",
"resolved": "https://registry.npmjs.org/react-ace/-/react-ace-15.0.0.tgz",
"integrity": "sha512-gdmS5ftF0hsbkcrAjZQqYbXwFg5JrHuxjki8mP6Bn9kfa6lHKfZk9sU9EfS0ifQ1CpGCmRxF/VC7GRvlJMBuZw==",
"license": "MIT",
"dependencies": {
"ace-builds": "^1.36.3",
"diff-match-patch": "^1.0.5",
"lodash.get": "^4.4.2",
"lodash.isequal": "^4.5.0",
"fast-equals": "^5.3.3",
"prop-types": "^15.8.1"
},
"peerDependencies": {
@@ -47593,8 +47566,10 @@
"license": "Apache-2.0",
"dependencies": {
"@math.gl/web-mercator": "^4.1.0",
"@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3",
"mapbox-gl": "^3.29.0",
"maplibre-gl": "^5.24.0",
"maplibre-gl": "^6.8.0",
"react-map-gl": "^8.1.2",
"supercluster": "^9.0.0"
},
@@ -47737,7 +47712,7 @@
"handlebars": "^4.7.9",
"lodash": "^4.18.1",
"lodash-es": "^4.18.1",
"maplibre-gl": "^5.24.0",
"maplibre-gl": "^6.8.0",
"mousetrap": "^1.6.5",
"ngeohash": "^0.6.4",
"prop-types": "^15.8.1",
@@ -78,7 +78,7 @@
"math-expression-evaluator": "^2.0.7",
"parse-ms": "^4.0.0",
"re-resizable": "^6.11.2",
"react-ace": "^14.0.1",
"react-ace": "^15.0.0",
"react-draggable": "^4.7.1",
"react-error-boundary": "^6.1.4",
"react-js-cron": "^6.0.2",
@@ -604,6 +604,61 @@ test('cleans up event listeners on unmount', async () => {
offSpy.mockRestore();
});
test('re-applies annotations only when their content actually changes across renders (react-ace 15 fast-equals regression guard)', async () => {
// react-ace's componentDidUpdate decides whether to call
// session.setAnnotations() by deep-comparing the new/old `annotations`
// prop (lib/ace.js, using an internal deep-equality helper -- lodash's
// isEqual through react-ace 14.x, fast-equals's deepEqual from 15.0.0
// onward). Superset's own AceEditorProvider/EditorWrapper always pass a
// freshly `.map()`-derived annotations array on every render, so this
// guards the actual behavior Superset relies on: a same-content-but-
// different-reference array must NOT re-trigger setAnnotations (or the
// editor would thrash on every keystroke-driven re-render), while a
// genuinely different array must still update the editor.
const ref = createRef<AceEditor>();
const annotationsV1 = [{ row: 0, column: 0, type: 'error', text: 'oops' }];
const { rerender, container } = render(
<SQLEditor ref={ref as React.Ref<never>} annotations={annotationsV1} />,
);
await waitFor(() => {
expect(container.querySelector(selector)).toBeInTheDocument();
});
const session = ref.current?.editor?.getSession();
expect(session).toBeDefined();
if (!session) return;
// The initial mount already applies annotations via componentDidMount,
// not componentDidUpdate, so start observing only from the first update.
const setAnnotationsSpy = jest.spyOn(session, 'setAnnotations');
// Same content, new array/object references -- must be a no-op.
const annotationsV1SameContent = [
{ row: 0, column: 0, type: 'error', text: 'oops' },
];
rerender(
<SQLEditor
ref={ref as React.Ref<never>}
annotations={annotationsV1SameContent}
/>,
);
expect(setAnnotationsSpy).not.toHaveBeenCalled();
// Genuinely different content -- must update, with the new value.
const annotationsV2 = [
{ row: 1, column: 2, type: 'warning', text: 'different' },
];
rerender(
<SQLEditor ref={ref as React.Ref<never>} annotations={annotationsV2} />,
);
expect(setAnnotationsSpy).toHaveBeenCalledTimes(1);
expect(setAnnotationsSpy).toHaveBeenCalledWith(annotationsV2);
setAnnotationsSpy.mockRestore();
});
test('does not move autocomplete popup if target container is document.body', async () => {
const ref = createRef<AceEditor>();
const { container } = render(<SQLEditor ref={ref as React.Ref<never>} />);
@@ -30,8 +30,10 @@
},
"dependencies": {
"@math.gl/web-mercator": "^4.1.0",
"@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3",
"mapbox-gl": "^3.29.0",
"maplibre-gl": "^5.24.0",
"maplibre-gl": "^6.8.0",
"react-map-gl": "^8.1.2",
"supercluster": "^9.0.0"
},
@@ -49,7 +49,7 @@
"handlebars": "^4.7.9",
"lodash": "^4.18.1",
"lodash-es": "^4.18.1",
"maplibre-gl": "^5.24.0",
"maplibre-gl": "^6.8.0",
"mousetrap": "^1.6.5",
"ngeohash": "^0.6.4",
"prop-types": "^15.8.1",
@@ -16,7 +16,13 @@
* specific language governing permissions and limitations
* under the License.
*/
import { Preset, VizType } from '@superset-ui/core';
import {
ChartLabel,
ChartMetadata,
ChartPlugin,
Preset,
VizType,
} from '@superset-ui/core';
import {
render,
cleanup,
@@ -47,12 +53,31 @@ jest.mock('scroll-into-view-if-needed', () => jest.fn());
jest.useFakeTimers({ advanceTimers: true });
// A minimal plugin carrying a "Featured" label, so tests can assert on the
// badge that VizTypeGallery overlays on its thumbnail.
class FeaturedTestChartPlugin extends ChartPlugin {
constructor() {
super({
metadata: new ChartMetadata({
name: 'Featured Test Chart',
thumbnail: '',
label: ChartLabel.Featured,
tags: ['Featured'],
}),
Chart: () => null,
});
}
}
class MainPreset extends Preset {
constructor() {
super({
name: 'Legacy charts',
plugins: [
new TableChartPlugin().configure({ key: VizType.Table }),
new FeaturedTestChartPlugin().configure({
key: 'featured_test_chart',
}),
new BigNumberTotalChartPlugin().configure({
key: VizType.BigNumberTotal,
}),
@@ -278,6 +303,25 @@ describe('VizTypeControl', () => {
).not.toBeInTheDocument();
});
test('anchors the Featured badge to the bottom-right of the thumbnail image', async () => {
// The badge is positioned relative to the thumbnail image only (not the
// whole tile), so it must hang off the image's bottom-right corner
// rather than its top edge.
await waitForRenderWrapper();
userEvent.click(screen.getByRole('tab', { name: 'All charts' }));
const visualizations = screen.getByTestId(getTestId('viz-row'));
const image = await within(visualizations).findByAltText(
'Featured Test Chart',
);
const badgeWrapper = image.nextElementSibling as HTMLElement;
expect(badgeWrapper).toHaveStyleRule('bottom', '4px');
expect(badgeWrapper).toHaveStyleRule('right', '4px');
expect(badgeWrapper).not.toHaveStyleRule('top', expect.anything());
expect(within(badgeWrapper).getByText('FEATURED')).toBeInTheDocument();
});
test('Thumbnail labels expose the full chart name via a title tooltip', async () => {
// Labels are clamped to a fixed two-line block so every tile is the same
// height; the full (possibly truncated) name must stay discoverable through
@@ -344,7 +344,7 @@ const ThumbnailImageWrapper = styled.div`
const ThumbnailLabelWrapper = styled.div`
position: absolute;
right: ${({ theme }) => theme.sizeUnit}px;
top: ${({ theme }) => theme.sizeUnit}px;
bottom: ${({ theme }) => theme.sizeUnit}px;
`;
const TitleLabelWrapper = styled.div`
@@ -153,6 +153,20 @@ test('renders a select and a VizTypeGallery', async () => {
expect(screen.getByText(/choose chart type/i)).toBeInTheDocument();
});
test('does not double up the vertical Steps icon-to-content gap', async () => {
// antd 6 added its own icon->content gap on `.ant-steps-item-wrapper`
// (column-gap), stacking on top of the pre-existing `margin-right` on
// `.ant-steps-item-icon` and shifting every step's content to the right.
const { container } = await renderComponent();
const styledContainer = container.firstChild;
expect(styledContainer).toHaveStyleRule('column-gap', '0', {
target: '.ant-steps-item-wrapper',
});
expect(styledContainer).toHaveStyleRule('margin-right', '8px', {
target: '.ant-steps-item-icon',
});
});
test('renders dataset help text when user lacks dataset write permissions', async () => {
await renderComponent();
expect(screen.queryByText('Add a dataset')).not.toBeInTheDocument();
@@ -141,6 +141,13 @@ const StyledContainer = styled.div`
display: none;
}
/* antd 6 added its own icon->content gap on this flex wrapper
(column-gap), on top of the .ant-steps-item-icon margin-right below,
doubling the gap. Zero it out so the icon's margin is the only gap. */
&&&& .ant-steps-item-wrapper {
column-gap: 0;
}
&&&& .ant-steps-item-icon {
margin-right: ${theme.marginXS}px;
width: ${theme.sizeUnit * 5}px;
+2 -1
View File
@@ -44,8 +44,9 @@ class SupersetMetastoreCache(BaseCache):
namespace: UUID,
codec: KeyValueCodec,
default_timeout: int = 300,
ignore_delete_many_errors: bool = False,
) -> None:
super().__init__(default_timeout)
super().__init__(default_timeout, ignore_delete_many_errors)
self.namespace = namespace
self.codec = codec
+25 -5
View File
@@ -80,7 +80,6 @@ from superset import db, is_feature_enabled
from superset.advanced_data_type.types import AdvancedDataTypeResponse
from superset.common.db_query_status import QueryStatus
from superset.common.grouping_sets import (
grouping_id_column,
grouping_marker_label,
grouping_sets_clause,
)
@@ -4867,10 +4866,31 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
and groupby_all_columns
and db_engine_spec.supports_grouping_sets
)
# Both the GROUPING() marker labels and the `grouping_sets` level
# definitions sent by the frontend (see buildQuery.ts) are expressed in
# terms of the column's logical/requested label (``.key``), not the
# engine-mutated SQL alias (``.name``). BigQuery, for example, mangles
# labels containing spaces (e.g. a Custom SQL column named "Test Row")
# into something like "Test_Row_a1b2c3" for `.name`, while `.key` keeps
# the original "Test Row". Keying by `.name` here would silently drop
# such columns from every rollup level (the `col in ...` guard below),
# producing an invalid ``GROUP BY GROUPING SETS`` clause that omits a
# selected, non-aggregated column.
groupby_columns_by_label = {
gby_expr.key: gby_expr for gby_expr in groupby_all_columns.values()
}
if use_grouping_sets:
# Route the marker through `make_sqla_column_compatible` like every
# other selected column: the SQL-level alias is engine-mutated if
# required (e.g. BigQuery rejects aliases with spaces), while
# `.key` keeps the unmutated marker label so it lines up with the
# `groupby_columns_by_label` keys above and with what the frontend
# looks for when splitting the combined result back per level.
select_exprs = select_exprs + [
grouping_id_column(gby_expr, grouping_marker_label(name))
for name, gby_expr in groupby_all_columns.items()
self.make_sqla_column_compatible(
sa.func.grouping(gby_expr), grouping_marker_label(label)
)
for label, gby_expr in groupby_columns_by_label.items()
]
# Expected output columns
@@ -4887,9 +4907,9 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
if use_grouping_sets:
gs_levels = [
[
groupby_all_columns[col]
groupby_columns_by_label[col]
for col in level
if col in groupby_all_columns
if col in groupby_columns_by_label
]
for level in grouping_sets or []
]
+1 -1
View File
@@ -325,7 +325,7 @@ def etag_cache( # noqa: C901
wrapper.uncached = f # type: ignore
wrapper.cache_timeout = timeout # type: ignore
wrapper.make_cache_key = cache._memoize_make_cache_key( # type: ignore # pylint: disable=protected-access
make_name=None, timeout=timeout, hash_method=configurable_hash_method
make_name=None, hash_method=configurable_hash_method
)
return wrapper
-4
View File
@@ -147,16 +147,12 @@ class SupersetCache(Cache):
def _memoize_make_cache_key(
self,
make_name: Callable[..., Any] | None = None,
timeout: Callable[..., Any] | None = None,
forced_update: bool = False,
hash_method: Callable[..., Any] = configurable_hash_method,
source_check: bool | None = False,
args_to_ignore: Any | None = None,
) -> Callable[..., Any]:
return super()._memoize_make_cache_key(
make_name=make_name,
timeout=timeout,
forced_update=forced_update,
hash_method=hash_method,
source_check=source_check,
args_to_ignore=args_to_ignore,
+97
View File
@@ -35,6 +35,10 @@ logger = logging.getLogger(__name__)
# Time to wait after scrolling for content to settle and load (in milliseconds)
SCROLL_SETTLE_TIMEOUT_MS = 1000
# Ceiling for un-clipping scrollable chart content (ag-Grid stabilization
# polling) before a screenshot, absent a report deadline to bound it against.
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS = 5.0
# Chromium can occasionally return a valid but uniformly blank PNG for an
# off-screen clip. Retry after forcing a compositor frame, but keep each CDP
# capture bounded so a wedged compositor cannot consume the report deadline.
@@ -437,6 +441,99 @@ CHART_CONTAINER_STATE_JS = f"""
}}
"""
CHART_CONTAINER_SELECTOR = ".chart-container"
# `.slice_container` (superset-frontend/src/components/Chart/Chart.tsx) is
# the one ancestor every chart type shares, directly inside `.chart-container`,
# with an explicit pixel height matching the dashboard tile. A locator-bounded
# capture (`element.screenshot()`, used for single-chart exports) clips to
# `.chart-container`'s own bounding box, which only has a `min-height` --
# so it stays exactly `.slice_container`'s fixed height unless that fixed
# height is lifted too. Un-clipping a scrollable *descendant* (the ag-Grid
# host, a table's own scroll body) is not enough on its own: the descendant
# can grow, but its ancestor's box does not, and the extra content just
# overflows the ancestor unseen by a bounding-box screenshot (#38090).
SLICE_CONTAINER_SELECTOR_FOR_EXPANSION = ".slice_container"
# Legacy/other chart-table implementations that scroll via an inline style
# rather than a stable class name (e.g. plugin-chart-table's sticky body,
# `superset-frontend/plugins/plugin-chart-table/src/DataTable/hooks/useSticky.tsx`)
# aren't reachable by a fixed class-selector list, so this catches any
# descendant of a chart that is *actually* clipping its own content
# (scrollHeight > clientHeight) rather than guessing at class names that may
# not exist in every plugin version. `.ant-table-body` is kept alongside it
# for a real Ant Design `<Table>` if one ever renders inside a chart.
GENERIC_SCROLLABLE_DESCENDANT_SELECTOR = (
f'{CHART_CONTAINER_SELECTOR} [style*="overflow"], '
f"{CHART_CONTAINER_SELECTOR} .ant-table-body"
)
# ag-Grid virtualizes rows for performance, so a plain height/overflow reset
# would still leave off-screen rows unrendered. `domLayout: "print"` is
# ag-Grid's own "render every row into the DOM" mode -- the same mode the
# client-side "download as image" export switches to via the GridApi that
# ThemedAgGridReact (superset-ui-core) stashes on the grid's host element
# specifically so screenshot/export code can reach it. The grid's own host
# element and its immediate parent (the ag-Grid table plugin's container,
# which sets an explicit pixel height via inline style -- see
# `plugin-chart-ag-grid-table/src/AgGridTable/index.tsx`) are reset for the
# same ancestor-box reason as `.slice_container` above.
#
# `page.screenshot(full_page=True)` already expands the outer dashboard
# scroll to include every below-the-fold chart (#31158); it has no effect on
# a chart's own internal scroll container, which is what this JS unrolls
# in-place before the page is captured.
EXPAND_SCROLLABLE_CONTENT_JS = f"""
async (maxWaitMs) => {{
const agGrids = Array.from(
document.querySelectorAll('{AG_GRID_HOST_SELECTOR}')
);
await Promise.all(agGrids.map(async (grid) => {{
const api = grid._agGridApi;
if (!api) {{ return; }}
api.setGridOption('domLayout', 'print');
if (api.resetRowHeights) {{ api.resetRowHeights(); }}
grid.style.height = 'auto';
if (grid.parentElement) {{ grid.parentElement.style.height = 'auto'; }}
// ag-Grid's autoHeight rows batch-measure asynchronously, so this
// polls for a stable scrollHeight instead of a fixed sleep. Five
// consecutive unchanged 100ms polls is a deliberate match for the
// client-side export's own
// waitForStableScrollHeight(agRootWrapper, 5000, 5) (downloadAsImage.tsx):
// always paid in full even when nothing is still settling, so both
// paths trust the measurement after the same wait rather than
// racing a batch that hasn't finished yet.
let lastHeight = grid.scrollHeight;
let stableCount = 0;
const deadline = Date.now() + maxWaitMs;
while (Date.now() < deadline && stableCount < 5) {{
await new Promise((resolve) => setTimeout(resolve, 100));
const height = grid.scrollHeight;
if (height === lastHeight) {{
stableCount += 1;
}} else {{
stableCount = 0;
lastHeight = height;
}}
}}
}}));
document.querySelectorAll('{SLICE_CONTAINER_SELECTOR_FOR_EXPANSION}').forEach(
(el) => {{ el.style.height = 'auto'; }}
);
document.querySelectorAll('{GENERIC_SCROLLABLE_DESCENDANT_SELECTOR}').forEach(
(el) => {{
if (el.scrollHeight > el.clientHeight) {{
el.style.overflow = 'visible';
el.style.height = 'auto';
el.style.maxHeight = 'none';
}}
}}
);
}}
"""
def combine_screenshot_tiles(
screenshot_tiles: list[bytes],
+68
View File
@@ -34,6 +34,8 @@ from superset.utils.screenshot_utils import (
CHART_CONTAINER_READY_JS,
CHART_CONTAINER_STATE_JS,
CHART_HOLDERS_READY_JS,
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
FIND_ALL_UNREADY_CHART_HOLDERS_JS,
FIND_CHART_HOLDER_STATES_JS,
FORCE_ALL_CHART_HOLDERS_IN_VIEW_JS,
@@ -238,6 +240,43 @@ class WebDriverPlaywright(WebDriverProxy):
else:
return element.screenshot(**timeout_kwargs)
@staticmethod
def _expand_scrollable_content(
page: Page,
log_context: str | None = None,
report_execution_context: ReportExecutionContext | None = None,
) -> None:
"""
Un-clip chart content that is fully present in the DOM but visually
cropped by a fixed height + internal scrollbar (e.g. a table taller
than the space its dashboard tile gives it) before the page is
captured.
The ag-Grid branch of this step polls for a stable row count, so it
is bounded by the report's own deadline the same way every other
wait in this method is, rather than an unconditional fixed sleep.
Best-effort: a failure here should not abort the screenshot, since a
clipped-but-present capture beats none at all.
"""
max_wait_seconds = (
report_execution_context.deadline.timeout_seconds(
"scrollable_content_expansion",
requested_seconds=EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
reserve_seconds=report_execution_context.readiness_reserve_seconds,
)
if report_execution_context
else EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS
)
try:
page.evaluate(EXPAND_SCROLLABLE_CONTENT_JS, max_wait_seconds * 1000)
except PlaywrightError:
logger.warning(
"Failed to expand scrollable chart content before screenshot%s",
f" [{log_context}]" if log_context else "",
exc_info=True,
)
@staticmethod
def _wait_for_charts_ready( # noqa: C901
page: Page,
@@ -712,6 +751,19 @@ class WebDriverPlaywright(WebDriverProxy):
unexpected_errors,
context_suffix,
)
# Un-clip scrollable/virtualized chart content (dense tables
# taller than their dashboard tile) before measuring height,
# so the tiling decision below sees the full content when
# possible. A chart whose ag-Grid hasn't fired GridReady yet
# at this point is re-expanded below, after readiness --
# `.chart-container` elements attaching (waited on above) is
# not the same as ag-Grid finishing its own internal init.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
# Detect large dashboards and use tiled screenshots if enabled
tiled_enabled = app.config.get("SCREENSHOT_TILED_ENABLED", False)
@@ -865,6 +917,14 @@ class WebDriverPlaywright(WebDriverProxy):
screenshot_started_at=screenshot_started_at,
report_execution_context=report_execution_context,
)
# Re-run now that readiness has confirmed every chart
# actually rendered: a grid whose GridReady hadn't
# fired yet at the earlier call above is expanded here.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
if selenium_animation_wait > 0:
if report_execution_context:
selenium_animation_wait = min(
@@ -929,6 +989,14 @@ class WebDriverPlaywright(WebDriverProxy):
screenshot_started_at=screenshot_started_at,
report_execution_context=report_execution_context,
)
# Re-run now that readiness has confirmed every chart
# actually rendered: a grid whose GridReady hadn't fired
# yet at the earlier call above is expanded here.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
if selenium_animation_wait > 0:
if report_execution_context:
selenium_animation_wait = min(
+99 -53
View File
@@ -24,9 +24,11 @@ request:
* :func:`collect_impact_pairs` pulls the distinct
``(dataset_id, transaction_id)`` pairs that need counts.
* :func:`batch_chart_counts` one SQL query joining
``dashboard_slices_version`` and ``slices_version`` to count
the matching charts validity-strategy-style.
* :func:`batch_chart_counts` counts the matching charts without a
join: dashboard membership comes from ``charts_attached_to_dashboard``'s
attach/detach windows over ``dashboard_slices_version``, and a
member-scoped ``slices_version`` scan supplies the chartdataset window;
the two are combined per pair by :func:`_count_attached_charts_at`.
* :func:`impact_for_record` pure projection from the pre-fetched
counts onto each record (returns ``None`` for non-Dashboard paths
or non-SqlaTable kinds, matching the ``impact`` computation).
@@ -38,6 +40,7 @@ inside another (no DB).
from __future__ import annotations
from collections.abc import Mapping, Sequence
from typing import Any
import sqlalchemy as sa
@@ -47,7 +50,14 @@ from superset.versioning.activity.kinds import (
chunked_ids,
ENTITY_ID_CHUNK_SIZE,
TABLE_KIND_TO_API,
Window,
)
from superset.versioning.baseline import OPERATION_DELETE
# Headroom left below SQLite's 999 bind-variable floor for the handful of scalar
# binds in the slice-scan WHERE (datasource_type, operation_type, the two tx
# bounds) once a member-id chunk and the dataset IN are accounted for.
_SCALAR_BIND_HEADROOM = 20
def collect_impact_pairs(
@@ -76,12 +86,17 @@ def batch_chart_counts(
distinct charts that were both on *dashboard_id* and pointing at
*dataset_id* at *target_tx*.
One SELECT against ``dashboard_slices_version`` ``slices_version``,
pulling the (slice, dataset, validity-window) state for every slice
ever on the dashboard whose dataset matches one of the requested
dataset_ids. The Python loop then applies the validity-strategy
predicate per pair. Replaces the previous N+1 shape that fired one
COUNT per related record.
No join: ``charts_attached_to_dashboard`` supplies each member chart's
``[attach, detach)`` windows from the association shadow (Continuum never
closes an M2M shadow's ``end_transaction_id``, so a validity-window filter
on it would count a chart removed before ``target_tx`` sc-119907), and a
member-scoped scan of ``slices_version`` supplies the chartdataset window,
whose ``end_transaction_id`` the validity backfill *does* close, so the
ordinary validity predicate is right there. The Python loop counts a slice
for a pair when both an attachment window and its chartdataset window
contain ``target_tx``. Replaces the previous N+1 shape that fired one COUNT
per related record, and the m2mslices join whose M2M validity window was
the buggy naive filter.
Returns ``{(dataset_id, target_tx): count}``; pairs whose count
would be zero are omitted so the caller's ``.get(key, 0)`` is
@@ -94,73 +109,104 @@ def batch_chart_counts(
from sqlalchemy_continuum import version_class
from superset.models.slice import Slice
from superset.versioning.membership import charts_attached_to_dashboard
metadata = version_class(Slice).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
slices_tbl = version_class(Slice).__table__
if m2m_tbl is None:
return {}
dataset_ids: set[int] = {dataset_id for dataset_id, _ in pairs}
# Bound both validity windows to the transaction range the page-set
# needs. Both the attachment (m2m) and the chart→dataset (slice) window
# must straddle a requested target_tx, so a row whose window starts
# after the newest target, or closes at/before the oldest one, can
# never contribute a match. Without this the join multiplies every
# attachment row by the full slice version history — an unbounded cross
# product on dashboards with long-lived, frequently-edited charts.
target_txs: set[int] = {target_tx for _, target_tx in pairs}
min_tx, max_tx = min(target_txs), max(target_txs)
# Chunk the datasource_id IN-clause to stay under SQLite's bind-variable
# floor (a dashboard pointing at very many datasets can exceed it).
rows: list[Any] = []
for chunk in chunked_ids(dataset_ids, ENTITY_ID_CHUNK_SIZE):
stmt = sa.select(
m2m_tbl.c.slice_id,
slices_tbl.c.datasource_id,
m2m_tbl.c.transaction_id.label("m2m_start"),
m2m_tbl.c.end_transaction_id.label("m2m_end"),
slices_tbl.c.transaction_id.label("slice_start"),
slices_tbl.c.end_transaction_id.label("slice_end"),
).where(
m2m_tbl.c.dashboard_id == dashboard_id,
m2m_tbl.c.operation_type != 2,
slices_tbl.c.id == m2m_tbl.c.slice_id,
slices_tbl.c.datasource_id.in_(chunk),
# Attachment membership per slice. charts_attached_to_dashboard owns the
# association-shadow read and the attach/detach window pairing — the single
# place that must never filter the M2M shadow by end_transaction_id, which
# Continuum never closes (sc-119907). Reused here so restore.py, the
# activity relationship walk, and this rollup share one implementation.
attach_windows: dict[int, list[Window]] = {}
for slice_id, window in charts_attached_to_dashboard(dashboard_id):
attach_windows.setdefault(slice_id, []).append(window)
if not attach_windows:
return {}
# Chart→dataset validity from the slice parent shadow, whose
# end_transaction_id the validity backfill *does* close, so the ordinary
# half-open validity predicate is correct here. Bounded on the DB side to
# this dashboard's member charts (the attach_windows keys) and the
# transaction range; the member-id IN-clause is chunked to stay under
# SQLite's 999 bind-variable floor.
#
# The requested-dataset prune is applied on the DB side too, but only when
# the dataset set co-binds with a full member chunk under that floor — a
# member chunk (<= ENTITY_ID_CHUNK_SIZE) plus the dataset IN plus the few
# scalar binds must stay < 999. When there are too many requested datasets,
# the DB-side dataset predicate is dropped and the combiner filters datasets
# in Python (it already keys on pairs_by_dataset), so a wide dashboard does
# not overflow the bind limit (sc-119907 review).
filter_datasets_in_sql = (
len(dataset_ids) <= 999 - ENTITY_ID_CHUNK_SIZE - _SCALAR_BIND_HEADROOM
)
slice_rows: list[Any] = []
for chunk in chunked_ids(set(attach_windows), ENTITY_ID_CHUNK_SIZE):
conditions = [
slices_tbl.c.id.in_(chunk),
slices_tbl.c.datasource_type == "table",
slices_tbl.c.operation_type != 2,
m2m_tbl.c.transaction_id <= max_tx,
sa.or_(
m2m_tbl.c.end_transaction_id.is_(None),
m2m_tbl.c.end_transaction_id > min_tx,
),
slices_tbl.c.operation_type != OPERATION_DELETE,
slices_tbl.c.transaction_id <= max_tx,
sa.or_(
slices_tbl.c.end_transaction_id.is_(None),
slices_tbl.c.end_transaction_id > min_tx,
),
)
rows.extend(db.session.connection().execute(stmt).mappings().all())
]
if filter_datasets_in_sql:
conditions.append(slices_tbl.c.datasource_id.in_(dataset_ids))
stmt = sa.select(
slices_tbl.c.id.label("slice_id"),
slices_tbl.c.datasource_id,
slices_tbl.c.transaction_id.label("slice_start"),
slices_tbl.c.end_transaction_id.label("slice_end"),
).where(*conditions)
slice_rows.extend(db.session.connection().execute(stmt).mappings().all())
# For each pair, collect the slice_ids whose two validity windows
# both straddle target_tx. ``set`` dedupes within a pair.
matches: dict[tuple[int, int], set[int]] = {}
pairs_by_dataset: dict[int, list[int]] = {}
for dataset_id, target_tx in pairs:
pairs_by_dataset.setdefault(dataset_id, []).append(target_tx)
for row in rows:
return _count_attached_charts_at(attach_windows, slice_rows, pairs_by_dataset)
def _count_attached_charts_at(
attach_windows: dict[int, list[Window]],
slice_rows: Sequence[Mapping[str, Any]],
pairs_by_dataset: dict[int, list[int]],
) -> dict[tuple[int, int], int]:
"""Pure combiner: for each ``(dataset_id, target_tx)``, count the distinct
charts whose attachment window and chartdataset window both contain
``target_tx``.
*attach_windows* maps ``slice_id`` to its ``[attach, detach)`` episodes
(from the association shadow see
:func:`~superset.versioning.activity.windows.attachment_windows`); a chart
removed before ``target_tx`` has no window containing it and is therefore
not counted. *slice_rows* are the chartdataset parent-shadow rows
(``slice_id``, ``datasource_id``, ``slice_start``, ``slice_end``), whose
``end_transaction_id`` (``slice_end``) the validity backfill does close, so
the half-open validity predicate is correct for them. Split out of
:func:`batch_chart_counts` so this membership logic is unit-testable
without a live shadow-table fixture.
"""
matches: dict[tuple[int, int], set[int]] = {}
for row in slice_rows:
windows = attach_windows.get(row["slice_id"])
if not windows:
continue
ds_id = row["datasource_id"]
for target_tx in pairs_by_dataset.get(ds_id, ()):
in_m2m = row["m2m_start"] <= target_tx and (
row["m2m_end"] is None or row["m2m_end"] > target_tx
)
in_attach = any(w.contains(target_tx) for w in windows)
in_slice = row["slice_start"] <= target_tx and (
row["slice_end"] is None or row["slice_end"] > target_tx
)
if in_m2m and in_slice:
if in_attach and in_slice:
matches.setdefault((ds_id, target_tx), set()).add(row["slice_id"])
return {pair: len(slice_ids) for pair, slice_ids in matches.items()}
+4 -91
View File
@@ -16,8 +16,10 @@
# under the License.
"""DB-touching helpers for the activity-view read path.
All Phase A relationship walks (``charts_attached_to_dashboard``,
``datasets_used_by_chart``, ``batch_datasets_used_by_charts``),
The Phase A relationship walks (``datasets_used_by_chart``,
``batch_datasets_used_by_charts``; the dashboard-membership walk
``charts_attached_to_dashboard`` lives in
:mod:`superset.versioning.membership`),
the Phase B change-record fetch (``fetch_change_records`` /
``_select_change_rows_for_kinds``), the name-denormalization helpers
(``_resolve_names_for_kind`` / ``apply_entity_name_denormalization``), the
@@ -39,7 +41,6 @@ from __future__ import annotations
import logging
from datetime import datetime
from heapq import heappush, heapreplace
from itertools import groupby
from typing import Any
from uuid import UUID
@@ -119,94 +120,6 @@ def first_tracked_tx(
# ---- Phase A: relationship-traversal queries ------------------------------
# ``operation_type`` values on a Continuum association shadow row
# (sqlalchemy_continuum.operation.Operation): INSERT attaches, DELETE detaches.
# UPDATE never occurs for a pure M2M association (there is nothing to update on
# a (dashboard, slice) pair); if it ever appeared it is ignored — neither
# opening nor closing a window — so an open attachment simply continues.
# These mirror the library enum's numeric values; ``test_m2m_op_constants_match_
# continuum`` pins them so a Continuum renumber fails loudly rather than silently.
_M2M_OP_INSERT = 0
_M2M_OP_DELETE = 2
def _attachment_windows(
rows: list[tuple[int, int, int]],
) -> list[tuple[int, Window]]:
"""Pair INSERT / DELETE association-version rows into ``[attach, detach)``
windows, one per attachment episode.
Each row is ``(slice_id, transaction_id, operation_type)``. Continuum
**never closes** an association shadow row's ``end_transaction_id`` — its
unit-of-work only *inserts* association versions
(``create_association_versions``); the validity backfill that sets
``end_transaction_id`` runs for parent objects, not for M2M links. So the
detach boundary lives on the DELETE row's ``transaction_id``, not on the
attach row's ``end_transaction_id`` (which stays NULL for the association's
whole life). An INSERT opens a window; the next DELETE closes it at its
transaction id; an attachment with no following DELETE stays open (the
chart is still on the dashboard). A DELETE at the same transaction as its
open (add-and-remove in one save) yields no window the chart was never
on a committed dashboard state.
"""
result: list[tuple[int, Window]] = []
# operation_type is part of the sort key so that, within one transaction,
# INSERT (0) sorts before DELETE (2): an add-and-remove in a single save is
# then seen open-before-close and collapses to no window (the DELETE finds
# ``tx == open_tx``, not ``>``). Do not drop it from the key.
rows_sorted = sorted(rows, key=lambda r: (r[0], r[1], r[2]))
for slice_id, group in groupby(rows_sorted, key=lambda r: r[0]):
open_tx: int | None = None
for _slice_id, tx, operation_type in group:
if operation_type == _M2M_OP_DELETE:
if open_tx is not None and tx > open_tx:
result.append((slice_id, Window(open_tx, tx)))
open_tx = None
elif operation_type == _M2M_OP_INSERT and open_tx is None:
open_tx = tx
if open_tx is not None:
result.append((slice_id, Window(open_tx, None)))
return result
def charts_attached_to_dashboard(dashboard_id: int) -> list[tuple[int, Window]]:
"""Return ``(slice_id, window)`` for every chart that has ever been on
*dashboard_id*, with each attachment episode's validity window in
transaction-id space.
Reads from ``dashboard_slices_version`` (Continuum's auto-generated M2M
shadow) and pairs its INSERT/DELETE rows via :func:`_attachment_windows`,
so a chart removed from the dashboard is bounded at the detach transaction
rather than open-ended otherwise the chart's edits made *after* removal
would surface in the dashboard's related history.
"""
# pylint: disable=import-outside-toplevel
from sqlalchemy_continuum import version_class
from superset.models.dashboard import Dashboard
metadata = version_class(Dashboard).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
if m2m_tbl is None:
return []
rows = (
db.session.connection()
.execute(
sa.select(
m2m_tbl.c.slice_id,
m2m_tbl.c.transaction_id,
m2m_tbl.c.operation_type,
).where(
m2m_tbl.c.dashboard_id == dashboard_id,
m2m_tbl.c.slice_id.is_not(None),
)
)
.all()
)
return _attachment_windows([(row[0], row[1], row[2]) for row in rows])
def datasets_used_by_chart(slice_id: int) -> list[tuple[int, Window]]:
"""Return ``(datasource_id, window)`` for every dataset that *slice_id*
has ever pointed at, with each association's validity window.
+1 -1
View File
@@ -35,13 +35,13 @@ from __future__ import annotations
from superset.versioning.activity.kinds import EntityWindows, Window
from superset.versioning.activity.queries import (
batch_datasets_used_by_charts,
charts_attached_to_dashboard,
datasets_used_by_chart,
)
from superset.versioning.activity.windows import (
intersect_windows,
merge_entity_windows,
)
from superset.versioning.membership import charts_attached_to_dashboard
def resolve_scope(
+66
View File
@@ -29,10 +29,76 @@ means "open-ended (current)" and behaves like positive infinity.
from __future__ import annotations
from itertools import groupby
from typing import Any
from superset.versioning.activity.kinds import EntityWindows, Window
# ``operation_type`` values on a Continuum association shadow row
# (sqlalchemy_continuum.operation.Operation): INSERT attaches, DELETE detaches.
# UPDATE never occurs for a pure M2M association (there is nothing to update on
# a (dashboard, slice) pair); if it ever appeared it is ignored — neither
# opening nor closing a window — so an open attachment simply continues.
# These mirror the library enum's numeric values; ``test_m2m_op_constants_match_
# continuum`` pins them so a Continuum renumber fails loudly rather than silently.
M2M_OP_INSERT = 0
M2M_OP_DELETE = 2
def attachment_windows(
rows: list[tuple[int, int, int]],
) -> list[tuple[int, Window]]:
"""Pair INSERT / DELETE association-version rows into ``[attach, detach)``
windows, one per attachment episode.
Each row is ``(assoc_id, transaction_id, operation_type)``. Continuum
**never closes** an association shadow row's ``end_transaction_id`` — its
unit-of-work only *inserts* association versions
(``create_association_versions``); the validity backfill that sets
``end_transaction_id`` runs for parent objects, not for M2M links. So the
detach boundary lives on the DELETE row's ``transaction_id``, not on the
attach row's ``end_transaction_id`` (which stays NULL for the association's
whole life). An INSERT opens a window; the next DELETE closes it at its
transaction id; an attachment with no following DELETE stays open (the
association is still live). A DELETE at the same transaction as its open
(add-and-remove in one save) yields no window the association was never
on a committed state. That last case is a deliberate divergence from
Continuum's own ``association_subquery`` reverter, which (selecting the
``MAX(tx) <= T`` row and excluding only DELETEs) would treat such a pair as
a member; the never-committed reading is the safer one for restore.
This is the M2M-correct counterpart to
:func:`~superset.versioning.changes.shadow_queries.shadow_rows_valid_at`,
whose ``end_transaction_id`` validity filter is right for parent/child
shadows but silently re-includes a detached association.
"""
result: list[tuple[int, Window]] = []
# operation_type is part of the sort key so that, within one transaction,
# INSERT (0) sorts before DELETE (2): an add-and-remove in a single save is
# then seen open-before-close and collapses to no window (the DELETE finds
# ``tx == open_tx``, not ``>``). Do not drop it from the key.
#
# Corollary / assumption: because INSERT is forced before DELETE within a
# transaction, this cannot represent a *remove-then-re-add* of the same
# association in one transaction (it would read the same as add-then-remove
# → no window). That relies on no write path emitting DELETE-then-INSERT
# for the same association within a single transaction — which holds today
# (a chart is detached or attached in a save, not both), so the case is
# latent, not live. Revisit this pairing if such a write path is added.
rows_sorted = sorted(rows, key=lambda r: (r[0], r[1], r[2]))
for assoc_id, group in groupby(rows_sorted, key=lambda r: r[0]):
open_tx: int | None = None
for _assoc_id, tx, operation_type in group:
if operation_type == M2M_OP_DELETE:
if open_tx is not None and tx > open_tx:
result.append((assoc_id, Window(open_tx, tx)))
open_tx = None
elif operation_type == M2M_OP_INSERT and open_tx is None:
open_tx = tx
if open_tx is not None:
result.append((assoc_id, Window(open_tx, None)))
return result
def intersect_windows(outer: Window, inner: Window) -> Window | None:
"""Intersect two half-open ``[start_tx, end_tx)`` windows.
+86
View File
@@ -0,0 +1,86 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""M2M dashboard-membership queries, shared across the versioning surfaces.
``charts_attached_to_dashboard`` reads the ``dashboard_slices_version``
association shadow and pairs its INSERT/DELETE rows into ``[attach, detach)``
windows. It must **never** filter that shadow by ``end_transaction_id``:
Continuum never closes an M2M association's ``end_transaction_id`` (see
:func:`~superset.versioning.activity.windows.attachment_windows`), so a
validity filter would re-include a chart removed before the queried tx.
This lives in a neutral module not the activity read-path module
(``activity/queries.py``) so the restore write path and the impact rollup
depend on it here rather than reaching up into the read path (sc-119907).
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import sqlalchemy as sa
from superset.extensions import db
if TYPE_CHECKING:
from superset.versioning.activity.kinds import Window
def charts_attached_to_dashboard(dashboard_id: int) -> list[tuple[int, Window]]:
"""Return ``(slice_id, window)`` for every chart that has ever been on
*dashboard_id*, with each attachment episode's validity window in
transaction-id space.
Reads from ``dashboard_slices_version`` (Continuum's auto-generated M2M
shadow) and pairs its INSERT/DELETE rows via
:func:`~superset.versioning.activity.windows.attachment_windows`,
so a chart removed from the dashboard is bounded at the detach transaction
rather than open-ended otherwise the chart's edits made *after* removal
would surface in the dashboard's related history.
"""
# pylint: disable=import-outside-toplevel
# attachment_windows is imported lazily (not at module top) so that this
# module stays a leaf: importing it must not pull the activity package,
# whose read-path modules (scope.py) import charts_attached_to_dashboard
# back from here — a module-top import created a circular import that only
# surfaced at runtime, when a restore imported this module for the first
# time (sc-119907).
from sqlalchemy_continuum import version_class
from superset.models.dashboard import Dashboard
from superset.versioning.activity.windows import attachment_windows
metadata = version_class(Dashboard).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
if m2m_tbl is None:
return []
rows = (
db.session.connection()
.execute(
sa.select(
m2m_tbl.c.slice_id,
m2m_tbl.c.transaction_id,
m2m_tbl.c.operation_type,
).where(
m2m_tbl.c.dashboard_id == dashboard_id,
m2m_tbl.c.slice_id.is_not(None),
)
)
.all()
)
return attachment_windows([(row[0], row[1], row[2]) for row in rows])
+24 -26
View File
@@ -195,42 +195,40 @@ def _restore_dashboard_membership(dashboard: Any, transaction_id: int) -> list[i
"""Reset *dashboard*'s chart membership to what it was at
*transaction_id*, reattaching only charts that still exist.
Reads the validity-windowed ``dashboard_slices_version`` shadow
(Continuum's auto-generated M2M table): a slice was a member at tx T
iff a non-DELETE row has ``transaction_id <= T`` and an open or
later-closing validity window.
Membership is derived from the ``dashboard_slices_version`` shadow
(Continuum's auto-generated M2M table) by pairing each slice's
INSERT/DELETE rows into ``[attach, detach)`` windows: a slice was a
member at tx T iff one of its attachment windows contains T. The
``shadow_rows_valid_at`` validity filter must **not** be used here
Continuum never closes an association shadow's ``end_transaction_id``,
so that filter would re-attach a chart that had been removed before T
(attached@1, removed@5, restore to tx10 the chart wrongly returns).
``shadow_rows_valid_at`` stays correct for parent/child shadows, whose
``end_transaction_id`` the validity backfill does close (sc-119907).
Returns the ids of snapshot members that no longer exist and were
skipped. Live charts' content is never touched — restoring a chart's
content is the chart's own restore endpoint's job.
"""
# pylint: disable=import-outside-toplevel
# Local imports: models.slice transitively imports models.core, which
# needs the initialised app — module-top import would recreate the
# bootstrap cycle documented in changes/listener.py; shadow_queries is
# imported lazily for the same reason (see queries.get_version).
# Local imports: models.slice transitively imports models.core, which needs
# the initialised app — a module-top import would recreate the bootstrap
# cycle documented in changes/listener.py. charts_attached_to_dashboard is
# imported lazily for the same reason: it pulls the window helpers, whose
# package transitively imports the versioning.changes listener graph, so a
# module-top import here would re-enter that same bootstrap cycle.
from superset.models.slice import Slice
from superset.versioning.changes import shadow_rows_valid_at
from superset.versioning.membership import charts_attached_to_dashboard
ver_cls = version_class(type(dashboard))
m2m_tbl = ver_cls.__table__.metadata.tables.get("dashboard_slices_version")
if m2m_tbl is None: # pragma: no cover — shadow tables always exist here
return []
# shadow_rows_valid_at owns the validity-window semantics (open or
# later-closing window, non-DELETE) — the same predicate the version
# snapshot's column/metric reconstruction uses.
# charts_attached_to_dashboard owns the association-shadow read and the
# attach/detach window pairing (the single place that must never filter the
# M2M shadow by end_transaction_id — Continuum never closes it). A slice was
# a member at transaction_id iff one of its windows contains it (sc-119907).
member_ids = sorted(
{
row["slice_id"]
for row in shadow_rows_valid_at(
db.session,
m2m_tbl,
"dashboard_id",
dashboard.id,
transaction_id,
)
if row["slice_id"] is not None
slice_id
for slice_id, window in charts_attached_to_dashboard(dashboard.id)
if window.contains(transaction_id)
}
)
if not member_ids:
+2 -1
View File
@@ -23,7 +23,8 @@ from unittest.mock import MagicMock
from urllib.parse import parse_qs, urlparse
import pytest
from flask import Response, escape, url_for
from flask import Response, url_for
from markupsafe import escape
from sqlalchemy import func
from superset import db, security_manager
@@ -18,10 +18,7 @@
import pytest
import rison
from flask import (
current_app,
escape, # noqa: F401
)
from flask import current_app
from superset.daos.dashboard import DashboardDAO
from superset.utils import json
@@ -201,6 +201,76 @@ class TestDashboardRestoreApi(SupersetTestCase):
f"got {restored_ids}"
)
def test_restore_to_tx_after_removal_does_not_reattach_chart(self) -> None:
"""The inverse of the re-attach case (sc-119907): restoring to a
snapshot captured *after* a chart was removed must NOT bring it back.
Continuum never closes an association shadow row's
``end_transaction_id``, so the removed chart's INSERT row still looks
"valid at" any later tx under a naive validity filter it would be
wrongly re-attached. Membership must be derived from paired
attach/detach windows: the window ``[attach, remove)`` does not contain
the post-removal target tx, so the chart stays off.
"""
from superset.daos.version import derive_version_uuid
_persist_fixture_state()
dashboard: Dashboard = (
db.session.query(Dashboard)
.filter(Dashboard.dashboard_title == "USA Births Names")
.first()
)
assert dashboard is not None
dashboard_uuid = str(dashboard.uuid)
dashboard_id = dashboard.id
entity_uuid = dashboard.uuid
assert entity_uuid is not None
original_slice_ids = sorted(s.id for s in dashboard.slices)
assert len(original_slice_ids) >= 2, (
f"fixture expected to attach >= 2 charts; got {original_slice_ids}"
)
slice_to_drop = dashboard.slices[0]
drop_id = slice_to_drop.id
# Remove the chart and commit — the detach is recorded in history.
dashboard.slices.remove(slice_to_drop)
db.session.commit()
# Touch the dashboard AFTER the removal so the restore target snapshot
# post-dates the detach.
dashboard.dashboard_title = "USA Births Names — post-removal snapshot"
db.session.commit()
ver_cls = version_class(Dashboard)
target_tx = (
db.session.query(ver_cls.transaction_id)
.filter(ver_cls.id == dashboard_id, ver_cls.uuid == entity_uuid)
.order_by(ver_cls.transaction_id.desc())
.limit(1)
.scalar()
)
assert target_tx is not None
target_uuid = str(derive_version_uuid(entity_uuid, target_tx))
self.login(ADMIN_USERNAME)
rv = self._restore(dashboard_uuid, target_uuid)
assert rv.status_code == 200, rv.data
db.session.expire_all()
dashboard = (
db.session.query(Dashboard).filter(Dashboard.id == dashboard_id).one()
)
restored_ids = sorted(s.id for s in dashboard.slices)
assert drop_id not in restored_ids, (
"restore re-attached a chart that was removed before the target "
f"snapshot: {drop_id} in {restored_ids}"
)
survivors = sorted(set(original_slice_ids) - {drop_id})
assert restored_ids == survivors, (
f"expected the surviving members {survivors}, got {restored_ids}"
)
def test_restore_preserves_live_chart_content(self) -> None:
"""Dashboard restore is membership-only: a member chart edited
AFTER the snapshot keeps its current content charts are shared
+74
View File
@@ -4888,6 +4888,80 @@ def test_get_sqla_query_dotted_struct_column_bigquery(
assert "`forecasts.original`" not in sql
def test_get_sqla_query_grouping_sets_preserves_bigquery_mutated_label(
mocker: MockerFixture,
session: Session,
) -> None:
"""
A pivot table with a non-additive metric issues a `grouping_sets` query
(see plugin-chart-pivot-table/src/plugin/buildQuery.ts), whose rollup
levels are expressed in terms of each column's original, unmutated label
(e.g. a Custom SQL row labelled "Test Row"). BigQuery mutates SQL aliases
that contain non-word characters, so the same column's `Label.name` ends
up as something like "Test_Row_<hash>". Looking up each level's columns by
that mutated name instead of the original label silently drops the column
from every rollup level, producing a `GROUP BY GROUPING SETS` clause that
doesn't cover a selected, non-aggregated column.
"""
bigquery = pytest.importorskip("sqlalchemy_bigquery")
from superset.connectors.sqla.models import SqlaTable, TableColumn
from superset.models.core import Database
SqlaTable.metadata.create_all(session.get_bind())
dialect = bigquery.BigQueryDialect()
@contextmanager
def fake_engine(*args, **kwargs):
engine = MagicMock()
engine.dialect = dialect
yield engine
database = Database(database_name="bq", sqlalchemy_uri="bigquery://project")
mocker.patch.object(database, "get_sqla_engine", new=fake_engine)
table = SqlaTable(
database=database,
schema=None,
table_name="orders",
columns=[TableColumn(column_name="amount", type="FLOAT")],
)
row_col: AdhocColumn = {
"sqlExpression": "some_column",
"label": "Test Row",
}
metric: AdhocMetric = {
"expressionType": "SIMPLE",
"column": {"column_name": "amount"},
"aggregate": "AVG",
"label": "avg_amount",
}
sqlaq = table.get_sqla_query(
columns=[row_col],
metrics=[metric],
extras={},
filter=[],
granularity=None,
is_timeseries=False,
grouping_sets=[["Test Row"], []],
)
sql = str(
sqlaq.sqla_query.compile(
dialect=dialect, compile_kwargs={"literal_binds": True}
)
)
# Before the fix, looking up each level's columns by the BigQuery-mutated
# alias (e.g. "Test_Row_<hash>") instead of the original "Test Row" label
# sent by the frontend silently dropped the column from every level,
# producing `GROUPING SETS ((), ())` -- a GROUP BY that doesn't cover the
# selected, non-aggregated "some_column" expression.
assert "GROUPING SETS((some_column), ())" in sql
def test_temporal_epoch_string_filter_is_coerced_for_bigquery() -> None:
"""
Drill-to-detail can send JavaScript timestamp strings for temporal values.
+2 -4
View File
@@ -129,7 +129,7 @@ def test_superset_cache_memoize_make_cache_key_uses_configurable_hash():
"_memoize_make_cache_key",
return_value=lambda *args, **kwargs: "cache_key",
) as mock_make_key:
cache._memoize_make_cache_key(make_name=None, timeout=300)
cache._memoize_make_cache_key(make_name=None)
mock_make_key.assert_called_once()
call_kwargs = mock_make_key.call_args[1]
@@ -145,9 +145,7 @@ def test_superset_cache_memoize_make_cache_key_allows_explicit_hash():
"_memoize_make_cache_key",
return_value=lambda *args, **kwargs: "cache_key",
) as mock_make_key:
cache._memoize_make_cache_key(
make_name=None, timeout=300, hash_method=hashlib.md5
)
cache._memoize_make_cache_key(make_name=None, hash_method=hashlib.md5)
mock_make_key.assert_called_once()
call_kwargs = mock_make_key.call_args[1]
@@ -1610,3 +1610,51 @@ def test_ag_grid_no_rows_overlay_is_a_terminal_empty_state() -> None:
CHART_CONTAINER_READY_JS,
):
assert ".ag-overlay-no-rows-wrapper:not(.ag-hidden)" in predicate
def test_expand_scrollable_content_js_unrolls_ag_grid_and_css_scroll() -> None:
"""The pre-capture DOM-expansion script must reach both flavors of
clipped table content: ag-Grid's row virtualization (needs its own API
to force a full render) and plain CSS overflow/height clipping (the
content already exists in the DOM and just needs the constraint lifted).
It must also lift the fixed heights on the shared `.chart-container` /
`.slice_container` / ag-Grid-wrapper ancestor chain, not just the
scrollable descendant itself -- a bounding-box capture
(`element.screenshot()`, used for single-chart exports) clips to the
ancestor's own box, which does not grow just because a descendant's
content does (#38090, reviewed by @aminghadersohi on #43979)."""
from superset.utils.screenshot_utils import (
AG_GRID_HOST_SELECTOR,
CHART_CONTAINER_SELECTOR,
EXPAND_SCROLLABLE_CONTENT_JS,
GENERIC_SCROLLABLE_DESCENDANT_SELECTOR,
SLICE_CONTAINER_SELECTOR_FOR_EXPANSION,
)
assert AG_GRID_HOST_SELECTOR in EXPAND_SCROLLABLE_CONTENT_JS
assert "setGridOption('domLayout', 'print')" in EXPAND_SCROLLABLE_CONTENT_JS
assert "grid._agGridApi" in EXPAND_SCROLLABLE_CONTENT_JS
# The grid's own host and its immediate parent (the ag-Grid table
# plugin's fixed-pixel-height wrapper) both get their height lifted --
# not just the descendant content inside the grid.
assert "grid.style.height = 'auto'" in EXPAND_SCROLLABLE_CONTENT_JS
assert "grid.parentElement.style.height = 'auto'" in EXPAND_SCROLLABLE_CONTENT_JS
assert SLICE_CONTAINER_SELECTOR_FOR_EXPANSION == ".slice_container"
assert SLICE_CONTAINER_SELECTOR_FOR_EXPANSION in EXPAND_SCROLLABLE_CONTENT_JS
assert CHART_CONTAINER_SELECTOR == ".chart-container"
assert GENERIC_SCROLLABLE_DESCENDANT_SELECTOR == (
'.chart-container [style*="overflow"], .chart-container .ant-table-body'
)
assert GENERIC_SCROLLABLE_DESCENDANT_SELECTOR in EXPAND_SCROLLABLE_CONTENT_JS
# Gated on actually clipping, so a non-scrollable match (e.g. a sticky
# table header with no overflow of its own) is left untouched.
assert "el.scrollHeight > el.clientHeight" in EXPAND_SCROLLABLE_CONTENT_JS
assert "overflow = 'visible'" in EXPAND_SCROLLABLE_CONTENT_JS
assert "maxHeight = 'none'" in EXPAND_SCROLLABLE_CONTENT_JS
# Takes an explicit wait budget rather than hardcoding one, so it can be
# bounded by a report's remaining deadline (see webdriver_test.py).
assert "async (maxWaitMs) =>" in EXPAND_SCROLLABLE_CONTENT_JS
assert "Date.now() + maxWaitMs" in EXPAND_SCROLLABLE_CONTENT_JS
+161 -11
View File
@@ -154,6 +154,84 @@ class TestWebDriverPlaywrightFallback:
"http://example.com", wait_until="networkidle"
)
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager")
@patch("superset.utils.webdriver.app")
def test_get_screenshot_expands_scrollable_content_before_capture(
self, mock_app, mock_browser_manager
):
"""A dense table taller than its dashboard tile renders fully in the
DOM but is visually clipped by a fixed height + internal scrollbar.
`get_screenshot` must un-clip that content (ag-Grid print layout /
CSS overflow reset, see EXPAND_SCROLLABLE_CONTENT_JS) before taking
the screenshot, and must do so *before* the capture call so the
expanded layout is what actually gets captured (#38090). It runs
twice: once before the tiling decision, and again after chart
readiness confirms every ag-Grid has actually mounted its API
(@aminghadersohi's review on #43979 -- the earlier call can miss a
grid whose GridReady hasn't fired yet)."""
from superset.utils.screenshot_utils import EXPAND_SCROLLABLE_CONTENT_JS
mock_user = MagicMock()
mock_user.username = "test_user"
mock_app.config = {
"WEBDRIVER_OPTION_ARGS": [],
"WEBDRIVER_WINDOW": {"pixel_density": 1},
"SCREENSHOT_PLAYWRIGHT_DEFAULT_TIMEOUT": 30000,
"SCREENSHOT_PLAYWRIGHT_WAIT_EVENT": "networkidle",
"SCREENSHOT_SELENIUM_HEADSTART": 0,
"SCREENSHOT_SELENIUM_ANIMATION_WAIT": 0,
"SCREENSHOT_REPLACE_UNEXPECTED_ERRORS": False,
"SCREENSHOT_TILED_ENABLED": False,
"SCREENSHOT_LOCATE_WAIT": 10,
"SCREENSHOT_LOAD_WAIT": 10,
}
mock_browser = MagicMock()
mock_context = MagicMock()
mock_page = MagicMock()
mock_element = MagicMock()
mock_browser_manager.get_browser.return_value = mock_browser
mock_browser.new_context.return_value = mock_context
mock_context.new_page.return_value = mock_page
mock_page.locator.return_value = mock_element
capture_order: list[str] = []
def evaluate_side_effect(script, *args, **kwargs):
if script == EXPAND_SCROLLABLE_CONTENT_JS:
capture_order.append("expand")
return None
# FIND_CHART_HOLDER_STATES_JS (readiness diagnostics) expects an
# iterable of holder states; every other call in this (non-tiled,
# non-report) path is only ever logged, not branched on.
return []
mock_page.evaluate.side_effect = evaluate_side_effect
mock_page.screenshot.side_effect = lambda **k: (
capture_order.append("capture") or b"fake_screenshot"
)
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
driver = WebDriverPlaywright("chrome")
result = driver.get_screenshot(
"http://example.com", "standalone", mock_user
)
assert result == b"fake_screenshot"
expand_calls = [
call
for call in mock_page.evaluate.call_args_list
if call.args[0] == EXPAND_SCROLLABLE_CONTENT_JS
]
assert len(expand_calls) == 2
# A concrete millisecond budget is passed as the second arg (bounded
# by the report deadline when one exists; a fixed default otherwise).
assert all(isinstance(call.args[1], float) for call in expand_calls)
assert capture_order == ["expand", "expand", "capture"]
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager")
@patch("superset.utils.webdriver.logger")
@@ -259,6 +337,70 @@ class TestWebDriverPlaywrightErrorHandling:
"Failed to capture unexpected errors%s", ""
)
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver.logger")
def test_expand_scrollable_content_swallows_playwright_error(self, mock_logger):
"""A failure while un-clipping scrollable content must not abort the
screenshot -- a clipped-but-present capture beats none at all."""
from superset.utils.webdriver import PlaywrightError
mock_page = MagicMock()
mock_page.evaluate.side_effect = PlaywrightError("boom")
WebDriverPlaywright._expand_scrollable_content(
mock_page, log_context="execution_id=abc-123"
)
mock_logger.warning.assert_called_once()
warning_args = mock_logger.warning.call_args.args
assert "Failed to expand scrollable chart content" in warning_args[0]
assert warning_args[1] == " [execution_id=abc-123]"
def test_expand_scrollable_content_defaults_wait_without_report_context(self):
"""Absent a report deadline to bound it against, the ag-Grid
stabilization poll gets the fixed default ceiling."""
from superset.utils.screenshot_utils import (
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
)
mock_page = MagicMock()
WebDriverPlaywright._expand_scrollable_content(mock_page)
mock_page.evaluate.assert_called_once_with(
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS * 1000,
)
def test_expand_scrollable_content_bounds_wait_to_report_deadline(self):
"""This step must respect the report's remaining budget like every
other wait in the capture path, rather than an unconditional fixed
sleep (Thread A / @aminghadersohi's review on #43979)."""
mock_page = MagicMock()
report_execution_context = _report_context()
# Only 2s left for this phase after other phases' reserves.
report_execution_context = report_execution_context.__class__(
**{
**report_execution_context.__dict__,
"deadline": report_execution_context.deadline.__class__(
total_seconds=2
+ report_execution_context.capture_reserve_seconds
+ report_execution_context.delivery_reserve_seconds
+ report_execution_context.cleanup_reserve_seconds,
started_at=0,
_clock=lambda: 0,
),
}
)
WebDriverPlaywright._expand_scrollable_content(
mock_page, report_execution_context=report_execution_context
)
max_wait_ms = mock_page.evaluate.call_args.args[1]
assert max_wait_ms == pytest.approx(2000)
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver.sync_playwright")
@patch("superset.utils.webdriver.logger")
@@ -979,12 +1121,16 @@ class TestWebDriverPlaywrightChartReadiness:
assert result == b"screenshot"
# Readiness diagnostics are emitted before polling so a task killed by
# an outer limit still leaves useful state in the logs.
assert mock_page.evaluate.call_count == 2
assert all(
"state: 'rendered'" in call.args[0]
# an outer limit still leaves useful state in the logs. Two additional
# evaluate() calls expand scrollable content before capture: once
# before the tiling decision, once after readiness (#38090).
readiness_calls = [
call
for call in mock_page.evaluate.call_args_list
)
if "state: 'rendered'" in call.args[0]
]
assert len(readiness_calls) == 2
assert mock_page.evaluate.call_count == 4
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager")
@@ -1599,8 +1745,9 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
# Small dashboard: 3 charts, 1000px height — below both thresholds
mock_page.evaluate.side_effect = [3, 1000, [], []]
# Small dashboard: 3 charts, 1000px height — below both thresholds.
# First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 3, 1000, [], []]
call_order: list[str] = []
@@ -1642,7 +1789,8 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
"SCREENSHOT_TILED_VIEWPORT_HEIGHT": 600,
}
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
mock_page.evaluate.side_effect = [25, 500, [], []]
# First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 500, [], []]
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
result = WebDriverPlaywright("chrome").get_screenshot(
@@ -1673,8 +1821,9 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
# Large dashboard: 25 charts, 6000px height
mock_page.evaluate.side_effect = [25, 6000]
# Large dashboard: 25 charts, 6000px height. First item is consumed
# by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 6000]
mock_take_tiled.return_value = b"tiled_screenshot"
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
@@ -1727,7 +1876,8 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
}
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
mock_page.evaluate.side_effect = [25, 6000]
# First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 6000]
# Empty bytes — falsy but not None; was silently passed through before the fix
mock_take_tiled.return_value = b""
# _get_screenshot("standalone") calls page.screenshot(full_page=True); it
+137 -12
View File
@@ -44,6 +44,8 @@ from superset.versioning.activity import (
Window,
)
from superset.versioning.activity.impact import (
_count_attached_charts_at,
batch_chart_counts,
collect_impact_pairs,
impact_for_record,
)
@@ -54,9 +56,6 @@ from superset.versioning.activity.orchestrator import (
_MAX_PAGE_SIZE,
)
from superset.versioning.activity.queries import (
_attachment_windows,
_M2M_OP_DELETE,
_M2M_OP_INSERT,
_merge_result_into_heap,
_record_sort_key,
BoundedRecordHeap,
@@ -68,7 +67,10 @@ from superset.versioning.activity.render import (
)
from superset.versioning.activity.scope import resolve_scope
from superset.versioning.activity.windows import (
attachment_windows,
intersect_windows,
M2M_OP_DELETE,
M2M_OP_INSERT,
merge_entity_windows,
row_within_any_window,
union_windows,
@@ -893,7 +895,7 @@ def test_build_summary_meta_headline_branches() -> None:
assert _build_summary("Dashboard", unknown) == "Dashboard updated"
# ---- _attachment_windows -------------------------------------------------
# ---- attachment_windows -------------------------------------------------
def test_attachment_windows_closes_at_detach() -> None:
@@ -903,7 +905,7 @@ def test_attachment_windows_closes_at_detach() -> None:
row's transaction id — an INSERT at t1 paired with a DELETE at t3 yields
the half-open window [t1, t3), so an edit at t2 (while attached) is
inside it and an edit at t4 (after removal) is not."""
windows = _attachment_windows([(7, 1, 0), (7, 3, 2)]) # INSERT@1, DELETE@3
windows = attachment_windows([(7, 1, 0), (7, 3, 2)]) # INSERT@1, DELETE@3
assert windows == [(7, Window(1, 3))]
window = windows[0][1]
@@ -916,7 +918,7 @@ def test_attachment_windows_still_attached_is_open_ended() -> None:
"""A chart with an INSERT and no following DELETE is still on the
dashboard, so its window is open-ended (end_tx = None) and every later
edit remains in scope."""
assert _attachment_windows([(7, 5, 0)]) == [(7, Window(5, None))]
assert attachment_windows([(7, 5, 0)]) == [(7, Window(5, None))]
def test_attachment_windows_reattach_cycles_and_ordering() -> None:
@@ -924,20 +926,30 @@ def test_attachment_windows_reattach_cycles_and_ordering() -> None:
is independent of the row order the query returns them in (the rows are
sorted by (slice_id, transaction_id) internally)."""
rows = [(7, 7, 2), (7, 1, 0), (7, 5, 0), (7, 3, 2)] # deliberately shuffled
assert _attachment_windows(rows) == [(7, Window(1, 3)), (7, Window(5, 7))]
assert attachment_windows(rows) == [(7, Window(1, 3)), (7, Window(5, 7))]
def test_attachment_windows_reattach_leaves_the_last_episode_open() -> None:
"""Attach@1, detach@5, re-attach@8 with no later detach: the first episode
is the closed window [1, 5) and the current attachment is open-ended
[8, None). An edit at tx 10 falls inside the live episode."""
windows = attachment_windows([(7, 1, 0), (7, 5, 2), (7, 8, 0)])
assert windows == [(7, Window(1, 5)), (7, Window(8, None))]
assert windows[1][1].contains(10)
assert not windows[0][1].contains(10)
def test_attachment_windows_add_and_remove_same_transaction() -> None:
"""Attaching and detaching in a single save (INSERT and DELETE at the
same transaction) leaves the chart on no committed dashboard state, so it
contributes no window (and no degenerate zero-width interval)."""
assert _attachment_windows([(7, 2, 0), (7, 2, 2)]) == []
assert attachment_windows([(7, 2, 0), (7, 2, 2)]) == []
def test_attachment_windows_separates_slices() -> None:
"""Each slice gets its own independent windows."""
rows = [(7, 1, 0), (7, 3, 2), (9, 2, 0)]
assert _attachment_windows(rows) == [
assert attachment_windows(rows) == [
(7, Window(1, 3)),
(9, Window(2, None)),
]
@@ -945,10 +957,123 @@ def test_attachment_windows_separates_slices() -> None:
def test_m2m_op_constants_match_continuum() -> None:
"""The association-shadow operation-type constants used by
_attachment_windows are the numeric values of Continuum's Operation enum.
attachment_windows are the numeric values of Continuum's Operation enum.
Pin them so a library renumber fails here loudly rather than silently
mis-pairing attach/detach rows."""
from sqlalchemy_continuum import Operation
assert _M2M_OP_INSERT == Operation.INSERT
assert _M2M_OP_DELETE == Operation.DELETE
assert M2M_OP_INSERT == Operation.INSERT
assert M2M_OP_DELETE == Operation.DELETE
# ---- _count_attached_charts_at (batch_chart_counts membership) -----------
def _slice_row(
slice_id: int, datasource_id: int, start: int, end: int | None
) -> dict[str, Any]:
"""A chart→dataset parent-shadow row as batch_chart_counts fetches it."""
return {
"slice_id": slice_id,
"datasource_id": datasource_id,
"slice_start": start,
"slice_end": end,
}
def test_count_attached_charts_excludes_chart_removed_before_target() -> None:
"""sc-119907: a chart attached@1 and removed@5 must NOT be counted for a
dataset rollup at target_tx=10. Its attachment window [1, 5) does not
contain 10, even though its (never-closed) association shadow row would
pass a naive end_transaction_id validity filter."""
attach_windows = {7: [Window(1, 5)]}
slice_rows = [_slice_row(7, 100, 1, None)] # chart→dataset open the whole time
result = _count_attached_charts_at(attach_windows, slice_rows, {100: [10]})
assert result == {} # zero-count pairs are omitted
def test_count_attached_charts_counts_chart_inside_its_window() -> None:
"""The same chart IS counted at a target inside its attachment window."""
attach_windows = {7: [Window(1, 5)]}
slice_rows = [_slice_row(7, 100, 1, None)]
result = _count_attached_charts_at(attach_windows, slice_rows, {100: [3]})
assert result == {(100, 3): 1}
def test_count_attached_charts_requires_both_windows() -> None:
"""A chart attached at target_tx but not yet pointing at the dataset (its
chartdataset window starts later) is not counted."""
attach_windows = {7: [Window(1, None)]}
slice_rows = [_slice_row(7, 100, 8, None)] # points at dataset only from tx8
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [3]}) == {}
def test_count_attached_charts_dedupes_within_pair() -> None:
"""Multiple parent-shadow rows for the same slice count the slice once."""
attach_windows = {7: [Window(1, None)]}
slice_rows = [_slice_row(7, 100, 1, 4), _slice_row(7, 100, 4, None)]
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [5]}) == {
(100, 5): 1
}
def test_count_attached_charts_ignores_slice_never_on_dashboard() -> None:
"""A slice pointing at the dataset but with no attachment window (never on
this dashboard) does not contribute guards the no-join fetch that may
return slices from other dashboards sharing the dataset."""
attach_windows: dict[int, list[Window]] = {}
slice_rows = [_slice_row(7, 100, 1, None)]
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [3]}) == {}
# ---- batch_chart_counts bind-variable floor (sc-119907) ------------------
def test_batch_chart_counts_stays_under_sqlite_bind_floor(app_context: None) -> None:
"""sc-119907: a wide dashboard (many member charts AND many requested
datasets) must not build a slice-scan statement that exceeds SQLite's 999
bind-variable floor. The member-id IN is chunked; the requested-dataset IN
is dropped to Python-side filtering when it would not co-bind under the
floor. Compiles every issued statement and asserts its bind count stays
safely under 999 the pre-fix code (dataset IN always on) bound
500 member + 600 dataset + scalars = ~1104 and would 500 on SQLite."""
from sqlalchemy.dialects import sqlite
member_windows = [(sid, Window(1, None)) for sid in range(1, 601)]
pairs = {(ds, 5) for ds in range(10_000, 10_600)} # 600 requested datasets
captured: list[Any] = []
class _FakeResult:
def mappings(self) -> "_FakeResult":
return self
def all(self) -> list[Any]:
return []
def _fake_execute(stmt: Any) -> "_FakeResult":
captured.append(stmt)
return _FakeResult()
with (
patch(
"superset.versioning.membership.charts_attached_to_dashboard",
return_value=member_windows,
),
patch("superset.versioning.activity.impact.db") as mock_db,
):
mock_db.session.connection.return_value.execute.side_effect = _fake_execute
batch_chart_counts(1, pairs)
assert captured, "expected at least one slice-scan statement"
for stmt in captured:
# render_postcompile expands ``IN (...)`` (an expanding bind in
# SQLAlchemy 2.0) into one param per element, so the count reflects
# what actually hits SQLite — a plain compile would show one bind per
# IN and hide the overflow.
compiled = stmt.compile(
dialect=sqlite.dialect(),
compile_kwargs={"render_postcompile": True},
)
n_binds = len(compiled.params)
assert n_binds < 999, f"statement binds {n_binds} params (SQLite floor 999)"