Compare commits

...
Author SHA1 Message Date
rusackasandClaude Sonnet 5 09f53d52d8 fix(security): suppress zizmor self-repository false positive for postgres setup step
cached-dependencies is a git submodule, so the $/ self-repository syntax
can't resolve it; keep the ./ form with a scoped zizmor suppression,
matching the pattern used elsewhere in this workflow for the same action.

Resolves code-scanning alert #2657.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 18:22:59 -07:00
986b648f1c fix(versioning): resolve dashboard M2M membership by attach/detach windows, not end_transaction_id (#44010)
Co-authored-by: Mike Bridge <michael.bridge@ext.preset.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-08 17:59:04 -07:00
dependabot[bot]andClaude Code acb29937ae chore(deps): bump the security group across 1 directory with 3 updates (#44030)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-08 17:52:32 -07:00
dependabot[bot]andClaude Code 2875c76d55 chore(deps): bump react-ace from 14.0.1 to 15.0.0 in /superset-frontend (#42833)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-08 17:52:09 -07:00
d0f139c210 feat(deps): bump flask-caching from 2.4.1 to 2.5.0 + bump flask from v2 to v3 (#43875)
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: hainenber <dotronghai96@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: rusackas <evan@rusackas.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: hainenber <dotronghai96@gmail.com>
Co-authored-by: Đỗ Trọng Hải <41283691+hainenber@users.noreply.github.com>
2026-09-08 17:20:53 -07:00
dependabot[bot] 9de50b85e5 chore(deps): bump the security group across 1 directory with 3 updates (#44029)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 17:04:39 -07:00
Joe LiandVitor Avila c510946df5 fix: BigQuery pivot-table Custom SQL row label with a space drops from GROUPING SETS (#43720)
Co-authored-by: Vitor Avila <vitor.avila@preset.io>
2026-09-08 16:58:49 -07:00
Joe LiandClaude Sonnet 5 91f4d01ca2 fix(chart-creation): stop antd v6 Steps from doubling the icon-content gap (#43730)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:57:55 -07:00
Evan Rusackas 3828dcb922 fix(security): suppress zizmor self-repository false positive for cached-dependencies in presto-hive workflow (#43976) 2026-09-08 16:53:30 -07:00
Evan Rusackas 362d3509cb fix(security): suppress zizmor self-repository false positive for cached-dependencies in presto-hive workflow (#43984) 2026-09-08 16:53:23 -07:00
Evan RusackasandClaude Sonnet 5 fa31e9c1c3 fix(security): suppress zizmor self-repository false positive for cached-dependencies (#43972)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:52:42 -07:00
Evan RusackasandClaude Sonnet 5 335a2a78ea fix(dashboard): expand clipped scrollable content before screenshot/PDF export (#43979)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:46:34 -07:00
Evan RusackasandClaude Sonnet 5 3564baabf6 fix(security): suppress zizmor self-repository false positive for setup-backend step (#44018)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-08 16:41:46 -07:00
36 changed files with 1227 additions and 356 deletions
@@ -164,7 +164,13 @@ jobs:
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Setup Postgres - name: Setup Postgres
uses: ./.github/actions/cached-dependencies # cached-dependencies is a git submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's gitlink. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with: with:
run: | run: |
setup-postgres setup-postgres
@@ -210,7 +216,7 @@ jobs:
persist-credentials: false persist-credentials: false
submodules: recursive submodules: recursive
- name: Setup Python - name: Setup Python
uses: ./.github/actions/setup-backend/ uses: $/.github/actions/setup-backend/
- name: Install dependencies - name: Install dependencies
uses: ./.github/actions/cached-dependencies uses: ./.github/actions/cached-dependencies
with: with:
@@ -82,7 +82,13 @@ jobs:
- name: Setup Python - name: Setup Python
uses: $/.github/actions/setup-backend/ uses: $/.github/actions/setup-backend/
- name: Setup Postgres - name: Setup Postgres
uses: ./.github/actions/cached-dependencies # cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with: with:
run: setup-postgres run: setup-postgres
- name: Start Celery worker - name: Start Celery worker
@@ -147,7 +153,13 @@ jobs:
with: with:
run: setup-postgres run: setup-postgres
- name: Start Celery worker - name: Start Celery worker
uses: ./.github/actions/cached-dependencies # cached-dependencies is a submodule (not a plain directory), and
# the $/ self-repository syntax resolves action files directly from
# the repository without performing a real (submodule-aware)
# checkout, so it can't see into a submodule's link. Keep this one
# on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with: with:
run: celery-worker run: celery-worker
- name: Python unit tests (PostgreSQL) - name: Python unit tests (PostgreSQL)
+1 -1
View File
@@ -51,7 +51,7 @@ jobs:
# checkout, so it can't see into a submodule's gitlink. Keep this one # checkout, so it can't see into a submodule's gitlink. Keep this one
# on the workspace-relative ./ form, consistent with every other # on the workspace-relative ./ form, consistent with every other
# workflow in the repo that references this action. # workflow in the repo that references this action.
uses: ./.github/actions/cached-dependencies uses: ./.github/actions/cached-dependencies # zizmor: ignore[self-repository] - $/ cannot resolve an action that lives in a submodule; ./ is required here
with: with:
run: npm-install run: npm-install
- name: lint - name: lint
+9 -9
View File
@@ -6924,9 +6924,9 @@ color-name@~1.1.4:
integrity sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA== integrity sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==
colord@^2.9.3: colord@^2.9.3:
version "2.9.3" version "2.10.0"
resolved "https://registry.yarnpkg.com/colord/-/colord-2.9.3.tgz#4f8ce919de456f1d5c1c368c307fe20f3e59fb43" resolved "https://registry.yarnpkg.com/colord/-/colord-2.10.0.tgz#56c9050e6b06b4b6c62ddec366a48d65ef57e860"
integrity sha512-jeC1axXpnb0/2nn/Y1LPuLdgXBLH7aDcHu4KEKfqw3CUhX7ZpfBSlPKyqXE6btIgEzfWtrX3/tyBCaCvXvMkOw== integrity sha512-AidJptpBJmjTclAp9BkLwJi0T93fo5epJnbaZslpg6QVzpHjAiveF55mE9AcUJiGMqRHgMDY8soMsQtuNYMHfw==
colorette@^1.2.0: colorette@^1.2.0:
version "1.4.0" version "1.4.0"
@@ -9646,9 +9646,9 @@ jiti@^1.20.0:
integrity sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A== integrity sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A==
joi@^17.9.2: joi@^17.9.2:
version "17.13.4" version "17.13.7"
resolved "https://registry.yarnpkg.com/joi/-/joi-17.13.4.tgz#ad6153d97ce558eb3a3b593e0d43eab51df1c474" resolved "https://registry.yarnpkg.com/joi/-/joi-17.13.7.tgz#92e212c50dbbbcb1a1592424f84083eb265cc778"
integrity sha512-1RuuER6kmt8K8I3nIWvPZKi5RQCb568ZPyY4Pwjlua+yo+63ZTmIwxLZH0heBmiKN4uxjvCiarDrjaeH84xicQ== integrity sha512-MF80Dm5Y2veNy8QWVx9Bj3ui4mo7+VPSPsR1M+oaHXV0Gx6zGX9a2F+OZG3Blby9tOlzU9Rs5FUimlEhbKtfnQ==
dependencies: dependencies:
"@hapi/hoek" "^9.3.0" "@hapi/hoek" "^9.3.0"
"@hapi/topo" "^5.1.0" "@hapi/topo" "^5.1.0"
@@ -14184,9 +14184,9 @@ svg-parser@^2.0.4:
integrity sha512-e4hG1hRwoOdRb37cIMSgzNsxyzKfayW6VOflrwvR+/bzrkyxY/31WkbgnQpgtrNp1SdpJvpUAGTa/ZoiPNDuRQ== integrity sha512-e4hG1hRwoOdRb37cIMSgzNsxyzKfayW6VOflrwvR+/bzrkyxY/31WkbgnQpgtrNp1SdpJvpUAGTa/ZoiPNDuRQ==
svgo@^3.0.2, svgo@^3.2.0: svgo@^3.0.2, svgo@^3.2.0:
version "3.3.4" version "3.3.5"
resolved "https://registry.yarnpkg.com/svgo/-/svgo-3.3.4.tgz#fd2aa10ff585b3bd2b83ce3602f5582bc0718bb5" resolved "https://registry.yarnpkg.com/svgo/-/svgo-3.3.5.tgz#8a3d9557ab2f386eca7e24760385849554985a1c"
integrity sha512-GsNRis4e8jxn2Y9ENz/8lbJ93CstG8svtMnuRaHbiF2LTJ5tK0/q3t/URPq9Zc7zVWBJnNnJMIp6bevK7bSmNg== integrity sha512-8SQMzdrvWaD8deUmrnYB+ASyxBVgWUOilg+A75nE/76WdLpj6LopCwiAVvkzkcqy/9b7t2Mg7faFLjg0ZRcZ3w==
dependencies: dependencies:
commander "^7.2.0" commander "^7.2.0"
css-select "^5.1.0" css-select "^5.1.0"
+1 -1
View File
@@ -54,7 +54,7 @@ dependencies = [
"deprecation>=2.1.0, <2.2.0", "deprecation>=2.1.0, <2.2.0",
"flask>=2.2.5, <4.0.0", "flask>=2.2.5, <4.0.0",
"flask-appbuilder>=5.2.2, <6.0.0", "flask-appbuilder>=5.2.2, <6.0.0",
"flask-caching>=2.4.1, <3", "flask-caching>=2.5.0, <3",
"flask-compress>=1.13, <2.0", "flask-compress>=1.13, <2.0",
"flask-talisman>=1.0.0, <2.0", "flask-talisman>=1.0.0, <2.0",
"flask-login>=0.6.0, < 1.0", "flask-login>=0.6.0, < 1.0",
+5 -4
View File
@@ -40,7 +40,7 @@ brotli==1.2.0
# via # via
# -r requirements/base.in # -r requirements/base.in
# flask-compress # flask-compress
cachelib==0.13.0 cachelib==0.17.0
# via # via
# flask-caching # flask-caching
# flask-session # flask-session
@@ -105,7 +105,7 @@ et-xmlfile==2.0.0
# via openpyxl # via openpyxl
filelock==3.20.3 filelock==3.20.3
# via -r requirements/base.in # via -r requirements/base.in
flask==2.3.3 flask==3.1.3
# via # via
# apache-superset (pyproject.toml) # apache-superset (pyproject.toml)
# flask-appbuilder # flask-appbuilder
@@ -124,9 +124,9 @@ flask-appbuilder==5.2.2
# via # via
# apache-superset (pyproject.toml) # apache-superset (pyproject.toml)
# apache-superset-core # apache-superset-core
flask-babel==3.1.0 flask-babel==4.0.0
# via flask-appbuilder # via flask-appbuilder
flask-caching==2.4.1 flask-caching==2.5.0
# via apache-superset (pyproject.toml) # via apache-superset (pyproject.toml)
flask-compress==1.24 flask-compress==1.24
# via apache-superset (pyproject.toml) # via apache-superset (pyproject.toml)
@@ -218,6 +218,7 @@ markdown-it-py==3.0.0
# via rich # via rich
markupsafe==3.0.2 markupsafe==3.0.2
# via # via
# flask
# jinja2 # jinja2
# mako # mako
# werkzeug # werkzeug
+5 -4
View File
@@ -94,7 +94,7 @@ brotli==1.2.0
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# flask-compress # flask-compress
cachelib==0.13.0 cachelib==0.17.0
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# flask-caching # flask-caching
@@ -247,7 +247,7 @@ filelock==3.20.3
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# virtualenv # virtualenv
flask==2.3.3 flask==3.1.3
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# apache-superset # apache-superset
@@ -269,11 +269,11 @@ flask-appbuilder==5.2.2
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# apache-superset # apache-superset
# apache-superset-core # apache-superset-core
flask-babel==3.1.0 flask-babel==4.0.0
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# flask-appbuilder # flask-appbuilder
flask-caching==2.4.1 flask-caching==2.5.0
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# apache-superset # apache-superset
@@ -524,6 +524,7 @@ markdown-it-py==3.0.0
markupsafe==3.0.2 markupsafe==3.0.2
# via # via
# -c requirements/base-constraint.txt # -c requirements/base-constraint.txt
# flask
# jinja2 # jinja2
# mako # mako
# werkzeug # werkzeug
+89 -114
View File
@@ -8007,11 +8007,12 @@
} }
}, },
"node_modules/@mapbox/jsonlint-lines-primitives": { "node_modules/@mapbox/jsonlint-lines-primitives": {
"version": "2.0.2", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/@mapbox/jsonlint-lines-primitives/-/jsonlint-lines-primitives-2.0.2.tgz", "resolved": "https://registry.npmjs.org/@mapbox/jsonlint-lines-primitives/-/jsonlint-lines-primitives-2.0.3.tgz",
"integrity": "sha512-rY0o9A5ECsTQRVhv7tL/OyDpGAoUB4tTvLiW1DSzQGq4bvTPhNw1VpSNjDJc5GFZ2XuyOtSWSVN05qOtcD71qQ==", "integrity": "sha512-0SElaV0uMxEnxzBhhX9WTuPyUeMsAN/SS0i16tjuba4/mio63MG9khjC1a0JAiPGXAwvwm4UfHJURCN7nyudQg==",
"license": "MIT",
"engines": { "engines": {
"node": ">= 0.6" "node": ">= 22"
} }
}, },
"node_modules/@mapbox/martini": { "node_modules/@mapbox/martini": {
@@ -8027,9 +8028,9 @@
"license": "ISC" "license": "ISC"
}, },
"node_modules/@mapbox/tiny-sdf": { "node_modules/@mapbox/tiny-sdf": {
"version": "2.1.0", "version": "2.2.0",
"resolved": "https://registry.npmjs.org/@mapbox/tiny-sdf/-/tiny-sdf-2.1.0.tgz", "resolved": "https://registry.npmjs.org/@mapbox/tiny-sdf/-/tiny-sdf-2.2.0.tgz",
"integrity": "sha512-uFJhNh36BR4OCuWIEiWaEix9CA2WzT6CAIcqVjWYpnx8+QDtS+oC4QehRrx5cX4mgWs37MmKnwUejeHxVymzNg==", "integrity": "sha512-LVL4wgI9YAum5V+LNVQO6QgFBPw7/MIIY4XJPNsPDMrjEwcE+JfKk1LuIl8GnF197ejVdC9QdPaxrx5gfgdGXg==",
"license": "BSD-2-Clause" "license": "BSD-2-Clause"
}, },
"node_modules/@mapbox/unitbezier": { "node_modules/@mapbox/unitbezier": {
@@ -8047,29 +8048,23 @@
"@mapbox/point-geometry": "~0.1.0" "@mapbox/point-geometry": "~0.1.0"
} }
}, },
"node_modules/@mapbox/whoots-js": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/@mapbox/whoots-js/-/whoots-js-3.1.0.tgz",
"integrity": "sha512-Es6WcD0nO5l+2BOQS4uLfNPYQaNDfbot3X1XUoloz+x0mPDS3eeORZJl06HXjwBG1fOGwCRnzK88LMdxKRrd6Q==",
"license": "ISC",
"engines": {
"node": ">=6.0.0"
}
},
"node_modules/@maplibre/geojson-vt": { "node_modules/@maplibre/geojson-vt": {
"version": "5.0.4", "version": "6.1.1",
"resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-5.0.4.tgz", "resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-6.1.1.tgz",
"integrity": "sha512-KGg9sma45S+stfH9vPCJk1J0lSDLWZgCT9Y8u8qWZJyjFlP8MNP1WGTxIMYJZjDvVT3PDn05kN1C95Sut1HpgQ==", "integrity": "sha512-FVMOcmSP/yqol45t7StApEyTL5/vmqBCuFhH9n+fFuINenhaX+YgHHIt1yJ86S8kln3uJLcMvmEU2cfn6E2eCQ==",
"license": "ISC"
},
"node_modules/@maplibre/maplibre-gl-style-spec": {
"version": "24.8.5",
"resolved": "https://registry.npmjs.org/@maplibre/maplibre-gl-style-spec/-/maplibre-gl-style-spec-24.8.5.tgz",
"integrity": "sha512-EzEJmMt6thioRH7GI9LWS7ahXTcAhAPGWCe6oTP2Ps4YnsXOOAfeqx854lZaiDnwURfHmcCKV1mr6oo0i23x6w==",
"license": "ISC", "license": "ISC",
"dependencies": { "dependencies": {
"@mapbox/jsonlint-lines-primitives": "~2.0.2", "kdbush": "^4.1.0"
"@mapbox/unitbezier": "^0.0.1", }
},
"node_modules/@maplibre/maplibre-gl-style-spec": {
"version": "26.4.2",
"resolved": "https://registry.npmjs.org/@maplibre/maplibre-gl-style-spec/-/maplibre-gl-style-spec-26.4.2.tgz",
"integrity": "sha512-6J0vZqMZvRAJKtdWJdDGHEh1YJ2ZHG08/GOur8gCArYhO8ZkM//OXqtI2AAEz4jc2G+Wq4MfcePg8qNK5TZ4Kg==",
"license": "ISC",
"dependencies": {
"@mapbox/jsonlint-lines-primitives": "^2.0.3",
"@mapbox/unitbezier": "^1.0.0",
"json-stringify-pretty-compact": "^4.0.0", "json-stringify-pretty-compact": "^4.0.0",
"minimist": "^1.2.8", "minimist": "^1.2.8",
"quickselect": "^3.0.0", "quickselect": "^3.0.0",
@@ -8081,10 +8076,16 @@
"gl-style-validate": "dist/gl-style-validate.mjs" "gl-style-validate": "dist/gl-style-validate.mjs"
} }
}, },
"node_modules/@maplibre/maplibre-gl-style-spec/node_modules/@mapbox/unitbezier": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/unitbezier/-/unitbezier-1.0.0.tgz",
"integrity": "sha512-fqd515fjBmANKGGsQ286E2Wvj/XvDFpGzwJxq4CI6jMQue6Oy04uCKp+JWKF00xRTmk6cEu1jPJ9p3xqH8YWqQ==",
"license": "BSD-2-Clause"
},
"node_modules/@maplibre/mlt": { "node_modules/@maplibre/mlt": {
"version": "1.1.8", "version": "1.2.1",
"resolved": "https://registry.npmjs.org/@maplibre/mlt/-/mlt-1.1.8.tgz", "resolved": "https://registry.npmjs.org/@maplibre/mlt/-/mlt-1.2.1.tgz",
"integrity": "sha512-8vtfYGidr1rNkv5IwIoU2lfe3Oy+Wa8HluzQYcQi9cveU9K3pweAal/poQj4GJ0K/EW4bTQp2wVAs09g2yDRZg==", "integrity": "sha512-5n5dgolE2EYxwCKgx8vlwURCB8A+kyfJJyThlYimjlGgOcOe2Bhw9VxxnnHC91OC9PgHg+nVdgVPTYPkIo62vg==",
"license": "(MIT OR Apache-2.0)", "license": "(MIT OR Apache-2.0)",
"dependencies": { "dependencies": {
"@mapbox/point-geometry": "^1.1.0" "@mapbox/point-geometry": "^1.1.0"
@@ -8097,18 +8098,14 @@
"license": "ISC" "license": "ISC"
}, },
"node_modules/@maplibre/vt-pbf": { "node_modules/@maplibre/vt-pbf": {
"version": "4.3.0", "version": "4.3.2",
"resolved": "https://registry.npmjs.org/@maplibre/vt-pbf/-/vt-pbf-4.3.0.tgz", "resolved": "https://registry.npmjs.org/@maplibre/vt-pbf/-/vt-pbf-4.3.2.tgz",
"integrity": "sha512-jIvp8F5hQCcreqOOpEt42TJMUlsrEcpf/kI1T2v85YrQRV6PPXUcEXUg5karKtH6oh47XJZ4kHu56pUkOuqA7w==", "integrity": "sha512-j6p0AdjvAR19Z3XaCysle7A4ZSo08tYOzxD0Y9NQylwPAkwJJeYub5b2eVucdeDh7erhv69DahoLOevDRERRUw==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@mapbox/point-geometry": "^1.1.0", "@mapbox/point-geometry": "^1.1.0",
"@mapbox/vector-tile": "^2.0.4",
"@maplibre/geojson-vt": "^5.0.4",
"@types/geojson": "^7946.0.16", "@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3", "pbf": "^5.1.0"
"pbf": "^4.0.1",
"supercluster": "^8.0.1"
} }
}, },
"node_modules/@maplibre/vt-pbf/node_modules/@mapbox/point-geometry": { "node_modules/@maplibre/vt-pbf/node_modules/@mapbox/point-geometry": {
@@ -8117,21 +8114,10 @@
"integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==", "integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/@maplibre/vt-pbf/node_modules/@mapbox/vector-tile": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-2.0.5.tgz",
"integrity": "sha512-pXj8m7KTsqZt+1jsE0xIpGvqTSbblfkuEJL/NJmNePMtEwxO8V3XMDo9WMSfDeqHvCtBI9Lmt4mGcGR10zecmw==",
"license": "BSD-3-Clause",
"dependencies": {
"@mapbox/point-geometry": "~1.1.0",
"@types/geojson": "^7946.0.16",
"pbf": "^4.0.2"
}
},
"node_modules/@maplibre/vt-pbf/node_modules/pbf": { "node_modules/@maplibre/vt-pbf/node_modules/pbf": {
"version": "4.0.2", "version": "5.1.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-4.0.2.tgz", "resolved": "https://registry.npmjs.org/pbf/-/pbf-5.1.2.tgz",
"integrity": "sha512-J0ajxARhZfpUEebxYs1vhMGMuLSXtBe1e+fFPDrf2uA2hgo+UshKfNUWOz92HJNz6/NFEXseQPddnHkTreWRqg==", "integrity": "sha512-mnvGdvOrIvJOBGUEdGkrVXjN8E/VkIJCkf2eS1DH2yv82ORUlLttmDt0rWY38yYZmVwciZwBUvHM20qxBZf40w==",
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
"resolve-protobuf-schema": "^2.1.0" "resolve-protobuf-schema": "^2.1.0"
@@ -28781,9 +28767,9 @@
} }
}, },
"node_modules/jest-process-manager/node_modules/joi": { "node_modules/jest-process-manager/node_modules/joi": {
"version": "17.13.4", "version": "17.13.7",
"resolved": "https://registry.npmjs.org/joi/-/joi-17.13.4.tgz", "resolved": "https://registry.npmjs.org/joi/-/joi-17.13.7.tgz",
"integrity": "sha512-1RuuER6kmt8K8I3nIWvPZKi5RQCb568ZPyY4Pwjlua+yo+63ZTmIwxLZH0heBmiKN4uxjvCiarDrjaeH84xicQ==", "integrity": "sha512-MF80Dm5Y2veNy8QWVx9Bj3ui4mo7+VPSPsR1M+oaHXV0Gx6zGX9a2F+OZG3Blby9tOlzU9Rs5FUimlEhbKtfnQ==",
"dev": true, "dev": true,
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
@@ -30783,9 +30769,9 @@
} }
}, },
"node_modules/joi": { "node_modules/joi": {
"version": "18.2.3", "version": "18.2.8",
"resolved": "https://registry.npmjs.org/joi/-/joi-18.2.3.tgz", "resolved": "https://registry.npmjs.org/joi/-/joi-18.2.8.tgz",
"integrity": "sha512-N5A3KTWQpPWT4ExxxPlUx7WmykGXRzhNidWhV41d6Abu9YfI2NyWCJuxdPnslJCPWtbRpSVOWSnSS6GakLM/Rg==", "integrity": "sha512-G2TX62h58ZHuwqetJgP2F4ualakqAmZtBYe3jWen7gxQRw5xApX6crnFtuB91WC0c3ESBnva+kGSnb3+6pIQDQ==",
"dev": true, "dev": true,
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
@@ -32288,7 +32274,8 @@
"version": "4.4.2", "version": "4.4.2",
"resolved": "https://registry.npmjs.org/lodash.get/-/lodash.get-4.4.2.tgz", "resolved": "https://registry.npmjs.org/lodash.get/-/lodash.get-4.4.2.tgz",
"integrity": "sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==", "integrity": "sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==",
"license": "MIT" "license": "MIT",
"peer": true
}, },
"node_modules/lodash.isempty": { "node_modules/lodash.isempty": {
"version": "4.4.0", "version": "4.4.0",
@@ -32300,7 +32287,8 @@
"version": "4.5.0", "version": "4.5.0",
"resolved": "https://registry.npmjs.org/lodash.isequal/-/lodash.isequal-4.5.0.tgz", "resolved": "https://registry.npmjs.org/lodash.isequal/-/lodash.isequal-4.5.0.tgz",
"integrity": "sha512-pDo3lu8Jhfjqls6GkMgpahsF9kCyayhgykjyLMNFTKWrpVdAQtYyB4muAMWozBB4ig/dtWAmsMxLEI8wuz+DYQ==", "integrity": "sha512-pDo3lu8Jhfjqls6GkMgpahsF9kCyayhgykjyLMNFTKWrpVdAQtYyB4muAMWozBB4ig/dtWAmsMxLEI8wuz+DYQ==",
"license": "MIT" "license": "MIT",
"peer": true
}, },
"node_modules/lodash.isplainobject": { "node_modules/lodash.isplainobject": {
"version": "4.0.6", "version": "4.0.6",
@@ -32575,27 +32563,25 @@
] ]
}, },
"node_modules/maplibre-gl": { "node_modules/maplibre-gl": {
"version": "5.24.0", "version": "6.8.0",
"resolved": "https://registry.npmjs.org/maplibre-gl/-/maplibre-gl-5.24.0.tgz", "resolved": "https://registry.npmjs.org/maplibre-gl/-/maplibre-gl-6.8.0.tgz",
"integrity": "sha512-ALyFxgtd5R+65UqZ/++lOqwWcC0SNho9c27fYSyLmG7AfnAul2o46F05aDJGPbFU57wos9dgcIySHs0Xe6ia3A==", "integrity": "sha512-+ZkjKTodsVLY0ewQThvXRxXQsclcsSgOm5LlnrBM3G8AloMJKt6Haw8mY4xrOl8T0WMqH6DTYjktZ9zGQXZd4w==",
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
"@mapbox/jsonlint-lines-primitives": "^2.0.2",
"@mapbox/point-geometry": "^1.1.0", "@mapbox/point-geometry": "^1.1.0",
"@mapbox/tiny-sdf": "^2.1.0", "@mapbox/tiny-sdf": "^2.2.0",
"@mapbox/unitbezier": "^0.0.1", "@mapbox/unitbezier": "^1.0.0",
"@mapbox/vector-tile": "^2.0.4", "@mapbox/vector-tile": "^3.0.0",
"@mapbox/whoots-js": "^3.1.0", "@maplibre/geojson-vt": "^6.1.1",
"@maplibre/geojson-vt": "^6.1.0", "@maplibre/maplibre-gl-style-spec": "^26.4.1",
"@maplibre/maplibre-gl-style-spec": "^24.8.1", "@maplibre/mlt": "^1.2.1",
"@maplibre/mlt": "^1.1.8", "@maplibre/vt-pbf": "^4.3.2",
"@maplibre/vt-pbf": "^4.3.0",
"@types/geojson": "^7946.0.16", "@types/geojson": "^7946.0.16",
"earcut": "^3.0.2", "earcut": "^3.2.3",
"gl-matrix": "^3.4.4", "gl-matrix": "^3.4.4",
"kdbush": "^4.0.2", "kdbush": "^4.1.0",
"murmurhash-js": "^1.0.0", "murmurhash-js": "^1.0.0",
"pbf": "^4.0.1", "pbf": "^5.1.2",
"potpack": "^2.1.0", "potpack": "^2.1.0",
"quickselect": "^3.0.0", "quickselect": "^3.0.0",
"tinyqueue": "^3.0.0" "tinyqueue": "^3.0.0"
@@ -32614,36 +32600,33 @@
"integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==", "integrity": "sha512-YGcBz1cg4ATXDCM/71L9xveh4dynfGmcLDqufR+nQQy3fKwsAZsWd/x4621/6uJaeB9mwOHE6hPeDgXz9uViUQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/maplibre-gl/node_modules/@mapbox/unitbezier": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/unitbezier/-/unitbezier-1.0.0.tgz",
"integrity": "sha512-fqd515fjBmANKGGsQ286E2Wvj/XvDFpGzwJxq4CI6jMQue6Oy04uCKp+JWKF00xRTmk6cEu1jPJ9p3xqH8YWqQ==",
"license": "BSD-2-Clause"
},
"node_modules/maplibre-gl/node_modules/@mapbox/vector-tile": { "node_modules/maplibre-gl/node_modules/@mapbox/vector-tile": {
"version": "2.0.5", "version": "3.0.0",
"resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-2.0.5.tgz", "resolved": "https://registry.npmjs.org/@mapbox/vector-tile/-/vector-tile-3.0.0.tgz",
"integrity": "sha512-pXj8m7KTsqZt+1jsE0xIpGvqTSbblfkuEJL/NJmNePMtEwxO8V3XMDo9WMSfDeqHvCtBI9Lmt4mGcGR10zecmw==", "integrity": "sha512-Qf10S1uIHMk20ri/IVBnpS+esUEkVaR5Hftmz88jTInrpmWgPGJfPe3LVjjlE77trLx8tH6qjTG7uWH9hIq/0Q==",
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
"@mapbox/point-geometry": "~1.1.0", "@mapbox/point-geometry": "~1.1.0",
"@types/geojson": "^7946.0.16", "@types/geojson": "^7946.0.16",
"pbf": "^4.0.2" "pbf": "^5.0.0"
}
},
"node_modules/maplibre-gl/node_modules/@maplibre/geojson-vt": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/@maplibre/geojson-vt/-/geojson-vt-6.1.0.tgz",
"integrity": "sha512-2eIY4gZxeKIVOZVNkAMb+5NgXhgsMQpOveTQAvnp53LYqHGJZDidk7Ew0Tged9PThidpbS+NFTh0g4zivhPDzQ==",
"license": "ISC",
"dependencies": {
"kdbush": "^4.0.2"
} }
}, },
"node_modules/maplibre-gl/node_modules/earcut": { "node_modules/maplibre-gl/node_modules/earcut": {
"version": "3.0.2", "version": "3.2.3",
"resolved": "https://registry.npmjs.org/earcut/-/earcut-3.0.2.tgz", "resolved": "https://registry.npmjs.org/earcut/-/earcut-3.2.3.tgz",
"integrity": "sha512-X7hshQbLyMJ/3RPhyObLARM2sNxxmRALLKx1+NVFFnQ9gKzmCrxm9+uLIAdBcvc8FNLpctqlQ2V6AE92Ol9UDQ==", "integrity": "sha512-vnS4AVwp1KHAF13i1vp1/2D5evWy3k5u/iW/B81QVsUZtV8cv2tU0b2VNFlqvh4kYwrFMDdjPCfAmfyJW9y14Q==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/maplibre-gl/node_modules/pbf": { "node_modules/maplibre-gl/node_modules/pbf": {
"version": "4.0.2", "version": "5.1.2",
"resolved": "https://registry.npmjs.org/pbf/-/pbf-4.0.2.tgz", "resolved": "https://registry.npmjs.org/pbf/-/pbf-5.1.2.tgz",
"integrity": "sha512-J0ajxARhZfpUEebxYs1vhMGMuLSXtBe1e+fFPDrf2uA2hgo+UshKfNUWOz92HJNz6/NFEXseQPddnHkTreWRqg==", "integrity": "sha512-mnvGdvOrIvJOBGUEdGkrVXjN8E/VkIJCkf2eS1DH2yv82ORUlLttmDt0rWY38yYZmVwciZwBUvHM20qxBZf40w==",
"license": "BSD-3-Clause", "license": "BSD-3-Clause",
"dependencies": { "dependencies": {
"resolve-protobuf-schema": "^2.1.0" "resolve-protobuf-schema": "^2.1.0"
@@ -42614,15 +42597,6 @@
"integrity": "sha512-Orov6g6BB1sDfYgzWfTHDOxamtX1bE/zo104Dh9e6fqJ3PooipYyfJ0pUmrZO2wAvO8YbEyeFrkV91XTsGMSrw==", "integrity": "sha512-Orov6g6BB1sDfYgzWfTHDOxamtX1bE/zo104Dh9e6fqJ3PooipYyfJ0pUmrZO2wAvO8YbEyeFrkV91XTsGMSrw==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/supercluster": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/supercluster/-/supercluster-8.0.1.tgz",
"integrity": "sha512-IiOea5kJ9iqzD2t7QJq/cREyLHTtSmUT6gQsweojg9WH2sYJqZK9SswTu6jrscO6D1G5v5vYZ9ru/eq85lXeZQ==",
"license": "ISC",
"dependencies": {
"kdbush": "^4.0.2"
}
},
"node_modules/supports-color": { "node_modules/supports-color": {
"version": "7.2.0", "version": "7.2.0",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz", "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
@@ -42715,9 +42689,9 @@
"dev": true "dev": true
}, },
"node_modules/svgo": { "node_modules/svgo": {
"version": "3.3.4", "version": "3.3.5",
"resolved": "https://registry.npmjs.org/svgo/-/svgo-3.3.4.tgz", "resolved": "https://registry.npmjs.org/svgo/-/svgo-3.3.5.tgz",
"integrity": "sha512-GsNRis4e8jxn2Y9ENz/8lbJ93CstG8svtMnuRaHbiF2LTJ5tK0/q3t/URPq9Zc7zVWBJnNnJMIp6bevK7bSmNg==", "integrity": "sha512-8SQMzdrvWaD8deUmrnYB+ASyxBVgWUOilg+A75nE/76WdLpj6LopCwiAVvkzkcqy/9b7t2Mg7faFLjg0ZRcZ3w==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
@@ -47102,7 +47076,7 @@
"math-expression-evaluator": "^2.0.7", "math-expression-evaluator": "^2.0.7",
"parse-ms": "^4.0.0", "parse-ms": "^4.0.0",
"re-resizable": "^6.11.2", "re-resizable": "^6.11.2",
"react-ace": "^14.0.1", "react-ace": "^15.0.0",
"react-draggable": "^4.7.1", "react-draggable": "^4.7.1",
"react-error-boundary": "^6.1.4", "react-error-boundary": "^6.1.4",
"react-js-cron": "^6.0.2", "react-js-cron": "^6.0.2",
@@ -47203,15 +47177,14 @@
} }
}, },
"packages/superset-ui-core/node_modules/react-ace": { "packages/superset-ui-core/node_modules/react-ace": {
"version": "14.0.1", "version": "15.0.0",
"resolved": "https://registry.npmjs.org/react-ace/-/react-ace-14.0.1.tgz", "resolved": "https://registry.npmjs.org/react-ace/-/react-ace-15.0.0.tgz",
"integrity": "sha512-z6YAZ20PNf/FqmYEic//G/UK6uw0rn21g58ASgHJHl9rfE4nITQLqthr9rHMVQK4ezwohJbp2dGrZpkq979PYQ==", "integrity": "sha512-gdmS5ftF0hsbkcrAjZQqYbXwFg5JrHuxjki8mP6Bn9kfa6lHKfZk9sU9EfS0ifQ1CpGCmRxF/VC7GRvlJMBuZw==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"ace-builds": "^1.36.3", "ace-builds": "^1.36.3",
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"lodash.get": "^4.4.2", "fast-equals": "^5.3.3",
"lodash.isequal": "^4.5.0",
"prop-types": "^15.8.1" "prop-types": "^15.8.1"
}, },
"peerDependencies": { "peerDependencies": {
@@ -47593,8 +47566,10 @@
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@math.gl/web-mercator": "^4.1.0", "@math.gl/web-mercator": "^4.1.0",
"@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3",
"mapbox-gl": "^3.29.0", "mapbox-gl": "^3.29.0",
"maplibre-gl": "^5.24.0", "maplibre-gl": "^6.8.0",
"react-map-gl": "^8.1.2", "react-map-gl": "^8.1.2",
"supercluster": "^9.0.0" "supercluster": "^9.0.0"
}, },
@@ -47737,7 +47712,7 @@
"handlebars": "^4.7.9", "handlebars": "^4.7.9",
"lodash": "^4.18.1", "lodash": "^4.18.1",
"lodash-es": "^4.18.1", "lodash-es": "^4.18.1",
"maplibre-gl": "^5.24.0", "maplibre-gl": "^6.8.0",
"mousetrap": "^1.6.5", "mousetrap": "^1.6.5",
"ngeohash": "^0.6.4", "ngeohash": "^0.6.4",
"prop-types": "^15.8.1", "prop-types": "^15.8.1",
@@ -78,7 +78,7 @@
"math-expression-evaluator": "^2.0.7", "math-expression-evaluator": "^2.0.7",
"parse-ms": "^4.0.0", "parse-ms": "^4.0.0",
"re-resizable": "^6.11.2", "re-resizable": "^6.11.2",
"react-ace": "^14.0.1", "react-ace": "^15.0.0",
"react-draggable": "^4.7.1", "react-draggable": "^4.7.1",
"react-error-boundary": "^6.1.4", "react-error-boundary": "^6.1.4",
"react-js-cron": "^6.0.2", "react-js-cron": "^6.0.2",
@@ -604,6 +604,61 @@ test('cleans up event listeners on unmount', async () => {
offSpy.mockRestore(); offSpy.mockRestore();
}); });
test('re-applies annotations only when their content actually changes across renders (react-ace 15 fast-equals regression guard)', async () => {
// react-ace's componentDidUpdate decides whether to call
// session.setAnnotations() by deep-comparing the new/old `annotations`
// prop (lib/ace.js, using an internal deep-equality helper -- lodash's
// isEqual through react-ace 14.x, fast-equals's deepEqual from 15.0.0
// onward). Superset's own AceEditorProvider/EditorWrapper always pass a
// freshly `.map()`-derived annotations array on every render, so this
// guards the actual behavior Superset relies on: a same-content-but-
// different-reference array must NOT re-trigger setAnnotations (or the
// editor would thrash on every keystroke-driven re-render), while a
// genuinely different array must still update the editor.
const ref = createRef<AceEditor>();
const annotationsV1 = [{ row: 0, column: 0, type: 'error', text: 'oops' }];
const { rerender, container } = render(
<SQLEditor ref={ref as React.Ref<never>} annotations={annotationsV1} />,
);
await waitFor(() => {
expect(container.querySelector(selector)).toBeInTheDocument();
});
const session = ref.current?.editor?.getSession();
expect(session).toBeDefined();
if (!session) return;
// The initial mount already applies annotations via componentDidMount,
// not componentDidUpdate, so start observing only from the first update.
const setAnnotationsSpy = jest.spyOn(session, 'setAnnotations');
// Same content, new array/object references -- must be a no-op.
const annotationsV1SameContent = [
{ row: 0, column: 0, type: 'error', text: 'oops' },
];
rerender(
<SQLEditor
ref={ref as React.Ref<never>}
annotations={annotationsV1SameContent}
/>,
);
expect(setAnnotationsSpy).not.toHaveBeenCalled();
// Genuinely different content -- must update, with the new value.
const annotationsV2 = [
{ row: 1, column: 2, type: 'warning', text: 'different' },
];
rerender(
<SQLEditor ref={ref as React.Ref<never>} annotations={annotationsV2} />,
);
expect(setAnnotationsSpy).toHaveBeenCalledTimes(1);
expect(setAnnotationsSpy).toHaveBeenCalledWith(annotationsV2);
setAnnotationsSpy.mockRestore();
});
test('does not move autocomplete popup if target container is document.body', async () => { test('does not move autocomplete popup if target container is document.body', async () => {
const ref = createRef<AceEditor>(); const ref = createRef<AceEditor>();
const { container } = render(<SQLEditor ref={ref as React.Ref<never>} />); const { container } = render(<SQLEditor ref={ref as React.Ref<never>} />);
@@ -30,8 +30,10 @@
}, },
"dependencies": { "dependencies": {
"@math.gl/web-mercator": "^4.1.0", "@math.gl/web-mercator": "^4.1.0",
"@types/geojson": "^7946.0.16",
"@types/supercluster": "^7.1.3",
"mapbox-gl": "^3.29.0", "mapbox-gl": "^3.29.0",
"maplibre-gl": "^5.24.0", "maplibre-gl": "^6.8.0",
"react-map-gl": "^8.1.2", "react-map-gl": "^8.1.2",
"supercluster": "^9.0.0" "supercluster": "^9.0.0"
}, },
@@ -49,7 +49,7 @@
"handlebars": "^4.7.9", "handlebars": "^4.7.9",
"lodash": "^4.18.1", "lodash": "^4.18.1",
"lodash-es": "^4.18.1", "lodash-es": "^4.18.1",
"maplibre-gl": "^5.24.0", "maplibre-gl": "^6.8.0",
"mousetrap": "^1.6.5", "mousetrap": "^1.6.5",
"ngeohash": "^0.6.4", "ngeohash": "^0.6.4",
"prop-types": "^15.8.1", "prop-types": "^15.8.1",
@@ -16,7 +16,13 @@
* specific language governing permissions and limitations * specific language governing permissions and limitations
* under the License. * under the License.
*/ */
import { Preset, VizType } from '@superset-ui/core'; import {
ChartLabel,
ChartMetadata,
ChartPlugin,
Preset,
VizType,
} from '@superset-ui/core';
import { import {
render, render,
cleanup, cleanup,
@@ -47,12 +53,31 @@ jest.mock('scroll-into-view-if-needed', () => jest.fn());
jest.useFakeTimers({ advanceTimers: true }); jest.useFakeTimers({ advanceTimers: true });
// A minimal plugin carrying a "Featured" label, so tests can assert on the
// badge that VizTypeGallery overlays on its thumbnail.
class FeaturedTestChartPlugin extends ChartPlugin {
constructor() {
super({
metadata: new ChartMetadata({
name: 'Featured Test Chart',
thumbnail: '',
label: ChartLabel.Featured,
tags: ['Featured'],
}),
Chart: () => null,
});
}
}
class MainPreset extends Preset { class MainPreset extends Preset {
constructor() { constructor() {
super({ super({
name: 'Legacy charts', name: 'Legacy charts',
plugins: [ plugins: [
new TableChartPlugin().configure({ key: VizType.Table }), new TableChartPlugin().configure({ key: VizType.Table }),
new FeaturedTestChartPlugin().configure({
key: 'featured_test_chart',
}),
new BigNumberTotalChartPlugin().configure({ new BigNumberTotalChartPlugin().configure({
key: VizType.BigNumberTotal, key: VizType.BigNumberTotal,
}), }),
@@ -278,6 +303,25 @@ describe('VizTypeControl', () => {
).not.toBeInTheDocument(); ).not.toBeInTheDocument();
}); });
test('anchors the Featured badge to the bottom-right of the thumbnail image', async () => {
// The badge is positioned relative to the thumbnail image only (not the
// whole tile), so it must hang off the image's bottom-right corner
// rather than its top edge.
await waitForRenderWrapper();
userEvent.click(screen.getByRole('tab', { name: 'All charts' }));
const visualizations = screen.getByTestId(getTestId('viz-row'));
const image = await within(visualizations).findByAltText(
'Featured Test Chart',
);
const badgeWrapper = image.nextElementSibling as HTMLElement;
expect(badgeWrapper).toHaveStyleRule('bottom', '4px');
expect(badgeWrapper).toHaveStyleRule('right', '4px');
expect(badgeWrapper).not.toHaveStyleRule('top', expect.anything());
expect(within(badgeWrapper).getByText('FEATURED')).toBeInTheDocument();
});
test('Thumbnail labels expose the full chart name via a title tooltip', async () => { test('Thumbnail labels expose the full chart name via a title tooltip', async () => {
// Labels are clamped to a fixed two-line block so every tile is the same // Labels are clamped to a fixed two-line block so every tile is the same
// height; the full (possibly truncated) name must stay discoverable through // height; the full (possibly truncated) name must stay discoverable through
@@ -344,7 +344,7 @@ const ThumbnailImageWrapper = styled.div`
const ThumbnailLabelWrapper = styled.div` const ThumbnailLabelWrapper = styled.div`
position: absolute; position: absolute;
right: ${({ theme }) => theme.sizeUnit}px; right: ${({ theme }) => theme.sizeUnit}px;
top: ${({ theme }) => theme.sizeUnit}px; bottom: ${({ theme }) => theme.sizeUnit}px;
`; `;
const TitleLabelWrapper = styled.div` const TitleLabelWrapper = styled.div`
@@ -153,6 +153,20 @@ test('renders a select and a VizTypeGallery', async () => {
expect(screen.getByText(/choose chart type/i)).toBeInTheDocument(); expect(screen.getByText(/choose chart type/i)).toBeInTheDocument();
}); });
test('does not double up the vertical Steps icon-to-content gap', async () => {
// antd 6 added its own icon->content gap on `.ant-steps-item-wrapper`
// (column-gap), stacking on top of the pre-existing `margin-right` on
// `.ant-steps-item-icon` and shifting every step's content to the right.
const { container } = await renderComponent();
const styledContainer = container.firstChild;
expect(styledContainer).toHaveStyleRule('column-gap', '0', {
target: '.ant-steps-item-wrapper',
});
expect(styledContainer).toHaveStyleRule('margin-right', '8px', {
target: '.ant-steps-item-icon',
});
});
test('renders dataset help text when user lacks dataset write permissions', async () => { test('renders dataset help text when user lacks dataset write permissions', async () => {
await renderComponent(); await renderComponent();
expect(screen.queryByText('Add a dataset')).not.toBeInTheDocument(); expect(screen.queryByText('Add a dataset')).not.toBeInTheDocument();
@@ -141,6 +141,13 @@ const StyledContainer = styled.div`
display: none; display: none;
} }
/* antd 6 added its own icon->content gap on this flex wrapper
(column-gap), on top of the .ant-steps-item-icon margin-right below,
doubling the gap. Zero it out so the icon's margin is the only gap. */
&&&& .ant-steps-item-wrapper {
column-gap: 0;
}
&&&& .ant-steps-item-icon { &&&& .ant-steps-item-icon {
margin-right: ${theme.marginXS}px; margin-right: ${theme.marginXS}px;
width: ${theme.sizeUnit * 5}px; width: ${theme.sizeUnit * 5}px;
+2 -1
View File
@@ -44,8 +44,9 @@ class SupersetMetastoreCache(BaseCache):
namespace: UUID, namespace: UUID,
codec: KeyValueCodec, codec: KeyValueCodec,
default_timeout: int = 300, default_timeout: int = 300,
ignore_delete_many_errors: bool = False,
) -> None: ) -> None:
super().__init__(default_timeout) super().__init__(default_timeout, ignore_delete_many_errors)
self.namespace = namespace self.namespace = namespace
self.codec = codec self.codec = codec
+25 -5
View File
@@ -80,7 +80,6 @@ from superset import db, is_feature_enabled
from superset.advanced_data_type.types import AdvancedDataTypeResponse from superset.advanced_data_type.types import AdvancedDataTypeResponse
from superset.common.db_query_status import QueryStatus from superset.common.db_query_status import QueryStatus
from superset.common.grouping_sets import ( from superset.common.grouping_sets import (
grouping_id_column,
grouping_marker_label, grouping_marker_label,
grouping_sets_clause, grouping_sets_clause,
) )
@@ -4867,10 +4866,31 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
and groupby_all_columns and groupby_all_columns
and db_engine_spec.supports_grouping_sets and db_engine_spec.supports_grouping_sets
) )
# Both the GROUPING() marker labels and the `grouping_sets` level
# definitions sent by the frontend (see buildQuery.ts) are expressed in
# terms of the column's logical/requested label (``.key``), not the
# engine-mutated SQL alias (``.name``). BigQuery, for example, mangles
# labels containing spaces (e.g. a Custom SQL column named "Test Row")
# into something like "Test_Row_a1b2c3" for `.name`, while `.key` keeps
# the original "Test Row". Keying by `.name` here would silently drop
# such columns from every rollup level (the `col in ...` guard below),
# producing an invalid ``GROUP BY GROUPING SETS`` clause that omits a
# selected, non-aggregated column.
groupby_columns_by_label = {
gby_expr.key: gby_expr for gby_expr in groupby_all_columns.values()
}
if use_grouping_sets: if use_grouping_sets:
# Route the marker through `make_sqla_column_compatible` like every
# other selected column: the SQL-level alias is engine-mutated if
# required (e.g. BigQuery rejects aliases with spaces), while
# `.key` keeps the unmutated marker label so it lines up with the
# `groupby_columns_by_label` keys above and with what the frontend
# looks for when splitting the combined result back per level.
select_exprs = select_exprs + [ select_exprs = select_exprs + [
grouping_id_column(gby_expr, grouping_marker_label(name)) self.make_sqla_column_compatible(
for name, gby_expr in groupby_all_columns.items() sa.func.grouping(gby_expr), grouping_marker_label(label)
)
for label, gby_expr in groupby_columns_by_label.items()
] ]
# Expected output columns # Expected output columns
@@ -4887,9 +4907,9 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
if use_grouping_sets: if use_grouping_sets:
gs_levels = [ gs_levels = [
[ [
groupby_all_columns[col] groupby_columns_by_label[col]
for col in level for col in level
if col in groupby_all_columns if col in groupby_columns_by_label
] ]
for level in grouping_sets or [] for level in grouping_sets or []
] ]
+1 -1
View File
@@ -325,7 +325,7 @@ def etag_cache( # noqa: C901
wrapper.uncached = f # type: ignore wrapper.uncached = f # type: ignore
wrapper.cache_timeout = timeout # type: ignore wrapper.cache_timeout = timeout # type: ignore
wrapper.make_cache_key = cache._memoize_make_cache_key( # type: ignore # pylint: disable=protected-access wrapper.make_cache_key = cache._memoize_make_cache_key( # type: ignore # pylint: disable=protected-access
make_name=None, timeout=timeout, hash_method=configurable_hash_method make_name=None, hash_method=configurable_hash_method
) )
return wrapper return wrapper
-4
View File
@@ -147,16 +147,12 @@ class SupersetCache(Cache):
def _memoize_make_cache_key( def _memoize_make_cache_key(
self, self,
make_name: Callable[..., Any] | None = None, make_name: Callable[..., Any] | None = None,
timeout: Callable[..., Any] | None = None,
forced_update: bool = False,
hash_method: Callable[..., Any] = configurable_hash_method, hash_method: Callable[..., Any] = configurable_hash_method,
source_check: bool | None = False, source_check: bool | None = False,
args_to_ignore: Any | None = None, args_to_ignore: Any | None = None,
) -> Callable[..., Any]: ) -> Callable[..., Any]:
return super()._memoize_make_cache_key( return super()._memoize_make_cache_key(
make_name=make_name, make_name=make_name,
timeout=timeout,
forced_update=forced_update,
hash_method=hash_method, hash_method=hash_method,
source_check=source_check, source_check=source_check,
args_to_ignore=args_to_ignore, args_to_ignore=args_to_ignore,
+97
View File
@@ -35,6 +35,10 @@ logger = logging.getLogger(__name__)
# Time to wait after scrolling for content to settle and load (in milliseconds) # Time to wait after scrolling for content to settle and load (in milliseconds)
SCROLL_SETTLE_TIMEOUT_MS = 1000 SCROLL_SETTLE_TIMEOUT_MS = 1000
# Ceiling for un-clipping scrollable chart content (ag-Grid stabilization
# polling) before a screenshot, absent a report deadline to bound it against.
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS = 5.0
# Chromium can occasionally return a valid but uniformly blank PNG for an # Chromium can occasionally return a valid but uniformly blank PNG for an
# off-screen clip. Retry after forcing a compositor frame, but keep each CDP # off-screen clip. Retry after forcing a compositor frame, but keep each CDP
# capture bounded so a wedged compositor cannot consume the report deadline. # capture bounded so a wedged compositor cannot consume the report deadline.
@@ -437,6 +441,99 @@ CHART_CONTAINER_STATE_JS = f"""
}} }}
""" """
CHART_CONTAINER_SELECTOR = ".chart-container"
# `.slice_container` (superset-frontend/src/components/Chart/Chart.tsx) is
# the one ancestor every chart type shares, directly inside `.chart-container`,
# with an explicit pixel height matching the dashboard tile. A locator-bounded
# capture (`element.screenshot()`, used for single-chart exports) clips to
# `.chart-container`'s own bounding box, which only has a `min-height` --
# so it stays exactly `.slice_container`'s fixed height unless that fixed
# height is lifted too. Un-clipping a scrollable *descendant* (the ag-Grid
# host, a table's own scroll body) is not enough on its own: the descendant
# can grow, but its ancestor's box does not, and the extra content just
# overflows the ancestor unseen by a bounding-box screenshot (#38090).
SLICE_CONTAINER_SELECTOR_FOR_EXPANSION = ".slice_container"
# Legacy/other chart-table implementations that scroll via an inline style
# rather than a stable class name (e.g. plugin-chart-table's sticky body,
# `superset-frontend/plugins/plugin-chart-table/src/DataTable/hooks/useSticky.tsx`)
# aren't reachable by a fixed class-selector list, so this catches any
# descendant of a chart that is *actually* clipping its own content
# (scrollHeight > clientHeight) rather than guessing at class names that may
# not exist in every plugin version. `.ant-table-body` is kept alongside it
# for a real Ant Design `<Table>` if one ever renders inside a chart.
GENERIC_SCROLLABLE_DESCENDANT_SELECTOR = (
f'{CHART_CONTAINER_SELECTOR} [style*="overflow"], '
f"{CHART_CONTAINER_SELECTOR} .ant-table-body"
)
# ag-Grid virtualizes rows for performance, so a plain height/overflow reset
# would still leave off-screen rows unrendered. `domLayout: "print"` is
# ag-Grid's own "render every row into the DOM" mode -- the same mode the
# client-side "download as image" export switches to via the GridApi that
# ThemedAgGridReact (superset-ui-core) stashes on the grid's host element
# specifically so screenshot/export code can reach it. The grid's own host
# element and its immediate parent (the ag-Grid table plugin's container,
# which sets an explicit pixel height via inline style -- see
# `plugin-chart-ag-grid-table/src/AgGridTable/index.tsx`) are reset for the
# same ancestor-box reason as `.slice_container` above.
#
# `page.screenshot(full_page=True)` already expands the outer dashboard
# scroll to include every below-the-fold chart (#31158); it has no effect on
# a chart's own internal scroll container, which is what this JS unrolls
# in-place before the page is captured.
EXPAND_SCROLLABLE_CONTENT_JS = f"""
async (maxWaitMs) => {{
const agGrids = Array.from(
document.querySelectorAll('{AG_GRID_HOST_SELECTOR}')
);
await Promise.all(agGrids.map(async (grid) => {{
const api = grid._agGridApi;
if (!api) {{ return; }}
api.setGridOption('domLayout', 'print');
if (api.resetRowHeights) {{ api.resetRowHeights(); }}
grid.style.height = 'auto';
if (grid.parentElement) {{ grid.parentElement.style.height = 'auto'; }}
// ag-Grid's autoHeight rows batch-measure asynchronously, so this
// polls for a stable scrollHeight instead of a fixed sleep. Five
// consecutive unchanged 100ms polls is a deliberate match for the
// client-side export's own
// waitForStableScrollHeight(agRootWrapper, 5000, 5) (downloadAsImage.tsx):
// always paid in full even when nothing is still settling, so both
// paths trust the measurement after the same wait rather than
// racing a batch that hasn't finished yet.
let lastHeight = grid.scrollHeight;
let stableCount = 0;
const deadline = Date.now() + maxWaitMs;
while (Date.now() < deadline && stableCount < 5) {{
await new Promise((resolve) => setTimeout(resolve, 100));
const height = grid.scrollHeight;
if (height === lastHeight) {{
stableCount += 1;
}} else {{
stableCount = 0;
lastHeight = height;
}}
}}
}}));
document.querySelectorAll('{SLICE_CONTAINER_SELECTOR_FOR_EXPANSION}').forEach(
(el) => {{ el.style.height = 'auto'; }}
);
document.querySelectorAll('{GENERIC_SCROLLABLE_DESCENDANT_SELECTOR}').forEach(
(el) => {{
if (el.scrollHeight > el.clientHeight) {{
el.style.overflow = 'visible';
el.style.height = 'auto';
el.style.maxHeight = 'none';
}}
}}
);
}}
"""
def combine_screenshot_tiles( def combine_screenshot_tiles(
screenshot_tiles: list[bytes], screenshot_tiles: list[bytes],
+68
View File
@@ -34,6 +34,8 @@ from superset.utils.screenshot_utils import (
CHART_CONTAINER_READY_JS, CHART_CONTAINER_READY_JS,
CHART_CONTAINER_STATE_JS, CHART_CONTAINER_STATE_JS,
CHART_HOLDERS_READY_JS, CHART_HOLDERS_READY_JS,
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
FIND_ALL_UNREADY_CHART_HOLDERS_JS, FIND_ALL_UNREADY_CHART_HOLDERS_JS,
FIND_CHART_HOLDER_STATES_JS, FIND_CHART_HOLDER_STATES_JS,
FORCE_ALL_CHART_HOLDERS_IN_VIEW_JS, FORCE_ALL_CHART_HOLDERS_IN_VIEW_JS,
@@ -238,6 +240,43 @@ class WebDriverPlaywright(WebDriverProxy):
else: else:
return element.screenshot(**timeout_kwargs) return element.screenshot(**timeout_kwargs)
@staticmethod
def _expand_scrollable_content(
page: Page,
log_context: str | None = None,
report_execution_context: ReportExecutionContext | None = None,
) -> None:
"""
Un-clip chart content that is fully present in the DOM but visually
cropped by a fixed height + internal scrollbar (e.g. a table taller
than the space its dashboard tile gives it) before the page is
captured.
The ag-Grid branch of this step polls for a stable row count, so it
is bounded by the report's own deadline the same way every other
wait in this method is, rather than an unconditional fixed sleep.
Best-effort: a failure here should not abort the screenshot, since a
clipped-but-present capture beats none at all.
"""
max_wait_seconds = (
report_execution_context.deadline.timeout_seconds(
"scrollable_content_expansion",
requested_seconds=EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
reserve_seconds=report_execution_context.readiness_reserve_seconds,
)
if report_execution_context
else EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS
)
try:
page.evaluate(EXPAND_SCROLLABLE_CONTENT_JS, max_wait_seconds * 1000)
except PlaywrightError:
logger.warning(
"Failed to expand scrollable chart content before screenshot%s",
f" [{log_context}]" if log_context else "",
exc_info=True,
)
@staticmethod @staticmethod
def _wait_for_charts_ready( # noqa: C901 def _wait_for_charts_ready( # noqa: C901
page: Page, page: Page,
@@ -712,6 +751,19 @@ class WebDriverPlaywright(WebDriverProxy):
unexpected_errors, unexpected_errors,
context_suffix, context_suffix,
) )
# Un-clip scrollable/virtualized chart content (dense tables
# taller than their dashboard tile) before measuring height,
# so the tiling decision below sees the full content when
# possible. A chart whose ag-Grid hasn't fired GridReady yet
# at this point is re-expanded below, after readiness --
# `.chart-container` elements attaching (waited on above) is
# not the same as ag-Grid finishing its own internal init.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
# Detect large dashboards and use tiled screenshots if enabled # Detect large dashboards and use tiled screenshots if enabled
tiled_enabled = app.config.get("SCREENSHOT_TILED_ENABLED", False) tiled_enabled = app.config.get("SCREENSHOT_TILED_ENABLED", False)
@@ -865,6 +917,14 @@ class WebDriverPlaywright(WebDriverProxy):
screenshot_started_at=screenshot_started_at, screenshot_started_at=screenshot_started_at,
report_execution_context=report_execution_context, report_execution_context=report_execution_context,
) )
# Re-run now that readiness has confirmed every chart
# actually rendered: a grid whose GridReady hadn't
# fired yet at the earlier call above is expanded here.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
if selenium_animation_wait > 0: if selenium_animation_wait > 0:
if report_execution_context: if report_execution_context:
selenium_animation_wait = min( selenium_animation_wait = min(
@@ -929,6 +989,14 @@ class WebDriverPlaywright(WebDriverProxy):
screenshot_started_at=screenshot_started_at, screenshot_started_at=screenshot_started_at,
report_execution_context=report_execution_context, report_execution_context=report_execution_context,
) )
# Re-run now that readiness has confirmed every chart
# actually rendered: a grid whose GridReady hadn't fired
# yet at the earlier call above is expanded here.
WebDriverPlaywright._expand_scrollable_content(
page,
log_context=log_context,
report_execution_context=report_execution_context,
)
if selenium_animation_wait > 0: if selenium_animation_wait > 0:
if report_execution_context: if report_execution_context:
selenium_animation_wait = min( selenium_animation_wait = min(
+99 -53
View File
@@ -24,9 +24,11 @@ request:
* :func:`collect_impact_pairs` pulls the distinct * :func:`collect_impact_pairs` pulls the distinct
``(dataset_id, transaction_id)`` pairs that need counts. ``(dataset_id, transaction_id)`` pairs that need counts.
* :func:`batch_chart_counts` one SQL query joining * :func:`batch_chart_counts` counts the matching charts without a
``dashboard_slices_version`` and ``slices_version`` to count join: dashboard membership comes from ``charts_attached_to_dashboard``'s
the matching charts validity-strategy-style. attach/detach windows over ``dashboard_slices_version``, and a
member-scoped ``slices_version`` scan supplies the chartdataset window;
the two are combined per pair by :func:`_count_attached_charts_at`.
* :func:`impact_for_record` pure projection from the pre-fetched * :func:`impact_for_record` pure projection from the pre-fetched
counts onto each record (returns ``None`` for non-Dashboard paths counts onto each record (returns ``None`` for non-Dashboard paths
or non-SqlaTable kinds, matching the ``impact`` computation). or non-SqlaTable kinds, matching the ``impact`` computation).
@@ -38,6 +40,7 @@ inside another (no DB).
from __future__ import annotations from __future__ import annotations
from collections.abc import Mapping, Sequence
from typing import Any from typing import Any
import sqlalchemy as sa import sqlalchemy as sa
@@ -47,7 +50,14 @@ from superset.versioning.activity.kinds import (
chunked_ids, chunked_ids,
ENTITY_ID_CHUNK_SIZE, ENTITY_ID_CHUNK_SIZE,
TABLE_KIND_TO_API, TABLE_KIND_TO_API,
Window,
) )
from superset.versioning.baseline import OPERATION_DELETE
# Headroom left below SQLite's 999 bind-variable floor for the handful of scalar
# binds in the slice-scan WHERE (datasource_type, operation_type, the two tx
# bounds) once a member-id chunk and the dataset IN are accounted for.
_SCALAR_BIND_HEADROOM = 20
def collect_impact_pairs( def collect_impact_pairs(
@@ -76,12 +86,17 @@ def batch_chart_counts(
distinct charts that were both on *dashboard_id* and pointing at distinct charts that were both on *dashboard_id* and pointing at
*dataset_id* at *target_tx*. *dataset_id* at *target_tx*.
One SELECT against ``dashboard_slices_version`` ``slices_version``, No join: ``charts_attached_to_dashboard`` supplies each member chart's
pulling the (slice, dataset, validity-window) state for every slice ``[attach, detach)`` windows from the association shadow (Continuum never
ever on the dashboard whose dataset matches one of the requested closes an M2M shadow's ``end_transaction_id``, so a validity-window filter
dataset_ids. The Python loop then applies the validity-strategy on it would count a chart removed before ``target_tx`` sc-119907), and a
predicate per pair. Replaces the previous N+1 shape that fired one member-scoped scan of ``slices_version`` supplies the chartdataset window,
COUNT per related record. whose ``end_transaction_id`` the validity backfill *does* close, so the
ordinary validity predicate is right there. The Python loop counts a slice
for a pair when both an attachment window and its chartdataset window
contain ``target_tx``. Replaces the previous N+1 shape that fired one COUNT
per related record, and the m2mslices join whose M2M validity window was
the buggy naive filter.
Returns ``{(dataset_id, target_tx): count}``; pairs whose count Returns ``{(dataset_id, target_tx): count}``; pairs whose count
would be zero are omitted so the caller's ``.get(key, 0)`` is would be zero are omitted so the caller's ``.get(key, 0)`` is
@@ -94,73 +109,104 @@ def batch_chart_counts(
from sqlalchemy_continuum import version_class from sqlalchemy_continuum import version_class
from superset.models.slice import Slice from superset.models.slice import Slice
from superset.versioning.membership import charts_attached_to_dashboard
metadata = version_class(Slice).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
slices_tbl = version_class(Slice).__table__ slices_tbl = version_class(Slice).__table__
if m2m_tbl is None:
return {}
dataset_ids: set[int] = {dataset_id for dataset_id, _ in pairs} dataset_ids: set[int] = {dataset_id for dataset_id, _ in pairs}
# Bound both validity windows to the transaction range the page-set
# needs. Both the attachment (m2m) and the chart→dataset (slice) window
# must straddle a requested target_tx, so a row whose window starts
# after the newest target, or closes at/before the oldest one, can
# never contribute a match. Without this the join multiplies every
# attachment row by the full slice version history — an unbounded cross
# product on dashboards with long-lived, frequently-edited charts.
target_txs: set[int] = {target_tx for _, target_tx in pairs} target_txs: set[int] = {target_tx for _, target_tx in pairs}
min_tx, max_tx = min(target_txs), max(target_txs) min_tx, max_tx = min(target_txs), max(target_txs)
# Chunk the datasource_id IN-clause to stay under SQLite's bind-variable
# floor (a dashboard pointing at very many datasets can exceed it). # Attachment membership per slice. charts_attached_to_dashboard owns the
rows: list[Any] = [] # association-shadow read and the attach/detach window pairing — the single
for chunk in chunked_ids(dataset_ids, ENTITY_ID_CHUNK_SIZE): # place that must never filter the M2M shadow by end_transaction_id, which
stmt = sa.select( # Continuum never closes (sc-119907). Reused here so restore.py, the
m2m_tbl.c.slice_id, # activity relationship walk, and this rollup share one implementation.
slices_tbl.c.datasource_id, attach_windows: dict[int, list[Window]] = {}
m2m_tbl.c.transaction_id.label("m2m_start"), for slice_id, window in charts_attached_to_dashboard(dashboard_id):
m2m_tbl.c.end_transaction_id.label("m2m_end"), attach_windows.setdefault(slice_id, []).append(window)
slices_tbl.c.transaction_id.label("slice_start"), if not attach_windows:
slices_tbl.c.end_transaction_id.label("slice_end"), return {}
).where(
m2m_tbl.c.dashboard_id == dashboard_id, # Chart→dataset validity from the slice parent shadow, whose
m2m_tbl.c.operation_type != 2, # end_transaction_id the validity backfill *does* close, so the ordinary
slices_tbl.c.id == m2m_tbl.c.slice_id, # half-open validity predicate is correct here. Bounded on the DB side to
slices_tbl.c.datasource_id.in_(chunk), # this dashboard's member charts (the attach_windows keys) and the
# transaction range; the member-id IN-clause is chunked to stay under
# SQLite's 999 bind-variable floor.
#
# The requested-dataset prune is applied on the DB side too, but only when
# the dataset set co-binds with a full member chunk under that floor — a
# member chunk (<= ENTITY_ID_CHUNK_SIZE) plus the dataset IN plus the few
# scalar binds must stay < 999. When there are too many requested datasets,
# the DB-side dataset predicate is dropped and the combiner filters datasets
# in Python (it already keys on pairs_by_dataset), so a wide dashboard does
# not overflow the bind limit (sc-119907 review).
filter_datasets_in_sql = (
len(dataset_ids) <= 999 - ENTITY_ID_CHUNK_SIZE - _SCALAR_BIND_HEADROOM
)
slice_rows: list[Any] = []
for chunk in chunked_ids(set(attach_windows), ENTITY_ID_CHUNK_SIZE):
conditions = [
slices_tbl.c.id.in_(chunk),
slices_tbl.c.datasource_type == "table", slices_tbl.c.datasource_type == "table",
slices_tbl.c.operation_type != 2, slices_tbl.c.operation_type != OPERATION_DELETE,
m2m_tbl.c.transaction_id <= max_tx,
sa.or_(
m2m_tbl.c.end_transaction_id.is_(None),
m2m_tbl.c.end_transaction_id > min_tx,
),
slices_tbl.c.transaction_id <= max_tx, slices_tbl.c.transaction_id <= max_tx,
sa.or_( sa.or_(
slices_tbl.c.end_transaction_id.is_(None), slices_tbl.c.end_transaction_id.is_(None),
slices_tbl.c.end_transaction_id > min_tx, slices_tbl.c.end_transaction_id > min_tx,
), ),
) ]
rows.extend(db.session.connection().execute(stmt).mappings().all()) if filter_datasets_in_sql:
conditions.append(slices_tbl.c.datasource_id.in_(dataset_ids))
stmt = sa.select(
slices_tbl.c.id.label("slice_id"),
slices_tbl.c.datasource_id,
slices_tbl.c.transaction_id.label("slice_start"),
slices_tbl.c.end_transaction_id.label("slice_end"),
).where(*conditions)
slice_rows.extend(db.session.connection().execute(stmt).mappings().all())
# For each pair, collect the slice_ids whose two validity windows
# both straddle target_tx. ``set`` dedupes within a pair.
matches: dict[tuple[int, int], set[int]] = {}
pairs_by_dataset: dict[int, list[int]] = {} pairs_by_dataset: dict[int, list[int]] = {}
for dataset_id, target_tx in pairs: for dataset_id, target_tx in pairs:
pairs_by_dataset.setdefault(dataset_id, []).append(target_tx) pairs_by_dataset.setdefault(dataset_id, []).append(target_tx)
for row in rows: return _count_attached_charts_at(attach_windows, slice_rows, pairs_by_dataset)
def _count_attached_charts_at(
attach_windows: dict[int, list[Window]],
slice_rows: Sequence[Mapping[str, Any]],
pairs_by_dataset: dict[int, list[int]],
) -> dict[tuple[int, int], int]:
"""Pure combiner: for each ``(dataset_id, target_tx)``, count the distinct
charts whose attachment window and chartdataset window both contain
``target_tx``.
*attach_windows* maps ``slice_id`` to its ``[attach, detach)`` episodes
(from the association shadow see
:func:`~superset.versioning.activity.windows.attachment_windows`); a chart
removed before ``target_tx`` has no window containing it and is therefore
not counted. *slice_rows* are the chartdataset parent-shadow rows
(``slice_id``, ``datasource_id``, ``slice_start``, ``slice_end``), whose
``end_transaction_id`` (``slice_end``) the validity backfill does close, so
the half-open validity predicate is correct for them. Split out of
:func:`batch_chart_counts` so this membership logic is unit-testable
without a live shadow-table fixture.
"""
matches: dict[tuple[int, int], set[int]] = {}
for row in slice_rows:
windows = attach_windows.get(row["slice_id"])
if not windows:
continue
ds_id = row["datasource_id"] ds_id = row["datasource_id"]
for target_tx in pairs_by_dataset.get(ds_id, ()): for target_tx in pairs_by_dataset.get(ds_id, ()):
in_m2m = row["m2m_start"] <= target_tx and ( in_attach = any(w.contains(target_tx) for w in windows)
row["m2m_end"] is None or row["m2m_end"] > target_tx
)
in_slice = row["slice_start"] <= target_tx and ( in_slice = row["slice_start"] <= target_tx and (
row["slice_end"] is None or row["slice_end"] > target_tx row["slice_end"] is None or row["slice_end"] > target_tx
) )
if in_m2m and in_slice: if in_attach and in_slice:
matches.setdefault((ds_id, target_tx), set()).add(row["slice_id"]) matches.setdefault((ds_id, target_tx), set()).add(row["slice_id"])
return {pair: len(slice_ids) for pair, slice_ids in matches.items()} return {pair: len(slice_ids) for pair, slice_ids in matches.items()}
+4 -91
View File
@@ -16,8 +16,10 @@
# under the License. # under the License.
"""DB-touching helpers for the activity-view read path. """DB-touching helpers for the activity-view read path.
All Phase A relationship walks (``charts_attached_to_dashboard``, The Phase A relationship walks (``datasets_used_by_chart``,
``datasets_used_by_chart``, ``batch_datasets_used_by_charts``), ``batch_datasets_used_by_charts``; the dashboard-membership walk
``charts_attached_to_dashboard`` lives in
:mod:`superset.versioning.membership`),
the Phase B change-record fetch (``fetch_change_records`` / the Phase B change-record fetch (``fetch_change_records`` /
``_select_change_rows_for_kinds``), the name-denormalization helpers ``_select_change_rows_for_kinds``), the name-denormalization helpers
(``_resolve_names_for_kind`` / ``apply_entity_name_denormalization``), the (``_resolve_names_for_kind`` / ``apply_entity_name_denormalization``), the
@@ -39,7 +41,6 @@ from __future__ import annotations
import logging import logging
from datetime import datetime from datetime import datetime
from heapq import heappush, heapreplace from heapq import heappush, heapreplace
from itertools import groupby
from typing import Any from typing import Any
from uuid import UUID from uuid import UUID
@@ -119,94 +120,6 @@ def first_tracked_tx(
# ---- Phase A: relationship-traversal queries ------------------------------ # ---- Phase A: relationship-traversal queries ------------------------------
# ``operation_type`` values on a Continuum association shadow row
# (sqlalchemy_continuum.operation.Operation): INSERT attaches, DELETE detaches.
# UPDATE never occurs for a pure M2M association (there is nothing to update on
# a (dashboard, slice) pair); if it ever appeared it is ignored — neither
# opening nor closing a window — so an open attachment simply continues.
# These mirror the library enum's numeric values; ``test_m2m_op_constants_match_
# continuum`` pins them so a Continuum renumber fails loudly rather than silently.
_M2M_OP_INSERT = 0
_M2M_OP_DELETE = 2
def _attachment_windows(
rows: list[tuple[int, int, int]],
) -> list[tuple[int, Window]]:
"""Pair INSERT / DELETE association-version rows into ``[attach, detach)``
windows, one per attachment episode.
Each row is ``(slice_id, transaction_id, operation_type)``. Continuum
**never closes** an association shadow row's ``end_transaction_id`` — its
unit-of-work only *inserts* association versions
(``create_association_versions``); the validity backfill that sets
``end_transaction_id`` runs for parent objects, not for M2M links. So the
detach boundary lives on the DELETE row's ``transaction_id``, not on the
attach row's ``end_transaction_id`` (which stays NULL for the association's
whole life). An INSERT opens a window; the next DELETE closes it at its
transaction id; an attachment with no following DELETE stays open (the
chart is still on the dashboard). A DELETE at the same transaction as its
open (add-and-remove in one save) yields no window the chart was never
on a committed dashboard state.
"""
result: list[tuple[int, Window]] = []
# operation_type is part of the sort key so that, within one transaction,
# INSERT (0) sorts before DELETE (2): an add-and-remove in a single save is
# then seen open-before-close and collapses to no window (the DELETE finds
# ``tx == open_tx``, not ``>``). Do not drop it from the key.
rows_sorted = sorted(rows, key=lambda r: (r[0], r[1], r[2]))
for slice_id, group in groupby(rows_sorted, key=lambda r: r[0]):
open_tx: int | None = None
for _slice_id, tx, operation_type in group:
if operation_type == _M2M_OP_DELETE:
if open_tx is not None and tx > open_tx:
result.append((slice_id, Window(open_tx, tx)))
open_tx = None
elif operation_type == _M2M_OP_INSERT and open_tx is None:
open_tx = tx
if open_tx is not None:
result.append((slice_id, Window(open_tx, None)))
return result
def charts_attached_to_dashboard(dashboard_id: int) -> list[tuple[int, Window]]:
"""Return ``(slice_id, window)`` for every chart that has ever been on
*dashboard_id*, with each attachment episode's validity window in
transaction-id space.
Reads from ``dashboard_slices_version`` (Continuum's auto-generated M2M
shadow) and pairs its INSERT/DELETE rows via :func:`_attachment_windows`,
so a chart removed from the dashboard is bounded at the detach transaction
rather than open-ended otherwise the chart's edits made *after* removal
would surface in the dashboard's related history.
"""
# pylint: disable=import-outside-toplevel
from sqlalchemy_continuum import version_class
from superset.models.dashboard import Dashboard
metadata = version_class(Dashboard).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
if m2m_tbl is None:
return []
rows = (
db.session.connection()
.execute(
sa.select(
m2m_tbl.c.slice_id,
m2m_tbl.c.transaction_id,
m2m_tbl.c.operation_type,
).where(
m2m_tbl.c.dashboard_id == dashboard_id,
m2m_tbl.c.slice_id.is_not(None),
)
)
.all()
)
return _attachment_windows([(row[0], row[1], row[2]) for row in rows])
def datasets_used_by_chart(slice_id: int) -> list[tuple[int, Window]]: def datasets_used_by_chart(slice_id: int) -> list[tuple[int, Window]]:
"""Return ``(datasource_id, window)`` for every dataset that *slice_id* """Return ``(datasource_id, window)`` for every dataset that *slice_id*
has ever pointed at, with each association's validity window. has ever pointed at, with each association's validity window.
+1 -1
View File
@@ -35,13 +35,13 @@ from __future__ import annotations
from superset.versioning.activity.kinds import EntityWindows, Window from superset.versioning.activity.kinds import EntityWindows, Window
from superset.versioning.activity.queries import ( from superset.versioning.activity.queries import (
batch_datasets_used_by_charts, batch_datasets_used_by_charts,
charts_attached_to_dashboard,
datasets_used_by_chart, datasets_used_by_chart,
) )
from superset.versioning.activity.windows import ( from superset.versioning.activity.windows import (
intersect_windows, intersect_windows,
merge_entity_windows, merge_entity_windows,
) )
from superset.versioning.membership import charts_attached_to_dashboard
def resolve_scope( def resolve_scope(
+66
View File
@@ -29,10 +29,76 @@ means "open-ended (current)" and behaves like positive infinity.
from __future__ import annotations from __future__ import annotations
from itertools import groupby
from typing import Any from typing import Any
from superset.versioning.activity.kinds import EntityWindows, Window from superset.versioning.activity.kinds import EntityWindows, Window
# ``operation_type`` values on a Continuum association shadow row
# (sqlalchemy_continuum.operation.Operation): INSERT attaches, DELETE detaches.
# UPDATE never occurs for a pure M2M association (there is nothing to update on
# a (dashboard, slice) pair); if it ever appeared it is ignored — neither
# opening nor closing a window — so an open attachment simply continues.
# These mirror the library enum's numeric values; ``test_m2m_op_constants_match_
# continuum`` pins them so a Continuum renumber fails loudly rather than silently.
M2M_OP_INSERT = 0
M2M_OP_DELETE = 2
def attachment_windows(
rows: list[tuple[int, int, int]],
) -> list[tuple[int, Window]]:
"""Pair INSERT / DELETE association-version rows into ``[attach, detach)``
windows, one per attachment episode.
Each row is ``(assoc_id, transaction_id, operation_type)``. Continuum
**never closes** an association shadow row's ``end_transaction_id`` — its
unit-of-work only *inserts* association versions
(``create_association_versions``); the validity backfill that sets
``end_transaction_id`` runs for parent objects, not for M2M links. So the
detach boundary lives on the DELETE row's ``transaction_id``, not on the
attach row's ``end_transaction_id`` (which stays NULL for the association's
whole life). An INSERT opens a window; the next DELETE closes it at its
transaction id; an attachment with no following DELETE stays open (the
association is still live). A DELETE at the same transaction as its open
(add-and-remove in one save) yields no window the association was never
on a committed state. That last case is a deliberate divergence from
Continuum's own ``association_subquery`` reverter, which (selecting the
``MAX(tx) <= T`` row and excluding only DELETEs) would treat such a pair as
a member; the never-committed reading is the safer one for restore.
This is the M2M-correct counterpart to
:func:`~superset.versioning.changes.shadow_queries.shadow_rows_valid_at`,
whose ``end_transaction_id`` validity filter is right for parent/child
shadows but silently re-includes a detached association.
"""
result: list[tuple[int, Window]] = []
# operation_type is part of the sort key so that, within one transaction,
# INSERT (0) sorts before DELETE (2): an add-and-remove in a single save is
# then seen open-before-close and collapses to no window (the DELETE finds
# ``tx == open_tx``, not ``>``). Do not drop it from the key.
#
# Corollary / assumption: because INSERT is forced before DELETE within a
# transaction, this cannot represent a *remove-then-re-add* of the same
# association in one transaction (it would read the same as add-then-remove
# → no window). That relies on no write path emitting DELETE-then-INSERT
# for the same association within a single transaction — which holds today
# (a chart is detached or attached in a save, not both), so the case is
# latent, not live. Revisit this pairing if such a write path is added.
rows_sorted = sorted(rows, key=lambda r: (r[0], r[1], r[2]))
for assoc_id, group in groupby(rows_sorted, key=lambda r: r[0]):
open_tx: int | None = None
for _assoc_id, tx, operation_type in group:
if operation_type == M2M_OP_DELETE:
if open_tx is not None and tx > open_tx:
result.append((assoc_id, Window(open_tx, tx)))
open_tx = None
elif operation_type == M2M_OP_INSERT and open_tx is None:
open_tx = tx
if open_tx is not None:
result.append((assoc_id, Window(open_tx, None)))
return result
def intersect_windows(outer: Window, inner: Window) -> Window | None: def intersect_windows(outer: Window, inner: Window) -> Window | None:
"""Intersect two half-open ``[start_tx, end_tx)`` windows. """Intersect two half-open ``[start_tx, end_tx)`` windows.
+86
View File
@@ -0,0 +1,86 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""M2M dashboard-membership queries, shared across the versioning surfaces.
``charts_attached_to_dashboard`` reads the ``dashboard_slices_version``
association shadow and pairs its INSERT/DELETE rows into ``[attach, detach)``
windows. It must **never** filter that shadow by ``end_transaction_id``:
Continuum never closes an M2M association's ``end_transaction_id`` (see
:func:`~superset.versioning.activity.windows.attachment_windows`), so a
validity filter would re-include a chart removed before the queried tx.
This lives in a neutral module not the activity read-path module
(``activity/queries.py``) so the restore write path and the impact rollup
depend on it here rather than reaching up into the read path (sc-119907).
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import sqlalchemy as sa
from superset.extensions import db
if TYPE_CHECKING:
from superset.versioning.activity.kinds import Window
def charts_attached_to_dashboard(dashboard_id: int) -> list[tuple[int, Window]]:
"""Return ``(slice_id, window)`` for every chart that has ever been on
*dashboard_id*, with each attachment episode's validity window in
transaction-id space.
Reads from ``dashboard_slices_version`` (Continuum's auto-generated M2M
shadow) and pairs its INSERT/DELETE rows via
:func:`~superset.versioning.activity.windows.attachment_windows`,
so a chart removed from the dashboard is bounded at the detach transaction
rather than open-ended otherwise the chart's edits made *after* removal
would surface in the dashboard's related history.
"""
# pylint: disable=import-outside-toplevel
# attachment_windows is imported lazily (not at module top) so that this
# module stays a leaf: importing it must not pull the activity package,
# whose read-path modules (scope.py) import charts_attached_to_dashboard
# back from here — a module-top import created a circular import that only
# surfaced at runtime, when a restore imported this module for the first
# time (sc-119907).
from sqlalchemy_continuum import version_class
from superset.models.dashboard import Dashboard
from superset.versioning.activity.windows import attachment_windows
metadata = version_class(Dashboard).__table__.metadata
m2m_tbl = metadata.tables.get("dashboard_slices_version")
if m2m_tbl is None:
return []
rows = (
db.session.connection()
.execute(
sa.select(
m2m_tbl.c.slice_id,
m2m_tbl.c.transaction_id,
m2m_tbl.c.operation_type,
).where(
m2m_tbl.c.dashboard_id == dashboard_id,
m2m_tbl.c.slice_id.is_not(None),
)
)
.all()
)
return attachment_windows([(row[0], row[1], row[2]) for row in rows])
+24 -26
View File
@@ -195,42 +195,40 @@ def _restore_dashboard_membership(dashboard: Any, transaction_id: int) -> list[i
"""Reset *dashboard*'s chart membership to what it was at """Reset *dashboard*'s chart membership to what it was at
*transaction_id*, reattaching only charts that still exist. *transaction_id*, reattaching only charts that still exist.
Reads the validity-windowed ``dashboard_slices_version`` shadow Membership is derived from the ``dashboard_slices_version`` shadow
(Continuum's auto-generated M2M table): a slice was a member at tx T (Continuum's auto-generated M2M table) by pairing each slice's
iff a non-DELETE row has ``transaction_id <= T`` and an open or INSERT/DELETE rows into ``[attach, detach)`` windows: a slice was a
later-closing validity window. member at tx T iff one of its attachment windows contains T. The
``shadow_rows_valid_at`` validity filter must **not** be used here
Continuum never closes an association shadow's ``end_transaction_id``,
so that filter would re-attach a chart that had been removed before T
(attached@1, removed@5, restore to tx10 the chart wrongly returns).
``shadow_rows_valid_at`` stays correct for parent/child shadows, whose
``end_transaction_id`` the validity backfill does close (sc-119907).
Returns the ids of snapshot members that no longer exist and were Returns the ids of snapshot members that no longer exist and were
skipped. Live charts' content is never touched — restoring a chart's skipped. Live charts' content is never touched — restoring a chart's
content is the chart's own restore endpoint's job. content is the chart's own restore endpoint's job.
""" """
# pylint: disable=import-outside-toplevel # pylint: disable=import-outside-toplevel
# Local imports: models.slice transitively imports models.core, which # Local imports: models.slice transitively imports models.core, which needs
# needs the initialised app — module-top import would recreate the # the initialised app — a module-top import would recreate the bootstrap
# bootstrap cycle documented in changes/listener.py; shadow_queries is # cycle documented in changes/listener.py. charts_attached_to_dashboard is
# imported lazily for the same reason (see queries.get_version). # imported lazily for the same reason: it pulls the window helpers, whose
# package transitively imports the versioning.changes listener graph, so a
# module-top import here would re-enter that same bootstrap cycle.
from superset.models.slice import Slice from superset.models.slice import Slice
from superset.versioning.changes import shadow_rows_valid_at from superset.versioning.membership import charts_attached_to_dashboard
ver_cls = version_class(type(dashboard)) # charts_attached_to_dashboard owns the association-shadow read and the
m2m_tbl = ver_cls.__table__.metadata.tables.get("dashboard_slices_version") # attach/detach window pairing (the single place that must never filter the
if m2m_tbl is None: # pragma: no cover — shadow tables always exist here # M2M shadow by end_transaction_id — Continuum never closes it). A slice was
return [] # a member at transaction_id iff one of its windows contains it (sc-119907).
# shadow_rows_valid_at owns the validity-window semantics (open or
# later-closing window, non-DELETE) — the same predicate the version
# snapshot's column/metric reconstruction uses.
member_ids = sorted( member_ids = sorted(
{ {
row["slice_id"] slice_id
for row in shadow_rows_valid_at( for slice_id, window in charts_attached_to_dashboard(dashboard.id)
db.session, if window.contains(transaction_id)
m2m_tbl,
"dashboard_id",
dashboard.id,
transaction_id,
)
if row["slice_id"] is not None
} }
) )
if not member_ids: if not member_ids:
+2 -1
View File
@@ -23,7 +23,8 @@ from unittest.mock import MagicMock
from urllib.parse import parse_qs, urlparse from urllib.parse import parse_qs, urlparse
import pytest import pytest
from flask import Response, escape, url_for from flask import Response, url_for
from markupsafe import escape
from sqlalchemy import func from sqlalchemy import func
from superset import db, security_manager from superset import db, security_manager
@@ -18,10 +18,7 @@
import pytest import pytest
import rison import rison
from flask import ( from flask import current_app
current_app,
escape, # noqa: F401
)
from superset.daos.dashboard import DashboardDAO from superset.daos.dashboard import DashboardDAO
from superset.utils import json from superset.utils import json
@@ -201,6 +201,76 @@ class TestDashboardRestoreApi(SupersetTestCase):
f"got {restored_ids}" f"got {restored_ids}"
) )
def test_restore_to_tx_after_removal_does_not_reattach_chart(self) -> None:
"""The inverse of the re-attach case (sc-119907): restoring to a
snapshot captured *after* a chart was removed must NOT bring it back.
Continuum never closes an association shadow row's
``end_transaction_id``, so the removed chart's INSERT row still looks
"valid at" any later tx under a naive validity filter it would be
wrongly re-attached. Membership must be derived from paired
attach/detach windows: the window ``[attach, remove)`` does not contain
the post-removal target tx, so the chart stays off.
"""
from superset.daos.version import derive_version_uuid
_persist_fixture_state()
dashboard: Dashboard = (
db.session.query(Dashboard)
.filter(Dashboard.dashboard_title == "USA Births Names")
.first()
)
assert dashboard is not None
dashboard_uuid = str(dashboard.uuid)
dashboard_id = dashboard.id
entity_uuid = dashboard.uuid
assert entity_uuid is not None
original_slice_ids = sorted(s.id for s in dashboard.slices)
assert len(original_slice_ids) >= 2, (
f"fixture expected to attach >= 2 charts; got {original_slice_ids}"
)
slice_to_drop = dashboard.slices[0]
drop_id = slice_to_drop.id
# Remove the chart and commit — the detach is recorded in history.
dashboard.slices.remove(slice_to_drop)
db.session.commit()
# Touch the dashboard AFTER the removal so the restore target snapshot
# post-dates the detach.
dashboard.dashboard_title = "USA Births Names — post-removal snapshot"
db.session.commit()
ver_cls = version_class(Dashboard)
target_tx = (
db.session.query(ver_cls.transaction_id)
.filter(ver_cls.id == dashboard_id, ver_cls.uuid == entity_uuid)
.order_by(ver_cls.transaction_id.desc())
.limit(1)
.scalar()
)
assert target_tx is not None
target_uuid = str(derive_version_uuid(entity_uuid, target_tx))
self.login(ADMIN_USERNAME)
rv = self._restore(dashboard_uuid, target_uuid)
assert rv.status_code == 200, rv.data
db.session.expire_all()
dashboard = (
db.session.query(Dashboard).filter(Dashboard.id == dashboard_id).one()
)
restored_ids = sorted(s.id for s in dashboard.slices)
assert drop_id not in restored_ids, (
"restore re-attached a chart that was removed before the target "
f"snapshot: {drop_id} in {restored_ids}"
)
survivors = sorted(set(original_slice_ids) - {drop_id})
assert restored_ids == survivors, (
f"expected the surviving members {survivors}, got {restored_ids}"
)
def test_restore_preserves_live_chart_content(self) -> None: def test_restore_preserves_live_chart_content(self) -> None:
"""Dashboard restore is membership-only: a member chart edited """Dashboard restore is membership-only: a member chart edited
AFTER the snapshot keeps its current content charts are shared AFTER the snapshot keeps its current content charts are shared
+74
View File
@@ -4888,6 +4888,80 @@ def test_get_sqla_query_dotted_struct_column_bigquery(
assert "`forecasts.original`" not in sql assert "`forecasts.original`" not in sql
def test_get_sqla_query_grouping_sets_preserves_bigquery_mutated_label(
mocker: MockerFixture,
session: Session,
) -> None:
"""
A pivot table with a non-additive metric issues a `grouping_sets` query
(see plugin-chart-pivot-table/src/plugin/buildQuery.ts), whose rollup
levels are expressed in terms of each column's original, unmutated label
(e.g. a Custom SQL row labelled "Test Row"). BigQuery mutates SQL aliases
that contain non-word characters, so the same column's `Label.name` ends
up as something like "Test_Row_<hash>". Looking up each level's columns by
that mutated name instead of the original label silently drops the column
from every rollup level, producing a `GROUP BY GROUPING SETS` clause that
doesn't cover a selected, non-aggregated column.
"""
bigquery = pytest.importorskip("sqlalchemy_bigquery")
from superset.connectors.sqla.models import SqlaTable, TableColumn
from superset.models.core import Database
SqlaTable.metadata.create_all(session.get_bind())
dialect = bigquery.BigQueryDialect()
@contextmanager
def fake_engine(*args, **kwargs):
engine = MagicMock()
engine.dialect = dialect
yield engine
database = Database(database_name="bq", sqlalchemy_uri="bigquery://project")
mocker.patch.object(database, "get_sqla_engine", new=fake_engine)
table = SqlaTable(
database=database,
schema=None,
table_name="orders",
columns=[TableColumn(column_name="amount", type="FLOAT")],
)
row_col: AdhocColumn = {
"sqlExpression": "some_column",
"label": "Test Row",
}
metric: AdhocMetric = {
"expressionType": "SIMPLE",
"column": {"column_name": "amount"},
"aggregate": "AVG",
"label": "avg_amount",
}
sqlaq = table.get_sqla_query(
columns=[row_col],
metrics=[metric],
extras={},
filter=[],
granularity=None,
is_timeseries=False,
grouping_sets=[["Test Row"], []],
)
sql = str(
sqlaq.sqla_query.compile(
dialect=dialect, compile_kwargs={"literal_binds": True}
)
)
# Before the fix, looking up each level's columns by the BigQuery-mutated
# alias (e.g. "Test_Row_<hash>") instead of the original "Test Row" label
# sent by the frontend silently dropped the column from every level,
# producing `GROUPING SETS ((), ())` -- a GROUP BY that doesn't cover the
# selected, non-aggregated "some_column" expression.
assert "GROUPING SETS((some_column), ())" in sql
def test_temporal_epoch_string_filter_is_coerced_for_bigquery() -> None: def test_temporal_epoch_string_filter_is_coerced_for_bigquery() -> None:
""" """
Drill-to-detail can send JavaScript timestamp strings for temporal values. Drill-to-detail can send JavaScript timestamp strings for temporal values.
+2 -4
View File
@@ -129,7 +129,7 @@ def test_superset_cache_memoize_make_cache_key_uses_configurable_hash():
"_memoize_make_cache_key", "_memoize_make_cache_key",
return_value=lambda *args, **kwargs: "cache_key", return_value=lambda *args, **kwargs: "cache_key",
) as mock_make_key: ) as mock_make_key:
cache._memoize_make_cache_key(make_name=None, timeout=300) cache._memoize_make_cache_key(make_name=None)
mock_make_key.assert_called_once() mock_make_key.assert_called_once()
call_kwargs = mock_make_key.call_args[1] call_kwargs = mock_make_key.call_args[1]
@@ -145,9 +145,7 @@ def test_superset_cache_memoize_make_cache_key_allows_explicit_hash():
"_memoize_make_cache_key", "_memoize_make_cache_key",
return_value=lambda *args, **kwargs: "cache_key", return_value=lambda *args, **kwargs: "cache_key",
) as mock_make_key: ) as mock_make_key:
cache._memoize_make_cache_key( cache._memoize_make_cache_key(make_name=None, hash_method=hashlib.md5)
make_name=None, timeout=300, hash_method=hashlib.md5
)
mock_make_key.assert_called_once() mock_make_key.assert_called_once()
call_kwargs = mock_make_key.call_args[1] call_kwargs = mock_make_key.call_args[1]
@@ -1610,3 +1610,51 @@ def test_ag_grid_no_rows_overlay_is_a_terminal_empty_state() -> None:
CHART_CONTAINER_READY_JS, CHART_CONTAINER_READY_JS,
): ):
assert ".ag-overlay-no-rows-wrapper:not(.ag-hidden)" in predicate assert ".ag-overlay-no-rows-wrapper:not(.ag-hidden)" in predicate
def test_expand_scrollable_content_js_unrolls_ag_grid_and_css_scroll() -> None:
"""The pre-capture DOM-expansion script must reach both flavors of
clipped table content: ag-Grid's row virtualization (needs its own API
to force a full render) and plain CSS overflow/height clipping (the
content already exists in the DOM and just needs the constraint lifted).
It must also lift the fixed heights on the shared `.chart-container` /
`.slice_container` / ag-Grid-wrapper ancestor chain, not just the
scrollable descendant itself -- a bounding-box capture
(`element.screenshot()`, used for single-chart exports) clips to the
ancestor's own box, which does not grow just because a descendant's
content does (#38090, reviewed by @aminghadersohi on #43979)."""
from superset.utils.screenshot_utils import (
AG_GRID_HOST_SELECTOR,
CHART_CONTAINER_SELECTOR,
EXPAND_SCROLLABLE_CONTENT_JS,
GENERIC_SCROLLABLE_DESCENDANT_SELECTOR,
SLICE_CONTAINER_SELECTOR_FOR_EXPANSION,
)
assert AG_GRID_HOST_SELECTOR in EXPAND_SCROLLABLE_CONTENT_JS
assert "setGridOption('domLayout', 'print')" in EXPAND_SCROLLABLE_CONTENT_JS
assert "grid._agGridApi" in EXPAND_SCROLLABLE_CONTENT_JS
# The grid's own host and its immediate parent (the ag-Grid table
# plugin's fixed-pixel-height wrapper) both get their height lifted --
# not just the descendant content inside the grid.
assert "grid.style.height = 'auto'" in EXPAND_SCROLLABLE_CONTENT_JS
assert "grid.parentElement.style.height = 'auto'" in EXPAND_SCROLLABLE_CONTENT_JS
assert SLICE_CONTAINER_SELECTOR_FOR_EXPANSION == ".slice_container"
assert SLICE_CONTAINER_SELECTOR_FOR_EXPANSION in EXPAND_SCROLLABLE_CONTENT_JS
assert CHART_CONTAINER_SELECTOR == ".chart-container"
assert GENERIC_SCROLLABLE_DESCENDANT_SELECTOR == (
'.chart-container [style*="overflow"], .chart-container .ant-table-body'
)
assert GENERIC_SCROLLABLE_DESCENDANT_SELECTOR in EXPAND_SCROLLABLE_CONTENT_JS
# Gated on actually clipping, so a non-scrollable match (e.g. a sticky
# table header with no overflow of its own) is left untouched.
assert "el.scrollHeight > el.clientHeight" in EXPAND_SCROLLABLE_CONTENT_JS
assert "overflow = 'visible'" in EXPAND_SCROLLABLE_CONTENT_JS
assert "maxHeight = 'none'" in EXPAND_SCROLLABLE_CONTENT_JS
# Takes an explicit wait budget rather than hardcoding one, so it can be
# bounded by a report's remaining deadline (see webdriver_test.py).
assert "async (maxWaitMs) =>" in EXPAND_SCROLLABLE_CONTENT_JS
assert "Date.now() + maxWaitMs" in EXPAND_SCROLLABLE_CONTENT_JS
+161 -11
View File
@@ -154,6 +154,84 @@ class TestWebDriverPlaywrightFallback:
"http://example.com", wait_until="networkidle" "http://example.com", wait_until="networkidle"
) )
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager")
@patch("superset.utils.webdriver.app")
def test_get_screenshot_expands_scrollable_content_before_capture(
self, mock_app, mock_browser_manager
):
"""A dense table taller than its dashboard tile renders fully in the
DOM but is visually clipped by a fixed height + internal scrollbar.
`get_screenshot` must un-clip that content (ag-Grid print layout /
CSS overflow reset, see EXPAND_SCROLLABLE_CONTENT_JS) before taking
the screenshot, and must do so *before* the capture call so the
expanded layout is what actually gets captured (#38090). It runs
twice: once before the tiling decision, and again after chart
readiness confirms every ag-Grid has actually mounted its API
(@aminghadersohi's review on #43979 -- the earlier call can miss a
grid whose GridReady hasn't fired yet)."""
from superset.utils.screenshot_utils import EXPAND_SCROLLABLE_CONTENT_JS
mock_user = MagicMock()
mock_user.username = "test_user"
mock_app.config = {
"WEBDRIVER_OPTION_ARGS": [],
"WEBDRIVER_WINDOW": {"pixel_density": 1},
"SCREENSHOT_PLAYWRIGHT_DEFAULT_TIMEOUT": 30000,
"SCREENSHOT_PLAYWRIGHT_WAIT_EVENT": "networkidle",
"SCREENSHOT_SELENIUM_HEADSTART": 0,
"SCREENSHOT_SELENIUM_ANIMATION_WAIT": 0,
"SCREENSHOT_REPLACE_UNEXPECTED_ERRORS": False,
"SCREENSHOT_TILED_ENABLED": False,
"SCREENSHOT_LOCATE_WAIT": 10,
"SCREENSHOT_LOAD_WAIT": 10,
}
mock_browser = MagicMock()
mock_context = MagicMock()
mock_page = MagicMock()
mock_element = MagicMock()
mock_browser_manager.get_browser.return_value = mock_browser
mock_browser.new_context.return_value = mock_context
mock_context.new_page.return_value = mock_page
mock_page.locator.return_value = mock_element
capture_order: list[str] = []
def evaluate_side_effect(script, *args, **kwargs):
if script == EXPAND_SCROLLABLE_CONTENT_JS:
capture_order.append("expand")
return None
# FIND_CHART_HOLDER_STATES_JS (readiness diagnostics) expects an
# iterable of holder states; every other call in this (non-tiled,
# non-report) path is only ever logged, not branched on.
return []
mock_page.evaluate.side_effect = evaluate_side_effect
mock_page.screenshot.side_effect = lambda **k: (
capture_order.append("capture") or b"fake_screenshot"
)
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
driver = WebDriverPlaywright("chrome")
result = driver.get_screenshot(
"http://example.com", "standalone", mock_user
)
assert result == b"fake_screenshot"
expand_calls = [
call
for call in mock_page.evaluate.call_args_list
if call.args[0] == EXPAND_SCROLLABLE_CONTENT_JS
]
assert len(expand_calls) == 2
# A concrete millisecond budget is passed as the second arg (bounded
# by the report deadline when one exists; a fixed default otherwise).
assert all(isinstance(call.args[1], float) for call in expand_calls)
assert capture_order == ["expand", "expand", "capture"]
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True) @patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager") @patch("superset.utils.webdriver._browser_manager")
@patch("superset.utils.webdriver.logger") @patch("superset.utils.webdriver.logger")
@@ -259,6 +337,70 @@ class TestWebDriverPlaywrightErrorHandling:
"Failed to capture unexpected errors%s", "" "Failed to capture unexpected errors%s", ""
) )
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver.logger")
def test_expand_scrollable_content_swallows_playwright_error(self, mock_logger):
"""A failure while un-clipping scrollable content must not abort the
screenshot -- a clipped-but-present capture beats none at all."""
from superset.utils.webdriver import PlaywrightError
mock_page = MagicMock()
mock_page.evaluate.side_effect = PlaywrightError("boom")
WebDriverPlaywright._expand_scrollable_content(
mock_page, log_context="execution_id=abc-123"
)
mock_logger.warning.assert_called_once()
warning_args = mock_logger.warning.call_args.args
assert "Failed to expand scrollable chart content" in warning_args[0]
assert warning_args[1] == " [execution_id=abc-123]"
def test_expand_scrollable_content_defaults_wait_without_report_context(self):
"""Absent a report deadline to bound it against, the ag-Grid
stabilization poll gets the fixed default ceiling."""
from superset.utils.screenshot_utils import (
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS,
)
mock_page = MagicMock()
WebDriverPlaywright._expand_scrollable_content(mock_page)
mock_page.evaluate.assert_called_once_with(
EXPAND_SCROLLABLE_CONTENT_JS,
EXPAND_SCROLLABLE_CONTENT_MAX_WAIT_SECONDS * 1000,
)
def test_expand_scrollable_content_bounds_wait_to_report_deadline(self):
"""This step must respect the report's remaining budget like every
other wait in the capture path, rather than an unconditional fixed
sleep (Thread A / @aminghadersohi's review on #43979)."""
mock_page = MagicMock()
report_execution_context = _report_context()
# Only 2s left for this phase after other phases' reserves.
report_execution_context = report_execution_context.__class__(
**{
**report_execution_context.__dict__,
"deadline": report_execution_context.deadline.__class__(
total_seconds=2
+ report_execution_context.capture_reserve_seconds
+ report_execution_context.delivery_reserve_seconds
+ report_execution_context.cleanup_reserve_seconds,
started_at=0,
_clock=lambda: 0,
),
}
)
WebDriverPlaywright._expand_scrollable_content(
mock_page, report_execution_context=report_execution_context
)
max_wait_ms = mock_page.evaluate.call_args.args[1]
assert max_wait_ms == pytest.approx(2000)
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True) @patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver.sync_playwright") @patch("superset.utils.webdriver.sync_playwright")
@patch("superset.utils.webdriver.logger") @patch("superset.utils.webdriver.logger")
@@ -979,12 +1121,16 @@ class TestWebDriverPlaywrightChartReadiness:
assert result == b"screenshot" assert result == b"screenshot"
# Readiness diagnostics are emitted before polling so a task killed by # Readiness diagnostics are emitted before polling so a task killed by
# an outer limit still leaves useful state in the logs. # an outer limit still leaves useful state in the logs. Two additional
assert mock_page.evaluate.call_count == 2 # evaluate() calls expand scrollable content before capture: once
assert all( # before the tiling decision, once after readiness (#38090).
"state: 'rendered'" in call.args[0] readiness_calls = [
call
for call in mock_page.evaluate.call_args_list for call in mock_page.evaluate.call_args_list
) if "state: 'rendered'" in call.args[0]
]
assert len(readiness_calls) == 2
assert mock_page.evaluate.call_count == 4
@patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True) @patch("superset.utils.webdriver.PLAYWRIGHT_AVAILABLE", True)
@patch("superset.utils.webdriver._browser_manager") @patch("superset.utils.webdriver._browser_manager")
@@ -1599,8 +1745,9 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager) mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
# Small dashboard: 3 charts, 1000px height — below both thresholds # Small dashboard: 3 charts, 1000px height — below both thresholds.
mock_page.evaluate.side_effect = [3, 1000, [], []] # First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 3, 1000, [], []]
call_order: list[str] = [] call_order: list[str] = []
@@ -1642,7 +1789,8 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
"SCREENSHOT_TILED_VIEWPORT_HEIGHT": 600, "SCREENSHOT_TILED_VIEWPORT_HEIGHT": 600,
} }
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager) mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
mock_page.evaluate.side_effect = [25, 500, [], []] # First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 500, [], []]
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context): with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
result = WebDriverPlaywright("chrome").get_screenshot( result = WebDriverPlaywright("chrome").get_screenshot(
@@ -1673,8 +1821,9 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager) mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
# Large dashboard: 25 charts, 6000px height # Large dashboard: 25 charts, 6000px height. First item is consumed
mock_page.evaluate.side_effect = [25, 6000] # by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 6000]
mock_take_tiled.return_value = b"tiled_screenshot" mock_take_tiled.return_value = b"tiled_screenshot"
with patch.object(WebDriverPlaywright, "auth", return_value=mock_context): with patch.object(WebDriverPlaywright, "auth", return_value=mock_context):
@@ -1727,7 +1876,8 @@ class TestWebDriverPlaywrightAnimationWaitOrder:
} }
mock_context, mock_page = self._make_pw_mocks(mock_browser_manager) mock_context, mock_page = self._make_pw_mocks(mock_browser_manager)
mock_page.evaluate.side_effect = [25, 6000] # First item is consumed by the pre-capture scrollable-content expansion.
mock_page.evaluate.side_effect = [None, 25, 6000]
# Empty bytes — falsy but not None; was silently passed through before the fix # Empty bytes — falsy but not None; was silently passed through before the fix
mock_take_tiled.return_value = b"" mock_take_tiled.return_value = b""
# _get_screenshot("standalone") calls page.screenshot(full_page=True); it # _get_screenshot("standalone") calls page.screenshot(full_page=True); it
+137 -12
View File
@@ -44,6 +44,8 @@ from superset.versioning.activity import (
Window, Window,
) )
from superset.versioning.activity.impact import ( from superset.versioning.activity.impact import (
_count_attached_charts_at,
batch_chart_counts,
collect_impact_pairs, collect_impact_pairs,
impact_for_record, impact_for_record,
) )
@@ -54,9 +56,6 @@ from superset.versioning.activity.orchestrator import (
_MAX_PAGE_SIZE, _MAX_PAGE_SIZE,
) )
from superset.versioning.activity.queries import ( from superset.versioning.activity.queries import (
_attachment_windows,
_M2M_OP_DELETE,
_M2M_OP_INSERT,
_merge_result_into_heap, _merge_result_into_heap,
_record_sort_key, _record_sort_key,
BoundedRecordHeap, BoundedRecordHeap,
@@ -68,7 +67,10 @@ from superset.versioning.activity.render import (
) )
from superset.versioning.activity.scope import resolve_scope from superset.versioning.activity.scope import resolve_scope
from superset.versioning.activity.windows import ( from superset.versioning.activity.windows import (
attachment_windows,
intersect_windows, intersect_windows,
M2M_OP_DELETE,
M2M_OP_INSERT,
merge_entity_windows, merge_entity_windows,
row_within_any_window, row_within_any_window,
union_windows, union_windows,
@@ -893,7 +895,7 @@ def test_build_summary_meta_headline_branches() -> None:
assert _build_summary("Dashboard", unknown) == "Dashboard updated" assert _build_summary("Dashboard", unknown) == "Dashboard updated"
# ---- _attachment_windows ------------------------------------------------- # ---- attachment_windows -------------------------------------------------
def test_attachment_windows_closes_at_detach() -> None: def test_attachment_windows_closes_at_detach() -> None:
@@ -903,7 +905,7 @@ def test_attachment_windows_closes_at_detach() -> None:
row's transaction id — an INSERT at t1 paired with a DELETE at t3 yields row's transaction id — an INSERT at t1 paired with a DELETE at t3 yields
the half-open window [t1, t3), so an edit at t2 (while attached) is the half-open window [t1, t3), so an edit at t2 (while attached) is
inside it and an edit at t4 (after removal) is not.""" inside it and an edit at t4 (after removal) is not."""
windows = _attachment_windows([(7, 1, 0), (7, 3, 2)]) # INSERT@1, DELETE@3 windows = attachment_windows([(7, 1, 0), (7, 3, 2)]) # INSERT@1, DELETE@3
assert windows == [(7, Window(1, 3))] assert windows == [(7, Window(1, 3))]
window = windows[0][1] window = windows[0][1]
@@ -916,7 +918,7 @@ def test_attachment_windows_still_attached_is_open_ended() -> None:
"""A chart with an INSERT and no following DELETE is still on the """A chart with an INSERT and no following DELETE is still on the
dashboard, so its window is open-ended (end_tx = None) and every later dashboard, so its window is open-ended (end_tx = None) and every later
edit remains in scope.""" edit remains in scope."""
assert _attachment_windows([(7, 5, 0)]) == [(7, Window(5, None))] assert attachment_windows([(7, 5, 0)]) == [(7, Window(5, None))]
def test_attachment_windows_reattach_cycles_and_ordering() -> None: def test_attachment_windows_reattach_cycles_and_ordering() -> None:
@@ -924,20 +926,30 @@ def test_attachment_windows_reattach_cycles_and_ordering() -> None:
is independent of the row order the query returns them in (the rows are is independent of the row order the query returns them in (the rows are
sorted by (slice_id, transaction_id) internally).""" sorted by (slice_id, transaction_id) internally)."""
rows = [(7, 7, 2), (7, 1, 0), (7, 5, 0), (7, 3, 2)] # deliberately shuffled rows = [(7, 7, 2), (7, 1, 0), (7, 5, 0), (7, 3, 2)] # deliberately shuffled
assert _attachment_windows(rows) == [(7, Window(1, 3)), (7, Window(5, 7))] assert attachment_windows(rows) == [(7, Window(1, 3)), (7, Window(5, 7))]
def test_attachment_windows_reattach_leaves_the_last_episode_open() -> None:
"""Attach@1, detach@5, re-attach@8 with no later detach: the first episode
is the closed window [1, 5) and the current attachment is open-ended
[8, None). An edit at tx 10 falls inside the live episode."""
windows = attachment_windows([(7, 1, 0), (7, 5, 2), (7, 8, 0)])
assert windows == [(7, Window(1, 5)), (7, Window(8, None))]
assert windows[1][1].contains(10)
assert not windows[0][1].contains(10)
def test_attachment_windows_add_and_remove_same_transaction() -> None: def test_attachment_windows_add_and_remove_same_transaction() -> None:
"""Attaching and detaching in a single save (INSERT and DELETE at the """Attaching and detaching in a single save (INSERT and DELETE at the
same transaction) leaves the chart on no committed dashboard state, so it same transaction) leaves the chart on no committed dashboard state, so it
contributes no window (and no degenerate zero-width interval).""" contributes no window (and no degenerate zero-width interval)."""
assert _attachment_windows([(7, 2, 0), (7, 2, 2)]) == [] assert attachment_windows([(7, 2, 0), (7, 2, 2)]) == []
def test_attachment_windows_separates_slices() -> None: def test_attachment_windows_separates_slices() -> None:
"""Each slice gets its own independent windows.""" """Each slice gets its own independent windows."""
rows = [(7, 1, 0), (7, 3, 2), (9, 2, 0)] rows = [(7, 1, 0), (7, 3, 2), (9, 2, 0)]
assert _attachment_windows(rows) == [ assert attachment_windows(rows) == [
(7, Window(1, 3)), (7, Window(1, 3)),
(9, Window(2, None)), (9, Window(2, None)),
] ]
@@ -945,10 +957,123 @@ def test_attachment_windows_separates_slices() -> None:
def test_m2m_op_constants_match_continuum() -> None: def test_m2m_op_constants_match_continuum() -> None:
"""The association-shadow operation-type constants used by """The association-shadow operation-type constants used by
_attachment_windows are the numeric values of Continuum's Operation enum. attachment_windows are the numeric values of Continuum's Operation enum.
Pin them so a library renumber fails here loudly rather than silently Pin them so a library renumber fails here loudly rather than silently
mis-pairing attach/detach rows.""" mis-pairing attach/detach rows."""
from sqlalchemy_continuum import Operation from sqlalchemy_continuum import Operation
assert _M2M_OP_INSERT == Operation.INSERT assert M2M_OP_INSERT == Operation.INSERT
assert _M2M_OP_DELETE == Operation.DELETE assert M2M_OP_DELETE == Operation.DELETE
# ---- _count_attached_charts_at (batch_chart_counts membership) -----------
def _slice_row(
slice_id: int, datasource_id: int, start: int, end: int | None
) -> dict[str, Any]:
"""A chart→dataset parent-shadow row as batch_chart_counts fetches it."""
return {
"slice_id": slice_id,
"datasource_id": datasource_id,
"slice_start": start,
"slice_end": end,
}
def test_count_attached_charts_excludes_chart_removed_before_target() -> None:
"""sc-119907: a chart attached@1 and removed@5 must NOT be counted for a
dataset rollup at target_tx=10. Its attachment window [1, 5) does not
contain 10, even though its (never-closed) association shadow row would
pass a naive end_transaction_id validity filter."""
attach_windows = {7: [Window(1, 5)]}
slice_rows = [_slice_row(7, 100, 1, None)] # chart→dataset open the whole time
result = _count_attached_charts_at(attach_windows, slice_rows, {100: [10]})
assert result == {} # zero-count pairs are omitted
def test_count_attached_charts_counts_chart_inside_its_window() -> None:
"""The same chart IS counted at a target inside its attachment window."""
attach_windows = {7: [Window(1, 5)]}
slice_rows = [_slice_row(7, 100, 1, None)]
result = _count_attached_charts_at(attach_windows, slice_rows, {100: [3]})
assert result == {(100, 3): 1}
def test_count_attached_charts_requires_both_windows() -> None:
"""A chart attached at target_tx but not yet pointing at the dataset (its
chartdataset window starts later) is not counted."""
attach_windows = {7: [Window(1, None)]}
slice_rows = [_slice_row(7, 100, 8, None)] # points at dataset only from tx8
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [3]}) == {}
def test_count_attached_charts_dedupes_within_pair() -> None:
"""Multiple parent-shadow rows for the same slice count the slice once."""
attach_windows = {7: [Window(1, None)]}
slice_rows = [_slice_row(7, 100, 1, 4), _slice_row(7, 100, 4, None)]
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [5]}) == {
(100, 5): 1
}
def test_count_attached_charts_ignores_slice_never_on_dashboard() -> None:
"""A slice pointing at the dataset but with no attachment window (never on
this dashboard) does not contribute guards the no-join fetch that may
return slices from other dashboards sharing the dataset."""
attach_windows: dict[int, list[Window]] = {}
slice_rows = [_slice_row(7, 100, 1, None)]
assert _count_attached_charts_at(attach_windows, slice_rows, {100: [3]}) == {}
# ---- batch_chart_counts bind-variable floor (sc-119907) ------------------
def test_batch_chart_counts_stays_under_sqlite_bind_floor(app_context: None) -> None:
"""sc-119907: a wide dashboard (many member charts AND many requested
datasets) must not build a slice-scan statement that exceeds SQLite's 999
bind-variable floor. The member-id IN is chunked; the requested-dataset IN
is dropped to Python-side filtering when it would not co-bind under the
floor. Compiles every issued statement and asserts its bind count stays
safely under 999 the pre-fix code (dataset IN always on) bound
500 member + 600 dataset + scalars = ~1104 and would 500 on SQLite."""
from sqlalchemy.dialects import sqlite
member_windows = [(sid, Window(1, None)) for sid in range(1, 601)]
pairs = {(ds, 5) for ds in range(10_000, 10_600)} # 600 requested datasets
captured: list[Any] = []
class _FakeResult:
def mappings(self) -> "_FakeResult":
return self
def all(self) -> list[Any]:
return []
def _fake_execute(stmt: Any) -> "_FakeResult":
captured.append(stmt)
return _FakeResult()
with (
patch(
"superset.versioning.membership.charts_attached_to_dashboard",
return_value=member_windows,
),
patch("superset.versioning.activity.impact.db") as mock_db,
):
mock_db.session.connection.return_value.execute.side_effect = _fake_execute
batch_chart_counts(1, pairs)
assert captured, "expected at least one slice-scan statement"
for stmt in captured:
# render_postcompile expands ``IN (...)`` (an expanding bind in
# SQLAlchemy 2.0) into one param per element, so the count reflects
# what actually hits SQLite — a plain compile would show one bind per
# IN and hide the overflow.
compiled = stmt.compile(
dialect=sqlite.dialect(),
compile_kwargs={"render_postcompile": True},
)
n_binds = len(compiled.params)
assert n_binds < 999, f"statement binds {n_binds} params (SQLite floor 999)"