Compare commits

..
Author SHA1 Message Date
Enzo Martellucci 1bac9807aa Merge branch 'master' into enxdev/fix/explore-chart-data 2026-08-28 18:49:26 +02:00
Enzo Martellucci 5f79743799 Merge branch 'master' into enxdev/fix/explore-chart-data 2026-08-28 18:44:22 +02:00
Enzo MartellucciandClaude Sonnet 5 2516bf0166 fix(explore): drop redundant .ant-tabs height rules from ResultsPaneOnDashboard
Tabs already emits height: 100% on .ant-tabs-body and .ant-tabs-content
when fullHeight is set, so the Wrapper's own .ant-tabs/.ant-tabs-body
overrides were dead weight.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-28 18:40:54 +02:00
Enzo Martellucci 3f61dd8bdc Merge branch 'master' into enxdev/fix/explore-chart-data 2026-08-28 18:39:22 +02:00
Enzo Martellucci 36f6c22660 Merge branch 'master' into enxdev/fix/explore-chart-data 2026-08-28 15:44:38 +02:00
Enzo Martellucci c9eebc0744 test(explore): cover fullHeight prop passed to ResultsPaneOnDashboard's Tabs
Splitting the stale-tab work into its own PR (#43648) took the only
test asserting fullHeight with it. Add coverage back here, reading
only Tabs' first call argument so the assertion doesn't depend on
React 18's legacy-context second argument, which React 19 drops.
2026-08-28 15:39:45 +02:00
Enzo Martellucci 304b9c10e0 Revert "fix(dashboard): reconcile stale results tab in Chart Data modal"
This reverts commit 50f4802bbf.
2026-08-28 15:00:52 +02:00
Enzo Martellucci b0528f0bf2 Revert "refactor(explore): dedupe stale results-tab fallback into a shared hook"
This reverts commit 7870fda6ab.
2026-08-28 15:00:52 +02:00
Enzo Martellucci b913ee27a0 Revert "test(explore): reset useResultsPane mock between ResultsPaneOnDashboard tests"
This reverts commit 13dd39abb1.
2026-08-28 15:00:52 +02:00
Enzo Martellucci 7de38c2af1 Revert "test(explore): stop asserting Tabs' legacy-context second argument"
This reverts commit 9ebbbc87f3.
2026-08-28 15:00:52 +02:00
Enzo MartellucciandClaude Sonnet 5 9ebbbc87f3 test(explore): stop asserting Tabs' legacy-context second argument
toHaveBeenCalledWith(matcher, expect.anything()) only passed because React
18 still calls function components with a legacy-context {} second
argument; React 19 drops it, which would break this assertion's arg-count
check for reasons unrelated to Tabs' actual props. Inspect the first call
argument directly instead, which is agnostic to whether a second one
exists.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-28 12:16:54 +02:00
Enzo MartellucciandClaude Sonnet 5 13dd39abb1 test(explore): reset useResultsPane mock between ResultsPaneOnDashboard tests
The last test's mockReturnValue overrides for useResultsPane persisted
after the test finished, since nothing restored the mock's implementation.
Restore it in afterEach so later tests keep getting the real hook.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-28 12:13:30 +02:00
Enzo MartellucciandClaude Sonnet 5 7870fda6ab refactor(explore): dedupe stale results-tab fallback into a shared hook
DataTablesPane and ResultsPaneOnDashboard each reconciled activeTabKey
against the current tab keys with an identical getStaleResultsTabFallback
+ useEffect pair. Extract useStaleResultsTabFallback so both call sites
share one implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-28 12:13:01 +02:00
Enzo Martellucci 4011845d92 Merge branch 'master' into enxdev/fix/explore-chart-data 2026-08-28 11:44:47 +02:00
Enzo MartellucciandClaude Sonnet 5 50f4802bbf fix(dashboard): reconcile stale results tab in Chart Data modal
Address PR #43454 review feedback: reuse DataTablesPane's stale-tab
fallback so ResultsPaneOnDashboard doesn't render blank when a mixed
chart's active results tab disappears, and drop CSS rules now
redundant with the fullHeight prop.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 12:16:21 +02:00
Enzo Martellucci fd07663dc1 fix(dashboard): stretch Chart Data modal results grid to fill available height 2026-08-24 13:32:15 +02:00
102 changed files with 572 additions and 6668 deletions
+8 -5
View File
@@ -24,6 +24,14 @@ updates:
- dependency-name: "@types/react-dom"
update-types: ["version-update:semver-major"]
- dependency-name: "react-icons"
# JSDOM v30 doesn't play well with Jest v30
# Source: https://jestjs.io/blog#known-issues
# GH thread: https://github.com/jsdom/jsdom/issues/3492
- dependency-name: "jest-environment-jsdom"
# `@swc/plugin-transform-imports` doesn't work with current Webpack-SWC hybrid setup
# See https://github.com/apache/superset/pull/37384#issuecomment-3793991389
# TODO: remove the plugin once Lodash usage has been migrated to a more readily tree-shakeable alternative
- dependency-name: "@swc/plugin-transform-imports"
# deck.gl and luma.gl share strict peer constraints across the root and
# plugin workspaces, and root overrides pin their transitive versions.
# Upgrade both families together in a manually validated change.
@@ -79,11 +87,6 @@ updates:
patterns:
- "ag-grid-react"
- "ag-grid-community"
swc:
patterns:
- "@swc/core"
- "@swc/plugin-emotion"
- "@swc/plugin-transform-imports"
open-pull-requests-limit: 30
versioning-strategy: increase
cooldown:
-21
View File
@@ -66,27 +66,6 @@ jobs:
- name: "Set up liccheck"
run: |
# liccheck (as of 0.9.2) still does a bare `import pkg_resources`
# without declaring setuptools as a dependency, relying on it
# having historically been bundled. setuptools 81+ (installed
# above via requirements/base.txt) dropped the pkg_resources
# subpackage entirely, so liccheck's own import breaks outright.
#
# Reinstalling an older setuptools would restore pkg_resources but
# would also downgrade the *real* setuptools install, which then
# trips liccheck's own working_set.resolve() -- it cross-checks
# requirements/base.txt's declared `setuptools==84.0.0` against
# what's actually installed, and a downgrade makes those disagree.
#
# Instead, vendor just the pkg_resources/ package files from an
# old setuptools wheel into site-packages, leaving the real
# setuptools install (and its dist-info metadata) untouched. This
# gives liccheck an importable pkg_resources whose own working-set
# scan still correctly reports the real installed setuptools
# version, so no conflict is raised.
pip download "setuptools<81" --no-deps -d /tmp/old-setuptools
python -m zipfile -e /tmp/old-setuptools/setuptools-*.whl /tmp/old-setuptools-extracted/
cp -r /tmp/old-setuptools-extracted/pkg_resources "$(python -c 'import site; print(site.getsitepackages()[0])')/"
uv pip install --system liccheck
- name: "Run liccheck"
run: |
-192
View File
@@ -1,192 +0,0 @@
# db_engine_specs tests against real databases (testcontainers)
name: Testcontainers
# Spins up real Docker containers (see tests/testcontainers/ for the current
# dialect list) via testcontainers-python, which catches real dialect/driver
# regressions -- the kind mocked db_engine_specs unit tests structurally
# cannot, e.g. apache/superset#42899 (Trino emitting OFFSET before LIMIT).
# Runs on a nightly cron (catches drift from a driver's own releases, not
# just from Superset's changes) and on pull_request, scoped via `paths` to
# only PRs that actually touch this test suite or the workflow itself, so
# unrelated PRs across the repo are never affected.
#
# A matrix entry can set `nightly_only: true` to run only on the cron (or a
# manual workflow_dispatch), never on pull_request -- for a dialect whose
# image is too heavy (a multi-service cluster, a many-GB image, a slow
# licensed installer) to justify adding its wall-clock/resource cost to
# every PR that merely touches this suite. Omit the field entirely for a
# normal dialect; it isn't nightly-only by default.
permissions:
contents: read
on:
schedule:
- cron: "0 5 * * *"
workflow_dispatch: {}
pull_request:
paths:
- ".github/workflows/testcontainers.yml"
- "tests/testcontainers/**"
- "superset/db_engine_specs/**"
- "pyproject.toml"
- "requirements/development.in"
- "requirements/development.txt"
concurrency:
# Scoped by ref, not just workflow name -- otherwise every PR run and the
# nightly cron share one group, and starting the workflow on another PR
# (or the nightly firing mid-PR-run) cancels an unrelated in-progress run.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
testcontainers:
runs-on: ubuntu-26.04
strategy:
fail-fast: false
matrix:
include:
# One job per dialect rather than one job for the whole suite: a
# single slow container would otherwise inflate the wall-clock
# time for every dialect, not just its own. Running in parallel
# means the suite's total time is bounded by the slowest dialect,
# not the sum of all of them. Db2's first-boot init is documented
# upstream as notably slow (a real instance bring-up, not just a
# process start) and untested locally here (no arm64 image), so
# it gets a wider timeout margin than the rest until real CI data
# says otherwise.
- dialect: cockroachdb
timeout: 10
- dialect: crate
timeout: 10
- dialect: trino
timeout: 10
- dialect: mssql
timeout: 10
- dialect: elasticsearch
timeout: 10
- dialect: oracle
timeout: 15
- dialect: db2
timeout: 25
- dialect: mariadb
timeout: 10
- dialect: timescaledb
timeout: 10
- dialect: yugabytedb
timeout: 10
- dialect: monetdb
timeout: 10
- dialect: mongodb
timeout: 10
- dialect: postgres
timeout: 10
- dialect: mysql
timeout: 10
- dialect: clickhouse
timeout: 10
# StarRocks' allin1-ubuntu image brings up both FE and BE in one
# container, which is a heavier bring-up than a single-process
# database -- wider margin until real CI data says otherwise.
- dialect: starrocks
timeout: 15
- dialect: databend
timeout: 10
- dialect: risingwave
timeout: 10
- dialect: firebird
timeout: 10
- dialect: ydb
timeout: 10
# OceanBase bootstraps a distributed-style cluster even in
# single-node MODE=MINI -- too heavy for every PR's CI budget, so
# it runs on the nightly cron / manual dispatch only.
- dialect: oceanbase
timeout: 20
nightly_only: true
timeout-minutes: ${{ matrix.timeout }}
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
# This job's matrix installs exactly one dialect's testcontainers
# driver for exactly this job, so treat that driver as required: a
# broken/missing import should fail the job, not silently skip to a
# misleadingly green, zero-tests-run result. See _driver.py.
SUPERSET_TESTCONTAINERS_STRICT: true
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Setup Python
uses: ./.github/actions/setup-backend/
with:
python-version: current
- name: Install db2 driver (ibm-db-sa)
# ibm-db (the db2 DBAPI) ships no Linux arm64 wheel, so it's kept out
# of the baseline dev install (requirements/development.in) to avoid
# breaking the multi-platform dev Docker image build. Install it here
# instead, only for this leg of the matrix.
if: matrix.dialect == 'db2'
run: uv pip install --system -e .[db2]
- name: Install oceanbase driver (oceanbase_py)
# oceanbase_py pins sqlalchemy-utils>=0.38.3,<0.39, which conflicts
# outright with Superset's own sqlalchemy-utils==0.42.1 pin -- kept
# out of the baseline dev install for the same reason as db2 above.
# Installed as its own standalone package (not via `-e .[oceanbase]`)
# so --no-deps only skips *oceanbase_py's* dependencies -- applied
# to `-e .[oceanbase]` instead, --no-deps blocks pip from installing
# anything the extras marker pulls in, including oceanbase_py
# itself, which "succeeds" without actually installing it
# (confirmed on real CI: the install step reported success, but the
# module was still missing). This job only needs oceanbase_py's
# dialect module importable, not its sqlalchemy-utils dependency
# satisfied, since nothing here calls into it.
if: >-
matrix.dialect == 'oceanbase' &&
(matrix.nightly_only != true ||
github.event_name == 'schedule' ||
github.event_name == 'workflow_dispatch')
run: uv pip install --system --no-deps "oceanbase_py>=0.0.1.2"
- name: Install Firebird client library (libfbclient2)
# sqlalchemy-firebird's driver (firebird-driver) is a pure-Python
# ctypes wrapper (its wheel is py3-none-any) that dynamically loads
# the native Firebird client library from the host at import time
# -- it doesn't bundle that library itself, so it has to come from
# the system package manager, only for this leg of the matrix.
if: matrix.dialect == 'firebird'
run: |
sudo apt-get update
sudo apt-get install -y libfbclient2
- name: Run testcontainers db_engine_specs tests (${{ matrix.dialect }})
# A job-level `if:` can't reference `matrix` (only github/inputs/
# needs/vars are available there), so the nightly_only skip has to
# live on the step instead. A dialect without `nightly_only` set
# evaluates the left side true (unset is null, and `null != true`
# is true) and always runs; one WITH it set only runs on the cron
# or a manual dispatch, never on pull_request.
if: >-
matrix.nightly_only != true ||
github.event_name == 'schedule' ||
github.event_name == 'workflow_dispatch'
run: |
pytest --durations-min=2 -v -m testcontainers \
./tests/testcontainers/db_engine_specs/test_${{ matrix.dialect }}.py \
--junit-xml=test-results/junit-testcontainers-${{ matrix.dialect }}.xml
- name: Upload JUnit test results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: junit-results-testcontainers-${{ matrix.dialect }}
path: test-results/
retention-days: 7
actions-timeline:
needs: [testcontainers]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@57fc93f20c6da7fbc14063c6d24a2a5627c799ad # v3.2.0
+1 -1
View File
@@ -25,7 +25,7 @@ assists people when migrating to a new version.
## Next
- `SAMPLES_ROW_LIMIT` is now the default for `/datasource/samples` requests without a valid explicit `per_page`, rather than a hard per-request ceiling; explicit limits are honored up to the existing global row-limit ceiling, matching `/chart/data` SAMPLES requests.
- The `cockroachdb` extra (`pip install apache-superset[cockroachdb]`) now installs `sqlalchemy-cockroachdb` instead of the abandoned `cockroachdb` package, whose SQLAlchemy dialect could not be imported under SQLAlchemy 2.0. Existing environments with the old package installed must `pip uninstall cockroachdb` before reinstalling the extra -- both packages register the same `cockroachdb` SQLAlchemy dialect entry point, so leaving the old one in place can still load the abandoned implementation.
- The `cockroachdb` extra (`pip install apache-superset[cockroachdb]`) now installs `sqlalchemy-cockroachdb` instead of the abandoned `cockroachdb` package, whose SQLAlchemy dialect could not be imported under SQLAlchemy 2.0. Existing environments with the old package installed should `pip uninstall cockroachdb && pip install sqlalchemy-cockroachdb` (or simply reinstall the extra) to restore CockroachDB connectivity.
### MCP tool results preserve stored string values
+2 -2
View File
@@ -62,7 +62,7 @@
"@superset-ui/core": "^0.20.4",
"@swc/core": "^1.16.1",
"antd": "^6.6.1",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"caniuse-lite": "^1.0.30001809",
"docusaurus-plugin-openapi-docs": "^5.2.0",
"docusaurus-theme-openapi-docs": "^5.2.0",
@@ -76,7 +76,7 @@
"react-svg-pan-zoom": "^3.13.1",
"react-table": "^7.8.0",
"remark-import-partial": "^0.0.2",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"storybook": "^10.5.10",
"swagger-ui-react": "^5.32.14",
"swc-loader": "^0.2.7",
+11 -11
View File
@@ -6529,10 +6529,10 @@ base64-js@^1.3.1, base64-js@^1.5.1:
resolved "https://registry.yarnpkg.com/base64-js/-/base64-js-1.5.1.tgz#1b1b440160a5bf7ad40b650f095963481903930a"
integrity sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.17, baseline-browser-mapping@^2.9.19:
version "2.11.17"
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.17.tgz#a168205490077c5d7c542f1610016af4ade8f875"
integrity sha512-KAUDn1OSS0fmPlGO+NOUMRcOQ/b/shUBH3OgkG73mPgdf+JD/BQ6fHboGxNOxnUmlwcq+lLq3dTkayRPuSfXwg==
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.16, baseline-browser-mapping@^2.9.19:
version "2.11.16"
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.16.tgz#0fa19a4ece2e34439ecaa3fdca8a59acfbd287fb"
integrity sha512-H/bNPUFHewJHyCTdjn1n3Pit5+2GmWT6mmeHImPX+8MA9NA6b67jO4gYmi4jTbCJb2otq34KMZnovndDPqJwhQ==
batch@0.6.1:
version "0.6.1"
@@ -6614,9 +6614,9 @@ boxen@^7.0.0:
wrap-ansi "^8.1.0"
brace-expansion@^1.1.7:
version "1.1.18"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.18.tgz#3ce74d89885136be1535341f8c3d4425c29a5cab"
integrity sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==
version "1.1.15"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.15.tgz#a6d90d54067236e5f42570a3b7378d594d9b7738"
integrity sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==
dependencies:
balanced-match "^1.0.0"
concat-map "0.0.1"
@@ -14106,10 +14106,10 @@ reselect@^4.0.0:
resolved "https://registry.yarnpkg.com/reselect/-/reselect-4.1.8.tgz#3f5dc671ea168dccdeb3e141236f69f02eaec524"
integrity sha512-ab9EmR80F/zQTMNeneUr4cv+jSwPJgIlvEmVwLerwrWVbpLlBuls9XHzIeTFy4cegU2NHBp3va0LKOzU5qFEYQ==
reselect@^5.1.0, reselect@^5.1.1, reselect@^5.3.0:
version "5.3.0"
resolved "https://registry.yarnpkg.com/reselect/-/reselect-5.3.0.tgz#0a3e3ed4436bdf2ab7c5e0f392dab2c062595d61"
integrity sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg==
reselect@^5.1.0, reselect@^5.1.1, reselect@^5.2.0:
version "5.2.0"
resolved "https://registry.yarnpkg.com/reselect/-/reselect-5.2.0.tgz#f380ef7664332d26ea06c1cba04bdbbdcaa955f1"
integrity sha512-AgZ3UOZm3YndfrJ4OYjgrT7bmCm/1iqkjvEfH/oYjzh6PD2qw4QuT3jjnXIrpdt4MTpMXclMT3lXbmRY+XRakw==
resize-observer-polyfill@1.5.1:
version "1.5.1"
+7 -21
View File
@@ -16,7 +16,7 @@
# under the License.
[build-system]
requires = ["setuptools>=84.0.0", "wheel"]
requires = ["setuptools>=40.9.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
@@ -67,7 +67,7 @@ dependencies = [
"flask-sqlalchemy>=3.1.1, <4.0",
"flask-wtf>=1.3.0, <2.0",
"geopy",
"greenlet<=3.5.5, >=3.5.5",
"greenlet<=3.5.4, >=3.5.4",
"gunicorn>=26.0.0, <27; sys_platform != 'win32'",
"hashids>=1.3.1, <2",
# holidays>=0.45 required for security fix
@@ -148,9 +148,10 @@ clickhouse = ["clickhouse-connect>=1.7.2, <2.0"]
# 2.0). sqlalchemy-cockroachdb is the actively maintained replacement,
# already linked from CockroachDbEngineSpec.metadata's docs_url, and
# registers the same `cockroachdb` SQLAlchemy dialect entry point.
# sqlalchemy-cockroachdb itself declares no DBAPI dependency (its own docs
# require picking one), so pull in the same psycopg2-binary pin as the
# `postgres` extra -- CockroachDB speaks the Postgres wire protocol.
# sqlalchemy-cockroachdb depends only on SQLAlchemy itself, not on a DBAPI
# driver, so psycopg2-binary is pinned alongside it (matching the `postgres`
# extra) to keep this extra self-contained -- CockroachDB speaks the
# PostgreSQL wire protocol, so psycopg2 is what actually opens connections.
cockroachdb = ["sqlalchemy-cockroachdb>=2.0.0, <3", "psycopg2-binary==2.9.12"]
crate = ["sqlalchemy-cratedb>=0.43.1, <1"]
# sqlalchemy-d1's only release (0.1.0, Nov 2025) pins sqlalchemy<2,>=1.4,
@@ -222,11 +223,6 @@ impala = ["impyla>=0.24.0, <0.25"]
# superset/db_engine_specs/kusto.py's known_incompatibilities metadata.
kusto = ["sqlalchemy-kusto>=3.1.2, <4"]
kylin = ["kylinpy>=2.8.4, <2.9"]
# MariaDB is a MySQL fork implementing the same wire protocol - connects via
# the plain mysql dialect, same driver as mysql.
mariadb = ["apache-superset[mysql]"]
monetdb = ["sqlalchemy-monetdb>=2.1.0, <3", "pymonetdb>=1.9.1, <2"]
mongodb = ["pymongosql>=0.7.3, <1"]
mssql = ["pymssql>=2.3.13, <3"]
# motherduck is an alias for duckdb - MotherDuck works via the duckdb driver
motherduck = ["apache-superset[duckdb]"]
@@ -275,9 +271,6 @@ tdengine = [
"taos-ws-py>=0.7.0"
]
teradata = ["teradatasql>=20.0.0.66"]
# TimescaleDB is a genuine Postgres extension, not a fork - connects via the
# plain postgresql dialect, same driver as postgres.
timescaledb = ["apache-superset[postgres]"]
thumbnails = [] # deprecated, will be removed in 7.0
vertica = ["sqlalchemy-vertica-python>= 0.6.3, < 0.7"]
netezza = ["nzalchemy>= 11.1.2, < 11.2"]
@@ -285,18 +278,11 @@ starrocks = ["starrocks>=1.3.4, <2"]
doris = ["pydoris>=1.2.0, <2.0.0"]
oceanbase = ["oceanbase_py>=0.0.1.2"]
ydb = ["ydb-sqlalchemy>=0.1.22", "ydb-sqlglot-plugin>=0.2.8"]
# YugabyteDB's YSQL layer is fully Postgres-wire compatible - connects via
# the plain postgresql dialect, same driver as postgres.
yugabytedb = ["apache-superset[postgres]"]
development = [
# no bounds for apache-superset-extensions-cli until a stable version
"apache-superset-extensions-cli",
"boto3",
# 7.0.0 raises `docker.errors.DockerException: ... Not supported URL
# scheme http+docker` against the requests/urllib3 versions pinned
# elsewhere in this file -- breaks testcontainers (tests/testcontainers/)
# before any container even starts. 7.2.0 is confirmed working.
"docker>=7.2.0",
"docker",
"flask-testing",
"freezegun",
"grpcio>=1.82.1",
+1 -7
View File
@@ -19,13 +19,7 @@ testpaths =
tests
python_files = *_test.py test_*.py *_tests.py *viz/utils.py
# `-p no:warnings` temporarily disabled in favor of more finely tuned `filterwarnings`.
# `not testcontainers` excludes tests/testcontainers/ by default: those spin up
# real Docker containers, and `testpaths = tests` would otherwise pull them into
# every plain `pytest` run. The dedicated CI job (testcontainers.yml) overrides
# this with an explicit `-m testcontainers` to run them.
addopts = -m "not testcontainers"
markers =
testcontainers: exercises a real database via testcontainers-python (needs Docker); excluded by default, see .github/workflows/testcontainers.yml
#addopts = -p no:warnings
asyncio_mode = auto
# `ignore` is effectively equivalent to `-p no:warnings`.
+2 -2
View File
@@ -52,11 +52,11 @@ marshmallow-sqlalchemy>=1.5.0
# needed for python 3.12 support
openapi-schema-validator>=0.6.3
# Pin setuptools <85 until all dependencies migrate from pkg_resources to importlib.metadata
# Pin setuptools <81 until all dependencies migrate from pkg_resources to importlib.metadata
# pkg_resources is deprecated and will be removed in setuptools 81+ (around 2025-11-30)
# Known affected packages: Preset's 'clients' package
# See docs/docs/contributing/pkg-resources-migration.md for details
setuptools<85
setuptools<81
# google-auth 2.53+ dropped its transitive dependency on cachetools, which is
# imported directly by superset.db_engine_specs.aws_iam. We declare cachetools
+2 -2
View File
@@ -163,7 +163,7 @@ google-auth==2.53.0
# via
# -r requirements/base.in
# shillelagh
greenlet==3.5.5
greenlet==3.5.4
# via
# apache-superset (pyproject.toml)
# shillelagh
@@ -366,7 +366,7 @@ rpds-py==0.25.0
# via
# jsonschema
# referencing
setuptools==84.0.0
setuptools==80.9.0
# via -r requirements/base.in
shillelagh==1.4.5
# via apache-superset (pyproject.toml)
+1 -28
View File
@@ -16,32 +16,5 @@
# specific language governing permissions and limitations
# under the License.
#
-e .[development,bigquery,clickhouse,cockroachdb,crate,databend,druid,duckdb,elasticsearch,fastmcp,firebird,gevent,gsheets,monetdb,mongodb,mssql,mysql,oracle,postgres,presto,prophet,risingwave,starrocks,trino,thumbnails,ydb]
-e .[development,bigquery,cockroachdb,druid,duckdb,fastmcp,gevent,gsheets,mysql,postgres,presto,prophet,trino,thumbnails]
-e ./superset-extensions-cli[test]
# testcontainers-backed db_engine_specs tests (tests/testcontainers/) --
# see .github/workflows/testcontainers.yml
#
# `db2` (the `ibm-db-sa`/`ibm-db` driver) and `oceanbase` (the `oceanbase_py`
# driver) are both deliberately left out of the baseline dev install above:
# `ibm-db` ships no Linux arm64 wheel, breaking the multi-platform
# (amd64+arm64) dev Docker image build; `oceanbase_py` pins
# `sqlalchemy-utils>=0.38.3,<0.39`, which conflicts outright with Superset's
# own `sqlalchemy-utils==0.42.1` pin -- there's no version of both that can
# coexist in one resolved environment. Both testcontainers CI jobs install
# their driver on demand instead, only for their own matrix leg -- see
# .github/workflows/testcontainers.yml.
#
# mariadb/timescaledb/yugabytedb need no testcontainers extra of their own:
# they reuse the postgres/mysql container classes pointed at a different
# image, and psycopg2-binary/mysqlclient are already pulled in above via
# the postgres/mysql extras. Plain postgres/mysql obviously need nothing
# extra either. clickhouse and starrocks also need no testcontainers extra:
# ClickHouseContainer has no driver import of its own (clickhouse-connect,
# pulled in above via the clickhouse extra, is all the test needs), and
# StarRocks has no dedicated testcontainers module at all -- its test uses
# a generic DockerContainer plus the same mysqlclient the mysql extra
# already provides. databend/risingwave/firebird/ydb are the same story:
# none has a dedicated testcontainers module, so each test uses a generic
# DockerContainer plus whatever driver its own extra above already
# provides.
testcontainers[cockroachdb,cratedb,mongodb,mssql,mysql,oracle,postgres,trino]>=4.15.0,<5
+9 -156
View File
@@ -12,17 +12,10 @@
# apache-superset
aiofile==3.9.0
# via py-key-value-aio
aiohappyeyeballs==2.7.1
# via aiohttp
aiohttp==3.14.3
# via ydb
aiosignal==1.4.0
# via aiohttp
alembic==1.15.2
# via
# -c requirements/base-constraint.txt
# flask-migrate
# starrocks
amqp==5.3.1
# via
# -c requirements/base-constraint.txt
@@ -31,8 +24,6 @@ annotated-types==0.7.0
# via
# -c requirements/base-constraint.txt
# pydantic
antlr4-python3-runtime==4.13.2
# via pymongosql
anyio==4.11.0
# via
# httpx
@@ -51,12 +42,9 @@ apsw==3.50.1.0
# shillelagh
astroid==3.3.10
# via pylint
asyncmy2==0.2.21
# via starrocks
attrs==25.3.0
# via
# -c requirements/base-constraint.txt
# aiohttp
# cattrs
# cyclopts
# jsonschema
@@ -77,7 +65,6 @@ backports-tarfile==1.2.0
backports-zstd==1.6.0
# via
# -c requirements/base-constraint.txt
# clickhouse-connect
# flask-compress
bcrypt==4.3.0
# via
@@ -130,11 +117,8 @@ celery==5.6.3
certifi==2026.5.20
# via
# -c requirements/base-constraint.txt
# clickhouse-connect
# elasticsearch
# httpcore
# httpx
# opensearch-py
# requests
cffi==2.0.0
# via
@@ -176,8 +160,6 @@ click-repl==0.3.0
# via
# -c requirements/base-constraint.txt
# celery
clickhouse-connect==1.7.2
# via apache-superset
cmdstanpy==1.1.0
# via prophet
colorama==0.4.6
@@ -189,8 +171,6 @@ contourpy==1.0.7
# via matplotlib
coverage==7.6.8
# via pytest-cov
crate==2.2.1
# via sqlalchemy-cratedb
cron-descriptor==1.4.5
# via
# -c requirements/base-constraint.txt
@@ -206,20 +186,14 @@ cryptography==50.0.0
# authlib
# google-auth
# joserfc
# oracledb
# paramiko
# pyjwt
# pymysql
# pyopenssl
# secretstorage
cycler==0.12.1
# via matplotlib
cyclopts==4.2.4
# via fastmcp-slim
databend-driver==0.34.2
# via databend-sqlalchemy
databend-sqlalchemy==0.5.5
# via apache-superset
db-dtypes==1.3.1
# via pandas-gbq
defusedxml==0.7.1
@@ -242,11 +216,8 @@ dnspython==2.7.0
# via
# -c requirements/base-constraint.txt
# email-validator
# pymongo
docker==7.2.0
# via
# apache-superset
# testcontainers
docker==7.0.0
# via apache-superset
docstring-parser==0.17.0
# via cyclopts
docutils==0.22.2
@@ -257,10 +228,6 @@ duckdb==1.5.5
# duckdb-engine
duckdb-engine==0.17.0
# via apache-superset
elasticsearch==7.17.13
# via elasticsearch-dbapi
elasticsearch-dbapi==0.2.13
# via apache-superset
email-validator==2.2.0
# via
# -c requirements/base-constraint.txt
@@ -270,8 +237,6 @@ et-xmlfile==2.0.0
# via
# -c requirements/base-constraint.txt
# openpyxl
events==0.5
# via opensearch-py
exceptiongroup==1.3.0
# via fastmcp-slim
fastmcp==3.4.7
@@ -282,10 +247,6 @@ filelock==3.20.3
# via
# -c requirements/base-constraint.txt
# virtualenv
firebird-base==2.0.3
# via firebird-driver
firebird-driver==2.0.3
# via sqlalchemy-firebird
flask==2.3.3
# via
# -c requirements/base-constraint.txt
@@ -366,18 +327,12 @@ fonttools==4.60.2
# via matplotlib
freezegun==1.5.1
# via apache-superset
frozenlist==1.8.0
# via
# aiohttp
# aiosignal
future==1.0.0
# via pyhive
geographiclib==2.0
# via
# -c requirements/base-constraint.txt
# geopy
geojson==3.3.0
# via sqlalchemy-cratedb
geopy==2.4.1
# via
# -c requirements/base-constraint.txt
@@ -420,7 +375,7 @@ googleapis-common-protos==1.66.0
# via
# google-api-core
# grpcio-status
greenlet==3.5.5
greenlet==3.5.4
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -434,7 +389,6 @@ grpcio==1.83.0
# apache-superset
# google-api-core
# grpcio-status
# ydb
grpcio-status==1.60.1
# via google-api-core
gunicorn==26.2.0
@@ -460,7 +414,6 @@ httpx==0.28.1
# via
# fastmcp-slim
# mcp
# testcontainers
httpx-sse==0.4.1
# via mcp
humanize==4.12.3
@@ -477,7 +430,6 @@ idna==3.15
# httpx
# requests
# url-normalize
# yarl
importlib-metadata==8.7.0
# via
# keyring
@@ -516,7 +468,6 @@ jmespath==1.1.0
# via
# boto3
# botocore
# pymongosql
joserfc==1.7.2
# via fastmcp-slim
jsonpath-ng==1.8.0
@@ -549,8 +500,6 @@ kombu==5.6.2
# via
# -c requirements/base-constraint.txt
# celery
lark==1.3.1
# via starrocks
lazy-object-proxy==1.10.0
# via openapi-spec-validator
limits==5.1.0
@@ -558,9 +507,7 @@ limits==5.1.0
# -c requirements/base-constraint.txt
# flask-limiter
lz4==4.4.5
# via
# clickhouse-connect
# trino
# via trino
mako==1.4.1
# via
# -c requirements/base-constraint.txt
@@ -620,10 +567,6 @@ msgspec==0.19.0
# via
# -c requirements/base-constraint.txt
# flask-session
multidict==6.7.1
# via
# aiohttp
# yarl
mysqlclient==2.2.8
# via apache-superset
nh3==0.3.6
@@ -662,22 +605,14 @@ openpyxl==3.1.5
# via
# -c requirements/base-constraint.txt
# pandas
opensearch-py==2.8.0
# via elasticsearch-dbapi
opentelemetry-api==1.39.1
# via fastmcp-slim
oracledb==4.0.2
# via
# apache-superset
# testcontainers
ordered-set==4.1.0
# via
# -c requirements/base-constraint.txt
# flask-limiter
orjson==3.11.9
# via
# crate
# trino
# via trino
packaging==25.0
# via
# -c requirements/base-constraint.txt
@@ -685,8 +620,8 @@ packaging==25.0
# apispec
# db-dtypes
# deprecation
# docker
# duckdb-engine
# elasticsearch-dbapi
# fastmcp-slim
# google-cloud-bigquery
# kombu
@@ -696,8 +631,6 @@ packaging==25.0
# pytest
# shillelagh
# sqlalchemy-bigquery
# sqlalchemy-firebird
# ydb
pandas==2.3.3
# via
# -c requirements/base-constraint.txt
@@ -730,7 +663,7 @@ pillow==12.3.0
# -c requirements/base-constraint.txt
# apache-superset
# matplotlib
pip==26.2.1
pip==25.1.1
# via apache-superset
platformdirs==4.3.8
# via
@@ -759,22 +692,16 @@ prompt-toolkit==3.0.51
# via
# -c requirements/base-constraint.txt
# click-repl
propcache==0.5.2
# via
# aiohttp
# yarl
prophet==1.4.0
# via apache-superset
proto-plus==1.25.0
# via google-api-core
protobuf==5.29.6
# via
# firebird-base
# google-api-core
# googleapis-common-protos
# grpcio-status
# proto-plus
# ydb
psutil==6.1.0
# via
# apache-superset
@@ -848,24 +775,6 @@ pyjwt==2.13.0
# mcp
pylint==3.3.7
# via apache-superset
pymonetdb==1.9.1
# via
# apache-superset
# sqlalchemy-monetdb
pymongo==4.17.0
# via
# pymongosql
# testcontainers
pymongosql==0.7.3
# via apache-superset
pymssql==2.3.13
# via
# apache-superset
# testcontainers
pymysql==1.2.0
# via
# starrocks
# testcontainers
pynacl==1.6.2
# via
# -c requirements/base-constraint.txt
@@ -910,13 +819,11 @@ python-dateutil==2.9.0.post0
# botocore
# celery
# croniter
# firebird-driver
# flask-appbuilder
# freezegun
# google-cloud-bigquery
# holidays
# matplotlib
# opensearch-py
# pandas
# pyhive
# shillelagh
@@ -927,7 +834,6 @@ python-dotenv==1.2.2
# apache-superset
# fastmcp-slim
# pydantic-settings
# testcontainers
python-ldap==3.4.7
# via apache-superset
python-multipart==0.0.29
@@ -971,7 +877,6 @@ requests==2.33.0
# google-api-core
# google-cloud-bigquery
# jsonschema-path
# opensearch-py
# pydruid
# pyhive
# requests-cache
@@ -1015,7 +920,7 @@ secretstorage==3.5.0
# via keyring
semver==3.0.4
# via apache-superset-extensions-cli
setuptools==84.0.0
setuptools==80.9.0
# via
# -c requirements/base-constraint.txt
# nodeenv
@@ -1050,9 +955,7 @@ sqlalchemy==2.0.52
# alembic
# apache-superset
# apache-superset-core
# databend-sqlalchemy
# duckdb-engine
# elasticsearch-dbapi
# flask-appbuilder
# flask-sqlalchemy
# marshmallow-sqlalchemy
@@ -1060,14 +963,7 @@ sqlalchemy==2.0.52
# sqlalchemy-bigquery
# sqlalchemy-cockroachdb
# sqlalchemy-continuum
# sqlalchemy-cratedb
# sqlalchemy-firebird
# sqlalchemy-monetdb
# sqlalchemy-risingwave
# sqlalchemy-utils
# starrocks
# testcontainers
# ydb-sqlalchemy
sqlalchemy-bigquery==1.17.2
# via apache-superset
sqlalchemy-cockroachdb==2.0.4
@@ -1076,16 +972,6 @@ sqlalchemy-continuum==1.7.0
# via
# -c requirements/base-constraint.txt
# apache-superset
sqlalchemy-cratedb==0.43.1
# via
# apache-superset
# testcontainers
sqlalchemy-firebird==2.2.0
# via apache-superset
sqlalchemy-monetdb==2.1.0
# via apache-superset
sqlalchemy-risingwave==2.1.0
# via apache-superset
sqlalchemy-utils==0.42.1
# via
# -c requirements/base-constraint.txt
@@ -1097,7 +983,6 @@ sqlglot==30.17.0
# -c requirements/base-constraint.txt
# apache-superset
# apache-superset-core
# ydb-sqlglot-plugin
sqloxide==0.1.51
# via apache-superset
sse-starlette==3.0.2
@@ -1110,8 +995,6 @@ starlette==1.3.1
# via
# fastmcp-slim
# mcp
starrocks==1.3.4
# via apache-superset
statsd==4.0.1
# via apache-superset
syntaqlite==0.9.0
@@ -1120,8 +1003,6 @@ tabulate==0.10.0
# via
# -c requirements/base-constraint.txt
# apache-superset
testcontainers==4.15.0
# via -r requirements/development.in
tiktoken==0.14.0
# via apache-superset
tomli-w==1.2.0
@@ -1133,14 +1014,10 @@ tqdm==4.67.1
# cmdstanpy
# prophet
trino==0.339.0
# via
# apache-superset
# testcontainers
# via apache-superset
typing-extensions==4.16.0
# via
# -c requirements/base-constraint.txt
# aiohttp
# aiosignal
# alembic
# anyio
# apache-superset
@@ -1152,7 +1029,6 @@ typing-extensions==4.16.0
# limits
# mcp
# opentelemetry-api
# oracledb
# py-key-value-aio
# pydantic
# pydantic-core
@@ -1161,7 +1037,6 @@ typing-extensions==4.16.0
# shillelagh
# sqlalchemy
# starlette
# testcontainers
# typing-inspection
typing-inspection==0.4.2
# via
@@ -1189,22 +1064,13 @@ urllib3==2.7.0
# via
# -c requirements/base-constraint.txt
# botocore
# clickhouse-connect
# crate
# docker
# elasticsearch
# opensearch-py
# requests
# requests-cache
# testcontainers
uvicorn==0.37.0
# via
# fastmcp-slim
# mcp
verlib2==0.3.2
# via
# crate
# sqlalchemy-cratedb
vine==5.1.0
# via
# -c requirements/base-constraint.txt
@@ -1238,7 +1104,6 @@ wrapt==1.17.2
# via
# -c requirements/base-constraint.txt
# deprecated
# testcontainers
wtforms==3.2.2
# via
# -c requirements/base-constraint.txt
@@ -1259,18 +1124,6 @@ xlsxwriter==3.2.9
# -c requirements/base-constraint.txt
# apache-superset
# pandas
yarl==1.24.5
# via aiohttp
ydb==3.31.4
# via
# ydb-dbapi
# ydb-sqlalchemy
ydb-dbapi==0.1.23
# via ydb-sqlalchemy
ydb-sqlalchemy==0.1.22
# via apache-superset
ydb-sqlglot-plugin==0.2.8
# via apache-superset
zipp==3.23.0
# via importlib-metadata
zope-event==5.0
+103 -54
View File
@@ -2086,6 +2086,14 @@
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/argparse": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
"integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
"dependencies": {
"sprintf-js": "~1.0.2"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
@@ -2099,6 +2107,18 @@
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/js-yaml": {
"version": "3.15.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"dependencies": {
"argparse": "^1.0.7",
"esprima": "^4.0.0"
},
"bin": {
"js-yaml": "bin/js-yaml.js"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
@@ -3195,9 +3215,9 @@
"integrity": "sha512-Fc8Ne62jJlKHiG/ajlonC4Sd66Pq68fFwK4ihJGNZpGqboc324SQk+lRvMzpPRuJOmfrJefdG8/7JdWX4bzJ2Q=="
},
"node_modules/brace-expansion": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz",
"integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==",
"dev": true,
"license": "MIT",
"peer": true,
@@ -3814,13 +3834,9 @@
}
},
"node_modules/d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-1.4.1.tgz",
"integrity": "sha512-p2sTHSLCJI2QKunbGb7ocOh7DgTAn8IrLx21QRc/BSnodXM4sv6aLQlnfpvehFMLZEfBc6g9pH9SWQccFYfJ9Q=="
},
"node_modules/d3-format": {
"version": "1.4.5",
@@ -4301,6 +4317,18 @@
"url": "https://opencollective.com/eslint"
}
},
"node_modules/esprima": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
"integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==",
"bin": {
"esparse": "bin/esparse.js",
"esvalidate": "bin/esvalidate.js"
},
"engines": {
"node": ">=4"
}
},
"node_modules/esquery": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/esquery/-/esquery-1.7.0.tgz",
@@ -4826,9 +4854,9 @@
"license": "MIT"
},
"node_modules/glob/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
@@ -5558,19 +5586,9 @@
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="
},
"node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/puzrin"
},
{
"type": "github",
"url": "https://github.com/sponsors/nodeca"
}
],
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
@@ -7830,6 +7848,11 @@
"node": ">=8"
}
},
"node_modules/sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="
},
"node_modules/sshpk": {
"version": "1.18.0",
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz",
@@ -7992,9 +8015,9 @@
"license": "MIT"
},
"node_modules/test-exclude/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
@@ -9819,7 +9842,7 @@
"debug": "4.4.0",
"execa": "4.1.0",
"istanbul-lib-coverage": "^3.0.0",
"js-yaml": "4.3.1",
"js-yaml": "4.1.1",
"nyc": "15.1.0",
"tinyglobby": "^0.2.14"
},
@@ -10098,7 +10121,7 @@
"requires": {
"@eslint/object-schema": "^3.0.5",
"debug": "^4.3.1",
"minimatch": ">=10"
"minimatch": "^10.2.4"
}
},
"@eslint/config-helpers": {
@@ -10190,10 +10213,18 @@
"camelcase": "^5.3.1",
"find-up": "^4.1.0",
"get-package-type": "^0.1.0",
"js-yaml": "4.3.1",
"js-yaml": "4.1.1",
"resolve-from": "^5.0.0"
},
"dependencies": {
"argparse": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
"integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
"requires": {
"sprintf-js": "~1.0.2"
}
},
"find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
@@ -10203,6 +10234,14 @@
"path-exists": "^4.0.0"
}
},
"js-yaml": {
"version": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"requires": {
"argparse": "^1.0.7",
"esprima": "^4.0.0"
}
},
"locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
@@ -11146,9 +11185,9 @@
"integrity": "sha512-Fc8Ne62jJlKHiG/ajlonC4Sd66Pq68fFwK4ihJGNZpGqboc324SQk+lRvMzpPRuJOmfrJefdG8/7JdWX4bzJ2Q=="
},
"brace-expansion": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz",
"integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==",
"dev": true,
"peer": true,
"requires": {
@@ -11591,9 +11630,9 @@
}
},
"d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA=="
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-1.4.1.tgz",
"integrity": "sha512-p2sTHSLCJI2QKunbGb7ocOh7DgTAn8IrLx21QRc/BSnodXM4sv6aLQlnfpvehFMLZEfBc6g9pH9SWQccFYfJ9Q=="
},
"d3-format": {
"version": "1.4.5",
@@ -11605,7 +11644,7 @@
"resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-1.4.0.tgz",
"integrity": "sha512-V9znK0zc3jOPV4VD2zZn0sDhZU3WAE2bmlxdIwwQPPzPjvyLkd8B3JUVdS1IDUFDkWZ72c9qnv1GK2ZagTZ8EA==",
"requires": {
"d3-color": "3.1.0"
"d3-color": "1"
}
},
"d3-scale": {
@@ -11853,7 +11892,7 @@
"imurmurhash": "^0.1.4",
"is-glob": "^4.0.0",
"json-stable-stringify-without-jsonify": "^1.0.1",
"minimatch": ">=10",
"minimatch": "^10.2.4",
"natural-compare": "^1.4.0",
"optionator": "^0.9.3"
},
@@ -11942,6 +11981,11 @@
"eslint-visitor-keys": "^5.0.1"
}
},
"esprima": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
"integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A=="
},
"esquery": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/esquery/-/esquery-1.7.0.tgz",
@@ -12290,7 +12334,7 @@
"fs.realpath": "^1.0.0",
"inflight": "^1.0.4",
"inherits": "2",
"minimatch": "<10",
"minimatch": "^3.1.1",
"once": "^1.3.0",
"path-is-absolute": "^1.0.0"
},
@@ -12301,9 +12345,9 @@
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="
},
"brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -12314,7 +12358,7 @@
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"requires": {
"brace-expansion": "1.1.18"
"brace-expansion": "^1.1.7"
}
}
}
@@ -12808,9 +12852,9 @@
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="
},
"js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"requires": {
"argparse": "^2.0.1"
}
@@ -13391,7 +13435,7 @@
"dev": true,
"peer": true,
"requires": {
"brace-expansion": ">=5.0.9"
"brace-expansion": "^5.0.5"
}
},
"minimist": {
@@ -14339,6 +14383,11 @@
"which": "^2.0.1"
}
},
"sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="
},
"sshpk": {
"version": "1.18.0",
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz",
@@ -14445,7 +14494,7 @@
"requires": {
"@istanbuljs/schema": "^0.1.2",
"glob": "^7.1.4",
"minimatch": "<10"
"minimatch": "^3.0.4"
},
"dependencies": {
"balanced-match": {
@@ -14454,9 +14503,9 @@
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="
},
"brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -14467,7 +14516,7 @@
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"requires": {
"brace-expansion": "1.1.18"
"brace-expansion": "^1.1.7"
}
}
}
+2 -8
View File
@@ -30,20 +30,14 @@
"overrides": {
"@babel/core": "^7.29.6",
"@cypress/code-coverage": {
"js-yaml": "4.3.1"
"js-yaml": "4.1.1"
},
"@cypress/request": "^3.0.0",
"cypress": {
"form-data": "^2.3.4"
},
"d3-interpolate": {
"d3-color": "3.1.0"
},
"minimatch@<10": {
"brace-expansion": "1.1.18"
},
"minimatch@>=10": {
"brace-expansion": ">=5.0.9"
"brace-expansion": ">=5.0.8"
},
"qs": "^6.14.2",
"uuid": "^11.1.1"
+1 -1
View File
@@ -77,7 +77,7 @@ module.exports = {
// @ant-design/colors and @ant-design/fast-color are allowed through because
// @ant-design/icons >= 6.3 deep-imports the ESM build of @ant-design/colors
// from its CJS output, so babel-jest must transform those files.
'node_modules/(?!@ant-design/(colors|fast-color)|@formatjs/.*|d3-(array|interpolate|color|time|scale|time-format|format|selection)|internmap|@mapbox/tiny-sdf|remark-gfm|(?!@ngrx|(?!deck.gl)|d3-scale)|markdown-table|micromark-*.|decode-named-character-reference|character-entities|mdast-util-*.|unist-util-*.|ccount|escape-string-regexp|nanoid|uuid|@rjsf/*.|@x0k/.*|echarts|zrender|fetch-mock|pretty-ms|parse-ms|ol|@babel/runtime|@emotion|cheerio|cheerio/lib|parse5|dom-serializer|entities|htmlparser2|rehype-sanitize|hast-util-sanitize|unified|unist-.*|hast-.*|hastscript|refractor|rehype-.*|remark-.*|mdast-.*|micromark-.*|parse-entities|character-reference-invalid|is-alphanumerical|is-alphabetical|is-decimal|is-hexadecimal|property-information|space-separated-tokens|comma-separated-tokens|bail|devlop|zwitch|longest-streak|geostyler|geostyler-.*|(?!geostyler)lodash|react-error-boundary|react-json-tree|react-base16-styling|lodash-es|rbush|quickselect|react-diff-viewer-continued|storybook/*.|json-stringify-pretty-compact|@x0k/json-schema-merge|content-disposition)',
'node_modules/(?!@ant-design/(colors|fast-color)|@formatjs/.*|d3-(array|interpolate|color|time|scale|time-format|format|selection)|internmap|@mapbox/tiny-sdf|remark-gfm|(?!@ngrx|(?!deck.gl)|d3-scale)|markdown-table|micromark-*.|decode-named-character-reference|character-entities|mdast-util-*.|unist-util-*.|ccount|escape-string-regexp|nanoid|uuid|@rjsf/*.|@x0k/.*|echarts|zrender|fetch-mock|pretty-ms|parse-ms|ol|@babel/runtime|@emotion|cheerio|cheerio/lib|parse5|dom-serializer|entities|htmlparser2|rehype-sanitize|hast-util-sanitize|unified|unist-.*|hast-.*|hastscript|refractor|rehype-.*|remark-.*|mdast-.*|micromark-.*|parse-entities|character-reference-invalid|is-alphanumerical|is-alphabetical|is-decimal|is-hexadecimal|property-information|space-separated-tokens|comma-separated-tokens|bail|devlop|zwitch|longest-streak|geostyler|geostyler-.*|(?!geostyler)lodash|react-error-boundary|react-json-tree|react-base16-styling|lodash-es|rbush|quickselect|react-diff-viewer-continued|storybook/*.|json-stringify-pretty-compact|@x0k/json-schema-merge)',
],
preset: 'ts-jest',
transform: {
+35 -35
View File
@@ -84,7 +84,7 @@
"antd": "^6.6.1",
"chrono-node": "^2.10.1",
"classnames": "^2.2.5",
"content-disposition": "^3.0.0",
"content-disposition": "^2.0.1",
"d3-scale": "^4.0.2",
"dayjs": "^1.11.23",
"dom-to-image-more": "^3.10.2",
@@ -218,13 +218,13 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"cheerio": "1.2.0",
"concurrently": "^10.0.5",
"copy-webpack-plugin": "^14.0.0",
"cross-env": "^10.1.0",
"css-loader": "^7.1.4",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-import-resolver-alias": "^1.1.2",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-i18n-strings": "file:eslint-rules/eslint-plugin-i18n-strings",
@@ -15753,9 +15753,9 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.11.17",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.17.tgz",
"integrity": "sha512-KAUDn1OSS0fmPlGO+NOUMRcOQ/b/shUBH3OgkG73mPgdf+JD/BQ6fHboGxNOxnUmlwcq+lLq3dTkayRPuSfXwg==",
"version": "2.11.16",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.16.tgz",
"integrity": "sha512-H/bNPUFHewJHyCTdjn1n3Pit5+2GmWT6mmeHImPX+8MA9NA6b67jO4gYmi4jTbCJb2otq34KMZnovndDPqJwhQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -17451,12 +17451,12 @@
}
},
"node_modules/content-disposition": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-3.0.0.tgz",
"integrity": "sha512-ZH/0Xs9rMIFWCOmGdmS9eHBTF62qqQYNz4nVjQhkdIO/a0fCP4UIM3mRz/wiqL0L14YgAz/1xio4OaSY4+ON/A==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-2.0.1.tgz",
"integrity": "sha512-e+H0ZXHSWYrENhQzw1LPuP4oF5MzVKmDU6d3hxlvaPEYLLg62MxtQNPRx4SYSuYJSBUgnQIG4HIN2tEtNv7Dog==",
"license": "MIT",
"engines": {
"node": ">=22"
"node": ">=18"
},
"funding": {
"type": "opencollective",
@@ -17804,9 +17804,9 @@
"license": "Python-2.0"
},
"node_modules/cosmiconfig/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -19858,9 +19858,9 @@
}
},
"node_modules/eslint": {
"version": "10.9.0",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.9.0.tgz",
"integrity": "sha512-5KeEOJZBfEVA47boFiBsf+6MmmJpffM7qEBg4pLla2e4nlKgdKlqCW0oSLOGsT8Wl5uCGJptLV1bkaiShj90Gw==",
"version": "10.8.1",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.8.1.tgz",
"integrity": "sha512-wqA7W2jbsC/BnV9Iv1UZpKVFkO1AdNoSmYW8NWG4HNOBbkAMvIqDZ27pI2f07dqn583NcIC44ckjAcOXDL1QbQ==",
"dev": true,
"license": "MIT",
"workspaces": [
@@ -26738,9 +26738,9 @@
"license": "MIT"
},
"node_modules/js-yaml": {
"version": "3.15.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"version": "3.15.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.0.tgz",
"integrity": "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -27662,9 +27662,9 @@
}
},
"node_modules/lerna/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -30733,9 +30733,9 @@
}
},
"node_modules/nwsapi": {
"version": "2.2.24",
"resolved": "https://registry.npmjs.org/nwsapi/-/nwsapi-2.2.24.tgz",
"integrity": "sha512-7YRhZ3jS45LwmSCT4b2sVFHt/WuovaktDU07QrtOBY2PXskss5a9jfmR9jptyumwXST+rFjrmppMY1KT/yn35A==",
"version": "2.2.23",
"resolved": "https://registry.npmjs.org/nwsapi/-/nwsapi-2.2.23.tgz",
"integrity": "sha512-7wfH4sLbt4M0gCDzGE6vzQBo0bfTKjU7Sfpqy/7gs1qBfYz2vEJH6vXcBKpO3+6Yu1telwd0t9HpyOoLEQQbIQ==",
"dev": true,
"license": "MIT"
},
@@ -34750,9 +34750,9 @@
}
},
"node_modules/react-diff-viewer-continued/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"funding": [
{
"type": "github",
@@ -36063,9 +36063,9 @@
"license": "MIT"
},
"node_modules/reselect": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.3.0.tgz",
"integrity": "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg==",
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.2.0.tgz",
"integrity": "sha512-AgZ3UOZm3YndfrJ4OYjgrT7bmCm/1iqkjvEfH/oYjzh6PD2qw4QuT3jjnXIrpdt4MTpMXclMT3lXbmRY+XRakw==",
"license": "MIT"
},
"node_modules/resize-observer-polyfill": {
@@ -38445,9 +38445,9 @@
"license": "ISC"
},
"node_modules/stylelint/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -43010,7 +43010,7 @@
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"rison": "^0.1.1",
"seedrandom": "^3.0.5",
"xss": "^1.0.15"
+5 -17
View File
@@ -161,7 +161,7 @@
"antd": "^6.6.1",
"chrono-node": "^2.10.1",
"classnames": "^2.2.5",
"content-disposition": "^3.0.0",
"content-disposition": "^2.0.1",
"d3-scale": "^4.0.2",
"dayjs": "^1.11.23",
"dom-to-image-more": "^3.10.2",
@@ -295,13 +295,13 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"cheerio": "1.2.0",
"concurrently": "^10.0.5",
"copy-webpack-plugin": "^14.0.0",
"cross-env": "^10.1.0",
"css-loader": "^7.1.4",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-import-resolver-alias": "^1.1.2",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-i18n-strings": "file:eslint-rules/eslint-plugin-i18n-strings",
@@ -383,9 +383,6 @@
"@great-expectations/jsonforms-antd-renderers": {
"antd": "$antd"
},
"@istanbuljs/load-nyc-config": {
"js-yaml": "^3.15.1"
},
"@jest/globals": "^30.4.0",
"@jest/types": "^30.4.0",
"@luma.gl/constants": "~9.2.5",
@@ -395,9 +392,6 @@
"@luma.gl/shadertools": "~9.2.5",
"@luma.gl/webgl": "~9.2.5",
"core-js": "^3.38.1",
"cosmiconfig": {
"js-yaml": "^4.3.1"
},
"dompurify": "^3.4.13",
"esbuild": "^0.28.1",
"eslint-plugin-import": {
@@ -414,22 +408,16 @@
"jest-mock": "^30.4.0",
"jest-runtime": "^30.4.0",
"jest-util": "^30.4.0",
"js-yaml-loader": {
"js-yaml": "^3.15.1"
},
"jspdf": "^4.2.0",
"lerna": {
"js-yaml": "^4.3.1"
"js-yaml": "^4.3.0"
},
"minimatch@>=10": {
"brace-expansion": ">=5.0.8"
},
"nanoid@>=3 <4": "3.3.18",
"nwsapi": "^2.2.24",
"nwsapi": "^2.2.13",
"puppeteer": "^22.4.1",
"react-diff-viewer-continued": {
"js-yaml": "^4.3.1"
},
"tar": "^7.5.16",
"typescript-json-schema": "^0.68.0",
"underscore": "^1.13.7",
@@ -89,7 +89,7 @@
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"rison": "^0.1.1",
"seedrandom": "^3.0.5",
"xss": "^1.0.15"
@@ -25,7 +25,6 @@ export enum VizType {
BoxPlot = 'box_plot',
Bubble = 'bubble_v2',
Bullet = 'bullet',
Butterfly = 'butterfly',
Calendar = 'cal_heatmap',
Cartodiagram = 'cartodiagram',
Chord = 'chord',
@@ -30,7 +30,6 @@ import type {
QueryFormData,
} from '../query';
import type { JsonResponse } from '../connection';
import type { MenuItem } from '../components/Menu';
/**
* A function which returns text (or marked-up text)
@@ -165,13 +164,6 @@ export interface SliceHeaderExtension {
dashboardId: number;
}
/**
* Interface for extensions to the Slice Header more-options menu
*/
export interface SliceHeaderMenuExtension extends SliceHeaderExtension {
sliceName: string;
}
/**
* Interface for extensions to Embed Modal
*/
@@ -270,9 +262,6 @@ export type Extensions = Partial<{
'sqleditor.extension.form': ComponentType<SQLFormExtensionProps>;
'sqleditor.extension.resultTable': ComponentType<SQLResultTableExtensionProps>;
'dashboard.slice.header': ComponentType<SliceHeaderExtension>;
'dashboard.slice.header.menu': (
context: SliceHeaderMenuExtension,
) => MenuItem[];
'sqleditor.extension.customAutocomplete': (
args: CustomAutoCompleteArgs,
) => CustomAutocomplete[] | undefined;
@@ -1,49 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import Echart from '../components/Echart';
import { ButterflyTransformedProps } from './types';
import { EventHandlers } from '../types';
export default function Butterfly(props: ButterflyTransformedProps) {
const { height, width, echartOptions, refs, onLegendStateChanged, formData } =
props;
const eventHandlers: EventHandlers = {
legendselectchanged: payload => {
onLegendStateChanged?.(payload.selected);
},
legendselectall: payload => {
onLegendStateChanged?.(payload.selected);
},
legendinverseselect: payload => {
onLegendStateChanged?.(payload.selected);
},
};
return (
<Echart
refs={refs}
height={height}
width={width}
echartOptions={echartOptions}
eventHandlers={eventHandlers}
vizType={formData.vizType}
/>
);
}
@@ -1,52 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
buildQueryContext,
ensureIsArray,
QueryFormData,
QueryFormOrderBy,
} from '@superset-ui/core';
import { buildSortMetricOrderby } from '@superset-ui/chart-controls';
export default function buildQuery(formData: QueryFormData) {
const columns = ensureIsArray(formData.groupby);
const baseMetrics = [
...ensureIsArray(formData.left_metric),
...ensureIsArray(formData.right_metric),
];
const { orderby, metrics } = buildSortMetricOrderby({
metrics: baseMetrics,
timeseriesLimitMetric: ensureIsArray(formData.orderby)[0],
order_desc: formData.order_desc,
});
const resolvedOrderby: QueryFormOrderBy[] | undefined = orderby.length
? orderby
: columns.length
? [[columns[0], true]]
: undefined;
return buildQueryContext(formData, baseQueryObject => [
{
...baseQueryObject,
columns,
metrics,
orderby: resolvedOrderby,
},
]);
}
@@ -1,29 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
DEFAULT_LEGEND_FORM_DATA,
DEFAULT_TITLE_FORM_DATA,
} from '../constants';
import { defaultXAxis } from '../defaults';
export const DEFAULT_FORM_DATA = {
...DEFAULT_LEGEND_FORM_DATA,
...DEFAULT_TITLE_FORM_DATA,
xAxisLabelRotation: defaultXAxis.xAxisLabelRotation,
};
@@ -1,242 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { ensureIsArray } from '@superset-ui/core';
import {
ControlPanelConfig,
ControlSubSectionHeader,
formatSelectOptions,
getStandardizedControls,
sections,
sharedControls,
} from '@superset-ui/chart-controls';
import {
legendSection,
showValueControl,
xAxisLabelRotation,
} from '../controls';
import { DEFAULT_FORM_DATA } from './constants';
const { xAxisTitleMargin, yAxisTitleMargin } = DEFAULT_FORM_DATA;
const config: ControlPanelConfig = {
controlPanelSections: [
{
label: t('Query'),
expanded: true,
controlSetRows: [
['groupby'],
[
{
name: 'left_metric',
config: {
...sharedControls.metric,
label: t('Left metric'),
description: t(
'Metric displayed on the left side of the butterfly chart',
),
},
},
],
[
{
name: 'right_metric',
config: {
...sharedControls.metric,
label: t('Right metric'),
description: t(
'Metric displayed on the right side of the butterfly chart',
),
},
},
],
['adhoc_filters'],
['row_limit'],
['orderby'],
[
{
name: 'order_desc',
config: {
...sharedControls.order_desc,
visibility: ({ controls }) => Boolean(controls.orderby.value),
},
},
],
],
},
{
label: t('Chart Options'),
expanded: true,
controlSetRows: [[showValueControl], ...legendSection],
},
{
label: t('Series settings'),
expanded: true,
controlSetRows: [
[
<ControlSubSectionHeader>
{t('Left series setting')}
</ControlSubSectionHeader>,
],
[
{
name: 'left_color',
config: {
label: t('Left color'),
type: 'ColorPickerControl',
default: { r: 84, g: 112, b: 198, a: 1 },
renderTrigger: true,
description: t('Color for bars on the left side of the chart'),
},
},
{
name: 'left_label',
config: {
label: t('Left label'),
type: 'TextControl',
renderTrigger: true,
description: t(
'Customize the label for the left series in tooltips and legend',
),
},
},
],
[
<ControlSubSectionHeader>
{t('Right series setting')}
</ControlSubSectionHeader>,
],
[
{
name: 'right_color',
config: {
label: t('Right color'),
type: 'ColorPickerControl',
default: { r: 145, g: 204, b: 117, a: 1 },
renderTrigger: true,
description: t('Color for bars on the right side of the chart'),
},
},
{
name: 'right_label',
config: {
label: t('Right label'),
type: 'TextControl',
renderTrigger: true,
description: t(
'Customize the label for the right series in tooltips and legend',
),
},
},
],
],
},
{
label: t('X Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'x_axis_label',
config: {
type: 'TextControl',
label: t('X Axis Label'),
renderTrigger: true,
default: '',
},
},
],
[
{
name: 'x_axis_title_margin',
config: {
type: 'SelectControl',
freeForm: true,
clearable: true,
label: t('X Axis title margin'),
renderTrigger: true,
default: xAxisTitleMargin,
choices: formatSelectOptions(sections.TITLE_MARGIN_OPTIONS),
},
},
],
['x_axis_format'],
['currency_format'],
],
},
{
label: t('Y Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'y_axis_label',
config: {
type: 'TextControl',
label: t('Y Axis Label'),
renderTrigger: true,
default: '',
},
},
],
[
{
name: 'y_axis_title_margin',
config: {
type: 'SelectControl',
freeForm: true,
clearable: true,
label: t('Y Axis title margin'),
renderTrigger: true,
default: yAxisTitleMargin,
choices: formatSelectOptions(sections.TITLE_MARGIN_OPTIONS),
},
},
],
[
{
name: xAxisLabelRotation.name,
config: {
...xAxisLabelRotation.config,
label: t('Rotate category label'),
description: t(
'Input field supports custom rotation. e.g. 30 for 30°',
),
},
},
],
],
},
],
controlOverrides: {
groupby: {
label: t('Categories'),
description: t('Dimension used for category labels on the vertical axis'),
multi: false,
},
},
formDataOverrides: formData => ({
...formData,
groupby: ensureIsArray(getStandardizedControls().shiftColumn()),
left_metric: getStandardizedControls().shiftMetric(),
right_metric: getStandardizedControls().shiftMetric(),
}),
};
export default config;
@@ -1,54 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { ChartMetadata, ChartPlugin } from '@superset-ui/core';
import buildQuery from './buildQuery';
import controlPanel from './controlPanel';
import transformProps from './transformProps';
import { EchartsButterflyChartProps, EchartsButterflyFormData } from './types';
export default class EchartsButterflyChartPlugin extends ChartPlugin<
EchartsButterflyFormData,
EchartsButterflyChartProps
> {
constructor() {
super({
buildQuery,
controlPanel,
loadChart: () => import('./Butterfly'),
metadata: new ChartMetadata({
credits: ['https://echarts.apache.org'],
category: t('Comparison'),
description: t(
'A butterfly chart compares two metrics across categories using horizontal bars ' +
'that extend left and right from a central axis.',
),
name: t('Butterfly Chart'),
tags: [
t('Categorical'),
t('Comparison'),
t('ECharts'),
t('Multi-Variables'),
],
thumbnail: '',
}),
transformProps,
});
}
}
@@ -1,298 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
CurrencyFormatter,
ensureIsArray,
getColumnLabel,
getMetricLabel,
getNumberFormatter,
NumberFormatter,
rgbToHex,
tooltipHtml,
} from '@superset-ui/core';
import type { ComposeOption } from 'echarts/core';
import type { BarSeriesOption } from 'echarts/charts';
import type { CallbackDataParams } from 'echarts/types/src/util/types';
import { EchartsButterflyChartProps, ButterflyTransformedProps } from './types';
import { DEFAULT_FORM_DATA } from './constants';
import { defaultGrid } from '../defaults';
import { getDefaultTooltip } from '../utils/tooltip';
import { Refs } from '../types';
import { NULL_STRING } from '../constants';
import { getChartPadding, getLegendProps } from '../utils/series';
import { resolveLegendLayout } from '../utils/legendLayout';
import { convertInteger } from '../utils/convertInteger';
type EChartsOption = ComposeOption<BarSeriesOption>;
const LABEL_LEFT = { position: 'left' as const };
const LABEL_RIGHT = { position: 'right' as const };
function formatCategory(value: unknown): string {
if (value == null) {
return NULL_STRING;
}
if (typeof value === 'string' || typeof value === 'number') {
return String(value);
}
return String(value);
}
function formatTooltip(
params: CallbackDataParams[],
formatter: NumberFormatter | CurrencyFormatter,
) {
const axisParams = params.filter(
param => param.seriesName && typeof param.value === 'number',
);
if (!axisParams.length) {
return '';
}
const title = axisParams[0].name;
const rows = axisParams.map(param => [
param.seriesName!,
formatter(Math.abs(param.value as number)),
]);
return tooltipHtml(rows, title);
}
export default function transformProps(
chartProps: EchartsButterflyChartProps,
): ButterflyTransformedProps {
const {
width,
height,
formData,
legendState,
queriesData,
hooks,
theme,
inContextMenu,
} = chartProps;
const refs: Refs = {};
const { data = [] } = queriesData[0];
const { setDataMask = () => {}, onContextMenu, onLegendStateChanged } = hooks;
const {
currencyFormat,
groupby,
leftMetric,
rightMetric,
leftColor = { r: 84, g: 112, b: 198, a: 1 },
rightColor = { r: 145, g: 204, b: 117, a: 1 },
leftLabel,
rightLabel,
xAxisLabel,
yAxisLabel,
xAxisFormat,
xAxisTitleMargin,
yAxisTitleMargin,
showLegend,
legendMargin,
legendOrientation,
legendType,
legendSort,
showValue,
xAxisLabelRotation,
}: EchartsButterflyChartProps['formData'] = {
...DEFAULT_FORM_DATA,
...formData,
};
const groupbyColumn = ensureIsArray(groupby)[0];
const categoryLabel = getColumnLabel(groupbyColumn);
const leftMetricLabel = leftMetric ? getMetricLabel(leftMetric) : '';
const rightMetricLabel = rightMetric ? getMetricLabel(rightMetric) : '';
const leftSeriesName = leftLabel || leftMetricLabel;
const rightSeriesName = rightLabel || rightMetricLabel;
const defaultFormatter = currencyFormat?.symbol
? new CurrencyFormatter({ d3Format: xAxisFormat, currency: currencyFormat })
: getNumberFormatter(xAxisFormat);
const categories = data.map(row => formatCategory(row[categoryLabel]));
const leftData = data.map(row => {
const value = Number(row[leftMetricLabel] ?? 0);
return {
value: -Math.abs(value),
label: LABEL_LEFT,
};
});
const rightData = data.map(row => {
const value = Number(row[rightMetricLabel] ?? 0);
return {
value: Math.abs(value),
label: LABEL_RIGHT,
};
});
const labelFormatter = (params: CallbackDataParams) => {
const value = Math.abs(params.value as number);
if (value === 0) {
return '';
}
return defaultFormatter(value);
};
const series: BarSeriesOption[] = [
{
name: leftSeriesName,
type: 'bar',
stack: 'Total',
label: {
show: showValue,
formatter: labelFormatter,
color: theme.colorText,
},
itemStyle: {
color: rgbToHex(leftColor.r, leftColor.g, leftColor.b),
},
data: leftData,
},
{
name: rightSeriesName,
type: 'bar',
stack: 'Total',
label: {
show: showValue,
formatter: labelFormatter,
color: theme.colorText,
},
itemStyle: {
color: rgbToHex(rightColor.r, rightColor.g, rightColor.b),
},
data: rightData,
},
];
const legendData = [leftSeriesName, rightSeriesName].sort((a, b) => {
if (!legendSort) {
return 0;
}
return legendSort === 'asc' ? a.localeCompare(b) : b.localeCompare(a);
});
const { effectiveLegendMargin, effectiveLegendType } = resolveLegendLayout({
chartHeight: height,
chartWidth: width,
legendItems: legendData,
legendMargin,
orientation: legendOrientation,
show: showLegend,
theme,
type: legendType,
});
const legendPadding = getChartPadding(
showLegend,
legendOrientation,
effectiveLegendMargin,
undefined,
true,
);
const echartOptions: EChartsOption = {
grid: {
...defaultGrid,
top:
theme.sizeUnit * 5 +
legendPadding.top +
convertInteger(xAxisTitleMargin),
bottom: theme.sizeUnit * 5 + legendPadding.bottom,
left:
theme.sizeUnit * 5 +
legendPadding.left +
convertInteger(yAxisTitleMargin),
right: theme.sizeUnit * 5 + legendPadding.right,
},
legend: {
...getLegendProps(
effectiveLegendType,
legendOrientation,
showLegend,
theme,
false,
legendState,
),
data: legendData,
},
xAxis: {
type: 'value',
position: 'top',
name: xAxisLabel,
nameLocation: 'middle',
nameGap: convertInteger(xAxisTitleMargin),
nameTextStyle: {
padding: [theme.sizeUnit * 4, 0, 0, 0],
},
splitLine: {
lineStyle: {
type: 'dashed',
},
},
axisLabel: {
formatter: (value: number) => defaultFormatter(Math.abs(value)),
},
},
yAxis: {
type: 'category',
name: yAxisLabel,
nameLocation: 'middle',
nameGap: convertInteger(yAxisTitleMargin),
nameTextStyle: {
padding: [0, theme.sizeUnit * 4, 0, 0],
},
axisLine: { show: false },
axisTick: { show: false },
splitLine: { show: false },
axisLabel: {
rotate: xAxisLabelRotation,
},
data: categories,
},
tooltip: {
...getDefaultTooltip(refs),
appendToBody: true,
trigger: 'axis',
axisPointer: { type: 'shadow' },
show: !inContextMenu,
formatter: (params: CallbackDataParams | CallbackDataParams[]) =>
formatTooltip(
ensureIsArray(params) as CallbackDataParams[],
defaultFormatter,
),
},
series,
};
return {
refs,
formData,
width,
height,
echartOptions,
setDataMask,
onContextMenu,
onLegendStateChanged,
};
}
@@ -1,52 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
ChartDataResponseResult,
ChartProps,
QueryFormColumn,
QueryFormData,
QueryFormMetric,
RgbaColor,
} from '@superset-ui/core';
import { BaseTransformedProps, LegendFormData, TitleFormData } from '../types';
export type EchartsButterflyFormData = QueryFormData &
LegendFormData &
TitleFormData & {
groupby: QueryFormColumn[];
leftMetric: QueryFormMetric;
rightMetric: QueryFormMetric;
leftColor: RgbaColor;
rightColor: RgbaColor;
leftLabel?: string;
rightLabel?: string;
xAxisLabel: string;
yAxisLabel: string;
xAxisFormat: string;
showValue: boolean;
xAxisLabelRotation: number;
};
export interface EchartsButterflyChartProps extends ChartProps {
formData: EchartsButterflyFormData;
queriesData: ChartDataResponseResult[];
}
export type ButterflyTransformedProps =
BaseTransformedProps<EchartsButterflyFormData>;
@@ -1276,15 +1276,10 @@ export default function transformProps(
// at the axis boundary.
...(showMaxLabel && {
showMaxLabel: true,
alignMaxLabel: 'right',
showMinLabel: true,
alignMinLabel: 'left',
}),
// The alignments assume the axis runs along the bottom; a horizontal
// chart puts this axis on the side, where they misplace the labels.
...(showMaxLabel &&
!isHorizontal && {
alignMaxLabel: 'right',
alignMinLabel: 'left',
}),
},
minorTick: { show: minorTicks },
axisTick: { show: axisTicks ? 'auto' : false },
@@ -46,7 +46,6 @@ export { default as EchartsSunburstChartPlugin } from './Sunburst';
export { default as EchartsBubbleChartPlugin } from './Bubble';
export { default as EchartsSankeyChartPlugin } from './Sankey';
export { default as EchartsWaterfallChartPlugin } from './Waterfall';
export { default as EchartsButterflyChartPlugin } from './Butterfly';
export { default as EchartsGanttChartPlugin } from './Gantt';
export { default as BoxPlotTransformProps } from './BoxPlot/transformProps';
@@ -63,7 +62,6 @@ export { default as HeatmapTransformProps } from './Heatmap/transformProps';
export { default as SunburstTransformProps } from './Sunburst/transformProps';
export { default as BubbleTransformProps } from './Bubble/transformProps';
export { default as WaterfallTransformProps } from './Waterfall/transformProps';
export { default as ButterflyTransformProps } from './Butterfly/transformProps';
export { default as HistogramTransformProps } from './Histogram/transformProps';
export { default as SankeyTransformProps } from './Sankey/transformProps';
export { default as GanttTransformProps } from './Gantt/transformProps';
@@ -1,85 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import buildQuery from '../../src/Butterfly/buildQuery';
const formData = {
datasource: '1__table',
viz_type: 'butterfly',
groupby: ['category'],
left_metric: 'left_sum',
right_metric: 'right_sum',
};
test('defaults to ordering by the category column', () => {
const [query] = buildQuery(formData).queries;
expect(query.columns).toEqual(['category']);
expect(query.metrics).toEqual(['left_sum', 'right_sum']);
expect(query.orderby).toEqual([['category', true]]);
});
test('wraps the sort metric in a valid orderby tuple', () => {
const sortMetric = {
expressionType: 'SIMPLE',
column: { column_name: 'left_sum' },
aggregate: 'SUM',
label: 'SUM(left_sum)',
};
const [query] = buildQuery({
...formData,
orderby: sortMetric,
order_desc: true,
}).queries;
expect(query.metrics).toEqual(['left_sum', 'right_sum', sortMetric]);
expect(query.orderby).toEqual([[sortMetric, false]]);
});
test('appends the sort metric when it is not already selected', () => {
const sortMetric = {
expressionType: 'SIMPLE',
column: { column_name: 'count' },
aggregate: 'SUM',
label: 'SUM(count)',
};
const [query] = buildQuery({
...formData,
orderby: sortMetric,
order_desc: false,
}).queries;
expect(query.metrics).toEqual(['left_sum', 'right_sum', sortMetric]);
expect(query.orderby).toEqual([[sortMetric, true]]);
});
test('leaves orderby unset when no category column is selected', () => {
const [query] = buildQuery({
...formData,
groupby: [],
}).queries;
expect(query.columns).toEqual([]);
expect(query.metrics).toEqual(['left_sum', 'right_sum']);
expect(query.orderby).toBeUndefined();
});
test('issues no metrics when none are selected', () => {
const [query] = buildQuery({
datasource: '1__table',
viz_type: 'butterfly',
groupby: ['category'],
}).queries;
expect(query.metrics).toEqual([]);
});
@@ -1,83 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { SqlaFormData } from '@superset-ui/core';
const mockShiftMetric = jest
.fn()
.mockReturnValueOnce('left_sum')
.mockReturnValueOnce('right_sum');
const mockShiftColumn = jest.fn(() => 'category');
jest.mock('@superset-ui/chart-controls', () => {
const actual = jest.requireActual('@superset-ui/chart-controls');
return {
...actual,
getStandardizedControls: jest.fn(() => ({
shiftMetric: mockShiftMetric,
shiftColumn: mockShiftColumn,
})),
};
});
// eslint-disable-next-line import/first
import controlPanel from '../../src/Butterfly/controlPanel';
const collectControlNames = () => {
const names = new Set<string>();
controlPanel.controlPanelSections?.forEach(section => {
section?.controlSetRows?.forEach(row => {
row.forEach(control => {
if (typeof control === 'string') {
names.add(control);
} else if (
control &&
typeof control === 'object' &&
'name' in control
) {
names.add(String(control.name));
}
});
});
});
return names;
};
test('exposes left and right metric controls', () => {
const controlNames = collectControlNames();
expect(controlNames.has('left_metric')).toBe(true);
expect(controlNames.has('right_metric')).toBe(true);
expect(controlNames.has('groupby')).toBe(true);
expect(controlNames.has('orderby')).toBe(true);
});
test('restricts categories to a single dimension', () => {
expect(controlPanel.controlOverrides?.groupby?.multi).toBe(false);
});
test('maps standardized controls to butterfly metrics', () => {
const dummyFormData = { someProp: 'test' } as unknown as SqlaFormData;
const formData = controlPanel.formDataOverrides?.(dummyFormData);
expect(formData?.someProp).toBe('test');
expect(formData?.groupby).toEqual(['category']);
expect(formData?.left_metric).toBe('left_sum');
expect(formData?.right_metric).toBe('right_sum');
expect(mockShiftMetric).toHaveBeenCalledTimes(2);
expect(mockShiftColumn).toHaveBeenCalledTimes(1);
});
@@ -1,202 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps } from '@superset-ui/core';
import { supersetTheme } from '@apache-superset/core/theme';
import {
EchartsButterflyChartProps,
ButterflyTransformedProps,
} from '../../src/Butterfly/types';
import transformProps from '../../src/Butterfly/transformProps';
import { NULL_STRING } from '../../src/constants';
type SeriesDataPoint = { value?: number } | number;
type ButterflyTestSeries = {
name?: string;
data?: SeriesDataPoint[];
itemStyle?: { color?: string };
label?: { show?: boolean };
};
type ButterflyTestEchartOptions = {
series?: ButterflyTestSeries[];
xAxis?: { name?: string; nameGap?: number };
yAxis?: {
name?: string;
nameGap?: number;
data?: string[];
axisLabel?: { rotate?: number };
};
legend?: { orient?: string; data?: string[] };
grid?: { left?: number; top?: number };
tooltip?: { show?: boolean };
};
const getEchartOptions = (
props: ButterflyTransformedProps,
): ButterflyTestEchartOptions =>
props.echartOptions as ButterflyTestEchartOptions;
const extractSeriesValues = (props: ButterflyTransformedProps) => {
const series = getEchartOptions(props).series ?? [];
return series.map(item =>
(item.data ?? []).map(entry =>
typeof entry === 'object' && entry !== null && 'value' in entry
? entry.value
: entry,
),
);
};
const extractSeriesNames = (props: ButterflyTransformedProps) => {
const series = getEchartOptions(props).series ?? [];
return series.map(item => item.name);
};
const data: Record<string, unknown>[] = [
{ category: 'A', left_sum: 10, right_sum: 25 },
{ category: 'B', left_sum: 5, right_sum: 19 },
];
const formData = {
groupby: ['category'],
left_metric: 'left_sum',
right_metric: 'right_sum',
left_color: { r: 84, g: 112, b: 198 },
right_color: { r: 145, g: 204, b: 117 },
showValue: true,
showLegend: true,
};
const createChartProps = (
overrides: Record<string, unknown> = {},
queryData: Record<string, unknown>[] = data,
) =>
new ChartProps({
formData: { ...formData, ...overrides },
width: 800,
height: 600,
queriesData: [{ data: queryData }],
theme: supersetTheme,
...((overrides.hooks ? { hooks: overrides.hooks } : {}) as object),
});
test('transforms chart props into diverging bar series', () => {
const transformedProps = transformProps(
createChartProps() as unknown as EchartsButterflyChartProps,
);
expect(extractSeriesValues(transformedProps)).toEqual([
[-10, -5],
[25, 19],
]);
});
test('uses absolute values for negative right-side metrics', () => {
const transformedProps = transformProps(
createChartProps({}, [
{ category: 'A', left_sum: -8, right_sum: -15 },
]) as unknown as EchartsButterflyChartProps,
);
expect(extractSeriesValues(transformedProps)).toEqual([[-8], [15]]);
});
test('formats null categories and missing metric values', () => {
const transformedProps = transformProps(
createChartProps({}, [
{ category: null, left_sum: undefined, right_sum: 7 },
]) as unknown as EchartsButterflyChartProps,
);
const { yAxis } = getEchartOptions(transformedProps);
expect(yAxis?.data).toEqual([NULL_STRING]);
const [leftValues, rightValues] = extractSeriesValues(transformedProps);
expect(Math.abs(leftValues[0] as number)).toBe(0);
expect(rightValues).toEqual([7]);
});
test('applies custom series labels, colors, and axis titles', () => {
const transformedProps = transformProps(
createChartProps({
left_label: 'Left side',
right_label: 'Right side',
left_color: { r: 255, g: 0, b: 0 },
right_color: { r: 0, g: 255, b: 0 },
x_axis_label: 'Value axis',
y_axis_label: 'Category axis',
}) as unknown as EchartsButterflyChartProps,
);
const { series, xAxis, yAxis } = getEchartOptions(transformedProps);
expect(extractSeriesNames(transformedProps)).toEqual([
'Left side',
'Right side',
]);
expect(series?.[0]?.itemStyle?.color).toBe('#ff0000');
expect(series?.[1]?.itemStyle?.color).toBe('#00ff00');
expect(xAxis?.name).toBe('Value axis');
expect(yAxis?.name).toBe('Category axis');
});
test('applies legend orientation, sort, and axis margin settings', () => {
const transformedProps = transformProps(
createChartProps({
legendOrientation: 'left',
legendSort: 'desc',
xAxisLabelRotation: 45,
x_axis_title_margin: 60,
y_axis_title_margin: 80,
}) as unknown as EchartsButterflyChartProps,
);
const { legend, xAxis, yAxis, grid } = getEchartOptions(transformedProps);
expect(legend?.orient).toBe('vertical');
expect(legend?.data).toEqual(['right_sum', 'left_sum']);
expect(xAxis?.nameGap).toBe(60);
expect(yAxis?.axisLabel?.rotate).toBe(45);
expect(yAxis?.nameGap).toBe(80);
expect(grid?.left).toBeGreaterThan(80);
expect(grid?.top).toBeGreaterThan(60);
});
test('hides value labels when showValue is false', () => {
const transformedProps = transformProps(
createChartProps({
showValue: false,
}) as unknown as EchartsButterflyChartProps,
);
const { series } = getEchartOptions(transformedProps);
expect(series?.[0]?.label?.show).toBe(false);
expect(series?.[1]?.label?.show).toBe(false);
});
test('hides tooltip while the context menu is open', () => {
const transformedProps = transformProps(
createChartProps({}, data) as unknown as EchartsButterflyChartProps,
);
const withContextMenu = transformProps({
...createChartProps(),
inContextMenu: true,
} as unknown as EchartsButterflyChartProps);
expect(getEchartOptions(transformedProps).tooltip?.show).toBe(true);
expect(getEchartOptions(withContextMenu).tooltip?.show).toBe(false);
});
@@ -2839,46 +2839,3 @@ test('applies gridlines to the value axis after a horizontal orientation swaps i
// and the gridlines belonging to it — end up on xAxis.
expect((echartOptions.xAxis as any).splitLine.show).toBe(false);
});
test('boundary label alignment is dropped when the orientation moves the time axis to the side', () => {
// The alignments position labels against the left and right edges of a
// bottom axis. A horizontal chart swaps the axes, so applying them there
// shifts the first label out of line with the rest (#43428 follow-up).
const monthData = [
{ __timestamp: Date.UTC(2003, 4, 1), sales: 100 },
{ __timestamp: Date.UTC(2003, 5, 1), sales: 200 },
];
const build = (orientation: OrientationType) =>
transformProps(
createTestChartProps({
formData: {
granularity_sqla: 'ds',
timeGrainSqla: TimeGranularity.MONTH,
xAxisTimeFormat: 'smart_date',
seriesType: EchartsTimeseriesSeriesType.Bar,
orientation,
},
queriesData: [
createTestQueryData(monthData, {
colnames: ['__timestamp', 'sales'],
coltypes: [GenericDataType.Temporal, GenericDataType.Numeric],
}),
],
}),
).echartOptions;
const vertical = build(OrientationType.Vertical).xAxis as any;
expect(vertical.axisLabel.alignMinLabel).toBe('left');
expect(vertical.axisLabel.alignMaxLabel).toBe('right');
// Horizontal swaps the axes, so the time axis is now yAxis.
const horizontal = build(OrientationType.Horizontal).yAxis as any;
expect(horizontal.axisLabel.alignMinLabel).toBeUndefined();
expect(horizontal.axisLabel.alignMaxLabel).toBeUndefined();
// The boundary labels themselves stay forced in both orientations.
expect(vertical.axisLabel.showMinLabel).toBe(true);
expect(vertical.axisLabel.showMaxLabel).toBe(true);
expect(horizontal.axisLabel.showMinLabel).toBe(true);
expect(horizontal.axisLabel.showMaxLabel).toBe(true);
});
@@ -462,14 +462,13 @@ export default function TableChart<D extends DataRecord = DataRecord>(
// only take relevant page size options
const pageSizeOptions = useMemo(() => {
const getServerPagination = (n: number) =>
n <= Math.max(rowCount, serverPageLength);
const getServerPagination = (n: number) => n <= rowCount;
return (
serverPagination ? SERVER_PAGE_SIZE_OPTIONS : PAGE_SIZE_OPTIONS
).filter(([n]) =>
serverPagination ? getServerPagination(n) : n <= 2 * data.length,
) as SizeOption[];
}, [data.length, rowCount, serverPageLength, serverPagination]);
}, [data.length, rowCount, serverPagination]);
const getValueRange = useCallback(
function getValueRange(key: string, alignPositiveNegative: boolean) {
@@ -67,23 +67,14 @@ async function renderAndWait(props = mockedProps) {
container = renderedContainer;
}
// A modal that wasn't handed an `etag` reads the dataset itself and can't save
// until that lands, so tests must wait before acting on the Save button.
async function waitForSaveEnabled() {
await waitFor(() =>
expect(screen.getByTestId('datasource-modal-save')).toBeEnabled(),
);
}
beforeEach(async () => {
beforeEach(() => {
fetchMock.clearHistory().removeRoutes();
cleanup();
renderAndWait();
fetchMock.post(SAVE_ENDPOINT, SAVE_PAYLOAD);
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
fetchMock.get(GET_DATASOURCE_ENDPOINT, { result: {} });
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
await waitForSaveEnabled();
});
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
@@ -127,7 +118,6 @@ describe('DatasourceModal', () => {
onDatasourceSave:
onDatasourceSave as unknown as typeof mockedProps.onDatasourceSave,
});
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
const okButton = await screen.findByRole('button', { name: 'Confirm' });
@@ -161,96 +151,6 @@ describe('DatasourceModal', () => {
putSpy.mockRestore();
});
test('sends the supplied etag as If-Match so a stale save is refused', async () => {
cleanup();
renderAndWait({ ...mockedProps, etag: '"v1"' } as typeof mockedProps);
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
await waitFor(() => {
const putCall = fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put');
expect(
new Headers(putCall?.options?.headers as HeadersInit).get('If-Match'),
).toEqual('"v1"');
});
});
test('reads the etag from the dataset when the caller supplies none', async () => {
cleanup();
fetchMock.clearHistory().removeRoutes();
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
fetchMock.get(GET_DATASOURCE_ENDPOINT, {
body: { result: {} },
headers: { ETag: '"v2"' },
});
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
// The form is seeded from the same read as the validator, so saving is
// unavailable until it lands.
expect(screen.getByTestId('datasource-modal-save')).toBeDisabled();
await screen.findByTestId('datasource-editor');
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
await waitFor(() => {
const putCall = fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put');
expect(
new Headers(putCall?.options?.headers as HeadersInit).get('If-Match'),
).toEqual('"v2"');
});
});
test('never saves unguarded while the validator read is in flight', async () => {
cleanup();
fetchMock.clearHistory().removeRoutes();
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
// A read that never resolves: the save path must stay closed rather than
// fall through to an unconditional PUT.
fetchMock.get(GET_DATASOURCE_ENDPOINT, new Promise(() => {}));
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
const saveButton = await screen.findByTestId('datasource-modal-save');
expect(saveButton).toBeDisabled();
fireEvent.click(saveButton);
expect(
fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put'),
).toBeUndefined();
});
test('shows a conflict dialog instead of a generic error on 412', async () => {
const putSpy = jest
.spyOn(SupersetClient, 'put')
.mockRejectedValue(new Response('', { status: 412 }));
try {
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
const conflictElements = await screen.findAllByText(
'Dataset changed since you opened it',
);
expect(conflictElements.length).toBeGreaterThan(0);
expect(
screen.queryByText('Error saving dataset'),
).not.toBeInTheDocument();
} finally {
putSpy.mockRestore();
}
});
test('shows sync columns checkbox when SQL changes', async () => {
cleanup();
const datasourceWithSQL = {
@@ -263,24 +163,15 @@ describe('DatasourceModal', () => {
};
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -317,24 +208,15 @@ describe('DatasourceModal', () => {
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL to trigger checkbox
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -387,24 +269,15 @@ describe('DatasourceModal', () => {
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL to trigger checkbox
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -21,7 +21,6 @@ import {
screen,
fireEvent,
act,
waitFor,
defaultStore as store,
} from 'spec/helpers/testing-library';
import fetchMock from 'fetch-mock';
@@ -73,9 +72,6 @@ test('DatasourceModal - should handle sync columns state without imperative moda
render(<DatasourceModal {...mockedProps} />, { store });
const saveButton = screen.getByTestId('datasource-modal-save');
// The modal fetches the current dataset version on open; save stays disabled
// until that settles
await waitFor(() => expect(saveButton).toBeEnabled());
// This should not throw any DOM errors
await act(async () => {
@@ -33,14 +33,12 @@ import {
Icons,
Button,
Checkbox,
Loading,
Modal,
AsyncEsmComponent,
} from '@superset-ui/core/components';
import withToasts from 'src/components/MessageToasts/withToasts';
import { ErrorMessageWithStackTrace } from 'src/components';
import type { DatasetObject } from 'src/features/datasets/types';
import { withCertificationFields } from '../utils';
import { mapSubjectValuesToIds } from 'src/features/subjects/SubjectPicker';
import type { DatasourceModalProps } from '../types';
@@ -93,18 +91,12 @@ export function buildExtraJsonObject(
const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
addSuccessToast,
datasource,
etag,
onDatasourceSave,
onHide,
show,
}) => {
const theme = useTheme();
const [currentDatasource, setCurrentDatasource] = useState(datasource);
// SQL of the server snapshot the form started from. The caller's, unless
// this modal read the dataset itself — then "did the SQL change?" has to be
// asked against the snapshot the payload is actually built from.
const [seededSql, setSeededSql] = useState<string | undefined>();
const [versionEtag, setVersionEtag] = useState(etag);
const [syncColumns, setSyncColumns] = useState(false);
const currencies = useSelector<
{
@@ -119,52 +111,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
const [isEditing, setIsEditing] = useState<boolean>(false);
const [modal, contextHolder] = Modal.useModal();
const [confirmModalOpen, setConfirmModalOpen] = useState(false);
const [isLoadingDatasource, setIsLoadingDatasource] = useState(false);
// Callers that read the dataset themselves (the dataset list) hand down the
// ETag of that read. The rest — Explore, where `datasource` comes from the
// page's bootstrap state — read it here, and must seed the form from the
// *same* response: a payload built from an older snapshot than the ETag
// guarding it would still be accepted, and would still clobber.
useEffect(() => {
setVersionEtag(etag);
if (etag || !show || !datasource.id) {
return undefined;
}
let cancelled = false;
setIsLoadingDatasource(true);
SupersetClient.get({
endpoint: `/api/v1/dataset/${datasource.id}`,
})
.then(({ json, response }) => {
if (cancelled) {
return;
}
const seeded = {
...datasource,
...json.result,
columns: withCertificationFields(json.result.columns),
};
setSeededSql(seeded.sql);
setCurrentDatasource(seeded);
setVersionEtag(response.headers.get('ETag') ?? undefined);
})
.catch(() => {
// The read failed outright, so there is no fresher snapshot to edit
// and no validator to send. Fall back to the caller's snapshot and an
// unconditional save, which is what this modal did before the guard.
})
.finally(() => {
if (!cancelled) {
setIsLoadingDatasource(false);
}
});
return () => {
cancelled = true;
};
}, [datasource.id, etag, show]);
const baselineSql = seededSql ?? datasource.sql;
const buildPayload = (datasource: Record<string, any>) => {
const payload: Record<string, any> = {
table_name: datasource.table_name,
@@ -251,13 +197,11 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
await SupersetClient.put({
endpoint: `/api/v1/dataset/${currentDatasource.id}?override_columns=${syncColumns}`,
jsonPayload: buildPayload(currentDatasource),
...(versionEtag ? { headers: { 'If-Match': versionEtag } } : {}),
});
const { json, response } = await SupersetClient.get({
const { json } = await SupersetClient.get({
endpoint: `/api/v1/dataset/${currentDatasource?.id}`,
});
setVersionEtag(response.headers.get('ETag') ?? undefined);
addSuccessToast(t('The dataset has been saved'));
// eslint-disable-next-line no-param-reassign
@@ -269,19 +213,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
onHide();
} catch (response) {
setIsSaving(false);
if ((response as Response)?.status === 412) {
modal.error({
title: t('Dataset changed since you opened it'),
okButtonProps: { danger: true, className: 'btn-danger' },
content: t(
'Someone else, or another one of your browser tabs, saved this ' +
'dataset after you opened it. Saving now would undo those ' +
'changes, so it was cancelled. Copy your edits, close this ' +
'dialog, and reopen the dataset to reapply them.',
),
});
return;
}
const error = await getClientErrorObject(response);
let errorResponse: SupersetError | undefined;
let errorText: string | undefined;
@@ -333,7 +264,7 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
here may affect other charts
in undesirable ways.`)}
/>
{baselineSql !== currentDatasource.sql && (
{datasource.sql !== currentDatasource.sql && (
<div
css={theme => ({
marginBottom: theme.marginMD,
@@ -367,14 +298,14 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
{t('Are you sure you want to save and apply changes?')}
</div>
),
[currentDatasource.sql, baselineSql, syncColumns],
[currentDatasource.sql, datasource.sql, syncColumns],
);
useEffect(() => {
if (baselineSql !== currentDatasource.sql) {
if (datasource.sql !== currentDatasource.sql) {
setSyncColumns(true);
}
}, [baselineSql, currentDatasource.sql]);
}, [datasource.sql, currentDatasource.sql]);
const onClickSave = () => {
setConfirmModalOpen(true);
@@ -425,7 +356,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
onClick={onClickSave}
disabled={
isSaving ||
isLoadingDatasource ||
errors.length > 0 ||
currentDatasource.is_managed_externally
}
@@ -451,18 +381,14 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
}}
draggable
>
{isLoadingDatasource ? (
<Loading />
) : (
<DatasourceEditor
showLoadingForImport
height={500}
datasource={currentDatasource}
onChange={onDatasourceChange}
setIsEditing={setIsEditing}
currencies={currencies}
/>
)}
<DatasourceEditor
showLoadingForImport
height={500}
datasource={currentDatasource}
onChange={onDatasourceChange}
setIsEditing={setIsEditing}
currencies={currencies}
/>
{contextHolder}
<Modal
title={t('Confirm save')}
@@ -20,5 +20,4 @@ import ChangeDatasourceModal from './ChangeDatasourceModal';
import DatasourceModal from './DatasourceModal';
export { ChangeDatasourceModal, DatasourceModal };
export { withCertificationFields } from './utils';
export type { DatasourceModalProps, ChangeDatasourceModalProps } from './types';
@@ -29,12 +29,6 @@ export interface DatasourceModalProps {
addSuccessToast: (msg: string) => void;
addDangerToast: (msg: string) => void;
datasource: DatasetObject;
/**
* ETag of the dataset read the form was seeded from. Replayed as `If-Match`
* on save so a stale form can't clobber a newer write. Fetched by the modal
* when the caller doesn't already have one.
*/
etag?: string;
onChange: () => {};
onDatasourceSave: (datasource: object, errors?: Array<any>) => {};
onHide: () => {};
@@ -27,7 +27,6 @@ import { nanoid } from 'nanoid';
import { SupersetClient } from '@superset-ui/core';
import { tn } from '@apache-superset/core/translation';
import rison from 'rison';
import type { ColumnObject } from 'src/features/datasets/types';
// Type definitions
@@ -249,29 +248,3 @@ export async function fetchSyncedColumns(
const { json } = await SupersetClient.get({ endpoint, signal });
return json as ColumnMetadata[];
}
/**
* Lift each column's certification out of its `extra` JSON into the flat
* fields the datasource editor binds to.
*/
export function withCertificationFields(columns: ColumnObject[] = []) {
return columns.map(column => {
// Malformed `extra` must not take out the whole column list, the way an
// uncaught parse would — same fallback as `hydrateMetricExtra`.
let parsedExtra;
try {
parsedExtra = JSON.parse(column.extra || '{}') || {};
} catch {
parsedExtra = {};
}
const {
certification: { details = '', certified_by: certifiedBy = '' } = {},
} = parsedExtra;
return {
...column,
certification_details: details || '',
certified_by: certifiedBy || '',
is_certified: details || certifiedBy,
};
});
}
@@ -23,7 +23,7 @@ import {
userEvent,
waitFor,
} from 'spec/helpers/testing-library';
import { FeatureFlag, VizType, getExtensionsRegistry } from '@superset-ui/core';
import { FeatureFlag, VizType } from '@superset-ui/core';
import mockState from 'spec/fixtures/mockState';
import { cachedSupersetGet } from 'src/utils/cachedSupersetGet';
import downloadAsImage from 'src/utils/downloadAsImage';
@@ -165,9 +165,6 @@ beforeEach(() => {
afterEach(() => {
Reflect.deleteProperty(document, 'fullscreenElement');
// TypedRegistry has no remove(); reset to a no-op so a registered slot does
// not leak into other tests (the empty array is guarded, so nothing injects).
getExtensionsRegistry().set('dashboard.slice.header.menu', () => []);
});
test('Should render', () => {
@@ -176,58 +173,6 @@ test('Should render', () => {
expect(screen.getByTestId(`slice_${SLICE_ID}-menu`)).toBeInTheDocument();
});
test('Injects dashboard.slice.header.menu items at the top of the menu', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => [
{ key: 'custom-ext', label: 'Custom Menu Extension' },
]);
renderWrapper();
openMenu();
const injected = screen.getByText('Custom Menu Extension');
expect(injected).toBeInTheDocument();
// Sits above the built-in entries.
const forceRefresh = screen.getByText('Force refresh');
expect(
injected.compareDocumentPosition(forceRefresh) &
Node.DOCUMENT_POSITION_FOLLOWING,
).toBeTruthy();
});
test('Injects nothing when dashboard.slice.header.menu returns no items', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => []);
renderWrapper();
openMenu();
expect(screen.queryByText('Custom Menu Extension')).not.toBeInTheDocument();
// The menu still renders its built-in entries unchanged (no dangling divider
// is added since the empty array is guarded).
expect(screen.getByText('Force refresh')).toBeInTheDocument();
});
test('Menu survives a dashboard.slice.header.menu extension that throws', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => {
throw new Error('boom');
});
renderWrapper();
openMenu();
// The throw is isolated: the built-in menu still renders.
expect(screen.getByText('Force refresh')).toBeInTheDocument();
expect(screen.getByText('Enter fullscreen')).toBeInTheDocument();
});
test('Injects nothing when the extension returns a non-array', () => {
getExtensionsRegistry().set(
'dashboard.slice.header.menu',
// JS registrations bypass the MenuItem[] type; a bad return must not crash.
(() => undefined) as never,
);
renderWrapper();
openMenu();
expect(screen.getByText('Force refresh')).toBeInTheDocument();
});
test('Should render default props', () => {
const props = createProps();
@@ -34,13 +34,11 @@ import {
isFeatureEnabled,
FeatureFlag,
getChartMetadataRegistry,
getExtensionsRegistry,
VizType,
BinaryQueryObjectFilterClause,
JsonObject,
QueryFormData,
} from '@superset-ui/core';
import { logging } from '@apache-superset/core/utils';
import { css, useTheme, styled } from '@apache-superset/core/theme';
import { useSelector } from 'react-redux';
import { Menu, MenuItem } from '@superset-ui/core/components/Menu';
@@ -167,8 +165,6 @@ const queueChartResize = () => {
}, 300);
};
const extensionsRegistry = getExtensionsRegistry();
const SliceHeaderControls = (
props: SliceHeaderControlsPropsWithRouter | SliceHeaderControlsProps,
) => {
@@ -518,26 +514,6 @@ const SliceHeaderControls = (
},
];
const sliceHeaderMenuExtension = extensionsRegistry.get(
'dashboard.slice.header.menu',
);
if (sliceHeaderMenuExtension) {
// Isolate the extension: a bad registration (throwing, or returning a
// non-array) must not take down the whole dashboard render.
try {
const extensionItems = sliceHeaderMenuExtension({
sliceId: slice.slice_id,
sliceName: slice.slice_name,
dashboardId,
});
if (Array.isArray(extensionItems) && extensionItems.length) {
newMenuItems.unshift(...extensionItems, { type: 'divider' });
}
} catch (error) {
logging.error('dashboard.slice.header.menu extension failed', error);
}
}
if (slice.description) {
newMenuItems.push({
key: MenuKeys.ToggleChartDescription,
@@ -28,14 +28,6 @@ const Wrapper = styled.div`
flex-direction: column;
height: 100%;
.ant-tabs {
height: 100%;
}
.ant-tabs-body {
height: 100%;
}
.ant-tabs-content {
display: flex;
flex-direction: column;
@@ -88,7 +80,12 @@ export const ResultsPaneOnDashboard = ({
return (
<Wrapper>
<Tabs activeKey={activeTabKey} onChange={setActiveTabKey} items={items} />
<Tabs
fullHeight
activeKey={activeTabKey}
onChange={setActiveTabKey}
items={items}
/>
</Wrapper>
);
};
@@ -25,9 +25,15 @@ import {
} from 'spec/helpers/testing-library';
import { ChartMetadata, ChartPlugin, VizType } from '@superset-ui/core';
import { setupAGGridModules } from '@superset-ui/core/components/ThemedAgGridReact';
import Tabs from '@superset-ui/core/components/Tabs';
import { ResultsPaneOnDashboard } from '../components';
import { createResultsPaneOnDashboardProps } from './fixture';
jest.mock('@superset-ui/core/components/Tabs', () => {
const actual = jest.requireActual('@superset-ui/core/components/Tabs');
return { __esModule: true, ...actual, default: jest.fn(actual.default) };
});
beforeAll(() => {
setupAGGridModules();
});
@@ -106,6 +112,8 @@ describe('ResultsPaneOnDashboard', () => {
expect(
await findByText('No results were returned for this query'),
).toBeVisible();
const [tabsProps] = (Tabs as unknown as jest.Mock).mock.calls[0];
expect(tabsProps).toEqual(expect.objectContaining({ fullHeight: true }));
});
test('render errorMessage', async () => {
@@ -297,12 +297,6 @@ test('Click on Edit dataset', async () => {
const props = createProps();
fetchMock.removeRoute(getDbWithQuery);
fetchMock.get(getDbWithQuery, { result: [] }, { name: getDbWithQuery });
fetchMock.removeRoute(getDatasetWithAllMockRouteName);
fetchMock.get(
getDatasetWithAll,
{ result: {} },
{ name: getDatasetWithAllMockRouteName },
);
render(<DatasourceControl {...props} />, {
useRedux: true,
useRouter: true,
@@ -313,9 +307,7 @@ test('Click on Edit dataset', async () => {
await userEvent.click(screen.getByText('Edit dataset'));
});
expect(
await screen.findByTestId('mock-datasource-editor'),
).toBeInTheDocument();
expect(screen.getByTestId('mock-datasource-editor')).toBeInTheDocument();
});
test('Edit dataset should be disabled when user is not admin', async () => {
@@ -43,6 +43,7 @@ import {
} from 'src/views/CRUD/utils';
import { SUBJECT_OPTION_FILTER_PROPS } from 'src/features/subjects/SubjectSelectLabel';
import { SubjectPile } from 'src/features/subjects/SubjectPile';
import { ColumnObject } from 'src/features/datasets/types';
import { useListViewResource } from 'src/views/CRUD/hooks';
import {
ActionButton,
@@ -61,7 +62,6 @@ import {
} from '@superset-ui/core/components';
import {
DatasourceModal,
withCertificationFields,
GenericLink,
ImportModal as ImportModelsModal,
ModifiedInfo,
@@ -496,8 +496,6 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
const [datasetCurrentlyEditing, setDatasetCurrentlyEditing] =
useState<Dataset | null>(null);
const [datasetCurrentlyEditingEtag, setDatasetCurrentlyEditingEtag] =
useState<string | undefined>();
const [datasetCurrentlyDuplicating, setDatasetCurrentlyDuplicating] =
useState<VirtualDataset | null>(null);
@@ -567,11 +565,24 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
SupersetClient.get({
endpoint: `/api/v1/dataset/${id}`,
})
.then(({ json = {}, response }) => {
setDatasetCurrentlyEditingEtag(
response.headers.get('ETag') ?? undefined,
.then(({ json = {} }) => {
const addCertificationFields = json.result.columns.map(
(column: ColumnObject) => {
const {
certification: {
details = '',
certified_by: certifiedBy = '',
} = {},
} = JSON.parse(column.extra || '{}') || {};
return {
...column,
certification_details: details || '',
certified_by: certifiedBy || '',
is_certified: details || certifiedBy,
};
},
);
json.result.columns = withCertificationFields(json.result.columns);
json.result.columns = [...addCertificationFields];
setDatasetCurrentlyEditing(json.result);
})
.catch(() => {
@@ -1513,7 +1524,6 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
{datasetCurrentlyEditing && (
<DatasourceModal
datasource={datasetCurrentlyEditing}
etag={datasetCurrentlyEditingEtag}
onDatasourceSave={refreshData}
onHide={closeDatasetEditModal}
show
@@ -65,7 +65,6 @@ import {
EchartsRoseChartPlugin,
EchartsTimePivotChartPlugin,
EchartsBulletChartPlugin,
EchartsButterflyChartPlugin,
} from '@superset-ui/plugin-chart-echarts';
import {
SelectFilterPlugin,
@@ -164,9 +163,6 @@ export default class MainPreset extends Preset {
new EchartsWaterfallChartPlugin().configure({
key: VizType.Waterfall,
}),
new EchartsButterflyChartPlugin().configure({
key: VizType.Butterfly,
}),
new EchartsHeatmapChartPlugin().configure({ key: VizType.Heatmap }),
new EchartsHistogramChartPlugin().configure({ key: VizType.Histogram }),
new SelectFilterPlugin().configure({ key: FilterPlugins.Select }),
+4 -4
View File
@@ -26,7 +26,7 @@
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.67.0",
"@typescript-eslint/parser": "^8.67.0",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.11.0",
"oxfmt": "^0.64.0",
@@ -1624,9 +1624,9 @@
}
},
"node_modules/eslint": {
"version": "10.9.0",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.9.0.tgz",
"integrity": "sha512-5KeEOJZBfEVA47boFiBsf+6MmmJpffM7qEBg4pLla2e4nlKgdKlqCW0oSLOGsT8Wl5uCGJptLV1bkaiShj90Gw==",
"version": "10.8.1",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.8.1.tgz",
"integrity": "sha512-wqA7W2jbsC/BnV9Iv1UZpKVFkO1AdNoSmYW8NWG4HNOBbkAMvIqDZ27pI2f07dqn583NcIC44ckjAcOXDL1QbQ==",
"dev": true,
"license": "MIT",
"workspaces": [
+1 -1
View File
@@ -34,7 +34,7 @@
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.67.0",
"@typescript-eslint/parser": "^8.67.0",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.11.0",
"oxfmt": "^0.64.0",
+6
View File
@@ -18,6 +18,7 @@ import logging
from flask import Response
from flask_appbuilder.api import expose, protect, safe
from flask_appbuilder.security.decorators import has_access_api
from superset.commands.dashboard.filter_state.create import CreateFilterStateCommand
from superset.commands.dashboard.filter_state.delete import DeleteFilterStateCommand
@@ -25,6 +26,7 @@ from superset.commands.dashboard.filter_state.get import GetFilterStateCommand
from superset.commands.dashboard.filter_state.update import UpdateFilterStateCommand
from superset.extensions import event_logger
from superset.temporary_cache.api import TemporaryCacheRestApi
from superset.views.base import api
logger = logging.getLogger(__name__)
@@ -46,6 +48,8 @@ class DashboardFilterStateRestApi(TemporaryCacheRestApi):
def get_delete_command(self) -> type[DeleteFilterStateCommand]:
return DeleteFilterStateCommand
@api
@has_access_api
@expose("/<int:pk>/filter_state", methods=("POST",))
@protect()
@safe
@@ -169,6 +173,8 @@ class DashboardFilterStateRestApi(TemporaryCacheRestApi):
"""
return super().post(pk)
@api
@has_access_api
@expose("/<int:pk>/filter_state/<string:key>", methods=("PUT",))
@protect()
@safe
+6 -54
View File
@@ -29,7 +29,7 @@ from flask_appbuilder.api import expose, protect, rison as parse_rison, safe
from flask_appbuilder.api.schemas import get_item_schema
from flask_appbuilder.const import API_RESULT_RES_KEY, API_SELECT_COLUMNS_RIS_KEY
from flask_appbuilder.models.sqla.interface import SQLAInterface
from flask_babel import gettext as _, ngettext
from flask_babel import ngettext
from jinja2.exceptions import TemplateError
from marshmallow import ValidationError
from sqlalchemy.orm.exc import MultipleResultsFound
@@ -95,20 +95,13 @@ from superset.subjects.filters import FilterRelatedSubjects, subject_type_filter
from superset.utils import json
from superset.utils.core import parse_boolean_string, send_export_zip
from superset.versioning.api_helpers import (
concurrency_token_from,
current_entity_etag_uuid,
current_entity_version_info,
entity_concurrency_token,
get_version_endpoint,
list_versions_endpoint,
lock_entity_for_update,
restore_version_endpoint,
)
from superset.versioning.etag import (
is_conditional_write,
raise_for_stale_write,
set_version_etag,
StaleEntityError,
)
from superset.versioning.etag import set_version_etag
from superset.versioning.schemas import VersionListItemSchema
from superset.views.base import DatasourceFilter
from superset.views.base_api import (
@@ -549,14 +542,6 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
schema:
type: boolean
name: override_columns
- in: header
schema:
type: string
name: If-Match
description: >-
Optional optimistic-concurrency guard. Pass the ``ETag`` returned
by a prior read of this dataset; the update is rejected with 412
if the dataset has changed since.
requestBody:
description: Dataset schema
required: true
@@ -633,17 +618,6 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
$ref: '#/components/responses/403'
404:
$ref: '#/components/responses/404'
412:
description: >-
The dataset changed since the version identified by the
request's ``If-Match`` header; the update was not applied.
content:
application/json:
schema:
type: object
properties:
message:
type: string
422:
$ref: '#/components/responses/422'
500:
@@ -660,32 +634,10 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
except ValidationError as error:
return self.response_400(message=error.messages)
# Serialise conditional saves on this dataset: the guard below reads
# the live version, the command writes, and the two must not interleave
# with another request's. Only a conditional save pays for the lock; an
# unconditional PUT behaves exactly as it did before the guard existed.
if is_conditional_write():
lock_entity_for_update(SqlaTable, pk)
# Live version identifiers before the update (empty + query-free when
# ``ENABLE_VERSIONING_CAPTURE`` is off).
old_info = current_entity_version_info(SqlaTable, pk)
try:
raise_for_stale_write(concurrency_token_from(old_info))
except StaleEntityError:
return set_version_etag(
self.response(
412,
message=_(
"The dataset was changed by another user or browser tab "
"after you opened it. Reopen it to pick up the latest "
"version, then reapply your changes."
),
),
concurrency_token_from(old_info),
)
try:
# Two commands, two commits, two Continuum transactions for an
# ``override_columns`` save — deliberately NOT merged into one
@@ -709,13 +661,13 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
new_info = current_entity_version_info(
SqlaTable, changed_model.id, changed_model.uuid
)
etag_version_uuid = concurrency_token_from(new_info)
etag_version_uuid = new_info.version_uuid
if override_columns:
RefreshDatasetCommand(pk).run()
# The ETag must reflect the entity's *current live* version,
# which after the refresh is the refresh's transaction —
# re-read it rather than reusing the pre-refresh uuid.
etag_version_uuid = entity_concurrency_token(
etag_version_uuid = current_entity_etag_uuid(
SqlaTable, changed_model.id, changed_model.uuid
)
response = self.response(
@@ -1748,7 +1700,7 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
return set_version_etag(
self.response(200, **response),
entity_concurrency_token(SqlaTable, table.id, table.uuid),
current_entity_etag_uuid(SqlaTable, table.id, table.uuid),
)
@expose("/<int:pk>/drill_info/", methods=("GET",))
-15
View File
@@ -41,7 +41,6 @@ import sqlalchemy.dialects
from flask import current_app as app
from sqlalchemy.engine.default import DefaultDialect
from sqlalchemy.exc import NoSuchModuleError
from sqlalchemy.sql import compiler as sqla_compiler
from superset import feature_flag_manager
from superset.db_engine_specs.base import BaseEngineSpec
@@ -160,16 +159,6 @@ def get_available_engine_specs() -> dict[type[BaseEngineSpec], set[str]]: # noq
continue
# installed 3rd-party dialects
#
# `ep.load()` runs arbitrary module-level code in the third-party package.
# Some dialects (e.g. sqlalchemy-monetdb) mutate SQLAlchemy's shared,
# process-global `compiler.OPERATORS` mapping in place on import instead
# of subclassing it, which would otherwise silently change SQL rendering
# (e.g. `!=` -> `<>`) for every dialect for the rest of the process, not
# just the misbehaving one. Snapshot/restore around each load so a
# buggy connector can't leak global compiler state into unrelated
# dialects just because it was enumerated here.
operators_snapshot = dict(sqla_compiler.OPERATORS)
for ep in entry_points(group="sqlalchemy.dialects"):
try:
dialect = ep.load()
@@ -204,10 +193,6 @@ def get_available_engine_specs() -> dict[type[BaseEngineSpec], set[str]]: # noq
if isinstance(driver, bytes):
driver = driver.decode()
drivers[backend].add(driver)
finally:
if sqla_compiler.OPERATORS != operators_snapshot:
sqla_compiler.OPERATORS.clear()
sqla_compiler.OPERATORS.update(operators_snapshot)
dbs_denylist = app.config["DBS_AVAILABLE_DENYLIST"]
if not feature_flag_manager.is_feature_enabled("ENABLE_SUPERSET_META_DB"):
+1 -4
View File
@@ -44,10 +44,7 @@ class CockroachDbEngineSpec(PostgresEngineSpec):
DatabaseCategory.TRADITIONAL_RDBMS,
DatabaseCategory.OPEN_SOURCE,
],
# sqlalchemy-cockroachdb declares no DBAPI dependency of its own (see
# the comment on the `cockroachdb` extra in pyproject.toml), so a
# plain `cockroachdb://` URL also needs psycopg2 installed to connect.
"pypi_packages": ["sqlalchemy-cockroachdb", "psycopg2-binary"],
"pypi_packages": ["sqlalchemy-cockroachdb", "psycopg2"],
"connection_string": "cockroachdb://root@{hostname}:{port}/{database}?sslmode=disable",
"default_port": 26257,
"docs_url": "https://github.com/cockroachdb/sqlalchemy-cockroachdb",
+1 -10
View File
@@ -26,7 +26,7 @@ from collections import defaultdict, deque
from datetime import datetime
from re import Pattern
from textwrap import dedent
from typing import Any, Callable, cast, Optional, TYPE_CHECKING
from typing import Any, cast, Optional, TYPE_CHECKING
from urllib import parse
import pandas as pd
@@ -182,15 +182,6 @@ class PrestoBaseEngineSpec(BaseEngineSpec, metaclass=ABCMeta):
# which raises a query error. Use = true/false instead.
use_equality_for_boolean_filters = True
# Presto/Trino's coordinator sends query results as JSON, which has no
# literal for NaN/Infinity/-Infinity, so REAL/DOUBLE columns holding
# those values arrive as quoted strings. Coerce them back to real
# floats so numeric post-processing (e.g. a pivot's mean) doesn't choke
# on a string value.
column_type_mutators: dict[types.TypeEngine, Callable[[Any], Any]] = {
types.FLOAT: lambda val: float(val) if isinstance(val, str) else val
}
column_type_mappings = (
(
re.compile(r"^boolean.*", re.IGNORECASE),
+1 -12
View File
@@ -22,8 +22,7 @@ import math
import threading
import time
from collections.abc import Sequence
from decimal import Decimal
from typing import Any, Callable, TYPE_CHECKING
from typing import Any, TYPE_CHECKING
import requests
from flask import copy_current_request_context, ctx, current_app as app, Flask, g
@@ -31,7 +30,6 @@ from flask_babel import gettext as __
from sqlalchemy.engine.reflection import Inspector
from sqlalchemy.engine.url import URL
from sqlalchemy.exc import NoSuchTableError
from sqlalchemy.types import DECIMAL, TypeEngine
from superset import cache_manager, db
from superset.common.db_query_status import QueryStatus
@@ -80,15 +78,6 @@ class TrinoEngineSpec(PrestoBaseEngineSpec):
"$.oauth2_client_info.secret": "OAuth2 client secret",
}
# Trino's DBAPI driver can return DECIMAL columns as plain strings
# (e.g. when a value's precision/scale can't be inferred from the
# column type alone), which later breaks numeric post-processing
# (e.g. pivot with a mean aggregate). Coerce them back to Decimal.
column_type_mutators: dict[TypeEngine, Callable[[Any], Any]] = {
**PrestoBaseEngineSpec.column_type_mutators,
DECIMAL: lambda val: Decimal(val) if isinstance(val, str) else val,
}
# The full set of columns Trino's "<table>$partitions" exposes for an
# Iceberg table. The real partition keys are nested in the "partition" ROW,
# so none of these are user partition columns.
-9
View File
@@ -30,7 +30,6 @@ from superset.commands.temporary_cache.exceptions import (
TemporaryCacheResourceNotFoundError,
)
from superset.constants import MODEL_API_RW_METHOD_PERMISSION_MAP
from superset.exceptions import SupersetTemplateException
from superset.explore.form_data.schemas import FormDataPostSchema, FormDataPutSchema
from superset.extensions import event_logger
from superset.views.base_api import BaseSupersetApi, requires_json, statsd_metrics
@@ -111,8 +110,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("PUT",))
@protect()
@@ -186,8 +183,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("GET",))
@protect()
@@ -239,8 +234,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("DELETE",))
@protect()
@@ -293,5 +286,3 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
+1 -10
View File
@@ -16,8 +16,6 @@
# under the License.
from typing import Optional
from jinja2.exceptions import TemplateError
from superset import security_manager
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
@@ -35,7 +33,6 @@ from superset.commands.exceptions import (
from superset.daos.chart import ChartDAO
from superset.daos.dataset import DatasetDAO
from superset.daos.query import QueryDAO
from superset.exceptions import SupersetTemplateException
from superset.utils.core import DatasourceType
@@ -56,13 +53,7 @@ def check_query_access(query_id: int) -> Optional[bool]:
# Access checks below, no need to validate them twice as they can be expensive.
query = QueryDAO.find_by_id(query_id, skip_base_filter=True)
if query:
try:
security_manager.raise_for_access(query=query)
except TemplateError as ex:
# raise_for_access() Jinja-renders the query's SQL to resolve
# the tables it touches; a malformed template surfaces here as
# a raw jinja2 exception rather than a Superset one.
raise SupersetTemplateException(str(ex)) from ex
security_manager.raise_for_access(query=query)
return True
raise QueryNotFoundValidationError()
-12
View File
@@ -4823,18 +4823,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
template_processor=template_processor
)
is_metric_filter = True
elif (
col_obj is None
and isinstance(flt_col, str)
and flt_col in adhoc_columns_by_label
):
sqla_col, _unused = self.adhoc_column_to_sqla(
col=adhoc_columns_by_label[flt_col],
template_processor=template_processor,
)
if isinstance(sqla_col, ColumnElement):
applied_adhoc_filters_columns.append(flt_col)
filter_grain = flt.get("grain")
# Check if this filter should be skipped because it was handled in
+1 -189
View File
@@ -16,7 +16,6 @@
# under the License.
from typing import Any, Optional
import pandas as pd
from flask_babel import gettext as _
from pandas import DataFrame
@@ -27,116 +26,6 @@ from superset.utils.pandas_postprocessing.utils import (
validate_column_args,
)
_PERCENT_MODES = frozenset({"percent_row", "percent_col", "percent_total"})
# Aggregate operator names that produce additive results across groups —
# the sum of the per-cell values equals the row/column/grand rollup the
# database would compute over the underlying rows. ``show_values_as``
# percent transforms divide each cell by that sum, so they are only
# meaningful when the sum IS the rollup. For non-additive operators
# (mean, median, min, max, etc.) the "percent of row" the exports would
# show is not the "percent of row" the chart's DB rollup would show,
# and the two disagree. See sadpandajoe's finding on #42976.
_ADDITIVE_OPERATORS = frozenset({"sum", "nansum", "count", "count_nonzero"})
def _div_preserving_nan(numerator: DataFrame, denominator: Any, axis: int) -> DataFrame:
"""Divide ``numerator`` by ``denominator``, preserving NaN numerators.
A genuine SQL NULL numerator must stay NaN (rendered blank) rather than
become ``0.0`` matches the client-side #42810 semantics guarding
against measured "0.0%" values for values that should stay blank.
"""
result = numerator.div(denominator, axis=axis)
return result.mask(numerator.isna(), other=float("nan"))
def _apply_percent_transform_to_group(g: DataFrame, mode: str) -> DataFrame:
"""Apply a percent-of-{row,col,total} transform to a single-metric block.
Called both for the whole DataFrame when it holds one metric, and
per-metric-group for MultiIndex / flat-multi-metric pivots. A zero or
NaN denominator produces NaN cells rather than ``Infinity``/``NaN``
from division-by-zero, matching the client's ``if (acc === null)
return null`` guard in ``fractionOf``.
"""
if mode == "percent_row":
row_totals = g.sum(axis=PandasAxis.COLUMN, skipna=True).replace(0, float("nan"))
return _div_preserving_nan(g, row_totals, axis=PandasAxis.ROW)
if mode == "percent_col":
col_totals = g.sum(axis=PandasAxis.ROW, skipna=True).replace(0, float("nan"))
return _div_preserving_nan(g, col_totals, axis=PandasAxis.COLUMN)
# percent_total
grand = g.sum(skipna=True).sum(skipna=True)
if pd.isna(grand) or grand == 0:
return g * float("nan")
return _div_preserving_nan(g, grand, axis=PandasAxis.ROW)
def _apply_show_values_as(df: DataFrame, mode: str) -> DataFrame:
"""Divide each metric cell by the appropriate rollup total.
Mirrors the client-side ``fractionOf`` semantic in
``plugin-chart-pivot-table/src/react-pivottable/utilities.ts:739``:
- ``percent_row``: cell / row-total (sum across the columns axis)
- ``percent_col``: cell / column-total (sum across the rows axis)
- ``percent_total``: cell / grand-total (sum of the metric block)
Per-metric isolation the totals are computed *within each metric* so
one metric's numerator is never divided by another metric's total,
matching the client's ``metricAxis`` handling. This applies to both
shapes ``pivot_table`` can produce:
- **MultiIndex columns** (level 0 = metric): iterate the level-0
groups explicitly. Explicit iteration avoids the deprecated
``df.groupby(level=0, axis=1)`` pattern (removed in pandas 3.x).
- **Flat columns with >1 column** (multi-metric pivot with no
``columns`` groupby each column IS a metric): treat each column
as its own single-column metric block.
- **Flat columns with 1 column** (single-metric pivot with no
``columns`` groupby): the whole block is one metric.
NULL preservation is *structural* only: cells that ``pivot_table``
left as ``NaN`` because the (row, column) group had no input rows
at all stay ``NaN`` in the output. Cells whose input rows all held
SQL NULL values have already collapsed to ``0.0`` inside
``pivot_table`` (pandas ``sum([NaN]) == 0``), so they render as
``0%``, not blank reconstructing "blank vs measured zero" from an
aggregated value is not possible at this layer.
"""
if df.empty:
# Empty pivots can carry an empty ``MultiIndex`` with zero level-0
# groups; iterating and concatenating produces
# ``ValueError: No objects to concatenate``. Nothing to transform.
return df
is_multi_metric_wide = isinstance(df.columns, pd.MultiIndex)
is_flat_multi_metric = not is_multi_metric_wide and df.shape[1] > 1
if is_multi_metric_wide:
# Iterate level-0 groups explicitly (pandas-3-safe).
metrics = df.columns.get_level_values(0).unique()
parts = []
for metric in metrics:
block = df.xs(metric, axis=PandasAxis.COLUMN, level=0, drop_level=False)
parts.append(_apply_percent_transform_to_group(block, mode))
# ``concat`` along columns preserves the MultiIndex; reorder to
# match the original column layout deterministically.
combined = pd.concat(parts, axis=PandasAxis.COLUMN)
return combined[df.columns]
if is_flat_multi_metric:
# Each flat column is its own metric — process independently.
parts = []
for col in df.columns:
block = df[[col]]
parts.append(_apply_percent_transform_to_group(block, mode))
return pd.concat(parts, axis=PandasAxis.COLUMN)[df.columns]
# Flat single-metric block — the whole DataFrame is one metric.
return _apply_percent_transform_to_group(df, mode)
def _restore_dropped_metric_columns(
df: DataFrame,
@@ -188,7 +77,7 @@ def _restore_dropped_metric_columns(
@validate_column_args("index", "columns")
def pivot( # pylint: disable=too-many-arguments # noqa: C901
def pivot( # pylint: disable=too-many-arguments
df: DataFrame,
index: list[str],
aggregates: dict[str, dict[str, Any]],
@@ -199,7 +88,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
combine_value_with_metric: bool = False,
marginal_distributions: Optional[bool] = None,
marginal_distribution_name: Optional[str] = None,
show_values_as: Optional[str] = None,
) -> DataFrame:
"""
Perform a pivot operation on a DataFrame.
@@ -223,13 +111,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
:param marginal_distributions: Add totals for row/column. Default to False
:param marginal_distribution_name: Name of row/column with marginal distribution.
Default to 'All'.
:param show_values_as: Optional post-pivot transform that expresses each
metric cell as a fraction of the row / column / grand total.
One of ``"percent_row"``, ``"percent_col"``, ``"percent_total"`` or
``None`` / ``"actual"`` (no-op, default). Mirrors the pivot chart's
client-side ``fractionOf`` semantic so server-side rendering paths
(CSV / XLSX exports, scheduled reports) can reproduce the browser
output. See #42809.
:return: A pivot table
:raises InvalidPostProcessingError: If the request in incorrect
"""
@@ -242,62 +123,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
_("Pivot operation must include at least one aggregate")
)
# Fail fast on ``show_values_as`` misconfiguration *before* running the
# (potentially expensive) ``pivot_table`` call: an unknown mode should
# not silently perform a full pivot only to raise at the end, and the
# ``marginal_distributions`` combination should reject before pandas
# gets a chance to raise its own margins-related errors. ``None`` /
# ``""`` / ``"actual"`` are the no-op sentinels — anything else must
# be a known percent mode.
percent_mode: Optional[str] = None
if show_values_as not in (None, "", "actual"):
if show_values_as not in _PERCENT_MODES:
raise InvalidPostProcessingError(
_(
"Unsupported show_values_as value: %(mode)s. "
"Expected one of: percent_row, percent_col, percent_total, actual.",
mode=show_values_as,
)
)
if marginal_distributions:
# The pivot would carry an "All" margin row and/or column;
# summing across the axis would double-count by including the
# margin as part of its own denominator. Combining ``margins``
# with ``show_values_as`` needs a first-class design (probably
# computing percentages on the non-margin subset and then
# re-inserting the margin totals as-is), which is out of scope
# here. Reject explicitly rather than silently returning wrong
# numbers.
raise InvalidPostProcessingError(
_(
"show_values_as is not yet supported when "
"marginal_distributions is enabled."
)
)
# ``show_values_as`` divides each cell by the sum of its axis, so
# it is only meaningful when that sum equals the rollup the DB
# would compute. For non-additive aggregates (mean, median, min,
# max, distinct count, …) the summed per-cell values are not the
# row/column/grand rollup, and the exports would disagree with
# the chart. Reject up front rather than emit numbers that mix
# with the DB rollup incorrectly.
non_additive = [
name
for name, cfg in aggregates.items()
if not isinstance(cfg.get("operator"), str)
or cfg["operator"] not in _ADDITIVE_OPERATORS
]
if non_additive:
raise InvalidPostProcessingError(
_(
"show_values_as is only supported for additive aggregates "
"(sum, count); got non-additive operator(s) for: "
"%(metrics)s.",
metrics=", ".join(non_additive),
)
)
percent_mode = show_values_as
if columns and column_fill_value:
df[columns] = df[columns].fillna(value=column_fill_value)
@@ -338,19 +163,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
elif pivot_key_set and not df.empty:
df = df.drop(df.columns.difference(pivot_key_set), axis=PandasAxis.COLUMN)
# Apply the ``show_values_as`` percent transform BEFORE the
# ``combine_value_with_metric`` reshape, not after. The reshape below
# swaps the column ``MultiIndex`` level order from ``(metric, category)``
# to ``(category, metric)``; if the percent transform runs against the
# post-reshape shape, its per-metric iteration
# (``df.columns.get_level_values(0).unique()``) walks categories thinking
# they are metrics, mixing metrics and producing wrong percentages. See
# sadpandajoe's finding on #42976. Running percent first keeps the
# per-metric-isolation invariant intact; the reshape then runs on the
# already-normalized values without changing them further.
if percent_mode is not None:
df = _apply_show_values_as(df, percent_mode)
if combine_value_with_metric:
# dropna=False preserves restored all-NaN metric rows that would otherwise
# be silently dropped by stack's default dropna=True behavior.
@@ -86,24 +86,16 @@ ALLOWLIST_CUMULATIVE_FUNCTIONS = (
PROPHET_TIME_GRAIN_MAP: dict[str, str] = {
TimeGrain.SECOND: "s",
TimeGrain.FIVE_SECONDS: "5s",
TimeGrain.THIRTY_SECONDS: "30s",
TimeGrain.MINUTE: "min",
TimeGrain.FIVE_MINUTES: "5min",
TimeGrain.TEN_MINUTES: "10min",
TimeGrain.FIFTEEN_MINUTES: "15min",
TimeGrain.THIRTY_MINUTES: "30min",
# An alternate ISO-8601 spelling of THIRTY_MINUTES that a number of engine
# specs expose instead; the two denote the same interval.
TimeGrain.HALF_HOUR: "30min",
TimeGrain.HOUR: "h",
TimeGrain.SIX_HOURS: "6h",
TimeGrain.DAY: "D",
TimeGrain.WEEK: "W",
TimeGrain.MONTH: "ME" if _PANDAS_VERSION >= (2, 2) else "M",
TimeGrain.QUARTER: "QE" if _PANDAS_VERSION >= (2, 2) else "Q",
# An alternate ISO-8601 spelling of QUARTER, as with HALF_HOUR above.
TimeGrain.QUARTER_YEAR: "QE" if _PANDAS_VERSION >= (2, 2) else "Q",
TimeGrain.YEAR: "YE" if _PANDAS_VERSION >= (2, 2) else "A",
TimeGrain.WEEK_STARTING_SUNDAY: "W-SUN",
TimeGrain.WEEK_STARTING_MONDAY: "W-MON",
-77
View File
@@ -72,11 +72,6 @@ class EntityVersionInfo:
version: int | None = None
transaction_id: int | None = None
version_uuid: str | None = None
#: Resolved uuid of the entity itself, carried so callers that need a
#: concurrency token for an entity with no version rows yet don't have to
#: re-run the ``SELECT uuid`` this helper already issued. Not part of the
#: API response.
entity_uuid: UUID | None = None
def _capture_enabled() -> bool:
@@ -128,7 +123,6 @@ def current_entity_version_info(
version=version,
transaction_id=transaction_id,
version_uuid=str(version_uuid) if version_uuid else None,
entity_uuid=entity_uuid,
)
@@ -150,77 +144,6 @@ def current_entity_etag_uuid(
return str(version_uuid) if version_uuid else None
# Sentinel Continuum transaction id for an entity that has no version rows
# yet. Continuum sequences start at 1, so it can never collide with a real
# one, and the derived uuid stops matching the moment the first version row
# lands — which is exactly the transition a concurrency guard must catch.
_UNVERSIONED_TRANSACTION_ID = 0
def unversioned_entity_token(entity_uuid: UUID) -> str:
"""Concurrency token for an entity Continuum hasn't versioned yet."""
return str(VersionDAO.derive_version_uuid(entity_uuid, _UNVERSIONED_TRANSACTION_ID))
def entity_concurrency_token(
model_cls: type[Model],
entity_id: int | None,
entity_uuid: UUID | None,
) -> str | None:
"""Resolve the optimistic-concurrency validator for *entity*.
Differs from :func:`current_entity_etag_uuid` in what it does for an
entity with no version rows: baseline rows are written lazily, on the
first update after the versioning migration, so a never-since-saved
entity has none. Reporting ``None`` there would leave the *first*
concurrent save on every such entity unguarded the exact case a
two-tab race hits on a pristine entity. Those entities get a
deterministic unversioned token instead.
``None`` still means "no validator exists": capture is off, or the
entity is missing.
"""
if entity_id is None or entity_uuid is None or not _capture_enabled():
return None
return current_entity_etag_uuid(
model_cls, entity_id, entity_uuid
) or unversioned_entity_token(entity_uuid)
def lock_entity_for_update(model_cls: type[Model], entity_id: int | None) -> None:
"""Row-lock *entity* so a conditional write's check and its update are atomic.
``If-Match`` is verified against a read taken before the update command
runs. Without a lock two overlapping requests can both read the same live
version, both pass the check, and then commit one after the other,
reintroducing the lost update the check exists to prevent. The lock is
held until the command commits, because both run in the same scoped
session.
Renders no ``FOR UPDATE`` on SQLite, which serialises writers anyway.
"""
try:
# The PUT route declares ``/<pk>`` (a string segment), so a non-numeric
# id must not raise a SQL cast error ahead of the command's 404.
entity_id = int(entity_id) # type: ignore[arg-type]
except (TypeError, ValueError):
return
db.session.execute(
sa.select(model_cls.id).where(model_cls.id == entity_id).with_for_update()
)
def concurrency_token_from(info: EntityVersionInfo) -> str | None:
"""Concurrency token for an already-resolved :class:`EntityVersionInfo`.
Lets a write endpoint reuse the pre-update version lookup it already
made rather than issuing a second one.
"""
if info.entity_uuid is None:
return None
return info.version_uuid or unversioned_entity_token(info.entity_uuid)
# Maps the versioned model class name to the keyword argument
# ``security_manager.raise_for_access`` expects for the per-resource
# gate. Slice → ``chart=``, Dashboard → ``dashboard=``, SqlaTable →
-44
View File
@@ -22,7 +22,6 @@ from typing import TYPE_CHECKING
from uuid import UUID
import sqlalchemy as sa
from flask import request
from flask_appbuilder import Model
from superset.extensions import db
@@ -77,46 +76,3 @@ def set_version_etag_by_uuid(
response,
VersionDAO.current_live_version_uuid(model_cls, entity_id, entity_uuid),
)
class StaleEntityError(Exception):
"""The request's ``If-Match`` doesn't match the entity's live version."""
def _entity_tag(tag: str) -> str:
"""Strip the content-coding suffix ``Flask-Compress`` appends to ETags.
A compressed response legitimately carries a different validator than the
identity one Flask-Compress rewrites ``"<uuid>"`` to ``"<uuid>:zstd"``
(see ``flask_compress``) so a client replaying the ETag it read never
matches the raw version uuid. Version uuids contain no ``:``, so cutting
at the first one recovers the entity identity from either form.
"""
return tag.split(":", 1)[0]
def is_conditional_write() -> bool:
"""Whether the request carries an ``If-Match`` precondition."""
return bool(request.if_match)
def raise_for_stale_write(current_version_uuid: str | None) -> None:
"""Enforce ``If-Match`` on a write request, if the client sent one.
Clients that read an entity's ``ETag`` may replay it as ``If-Match`` on a
subsequent write to get optimistic concurrency: the write is rejected when
the entity moved on in the meantime, instead of silently clobbering
whatever landed in between.
The condition is skipped rather than failing closed when the caller
has no validator to offer (``ENABLE_VERSIONING_CAPTURE`` off). Failing
closed there would block every conditional write on deployments running
without version capture, and those are no worse off than before they sent
the header.
"""
if_match = request.if_match
if not if_match or if_match.star_tag or current_version_uuid is None:
return
live = _entity_tag(str(current_version_uuid))
if not any(_entity_tag(tag) == live for tag in if_match.as_set(True)):
raise StaleEntityError()
+1
View File
@@ -687,6 +687,7 @@ class Superset(BaseSupersetView):
return json_success(json.dumps(sanitize_datasource_data(datasource.data)))
@event_logger.log_this
@has_access
@expose("/language_pack/<lang>/")
def language_pack(self, lang: str) -> FlaskResponse:
# Only allow expected language formats like "en", "pt_BR", etc.
@@ -1,340 +1,295 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from unittest.mock import patch # noqa: F401
import pytest
from flask.ctx import AppContext
from flask_appbuilder.security.sqla.models import User
from sqlalchemy.orm import Session # noqa: F401
from superset import db
from superset.commands.dashboard.exceptions import (
DashboardAccessDeniedError, # noqa: F401
)
from superset.commands.temporary_cache.entry import Entry
from superset.extensions import cache_manager
from superset.models.dashboard import Dashboard
from superset.temporary_cache.utils import cache_key
from superset.utils import json
from tests.integration_tests.fixtures.world_bank_dashboard import (
load_world_bank_dashboard_with_slices, # noqa: F401
load_world_bank_data, # noqa: F401
)
from tests.integration_tests.test_app import app # noqa: F401
KEY = "test-key"
INITIAL_VALUE = json.dumps({"test": "initial value"})
UPDATED_VALUE = json.dumps({"test": "updated value"})
@pytest.fixture
def dashboard_id(app_context: AppContext, load_world_bank_dashboard_with_slices) -> int: # noqa: F811
dashboard = db.session.query(Dashboard).filter_by(slug="world_health").one()
return dashboard.id
@pytest.fixture
def admin_id(app_context: AppContext) -> int:
admin = db.session.query(User).filter_by(username="admin").one_or_none()
return admin.id
@pytest.fixture(autouse=True)
def cache(dashboard_id, admin_id):
entry: Entry = {"owner": admin_id, "value": INITIAL_VALUE}
cache_manager.filter_state_cache.set(cache_key(dashboard_id, KEY), entry)
def test_post(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": INITIAL_VALUE,
},
)
assert resp.status_code == 201
def test_post_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_post_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": "foo",
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 400
def test_post_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 404
def test_post_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_post_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_post_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 200
def test_put_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_put_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_put_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": "foo",
},
)
assert resp.status_code == 400
def test_put_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 404
@patch("superset.commands.dashboard.filter_state.create.check_access")
def test_post_authenticated_user_with_access(
mock_check_access, test_client, login_as, dashboard_id: int
):
login_as("alpha")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 201
mock_check_access.assert_called_once_with(dashboard_id)
@patch("superset.commands.dashboard.filter_state.create.check_access")
@patch("superset.commands.dashboard.filter_state.update.check_access")
def test_put_authenticated_user_with_access(
mock_update_check_access,
mock_create_check_access,
test_client,
login_as,
dashboard_id: int,
):
login_as("alpha")
payload = {
"value": INITIAL_VALUE,
}
post_resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert post_resp.status_code == 201
key = json.loads(post_resp.data.decode("utf-8"))["key"]
put_payload = {
"value": UPDATED_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{key}", json=put_payload
)
assert resp.status_code == 200
mock_create_check_access.assert_called_once_with(dashboard_id)
mock_update_check_access.assert_called_once_with(dashboard_id)
def test_get_key_not_found(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/unknown-key/")
assert resp.status_code == 404
def test_get_dashboard_not_found(test_client, login_as_admin):
resp = test_client.get(f"api/v1/dashboard/{-1}/filter_state/{KEY}")
assert resp.status_code == 404
def test_get_dashboard_filter_state(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
data = json.loads(resp.data.decode("utf-8"))
assert INITIAL_VALUE == data.get("value")
def test_get_access_denied(test_client, login_as, dashboard_id):
login_as("gamma")
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete(test_client, login_as_admin, dashboard_id: int):
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
def test_delete_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete_not_owner(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from unittest.mock import patch # noqa: F401
import pytest
from flask.ctx import AppContext
from flask_appbuilder.security.sqla.models import User
from sqlalchemy.orm import Session # noqa: F401
from superset import db
from superset.commands.dashboard.exceptions import (
DashboardAccessDeniedError, # noqa: F401
)
from superset.commands.temporary_cache.entry import Entry
from superset.extensions import cache_manager
from superset.models.dashboard import Dashboard
from superset.temporary_cache.utils import cache_key
from superset.utils import json
from tests.integration_tests.fixtures.world_bank_dashboard import (
load_world_bank_dashboard_with_slices, # noqa: F401
load_world_bank_data, # noqa: F401
)
from tests.integration_tests.test_app import app # noqa: F401
KEY = "test-key"
INITIAL_VALUE = json.dumps({"test": "initial value"})
UPDATED_VALUE = json.dumps({"test": "updated value"})
@pytest.fixture
def dashboard_id(app_context: AppContext, load_world_bank_dashboard_with_slices) -> int: # noqa: F811
dashboard = db.session.query(Dashboard).filter_by(slug="world_health").one()
return dashboard.id
@pytest.fixture
def admin_id(app_context: AppContext) -> int:
admin = db.session.query(User).filter_by(username="admin").one_or_none()
return admin.id
@pytest.fixture(autouse=True)
def cache(dashboard_id, admin_id):
entry: Entry = {"owner": admin_id, "value": INITIAL_VALUE}
cache_manager.filter_state_cache.set(cache_key(dashboard_id, KEY), entry)
def test_post(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": INITIAL_VALUE,
},
)
assert resp.status_code == 201
def test_post_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_post_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": "foo",
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 400
def test_post_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 404
def test_post_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_post_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_post_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 200
def test_put_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_put_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_put_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": "foo",
},
)
assert resp.status_code == 400
def test_put_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 404
def test_get_key_not_found(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/unknown-key/")
assert resp.status_code == 404
def test_get_dashboard_not_found(test_client, login_as_admin):
resp = test_client.get(f"api/v1/dashboard/{-1}/filter_state/{KEY}")
assert resp.status_code == 404
def test_get_dashboard_filter_state(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
data = json.loads(resp.data.decode("utf-8"))
assert INITIAL_VALUE == data.get("value")
def test_get_access_denied(test_client, login_as, dashboard_id):
login_as("gamma")
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete(test_client, login_as_admin, dashboard_id: int):
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
def test_delete_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete_not_owner(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
@@ -2487,15 +2487,8 @@ class TestDatabaseApi(SupersetTestCase):
}
assert response == expected_response
# Uses a `dialect+driver://` URI (rather than the bare `broken://`
# above) to also cover engine-name extraction stripping the driver
# suffix. The dialect itself ("broken") must stay one that no
# installed extra ever registers a real SQLAlchemy plugin for --
# this PR's own testcontainers extras (mssql, oracle, db2, ...)
# install real drivers for those dialects, which would make this
# URI actually attempt a connection instead of failing to load.
data = {
"sqlalchemy_uri": "broken+driver://url",
"sqlalchemy_uri": "mssql+pymssql://url",
"database_name": "examples",
"impersonate_user": False,
"server_cert": None,
@@ -2507,7 +2500,7 @@ class TestDatabaseApi(SupersetTestCase):
expected_response = {
"errors": [
{
"message": "Could not load database driver for: broken",
"message": "Could not load database driver for: mssql",
"error_type": "GENERIC_COMMAND_ERROR",
"level": "warning",
"extra": {
@@ -1750,10 +1750,6 @@ class TestRolePermission(SupersetTestCase):
# user/tenant data) as content-addressed scripts; must load for
# anonymous principals (login page, embedded dashboards).
["Superset", "language_pack_script"],
# Language pack endpoint serves JS bundle translations, no auth
# needed; embedded dashboards fetch this without a guest-token
# header, so it must be reachable unauthenticated.
["Superset", "language_pack"],
]
unsecured_views = []
for view_class in appbuilder.baseviews:
-16
View File
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
@@ -1,47 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Shared import guard for the per-dialect testcontainers modules
(tests/testcontainers/db_engine_specs/test_*.py), each of which needs its
own optional `testcontainers[...]` driver submodule to even import.
"""
import importlib
import os
def require_driver(module_name: str) -> None:
"""
Import `module_name`, a dialect's `testcontainers` driver submodule.
Most environments treat that driver as optional: a bare local `pytest`
run, or another CI job that never installed the `testcontainers` extras,
should skip the module rather than fail collection outright.
The dedicated per-dialect CI job (.github/workflows/testcontainers.yml)
sets SUPERSET_TESTCONTAINERS_STRICT, because there the driver is not
optional -- that job's matrix installs exactly this one driver for
exactly this one module. A broken or missing import there means the job
is misconfigured, and should fail loudly instead of silently reporting
a misleadingly green, zero-tests-run result.
"""
if os.environ.get("SUPERSET_TESTCONTAINERS_STRICT"):
importlib.import_module(module_name)
else:
import pytest
pytest.importorskip(module_name)
@@ -1,68 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Shared body for the "paginated query returns correct rows in order" test
that db_engine_specs.{cockroachdb,crate,db2,mssql,oracle,trino}'s
testcontainers suites each run against their own real instance: a plain
SQLAlchemy Core LIMIT/OFFSET query, compiled and executed for real. Mocked
tests cannot catch a dialect compiling this incorrectly (see
apache/superset#42899, where Trino emitted OFFSET before LIMIT) -- only
real execution can.
Each call site keeps its own test function (and dialect-specific docstring)
so failures still report against the right module; this only factors out
the identical table setup/assert body, via an optional post-insert hook for
dialects (CrateDB) that need one, and an optional extra-table-args hook for
dialects (ClickHouse) whose CREATE TABLE requires a schema item a plain
Column/primary key can't express.
"""
from collections.abc import Callable
from typing import Any
from sqlalchemy import Column, insert, Integer, MetaData, select, Table as SATable
from sqlalchemy.engine import Connection, Engine
def assert_paginated_query_returns_correct_rows_in_order(
engine: Engine,
after_insert: Callable[[Connection], None] | None = None,
extra_table_args: tuple[Any, ...] = (),
) -> None:
metadata = MetaData()
t = SATable(
"pilot_pagination",
metadata,
# autoincrement=False: a single-column integer primary key otherwise
# implicitly becomes AUTO_INCREMENT on MySQL/MariaDB. That column
# type treats an explicit 0 as NULL by default (NO_AUTO_VALUE_ON_ZERO
# is off), so the id=0 row below would silently get auto-assigned 1,
# colliding with the explicit id=1 row in the same batch insert.
Column("id", Integer, primary_key=True, autoincrement=False),
*extra_table_args,
)
metadata.create_all(engine)
with engine.begin() as conn:
conn.execute(insert(t), [{"id": i} for i in range(10)])
if after_insert is not None:
after_insert(conn)
with engine.connect() as conn:
stmt = select(t.c.id).order_by(t.c.id).limit(3).offset(4)
rows = conn.execute(stmt).fetchall()
assert [row.id for row in rows] == [4, 5, 6]
@@ -1,130 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.clickhouse against a real ClickHouse instance, spun
up on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Superset's recommended ClickHouse connector is `clickhouse-connect`
(`ClickHouseConnectEngineSpec`, engine "clickhousedb"), which talks HTTP,
not `ClickHouseContainer`'s own documented `clickhouse_driver` (a different,
native-TCP-protocol package Superset doesn't use at all). The container
exposes both the native TCP port (9000) and the HTTP port (8123); this test
connects over the HTTP port to match Superset's actual driver.
Unlike every other dialect in this suite, ClickHouse tables have no real
primary key/constraint concept -- CREATE TABLE requires an explicit engine
(e.g. MergeTree), or clickhouse-connect's DDL compiler raises a CompileError
rather than defaulting to one.
`superset.db_engine_specs.clickhouse` runs module-level setup code (default
type-formatting overrides) that dereferences `current_app.config` whenever
clickhouse-connect is installed, so importing it outside a Flask app context
raises RuntimeError the first time it's imported in a process.
`tests/unit_tests/db_engine_specs/test_clickhouse.py` gets an app context
for free from that suite's autouse fixture; this suite has no such fixture,
so this test pushes one explicitly around just that one-time import, reusing
the real app instance `tests/conftest.py` already builds for the rest of the
test run rather than constructing a second one.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.clickhouse")
require_driver("clickhouse_connect")
from clickhouse_connect.cc_sqlalchemy.ddl.tableengine import MergeTree # noqa: E402
from testcontainers.community.clickhouse import ClickHouseContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
HTTP_PORT = 8123
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with ClickHouseContainer("clickhouse/clickhouse-server:latest") as container:
host = container.get_container_host_ip()
port = container.get_exposed_port(HTTP_PORT)
yield create_engine(
f"clickhousedb://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(
engine, extra_table_args=(MergeTree(order_by="id"),)
)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
ClickHouseConnectEngineSpec.get_columns wraps a real SQLAlchemy
Inspector; this exercises that against actual server-reported column
metadata rather than a mocked Inspector.
"""
from tests.integration_tests.test_app import app
with app.app_context():
from superset.db_engine_specs.clickhouse import ClickHouseConnectEngineSpec
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
MergeTree(order_by="id"),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = ClickHouseConnectEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = ClickHouseConnectEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,97 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.cockroachdb against a real CockroachDB instance,
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml -- these exercise real SQL execution
and dialect introspection, which mocked unit tests structurally cannot.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.cockroachdb import CockroachDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.cockroachdb")
from testcontainers.community.cockroachdb import CockroachDBContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
# sqlalchemy-cockroachdb registers its dialect under the plain
# "cockroachdb" name; the container's own default ("cockroachdb+psycopg2")
# matches the abandoned `cockroachdb` package instead (see #43501).
with CockroachDBContainer(dialect="cockroachdb") as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
CockroachDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported column metadata
rather than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = CockroachDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = CockroachDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,103 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.crate against a real CrateDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
crate/crate only publishes an amd64 image (no arm64 build), and requires a
host CPU supporting the x86-64-v3 instruction set -- QEMU emulation on
Apple Silicon cannot satisfy that, so this file cannot run locally on an
Apple Silicon machine even with `docker pull --platform linux/amd64`. It
runs natively on GitHub Actions' x86_64 runners.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Connection, Engine
from superset.db_engine_specs.crate import CrateEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.cratedb")
from testcontainers.community.cratedb import CrateDBContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with CrateDBContainer() as container:
yield create_engine(container.get_connection_url())
def _refresh_pilot_pagination(conn: Connection) -> None:
# CrateDB is eventually consistent: a row is not guaranteed visible to
# subsequent selects immediately after insert.
conn.exec_driver_sql("REFRESH TABLE pilot_pagination")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(
engine, after_insert=_refresh_pilot_pagination
)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
CrateEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = CrateEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = CrateEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,114 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.databend against a real Databend instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Databend has no dedicated testcontainers module, so this uses a generic
DockerContainer against the official `datafuselabs/databend` standalone
image. Superset's DatabendEngineSpec defaults to `sslmode=require`
(`encryption_parameters`), but the local standalone image has no TLS
listener, so this connects with `sslmode=disable` explicitly.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.databend import DatabendEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("databend_sqlalchemy")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
HTTP_PORT = 8000
DBNAME = "default"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("datafuselabs/databend")
container.with_exposed_ports(HTTP_PORT)
# The image's own startup banner documents this exact line as proof its
# HTTP query endpoint is bound and ready.
container.waiting_for(LogMessageWaitStrategy(f"listened at 0.0.0.0:{HTTP_PORT}"))
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(HTTP_PORT)
# "root" with no password is the image's builtin user -- confirmed
# directly against a running container, not from the image's own
# doc text, which only shows ${USER}/${PASSWORD} placeholders.
yield create_engine(f"databend://root:@{host}:{port}/{DBNAME}?sslmode=disable")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
DatabendEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = DatabendEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = DatabendEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,95 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.db2 against a real IBM Db2 instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
icr.io/db2_community/db2 only publishes amd64/ppc64le/s390x images (no
arm64 build), so this cannot run locally on an Apple Silicon machine. It
runs natively on GitHub Actions' x86_64 runners. Db2 is also a notably slow
starter (a full instance bring-up, not just a process start) -- expect this
module alone to take several minutes.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.db2 import Db2EngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.db2")
from testcontainers.community.db2 import Db2Container # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with Db2Container() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
Db2EngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = Db2EngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = Db2EngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,107 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.elasticsearch against a real Elasticsearch instance,
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml.
Unlike the SQL-native dialects in this directory, Elasticsearch has no
CREATE TABLE / INSERT: indices and documents get created via its REST API
(elasticsearch-dbapi's SQLAlchemy dialect is read-focused, translating SQL
to the _sql endpoint), matching how Superset actually encounters
Elasticsearch in practice -- data arrives via ingestion tooling, not
through Superset itself.
"""
from collections.abc import Iterator
import pytest
import requests
from sqlalchemy import create_engine, inspect, text
from sqlalchemy.engine import Engine
from superset.db_engine_specs.elasticsearch import ElasticSearchEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.elasticsearch")
from testcontainers.community.elasticsearch import ElasticSearchContainer # noqa: E402
INDEX = "pilot_pagination"
def _index_document(
base_url: str, index: str, doc_id: int, body: dict[str, int]
) -> None:
response = requests.put(f"{base_url}/{index}/_doc/{doc_id}", json=body, timeout=10)
response.raise_for_status()
def _refresh(base_url: str, index: str) -> None:
response = requests.post(f"{base_url}/{index}/_refresh", timeout=10)
response.raise_for_status()
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with ElasticSearchContainer("elasticsearch:8.11.0") as container:
host = container.get_container_host_ip()
port = container.get_exposed_port(container.port)
base_url = f"http://{host}:{port}"
for i in range(10):
_index_document(base_url, INDEX, i, {"id": i})
_refresh(base_url, INDEX)
yield create_engine(f"elasticsearch+http://{host}:{port}/")
def test_ordered_limited_query_returns_correct_rows(engine: Engine) -> None:
"""
A plain LIMIT query, compiled and executed against a real instance.
Mocked tests cannot catch a dialect compiling this incorrectly (see
apache/superset#42899, where Trino emitted OFFSET before LIMIT) -- only
real execution can. No OFFSET here: Elasticsearch's SQL layer genuinely
doesn't support it (a protocol limitation, not a bug -- confirmed
against a real instance, which raises a parsing_exception on OFFSET).
ElasticSearchEngineSpec.supports_offset = False documents this already.
"""
with engine.connect() as conn:
rows = conn.execute(
text(f"SELECT id FROM {INDEX} ORDER BY id LIMIT 3") # noqa: S608
).fetchall()
assert [row.id for row in rows] == [0, 1, 2]
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
ElasticSearchEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported field mappings
rather than a mocked Inspector.
"""
inspector = inspect(engine)
columns = ElasticSearchEngineSpec.get_columns(inspector, Table(INDEX))
by_name = {col["column_name"]: col for col in columns}
assert "id" in by_name
spec = ElasticSearchEngineSpec.get_column_spec(str(by_name["id"]["type"]))
assert spec is not None
@@ -1,146 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.firebird against a real Firebird instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Firebird references a database *file* rather than a server-managed named
database -- the connection URI is `firebird://user:pass@host:port/<path>`,
where <path> is the path to a .fdb file on the server. This uses the
well-known `jacobalberty/firebird` image, which creates that file (per
FIREBIRD_DATABASE) under /firebird/data on first boot.
FirebirdEngineSpec sets `limit_method = LimitMethod.FETCH_MANY` with a
comment claiming Firebird "uses FIRST to limit" -- stale relative to the
modern sqlalchemy-firebird driver, which compiles real ROWS-based
pagination (confirmed via an offline dialect compile: `SELECT ... ROWS
4 + 1 TO 4 + 3`, correctly ordered, not a Trino-style bug). That staleness
affects what Superset's own query layer emits, not what this suite's
direct dialect-compilation check exercises.
Could not be verified against a real running instance in this
environment: `firebird-driver` is a pure-Python ctypes wrapper (its wheel
is `py3-none-any`, confirmed by downloading it directly) that dynamically
loads the native Firebird client library (`libfbclient`) from the host at
import time -- it doesn't bundle that library itself. This machine has no
Homebrew formula or straightforward install path for it. The container
itself was confirmed to start and pass its own healthcheck; CI installs
the `libfbclient2` system package separately (see
.github/workflows/testcontainers.yml) for the actual client-library
dependency this driver needs.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.firebird import FirebirdEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("firebird.driver")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import HealthcheckWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 3050
PASSWORD = "masterkey" # noqa: S105 -- fixed test-fixture password, not a secret
DB_FILE = "test.fdb"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("jacobalberty/firebird")
container.with_exposed_ports(PORT)
container.with_env("ISC_PASSWORD", PASSWORD)
container.with_env("FIREBIRD_DATABASE", DB_FILE)
# The image logs nothing beyond a single startup banner line and never
# prints a distinct "ready" message -- it ships its own Docker
# HEALTHCHECK instead, confirmed via `docker ps` reporting (healthy).
container.waiting_for(HealthcheckWaitStrategy())
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
eng = create_engine(
f"firebird://sysdba:{PASSWORD}@{host}:{port}//firebird/data/{DB_FILE}"
)
yield eng
# firebird-driver registers its own atexit handler that talks to
# the Firebird subsystem to shut it down cleanly. Without disposing
# here first, that handler fires at interpreter exit against a
# server the container has *already* torn down -- confirmed on
# real CI as a segfault (exit code 139) after both tests had
# already passed. Disposing while the server is still up lets the
# driver close out normally, so the later atexit call has nothing
# left to talk to.
eng.dispose()
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
FirebirdEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = FirebirdEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = FirebirdEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,117 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mariadb against a real MariaDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
MariaDB is a MySQL fork implementing the same wire protocol: connects via
the plain "mysql" dialect with mysqlclient, same as vanilla MySQL, just
pointed at the mariadb image instead of mysql:latest.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient. CI installs it via apt on
Linux, where this does not occur.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mariadb import MariaDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mysql")
from testcontainers.community.mysql import MySqlContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MySqlContainer("mariadb:11") as container:
# get_connection_url() has no host override and defaults to
# get_container_host_ip(), which is the literal string "localhost"
# on native Linux Docker (e.g. GitHub Actions runners). MySQLdb
# (mysqlclient) treats a "localhost" host specially and attempts a
# Unix socket connection instead of TCP, which fails since there's
# no local MySQL socket -- the container is reached over the
# network. Only rewrite that specific local case to 127.0.0.1; a
# remote Docker daemon reports its own real host/IP here, which
# must be preserved so the suite can still reach it.
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(container.port)
yield create_engine(
f"mysql://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MariaDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MariaDBEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MariaDBEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,127 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.monetdb against a real MonetDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
monetdb/monetdb publishes an amd64-only image, so this needs Rosetta/QEMU
emulation on Apple Silicon -- unlike CrateDB's x86-64-v3 CPU requirement,
this one actually runs fine under emulation (verified locally). No native
testcontainers module exists for MonetDB, so this uses a generic
DockerContainer with the documented MDB_* environment variables and waits
for the daemon's own startup log line.
"""
import re
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.monetdb import MonetDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("sqlalchemy_monetdb")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import ( # noqa: E402
CompositeWaitStrategy,
LogMessageWaitStrategy,
PortWaitStrategy,
)
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 50000
PASSWORD = "monetdb" # noqa: S105 -- fixed test-fixture password, not a secret
DBNAME = "test"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("monetdb/monetdb:latest")
container.with_exposed_ports(PORT)
container.with_env("MDB_DB_ADMIN_PASS", PASSWORD)
container.with_env("MDB_CREATE_DBS", DBNAME)
# The "Starting MonetDB daemon" log line is emitted before the image
# actually runs `monetdbd start -n`, so it alone isn't proof the server
# is accepting connections yet. Follow it with a port-connect check,
# which only succeeds once monetdbd is really listening.
container.waiting_for(
CompositeWaitStrategy(
LogMessageWaitStrategy(re.compile("Starting MonetDB daemon")),
PortWaitStrategy(PORT),
)
)
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
yield create_engine(f"monetdb://monetdb:{PASSWORD}@{host}:{port}/{DBNAME}")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MonetDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MonetDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MonetDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,141 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mongodb against a real MongoDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
MongoDB is schemaless, and Superset talks to it via `pymongosql`, a
SQL-to-MongoDB translation layer (dialect requires a `?mode=superset` query
param -- not part of testcontainers' own MongoDbContainer.get_connection_url()).
Documents get inserted via the native pymongo driver, not SQL INSERT,
matching how Superset actually encounters MongoDB in practice and avoiding
any assumption about pymongosql's own INSERT/DDL support.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
column,
create_engine,
inspect,
Integer,
select,
table,
text,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mongodb import MongoDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mongodb")
require_driver("pymongosql")
from testcontainers.community.mongodb import MongoDbContainer # noqa: E402
COLLECTION = "pilot_pagination"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MongoDbContainer("mongo:7.0.7") as container:
client = container.get_connection_client()
client[container.dbname][COLLECTION].insert_many([{"id": i} for i in range(10)])
# MongoDbContainer.get_connection_url() has no database path segment
# or query string at all (it only builds user:pass@host:port), so
# naively appending "&mode=superset" glues it straight onto the port
# number instead of starting a query string. Build the full URL
# ourselves instead of relying on string concatenation.
host = container.get_container_host_ip()
port = container.get_exposed_port(container.port)
# authSource=admin is required: MongoDbContainer creates its root
# user via MONGO_INITDB_ROOT_USERNAME, which lives in the `admin`
# database, not in `dbname` -- without it, auth fails against
# whatever database is in the URL path.
yield create_engine(
f"mongodb://{container.username}:{container.password}@{host}:{port}"
f"/{container.dbname}?mode=superset&authSource=admin"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed
against a real instance. Mocked tests cannot catch a dialect compiling
this incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can. Unlike Elasticsearch's SQL
layer (which has no OFFSET support at all), pymongosql maps OFFSET to
MongoDB's native `skip`, so this dialect supports it.
Compiled with `literal_binds=True`, matching how Superset actually
issues chart/SQL Lab queries (see `models/helpers.py`'s
`get_query_str_extended`): pymongosql's SQL-to-Mongo AST parser reads
LIMIT/OFFSET straight off the compiled SQL text ahead of parameter
substitution, so a bound `LIMIT ?`/`OFFSET ?` placeholder is rejected
("invalid literal for int() with base 10: '?'") and the clause is
silently dropped -- unlike its WHERE-clause parameter handling, which
does substitute correctly. Literal binds sidestep that and exercise
the dialect's actual LIMIT/OFFSET compilation, per this test's intent.
Uses a bare `column("id")`/`table(...)` pair rather than a full
`Table`-bound column: SQLAlchemy always qualifies a Table-bound column
reference as `pilot_pagination.id` once there's a FROM clause, and
pymongosql's projection builder takes that qualified text completely
literally as a MongoDB field path -- `{"pilot_pagination.id": 1}` reads
a *nested* field under a top-level `pilot_pagination` key, which
doesn't exist on these flat documents, silently projecting None instead
of raising. An unbound column compiles unqualified ("id"), which
resolves correctly, while still exercising the dialect's own
LIMIT/OFFSET compilation via a real Core `select()`.
"""
id_col = column("id")
stmt = (
select(id_col)
.select_from(table(COLLECTION))
.order_by(id_col)
.limit(3)
.offset(4)
)
compiled = stmt.compile(engine, compile_kwargs={"literal_binds": True})
with engine.connect() as conn:
rows = conn.execute(text(str(compiled))).fetchall()
assert [row.id for row in rows] == [4, 5, 6]
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MongoDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector, which
pymongosql implements by sampling real documents to infer column types
-- this exercises that against an actual running instance rather than
a mocked Inspector.
"""
inspector = inspect(engine)
columns = MongoDBEngineSpec.get_columns(inspector, Table(COLLECTION))
by_name = {col["column_name"]: col for col in columns}
assert "id" in by_name
spec = MongoDBEngineSpec.get_column_spec(str(by_name["id"]["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,93 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mssql against a real SQL Server instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
mcr.microsoft.com/mssql/server only publishes an amd64 image (SQL Server on
Linux has no ARM build), so this cannot run locally on an Apple Silicon
machine. It runs natively on GitHub Actions' x86_64 runners.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mssql import MssqlEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mssql")
from testcontainers.community.mssql import SqlServerContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with SqlServerContainer() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MssqlEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MssqlEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MssqlEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,117 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mysql against a real MySQL instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Plain MySQL itself was never covered by this suite: MariaDB and StarRocks
both reuse `MySQLEngineSpec`'s plain "mysql" dialect via mysqlclient, but
neither stands in for vanilla MySQL server's own dialect quirks.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient. CI installs it via apt on
Linux, where this does not occur.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mysql import MySQLEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mysql")
from testcontainers.community.mysql import MySqlContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MySqlContainer("mysql:8.0") as container:
# get_connection_url() has no host override and defaults to
# get_container_host_ip(), which is the literal string "localhost"
# on native Linux Docker (e.g. GitHub Actions runners). MySQLdb
# (mysqlclient) treats a "localhost" host specially and attempts a
# Unix socket connection instead of TCP, which fails since there's
# no local MySQL socket -- the container is reached over the
# network. Only rewrite that specific local case to 127.0.0.1; a
# remote Docker daemon reports its own real host/IP here, which
# must be preserved so the suite can still reach it.
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(container.port)
yield create_engine(
f"mysql://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MySQLEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MySQLEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MySQLEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,221 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.oceanbase against a real OceanBase instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml,
on the nightly cron / manual dispatch only (see `nightly_only: true` on
this dialect's matrix entry) -- OceanBase bootstraps a distributed-style
cluster even in single-node MODE=MINI, a substantially heavier first-boot
than a single-process database, not a good fit for every PR's CI budget.
OceanBaseEngineSpec extends MySQLEngineSpec and its dialect
(oceanbase_py.sqlalchemy.dialect.OceanBaseDialect) extends
MySQLDialect_mysqldb directly with no custom DDL or LIMIT/OFFSET compiler,
so this follows the same mysqlclient-based pattern as MariaDB/MySQL/
StarRocks in this suite -- including the same "localhost" -> "127.0.0.1"
fix MySQLdb needs on native Linux Docker.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient, and this dialect wasn't
pulled/run locally at all given its heavier resource footprint -- CI-only
verification, matching the nightly_only gating.
oceanbase_py.sqlalchemy.dialect.OceanBaseDialect has real bugs, all
confirmed on real CI, in both has_table() (called by create_all()'s
default checkfirst=True) and get_columns() (called by
OceanBaseEngineSpec.get_columns(), which this suite's second test needs
to actually exercise):
1. Both pass a raw string straight to Connection.execute() (e.g.
`connection.execute(f"DESCRIBE {full_name}")`), which SQLAlchemy 2.0
rejects outright (ObjectNotExecutableError). Every *other* raw-SQL
method in the same dialect module correctly uses
`connection.exec_driver_sql(...)` instead.
2. has_table() never catches the error DESCRIBE raises for a table that
doesn't exist (1146) -- so even with (1) fixed, it can only ever
return True, raising instead of returning False for exactly the case
checkfirst exists to handle.
This test monkeypatches both methods to do what the rest of the dialect's
raw-SQL methods already do, plus has_table()'s missing not-found
handling, rather than working around any of this from the test side.
"""
from collections.abc import Iterator
from typing import Any
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Connection, Engine, URL
from sqlalchemy.exc import NoSuchTableError, ProgrammingError
from superset.db_engine_specs.oceanbase import OceanBaseEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("oceanbase_py")
from oceanbase_py.sqlalchemy import datatype # noqa: E402
from oceanbase_py.sqlalchemy.dialect import OceanBaseDialect # noqa: E402
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
def _has_table(
self: OceanBaseDialect,
connection: Connection,
table_name: str,
schema: str | None = None,
**kw: object,
) -> bool:
if schema is None:
schema = self.default_schema_name
quote = self.identifier_preparer.quote_identifier
full_name = quote(table_name)
if schema:
full_name = f"{quote(schema)}.{full_name}"
try:
res = connection.exec_driver_sql(f"DESCRIBE {full_name}")
except ProgrammingError:
# The original never catches this at all -- DESCRIBE on a
# nonexistent table raises 1146 ("table doesn't exist") rather
# than returning an empty result set, so the unpatched method can
# only ever return True, and raises instead of returning False for
# exactly the case create_all()'s checkfirst exists to handle
# (confirmed on real CI).
return False
return res.first() is not None
def _get_columns(
self: OceanBaseDialect,
connection: Connection,
table_name: str,
schema: str | None = None,
**kw: object,
) -> list[dict[str, Any]]:
# Same connection.execute(raw string) anti-pattern as has_table(),
# confirmed on real CI as the same ObjectNotExecutableError -- this is
# the actual column-introspection call OceanBaseEngineSpec.get_columns
# (and this test) needs, so it gets the same exec_driver_sql fix.
if not self.has_table(connection, table_name, schema):
raise NoSuchTableError(f"schema={schema}, table={table_name}")
schema = schema or self._get_default_schema_name(connection)
quote = self.identifier_preparer.quote_identifier
full_name = quote(table_name)
if schema:
full_name = f"{quote(schema)}.{full_name}"
res = connection.exec_driver_sql(f"SHOW COLUMNS FROM {full_name}")
return [
{
"name": record.Field,
"type": datatype.parse_sql_type(record.Type),
"nullable": record.Null == "YES",
"default": record.Default,
}
for record in res
]
OceanBaseDialect.has_table = _has_table
OceanBaseDialect.get_columns = _get_columns
PORT = 2881
PASSWORD = "pilot" # noqa: S105 -- fixed test-fixture password, not a secret
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("oceanbase/oceanbase-ce")
container.with_exposed_ports(PORT)
container.with_env("MODE", "MINI")
container.with_env("OB_TENANT_PASSWORD", PASSWORD)
container.waiting_for(LogMessageWaitStrategy("boot success!"))
with container:
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(PORT)
# OceanBase usernames for a MySQL-mode tenant use "user@tenant"
# (e.g. "root@test"), a literal "@" that URL.create() percent-encodes
# correctly -- an f-string would produce a second "@" that breaks
# the URL's own host/user boundary parsing.
yield create_engine(
URL.create(
"oceanbase",
username="root@test",
password=PASSWORD,
host=host,
port=int(port),
database="test",
)
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
OceanBaseEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = OceanBaseEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = OceanBaseEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,95 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.oracle against a real Oracle instance (gvenzl/oracle-free),
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml.
gvenzl/oracle-free ships with its datafiles pre-baked into the image, so
once the (large-ish, ~1GB) image is pulled, container startup is fast --
under 15s measured locally. Almost all the wall-clock cost here is the
image pull itself, same as any other dialect's container.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.oracle import OracleEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.oracle")
from testcontainers.community.oracle import OracleDbContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with OracleDbContainer() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
OracleEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = OracleEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = OracleEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,99 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.postgres against a real PostgreSQL instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml
-- these exercise real SQL execution and dialect introspection, which
mocked unit tests structurally cannot.
Plain Postgres itself was never covered by this suite: CockroachDB,
TimescaleDB and YugabyteDB all speak the Postgres wire protocol and already
exercise `PostgresContainer`/the "postgresql" dialect, but none of them
stand in for vanilla PostgreSQL's own dialect quirks.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.postgres import PostgresEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.postgres")
from testcontainers.community.postgres import PostgresContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with PostgresContainer("postgres:17-alpine") as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
PostgresEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = PostgresEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = PostgresEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,127 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.risingwave against a real RisingWave instance, spun
up on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
RisingWave speaks the Postgres wire protocol, but doesn't run the real
Postgres server binary or its POSTGRES_PASSWORD-style bootstrap env vars,
so this can't reuse `PostgresContainer` the way TimescaleDB/YugabyteDB do
-- it needs a generic DockerContainer against the official
`risingwavelabs/risingwave` single-binary playground image instead.
`RisingWaveDbEngineSpec` extends `PostgresEngineSpec`, and
`sqlalchemy-risingwave`'s dialect is a genuine subclass of SQLAlchemy's own
Postgres dialect (via psycopg2), so DDL/pagination compile with standard
Postgres semantics -- no ClickHouse-style mandatory table option needed.
"""
import re
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
text,
)
from sqlalchemy.engine import Connection, Engine
from superset.db_engine_specs.risingwave import RisingWaveDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("sqlalchemy_risingwave")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 4566
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("risingwavelabs/risingwave")
container.with_exposed_ports(PORT)
container.with_command("playground")
# The actual startup banner reads "RisingWave standalone mode is
# ready." -- confirmed against a real container's logs.
container.waiting_for(
LogMessageWaitStrategy(re.compile("RisingWave standalone mode is ready"))
)
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
yield create_engine(f"risingwave://root@{host}:{port}/dev")
def _flush(conn: Connection) -> None:
# RisingWave's storage engine checkpoints asynchronously: without an
# explicit FLUSH, a SELECT immediately after INSERT can see zero rows
# -- confirmed against a real instance (a bare INSERT commits fine, but
# the data isn't visible to a subsequent query until flushed).
conn.execute(text("FLUSH"))
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine, after_insert=_flush)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
RisingWaveDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported column metadata
rather than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = RisingWaveDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = RisingWaveDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,156 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.starrocks against a real StarRocks instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
StarRocks has no dedicated testcontainers module, so this uses a generic
DockerContainer against the official `starrocks/allin1-ubuntu` image, which
brings up both the FE (query frontend, MySQL wire protocol on port 9030)
and BE (execution backend) in a single container -- a heavier bring-up than
a single-process database. `root` has no password by default and no
database exists yet, so the fixture creates one itself before yielding an
engine pointed at it. The FE's query port accepts connections, and can even
run metadata statements like CREATE DATABASE, before the BE has registered
with it -- an actual CREATE TABLE/INSERT then fails with "Backend node not
found" -- so the fixture retries a real create-table-and-insert probe
against a throwaway table rather than trusting the open port or a bare
CREATE DATABASE as a readiness signal.
Not verified locally in this environment: the `allin1-ubuntu` image is
multiple GB and was skipped here to keep local Docker resource usage low,
per session guidance to lean on CI (which has no such constraint) for
dialects with unusually heavy images.
"""
import time
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
text,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.starrocks import StarRocksEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("starrocks")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import PortWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
QUERY_PORT = 9030
DBNAME = "pilot"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("starrocks/allin1-ubuntu")
container.with_exposed_ports(QUERY_PORT)
container.waiting_for(PortWaitStrategy(QUERY_PORT))
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(QUERY_PORT)
bootstrap_engine = create_engine(
f"starrocks://root:@{host}:{port}/default_catalog.information_schema"
)
# The FE's query port accepts connections, and can even run metadata
# statements like CREATE DATABASE, before any BE (execution backend)
# has registered with it -- an actual table create/insert then fails
# with "Backend node not found". Probe with the real operations the
# tests below need, in a throwaway table, so readiness is confirmed
# for what actually matters rather than just the FE's own port.
last_error: Exception | None = None
for _ in range(60):
try:
with bootstrap_engine.begin() as conn:
conn.execute(text(f"CREATE DATABASE IF NOT EXISTS {DBNAME}"))
probe_engine = create_engine(
f"starrocks://root:@{host}:{port}/default_catalog.{DBNAME}"
)
with probe_engine.begin() as conn:
conn.execute(
text("CREATE TABLE IF NOT EXISTS pilot_ready (id INT)")
)
conn.execute(text("INSERT INTO pilot_ready VALUES (1)"))
conn.execute(text("DROP TABLE pilot_ready"))
break
except Exception as ex: # noqa: BLE001 -- retry on any not-ready-yet error
last_error = ex
time.sleep(2)
else:
raise RuntimeError(
"StarRocks FE/BE never became ready to create and use a table"
) from last_error
yield create_engine(f"starrocks://root:@{host}:{port}/default_catalog.{DBNAME}")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
StarRocksEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = StarRocksEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = StarRocksEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,98 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.timescaledb against a real TimescaleDB instance,
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml -- these exercise real SQL execution
and dialect introspection, which mocked unit tests structurally cannot.
TimescaleDB is a genuine Postgres extension, not a fork: connects via the
plain "postgresql" dialect with psycopg2, same as vanilla Postgres, just
pointed at the timescale/timescaledb image instead of postgres:latest.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.timescaledb import TimescaleDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.postgres")
from testcontainers.community.postgres import PostgresContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with PostgresContainer("timescale/timescaledb:2.29.2-pg16") as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
TimescaleDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported column metadata
rather than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = TimescaleDBEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = TimescaleDBEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,104 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.trino against a real Trino instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Only Presto is covered by existing docker-compose-based integration CI;
Trino, despite sharing lineage with Presto, is not.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.trino import TrinoEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.trino")
from testcontainers.community.trino import TrinoContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with TrinoContainer() as container:
# TrinoContainer.get_connection_url() (testcontainers 4.15.0) returns
# the container-internal port (e.g. 8080) instead of the Docker-
# mapped host port, so the URL it builds cannot actually connect.
# Build it manually with get_exposed_port() instead. Filed upstream:
# https://github.com/testcontainers/testcontainers-python/issues
url = (
f"trino://{container.user}@{container.get_container_host_ip()}"
f":{container.get_exposed_port(container.port)}/memory/default"
)
yield create_engine(url)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. This is the exact bug class in apache/superset#42899,
where Trino emitted OFFSET before LIMIT for paginated queries -- a
dialect-compiler bug invisible to mocked tests, only catchable by
actually executing the compiled SQL.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
TrinoEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = TrinoEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = TrinoEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,148 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.ydb against a real YDB instance, spun up on demand
via testcontainers. Run via .github/workflows/testcontainers.yml.
YDB has no dedicated testcontainers module, so this uses a generic
DockerContainer against the official `ydbplatform/local-ydb` image, which
needs no auth for local/anonymous access -- YDBEngineSpec's own
`sqlalchemy_uri_placeholder` ("ydb://{host}:{port}/{database_name}") has
no username/password at all.
"""
import time
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.ydb import YDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("ydb_sqlalchemy")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import PortWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
GRPC_PORT = 2136
DATABASE = "/local"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("ydbplatform/local-ydb")
container.with_exposed_ports(GRPC_PORT)
# YDB's gRPC client does endpoint discovery: it asks the server for its
# "real" endpoints and reconnects to whatever comes back, rather than
# just using the address it was originally given. By default that's
# the container's own internal Docker hostname (e.g. "6abbb4bb0ab7"),
# which isn't reachable from the host. Binding the same port number on
# the host as inside the container, plus advertising "localhost" as
# the container's own hostname, makes the discovered endpoint
# ("localhost:2136") actually resolve to something reachable.
container.with_bind_ports(GRPC_PORT, GRPC_PORT)
container.with_kwargs(hostname="localhost")
container.with_env("YDB_USE_IN_MEMORY_PDISKS", "true")
container.waiting_for(PortWaitStrategy(GRPC_PORT))
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(GRPC_PORT)
eng = create_engine(f"yql://{host}:{port}{DATABASE}")
# The gRPC port opens, and even a bare SELECT succeeds, before YDB's
# storage pools are fully initialized -- an actual CREATE TABLE can
# still fail with "database doesn't have storage pools at all to
# create tablet channels" (confirmed on a real instance). Probe with
# metadata.create_all()/drop_all() specifically, the same call the
# real tests below make: a raw `text("CREATE TABLE ...")` hits a
# separate, unrelated error ("Scheme operations cannot be executed
# inside transaction") that create_all()'s own DDL execution path
# doesn't, even with AUTOCOMMIT set on a manually-opened connection.
probe_metadata = MetaData()
SATable("pilot_ready", probe_metadata, Column("id", Integer, primary_key=True))
last_error: Exception | None = None
for _ in range(30):
try:
probe_metadata.create_all(eng)
probe_metadata.drop_all(eng)
break
except Exception as ex: # noqa: BLE001 -- retry on any not-ready-yet error
last_error = ex
time.sleep(2)
else:
raise RuntimeError(
"YDB never became ready to create and use a table"
) from last_error
yield eng
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
YDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = YDBEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = YDBEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,119 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.yugabytedb against a real YugabyteDB instance, spun
up on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
YugabyteDB's YSQL layer is fully Postgres-wire compatible (postgresql+
psycopg2, port 5433), but the image doesn't ship a plain `psql` binary
(only its own `ysqlsh`), so testcontainers' PostgresContainer can't be
reused directly -- its built-in readiness check execs `psql`, which would
fail here. This uses a generic DockerContainer, starting the node via
`yugabyted start --background=false` and waiting for yugabyted's own final
startup message instead.
"""
import re
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.yugabytedb import YugabyteDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
YSQL_PORT = 5433
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("yugabytedb/yugabyte:latest")
container.with_exposed_ports(YSQL_PORT)
container.with_command("bin/yugabyted start --background=false")
container.waiting_for(
LogMessageWaitStrategy(
re.compile("Data placement constraint successfully verified")
)
)
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(YSQL_PORT)
# Default single-node credentials/database, per yugabyted's own
# documented quickstart defaults -- no env vars needed to set them.
yield create_engine(
f"postgresql+psycopg2://yugabyte:yugabyte@{host}:{port}/yugabyte"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
YugabyteDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = YugabyteDBEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = YugabyteDBEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,66 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Unit tests for DashboardFilterStateRestApi."""
import inspect
from superset.commands.dashboard.filter_state.create import CreateFilterStateCommand
from superset.commands.dashboard.filter_state.delete import DeleteFilterStateCommand
from superset.commands.dashboard.filter_state.get import GetFilterStateCommand
from superset.commands.dashboard.filter_state.update import UpdateFilterStateCommand
from superset.dashboards.filter_state.api import DashboardFilterStateRestApi
from superset.temporary_cache.api import TemporaryCacheRestApi
def test_dashboard_filter_state_rest_api_inheritance():
"""Ensure DashboardFilterStateRestApi correctly subclasses TemporaryCacheRestApi."""
assert issubclass(DashboardFilterStateRestApi, TemporaryCacheRestApi)
assert (
DashboardFilterStateRestApi.class_permission_name
== "DashboardFilterStateRestApi"
)
assert DashboardFilterStateRestApi.resource_name == "dashboard"
assert DashboardFilterStateRestApi.openapi_spec_tag == "Dashboard Filter State"
def test_dashboard_filter_state_command_factories():
"""Ensure factory methods return the expected command classes."""
api = DashboardFilterStateRestApi()
assert api.get_create_command() is CreateFilterStateCommand
assert api.get_update_command() is UpdateFilterStateCommand
assert api.get_get_command() is GetFilterStateCommand
assert api.get_delete_command() is DeleteFilterStateCommand
def test_post_put_methods_have_no_has_access_api_or_api_decorator():
"""
Ensure post and put methods are not decorated with @has_access_api or @api.
Because DashboardFilterStateRestApi is a temporary cache API, permission
verification is handled dynamically at the command level via
CheckAccessDataCommand. @has_access_api causes 401 Unauthorized for regular
users due to missing FAB permissions. The @api wrapper would catch and convert
uncaught auth errors into 500s.
"""
source_post = inspect.getsource(DashboardFilterStateRestApi.post)
source_put = inspect.getsource(DashboardFilterStateRestApi.put)
assert "has_access_api" not in source_post
assert "has_access_api" not in source_put
assert not any(line.strip().startswith("@api") for line in source_post.splitlines())
assert not any(line.strip().startswith("@api") for line in source_put.splitlines())
-85
View File
@@ -217,91 +217,6 @@ def test_handle_filters_args_returns_request_scoped_filters(
assert fresh_filters.get_joined_filters.call_count == 2
def _create_dataset(name: str) -> Any:
from superset.connectors.sqla.models import SqlaTable
from superset.models.core import Database
SqlaTable.metadata.create_all(db.session.get_bind())
dataset = SqlaTable(
table_name=name,
database=Database(database_name=f"{name}_db", sqlalchemy_uri="sqlite://"),
)
db.session.add(dataset)
db.session.flush()
return dataset
def test_put_dataset_rejects_stale_if_match(
session: Session,
client: Any,
full_api_access: None,
) -> None:
"""
A PUT carrying an ``If-Match`` from an older version is refused with 412.
"""
from superset.versioning.api_helpers import EntityVersionInfo
dataset = _create_dataset("test_put_stale_if_match")
with patch(
"superset.datasets.api.current_entity_version_info",
return_value=EntityVersionInfo(
version=1,
transaction_id=2,
version_uuid="new",
entity_uuid=dataset.uuid,
),
):
response = client.put(
f"/api/v1/dataset/{dataset.id}",
json={"description": "from a stale tab"},
headers={"If-Match": '"old"'},
)
assert response.status_code == 412
assert response.headers["ETag"] == '"new"'
db.session.expire(dataset)
assert dataset.description is None
def test_put_dataset_guards_a_dataset_with_no_version_rows(
session: Session,
client: Any,
full_api_access: None,
) -> None:
"""Baseline rows are written lazily on the first update, so a dataset that
has never been saved has no version rows it must still be guarded, or
the first concurrent save on every pristine dataset goes unprotected.
"""
from superset.versioning.api_helpers import (
EntityVersionInfo,
unversioned_entity_token,
)
dataset = _create_dataset("test_put_unversioned_guard")
entity_uuid = dataset.uuid
with patch(
"superset.datasets.api.current_entity_version_info",
# A dataset that has since been versioned by another tab's save.
return_value=EntityVersionInfo(
version=0,
transaction_id=1,
version_uuid="written-by-the-other-tab",
entity_uuid=entity_uuid,
),
):
response = client.put(
f"/api/v1/dataset/{dataset.id}",
json={"description": "from the tab that opened first"},
headers={"If-Match": f'"{unversioned_entity_token(entity_uuid)}"'},
)
assert response.status_code == 412
db.session.expire(dataset)
assert dataset.description is None
def test_get_dataset_exposes_certification_metadata(
session: Session,
client: Any,
@@ -137,56 +137,6 @@ def test_get_available_engine_specs_keeps_valid_third_party_dialect(
assert available[SqliteEngineSpec] == {"valid_driver"}
def test_get_available_engine_specs_restores_compiler_operators(
mocker: MockerFixture,
) -> None:
"""
A third-party ``sqlalchemy.dialects`` entry point that mutates SQLAlchemy's
shared, process-global ``compiler.OPERATORS`` mapping on import (as
``sqlalchemy-monetdb`` does, in place, rather than subclassing) must not be
allowed to leak that change into every other dialect for the rest of the
process.
Regression test: enumerating a real "monetdb" entry point here (to build the
"available databases" list) silently changed ``!=`` rendering to ``<>`` for
postgres/mysql/sqlite/etc. too, for the remainder of the process.
"""
from sqlalchemy.sql import compiler as sqla_compiler, operators
mocker.patch(
"superset.db_engine_specs.load_engine_specs",
return_value=iter([]),
)
pristine = dict(sqla_compiler.OPERATORS)
assert pristine[operators.ne] != " <> "
class MisbehavingDialect(DefaultDialect):
name = "misbehaving"
driver = "misbehaving_driver"
def load_and_mutate_globally() -> type[MisbehavingDialect]:
# Mirrors sqlalchemy-monetdb's `base.py`: grabs a reference to the
# shared dict (not a copy) and mutates it in place.
sqla_compiler.OPERATORS[operators.ne] = " <> "
return MisbehavingDialect
entry_point = mocker.MagicMock()
entry_point.name = "misbehaving"
entry_point.load.side_effect = load_and_mutate_globally
mocker.patch(
"superset.db_engine_specs.entry_points",
return_value=[entry_point],
)
try:
get_available_engine_specs()
assert sqla_compiler.OPERATORS[operators.ne] == pristine[operators.ne]
finally:
sqla_compiler.OPERATORS.clear()
sqla_compiler.OPERATORS.update(pristine)
@pytest.mark.parametrize(
"app",
[{"DBS_AVAILABLE_DENYLIST": {"databricks": {"pyhive", "pyodbc"}}}],
@@ -14,11 +14,9 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import math
from datetime import datetime
from typing import Any, Optional
from unittest import mock
from unittest.mock import Mock
import pytest
import pytz
@@ -92,37 +90,6 @@ def test_get_column_spec(
assert_column_spec(spec, native_type, sqla_type, attrs, generic_type, is_dttm)
@pytest.mark.parametrize(
"string_value,expected_float",
[
("NaN", math.nan),
("Infinity", math.inf),
("-Infinity", -math.inf),
],
)
def test_column_type_mutator_double_special_values(
string_value: str, expected_float: float
) -> None:
"""
Presto's coordinator sends results as JSON, which has no literal for
NaN/Infinity/-Infinity, so REAL/DOUBLE columns holding those values
arrive as quoted strings. They must be coerced back to real floats
(inherited from PrestoBaseEngineSpec, shared with TrinoEngineSpec).
"""
from superset.db_engine_specs.presto import PrestoEngineSpec
mock_cursor = Mock()
mock_cursor.fetchall.return_value = [[string_value]]
mock_cursor.description = [("val", "double")]
(result_value,) = PrestoEngineSpec.fetch_data(mock_cursor)[0]
assert isinstance(result_value, float)
if math.isnan(expected_float):
assert math.isnan(result_value)
else:
assert result_value == expected_float
def test_get_schema_from_engine_params() -> None:
"""
Test the ``get_schema_from_engine_params`` method.
@@ -18,10 +18,8 @@
from __future__ import annotations
import copy
import math
from collections import namedtuple
from datetime import datetime
from decimal import Decimal
from typing import Any, Optional
from unittest.mock import MagicMock, Mock, patch
@@ -328,96 +326,6 @@ def test_convert_dttm(
assert_convert_dttm(TrinoEngineSpec, target_type, expected_result, dttm)
@pytest.mark.parametrize(
"data,description,expected_result",
[
(
[["1.846619834", "abc"]],
[("dec", "decimal(12,9)"), ("str", "varchar(3)")],
[(Decimal("1.846619834"), "abc")],
),
(
[[Decimal("1.846619834"), "abc"]],
[("dec", "decimal(12,9)"), ("str", "varchar(3)")],
[(Decimal("1.846619834"), "abc")],
),
(
[["1.846619834", "abc"]],
[("dec", "decimal(12)"), ("str", "varchar(3)")],
[(Decimal("1.846619834"), "abc")],
),
(
[["1.846619834", "abc"]],
[("dec", "decimal"), ("str", "varchar(3)")],
[(Decimal("1.846619834"), "abc")],
),
(
[["1.846619834", "abc"]],
[("dec", "varchar(255)"), ("str", "varchar(3)")],
[["1.846619834", "abc"]],
),
(
[["1.846619834", "abc"]],
[("val", "double"), ("str", "varchar(3)")],
[(1.846619834, "abc")],
),
(
[[1.846619834, "abc"]],
[("val", "real"), ("str", "varchar(3)")],
[(1.846619834, "abc")],
),
],
)
def test_column_type_mutator(
data: list[Any],
description: list[Any],
expected_result: list[Any],
) -> None:
"""
Trino's DBAPI driver can return DECIMAL columns as plain strings.
Superset must coerce those back to ``Decimal`` at fetch time so that
downstream numeric post-processing (e.g. a pivot with a mean
aggregate) doesn't choke on a string value.
"""
from superset.db_engine_specs.trino import TrinoEngineSpec
mock_cursor = Mock()
mock_cursor.fetchall.return_value = data
mock_cursor.description = description
assert TrinoEngineSpec.fetch_data(mock_cursor) == expected_result
@pytest.mark.parametrize(
"string_value,expected_float",
[
("NaN", math.nan),
("Infinity", math.inf),
("-Infinity", -math.inf),
],
)
def test_column_type_mutator_double_special_values(
string_value: str, expected_float: float
) -> None:
"""
Trino's wire protocol has no JSON literal for NaN/Infinity/-Infinity, so
REAL/DOUBLE columns holding those values arrive as quoted strings. They
must be coerced back to real floats, same as string-typed DECIMALs.
"""
from superset.db_engine_specs.trino import TrinoEngineSpec
mock_cursor = Mock()
mock_cursor.fetchall.return_value = [[string_value]]
mock_cursor.description = [("val", "double")]
(result_value,) = TrinoEngineSpec.fetch_data(mock_cursor)[0]
assert isinstance(result_value, float)
if math.isnan(expected_float):
assert math.isnan(result_value)
else:
assert result_value == expected_float
def test_get_extra_table_metadata(mocker: MockerFixture) -> None:
from superset.db_engine_specs.trino import TrinoEngineSpec
+1 -24
View File
@@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
from flask_appbuilder.security.sqla.models import User
from jinja2.exceptions import TemplateSyntaxError
from pytest import raises # noqa: PT013
from pytest_mock import MockerFixture
@@ -31,7 +30,7 @@ from superset.commands.exceptions import (
DatasourceNotFoundValidationError,
QueryNotFoundValidationError,
)
from superset.exceptions import SupersetSecurityException, SupersetTemplateException
from superset.exceptions import SupersetSecurityException
from superset.utils.core import DatasourceType, override_user
dataset_find_by_id = "superset.daos.dataset.DatasetDAO.find_by_id"
@@ -341,28 +340,6 @@ def test_query_has_access(mocker: MockerFixture) -> None:
)
def test_query_malformed_jinja_template(mocker: MockerFixture) -> None:
"""
``raise_for_access(query=...)`` Jinja-renders the query's SQL to resolve
the tables it touches. A malformed template must surface as a
``SupersetTemplateException``, not the raw ``jinja2`` exception.
"""
from superset.explore.utils import check_datasource_access
from superset.models.sql_lab import Query
mocker.patch(query_find_by_id, return_value=Query())
mocker.patch(
raise_for_access,
side_effect=TemplateSyntaxError("unexpected end of template", lineno=1),
)
with raises(SupersetTemplateException): # noqa: PT012
check_datasource_access(
datasource_id=1,
datasource_type=DatasourceType.QUERY,
)
def test_query_no_access(mocker: MockerFixture, client) -> None:
from superset.connectors.sqla.models import SqlaTable
from superset.explore.utils import check_datasource_access
-53
View File
@@ -5054,56 +5054,3 @@ def test_adhoc_type_probe_does_not_get_sampling_retry(
table.adhoc_column_to_sqla(adhoc_col)
retry.assert_not_called()
def test_filter_adhoc_column(database: Database) -> None:
"""
Test that filter works with adhoc column labels.
When filter contains a string that matches the label of an adhoc column
in the columns list, it should correctly convert to a SQLAlchemy column
instead of raising QueryObjectValidationError.
"""
from superset.connectors.sqla.models import SqlaTable, TableColumn
table = SqlaTable(
table_name="test_table",
database=database,
columns=[
TableColumn(column_name="name", type="TEXT"),
TableColumn(column_name="real_name", type="TEXT"),
],
)
# Should not raise QueryObjectValidationError
result = table.get_sqla_query(
columns=[
"name",
{
"expressionType": "SQL",
"label": "full_name",
"sqlExpression": "real_name",
},
],
orderby=[],
metrics=[],
extras={},
filter=[
{"col": "full_name", "op": "ILIKE", "val": "Zona%"}
], # Filter by adhoc column label
granularity=None,
is_timeseries=False,
)
assert result is not None
# Verify the WHERE predicate uses the resolved adhoc expression and value.
with database.get_sqla_engine() as engine:
sql = str(
result.sqla_query.compile(
dialect=engine.dialect,
compile_kwargs={"literal_binds": True},
)
)
assert "real_name AS full_name" in sql
assert "WHERE" in sql
assert "lower(real_name) LIKE lower('Zona%')" in sql
@@ -446,428 +446,3 @@ def test_pivot_only_entirely_absent_metrics_are_restored():
assert ("metric_partial", "A") in df.columns
assert ("metric_partial", "B") not in df.columns
assert df[("metric_partial", "A")].iloc[0] == 1.0
# --- show_values_as regression tests (#42809) --------------------------------
#
# ``show_values_as`` expresses each metric cell as a fraction of the row,
# column, or grand total after pivoting. Mirrors the client-side
# ``fractionOf`` semantic in
# ``plugin-chart-pivot-table/src/react-pivottable/utilities.ts:739`` so
# server-side rendering paths (CSV / XLSX exports, scheduled reports)
# match the browser output. See #42809.
#
# Fixture: a tiny 3-column DataFrame that keeps row/col/grand totals easy
# to eyeball. Two rows (``r1``, ``r2``), two columns (``c1``, ``c2``),
# single metric ``v``. Grand total is 100 so every percent-of-total
# assertion is trivially checkable.
def _show_values_as_fixture() -> DataFrame:
"""Long-format input that pivots to::
v
col c1 c2
row
r1 10 20
r2 30 40
row totals: r1=30, r2=70; col totals: c1=40, c2=60; grand=100.
"""
return DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"v": [10, 20, 30, 40],
}
)
def test_pivot_show_values_as_actual_is_noop() -> None:
"""``show_values_as='actual'`` (and ``None``) leaves values unchanged."""
df = _show_values_as_fixture()
aggregates = {"v": {"operator": "sum"}}
baseline = pivot(df=df, index=["row"], columns=["col"], aggregates=aggregates)
for mode in (None, "actual"):
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates=aggregates,
show_values_as=mode,
)
pd.testing.assert_frame_equal(result, baseline)
def test_pivot_show_values_as_percent_row() -> None:
"""Each cell = cell / row-total; each row sums to 1.0."""
result = pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_row",
)
# r1: 10/30, 20/30; r2: 30/70, 40/70
assert result.loc["r1", ("v", "c1")] == pytest.approx(10 / 30)
assert result.loc["r1", ("v", "c2")] == pytest.approx(20 / 30)
assert result.loc["r2", ("v", "c1")] == pytest.approx(30 / 70)
assert result.loc["r2", ("v", "c2")] == pytest.approx(40 / 70)
assert result.sum(axis=1).tolist() == pytest.approx([1.0, 1.0])
def test_pivot_show_values_as_percent_col() -> None:
"""Each cell = cell / column-total; each column sums to 1.0."""
result = pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_col",
)
# c1 total=40: 10/40, 30/40; c2 total=60: 20/60, 40/60
assert result.loc["r1", ("v", "c1")] == pytest.approx(10 / 40)
assert result.loc["r2", ("v", "c1")] == pytest.approx(30 / 40)
assert result.loc["r1", ("v", "c2")] == pytest.approx(20 / 60)
assert result.loc["r2", ("v", "c2")] == pytest.approx(40 / 60)
assert result.sum(axis=0).tolist() == pytest.approx([1.0, 1.0])
def test_pivot_show_values_as_percent_total() -> None:
"""Each cell = cell / grand-total; the whole frame sums to 1.0."""
result = pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_total",
)
# grand=100: each cell divided by 100
assert result.loc["r1", ("v", "c1")] == pytest.approx(0.10)
assert result.loc["r1", ("v", "c2")] == pytest.approx(0.20)
assert result.loc["r2", ("v", "c1")] == pytest.approx(0.30)
assert result.loc["r2", ("v", "c2")] == pytest.approx(0.40)
assert result.values.sum() == pytest.approx(1.0)
def test_pivot_show_values_as_preserves_nan_numerator() -> None:
"""A NaN/NULL numerator stays NaN — matches the client-side #42810 guard
that a genuine SQL NULL should render blank, not "0.0%".
The fixture uses a **missing** (row, col) combination ``r1`` has no
``c2`` row so ``pivot_table`` produces a genuine NaN cell for
(``r1``, ``c2``). Using a ``NaN`` *input value* with ``operator='sum'``
would not exercise this path because ``pandas`` ``.sum(skipna=True)``
on a single-value ``[NaN]`` group returns ``0.0``, not ``NaN``.
"""
df = DataFrame(
{
"row": ["r1", "r2", "r2"], # r1 has no c2 row → post-pivot NaN
"col": ["c1", "c1", "c2"],
"v": [10, 30, 40],
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_row",
)
# The genuinely-NaN cell stays NaN through the percent transform.
assert pd.isna(result.loc["r1", ("v", "c2")])
# The other cell in the same row divides correctly against just its
# own value (row total is 10 since c2 is NaN and skipna=True).
assert result.loc["r1", ("v", "c1")] == pytest.approx(1.0)
def test_pivot_show_values_as_percent_total_zero_grand_total_yields_nan() -> None:
"""Grand total of zero yields NaN cells rather than Infinity — matches the
client's ``if (acc === null) return null`` division-by-zero guard."""
df = DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"v": [0, 0, 0, 0],
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_total",
)
# No cell should be Infinity or a real number; all should be NaN.
assert result.isna().values.all()
def test_pivot_show_values_as_percent_row_multi_metric_keeps_metrics_separate() -> None:
"""On a multi-metric pivot (``MultiIndex`` columns), per-row totals are
computed *within each metric*. Metric A's percentages must sum to 1.0
per row independent of metric B's values."""
df = DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"a": [10, 20, 30, 40],
"b": [1, 3, 5, 7],
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"a": {"operator": "sum"}, "b": {"operator": "sum"}},
show_values_as="percent_row",
)
# Metric ``a``: row totals 30 and 70; each row of ``a`` sums to 1.
assert result["a"].sum(axis=1).tolist() == pytest.approx([1.0, 1.0])
# Metric ``b``: row totals 4 and 12; each row of ``b`` sums to 1.
assert result["b"].sum(axis=1).tolist() == pytest.approx([1.0, 1.0])
# Metric ``a`` percentages must not be contaminated by metric ``b`` values.
assert result.loc["r1", ("a", "c1")] == pytest.approx(10 / 30)
assert result.loc["r1", ("b", "c1")] == pytest.approx(1 / 4)
def test_pivot_show_values_as_invalid_mode_raises() -> None:
"""An unknown ``show_values_as`` value raises ``InvalidPostProcessingError``
rather than silently falling through to a no-op."""
with pytest.raises(InvalidPostProcessingError):
pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_of_moon",
)
def test_pivot_show_values_as_empty_string_is_noop() -> None:
"""Empty-string ``show_values_as`` is treated as a no-op alongside
``None`` and ``"actual"`` it must NOT reach the percent-mode
validator (which would raise on it) or silently divide.
"""
df = _show_values_as_fixture()
aggregates = {"v": {"operator": "sum"}}
baseline = pivot(df=df, index=["row"], columns=["col"], aggregates=aggregates)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates=aggregates,
show_values_as="",
)
pd.testing.assert_frame_equal(result, baseline)
def test_pivot_show_values_as_percent_total_flat_multi_metric() -> None:
"""A multi-metric pivot with **no** ``columns`` groupby produces a
**flat** column index each column IS its own metric. ``percent_total``
must divide each metric column by its OWN grand total (never mixing
metrics), otherwise one metric's magnitude changes another metric's
percentages.
"""
df = DataFrame({"row": ["r1", "r2"], "a": [10, 30], "b": [1, 3]})
result = pivot(
df=df,
index=["row"],
aggregates={"a": {"operator": "sum"}, "b": {"operator": "sum"}},
show_values_as="percent_total",
)
# Each metric column sums to 1.0 independently.
assert result["a"].sum() == pytest.approx(1.0)
assert result["b"].sum() == pytest.approx(1.0)
# And metric a's magnitude (10, 30 → grand 40) doesn't leak into
# metric b's percentages (which use grand 4).
assert result.loc["r1", "a"] == pytest.approx(10 / 40)
assert result.loc["r1", "b"] == pytest.approx(1 / 4)
def test_pivot_show_values_as_percent_row_zero_row_total_yields_nan() -> None:
"""A row whose values sum to zero yields NaN cells in that row rather
than ``Infinity``/``NaN`` from division-by-zero. Other rows still
divide correctly.
"""
df = DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"v": [0, 0, 30, 40], # r1's row-total is 0
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_row",
)
assert pd.isna(result.loc["r1", ("v", "c1")])
assert pd.isna(result.loc["r1", ("v", "c2")])
# r2 still divides correctly against its own row-total (70).
assert result.loc["r2", ("v", "c1")] == pytest.approx(30 / 70)
def test_pivot_show_values_as_percent_col_zero_col_total_yields_nan() -> None:
"""A column whose values sum to zero yields NaN cells in that column
rather than ``Infinity``/``NaN`` from division-by-zero. Other columns
still divide correctly.
"""
df = DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"v": [0, 20, 0, 40], # c1's column-total is 0
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_col",
)
assert pd.isna(result.loc["r1", ("v", "c1")])
assert pd.isna(result.loc["r2", ("v", "c1")])
# c2 still divides correctly against its own column-total (60).
assert result.loc["r1", ("v", "c2")] == pytest.approx(20 / 60)
def test_pivot_show_values_as_with_marginal_distributions_raises() -> None:
"""``show_values_as`` combined with ``marginal_distributions`` would
include the ``All`` margin row/column in the row/column/grand-total
denominators, producing wrong percentages. Combining the two needs a
first-class design; for now the combination raises loudly rather than
silently returning wrong numbers.
"""
with pytest.raises(InvalidPostProcessingError, match="marginal_distributions"):
pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
marginal_distributions=True,
show_values_as="percent_row",
)
def test_pivot_show_values_as_with_combine_value_with_metric_preserves_per_metric() -> (
None
):
"""Regression test for sadpandajoe's finding on #42976.
``combine_value_with_metric`` reshapes the column ``MultiIndex`` from
``(metric, category)`` to ``(category, metric)``. Historically the
``show_values_as`` transform ran *after* this reshape, so its per-metric
iteration walked categories thinking they were metrics mixing metric
magnitudes and producing wrong percentages (e.g. metric ``a``'s row
would sum to ~1.78 instead of 1.0 because metric ``b``'s values leaked
into ``a``'s denominators).
The transform now runs *before* the reshape so per-metric isolation
stays intact regardless of the final column layout.
"""
df = DataFrame(
{
"row": ["r1", "r1", "r2", "r2"],
"col": ["c1", "c2", "c1", "c2"],
"a": [10, 20, 30, 40],
"b": [1, 3, 5, 7],
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"a": {"operator": "sum"}, "b": {"operator": "sum"}},
combine_value_with_metric=True,
show_values_as="percent_row",
)
# After combine_value_with_metric, the column MultiIndex is
# ``(category, metric)``. Per-metric row sums are pulled via cross-section
# on level 1 (the metric axis).
for metric, expected in (("a", [1.0, 1.0]), ("b", [1.0, 1.0])):
per_metric = result.xs(metric, axis=1, level=1)
assert per_metric.sum(axis=1).tolist() == pytest.approx(expected), (
f"metric {metric!r} rows must each sum to 1.0 after "
"percent_row on a combined pivot; got contamination from "
"other metrics"
)
# And the actual values match the natural per-metric percentages,
# not the mixed-metric ones that the bug produced.
assert result.loc["r1", ("c1", "a")] == pytest.approx(10 / 30)
assert result.loc["r1", ("c1", "b")] == pytest.approx(1 / 4)
def test_pivot_show_values_as_rejects_non_additive_aggregate() -> None:
"""``show_values_as`` requires additive aggregates.
For a ``mean`` aggregate, the summed per-cell values are not the
row/column/grand rollup the DB would compute over the underlying
rows, so ``cell / sum(cells)`` disagrees with the "share of the
real row total" the chart shows. Reject up front rather than emit
numbers that mix with the DB rollup incorrectly.
"""
with pytest.raises(InvalidPostProcessingError, match="additive"):
pivot(
df=_show_values_as_fixture(),
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "mean"}},
show_values_as="percent_row",
)
def test_pivot_show_values_as_on_empty_pivot_returns_empty_frame() -> None:
"""Empty inputs must not crash the percent transform.
An empty pivot with a column grouping has a ``MultiIndex`` with zero
level-0 groups; the metric-iteration loop then feeds ``pd.concat``
an empty list and raises ``ValueError: No objects to concatenate``.
The empty frame should pass through unchanged.
"""
empty = DataFrame({"row": [], "col": [], "v": []}).astype(
{"row": str, "col": str, "v": float}
)
result = pivot(
df=empty,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_row",
)
assert result.empty
def test_pivot_show_values_as_preserves_structural_nan() -> None:
"""Structurally-missing cells (no input rows for that (row, col)) stay NaN.
NULL preservation is scoped to the structural case: cells that
``pivot_table`` left as ``NaN`` because no input row exists for that
(row, column) group must render as blank (``NaN``), not as ``0%``.
Value-is-NULL cells are a separate case documented on
``_apply_show_values_as``.
"""
df = DataFrame(
{
"row": ["r1", "r2", "r2"],
"col": ["c1", "c1", "c2"],
"v": [10.0, 30.0, 40.0],
}
)
result = pivot(
df=df,
index=["row"],
columns=["col"],
aggregates={"v": {"operator": "sum"}},
show_values_as="percent_row",
)
# r1 has no c2 row → cell is structurally missing → stays NaN.
assert pd.isna(result.loc["r1", ("v", "c2")])
# r1's row-total is just c1 (10.0), so c1 is 100%.
assert result.loc["r1", ("v", "c1")] == pytest.approx(1.0)
@@ -392,49 +392,3 @@ def test_prophet_does_not_clamp_yhat_below_zero_for_negative_actuals():
forecast_periods = 2
forecast_yhat = result["balance__yhat"].iloc[-forecast_periods:]
assert (forecast_yhat < 0).any()
def test_prophet_every_time_grain_is_mapped():
"""No `TimeGrain` may be missing from `PROPHET_TIME_GRAIN_MAP`.
Engine specs offer these grains in the Time Grain control, and the
`ChartDataProphetOptionsSchema.time_grain` field advertises them in the
OpenAPI spec, so an unmapped grain surfaces as "Unsupported time grain"
on a value the UI and the API both present as valid.
"""
from superset.constants import TimeGrain
from superset.utils.pandas_postprocessing.utils import PROPHET_TIME_GRAIN_MAP
unmapped = [
grain.name for grain in TimeGrain if grain not in PROPHET_TIME_GRAIN_MAP
]
assert not unmapped, f"TimeGrain values missing a pandas frequency: {unmapped}"
@pytest.mark.parametrize(
"time_grain",
["PT5S", "PT30S", "PT0.5H", "PT6H", "P0.25Y"],
)
def test_prophet_previously_unmapped_time_grains(time_grain):
"""These five grains raised "Unsupported time grain" despite being offered
by engine specs such as Postgres, SQLite, Presto and Druid."""
df = prophet(
df=prophet_df, time_grain=time_grain, periods=3, confidence_interval=0.9
)
assert {"a__yhat", "a__yhat_upper", "a__yhat_lower"} <= set(df.columns)
def test_prophet_alias_time_grains_match_their_canonical_form():
"""`HALF_HOUR`/`QUARTER_YEAR` are alternate ISO-8601 spellings of
`THIRTY_MINUTES`/`QUARTER` and must resolve to the same frequency."""
from superset.constants import TimeGrain
from superset.utils.pandas_postprocessing.utils import PROPHET_TIME_GRAIN_MAP
assert (
PROPHET_TIME_GRAIN_MAP[TimeGrain.HALF_HOUR]
== PROPHET_TIME_GRAIN_MAP[TimeGrain.THIRTY_MINUTES]
)
assert (
PROPHET_TIME_GRAIN_MAP[TimeGrain.QUARTER_YEAR]
== PROPHET_TIME_GRAIN_MAP[TimeGrain.QUARTER]
)

Some files were not shown because too many files have changed in this diff Show More