Compare commits

..
Author SHA1 Message Date
Elizabeth Thompson 1cadf39ce8 test(sql): cover SqlglotError fallback branch in parse_predicate
The regression test only exercised the ParseError branch, leaving the
generic sqlglot.errors.SqlglotError fallback in
SQLStatement.parse_predicate uncovered and dropping line coverage below
the 100% gate. Add a test that mocks sqlglot.parse_one to raise a bare
SqlglotError and asserts it is converted to a SupersetParseError.
2026-08-28 22:14:52 +00:00
Elizabeth Thompson 876b8641e2 fix(sql): catch sqlglot ParseError when parsing RLS predicates
SQLStatement.parse_predicate called sqlglot.parse_one unguarded, so a
syntactically invalid RLS predicate raised a raw sqlglot ParseError.
Reachable via apply_rls (e.g. POST /api/v1/sqllab/estimate with
RLS_IN_SQLLAB enabled), this surfaced as an opaque 500 instead of a
typed 422.

Wrap the call to convert ParseError/SqlglotError into SupersetParseError,
mirroring the existing idiom in SQLStatement._parse.
2026-08-28 16:49:17 +00:00
117 changed files with 699 additions and 6660 deletions
+8 -5
View File
@@ -24,6 +24,14 @@ updates:
- dependency-name: "@types/react-dom"
update-types: ["version-update:semver-major"]
- dependency-name: "react-icons"
# JSDOM v30 doesn't play well with Jest v30
# Source: https://jestjs.io/blog#known-issues
# GH thread: https://github.com/jsdom/jsdom/issues/3492
- dependency-name: "jest-environment-jsdom"
# `@swc/plugin-transform-imports` doesn't work with current Webpack-SWC hybrid setup
# See https://github.com/apache/superset/pull/37384#issuecomment-3793991389
# TODO: remove the plugin once Lodash usage has been migrated to a more readily tree-shakeable alternative
- dependency-name: "@swc/plugin-transform-imports"
# deck.gl and luma.gl share strict peer constraints across the root and
# plugin workspaces, and root overrides pin their transitive versions.
# Upgrade both families together in a manually validated change.
@@ -79,11 +87,6 @@ updates:
patterns:
- "ag-grid-react"
- "ag-grid-community"
swc:
patterns:
- "@swc/core"
- "@swc/plugin-emotion"
- "@swc/plugin-transform-imports"
open-pull-requests-limit: 30
versioning-strategy: increase
cooldown:
-21
View File
@@ -66,27 +66,6 @@ jobs:
- name: "Set up liccheck"
run: |
# liccheck (as of 0.9.2) still does a bare `import pkg_resources`
# without declaring setuptools as a dependency, relying on it
# having historically been bundled. setuptools 81+ (installed
# above via requirements/base.txt) dropped the pkg_resources
# subpackage entirely, so liccheck's own import breaks outright.
#
# Reinstalling an older setuptools would restore pkg_resources but
# would also downgrade the *real* setuptools install, which then
# trips liccheck's own working_set.resolve() -- it cross-checks
# requirements/base.txt's declared `setuptools==84.0.0` against
# what's actually installed, and a downgrade makes those disagree.
#
# Instead, vendor just the pkg_resources/ package files from an
# old setuptools wheel into site-packages, leaving the real
# setuptools install (and its dist-info metadata) untouched. This
# gives liccheck an importable pkg_resources whose own working-set
# scan still correctly reports the real installed setuptools
# version, so no conflict is raised.
pip download "setuptools<81" --no-deps -d /tmp/old-setuptools
python -m zipfile -e /tmp/old-setuptools/setuptools-*.whl /tmp/old-setuptools-extracted/
cp -r /tmp/old-setuptools-extracted/pkg_resources "$(python -c 'import site; print(site.getsitepackages()[0])')/"
uv pip install --system liccheck
- name: "Run liccheck"
run: |
-192
View File
@@ -1,192 +0,0 @@
# db_engine_specs tests against real databases (testcontainers)
name: Testcontainers
# Spins up real Docker containers (see tests/testcontainers/ for the current
# dialect list) via testcontainers-python, which catches real dialect/driver
# regressions -- the kind mocked db_engine_specs unit tests structurally
# cannot, e.g. apache/superset#42899 (Trino emitting OFFSET before LIMIT).
# Runs on a nightly cron (catches drift from a driver's own releases, not
# just from Superset's changes) and on pull_request, scoped via `paths` to
# only PRs that actually touch this test suite or the workflow itself, so
# unrelated PRs across the repo are never affected.
#
# A matrix entry can set `nightly_only: true` to run only on the cron (or a
# manual workflow_dispatch), never on pull_request -- for a dialect whose
# image is too heavy (a multi-service cluster, a many-GB image, a slow
# licensed installer) to justify adding its wall-clock/resource cost to
# every PR that merely touches this suite. Omit the field entirely for a
# normal dialect; it isn't nightly-only by default.
permissions:
contents: read
on:
schedule:
- cron: "0 5 * * *"
workflow_dispatch: {}
pull_request:
paths:
- ".github/workflows/testcontainers.yml"
- "tests/testcontainers/**"
- "superset/db_engine_specs/**"
- "pyproject.toml"
- "requirements/development.in"
- "requirements/development.txt"
concurrency:
# Scoped by ref, not just workflow name -- otherwise every PR run and the
# nightly cron share one group, and starting the workflow on another PR
# (or the nightly firing mid-PR-run) cancels an unrelated in-progress run.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
testcontainers:
runs-on: ubuntu-26.04
strategy:
fail-fast: false
matrix:
include:
# One job per dialect rather than one job for the whole suite: a
# single slow container would otherwise inflate the wall-clock
# time for every dialect, not just its own. Running in parallel
# means the suite's total time is bounded by the slowest dialect,
# not the sum of all of them. Db2's first-boot init is documented
# upstream as notably slow (a real instance bring-up, not just a
# process start) and untested locally here (no arm64 image), so
# it gets a wider timeout margin than the rest until real CI data
# says otherwise.
- dialect: cockroachdb
timeout: 10
- dialect: crate
timeout: 10
- dialect: trino
timeout: 10
- dialect: mssql
timeout: 10
- dialect: elasticsearch
timeout: 10
- dialect: oracle
timeout: 15
- dialect: db2
timeout: 25
- dialect: mariadb
timeout: 10
- dialect: timescaledb
timeout: 10
- dialect: yugabytedb
timeout: 10
- dialect: monetdb
timeout: 10
- dialect: mongodb
timeout: 10
- dialect: postgres
timeout: 10
- dialect: mysql
timeout: 10
- dialect: clickhouse
timeout: 10
# StarRocks' allin1-ubuntu image brings up both FE and BE in one
# container, which is a heavier bring-up than a single-process
# database -- wider margin until real CI data says otherwise.
- dialect: starrocks
timeout: 15
- dialect: databend
timeout: 10
- dialect: risingwave
timeout: 10
- dialect: firebird
timeout: 10
- dialect: ydb
timeout: 10
# OceanBase bootstraps a distributed-style cluster even in
# single-node MODE=MINI -- too heavy for every PR's CI budget, so
# it runs on the nightly cron / manual dispatch only.
- dialect: oceanbase
timeout: 20
nightly_only: true
timeout-minutes: ${{ matrix.timeout }}
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
# This job's matrix installs exactly one dialect's testcontainers
# driver for exactly this job, so treat that driver as required: a
# broken/missing import should fail the job, not silently skip to a
# misleadingly green, zero-tests-run result. See _driver.py.
SUPERSET_TESTCONTAINERS_STRICT: true
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Setup Python
uses: ./.github/actions/setup-backend/
with:
python-version: current
- name: Install db2 driver (ibm-db-sa)
# ibm-db (the db2 DBAPI) ships no Linux arm64 wheel, so it's kept out
# of the baseline dev install (requirements/development.in) to avoid
# breaking the multi-platform dev Docker image build. Install it here
# instead, only for this leg of the matrix.
if: matrix.dialect == 'db2'
run: uv pip install --system -e .[db2]
- name: Install oceanbase driver (oceanbase_py)
# oceanbase_py pins sqlalchemy-utils>=0.38.3,<0.39, which conflicts
# outright with Superset's own sqlalchemy-utils==0.42.1 pin -- kept
# out of the baseline dev install for the same reason as db2 above.
# Installed as its own standalone package (not via `-e .[oceanbase]`)
# so --no-deps only skips *oceanbase_py's* dependencies -- applied
# to `-e .[oceanbase]` instead, --no-deps blocks pip from installing
# anything the extras marker pulls in, including oceanbase_py
# itself, which "succeeds" without actually installing it
# (confirmed on real CI: the install step reported success, but the
# module was still missing). This job only needs oceanbase_py's
# dialect module importable, not its sqlalchemy-utils dependency
# satisfied, since nothing here calls into it.
if: >-
matrix.dialect == 'oceanbase' &&
(matrix.nightly_only != true ||
github.event_name == 'schedule' ||
github.event_name == 'workflow_dispatch')
run: uv pip install --system --no-deps "oceanbase_py>=0.0.1.2"
- name: Install Firebird client library (libfbclient2)
# sqlalchemy-firebird's driver (firebird-driver) is a pure-Python
# ctypes wrapper (its wheel is py3-none-any) that dynamically loads
# the native Firebird client library from the host at import time
# -- it doesn't bundle that library itself, so it has to come from
# the system package manager, only for this leg of the matrix.
if: matrix.dialect == 'firebird'
run: |
sudo apt-get update
sudo apt-get install -y libfbclient2
- name: Run testcontainers db_engine_specs tests (${{ matrix.dialect }})
# A job-level `if:` can't reference `matrix` (only github/inputs/
# needs/vars are available there), so the nightly_only skip has to
# live on the step instead. A dialect without `nightly_only` set
# evaluates the left side true (unset is null, and `null != true`
# is true) and always runs; one WITH it set only runs on the cron
# or a manual dispatch, never on pull_request.
if: >-
matrix.nightly_only != true ||
github.event_name == 'schedule' ||
github.event_name == 'workflow_dispatch'
run: |
pytest --durations-min=2 -v -m testcontainers \
./tests/testcontainers/db_engine_specs/test_${{ matrix.dialect }}.py \
--junit-xml=test-results/junit-testcontainers-${{ matrix.dialect }}.xml
- name: Upload JUnit test results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: junit-results-testcontainers-${{ matrix.dialect }}
path: test-results/
retention-days: 7
actions-timeline:
needs: [testcontainers]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@57fc93f20c6da7fbc14063c6d24a2a5627c799ad # v3.2.0
+1 -1
View File
@@ -25,7 +25,7 @@ assists people when migrating to a new version.
## Next
- `SAMPLES_ROW_LIMIT` is now the default for `/datasource/samples` requests without a valid explicit `per_page`, rather than a hard per-request ceiling; explicit limits are honored up to the existing global row-limit ceiling, matching `/chart/data` SAMPLES requests.
- The `cockroachdb` extra (`pip install apache-superset[cockroachdb]`) now installs `sqlalchemy-cockroachdb` instead of the abandoned `cockroachdb` package, whose SQLAlchemy dialect could not be imported under SQLAlchemy 2.0. Existing environments with the old package installed must `pip uninstall cockroachdb` before reinstalling the extra -- both packages register the same `cockroachdb` SQLAlchemy dialect entry point, so leaving the old one in place can still load the abandoned implementation.
- The `cockroachdb` extra (`pip install apache-superset[cockroachdb]`) now installs `sqlalchemy-cockroachdb` instead of the abandoned `cockroachdb` package, whose SQLAlchemy dialect could not be imported under SQLAlchemy 2.0. Existing environments with the old package installed should `pip uninstall cockroachdb && pip install sqlalchemy-cockroachdb` (or simply reinstall the extra) to restore CockroachDB connectivity.
### MCP tool results preserve stored string values
+2 -2
View File
@@ -62,7 +62,7 @@
"@superset-ui/core": "^0.20.4",
"@swc/core": "^1.16.1",
"antd": "^6.6.1",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"caniuse-lite": "^1.0.30001809",
"docusaurus-plugin-openapi-docs": "^5.2.0",
"docusaurus-theme-openapi-docs": "^5.2.0",
@@ -76,7 +76,7 @@
"react-svg-pan-zoom": "^3.13.1",
"react-table": "^7.8.0",
"remark-import-partial": "^0.0.2",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"storybook": "^10.5.10",
"swagger-ui-react": "^5.32.14",
"swc-loader": "^0.2.7",
+11 -11
View File
@@ -6529,10 +6529,10 @@ base64-js@^1.3.1, base64-js@^1.5.1:
resolved "https://registry.yarnpkg.com/base64-js/-/base64-js-1.5.1.tgz#1b1b440160a5bf7ad40b650f095963481903930a"
integrity sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.17, baseline-browser-mapping@^2.9.19:
version "2.11.17"
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.17.tgz#a168205490077c5d7c542f1610016af4ade8f875"
integrity sha512-KAUDn1OSS0fmPlGO+NOUMRcOQ/b/shUBH3OgkG73mPgdf+JD/BQ6fHboGxNOxnUmlwcq+lLq3dTkayRPuSfXwg==
baseline-browser-mapping@^2.10.38, baseline-browser-mapping@^2.11.16, baseline-browser-mapping@^2.9.19:
version "2.11.16"
resolved "https://registry.yarnpkg.com/baseline-browser-mapping/-/baseline-browser-mapping-2.11.16.tgz#0fa19a4ece2e34439ecaa3fdca8a59acfbd287fb"
integrity sha512-H/bNPUFHewJHyCTdjn1n3Pit5+2GmWT6mmeHImPX+8MA9NA6b67jO4gYmi4jTbCJb2otq34KMZnovndDPqJwhQ==
batch@0.6.1:
version "0.6.1"
@@ -6614,9 +6614,9 @@ boxen@^7.0.0:
wrap-ansi "^8.1.0"
brace-expansion@^1.1.7:
version "1.1.18"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.18.tgz#3ce74d89885136be1535341f8c3d4425c29a5cab"
integrity sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==
version "1.1.15"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.15.tgz#a6d90d54067236e5f42570a3b7378d594d9b7738"
integrity sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==
dependencies:
balanced-match "^1.0.0"
concat-map "0.0.1"
@@ -14106,10 +14106,10 @@ reselect@^4.0.0:
resolved "https://registry.yarnpkg.com/reselect/-/reselect-4.1.8.tgz#3f5dc671ea168dccdeb3e141236f69f02eaec524"
integrity sha512-ab9EmR80F/zQTMNeneUr4cv+jSwPJgIlvEmVwLerwrWVbpLlBuls9XHzIeTFy4cegU2NHBp3va0LKOzU5qFEYQ==
reselect@^5.1.0, reselect@^5.1.1, reselect@^5.3.0:
version "5.3.0"
resolved "https://registry.yarnpkg.com/reselect/-/reselect-5.3.0.tgz#0a3e3ed4436bdf2ab7c5e0f392dab2c062595d61"
integrity sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg==
reselect@^5.1.0, reselect@^5.1.1, reselect@^5.2.0:
version "5.2.0"
resolved "https://registry.yarnpkg.com/reselect/-/reselect-5.2.0.tgz#f380ef7664332d26ea06c1cba04bdbbdcaa955f1"
integrity sha512-AgZ3UOZm3YndfrJ4OYjgrT7bmCm/1iqkjvEfH/oYjzh6PD2qw4QuT3jjnXIrpdt4MTpMXclMT3lXbmRY+XRakw==
resize-observer-polyfill@1.5.1:
version "1.5.1"
+7 -21
View File
@@ -16,7 +16,7 @@
# under the License.
[build-system]
requires = ["setuptools>=84.0.0", "wheel"]
requires = ["setuptools>=40.9.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
@@ -67,7 +67,7 @@ dependencies = [
"flask-sqlalchemy>=3.1.1, <4.0",
"flask-wtf>=1.3.0, <2.0",
"geopy",
"greenlet<=3.5.5, >=3.5.5",
"greenlet<=3.5.4, >=3.5.4",
"gunicorn>=26.0.0, <27; sys_platform != 'win32'",
"hashids>=1.3.1, <2",
# holidays>=0.45 required for security fix
@@ -148,9 +148,10 @@ clickhouse = ["clickhouse-connect>=1.7.2, <2.0"]
# 2.0). sqlalchemy-cockroachdb is the actively maintained replacement,
# already linked from CockroachDbEngineSpec.metadata's docs_url, and
# registers the same `cockroachdb` SQLAlchemy dialect entry point.
# sqlalchemy-cockroachdb itself declares no DBAPI dependency (its own docs
# require picking one), so pull in the same psycopg2-binary pin as the
# `postgres` extra -- CockroachDB speaks the Postgres wire protocol.
# sqlalchemy-cockroachdb depends only on SQLAlchemy itself, not on a DBAPI
# driver, so psycopg2-binary is pinned alongside it (matching the `postgres`
# extra) to keep this extra self-contained -- CockroachDB speaks the
# PostgreSQL wire protocol, so psycopg2 is what actually opens connections.
cockroachdb = ["sqlalchemy-cockroachdb>=2.0.0, <3", "psycopg2-binary==2.9.12"]
crate = ["sqlalchemy-cratedb>=0.43.1, <1"]
# sqlalchemy-d1's only release (0.1.0, Nov 2025) pins sqlalchemy<2,>=1.4,
@@ -222,11 +223,6 @@ impala = ["impyla>=0.24.0, <0.25"]
# superset/db_engine_specs/kusto.py's known_incompatibilities metadata.
kusto = ["sqlalchemy-kusto>=3.1.2, <4"]
kylin = ["kylinpy>=2.8.4, <2.9"]
# MariaDB is a MySQL fork implementing the same wire protocol - connects via
# the plain mysql dialect, same driver as mysql.
mariadb = ["apache-superset[mysql]"]
monetdb = ["sqlalchemy-monetdb>=2.1.0, <3", "pymonetdb>=1.9.1, <2"]
mongodb = ["pymongosql>=0.7.3, <1"]
mssql = ["pymssql>=2.3.13, <3"]
# motherduck is an alias for duckdb - MotherDuck works via the duckdb driver
motherduck = ["apache-superset[duckdb]"]
@@ -275,9 +271,6 @@ tdengine = [
"taos-ws-py>=0.7.0"
]
teradata = ["teradatasql>=20.0.0.66"]
# TimescaleDB is a genuine Postgres extension, not a fork - connects via the
# plain postgresql dialect, same driver as postgres.
timescaledb = ["apache-superset[postgres]"]
thumbnails = [] # deprecated, will be removed in 7.0
vertica = ["sqlalchemy-vertica-python>= 0.6.3, < 0.7"]
netezza = ["nzalchemy>= 11.1.2, < 11.2"]
@@ -285,18 +278,11 @@ starrocks = ["starrocks>=1.3.4, <2"]
doris = ["pydoris>=1.2.0, <2.0.0"]
oceanbase = ["oceanbase_py>=0.0.1.2"]
ydb = ["ydb-sqlalchemy>=0.1.22", "ydb-sqlglot-plugin>=0.2.8"]
# YugabyteDB's YSQL layer is fully Postgres-wire compatible - connects via
# the plain postgresql dialect, same driver as postgres.
yugabytedb = ["apache-superset[postgres]"]
development = [
# no bounds for apache-superset-extensions-cli until a stable version
"apache-superset-extensions-cli",
"boto3",
# 7.0.0 raises `docker.errors.DockerException: ... Not supported URL
# scheme http+docker` against the requests/urllib3 versions pinned
# elsewhere in this file -- breaks testcontainers (tests/testcontainers/)
# before any container even starts. 7.2.0 is confirmed working.
"docker>=7.2.0",
"docker",
"flask-testing",
"freezegun",
"grpcio>=1.82.1",
+1 -7
View File
@@ -19,13 +19,7 @@ testpaths =
tests
python_files = *_test.py test_*.py *_tests.py *viz/utils.py
# `-p no:warnings` temporarily disabled in favor of more finely tuned `filterwarnings`.
# `not testcontainers` excludes tests/testcontainers/ by default: those spin up
# real Docker containers, and `testpaths = tests` would otherwise pull them into
# every plain `pytest` run. The dedicated CI job (testcontainers.yml) overrides
# this with an explicit `-m testcontainers` to run them.
addopts = -m "not testcontainers"
markers =
testcontainers: exercises a real database via testcontainers-python (needs Docker); excluded by default, see .github/workflows/testcontainers.yml
#addopts = -p no:warnings
asyncio_mode = auto
# `ignore` is effectively equivalent to `-p no:warnings`.
+2 -2
View File
@@ -52,11 +52,11 @@ marshmallow-sqlalchemy>=1.5.0
# needed for python 3.12 support
openapi-schema-validator>=0.6.3
# Pin setuptools <85 until all dependencies migrate from pkg_resources to importlib.metadata
# Pin setuptools <81 until all dependencies migrate from pkg_resources to importlib.metadata
# pkg_resources is deprecated and will be removed in setuptools 81+ (around 2025-11-30)
# Known affected packages: Preset's 'clients' package
# See docs/docs/contributing/pkg-resources-migration.md for details
setuptools<85
setuptools<81
# google-auth 2.53+ dropped its transitive dependency on cachetools, which is
# imported directly by superset.db_engine_specs.aws_iam. We declare cachetools
+2 -2
View File
@@ -163,7 +163,7 @@ google-auth==2.53.0
# via
# -r requirements/base.in
# shillelagh
greenlet==3.5.5
greenlet==3.5.4
# via
# apache-superset (pyproject.toml)
# shillelagh
@@ -366,7 +366,7 @@ rpds-py==0.25.0
# via
# jsonschema
# referencing
setuptools==84.0.0
setuptools==80.9.0
# via -r requirements/base.in
shillelagh==1.4.5
# via apache-superset (pyproject.toml)
+1 -28
View File
@@ -16,32 +16,5 @@
# specific language governing permissions and limitations
# under the License.
#
-e .[development,bigquery,clickhouse,cockroachdb,crate,databend,druid,duckdb,elasticsearch,fastmcp,firebird,gevent,gsheets,monetdb,mongodb,mssql,mysql,oracle,postgres,presto,prophet,risingwave,starrocks,trino,thumbnails,ydb]
-e .[development,bigquery,cockroachdb,druid,duckdb,fastmcp,gevent,gsheets,mysql,postgres,presto,prophet,trino,thumbnails]
-e ./superset-extensions-cli[test]
# testcontainers-backed db_engine_specs tests (tests/testcontainers/) --
# see .github/workflows/testcontainers.yml
#
# `db2` (the `ibm-db-sa`/`ibm-db` driver) and `oceanbase` (the `oceanbase_py`
# driver) are both deliberately left out of the baseline dev install above:
# `ibm-db` ships no Linux arm64 wheel, breaking the multi-platform
# (amd64+arm64) dev Docker image build; `oceanbase_py` pins
# `sqlalchemy-utils>=0.38.3,<0.39`, which conflicts outright with Superset's
# own `sqlalchemy-utils==0.42.1` pin -- there's no version of both that can
# coexist in one resolved environment. Both testcontainers CI jobs install
# their driver on demand instead, only for their own matrix leg -- see
# .github/workflows/testcontainers.yml.
#
# mariadb/timescaledb/yugabytedb need no testcontainers extra of their own:
# they reuse the postgres/mysql container classes pointed at a different
# image, and psycopg2-binary/mysqlclient are already pulled in above via
# the postgres/mysql extras. Plain postgres/mysql obviously need nothing
# extra either. clickhouse and starrocks also need no testcontainers extra:
# ClickHouseContainer has no driver import of its own (clickhouse-connect,
# pulled in above via the clickhouse extra, is all the test needs), and
# StarRocks has no dedicated testcontainers module at all -- its test uses
# a generic DockerContainer plus the same mysqlclient the mysql extra
# already provides. databend/risingwave/firebird/ydb are the same story:
# none has a dedicated testcontainers module, so each test uses a generic
# DockerContainer plus whatever driver its own extra above already
# provides.
testcontainers[cockroachdb,cratedb,mongodb,mssql,mysql,oracle,postgres,trino]>=4.15.0,<5
+9 -156
View File
@@ -12,17 +12,10 @@
# apache-superset
aiofile==3.9.0
# via py-key-value-aio
aiohappyeyeballs==2.7.1
# via aiohttp
aiohttp==3.14.3
# via ydb
aiosignal==1.4.0
# via aiohttp
alembic==1.15.2
# via
# -c requirements/base-constraint.txt
# flask-migrate
# starrocks
amqp==5.3.1
# via
# -c requirements/base-constraint.txt
@@ -31,8 +24,6 @@ annotated-types==0.7.0
# via
# -c requirements/base-constraint.txt
# pydantic
antlr4-python3-runtime==4.13.2
# via pymongosql
anyio==4.11.0
# via
# httpx
@@ -51,12 +42,9 @@ apsw==3.50.1.0
# shillelagh
astroid==3.3.10
# via pylint
asyncmy2==0.2.21
# via starrocks
attrs==25.3.0
# via
# -c requirements/base-constraint.txt
# aiohttp
# cattrs
# cyclopts
# jsonschema
@@ -77,7 +65,6 @@ backports-tarfile==1.2.0
backports-zstd==1.6.0
# via
# -c requirements/base-constraint.txt
# clickhouse-connect
# flask-compress
bcrypt==4.3.0
# via
@@ -130,11 +117,8 @@ celery==5.6.3
certifi==2026.5.20
# via
# -c requirements/base-constraint.txt
# clickhouse-connect
# elasticsearch
# httpcore
# httpx
# opensearch-py
# requests
cffi==2.0.0
# via
@@ -176,8 +160,6 @@ click-repl==0.3.0
# via
# -c requirements/base-constraint.txt
# celery
clickhouse-connect==1.7.2
# via apache-superset
cmdstanpy==1.1.0
# via prophet
colorama==0.4.6
@@ -189,8 +171,6 @@ contourpy==1.0.7
# via matplotlib
coverage==7.6.8
# via pytest-cov
crate==2.2.1
# via sqlalchemy-cratedb
cron-descriptor==1.4.5
# via
# -c requirements/base-constraint.txt
@@ -206,20 +186,14 @@ cryptography==50.0.0
# authlib
# google-auth
# joserfc
# oracledb
# paramiko
# pyjwt
# pymysql
# pyopenssl
# secretstorage
cycler==0.12.1
# via matplotlib
cyclopts==4.2.4
# via fastmcp-slim
databend-driver==0.34.2
# via databend-sqlalchemy
databend-sqlalchemy==0.5.5
# via apache-superset
db-dtypes==1.3.1
# via pandas-gbq
defusedxml==0.7.1
@@ -242,11 +216,8 @@ dnspython==2.7.0
# via
# -c requirements/base-constraint.txt
# email-validator
# pymongo
docker==7.2.0
# via
# apache-superset
# testcontainers
docker==7.0.0
# via apache-superset
docstring-parser==0.17.0
# via cyclopts
docutils==0.22.2
@@ -257,10 +228,6 @@ duckdb==1.5.5
# duckdb-engine
duckdb-engine==0.17.0
# via apache-superset
elasticsearch==7.17.13
# via elasticsearch-dbapi
elasticsearch-dbapi==0.2.13
# via apache-superset
email-validator==2.2.0
# via
# -c requirements/base-constraint.txt
@@ -270,8 +237,6 @@ et-xmlfile==2.0.0
# via
# -c requirements/base-constraint.txt
# openpyxl
events==0.5
# via opensearch-py
exceptiongroup==1.3.0
# via fastmcp-slim
fastmcp==3.4.7
@@ -282,10 +247,6 @@ filelock==3.20.3
# via
# -c requirements/base-constraint.txt
# virtualenv
firebird-base==2.0.3
# via firebird-driver
firebird-driver==2.0.3
# via sqlalchemy-firebird
flask==2.3.3
# via
# -c requirements/base-constraint.txt
@@ -366,18 +327,12 @@ fonttools==4.60.2
# via matplotlib
freezegun==1.5.1
# via apache-superset
frozenlist==1.8.0
# via
# aiohttp
# aiosignal
future==1.0.0
# via pyhive
geographiclib==2.0
# via
# -c requirements/base-constraint.txt
# geopy
geojson==3.3.0
# via sqlalchemy-cratedb
geopy==2.4.1
# via
# -c requirements/base-constraint.txt
@@ -420,7 +375,7 @@ googleapis-common-protos==1.66.0
# via
# google-api-core
# grpcio-status
greenlet==3.5.5
greenlet==3.5.4
# via
# -c requirements/base-constraint.txt
# apache-superset
@@ -434,7 +389,6 @@ grpcio==1.83.0
# apache-superset
# google-api-core
# grpcio-status
# ydb
grpcio-status==1.60.1
# via google-api-core
gunicorn==26.2.0
@@ -460,7 +414,6 @@ httpx==0.28.1
# via
# fastmcp-slim
# mcp
# testcontainers
httpx-sse==0.4.1
# via mcp
humanize==4.12.3
@@ -477,7 +430,6 @@ idna==3.15
# httpx
# requests
# url-normalize
# yarl
importlib-metadata==8.7.0
# via
# keyring
@@ -516,7 +468,6 @@ jmespath==1.1.0
# via
# boto3
# botocore
# pymongosql
joserfc==1.7.2
# via fastmcp-slim
jsonpath-ng==1.8.0
@@ -549,8 +500,6 @@ kombu==5.6.2
# via
# -c requirements/base-constraint.txt
# celery
lark==1.3.1
# via starrocks
lazy-object-proxy==1.10.0
# via openapi-spec-validator
limits==5.1.0
@@ -558,9 +507,7 @@ limits==5.1.0
# -c requirements/base-constraint.txt
# flask-limiter
lz4==4.4.5
# via
# clickhouse-connect
# trino
# via trino
mako==1.4.1
# via
# -c requirements/base-constraint.txt
@@ -620,10 +567,6 @@ msgspec==0.19.0
# via
# -c requirements/base-constraint.txt
# flask-session
multidict==6.7.1
# via
# aiohttp
# yarl
mysqlclient==2.2.8
# via apache-superset
nh3==0.3.6
@@ -662,22 +605,14 @@ openpyxl==3.1.5
# via
# -c requirements/base-constraint.txt
# pandas
opensearch-py==2.8.0
# via elasticsearch-dbapi
opentelemetry-api==1.39.1
# via fastmcp-slim
oracledb==4.0.2
# via
# apache-superset
# testcontainers
ordered-set==4.1.0
# via
# -c requirements/base-constraint.txt
# flask-limiter
orjson==3.11.9
# via
# crate
# trino
# via trino
packaging==25.0
# via
# -c requirements/base-constraint.txt
@@ -685,8 +620,8 @@ packaging==25.0
# apispec
# db-dtypes
# deprecation
# docker
# duckdb-engine
# elasticsearch-dbapi
# fastmcp-slim
# google-cloud-bigquery
# kombu
@@ -696,8 +631,6 @@ packaging==25.0
# pytest
# shillelagh
# sqlalchemy-bigquery
# sqlalchemy-firebird
# ydb
pandas==2.3.3
# via
# -c requirements/base-constraint.txt
@@ -730,7 +663,7 @@ pillow==12.3.0
# -c requirements/base-constraint.txt
# apache-superset
# matplotlib
pip==26.2.1
pip==25.1.1
# via apache-superset
platformdirs==4.3.8
# via
@@ -759,22 +692,16 @@ prompt-toolkit==3.0.51
# via
# -c requirements/base-constraint.txt
# click-repl
propcache==0.5.2
# via
# aiohttp
# yarl
prophet==1.4.0
# via apache-superset
proto-plus==1.25.0
# via google-api-core
protobuf==5.29.6
# via
# firebird-base
# google-api-core
# googleapis-common-protos
# grpcio-status
# proto-plus
# ydb
psutil==6.1.0
# via
# apache-superset
@@ -848,24 +775,6 @@ pyjwt==2.13.0
# mcp
pylint==3.3.7
# via apache-superset
pymonetdb==1.9.1
# via
# apache-superset
# sqlalchemy-monetdb
pymongo==4.17.0
# via
# pymongosql
# testcontainers
pymongosql==0.7.3
# via apache-superset
pymssql==2.3.13
# via
# apache-superset
# testcontainers
pymysql==1.2.0
# via
# starrocks
# testcontainers
pynacl==1.6.2
# via
# -c requirements/base-constraint.txt
@@ -910,13 +819,11 @@ python-dateutil==2.9.0.post0
# botocore
# celery
# croniter
# firebird-driver
# flask-appbuilder
# freezegun
# google-cloud-bigquery
# holidays
# matplotlib
# opensearch-py
# pandas
# pyhive
# shillelagh
@@ -927,7 +834,6 @@ python-dotenv==1.2.2
# apache-superset
# fastmcp-slim
# pydantic-settings
# testcontainers
python-ldap==3.4.7
# via apache-superset
python-multipart==0.0.29
@@ -971,7 +877,6 @@ requests==2.33.0
# google-api-core
# google-cloud-bigquery
# jsonschema-path
# opensearch-py
# pydruid
# pyhive
# requests-cache
@@ -1015,7 +920,7 @@ secretstorage==3.5.0
# via keyring
semver==3.0.4
# via apache-superset-extensions-cli
setuptools==84.0.0
setuptools==80.9.0
# via
# -c requirements/base-constraint.txt
# nodeenv
@@ -1050,9 +955,7 @@ sqlalchemy==2.0.52
# alembic
# apache-superset
# apache-superset-core
# databend-sqlalchemy
# duckdb-engine
# elasticsearch-dbapi
# flask-appbuilder
# flask-sqlalchemy
# marshmallow-sqlalchemy
@@ -1060,14 +963,7 @@ sqlalchemy==2.0.52
# sqlalchemy-bigquery
# sqlalchemy-cockroachdb
# sqlalchemy-continuum
# sqlalchemy-cratedb
# sqlalchemy-firebird
# sqlalchemy-monetdb
# sqlalchemy-risingwave
# sqlalchemy-utils
# starrocks
# testcontainers
# ydb-sqlalchemy
sqlalchemy-bigquery==1.17.2
# via apache-superset
sqlalchemy-cockroachdb==2.0.4
@@ -1076,16 +972,6 @@ sqlalchemy-continuum==1.7.0
# via
# -c requirements/base-constraint.txt
# apache-superset
sqlalchemy-cratedb==0.43.1
# via
# apache-superset
# testcontainers
sqlalchemy-firebird==2.2.0
# via apache-superset
sqlalchemy-monetdb==2.1.0
# via apache-superset
sqlalchemy-risingwave==2.1.0
# via apache-superset
sqlalchemy-utils==0.42.1
# via
# -c requirements/base-constraint.txt
@@ -1097,7 +983,6 @@ sqlglot==30.17.0
# -c requirements/base-constraint.txt
# apache-superset
# apache-superset-core
# ydb-sqlglot-plugin
sqloxide==0.1.51
# via apache-superset
sse-starlette==3.0.2
@@ -1110,8 +995,6 @@ starlette==1.3.1
# via
# fastmcp-slim
# mcp
starrocks==1.3.4
# via apache-superset
statsd==4.0.1
# via apache-superset
syntaqlite==0.9.0
@@ -1120,8 +1003,6 @@ tabulate==0.10.0
# via
# -c requirements/base-constraint.txt
# apache-superset
testcontainers==4.15.0
# via -r requirements/development.in
tiktoken==0.14.0
# via apache-superset
tomli-w==1.2.0
@@ -1133,14 +1014,10 @@ tqdm==4.67.1
# cmdstanpy
# prophet
trino==0.339.0
# via
# apache-superset
# testcontainers
# via apache-superset
typing-extensions==4.16.0
# via
# -c requirements/base-constraint.txt
# aiohttp
# aiosignal
# alembic
# anyio
# apache-superset
@@ -1152,7 +1029,6 @@ typing-extensions==4.16.0
# limits
# mcp
# opentelemetry-api
# oracledb
# py-key-value-aio
# pydantic
# pydantic-core
@@ -1161,7 +1037,6 @@ typing-extensions==4.16.0
# shillelagh
# sqlalchemy
# starlette
# testcontainers
# typing-inspection
typing-inspection==0.4.2
# via
@@ -1189,22 +1064,13 @@ urllib3==2.7.0
# via
# -c requirements/base-constraint.txt
# botocore
# clickhouse-connect
# crate
# docker
# elasticsearch
# opensearch-py
# requests
# requests-cache
# testcontainers
uvicorn==0.37.0
# via
# fastmcp-slim
# mcp
verlib2==0.3.2
# via
# crate
# sqlalchemy-cratedb
vine==5.1.0
# via
# -c requirements/base-constraint.txt
@@ -1238,7 +1104,6 @@ wrapt==1.17.2
# via
# -c requirements/base-constraint.txt
# deprecated
# testcontainers
wtforms==3.2.2
# via
# -c requirements/base-constraint.txt
@@ -1259,18 +1124,6 @@ xlsxwriter==3.2.9
# -c requirements/base-constraint.txt
# apache-superset
# pandas
yarl==1.24.5
# via aiohttp
ydb==3.31.4
# via
# ydb-dbapi
# ydb-sqlalchemy
ydb-dbapi==0.1.23
# via ydb-sqlalchemy
ydb-sqlalchemy==0.1.22
# via apache-superset
ydb-sqlglot-plugin==0.2.8
# via apache-superset
zipp==3.23.0
# via importlib-metadata
zope-event==5.0
@@ -390,3 +390,19 @@ def get_session() -> scoped_session:
:returns: The SQLAlchemy scoped session instance.
"""
raise NotImplementedError("Function will be replaced during initialization")
__all__ = [
"Dataset",
"Database",
"Chart",
"Dashboard",
"User",
"Role",
"Group",
"Tag",
"KeyValue",
"Subject",
"CoreModel",
"get_session",
]
@@ -183,3 +183,10 @@ def prompt(
"MCP prompt decorator not initialized. "
"This decorator should be replaced during Superset startup."
)
__all__ = [
"tool",
"prompt",
"ToolAnnotations",
]
@@ -55,3 +55,9 @@ class SavedQueryDAO(BaseDAO[SavedQuery]):
model_cls = None
base_filter = None
id_column_name = "id"
__all__ = [
"QueryDAO",
"SavedQueryDAO",
]
@@ -71,3 +71,9 @@ class SavedQuery(CoreModel):
database_id: int | None
description: str | None
user_id: int | None
__all__ = [
"Query",
"SavedQuery",
]
@@ -46,3 +46,6 @@ def get_sqlglot_dialect(database: "Database") -> Dialects:
:returns: The SQLGlot dialect enum corresponding to the database.
"""
raise NotImplementedError("Function will be replaced during initialization")
__all__ = ["get_sqlglot_dialect"]
@@ -165,3 +165,13 @@ class AsyncQueryHandle:
:returns: True if cancellation was successful
"""
raise NotImplementedError("Method will be replaced during initialization")
__all__ = [
"QueryStatus",
"QueryOptions",
"QueryResult",
"StatementResult",
"AsyncQueryHandle",
"CacheOptions",
]
@@ -27,3 +27,6 @@ class RestApi(BaseApi):
"""
allow_browser_login = True
__all__ = ["RestApi"]
@@ -98,3 +98,6 @@ def api(
"API decorator not initialized. "
"This decorator should be replaced during Superset startup."
)
__all__ = ["api"]
@@ -164,3 +164,6 @@ class AbstractSemanticViewDAO(BaseDAO[SemanticViewModel]):
:return: SemanticViewModel instance or None
"""
...
__all__ = ["AbstractSemanticLayerDAO", "AbstractSemanticViewDAO"]
@@ -97,3 +97,6 @@ def semantic_layer(
"Semantic layer decorator not initialized. "
"This decorator should be replaced during Superset startup."
)
__all__ = ["semantic_layer"]
@@ -80,3 +80,6 @@ class SemanticViewModel(CoreModel):
semantic_layer_uuid: UUID
created_on: datetime | None
changed_on: datetime | None
__all__ = ["SemanticLayerModel", "SemanticViewModel"]
@@ -71,3 +71,6 @@ class TaskDAO(BaseDAO[Task]):
:returns: Task instance or None if not found or not active
"""
...
__all__ = ["TaskDAO"]
@@ -144,3 +144,9 @@ def get_context() -> TaskContext:
)
"""
raise NotImplementedError("Function will be replaced during initialization")
__all__ = [
"task",
"get_context",
]
@@ -161,3 +161,9 @@ class TaskSubscriber(CoreModel):
changed_on: datetime | None
created_by_fk: int | None
changed_by_fk: int | None
__all__ = [
"Task",
"TaskSubscriber",
]
@@ -226,3 +226,12 @@ class TaskContext(ABC):
cleanup_partial_work()
"""
...
__all__ = [
"TaskStatus",
"TaskScope",
"TaskProperties",
"TaskContext",
"TaskOptions",
]
+103 -54
View File
@@ -2086,6 +2086,14 @@
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/argparse": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
"integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
"dependencies": {
"sprintf-js": "~1.0.2"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
@@ -2099,6 +2107,18 @@
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/js-yaml": {
"version": "3.15.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"dependencies": {
"argparse": "^1.0.7",
"esprima": "^4.0.0"
},
"bin": {
"js-yaml": "bin/js-yaml.js"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
@@ -3195,9 +3215,9 @@
"integrity": "sha512-Fc8Ne62jJlKHiG/ajlonC4Sd66Pq68fFwK4ihJGNZpGqboc324SQk+lRvMzpPRuJOmfrJefdG8/7JdWX4bzJ2Q=="
},
"node_modules/brace-expansion": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz",
"integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==",
"dev": true,
"license": "MIT",
"peer": true,
@@ -3814,13 +3834,9 @@
}
},
"node_modules/d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-1.4.1.tgz",
"integrity": "sha512-p2sTHSLCJI2QKunbGb7ocOh7DgTAn8IrLx21QRc/BSnodXM4sv6aLQlnfpvehFMLZEfBc6g9pH9SWQccFYfJ9Q=="
},
"node_modules/d3-format": {
"version": "1.4.5",
@@ -4301,6 +4317,18 @@
"url": "https://opencollective.com/eslint"
}
},
"node_modules/esprima": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
"integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==",
"bin": {
"esparse": "bin/esparse.js",
"esvalidate": "bin/esvalidate.js"
},
"engines": {
"node": ">=4"
}
},
"node_modules/esquery": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/esquery/-/esquery-1.7.0.tgz",
@@ -4826,9 +4854,9 @@
"license": "MIT"
},
"node_modules/glob/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
@@ -5558,19 +5586,9 @@
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="
},
"node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/puzrin"
},
{
"type": "github",
"url": "https://github.com/sponsors/nodeca"
}
],
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
@@ -7830,6 +7848,11 @@
"node": ">=8"
}
},
"node_modules/sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="
},
"node_modules/sshpk": {
"version": "1.18.0",
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz",
@@ -7992,9 +8015,9 @@
"license": "MIT"
},
"node_modules/test-exclude/node_modules/brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
@@ -9819,7 +9842,7 @@
"debug": "4.4.0",
"execa": "4.1.0",
"istanbul-lib-coverage": "^3.0.0",
"js-yaml": "4.3.1",
"js-yaml": "4.1.1",
"nyc": "15.1.0",
"tinyglobby": "^0.2.14"
},
@@ -10098,7 +10121,7 @@
"requires": {
"@eslint/object-schema": "^3.0.5",
"debug": "^4.3.1",
"minimatch": ">=10"
"minimatch": "^10.2.4"
}
},
"@eslint/config-helpers": {
@@ -10190,10 +10213,18 @@
"camelcase": "^5.3.1",
"find-up": "^4.1.0",
"get-package-type": "^0.1.0",
"js-yaml": "4.3.1",
"js-yaml": "4.1.1",
"resolve-from": "^5.0.0"
},
"dependencies": {
"argparse": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
"integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
"requires": {
"sprintf-js": "~1.0.2"
}
},
"find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
@@ -10203,6 +10234,14 @@
"path-exists": "^4.0.0"
}
},
"js-yaml": {
"version": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"requires": {
"argparse": "^1.0.7",
"esprima": "^4.0.0"
}
},
"locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
@@ -11146,9 +11185,9 @@
"integrity": "sha512-Fc8Ne62jJlKHiG/ajlonC4Sd66Pq68fFwK4ihJGNZpGqboc324SQk+lRvMzpPRuJOmfrJefdG8/7JdWX4bzJ2Q=="
},
"brace-expansion": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz",
"integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==",
"dev": true,
"peer": true,
"requires": {
@@ -11591,9 +11630,9 @@
}
},
"d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA=="
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-1.4.1.tgz",
"integrity": "sha512-p2sTHSLCJI2QKunbGb7ocOh7DgTAn8IrLx21QRc/BSnodXM4sv6aLQlnfpvehFMLZEfBc6g9pH9SWQccFYfJ9Q=="
},
"d3-format": {
"version": "1.4.5",
@@ -11605,7 +11644,7 @@
"resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-1.4.0.tgz",
"integrity": "sha512-V9znK0zc3jOPV4VD2zZn0sDhZU3WAE2bmlxdIwwQPPzPjvyLkd8B3JUVdS1IDUFDkWZ72c9qnv1GK2ZagTZ8EA==",
"requires": {
"d3-color": "3.1.0"
"d3-color": "1"
}
},
"d3-scale": {
@@ -11853,7 +11892,7 @@
"imurmurhash": "^0.1.4",
"is-glob": "^4.0.0",
"json-stable-stringify-without-jsonify": "^1.0.1",
"minimatch": ">=10",
"minimatch": "^10.2.4",
"natural-compare": "^1.4.0",
"optionator": "^0.9.3"
},
@@ -11942,6 +11981,11 @@
"eslint-visitor-keys": "^5.0.1"
}
},
"esprima": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
"integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A=="
},
"esquery": {
"version": "1.7.0",
"resolved": "https://registry.npmjs.org/esquery/-/esquery-1.7.0.tgz",
@@ -12290,7 +12334,7 @@
"fs.realpath": "^1.0.0",
"inflight": "^1.0.4",
"inherits": "2",
"minimatch": "<10",
"minimatch": "^3.1.1",
"once": "^1.3.0",
"path-is-absolute": "^1.0.0"
},
@@ -12301,9 +12345,9 @@
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="
},
"brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -12314,7 +12358,7 @@
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"requires": {
"brace-expansion": "1.1.18"
"brace-expansion": "^1.1.7"
}
}
}
@@ -12808,9 +12852,9 @@
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="
},
"js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"requires": {
"argparse": "^2.0.1"
}
@@ -13391,7 +13435,7 @@
"dev": true,
"peer": true,
"requires": {
"brace-expansion": ">=5.0.9"
"brace-expansion": "^5.0.5"
}
},
"minimist": {
@@ -14339,6 +14383,11 @@
"which": "^2.0.1"
}
},
"sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="
},
"sshpk": {
"version": "1.18.0",
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz",
@@ -14445,7 +14494,7 @@
"requires": {
"@istanbuljs/schema": "^0.1.2",
"glob": "^7.1.4",
"minimatch": "<10"
"minimatch": "^3.0.4"
},
"dependencies": {
"balanced-match": {
@@ -14454,9 +14503,9 @@
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="
},
"brace-expansion": {
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"version": "1.1.16",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.16.tgz",
"integrity": "sha512-IDw48K2/2kRkg9LdJxurvq3lV3aBgq0REY89duEqFRthjlPdXHKMj7EnQOXVckxzgisinf3nHfrcE2FufFLXMw==",
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -14467,7 +14516,7 @@
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"requires": {
"brace-expansion": "1.1.18"
"brace-expansion": "^1.1.7"
}
}
}
+2 -8
View File
@@ -30,20 +30,14 @@
"overrides": {
"@babel/core": "^7.29.6",
"@cypress/code-coverage": {
"js-yaml": "4.3.1"
"js-yaml": "4.1.1"
},
"@cypress/request": "^3.0.0",
"cypress": {
"form-data": "^2.3.4"
},
"d3-interpolate": {
"d3-color": "3.1.0"
},
"minimatch@<10": {
"brace-expansion": "1.1.18"
},
"minimatch@>=10": {
"brace-expansion": ">=5.0.9"
"brace-expansion": ">=5.0.8"
},
"qs": "^6.14.2",
"uuid": "^11.1.1"
+1 -1
View File
@@ -77,7 +77,7 @@ module.exports = {
// @ant-design/colors and @ant-design/fast-color are allowed through because
// @ant-design/icons >= 6.3 deep-imports the ESM build of @ant-design/colors
// from its CJS output, so babel-jest must transform those files.
'node_modules/(?!@ant-design/(colors|fast-color)|@formatjs/.*|d3-(array|interpolate|color|time|scale|time-format|format|selection)|internmap|@mapbox/tiny-sdf|remark-gfm|(?!@ngrx|(?!deck.gl)|d3-scale)|markdown-table|micromark-*.|decode-named-character-reference|character-entities|mdast-util-*.|unist-util-*.|ccount|escape-string-regexp|nanoid|uuid|@rjsf/*.|@x0k/.*|echarts|zrender|fetch-mock|pretty-ms|parse-ms|ol|@babel/runtime|@emotion|cheerio|cheerio/lib|parse5|dom-serializer|entities|htmlparser2|rehype-sanitize|hast-util-sanitize|unified|unist-.*|hast-.*|hastscript|refractor|rehype-.*|remark-.*|mdast-.*|micromark-.*|parse-entities|character-reference-invalid|is-alphanumerical|is-alphabetical|is-decimal|is-hexadecimal|property-information|space-separated-tokens|comma-separated-tokens|bail|devlop|zwitch|longest-streak|geostyler|geostyler-.*|(?!geostyler)lodash|react-error-boundary|react-json-tree|react-base16-styling|lodash-es|rbush|quickselect|react-diff-viewer-continued|storybook/*.|json-stringify-pretty-compact|@x0k/json-schema-merge|content-disposition)',
'node_modules/(?!@ant-design/(colors|fast-color)|@formatjs/.*|d3-(array|interpolate|color|time|scale|time-format|format|selection)|internmap|@mapbox/tiny-sdf|remark-gfm|(?!@ngrx|(?!deck.gl)|d3-scale)|markdown-table|micromark-*.|decode-named-character-reference|character-entities|mdast-util-*.|unist-util-*.|ccount|escape-string-regexp|nanoid|uuid|@rjsf/*.|@x0k/.*|echarts|zrender|fetch-mock|pretty-ms|parse-ms|ol|@babel/runtime|@emotion|cheerio|cheerio/lib|parse5|dom-serializer|entities|htmlparser2|rehype-sanitize|hast-util-sanitize|unified|unist-.*|hast-.*|hastscript|refractor|rehype-.*|remark-.*|mdast-.*|micromark-.*|parse-entities|character-reference-invalid|is-alphanumerical|is-alphabetical|is-decimal|is-hexadecimal|property-information|space-separated-tokens|comma-separated-tokens|bail|devlop|zwitch|longest-streak|geostyler|geostyler-.*|(?!geostyler)lodash|react-error-boundary|react-json-tree|react-base16-styling|lodash-es|rbush|quickselect|react-diff-viewer-continued|storybook/*.|json-stringify-pretty-compact|@x0k/json-schema-merge)',
],
preset: 'ts-jest',
transform: {
+35 -35
View File
@@ -84,7 +84,7 @@
"antd": "^6.6.1",
"chrono-node": "^2.10.1",
"classnames": "^2.2.5",
"content-disposition": "^3.0.0",
"content-disposition": "^2.0.1",
"d3-scale": "^4.0.2",
"dayjs": "^1.11.23",
"dom-to-image-more": "^3.10.2",
@@ -218,13 +218,13 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"cheerio": "1.2.0",
"concurrently": "^10.0.5",
"copy-webpack-plugin": "^14.0.0",
"cross-env": "^10.1.0",
"css-loader": "^7.1.4",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-import-resolver-alias": "^1.1.2",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-i18n-strings": "file:eslint-rules/eslint-plugin-i18n-strings",
@@ -15753,9 +15753,9 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.11.17",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.17.tgz",
"integrity": "sha512-KAUDn1OSS0fmPlGO+NOUMRcOQ/b/shUBH3OgkG73mPgdf+JD/BQ6fHboGxNOxnUmlwcq+lLq3dTkayRPuSfXwg==",
"version": "2.11.16",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.16.tgz",
"integrity": "sha512-H/bNPUFHewJHyCTdjn1n3Pit5+2GmWT6mmeHImPX+8MA9NA6b67jO4gYmi4jTbCJb2otq34KMZnovndDPqJwhQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
@@ -17451,12 +17451,12 @@
}
},
"node_modules/content-disposition": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-3.0.0.tgz",
"integrity": "sha512-ZH/0Xs9rMIFWCOmGdmS9eHBTF62qqQYNz4nVjQhkdIO/a0fCP4UIM3mRz/wiqL0L14YgAz/1xio4OaSY4+ON/A==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-2.0.1.tgz",
"integrity": "sha512-e+H0ZXHSWYrENhQzw1LPuP4oF5MzVKmDU6d3hxlvaPEYLLg62MxtQNPRx4SYSuYJSBUgnQIG4HIN2tEtNv7Dog==",
"license": "MIT",
"engines": {
"node": ">=22"
"node": ">=18"
},
"funding": {
"type": "opencollective",
@@ -17804,9 +17804,9 @@
"license": "Python-2.0"
},
"node_modules/cosmiconfig/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -19858,9 +19858,9 @@
}
},
"node_modules/eslint": {
"version": "10.9.0",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.9.0.tgz",
"integrity": "sha512-5KeEOJZBfEVA47boFiBsf+6MmmJpffM7qEBg4pLla2e4nlKgdKlqCW0oSLOGsT8Wl5uCGJptLV1bkaiShj90Gw==",
"version": "10.8.1",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.8.1.tgz",
"integrity": "sha512-wqA7W2jbsC/BnV9Iv1UZpKVFkO1AdNoSmYW8NWG4HNOBbkAMvIqDZ27pI2f07dqn583NcIC44ckjAcOXDL1QbQ==",
"dev": true,
"license": "MIT",
"workspaces": [
@@ -26738,9 +26738,9 @@
"license": "MIT"
},
"node_modules/js-yaml": {
"version": "3.15.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.1.tgz",
"integrity": "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag==",
"version": "3.15.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.0.tgz",
"integrity": "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -27662,9 +27662,9 @@
}
},
"node_modules/lerna/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -30733,9 +30733,9 @@
}
},
"node_modules/nwsapi": {
"version": "2.2.24",
"resolved": "https://registry.npmjs.org/nwsapi/-/nwsapi-2.2.24.tgz",
"integrity": "sha512-7YRhZ3jS45LwmSCT4b2sVFHt/WuovaktDU07QrtOBY2PXskss5a9jfmR9jptyumwXST+rFjrmppMY1KT/yn35A==",
"version": "2.2.23",
"resolved": "https://registry.npmjs.org/nwsapi/-/nwsapi-2.2.23.tgz",
"integrity": "sha512-7wfH4sLbt4M0gCDzGE6vzQBo0bfTKjU7Sfpqy/7gs1qBfYz2vEJH6vXcBKpO3+6Yu1telwd0t9HpyOoLEQQbIQ==",
"dev": true,
"license": "MIT"
},
@@ -34750,9 +34750,9 @@
}
},
"node_modules/react-diff-viewer-continued/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"funding": [
{
"type": "github",
@@ -36063,9 +36063,9 @@
"license": "MIT"
},
"node_modules/reselect": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.3.0.tgz",
"integrity": "sha512-XGoLeRAVzUTcJ1qkxPQhDJyIZ5d6zzZD9nT7AEZOaaU9UbWclhycElmhO+VD5bFeLuzhPBaOV2oXC8uG35ZSpg==",
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.2.0.tgz",
"integrity": "sha512-AgZ3UOZm3YndfrJ4OYjgrT7bmCm/1iqkjvEfH/oYjzh6PD2qw4QuT3jjnXIrpdt4MTpMXclMT3lXbmRY+XRakw==",
"license": "MIT"
},
"node_modules/resize-observer-polyfill": {
@@ -38445,9 +38445,9 @@
"license": "ISC"
},
"node_modules/stylelint/node_modules/js-yaml": {
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz",
"integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==",
"dev": true,
"funding": [
{
@@ -43010,7 +43010,7 @@
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"rison": "^0.1.1",
"seedrandom": "^3.0.5",
"xss": "^1.0.15"
+5 -17
View File
@@ -161,7 +161,7 @@
"antd": "^6.6.1",
"chrono-node": "^2.10.1",
"classnames": "^2.2.5",
"content-disposition": "^3.0.0",
"content-disposition": "^2.0.1",
"d3-scale": "^4.0.2",
"dayjs": "^1.11.23",
"dom-to-image-more": "^3.10.2",
@@ -295,13 +295,13 @@
"babel-loader": "^10.1.1",
"babel-plugin-dynamic-import-node": "^2.3.3",
"babel-plugin-jsx-remove-data-test-id": "^3.0.0",
"baseline-browser-mapping": "^2.11.17",
"baseline-browser-mapping": "^2.11.16",
"cheerio": "1.2.0",
"concurrently": "^10.0.5",
"copy-webpack-plugin": "^14.0.0",
"cross-env": "^10.1.0",
"css-loader": "^7.1.4",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-import-resolver-alias": "^1.1.2",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-i18n-strings": "file:eslint-rules/eslint-plugin-i18n-strings",
@@ -383,9 +383,6 @@
"@great-expectations/jsonforms-antd-renderers": {
"antd": "$antd"
},
"@istanbuljs/load-nyc-config": {
"js-yaml": "^3.15.1"
},
"@jest/globals": "^30.4.0",
"@jest/types": "^30.4.0",
"@luma.gl/constants": "~9.2.5",
@@ -395,9 +392,6 @@
"@luma.gl/shadertools": "~9.2.5",
"@luma.gl/webgl": "~9.2.5",
"core-js": "^3.38.1",
"cosmiconfig": {
"js-yaml": "^4.3.1"
},
"dompurify": "^3.4.13",
"esbuild": "^0.28.1",
"eslint-plugin-import": {
@@ -414,22 +408,16 @@
"jest-mock": "^30.4.0",
"jest-runtime": "^30.4.0",
"jest-util": "^30.4.0",
"js-yaml-loader": {
"js-yaml": "^3.15.1"
},
"jspdf": "^4.2.0",
"lerna": {
"js-yaml": "^4.3.1"
"js-yaml": "^4.3.0"
},
"minimatch@>=10": {
"brace-expansion": ">=5.0.8"
},
"nanoid@>=3 <4": "3.3.18",
"nwsapi": "^2.2.24",
"nwsapi": "^2.2.13",
"puppeteer": "^22.4.1",
"react-diff-viewer-continued": {
"js-yaml": "^4.3.1"
},
"tar": "^7.5.16",
"typescript-json-schema": "^0.68.0",
"underscore": "^1.13.7",
@@ -89,7 +89,7 @@
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-gfm": "^4.0.1",
"reselect": "^5.3.0",
"reselect": "^5.2.0",
"rison": "^0.1.1",
"seedrandom": "^3.0.5",
"xss": "^1.0.15"
@@ -25,7 +25,6 @@ export enum VizType {
BoxPlot = 'box_plot',
Bubble = 'bubble_v2',
Bullet = 'bullet',
Butterfly = 'butterfly',
Calendar = 'cal_heatmap',
Cartodiagram = 'cartodiagram',
Chord = 'chord',
@@ -30,7 +30,6 @@ import type {
QueryFormData,
} from '../query';
import type { JsonResponse } from '../connection';
import type { MenuItem } from '../components/Menu';
/**
* A function which returns text (or marked-up text)
@@ -165,13 +164,6 @@ export interface SliceHeaderExtension {
dashboardId: number;
}
/**
* Interface for extensions to the Slice Header more-options menu
*/
export interface SliceHeaderMenuExtension extends SliceHeaderExtension {
sliceName: string;
}
/**
* Interface for extensions to Embed Modal
*/
@@ -270,9 +262,6 @@ export type Extensions = Partial<{
'sqleditor.extension.form': ComponentType<SQLFormExtensionProps>;
'sqleditor.extension.resultTable': ComponentType<SQLResultTableExtensionProps>;
'dashboard.slice.header': ComponentType<SliceHeaderExtension>;
'dashboard.slice.header.menu': (
context: SliceHeaderMenuExtension,
) => MenuItem[];
'sqleditor.extension.customAutocomplete': (
args: CustomAutoCompleteArgs,
) => CustomAutocomplete[] | undefined;
@@ -1,49 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import Echart from '../components/Echart';
import { ButterflyTransformedProps } from './types';
import { EventHandlers } from '../types';
export default function Butterfly(props: ButterflyTransformedProps) {
const { height, width, echartOptions, refs, onLegendStateChanged, formData } =
props;
const eventHandlers: EventHandlers = {
legendselectchanged: payload => {
onLegendStateChanged?.(payload.selected);
},
legendselectall: payload => {
onLegendStateChanged?.(payload.selected);
},
legendinverseselect: payload => {
onLegendStateChanged?.(payload.selected);
},
};
return (
<Echart
refs={refs}
height={height}
width={width}
echartOptions={echartOptions}
eventHandlers={eventHandlers}
vizType={formData.vizType}
/>
);
}
@@ -1,52 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
buildQueryContext,
ensureIsArray,
QueryFormData,
QueryFormOrderBy,
} from '@superset-ui/core';
import { buildSortMetricOrderby } from '@superset-ui/chart-controls';
export default function buildQuery(formData: QueryFormData) {
const columns = ensureIsArray(formData.groupby);
const baseMetrics = [
...ensureIsArray(formData.left_metric),
...ensureIsArray(formData.right_metric),
];
const { orderby, metrics } = buildSortMetricOrderby({
metrics: baseMetrics,
timeseriesLimitMetric: ensureIsArray(formData.orderby)[0],
order_desc: formData.order_desc,
});
const resolvedOrderby: QueryFormOrderBy[] | undefined = orderby.length
? orderby
: columns.length
? [[columns[0], true]]
: undefined;
return buildQueryContext(formData, baseQueryObject => [
{
...baseQueryObject,
columns,
metrics,
orderby: resolvedOrderby,
},
]);
}
@@ -1,29 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
DEFAULT_LEGEND_FORM_DATA,
DEFAULT_TITLE_FORM_DATA,
} from '../constants';
import { defaultXAxis } from '../defaults';
export const DEFAULT_FORM_DATA = {
...DEFAULT_LEGEND_FORM_DATA,
...DEFAULT_TITLE_FORM_DATA,
xAxisLabelRotation: defaultXAxis.xAxisLabelRotation,
};
@@ -1,242 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { ensureIsArray } from '@superset-ui/core';
import {
ControlPanelConfig,
ControlSubSectionHeader,
formatSelectOptions,
getStandardizedControls,
sections,
sharedControls,
} from '@superset-ui/chart-controls';
import {
legendSection,
showValueControl,
xAxisLabelRotation,
} from '../controls';
import { DEFAULT_FORM_DATA } from './constants';
const { xAxisTitleMargin, yAxisTitleMargin } = DEFAULT_FORM_DATA;
const config: ControlPanelConfig = {
controlPanelSections: [
{
label: t('Query'),
expanded: true,
controlSetRows: [
['groupby'],
[
{
name: 'left_metric',
config: {
...sharedControls.metric,
label: t('Left metric'),
description: t(
'Metric displayed on the left side of the butterfly chart',
),
},
},
],
[
{
name: 'right_metric',
config: {
...sharedControls.metric,
label: t('Right metric'),
description: t(
'Metric displayed on the right side of the butterfly chart',
),
},
},
],
['adhoc_filters'],
['row_limit'],
['orderby'],
[
{
name: 'order_desc',
config: {
...sharedControls.order_desc,
visibility: ({ controls }) => Boolean(controls.orderby.value),
},
},
],
],
},
{
label: t('Chart Options'),
expanded: true,
controlSetRows: [[showValueControl], ...legendSection],
},
{
label: t('Series settings'),
expanded: true,
controlSetRows: [
[
<ControlSubSectionHeader>
{t('Left series setting')}
</ControlSubSectionHeader>,
],
[
{
name: 'left_color',
config: {
label: t('Left color'),
type: 'ColorPickerControl',
default: { r: 84, g: 112, b: 198, a: 1 },
renderTrigger: true,
description: t('Color for bars on the left side of the chart'),
},
},
{
name: 'left_label',
config: {
label: t('Left label'),
type: 'TextControl',
renderTrigger: true,
description: t(
'Customize the label for the left series in tooltips and legend',
),
},
},
],
[
<ControlSubSectionHeader>
{t('Right series setting')}
</ControlSubSectionHeader>,
],
[
{
name: 'right_color',
config: {
label: t('Right color'),
type: 'ColorPickerControl',
default: { r: 145, g: 204, b: 117, a: 1 },
renderTrigger: true,
description: t('Color for bars on the right side of the chart'),
},
},
{
name: 'right_label',
config: {
label: t('Right label'),
type: 'TextControl',
renderTrigger: true,
description: t(
'Customize the label for the right series in tooltips and legend',
),
},
},
],
],
},
{
label: t('X Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'x_axis_label',
config: {
type: 'TextControl',
label: t('X Axis Label'),
renderTrigger: true,
default: '',
},
},
],
[
{
name: 'x_axis_title_margin',
config: {
type: 'SelectControl',
freeForm: true,
clearable: true,
label: t('X Axis title margin'),
renderTrigger: true,
default: xAxisTitleMargin,
choices: formatSelectOptions(sections.TITLE_MARGIN_OPTIONS),
},
},
],
['x_axis_format'],
['currency_format'],
],
},
{
label: t('Y Axis'),
expanded: true,
controlSetRows: [
[
{
name: 'y_axis_label',
config: {
type: 'TextControl',
label: t('Y Axis Label'),
renderTrigger: true,
default: '',
},
},
],
[
{
name: 'y_axis_title_margin',
config: {
type: 'SelectControl',
freeForm: true,
clearable: true,
label: t('Y Axis title margin'),
renderTrigger: true,
default: yAxisTitleMargin,
choices: formatSelectOptions(sections.TITLE_MARGIN_OPTIONS),
},
},
],
[
{
name: xAxisLabelRotation.name,
config: {
...xAxisLabelRotation.config,
label: t('Rotate category label'),
description: t(
'Input field supports custom rotation. e.g. 30 for 30°',
),
},
},
],
],
},
],
controlOverrides: {
groupby: {
label: t('Categories'),
description: t('Dimension used for category labels on the vertical axis'),
multi: false,
},
},
formDataOverrides: formData => ({
...formData,
groupby: ensureIsArray(getStandardizedControls().shiftColumn()),
left_metric: getStandardizedControls().shiftMetric(),
right_metric: getStandardizedControls().shiftMetric(),
}),
};
export default config;
@@ -1,54 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { ChartMetadata, ChartPlugin } from '@superset-ui/core';
import buildQuery from './buildQuery';
import controlPanel from './controlPanel';
import transformProps from './transformProps';
import { EchartsButterflyChartProps, EchartsButterflyFormData } from './types';
export default class EchartsButterflyChartPlugin extends ChartPlugin<
EchartsButterflyFormData,
EchartsButterflyChartProps
> {
constructor() {
super({
buildQuery,
controlPanel,
loadChart: () => import('./Butterfly'),
metadata: new ChartMetadata({
credits: ['https://echarts.apache.org'],
category: t('Comparison'),
description: t(
'A butterfly chart compares two metrics across categories using horizontal bars ' +
'that extend left and right from a central axis.',
),
name: t('Butterfly Chart'),
tags: [
t('Categorical'),
t('Comparison'),
t('ECharts'),
t('Multi-Variables'),
],
thumbnail: '',
}),
transformProps,
});
}
}
@@ -1,298 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
CurrencyFormatter,
ensureIsArray,
getColumnLabel,
getMetricLabel,
getNumberFormatter,
NumberFormatter,
rgbToHex,
tooltipHtml,
} from '@superset-ui/core';
import type { ComposeOption } from 'echarts/core';
import type { BarSeriesOption } from 'echarts/charts';
import type { CallbackDataParams } from 'echarts/types/src/util/types';
import { EchartsButterflyChartProps, ButterflyTransformedProps } from './types';
import { DEFAULT_FORM_DATA } from './constants';
import { defaultGrid } from '../defaults';
import { getDefaultTooltip } from '../utils/tooltip';
import { Refs } from '../types';
import { NULL_STRING } from '../constants';
import { getChartPadding, getLegendProps } from '../utils/series';
import { resolveLegendLayout } from '../utils/legendLayout';
import { convertInteger } from '../utils/convertInteger';
type EChartsOption = ComposeOption<BarSeriesOption>;
const LABEL_LEFT = { position: 'left' as const };
const LABEL_RIGHT = { position: 'right' as const };
function formatCategory(value: unknown): string {
if (value == null) {
return NULL_STRING;
}
if (typeof value === 'string' || typeof value === 'number') {
return String(value);
}
return String(value);
}
function formatTooltip(
params: CallbackDataParams[],
formatter: NumberFormatter | CurrencyFormatter,
) {
const axisParams = params.filter(
param => param.seriesName && typeof param.value === 'number',
);
if (!axisParams.length) {
return '';
}
const title = axisParams[0].name;
const rows = axisParams.map(param => [
param.seriesName!,
formatter(Math.abs(param.value as number)),
]);
return tooltipHtml(rows, title);
}
export default function transformProps(
chartProps: EchartsButterflyChartProps,
): ButterflyTransformedProps {
const {
width,
height,
formData,
legendState,
queriesData,
hooks,
theme,
inContextMenu,
} = chartProps;
const refs: Refs = {};
const { data = [] } = queriesData[0];
const { setDataMask = () => {}, onContextMenu, onLegendStateChanged } = hooks;
const {
currencyFormat,
groupby,
leftMetric,
rightMetric,
leftColor = { r: 84, g: 112, b: 198, a: 1 },
rightColor = { r: 145, g: 204, b: 117, a: 1 },
leftLabel,
rightLabel,
xAxisLabel,
yAxisLabel,
xAxisFormat,
xAxisTitleMargin,
yAxisTitleMargin,
showLegend,
legendMargin,
legendOrientation,
legendType,
legendSort,
showValue,
xAxisLabelRotation,
}: EchartsButterflyChartProps['formData'] = {
...DEFAULT_FORM_DATA,
...formData,
};
const groupbyColumn = ensureIsArray(groupby)[0];
const categoryLabel = getColumnLabel(groupbyColumn);
const leftMetricLabel = leftMetric ? getMetricLabel(leftMetric) : '';
const rightMetricLabel = rightMetric ? getMetricLabel(rightMetric) : '';
const leftSeriesName = leftLabel || leftMetricLabel;
const rightSeriesName = rightLabel || rightMetricLabel;
const defaultFormatter = currencyFormat?.symbol
? new CurrencyFormatter({ d3Format: xAxisFormat, currency: currencyFormat })
: getNumberFormatter(xAxisFormat);
const categories = data.map(row => formatCategory(row[categoryLabel]));
const leftData = data.map(row => {
const value = Number(row[leftMetricLabel] ?? 0);
return {
value: -Math.abs(value),
label: LABEL_LEFT,
};
});
const rightData = data.map(row => {
const value = Number(row[rightMetricLabel] ?? 0);
return {
value: Math.abs(value),
label: LABEL_RIGHT,
};
});
const labelFormatter = (params: CallbackDataParams) => {
const value = Math.abs(params.value as number);
if (value === 0) {
return '';
}
return defaultFormatter(value);
};
const series: BarSeriesOption[] = [
{
name: leftSeriesName,
type: 'bar',
stack: 'Total',
label: {
show: showValue,
formatter: labelFormatter,
color: theme.colorText,
},
itemStyle: {
color: rgbToHex(leftColor.r, leftColor.g, leftColor.b),
},
data: leftData,
},
{
name: rightSeriesName,
type: 'bar',
stack: 'Total',
label: {
show: showValue,
formatter: labelFormatter,
color: theme.colorText,
},
itemStyle: {
color: rgbToHex(rightColor.r, rightColor.g, rightColor.b),
},
data: rightData,
},
];
const legendData = [leftSeriesName, rightSeriesName].sort((a, b) => {
if (!legendSort) {
return 0;
}
return legendSort === 'asc' ? a.localeCompare(b) : b.localeCompare(a);
});
const { effectiveLegendMargin, effectiveLegendType } = resolveLegendLayout({
chartHeight: height,
chartWidth: width,
legendItems: legendData,
legendMargin,
orientation: legendOrientation,
show: showLegend,
theme,
type: legendType,
});
const legendPadding = getChartPadding(
showLegend,
legendOrientation,
effectiveLegendMargin,
undefined,
true,
);
const echartOptions: EChartsOption = {
grid: {
...defaultGrid,
top:
theme.sizeUnit * 5 +
legendPadding.top +
convertInteger(xAxisTitleMargin),
bottom: theme.sizeUnit * 5 + legendPadding.bottom,
left:
theme.sizeUnit * 5 +
legendPadding.left +
convertInteger(yAxisTitleMargin),
right: theme.sizeUnit * 5 + legendPadding.right,
},
legend: {
...getLegendProps(
effectiveLegendType,
legendOrientation,
showLegend,
theme,
false,
legendState,
),
data: legendData,
},
xAxis: {
type: 'value',
position: 'top',
name: xAxisLabel,
nameLocation: 'middle',
nameGap: convertInteger(xAxisTitleMargin),
nameTextStyle: {
padding: [theme.sizeUnit * 4, 0, 0, 0],
},
splitLine: {
lineStyle: {
type: 'dashed',
},
},
axisLabel: {
formatter: (value: number) => defaultFormatter(Math.abs(value)),
},
},
yAxis: {
type: 'category',
name: yAxisLabel,
nameLocation: 'middle',
nameGap: convertInteger(yAxisTitleMargin),
nameTextStyle: {
padding: [0, theme.sizeUnit * 4, 0, 0],
},
axisLine: { show: false },
axisTick: { show: false },
splitLine: { show: false },
axisLabel: {
rotate: xAxisLabelRotation,
},
data: categories,
},
tooltip: {
...getDefaultTooltip(refs),
appendToBody: true,
trigger: 'axis',
axisPointer: { type: 'shadow' },
show: !inContextMenu,
formatter: (params: CallbackDataParams | CallbackDataParams[]) =>
formatTooltip(
ensureIsArray(params) as CallbackDataParams[],
defaultFormatter,
),
},
series,
};
return {
refs,
formData,
width,
height,
echartOptions,
setDataMask,
onContextMenu,
onLegendStateChanged,
};
}
@@ -1,52 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import {
ChartDataResponseResult,
ChartProps,
QueryFormColumn,
QueryFormData,
QueryFormMetric,
RgbaColor,
} from '@superset-ui/core';
import { BaseTransformedProps, LegendFormData, TitleFormData } from '../types';
export type EchartsButterflyFormData = QueryFormData &
LegendFormData &
TitleFormData & {
groupby: QueryFormColumn[];
leftMetric: QueryFormMetric;
rightMetric: QueryFormMetric;
leftColor: RgbaColor;
rightColor: RgbaColor;
leftLabel?: string;
rightLabel?: string;
xAxisLabel: string;
yAxisLabel: string;
xAxisFormat: string;
showValue: boolean;
xAxisLabelRotation: number;
};
export interface EchartsButterflyChartProps extends ChartProps {
formData: EchartsButterflyFormData;
queriesData: ChartDataResponseResult[];
}
export type ButterflyTransformedProps =
BaseTransformedProps<EchartsButterflyFormData>;
@@ -1276,15 +1276,10 @@ export default function transformProps(
// at the axis boundary.
...(showMaxLabel && {
showMaxLabel: true,
alignMaxLabel: 'right',
showMinLabel: true,
alignMinLabel: 'left',
}),
// The alignments assume the axis runs along the bottom; a horizontal
// chart puts this axis on the side, where they misplace the labels.
...(showMaxLabel &&
!isHorizontal && {
alignMaxLabel: 'right',
alignMinLabel: 'left',
}),
},
minorTick: { show: minorTicks },
axisTick: { show: axisTicks ? 'auto' : false },
@@ -46,7 +46,6 @@ export { default as EchartsSunburstChartPlugin } from './Sunburst';
export { default as EchartsBubbleChartPlugin } from './Bubble';
export { default as EchartsSankeyChartPlugin } from './Sankey';
export { default as EchartsWaterfallChartPlugin } from './Waterfall';
export { default as EchartsButterflyChartPlugin } from './Butterfly';
export { default as EchartsGanttChartPlugin } from './Gantt';
export { default as BoxPlotTransformProps } from './BoxPlot/transformProps';
@@ -63,7 +62,6 @@ export { default as HeatmapTransformProps } from './Heatmap/transformProps';
export { default as SunburstTransformProps } from './Sunburst/transformProps';
export { default as BubbleTransformProps } from './Bubble/transformProps';
export { default as WaterfallTransformProps } from './Waterfall/transformProps';
export { default as ButterflyTransformProps } from './Butterfly/transformProps';
export { default as HistogramTransformProps } from './Histogram/transformProps';
export { default as SankeyTransformProps } from './Sankey/transformProps';
export { default as GanttTransformProps } from './Gantt/transformProps';
@@ -1,85 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import buildQuery from '../../src/Butterfly/buildQuery';
const formData = {
datasource: '1__table',
viz_type: 'butterfly',
groupby: ['category'],
left_metric: 'left_sum',
right_metric: 'right_sum',
};
test('defaults to ordering by the category column', () => {
const [query] = buildQuery(formData).queries;
expect(query.columns).toEqual(['category']);
expect(query.metrics).toEqual(['left_sum', 'right_sum']);
expect(query.orderby).toEqual([['category', true]]);
});
test('wraps the sort metric in a valid orderby tuple', () => {
const sortMetric = {
expressionType: 'SIMPLE',
column: { column_name: 'left_sum' },
aggregate: 'SUM',
label: 'SUM(left_sum)',
};
const [query] = buildQuery({
...formData,
orderby: sortMetric,
order_desc: true,
}).queries;
expect(query.metrics).toEqual(['left_sum', 'right_sum', sortMetric]);
expect(query.orderby).toEqual([[sortMetric, false]]);
});
test('appends the sort metric when it is not already selected', () => {
const sortMetric = {
expressionType: 'SIMPLE',
column: { column_name: 'count' },
aggregate: 'SUM',
label: 'SUM(count)',
};
const [query] = buildQuery({
...formData,
orderby: sortMetric,
order_desc: false,
}).queries;
expect(query.metrics).toEqual(['left_sum', 'right_sum', sortMetric]);
expect(query.orderby).toEqual([[sortMetric, true]]);
});
test('leaves orderby unset when no category column is selected', () => {
const [query] = buildQuery({
...formData,
groupby: [],
}).queries;
expect(query.columns).toEqual([]);
expect(query.metrics).toEqual(['left_sum', 'right_sum']);
expect(query.orderby).toBeUndefined();
});
test('issues no metrics when none are selected', () => {
const [query] = buildQuery({
datasource: '1__table',
viz_type: 'butterfly',
groupby: ['category'],
}).queries;
expect(query.metrics).toEqual([]);
});
@@ -1,83 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { SqlaFormData } from '@superset-ui/core';
const mockShiftMetric = jest
.fn()
.mockReturnValueOnce('left_sum')
.mockReturnValueOnce('right_sum');
const mockShiftColumn = jest.fn(() => 'category');
jest.mock('@superset-ui/chart-controls', () => {
const actual = jest.requireActual('@superset-ui/chart-controls');
return {
...actual,
getStandardizedControls: jest.fn(() => ({
shiftMetric: mockShiftMetric,
shiftColumn: mockShiftColumn,
})),
};
});
// eslint-disable-next-line import/first
import controlPanel from '../../src/Butterfly/controlPanel';
const collectControlNames = () => {
const names = new Set<string>();
controlPanel.controlPanelSections?.forEach(section => {
section?.controlSetRows?.forEach(row => {
row.forEach(control => {
if (typeof control === 'string') {
names.add(control);
} else if (
control &&
typeof control === 'object' &&
'name' in control
) {
names.add(String(control.name));
}
});
});
});
return names;
};
test('exposes left and right metric controls', () => {
const controlNames = collectControlNames();
expect(controlNames.has('left_metric')).toBe(true);
expect(controlNames.has('right_metric')).toBe(true);
expect(controlNames.has('groupby')).toBe(true);
expect(controlNames.has('orderby')).toBe(true);
});
test('restricts categories to a single dimension', () => {
expect(controlPanel.controlOverrides?.groupby?.multi).toBe(false);
});
test('maps standardized controls to butterfly metrics', () => {
const dummyFormData = { someProp: 'test' } as unknown as SqlaFormData;
const formData = controlPanel.formDataOverrides?.(dummyFormData);
expect(formData?.someProp).toBe('test');
expect(formData?.groupby).toEqual(['category']);
expect(formData?.left_metric).toBe('left_sum');
expect(formData?.right_metric).toBe('right_sum');
expect(mockShiftMetric).toHaveBeenCalledTimes(2);
expect(mockShiftColumn).toHaveBeenCalledTimes(1);
});
@@ -1,202 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps } from '@superset-ui/core';
import { supersetTheme } from '@apache-superset/core/theme';
import {
EchartsButterflyChartProps,
ButterflyTransformedProps,
} from '../../src/Butterfly/types';
import transformProps from '../../src/Butterfly/transformProps';
import { NULL_STRING } from '../../src/constants';
type SeriesDataPoint = { value?: number } | number;
type ButterflyTestSeries = {
name?: string;
data?: SeriesDataPoint[];
itemStyle?: { color?: string };
label?: { show?: boolean };
};
type ButterflyTestEchartOptions = {
series?: ButterflyTestSeries[];
xAxis?: { name?: string; nameGap?: number };
yAxis?: {
name?: string;
nameGap?: number;
data?: string[];
axisLabel?: { rotate?: number };
};
legend?: { orient?: string; data?: string[] };
grid?: { left?: number; top?: number };
tooltip?: { show?: boolean };
};
const getEchartOptions = (
props: ButterflyTransformedProps,
): ButterflyTestEchartOptions =>
props.echartOptions as ButterflyTestEchartOptions;
const extractSeriesValues = (props: ButterflyTransformedProps) => {
const series = getEchartOptions(props).series ?? [];
return series.map(item =>
(item.data ?? []).map(entry =>
typeof entry === 'object' && entry !== null && 'value' in entry
? entry.value
: entry,
),
);
};
const extractSeriesNames = (props: ButterflyTransformedProps) => {
const series = getEchartOptions(props).series ?? [];
return series.map(item => item.name);
};
const data: Record<string, unknown>[] = [
{ category: 'A', left_sum: 10, right_sum: 25 },
{ category: 'B', left_sum: 5, right_sum: 19 },
];
const formData = {
groupby: ['category'],
left_metric: 'left_sum',
right_metric: 'right_sum',
left_color: { r: 84, g: 112, b: 198 },
right_color: { r: 145, g: 204, b: 117 },
showValue: true,
showLegend: true,
};
const createChartProps = (
overrides: Record<string, unknown> = {},
queryData: Record<string, unknown>[] = data,
) =>
new ChartProps({
formData: { ...formData, ...overrides },
width: 800,
height: 600,
queriesData: [{ data: queryData }],
theme: supersetTheme,
...((overrides.hooks ? { hooks: overrides.hooks } : {}) as object),
});
test('transforms chart props into diverging bar series', () => {
const transformedProps = transformProps(
createChartProps() as unknown as EchartsButterflyChartProps,
);
expect(extractSeriesValues(transformedProps)).toEqual([
[-10, -5],
[25, 19],
]);
});
test('uses absolute values for negative right-side metrics', () => {
const transformedProps = transformProps(
createChartProps({}, [
{ category: 'A', left_sum: -8, right_sum: -15 },
]) as unknown as EchartsButterflyChartProps,
);
expect(extractSeriesValues(transformedProps)).toEqual([[-8], [15]]);
});
test('formats null categories and missing metric values', () => {
const transformedProps = transformProps(
createChartProps({}, [
{ category: null, left_sum: undefined, right_sum: 7 },
]) as unknown as EchartsButterflyChartProps,
);
const { yAxis } = getEchartOptions(transformedProps);
expect(yAxis?.data).toEqual([NULL_STRING]);
const [leftValues, rightValues] = extractSeriesValues(transformedProps);
expect(Math.abs(leftValues[0] as number)).toBe(0);
expect(rightValues).toEqual([7]);
});
test('applies custom series labels, colors, and axis titles', () => {
const transformedProps = transformProps(
createChartProps({
left_label: 'Left side',
right_label: 'Right side',
left_color: { r: 255, g: 0, b: 0 },
right_color: { r: 0, g: 255, b: 0 },
x_axis_label: 'Value axis',
y_axis_label: 'Category axis',
}) as unknown as EchartsButterflyChartProps,
);
const { series, xAxis, yAxis } = getEchartOptions(transformedProps);
expect(extractSeriesNames(transformedProps)).toEqual([
'Left side',
'Right side',
]);
expect(series?.[0]?.itemStyle?.color).toBe('#ff0000');
expect(series?.[1]?.itemStyle?.color).toBe('#00ff00');
expect(xAxis?.name).toBe('Value axis');
expect(yAxis?.name).toBe('Category axis');
});
test('applies legend orientation, sort, and axis margin settings', () => {
const transformedProps = transformProps(
createChartProps({
legendOrientation: 'left',
legendSort: 'desc',
xAxisLabelRotation: 45,
x_axis_title_margin: 60,
y_axis_title_margin: 80,
}) as unknown as EchartsButterflyChartProps,
);
const { legend, xAxis, yAxis, grid } = getEchartOptions(transformedProps);
expect(legend?.orient).toBe('vertical');
expect(legend?.data).toEqual(['right_sum', 'left_sum']);
expect(xAxis?.nameGap).toBe(60);
expect(yAxis?.axisLabel?.rotate).toBe(45);
expect(yAxis?.nameGap).toBe(80);
expect(grid?.left).toBeGreaterThan(80);
expect(grid?.top).toBeGreaterThan(60);
});
test('hides value labels when showValue is false', () => {
const transformedProps = transformProps(
createChartProps({
showValue: false,
}) as unknown as EchartsButterflyChartProps,
);
const { series } = getEchartOptions(transformedProps);
expect(series?.[0]?.label?.show).toBe(false);
expect(series?.[1]?.label?.show).toBe(false);
});
test('hides tooltip while the context menu is open', () => {
const transformedProps = transformProps(
createChartProps({}, data) as unknown as EchartsButterflyChartProps,
);
const withContextMenu = transformProps({
...createChartProps(),
inContextMenu: true,
} as unknown as EchartsButterflyChartProps);
expect(getEchartOptions(transformedProps).tooltip?.show).toBe(true);
expect(getEchartOptions(withContextMenu).tooltip?.show).toBe(false);
});
@@ -2839,46 +2839,3 @@ test('applies gridlines to the value axis after a horizontal orientation swaps i
// and the gridlines belonging to it — end up on xAxis.
expect((echartOptions.xAxis as any).splitLine.show).toBe(false);
});
test('boundary label alignment is dropped when the orientation moves the time axis to the side', () => {
// The alignments position labels against the left and right edges of a
// bottom axis. A horizontal chart swaps the axes, so applying them there
// shifts the first label out of line with the rest (#43428 follow-up).
const monthData = [
{ __timestamp: Date.UTC(2003, 4, 1), sales: 100 },
{ __timestamp: Date.UTC(2003, 5, 1), sales: 200 },
];
const build = (orientation: OrientationType) =>
transformProps(
createTestChartProps({
formData: {
granularity_sqla: 'ds',
timeGrainSqla: TimeGranularity.MONTH,
xAxisTimeFormat: 'smart_date',
seriesType: EchartsTimeseriesSeriesType.Bar,
orientation,
},
queriesData: [
createTestQueryData(monthData, {
colnames: ['__timestamp', 'sales'],
coltypes: [GenericDataType.Temporal, GenericDataType.Numeric],
}),
],
}),
).echartOptions;
const vertical = build(OrientationType.Vertical).xAxis as any;
expect(vertical.axisLabel.alignMinLabel).toBe('left');
expect(vertical.axisLabel.alignMaxLabel).toBe('right');
// Horizontal swaps the axes, so the time axis is now yAxis.
const horizontal = build(OrientationType.Horizontal).yAxis as any;
expect(horizontal.axisLabel.alignMinLabel).toBeUndefined();
expect(horizontal.axisLabel.alignMaxLabel).toBeUndefined();
// The boundary labels themselves stay forced in both orientations.
expect(vertical.axisLabel.showMinLabel).toBe(true);
expect(vertical.axisLabel.showMaxLabel).toBe(true);
expect(horizontal.axisLabel.showMinLabel).toBe(true);
expect(horizontal.axisLabel.showMaxLabel).toBe(true);
});
@@ -462,14 +462,13 @@ export default function TableChart<D extends DataRecord = DataRecord>(
// only take relevant page size options
const pageSizeOptions = useMemo(() => {
const getServerPagination = (n: number) =>
n <= Math.max(rowCount, serverPageLength);
const getServerPagination = (n: number) => n <= rowCount;
return (
serverPagination ? SERVER_PAGE_SIZE_OPTIONS : PAGE_SIZE_OPTIONS
).filter(([n]) =>
serverPagination ? getServerPagination(n) : n <= 2 * data.length,
) as SizeOption[];
}, [data.length, rowCount, serverPageLength, serverPagination]);
}, [data.length, rowCount, serverPagination]);
const getValueRange = useCallback(
function getValueRange(key: string, alignPositiveNegative: boolean) {
@@ -67,23 +67,14 @@ async function renderAndWait(props = mockedProps) {
container = renderedContainer;
}
// A modal that wasn't handed an `etag` reads the dataset itself and can't save
// until that lands, so tests must wait before acting on the Save button.
async function waitForSaveEnabled() {
await waitFor(() =>
expect(screen.getByTestId('datasource-modal-save')).toBeEnabled(),
);
}
beforeEach(async () => {
beforeEach(() => {
fetchMock.clearHistory().removeRoutes();
cleanup();
renderAndWait();
fetchMock.post(SAVE_ENDPOINT, SAVE_PAYLOAD);
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
fetchMock.get(GET_DATASOURCE_ENDPOINT, { result: {} });
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
await waitForSaveEnabled();
});
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
@@ -127,7 +118,6 @@ describe('DatasourceModal', () => {
onDatasourceSave:
onDatasourceSave as unknown as typeof mockedProps.onDatasourceSave,
});
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
const okButton = await screen.findByRole('button', { name: 'Confirm' });
@@ -161,96 +151,6 @@ describe('DatasourceModal', () => {
putSpy.mockRestore();
});
test('sends the supplied etag as If-Match so a stale save is refused', async () => {
cleanup();
renderAndWait({ ...mockedProps, etag: '"v1"' } as typeof mockedProps);
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
await waitFor(() => {
const putCall = fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put');
expect(
new Headers(putCall?.options?.headers as HeadersInit).get('If-Match'),
).toEqual('"v1"');
});
});
test('reads the etag from the dataset when the caller supplies none', async () => {
cleanup();
fetchMock.clearHistory().removeRoutes();
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
fetchMock.get(GET_DATASOURCE_ENDPOINT, {
body: { result: {} },
headers: { ETag: '"v2"' },
});
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
// The form is seeded from the same read as the validator, so saving is
// unavailable until it lands.
expect(screen.getByTestId('datasource-modal-save')).toBeDisabled();
await screen.findByTestId('datasource-editor');
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
await waitFor(() => {
const putCall = fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put');
expect(
new Headers(putCall?.options?.headers as HeadersInit).get('If-Match'),
).toEqual('"v2"');
});
});
test('never saves unguarded while the validator read is in flight', async () => {
cleanup();
fetchMock.clearHistory().removeRoutes();
fetchMock.put(SAVE_DATASOURCE_ENDPOINT, {});
// A read that never resolves: the save path must stay closed rather than
// fall through to an unconditional PUT.
fetchMock.get(GET_DATASOURCE_ENDPOINT, new Promise(() => {}));
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
renderAndWait();
const saveButton = await screen.findByTestId('datasource-modal-save');
expect(saveButton).toBeDisabled();
fireEvent.click(saveButton);
expect(
fetchMock.callHistory
.calls()
.find(call => call.options?.method === 'put'),
).toBeUndefined();
});
test('shows a conflict dialog instead of a generic error on 412', async () => {
const putSpy = jest
.spyOn(SupersetClient, 'put')
.mockRejectedValue(new Response('', { status: 412 }));
try {
fireEvent.click(screen.getByTestId('datasource-modal-save'));
fireEvent.click(await screen.findByRole('button', { name: 'Confirm' }));
const conflictElements = await screen.findAllByText(
'Dataset changed since you opened it',
);
expect(conflictElements.length).toBeGreaterThan(0);
expect(
screen.queryByText('Error saving dataset'),
).not.toBeInTheDocument();
} finally {
putSpy.mockRestore();
}
});
test('shows sync columns checkbox when SQL changes', async () => {
cleanup();
const datasourceWithSQL = {
@@ -263,24 +163,15 @@ describe('DatasourceModal', () => {
};
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -317,24 +208,15 @@ describe('DatasourceModal', () => {
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL to trigger checkbox
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -387,24 +269,15 @@ describe('DatasourceModal', () => {
fetchMock.get(GET_DATABASE_ENDPOINT, { result: [] });
const { rerender } = render(
<DatasourceModal
{...mockedProps}
datasource={datasourceWithSQL}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={datasourceWithSQL} />,
{ store, useRouter: true },
);
// Update with modified SQL to trigger checkbox
rerender(
<DatasourceModal
{...mockedProps}
datasource={modifiedDatasource}
etag='"v1"'
/>,
<DatasourceModal {...mockedProps} datasource={modifiedDatasource} />,
);
await waitForSaveEnabled();
const saveButton = screen.getByTestId('datasource-modal-save');
fireEvent.click(saveButton);
@@ -21,7 +21,6 @@ import {
screen,
fireEvent,
act,
waitFor,
defaultStore as store,
} from 'spec/helpers/testing-library';
import fetchMock from 'fetch-mock';
@@ -73,9 +72,6 @@ test('DatasourceModal - should handle sync columns state without imperative moda
render(<DatasourceModal {...mockedProps} />, { store });
const saveButton = screen.getByTestId('datasource-modal-save');
// The modal fetches the current dataset version on open; save stays disabled
// until that settles
await waitFor(() => expect(saveButton).toBeEnabled());
// This should not throw any DOM errors
await act(async () => {
@@ -33,14 +33,12 @@ import {
Icons,
Button,
Checkbox,
Loading,
Modal,
AsyncEsmComponent,
} from '@superset-ui/core/components';
import withToasts from 'src/components/MessageToasts/withToasts';
import { ErrorMessageWithStackTrace } from 'src/components';
import type { DatasetObject } from 'src/features/datasets/types';
import { withCertificationFields } from '../utils';
import { mapSubjectValuesToIds } from 'src/features/subjects/SubjectPicker';
import type { DatasourceModalProps } from '../types';
@@ -93,18 +91,12 @@ export function buildExtraJsonObject(
const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
addSuccessToast,
datasource,
etag,
onDatasourceSave,
onHide,
show,
}) => {
const theme = useTheme();
const [currentDatasource, setCurrentDatasource] = useState(datasource);
// SQL of the server snapshot the form started from. The caller's, unless
// this modal read the dataset itself — then "did the SQL change?" has to be
// asked against the snapshot the payload is actually built from.
const [seededSql, setSeededSql] = useState<string | undefined>();
const [versionEtag, setVersionEtag] = useState(etag);
const [syncColumns, setSyncColumns] = useState(false);
const currencies = useSelector<
{
@@ -119,52 +111,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
const [isEditing, setIsEditing] = useState<boolean>(false);
const [modal, contextHolder] = Modal.useModal();
const [confirmModalOpen, setConfirmModalOpen] = useState(false);
const [isLoadingDatasource, setIsLoadingDatasource] = useState(false);
// Callers that read the dataset themselves (the dataset list) hand down the
// ETag of that read. The rest — Explore, where `datasource` comes from the
// page's bootstrap state — read it here, and must seed the form from the
// *same* response: a payload built from an older snapshot than the ETag
// guarding it would still be accepted, and would still clobber.
useEffect(() => {
setVersionEtag(etag);
if (etag || !show || !datasource.id) {
return undefined;
}
let cancelled = false;
setIsLoadingDatasource(true);
SupersetClient.get({
endpoint: `/api/v1/dataset/${datasource.id}`,
})
.then(({ json, response }) => {
if (cancelled) {
return;
}
const seeded = {
...datasource,
...json.result,
columns: withCertificationFields(json.result.columns),
};
setSeededSql(seeded.sql);
setCurrentDatasource(seeded);
setVersionEtag(response.headers.get('ETag') ?? undefined);
})
.catch(() => {
// The read failed outright, so there is no fresher snapshot to edit
// and no validator to send. Fall back to the caller's snapshot and an
// unconditional save, which is what this modal did before the guard.
})
.finally(() => {
if (!cancelled) {
setIsLoadingDatasource(false);
}
});
return () => {
cancelled = true;
};
}, [datasource.id, etag, show]);
const baselineSql = seededSql ?? datasource.sql;
const buildPayload = (datasource: Record<string, any>) => {
const payload: Record<string, any> = {
table_name: datasource.table_name,
@@ -251,13 +197,11 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
await SupersetClient.put({
endpoint: `/api/v1/dataset/${currentDatasource.id}?override_columns=${syncColumns}`,
jsonPayload: buildPayload(currentDatasource),
...(versionEtag ? { headers: { 'If-Match': versionEtag } } : {}),
});
const { json, response } = await SupersetClient.get({
const { json } = await SupersetClient.get({
endpoint: `/api/v1/dataset/${currentDatasource?.id}`,
});
setVersionEtag(response.headers.get('ETag') ?? undefined);
addSuccessToast(t('The dataset has been saved'));
// eslint-disable-next-line no-param-reassign
@@ -269,19 +213,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
onHide();
} catch (response) {
setIsSaving(false);
if ((response as Response)?.status === 412) {
modal.error({
title: t('Dataset changed since you opened it'),
okButtonProps: { danger: true, className: 'btn-danger' },
content: t(
'Someone else, or another one of your browser tabs, saved this ' +
'dataset after you opened it. Saving now would undo those ' +
'changes, so it was cancelled. Copy your edits, close this ' +
'dialog, and reopen the dataset to reapply them.',
),
});
return;
}
const error = await getClientErrorObject(response);
let errorResponse: SupersetError | undefined;
let errorText: string | undefined;
@@ -333,7 +264,7 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
here may affect other charts
in undesirable ways.`)}
/>
{baselineSql !== currentDatasource.sql && (
{datasource.sql !== currentDatasource.sql && (
<div
css={theme => ({
marginBottom: theme.marginMD,
@@ -367,14 +298,14 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
{t('Are you sure you want to save and apply changes?')}
</div>
),
[currentDatasource.sql, baselineSql, syncColumns],
[currentDatasource.sql, datasource.sql, syncColumns],
);
useEffect(() => {
if (baselineSql !== currentDatasource.sql) {
if (datasource.sql !== currentDatasource.sql) {
setSyncColumns(true);
}
}, [baselineSql, currentDatasource.sql]);
}, [datasource.sql, currentDatasource.sql]);
const onClickSave = () => {
setConfirmModalOpen(true);
@@ -425,7 +356,6 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
onClick={onClickSave}
disabled={
isSaving ||
isLoadingDatasource ||
errors.length > 0 ||
currentDatasource.is_managed_externally
}
@@ -451,18 +381,14 @@ const DatasourceModal: FunctionComponent<DatasourceModalProps> = ({
}}
draggable
>
{isLoadingDatasource ? (
<Loading />
) : (
<DatasourceEditor
showLoadingForImport
height={500}
datasource={currentDatasource}
onChange={onDatasourceChange}
setIsEditing={setIsEditing}
currencies={currencies}
/>
)}
<DatasourceEditor
showLoadingForImport
height={500}
datasource={currentDatasource}
onChange={onDatasourceChange}
setIsEditing={setIsEditing}
currencies={currencies}
/>
{contextHolder}
<Modal
title={t('Confirm save')}
@@ -20,5 +20,4 @@ import ChangeDatasourceModal from './ChangeDatasourceModal';
import DatasourceModal from './DatasourceModal';
export { ChangeDatasourceModal, DatasourceModal };
export { withCertificationFields } from './utils';
export type { DatasourceModalProps, ChangeDatasourceModalProps } from './types';
@@ -29,12 +29,6 @@ export interface DatasourceModalProps {
addSuccessToast: (msg: string) => void;
addDangerToast: (msg: string) => void;
datasource: DatasetObject;
/**
* ETag of the dataset read the form was seeded from. Replayed as `If-Match`
* on save so a stale form can't clobber a newer write. Fetched by the modal
* when the caller doesn't already have one.
*/
etag?: string;
onChange: () => {};
onDatasourceSave: (datasource: object, errors?: Array<any>) => {};
onHide: () => {};
@@ -27,7 +27,6 @@ import { nanoid } from 'nanoid';
import { SupersetClient } from '@superset-ui/core';
import { tn } from '@apache-superset/core/translation';
import rison from 'rison';
import type { ColumnObject } from 'src/features/datasets/types';
// Type definitions
@@ -249,29 +248,3 @@ export async function fetchSyncedColumns(
const { json } = await SupersetClient.get({ endpoint, signal });
return json as ColumnMetadata[];
}
/**
* Lift each column's certification out of its `extra` JSON into the flat
* fields the datasource editor binds to.
*/
export function withCertificationFields(columns: ColumnObject[] = []) {
return columns.map(column => {
// Malformed `extra` must not take out the whole column list, the way an
// uncaught parse would — same fallback as `hydrateMetricExtra`.
let parsedExtra;
try {
parsedExtra = JSON.parse(column.extra || '{}') || {};
} catch {
parsedExtra = {};
}
const {
certification: { details = '', certified_by: certifiedBy = '' } = {},
} = parsedExtra;
return {
...column,
certification_details: details || '',
certified_by: certifiedBy || '',
is_certified: details || certifiedBy,
};
});
}
@@ -23,7 +23,7 @@ import {
userEvent,
waitFor,
} from 'spec/helpers/testing-library';
import { FeatureFlag, VizType, getExtensionsRegistry } from '@superset-ui/core';
import { FeatureFlag, VizType } from '@superset-ui/core';
import mockState from 'spec/fixtures/mockState';
import { cachedSupersetGet } from 'src/utils/cachedSupersetGet';
import downloadAsImage from 'src/utils/downloadAsImage';
@@ -165,9 +165,6 @@ beforeEach(() => {
afterEach(() => {
Reflect.deleteProperty(document, 'fullscreenElement');
// TypedRegistry has no remove(); reset to a no-op so a registered slot does
// not leak into other tests (the empty array is guarded, so nothing injects).
getExtensionsRegistry().set('dashboard.slice.header.menu', () => []);
});
test('Should render', () => {
@@ -176,58 +173,6 @@ test('Should render', () => {
expect(screen.getByTestId(`slice_${SLICE_ID}-menu`)).toBeInTheDocument();
});
test('Injects dashboard.slice.header.menu items at the top of the menu', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => [
{ key: 'custom-ext', label: 'Custom Menu Extension' },
]);
renderWrapper();
openMenu();
const injected = screen.getByText('Custom Menu Extension');
expect(injected).toBeInTheDocument();
// Sits above the built-in entries.
const forceRefresh = screen.getByText('Force refresh');
expect(
injected.compareDocumentPosition(forceRefresh) &
Node.DOCUMENT_POSITION_FOLLOWING,
).toBeTruthy();
});
test('Injects nothing when dashboard.slice.header.menu returns no items', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => []);
renderWrapper();
openMenu();
expect(screen.queryByText('Custom Menu Extension')).not.toBeInTheDocument();
// The menu still renders its built-in entries unchanged (no dangling divider
// is added since the empty array is guarded).
expect(screen.getByText('Force refresh')).toBeInTheDocument();
});
test('Menu survives a dashboard.slice.header.menu extension that throws', () => {
getExtensionsRegistry().set('dashboard.slice.header.menu', () => {
throw new Error('boom');
});
renderWrapper();
openMenu();
// The throw is isolated: the built-in menu still renders.
expect(screen.getByText('Force refresh')).toBeInTheDocument();
expect(screen.getByText('Enter fullscreen')).toBeInTheDocument();
});
test('Injects nothing when the extension returns a non-array', () => {
getExtensionsRegistry().set(
'dashboard.slice.header.menu',
// JS registrations bypass the MenuItem[] type; a bad return must not crash.
(() => undefined) as never,
);
renderWrapper();
openMenu();
expect(screen.getByText('Force refresh')).toBeInTheDocument();
});
test('Should render default props', () => {
const props = createProps();
@@ -34,13 +34,11 @@ import {
isFeatureEnabled,
FeatureFlag,
getChartMetadataRegistry,
getExtensionsRegistry,
VizType,
BinaryQueryObjectFilterClause,
JsonObject,
QueryFormData,
} from '@superset-ui/core';
import { logging } from '@apache-superset/core/utils';
import { css, useTheme, styled } from '@apache-superset/core/theme';
import { useSelector } from 'react-redux';
import { Menu, MenuItem } from '@superset-ui/core/components/Menu';
@@ -167,8 +165,6 @@ const queueChartResize = () => {
}, 300);
};
const extensionsRegistry = getExtensionsRegistry();
const SliceHeaderControls = (
props: SliceHeaderControlsPropsWithRouter | SliceHeaderControlsProps,
) => {
@@ -518,26 +514,6 @@ const SliceHeaderControls = (
},
];
const sliceHeaderMenuExtension = extensionsRegistry.get(
'dashboard.slice.header.menu',
);
if (sliceHeaderMenuExtension) {
// Isolate the extension: a bad registration (throwing, or returning a
// non-array) must not take down the whole dashboard render.
try {
const extensionItems = sliceHeaderMenuExtension({
sliceId: slice.slice_id,
sliceName: slice.slice_name,
dashboardId,
});
if (Array.isArray(extensionItems) && extensionItems.length) {
newMenuItems.unshift(...extensionItems, { type: 'divider' });
}
} catch (error) {
logging.error('dashboard.slice.header.menu extension failed', error);
}
}
if (slice.description) {
newMenuItems.push({
key: MenuKeys.ToggleChartDescription,
@@ -297,12 +297,6 @@ test('Click on Edit dataset', async () => {
const props = createProps();
fetchMock.removeRoute(getDbWithQuery);
fetchMock.get(getDbWithQuery, { result: [] }, { name: getDbWithQuery });
fetchMock.removeRoute(getDatasetWithAllMockRouteName);
fetchMock.get(
getDatasetWithAll,
{ result: {} },
{ name: getDatasetWithAllMockRouteName },
);
render(<DatasourceControl {...props} />, {
useRedux: true,
useRouter: true,
@@ -313,9 +307,7 @@ test('Click on Edit dataset', async () => {
await userEvent.click(screen.getByText('Edit dataset'));
});
expect(
await screen.findByTestId('mock-datasource-editor'),
).toBeInTheDocument();
expect(screen.getByTestId('mock-datasource-editor')).toBeInTheDocument();
});
test('Edit dataset should be disabled when user is not admin', async () => {
@@ -43,6 +43,7 @@ import {
} from 'src/views/CRUD/utils';
import { SUBJECT_OPTION_FILTER_PROPS } from 'src/features/subjects/SubjectSelectLabel';
import { SubjectPile } from 'src/features/subjects/SubjectPile';
import { ColumnObject } from 'src/features/datasets/types';
import { useListViewResource } from 'src/views/CRUD/hooks';
import {
ActionButton,
@@ -61,7 +62,6 @@ import {
} from '@superset-ui/core/components';
import {
DatasourceModal,
withCertificationFields,
GenericLink,
ImportModal as ImportModelsModal,
ModifiedInfo,
@@ -496,8 +496,6 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
const [datasetCurrentlyEditing, setDatasetCurrentlyEditing] =
useState<Dataset | null>(null);
const [datasetCurrentlyEditingEtag, setDatasetCurrentlyEditingEtag] =
useState<string | undefined>();
const [datasetCurrentlyDuplicating, setDatasetCurrentlyDuplicating] =
useState<VirtualDataset | null>(null);
@@ -567,11 +565,24 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
SupersetClient.get({
endpoint: `/api/v1/dataset/${id}`,
})
.then(({ json = {}, response }) => {
setDatasetCurrentlyEditingEtag(
response.headers.get('ETag') ?? undefined,
.then(({ json = {} }) => {
const addCertificationFields = json.result.columns.map(
(column: ColumnObject) => {
const {
certification: {
details = '',
certified_by: certifiedBy = '',
} = {},
} = JSON.parse(column.extra || '{}') || {};
return {
...column,
certification_details: details || '',
certified_by: certifiedBy || '',
is_certified: details || certifiedBy,
};
},
);
json.result.columns = withCertificationFields(json.result.columns);
json.result.columns = [...addCertificationFields];
setDatasetCurrentlyEditing(json.result);
})
.catch(() => {
@@ -1513,7 +1524,6 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
{datasetCurrentlyEditing && (
<DatasourceModal
datasource={datasetCurrentlyEditing}
etag={datasetCurrentlyEditingEtag}
onDatasourceSave={refreshData}
onHide={closeDatasetEditModal}
show
@@ -65,7 +65,6 @@ import {
EchartsRoseChartPlugin,
EchartsTimePivotChartPlugin,
EchartsBulletChartPlugin,
EchartsButterflyChartPlugin,
} from '@superset-ui/plugin-chart-echarts';
import {
SelectFilterPlugin,
@@ -164,9 +163,6 @@ export default class MainPreset extends Preset {
new EchartsWaterfallChartPlugin().configure({
key: VizType.Waterfall,
}),
new EchartsButterflyChartPlugin().configure({
key: VizType.Butterfly,
}),
new EchartsHeatmapChartPlugin().configure({ key: VizType.Heatmap }),
new EchartsHistogramChartPlugin().configure({ key: VizType.Histogram }),
new SelectFilterPlugin().configure({ key: FilterPlugins.Select }),
+4 -4
View File
@@ -26,7 +26,7 @@
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.67.0",
"@typescript-eslint/parser": "^8.67.0",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.11.0",
"oxfmt": "^0.64.0",
@@ -1624,9 +1624,9 @@
}
},
"node_modules/eslint": {
"version": "10.9.0",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.9.0.tgz",
"integrity": "sha512-5KeEOJZBfEVA47boFiBsf+6MmmJpffM7qEBg4pLla2e4nlKgdKlqCW0oSLOGsT8Wl5uCGJptLV1bkaiShj90Gw==",
"version": "10.8.1",
"resolved": "https://registry.npmjs.org/eslint/-/eslint-10.8.1.tgz",
"integrity": "sha512-wqA7W2jbsC/BnV9Iv1UZpKVFkO1AdNoSmYW8NWG4HNOBbkAMvIqDZ27pI2f07dqn583NcIC44ckjAcOXDL1QbQ==",
"dev": true,
"license": "MIT",
"workspaces": [
+1 -1
View File
@@ -34,7 +34,7 @@
"@types/ws": "^8.18.1",
"@typescript-eslint/eslint-plugin": "^8.67.0",
"@typescript-eslint/parser": "^8.67.0",
"eslint": "^10.9.0",
"eslint": "^10.8.1",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.11.0",
"oxfmt": "^0.64.0",
+6
View File
@@ -18,6 +18,7 @@ import logging
from flask import Response
from flask_appbuilder.api import expose, protect, safe
from flask_appbuilder.security.decorators import has_access_api
from superset.commands.dashboard.filter_state.create import CreateFilterStateCommand
from superset.commands.dashboard.filter_state.delete import DeleteFilterStateCommand
@@ -25,6 +26,7 @@ from superset.commands.dashboard.filter_state.get import GetFilterStateCommand
from superset.commands.dashboard.filter_state.update import UpdateFilterStateCommand
from superset.extensions import event_logger
from superset.temporary_cache.api import TemporaryCacheRestApi
from superset.views.base import api
logger = logging.getLogger(__name__)
@@ -46,6 +48,8 @@ class DashboardFilterStateRestApi(TemporaryCacheRestApi):
def get_delete_command(self) -> type[DeleteFilterStateCommand]:
return DeleteFilterStateCommand
@api
@has_access_api
@expose("/<int:pk>/filter_state", methods=("POST",))
@protect()
@safe
@@ -169,6 +173,8 @@ class DashboardFilterStateRestApi(TemporaryCacheRestApi):
"""
return super().post(pk)
@api
@has_access_api
@expose("/<int:pk>/filter_state/<string:key>", methods=("PUT",))
@protect()
@safe
+6 -54
View File
@@ -29,7 +29,7 @@ from flask_appbuilder.api import expose, protect, rison as parse_rison, safe
from flask_appbuilder.api.schemas import get_item_schema
from flask_appbuilder.const import API_RESULT_RES_KEY, API_SELECT_COLUMNS_RIS_KEY
from flask_appbuilder.models.sqla.interface import SQLAInterface
from flask_babel import gettext as _, ngettext
from flask_babel import ngettext
from jinja2.exceptions import TemplateError
from marshmallow import ValidationError
from sqlalchemy.orm.exc import MultipleResultsFound
@@ -95,20 +95,13 @@ from superset.subjects.filters import FilterRelatedSubjects, subject_type_filter
from superset.utils import json
from superset.utils.core import parse_boolean_string, send_export_zip
from superset.versioning.api_helpers import (
concurrency_token_from,
current_entity_etag_uuid,
current_entity_version_info,
entity_concurrency_token,
get_version_endpoint,
list_versions_endpoint,
lock_entity_for_update,
restore_version_endpoint,
)
from superset.versioning.etag import (
is_conditional_write,
raise_for_stale_write,
set_version_etag,
StaleEntityError,
)
from superset.versioning.etag import set_version_etag
from superset.versioning.schemas import VersionListItemSchema
from superset.views.base import DatasourceFilter
from superset.views.base_api import (
@@ -549,14 +542,6 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
schema:
type: boolean
name: override_columns
- in: header
schema:
type: string
name: If-Match
description: >-
Optional optimistic-concurrency guard. Pass the ``ETag`` returned
by a prior read of this dataset; the update is rejected with 412
if the dataset has changed since.
requestBody:
description: Dataset schema
required: true
@@ -633,17 +618,6 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
$ref: '#/components/responses/403'
404:
$ref: '#/components/responses/404'
412:
description: >-
The dataset changed since the version identified by the
request's ``If-Match`` header; the update was not applied.
content:
application/json:
schema:
type: object
properties:
message:
type: string
422:
$ref: '#/components/responses/422'
500:
@@ -660,32 +634,10 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
except ValidationError as error:
return self.response_400(message=error.messages)
# Serialise conditional saves on this dataset: the guard below reads
# the live version, the command writes, and the two must not interleave
# with another request's. Only a conditional save pays for the lock; an
# unconditional PUT behaves exactly as it did before the guard existed.
if is_conditional_write():
lock_entity_for_update(SqlaTable, pk)
# Live version identifiers before the update (empty + query-free when
# ``ENABLE_VERSIONING_CAPTURE`` is off).
old_info = current_entity_version_info(SqlaTable, pk)
try:
raise_for_stale_write(concurrency_token_from(old_info))
except StaleEntityError:
return set_version_etag(
self.response(
412,
message=_(
"The dataset was changed by another user or browser tab "
"after you opened it. Reopen it to pick up the latest "
"version, then reapply your changes."
),
),
concurrency_token_from(old_info),
)
try:
# Two commands, two commits, two Continuum transactions for an
# ``override_columns`` save — deliberately NOT merged into one
@@ -709,13 +661,13 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
new_info = current_entity_version_info(
SqlaTable, changed_model.id, changed_model.uuid
)
etag_version_uuid = concurrency_token_from(new_info)
etag_version_uuid = new_info.version_uuid
if override_columns:
RefreshDatasetCommand(pk).run()
# The ETag must reflect the entity's *current live* version,
# which after the refresh is the refresh's transaction —
# re-read it rather than reusing the pre-refresh uuid.
etag_version_uuid = entity_concurrency_token(
etag_version_uuid = current_entity_etag_uuid(
SqlaTable, changed_model.id, changed_model.uuid
)
response = self.response(
@@ -1748,7 +1700,7 @@ class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi):
return set_version_etag(
self.response(200, **response),
entity_concurrency_token(SqlaTable, table.id, table.uuid),
current_entity_etag_uuid(SqlaTable, table.id, table.uuid),
)
@expose("/<int:pk>/drill_info/", methods=("GET",))
-15
View File
@@ -41,7 +41,6 @@ import sqlalchemy.dialects
from flask import current_app as app
from sqlalchemy.engine.default import DefaultDialect
from sqlalchemy.exc import NoSuchModuleError
from sqlalchemy.sql import compiler as sqla_compiler
from superset import feature_flag_manager
from superset.db_engine_specs.base import BaseEngineSpec
@@ -160,16 +159,6 @@ def get_available_engine_specs() -> dict[type[BaseEngineSpec], set[str]]: # noq
continue
# installed 3rd-party dialects
#
# `ep.load()` runs arbitrary module-level code in the third-party package.
# Some dialects (e.g. sqlalchemy-monetdb) mutate SQLAlchemy's shared,
# process-global `compiler.OPERATORS` mapping in place on import instead
# of subclassing it, which would otherwise silently change SQL rendering
# (e.g. `!=` -> `<>`) for every dialect for the rest of the process, not
# just the misbehaving one. Snapshot/restore around each load so a
# buggy connector can't leak global compiler state into unrelated
# dialects just because it was enumerated here.
operators_snapshot = dict(sqla_compiler.OPERATORS)
for ep in entry_points(group="sqlalchemy.dialects"):
try:
dialect = ep.load()
@@ -204,10 +193,6 @@ def get_available_engine_specs() -> dict[type[BaseEngineSpec], set[str]]: # noq
if isinstance(driver, bytes):
driver = driver.decode()
drivers[backend].add(driver)
finally:
if sqla_compiler.OPERATORS != operators_snapshot:
sqla_compiler.OPERATORS.clear()
sqla_compiler.OPERATORS.update(operators_snapshot)
dbs_denylist = app.config["DBS_AVAILABLE_DENYLIST"]
if not feature_flag_manager.is_feature_enabled("ENABLE_SUPERSET_META_DB"):
+1 -4
View File
@@ -44,10 +44,7 @@ class CockroachDbEngineSpec(PostgresEngineSpec):
DatabaseCategory.TRADITIONAL_RDBMS,
DatabaseCategory.OPEN_SOURCE,
],
# sqlalchemy-cockroachdb declares no DBAPI dependency of its own (see
# the comment on the `cockroachdb` extra in pyproject.toml), so a
# plain `cockroachdb://` URL also needs psycopg2 installed to connect.
"pypi_packages": ["sqlalchemy-cockroachdb", "psycopg2-binary"],
"pypi_packages": ["sqlalchemy-cockroachdb", "psycopg2"],
"connection_string": "cockroachdb://root@{hostname}:{port}/{database}?sslmode=disable",
"default_port": 26257,
"docs_url": "https://github.com/cockroachdb/sqlalchemy-cockroachdb",
+1 -10
View File
@@ -26,7 +26,7 @@ from collections import defaultdict, deque
from datetime import datetime
from re import Pattern
from textwrap import dedent
from typing import Any, Callable, cast, Optional, TYPE_CHECKING
from typing import Any, cast, Optional, TYPE_CHECKING
from urllib import parse
import pandas as pd
@@ -182,15 +182,6 @@ class PrestoBaseEngineSpec(BaseEngineSpec, metaclass=ABCMeta):
# which raises a query error. Use = true/false instead.
use_equality_for_boolean_filters = True
# Presto/Trino's coordinator sends query results as JSON, which has no
# literal for NaN/Infinity/-Infinity, so REAL/DOUBLE columns holding
# those values arrive as quoted strings. Coerce them back to real
# floats so numeric post-processing (e.g. a pivot's mean) doesn't choke
# on a string value.
column_type_mutators: dict[types.TypeEngine, Callable[[Any], Any]] = {
types.FLOAT: lambda val: float(val) if isinstance(val, str) else val
}
column_type_mappings = (
(
re.compile(r"^boolean.*", re.IGNORECASE),
+1 -12
View File
@@ -22,8 +22,7 @@ import math
import threading
import time
from collections.abc import Sequence
from decimal import Decimal
from typing import Any, Callable, TYPE_CHECKING
from typing import Any, TYPE_CHECKING
import requests
from flask import copy_current_request_context, ctx, current_app as app, Flask, g
@@ -31,7 +30,6 @@ from flask_babel import gettext as __
from sqlalchemy.engine.reflection import Inspector
from sqlalchemy.engine.url import URL
from sqlalchemy.exc import NoSuchTableError
from sqlalchemy.types import DECIMAL, TypeEngine
from superset import cache_manager, db
from superset.common.db_query_status import QueryStatus
@@ -80,15 +78,6 @@ class TrinoEngineSpec(PrestoBaseEngineSpec):
"$.oauth2_client_info.secret": "OAuth2 client secret",
}
# Trino's DBAPI driver can return DECIMAL columns as plain strings
# (e.g. when a value's precision/scale can't be inferred from the
# column type alone), which later breaks numeric post-processing
# (e.g. pivot with a mean aggregate). Coerce them back to Decimal.
column_type_mutators: dict[TypeEngine, Callable[[Any], Any]] = {
**PrestoBaseEngineSpec.column_type_mutators,
DECIMAL: lambda val: Decimal(val) if isinstance(val, str) else val,
}
# The full set of columns Trino's "<table>$partitions" exposes for an
# Iceberg table. The real partition keys are nested in the "partition" ROW,
# so none of these are user partition columns.
-9
View File
@@ -30,7 +30,6 @@ from superset.commands.temporary_cache.exceptions import (
TemporaryCacheResourceNotFoundError,
)
from superset.constants import MODEL_API_RW_METHOD_PERMISSION_MAP
from superset.exceptions import SupersetTemplateException
from superset.explore.form_data.schemas import FormDataPostSchema, FormDataPutSchema
from superset.extensions import event_logger
from superset.views.base_api import BaseSupersetApi, requires_json, statsd_metrics
@@ -111,8 +110,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("PUT",))
@protect()
@@ -186,8 +183,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("GET",))
@protect()
@@ -239,8 +234,6 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
@expose("/form_data/<string:key>", methods=("DELETE",))
@protect()
@@ -293,5 +286,3 @@ class ExploreFormDataRestApi(BaseSupersetApi):
return self.response(403, message=str(ex))
except TemporaryCacheResourceNotFoundError as ex:
return self.response(404, message=str(ex))
except SupersetTemplateException as ex:
return self.response(ex.status, message=str(ex))
+1 -10
View File
@@ -16,8 +16,6 @@
# under the License.
from typing import Optional
from jinja2.exceptions import TemplateError
from superset import security_manager
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
@@ -35,7 +33,6 @@ from superset.commands.exceptions import (
from superset.daos.chart import ChartDAO
from superset.daos.dataset import DatasetDAO
from superset.daos.query import QueryDAO
from superset.exceptions import SupersetTemplateException
from superset.utils.core import DatasourceType
@@ -56,13 +53,7 @@ def check_query_access(query_id: int) -> Optional[bool]:
# Access checks below, no need to validate them twice as they can be expensive.
query = QueryDAO.find_by_id(query_id, skip_base_filter=True)
if query:
try:
security_manager.raise_for_access(query=query)
except TemplateError as ex:
# raise_for_access() Jinja-renders the query's SQL to resolve
# the tables it touches; a malformed template surfaces here as
# a raw jinja2 exception rather than a Superset one.
raise SupersetTemplateException(str(ex)) from ex
security_manager.raise_for_access(query=query)
return True
raise QueryNotFoundValidationError()
-12
View File
@@ -4823,18 +4823,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
template_processor=template_processor
)
is_metric_filter = True
elif (
col_obj is None
and isinstance(flt_col, str)
and flt_col in adhoc_columns_by_label
):
sqla_col, _unused = self.adhoc_column_to_sqla(
col=adhoc_columns_by_label[flt_col],
template_processor=template_processor,
)
if isinstance(sqla_col, ColumnElement):
applied_adhoc_filters_columns.append(flt_col)
filter_grain = flt.get("grain")
# Check if this filter should be skipped because it was handled in
+19 -1
View File
@@ -1539,7 +1539,25 @@ class SQLStatement(BaseSQLStatement[exp.Expression]):
:return: The parsed predicate.
"""
_check_script_length(predicate, self.engine)
return sqlglot.parse_one(predicate, dialect=self._dialect)
try:
return sqlglot.parse_one(predicate, dialect=self._dialect)
except sqlglot.errors.ParseError as ex:
kwargs = (
{
"highlight": ex.errors[0]["highlight"],
"line": ex.errors[0]["line"],
"column": ex.errors[0]["col"],
}
if ex.errors
else {}
)
raise SupersetParseError(predicate, self.engine, **kwargs) from ex
except sqlglot.errors.SqlglotError as ex:
raise SupersetParseError(
predicate,
self.engine,
message="Unable to parse predicate",
) from ex
def apply_rls(
self,
+1 -189
View File
@@ -16,7 +16,6 @@
# under the License.
from typing import Any, Optional
import pandas as pd
from flask_babel import gettext as _
from pandas import DataFrame
@@ -27,116 +26,6 @@ from superset.utils.pandas_postprocessing.utils import (
validate_column_args,
)
_PERCENT_MODES = frozenset({"percent_row", "percent_col", "percent_total"})
# Aggregate operator names that produce additive results across groups —
# the sum of the per-cell values equals the row/column/grand rollup the
# database would compute over the underlying rows. ``show_values_as``
# percent transforms divide each cell by that sum, so they are only
# meaningful when the sum IS the rollup. For non-additive operators
# (mean, median, min, max, etc.) the "percent of row" the exports would
# show is not the "percent of row" the chart's DB rollup would show,
# and the two disagree. See sadpandajoe's finding on #42976.
_ADDITIVE_OPERATORS = frozenset({"sum", "nansum", "count", "count_nonzero"})
def _div_preserving_nan(numerator: DataFrame, denominator: Any, axis: int) -> DataFrame:
"""Divide ``numerator`` by ``denominator``, preserving NaN numerators.
A genuine SQL NULL numerator must stay NaN (rendered blank) rather than
become ``0.0`` matches the client-side #42810 semantics guarding
against measured "0.0%" values for values that should stay blank.
"""
result = numerator.div(denominator, axis=axis)
return result.mask(numerator.isna(), other=float("nan"))
def _apply_percent_transform_to_group(g: DataFrame, mode: str) -> DataFrame:
"""Apply a percent-of-{row,col,total} transform to a single-metric block.
Called both for the whole DataFrame when it holds one metric, and
per-metric-group for MultiIndex / flat-multi-metric pivots. A zero or
NaN denominator produces NaN cells rather than ``Infinity``/``NaN``
from division-by-zero, matching the client's ``if (acc === null)
return null`` guard in ``fractionOf``.
"""
if mode == "percent_row":
row_totals = g.sum(axis=PandasAxis.COLUMN, skipna=True).replace(0, float("nan"))
return _div_preserving_nan(g, row_totals, axis=PandasAxis.ROW)
if mode == "percent_col":
col_totals = g.sum(axis=PandasAxis.ROW, skipna=True).replace(0, float("nan"))
return _div_preserving_nan(g, col_totals, axis=PandasAxis.COLUMN)
# percent_total
grand = g.sum(skipna=True).sum(skipna=True)
if pd.isna(grand) or grand == 0:
return g * float("nan")
return _div_preserving_nan(g, grand, axis=PandasAxis.ROW)
def _apply_show_values_as(df: DataFrame, mode: str) -> DataFrame:
"""Divide each metric cell by the appropriate rollup total.
Mirrors the client-side ``fractionOf`` semantic in
``plugin-chart-pivot-table/src/react-pivottable/utilities.ts:739``:
- ``percent_row``: cell / row-total (sum across the columns axis)
- ``percent_col``: cell / column-total (sum across the rows axis)
- ``percent_total``: cell / grand-total (sum of the metric block)
Per-metric isolation the totals are computed *within each metric* so
one metric's numerator is never divided by another metric's total,
matching the client's ``metricAxis`` handling. This applies to both
shapes ``pivot_table`` can produce:
- **MultiIndex columns** (level 0 = metric): iterate the level-0
groups explicitly. Explicit iteration avoids the deprecated
``df.groupby(level=0, axis=1)`` pattern (removed in pandas 3.x).
- **Flat columns with >1 column** (multi-metric pivot with no
``columns`` groupby each column IS a metric): treat each column
as its own single-column metric block.
- **Flat columns with 1 column** (single-metric pivot with no
``columns`` groupby): the whole block is one metric.
NULL preservation is *structural* only: cells that ``pivot_table``
left as ``NaN`` because the (row, column) group had no input rows
at all stay ``NaN`` in the output. Cells whose input rows all held
SQL NULL values have already collapsed to ``0.0`` inside
``pivot_table`` (pandas ``sum([NaN]) == 0``), so they render as
``0%``, not blank reconstructing "blank vs measured zero" from an
aggregated value is not possible at this layer.
"""
if df.empty:
# Empty pivots can carry an empty ``MultiIndex`` with zero level-0
# groups; iterating and concatenating produces
# ``ValueError: No objects to concatenate``. Nothing to transform.
return df
is_multi_metric_wide = isinstance(df.columns, pd.MultiIndex)
is_flat_multi_metric = not is_multi_metric_wide and df.shape[1] > 1
if is_multi_metric_wide:
# Iterate level-0 groups explicitly (pandas-3-safe).
metrics = df.columns.get_level_values(0).unique()
parts = []
for metric in metrics:
block = df.xs(metric, axis=PandasAxis.COLUMN, level=0, drop_level=False)
parts.append(_apply_percent_transform_to_group(block, mode))
# ``concat`` along columns preserves the MultiIndex; reorder to
# match the original column layout deterministically.
combined = pd.concat(parts, axis=PandasAxis.COLUMN)
return combined[df.columns]
if is_flat_multi_metric:
# Each flat column is its own metric — process independently.
parts = []
for col in df.columns:
block = df[[col]]
parts.append(_apply_percent_transform_to_group(block, mode))
return pd.concat(parts, axis=PandasAxis.COLUMN)[df.columns]
# Flat single-metric block — the whole DataFrame is one metric.
return _apply_percent_transform_to_group(df, mode)
def _restore_dropped_metric_columns(
df: DataFrame,
@@ -188,7 +77,7 @@ def _restore_dropped_metric_columns(
@validate_column_args("index", "columns")
def pivot( # pylint: disable=too-many-arguments # noqa: C901
def pivot( # pylint: disable=too-many-arguments
df: DataFrame,
index: list[str],
aggregates: dict[str, dict[str, Any]],
@@ -199,7 +88,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
combine_value_with_metric: bool = False,
marginal_distributions: Optional[bool] = None,
marginal_distribution_name: Optional[str] = None,
show_values_as: Optional[str] = None,
) -> DataFrame:
"""
Perform a pivot operation on a DataFrame.
@@ -223,13 +111,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
:param marginal_distributions: Add totals for row/column. Default to False
:param marginal_distribution_name: Name of row/column with marginal distribution.
Default to 'All'.
:param show_values_as: Optional post-pivot transform that expresses each
metric cell as a fraction of the row / column / grand total.
One of ``"percent_row"``, ``"percent_col"``, ``"percent_total"`` or
``None`` / ``"actual"`` (no-op, default). Mirrors the pivot chart's
client-side ``fractionOf`` semantic so server-side rendering paths
(CSV / XLSX exports, scheduled reports) can reproduce the browser
output. See #42809.
:return: A pivot table
:raises InvalidPostProcessingError: If the request in incorrect
"""
@@ -242,62 +123,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
_("Pivot operation must include at least one aggregate")
)
# Fail fast on ``show_values_as`` misconfiguration *before* running the
# (potentially expensive) ``pivot_table`` call: an unknown mode should
# not silently perform a full pivot only to raise at the end, and the
# ``marginal_distributions`` combination should reject before pandas
# gets a chance to raise its own margins-related errors. ``None`` /
# ``""`` / ``"actual"`` are the no-op sentinels — anything else must
# be a known percent mode.
percent_mode: Optional[str] = None
if show_values_as not in (None, "", "actual"):
if show_values_as not in _PERCENT_MODES:
raise InvalidPostProcessingError(
_(
"Unsupported show_values_as value: %(mode)s. "
"Expected one of: percent_row, percent_col, percent_total, actual.",
mode=show_values_as,
)
)
if marginal_distributions:
# The pivot would carry an "All" margin row and/or column;
# summing across the axis would double-count by including the
# margin as part of its own denominator. Combining ``margins``
# with ``show_values_as`` needs a first-class design (probably
# computing percentages on the non-margin subset and then
# re-inserting the margin totals as-is), which is out of scope
# here. Reject explicitly rather than silently returning wrong
# numbers.
raise InvalidPostProcessingError(
_(
"show_values_as is not yet supported when "
"marginal_distributions is enabled."
)
)
# ``show_values_as`` divides each cell by the sum of its axis, so
# it is only meaningful when that sum equals the rollup the DB
# would compute. For non-additive aggregates (mean, median, min,
# max, distinct count, …) the summed per-cell values are not the
# row/column/grand rollup, and the exports would disagree with
# the chart. Reject up front rather than emit numbers that mix
# with the DB rollup incorrectly.
non_additive = [
name
for name, cfg in aggregates.items()
if not isinstance(cfg.get("operator"), str)
or cfg["operator"] not in _ADDITIVE_OPERATORS
]
if non_additive:
raise InvalidPostProcessingError(
_(
"show_values_as is only supported for additive aggregates "
"(sum, count); got non-additive operator(s) for: "
"%(metrics)s.",
metrics=", ".join(non_additive),
)
)
percent_mode = show_values_as
if columns and column_fill_value:
df[columns] = df[columns].fillna(value=column_fill_value)
@@ -338,19 +163,6 @@ def pivot( # pylint: disable=too-many-arguments # noqa: C901
elif pivot_key_set and not df.empty:
df = df.drop(df.columns.difference(pivot_key_set), axis=PandasAxis.COLUMN)
# Apply the ``show_values_as`` percent transform BEFORE the
# ``combine_value_with_metric`` reshape, not after. The reshape below
# swaps the column ``MultiIndex`` level order from ``(metric, category)``
# to ``(category, metric)``; if the percent transform runs against the
# post-reshape shape, its per-metric iteration
# (``df.columns.get_level_values(0).unique()``) walks categories thinking
# they are metrics, mixing metrics and producing wrong percentages. See
# sadpandajoe's finding on #42976. Running percent first keeps the
# per-metric-isolation invariant intact; the reshape then runs on the
# already-normalized values without changing them further.
if percent_mode is not None:
df = _apply_show_values_as(df, percent_mode)
if combine_value_with_metric:
# dropna=False preserves restored all-NaN metric rows that would otherwise
# be silently dropped by stack's default dropna=True behavior.
@@ -86,24 +86,16 @@ ALLOWLIST_CUMULATIVE_FUNCTIONS = (
PROPHET_TIME_GRAIN_MAP: dict[str, str] = {
TimeGrain.SECOND: "s",
TimeGrain.FIVE_SECONDS: "5s",
TimeGrain.THIRTY_SECONDS: "30s",
TimeGrain.MINUTE: "min",
TimeGrain.FIVE_MINUTES: "5min",
TimeGrain.TEN_MINUTES: "10min",
TimeGrain.FIFTEEN_MINUTES: "15min",
TimeGrain.THIRTY_MINUTES: "30min",
# An alternate ISO-8601 spelling of THIRTY_MINUTES that a number of engine
# specs expose instead; the two denote the same interval.
TimeGrain.HALF_HOUR: "30min",
TimeGrain.HOUR: "h",
TimeGrain.SIX_HOURS: "6h",
TimeGrain.DAY: "D",
TimeGrain.WEEK: "W",
TimeGrain.MONTH: "ME" if _PANDAS_VERSION >= (2, 2) else "M",
TimeGrain.QUARTER: "QE" if _PANDAS_VERSION >= (2, 2) else "Q",
# An alternate ISO-8601 spelling of QUARTER, as with HALF_HOUR above.
TimeGrain.QUARTER_YEAR: "QE" if _PANDAS_VERSION >= (2, 2) else "Q",
TimeGrain.YEAR: "YE" if _PANDAS_VERSION >= (2, 2) else "A",
TimeGrain.WEEK_STARTING_SUNDAY: "W-SUN",
TimeGrain.WEEK_STARTING_MONDAY: "W-MON",
-77
View File
@@ -72,11 +72,6 @@ class EntityVersionInfo:
version: int | None = None
transaction_id: int | None = None
version_uuid: str | None = None
#: Resolved uuid of the entity itself, carried so callers that need a
#: concurrency token for an entity with no version rows yet don't have to
#: re-run the ``SELECT uuid`` this helper already issued. Not part of the
#: API response.
entity_uuid: UUID | None = None
def _capture_enabled() -> bool:
@@ -128,7 +123,6 @@ def current_entity_version_info(
version=version,
transaction_id=transaction_id,
version_uuid=str(version_uuid) if version_uuid else None,
entity_uuid=entity_uuid,
)
@@ -150,77 +144,6 @@ def current_entity_etag_uuid(
return str(version_uuid) if version_uuid else None
# Sentinel Continuum transaction id for an entity that has no version rows
# yet. Continuum sequences start at 1, so it can never collide with a real
# one, and the derived uuid stops matching the moment the first version row
# lands — which is exactly the transition a concurrency guard must catch.
_UNVERSIONED_TRANSACTION_ID = 0
def unversioned_entity_token(entity_uuid: UUID) -> str:
"""Concurrency token for an entity Continuum hasn't versioned yet."""
return str(VersionDAO.derive_version_uuid(entity_uuid, _UNVERSIONED_TRANSACTION_ID))
def entity_concurrency_token(
model_cls: type[Model],
entity_id: int | None,
entity_uuid: UUID | None,
) -> str | None:
"""Resolve the optimistic-concurrency validator for *entity*.
Differs from :func:`current_entity_etag_uuid` in what it does for an
entity with no version rows: baseline rows are written lazily, on the
first update after the versioning migration, so a never-since-saved
entity has none. Reporting ``None`` there would leave the *first*
concurrent save on every such entity unguarded the exact case a
two-tab race hits on a pristine entity. Those entities get a
deterministic unversioned token instead.
``None`` still means "no validator exists": capture is off, or the
entity is missing.
"""
if entity_id is None or entity_uuid is None or not _capture_enabled():
return None
return current_entity_etag_uuid(
model_cls, entity_id, entity_uuid
) or unversioned_entity_token(entity_uuid)
def lock_entity_for_update(model_cls: type[Model], entity_id: int | None) -> None:
"""Row-lock *entity* so a conditional write's check and its update are atomic.
``If-Match`` is verified against a read taken before the update command
runs. Without a lock two overlapping requests can both read the same live
version, both pass the check, and then commit one after the other,
reintroducing the lost update the check exists to prevent. The lock is
held until the command commits, because both run in the same scoped
session.
Renders no ``FOR UPDATE`` on SQLite, which serialises writers anyway.
"""
try:
# The PUT route declares ``/<pk>`` (a string segment), so a non-numeric
# id must not raise a SQL cast error ahead of the command's 404.
entity_id = int(entity_id) # type: ignore[arg-type]
except (TypeError, ValueError):
return
db.session.execute(
sa.select(model_cls.id).where(model_cls.id == entity_id).with_for_update()
)
def concurrency_token_from(info: EntityVersionInfo) -> str | None:
"""Concurrency token for an already-resolved :class:`EntityVersionInfo`.
Lets a write endpoint reuse the pre-update version lookup it already
made rather than issuing a second one.
"""
if info.entity_uuid is None:
return None
return info.version_uuid or unversioned_entity_token(info.entity_uuid)
# Maps the versioned model class name to the keyword argument
# ``security_manager.raise_for_access`` expects for the per-resource
# gate. Slice → ``chart=``, Dashboard → ``dashboard=``, SqlaTable →
-44
View File
@@ -22,7 +22,6 @@ from typing import TYPE_CHECKING
from uuid import UUID
import sqlalchemy as sa
from flask import request
from flask_appbuilder import Model
from superset.extensions import db
@@ -77,46 +76,3 @@ def set_version_etag_by_uuid(
response,
VersionDAO.current_live_version_uuid(model_cls, entity_id, entity_uuid),
)
class StaleEntityError(Exception):
"""The request's ``If-Match`` doesn't match the entity's live version."""
def _entity_tag(tag: str) -> str:
"""Strip the content-coding suffix ``Flask-Compress`` appends to ETags.
A compressed response legitimately carries a different validator than the
identity one Flask-Compress rewrites ``"<uuid>"`` to ``"<uuid>:zstd"``
(see ``flask_compress``) so a client replaying the ETag it read never
matches the raw version uuid. Version uuids contain no ``:``, so cutting
at the first one recovers the entity identity from either form.
"""
return tag.split(":", 1)[0]
def is_conditional_write() -> bool:
"""Whether the request carries an ``If-Match`` precondition."""
return bool(request.if_match)
def raise_for_stale_write(current_version_uuid: str | None) -> None:
"""Enforce ``If-Match`` on a write request, if the client sent one.
Clients that read an entity's ``ETag`` may replay it as ``If-Match`` on a
subsequent write to get optimistic concurrency: the write is rejected when
the entity moved on in the meantime, instead of silently clobbering
whatever landed in between.
The condition is skipped rather than failing closed when the caller
has no validator to offer (``ENABLE_VERSIONING_CAPTURE`` off). Failing
closed there would block every conditional write on deployments running
without version capture, and those are no worse off than before they sent
the header.
"""
if_match = request.if_match
if not if_match or if_match.star_tag or current_version_uuid is None:
return
live = _entity_tag(str(current_version_uuid))
if not any(_entity_tag(tag) == live for tag in if_match.as_set(True)):
raise StaleEntityError()
+1
View File
@@ -687,6 +687,7 @@ class Superset(BaseSupersetView):
return json_success(json.dumps(sanitize_datasource_data(datasource.data)))
@event_logger.log_this
@has_access
@expose("/language_pack/<lang>/")
def language_pack(self, lang: str) -> FlaskResponse:
# Only allow expected language formats like "en", "pt_BR", etc.
@@ -1,340 +1,295 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from unittest.mock import patch # noqa: F401
import pytest
from flask.ctx import AppContext
from flask_appbuilder.security.sqla.models import User
from sqlalchemy.orm import Session # noqa: F401
from superset import db
from superset.commands.dashboard.exceptions import (
DashboardAccessDeniedError, # noqa: F401
)
from superset.commands.temporary_cache.entry import Entry
from superset.extensions import cache_manager
from superset.models.dashboard import Dashboard
from superset.temporary_cache.utils import cache_key
from superset.utils import json
from tests.integration_tests.fixtures.world_bank_dashboard import (
load_world_bank_dashboard_with_slices, # noqa: F401
load_world_bank_data, # noqa: F401
)
from tests.integration_tests.test_app import app # noqa: F401
KEY = "test-key"
INITIAL_VALUE = json.dumps({"test": "initial value"})
UPDATED_VALUE = json.dumps({"test": "updated value"})
@pytest.fixture
def dashboard_id(app_context: AppContext, load_world_bank_dashboard_with_slices) -> int: # noqa: F811
dashboard = db.session.query(Dashboard).filter_by(slug="world_health").one()
return dashboard.id
@pytest.fixture
def admin_id(app_context: AppContext) -> int:
admin = db.session.query(User).filter_by(username="admin").one_or_none()
return admin.id
@pytest.fixture(autouse=True)
def cache(dashboard_id, admin_id):
entry: Entry = {"owner": admin_id, "value": INITIAL_VALUE}
cache_manager.filter_state_cache.set(cache_key(dashboard_id, KEY), entry)
def test_post(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": INITIAL_VALUE,
},
)
assert resp.status_code == 201
def test_post_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_post_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": "foo",
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 400
def test_post_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 404
def test_post_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_post_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_post_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 200
def test_put_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_put_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_put_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": "foo",
},
)
assert resp.status_code == 400
def test_put_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 404
@patch("superset.commands.dashboard.filter_state.create.check_access")
def test_post_authenticated_user_with_access(
mock_check_access, test_client, login_as, dashboard_id: int
):
login_as("alpha")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 201
mock_check_access.assert_called_once_with(dashboard_id)
@patch("superset.commands.dashboard.filter_state.create.check_access")
@patch("superset.commands.dashboard.filter_state.update.check_access")
def test_put_authenticated_user_with_access(
mock_update_check_access,
mock_create_check_access,
test_client,
login_as,
dashboard_id: int,
):
login_as("alpha")
payload = {
"value": INITIAL_VALUE,
}
post_resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert post_resp.status_code == 201
key = json.loads(post_resp.data.decode("utf-8"))["key"]
put_payload = {
"value": UPDATED_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{key}", json=put_payload
)
assert resp.status_code == 200
mock_create_check_access.assert_called_once_with(dashboard_id)
mock_update_check_access.assert_called_once_with(dashboard_id)
def test_get_key_not_found(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/unknown-key/")
assert resp.status_code == 404
def test_get_dashboard_not_found(test_client, login_as_admin):
resp = test_client.get(f"api/v1/dashboard/{-1}/filter_state/{KEY}")
assert resp.status_code == 404
def test_get_dashboard_filter_state(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
data = json.loads(resp.data.decode("utf-8"))
assert INITIAL_VALUE == data.get("value")
def test_get_access_denied(test_client, login_as, dashboard_id):
login_as("gamma")
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete(test_client, login_as_admin, dashboard_id: int):
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
def test_delete_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete_not_owner(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from unittest.mock import patch # noqa: F401
import pytest
from flask.ctx import AppContext
from flask_appbuilder.security.sqla.models import User
from sqlalchemy.orm import Session # noqa: F401
from superset import db
from superset.commands.dashboard.exceptions import (
DashboardAccessDeniedError, # noqa: F401
)
from superset.commands.temporary_cache.entry import Entry
from superset.extensions import cache_manager
from superset.models.dashboard import Dashboard
from superset.temporary_cache.utils import cache_key
from superset.utils import json
from tests.integration_tests.fixtures.world_bank_dashboard import (
load_world_bank_dashboard_with_slices, # noqa: F401
load_world_bank_data, # noqa: F401
)
from tests.integration_tests.test_app import app # noqa: F401
KEY = "test-key"
INITIAL_VALUE = json.dumps({"test": "initial value"})
UPDATED_VALUE = json.dumps({"test": "updated value"})
@pytest.fixture
def dashboard_id(app_context: AppContext, load_world_bank_dashboard_with_slices) -> int: # noqa: F811
dashboard = db.session.query(Dashboard).filter_by(slug="world_health").one()
return dashboard.id
@pytest.fixture
def admin_id(app_context: AppContext) -> int:
admin = db.session.query(User).filter_by(username="admin").one_or_none()
return admin.id
@pytest.fixture(autouse=True)
def cache(dashboard_id, admin_id):
entry: Entry = {"owner": admin_id, "value": INITIAL_VALUE}
cache_manager.filter_state_cache.set(cache_key(dashboard_id, KEY), entry)
def test_post(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": INITIAL_VALUE,
},
)
assert resp.status_code == 201
def test_post_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_post_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": "foo",
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 400
def test_post_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
assert resp.status_code == 404
def test_post_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_post_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_post_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.post(
f"api/v1/dashboard/{dashboard_id}/filter_state", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 200
def test_put_same_key_for_same_tab_id(test_client, login_as_admin, dashboard_id: int):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key == second_key
def test_put_different_key_for_different_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=1", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}?tab_id=2", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_different_key_for_no_tab_id(
test_client, login_as_admin, dashboard_id: int
):
payload = {
"value": INITIAL_VALUE,
}
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
first_key = data.get("key")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}", json=payload
)
data = json.loads(resp.data.decode("utf-8"))
second_key = data.get("key")
assert first_key != second_key
def test_put_bad_request_non_string(test_client, login_as_admin, dashboard_id: int):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": 1234,
},
)
assert resp.status_code == 400
def test_put_bad_request_non_json_string(
test_client, login_as_admin, dashboard_id: int
):
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": "foo",
},
)
assert resp.status_code == 400
def test_put_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.put(
f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}",
json={
"value": UPDATED_VALUE,
},
)
assert resp.status_code == 404
def test_get_key_not_found(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/unknown-key/")
assert resp.status_code == 404
def test_get_dashboard_not_found(test_client, login_as_admin):
resp = test_client.get(f"api/v1/dashboard/{-1}/filter_state/{KEY}")
assert resp.status_code == 404
def test_get_dashboard_filter_state(test_client, login_as_admin, dashboard_id: int):
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
data = json.loads(resp.data.decode("utf-8"))
assert INITIAL_VALUE == data.get("value")
def test_get_access_denied(test_client, login_as, dashboard_id):
login_as("gamma")
resp = test_client.get(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete(test_client, login_as_admin, dashboard_id: int):
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 200
def test_delete_access_denied(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
def test_delete_not_owner(test_client, login_as, dashboard_id: int):
login_as("gamma")
resp = test_client.delete(f"api/v1/dashboard/{dashboard_id}/filter_state/{KEY}")
assert resp.status_code == 404
@@ -2487,15 +2487,8 @@ class TestDatabaseApi(SupersetTestCase):
}
assert response == expected_response
# Uses a `dialect+driver://` URI (rather than the bare `broken://`
# above) to also cover engine-name extraction stripping the driver
# suffix. The dialect itself ("broken") must stay one that no
# installed extra ever registers a real SQLAlchemy plugin for --
# this PR's own testcontainers extras (mssql, oracle, db2, ...)
# install real drivers for those dialects, which would make this
# URI actually attempt a connection instead of failing to load.
data = {
"sqlalchemy_uri": "broken+driver://url",
"sqlalchemy_uri": "mssql+pymssql://url",
"database_name": "examples",
"impersonate_user": False,
"server_cert": None,
@@ -2507,7 +2500,7 @@ class TestDatabaseApi(SupersetTestCase):
expected_response = {
"errors": [
{
"message": "Could not load database driver for: broken",
"message": "Could not load database driver for: mssql",
"error_type": "GENERIC_COMMAND_ERROR",
"level": "warning",
"extra": {
@@ -1750,10 +1750,6 @@ class TestRolePermission(SupersetTestCase):
# user/tenant data) as content-addressed scripts; must load for
# anonymous principals (login page, embedded dashboards).
["Superset", "language_pack_script"],
# Language pack endpoint serves JS bundle translations, no auth
# needed; embedded dashboards fetch this without a guest-token
# header, so it must be reachable unauthenticated.
["Superset", "language_pack"],
]
unsecured_views = []
for view_class in appbuilder.baseviews:
-16
View File
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
@@ -1,47 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Shared import guard for the per-dialect testcontainers modules
(tests/testcontainers/db_engine_specs/test_*.py), each of which needs its
own optional `testcontainers[...]` driver submodule to even import.
"""
import importlib
import os
def require_driver(module_name: str) -> None:
"""
Import `module_name`, a dialect's `testcontainers` driver submodule.
Most environments treat that driver as optional: a bare local `pytest`
run, or another CI job that never installed the `testcontainers` extras,
should skip the module rather than fail collection outright.
The dedicated per-dialect CI job (.github/workflows/testcontainers.yml)
sets SUPERSET_TESTCONTAINERS_STRICT, because there the driver is not
optional -- that job's matrix installs exactly this one driver for
exactly this one module. A broken or missing import there means the job
is misconfigured, and should fail loudly instead of silently reporting
a misleadingly green, zero-tests-run result.
"""
if os.environ.get("SUPERSET_TESTCONTAINERS_STRICT"):
importlib.import_module(module_name)
else:
import pytest
pytest.importorskip(module_name)
@@ -1,68 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Shared body for the "paginated query returns correct rows in order" test
that db_engine_specs.{cockroachdb,crate,db2,mssql,oracle,trino}'s
testcontainers suites each run against their own real instance: a plain
SQLAlchemy Core LIMIT/OFFSET query, compiled and executed for real. Mocked
tests cannot catch a dialect compiling this incorrectly (see
apache/superset#42899, where Trino emitted OFFSET before LIMIT) -- only
real execution can.
Each call site keeps its own test function (and dialect-specific docstring)
so failures still report against the right module; this only factors out
the identical table setup/assert body, via an optional post-insert hook for
dialects (CrateDB) that need one, and an optional extra-table-args hook for
dialects (ClickHouse) whose CREATE TABLE requires a schema item a plain
Column/primary key can't express.
"""
from collections.abc import Callable
from typing import Any
from sqlalchemy import Column, insert, Integer, MetaData, select, Table as SATable
from sqlalchemy.engine import Connection, Engine
def assert_paginated_query_returns_correct_rows_in_order(
engine: Engine,
after_insert: Callable[[Connection], None] | None = None,
extra_table_args: tuple[Any, ...] = (),
) -> None:
metadata = MetaData()
t = SATable(
"pilot_pagination",
metadata,
# autoincrement=False: a single-column integer primary key otherwise
# implicitly becomes AUTO_INCREMENT on MySQL/MariaDB. That column
# type treats an explicit 0 as NULL by default (NO_AUTO_VALUE_ON_ZERO
# is off), so the id=0 row below would silently get auto-assigned 1,
# colliding with the explicit id=1 row in the same batch insert.
Column("id", Integer, primary_key=True, autoincrement=False),
*extra_table_args,
)
metadata.create_all(engine)
with engine.begin() as conn:
conn.execute(insert(t), [{"id": i} for i in range(10)])
if after_insert is not None:
after_insert(conn)
with engine.connect() as conn:
stmt = select(t.c.id).order_by(t.c.id).limit(3).offset(4)
rows = conn.execute(stmt).fetchall()
assert [row.id for row in rows] == [4, 5, 6]
@@ -1,130 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.clickhouse against a real ClickHouse instance, spun
up on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Superset's recommended ClickHouse connector is `clickhouse-connect`
(`ClickHouseConnectEngineSpec`, engine "clickhousedb"), which talks HTTP,
not `ClickHouseContainer`'s own documented `clickhouse_driver` (a different,
native-TCP-protocol package Superset doesn't use at all). The container
exposes both the native TCP port (9000) and the HTTP port (8123); this test
connects over the HTTP port to match Superset's actual driver.
Unlike every other dialect in this suite, ClickHouse tables have no real
primary key/constraint concept -- CREATE TABLE requires an explicit engine
(e.g. MergeTree), or clickhouse-connect's DDL compiler raises a CompileError
rather than defaulting to one.
`superset.db_engine_specs.clickhouse` runs module-level setup code (default
type-formatting overrides) that dereferences `current_app.config` whenever
clickhouse-connect is installed, so importing it outside a Flask app context
raises RuntimeError the first time it's imported in a process.
`tests/unit_tests/db_engine_specs/test_clickhouse.py` gets an app context
for free from that suite's autouse fixture; this suite has no such fixture,
so this test pushes one explicitly around just that one-time import, reusing
the real app instance `tests/conftest.py` already builds for the rest of the
test run rather than constructing a second one.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.clickhouse")
require_driver("clickhouse_connect")
from clickhouse_connect.cc_sqlalchemy.ddl.tableengine import MergeTree # noqa: E402
from testcontainers.community.clickhouse import ClickHouseContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
HTTP_PORT = 8123
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with ClickHouseContainer("clickhouse/clickhouse-server:latest") as container:
host = container.get_container_host_ip()
port = container.get_exposed_port(HTTP_PORT)
yield create_engine(
f"clickhousedb://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(
engine, extra_table_args=(MergeTree(order_by="id"),)
)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
ClickHouseConnectEngineSpec.get_columns wraps a real SQLAlchemy
Inspector; this exercises that against actual server-reported column
metadata rather than a mocked Inspector.
"""
from tests.integration_tests.test_app import app
with app.app_context():
from superset.db_engine_specs.clickhouse import ClickHouseConnectEngineSpec
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
MergeTree(order_by="id"),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = ClickHouseConnectEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = ClickHouseConnectEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,97 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.cockroachdb against a real CockroachDB instance,
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml -- these exercise real SQL execution
and dialect introspection, which mocked unit tests structurally cannot.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.cockroachdb import CockroachDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.cockroachdb")
from testcontainers.community.cockroachdb import CockroachDBContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
# sqlalchemy-cockroachdb registers its dialect under the plain
# "cockroachdb" name; the container's own default ("cockroachdb+psycopg2")
# matches the abandoned `cockroachdb` package instead (see #43501).
with CockroachDBContainer(dialect="cockroachdb") as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
CockroachDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported column metadata
rather than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = CockroachDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = CockroachDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,103 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.crate against a real CrateDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
crate/crate only publishes an amd64 image (no arm64 build), and requires a
host CPU supporting the x86-64-v3 instruction set -- QEMU emulation on
Apple Silicon cannot satisfy that, so this file cannot run locally on an
Apple Silicon machine even with `docker pull --platform linux/amd64`. It
runs natively on GitHub Actions' x86_64 runners.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Connection, Engine
from superset.db_engine_specs.crate import CrateEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.cratedb")
from testcontainers.community.cratedb import CrateDBContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with CrateDBContainer() as container:
yield create_engine(container.get_connection_url())
def _refresh_pilot_pagination(conn: Connection) -> None:
# CrateDB is eventually consistent: a row is not guaranteed visible to
# subsequent selects immediately after insert.
conn.exec_driver_sql("REFRESH TABLE pilot_pagination")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(
engine, after_insert=_refresh_pilot_pagination
)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
CrateEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = CrateEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = CrateEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,114 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.databend against a real Databend instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Databend has no dedicated testcontainers module, so this uses a generic
DockerContainer against the official `datafuselabs/databend` standalone
image. Superset's DatabendEngineSpec defaults to `sslmode=require`
(`encryption_parameters`), but the local standalone image has no TLS
listener, so this connects with `sslmode=disable` explicitly.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.databend import DatabendEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("databend_sqlalchemy")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
HTTP_PORT = 8000
DBNAME = "default"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("datafuselabs/databend")
container.with_exposed_ports(HTTP_PORT)
# The image's own startup banner documents this exact line as proof its
# HTTP query endpoint is bound and ready.
container.waiting_for(LogMessageWaitStrategy(f"listened at 0.0.0.0:{HTTP_PORT}"))
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(HTTP_PORT)
# "root" with no password is the image's builtin user -- confirmed
# directly against a running container, not from the image's own
# doc text, which only shows ${USER}/${PASSWORD} placeholders.
yield create_engine(f"databend://root:@{host}:{port}/{DBNAME}?sslmode=disable")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
DatabendEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = DatabendEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = DatabendEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,95 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.db2 against a real IBM Db2 instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
icr.io/db2_community/db2 only publishes amd64/ppc64le/s390x images (no
arm64 build), so this cannot run locally on an Apple Silicon machine. It
runs natively on GitHub Actions' x86_64 runners. Db2 is also a notably slow
starter (a full instance bring-up, not just a process start) -- expect this
module alone to take several minutes.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.db2 import Db2EngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.db2")
from testcontainers.community.db2 import Db2Container # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with Db2Container() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
Db2EngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = Db2EngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = Db2EngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,107 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.elasticsearch against a real Elasticsearch instance,
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml.
Unlike the SQL-native dialects in this directory, Elasticsearch has no
CREATE TABLE / INSERT: indices and documents get created via its REST API
(elasticsearch-dbapi's SQLAlchemy dialect is read-focused, translating SQL
to the _sql endpoint), matching how Superset actually encounters
Elasticsearch in practice -- data arrives via ingestion tooling, not
through Superset itself.
"""
from collections.abc import Iterator
import pytest
import requests
from sqlalchemy import create_engine, inspect, text
from sqlalchemy.engine import Engine
from superset.db_engine_specs.elasticsearch import ElasticSearchEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.elasticsearch")
from testcontainers.community.elasticsearch import ElasticSearchContainer # noqa: E402
INDEX = "pilot_pagination"
def _index_document(
base_url: str, index: str, doc_id: int, body: dict[str, int]
) -> None:
response = requests.put(f"{base_url}/{index}/_doc/{doc_id}", json=body, timeout=10)
response.raise_for_status()
def _refresh(base_url: str, index: str) -> None:
response = requests.post(f"{base_url}/{index}/_refresh", timeout=10)
response.raise_for_status()
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with ElasticSearchContainer("elasticsearch:8.11.0") as container:
host = container.get_container_host_ip()
port = container.get_exposed_port(container.port)
base_url = f"http://{host}:{port}"
for i in range(10):
_index_document(base_url, INDEX, i, {"id": i})
_refresh(base_url, INDEX)
yield create_engine(f"elasticsearch+http://{host}:{port}/")
def test_ordered_limited_query_returns_correct_rows(engine: Engine) -> None:
"""
A plain LIMIT query, compiled and executed against a real instance.
Mocked tests cannot catch a dialect compiling this incorrectly (see
apache/superset#42899, where Trino emitted OFFSET before LIMIT) -- only
real execution can. No OFFSET here: Elasticsearch's SQL layer genuinely
doesn't support it (a protocol limitation, not a bug -- confirmed
against a real instance, which raises a parsing_exception on OFFSET).
ElasticSearchEngineSpec.supports_offset = False documents this already.
"""
with engine.connect() as conn:
rows = conn.execute(
text(f"SELECT id FROM {INDEX} ORDER BY id LIMIT 3") # noqa: S608
).fetchall()
assert [row.id for row in rows] == [0, 1, 2]
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
ElasticSearchEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported field mappings
rather than a mocked Inspector.
"""
inspector = inspect(engine)
columns = ElasticSearchEngineSpec.get_columns(inspector, Table(INDEX))
by_name = {col["column_name"]: col for col in columns}
assert "id" in by_name
spec = ElasticSearchEngineSpec.get_column_spec(str(by_name["id"]["type"]))
assert spec is not None
@@ -1,146 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.firebird against a real Firebird instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Firebird references a database *file* rather than a server-managed named
database -- the connection URI is `firebird://user:pass@host:port/<path>`,
where <path> is the path to a .fdb file on the server. This uses the
well-known `jacobalberty/firebird` image, which creates that file (per
FIREBIRD_DATABASE) under /firebird/data on first boot.
FirebirdEngineSpec sets `limit_method = LimitMethod.FETCH_MANY` with a
comment claiming Firebird "uses FIRST to limit" -- stale relative to the
modern sqlalchemy-firebird driver, which compiles real ROWS-based
pagination (confirmed via an offline dialect compile: `SELECT ... ROWS
4 + 1 TO 4 + 3`, correctly ordered, not a Trino-style bug). That staleness
affects what Superset's own query layer emits, not what this suite's
direct dialect-compilation check exercises.
Could not be verified against a real running instance in this
environment: `firebird-driver` is a pure-Python ctypes wrapper (its wheel
is `py3-none-any`, confirmed by downloading it directly) that dynamically
loads the native Firebird client library (`libfbclient`) from the host at
import time -- it doesn't bundle that library itself. This machine has no
Homebrew formula or straightforward install path for it. The container
itself was confirmed to start and pass its own healthcheck; CI installs
the `libfbclient2` system package separately (see
.github/workflows/testcontainers.yml) for the actual client-library
dependency this driver needs.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.firebird import FirebirdEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("firebird.driver")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import HealthcheckWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 3050
PASSWORD = "masterkey" # noqa: S105 -- fixed test-fixture password, not a secret
DB_FILE = "test.fdb"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("jacobalberty/firebird")
container.with_exposed_ports(PORT)
container.with_env("ISC_PASSWORD", PASSWORD)
container.with_env("FIREBIRD_DATABASE", DB_FILE)
# The image logs nothing beyond a single startup banner line and never
# prints a distinct "ready" message -- it ships its own Docker
# HEALTHCHECK instead, confirmed via `docker ps` reporting (healthy).
container.waiting_for(HealthcheckWaitStrategy())
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
eng = create_engine(
f"firebird://sysdba:{PASSWORD}@{host}:{port}//firebird/data/{DB_FILE}"
)
yield eng
# firebird-driver registers its own atexit handler that talks to
# the Firebird subsystem to shut it down cleanly. Without disposing
# here first, that handler fires at interpreter exit against a
# server the container has *already* torn down -- confirmed on
# real CI as a segfault (exit code 139) after both tests had
# already passed. Disposing while the server is still up lets the
# driver close out normally, so the later atexit call has nothing
# left to talk to.
eng.dispose()
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
FirebirdEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = FirebirdEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = FirebirdEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,117 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mariadb against a real MariaDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
MariaDB is a MySQL fork implementing the same wire protocol: connects via
the plain "mysql" dialect with mysqlclient, same as vanilla MySQL, just
pointed at the mariadb image instead of mysql:latest.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient. CI installs it via apt on
Linux, where this does not occur.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mariadb import MariaDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mysql")
from testcontainers.community.mysql import MySqlContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MySqlContainer("mariadb:11") as container:
# get_connection_url() has no host override and defaults to
# get_container_host_ip(), which is the literal string "localhost"
# on native Linux Docker (e.g. GitHub Actions runners). MySQLdb
# (mysqlclient) treats a "localhost" host specially and attempts a
# Unix socket connection instead of TCP, which fails since there's
# no local MySQL socket -- the container is reached over the
# network. Only rewrite that specific local case to 127.0.0.1; a
# remote Docker daemon reports its own real host/IP here, which
# must be preserved so the suite can still reach it.
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(container.port)
yield create_engine(
f"mysql://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MariaDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MariaDBEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MariaDBEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,127 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.monetdb against a real MonetDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
monetdb/monetdb publishes an amd64-only image, so this needs Rosetta/QEMU
emulation on Apple Silicon -- unlike CrateDB's x86-64-v3 CPU requirement,
this one actually runs fine under emulation (verified locally). No native
testcontainers module exists for MonetDB, so this uses a generic
DockerContainer with the documented MDB_* environment variables and waits
for the daemon's own startup log line.
"""
import re
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.monetdb import MonetDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("sqlalchemy_monetdb")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import ( # noqa: E402
CompositeWaitStrategy,
LogMessageWaitStrategy,
PortWaitStrategy,
)
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 50000
PASSWORD = "monetdb" # noqa: S105 -- fixed test-fixture password, not a secret
DBNAME = "test"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("monetdb/monetdb:latest")
container.with_exposed_ports(PORT)
container.with_env("MDB_DB_ADMIN_PASS", PASSWORD)
container.with_env("MDB_CREATE_DBS", DBNAME)
# The "Starting MonetDB daemon" log line is emitted before the image
# actually runs `monetdbd start -n`, so it alone isn't proof the server
# is accepting connections yet. Follow it with a port-connect check,
# which only succeeds once monetdbd is really listening.
container.waiting_for(
CompositeWaitStrategy(
LogMessageWaitStrategy(re.compile("Starting MonetDB daemon")),
PortWaitStrategy(PORT),
)
)
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
yield create_engine(f"monetdb://monetdb:{PASSWORD}@{host}:{port}/{DBNAME}")
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MonetDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MonetDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MonetDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,141 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mongodb against a real MongoDB instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
MongoDB is schemaless, and Superset talks to it via `pymongosql`, a
SQL-to-MongoDB translation layer (dialect requires a `?mode=superset` query
param -- not part of testcontainers' own MongoDbContainer.get_connection_url()).
Documents get inserted via the native pymongo driver, not SQL INSERT,
matching how Superset actually encounters MongoDB in practice and avoiding
any assumption about pymongosql's own INSERT/DDL support.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
column,
create_engine,
inspect,
Integer,
select,
table,
text,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mongodb import MongoDBEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mongodb")
require_driver("pymongosql")
from testcontainers.community.mongodb import MongoDbContainer # noqa: E402
COLLECTION = "pilot_pagination"
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MongoDbContainer("mongo:7.0.7") as container:
client = container.get_connection_client()
client[container.dbname][COLLECTION].insert_many([{"id": i} for i in range(10)])
# MongoDbContainer.get_connection_url() has no database path segment
# or query string at all (it only builds user:pass@host:port), so
# naively appending "&mode=superset" glues it straight onto the port
# number instead of starting a query string. Build the full URL
# ourselves instead of relying on string concatenation.
host = container.get_container_host_ip()
port = container.get_exposed_port(container.port)
# authSource=admin is required: MongoDbContainer creates its root
# user via MONGO_INITDB_ROOT_USERNAME, which lives in the `admin`
# database, not in `dbname` -- without it, auth fails against
# whatever database is in the URL path.
yield create_engine(
f"mongodb://{container.username}:{container.password}@{host}:{port}"
f"/{container.dbname}?mode=superset&authSource=admin"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed
against a real instance. Mocked tests cannot catch a dialect compiling
this incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can. Unlike Elasticsearch's SQL
layer (which has no OFFSET support at all), pymongosql maps OFFSET to
MongoDB's native `skip`, so this dialect supports it.
Compiled with `literal_binds=True`, matching how Superset actually
issues chart/SQL Lab queries (see `models/helpers.py`'s
`get_query_str_extended`): pymongosql's SQL-to-Mongo AST parser reads
LIMIT/OFFSET straight off the compiled SQL text ahead of parameter
substitution, so a bound `LIMIT ?`/`OFFSET ?` placeholder is rejected
("invalid literal for int() with base 10: '?'") and the clause is
silently dropped -- unlike its WHERE-clause parameter handling, which
does substitute correctly. Literal binds sidestep that and exercise
the dialect's actual LIMIT/OFFSET compilation, per this test's intent.
Uses a bare `column("id")`/`table(...)` pair rather than a full
`Table`-bound column: SQLAlchemy always qualifies a Table-bound column
reference as `pilot_pagination.id` once there's a FROM clause, and
pymongosql's projection builder takes that qualified text completely
literally as a MongoDB field path -- `{"pilot_pagination.id": 1}` reads
a *nested* field under a top-level `pilot_pagination` key, which
doesn't exist on these flat documents, silently projecting None instead
of raising. An unbound column compiles unqualified ("id"), which
resolves correctly, while still exercising the dialect's own
LIMIT/OFFSET compilation via a real Core `select()`.
"""
id_col = column("id")
stmt = (
select(id_col)
.select_from(table(COLLECTION))
.order_by(id_col)
.limit(3)
.offset(4)
)
compiled = stmt.compile(engine, compile_kwargs={"literal_binds": True})
with engine.connect() as conn:
rows = conn.execute(text(str(compiled))).fetchall()
assert [row.id for row in rows] == [4, 5, 6]
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MongoDBEngineSpec.get_columns wraps a real SQLAlchemy Inspector, which
pymongosql implements by sampling real documents to infer column types
-- this exercises that against an actual running instance rather than
a mocked Inspector.
"""
inspector = inspect(engine)
columns = MongoDBEngineSpec.get_columns(inspector, Table(COLLECTION))
by_name = {col["column_name"]: col for col in columns}
assert "id" in by_name
spec = MongoDBEngineSpec.get_column_spec(str(by_name["id"]["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,93 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mssql against a real SQL Server instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
mcr.microsoft.com/mssql/server only publishes an amd64 image (SQL Server on
Linux has no ARM build), so this cannot run locally on an Apple Silicon
machine. It runs natively on GitHub Actions' x86_64 runners.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mssql import MssqlEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mssql")
from testcontainers.community.mssql import SqlServerContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with SqlServerContainer() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MssqlEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MssqlEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MssqlEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,117 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.mysql against a real MySQL instance, spun up on
demand via testcontainers. Run via .github/workflows/testcontainers.yml.
Plain MySQL itself was never covered by this suite: MariaDB and StarRocks
both reuse `MySQLEngineSpec`'s plain "mysql" dialect via mysqlclient, but
neither stands in for vanilla MySQL server's own dialect quirks.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient. CI installs it via apt on
Linux, where this does not occur.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.mysql import MySQLEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.mysql")
from testcontainers.community.mysql import MySqlContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with MySqlContainer("mysql:8.0") as container:
# get_connection_url() has no host override and defaults to
# get_container_host_ip(), which is the literal string "localhost"
# on native Linux Docker (e.g. GitHub Actions runners). MySQLdb
# (mysqlclient) treats a "localhost" host specially and attempts a
# Unix socket connection instead of TCP, which fails since there's
# no local MySQL socket -- the container is reached over the
# network. Only rewrite that specific local case to 127.0.0.1; a
# remote Docker daemon reports its own real host/IP here, which
# must be preserved so the suite can still reach it.
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(container.port)
yield create_engine(
f"mysql://{container.username}:{container.password}"
f"@{host}:{port}/{container.dbname}"
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
MySQLEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = MySQLEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = MySQLEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,221 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.oceanbase against a real OceanBase instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml,
on the nightly cron / manual dispatch only (see `nightly_only: true` on
this dialect's matrix entry) -- OceanBase bootstraps a distributed-style
cluster even in single-node MODE=MINI, a substantially heavier first-boot
than a single-process database, not a good fit for every PR's CI budget.
OceanBaseEngineSpec extends MySQLEngineSpec and its dialect
(oceanbase_py.sqlalchemy.dialect.OceanBaseDialect) extends
MySQLDialect_mysqldb directly with no custom DDL or LIMIT/OFFSET compiler,
so this follows the same mysqlclient-based pattern as MariaDB/MySQL/
StarRocks in this suite -- including the same "localhost" -> "127.0.0.1"
fix MySQLdb needs on native Linux Docker.
Could not be verified locally in this environment: mysqlclient (MySQLdb)
has a pre-existing, unrelated native-library linking issue against this
machine's Homebrew-installed libmysqlclient, and this dialect wasn't
pulled/run locally at all given its heavier resource footprint -- CI-only
verification, matching the nightly_only gating.
oceanbase_py.sqlalchemy.dialect.OceanBaseDialect has real bugs, all
confirmed on real CI, in both has_table() (called by create_all()'s
default checkfirst=True) and get_columns() (called by
OceanBaseEngineSpec.get_columns(), which this suite's second test needs
to actually exercise):
1. Both pass a raw string straight to Connection.execute() (e.g.
`connection.execute(f"DESCRIBE {full_name}")`), which SQLAlchemy 2.0
rejects outright (ObjectNotExecutableError). Every *other* raw-SQL
method in the same dialect module correctly uses
`connection.exec_driver_sql(...)` instead.
2. has_table() never catches the error DESCRIBE raises for a table that
doesn't exist (1146) -- so even with (1) fixed, it can only ever
return True, raising instead of returning False for exactly the case
checkfirst exists to handle.
This test monkeypatches both methods to do what the rest of the dialect's
raw-SQL methods already do, plus has_table()'s missing not-found
handling, rather than working around any of this from the test side.
"""
from collections.abc import Iterator
from typing import Any
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Connection, Engine, URL
from sqlalchemy.exc import NoSuchTableError, ProgrammingError
from superset.db_engine_specs.oceanbase import OceanBaseEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("oceanbase_py")
from oceanbase_py.sqlalchemy import datatype # noqa: E402
from oceanbase_py.sqlalchemy.dialect import OceanBaseDialect # noqa: E402
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
def _has_table(
self: OceanBaseDialect,
connection: Connection,
table_name: str,
schema: str | None = None,
**kw: object,
) -> bool:
if schema is None:
schema = self.default_schema_name
quote = self.identifier_preparer.quote_identifier
full_name = quote(table_name)
if schema:
full_name = f"{quote(schema)}.{full_name}"
try:
res = connection.exec_driver_sql(f"DESCRIBE {full_name}")
except ProgrammingError:
# The original never catches this at all -- DESCRIBE on a
# nonexistent table raises 1146 ("table doesn't exist") rather
# than returning an empty result set, so the unpatched method can
# only ever return True, and raises instead of returning False for
# exactly the case create_all()'s checkfirst exists to handle
# (confirmed on real CI).
return False
return res.first() is not None
def _get_columns(
self: OceanBaseDialect,
connection: Connection,
table_name: str,
schema: str | None = None,
**kw: object,
) -> list[dict[str, Any]]:
# Same connection.execute(raw string) anti-pattern as has_table(),
# confirmed on real CI as the same ObjectNotExecutableError -- this is
# the actual column-introspection call OceanBaseEngineSpec.get_columns
# (and this test) needs, so it gets the same exec_driver_sql fix.
if not self.has_table(connection, table_name, schema):
raise NoSuchTableError(f"schema={schema}, table={table_name}")
schema = schema or self._get_default_schema_name(connection)
quote = self.identifier_preparer.quote_identifier
full_name = quote(table_name)
if schema:
full_name = f"{quote(schema)}.{full_name}"
res = connection.exec_driver_sql(f"SHOW COLUMNS FROM {full_name}")
return [
{
"name": record.Field,
"type": datatype.parse_sql_type(record.Type),
"nullable": record.Null == "YES",
"default": record.Default,
}
for record in res
]
OceanBaseDialect.has_table = _has_table
OceanBaseDialect.get_columns = _get_columns
PORT = 2881
PASSWORD = "pilot" # noqa: S105 -- fixed test-fixture password, not a secret
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("oceanbase/oceanbase-ce")
container.with_exposed_ports(PORT)
container.with_env("MODE", "MINI")
container.with_env("OB_TENANT_PASSWORD", PASSWORD)
container.waiting_for(LogMessageWaitStrategy("boot success!"))
with container:
host = container.get_container_host_ip()
if host == "localhost":
host = "127.0.0.1"
port = container.get_exposed_port(PORT)
# OceanBase usernames for a MySQL-mode tenant use "user@tenant"
# (e.g. "root@test"), a literal "@" that URL.create() percent-encodes
# correctly -- an f-string would produce a second "@" that breaks
# the URL's own host/user boundary parsing.
yield create_engine(
URL.create(
"oceanbase",
username="root@test",
password=PASSWORD,
host=host,
port=int(port),
database="test",
)
)
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
OceanBaseEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = OceanBaseEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = OceanBaseEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,95 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.oracle against a real Oracle instance (gvenzl/oracle-free),
spun up on demand via testcontainers. Run via
.github/workflows/testcontainers.yml.
gvenzl/oracle-free ships with its datafiles pre-baked into the image, so
once the (large-ish, ~1GB) image is pulled, container startup is fast --
under 15s measured locally. Almost all the wall-clock cost here is the
image pull itself, same as any other dialect's container.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.oracle import OracleEngineSpec
from superset.sql.parse import Table
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.oracle")
from testcontainers.community.oracle import OracleDbContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with OracleDbContainer() as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
OracleEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = OracleEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = OracleEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
@@ -1,99 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.postgres against a real PostgreSQL instance, spun up
on demand via testcontainers. Run via .github/workflows/testcontainers.yml
-- these exercise real SQL execution and dialect introspection, which
mocked unit tests structurally cannot.
Plain Postgres itself was never covered by this suite: CockroachDB,
TimescaleDB and YugabyteDB all speak the Postgres wire protocol and already
exercise `PostgresContainer`/the "postgresql" dialect, but none of them
stand in for vanilla PostgreSQL's own dialect quirks.
"""
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
)
from sqlalchemy.engine import Engine
from superset.db_engine_specs.postgres import PostgresEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.community.postgres")
from testcontainers.community.postgres import PostgresContainer # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
with PostgresContainer("postgres:17-alpine") as container:
yield create_engine(container.get_connection_url())
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
PostgresEngineSpec.get_columns wraps a real SQLAlchemy Inspector; this
exercises that against actual server-reported column metadata rather
than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = PostgresEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = PostgresEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)
@@ -1,127 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests db_engine_specs.risingwave against a real RisingWave instance, spun
up on demand via testcontainers. Run via .github/workflows/testcontainers.yml.
RisingWave speaks the Postgres wire protocol, but doesn't run the real
Postgres server binary or its POSTGRES_PASSWORD-style bootstrap env vars,
so this can't reuse `PostgresContainer` the way TimescaleDB/YugabyteDB do
-- it needs a generic DockerContainer against the official
`risingwavelabs/risingwave` single-binary playground image instead.
`RisingWaveDbEngineSpec` extends `PostgresEngineSpec`, and
`sqlalchemy-risingwave`'s dialect is a genuine subclass of SQLAlchemy's own
Postgres dialect (via psycopg2), so DDL/pagination compile with standard
Postgres semantics -- no ClickHouse-style mandatory table option needed.
"""
import re
from collections.abc import Iterator
import pytest
from sqlalchemy import (
Column,
create_engine,
inspect,
Integer,
MetaData,
Table as SATable,
text,
)
from sqlalchemy.engine import Connection, Engine
from superset.db_engine_specs.risingwave import RisingWaveDbEngineSpec
from superset.sql.parse import Table
from superset.utils.core import GenericDataType
pytestmark = pytest.mark.testcontainers
from ._driver import require_driver # noqa: E402
require_driver("testcontainers.core.container")
require_driver("sqlalchemy_risingwave")
from testcontainers.core.container import DockerContainer # noqa: E402
from testcontainers.core.wait_strategies import LogMessageWaitStrategy # noqa: E402
from ._pagination import ( # noqa: E402
assert_paginated_query_returns_correct_rows_in_order,
)
PORT = 4566
@pytest.fixture(scope="module")
def engine() -> Iterator[Engine]:
container = DockerContainer("risingwavelabs/risingwave")
container.with_exposed_ports(PORT)
container.with_command("playground")
# The actual startup banner reads "RisingWave standalone mode is
# ready." -- confirmed against a real container's logs.
container.waiting_for(
LogMessageWaitStrategy(re.compile("RisingWave standalone mode is ready"))
)
with container:
host = container.get_container_host_ip()
port = container.get_exposed_port(PORT)
yield create_engine(f"risingwave://root@{host}:{port}/dev")
def _flush(conn: Connection) -> None:
# RisingWave's storage engine checkpoints asynchronously: without an
# explicit FLUSH, a SELECT immediately after INSERT can see zero rows
# -- confirmed against a real instance (a bare INSERT commits fine, but
# the data isn't visible to a subsequent query until flushed).
conn.execute(text("FLUSH"))
def test_paginated_query_returns_correct_rows_in_order(engine: Engine) -> None:
"""
A plain SQLAlchemy Core LIMIT/OFFSET query, compiled and executed against
a real instance. Mocked tests cannot catch a dialect compiling this
incorrectly (see apache/superset#42899, where Trino emitted OFFSET
before LIMIT) -- only real execution can.
"""
assert_paginated_query_returns_correct_rows_in_order(engine, after_insert=_flush)
def test_get_columns_maps_native_types(engine: Engine) -> None:
"""
RisingWaveDbEngineSpec.get_columns wraps a real SQLAlchemy Inspector;
this exercises that against actual server-reported column metadata
rather than a mocked Inspector.
"""
metadata = MetaData()
SATable(
"pilot_types",
metadata,
Column("id", Integer, primary_key=True),
Column("amount", Integer),
)
metadata.create_all(engine)
inspector = inspect(engine)
columns = RisingWaveDbEngineSpec.get_columns(inspector, Table("pilot_types"))
by_name = {col["column_name"]: col for col in columns}
assert set(by_name) == {"id", "amount"}
for col in by_name.values():
spec = RisingWaveDbEngineSpec.get_column_spec(str(col["type"]))
assert spec is not None
assert spec.generic_type == GenericDataType.NUMERIC
assert isinstance(spec.sqla_type, Integer)

Some files were not shown because too many files have changed in this diff Show More