mirror of
https://github.com/apache/superset.git
synced 2026-09-08 00:04:36 +00:00
feat(ci): expand testcontainers coverage to databend, risingwave, firebird, ydb, oceanbase, vertica
Stacked on feat/testcontainers-nightly-only-gating. All six extras already
existed in pyproject.toml. oceanbase and vertica run nightly_only: true
(heavy first-boot and a ~12GB RAM floor, respectively), so they don't run
per-PR; databend/risingwave/firebird/ydb run on every PR like the rest of
this suite.
- oceanbase_py pins sqlalchemy-utils<0.39, which conflicts outright with
Superset's own sqlalchemy-utils==0.42.1 pin -- kept out of the baseline
dev install (same reason as db2's ibm-db-sa) and installed on demand,
--no-deps, only for its own CI leg (it never actually imports
sqlalchemy_utils itself, so the version mismatch is inert at runtime).
- databend: connects to the local standalone image's builtin `root` user
(no password) with sslmode=disable, since Superset's default
encryption_parameters assume TLS the local image doesn't have.
- risingwave: RisingWave's storage engine checkpoints asynchronously --
a SELECT immediately after INSERT can see zero rows without an explicit
FLUSH (confirmed on a real instance). Uses the shared _pagination.py
helper's after_insert hook (originally added for CrateDB) to do that.
- firebird: sqlalchemy-firebird's driver is a pure-Python ctypes wrapper
(py3-none-any wheel, confirmed by downloading it directly) that
dynamically loads the native libfbclient from the host rather than
bundling it -- CI installs that system package on demand. Also confirms
in the test docstring that FirebirdEngineSpec's `limit_method =
LimitMethod.FETCH_MANY` (comment: "uses FIRST to limit") is stale
against the modern driver, which compiles real ROWS-based pagination.
- ydb: needed three real fixes to make a generic DockerContainer usable
at all. (1) YDB's gRPC client does endpoint discovery and reconnects to
whatever the server reports, which by default is the container's own
internal Docker hostname -- fixed by binding the same port on the host
as inside the container and advertising "localhost" as the container's
own hostname, so the discovered endpoint is actually reachable. (2) The
gRPC port opens before storage pools are fully initialized, so an early
CREATE TABLE fails; the fixture retries a real metadata.create_all()
probe rather than trusting the open port. (3) YDB rejects DDL inside an
explicit transaction ("Scheme operations cannot be executed inside
transaction") -- confirmed this only affects a raw text("CREATE
TABLE..."), not metadata.create_all()'s own DDL execution path, which
already does the right thing.
This commit is contained in:
@@ -90,6 +90,25 @@ jobs:
|
||||
# database -- wider margin until real CI data says otherwise.
|
||||
- dialect: starrocks
|
||||
timeout: 15
|
||||
- dialect: databend
|
||||
timeout: 10
|
||||
- dialect: risingwave
|
||||
timeout: 10
|
||||
- dialect: firebird
|
||||
timeout: 10
|
||||
- dialect: ydb
|
||||
timeout: 10
|
||||
# OceanBase bootstraps a distributed-style cluster even in
|
||||
# single-node MODE=MINI, and Vertica Community Edition has a
|
||||
# well-documented ~12GB RAM floor to even start -- both too heavy
|
||||
# for every PR's CI budget, so both run on the nightly cron /
|
||||
# manual dispatch only.
|
||||
- dialect: oceanbase
|
||||
timeout: 20
|
||||
nightly_only: true
|
||||
- dialect: vertica
|
||||
timeout: 20
|
||||
nightly_only: true
|
||||
timeout-minutes: ${{ matrix.timeout }}
|
||||
env:
|
||||
PYTHONPATH: ${{ github.workspace }}
|
||||
@@ -116,6 +135,25 @@ jobs:
|
||||
# instead, only for this leg of the matrix.
|
||||
if: matrix.dialect == 'db2'
|
||||
run: uv pip install --system -e .[db2]
|
||||
- name: Install oceanbase driver (oceanbase_py)
|
||||
# oceanbase_py pins sqlalchemy-utils>=0.38.3,<0.39, which conflicts
|
||||
# outright with Superset's own sqlalchemy-utils==0.42.1 pin -- kept
|
||||
# out of the baseline dev install for the same reason as db2 above.
|
||||
# --no-deps sidesteps that pin entirely: this job only needs
|
||||
# oceanbase_py's dialect module importable, not its sqlalchemy-utils
|
||||
# dependency satisfied, since nothing here calls into it.
|
||||
if: matrix.dialect == 'oceanbase'
|
||||
run: uv pip install --system --no-deps -e .[oceanbase]
|
||||
- name: Install Firebird client library (libfbclient2)
|
||||
# sqlalchemy-firebird's driver (firebird-driver) is a pure-Python
|
||||
# ctypes wrapper (its wheel is py3-none-any) that dynamically loads
|
||||
# the native Firebird client library from the host at import time
|
||||
# -- it doesn't bundle that library itself, so it has to come from
|
||||
# the system package manager, only for this leg of the matrix.
|
||||
if: matrix.dialect == 'firebird'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libfbclient2
|
||||
- name: Run testcontainers db_engine_specs tests (${{ matrix.dialect }})
|
||||
# A job-level `if:` can't reference `matrix` (only github/inputs/
|
||||
# needs/vars are available there), so the nightly_only skip has to
|
||||
|
||||
Reference in New Issue
Block a user