feat(ci): expand testcontainers coverage to databend, risingwave, firebird, ydb, oceanbase, vertica

Stacked on feat/testcontainers-nightly-only-gating. All six extras already
existed in pyproject.toml. oceanbase and vertica run nightly_only: true
(heavy first-boot and a ~12GB RAM floor, respectively), so they don't run
per-PR; databend/risingwave/firebird/ydb run on every PR like the rest of
this suite.

- oceanbase_py pins sqlalchemy-utils<0.39, which conflicts outright with
  Superset's own sqlalchemy-utils==0.42.1 pin -- kept out of the baseline
  dev install (same reason as db2's ibm-db-sa) and installed on demand,
  --no-deps, only for its own CI leg (it never actually imports
  sqlalchemy_utils itself, so the version mismatch is inert at runtime).
- databend: connects to the local standalone image's builtin `root` user
  (no password) with sslmode=disable, since Superset's default
  encryption_parameters assume TLS the local image doesn't have.
- risingwave: RisingWave's storage engine checkpoints asynchronously --
  a SELECT immediately after INSERT can see zero rows without an explicit
  FLUSH (confirmed on a real instance). Uses the shared _pagination.py
  helper's after_insert hook (originally added for CrateDB) to do that.
- firebird: sqlalchemy-firebird's driver is a pure-Python ctypes wrapper
  (py3-none-any wheel, confirmed by downloading it directly) that
  dynamically loads the native libfbclient from the host rather than
  bundling it -- CI installs that system package on demand. Also confirms
  in the test docstring that FirebirdEngineSpec's `limit_method =
  LimitMethod.FETCH_MANY` (comment: "uses FIRST to limit") is stale
  against the modern driver, which compiles real ROWS-based pagination.
- ydb: needed three real fixes to make a generic DockerContainer usable
  at all. (1) YDB's gRPC client does endpoint discovery and reconnects to
  whatever the server reports, which by default is the container's own
  internal Docker hostname -- fixed by binding the same port on the host
  as inside the container and advertising "localhost" as the container's
  own hostname, so the discovered endpoint is actually reachable. (2) The
  gRPC port opens before storage pools are fully initialized, so an early
  CREATE TABLE fails; the fixture retries a real metadata.create_all()
  probe rather than trusting the open port. (3) YDB rejects DDL inside an
  explicit transaction ("Scheme operations cannot be executed inside
  transaction") -- confirmed this only affects a raw text("CREATE
  TABLE..."), not metadata.create_all()'s own DDL execution path, which
  already does the right thing.
This commit is contained in:
Superset Dev
2026-08-29 10:10:03 -07:00
committed by rusackas
parent 0f1d2a9e00
commit 770dd5caef
9 changed files with 898 additions and 7 deletions
+38
View File
@@ -90,6 +90,25 @@ jobs:
# database -- wider margin until real CI data says otherwise.
- dialect: starrocks
timeout: 15
- dialect: databend
timeout: 10
- dialect: risingwave
timeout: 10
- dialect: firebird
timeout: 10
- dialect: ydb
timeout: 10
# OceanBase bootstraps a distributed-style cluster even in
# single-node MODE=MINI, and Vertica Community Edition has a
# well-documented ~12GB RAM floor to even start -- both too heavy
# for every PR's CI budget, so both run on the nightly cron /
# manual dispatch only.
- dialect: oceanbase
timeout: 20
nightly_only: true
- dialect: vertica
timeout: 20
nightly_only: true
timeout-minutes: ${{ matrix.timeout }}
env:
PYTHONPATH: ${{ github.workspace }}
@@ -116,6 +135,25 @@ jobs:
# instead, only for this leg of the matrix.
if: matrix.dialect == 'db2'
run: uv pip install --system -e .[db2]
- name: Install oceanbase driver (oceanbase_py)
# oceanbase_py pins sqlalchemy-utils>=0.38.3,<0.39, which conflicts
# outright with Superset's own sqlalchemy-utils==0.42.1 pin -- kept
# out of the baseline dev install for the same reason as db2 above.
# --no-deps sidesteps that pin entirely: this job only needs
# oceanbase_py's dialect module importable, not its sqlalchemy-utils
# dependency satisfied, since nothing here calls into it.
if: matrix.dialect == 'oceanbase'
run: uv pip install --system --no-deps -e .[oceanbase]
- name: Install Firebird client library (libfbclient2)
# sqlalchemy-firebird's driver (firebird-driver) is a pure-Python
# ctypes wrapper (its wheel is py3-none-any) that dynamically loads
# the native Firebird client library from the host at import time
# -- it doesn't bundle that library itself, so it has to come from
# the system package manager, only for this leg of the matrix.
if: matrix.dialect == 'firebird'
run: |
sudo apt-get update
sudo apt-get install -y libfbclient2
- name: Run testcontainers db_engine_specs tests (${{ matrix.dialect }})
# A job-level `if:` can't reference `matrix` (only github/inputs/
# needs/vars are available there), so the nightly_only skip has to