- requirements/development.in + testcontainers.yml: scope the `db2` extra
(`ibm-db-sa`/`ibm-db`) out of the baseline dev install. `ibm-db` ships no
Linux arm64 wheel, so bundling it there broke the multi-platform dev
Docker image build on push. The testcontainers CI job now installs it
directly, only for its own db2 matrix leg.
- cockroachdb.py: list `psycopg2-binary` alongside `sqlalchemy-cockroachdb`
in the `pypi_packages` metadata, since a plain `cockroachdb://` URL can't
connect without a DBAPI and sqlalchemy-cockroachdb doesn't install one.
- test_cockroachdb.py/test_crate.py/test_trino.py: fix stale docstring
references to the old `nightly-testcontainers.yml` filename.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- pyproject.toml: pin psycopg2-binary alongside sqlalchemy-cockroachdb --
the latter declares no DBAPI dependency of its own, so the documented
`apache-superset[cockroachdb]` install couldn't actually connect.
- testcontainers.yml: scope the concurrency group by ref so a PR run and
the nightly cron (or two different PRs) no longer cancel each other.
- test_cockroachdb.py: assert the actual generic/SQLAlchemy type, matching
the Trino test, instead of only checking a column spec was found.
- pytest.ini + new `testcontainers` marker + _driver.py: exclude
tests/testcontainers/ from a plain `pytest` run by default (it needs
Docker), while the dedicated CI job now sets
SUPERSET_TESTCONTAINERS_STRICT so a broken/missing driver import fails
that job instead of silently skipping to a green, zero-tests-run result.
- UPDATING.md: the migration note now says to uninstall the old
`cockroachdb` package outright, since reinstalling the extra alone can
leave both packages registering the same dialect entry point.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The "paginated query returns correct rows in order" test body was
duplicated verbatim across cockroachdb/crate/db2/mssql/oracle/trino's
testcontainers suites. Factor the shared table-setup/assert logic into
_pagination.py, with an optional post-insert hook for CrateDB's
eventual-consistency REFRESH TABLE step; each call site keeps its own
test function and dialect-specific docstring.
Addresses bito-code-review feedback on #43502.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pilot for real-container testing of db_engine_specs against actual
databases (CockroachDB, CrateDB, Trino), via testcontainers-python.
Existing unit tests mock the driver/dialect layer entirely, which cannot
catch real SQL-compilation or type-mapping bugs -- e.g. #42899, where
Trino emitted OFFSET before LIMIT for paginated queries.
Runs nightly (.github/workflows/nightly-testcontainers.yml), not on every
merge: container pulls and startup are slower and more flake-prone than
the existing mocked unit tests, and this measures that cost/signal
tradeoff before considering wider adoption.
Building this surfaced two real bugs, fixed/documented separately:
- The cockroachdb dialect was completely broken under SQLAlchemy 2.0 due
to a dead upstream package -- fixed in #43501.
- testcontainers-python's TrinoContainer.get_connection_url() returns the
container-internal port instead of the Docker-mapped host port; worked
around locally, filed upstream.
