Compare commits

..

5 Commits

Author SHA1 Message Date
Evan
a48f83258d test: trigger connection failure via engine.connect for SQLAlchemy 2.0
The connection check now uses engine.connect() + connection.execute()
instead of the removed Engine.execute(), so the test must inject the
failure on engine.connect.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:53:41 -07:00
Evan
e73633697d test: read re-encrypt bind params from positional args, not kwargs
The migrator passes bind params positionally to conn.execute(stmt, params),
so the assertion must read call_args.args[1] rather than call_args.kwargs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:53:41 -07:00
Evan
69e4f6e254 test: pass real SQLAlchemy Row to _re_encrypt_row in integration tests
The re-encrypt path reads column values via row._mapping (SQLAlchemy 2.0
Row API), but the integration tests passed a plain dict, which has no
_mapping, raising AttributeError. Build a genuine Row via a small helper
that handles both the 1.4 and 2.0 Row constructor signatures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:53:41 -07:00
Evan
410768e62e test: update encrypt re-encrypt fixtures for SQLAlchemy 2.0 Row API
The migrator now reads columns via row._mapping[...] and binds the
UPDATE params as a single positional dict. Wrap the fixture rows in a
_Row stand-in and read the bind dict positionally so the unit tests
match the production interface.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 14:53:41 -07:00
Claude Code
7c7689c14f refactor: make import/expression layer SQLAlchemy 2.0-compatible
Forward-compatible groundwork for the SQLAlchemy 1.4 -> 2.0 bump. Every
change here is backward-compatible with the current SQLAlchemy 1.4.54 /
Flask-SQLAlchemy 2.5.1 baseline, so it does not regress anything while
shrinking the eventual version bump.

Removed-in-2.0 API fixes (these break on 2.0 but still work on 1.4):
- `sqlalchemy.orm.eagerload` (removed alias of `joinedload`) in the
  security manager.
- `sqlalchemy.sql.visitors.VisitableType` (removed) -> `sqlalchemy.types.TypeEngine`
  in dataset importer and mock_data type hints.
- `flask_sqlalchemy.BaseQuery` (removed in FSA 3.x, required by SA 2.0) ->
  import from `flask_sqlalchemy.query.Query` with a fallback to the FSA 2.x
  path, in the query/saved-query filters and the report command tests.
- `Engine.execute` / raw-string execution -> wrap startup health check in a
  `Connection` + `text()`, and the secrets re-encryptor in `text()` with a
  single bind dict and `row._mapping[...]` access.

Prospective Annotated Declarative prep (no-ops on 1.4, ease the 2.0 flip):
- `__allow_unmapped__` on the FAB declarative base so legacy 1.x annotations
  are tolerated during incremental migration.
- `Mapped[...]` return annotations on `created_by_fk`/`changed_by_fk` and
  `BaseDatasource.slices`.

The `select()` and `case()` syntax changes from the same effort already
landed via #40276 / #40275, so they are intentionally excluded here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-23 14:53:39 -07:00
73 changed files with 306 additions and 1199 deletions

View File

@@ -31,7 +31,7 @@ jobs:
checks: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: true
ref: master
@@ -40,7 +40,7 @@ jobs:
uses: ./.github/actions/setup-supersetbot/
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.10"

View File

@@ -22,7 +22,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -25,7 +25,7 @@ jobs:
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check and notify

View File

@@ -26,7 +26,7 @@ jobs:
frontend: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -58,7 +58,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -27,7 +27,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: "Checkout Repository"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: "Dependency Review"
@@ -51,7 +51,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- name: "Checkout Repository"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -30,7 +30,7 @@ jobs:
docker: ${{ steps.check.outputs.docker }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -71,7 +71,7 @@ jobs:
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -177,7 +177,7 @@ jobs:
timeout-minutes: 30
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Free up disk space

View File

@@ -23,7 +23,7 @@ jobs:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
# Note: registry-url is intentionally omitted. When set, actions/setup-node

View File

@@ -21,7 +21,7 @@ jobs:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6

View File

@@ -32,7 +32,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -27,7 +27,7 @@ jobs:
security-events: write
steps:
- name: Checkout Repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -16,7 +16,7 @@ jobs:
issues: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -12,7 +12,7 @@ jobs:
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -18,7 +18,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -21,7 +21,7 @@ jobs:
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -28,7 +28,7 @@ jobs:
python-version: ${{ github.event_name == 'pull_request' && fromJSON('["current"]') || fromJSON('["current", "previous", "next"]') }}
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -33,7 +33,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
# pulls all commits (needed for lerna / semantic release to correctly version)

View File

@@ -152,7 +152,7 @@ jobs:
- name: Checkout PR code (only if build needed)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.build_needed == 'true'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ steps.check.outputs.target_sha }}
persist-credentials: false

View File

@@ -41,7 +41,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -60,7 +60,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.event.workflow_run.head_sha || github.sha }}"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ github.event.workflow_run.head_sha || github.sha }}
persist-credentials: false

View File

@@ -28,7 +28,7 @@ jobs:
name: Link Checking
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
# Do not bump this linkinator-action version without opening
@@ -73,7 +73,7 @@ jobs:
working-directory: docs
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
@@ -112,7 +112,7 @@ jobs:
working-directory: docs
steps:
- name: "Checkout PR head: ${{ github.event.workflow_run.head_sha }}"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ github.event.workflow_run.head_sha }}
persist-credentials: false

View File

@@ -38,7 +38,7 @@ jobs:
frontend: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -97,21 +97,21 @@ jobs:
# Conditional checkout based on context
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
@@ -207,21 +207,21 @@ jobs:
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge

View File

@@ -31,7 +31,7 @@ jobs:
working-directory: superset-extensions-cli
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -27,7 +27,7 @@ jobs:
should-run: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout Code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
@@ -110,7 +110,7 @@ jobs:
id-token: write
steps:
- name: Checkout Code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0

View File

@@ -19,7 +19,7 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -29,7 +29,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.ref || github.ref_name }}
persist-credentials: true

View File

@@ -34,7 +34,7 @@ jobs:
frontend: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -83,21 +83,21 @@ jobs:
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge

View File

@@ -29,7 +29,7 @@ jobs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -72,7 +72,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
@@ -157,7 +157,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
@@ -207,7 +207,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -25,7 +25,7 @@ jobs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -72,7 +72,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
@@ -127,7 +127,7 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -30,7 +30,7 @@ jobs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Check for file changes
@@ -55,7 +55,7 @@ jobs:
PYTHONPATH: ${{ github.workspace }}
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -25,7 +25,7 @@ jobs:
pull-requests: read
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive
@@ -61,7 +61,7 @@ jobs:
pull-requests: read
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
submodules: recursive

View File

@@ -25,7 +25,7 @@ jobs:
timeout-minutes: 20
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Install dependencies

View File

@@ -38,7 +38,7 @@ jobs:
});
- name: "Checkout ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -27,7 +27,7 @@ jobs:
# zizmor: ignore[artipacked] - required persisted credentials to push synced requirement changes back to remote
- name: Checkout source code
if: ${{ steps.dependabot-metadata.outputs.package-ecosystem == 'pip' }}
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ github.event.pull_request.head.sha }}
persist-credentials: true

View File

@@ -54,7 +54,7 @@ jobs:
fail-fast: false
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
@@ -120,7 +120,7 @@ jobs:
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0

View File

@@ -32,7 +32,7 @@ jobs:
name: Generate Reports
steps:
- name: Checkout Repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

View File

@@ -36,7 +36,7 @@
"@great-expectations/jsonforms-antd-renderers": "^2.2.10",
"@jsonforms/core": "^3.7.0",
"@jsonforms/react": "^3.7.0",
"@jsonforms/vanilla-renderers": "^3.8.0",
"@jsonforms/vanilla-renderers": "^3.7.0",
"@luma.gl/constants": "~9.2.5",
"@luma.gl/core": "~9.2.5",
"@luma.gl/engine": "~9.2.5",
@@ -187,7 +187,7 @@
"@storybook/test-runner": "0.24.4",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.41",
"@swc/plugin-emotion": "^14.13.0",
"@swc/plugin-emotion": "^14.12.0",
"@swc/plugin-transform-imports": "^12.5.0",
"@testing-library/dom": "^9.3.4",
"@testing-library/jest-dom": "^6.9.1",
@@ -272,7 +272,7 @@
"source-map": "^0.7.6",
"source-map-support": "^0.5.21",
"speed-measure-webpack-plugin": "^1.6.0",
"storybook": "10.4.6",
"storybook": "10.4.5",
"style-loader": "^4.0.0",
"swc-loader": "^0.2.7",
"terser-webpack-plugin": "^5.6.1",
@@ -5387,41 +5387,41 @@
}
},
"node_modules/@jsonforms/core": {
"version": "3.8.0",
"resolved": "https://registry.npmjs.org/@jsonforms/core/-/core-3.8.0.tgz",
"integrity": "sha512-XSvaZuQSs/MceG5nDDcrE879onPHkGBy0xEuLeZMUkSM/M8wc1dEUrJtMOZVNSITocm9YXFY1qQ5gnsPP38zAg==",
"version": "3.7.0",
"resolved": "https://registry.npmjs.org/@jsonforms/core/-/core-3.7.0.tgz",
"integrity": "sha512-CE9viWtwi9QWLqlWLeOul1/R1GRAyOA9y6OoUpsCc0FhyR+g5p29F3k0fUExHWxL0Sf4KHcXYkfhtqfRBPS8ww==",
"license": "MIT",
"dependencies": {
"@types/json-schema": "^7.0.3",
"ajv": "^8.18.0",
"ajv": "^8.6.1",
"ajv-formats": "^2.1.0",
"lodash": "^4.17.21"
}
},
"node_modules/@jsonforms/react": {
"version": "3.8.0",
"resolved": "https://registry.npmjs.org/@jsonforms/react/-/react-3.8.0.tgz",
"integrity": "sha512-k81+yWLpCQl+3XizS1bLjXoBwYhW1OAkjSXFA8W5qNtfPZjSOXDgtiuMOGYDv4b60tu2e9RB8h2P2O7QhfkhiA==",
"version": "3.7.0",
"resolved": "https://registry.npmjs.org/@jsonforms/react/-/react-3.7.0.tgz",
"integrity": "sha512-HkY7qAx8vW97wPEgZ7GxCB3iiXG1c95GuObxtcDHGPBJWMwnxWBnVYJmv5h7nthrInKsQKHZL5OusnC/sj/1GQ==",
"license": "MIT",
"dependencies": {
"lodash": "^4.17.21"
},
"peerDependencies": {
"@jsonforms/core": "3.8.0",
"@jsonforms/core": "3.7.0",
"react": "^16.12.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/@jsonforms/vanilla-renderers": {
"version": "3.8.0",
"resolved": "https://registry.npmjs.org/@jsonforms/vanilla-renderers/-/vanilla-renderers-3.8.0.tgz",
"integrity": "sha512-s75TG4hSYgYLN9IRVhYtGjijqyhVXijgDhb2WnMqY+Ki7MQkLn9U7yg/l89NEpwzWS1sv0DxKUxriqVUq382og==",
"version": "3.7.0",
"resolved": "https://registry.npmjs.org/@jsonforms/vanilla-renderers/-/vanilla-renderers-3.7.0.tgz",
"integrity": "sha512-RdXQGsheARUJVbaTe6SqGw9W4/yrm0BgUok6OKUj8krp1NF4fqXc5UbYGHFksMR/p7LCuoYHCtQzKLXEfxJbDw==",
"license": "MIT",
"dependencies": {
"lodash": "^4.17.21"
},
"peerDependencies": {
"@jsonforms/core": "3.8.0",
"@jsonforms/react": "3.8.0",
"@jsonforms/core": "3.7.0",
"@jsonforms/react": "3.7.0",
"react": "^16.12.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
@@ -10819,9 +10819,9 @@
}
},
"node_modules/@swc/plugin-emotion": {
"version": "14.13.0",
"resolved": "https://registry.npmjs.org/@swc/plugin-emotion/-/plugin-emotion-14.13.0.tgz",
"integrity": "sha512-UT1l9tr934HjnktUiMGbw1rWrIXUhAByTB0DwZJwHmS8KWox+wNBIK4ZkJ2tKVU/PnQZRni+R9e6xklFkmgSYg==",
"version": "14.12.0",
"resolved": "https://registry.npmjs.org/@swc/plugin-emotion/-/plugin-emotion-14.12.0.tgz",
"integrity": "sha512-lyAQgTeDkowq/4+8JYaviVOL4jXSdObz+uuk84DjM0z4qoiMpI6xoDVp7/tjWeVjmLc2U6Qp3hDuwWMZ5xe88Q==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
@@ -39527,9 +39527,9 @@
}
},
"node_modules/storybook": {
"version": "10.4.6",
"resolved": "https://registry.npmjs.org/storybook/-/storybook-10.4.6.tgz",
"integrity": "sha512-6wkA6LxfDSSilloITsrFOJfsnw0mDUP2h8Ls+lRt8oRsudtz2RWFhLv+Toiwg6NW7hUpdTDc2hzR7DztJid6+A==",
"version": "10.4.5",
"resolved": "https://registry.npmjs.org/storybook/-/storybook-10.4.5.tgz",
"integrity": "sha512-QZuv1gS9Tf9RMCjDw5JOfv1XSB5IhU0uhSKQNS7l/N9zDpmSydirCspkCNT9e0zkFfPkZ9vmQUTzHY/BA07saA==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -39540,7 +39540,7 @@
"@vitest/expect": "3.2.4",
"@vitest/spy": "3.2.4",
"@webcontainer/env": "^1.1.1",
"esbuild": "^0.18.0 || ^0.19.0 || ^0.20.0 || ^0.21.0 || ^0.22.0 || ^0.23.0 || ^0.24.0 || ^0.25.0 || ^0.26.0 || ^0.27.0 || ^0.28.0",
"esbuild": "^0.18.0 || ^0.19.0 || ^0.20.0 || ^0.21.0 || ^0.22.0 || ^0.23.0 || ^0.24.0 || ^0.25.0 || ^0.26.0 || ^0.27.0",
"open": "^10.2.0",
"oxc-parser": "^0.127.0",
"oxc-resolver": "^11.19.1",

View File

@@ -119,7 +119,7 @@
"@great-expectations/jsonforms-antd-renderers": "^2.2.10",
"@jsonforms/core": "^3.7.0",
"@jsonforms/react": "^3.7.0",
"@jsonforms/vanilla-renderers": "^3.8.0",
"@jsonforms/vanilla-renderers": "^3.7.0",
"@luma.gl/constants": "~9.2.5",
"@luma.gl/core": "~9.2.5",
"@luma.gl/engine": "~9.2.5",
@@ -270,7 +270,7 @@
"@storybook/test-runner": "0.24.4",
"@svgr/webpack": "^8.1.0",
"@swc/core": "^1.15.41",
"@swc/plugin-emotion": "^14.13.0",
"@swc/plugin-emotion": "^14.12.0",
"@swc/plugin-transform-imports": "^12.5.0",
"@testing-library/dom": "^9.3.4",
"@testing-library/jest-dom": "^6.9.1",
@@ -355,7 +355,7 @@
"source-map": "^0.7.6",
"source-map-support": "^0.5.21",
"speed-measure-webpack-plugin": "^1.6.0",
"storybook": "10.4.6",
"storybook": "10.4.5",
"style-loader": "^4.0.0",
"swc-loader": "^0.2.7",
"terser-webpack-plugin": "^5.6.1",

View File

@@ -21,12 +21,10 @@
import d3 from 'd3';
import { extent as d3Extent } from 'd3-array';
import {
BinaryQueryObjectFilterClause,
CategoricalColorNamespace,
ContextMenuFilters,
DataMask,
ValueFormatter,
getNumberFormatter,
getSequentialSchemeRegistry,
CategoricalColorNamespace,
} from '@superset-ui/core';
import countries, { countryOptions } from './countries';
@@ -67,28 +65,9 @@ interface CountryMapProps {
formatter: ValueFormatter;
colorScheme: string;
sliceId: number;
onContextMenu?: (
clientX: number,
clientY: number,
data: ContextMenuFilters,
) => void;
emitCrossFilters?: boolean;
setDataMask?: (dataMask: DataMask) => void;
filterState?: {
selectedValues?: string[];
extraFormData?: {
filters?: BinaryQueryObjectFilterClause[];
};
};
entity?: string;
}
const maps: Record<string, GeoData> = {};
// Store zoom state per chart instance using element as key to enable garbage collection
const zoomStates = new WeakMap<
HTMLElement,
{ scale: number; translate: [number, number] }
>();
function CountryMap(element: HTMLElement, props: CountryMapProps) {
const {
@@ -96,15 +75,10 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
width,
height,
country,
entity,
linearColorScheme,
formatter,
colorScheme,
sliceId,
filterState,
emitCrossFilters,
onContextMenu,
setDataMask,
} = props;
const container = element;
@@ -125,15 +99,7 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
? colorScale(d.country_id, sliceId)
: (linearColorScale(d.metric) ?? '');
});
const colorFn = (feature: GeoFeature): string => {
if (!feature?.properties) return '#d9d9d9';
const iso = feature.properties.ISO;
return colorMap[iso] || '#d9d9d9';
};
// Check if dashboard is in edit mode
const isEditMode = container.closest('.dashboard--editing') !== null;
const colorFn = (d: GeoFeature) => colorMap[d.properties.ISO] || 'none';
const path = d3.geo.path();
const div = d3.select(container);
@@ -146,11 +112,6 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
.attr('width', width)
.attr('height', height)
.attr('preserveAspectRatio', 'xMidYMid meet');
// Only set grab cursor if not in edit mode
if (!isEditMode) {
svg.style('cursor', 'grab');
}
const backgroundRect = svg
.append('rect')
.attr('class', 'background')
@@ -158,65 +119,40 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
.attr('height', height);
const g = svg.append('g');
const mapLayer = g.append('g').classed('map-layer', true);
// Add hover popup for tooltip
const hoverPopup = div.append('div').attr('class', 'hover-popup');
// Track mouse position to distinguish clicks from drags
let mousedownPos: { x: number; y: number } | null = null;
let centered: GeoFeature | null;
// Cross-filter support
const getCrossFilterDataMask = (
source: GeoFeature,
): { dataMask: DataMask; isCurrentValueSelected: boolean } | undefined => {
if (!entity) return undefined;
const clicked = function clicked(d: GeoFeature) {
const hasCenter = d && centered !== d;
let x: number;
let y: number;
let k: number;
const halfWidth = width / 2;
const halfHeight = height / 2;
const selected = filterState?.selectedValues || [];
const iso = source?.properties?.ISO;
if (!iso) return undefined;
const isSelected = selected.includes(iso);
const values = isSelected ? [] : [iso];
return {
dataMask: {
extraFormData: {
filters: values.length
? [{ col: entity, op: 'IN', val: values }]
: [],
},
filterState: {
value: values.length ? values : null,
selectedValues: values.length ? values : null,
},
},
isCurrentValueSelected: isSelected,
};
};
// Handle right-click context menu
const handleContextMenu = (feature: GeoFeature): void => {
const pointerEvent = d3.event;
if (typeof onContextMenu === 'function') {
pointerEvent?.preventDefault();
if (hasCenter) {
const centroid = path.centroid(d);
[x, y] = centroid;
k = 4;
centered = d;
} else {
x = halfWidth;
y = halfHeight;
k = 1;
centered = null;
}
const iso = feature?.properties?.ISO;
if (!iso || typeof onContextMenu !== 'function' || !entity) return;
const drillVal = iso;
const drillToDetailFilters = [
{ col: entity, op: '==', val: drillVal, formattedVal: drillVal },
];
const drillByFilters = [{ col: entity, op: '==', val: drillVal }];
onContextMenu(pointerEvent.clientX, pointerEvent.clientY, {
drillToDetail: drillToDetailFilters,
crossFilter: getCrossFilterDataMask(feature),
drillBy: { filters: drillByFilters, groupbyFieldName: 'entity' },
});
g.transition()
.duration(750)
.attr(
'transform',
`translate(${halfWidth},${halfHeight})scale(${k})translate(${-x},${-y})`,
);
};
backgroundRect.on('click', clicked);
const getNameOfRegion = function getNameOfRegion(
feature: GeoFeature,
): string {
@@ -229,7 +165,7 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
return '';
};
const updatePopupPosition = (): void => {
const updatePopupPosition = () => {
const svgHeight = svg.node().getBoundingClientRect().height;
const [x, y] = d3.mouse(svg.node());
hoverPopup
@@ -239,135 +175,36 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
.classed('popup-at-bottom', y > (svgHeight * 2) / 3);
};
const mouseenter = function mouseenter(
this: SVGPathElement,
d: GeoFeature,
): void {
const mouseenter = function mouseenter(this: SVGPathElement, d: GeoFeature) {
// Darken color
let c: string = colorFn(d);
if (c) {
if (c !== 'none') {
c = d3.rgb(c).darker().toString();
}
d3.select(this).style('fill', c);
// Display information popup
const result = data.filter(r => r.country_id === d?.properties?.ISO);
const regionName = escapeHtml(getNameOfRegion(d));
const metricValue =
result.length > 0 ? escapeHtml(String(formatter(result[0].metric))) : '';
const result = data.filter(
region => region.country_id === d.properties.ISO,
);
hoverPopup
.style('display', 'block')
.html(`<div><strong>${regionName}</strong><br>${metricValue}</div>`);
.html(
`<div><strong>${getNameOfRegion(d)}</strong><br>${result.length > 0 ? formatter(result[0].metric) : ''}</div>`,
);
updatePopupPosition();
};
// Mouse move handler to update tooltip position
const mousemove = function mousemove(): void {
const mousemove = function mousemove() {
updatePopupPosition();
};
const mouseout = function mouseout(this: SVGPathElement): void {
d3.select(this).style('fill', (d: GeoFeature) => colorFn(d));
const mouseout = function mouseout(this: SVGPathElement) {
d3.select(this).style('fill', colorFn);
hoverPopup.style('display', 'none');
};
// Only enable zoom if not in edit mode
if (!isEditMode) {
// Zoom with panning bounds
const zoom = d3.behavior
.zoom()
.scaleExtent([1, 4])
.on('zoomstart', () => {
svg.style('cursor', 'grabbing');
})
.on('zoom', () => {
const { translate, scale } = d3.event;
let [tx, ty] = translate;
const scaledW = width * scale;
const scaledH = height * scale;
const minX = Math.min(0, width - scaledW);
const maxX = 0;
const minY = Math.min(0, height - scaledH);
const maxY = 0;
tx = Math.max(Math.min(tx, maxX), minX);
ty = Math.max(Math.min(ty, maxY), minY);
// Sync D3's internal translate state with the clamped values so the
// next wheel/zoom event starts from the constrained position rather
// than the unclamped one (otherwise the view jumps).
zoom.translate([tx, ty]);
g.attr('transform', `translate(${tx}, ${ty}) scale(${scale})`);
const prev = zoomStates.get(element);
const changed =
!prev ||
prev.scale !== scale ||
prev.translate[0] !== tx ||
prev.translate[1] !== ty;
if (changed) {
zoomStates.set(element, { scale, translate: [tx, ty] });
}
})
.on('zoomend', () => {
svg.style('cursor', 'grab');
});
d3.select(svg.node()).call(zoom);
// Restore previous zoom state if it exists
const savedZoom = zoomStates.get(element);
if (savedZoom) {
const { scale, translate } = savedZoom;
zoom.scale(scale).translate(translate);
g.attr(
'transform',
`translate(${translate[0]}, ${translate[1]}) scale(${scale})`,
);
}
}
// Visual highlighting for selected regions
function highlightSelectedRegion(
selectedValues: string[] | null = null,
): void {
const selected = selectedValues || filterState?.selectedValues || [];
mapLayer
.selectAll('path.region')
.style('fill-opacity', (d: GeoFeature) => {
const iso = d?.properties?.ISO;
return selected.length === 0 || selected.includes(iso) ? 1 : 0.3;
})
.style('stroke', (d: GeoFeature) => {
const iso = d?.properties?.ISO;
return selected.includes(iso) ? '#222' : null;
})
.style('stroke-width', (d: GeoFeature) => {
const iso = d?.properties?.ISO;
return selected.includes(iso) ? '1.5px' : '0.5px';
});
}
// Click handler for cross-filters
const handleClick = (feature: GeoFeature): void => {
if (!entity || !emitCrossFilters || typeof setDataMask !== 'function') {
return;
}
const result = getCrossFilterDataMask(feature);
if (!result) return;
const { dataMask, isCurrentValueSelected } = result;
setDataMask(dataMask);
const iso = feature?.properties?.ISO;
const newSelection = isCurrentValueSelected || !iso ? [] : [iso];
highlightSelectedRegion(newSelection);
};
function drawMap(mapData: GeoData): void {
function drawMap(mapData: GeoData) {
const { features } = mapData;
const center = d3.geo.centroid(mapData);
const scale = 100;
@@ -378,11 +215,13 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
.translate([width / 2, height / 2]);
path.projection(projection);
// Compute scale that fits container.
const bounds = path.bounds(mapData);
const hscale = (scale * width) / (bounds[1][0] - bounds[0][0]);
const vscale = (scale * height) / (bounds[1][1] - bounds[0][1]);
const newScale = Math.min(hscale, vscale);
const newScale = hscale < vscale ? hscale : vscale;
// Compute bounds and offset using the updated scale.
projection.scale(newScale);
const newBounds = path.bounds(mapData);
projection.translate([
@@ -390,45 +229,20 @@ function CountryMap(element: HTMLElement, props: CountryMapProps) {
height - (newBounds[0][1] + newBounds[1][1]) / 2,
]);
const sel = mapLayer.selectAll('path.region').data(features);
sel
// Draw each province as a path
mapLayer
.selectAll('path')
.data(features)
.enter()
.append('path')
.attr('class', 'region')
.attr('vector-effect', 'non-scaling-stroke');
// Apply attributes and event handlers to all elements (enter + update)
mapLayer
.selectAll('path.region')
.attr('d', path)
.attr('class', 'region')
.attr('vector-effect', 'non-scaling-stroke')
.style('fill', colorFn)
.on('mouseenter', mouseenter)
.on('mousemove', mousemove)
.on('mouseout', mouseout)
.on('contextmenu', handleContextMenu)
.on('mousedown', function mousedown() {
const pos = d3.mouse(svg.node());
mousedownPos = { x: pos[0], y: pos[1] };
})
.on('click', function click(feature: GeoFeature) {
if (mousedownPos) {
const pos = d3.mouse(svg.node());
const dx = Math.abs(pos[0] - mousedownPos.x);
const dy = Math.abs(pos[1] - mousedownPos.y);
const dragThreshold = 5;
if (dx < dragThreshold && dy < dragThreshold) {
handleClick(feature);
}
mousedownPos = null;
}
});
sel.exit().remove();
highlightSelectedRegion();
.on('click', clicked);
}
const map = maps[country];

View File

@@ -17,7 +17,7 @@
* under the License.
*/
import { t } from '@apache-superset/core/translation';
import { ChartMetadata, ChartPlugin, Behavior } from '@superset-ui/core';
import { ChartMetadata, ChartPlugin } from '@superset-ui/core';
import transformProps from './transformProps';
import exampleUsa from './images/exampleUsa.jpg';
import exampleUsaDark from './images/exampleUsa-dark.jpg';
@@ -49,11 +49,6 @@ const metadata = new ChartMetadata({
thumbnail,
thumbnailDark,
useLegacyApi: true,
behaviors: [
Behavior.InteractiveChart,
Behavior.DrillToDetail,
Behavior.DrillBy,
],
});
export default class CountryMapChartPlugin extends ChartPlugin {

View File

@@ -19,18 +19,8 @@
import { ChartProps, getValueFormatter } from '@superset-ui/core';
export default function transformProps(chartProps: ChartProps) {
const { width, height, formData, queriesData, datasource } = chartProps;
const {
width,
height,
formData,
queriesData,
datasource,
hooks = {},
filterState,
emitCrossFilters,
} = chartProps;
const {
entity,
linearColorScheme,
numberFormat,
currencyFormat,
@@ -59,8 +49,6 @@ export default function transformProps(chartProps: ChartProps) {
detectedCurrency,
);
const { onContextMenu, setDataMask } = hooks;
return {
width,
height,
@@ -71,10 +59,5 @@ export default function transformProps(chartProps: ChartProps) {
colorScheme,
sliceId,
formatter,
entity,
onContextMenu,
setDataMask,
emitCrossFilters,
filterState,
};
}

View File

@@ -133,11 +133,10 @@ describe('CountryMap (legacy d3)', () => {
expect(popup!).toHaveStyle({ display: 'none' });
});
test('emits a cross-filter data mask when a region is clicked', () => {
test('shows tooltip on mouseenter/mousemove/mouseout', async () => {
d3Any.json.mockImplementation((_url: string, cb: D3JsonCallback) =>
cb(null, mockMapData),
);
const setDataMask = jest.fn();
render(
<ReactCountryMap
@@ -148,101 +147,19 @@ describe('CountryMap (legacy d3)', () => {
linearColorScheme="bnbColors"
colorScheme=""
formatter={jest.fn().mockReturnValue('100')}
entity="country_code"
emitCrossFilters
setDataMask={setDataMask}
filterState={{ selectedValues: [] }}
/>,
);
const region = document.querySelector('path.region');
expect(region).not.toBeNull();
// A click is only treated as a selection when it follows a mousedown
// without dragging beyond the threshold (d3.mouse is mocked to a fixed
// position, so the down/up positions match).
fireEvent.mouseDown(region!);
fireEvent.click(region!);
const popup = document.querySelector('.hover-popup');
expect(popup).not.toBeNull();
expect(setDataMask).toHaveBeenCalledTimes(1);
expect(setDataMask).toHaveBeenCalledWith(
expect.objectContaining({
extraFormData: {
filters: [{ col: 'country_code', op: 'IN', val: ['CAN'] }],
},
filterState: expect.objectContaining({ value: ['CAN'] }),
}),
);
});
fireEvent.mouseEnter(region!);
expect(popup!).toHaveStyle({ display: 'block' });
test('does not emit a cross-filter when emitCrossFilters is disabled', () => {
d3Any.json.mockImplementation((_url: string, cb: D3JsonCallback) =>
cb(null, mockMapData),
);
const setDataMask = jest.fn();
render(
<ReactCountryMap
width={500}
height={300}
data={[{ country_id: 'CAN', metric: 100 }]}
country="canada"
linearColorScheme="bnbColors"
colorScheme=""
formatter={jest.fn().mockReturnValue('100')}
entity="country_code"
emitCrossFilters={false}
setDataMask={setDataMask}
filterState={{ selectedValues: [] }}
/>,
);
const region = document.querySelector('path.region');
fireEvent.mouseDown(region!);
fireEvent.click(region!);
expect(setDataMask).not.toHaveBeenCalled();
});
test('opens the context menu with drill-by keyed on the entity control', () => {
d3Any.json.mockImplementation((_url: string, cb: D3JsonCallback) =>
cb(null, mockMapData),
);
const onContextMenu = jest.fn();
render(
<ReactCountryMap
width={500}
height={300}
data={[{ country_id: 'CAN', metric: 100 }]}
country="canada"
linearColorScheme="bnbColors"
colorScheme=""
formatter={jest.fn().mockReturnValue('100')}
entity="country_code"
onContextMenu={onContextMenu}
filterState={{ selectedValues: [] }}
/>,
);
const region = document.querySelector('path.region');
expect(region).not.toBeNull();
fireEvent.contextMenu(region!, { clientX: 123, clientY: 45 });
expect(onContextMenu).toHaveBeenCalledTimes(1);
const [[clientX, clientY, payload]] = onContextMenu.mock.calls;
expect(clientX).toBe(123);
expect(clientY).toBe(45);
expect(payload.drillToDetail).toEqual([
{ col: 'country_code', op: '==', val: 'CAN', formattedVal: 'CAN' },
]);
// groupbyFieldName must be the form-data control key ('entity'), not the
// selected column value ('country_code'), so DrillByModal can map the
// selection back to the chart control.
expect(payload.drillBy).toEqual({
filters: [{ col: 'country_code', op: '==', val: 'CAN' }],
groupbyFieldName: 'entity',
});
fireEvent.mouseOut(region!);
expect(popup!).toHaveStyle({ display: 'none' });
});
});

View File

@@ -1,76 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps } from '@superset-ui/core';
import transformProps from '../src/transformProps';
const onContextMenu = jest.fn();
const setDataMask = jest.fn();
const createProps = (formDataOverrides = {}, chartPropsOverrides = {}) =>
({
width: 800,
height: 600,
formData: {
entity: 'country_code',
linearColorScheme: 'bnbColors',
numberFormat: '.2f',
selectCountry: 'France',
colorScheme: '',
sliceId: 1,
metric: 'count',
...formDataOverrides,
},
queriesData: [{ data: [{ country_id: 'FRA', metric: 10 }] }],
datasource: { currencyFormats: {}, columnFormats: {} },
hooks: { onContextMenu, setDataMask },
filterState: { selectedValues: ['FRA'] },
emitCrossFilters: true,
...chartPropsOverrides,
}) as unknown as ChartProps;
test('forwards cross-filter hooks and state to the chart', () => {
const transformed = transformProps(createProps());
expect(transformed).toMatchObject({
width: 800,
height: 600,
entity: 'country_code',
onContextMenu,
setDataMask,
emitCrossFilters: true,
filterState: { selectedValues: ['FRA'] },
data: [{ country_id: 'FRA', metric: 10 }],
});
});
test('lowercases the selected country for map lookup', () => {
const transformed = transformProps(createProps());
expect(transformed.country).toBe('france');
});
test('passes a null country when none is selected', () => {
const transformed = transformProps(createProps({ selectCountry: undefined }));
expect(transformed.country).toBeNull();
});
test('defaults hooks to an empty object when none are provided', () => {
const transformed = transformProps(createProps({}, { hooks: undefined }));
expect(transformed.onContextMenu).toBeUndefined();
expect(transformed.setDataMask).toBeUndefined();
});

View File

@@ -160,20 +160,6 @@ const config: ControlPanelConfig = {
},
],
['zoomable'],
[
{
name: 'y_axis_slider',
config: {
type: 'CheckboxControl',
label: t('Y-axis range slider'),
default: false,
renderTrigger: true,
description: t(
'Show a draggable slider to control the visible range of the Y-axis.',
),
},
},
],
],
},
],

View File

@@ -74,7 +74,6 @@ export default function transformProps(
yAxisTitlePosition,
sliceId,
zoomable,
yAxisSlider,
} = formData as BoxPlotQueryFormData;
const refs: Refs = {};
const colorFn = CategoricalColorNamespace.getScale(colorScheme as string);
@@ -258,28 +257,6 @@ export default function transformProps(
convertInteger(yAxisTitleMargin),
convertInteger(xAxisTitleMargin),
);
const dataZoom = [
...(zoomable
? [
{
type: 'inside',
zoomOnMouseWheel: false,
moveOnMouseWheel: true,
},
]
: []),
...(yAxisSlider
? [
{
type: 'slider',
show: true,
yAxisIndex: [0],
// Adjust the axis window without dropping data points outside the range.
filterMode: 'none',
},
]
: []),
];
const echartOptions: EChartsCoreOption = {
grid: {
...defaultGrid,
@@ -321,7 +298,15 @@ export default function transformProps(
},
},
},
dataZoom,
dataZoom: zoomable
? [
{
type: 'inside',
zoomOnMouseWheel: false,
moveOnMouseWheel: true,
},
]
: [],
};
return {

View File

@@ -30,7 +30,6 @@ export type BoxPlotQueryFormData = QueryFormData & {
numberFormat?: string;
whiskerOptions?: BoxPlotFormDataWhiskerOptions;
xTickLayout?: BoxPlotFormXTickLayout;
yAxisSlider?: boolean;
} & TitleFormData;
export type BoxPlotFormDataWhiskerOptions =

View File

@@ -71,15 +71,6 @@ describe('BoxPlot transformProps', () => {
theme: supersetTheme,
});
const buildChartProps = (formDataOverrides: Partial<SqlaFormData> = {}) =>
new ChartProps({
formData: { ...formData, ...formDataOverrides },
width: 800,
height: 600,
queriesData: chartProps.queriesData,
theme: supersetTheme,
}) as EchartsBoxPlotChartProps;
test('should transform chart props for viz', () => {
expect(transformProps(chartProps as EchartsBoxPlotChartProps)).toEqual(
expect.objectContaining({
@@ -134,41 +125,4 @@ describe('BoxPlot transformProps', () => {
}),
);
});
test('should add a vertical Y-axis slider to dataZoom when yAxisSlider is enabled', () => {
const { echartOptions } = transformProps(
buildChartProps({ yAxisSlider: true }),
);
expect((echartOptions as any).dataZoom).toEqual(
expect.arrayContaining([
expect.objectContaining({
type: 'slider',
show: true,
yAxisIndex: [0],
filterMode: 'none',
}),
]),
);
});
test('should not add a Y-axis slider when yAxisSlider is disabled', () => {
const { echartOptions } = transformProps(
buildChartProps({ yAxisSlider: false }),
);
expect((echartOptions as any).dataZoom).not.toContainEqual(
expect.objectContaining({ type: 'slider' }),
);
});
test('should combine zoomable and yAxisSlider dataZoom entries', () => {
const { echartOptions } = transformProps(
buildChartProps({ zoomable: true, yAxisSlider: true }),
);
expect((echartOptions as any).dataZoom).toEqual(
expect.arrayContaining([
expect.objectContaining({ type: 'inside' }),
expect.objectContaining({ type: 'slider', yAxisIndex: [0] }),
]),
);
});
});

View File

@@ -14,10 +14,22 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import flask_appbuilder
from werkzeug.local import LocalProxy
from superset.app import create_app # noqa: F401
from superset.extensions import (
# SQLAlchemy 2.0 enables "Annotated Declarative" mapping, which inspects class
# attribute type annotations and requires mapped attributes to use ``Mapped[...]``.
# Superset's models (and Flask-AppBuilder mixins) still carry legacy 1.x style
# annotations that are not wrapped in ``Mapped[...]``. Setting ``__allow_unmapped__``
# on the shared declarative base preserves the legacy behavior so those annotations
# are ignored by the ORM. This must run before any model class is defined (i.e.
# before importing ``superset.app``), since the annotation check happens at class
# creation time. Models can be migrated incrementally to the typed ``Mapped[...]``
# form.
flask_appbuilder.Model.__allow_unmapped__ = True
from superset.app import create_app # noqa: E402, F401
from superset.extensions import ( # noqa: E402
appbuilder, # noqa: F401
cache_manager,
db, # noqa: F401
@@ -28,7 +40,7 @@ from superset.extensions import (
security_manager, # noqa: F401
talisman, # noqa: F401
)
from superset.security import SupersetSecurityManager # noqa: F401
from superset.security import SupersetSecurityManager # noqa: E402, F401
# All of the fields located here should be considered legacy. The correct way to
# declare "global" dependencies is to define it in extensions.py,

View File

@@ -28,7 +28,7 @@ from flask import current_app as app
from pandas.errors import OutOfBoundsDatetime
from sqlalchemy import BigInteger, Boolean, Date, DateTime, Float, String, Text
from sqlalchemy.exc import MultipleResultsFound
from sqlalchemy.sql.visitors import VisitableType
from sqlalchemy.types import TypeEngine
from superset import db, security_manager
from superset.commands.dataset.exceptions import (
@@ -94,7 +94,7 @@ type_map = {
}
def get_sqla_type(native_type: str) -> VisitableType:
def get_sqla_type(native_type: str) -> TypeEngine:
if native_type.upper() in type_map:
return type_map[native_type.upper()]
@@ -107,7 +107,7 @@ def get_sqla_type(native_type: str) -> VisitableType:
)
def get_dtype(df: pd.DataFrame, dataset: SqlaTable) -> dict[str, VisitableType]:
def get_dtype(df: pd.DataFrame, dataset: SqlaTable) -> dict[str, TypeEngine]:
return {
column.column_name: get_sqla_type(column.type)
for column in dataset.columns

View File

@@ -524,11 +524,7 @@ class BaseReportState:
self._update_query_context()
try:
csv_data = get_chart_csv_data(
chart_url=url,
auth_cookies=auth_cookies,
timeout=app.config["ALERT_REPORTS_CSV_REQUEST_TIMEOUT"],
)
csv_data = get_chart_csv_data(chart_url=url, auth_cookies=auth_cookies)
elapsed_seconds = (datetime.utcnow() - start_time).total_seconds()
logger.info(
"CSV data generation from %s as user %s took %.2fs - execution_id: %s",
@@ -578,11 +574,7 @@ class BaseReportState:
self._update_query_context()
try:
dataframe = get_chart_dataframe(
url,
auth_cookies,
timeout=app.config["ALERT_REPORTS_CSV_REQUEST_TIMEOUT"],
)
dataframe = get_chart_dataframe(url, auth_cookies)
elapsed_seconds = (datetime.utcnow() - start_time).total_seconds()
logger.info(
"DataFrame generation from %s as user %s took %.2fs - execution_id: %s",

View File

@@ -1153,12 +1153,6 @@ SCREENSHOT_LOCATE_WAIT = int(timedelta(seconds=10).total_seconds())
# Time before selenium times out after waiting for all DOM class elements named
# "loading" are gone.
SCREENSHOT_LOAD_WAIT = int(timedelta(minutes=1).total_seconds())
# Maximum time (in seconds) selenium waits for an initial page navigation
# (driver.get) to complete. Without it the navigation blocks indefinitely when
# the target page never finishes loading (e.g. an unreachable WEBDRIVER_BASEURL),
# which leaves the report schedule stuck in the WORKING state. Set to None to
# disable (not recommended).
SCREENSHOT_PAGE_LOAD_WAIT = int(timedelta(minutes=2).total_seconds())
# Selenium destroy retries
SCREENSHOT_SELENIUM_RETRIES = 5
# Give selenium an headstart, in seconds
@@ -1742,11 +1736,6 @@ SMTP_MAIL_FROM = "superset@superset.com"
# If True creates a default SSL context with ssl.Purpose.CLIENT_AUTH using the
# default system root CA certificates.
SMTP_SSL_SERVER_AUTH = False
# Socket timeout (in seconds) for the SMTP connection used when sending
# alert/report emails. Without a timeout the underlying socket blocks
# indefinitely if the SMTP server becomes unreachable, which leaves report
# schedules stuck in the WORKING state. Set to None to disable (not recommended).
SMTP_TIMEOUT = 30
ENABLE_CHUNK_ENCODING = False
# Whether to bump the logging level to ERROR on the flask_appbuilder package
@@ -2095,12 +2084,6 @@ ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
# Max tries to run queries to prevent false errors caused by transient errors
# being returned to users. Set to a value >1 to enable retries.
ALERT_REPORTS_QUERY_EXECUTION_MAX_TRIES = 1
# Socket timeout (in seconds) for the HTTP request that fetches chart data when
# generating CSV/dataframe report attachments. Without a timeout the request
# blocks indefinitely if the Superset webserver is unreachable from the worker,
# which leaves the report schedule stuck in the WORKING state. Set to None to
# disable (not recommended).
ALERT_REPORTS_CSV_REQUEST_TIMEOUT = 60
# Custom width for screenshots
ALERT_REPORTS_MIN_CUSTOM_SCREENSHOT_WIDTH = 600
ALERT_REPORTS_MAX_CUSTOM_SCREENSHOT_WIDTH = 2400
@@ -2145,12 +2128,6 @@ SLACK_CACHE_TIMEOUT = int(timedelta(days=1).total_seconds())
# For workspaces with 10k+ channels, consider increasing to 10
SLACK_API_RATE_LIMIT_RETRY_COUNT = 2
# Timeout (in seconds) for outbound Slack API calls. The Slack SDK defaults to 30s;
# exposing it here lets operators grant more time for large file uploads (multi-MB
# CSVs, PDFs, screenshot sets) to congested or rate-limited Slack endpoints without
# patching code, consistent with the SMTP/CSV/screenshot timeouts.
SLACK_API_TIMEOUT = 30
# The webdriver to use for generating reports when using Selenium (not Playwright).
# This setting is ignored when PLAYWRIGHT_REPORTS_AND_THUMBNAILS is enabled, as
# Playwright always uses Chromium regardless of this value.

View File

@@ -335,7 +335,7 @@ class BaseDatasource(
return self.kind == DatasourceKind.VIRTUAL
@declared_attr
def slices(self) -> RelationshipProperty:
def slices(self) -> Mapped[list["Slice"]]:
return relationship(
"Slice",
overlaps="table",

View File

@@ -31,12 +31,7 @@ from superset.exceptions import SupersetSecurityException
from superset.extensions import cache_manager
from superset.superset_typing import FlaskResponse
from superset.utils import json
from superset.utils.core import (
apply_max_row_limit,
DatasourceType,
parse_boolean_string,
SqlExpressionType,
)
from superset.utils.core import apply_max_row_limit, DatasourceType, SqlExpressionType
from superset.views.base_api import BaseSupersetApi, statsd_metrics
logger = logging.getLogger(__name__)
@@ -130,63 +125,13 @@ class DatasourceRestApi(BaseSupersetApi):
row_limit = apply_max_row_limit(app.config["FILTER_SELECT_ROW_LIMIT"])
denormalize_column = not datasource.normalize_columns
# Cache distinct column-value results so a dashboard with many filters
# backed by the same (often heavy) virtual dataset doesn't re-execute
# the wrapping query per filter (#39342).
#
# Key fields:
# - ``rls`` — full RLS fingerprint via
# ``security_manager.get_rls_cache_key`` (the canonical helper used
# by viz.py and query_context_processor.py). This is the sole
# security-isolation field — two users with identical effective
# RLS share a cache entry (intentional: they would see identical
# filtered values anyway), while users with different RLS, guest
# sessions with different guest-token RLS, and anonymous sessions
# with no RLS each get their own partition. We deliberately do
# NOT include the raw user id; doing so would defeat the
# intended cross-user cache sharing without adding any real
# security boundary beyond what the RLS fingerprint already
# provides.
# - ``changed_on`` — auto-busts cached entries when the dataset's
# underlying SQL is edited.
# - ``uid`` / ``col`` / ``limit`` / ``denorm`` — basic query-shape
# isolation so different inputs never collide.
force = parse_boolean_string(request.args.get("force"))
cache_key = (
"col_values:"
+ hashlib.sha256(
json.dumps(
{
"uid": datasource.uid,
"col": column_name,
"limit": row_limit,
"denorm": denormalize_column,
"rls": security_manager.get_rls_cache_key(datasource),
"changed_on": str(getattr(datasource, "changed_on", "")),
},
sort_keys=True,
).encode()
).hexdigest()
)
if (
not force
and (cached := cache_manager.data_cache.get(cache_key)) is not None
):
logger.debug(
"column-values cache HIT: uid=%s col=%s", datasource.uid, column_name
)
response = self.response(200, result=cached)
response.headers["X-Cache-Status"] = "HIT"
return response
try:
payload = datasource.values_for_column(
column_name=column_name,
limit=row_limit,
denormalize_column=denormalize_column,
)
return self.response(200, result=payload)
except KeyError:
return self.response(
400, message=f"Column name {column_name} does not exist"
@@ -200,31 +145,6 @@ class DatasourceRestApi(BaseSupersetApi):
),
)
# Warn before caching very large payloads (high-cardinality columns)
# so operators can spot cache-memory pressure before Redis OOMs.
# Threshold is operator-tunable; defaults to 100k rows.
warn_threshold = app.config.get("FILTER_VALUES_CACHE_WARN_THRESHOLD", 100_000)
if (payload_size := len(payload)) > warn_threshold:
logger.warning(
"column-values payload exceeds cache-warn threshold: "
"uid=%s col=%s rows=%d threshold=%d",
datasource.uid,
column_name,
payload_size,
warn_threshold,
)
timeout = datasource.cache_timeout or app.config.get(
"CACHE_DEFAULT_TIMEOUT", 300
)
cache_manager.data_cache.set(cache_key, payload, timeout=timeout)
logger.debug(
"column-values cache MISS: uid=%s col=%s", datasource.uid, column_name
)
response = self.response(200, result=payload)
response.headers["X-Cache-Status"] = "MISS"
return response
@expose(
"/<datasource_type>/<int:datasource_id>/validate_expression/",
methods=("POST",),

View File

@@ -815,7 +815,8 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
try:
with self.superset_app.app_context():
# Simple connection test
db.engine.execute(text("SELECT 1"))
with db.engine.connect() as connection:
connection.execute(text("SELECT 1"))
except Exception:
db_uri = self.database_uri

View File

@@ -60,7 +60,7 @@ from sqlalchemy import and_, Column, or_, UniqueConstraint
from sqlalchemy.exc import MultipleResultsFound
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy.ext.hybrid import hybrid_property
from sqlalchemy.orm import Mapper, Session, validates, with_loader_criteria
from sqlalchemy.orm import Mapped, Mapper, Session, validates, with_loader_criteria
from sqlalchemy.orm.session import ORMExecuteState
from sqlalchemy.sql.elements import ColumnElement, Grouping, literal_column, TextClause
from sqlalchemy.sql.expression import Label, Select, TextAsFrom
@@ -583,7 +583,7 @@ class AuditMixinNullable(AuditMixin):
)
@declared_attr
def created_by_fk(self) -> sa.Column: # pylint: disable=arguments-renamed
def created_by_fk(self) -> Mapped[Optional[int]]: # pylint: disable=arguments-renamed
return sa.Column(
sa.Integer,
sa.ForeignKey("ab_user.id"),
@@ -592,7 +592,7 @@ class AuditMixinNullable(AuditMixin):
)
@declared_attr
def changed_by_fk(self) -> sa.Column: # pylint: disable=arguments-renamed
def changed_by_fk(self) -> Mapped[Optional[int]]: # pylint: disable=arguments-renamed
return sa.Column(
sa.Integer,
sa.ForeignKey("ab_user.id"),

View File

@@ -16,7 +16,12 @@
# under the License.
from typing import Any
from flask_sqlalchemy import BaseQuery
try:
# Flask-SQLAlchemy 3.x (required by SQLAlchemy 2.0)
from flask_sqlalchemy.query import Query as BaseQuery
except ImportError: # pragma: no cover
# Flask-SQLAlchemy 2.x
from flask_sqlalchemy import BaseQuery
from superset import security_manager
from superset.models.sql_lab import Query

View File

@@ -18,10 +18,16 @@ from typing import Any
from flask import g, has_request_context, request
from flask_babel import lazy_gettext as _
from flask_sqlalchemy import BaseQuery
from sqlalchemy import or_
from sqlalchemy.orm.query import Query
try:
# Flask-SQLAlchemy 3.x (required by SQLAlchemy 2.0)
from flask_sqlalchemy.query import Query as BaseQuery
except ImportError: # pragma: no cover
# Flask-SQLAlchemy 2.x
from flask_sqlalchemy import BaseQuery
from superset import security_manager
from superset.models.sql_lab import SavedQuery
from superset.tags.filters import BaseTagIdFilter, BaseTagNameFilter

View File

@@ -73,21 +73,9 @@ def stringify_values(array: NDArray[Any]) -> NDArray[Any]:
obj[na_obj] = None
else:
try:
val = obj.item()
if isinstance(val, (dict, list)):
try:
# Use json.dumps for valid double-quoted JSON.
# str() gives single-quoted repr like {'a': 1}
# which breaks the frontend cell viewer.
obj[...] = stringify(val)
except TypeError:
# Non-JSON-serializable value (e.g. bytes, custom
# objects): fall back to str() to avoid crashing.
obj[...] = str(val)
else:
# for simple string conversions
# this handles odd character types better
obj[...] = obj.astype(str)
# for simple string conversions
# this handles odd character types better
obj[...] = obj.astype(str)
except ValueError:
obj[...] = stringify(obj)

View File

@@ -54,7 +54,7 @@ from flask_login import AnonymousUserMixin, LoginManager
from jwt.api_jwt import _jwt_global_obj
from sqlalchemy import and_, func as sa_func, inspect, or_
from sqlalchemy.engine.base import Connection
from sqlalchemy.orm import eagerload, joinedload
from sqlalchemy.orm import joinedload
from sqlalchemy.orm.exc import MultipleResultsFound
from sqlalchemy.orm.mapper import Mapper
from sqlalchemy.orm.query import Query as SqlaQuery
@@ -1914,8 +1914,8 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
pvms = (
self.session.query(self.permissionview_model)
.options(
eagerload(self.permissionview_model.permission),
eagerload(self.permissionview_model.view_menu),
joinedload(self.permissionview_model.permission),
joinedload(self.permissionview_model.view_menu),
)
.all()
)

View File

@@ -938,11 +938,6 @@ def send_mime_email(
smtp_starttls = config["SMTP_STARTTLS"]
smtp_ssl = config["SMTP_SSL"]
smtp_ssl_server_auth = config["SMTP_SSL_SERVER_AUTH"]
# A missing timeout means the socket blocks forever when the SMTP server is
# unreachable, wedging the report schedule in the WORKING state. Fall back to
# the key being absent for backwards compatibility with custom configs.
# Keep this fallback in sync with the SMTP_TIMEOUT default in config.py.
smtp_timeout = config.get("SMTP_TIMEOUT", 30)
if dryrun:
logger.info("Dryrun enabled, email notification content is below:")
@@ -953,27 +948,17 @@ def send_mime_email(
# root CA certificates
ssl_context = ssl.create_default_context() if smtp_ssl_server_auth else None
smtp = (
smtplib.SMTP_SSL(
smtp_host, smtp_port, context=ssl_context, timeout=smtp_timeout
)
smtplib.SMTP_SSL(smtp_host, smtp_port, context=ssl_context)
if smtp_ssl
else smtplib.SMTP(smtp_host, smtp_port, timeout=smtp_timeout)
else smtplib.SMTP(smtp_host, smtp_port)
)
try:
if smtp_starttls:
smtp.starttls(context=ssl_context)
if smtp_user and smtp_password:
smtp.login(smtp_user, smtp_password)
logger.debug("Sent an email to %s", str(e_to))
smtp.sendmail(e_from, e_to, mime_msg.as_string())
finally:
# Always release the socket; the new timeout means starttls/login/
# sendmail can raise, and a skipped quit() would leak connections in
# the long-lived worker process.
try:
smtp.quit()
except smtplib.SMTPException:
pass
if smtp_starttls:
smtp.starttls(context=ssl_context)
if smtp_user and smtp_password:
smtp.login(smtp_user, smtp_password)
logger.debug("Sent an email to %s", str(e_to))
smtp.sendmail(e_from, e_to, mime_msg.as_string())
smtp.quit()
def recipients_string_to_list(address_string: str | None) -> list[str]:

View File

@@ -90,18 +90,14 @@ def df_to_escaped_csv(df: pd.DataFrame, **kwargs: Any) -> Any:
def get_chart_csv_data(
chart_url: str,
auth_cookies: Optional[dict[str, str]] = None,
timeout: Optional[float] = None,
chart_url: str, auth_cookies: Optional[dict[str, str]] = None
) -> Optional[bytes]:
content = None
if auth_cookies:
opener = urllib.request.build_opener()
cookie_str = ";".join([f"{key}={val}" for key, val in auth_cookies.items()])
opener.addheaders.append(("Cookie", cookie_str))
# A missing timeout means the socket blocks forever when the Superset
# webserver is unreachable, wedging the report schedule in WORKING.
response = opener.open(chart_url, timeout=timeout)
response = opener.open(chart_url)
content = response.read()
if response.getcode() != 200:
raise URLError(response.getcode())
@@ -111,13 +107,11 @@ def get_chart_csv_data(
def get_chart_dataframe(
chart_url: str,
auth_cookies: Optional[dict[str, str]] = None,
timeout: Optional[float] = None,
chart_url: str, auth_cookies: Optional[dict[str, str]] = None
) -> Optional[pd.DataFrame]:
# Disable all the unnecessary-lambda violations in this function
# pylint: disable=unnecessary-lambda
content = get_chart_csv_data(chart_url, auth_cookies, timeout)
content = get_chart_csv_data(chart_url, auth_cookies)
if content is None:
return None

View File

@@ -430,7 +430,7 @@ class SecretsMigrator:
re_encrypted_columns = {}
for column_name, encrypted_type in columns.items():
raw_value = self._read_bytes(column_name, row[column_name])
raw_value = self._read_bytes(column_name, row._mapping[column_name])
# NULL values aren't encrypted; there is nothing to migrate.
if raw_value is None:
@@ -508,13 +508,12 @@ class SecretsMigrator:
set_cols = ",".join(f"{name} = :{name}" for name in re_encrypted_columns)
where_clause = " AND ".join(f"{pk} = :_pk_{pk}" for pk in pk_columns)
pk_bind = {f"_pk_{pk}": row[pk] for pk in pk_columns}
pk_bind = {f"_pk_{pk}": row._mapping[pk] for pk in pk_columns}
conn.execute(
text(
f"UPDATE {table_name} SET {set_cols} WHERE {where_clause}" # noqa: S608
),
**pk_bind,
**re_encrypted_columns,
{**pk_bind, **re_encrypted_columns},
)
def run(self) -> ReEncryptStats:

View File

@@ -31,7 +31,7 @@ from flask_appbuilder import Model
from sqlalchemy import Column, inspect, MetaData, Table as DBTable, text
from sqlalchemy.dialects import postgresql
from sqlalchemy.sql import func
from sqlalchemy.sql.visitors import VisitableType
from sqlalchemy.types import TypeEngine
from superset import db
from superset.sql.parse import Table
@@ -42,7 +42,7 @@ logger = logging.getLogger(__name__)
class ColumnInfo(TypedDict):
name: str
type: VisitableType
type: TypeEngine
nullable: bool
default: Optional[Any]
autoincrement: str

View File

@@ -48,11 +48,7 @@ def get_slack_client() -> WebClient:
token: str = app.config["SLACK_API_TOKEN"]
if callable(token):
token = token()
client = WebClient(
token=token,
proxy=app.config["SLACK_PROXY"],
timeout=app.config["SLACK_API_TIMEOUT"],
)
client = WebClient(token=token, proxy=app.config["SLACK_PROXY"])
max_retry_count = app.config.get("SLACK_API_RATE_LIMIT_RETRY_COUNT", 2)
rate_limit_handler = RateLimitErrorRetryHandler(max_retry_count=max_retry_count)

View File

@@ -502,23 +502,9 @@ class WebDriverSelenium(WebDriverProxy):
self._driver = self._create()
if not self._driver:
raise RuntimeError("WebDriver creation failed")
try:
self._driver.set_window_size(*self._window)
# Bound driver.get() so an unreachable page raises a
# TimeoutException instead of blocking the worker (and the
# report schedule) forever.
page_load_wait = app.config["SCREENSHOT_PAGE_LOAD_WAIT"]
if page_load_wait is not None:
self._driver.set_page_load_timeout(page_load_wait)
if self._user:
self._auth(self._user)
except Exception:
# A failure mid-setup (e.g. the new page-load timeout or auth
# raising) would otherwise leave a partially initialized,
# unauthenticated driver cached for reuse. Tear it down so the
# next access recreates it cleanly.
self._destroy()
raise
self._driver.set_window_size(*self._window)
if self._user:
self._auth(self._user)
return self._driver
def _create_firefox_driver(

View File

@@ -15,7 +15,6 @@
# specific language governing permissions and limitations
# under the License.
from datetime import datetime
from unittest.mock import ANY, patch
import pytest
@@ -24,21 +23,12 @@ from sqlalchemy.sql.elements import TextClause
from superset import db, security_manager
from superset.connectors.sqla.models import SqlaTable
from superset.daos.exceptions import DatasourceTypeNotSupportedError
from superset.extensions import cache_manager
from superset.utils import json
from tests.integration_tests.base_tests import SupersetTestCase
from tests.integration_tests.constants import ADMIN_USERNAME, GAMMA_USERNAME
class TestDatasourceApi(SupersetTestCase):
def setUp(self):
# Clear the column-values cache before every test so that
# ``get_column_values`` always re-runs ``values_for_column`` rather
# than returning a payload populated by a previous test. Prevents
# order-dependent flakes now that the endpoint caches its result.
super().setUp()
cache_manager.data_cache.clear()
def get_virtual_dataset(self):
return (
db.session.query(SqlaTable)
@@ -215,123 +205,6 @@ class TestDatasourceApi(SupersetTestCase):
response = json.loads(rv.data.decode("utf-8"))
assert response["result"] == []
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_cache_hit_skips_query(self, values_for_column_mock):
"""Regression test for #39342.
Two identical requests for the same column on the same datasource
should hit ``values_for_column`` exactly once — the second request
returns the cached payload.
"""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["a", "b", "c"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
url = f"api/v1/datasource/table/{table.id}/column/col2/values/"
rv1 = self.client.get(url)
rv2 = self.client.get(url)
assert rv1.status_code == 200
assert rv2.status_code == 200
assert json.loads(rv2.data.decode("utf-8"))["result"] == ["a", "b", "c"]
assert values_for_column_mock.call_count == 1
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_force_bypasses_cache(self, values_for_column_mock):
"""``?force=true`` should bypass the cache and re-query the source."""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["a", "b"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
url = f"api/v1/datasource/table/{table.id}/column/col2/values/"
self.client.get(url)
self.client.get(f"{url}?force=true")
assert values_for_column_mock.call_count == 2
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_cache_isolated_per_column(self, values_for_column_mock):
"""Different columns on the same datasource must not share a cache
entry — otherwise filter values would be silently swapped."""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["x"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
self.client.get(f"api/v1/datasource/table/{table.id}/column/col1/values/")
self.client.get(f"api/v1/datasource/table/{table.id}/column/col2/values/")
assert values_for_column_mock.call_count == 2
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_cache_busts_on_changed_on(self, values_for_column_mock):
"""Editing the underlying virtual dataset SQL bumps ``changed_on``,
which is part of the cache key — so the next request must miss the
cache and re-run the query."""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["v"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
url = f"api/v1/datasource/table/{table.id}/column/col2/values/"
self.client.get(url)
# Simulate an edit to the dataset; ``changed_on`` is what the cache
# key reads, so any new value forces a miss.
table.changed_on = datetime(2030, 1, 1)
db.session.flush()
self.client.get(url)
assert values_for_column_mock.call_count == 2
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.datasource.api.security_manager.get_rls_cache_key")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_cache_isolated_per_rls_context(
self, values_for_column_mock, get_rls_cache_key_mock
):
"""RLS safety for guest/embedded sessions. ``get_user_id()`` returns
``None`` for guest users, so two embedded dashboards with different
guest-token RLS would otherwise collide on ``user=None``. Including
the RLS fingerprint in the cache key keeps them separate."""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["v"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
url = f"api/v1/datasource/table/{table.id}/column/col2/values/"
get_rls_cache_key_mock.return_value = ["dept='A'"]
self.client.get(url)
get_rls_cache_key_mock.return_value = ["dept='B'"]
self.client.get(url)
assert values_for_column_mock.call_count == 2
@pytest.mark.usefixtures("app_context", "virtual_dataset")
@patch("superset.models.helpers.ExploreMixin.values_for_column")
def test_get_column_values_response_advertises_cache_status(
self, values_for_column_mock
):
"""The ``X-Cache-Status`` response header should advertise MISS on
the populating request and HIT on the next identical request, so
operators can debug cache behavior from logs or browser devtools."""
cache_manager.data_cache.clear()
values_for_column_mock.return_value = ["v"]
self.login(ADMIN_USERNAME)
table = self.get_virtual_dataset()
url = f"api/v1/datasource/table/{table.id}/column/col2/values/"
rv_miss = self.client.get(url)
rv_hit = self.client.get(url)
assert rv_miss.headers.get("X-Cache-Status") == "MISS"
assert rv_hit.headers.get("X-Cache-Status") == "HIT"
@patch("superset.datasource.api.security_manager.can_access")
@patch("superset.datasource.api.GetCombinedDatasourceListCommand.run")
def test_combined_list_invalid_order_column(

View File

@@ -193,9 +193,7 @@ class TestEmailSmtp(SupersetTestCase):
msg = MIMEMultipart()
utils.send_mime_email("from", "to", msg, current_app.config, dryrun=False)
mock_smtp.assert_called_with(
current_app.config["SMTP_HOST"],
current_app.config["SMTP_PORT"],
timeout=current_app.config["SMTP_TIMEOUT"],
current_app.config["SMTP_HOST"], current_app.config["SMTP_PORT"]
)
assert mock_smtp.return_value.starttls.called
mock_smtp.return_value.login.assert_called_with(
@@ -220,7 +218,6 @@ class TestEmailSmtp(SupersetTestCase):
current_app.config["SMTP_HOST"],
current_app.config["SMTP_PORT"],
context=None,
timeout=current_app.config["SMTP_TIMEOUT"],
)
@mock.patch("smtplib.SMTP_SSL")
@@ -238,7 +235,6 @@ class TestEmailSmtp(SupersetTestCase):
current_app.config["SMTP_HOST"],
current_app.config["SMTP_PORT"],
context=mock.ANY,
timeout=current_app.config["SMTP_TIMEOUT"],
)
called_context = mock_smtp_ssl.call_args.kwargs["context"]
assert called_context.verify_mode == ssl.CERT_REQUIRED
@@ -270,9 +266,7 @@ class TestEmailSmtp(SupersetTestCase):
)
assert not mock_smtp_ssl.called
mock_smtp.assert_called_with(
current_app.config["SMTP_HOST"],
current_app.config["SMTP_PORT"],
timeout=current_app.config["SMTP_TIMEOUT"],
current_app.config["SMTP_HOST"], current_app.config["SMTP_PORT"]
)
assert not mock_smtp.login.called
current_app.config["SMTP_USER"] = smtp_user
@@ -287,36 +281,6 @@ class TestEmailSmtp(SupersetTestCase):
assert not mock_smtp.called
assert not mock_smtp_ssl.called
@mock.patch("smtplib.SMTP_SSL")
@mock.patch("smtplib.SMTP")
def test_send_mime_respects_custom_timeout(
self, mock_smtp: mock.Mock, mock_smtp_ssl: mock.Mock
) -> None:
"""A configured SMTP_TIMEOUT must reach the smtplib client.
A missing timeout would block the worker forever when the SMTP server
is unreachable, wedging the report schedule in WORKING (issue #40047).
"""
config = {**current_app.config, "SMTP_TIMEOUT": 7, "SMTP_SSL": False}
mock_smtp.return_value = mock.Mock()
utils.send_mime_email("from", ["to"], MIMEMultipart(), config, dryrun=False)
assert mock_smtp.call_args.kwargs["timeout"] == 7
@mock.patch("smtplib.SMTP_SSL")
@mock.patch("smtplib.SMTP")
def test_send_mime_timeout_defaults_when_unset(
self, mock_smtp: mock.Mock, mock_smtp_ssl: mock.Mock
) -> None:
"""An absent SMTP_TIMEOUT key falls back to the 30s default.
Custom configs predating SMTP_TIMEOUT must still get a finite timeout.
"""
config = {k: v for k, v in current_app.config.items() if k != "SMTP_TIMEOUT"}
config["SMTP_SSL"] = False
mock_smtp.return_value = mock.Mock()
utils.send_mime_email("from", ["to"], MIMEMultipart(), config, dryrun=False)
assert mock_smtp.call_args.kwargs["timeout"] == 30
if __name__ == "__main__":
unittest.main()

View File

@@ -23,7 +23,6 @@ from uuid import uuid4
import pytest
from flask.ctx import AppContext
from flask_appbuilder.security.sqla.models import User
from flask_sqlalchemy import BaseQuery
from freezegun import freeze_time
from slack_sdk.errors import (
BotUserAccessError,
@@ -37,6 +36,13 @@ from slack_sdk.errors import (
)
from sqlalchemy.sql import func, text
try:
# Flask-SQLAlchemy 3.x (required by SQLAlchemy 2.0)
from flask_sqlalchemy.query import Query as BaseQuery
except ImportError: # pragma: no cover
# Flask-SQLAlchemy 2.x
from flask_sqlalchemy import BaseQuery
from superset import db
from superset.commands.report.exceptions import (
AlertQueryError,
@@ -2007,11 +2013,7 @@ def test_slack_token_callable_chart_report(
TEST_ID, create_report_slack_chart.id, datetime.utcnow()
).run()
slack_token_mock.assert_called()
slack_client_mock_class.assert_called_with(
token="cool_code", # noqa: S106
proxy=None,
timeout=30,
)
slack_client_mock_class.assert_called_with(token="cool_code", proxy=None) # noqa: S106
assert_log(ReportState.SUCCESS)

View File

@@ -31,6 +31,28 @@ from superset.utils.encrypt import (
from tests.integration_tests.base_tests import SupersetTestCase
def make_row(values: dict[str, Any]) -> Any:
"""Build a genuine SQLAlchemy ``Row`` from a mapping.
``SecretsMigrator._re_encrypt_row`` consumes the ``Row`` objects yielded by
``conn.execute(...)``, reading column values through ``row._mapping`` per the
SQLAlchemy 2.0 Row API. Tests must therefore pass a real ``Row`` rather than a
plain ``dict`` (which lacks ``_mapping``). The constructor signature differs
between SQLAlchemy 1.4 and 2.0, so both are handled here.
"""
from sqlalchemy.engine.result import SimpleResultMetaData
from sqlalchemy.engine.row import Row
metadata = SimpleResultMetaData(tuple(values))
data = tuple(values.values())
try:
# SQLAlchemy 2.0: Row(parent, processors, key_to_index, data)
return Row(metadata, None, metadata._key_to_index, data)
except AttributeError:
# SQLAlchemy 1.4: Row(parent, processors, keymap, key_style, data)
return Row(metadata, None, metadata._keymap, Row._default_key_style, data)
class CustomEncFieldAdapter(AbstractEncryptedFieldAdapter):
def create(
self,
@@ -224,7 +246,8 @@ class EncryptedFieldTest(SupersetTestCase):
current_field = encrypted_field_factory.create(String(1024))
conn = MagicMock()
row = {"uuid": b"\x00" * 16, "configuration": ciphertext}
pk_value = b"\x00" * 16
row = make_row({"uuid": pk_value, "configuration": ciphertext})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -239,9 +262,11 @@ class EncryptedFieldTest(SupersetTestCase):
assert conn.execute.call_count == 1
stmt = str(conn.execute.call_args.args[0])
assert "WHERE uuid = :_pk_uuid" in stmt
kwargs = conn.execute.call_args.kwargs
assert kwargs["_pk_uuid"] == row["uuid"]
assert "configuration" in kwargs
# The migrator passes bind params positionally (conn.execute(stmt, params)),
# so read them from args[1] rather than kwargs.
params = conn.execute.call_args.args[1]
assert params["_pk_uuid"] == pk_value
assert "configuration" in params
assert stats == ReEncryptStats(re_encrypted=1, skipped=0, failed=0)
def test_re_encrypt_row_is_idempotent(self):
@@ -264,7 +289,7 @@ class EncryptedFieldTest(SupersetTestCase):
assert field.process_result_value(ciphertext, dialect) == "hunter2"
conn = MagicMock()
row = {"uuid": b"\x00" * 16, "configuration": ciphertext}
row = make_row({"uuid": b"\x00" * 16, "configuration": ciphertext})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -308,7 +333,7 @@ class EncryptedFieldTest(SupersetTestCase):
ciphertext = field.process_bind_param("hunter2", dialect)
conn = MagicMock()
row = {"uuid": b"\x00" * 16, "configuration": ciphertext}
row = make_row({"uuid": b"\x00" * 16, "configuration": ciphertext})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -342,7 +367,7 @@ class EncryptedFieldTest(SupersetTestCase):
field = encrypted_field_factory.create(String(1024))
conn = MagicMock()
row = {"uuid": b"\x00" * 16, "configuration": b"not-valid-ciphertext"}
row = make_row({"uuid": b"\x00" * 16, "configuration": b"not-valid-ciphertext"})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -374,7 +399,7 @@ class EncryptedFieldTest(SupersetTestCase):
field = encrypted_field_factory.create(String(1024))
conn = MagicMock()
row = {"uuid": b"\x00" * 16, "configuration": None}
row = make_row({"uuid": b"\x00" * 16, "configuration": None})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001

View File

@@ -195,7 +195,7 @@ class TestSupersetAppInitializer:
mock_app.app_context.return_value.__enter__.return_value = MagicMock()
with patch("superset.initialization.db") as mock_db:
mock_db.engine.execute.side_effect = Exception("Connection Failed")
mock_db.engine.connect.side_effect = Exception("Connection Failed")
with patch.object(
SupersetAppInitializer,
@@ -219,7 +219,7 @@ class TestSupersetAppInitializer:
mock_app.app_context.return_value.__enter__.return_value = MagicMock()
with patch("superset.initialization.db") as mock_db:
mock_db.engine.execute.side_effect = Exception("Connection Failed")
mock_db.engine.connect.side_effect = Exception("Connection Failed")
with patch.object(
SupersetAppInitializer,

View File

@@ -450,93 +450,3 @@ def test_stringify_extension_columns() -> None:
# plain binary BLOBs and other types are left untouched
assert pa.types.is_binary(result.schema.field("blob").type)
assert pa.types.is_integer(result.schema.field("n").type)
def test_stringify_values_dict_and_list_produce_valid_json() -> None:
"""
ClickHouse native JSON and Map types return Python dicts. When stringified for
Arrow array storage they must produce valid double-quoted JSON strings, not
Python's single-quoted repr. Single-quoted strings pass the cheap '{' prefix
check in the frontend's safeJsonObjectParse but then fail JSONbig.parse(),
so the SQL Lab cell viewer never activates.
"""
data = np.array(
[
{"key": "value", "nested": {"a": 1}},
# str() gives ['a', 'b'] (single-quoted, invalid JSON);
# json.dumps gives ["a", "b"] (double-quoted, valid JSON).
["a", "b"],
{"items": [1, 2, 3], "d": "Hello, World!"},
None,
],
dtype=object,
)
result = stringify_values(data)
# Must be valid JSON strings (double-quoted), not Python repr (single-quoted)
assert result[0] == '{"key": "value", "nested": {"a": 1}}'
assert result[1] == '["a", "b"]'
assert result[2] == '{"items": [1, 2, 3], "d": "Hello, World!"}'
assert result[3] is None
# Parseable by a JSON parser — confirms the frontend's JSON.parse would succeed
parsed = superset_json.loads(result[0])
assert parsed == {"key": "value", "nested": {"a": 1}}
parsed = superset_json.loads(result[1])
assert parsed == ["a", "b"]
def test_clickhouse_json_column_in_pa_table_is_valid_json() -> None:
"""
Verify that ClickHouse-style heterogeneous dict columns produce valid JSON
strings in the Arrow table used by the msgpack serialization path.
When clickhouse-connect returns Python dicts for JSON/Map type columns,
SupersetResultSet must serialize them with json.dumps (not str()) so that
the SQL Lab grid's cell viewer can call JSON.parse on the value.
"""
data = [
(1, {"a": {"b": 42}, "c": [1, 2, 3], "d": "Hello, World!"}),
(2, {"e": 5}),
(3, None),
]
description = [
("id", 3, None, None, None, None, None),
("json_col", None, None, None, None, None, None),
]
result_set = SupersetResultSet(data, description, BaseEngineSpec) # type: ignore
df_from_pa = SupersetResultSet.convert_table_to_df(result_set.pa_table)
val0 = df_from_pa["json_col"].iloc[0]
val1 = df_from_pa["json_col"].iloc[1]
# Values in pa_table must be valid JSON strings (parseable by JSON.parse)
assert isinstance(val0, str)
assert isinstance(val1, str)
# Double-quoted JSON, not single-quoted Python repr
parsed0 = superset_json.loads(val0)
assert parsed0 == {"a": {"b": 42}, "c": [1, 2, 3], "d": "Hello, World!"}
parsed1 = superset_json.loads(val1)
assert parsed1 == {"e": 5}
def test_stringify_values_non_serializable_dict_falls_back_to_str() -> None:
"""
When a dict/list contains a value that json.dumps cannot serialize (e.g. bytes),
stringify_values must fall back to str() rather than raising TypeError and crashing
the result-set construction path.
"""
class _Unserializable:
def __repr__(self) -> str:
return "unserializable"
data = np.array(
[{"key": _Unserializable()}],
dtype=object,
)
# Must not raise — falls back to str()
result = stringify_values(data)
assert result[0] == str({"key": _Unserializable()})

View File

@@ -16,9 +16,6 @@
# under the License.
from typing import Any
from unittest import mock
import pandas as pd
import pyarrow as pa
import pytest # noqa: F401
@@ -28,7 +25,6 @@ from superset.utils import csv, json
from superset.utils.core import GenericDataType
from superset.utils.csv import (
df_to_escaped_csv,
get_chart_csv_data,
get_chart_dataframe,
)
@@ -79,33 +75,20 @@ def test_escape_value():
assert result == "'\rfoo"
def fake_get_chart_csv_data_none(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
"""Return ``None`` to mock a fetch that yields no payload."""
def fake_get_chart_csv_data_none(chart_url, auth_cookies=None):
return None
def fake_get_chart_csv_data_empty(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
"""Return an encoded empty-result payload for dataframe-empty scenarios."""
fake_result: dict[str, Any] = {
def fake_get_chart_csv_data_empty(chart_url, auth_cookies=None):
# Return JSON with empty data so that the resulting DataFrame is empty
fake_result = {
"result": [{"data": {}, "coltypes": [], "colnames": [], "indexnames": []}]
}
return json.dumps(fake_result).encode("utf-8")
def fake_get_chart_csv_data_valid(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
"""Return a non-temporal payload used to verify dataframe construction."""
def fake_get_chart_csv_data_valid(chart_url, auth_cookies=None):
# Return JSON with non-temporal data and valid indexnames so that they are used.
fake_result = {
"result": [
{
@@ -120,11 +103,7 @@ def fake_get_chart_csv_data_valid(
return json.dumps(fake_result).encode("utf-8")
def fake_get_chart_csv_data_temporal(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
def fake_get_chart_csv_data_temporal(chart_url, auth_cookies=None):
"""
Return JSON with a temporal column and valid indexnames
so that a MultiIndex is built.
@@ -143,12 +122,8 @@ def fake_get_chart_csv_data_temporal(
return json.dumps(fake_result).encode("utf-8")
def fake_get_chart_csv_data_hierarchical(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
"""Return hierarchical-column mock data for MultiIndex assertions."""
def fake_get_chart_csv_data_hierarchical(chart_url, auth_cookies=None):
# Return JSON with hierarchical column (list-based) and matching index names.
fake_result = {
"result": [
{
@@ -163,11 +138,7 @@ def fake_get_chart_csv_data_hierarchical(
return json.dumps(fake_result).encode("utf-8")
def fake_get_chart_csv_data_with_na_values(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
def fake_get_chart_csv_data_with_na_values(chart_url, auth_cookies=None):
# Return JSON with data containing "NA" string value that will be treated as null
fake_result = {
"result": [
@@ -409,41 +380,3 @@ def test_get_chart_dataframe_preserves_na_string_values(
last_name_values = df[("last_name",)].values
assert last_name_values[0] == "Smith"
assert last_name_values[1] == "NA"
def test_get_chart_csv_data_passes_timeout_to_opener(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The timeout argument must reach the urllib opener's open() call."""
# Without a timeout the request blocks forever when the webserver is
# unreachable, wedging the report schedule in WORKING (issue #40047).
mock_response = mock.Mock()
mock_response.read.return_value = b"data"
mock_response.getcode.return_value = 200
mock_opener = mock.Mock()
mock_opener.open.return_value = mock_response
mock_opener.addheaders = []
monkeypatch.setattr(
"urllib.request.build_opener", mock.Mock(return_value=mock_opener)
)
get_chart_csv_data("http://dummy-url", auth_cookies={"session": "x"}, timeout=42)
mock_opener.open.assert_called_once_with("http://dummy-url", timeout=42)
def test_get_chart_dataframe_forwards_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
"""get_chart_dataframe must forward its timeout down to get_chart_csv_data."""
captured: dict[str, float | None] = {}
def fake(
chart_url: str,
auth_cookies: dict[str, str] | None = None,
timeout: float | None = None,
) -> bytes | None:
captured["timeout"] = timeout
return None
monkeypatch.setattr(csv, "get_chart_csv_data", fake)
get_chart_dataframe("http://dummy-url", timeout=99)
assert captured["timeout"] == 99

View File

@@ -57,6 +57,18 @@ def _engine_migrator(target_engine: type) -> SecretsMigrator:
return migrator
class _Row:
"""Minimal stand-in for a SQLAlchemy ``Row``.
``_re_encrypt_row`` accesses columns via ``row._mapping[...]`` (the
SQLAlchemy 2.0-compatible idiom), so the fixtures wrap their column dicts
in an object exposing that attribute rather than passing a bare ``dict``.
"""
def __init__(self, mapping: dict[str, object]) -> None:
self._mapping = mapping
def test_default_engine_is_aes_cbc() -> None:
"""Without config, the adapter keeps the historical AES-CBC engine."""
field = SQLAlchemyUtilsAdapter().create(SECRET, String(128))
@@ -156,7 +168,7 @@ def test_engine_migration_cbc_to_gcm_re_encrypts() -> None:
migrator = _engine_migrator(AesGcmEngine)
conn = MagicMock()
row = {"id": 1, "password": ciphertext}
row = _Row({"id": 1, "password": ciphertext})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -165,7 +177,7 @@ def test_engine_migration_cbc_to_gcm_re_encrypts() -> None:
assert stats == ReEncryptStats(re_encrypted=1)
assert conn.execute.call_count == 1
new_value = conn.execute.call_args.kwargs["password"]
new_value = conn.execute.call_args.args[1]["password"]
# The stored value changed and now decrypts as GCM back to the plaintext.
assert new_value != ciphertext
gcm = _encrypted_type(AesGcmEngine)
@@ -183,7 +195,7 @@ def test_engine_migration_idempotent_for_already_target() -> None:
migrator = _engine_migrator(AesGcmEngine)
conn = MagicMock()
row = {"id": 1, "password": gcm_value}
row = _Row({"id": 1, "password": gcm_value})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -206,7 +218,7 @@ def test_engine_migration_reads_cbc_after_config_already_flipped() -> None:
migrator = _engine_migrator(AesGcmEngine)
conn = MagicMock()
row = {"id": 1, "password": cbc_value}
row = _Row({"id": 1, "password": cbc_value})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -214,7 +226,7 @@ def test_engine_migration_reads_cbc_after_config_already_flipped() -> None:
)
assert stats == ReEncryptStats(re_encrypted=1)
new_value = conn.execute.call_args.kwargs["password"]
new_value = conn.execute.call_args.args[1]["password"]
assert gcm_column.process_result_value(new_value, DIALECT) == "hunter2"
@@ -231,7 +243,7 @@ def test_engine_migration_gcm_to_cbc_rolls_back() -> None:
migrator = _engine_migrator(AesEngine)
conn = MagicMock()
row = {"id": 1, "password": gcm_value}
row = _Row({"id": 1, "password": gcm_value})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -239,7 +251,7 @@ def test_engine_migration_gcm_to_cbc_rolls_back() -> None:
)
assert stats == ReEncryptStats(re_encrypted=1)
new_value = conn.execute.call_args.kwargs["password"]
new_value = conn.execute.call_args.args[1]["password"]
assert new_value != gcm_value
# The rolled-back value now decrypts as AES-CBC back to the plaintext.
assert _encrypted_type(AesEngine).process_result_value(new_value, DIALECT) == (
@@ -272,7 +284,7 @@ def test_rollback_authenticated_probe_wins_over_spurious_cbc_skip() -> None:
spurious_target.process_bind_param.return_value = b"new-cbc-ciphertext"
conn = MagicMock()
row = {"id": 1, "password": gcm_value}
row = _Row({"id": 1, "password": gcm_value})
stats = ReEncryptStats()
with mock.patch.object(migrator, "_target_type", return_value=spurious_target):
@@ -302,7 +314,7 @@ def test_combined_key_rotation_and_engine_migration() -> None:
migrator._previous_secret_key = old_key # noqa: SLF001 # rotate key too
conn = MagicMock()
row = {"id": 1, "password": old_value}
row = _Row({"id": 1, "password": old_value})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -310,7 +322,7 @@ def test_combined_key_rotation_and_engine_migration() -> None:
)
assert stats == ReEncryptStats(re_encrypted=1)
new_value = conn.execute.call_args.kwargs["password"]
new_value = conn.execute.call_args.args[1]["password"]
# The migrated value decrypts as GCM under the *current* key.
assert _encrypted_type(AesGcmEngine).process_result_value(new_value, DIALECT) == (
"hunter2"
@@ -346,7 +358,7 @@ def test_key_rotation_for_aes_gcm_column() -> None:
migrator = _key_rotation_migrator(previous_secret_key=old_key)
conn = MagicMock()
row = {"id": 1, "password": old_value}
row = _Row({"id": 1, "password": old_value})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001
@@ -354,7 +366,7 @@ def test_key_rotation_for_aes_gcm_column() -> None:
)
assert stats == ReEncryptStats(re_encrypted=1)
new_value = conn.execute.call_args.kwargs["password"]
new_value = conn.execute.call_args.args[1]["password"]
assert gcm_column.process_result_value(new_value, DIALECT) == "hunter2"
@@ -362,7 +374,7 @@ def test_engine_migration_unreadable_value_counts_as_failure() -> None:
"""A value no engine/key can read is a failure, not a silent pass-through."""
migrator = _engine_migrator(AesGcmEngine)
conn = MagicMock()
row = {"id": 1, "password": b"not-valid-ciphertext"}
row = _Row({"id": 1, "password": b"not-valid-ciphertext"})
stats = ReEncryptStats()
migrator._re_encrypt_row( # noqa: SLF001

View File

@@ -273,36 +273,6 @@ class TestWebDriverSelenium:
# Should create driver without errors
mock_driver_class.assert_called_once()
@patch("superset.utils.webdriver.app")
def test_driver_sets_page_load_timeout(self, mock_app_patch: MagicMock) -> None:
"""driver.get() must be bounded so it can't block forever (#40047)."""
mock_app_patch.config = {
"SCREENSHOT_LOCATE_WAIT": 10,
"SCREENSHOT_LOAD_WAIT": 10,
"SCREENSHOT_PAGE_LOAD_WAIT": 120,
}
mock_driver = MagicMock()
driver = WebDriverSelenium(driver_type="chrome", window=(800, 600))
with patch.object(driver, "_create", return_value=mock_driver):
assert driver.driver is mock_driver
mock_driver.set_page_load_timeout.assert_called_once_with(120)
@patch("superset.utils.webdriver.app")
def test_driver_skips_page_load_timeout_when_none(
self, mock_app_patch: MagicMock
) -> None:
"""Setting SCREENSHOT_PAGE_LOAD_WAIT to None disables the bound."""
mock_app_patch.config = {
"SCREENSHOT_LOCATE_WAIT": 10,
"SCREENSHOT_LOAD_WAIT": 10,
"SCREENSHOT_PAGE_LOAD_WAIT": None,
}
mock_driver = MagicMock()
driver = WebDriverSelenium(driver_type="chrome", window=(800, 600))
with patch.object(driver, "_create", return_value=mock_driver):
assert driver.driver is mock_driver
mock_driver.set_page_load_timeout.assert_not_called()
class TestPlaywrightAvailabilityCheck:
"""Test comprehensive Playwright availability checking."""