Compare commits

..
Author SHA1 Message Date
EvanandClaude Sonnet 4.6 0052909f10 fix(version): check EXPOSE_VERSION_INFO before calling get_version_metadata
- Skip get_version_metadata() when the flag is False: read VERSION_STRING
  directly from app config so git subprocesses are never invoked on
  unauthenticated /version requests when version details are redacted
- Add missing build_number assertion in the positive-flag test case
- Update the disabled-flag test to match new code path (no longer mocks
  get_version_metadata; sets VERSION_STRING directly in app config)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 02:32:31 -07:00
Claude Code 898b3c498f feat(config): add Cross-Origin-Resource-Policy default header
Adds a conservative `Cross-Origin-Resource-Policy: same-site` default to
DEFAULT_HTTP_HEADERS as a defense-in-depth response-header hardening. The
header is applied through the existing DEFAULT_HTTP_HEADERS mechanism, so it
is only set when a response does not already carry the header and operators
can override it via config.

`same-site` is used rather than the stricter `same-origin` so documented
same-site embedding flows (e.g. the Embedded SDK) keep working unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 02:32:31 -07:00
Claude Code 81a4665813 feat(config): add EXPOSE_VERSION_INFO to control /version detail
The unauthenticated /version endpoint returns the version string along with
the Git SHA, full SHA, build number, and branch name when available. Add an
EXPOSE_VERSION_INFO config option (default True, preserving existing behavior)
that, when set to False, reduces the response to just the version string and
omits the build-specific details.

The gating is applied in the endpoint itself so the change is opt-in and
non-breaking for existing deployments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 02:32:30 -07:00
105 changed files with 680 additions and 4279 deletions
-13
View File
@@ -38,19 +38,6 @@ jobs:
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
# Authenticate the Docker daemon so the python:slim pull in
# uv-pip-compile.sh uses our (much higher) authenticated rate limit
# instead of the shared-runner anonymous one. Best-effort: on fork PRs the
# secrets are unavailable, so this no-ops and the pull falls back to
# anonymous (covered by the retry loop in the script).
- name: Login to Docker Hub
if: steps.check.outputs.python
continue-on-error: true
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ secrets.DOCKERHUB_USER }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Run uv
if: steps.check.outputs.python
run: ./scripts/uv-pip-compile.sh
@@ -12,11 +12,6 @@ on:
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
validate-all-ghas:
-5
View File
@@ -2,11 +2,6 @@ name: "Pull Request Labeler"
on:
- pull_request_target
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
labeler:
permissions:
-5
View File
@@ -8,11 +8,6 @@ on:
# Possible values: https://help.github.com/en/actions/reference/events-that-trigger-workflows#pull-request-event-pull_request
types: [opened, edited, reopened, synchronize]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
lint-check:
runs-on: ubuntu-24.04
+44 -94
View File
@@ -1,16 +1,12 @@
name: E2E
on:
# Gated behind pre-commit: this workflow runs only after the "pre-commit
# checks" workflow completes, and (via the job-level `if` below) only when
# it succeeded. That keeps the expensive Cypress/Playwright runners from
# spinning up while a PR still has formatting/lint/type errors that
# pre-commit catches in a fraction of the time. pre-commit itself runs on
# push (master/release) and pull_request, so this preserves coverage for
# both event types.
workflow_run:
workflows: ["pre-commit checks"]
types: [completed]
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
workflow_dispatch:
inputs:
use_dashboard:
@@ -27,46 +23,11 @@ on:
default: ''
concurrency:
# workflow_run has no PR number in context; key on the originating branch.
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch || github.run_id }}
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
changes:
# The pre-commit gate: only proceed when pre-commit succeeded (or on a
# manual dispatch). On failure this job is skipped, and every downstream
# job (needs: changes) is skipped with it — no runners are provisioned.
if: >-
github.event_name == 'workflow_dispatch' ||
github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
outputs:
python: ${{ steps.check.outputs.python }}
frontend: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_sha || github.sha }}
# The shared change-detector action reads the live event context, which
# under workflow_run points at the default branch. Call the script
# directly instead, passing the originating event/SHA/PR via WF_RUN_*.
- name: Check for file changes
id: check
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WF_RUN_EVENT: ${{ github.event.workflow_run.event }}
WF_RUN_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
WF_RUN_PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
run: python scripts/change_detector.py
cypress-matrix:
needs: changes
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.frontend == 'true'
# Somehow one test flakes on 24.04 for unknown reasons, this is the only GHA left on 22.04
runs-on: ubuntu-22.04
permissions:
@@ -79,14 +40,9 @@ jobs:
# https://github.com/cypress-io/github-action/issues/48
fail-fast: false
matrix:
parallel_id: [0, 1]
parallel_id: [0, 1, 2, 3, 4, 5]
browser: ["chrome"]
app_root: ${{ github.event.workflow_run.event == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
# The /app/prefix variant (push events only) is smoke-tested on a single
# shard rather than the full matrix, so exclude it from the other shards.
exclude:
- parallel_id: 1
app_root: "/app/prefix"
app_root: ${{ github.event_name == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -111,13 +67,13 @@ jobs:
steps:
# -------------------------------------------------------
# Conditional checkout based on context
- name: Checkout (gated by pre-commit via workflow_run)
if: github.event_name == 'workflow_run'
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event.workflow_run.head_sha }}
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -133,38 +89,51 @@ jobs:
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: cypress-install
- name: Run Cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
CYPRESS_BROWSER: ${{ matrix.browser }}
PARALLEL_ID: ${{ matrix.parallel_id }}
PARALLELISM: 2
PARALLELISM: 6
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
NODE_OPTIONS: "--max-old-space-size=4096"
with:
@@ -185,8 +154,6 @@ jobs:
name: cypress-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}-${{ matrix.parallel_id }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}
playwright-tests:
needs: changes
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.frontend == 'true'
runs-on: ubuntu-22.04
permissions:
contents: read
@@ -195,7 +162,7 @@ jobs:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ${{ github.event.workflow_run.event == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
app_root: ${{ github.event_name == 'push' && fromJSON('["", "/app/prefix"]') || fromJSON('[""]') }}
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -218,13 +185,13 @@ jobs:
steps:
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout (gated by pre-commit via workflow_run)
if: github.event_name == 'workflow_run'
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event.workflow_run.head_sha }}
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -240,37 +207,51 @@ jobs:
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright_testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Build embedded SDK
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-embedded-sdk
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Required Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
@@ -292,34 +273,3 @@ jobs:
${{ github.workspace }}/superset-frontend/playwright-results/
${{ github.workspace }}/superset-frontend/test-results/
name: playwright-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}
# workflow_run runs don't attach their checks to the originating PR, so post
# an aggregate commit status back onto the PR head SHA. Make THIS the required
# status check in branch protection (in place of the individual E2E jobs).
report-status:
needs: [cypress-matrix, playwright-tests]
if: always() && github.event_name == 'workflow_run'
runs-on: ubuntu-24.04
permissions:
statuses: write
steps:
- name: Report aggregate E2E status to PR commit
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
// 'skipped' is acceptable: the change-detector legitimately skips
// jobs when no relevant files changed. Only real failures fail.
const results = [
'${{ needs.cypress-matrix.result }}',
'${{ needs.playwright-tests.result }}',
];
const ok = results.every((r) => r === 'success' || r === 'skipped');
await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: context.payload.workflow_run.head_sha,
state: ok ? 'success' : 'failure',
context: 'E2E / required',
description: ok ? 'E2E passed (or skipped)' : 'E2E failed',
target_url: `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`,
});
+15 -21
View File
@@ -23,30 +23,9 @@ concurrency:
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
outputs:
python: ${{ steps.check.outputs.python }}
frontend: ${{ steps.check.outputs.frontend }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
# NOTE: Required Playwright tests are in superset-e2e.yml (E2E / playwright-tests)
# This workflow contains only experimental tests that run in shadow mode
playwright-tests-experimental:
needs: changes
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.frontend == 'true'
runs-on: ubuntu-22.04
continue-on-error: true
permissions:
@@ -101,43 +80,58 @@ jobs:
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright_testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Build embedded SDK
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-embedded-sdk
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Experimental Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
with:
run: playwright-run "${{ matrix.app_root }}" experimental/
- name: Run Playwright (Embedded Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
@@ -14,27 +14,7 @@ concurrency:
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
outputs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
test-mysql:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -47,8 +27,6 @@ jobs:
services:
mysql:
image: mysql:8.0
# Authenticated pulls use our higher Docker Hub rate limit. Empty on
# fork PRs (secrets unavailable) -> runner falls back to anonymous.
env:
MYSQL_ROOT_PASSWORD: root
ports:
@@ -69,17 +47,26 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup MySQL
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: setup-mysql
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python integration tests (MySQL)
if: steps.check.outputs.python
run: |
./scripts/python_tests.sh
- name: Upload code coverage
@@ -90,6 +77,7 @@ jobs:
use_oidc: true
slug: apache/superset
- name: Generate database diagnostics for docs
if: steps.check.outputs.python
env:
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: |
@@ -112,14 +100,13 @@ jobs:
print(f'Generated diagnostics for {len(docs)} databases')
"
- name: Upload database diagnostics artifact
if: steps.check.outputs.python
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: database-diagnostics
path: databases-diagnostics.json
retention-days: 7
test-postgres:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -151,20 +138,29 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
with:
python-version: ${{ matrix.python-version }}
- name: Setup Postgres
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: |
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python integration tests (PostgreSQL)
if: steps.check.outputs.python
run: |
./scripts/python_tests.sh
- name: Upload code coverage
@@ -176,8 +172,6 @@ jobs:
slug: apache/superset
test-sqlite:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -200,19 +194,28 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Install dependencies
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: |
# sqlite needs this working directory
mkdir ${{ github.workspace }}/.temp
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python integration tests (SQLite)
if: steps.check.outputs.python
run: |
./scripts/python_tests.sh
- name: Upload code coverage
@@ -15,27 +15,7 @@ concurrency:
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
outputs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
test-postgres-presto:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -74,17 +54,28 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python == 'true'
- name: Setup Postgres
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
run: |
echo "${{ steps.check.outputs.python }}"
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
run: |
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
- name: Upload code coverage
@@ -96,8 +87,6 @@ jobs:
slug: apache/superset
test-postgres-hive:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -128,23 +117,35 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Create csv upload directory
if: steps.check.outputs.python
run: sudo mkdir -p /tmp/.superset/uploads
- name: Give write access to the csv upload directory
if: steps.check.outputs.python
run: sudo chown -R $USER:$USER /tmp/.superset
- name: Start hadoop and hive
if: steps.check.outputs.python
run: docker compose -f scripts/databases/hive/docker-compose.yml up -d
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup Postgres
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
run: |
pip install -e .[hive]
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
+8 -20
View File
@@ -15,27 +15,7 @@ concurrency:
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: read
outputs:
python: ${{ steps.check.outputs.python }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
unit-tests:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
@@ -50,17 +30,25 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
with:
python-version: ${{ matrix.python-version }}
- name: Python unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear --maxfail=50
- name: Python 100% coverage unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
+19
View File
@@ -34,6 +34,25 @@ The embedded dashboard page now validates the origin of incoming `postMessage` e
Enforcement only applies when the Allowed Domains list is non-empty. If the list is empty (the default), any origin is accepted, so there is no behavior change for embeds that did not configure Allowed Domains.
### New `EXPOSE_VERSION_INFO` config to control `/version` detail
A new `EXPOSE_VERSION_INFO` config option controls how much detail the unauthenticated `/version` endpoint returns. It defaults to `True`, which preserves the existing behavior: the endpoint returns the full version metadata, including the Git SHA, full SHA, build number, and branch name when available.
Operators who prefer not to expose build-specific details to unauthenticated callers can set the following in `superset_config.py`:
```python
EXPOSE_VERSION_INFO = False
```
When disabled, `/version` returns only the human-readable `version_string` and omits the Git SHA, full SHA, build number, and branch name. Because the default is `True`, this change is non-breaking for existing deployments.
As an additional defense-in-depth hardening, Superset now sends a `Cross-Origin-Resource-Policy: same-site` response header by default via `DEFAULT_HTTP_HEADERS`. `same-site` is deliberately chosen over the stricter `same-origin` so that documented same-site embedding flows (e.g. the Embedded SDK, where a Superset subdomain is framed by a sibling application subdomain) continue to work unchanged. Deployments that serve Superset responses or static assets as subresources to a _cross-site_ origin may need to relax or remove this header. Because it is applied through `DEFAULT_HTTP_HEADERS`, the header is only set when a response does not already carry one, so it can be overridden per-response or by replacing the config value:
```python
# Relax to permit cross-site consumers, or set to "same-origin" to harden further.
DEFAULT_HTTP_HEADERS = {"Cross-Origin-Resource-Policy": "cross-origin"}
```
### Dataset import validates catalog against the target connection
Importing a dataset now validates the `catalog` field against the target database connection. When the connection has multi-catalog disabled (`allow_multi_catalog` off) and the dataset's catalog is not the connection's default catalog, the import fails instead of silently persisting the non-default catalog. This matches the validation already enforced on the dataset update path and prevents imported datasets from querying an unintended database.
+1 -17
View File
@@ -80,23 +80,7 @@ case "${1}" in
;;
app)
echo "Starting web app (using development server)..."
# Environment-based debugger control for security
# Only enable Werkzeug interactive debugger when explicitly requested
# Modern Werkzeug (3.0+) includes PIN protection, but defense-in-depth approach
# Override FLASK_DEBUG so the effective state matches SUPERSET_DEBUG_ENABLED even
# when FLASK_DEBUG=true is inherited from docker/.env or .flaskenv
if [[ "${SUPERSET_DEBUG_ENABLED:-}" == "true" ]]; then
export FLASK_DEBUG=1
DEBUGGER_FLAG="--debugger"
echo " ⚠️ Werkzeug debugger enabled (requires PIN for /console access)"
else
export FLASK_DEBUG=0
DEBUGGER_FLAG="--no-debugger"
echo " 🔒 Werkzeug debugger disabled (set SUPERSET_DEBUG_ENABLED=true to enable)"
fi
flask run -p $PORT --reload $DEBUGGER_FLAG --host=0.0.0.0 --exclude-patterns "*/node_modules/*:*/.venv/*:*/build/*:*/__pycache__/*:*/superset-frontend/*"
flask run -p $PORT --reload --debugger --host=0.0.0.0 --exclude-patterns "*/node_modules/*:*/.venv/*:*/build/*:*/__pycache__/*:*/superset-frontend/*"
;;
app-gunicorn)
echo "Starting web app..."
+1 -8
View File
@@ -157,15 +157,8 @@ superset load_examples
superset init
# To start a development web server on port 8088, use -p to bind to another port
superset run -p 8088 --with-threads --reload
# For debugging with interactive console (⚠️ localhost only)
# superset run -p 8088 --with-threads --reload --debugger
superset run -p 8088 --with-threads --reload --debugger
```
:::warning Security Note
The `--debugger` flag enables Werkzeug's interactive console at `/console`. Only use this for local development and never bind to `0.0.0.0` or expose the server to networks when debugging is enabled.
:::
If everything worked, you should be able to navigate to `hostname:port` in your browser (e.g.
locally by default at `localhost:8088`) and login using the username and password you created.
@@ -157,15 +157,8 @@ superset load_examples
superset init
# To start a development web server on port 8088, use -p to bind to another port
superset run -p 8088 --with-threads --reload
# For debugging with interactive console (⚠️ localhost only)
# superset run -p 8088 --with-threads --reload --debugger
superset run -p 8088 --with-threads --reload --debugger
```
:::warning Security Note
The `--debugger` flag enables Werkzeug's interactive console at `/console`. Only use this for local development and never bind to `0.0.0.0` or expose the server to networks when debugging is enabled.
:::
If everything worked, you should be able to navigate to `hostname:port` in your browser (e.g.
locally by default at `localhost:8088`) and login using the username and password you created.
@@ -102,8 +102,6 @@ Affecting the Docker build process:
save some precious time on startup by `SUPERSET_LOAD_EXAMPLES=no docker compose up`
- **SUPERSET_LOG_LEVEL (default=info)**: Can be set to debug, info, warning, error, critical
for more verbose logging
- **SUPERSET_DEBUG_ENABLED (default=false)**: Enable Werkzeug debugger with interactive console.
Set to `true` for debugging: `SUPERSET_DEBUG_ENABLED=true docker compose up`
For more env vars that affect your configuration, see this
[superset_config.py](https://github.com/apache/superset/blob/master/docker/pythonpath_dev/superset_config.py)
+1 -2
View File
@@ -154,7 +154,7 @@ fastmcp = [
]
firebird = ["sqlalchemy-firebird>=0.7.0, <2.2"]
firebolt = ["firebolt-sqlalchemy>=1.0.0, <2"]
gevent = ["gevent>=26.4.0"]
gevent = ["gevent>=23.9.1"]
gsheets = ["shillelagh[gsheetsapi]>=1.4.4, <2"]
hana = ["hdbcli==2.28.20", "sqlalchemy_hana==0.4.0"]
hive = [
@@ -456,7 +456,6 @@ authorized_licenses = [
"isc license (iscl)",
"isc license",
"mit",
"mit and psf-2.0",
"mit-cmu",
"mozilla public license 2.0 (mpl 2.0)",
"osi approved",
+1 -1
View File
@@ -161,7 +161,7 @@ geopy==2.4.1
# via apache-superset (pyproject.toml)
google-auth==2.43.0
# via shillelagh
greenlet==3.5.0
greenlet==3.1.1
# via
# apache-superset (pyproject.toml)
# shillelagh
+2 -2
View File
@@ -331,7 +331,7 @@ geopy==2.4.1
# via
# -c requirements/base-constraint.txt
# apache-superset
gevent==26.4.0
gevent==24.2.1
# via apache-superset
google-api-core==2.23.0
# via
@@ -373,7 +373,7 @@ googleapis-common-protos==1.66.0
# via
# google-api-core
# grpcio-status
greenlet==3.5.0
greenlet==3.1.1
# via
# -c requirements/base-constraint.txt
# apache-superset
-16
View File
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-34
View File
@@ -109,37 +109,6 @@ def is_int(s: str) -> bool:
return bool(re.match(r"^-?\d+$", s))
def resolve_workflow_run_files(repo: str, sha: str) -> Optional[List[str]]:
"""Resolve changed files for a workflow_run-triggered run.
When a workflow is gated behind another (e.g. running only after
pre-commit succeeds), GitHub re-dispatches it as a `workflow_run` event
whose context points at the default branch rather than the originating
diff. Recover the original event and head SHA from the workflow_run
payload, exposed via the WF_RUN_* env vars. Returns ``None`` (meaning
"assume all changed") when the diff can't be resolved.
"""
original_event = os.getenv("WF_RUN_EVENT") or "push"
print("ORIGINAL_EVENT", original_event)
if original_event == "pull_request":
pr_number = os.getenv("WF_RUN_PR_NUMBER", "")
if not is_int(pr_number):
# Fork PRs don't populate workflow_run.pull_requests, so we can't
# resolve the diff -> assume all changed (run everything).
print("workflow_run without PR context, assuming all changed")
return None
files = fetch_changed_files_pr(repo, pr_number)
print("PR files:")
print_files(files)
return files
head_sha = os.getenv("WF_RUN_HEAD_SHA") or sha
files = fetch_changed_files_push(repo, head_sha)
print("Files touched since previous commit:")
print_files(files)
return files
def main(event_type: str, sha: str, repo: str) -> None:
"""Main function to check for file changes based on event context."""
print("SHA:", sha)
@@ -157,9 +126,6 @@ def main(event_type: str, sha: str, repo: str) -> None:
print("Files touched since previous commit:")
print_files(files)
elif event_type == "workflow_run":
files = resolve_workflow_run_files(repo, sha)
elif event_type in ("workflow_dispatch", "schedule"):
# Manual or cron-triggered runs aren't tied to a specific diff, so
# treat every group as changed. `files = None` makes the loop below
@@ -18,31 +18,14 @@
"""
Check that source-code changes don't cause translation regressions.
What counts as a regression
---------------------------
A regression is an *existing translation that a source change invalidated* —
i.e. a string was renamed/reworded so its committed translation no longer
applies. ``babel_update.sh`` (``pybabel update --ignore-obsolete``) surfaces
exactly these as **newly fuzzy** entries: the old translation is fuzzy-matched
onto the new ``msgid`` and flagged ``#, fuzzy``.
Crucially, *deleting* a translatable string is **not** a regression. With
``--ignore-obsolete`` a removed string is dropped from the catalogs entirely;
no fuzzy entry is created. So a PR that intentionally removes a string (e.g. a
security fix that stops rendering a value) legitimately lowers the translated
count without introducing any fuzzies, and must not be flagged. We therefore
key the check on the **increase in fuzzy entries**, not on a drop in the
translated count (a drop happens identically for a benign deletion and a real
rename, so it cannot distinguish the two).
Usage
-----
Count translated + fuzzy entries in all .po files and write JSON to stdout:
Count non-fuzzy translated entries in all .po files and write JSON to stdout:
python check_translation_regression.py --count
Compare the current .po state against a previously-recorded baseline and fail
if a source change invalidated existing translations (new fuzzies):
if any language lost translations:
python check_translation_regression.py --compare /path/to/before.json
@@ -67,8 +50,8 @@ Typical CI workflow
Running babel_update on the base branch first isolates regressions caused by
the PR's source diff from any pre-existing drift on the base branch, while the
PR worktree run still allows committed .po updates to resolve the fuzzies (and
thus clear the regression) before merging.
PR worktree run still allows committed .po updates to restore lost
translations.
"""
import argparse
@@ -88,13 +71,8 @@ DEFAULT_TRANSLATIONS_DIR = (
SKIP_LANGS = {"en"}
def count_stats(po_file: Path) -> dict[str, int]:
"""Return ``{"translated": int, "fuzzy": int}`` for a .po file.
``translated`` is the number of non-fuzzy translated messages; ``fuzzy`` is
the number of fuzzy translations. The fuzzy count is what the regression
check keys on — a source rename invalidates an existing translation by
making it fuzzy, whereas a deletion simply drops it (``--ignore-obsolete``).
def count_translated(po_file: Path) -> int:
"""Return the number of non-fuzzy translated messages in a .po file.
Raises:
subprocess.CalledProcessError: if ``msgfmt`` fails (e.g. malformed
@@ -112,50 +90,29 @@ def count_stats(po_file: Path) -> dict[str, int]:
check=True,
)
# stderr: "123 translated messages, 4 fuzzy translations, 56 untranslated messages."
# The fuzzy and untranslated clauses are omitted by msgfmt when they are 0.
translated_match = re.search(r"(\d+) translated message", result.stderr)
if not translated_match:
match = re.search(r"(\d+) translated message", result.stderr)
if not match:
raise RuntimeError(
f"Could not parse msgfmt --statistics output for {po_file}: "
f"{result.stderr!r}"
)
fuzzy_match = re.search(r"(\d+) fuzzy translation", result.stderr)
return {
"translated": int(translated_match.group(1)),
"fuzzy": int(fuzzy_match.group(1)) if fuzzy_match else 0,
}
return int(match.group(1))
def get_counts(
translations_dir: Path,
failures: Optional[set[str]] = None,
) -> dict[str, dict[str, int]]:
"""Count translated/fuzzy entries for every ``.po`` file in a directory.
If ``failures`` is provided, the name of each language whose ``.po`` file
is present on disk but could not be counted (msgfmt non-zero exit, or
unparseable output) is added to it. Such a language is deliberately absent
from the returned mapping — but, unlike a language whose catalog was simply
deleted, it must not be mistaken for an intentional removal: a caller that
cares about the distinction (see :func:`cmd_compare`) can inspect
``failures`` and treat it as a hard error.
"""
counts: dict[str, dict[str, int]] = {}
def get_counts(translations_dir: Path) -> dict[str, int]:
counts: dict[str, int] = {}
for po_file in sorted(translations_dir.glob("*/LC_MESSAGES/messages.po")):
lang = po_file.parent.parent.name
if lang in SKIP_LANGS:
continue
try:
counts[lang] = count_stats(po_file)
counts[lang] = count_translated(po_file)
except (subprocess.CalledProcessError, RuntimeError) as exc:
# A malformed .po file (msgfmt non-zero exit, or stderr we
# can't parse) is a real problem worth seeing, but it shouldn't
# take the whole regression check down with it — that would
# hide every other language's status. Skip and warn here; the
# caller is told which langs failed via ``failures`` so it can
# decide whether a present-but-uncountable catalog is fatal.
if failures is not None:
failures.add(lang)
# hide every other language's status. Skip and warn instead;
# the missing lang will not appear in the comparison output.
print(
f"WARNING: skipping {lang}{po_file} could not be counted: {exc}",
file=sys.stderr,
@@ -163,42 +120,18 @@ def get_counts(
return counts
def _normalize(entry: object) -> dict[str, int]:
"""Coerce a baseline entry into ``{"translated", "fuzzy"}``.
Tolerates the legacy baseline format where each language mapped directly to
an integer translated count (no fuzzy data); such entries contribute a
fuzzy baseline of 0.
"""
if isinstance(entry, dict):
return {
"translated": int(entry.get("translated", 0)),
"fuzzy": int(entry.get("fuzzy", 0)),
}
if isinstance(entry, int):
return {"translated": entry, "fuzzy": 0}
raise TypeError(f"Unsupported baseline entry: {entry!r}")
def build_regression_report(regressions: list[tuple[str, int, int]]) -> str:
"""Build a markdown report for posting as a PR comment.
Each regression tuple is ``(lang, before_fuzzy, after_fuzzy)``.
"""
"""Build a markdown report for posting as a PR comment."""
rows = "\n".join(
f"| `{lang}` | {b} | {a} | +{a - b} |" for lang, b, a in regressions
f"| `{lang}` | {b} | {a} | -{b - a} |" for lang, b, a in regressions
)
affected = ", ".join(f"`{lang}`" for lang, _, _ in regressions)
return (
"## ⚠️ Translation Regression Detected\n\n"
f"A source change in this PR renamed or reworded strings, invalidating "
f"existing translations (they are now `#, fuzzy`) in {affected}. Please "
f"resolve the affected `.po` files before merging.\n\n"
"_Note: intentionally **deleting** a translatable string is not a "
"regression and is not flagged here — only translations invalidated by "
"a renamed/reworded source string are._\n\n"
"| Language | Fuzzy before | Fuzzy after | New |\n"
"|----------|-------------:|------------:|----:|\n"
f"This PR causes existing translations to become fuzzy or be removed "
f"in {affected}. Please fix the affected `.po` files before merging.\n\n"
"| Language | Before | After | Lost |\n"
"|----------|-------:|------:|-----:|\n"
f"{rows}\n\n"
"### How to fix\n\n"
"**1. Install dependencies** (if not already set up):\n\n"
@@ -236,49 +169,26 @@ def cmd_compare(
report_path: Optional[str] = None,
) -> None:
with open(before_path) as f:
before_raw: dict[str, object] = json.load(f)
before = {lang: _normalize(entry) for lang, entry in before_raw.items()}
before: dict[str, int] = json.load(f)
failures: set[str] = set()
after = get_counts(translations_dir, failures=failures)
after = get_counts(translations_dir)
# A baseline language whose catalog is *missing* from `after` is fine —
# that's an intentional catalog deletion (handled below like any other
# deletion). But a language whose .po file is still present yet could not
# be counted (msgfmt failed / output unparseable) is a hard error: leaving
# it out silently would let a corrupt catalog pass as "no regression".
broken = sorted(lang for lang in failures if lang in before)
if broken:
print("Translation check failed!\n")
for lang in broken:
print(f" {lang}: catalog present but could not be counted (msgfmt error)")
print(
"\nFix the malformed .po file(s) above before merging — a catalog "
"that cannot be parsed must not be silently dropped."
)
sys.exit(1)
# A regression is an *increase* in fuzzy entries: the PR's source diff
# renamed/reworded strings, leaving their committed translations stranded.
# A plain drop in the translated count is NOT used — deleting a string
# lowers it identically to a rename but is a legitimate change, and with
# `pybabel update --ignore-obsolete` a deletion creates no fuzzy entry.
regressions: list[tuple[str, int, int]] = []
for lang, before_stats in sorted(before.items()):
after_stats = after.get(lang, {"translated": 0, "fuzzy": 0})
if after_stats["fuzzy"] > before_stats["fuzzy"]:
regressions.append((lang, before_stats["fuzzy"], after_stats["fuzzy"]))
for lang, before_count in sorted(before.items()):
after_count = after.get(lang, 0)
if after_count < before_count:
regressions.append((lang, before_count, after_count))
if regressions:
print("Translation regression detected!\n")
for lang, b, a in regressions:
print(
f" {lang}: {a - b} translation(s) invalidated "
f"(fuzzy {b} -> {a}) by a renamed/reworded source string"
)
lost = b - a
print(f" {lang}: {b} -> {a} (-{lost} string(s) became fuzzy or removed)")
print(
"\nResolve the newly-fuzzy entries in the affected .po files "
"before merging."
"\nStrings renamed or deleted by this PR invalidated existing translations."
)
print(
"Update the affected .po files to restore the lost entries before merging."
)
if report_path:
Path(report_path).write_text(
@@ -289,15 +199,15 @@ def cmd_compare(
# All good — print a summary so it's easy to read in CI logs.
print("No translation regressions.\n")
for lang in sorted(after):
before_stats = before.get(lang, {"translated": 0, "fuzzy": 0})
after_stats = after[lang]
t_delta = after_stats["translated"] - before_stats["translated"]
f_delta = after_stats["fuzzy"] - before_stats["fuzzy"]
print(
f" {lang}: translated {before_stats['translated']} -> "
f"{after_stats['translated']} ({t_delta:+d}), fuzzy "
f"{before_stats['fuzzy']} -> {after_stats['fuzzy']} ({f_delta:+d})"
)
b = before.get(lang, 0)
a = after[lang]
if a > b:
delta = f"+{a - b}"
elif a == b:
delta = "no change"
else:
delta = f"-{b - a}"
print(f" {lang}: {b} -> {a} ({delta})")
def main() -> None:
+1 -22
View File
@@ -31,32 +31,11 @@ if [ -z "$RUNNING_IN_DOCKER" ]; then
echo "Running in Docker (Python ${PYTHON_VERSION} on Linux)..."
IMAGE="python:${PYTHON_VERSION}-slim"
# Pre-pull the image with a few retries to absorb transient Docker Hub
# registry failures ("context deadline exceeded" / anonymous rate-limit blips
# on shared CI runners). Without this a flaky pull fails the whole
# check-python-deps job on an infrastructure hiccup rather than a real
# dependency drift. The pull is in the `until` condition so `set -e` does not
# abort on an individual failed attempt.
attempt=1
max_attempts=4
until docker pull "$IMAGE"; do
if [ "$attempt" -ge "$max_attempts" ]; then
echo "docker pull $IMAGE failed after ${max_attempts} attempts" >&2
exit 1
fi
delay=$((attempt * 10))
echo "docker pull $IMAGE failed (attempt ${attempt}/${max_attempts}); retrying in ${delay}s..." >&2
sleep "$delay"
attempt=$((attempt + 1))
done
docker run --rm \
-v "$(pwd)":/app \
-w /app \
-e RUNNING_IN_DOCKER=1 \
"$IMAGE" \
python:${PYTHON_VERSION}-slim \
bash -c "pip install uv && ./scripts/uv-pip-compile.sh $*"
exit $?
+1 -1
View File
@@ -80,7 +80,7 @@ const restrictedImportsRules = {
'no-jest-mock-console': {
name: 'jest-mock-console',
message: 'Please use native Jest spies, i.e. jest.spyOn(console, "warn")',
},
}
};
module.exports = {
@@ -19,9 +19,7 @@
import { getTimeFormatter } from '@superset-ui/core';
// Cal-Heatmap provides local timestamps (UTC shifted by the browser's timezone
// offset). We subtract that offset so the formatter displays the correct UTC
// date regardless of the browser's timezone.
// Cal-Heatmap provides local timestamps. We subtract the offset so that utcFormat displays the correct local date.
export const getFormattedUTCTime = (
ts: number | string,
timeFormat?: string,
@@ -299,23 +299,18 @@ var CalHeatMap = function () {
// Takes the fetched "data" object as argument, must return a json object
// formatted like {timestamp:count, timestamp2:count2},
afterLoadData: function (timestamps) {
// Use the DST-aware timezone offset for each individual timestamp so that
// every data point is shifted by its own local offset (not a fixed
// standard-time offset). This prevents data from landing in phantom hours
// during DST transitions and keeps the offset consistent with what
// getFormattedUTCTime undoes when formatting the tooltip.
//
// Around DST transitions two distinct UTC timestamps can shift to the
// same adjusted key (e.g. the "spring forward" hour that doesn't exist
// locally). Accumulate values on collision so no datapoints are silently
// dropped in hourly/minutely views.
// See https://github.com/wa0x6e/cal-heatmap/issues/126#issuecomment-373301803
const stdTimezoneOffset = date => {
const jan = new Date(date.getFullYear(), 0, 1);
const jul = new Date(date.getFullYear(), 6, 1);
return Math.max(jan.getTimezoneOffset(), jul.getTimezoneOffset());
};
const offset = stdTimezoneOffset(new Date()) * 60;
let results = {};
for (let timestamp in timestamps) {
const value = timestamps[timestamp];
const ts = parseInt(timestamp, 10);
const offset = new Date(ts * 1000).getTimezoneOffset() * 60;
const adjustedTs = ts + offset;
results[adjustedTs] = (results[adjustedTs] || 0) + value;
timestamp = parseInt(timestamp, 10);
results[timestamp + offset] = value;
}
return results;
},
@@ -4010,10 +4005,6 @@ function mergeRecursive(obj1, obj2) {
/*jshint forin:false */
for (var p in obj2) {
// Skip keys that could pollute the object prototype.
if (p === '__proto__' || p === 'constructor' || p === 'prototype') {
continue;
}
try {
// Property in destination object set; update its value.
if (obj2[p].constructor === Object) {
@@ -19,71 +19,78 @@
import { getFormattedUTCTime, convertUTCTimestampToLocal } from '../src/utils';
test('getFormattedUTCTime formats local timestamp for display as UTC date', () => {
const utcTimestamp = 1420070400000; // 2015-01-01 00:00:00 UTC
const localTimestamp = convertUTCTimestampToLocal(utcTimestamp);
// Cal-Heatmap's afterLoadData adjusts timestamps similarly, so
// getFormattedUTCTime receives already-adjusted timestamps and
// formats them directly. The date component should be correct.
const formattedTime = getFormattedUTCTime(localTimestamp, '%Y-%m-%d');
describe('getFormattedUTCTime', () => {
test('formats local timestamp for display as UTC date', () => {
const utcTimestamp = 1420070400000; // 2015-01-01 00:00:00 UTC
const localTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const formattedTime = getFormattedUTCTime(
localTimestamp,
'%Y-%m-%d %H:%M:%S',
);
expect(formattedTime).toEqual('2015-01-01');
expect(formattedTime).toEqual('2015-01-01 00:00:00');
});
});
test('convertUTCTimestampToLocal adjusts timestamp so local Date shows UTC date', () => {
const utcTimestamp = 1704067200000;
const adjustedTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const adjustedDate = new Date(adjustedTimestamp);
describe('convertUTCTimestampToLocal', () => {
test('adjusts timestamp so local Date shows UTC date', () => {
const utcTimestamp = 1704067200000;
const adjustedTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const adjustedDate = new Date(adjustedTimestamp);
expect(adjustedDate.getFullYear()).toEqual(2024);
expect(adjustedDate.getMonth()).toEqual(0);
expect(adjustedDate.getDate()).toEqual(1);
expect(adjustedDate.getFullYear()).toEqual(2024);
expect(adjustedDate.getMonth()).toEqual(0);
expect(adjustedDate.getDate()).toEqual(1);
});
test('handles month boundaries', () => {
const utcTimestamp = 1706745600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(utcTimestamp));
expect(adjustedDate.getFullYear()).toEqual(2024);
expect(adjustedDate.getMonth()).toEqual(1);
expect(adjustedDate.getDate()).toEqual(1);
});
test('handles year boundaries', () => {
const utcTimestamp = 1735689600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(utcTimestamp));
expect(adjustedDate.getFullYear()).toEqual(2025);
expect(adjustedDate.getMonth()).toEqual(0);
expect(adjustedDate.getDate()).toEqual(1);
});
test('adds timezone offset to timestamp', () => {
const utcTimestamp = 1704067200000;
const adjustedTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const expectedOffset =
new Date(utcTimestamp).getTimezoneOffset() * 60 * 1000;
expect(adjustedTimestamp - utcTimestamp).toEqual(expectedOffset);
});
});
test('convertUTCTimestampToLocal handles month boundaries', () => {
const utcTimestamp = 1706745600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(utcTimestamp));
describe('integration', () => {
test('fixes timezone bug for CalHeatMap', () => {
const febFirst2024UTC = 1706745600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(febFirst2024UTC));
expect(adjustedDate.getFullYear()).toEqual(2024);
expect(adjustedDate.getMonth()).toEqual(1);
expect(adjustedDate.getDate()).toEqual(1);
});
test('convertUTCTimestampToLocal handles year boundaries', () => {
const utcTimestamp = 1735689600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(utcTimestamp));
expect(adjustedDate.getFullYear()).toEqual(2025);
expect(adjustedDate.getMonth()).toEqual(0);
expect(adjustedDate.getDate()).toEqual(1);
});
test('convertUTCTimestampToLocal adds timezone offset to timestamp', () => {
const utcTimestamp = 1704067200000;
const adjustedTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const expectedOffset = new Date(utcTimestamp).getTimezoneOffset() * 60 * 1000;
expect(adjustedTimestamp - utcTimestamp).toEqual(expectedOffset);
});
test('convertUTCTimestampToLocal fixes timezone bug for CalHeatMap', () => {
const febFirst2024UTC = 1706745600000;
const adjustedDate = new Date(convertUTCTimestampToLocal(febFirst2024UTC));
expect(adjustedDate.getMonth()).toEqual(1);
expect(adjustedDate.getDate()).toEqual(1);
});
test('convertUTCTimestampToLocal and getFormattedUTCTime work together to display dates correctly', () => {
const utcTimestamp = 1704067200000;
// convertUTCTimestampToLocal adjusts UTC for Cal-Heatmap (which interprets as local)
const localTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const calHeatmapDate = new Date(localTimestamp);
expect(calHeatmapDate.getMonth()).toEqual(0);
expect(calHeatmapDate.getDate()).toEqual(1);
// getFormattedUTCTime receives LOCAL timestamp (from Cal-Heatmap) and formats it
const formattedTime = getFormattedUTCTime(localTimestamp, '%Y-%m-%d');
expect(formattedTime).toContain('2024-01-01');
expect(adjustedDate.getMonth()).toEqual(1);
expect(adjustedDate.getDate()).toEqual(1);
});
test('both functions work together to display dates correctly', () => {
const utcTimestamp = 1704067200000;
// convertUTCTimestampToLocal adjusts UTC for Cal-Heatmap (which interprets as local)
const localTimestamp = convertUTCTimestampToLocal(utcTimestamp);
const calHeatmapDate = new Date(localTimestamp);
expect(calHeatmapDate.getMonth()).toEqual(0);
expect(calHeatmapDate.getDate()).toEqual(1);
// getFormattedUTCTime receives LOCAL timestamp (from Cal-Heatmap) and formats it
const formattedTime = getFormattedUTCTime(localTimestamp, '%Y-%m-%d');
expect(formattedTime).toContain('2024-01-01');
});
});
@@ -275,29 +275,29 @@ export function wrapTooltip(chart) {
});
}
// Builds the sanitized HTML for an annotation layer's tooltip. Title and
// description values come from the annotation data source, so the output is
// run through dompurify before being inserted into the DOM by d3-tip.
export function generateAnnotationTooltipContent(layer, d) {
const title =
d[layer.titleColumn] && d[layer.titleColumn].length > 0
? `${d[layer.titleColumn]} - ${layer.name}`
: layer.name;
const body = Array.isArray(layer.descriptionColumns)
? layer.descriptionColumns.map(c => d[c])
: Object.values(d);
return dompurify.sanitize(
`<div><strong>${title}</strong></div><br/><div>${body.join(', ')}</div>`,
);
}
export function tipFactory(layer) {
return d3tip()
.attr('class', `d3-tip ${layer.annotationTipClass || ''}`)
.direction('n')
.offset([-5, 0])
.html(d => (d ? generateAnnotationTooltipContent(layer, d) : ''));
.html(d => {
if (!d) {
return '';
}
const rawTitle =
d[layer.titleColumn] && d[layer.titleColumn].length > 0
? `${d[layer.titleColumn]} - ${layer.name}`
: layer.name;
const rawBody = Array.isArray(layer.descriptionColumns)
? layer.descriptionColumns.map(c => d[c])
: Object.values(d);
return dompurify.sanitize(
`<div><strong>${rawTitle}</strong></div><br/><div>${rawBody.join(
', ',
)}</div>`,
);
});
}
export function getMaxLabelSize(svg, axisClass) {
@@ -24,7 +24,6 @@ import {
import {
computeYDomain,
generateAnnotationTooltipContent,
generateBubbleTooltipContent,
generateMultiLineTooltipContent,
getTimeOrNumberFormatter,
@@ -126,42 +125,6 @@ describe('nvd3/utils', () => {
);
});
describe('generateMultiLineTooltipContent()', () => {
const identity = (value: any) => value;
test('renders the series key in the tooltip markup', () => {
const tooltip = generateMultiLineTooltipContent(
{
value: 'x-value',
series: [{ key: 'Region A', color: '#fff', value: 1 }],
},
identity,
[identity],
);
expect(tooltip).toContain('Region A');
});
test('strips a script payload from a malicious series key', () => {
const tooltip = generateMultiLineTooltipContent(
{
value: 'x-value',
series: [
{
key: '<img src=x onerror="alert(1)">',
color: '#fff',
value: 1,
},
],
},
identity,
[identity],
);
// DOMPurify removes the event handler that would execute on render.
expect(tooltip).not.toContain('onerror');
expect(tooltip).not.toContain('alert(1)');
});
});
describe('getTimeOrNumberFormatter(format)', () => {
test('is a function', () => {
expect(typeof getTimeOrNumberFormatter).toBe('function');
@@ -313,46 +276,4 @@ describe('nvd3/utils', () => {
expect(html).toContain('payload');
});
});
describe('generateAnnotationTooltipContent()', () => {
const layer = {
name: 'My annotations',
titleColumn: 'title',
descriptionColumns: ['description'],
};
test('renders the annotation title and description', () => {
const html = generateAnnotationTooltipContent(layer, {
title: 'Release',
description: 'Shipped v1',
});
expect(html).toContain('Release - My annotations');
expect(html).toContain('Shipped v1');
});
test('falls back to the layer name when the title column is empty', () => {
const html = generateAnnotationTooltipContent(layer, {
title: '',
description: 'Shipped v1',
});
expect(html).toContain('My annotations');
});
test('strips an event-handler payload from the title column', () => {
const html = generateAnnotationTooltipContent(layer, {
title: '<img src=x onerror="alert(1)">',
description: 'ok',
});
expect(html).not.toContain('onerror');
expect(html).not.toContain('alert(1)');
});
test('strips a script payload from a description column', () => {
const html = generateAnnotationTooltipContent(layer, {
title: 'Release',
description: '<script>alert(document.cookie)</script>',
});
expect(html).not.toContain('<script>');
});
});
});
@@ -288,7 +288,9 @@ describe('BigNumberWithTrendline transformProps', () => {
height: 300,
queriesData: [
{
data: [{ __timestamp: 1, value: 100 }] as unknown as BigNumberDatum[],
data: [
{ __timestamp: 1, value: 100 },
] as unknown as BigNumberDatum[],
colnames: ['__timestamp', 'value'],
coltypes: ['TEMPORAL', 'NUMERIC'],
},
@@ -284,11 +284,8 @@ function Echart(
// setOption(notMerge:true) replaces the dataZoom config, dropping any
// range the user has engaged. Preserve it across the call.
const previousZoom = notMerge
? (
chartRef.current?.getOption() as {
dataZoom?: DataZoomComponentOption[];
}
)?.dataZoom
? (chartRef.current?.getOption() as { dataZoom?: DataZoomComponentOption[] })
?.dataZoom
: undefined;
chartRef.current?.setOption(themedEchartOptions, {
notMerge,
@@ -188,9 +188,7 @@ function CollectionControl({
// Two items can collide when keyAccessor returns falsy and the index
// fallback is used — breaking dnd-kit reordering and React reconciliation.
// Assign a stable nanoid per item ref when no key is available.
const generatedIdsRef = useRef<WeakMap<CollectionItem, string>>(
new WeakMap(),
);
const generatedIdsRef = useRef<WeakMap<CollectionItem, string>>(new WeakMap());
const itemIds = useMemo(
() =>
value.map(item => {
@@ -287,14 +287,14 @@ function UsersList({ user }: UsersListProps) {
id: 'login_count',
Header: t('Login count'),
hidden: true,
Cell: ({ row: { original } }: any) => original.login_count ?? 0,
Cell: ({ row: { original } }: any) => original.login_count,
},
{
accessor: 'fail_login_count',
id: 'fail_login_count',
Header: t('Fail login count'),
hidden: true,
Cell: ({ row: { original } }: any) => original.fail_login_count ?? 0,
Cell: ({ row: { original } }: any) => original.fail_login_count,
},
{
accessor: 'created_on',
+4 -7
View File
@@ -58,7 +58,6 @@ from superset.charts.schemas import (
from superset.commands.chart.create import CreateChartCommand
from superset.commands.chart.delete import DeleteChartCommand
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
ChartCreateFailedError,
ChartDeleteFailedError,
ChartForbiddenError,
@@ -441,8 +440,6 @@ class ChartRestApi(BaseSupersetModelRestApi):
response = self.response_404()
except ChartForbiddenError:
response = self.response_403()
except DashboardsForbiddenError as ex:
response = self.response(ex.status, message=ex.message)
except TagForbiddenError as ex:
response = self.response(403, message=str(ex))
except ChartInvalidError as ex:
@@ -975,7 +972,7 @@ class ChartRestApi(BaseSupersetModelRestApi):
AddFavoriteChartCommand(pk).run()
except ChartNotFoundError:
return self.response_404()
except (ChartAccessDeniedError, ChartForbiddenError):
except ChartForbiddenError:
return self.response_403()
return self.response(200, result="OK")
@@ -1020,9 +1017,9 @@ class ChartRestApi(BaseSupersetModelRestApi):
try:
DelFavoriteChartCommand(pk).run()
except ChartNotFoundError:
return self.response_404()
except (ChartAccessDeniedError, ChartForbiddenError):
return self.response_403()
self.response_404()
except ChartForbiddenError:
self.response_403()
return self.response(200, result="OK")
-5
View File
@@ -27,7 +27,6 @@ from superset import security_manager
from superset.commands.base import BaseCommand, CreateMixin
from superset.commands.chart.exceptions import (
ChartCreateFailedError,
ChartForbiddenError,
ChartInvalidError,
DashboardsForbiddenError,
DashboardsNotFoundValidationError,
@@ -35,7 +34,6 @@ from superset.commands.chart.exceptions import (
from superset.commands.utils import get_datasource_by_id
from superset.daos.chart import ChartDAO
from superset.daos.dashboard import DashboardDAO
from superset.exceptions import SupersetSecurityException
from superset.utils import json
from superset.utils.decorators import on_error, transaction
@@ -71,9 +69,6 @@ class CreateChartCommand(CreateMixin, BaseCommand):
try:
datasource = get_datasource_by_id(datasource_id, datasource_type)
self._properties["datasource_name"] = datasource.name
security_manager.raise_for_access(datasource=datasource)
except SupersetSecurityException as ex:
raise ChartForbiddenError() from ex
except ValidationError as ex:
exceptions.append(ex)
+1 -7
View File
@@ -17,15 +17,12 @@
import logging
from functools import partial
from superset import security_manager
from superset.commands.base import BaseCommand
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
ChartFaveError,
ChartNotFoundError,
)
from superset.daos.chart import ChartDAO
from superset.exceptions import SupersetSecurityException
from superset.models.slice import Slice
from superset.utils.decorators import on_error, transaction
@@ -47,8 +44,5 @@ class AddFavoriteChartCommand(BaseCommand):
chart = ChartDAO.find_by_id(self._chart_id)
if not chart:
raise ChartNotFoundError()
try:
security_manager.raise_for_access(chart=chart)
except SupersetSecurityException as ex:
raise ChartAccessDeniedError() from ex
self._chart = chart
+1 -7
View File
@@ -17,15 +17,12 @@
import logging
from functools import partial
from superset import security_manager
from superset.commands.base import BaseCommand
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
ChartNotFoundError,
ChartUnfaveError,
)
from superset.daos.chart import ChartDAO
from superset.exceptions import SupersetSecurityException
from superset.models.slice import Slice
from superset.utils.decorators import on_error, transaction
@@ -47,8 +44,5 @@ class DelFavoriteChartCommand(BaseCommand):
chart = ChartDAO.find_by_id(self._chart_id)
if not chart:
raise ChartNotFoundError()
try:
security_manager.raise_for_access(chart=chart)
except SupersetSecurityException as ex:
raise ChartAccessDeniedError() from ex
self._chart = chart
+5 -16
View File
@@ -30,7 +30,6 @@ from superset.commands.chart.exceptions import (
ChartInvalidError,
ChartNotFoundError,
ChartUpdateFailedError,
DashboardsForbiddenError,
DashboardsNotFoundValidationError,
DatasourceTypeUpdateRequiredValidationError,
)
@@ -74,10 +73,10 @@ class UpdateChartCommand(UpdateMixin, BaseCommand):
return ChartDAO.update(self._model, self._properties)
def _validate_new_dashboard_access(
self, requested_dashboards: list[Dashboard], exceptions: list[ValidationError]
self, requested_dashboards: list[Dashboard], exceptions: list[Exception]
) -> None:
"""
Validate user has ownership of any NEW dashboard relationships.
Validate user has access to any NEW dashboard relationships.
Existing relationships are preserved to maintain chart ownership rights.
"""
if not self._model:
@@ -87,20 +86,14 @@ class UpdateChartCommand(UpdateMixin, BaseCommand):
requested_dashboard_ids = {d.id for d in requested_dashboards}
if new_dashboard_ids := requested_dashboard_ids - existing_dashboard_ids:
# For NEW dashboard relationships, verify user has ownership
# For NEW dashboard relationships, verify user has access
accessible_dashboards = DashboardDAO.find_by_ids(list(new_dashboard_ids))
unauthorized_dashboard_ids = new_dashboard_ids - {
d.id for d in accessible_dashboards
}
accessible_dashboard_ids = {d.id for d in accessible_dashboards}
unauthorized_dashboard_ids = new_dashboard_ids - accessible_dashboard_ids
if unauthorized_dashboard_ids:
exceptions.append(DashboardsNotFoundValidationError())
# Additional ownership check - must match CreateChartCommand behavior
for dash in accessible_dashboards:
if not security_manager.is_owner(dash):
raise DashboardsForbiddenError()
def validate(self) -> None: # noqa: C901
exceptions: list[ValidationError] = []
dashboard_ids = self._properties.get("dashboards")
@@ -109,7 +102,6 @@ class UpdateChartCommand(UpdateMixin, BaseCommand):
# Validate if datasource_id is provided datasource_type is required
datasource_id = self._properties.get("datasource_id")
datasource_type = ""
if datasource_id is not None:
datasource_type = self._properties.get("datasource_type", "")
if not datasource_type:
@@ -146,9 +138,6 @@ class UpdateChartCommand(UpdateMixin, BaseCommand):
try:
datasource = get_datasource_by_id(datasource_id, datasource_type)
self._properties["datasource_name"] = datasource.name
security_manager.raise_for_access(datasource=datasource)
except SupersetSecurityException as ex:
raise ChartForbiddenError() from ex
except ValidationError as ex:
exceptions.append(ex)
+6 -13
View File
@@ -23,13 +23,11 @@ from flask import g
from superset.commands.base import BaseCommand
from superset.commands.chart.data.get_data_command import ChartDataCommand
from superset.commands.chart.exceptions import (
ChartAccessDeniedError,
ChartInvalidError,
WarmUpCacheChartNotFoundError,
)
from superset.common.db_query_status import QueryStatus
from superset.exceptions import SupersetSecurityException
from superset.extensions import db, security_manager
from superset.extensions import db
from superset.models.slice import Slice
from superset.utils import json
from superset.utils.core import error_msg_from_exception, QueryObjectFilterClause
@@ -126,13 +124,8 @@ class ChartWarmUpCacheCommand(BaseCommand):
def validate(self) -> None:
if isinstance(self._chart_or_id, Slice):
chart = self._chart_or_id
else:
chart = db.session.query(Slice).filter_by(id=self._chart_or_id).scalar()
if not chart:
raise WarmUpCacheChartNotFoundError()
self._chart_or_id = chart
try:
security_manager.raise_for_access(chart=chart)
except SupersetSecurityException as ex:
raise ChartAccessDeniedError() from ex
return
chart = db.session.query(Slice).filter_by(id=self._chart_or_id).scalar()
if not chart:
raise WarmUpCacheChartNotFoundError()
self._chart_or_id = chart
+1 -4
View File
@@ -48,10 +48,7 @@ class DeleteEmbeddedDashboardCommand(BaseCommand):
return EmbeddedDashboardDAO.delete(self._dashboard.embedded)
def validate(self) -> None:
try:
security_manager.raise_for_ownership(self._dashboard)
except SupersetSecurityException as ex:
raise DashboardForbiddenError() from ex
pass
class DeleteDashboardCommand(BaseCommand):
-8
View File
@@ -102,14 +102,6 @@ class CreateDatasetCommand(CreateMixin, BaseCommand):
field_name="sql",
)
)
elif database:
try:
security_manager.raise_for_access(
database=database,
table=table,
)
except SupersetSecurityException as ex:
exceptions.append(DatasetDataAccessIsNotAllowed(ex.error.message))
try:
owners = self.populate_owners(owner_ids)
self._properties["owners"] = owners
+1 -7
View File
@@ -22,10 +22,8 @@ from flask_appbuilder.models.sqla import Model
from flask_babel import gettext as __
from marshmallow import ValidationError
from superset import security_manager
from superset.commands.base import BaseCommand, CreateMixin
from superset.commands.dataset.exceptions import (
DatasetAccessDeniedError,
DatasetDuplicateFailedError,
DatasetExistsValidationError,
DatasetInvalidError,
@@ -35,7 +33,7 @@ from superset.commands.exceptions import DatasourceTypeInvalidError
from superset.connectors.sqla.models import SqlaTable, SqlMetric, TableColumn
from superset.daos.dataset import DatasetDAO
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import SupersetErrorException, SupersetSecurityException
from superset.exceptions import SupersetErrorException
from superset.extensions import db
from superset.models.core import Database
from superset.sql.parse import Table
@@ -112,10 +110,6 @@ class DuplicateDatasetCommand(CreateMixin, BaseCommand):
if not base_model:
exceptions.append(DatasetNotFoundError())
else:
try:
security_manager.raise_for_access(datasource=base_model)
except SupersetSecurityException as ex:
raise DatasetAccessDeniedError() from ex
self._base_model = base_model
if self._base_model and self._base_model.kind != "virtual":
+2 -49
View File
@@ -44,14 +44,10 @@ from superset.commands.dataset.exceptions import (
DatasetUpdateFailedError,
MultiCatalogDisabledValidationError,
)
from superset.connectors.sqla.models import SqlaTable, validate_stored_expression
from superset.connectors.sqla.models import SqlaTable
from superset.daos.dataset import DatasetDAO
from superset.datasets.schemas import FolderSchema
from superset.exceptions import (
QueryClauseValidationException,
SupersetParseError,
SupersetSecurityException,
)
from superset.exceptions import SupersetParseError, SupersetSecurityException
from superset.models.core import Database
from superset.sql.parse import Table
from superset.utils.decorators import on_error, transaction
@@ -216,11 +212,9 @@ class UpdateDatasetCommand(UpdateMixin, BaseCommand):
self._model = cast(SqlaTable, self._model)
if columns := self._properties.get("columns"):
self._validate_columns(columns, exceptions)
self._validate_expressions(columns, "columns", exceptions)
if metrics := self._properties.get("metrics"):
self._validate_metrics(metrics, exceptions)
self._validate_expressions(metrics, "metrics", exceptions)
if folders := self._properties.get("folders"):
valid_uuids: set[UUID] = set()
@@ -289,47 +283,6 @@ class UpdateDatasetCommand(UpdateMixin, BaseCommand):
if not DatasetDAO.validate_metrics_uniqueness(self._model_id, metric_names):
exceptions.append(DatasetMetricsExistsValidationError())
def _validate_expressions(
self,
items: list[dict[str, Any]],
label: str,
exceptions: list[ValidationError],
) -> None:
"""
Run each item's SQL expression through the parser-based validator that
already governs adhoc expressions, so stored column and metric
expressions cannot smuggle sub-queries, set operations, or
multi-statement SQL into chart queries.
"""
self._model = cast(SqlaTable, self._model)
# `_validate_dataset_source` runs first and rebinds
# `self._properties["database"]` from the request's `database_id`
# to the resolved `Database` model when the user is repointing the
# dataset; otherwise the key is absent and we fall back to the
# currently-bound database on the model.
database = self._properties.get("database") or self._model.database
catalog = self._properties.get("catalog", self._model.catalog)
schema = self._properties.get("schema", self._model.schema)
for idx, item in enumerate(items):
expression = item.get("expression")
if not expression:
continue
try:
validate_stored_expression(database, catalog, schema, expression)
except (SupersetSecurityException, QueryClauseValidationException) as ex:
message = (
ex.error.message
if isinstance(ex, SupersetSecurityException)
else ex.message
)
exceptions.append(
ValidationError(
message,
field_name=f"{label}.{idx}.expression",
)
)
@staticmethod
def _get_duplicates(data: list[dict[str, Any]], key: str) -> list[str]:
duplicates = [
+2 -10
View File
@@ -20,13 +20,9 @@ from typing import Any, Optional
from superset.commands.base import BaseCommand
from superset.commands.chart.warm_up_cache import ChartWarmUpCacheCommand
from superset.commands.dataset.exceptions import (
DatasetAccessDeniedError,
WarmUpCacheTableNotFoundError,
)
from superset.commands.dataset.exceptions import WarmUpCacheTableNotFoundError
from superset.connectors.sqla.models import SqlaTable
from superset.exceptions import SupersetSecurityException
from superset.extensions import db, security_manager
from superset.extensions import db
from superset.models.core import Database
from superset.models.slice import Slice
@@ -67,10 +63,6 @@ class DatasetWarmUpCacheCommand(BaseCommand):
).one_or_none()
if not table:
raise WarmUpCacheTableNotFoundError()
try:
security_manager.raise_for_access(datasource=table)
except SupersetSecurityException as ex:
raise DatasetAccessDeniedError() from ex
self._charts = (
db.session.query(Slice)
.filter_by(datasource_id=table.id, datasource_type=table.type)
+8 -38
View File
@@ -22,7 +22,6 @@ from flask import current_app as app
from flask_babel import gettext as _
from marshmallow import ValidationError
from superset import security_manager
from superset.commands.base import BaseCommand
from superset.commands.report.exceptions import (
ChartNotFoundValidationError,
@@ -30,14 +29,11 @@ from superset.commands.report.exceptions import (
DashboardNotFoundValidationError,
DashboardNotSavedValidationError,
ReportScheduleEitherChartOrDashboardError,
ReportScheduleForbiddenError,
ReportScheduleFrequencyNotAllowed,
ReportScheduleOnlyChartOrDashboardError,
)
from superset.daos.base import BaseDAO
from superset.daos.chart import ChartDAO
from superset.daos.dashboard import DashboardDAO
from superset.exceptions import SupersetSecurityException
from superset.reports.models import (
ReportCreationMethod,
ReportScheduleType,
@@ -57,26 +53,6 @@ class BaseReportScheduleCommand(BaseCommand):
def validate(self) -> None:
pass
def _check_object_access(
self,
object_id: int,
*,
kind: str,
dao: type[BaseDAO[Any]],
not_found_exc: type[ValidationError],
exceptions: list[ValidationError],
) -> None:
"""Validate the object exists and the current user can access it."""
obj = dao.find_by_id(object_id)
if not obj:
exceptions.append(not_found_exc())
else:
try:
security_manager.raise_for_access(**{kind: obj})
except SupersetSecurityException as ex:
raise ReportScheduleForbiddenError() from ex
self._properties[kind] = obj
def validate_chart_dashboard(
self, exceptions: list[ValidationError], update: bool = False
) -> None:
@@ -98,21 +74,15 @@ class BaseReportScheduleCommand(BaseCommand):
exceptions.append(ReportScheduleOnlyChartOrDashboardError())
if chart_id:
self._check_object_access(
chart_id,
kind="chart",
dao=ChartDAO,
not_found_exc=ChartNotFoundValidationError,
exceptions=exceptions,
)
chart = ChartDAO.find_by_id(chart_id)
if not chart:
exceptions.append(ChartNotFoundValidationError())
self._properties["chart"] = chart
elif dashboard_id:
self._check_object_access(
dashboard_id,
kind="dashboard",
dao=DashboardDAO,
not_found_exc=DashboardNotFoundValidationError,
exceptions=exceptions,
)
dashboard = DashboardDAO.find_by_id(dashboard_id)
if not dashboard:
exceptions.append(DashboardNotFoundValidationError())
self._properties["dashboard"] = dashboard
elif not update:
exceptions.append(ReportScheduleEitherChartOrDashboardError())
-35
View File
@@ -60,44 +60,9 @@ class CreateCustomTagCommand(CreateMixin, BaseCommand):
exceptions.append(
TagCreateFailedError(f"invalid object type {self._object_type}")
)
# Validate user has access to the target object
if object_type:
self._validate_object_access(object_type, self._object_id, exceptions)
if exceptions:
raise TagInvalidError(exceptions=exceptions)
def _validate_object_access(
self, object_type: ObjectType, object_id: int, exceptions: list[Any]
) -> None:
"""Validate that the current user has access to the target object."""
# Skip base filter so we can distinguish "not found" from "no access"
target_object = to_object_model(object_type, object_id, skip_base_filter=True)
if not target_object:
# Allow operation on stale references; no object to authorize against
return
try:
if object_type == ObjectType.dashboard:
security_manager.raise_for_access(dashboard=target_object)
elif object_type == ObjectType.chart:
security_manager.raise_for_access(chart=target_object)
elif object_type == ObjectType.query:
security_manager.raise_for_access(query=target_object)
elif object_type == ObjectType.dataset:
security_manager.raise_for_access(datasource=target_object)
else:
exceptions.append(
TagCreateFailedError(
f"Access validation not supported for {object_type}"
)
)
except SupersetSecurityException:
exceptions.append(
TagCreateFailedError(f"Access denied for {object_type} {object_id}")
)
class CreateCustomTagWithRelationshipsCommand(CreateMixin, BaseCommand):
def __init__(self, data: dict[str, Any], bulk_create: bool = False):
+1 -39
View File
@@ -16,9 +16,7 @@
# under the License.
import logging
from functools import partial
from typing import Any
from superset import security_manager
from superset.commands.base import BaseCommand
from superset.commands.tag.exceptions import (
TagDeleteFailedError,
@@ -27,9 +25,8 @@ from superset.commands.tag.exceptions import (
TagInvalidError,
TagNotFoundError,
)
from superset.commands.tag.utils import to_object_model, to_object_type
from superset.commands.tag.utils import to_object_type
from superset.daos.tag import TagDAO
from superset.exceptions import SupersetSecurityException
from superset.tags.models import ObjectType
from superset.utils.decorators import on_error, transaction
from superset.views.base import DeleteMixin
@@ -74,9 +71,6 @@ class DeleteTaggedObjectCommand(DeleteMixin, BaseCommand):
)
)
else:
# Validate user has access to the target object
self._validate_object_access(object_type, self._object_id, exceptions)
tagged_object = TagDAO.find_tagged_object(
object_type=object_type, object_id=self._object_id, tag_id=tag.id
)
@@ -91,38 +85,6 @@ class DeleteTaggedObjectCommand(DeleteMixin, BaseCommand):
if exceptions:
raise TagInvalidError(exceptions=exceptions)
def _validate_object_access(
self, object_type: ObjectType, object_id: int, exceptions: list[Any]
) -> None:
"""Validate that the current user has access to the target object."""
# Skip base filter so we can distinguish "not found" from "no access"
target_object = to_object_model(object_type, object_id, skip_base_filter=True)
if not target_object:
# Allow operation on stale references; no object to authorize against
return
try:
if object_type == ObjectType.dashboard:
security_manager.raise_for_access(dashboard=target_object)
elif object_type == ObjectType.chart:
security_manager.raise_for_access(chart=target_object)
elif object_type == ObjectType.query:
security_manager.raise_for_access(query=target_object)
elif object_type == ObjectType.dataset:
security_manager.raise_for_access(datasource=target_object)
else:
exceptions.append(
TaggedObjectDeleteFailedError(
f"Access validation not supported for {object_type}"
)
)
except SupersetSecurityException:
exceptions.append(
TaggedObjectDeleteFailedError(
f"Access denied for {object_type} {object_id}"
)
)
class DeleteTagsCommand(DeleteMixin, BaseCommand):
def __init__(self, tags: list[str]):
+6 -11
View File
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
from typing import Any, Optional, Union
from typing import Optional, Union
from superset.daos.chart import ChartDAO
from superset.daos.dashboard import DashboardDAO
@@ -36,17 +36,12 @@ def to_object_type(object_type: Union[ObjectType, int, str]) -> Optional[ObjectT
def to_object_model(
object_type: ObjectType, object_id: int, skip_base_filter: bool = False
) -> Optional[Union[Dashboard, SavedQuery, Slice, Any]]:
object_type: ObjectType, object_id: int
) -> Optional[Union[Dashboard, SavedQuery, Slice]]:
if ObjectType.dashboard == object_type:
return DashboardDAO.find_by_id(object_id, skip_base_filter=skip_base_filter)
return DashboardDAO.find_by_id(object_id)
if ObjectType.query == object_type:
return SavedQueryDAO.find_by_id(object_id, skip_base_filter=skip_base_filter)
return SavedQueryDAO.find_by_id(object_id)
if ObjectType.chart == object_type:
return ChartDAO.find_by_id(object_id, skip_base_filter=skip_base_filter)
if ObjectType.dataset == object_type:
# Imported lazily to avoid a circular import via superset.views.base
from superset.daos.dataset import DatasetDAO
return DatasetDAO.find_by_id(object_id, skip_base_filter=skip_base_filter)
return ChartDAO.find_by_id(object_id)
return None
+21 -2
View File
@@ -156,6 +156,14 @@ VERSION_SHA = _try_json_readsha(VERSION_INFO_FILE, VERSION_SHA_LENGTH)
# can be replaced at build time to expose build information.
BUILD_NUMBER = None
# Controls how much detail the unauthenticated ``/version`` endpoint returns.
# When True (default, preserves existing behavior) the endpoint returns the full
# version metadata, including the Git SHA and branch name when available. Set to
# False to return only the human-readable version string and omit the Git SHA,
# full SHA, build number, and branch name, so deployment-specific build details
# are not exposed to unauthenticated callers.
EXPOSE_VERSION_INFO = True
# default viz used in chart explorer & SQL Lab explore
DEFAULT_VIZ_TYPE = "table"
@@ -1193,7 +1201,7 @@ HTML_SANITIZATION_SCHEMA_EXTENSIONS: dict[str, Any] = {}
# than 6 slices in dashboard, a lot of time fetch requests are queued up and wait for
# next available socket. PR #5039 added domain sharding for Superset,
# and this feature can be enabled by configuration only (by default Superset
# doesn't allow cross-domain request). This feature is deprecated, and will be removed
# doesn't allow cross-domain request). This feature is deprecated, annd will be removed
# in the next major version of Superset, as enabling HTTP2 will serve the same goals.
SUPERSET_WEBSERVER_DOMAINS = None # deprecated
@@ -1449,7 +1457,18 @@ CELERY_CONFIG: type[CeleryConfig] | None = CeleryConfig
# within the app
# OVERRIDE_HTTP_HEADERS: sets override values for HTTP headers. These values will
# override anything set within the app
DEFAULT_HTTP_HEADERS: dict[str, Any] = {}
#
# As a defense-in-depth default, Superset sends a conservative
# `Cross-Origin-Resource-Policy` header on its responses. `same-site` is used
# (rather than the stricter `same-origin`) so that same-site embedding patterns
# such as the Embedded SDK, where a Superset subdomain is framed by a sibling
# application subdomain, keep working out of the box. Because this is set through
# DEFAULT_HTTP_HEADERS, the value is only applied when the response does not
# already carry the header, so operators can override it (per-response or by
# replacing this config value) to suit their cross-origin requirements.
DEFAULT_HTTP_HEADERS: dict[str, Any] = {
"Cross-Origin-Resource-Policy": "same-site",
}
OVERRIDE_HTTP_HEADERS: dict[str, Any] = {}
HTTP_HEADERS: dict[str, Any] = {}
+1 -74
View File
@@ -19,7 +19,6 @@ from __future__ import annotations
import builtins
import logging
import re
from collections import defaultdict
from collections.abc import Hashable
from dataclasses import dataclass, field
@@ -79,13 +78,11 @@ from superset.connectors.sqla.utils import (
)
from superset.daos.exceptions import DatasourceNotFound
from superset.db_engine_specs.base import BaseEngineSpec, TimestampExpression
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import (
ColumnNotFoundException,
DatasetInvalidPermissionEvaluationException,
QueryObjectValidationError,
SupersetGenericDBErrorException,
SupersetParseError,
SupersetSecurityException,
SupersetSyntaxErrorException,
)
@@ -104,11 +101,10 @@ from superset.models.helpers import (
ImportExportMixin,
QueryResult,
SQLA_QUERY_KEYS,
validate_adhoc_subquery,
)
from superset.models.slice import Slice
from superset.models.sql_types.base import CurrencyType
from superset.sql.parse import sanitize_clause, SQLStatement, Table
from superset.sql.parse import Table
from superset.superset_typing import (
AdhocColumn,
AdhocMetric,
@@ -871,75 +867,6 @@ class AnnotationDatasource(BaseDatasource):
raise NotImplementedError()
_JINJA_BLOCK_RE = re.compile(
r"\{\{.*?\}\}|\{%.*?%\}|\{#.*?#\}",
re.DOTALL,
)
def validate_stored_expression(
database: Database,
catalog: str | None,
schema: str | None,
expression: str | None,
) -> None:
"""
Apply the adhoc-expression validator to a stored column or metric expression.
Wrapping in a synthetic ``SELECT <expr>`` reuses the column-position parser
rules already enforced for adhoc expressions, so the same policy on
sub-queries, set operations, and multi-statement SQL applies to stored
expressions when they are saved.
Balanced Jinja blocks (``{{ ... }}``, ``{% ... %}``, ``{# ... #}``) are
replaced with a numeric placeholder before parsing so the surrounding SQL
is still inspected; structural attacks smuggled in the non-templated
portion of an otherwise-templated expression are still rejected.
Expressions whose substituted skeleton is unparseable (typically due to
``{% if %}`` control-flow templating) fall back to deferring validation
to query time, when the template processor has a real context.
"""
if not expression:
return
skeleton = _JINJA_BLOCK_RE.sub(" NULL ", expression)
contains_jinja = skeleton != expression
engine = database.backend
wrapped = f"SELECT {skeleton}"
try:
parsed = SQLStatement(wrapped, engine)
except SupersetParseError as ex:
if contains_jinja:
return
raise SupersetSecurityException(
SupersetError(
error_type=SupersetErrorType.ADHOC_SUBQUERY_NOT_ALLOWED_ERROR,
message=_(
"Custom SQL fields cannot be parsed as a single SQL statement."
),
level=ErrorLevel.ERROR,
)
) from ex
if parsed.is_set_operation():
raise SupersetSecurityException(
SupersetError(
error_type=SupersetErrorType.ADHOC_SUBQUERY_NOT_ALLOWED_ERROR,
message=_("Custom SQL fields cannot contain set operations."),
level=ErrorLevel.ERROR,
)
)
validate_adhoc_subquery(
wrapped,
database,
catalog,
schema or "",
engine,
)
sanitize_clause(wrapped, engine)
class TableColumn(AuditMixinNullable, ImportExportMixin, CertificationMixin, Model):
"""ORM object for table columns, each table can have multiple columns"""
+7 -30
View File
@@ -80,24 +80,13 @@ class ColumnOperatorEnum(str, Enum):
return op_func(column, value)
def _escape_like(value: str) -> str:
"""Escape LIKE/ILIKE wildcards to prevent wildcard injection."""
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
# Define operator_map as a module-level dict after the enum is defined
operator_map: Dict[ColumnOperatorEnum, Any] = {
ColumnOperatorEnum.eq: lambda col, val: col == val,
ColumnOperatorEnum.ne: lambda col, val: col != val,
ColumnOperatorEnum.sw: lambda col, val: col.like(
f"{_escape_like(val)}%", escape="\\"
),
ColumnOperatorEnum.ew: lambda col, val: col.like(
f"%{_escape_like(val)}", escape="\\"
),
ColumnOperatorEnum.ct: lambda col, val: col.ilike(
f"%{_escape_like(val)}%", escape="\\"
),
ColumnOperatorEnum.sw: lambda col, val: col.like(f"{val}%"),
ColumnOperatorEnum.ew: lambda col, val: col.like(f"%{val}"),
ColumnOperatorEnum.ct: lambda col, val: col.ilike(f"%{val}%"),
ColumnOperatorEnum.in_: lambda col, val: col.in_(
val if isinstance(val, (list, tuple)) else [val]
),
@@ -108,12 +97,8 @@ operator_map: Dict[ColumnOperatorEnum, Any] = {
ColumnOperatorEnum.gte: lambda col, val: col >= val,
ColumnOperatorEnum.lt: lambda col, val: col < val,
ColumnOperatorEnum.lte: lambda col, val: col <= val,
ColumnOperatorEnum.like: lambda col, val: col.like(
f"%{_escape_like(val)}%", escape="\\"
),
ColumnOperatorEnum.ilike: lambda col, val: col.ilike(
f"%{_escape_like(val)}%", escape="\\"
),
ColumnOperatorEnum.like: lambda col, val: col.like(f"%{val}%"),
ColumnOperatorEnum.ilike: lambda col, val: col.ilike(f"%{val}%"),
ColumnOperatorEnum.is_null: lambda col, _: col.is_(None),
ColumnOperatorEnum.is_not_null: lambda col, _: col.isnot(None),
}
@@ -672,11 +657,7 @@ class BaseDAO(CoreBaseDAO[T], Generic[T]):
for column_name in search_columns:
if hasattr(cls.model_cls, column_name):
column = getattr(cls.model_cls, column_name)
search_filters.append(
cast(column, Text).ilike(
f"%{_escape_like(search)}%", escape="\\"
)
)
search_filters.append(cast(column, Text).ilike(f"%{search}%"))
if search_filters:
query = query.filter(or_(*search_filters))
if custom_filters:
@@ -743,11 +724,7 @@ class BaseDAO(CoreBaseDAO[T], Generic[T]):
for column_name in search_columns:
if hasattr(cls.model_cls, column_name):
column = getattr(cls.model_cls, column_name)
search_filters.append(
cast(column, Text).ilike(
f"%{_escape_like(search)}%", escape="\\"
)
)
search_filters.append(cast(column, Text).ilike(f"%{search}%"))
if search_filters:
query = query.filter(or_(*search_filters))
if custom_filters:
+1 -1
View File
@@ -161,7 +161,7 @@ class DashboardDAO(BaseDAO[Dashboard]):
return dashboard
@staticmethod
def get_datasets_for_dashboard(id_or_slug: str) -> list[tuple[Any, dict[str, Any]]]:
def get_datasets_for_dashboard(id_or_slug: str) -> list[Any]:
dashboard = DashboardDAO.get_by_id_or_slug(id_or_slug)
return dashboard.datasets_trimmed_for_slices()
+2 -24
View File
@@ -570,31 +570,12 @@ class DashboardRestApi(CustomTagsOptimizationMixin, BaseSupersetModelRestApi):
try:
datasets = DashboardDAO.get_datasets_for_dashboard(id_or_slug)
result = [
self._serialize_dashboard_dataset(datasource, payload)
for datasource, payload in datasets
self.dashboard_dataset_schema.dump(dataset) for dataset in datasets
]
return self.response(200, result=result)
except (TypeError, ValueError) as err:
raise DatasetValidationError(err) from err
def _serialize_dashboard_dataset(
self, datasource: Any, payload: dict[str, Any]
) -> dict[str, Any]:
serialized = self.dashboard_dataset_schema.dump(payload)
if not security_manager.can_access_datasource(datasource):
for key in (
"sql",
"select_star",
"fetch_values_predicate",
"template_params",
"params",
):
serialized.pop(key, None)
for collection_key in ("columns", "metrics"):
for item in serialized.get(collection_key) or ():
item.pop("expression", None)
return serialized
@expose("/<id_or_slug>/tabs", methods=("GET",))
@protect()
@safe
@@ -2150,10 +2131,7 @@ class DashboardRestApi(CustomTagsOptimizationMixin, BaseSupersetModelRestApi):
500:
$ref: '#/components/responses/500'
"""
try:
DeleteEmbeddedDashboardCommand(dashboard).run()
except DashboardForbiddenError:
return self.response_403()
DeleteEmbeddedDashboardCommand(dashboard).run()
return self.response(200, message="OK")
@expose("/<id_or_slug>/copy/", methods=("POST",))
-2
View File
@@ -1327,7 +1327,6 @@ class DatabaseRestApi(BaseSupersetModelRestApi):
"viz_type": chart.viz_type,
}
for chart in data["charts"]
if security_manager.can_access_chart(chart)
]
dashboards = [
{
@@ -1337,7 +1336,6 @@ class DatabaseRestApi(BaseSupersetModelRestApi):
"title": dashboard.dashboard_title,
}
for dashboard in data["dashboards"]
if security_manager.can_access_dashboard(dashboard)
]
sqllab_tab_states = [
{"id": tab_state.id, "label": tab_state.label, "active": tab_state.active}
-2
View File
@@ -828,7 +828,6 @@ class DatasetRestApi(BaseSupersetModelRestApi):
"viz_type": chart.viz_type,
}
for chart in data["charts"]
if security_manager.can_access_chart(chart)
]
dashboards = [
{
@@ -838,7 +837,6 @@ class DatasetRestApi(BaseSupersetModelRestApi):
"title": dashboard.dashboard_title,
}
for dashboard in data["dashboards"]
if security_manager.can_access_dashboard(dashboard)
]
return self.response(
200,
+2 -4
View File
@@ -1746,8 +1746,7 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
raise cls.get_dbapi_mapped_exception(ex) from ex
if schema and cls.try_remove_schema_from_table_name:
escaped_schema = re.escape(schema)
tables = {re.sub(f"^{escaped_schema}\\.", "", table) for table in tables}
tables = {re.sub(f"^{schema}\\.", "", table) for table in tables}
return tables
@classmethod
@@ -1775,8 +1774,7 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
raise cls.get_dbapi_mapped_exception(ex) from ex
if schema and cls.try_remove_schema_from_table_name:
escaped_schema = re.escape(schema)
views = {re.sub(f"^{escaped_schema}\\.", "", view) for view in views}
views = {re.sub(f"^{schema}\\.", "", view) for view in views}
return views
@classmethod
+4 -51
View File
@@ -95,11 +95,10 @@ def context_addons() -> dict[str, Any]:
return current_app.config.get("JINJA_CONTEXT_ADDONS", {})
class Filter(TypedDict, total=False):
class Filter(TypedDict):
op: str # pylint: disable=C0103
col: str
val: Union[None, Any, list[Any]]
escaped_val: Union[None, Any, list[Any]]
@dataclass
@@ -346,57 +345,17 @@ class ExtraCache:
return return_val
def _escape_value(self, val: Any) -> Any:
"""Return a dialect-quoted form of ``val`` suitable for direct SQL
interpolation. When no dialect is configured the value is returned
unchanged so callers see the raw value as before. Strings are
passed through SQLAlchemy's ``String`` literal processor (with the
surrounding quotes stripped, mirroring ``url_param``). Lists are
processed element-wise; non-string members are left as-is.
"""
if not self.dialect:
return val
if isinstance(val, str):
return String().literal_processor(dialect=self.dialect)(value=val)[1:-1]
if isinstance(val, list):
return [
String().literal_processor(dialect=self.dialect)(value=v)[1:-1]
if isinstance(v, str)
else v
for v in val
]
return val
def get_filters(self, column: str, remove_filter: bool = False) -> list[Filter]:
"""Get the filters applied to the given column. In addition
to returning values like the filter_values function
the get_filters function returns the operator specified in the explorer UI.
Each filter dict additionally carries an ``escaped_val`` key when a
SQL dialect is available. Templates that interpolate the value into
a SQL string (for example a ``LIKE`` clause) should reference
``escaped_val`` so the value is rendered through the dialect's
literal processor. ``val`` continues to expose the raw value for
non-SQL uses such as comparison, logging, or ``where_in``.
This is useful if:
- you want to handle more than the IN operator in your SQL clause
- you want to handle generating custom SQL conditions for a filter
- you want to have the ability for filter inside the main query for speed
purposes
Always use the ``where_in`` filter for list membership rather than
building SQL by hand. The filter renders values with dialect-safe quoting
(via SQLAlchemy's ``literal_binds`` compilation) instead of interpolating
them directly into the SQL string.
.. warning::
Do not manually escape filter values (for example, with
``replace("'", "''")``). Hand-rolled escaping is error-prone and easy
to get wrong across dialects. Rely on the ``where_in`` filter so values
are quoted safely by the engine.
Usage example::
@@ -420,7 +379,7 @@ class ExtraCache:
{%- endif -%}
{%- if filter.get('op') == 'LIKE' -%}
AND
full_name LIKE '{{ filter.get('escaped_val') }}'
full_name LIKE '{{ filter.get('val') | replace("'", "''") }}'
{%- endif -%}
{%- endfor -%}
UNION ALL
@@ -487,10 +446,7 @@ class ExtraCache:
) and not isinstance(val, list):
val = [val]
entry: Filter = {"op": op, "col": column, "val": val}
if self.dialect:
entry["escaped_val"] = self._escape_value(val)
filters.append(entry)
filters.append({"op": op, "col": column, "val": val})
# Drill-to-detail queries send filters in native {col, op, val} format
# rather than adhoc_filters, so get_form_data() above finds nothing.
@@ -525,10 +481,7 @@ class ExtraCache:
self.removed_filters.append(column)
if column not in self.applied_filters:
self.applied_filters.append(column)
entry: Filter = {"op": op, "col": column, "val": val}
if self.dialect:
entry["escaped_val"] = self._escape_value(val)
filters.append(entry)
filters.append({"op": op, "col": column, "val": val})
return filters
# pylint: disable=too-many-arguments
+2 -2
View File
@@ -59,7 +59,7 @@
* 4. Run npm publish with appropriate access rights
*/
const { spawn, execSync, execFileSync } = require('child_process');
const { spawn, execSync } = require('child_process');
const path = require('path');
const fs = require('fs');
@@ -176,7 +176,7 @@ function checkEnvironment() {
// Check if Superset is installed
try {
execFileSync(python, ['-c', 'import superset'], {
execSync(`${python} -c "import superset"`, {
env: { ...process.env, PYTHONPATH: supersetRoot },
stdio: 'ignore'
});
-153
View File
@@ -274,125 +274,6 @@ def merge_extra_form_data_filters_into_query(
merge_form_data_filters_into_query(query, extra_query_form_data)
def _deck_gl_spatial_cols(spatial: dict[str, Any] | None) -> list[str]:
"""Return the column names referenced by a single Deck.gl spatial control."""
if not isinstance(spatial, dict):
return []
spatial_type = spatial.get("type")
if spatial_type == "latlong":
return [c for c in [spatial.get("lonCol"), spatial.get("latCol")] if c]
if spatial_type == "delimited":
return [c for c in [spatial.get("lonlatCol")] if c]
if spatial_type == "geohash":
return [c for c in [spatial.get("geohashCol")] if c]
return []
def _is_metric_ref(value: Any) -> bool:
"""Return True if value is a metric reference (dict or non-numeric string).
Deck.gl size/metric fields hold either a dict metric definition or a
simple saved-metric string key (e.g. "count"). Scalar numeric strings
like "100" are fixed display settings and must not be treated as metrics.
Note: float() accepts "inf", "-inf", and "nan", so those strings would be
excluded here too they are not valid metric names in practice.
"""
if isinstance(value, dict):
return True
if isinstance(value, str) and value:
try:
float(value)
return False
except ValueError:
return True
return False
def _deck_gl_null_filters(form_data: dict[str, Any]) -> list[dict[str, Any]]:
"""Build IS NOT NULL simple filters for Deck.gl spatial and data columns.
Mirrors BaseDeckGLViz.add_null_filters() behavior: spatial control columns,
line_column, and the geojson column are filtered for non-null values by
default.
"""
seen: set[str] = set()
result: list[dict[str, Any]] = []
for key in ("spatial", "start_spatial", "end_spatial"):
for col in _deck_gl_spatial_cols(form_data.get(key)):
if col not in seen:
seen.add(col)
result.append({"col": col, "op": "IS NOT NULL", "val": ""})
for field in ("line_column", "geojson"):
data_col = form_data.get(field)
if isinstance(data_col, str) and data_col and data_col not in seen:
seen.add(data_col)
result.append({"col": data_col, "op": "IS NOT NULL", "val": ""})
return result
def _resolve_deck_gl_metrics(
form_data: dict[str, Any], viz_type: str = ""
) -> list[Any]:
"""Extract metrics for Deck.gl chart types.
deck_geojson.query_obj() forces metrics=[] regardless of form_data.
For other types, size/metric values are included when they are metric
references (dicts or non-numeric strings); numeric scalars like "100"
are fixed display settings and are excluded.
deck_scatter and deck_polygon can additionally store metric-backed
values in point_radius_fixed (radius for scatter, elevation for polygon).
"""
if viz_type == "deck_geojson":
return []
metrics: list[Any] = []
for field in ("size", "metric"):
m = form_data.get(field)
if _is_metric_ref(m):
metrics.append(m)
prf = form_data.get("point_radius_fixed")
if isinstance(prf, dict) and prf.get("type") == "metric":
value = prf.get("value")
if value:
metrics.append(value)
elif isinstance(prf, str) and _is_metric_ref(prf):
# Legacy deck_scatter: point_radius_fixed as a bare non-numeric metric key
logger.debug("Legacy point_radius_fixed string metric encountered: %s", prf)
metrics.append(prf)
return metrics
def resolve_deck_gl_columns(form_data: dict[str, Any]) -> list[str]:
"""Extract SQL column names for Deck.gl chart types from form_data.
Deck.gl charts use spatial controls (lat/lon pairs, geohash, etc.)
rather than the standard metrics/groupby structure. This function
maps those spatial control configs to the actual column names
needed by the SQL query.
"""
seen: set[str] = set()
columns: list[str] = []
def _add(col: str | None) -> None:
if col and isinstance(col, str) and col not in seen:
seen.add(col)
columns.append(col)
# Most Deck.gl types use "spatial"; arc charts use start/end spatial
for key in ("spatial", "start_spatial", "end_spatial"):
for col in _deck_gl_spatial_cols(form_data.get(key)):
_add(col)
# deck_path / deck_polygon use a line column; deck_geojson uses geojson
for field in ("line_column", "geojson", "dimension"):
_add(form_data.get(field))
for col in form_data.get("js_columns") or []:
if isinstance(col, str):
_add(col)
return columns
def resolve_metrics(form_data: dict[str, Any], viz_type: str) -> list[Any]:
"""Extract metrics from form_data, handling chart-type-specific fields."""
if viz_type == "bubble":
@@ -531,12 +412,6 @@ def _build_mixed_timeseries_secondary(
return qd
# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
{"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
)
def build_query_dicts_from_form_data(
form_data: dict[str, Any],
datasource_id: Any,
@@ -562,34 +437,6 @@ def build_query_dicts_from_form_data(
or (getattr(chart, "viz_type", "") if chart else "")
or ""
)
# Deck.gl charts use spatial column configs rather than the standard
# metrics / groupby fields. Extract columns from the spatial controls.
if viz_type.startswith("deck_"):
deck_columns = resolve_deck_gl_columns(form_data)
deck_metrics = _resolve_deck_gl_metrics(form_data, viz_type)
qd = _build_single_query_dict(
form_data,
deck_columns,
deck_metrics,
row_limit=row_limit,
order_desc=order_desc,
)
if deck_metrics:
# Mirror BaseDeckGLViz.query_obj(): order by first metric descending
qd["orderby"] = [(deck_metrics[0], not form_data.get("order_desc", True))]
if viz_type in _DECK_TIMESERIES_VIZ_TYPES and (
time_grain := form_data.get("time_grain_sqla")
):
qd["is_timeseries"] = True
qd["granularity"] = form_data.get("granularity_sqla")
qd.setdefault("extras", {})["time_grain_sqla"] = time_grain
if form_data.get("filter_nulls", True):
null_filters = _deck_gl_null_filters(form_data)
if null_filters:
qd["filters"] = [*(qd.get("filters") or []), *null_filters]
return [qd]
is_timeseries = (
viz_type.startswith("echarts_timeseries") or viz_type == "mixed_timeseries"
)
@@ -522,10 +522,31 @@ async def get_chart_data( # noqa: C901
# groupby-like fields (entity, series, columns):
# world_map, treemap_v2, sunburst_v2, gauge_chart
# Bubble charts use x/y/size as separate metric fields.
# Deck.gl charts (deck_arc, deck_scatter, etc.) use spatial
# column configs (lat/lon, geohash, etc.) instead.
viz_type = chart.viz_type or ""
# Deck.gl chart types store spatial data (lat/lon)
# rather than traditional metrics/groupby. They
# require a saved query_context to retrieve data.
# Match by prefix to cover all current and future
# deck.gl viz types (deck_arc, deck_scatter, etc.).
if viz_type.startswith("deck_"):
await ctx.warning(
"Chart %s is a deck.gl visualization (%s) with no "
"saved query_context. Data retrieval requires "
"re-saving the chart in Superset." % (chart.id, viz_type)
)
return ChartError(
error=(
f"Chart {chart.id} is a deck.gl visualization "
f"(type: {viz_type}) with no saved query_context. "
f"Deck.gl charts use spatial data (lat/lon) that "
f"cannot be reconstructed from form_data alone. "
f"Please open this chart in Superset and re-save "
f"it to generate a query_context."
),
error_type="MissingQueryContext",
)
fallback_queries = build_query_dicts_from_form_data(
form_data,
chart.datasource_id,
+3 -5
View File
@@ -267,15 +267,13 @@ class Dashboard(CoreDashboard, AuditMixinNullable, ImportExportMixin):
"is_managed_externally": self.is_managed_externally,
}
def datasets_trimmed_for_slices(
self,
) -> list[tuple[BaseDatasource, dict[str, Any]]]:
def datasets_trimmed_for_slices(self) -> list[dict[str, Any]]:
slices_by_datasource: dict[int, set[Slice]] = defaultdict(set)
for slc in self.slices:
slices_by_datasource[slc.datasource_id].add(slc)
result: list[tuple[BaseDatasource, dict[str, Any]]] = []
result: list[dict[str, Any]] = []
for _, slices in slices_by_datasource.items():
# Use the eagerly-loaded datasource from any slice in the group
@@ -283,7 +281,7 @@ class Dashboard(CoreDashboard, AuditMixinNullable, ImportExportMixin):
if datasource:
# Filter out unneeded fields from the datasource payload
result.append((datasource, datasource.data_for_slices(list(slices))))
result.append(datasource.data_for_slices(list(slices)))
return result
-80
View File
@@ -90,8 +90,6 @@ from superset.exceptions import (
InvalidPostProcessingError,
QueryClauseValidationException,
QueryObjectValidationError,
SupersetDisallowedSQLFunctionException,
SupersetDisallowedSQLTableException,
SupersetErrorException,
SupersetErrorsException,
SupersetSecurityException,
@@ -1194,51 +1192,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
expression = sanitize_clause(expression, engine)
except QueryClauseValidationException as ex:
raise QueryObjectValidationError(ex.message) from ex
# Adhoc expressions are user-controlled SQL that ends up inside a
# `literal_column(...)`. Apply the operator-configured
# `DISALLOWED_SQL_FUNCTIONS` / `DISALLOWED_SQL_TABLES` gates at the
# validation step so a dangerous function call (e.g. `version()`,
# `pg_read_file(...)`, `query_to_xml(...)`) is rejected before the
# expression is incorporated into the final SQL. This complements
# the same gate applied at query-execution time and gives the
# adhoc-expression path defense in depth.
disallowed_functions = app.config["DISALLOWED_SQL_FUNCTIONS"].get(
engine, set()
)
disallowed_tables = app.config["DISALLOWED_SQL_TABLES"].get(engine, set())
if disallowed_functions or disallowed_tables:
# `_process_select_expression` (and siblings) pre-wraps the
# input with `SELECT ...`; other callers pass bare
# expressions. Detect and don't double-wrap, otherwise
# `SELECT SELECT ...` fails the sqlglot parse.
sql_to_check = (
expression
if expression.strip().upper().startswith("SELECT")
else f"SELECT {expression}"
)
parsed = SQLScript(sql_to_check, engine=engine)
if disallowed_functions and parsed.check_functions_present(
disallowed_functions
):
raise SupersetDisallowedSQLFunctionException(disallowed_functions)
if disallowed_tables and parsed.check_tables_present(disallowed_tables):
# Report only the tables actually found in the expression,
# mirroring the canonical execution-time gate in
# `superset.sql_lab._validate_query` so the user-facing
# error doesn't echo the operator's full denylist.
present_tables = {
table.table.lower()
for statement in parsed.statements
for table in statement.tables
}
found_tables = {
table
for table in disallowed_tables
if table.lower() in present_tables
}
raise SupersetDisallowedSQLTableException(
found_tables or disallowed_tables
)
return expression
def _process_select_expression(
@@ -1449,36 +1402,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
if is_alias_used_in_orderby(col):
col.name = f"{col.name}__"
def _raise_for_disallowed_sql(self, sql: str) -> None:
"""
Mirror the DISALLOWED_SQL_* gate that sql_lab.execute_sql_statement
enforces so both query surfaces honour the same denylist.
"""
engine = self.db_engine_spec.engine
disallowed_functions = app.config["DISALLOWED_SQL_FUNCTIONS"].get(engine, set())
disallowed_tables = app.config["DISALLOWED_SQL_TABLES"].get(engine, set())
if not (disallowed_functions or disallowed_tables):
return
parsed_script = SQLScript(sql, engine=engine)
if disallowed_functions and parsed_script.check_functions_present(
disallowed_functions
):
raise SupersetDisallowedSQLFunctionException(disallowed_functions)
if disallowed_tables and parsed_script.check_tables_present(disallowed_tables):
# Report only the tables actually found in the query, mirroring the
# canonical execution-time gate so the user-facing error doesn't
# echo the operator's full denylist.
present_tables = {
table.table.lower()
for statement in parsed_script.statements
for table in statement.tables
}
found_tables = {
table for table in disallowed_tables if table.lower() in present_tables
}
raise SupersetDisallowedSQLTableException(found_tables or disallowed_tables)
def query(self, query_obj: QueryObjectDict) -> QueryResult:
"""
Executes the query and returns a dataframe.
@@ -1489,9 +1412,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
qry_start_dttm = datetime.now()
query_str_ext = self.get_query_str_extended(query_obj)
sql = query_str_ext.sql
self._raise_for_disallowed_sql(sql)
status = QueryStatus.SUCCESS
errors = None
error_message = None
+1 -16
View File
@@ -83,18 +83,6 @@ class GuestTokenCreateSchema(PermissiveSchema):
user = fields.Nested(UserSchema)
resources = fields.List(fields.Nested(ResourceSchema), required=True)
rls = fields.List(fields.Nested(RlsRuleSchema), required=True)
datasets = fields.List(
fields.Integer(),
load_default=None,
allow_none=True,
metadata={
"description": (
"Optional allowlist of dataset IDs the guest may access. "
"When omitted all datasets linked to the embedded dashboard "
"are accessible, preserving the default behaviour."
)
},
)
class RoleResponseSchema(PermissiveSchema):
@@ -199,10 +187,7 @@ class SecurityRestApi(BaseSupersetApi):
# make sure username doesn't reference an existing user
# check rls rules for validity?
token = self.appbuilder.sm.create_guest_access_token(
body.get("user", {}),
body["resources"],
body["rls"],
**({"datasets": body["datasets"]} if "datasets" in body else {}),
body["user"], body["resources"], body["rls"]
)
return self.response(200, token=token)
except EmbeddedDashboardNotFoundError as error:
+1 -14
View File
@@ -45,9 +45,7 @@ class GuestTokenRlsRule(TypedDict):
clause: str
class _GuestTokenRequired(TypedDict):
"""Required JWT claims for a guest token payload."""
class GuestToken(TypedDict):
iat: float
exp: float
user: GuestTokenUser
@@ -55,17 +53,6 @@ class _GuestTokenRequired(TypedDict):
rls_rules: list[GuestTokenRlsRule]
class GuestToken(_GuestTokenRequired, total=False):
"""JWT claims for an embedded guest token.
``datasets`` is an optional allowlist of dataset IDs the guest may access.
When absent the guest can access all datasets linked to the embedded dashboard,
preserving existing behaviour. When present only the listed IDs are permitted.
"""
datasets: list[int]
class GuestUser(AnonymousUserMixin):
"""
Used as the "anonymous" user in case of guest authentication (embedded)
+2 -25
View File
@@ -3208,23 +3208,6 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
self.get_datasource_access_error_object(datasource)
)
# When the guest token carries a dataset allowlist, restrict access
# to only those dataset IDs even if the chart/dashboard check above
# would otherwise grant it. Tokens without the ``datasets`` claim
# retain the existing behaviour (all dashboard datasets accessible).
if guest_user := self.get_current_guest_user_if_guest():
allowed_datasets: Optional[list[int]] = guest_user.guest_token.get(
"datasets"
)
if allowed_datasets is not None and (
not isinstance(allowed_datasets, list)
or not all(isinstance(d, int) for d in allowed_datasets)
or datasource.id not in allowed_datasets
):
raise SupersetSecurityException(
self.get_datasource_access_error_object(datasource)
)
if dashboard:
if self.is_guest_user():
# Guest user is currently used for embedded dashboards only. If the guest # noqa: E501
@@ -3554,7 +3537,6 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
user: GuestTokenUser,
resources: GuestTokenResources,
rls: list[GuestTokenRlsRule],
datasets: Optional[list[int]] = None,
) -> bytes:
secret = get_conf()["GUEST_TOKEN_JWT_SECRET"]
algo = get_conf()["GUEST_TOKEN_JWT_ALGO"]
@@ -3563,7 +3545,7 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
# calculate expiration time
now = self._get_current_epoch_time()
exp = now + exp_seconds
claims: dict[str, Any] = {
claims = {
"user": user,
"resources": resources,
"rls_rules": rls,
@@ -3573,8 +3555,6 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
"aud": audience,
"type": "guest",
}
if datasets is not None:
claims["datasets"] = datasets
return self.pyjwt_for_guest_token.encode(claims, secret, algorithm=algo)
def get_guest_user_from_request(self, req: Request) -> Optional[GuestUser]:
@@ -3645,10 +3625,7 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
return hasattr(user, "is_guest_user") and user.is_guest_user
def get_current_guest_user_if_guest(self) -> Optional[GuestUser]:
user = getattr(g, "user", None)
if isinstance(user, GuestUser):
return user
return None
return g.user if self.is_guest_user() else None
def has_guest_access(self, dashboard: "Dashboard") -> bool:
user = self.get_current_guest_user_if_guest()
+3 -11
View File
@@ -923,11 +923,9 @@ class SQLStatement(BaseSQLStatement[exp.Expression]):
:return: A new SQLStatement with the create table statement.
"""
table_expr = exp.Table(
this=exp.Identifier(this=table.table, quoted=True),
db=exp.Identifier(this=table.schema, quoted=True) if table.schema else None,
catalog=exp.Identifier(this=table.catalog, quoted=True)
if table.catalog
else None,
this=exp.Identifier(this=table.table),
db=exp.Identifier(this=table.schema) if table.schema else None,
catalog=exp.Identifier(this=table.catalog) if table.catalog else None,
)
create_table = exp.Create(
this=table_expr,
@@ -952,12 +950,6 @@ class SQLStatement(BaseSQLStatement[exp.Expression]):
)
)
def is_set_operation(self) -> bool:
"""
Check if the statement is a top-level set operation (UNION/INTERSECT/EXCEPT).
"""
return isinstance(self._parsed, exp.SetOperation)
def parse_predicate(self, predicate: str) -> exp.Expression:
"""
Parse a predicate string into an AST.
+2 -12
View File
@@ -14,17 +14,10 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from marshmallow import fields, Schema, validate
from marshmallow import fields, Schema
from superset.databases.schemas import ImportV1DatabaseSchema
# Restricts the optional CTAS target name to a bare SQL identifier. Shared by the
# SQL Lab execute payload schemas so both request paths validate it identically.
tmp_table_name_validator = validate.Regexp(
r"^([A-Za-z_][A-Za-z0-9_]*)?\Z",
error="tmp_table_name must contain only letters, digits, and underscores",
)
sql_lab_get_results_schema = {
"type": "object",
"properties": {
@@ -76,10 +69,7 @@ class ExecutePayloadSchema(Schema):
tab = fields.String(allow_none=True)
ctas_method = fields.String(allow_none=True)
templateParams = fields.String(allow_none=True) # noqa: N815
tmp_table_name = fields.String(
allow_none=True,
validate=tmp_table_name_validator,
)
tmp_table_name = fields.String(allow_none=True)
select_as_cta = fields.Boolean(allow_none=True)
runAsync = fields.Boolean(allow_none=True) # noqa: N815
expand_data = fields.Boolean(allow_none=True)
+1 -4
View File
@@ -14,9 +14,6 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Keep Babel in sync with requirements/base.txt — CI extracts/updates the
# catalogs with that version (via requirements/development.txt), so a different
# pin here makes local `babel_update.sh` runs produce spurious reformatting diffs.
Babel==2.17.0
Babel==2.9.1
jinja2==3.1.6
polib>=1.2.0
+1 -13
View File
@@ -118,14 +118,6 @@ def process_html_links(html_content: str) -> str:
return html_content
# Characters that browsers remove from URLs during parsing per the WHATWG
# URL spec (TAB, LF, CR). Both literal and percent-encoded forms must be
# stripped before any structural check, otherwise a path like
# ``/%09///host`` slips past a leading-``//`` guard because the percent-
# encoded TAB only disappears after the browser parses the URL.
_URL_STRIPPED_CONTROL_CHARS = re.compile(r"[\t\n\r]|%09|%0[ADad]")
def is_safe_redirect_url(url: str) -> bool:
"""
Return True if *url* is an internal Superset URL (safe to redirect to
@@ -134,11 +126,7 @@ def is_safe_redirect_url(url: str) -> bool:
if not url or not url.strip():
return False
# Normalize the URL the same way a browser will before parsing: drop
# the TAB/LF/CR characters that the WHATWG URL parser removes, plus
# their percent-encoded forms (which some browsers also strip when
# following a Location header).
stripped = _URL_STRIPPED_CONTROL_CHARS.sub("", url.strip())
stripped = url.strip()
# Block protocol-relative URLs
if stripped.startswith("//") or stripped.startswith("\\\\"):
+6 -13
View File
@@ -24,14 +24,13 @@ from flask_appbuilder.api import rison as parse_rison
from flask_appbuilder.security.decorators import has_access_api
from flask_babel import lazy_gettext as _
from superset import event_logger
from superset import db, event_logger
from superset.commands.chart.exceptions import (
ChartNotFoundError,
TimeRangeAmbiguousError,
TimeRangeParseFailError,
)
from superset.daos.chart import ChartDAO
from superset.legacy import update_time_range
from superset.models.slice import Slice
from superset.superset_typing import FlaskResponse
from superset.utils import json
from superset.utils.date_parser import get_since_until
@@ -86,17 +85,11 @@ class Api(BaseSupersetView):
Get the form_data stored in the database for existing slice.
params: slice_id: integer
"""
form_data: dict[str, Any] = {}
form_data = {}
if slice_id := request.args.get("slice_id"):
# Reuse ChartDAO.get_by_id_or_uuid so this endpoint applies the
# same ChartFilter (dataset-scoped) as ChartRestApi.get. Both a
# missing chart and a chart the caller cannot access surface as
# ChartNotFoundError, mapped to 404 so the status code cannot be
# used to distinguish the two cases.
try:
form_data = ChartDAO.get_by_id_or_uuid(slice_id).form_data.copy()
except ChartNotFoundError:
return self.json_response({}, 404)
slc = db.session.query(Slice).filter_by(id=slice_id).one_or_none()
if slc:
form_data = slc.form_data.copy()
update_time_range(form_data)
+1 -13
View File
@@ -18,7 +18,7 @@ from __future__ import annotations
import functools
import logging
from typing import Any, Callable, cast, Optional
from typing import Any, Callable, cast
from flask import request, Response
from flask_appbuilder import Model, ModelRestApi
@@ -566,14 +566,6 @@ class BaseSupersetModelRestApi(BaseSupersetApiMixin, ModelRestApi):
self.send_stats_metrics(response, self.delete.__name__, duration)
return response
def ensure_owners_write_access(self, column_name: str) -> Optional[Response]:
"""Restrict the owners related field to users with write access."""
if column_name == "owners" and not security_manager.can_access(
"can_write", self.class_permission_name
):
return self.response_403()
return None
@expose("/related/<column_name>", methods=("GET",))
@protect()
@safe
@@ -608,15 +600,11 @@ class BaseSupersetModelRestApi(BaseSupersetApiMixin, ModelRestApi):
$ref: '#/components/responses/400'
401:
$ref: '#/components/responses/401'
403:
$ref: '#/components/responses/403'
404:
$ref: '#/components/responses/404'
500:
$ref: '#/components/responses/500'
"""
if response := self.ensure_owners_write_access(column_name):
return response
if column_name not in self.allowed_rel_fields:
self.incr_stats("error", self.related.__name__)
return self.response_404()
+5 -12
View File
@@ -14,10 +14,8 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from __future__ import annotations
import logging
from typing import Any, Iterable, Optional, TYPE_CHECKING
from typing import Any, Iterable, Optional
from flask import current_app as app
@@ -30,9 +28,6 @@ from superset.daos.datasource import DatasourceDAO
from superset.utils.core import QueryStatus
from superset.views.datasource.schemas import SamplesPayloadSchema
if TYPE_CHECKING:
from superset.daos.datasource import Datasource
logger = logging.getLogger(__name__)
@@ -100,14 +95,12 @@ def get_samples( # pylint: disable=too-many-arguments
page: int = 1,
per_page: int = 1000,
payload: SamplesPayloadSchema | None = None,
datasource: Datasource | None = None,
dashboard_id: int | None = None,
) -> dict[str, Any]:
if datasource is None:
datasource = DatasourceDAO.get_datasource(
datasource_type=datasource_type,
database_id_or_uuid=str(datasource_id),
)
datasource = DatasourceDAO.get_datasource(
datasource_type=datasource_type,
database_id_or_uuid=str(datasource_id),
)
form_data = {"dashboardId": dashboard_id} if dashboard_id else None
limit_clause = get_limit_clause(page, per_page)
-24
View File
@@ -36,7 +36,6 @@ from superset.connectors.sqla.utils import get_physical_table_metadata
from superset.daos.dashboard import DashboardDAO
from superset.daos.dataset import DatasetDAO
from superset.daos.datasource import DatasourceDAO
from superset.daos.exceptions import DatasourceNotFound, DatasourceTypeNotSupportedError
from superset.exceptions import SupersetException, SupersetSecurityException
from superset.models.core import Database
from superset.sql.parse import Table
@@ -224,28 +223,6 @@ class Datasource(BaseSupersetView):
dashboard,
):
return json_error_response(_("Forbidden"), status=403)
else:
# Pre-fetch and access-check only for table-type datasources.
# Non-table types (query, saved_query) use a different access model;
# passing them to raise_for_access(datasource=...) would check the
# wrong attributes. Let get_samples() handle the lookup for those types.
if params["datasource_type"] in {
DatasourceType.TABLE.value,
DatasourceType.DATASET.value,
}:
try:
dataset = DatasourceDAO.get_datasource(
datasource_type=params["datasource_type"],
database_id_or_uuid=params["datasource_id"],
)
except (DatasourceNotFound, DatasourceTypeNotSupportedError):
return self.response_404()
try:
security_manager.raise_for_access(datasource=dataset)
except SupersetSecurityException:
return json_error_response(_("Forbidden"), status=403)
else:
dataset = None
rv = get_samples(
datasource_type=params["datasource_type"],
@@ -254,7 +231,6 @@ class Datasource(BaseSupersetView):
page=params["page"],
per_page=params["per_page"],
payload=payload,
datasource=dataset,
dashboard_id=dashboard_id,
)
return self.json_response({"result": rv})
+10 -2
View File
@@ -37,9 +37,17 @@ def health() -> FlaskResponse:
@talisman(force_https=False)
def version() -> FlaskResponse:
"""
Return comprehensive version information including Git SHA
and branch when available.
Return version information for the running Superset instance.
When ``EXPOSE_VERSION_INFO`` is True (default) this returns the full
version metadata, including the Git SHA and branch name when available.
When it is False, only the human-readable version string is returned and
build-specific details (Git SHA, full SHA, build number, branch name) are
omitted so they are not exposed to unauthenticated callers.
"""
if not app.config.get("EXPOSE_VERSION_INFO", True):
return jsonify({"version_string": app.config.get("VERSION_STRING", "unknown")})
from superset.utils.version import get_version_metadata
return jsonify(get_version_metadata())
+1 -6
View File
@@ -17,8 +17,6 @@
from marshmallow import fields, Schema
from superset.sqllab.schemas import tmp_table_name_validator
class SqlJsonPayloadSchema(Schema):
database_id = fields.Integer(required=True)
@@ -30,10 +28,7 @@ class SqlJsonPayloadSchema(Schema):
tab = fields.String(allow_none=True)
ctas_method = fields.String(allow_none=True)
templateParams = fields.String(allow_none=True) # noqa: N815
tmp_table_name = fields.String(
allow_none=True,
validate=tmp_table_name_validator,
)
tmp_table_name = fields.String(allow_none=True)
select_as_cta = fields.Boolean(allow_none=True)
runAsync = fields.Boolean(allow_none=True) # noqa: N815
expand_data = fields.Boolean(allow_none=True)
+13 -40
View File
@@ -226,14 +226,10 @@ class TableSchemaView(BaseSupersetView):
def post(self) -> FlaskResponse:
try:
table = json.loads(request.form["table"])
tab_state_id = table["queryEditorId"]
owner_id = _get_owner_id(tab_state_id)
if owner_id is None or owner_id != get_user_id():
return json_error_response(__("Forbidden"), status=403)
# delete any existing table schema
db.session.query(TableSchema).filter(
TableSchema.tab_state_id == tab_state_id,
TableSchema.tab_state_id == table["queryEditorId"],
TableSchema.database_id == table["dbId"],
TableSchema.catalog == table.get("catalog"),
TableSchema.schema == table["schema"],
@@ -241,7 +237,7 @@ class TableSchemaView(BaseSupersetView):
).delete(synchronize_session=False)
table_schema = TableSchema(
tab_state_id=tab_state_id,
tab_state_id=table["queryEditorId"],
database_id=table["dbId"],
catalog=table.get("catalog"),
schema=table["schema"],
@@ -260,19 +256,9 @@ class TableSchemaView(BaseSupersetView):
@expose("/<int:table_schema_id>", methods=("DELETE",))
def delete(self, table_schema_id: int) -> FlaskResponse:
try:
tab_state_id = (
db.session.query(TableSchema.tab_state_id)
.filter_by(id=table_schema_id)
.scalar()
)
if tab_state_id is None:
return json_error_response(__("Not found"), status=404)
owner_id = _get_owner_id(tab_state_id)
if owner_id is None or owner_id != get_user_id():
return json_error_response(__("Forbidden"), status=403)
db.session.query(TableSchema).filter_by(id=table_schema_id).delete(
synchronize_session=False
)
db.session.query(TableSchema).filter(
TableSchema.id == table_schema_id
).delete(synchronize_session=False)
db.session.commit()
return json_success(json.dumps("OK"))
except Exception as ex: # pylint: disable=broad-except
@@ -282,24 +268,11 @@ class TableSchemaView(BaseSupersetView):
@has_access_api
@expose("/<int:table_schema_id>/expanded", methods=("POST",))
def expanded(self, table_schema_id: int) -> FlaskResponse:
try:
tab_state_id = (
db.session.query(TableSchema.tab_state_id)
.filter_by(id=table_schema_id)
.scalar()
)
if tab_state_id is None:
return json_error_response(__("Not found"), status=404)
owner_id = _get_owner_id(tab_state_id)
if owner_id is None or owner_id != get_user_id():
return json_error_response(__("Forbidden"), status=403)
payload = json.loads(request.form["expanded"])
db.session.query(TableSchema).filter_by(id=table_schema_id).update(
{"expanded": payload}
)
db.session.commit()
response = json.dumps({"id": table_schema_id, "expanded": payload})
return json_success(response)
except Exception as ex: # pylint: disable=broad-except
db.session.rollback()
return json_error_response(error_msg_from_exception(ex), 400)
payload = json.loads(request.form["expanded"])
(
db.session.query(TableSchema)
.filter_by(id=table_schema_id)
.update({"expanded": payload})
)
response = json.dumps({"id": table_schema_id, "expanded": payload})
return json_success(response)
-4
View File
@@ -169,13 +169,9 @@ class UserRestApi(BaseSupersetApi):
resource_name = "user"
openapi_spec_tag = "User"
# Enable browser login for all user endpoints to support avatar access and other
# user-related functionality that may be called from browser contexts
allow_browser_login = True
openapi_spec_component_schemas = (UserResponseSchema,)
@expose("/<int:user_id>/avatar.png", methods=("GET",))
@protect()
@safe
def avatar(self, user_id: int) -> Response:
"""Get a redirect to the avatar's URL for the user with the given ID.
+1 -3
View File
@@ -243,12 +243,10 @@ def get_form_data(
# or if form_data only contains slice_id and additional filters
if slice_id and (use_slice_data or valid_slice_id):
slc = db.session.query(Slice).filter_by(id=slice_id).one_or_none()
if slc and security_manager.can_access_chart(slc):
if slc:
slice_form_data = slc.form_data.copy()
slice_form_data.update(form_data)
form_data = slice_form_data
else:
slc = None
update_time_range(form_data)
return form_data, slc
+109 -29
View File
@@ -130,18 +130,6 @@ def quote_f(value: Optional[str]):
return inspector.engine.dialect.identifier_preparer.quote_identifier(value)
def expected_cta_sql(
ctas_method: CTASMethod, table: str, schema: Optional[str] = None
) -> str:
target = quote_f(table)
if schema:
target = f"{quote_f(schema)}.{target}"
return (
f"CREATE {ctas_method.name} {target} AS\n"
"SELECT\n name\nFROM birth_names\nLIMIT 1"
)
def cta_result(ctas_method: CTASMethod):
if backend() != "presto":
return [], []
@@ -237,7 +225,31 @@ def test_run_sync_query_cta_no_data(test_client):
@pytest.mark.usefixtures("load_birth_names_data", "login_as_admin")
@pytest.mark.parametrize("ctas_method", [CTASMethod.TABLE, CTASMethod.VIEW])
@pytest.mark.parametrize(
"ctas_method, expected",
[
(
CTASMethod.TABLE,
"""
CREATE TABLE sqllab_test_db.test_sync_cta_table AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
(
CTASMethod.VIEW,
"""
CREATE VIEW sqllab_test_db.test_sync_cta_view AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
],
)
@mock.patch( # noqa: PT008
"superset.sqllab.sqllab_execution_context.get_cta_schema_name",
lambda d, u, s, sql: CTAS_SCHEMA_NAME,
@@ -245,13 +257,12 @@ def test_run_sync_query_cta_no_data(test_client):
def test_run_sync_query_cta_config(
test_client,
ctas_method: CTASMethod,
expected: str,
) -> None:
db_backend = backend()
if db_backend == "sqlite":
if backend() == "sqlite":
# sqlite doesn't support schemas
return
tmp_table_name = f"{TEST_SYNC_CTA}_{ctas_method.name.lower()}"
expected = expected_cta_sql(ctas_method, tmp_table_name, CTAS_SCHEMA_NAME)
result = run_sql(
test_client, QUERY, cta=True, ctas_method=ctas_method, tmp_table=tmp_table_name
)
@@ -270,7 +281,31 @@ def test_run_sync_query_cta_config(
@pytest.mark.usefixtures("load_birth_names_data", "login_as_admin")
@pytest.mark.parametrize("ctas_method", [CTASMethod.TABLE, CTASMethod.VIEW])
@pytest.mark.parametrize(
"ctas_method, expected",
[
(
CTASMethod.TABLE,
"""
CREATE TABLE sqllab_test_db.test_async_cta_config_table AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
(
CTASMethod.VIEW,
"""
CREATE VIEW sqllab_test_db.test_async_cta_config_view AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
],
)
@mock.patch( # noqa: PT008
"superset.sqllab.sqllab_execution_context.get_cta_schema_name",
lambda d, u, s, sql: CTAS_SCHEMA_NAME,
@@ -278,13 +313,12 @@ def test_run_sync_query_cta_config(
def test_run_async_query_cta_config(
test_client,
ctas_method: CTASMethod,
expected: str,
) -> None:
db_backend = backend()
if db_backend == "sqlite":
if backend() == "sqlite":
# sqlite doesn't support schemas
return
tmp_table_name = f"{TEST_ASYNC_CTA_CONFIG}_{ctas_method.name.lower()}"
expected = expected_cta_sql(ctas_method, tmp_table_name, CTAS_SCHEMA_NAME)
result = run_sql(
test_client,
QUERY,
@@ -307,14 +341,37 @@ def test_run_async_query_cta_config(
@pytest.mark.usefixtures("load_birth_names_data", "login_as_admin")
@pytest.mark.parametrize("ctas_method", [CTASMethod.TABLE, CTASMethod.VIEW])
@pytest.mark.parametrize(
"ctas_method, expected",
[
(
CTASMethod.TABLE,
"""
CREATE TABLE test_async_cta_table AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
(
CTASMethod.VIEW,
"""
CREATE VIEW test_async_cta_view AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
],
)
def test_run_async_cta_query(
test_client,
ctas_method: CTASMethod,
expected: str,
) -> None:
db_backend = backend()
table_name = f"{TEST_ASYNC_CTA}_{ctas_method.name.lower()}"
expected = expected_cta_sql(ctas_method, table_name)
result = run_sql(
test_client,
QUERY,
@@ -331,7 +388,7 @@ def test_run_async_cta_query(
assert query.executed_sql == expected
assert QUERY == query.sql
assert query.rows == (1 if db_backend == "presto" else 0)
assert query.rows == (1 if backend() == "presto" else 0)
assert query.select_as_cta
assert query.select_as_cta_used
@@ -339,14 +396,37 @@ def test_run_async_cta_query(
@pytest.mark.usefixtures("load_birth_names_data", "login_as_admin")
@pytest.mark.parametrize("ctas_method", [CTASMethod.TABLE, CTASMethod.VIEW])
@pytest.mark.parametrize(
"ctas_method, expected",
[
(
CTASMethod.TABLE,
"""
CREATE TABLE test_async_lower_limit_table AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
(
CTASMethod.VIEW,
"""
CREATE VIEW test_async_lower_limit_view AS
SELECT
name
FROM birth_names
LIMIT 1
""".strip(),
),
],
)
def test_run_async_cta_query_with_lower_limit(
test_client,
ctas_method: CTASMethod,
expected: str,
) -> None:
db_backend = backend()
tmp_table = f"{TEST_ASYNC_LOWER_LIMIT}_{ctas_method.name.lower()}"
expected = expected_cta_sql(ctas_method, tmp_table)
result = run_sql(
test_client,
QUERY,
@@ -361,14 +441,14 @@ def test_run_async_cta_query_with_lower_limit(
sqlite_select_sql = f"SELECT\n *\nFROM {tmp_table}\nLIMIT {query.limit}\nOFFSET 0"
assert query.select_sql == (
sqlite_select_sql
if db_backend == "sqlite"
if backend() == "sqlite"
else get_select_star(tmp_table, query.limit)
)
assert query.executed_sql == expected
assert QUERY == query.sql
assert query.rows == (1 if db_backend == "presto" else 0)
assert query.rows == (1 if backend() == "presto" else 0)
assert query.limit == 50000
assert query.select_as_cta
assert query.select_as_cta_used
@@ -1769,52 +1769,6 @@ class TestChartApi(ApiOwnersTestCaseMixin, InsertChartMixin, SupersetTestCase):
if slice:
assert data["slice_id"] == slice.id
@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
def test_query_form_data_no_data_access(self):
"""
Chart API: query_form_data must refuse callers without
datasource_access on the chart's underlying dataset. Mirrors
the existing test_get_chart_no_data_access guard on
ChartRestApi (which also returns 404 to avoid leaking chart
existence to unauthorised callers).
"""
self.login(GAMMA_USERNAME)
chart_no_access = (
db.session.query(Slice)
.filter_by(slice_name="Girl Name Cloud")
.one_or_none()
)
assert chart_no_access is not None, (
"fixture load_birth_names_dashboard_with_slices did not "
"create the 'Girl Name Cloud' slice"
)
uri = f"api/v1/form_data/?slice_id={chart_no_access.id}"
rv = self.client.get(uri)
# Match ChartRestApi.get: 404 for both missing AND forbidden so
# the endpoint cannot be used to enumerate chart IDs.
assert rv.status_code == 404, (
f"Gamma user without datasource_access should get 404 "
f"(status={rv.status_code}, body={rv.data[:200]!r})"
)
# Defence in depth: even if a future regression returns a non-
# 200 status with a partially-filled error envelope, ensure the
# caller cannot recover form_data fields.
assert b"datasource" not in rv.data
assert b"adhoc_filters" not in rv.data
assert b"viz_type" not in rv.data
def test_query_form_data_missing_slice(self):
"""
Chart API: a non-existent slice_id must return the same 404 as a
forbidden one, so the status code cannot be used to enumerate
which slice IDs exist.
"""
self.login(ADMIN_USERNAME)
max_id = db.session.query(func.max(Slice.id)).scalar() or 0
uri = f"api/v1/form_data/?slice_id={max_id + 10_000}"
rv = self.client.get(uri)
assert rv.status_code == 404
@pytest.mark.usefixtures(
"load_unicode_dashboard_with_slice",
"load_energy_table_with_slice",
@@ -676,22 +676,13 @@ class TestFavoriteChartCommand(SupersetTestCase):
# Assert that the chart exists
assert example_chart is not None
# Grant gamma read access to the datasource so the access check passes.
# Faving requires datasource access but not ownership.
if example_chart.datasource:
self.grant_role_access_to_table(example_chart.datasource, "Gamma")
with override_user(security_manager.find_user("gamma")):
AddFavoriteChartCommand(example_chart.id).run()
ids = ChartDAO.favorited_ids([example_chart])
try:
with override_user(security_manager.find_user("gamma")):
AddFavoriteChartCommand(example_chart.id).run()
ids = ChartDAO.favorited_ids([example_chart])
assert example_chart.id in ids
assert example_chart.id in ids
DelFavoriteChartCommand(example_chart.id).run()
ids = ChartDAO.favorited_ids([example_chart])
DelFavoriteChartCommand(example_chart.id).run()
ids = ChartDAO.favorited_ids([example_chart])
assert example_chart.id not in ids
finally:
if example_chart.datasource:
self.revoke_role_access_to_table("Gamma", example_chart.datasource)
assert example_chart.id not in ids
@@ -809,75 +809,6 @@ def test_base_dao_list_search(user_with_data: Session) -> None:
user_with_data.commit()
def test_base_dao_list_search_wildcard_injection(user_with_data: Session) -> None:
"""Verify that search='%' is treated literally and does not match all rows."""
users = []
for i in range(3):
user = User(
id=420 + i,
username=f"nowildcard_{i}",
first_name=f"NoWild{i}",
last_name="Card",
email=f"nowild{i}@example.com",
active=True,
)
users.append(user)
user_with_data.add(user)
user_with_data.commit()
# '%' must be escaped; it should not match any of our plain-name users
results, total = UserDAO.list(search="%", search_columns=["username", "first_name"])
result_usernames = {r.username for r in results}
for user in users:
assert user.username not in result_usernames, (
f"search='%' should not match '{user.username}' — LIKE wildcard injection"
)
for user in users:
user_with_data.delete(user)
user_with_data.commit()
def test_base_dao_list_column_operator_wildcard_injection(
user_with_data: Session,
) -> None:
"""Verify that column_operators with '%' values are treated literally."""
users = []
for i in range(3):
user = User(
id=430 + i,
username=f"opwildcard_{i}",
first_name=f"OpWild{i}",
last_name="Card",
email=f"opwild{i}@example.com",
active=True,
)
users.append(user)
user_with_data.add(user)
user_with_data.commit()
# Each wildcard operator with '%' as the value must not match plain-name rows
for opr in (
ColumnOperatorEnum.ct,
ColumnOperatorEnum.like,
ColumnOperatorEnum.ilike,
ColumnOperatorEnum.sw,
ColumnOperatorEnum.ew,
):
results, _ = UserDAO.list(
column_operators=[ColumnOperator(col="username", opr=opr, value="%")]
)
result_usernames = {r.username for r in results}
for user in users:
assert user.username not in result_usernames, (
f"opr={opr!r} with value='%' should not match '{user.username}'"
)
for user in users:
user_with_data.delete(user)
user_with_data.commit()
def test_base_dao_list_custom_filter(user_with_data: Session) -> None:
"""Test BaseDAO.list with custom filters."""
# Create users with specific attributes
@@ -295,7 +295,6 @@ class TestDashboardApi(ApiOwnersTestCaseMixin, InsertChartMixin, SupersetTestCas
assert actual_dataset_ids == expected_dataset_ids
expected_values = [0, 1] if backend() == "presto" else [0, 1, 2]
assert result[0]["column_types"] == expected_values
assert "sql" in result[0]
logger_mock.warning.assert_not_called()
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@@ -316,31 +315,6 @@ class TestDashboardApi(ApiOwnersTestCaseMixin, InsertChartMixin, SupersetTestCas
for excluded_key in ["database", "owners"]:
assert excluded_key not in dataset
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@patch("superset.dashboards.api.security_manager.can_access_datasource")
def test_get_dashboard_datasets_strips_definition_without_datasource_access(
self, can_access_datasource_mock
):
can_access_datasource_mock.return_value = False
self.login(ADMIN_USERNAME)
uri = "api/v1/dashboard/world_health/datasets"
response = self.get_assert_metric(uri, "get_datasets")
assert response.status_code == 200
data = json.loads(response.data.decode("utf-8"))
for dataset in data["result"]:
for excluded_key in [
"sql",
"select_star",
"fetch_values_predicate",
"template_params",
"params",
]:
assert excluded_key not in dataset
for column in dataset.get("columns") or []:
assert "expression" not in column
for metric in dataset.get("metrics") or []:
assert "expression" not in metric
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@patch("superset.utils.log.logger")
def test_get_dashboard_datasets_not_found(self, logger_mock):
@@ -569,14 +569,15 @@ class TestCreateDatasetCommand(SupersetTestCase):
with self.assertRaises(DatasetInvalidError): # noqa: PT027
CreateDatasetCommand({"table_name": "table", "database": 9999}).run()
@patch("superset.commands.utils.g")
@patch("superset.models.core.Database.get_table")
def test_get_table_from_database_error(self, get_table_mock):
def test_get_table_from_database_error(self, get_table_mock, mock_g):
get_table_mock.side_effect = SQLAlchemyError
with override_user(security_manager.find_user("admin")):
with self.assertRaises(DatasetInvalidError): # noqa: PT027
CreateDatasetCommand(
{"table_name": "table", "database": get_example_database().id}
).run()
mock_g.user = security_manager.find_user("admin")
with self.assertRaises(DatasetInvalidError): # noqa: PT027
CreateDatasetCommand(
{"table_name": "table", "database": get_example_database().id}
).run()
def test_create_dataset_command(self):
examples_db = get_example_database()
@@ -634,10 +635,9 @@ class TestDatasetWarmUpCacheCommand(SupersetTestCase):
)
.all()
)
with override_user(security_manager.find_user("admin")):
results = DatasetWarmUpCacheCommand(
get_example_database().database_name, "birth_names", None, None
).run()
results = DatasetWarmUpCacheCommand(
get_example_database().database_name, "birth_names", None, None
).run()
assert len(results) == len(birth_charts)
for chart_result in results:
assert "chart_id" in chart_result
@@ -39,7 +39,6 @@ from superset.models.dashboard import Dashboard
from superset.models.slice import Slice # noqa: F401
from superset.tags.models import ObjectType, Tag, TaggedObject, TagType
from tests.integration_tests.base_tests import SupersetTestCase
from tests.integration_tests.constants import ADMIN_USERNAME
from tests.integration_tests.fixtures.importexport import (
chart_config, # noqa: F401
dashboard_config, # noqa: F401
@@ -63,7 +62,6 @@ class TestCreateCustomTagCommand(SupersetTestCase):
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@pytest.mark.usefixtures("with_tagging_system_feature")
def test_create_custom_tag_command(self):
self.login(ADMIN_USERNAME)
example_dashboard = (
db.session.query(Dashboard).filter_by(slug="world_health").one()
)
@@ -98,7 +96,6 @@ class TestDeleteTagsCommand(SupersetTestCase):
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@pytest.mark.usefixtures("with_tagging_system_feature")
def test_delete_tags_command(self):
self.login(ADMIN_USERNAME)
example_dashboard = (
db.session.query(Dashboard)
.filter_by(dashboard_title="World Bank's Data")
@@ -133,7 +130,6 @@ class TestDeleteTaggedObjectCommand(SupersetTestCase):
@pytest.mark.usefixtures("load_world_bank_dashboard_with_slices")
@pytest.mark.usefixtures("with_tagging_system_feature")
def test_delete_tags_command(self):
self.login(ADMIN_USERNAME)
# create tagged objects
example_dashboard = (
db.session.query(Dashboard).filter_by(slug="world_health").one()
@@ -1,147 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Unit tests for per-object datasource access checks in chart create/update."""
from unittest.mock import MagicMock, patch
import pytest
from superset.commands.chart.exceptions import ChartForbiddenError
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import SupersetSecurityException
def _security_exception() -> SupersetSecurityException:
return SupersetSecurityException(
SupersetError(
error_type=SupersetErrorType.DATASOURCE_SECURITY_ACCESS_ERROR,
message="Access denied",
level=ErrorLevel.ERROR,
)
)
# ---------------------------------------------------------------------------
# CreateChartCommand
# ---------------------------------------------------------------------------
def test_create_chart_command_forbidden_when_no_datasource_access() -> None:
"""CreateChartCommand.validate() must raise ChartForbiddenError when the
caller lacks access to the chart's datasource."""
from superset.commands.chart.create import CreateChartCommand
with patch(
"superset.commands.chart.create.get_datasource_by_id",
return_value=MagicMock(name="datasource"),
):
with patch(
"superset.commands.chart.create.security_manager.raise_for_access",
side_effect=_security_exception(),
):
with patch(
"superset.commands.chart.create.CreateChartCommand.populate_owners",
return_value=[],
):
command = CreateChartCommand(
{
"slice_name": "test",
"viz_type": "bar",
"datasource_id": 1,
"datasource_type": "table",
}
)
with pytest.raises(ChartForbiddenError):
command.validate()
def test_create_chart_command_allowed_when_access_passes() -> None:
"""CreateChartCommand.validate() must not raise when the caller has access."""
from superset.commands.chart.create import CreateChartCommand
mock_datasource = MagicMock()
mock_datasource.name = "test_table"
with patch(
"superset.commands.chart.create.get_datasource_by_id",
return_value=mock_datasource,
):
with patch("superset.commands.chart.create.security_manager.raise_for_access"):
with patch(
"superset.commands.chart.create.CreateChartCommand.populate_owners",
return_value=[],
):
with patch(
"superset.commands.chart.create.DashboardDAO.find_by_ids",
return_value=[],
):
command = CreateChartCommand(
{
"slice_name": "test",
"viz_type": "bar",
"datasource_id": 1,
"datasource_type": "table",
}
)
command.validate() # should not raise
# ---------------------------------------------------------------------------
# UpdateChartCommand
# ---------------------------------------------------------------------------
def test_update_chart_command_forbidden_when_no_datasource_access() -> None:
"""UpdateChartCommand.validate() must raise ChartForbiddenError when the
caller lacks access to the new datasource."""
from superset.commands.chart.update import UpdateChartCommand
mock_chart = MagicMock()
mock_chart.id = 1
mock_chart.owners = []
mock_chart.dashboards = []
mock_chart.tags = []
with patch(
"superset.commands.chart.update.ChartDAO.find_by_id",
return_value=mock_chart,
):
with patch(
"superset.commands.chart.update.security_manager.raise_for_ownership"
):
with patch(
"superset.commands.chart.update.UpdateChartCommand.compute_owners",
return_value=[],
):
with patch("superset.commands.chart.update.validate_tags"):
with patch(
"superset.commands.chart.update.get_datasource_by_id",
return_value=MagicMock(name="datasource"),
):
with patch(
"superset.commands.chart.update.security_manager.raise_for_access",
side_effect=_security_exception(),
):
command = UpdateChartCommand(
1,
{
"datasource_id": 2,
"datasource_type": "table",
},
)
with pytest.raises(ChartForbiddenError):
command.validate()
@@ -22,12 +22,6 @@ from superset.commands.chart.warm_up_cache import ChartWarmUpCacheCommand
from superset.models.slice import Slice
@pytest.fixture(autouse=True)
def mock_security_manager():
with patch("superset.commands.chart.warm_up_cache.security_manager"):
yield
@patch("superset.commands.chart.warm_up_cache.get_dashboard_extra_filters")
@patch("superset.commands.chart.warm_up_cache.ChartDataCommand")
def test_applies_dashboard_filters_to_non_legacy_chart(
@@ -26,7 +26,7 @@ from superset.exceptions import SupersetParseError
from superset.models.core import Database
def test_create_dataset_invalid_sql_parse_error() -> None:
def test_create_dataset_invalid_sql_parse_error():
"""Test that invalid SQL returns a 4xx error when caught as SupersetParseError."""
mock_database = Mock(spec=Database)
mock_database.id = 1
@@ -75,7 +75,7 @@ def test_create_dataset_invalid_sql_parse_error() -> None:
)
def test_create_dataset_valid_sql_with_access_error() -> None:
def test_create_dataset_valid_sql_with_access_error():
"""
Test that security exceptions work correctly
"""
@@ -136,10 +136,7 @@ def test_create_dataset_valid_sql_with_access_error() -> None:
)
@patch("superset.commands.dataset.create.security_manager")
def test_create_dataset_physical_table_no_parse_error(
mock_security_manager: Mock,
) -> None:
def test_create_dataset_physical_table_no_parse_error():
"""Test that physical tables (no SQL) don't trigger parsing."""
mock_database = Mock(spec=Database)
mock_database.id = 1
@@ -1,103 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Unit tests for per-object access check in DuplicateDatasetCommand."""
from unittest.mock import MagicMock, patch
import pytest
from superset.commands.dataset.exceptions import DatasetAccessDeniedError
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import SupersetSecurityException
def _security_exception() -> SupersetSecurityException:
return SupersetSecurityException(
SupersetError(
error_type=SupersetErrorType.DATASOURCE_SECURITY_ACCESS_ERROR,
message="Access denied to dataset",
level=ErrorLevel.ERROR,
)
)
def test_duplicate_dataset_forbidden_when_no_access() -> None:
"""DuplicateDatasetCommand.validate() must raise DatasetAccessDeniedError
when the caller lacks read access to the source dataset."""
from superset.commands.dataset.duplicate import DuplicateDatasetCommand
mock_dataset = MagicMock()
mock_dataset.id = 1
mock_dataset.kind = "virtual"
with patch(
"superset.commands.dataset.duplicate.DatasetDAO.find_by_id",
return_value=mock_dataset,
):
with patch(
"superset.commands.dataset.duplicate.security_manager.raise_for_access",
side_effect=_security_exception(),
):
with patch(
"superset.commands.dataset.duplicate.DuplicateDatasetCommand.populate_owners",
return_value=[],
):
command = DuplicateDatasetCommand(
{
"base_model_id": 1,
"table_name": "duplicate_name",
"is_managed_externally": False,
}
)
with pytest.raises(DatasetAccessDeniedError):
command.validate()
def test_duplicate_dataset_access_check_passes_through() -> None:
"""DuplicateDatasetCommand.validate() must not raise DatasetAccessDeniedError
when security_manager.raise_for_access() does not raise."""
from superset.commands.dataset.duplicate import DuplicateDatasetCommand
mock_dataset = MagicMock()
mock_dataset.id = 1
mock_dataset.kind = "virtual"
with patch(
"superset.commands.dataset.duplicate.DatasetDAO.find_by_id",
return_value=mock_dataset,
):
with patch(
"superset.commands.dataset.duplicate.security_manager.raise_for_access"
) as mock_access:
with patch(
"superset.commands.dataset.duplicate.DatasetDAO.find_one_or_none",
return_value=None,
):
with patch(
"superset.commands.dataset.duplicate.DuplicateDatasetCommand.populate_owners",
return_value=[],
):
command = DuplicateDatasetCommand(
{
"base_model_id": 1,
"table_name": "new_unique_name",
"is_managed_externally": False,
}
)
command.validate() # should not raise
# Confirm access check was called with the base dataset
mock_access.assert_called_once_with(datasource=mock_dataset)
@@ -1,115 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Unit tests for DatasetWarmUpCacheCommand access control."""
from unittest.mock import MagicMock, patch
import pytest
from superset.commands.dataset.exceptions import (
DatasetAccessDeniedError,
WarmUpCacheTableNotFoundError,
)
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
from superset.exceptions import SupersetSecurityException
def _security_exception() -> SupersetSecurityException:
return SupersetSecurityException(
SupersetError(
error_type=SupersetErrorType.DATASOURCE_SECURITY_ACCESS_ERROR,
message="Access denied to table",
level=ErrorLevel.ERROR,
)
)
def _mock_table() -> MagicMock:
table = MagicMock()
table.id = 1
table.type = "table"
return table
def test_warm_up_cache_raises_not_found_when_table_missing() -> None:
"""validate() must raise WarmUpCacheTableNotFoundError when the table
does not exist in the given database."""
from superset.commands.dataset.warm_up_cache import DatasetWarmUpCacheCommand
with patch("superset.commands.dataset.warm_up_cache.db") as mock_db:
q = mock_db.session.query.return_value
q.join.return_value.filter.return_value.one_or_none.return_value = None
command = DatasetWarmUpCacheCommand(
db_name="mydb",
table_name="nonexistent",
dashboard_id=None,
extra_filters=None,
)
with pytest.raises(WarmUpCacheTableNotFoundError):
command.validate()
def test_warm_up_cache_raises_access_denied_when_no_permission() -> None:
"""validate() must raise DatasetAccessDeniedError when the caller lacks
access to the dataset."""
from superset.commands.dataset.warm_up_cache import DatasetWarmUpCacheCommand
mock_table = _mock_table()
with patch("superset.commands.dataset.warm_up_cache.db") as mock_db:
q = mock_db.session.query.return_value
q.join.return_value.filter.return_value.one_or_none.return_value = mock_table
with patch(
"superset.commands.dataset.warm_up_cache.security_manager.raise_for_access",
side_effect=_security_exception(),
):
command = DatasetWarmUpCacheCommand(
db_name="mydb",
table_name="secret_table",
dashboard_id=None,
extra_filters=None,
)
with pytest.raises(DatasetAccessDeniedError):
command.validate()
def test_warm_up_cache_populates_charts_when_access_granted() -> None:
"""validate() must populate _charts when the caller has access."""
from superset.commands.dataset.warm_up_cache import DatasetWarmUpCacheCommand
mock_table = _mock_table()
mock_charts = [MagicMock(), MagicMock()]
with patch("superset.commands.dataset.warm_up_cache.db") as mock_db:
# First query() call returns table; second returns chart list
q = mock_db.session.query.return_value
q.join.return_value.filter.return_value.one_or_none.return_value = mock_table
q.filter_by.return_value.all.return_value = mock_charts
with patch(
"superset.commands.dataset.warm_up_cache.security_manager.raise_for_access"
):
command = DatasetWarmUpCacheCommand(
db_name="mydb",
table_name="allowed_table",
dashboard_id=None,
extra_filters=None,
)
command.validate()
assert command._charts == mock_charts
@@ -240,166 +240,6 @@ def test_update_dataset_validation_errors(
assert any(error_msg in str(exc) for exc in excinfo.value._exceptions)
@pytest.mark.parametrize(
("payload", "field"),
[
(
{
"columns": [
{
"column_name": "evil_col",
"expression": "1; DROP TABLE users",
}
]
},
"columns.0.expression",
),
(
{
"metrics": [
{
"metric_name": "evil_metric",
"expression": "1 UNION SELECT password FROM ab_user",
}
]
},
"metrics.0.expression",
),
],
)
def test_update_dataset_rejects_malicious_expression(
payload: dict[str, Any],
field: str,
mocker: MockerFixture,
) -> None:
"""
Stored column and metric ``expression`` strings are routed through the
same validator as adhoc SQL fields, and command-level validation
surfaces the parser's verdict as a field-level ``ValidationError``.
"""
mock_dataset_dao = mocker.patch("superset.commands.dataset.update.DatasetDAO")
mocker.patch(
"superset.commands.dataset.update.security_manager.raise_for_ownership",
)
mocker.patch("superset.commands.utils.security_manager.is_admin", return_value=True)
mocker.patch(
"superset.commands.utils.security_manager.get_user_by_id", return_value=None
)
mock_database = mocker.MagicMock()
mock_database.id = 1
mock_database.backend = "sqlite"
mock_database.allow_multi_catalog = False
mock_database.get_default_catalog.return_value = "catalog"
mock_dataset = mocker.MagicMock()
mock_dataset.database = mock_database
mock_dataset.catalog = "catalog"
mock_dataset.schema = None
mock_dataset_dao.find_by_id.return_value = mock_dataset
mock_dataset_dao.get_database_by_id.return_value = mock_database
mock_dataset_dao.validate_update_uniqueness.return_value = True
mock_dataset_dao.validate_columns_exist.return_value = True
mock_dataset_dao.validate_columns_uniqueness.return_value = True
mock_dataset_dao.validate_metrics_exist.return_value = True
mock_dataset_dao.validate_metrics_uniqueness.return_value = True
with pytest.raises(DatasetInvalidError) as excinfo:
UpdateDatasetCommand(1, payload).run()
expression_errors = [
exc
for exc in excinfo.value._exceptions
if isinstance(exc, ValidationError) and field in (exc.field_name or "")
]
assert expression_errors, (
f"Expected a field-level ValidationError on '{field}'. Got: "
f"{[(type(e).__name__, getattr(e, 'field_name', None), str(e)) for e in excinfo.value._exceptions]}" # noqa: E501
)
def test_update_dataset_accepts_benign_expression(mocker: MockerFixture) -> None:
"""
A well-formed stored expression (e.g. CASE) passes the validator: the
command's ``validate()`` collects no expression-level errors. We
invoke ``validate()`` directly to isolate the validator from the
rest of the run-path (commit, audit, etc.).
"""
mock_dataset_dao = mocker.patch("superset.commands.dataset.update.DatasetDAO")
mocker.patch(
"superset.commands.dataset.update.security_manager.raise_for_ownership",
)
mocker.patch("superset.commands.utils.security_manager.is_admin", return_value=True)
mocker.patch(
"superset.commands.utils.security_manager.get_user_by_id", return_value=None
)
mock_database = mocker.MagicMock()
mock_database.id = 1
mock_database.backend = "sqlite"
mock_database.allow_multi_catalog = False
mock_database.get_default_catalog.return_value = "catalog"
mock_dataset = mocker.MagicMock()
mock_dataset.database = mock_database
mock_dataset.catalog = "catalog"
mock_dataset.schema = None
mock_dataset_dao.find_by_id.return_value = mock_dataset
mock_dataset_dao.get_database_by_id.return_value = mock_database
mock_dataset_dao.validate_update_uniqueness.return_value = True
mock_dataset_dao.validate_columns_exist.return_value = True
mock_dataset_dao.validate_columns_uniqueness.return_value = True
payload = {
"columns": [
{
"column_name": "case_col",
"expression": "CASE WHEN amount > 0 THEN 'a' ELSE 'b' END",
}
]
}
UpdateDatasetCommand(1, payload).validate()
def test_update_dataset_accepts_jinja_expression(mocker: MockerFixture) -> None:
"""
Stored column/metric expressions can use Jinja templating (e.g.
``{{ current_username() }}``). At save time there is no template
context, so the parser-based gate is bypassed; the same validator
re-runs on the rendered SQL at query time.
"""
mock_dataset_dao = mocker.patch("superset.commands.dataset.update.DatasetDAO")
mocker.patch(
"superset.commands.dataset.update.security_manager.raise_for_ownership",
)
mocker.patch("superset.commands.utils.security_manager.is_admin", return_value=True)
mocker.patch(
"superset.commands.utils.security_manager.get_user_by_id", return_value=None
)
mock_database = mocker.MagicMock()
mock_database.id = 1
mock_database.backend = "sqlite"
mock_database.allow_multi_catalog = False
mock_database.get_default_catalog.return_value = "catalog"
mock_dataset = mocker.MagicMock()
mock_dataset.database = mock_database
mock_dataset.catalog = "catalog"
mock_dataset.schema = None
mock_dataset_dao.find_by_id.return_value = mock_dataset
mock_dataset_dao.get_database_by_id.return_value = mock_database
mock_dataset_dao.validate_update_uniqueness.return_value = True
mock_dataset_dao.validate_columns_exist.return_value = True
mock_dataset_dao.validate_columns_uniqueness.return_value = True
payload = {
"columns": [
{
"column_name": "user_match",
"expression": (
"case when '{{ current_username() }}' = 'abc' "
"then 'yes' else 'no' end"
),
}
]
}
UpdateDatasetCommand(1, payload).validate()
@with_feature_flags(DATASET_FOLDERS=True)
def test_validate_folders(mocker: MockerFixture) -> None:
"""
@@ -17,8 +17,6 @@
from unittest.mock import MagicMock, patch
from marshmallow import ValidationError
from superset.commands.report.create import CreateReportScheduleCommand
from superset.commands.report.exceptions import ReportScheduleUserEmailNotFoundError
from superset.reports.models import (
@@ -47,7 +45,7 @@ def test_populate_recipients_chart_creation_with_user_email() -> None:
],
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that recipients were overridden
@@ -75,7 +73,7 @@ def test_populate_recipients_dashboard_creation_with_user_email() -> None:
# No recipients provided initially
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that recipients were set
@@ -106,7 +104,7 @@ def test_populate_recipients_alerts_reports_keeps_original() -> None:
"recipients": original_recipients,
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that recipients were NOT changed
@@ -127,7 +125,7 @@ def test_populate_recipients_chart_creation_no_user_email() -> None:
"creation_method": ReportCreationMethod.CHARTS,
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that validation error was added
@@ -151,7 +149,7 @@ def test_populate_recipients_dashboard_creation_no_user() -> None:
"creation_method": ReportCreationMethod.DASHBOARDS,
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that validation error was added
@@ -173,7 +171,7 @@ def test_populate_recipients_no_creation_method() -> None:
"recipients": original_recipients,
}
exceptions: list[ValidationError] = []
exceptions: list[Exception] = []
command._populate_recipients(exceptions)
# Check that recipients were NOT changed
+25
View File
@@ -312,3 +312,28 @@ def test_full_setting(
assert dttm_col.is_dttm
assert dttm_col.python_date_format == "epoch_s"
assert dttm_col.expression == "CAST(dttm as INTEGER)"
def test_expose_version_info_defaults_to_true() -> None:
"""
The /version endpoint preserves its existing behavior by default. Operators
can set EXPOSE_VERSION_INFO = False to omit build-specific details.
"""
from superset import config
assert config.EXPOSE_VERSION_INFO is True
def test_default_cross_origin_resource_policy_header() -> None:
"""
Superset ships a conservative `Cross-Origin-Resource-Policy: same-site`
default through DEFAULT_HTTP_HEADERS. `same-site` (rather than `same-origin`)
is chosen so documented same-site embedding flows, such as the Embedded SDK,
keep working while still providing a defense-in-depth default that operators
can override.
"""
from superset import config
assert (
config.DEFAULT_HTTP_HEADERS.get("Cross-Origin-Resource-Policy") == "same-site"
)
+3 -234
View File
@@ -22,19 +22,10 @@ from sqlalchemy import create_engine
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm.session import Session
from superset.connectors.sqla.models import (
SqlaTable,
TableColumn,
validate_stored_expression,
)
from superset.connectors.sqla.models import SqlaTable, TableColumn
from superset.daos.dataset import DatasetDAO
from superset.daos.exceptions import DatasourceNotFound
from superset.exceptions import (
OAuth2RedirectError,
SupersetDisallowedSQLFunctionException,
SupersetDisallowedSQLTableException,
SupersetSecurityException,
)
from superset.exceptions import OAuth2RedirectError
from superset.models.core import Database
from superset.sql.parse import Table
from superset.superset_typing import QueryObjectDict
@@ -83,116 +74,6 @@ def test_query_bubbles_errors(mocker: MockerFixture) -> None:
sqla_table.query(query_obj)
def _query_obj() -> QueryObjectDict:
return {
"granularity": None,
"from_dttm": None,
"to_dttm": None,
"groupby": ["id"],
"metrics": [],
"is_timeseries": False,
"filter": [],
}
def _build_sqla_table_for_query(
mocker: MockerFixture, sql: str, engine: str = "postgresql"
) -> SqlaTable:
db_engine_spec = mocker.MagicMock()
db_engine_spec.engine = engine
database = mocker.MagicMock()
database.db_engine_spec = db_engine_spec
sqla_table = SqlaTable(
table_name="my_sqla_table",
columns=[],
metrics=[],
database=database,
)
mocker.patch.object(
SqlaTable,
"db_engine_spec",
new=property(lambda self: db_engine_spec),
)
mocker.patch.object(
sqla_table,
"get_query_str_extended",
return_value=mocker.MagicMock(sql=sql, labels_expected=[]),
)
return sqla_table
def test_query_blocks_disallowed_function_on_chart_data_path(
mocker: MockerFixture,
) -> None:
mocker.patch.dict(
"flask.current_app.config",
{
"DISALLOWED_SQL_FUNCTIONS": {"postgresql": {"version"}},
"DISALLOWED_SQL_TABLES": {},
},
clear=False,
)
sqla_table = _build_sqla_table_for_query(mocker, "SELECT version()")
with pytest.raises(SupersetDisallowedSQLFunctionException):
sqla_table.query(_query_obj())
sqla_table.database.get_df.assert_not_called() # type: ignore[attr-defined]
def test_query_blocks_disallowed_table_on_chart_data_path(
mocker: MockerFixture,
) -> None:
mocker.patch.dict(
"flask.current_app.config",
{
"DISALLOWED_SQL_FUNCTIONS": {},
"DISALLOWED_SQL_TABLES": {"postgresql": {"pg_authid"}},
},
clear=False,
)
sqla_table = _build_sqla_table_for_query(mocker, "SELECT rolname FROM pg_authid")
with pytest.raises(SupersetDisallowedSQLTableException):
sqla_table.query(_query_obj())
sqla_table.database.get_df.assert_not_called() # type: ignore[attr-defined]
def test_query_disallowed_table_error_reports_only_matched_tables(
mocker: MockerFixture,
) -> None:
mocker.patch.dict(
"flask.current_app.config",
{
"DISALLOWED_SQL_FUNCTIONS": {},
"DISALLOWED_SQL_TABLES": {
"postgresql": {"pg_authid", "pg_shadow", "pg_stat_activity"}
},
},
clear=False,
)
sqla_table = _build_sqla_table_for_query(mocker, "SELECT rolname FROM pg_authid")
with pytest.raises(SupersetDisallowedSQLTableException) as excinfo:
sqla_table.query(_query_obj())
message = str(excinfo.value)
assert "pg_authid" in message
assert "pg_shadow" not in message
assert "pg_stat_activity" not in message
def test_query_allows_benign_sql_on_chart_data_path(mocker: MockerFixture) -> None:
mocker.patch.dict(
"flask.current_app.config",
{
"DISALLOWED_SQL_FUNCTIONS": {"postgresql": {"version"}},
"DISALLOWED_SQL_TABLES": {"postgresql": {"pg_authid"}},
},
clear=False,
)
sqla_table = _build_sqla_table_for_query(mocker, "SELECT id FROM my_sqla_table")
sqla_table.database.get_df.return_value = pd.DataFrame() # type: ignore[attr-defined]
result = sqla_table.query(_query_obj())
sqla_table.database.get_df.assert_called_once() # type: ignore[attr-defined]
assert result is not None
def test_permissions_without_catalog() -> None:
"""
Test permissions when the table has no catalog.
@@ -388,7 +269,7 @@ def test_dataset_uniqueness(session: Session) -> None:
def test_normalize_prequery_result_type_custom_sql() -> None:
"""
Test that the `_normalize_prequery_result_type` can handle custom SQL.
Test that the `_normalize_prequery_result_type` can hanndle custom SQL.
"""
sqla_table = SqlaTable(
table_name="my_sqla_table",
@@ -1096,115 +977,3 @@ def test_owners_data_includes_email(mocker: MockerFixture) -> None:
"id": 1,
"email": "john@example.com",
}
def _database_for_expression(mocker: MockerFixture) -> Database:
database = mocker.MagicMock(spec=Database)
database.backend = "sqlite"
database.allow_multi_catalog = False
return database
def test_validate_stored_expression_rejects_multi_statement(
mocker: MockerFixture,
) -> None:
database = _database_for_expression(mocker)
with pytest.raises(SupersetSecurityException):
validate_stored_expression(database, None, None, "1; DROP TABLE users")
def test_validate_stored_expression_rejects_set_operation(
mocker: MockerFixture,
) -> None:
database = _database_for_expression(mocker)
with pytest.raises(SupersetSecurityException):
validate_stored_expression(
database, None, None, "1 UNION SELECT password FROM ab_user"
)
def test_validate_stored_expression_accepts_case_expression(
mocker: MockerFixture,
) -> None:
database = _database_for_expression(mocker)
validate_stored_expression(
database, None, None, "CASE WHEN amount > 0 THEN 'a' ELSE 'b' END"
)
def test_validate_stored_expression_rejects_subquery(
mocker: MockerFixture,
) -> None:
"""
With ``ALLOW_ADHOC_SUBQUERY=False`` (the default), a stored
expression that contains a sub-query is rejected by the same
``validate_adhoc_subquery`` gate that already covers adhoc SQL.
Locks in the sub-query branch so a future refactor that
removes the ``validate_adhoc_subquery`` call gets a red test.
"""
database = _database_for_expression(mocker)
mocker.patch("superset.models.helpers.is_feature_enabled", return_value=False)
with pytest.raises(SupersetSecurityException):
validate_stored_expression(
database,
None,
None,
"(SELECT password FROM ab_user LIMIT 1)",
)
@pytest.mark.parametrize(
"expression",
[
"case when '{{ current_username() }}' = 'abc' then 'yes' else 'no' end",
"SUM(price) * {{ url_param('multiplier') }}",
"{# comment #} amount",
"{% if 1 %}amount{% endif %}",
],
)
def test_validate_stored_expression_accepts_jinja(
mocker: MockerFixture, expression: str
) -> None:
"""
Stored expressions can contain Jinja templating. Balanced Jinja blocks
are replaced with a placeholder so the surrounding SQL is still parsed;
skeletons whose control flow leaves them unparseable defer to runtime.
"""
database = _database_for_expression(mocker)
validate_stored_expression(database, None, None, expression)
def test_validate_stored_expression_rejects_set_op_around_jinja(
mocker: MockerFixture,
) -> None:
"""
A ``UNION`` smuggled around a Jinja block must still be rejected: the
Jinja substitution leaves the set operator visible to the parser.
"""
database = _database_for_expression(mocker)
with pytest.raises(SupersetSecurityException):
validate_stored_expression(
database,
None,
None,
"'{{ current_username() }}' UNION SELECT password FROM ab_user",
)
def test_validate_stored_expression_rejects_subquery_around_jinja(
mocker: MockerFixture,
) -> None:
"""
Sub-queries combined with a Jinja comment block must still be rejected:
stripping the ``{# ... #}`` block leaves the sub-query visible to the
``validate_adhoc_subquery`` gate.
"""
database = _database_for_expression(mocker)
mocker.patch("superset.models.helpers.is_feature_enabled", return_value=False)
with pytest.raises(SupersetSecurityException):
validate_stored_expression(
database,
None,
None,
"(SELECT password FROM ab_user LIMIT 1) {# x #}",
)
@@ -1283,59 +1283,3 @@ def test_start_oauth2_dance_falls_back_to_url_for(mocker: MockerFixture) -> None
error = exc_info.value.error
assert error.extra["redirect_uri"] == fallback_uri
def test_get_table_names_strips_schema_with_regex_metacharacters(
mocker: MockerFixture,
) -> None:
"""
Test that get_table_names strips a schema prefix containing regex
metacharacters without raising and without mangling unrelated names.
"""
schema = "a.b(c)"
inspector = mocker.MagicMock()
inspector.get_table_names.return_value = [
f"{schema}.orders",
"axbc.other",
]
database = mocker.MagicMock()
spec = BaseEngineSpec
mocker.patch.object(spec, "try_remove_schema_from_table_name", True)
tables = spec.get_table_names(database, inspector, schema)
# The real schema prefix is stripped; the look-alike name is left intact
# because the metacharacters are escaped before being used as a regex.
# "axbc.other" would match the old unescaped pattern ^a.b(c)\. and be
# incorrectly stripped — the escaped version correctly preserves it.
assert tables == {"orders", "axbc.other"}
def test_get_view_names_strips_schema_with_regex_metacharacters(
mocker: MockerFixture,
) -> None:
"""
Test that get_view_names strips a schema prefix containing regex
metacharacters without raising and without mangling unrelated names.
"""
schema = "a.b(c)"
inspector = mocker.MagicMock()
inspector.get_view_names.return_value = [
f"{schema}.report",
"axbc.other",
]
database = mocker.MagicMock()
spec = BaseEngineSpec
mocker.patch.object(spec, "try_remove_schema_from_table_name", True)
views = spec.get_view_names(database, inspector, schema)
# "axbc.other" would match the old unescaped pattern ^a.b(c)\. and be
# incorrectly stripped — the escaped version correctly preserves it.
assert views == {"report", "axbc.other"}
-118
View File
@@ -354,124 +354,6 @@ def test_filter_values_query_context_filters() -> None:
assert cache.applied_filters == ["name"]
def test_get_filters_escaped_val_string_adhoc() -> None:
"""
``get_filters`` exposes an ``escaped_val`` companion for string values
when a dialect is configured, while leaving ``val`` raw.
"""
with current_app.test_request_context(
data={
"form_data": json.dumps(
{
"adhoc_filters": [
{
"clause": "WHERE",
"comparator": "O'Brien",
"expressionType": "SIMPLE",
"operator": "LIKE",
"subject": "name",
}
],
}
)
}
):
cache = ExtraCache(dialect=dialect())
result = cache.get_filters("name")
assert result == [
{
"op": "LIKE",
"col": "name",
"val": "O'Brien",
"escaped_val": "O''Brien",
}
]
def test_get_filters_escaped_val_list_adhoc() -> None:
"""
``get_filters`` produces an ``escaped_val`` list with every string
member dialect-escaped; non-string members pass through untouched.
"""
with current_app.test_request_context(
data={
"form_data": json.dumps(
{
"adhoc_filters": [
{
"clause": "WHERE",
"comparator": ["O'Brien", "Smith"],
"expressionType": "SIMPLE",
"operator": "in",
"subject": "name",
}
],
}
)
}
):
cache = ExtraCache(dialect=dialect())
result = cache.get_filters("name")
assert result == [
{
"op": "IN",
"col": "name",
"val": ["O'Brien", "Smith"],
"escaped_val": ["O''Brien", "Smith"],
}
]
def test_get_filters_escaped_val_query_context_filters() -> None:
"""
The ``escaped_val`` companion is also populated when filters arrive via
the drill-to-detail ``query_context_filters`` path.
"""
cache = ExtraCache(
dialect=dialect(),
query_context_filters=[
{"col": "name", "op": "LIKE", "val": "O'Brien"},
],
)
assert cache.get_filters("name") == [
{
"op": "LIKE",
"col": "name",
"val": "O'Brien",
"escaped_val": "O''Brien",
}
]
def test_get_filters_no_escaped_val_without_dialect() -> None:
"""
Without a dialect ``get_filters`` returns the original schema, with no
``escaped_val`` key preserving backwards compatibility for callers
that did not opt into a dialect-aware processor.
"""
with current_app.test_request_context(
data={
"form_data": json.dumps(
{
"adhoc_filters": [
{
"clause": "WHERE",
"comparator": "O'Brien",
"expressionType": "SIMPLE",
"operator": "LIKE",
"subject": "name",
}
],
}
)
}
):
cache = ExtraCache()
result = cache.get_filters("name")
assert result == [{"op": "LIKE", "col": "name", "val": "O'Brien"}]
assert "escaped_val" not in result[0]
def test_url_param_query() -> None:
"""
Test the ``url_param`` macro.
@@ -18,9 +18,6 @@
from unittest.mock import MagicMock, patch
from superset.mcp_service.chart.chart_helpers import (
_deck_gl_null_filters,
_is_metric_ref,
_resolve_deck_gl_metrics,
apply_form_data_filters_to_query,
build_query_dicts_from_form_data,
extract_form_data_key_from_url,
@@ -29,7 +26,6 @@ from superset.mcp_service.chart.chart_helpers import (
merge_extra_form_data_filters_into_query,
merge_form_data_filters_into_query,
prepare_form_data_for_query,
resolve_deck_gl_columns,
)
@@ -289,650 +285,3 @@ def test_merge_extra_form_data_filters_into_query_adds_only_extra_predicates(
assert query["time_range"] == "No filter"
assert query["granularity"] == "updated_at"
assert query["time_grain_sqla"] == "P1D"
# ---------------------------------------------------------------------------
# resolve_deck_gl_columns
# ---------------------------------------------------------------------------
def test_resolve_deck_gl_columns_latlong():
form_data = {
"spatial": {"type": "latlong", "lonCol": "longitude", "latCol": "latitude"},
}
assert resolve_deck_gl_columns(form_data) == ["longitude", "latitude"]
def test_resolve_deck_gl_columns_delimited():
form_data = {
"spatial": {"type": "delimited", "lonlatCol": "coords"},
}
assert resolve_deck_gl_columns(form_data) == ["coords"]
def test_resolve_deck_gl_columns_geohash():
form_data = {
"spatial": {"type": "geohash", "geohashCol": "geo"},
}
assert resolve_deck_gl_columns(form_data) == ["geo"]
def test_resolve_deck_gl_columns_arc_start_end():
form_data = {
"start_spatial": {
"type": "latlong",
"lonCol": "start_lon",
"latCol": "start_lat",
},
"end_spatial": {"type": "latlong", "lonCol": "end_lon", "latCol": "end_lat"},
}
cols = resolve_deck_gl_columns(form_data)
assert cols == ["start_lon", "start_lat", "end_lon", "end_lat"]
def test_resolve_deck_gl_columns_path_line_column():
form_data = {
"line_column": "path_wkt",
}
assert resolve_deck_gl_columns(form_data) == ["path_wkt"]
def test_resolve_deck_gl_columns_geojson():
form_data = {
"geojson": "geom_col",
}
assert resolve_deck_gl_columns(form_data) == ["geom_col"]
def test_resolve_deck_gl_columns_with_dimension_and_js_columns():
form_data = {
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"dimension": "category",
"js_columns": ["name", "value"],
}
cols = resolve_deck_gl_columns(form_data)
assert "lon" in cols
assert "lat" in cols
assert "category" in cols
assert "name" in cols
assert "value" in cols
def test_resolve_deck_gl_columns_deduplicates():
form_data = {
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"dimension": "lon", # same as lonCol — should not duplicate
}
cols = resolve_deck_gl_columns(form_data)
assert cols.count("lon") == 1
def test_resolve_deck_gl_columns_empty():
assert resolve_deck_gl_columns({}) == []
def test_resolve_deck_gl_columns_ignores_non_string_js_columns():
form_data = {
"js_columns": [42, None, "valid_col"],
}
assert resolve_deck_gl_columns(form_data) == ["valid_col"]
# ---------------------------------------------------------------------------
# build_query_dicts_from_form_data — Deck.gl branch
# ---------------------------------------------------------------------------
def test_build_query_dicts_deck_scatter_latlong(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert len(queries) == 1
assert queries[0]["columns"] == ["lon", "lat"]
assert queries[0]["metrics"] == []
def test_build_query_dicts_deck_scatter_with_size_metric(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
metric = {
"expressionType": "SIMPLE",
"column": {"column_name": "sales"},
"aggregate": "SUM",
}
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"size": metric,
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert len(queries) == 1
assert queries[0]["columns"] == ["lon", "lat"]
assert queries[0]["metrics"] == [metric]
def test_build_query_dicts_deck_arc(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_arc",
"start_spatial": {
"type": "latlong",
"lonCol": "origin_lon",
"latCol": "origin_lat",
},
"end_spatial": {"type": "latlong", "lonCol": "dest_lon", "latCol": "dest_lat"},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert len(queries) == 1
assert queries[0]["columns"] == ["origin_lon", "origin_lat", "dest_lon", "dest_lat"]
assert queries[0]["metrics"] == []
def test_build_query_dicts_deck_geojson(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_geojson",
"geojson": "geometry",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert len(queries) == 1
assert queries[0]["columns"] == ["geometry"]
assert queries[0]["metrics"] == []
def test_build_query_dicts_deck_hex_geohash(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_hex",
"spatial": {"type": "geohash", "geohashCol": "geohash"},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert len(queries) == 1
assert queries[0]["columns"] == ["geohash"]
def test_build_query_dicts_deck_path_with_row_limit(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_path",
"line_column": "path_col",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table", row_limit=50)
assert queries[0]["columns"] == ["path_col"]
assert queries[0]["row_limit"] == 50
# ---------------------------------------------------------------------------
# resolve_deck_gl_columns — display-only fields excluded
# ---------------------------------------------------------------------------
def test_resolve_deck_gl_columns_ignores_tooltip_contents():
# tooltip_contents are display-only; BaseDeckGLViz.query_obj() does not
# include them in columns/groupby, so the fallback should not either.
form_data = {
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"tooltip_contents": ["name", "category"],
}
cols = resolve_deck_gl_columns(form_data)
assert "name" not in cols
assert "category" not in cols
def test_resolve_deck_gl_columns_ignores_cross_filter_column():
form_data = {
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"cross_filter_column": "region",
}
cols = resolve_deck_gl_columns(form_data)
assert "region" not in cols
# ---------------------------------------------------------------------------
# _is_metric_ref
# ---------------------------------------------------------------------------
def test_is_metric_ref_dict():
assert _is_metric_ref({"expressionType": "SIMPLE"}) is True
def test_is_metric_ref_string_key():
assert _is_metric_ref("count") is True
assert _is_metric_ref("sum__sales") is True
def test_is_metric_ref_numeric_string_excluded():
assert _is_metric_ref("100") is False
assert _is_metric_ref("3.14") is False
assert _is_metric_ref("0") is False
def test_is_metric_ref_integer_excluded():
assert _is_metric_ref(100) is False
def test_is_metric_ref_none_and_empty():
assert _is_metric_ref(None) is False
assert _is_metric_ref("") is False
# ---------------------------------------------------------------------------
# _resolve_deck_gl_metrics (Fix 2)
# ---------------------------------------------------------------------------
def test_resolve_deck_gl_metrics_no_metrics():
assert _resolve_deck_gl_metrics({}) == []
def test_resolve_deck_gl_metrics_size_field():
metric = {"expressionType": "SIMPLE", "aggregate": "COUNT", "column": None}
result = _resolve_deck_gl_metrics({"size": metric})
assert result == [metric]
def test_resolve_deck_gl_metrics_metric_field():
metric = {"expressionType": "SIMPLE", "aggregate": "SUM"}
result = _resolve_deck_gl_metrics({"metric": metric})
assert result == [metric]
def test_resolve_deck_gl_metrics_point_radius_fixed_metric():
prf_metric = {"expressionType": "SIMPLE", "aggregate": "AVG"}
prf = {"type": "metric", "value": prf_metric}
result = _resolve_deck_gl_metrics({"point_radius_fixed": prf})
assert result == [prf_metric]
def test_resolve_deck_gl_metrics_point_radius_fixed_not_metric():
prf = {"type": "fix", "value": 100}
result = _resolve_deck_gl_metrics({"point_radius_fixed": prf})
assert result == []
def test_resolve_deck_gl_metrics_polygon_both_metric_and_prf():
base_metric = {"expressionType": "SIMPLE", "aggregate": "SUM"}
elevation_metric = {"expressionType": "SIMPLE", "aggregate": "AVG"}
prf = {"type": "metric", "value": elevation_metric}
result = _resolve_deck_gl_metrics(
{"metric": base_metric, "point_radius_fixed": prf}
)
assert result == [base_metric, elevation_metric]
def test_resolve_deck_gl_metrics_geojson_returns_empty():
# deck_geojson.query_obj() forces metrics=[] regardless of form_data
metric = {"expressionType": "SIMPLE", "aggregate": "SUM"}
result = _resolve_deck_gl_metrics(
{"size": metric, "metric": metric}, "deck_geojson"
)
assert result == []
def test_resolve_deck_gl_metrics_scalar_size_excluded():
# Numeric string size values (fixed display settings) must not be metrics
result = _resolve_deck_gl_metrics({"size": "100"}, "deck_hex")
assert result == []
def test_resolve_deck_gl_metrics_integer_size_excluded():
result = _resolve_deck_gl_metrics({"size": 100}, "deck_path")
assert result == []
def test_resolve_deck_gl_metrics_string_metric_included():
# Non-numeric string metrics (saved metric keys) must be preserved
result = _resolve_deck_gl_metrics({"size": "count"}, "deck_hex")
assert result == ["count"]
def test_resolve_deck_gl_metrics_string_metric_field():
result = _resolve_deck_gl_metrics({"metric": "sum__sales"}, "deck_arc")
assert result == ["sum__sales"]
def test_resolve_deck_gl_metrics_string_point_radius_fixed():
# Legacy deck_scatter: point_radius_fixed as a bare metric key string
result = _resolve_deck_gl_metrics({"point_radius_fixed": "count"}, "deck_scatter")
assert result == ["count"]
def test_resolve_deck_gl_metrics_numeric_point_radius_fixed_excluded():
# Numeric string point_radius_fixed is a fixed pixel radius, not a metric
result = _resolve_deck_gl_metrics({"point_radius_fixed": "100"}, "deck_scatter")
assert result == []
def test_resolve_deck_gl_metrics_non_string_point_radius_fixed_excluded():
# Non-string point_radius_fixed values (int, None, list) are excluded by
# the isinstance(prf, str) guard in the elif branch
assert _resolve_deck_gl_metrics({"point_radius_fixed": 100}, "deck_scatter") == []
assert _resolve_deck_gl_metrics({"point_radius_fixed": None}, "deck_scatter") == []
assert (
_resolve_deck_gl_metrics({"point_radius_fixed": ["bad"]}, "deck_scatter") == []
)
# ---------------------------------------------------------------------------
# _deck_gl_null_filters (Fix 3)
# ---------------------------------------------------------------------------
def test_deck_gl_null_filters_latlong():
form_data = {
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
}
result = _deck_gl_null_filters(form_data)
assert result == [
{"col": "lon", "op": "IS NOT NULL", "val": ""},
{"col": "lat", "op": "IS NOT NULL", "val": ""},
]
def test_deck_gl_null_filters_arc_start_end():
form_data = {
"start_spatial": {"type": "latlong", "lonCol": "s_lon", "latCol": "s_lat"},
"end_spatial": {"type": "latlong", "lonCol": "e_lon", "latCol": "e_lat"},
}
result = _deck_gl_null_filters(form_data)
assert result == [
{"col": "s_lon", "op": "IS NOT NULL", "val": ""},
{"col": "s_lat", "op": "IS NOT NULL", "val": ""},
{"col": "e_lon", "op": "IS NOT NULL", "val": ""},
{"col": "e_lat", "op": "IS NOT NULL", "val": ""},
]
def test_deck_gl_null_filters_line_column():
form_data = {"line_column": "path_col"}
result = _deck_gl_null_filters(form_data)
assert result == [{"col": "path_col", "op": "IS NOT NULL", "val": ""}]
def test_deck_gl_null_filters_empty():
assert _deck_gl_null_filters({}) == []
def test_deck_gl_null_filters_geojson_column():
# geojson column gets an IS NOT NULL filter just like spatial columns
form_data = {"geojson": "geometry"}
assert _deck_gl_null_filters(form_data) == [
{"col": "geometry", "op": "IS NOT NULL", "val": ""}
]
# ---------------------------------------------------------------------------
# build_query_dicts_from_form_data — null filters behavior (Fix 3)
# ---------------------------------------------------------------------------
def test_build_query_dicts_deck_scatter_adds_null_filters_by_default(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert {"col": "lon", "op": "IS NOT NULL", "val": ""} in queries[0]["filters"]
assert {"col": "lat", "op": "IS NOT NULL", "val": ""} in queries[0]["filters"]
def test_build_query_dicts_deck_scatter_filter_nulls_false(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"filter_nulls": False,
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
null_filters = [
f for f in queries[0].get("filters", []) if f.get("op") == "IS NOT NULL"
]
assert null_filters == []
def test_build_query_dicts_deck_scatter_point_radius_fixed_metric(monkeypatch):
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
radius_metric = {
"expressionType": "SIMPLE",
"aggregate": "AVG",
"column": {"column_name": "radius"},
}
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"point_radius_fixed": {"type": "metric", "value": radius_metric},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["metrics"] == [radius_metric]
def test_build_query_dicts_deck_geojson_scalar_size_produces_no_metrics(monkeypatch):
# Regression: deck_geojson fixture has size='100' (scalar, not a metric).
# The fallback must produce metrics=[] to match DeckGeoJson.query_obj().
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_geojson",
"geojson": "geometry",
"size": "100",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["metrics"] == []
def test_build_query_dicts_deck_path_scalar_size_produces_no_metrics(monkeypatch):
# deck_path fixture also has size='100' — scalar must not become a metric.
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_path",
"line_column": "path_col",
"size": "100",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["metrics"] == []
def test_build_query_dicts_deck_geojson_adds_geojson_null_filter(monkeypatch):
# deck_geojson should add IS NOT NULL on the geojson column when filter_nulls
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_geojson",
"geojson": "geometry_col",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert {"col": "geometry_col", "op": "IS NOT NULL", "val": ""} in queries[0][
"filters"
]
def test_build_query_dicts_deck_hex_string_metric(monkeypatch):
# Non-numeric string size (saved metric key) must be included as a metric
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_hex",
"spatial": {"type": "geohash", "geohashCol": "geo"},
"size": "count",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["metrics"] == ["count"]
def test_build_query_dicts_deck_scatter_string_point_radius_fixed(monkeypatch):
# Legacy deck_scatter with point_radius_fixed as a bare metric key string
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"point_radius_fixed": "count",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["metrics"] == ["count"]
def test_build_query_dicts_deck_hex_orderby_when_metrics_present(monkeypatch):
# Mirrors BaseDeckGLViz.query_obj(): orderby set from first metric (desc by default)
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
metric = {"expressionType": "SIMPLE", "aggregate": "COUNT", "column": None}
form_data = {
"viz_type": "deck_hex",
"spatial": {"type": "geohash", "geohashCol": "geo"},
"size": metric,
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["orderby"] == [(metric, False)]
def test_build_query_dicts_deck_scatter_no_orderby_without_metrics(monkeypatch):
# No metrics → no orderby (pure spatial column query)
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_scatter",
"spatial": {"type": "latlong", "lonCol": "lon", "latCol": "lat"},
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert "orderby" not in queries[0]
def test_build_query_dicts_deck_arc_time_grain(monkeypatch):
# deck_arc with time_grain_sqla → is_timeseries, granularity, extras set
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_arc",
"spatial": {"type": "latlong", "lonCol": "start_lon", "latCol": "start_lat"},
"end_spatial": {
"type": "latlong",
"lonCol": "end_lon",
"latCol": "end_lat",
},
"granularity_sqla": "ts",
"time_grain_sqla": "P1D",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert queries[0]["is_timeseries"] is True
assert queries[0]["granularity"] == "ts"
assert queries[0].get("extras", {}).get("time_grain_sqla") == "P1D"
def test_build_query_dicts_deck_geojson_ignores_time_grain(monkeypatch):
# deck_geojson is not in _DECK_TIMESERIES_VIZ_TYPES; time grain fields not added
monkeypatch.setattr(
"superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
lambda datasource_id, datasource_type: "base",
)
form_data = {
"viz_type": "deck_geojson",
"geojson": "geometry",
"granularity_sqla": "ts",
"time_grain_sqla": "P1D",
"adhoc_filters": [],
}
queries = build_query_dicts_from_form_data(form_data, 1, "table")
assert "is_timeseries" not in queries[0]
assert queries[0].get("extras", {}).get("time_grain_sqla") is None
-144
View File
@@ -32,7 +32,6 @@ from sqlalchemy.sql.elements import ColumnElement
from superset.superset_typing import AdhocColumn
from superset.utils.core import GenericDataType
from tests.unit_tests.conftest import with_feature_flags
if TYPE_CHECKING:
from superset.jinja_context import BaseTemplateProcessor
@@ -2704,146 +2703,3 @@ def test_format_time_humanized_skips_activation_for_english(
instance._format_time_humanized(datetime.now() - timedelta(hours=2))
mock_activate.assert_not_called()
# -----------------------------------------------------------------------------
# _process_sql_expression denylist gate (adhoc expression validation)
# -----------------------------------------------------------------------------
def _patch_disallowed(
mocker: MockerFixture,
functions: dict[str, set[str]] | None = None,
tables: dict[str, set[str]] | None = None,
) -> None:
"""Inject the engine-keyed denylists into the live Flask app config that
`_process_sql_expression` consults via `app.config[...]`."""
mocker.patch.dict(
"flask.current_app.config",
{
"DISALLOWED_SQL_FUNCTIONS": functions or {},
"DISALLOWED_SQL_TABLES": tables or {},
},
clear=False,
)
def test_process_sql_expression_rejects_disallowed_function(
mocker: MockerFixture, database: Database
) -> None:
"""Adhoc expressions are user-controlled SQL incorporated into the final
query via `literal_column(...)`. A function name on the operator's
DISALLOWED_SQL_FUNCTIONS list must be rejected at validation time,
before the rendered SQL is handed to the database."""
from superset.connectors.sqla.models import SqlaTable
from superset.exceptions import SupersetDisallowedSQLFunctionException
_patch_disallowed(mocker, functions={"postgresql": {"version"}})
table = SqlaTable(database=database, schema=None, table_name="t")
with pytest.raises(SupersetDisallowedSQLFunctionException):
table._process_sql_expression(
expression="version()",
database_id=database.id,
engine="postgresql",
schema="",
template_processor=None,
)
def test_process_sql_expression_rejects_disallowed_function_in_aggregate(
mocker: MockerFixture, database: Database
) -> None:
"""A denylisted function wrapped in a legitimate aggregate
(`MAX(version())`) must still be rejected: the wrapper is the obvious
bypass attempt."""
from superset.connectors.sqla.models import SqlaTable
from superset.exceptions import SupersetDisallowedSQLFunctionException
_patch_disallowed(mocker, functions={"postgresql": {"version"}})
table = SqlaTable(database=database, schema=None, table_name="t")
with pytest.raises(SupersetDisallowedSQLFunctionException):
table._process_sql_expression(
expression="MAX(version())",
database_id=database.id,
engine="postgresql",
schema="",
template_processor=None,
)
@with_feature_flags(ALLOW_ADHOC_SUBQUERY=True)
def test_process_sql_expression_rejects_disallowed_table(
mocker: MockerFixture, database: Database
) -> None:
"""Adhoc subqueries that reference a denylisted table must be rejected,
and the raised exception must carry only the tables actually found in
the expression (matching the canonical execution-time gate in
`superset.sql_lab._validate_query`). The branch is only reachable when
`ALLOW_ADHOC_SUBQUERY=True`; otherwise `validate_adhoc_subquery` rejects
the subquery first."""
from superset.connectors.sqla.models import SqlaTable
from superset.exceptions import SupersetDisallowedSQLTableException
_patch_disallowed(
mocker, tables={"postgresql": {"pg_authid", "pg_shadow", "pg_stat_activity"}}
)
table = SqlaTable(database=database, schema=None, table_name="t")
with pytest.raises(SupersetDisallowedSQLTableException) as exc_info:
table._process_sql_expression(
expression="(SELECT id FROM pg_authid)",
database_id=database.id,
engine="postgresql",
schema="",
template_processor=None,
)
# Assert on substring (set repr ordering): only the offending table is
# echoed back to the user, not the full operator denylist.
message = exc_info.value.error.message
assert "pg_authid" in message
assert "pg_shadow" not in message
assert "pg_stat_activity" not in message
def test_process_sql_expression_allows_benign_expression(
mocker: MockerFixture, database: Database
) -> None:
"""Negative control: a benign aggregate over a regular column must pass
even when denylists are configured."""
from superset.connectors.sqla.models import SqlaTable
_patch_disallowed(
mocker,
functions={"postgresql": {"version"}},
tables={"postgresql": {"pg_authid"}},
)
table = SqlaTable(database=database, schema=None, table_name="t")
result = table._process_sql_expression(
expression="SUM(amount)",
database_id=database.id,
engine="postgresql",
schema="",
template_processor=None,
)
assert result is not None
assert "SUM" in result.upper()
def test_process_sql_expression_no_gate_when_denylists_empty(
mocker: MockerFixture, database: Database
) -> None:
"""When neither DISALLOWED_SQL_FUNCTIONS nor DISALLOWED_SQL_TABLES has an
entry for the engine, the new gate must not run an extra parse: any
SQL that passes the pre-existing `sanitize_clause` validation is
accepted."""
from superset.connectors.sqla.models import SqlaTable
_patch_disallowed(mocker, functions={}, tables={})
table = SqlaTable(database=database, schema=None, table_name="t")
result = table._process_sql_expression(
expression="version()",
database_id=database.id,
engine="postgresql",
schema="",
template_processor=None,
)
assert result is not None
@@ -1,103 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from unittest import mock
from scripts import change_detector
REPO = "apache/superset"
def test_resolve_workflow_run_files_pull_request(monkeypatch) -> None:
"""A workflow_run originating from a pull_request resolves the PR diff."""
monkeypatch.setenv("WF_RUN_EVENT", "pull_request")
monkeypatch.setenv("WF_RUN_PR_NUMBER", "123")
with (
mock.patch.object(
change_detector,
"fetch_changed_files_pr",
return_value=["superset/foo.py"],
) as fetch_pr,
mock.patch.object(change_detector, "fetch_changed_files_push") as fetch_push,
):
files = change_detector.resolve_workflow_run_files(REPO, "deadbeef")
assert files == ["superset/foo.py"]
fetch_pr.assert_called_once_with(REPO, "123")
fetch_push.assert_not_called()
def test_resolve_workflow_run_files_push(monkeypatch) -> None:
"""A workflow_run originating from a push resolves the push diff via head SHA."""
monkeypatch.setenv("WF_RUN_EVENT", "push")
monkeypatch.setenv("WF_RUN_HEAD_SHA", "abc123")
with (
mock.patch.object(
change_detector,
"fetch_changed_files_push",
return_value=["superset-frontend/bar.tsx"],
) as fetch_push,
mock.patch.object(change_detector, "fetch_changed_files_pr") as fetch_pr,
):
files = change_detector.resolve_workflow_run_files(REPO, "fallback-sha")
assert files == ["superset-frontend/bar.tsx"]
# The originating head SHA wins over the (default-branch) fallback SHA.
fetch_push.assert_called_once_with(REPO, "abc123")
fetch_pr.assert_not_called()
def test_resolve_workflow_run_files_push_defaults_to_fallback_sha(
monkeypatch,
) -> None:
"""Without WF_RUN_HEAD_SHA the push path falls back to the passed SHA."""
monkeypatch.delenv("WF_RUN_EVENT", raising=False)
monkeypatch.delenv("WF_RUN_HEAD_SHA", raising=False)
with mock.patch.object(
change_detector,
"fetch_changed_files_push",
return_value=[],
) as fetch_push:
change_detector.resolve_workflow_run_files(REPO, "fallback-sha")
fetch_push.assert_called_once_with(REPO, "fallback-sha")
def test_resolve_workflow_run_files_pr_missing_number(monkeypatch) -> None:
"""A fork PR without a resolvable number returns None (assume all changed)."""
monkeypatch.setenv("WF_RUN_EVENT", "pull_request")
monkeypatch.delenv("WF_RUN_PR_NUMBER", raising=False)
with mock.patch.object(change_detector, "fetch_changed_files_pr") as fetch_pr:
files = change_detector.resolve_workflow_run_files(REPO, "deadbeef")
assert files is None
fetch_pr.assert_not_called()
def test_resolve_workflow_run_files_pr_invalid_number(monkeypatch) -> None:
"""A non-integer PR number is treated as unresolvable and returns None."""
monkeypatch.setenv("WF_RUN_EVENT", "pull_request")
monkeypatch.setenv("WF_RUN_PR_NUMBER", "not-a-number")
with mock.patch.object(change_detector, "fetch_changed_files_pr") as fetch_pr:
files = change_detector.resolve_workflow_run_files(REPO, "deadbeef")
assert files is None
fetch_pr.assert_not_called()
@@ -1,179 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Tests for ``scripts/translations/check_translation_regression.py``.
The script is not installed as a package, so it is loaded via importlib from
its on-disk path.
"""
import importlib.util
import json # noqa: TID251 - testing a standalone script that uses stdlib json
from collections.abc import Mapping
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
_SCRIPT_PATH = (
Path(__file__).resolve().parents[4]
/ "scripts"
/ "translations"
/ "check_translation_regression.py"
)
_spec = importlib.util.spec_from_file_location(
"check_translation_regression", _SCRIPT_PATH
)
assert _spec is not None, f"Could not load {_SCRIPT_PATH}"
assert _spec.loader is not None, f"No loader on spec for {_SCRIPT_PATH}"
check_translation_regression = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(check_translation_regression)
def _compare(
tmp_path: Path,
before: Mapping[str, object],
after: Mapping[str, object],
failures: set[str] | None = None,
) -> None:
"""Run cmd_compare with a baseline file and a mocked 'after' state.
``failures`` simulates languages whose .po file was present but could not
be counted; cmd_compare passes a ``failures`` set into get_counts, so the
mock populates it (mirroring the real get_counts contract).
"""
def fake_get_counts(
_dir: Path, failures: set[str] | None = None
) -> Mapping[str, object]:
if failures is not None and _simulated_failures:
failures.update(_simulated_failures)
return after
_simulated_failures = failures or set()
before_file = tmp_path / "before.json"
before_file.write_text(json.dumps(before), encoding="utf-8")
with patch.object(
check_translation_regression, "get_counts", side_effect=fake_get_counts
):
check_translation_regression.cmd_compare(str(before_file), tmp_path)
def test_deleting_a_translated_string_is_not_a_regression(tmp_path: Path) -> None:
# Translated count drops by 1 (string removed from source) but no new
# fuzzy is introduced -> intentional deletion, must NOT be flagged.
before = {"fr": {"translated": 100, "fuzzy": 5}}
after = {"fr": {"translated": 99, "fuzzy": 5}}
_compare(tmp_path, before, after) # no SystemExit
def test_renaming_a_string_flags_a_regression(tmp_path: Path) -> None:
# A reworded source string strands its translation as fuzzy: translated
# drops by 1 AND fuzzy rises by 1 -> real regression.
before = {"fr": {"translated": 100, "fuzzy": 5}}
after = {"fr": {"translated": 99, "fuzzy": 6}}
with pytest.raises(SystemExit) as exc:
_compare(tmp_path, before, after)
assert exc.value.code == 1
def test_no_change_is_clean(tmp_path: Path) -> None:
stats = {"fr": {"translated": 100, "fuzzy": 5}}
_compare(tmp_path, stats, dict(stats)) # no SystemExit
def test_adding_strings_does_not_offset_a_regression(tmp_path: Path) -> None:
# Even when the PR adds new (untranslated) strings, a fresh fuzzy elsewhere
# is still a regression — additions must not mask it.
before = {"de": {"translated": 200, "fuzzy": 10}}
after = {"de": {"translated": 205, "fuzzy": 11}}
with pytest.raises(SystemExit) as exc:
_compare(tmp_path, before, after)
assert exc.value.code == 1
def test_legacy_integer_baseline_is_tolerated(tmp_path: Path) -> None:
# Older baselines stored a bare translated count with no fuzzy data.
before = {"fr": 100}
after = {"fr": {"translated": 90, "fuzzy": 0}}
_compare(tmp_path, before, after) # deletion-only, no SystemExit
def test_deleting_an_entire_catalog_is_not_a_regression(tmp_path: Path) -> None:
# The whole `fr` catalog was intentionally removed: it is absent from
# `after` and did NOT fail to count -> a legitimate deletion, not flagged.
before = {"fr": {"translated": 100, "fuzzy": 5}}
after: dict[str, object] = {}
_compare(tmp_path, before, after) # no SystemExit
def test_uncountable_baseline_catalog_is_a_hard_failure(tmp_path: Path) -> None:
# `fr` still exists on disk but msgfmt could not count it (malformed .po),
# so it is missing from `after` AND reported as a failure. This must NOT be
# mistaken for an intentional deletion — it is a hard error.
before = {"fr": {"translated": 100, "fuzzy": 5}}
after: dict[str, object] = {}
with pytest.raises(SystemExit) as exc:
_compare(tmp_path, before, after, failures={"fr"})
assert exc.value.code == 1
def test_uncountable_new_catalog_not_in_baseline_is_ignored(tmp_path: Path) -> None:
# A catalog that fails to count but was not in the baseline can't be a
# regression of a previously-good translation, so it does not fail the run.
before = {"fr": {"translated": 100, "fuzzy": 5}}
after = {"fr": {"translated": 100, "fuzzy": 5}}
_compare(tmp_path, before, after, failures={"zz"}) # no SystemExit
def test_writes_report_on_regression(tmp_path: Path) -> None:
before_file = tmp_path / "before.json"
before_file.write_text(json.dumps({"ja": {"translated": 50, "fuzzy": 2}}))
report = tmp_path / "report.md"
after = {"ja": {"translated": 49, "fuzzy": 3}}
with patch.object(check_translation_regression, "get_counts", return_value=after):
with pytest.raises(SystemExit):
check_translation_regression.cmd_compare(
str(before_file), tmp_path, str(report)
)
body = report.read_text(encoding="utf-8")
assert "Translation Regression Detected" in body
assert "`ja`" in body
# The report must make clear deletions are not flagged.
assert "deleting" in body.lower()
def test_count_stats_parses_translated_and_fuzzy() -> None:
stderr = (
"3731 translated messages, 1009 fuzzy translations, 175 untranslated messages."
)
mock_result = MagicMock(stderr=stderr)
with patch.object(
check_translation_regression.subprocess, "run", return_value=mock_result
):
stats = check_translation_regression.count_stats(Path("dummy.po"))
assert stats == {"translated": 3731, "fuzzy": 1009}
def test_count_stats_defaults_fuzzy_to_zero_when_absent() -> None:
# msgfmt omits the fuzzy clause entirely when there are none.
mock_result = MagicMock(stderr="42 translated messages.")
with patch.object(
check_translation_regression.subprocess, "run", return_value=mock_result
):
stats = check_translation_regression.count_stats(Path("dummy.po"))
assert stats == {"translated": 42, "fuzzy": 0}
@@ -1,273 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Tests for the optional dataset allowlist in guest tokens.
Covers token creation (JWT claims), schema deserialization, and the access
enforcement gate inside raise_for_access().
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from superset.exceptions import SupersetSecurityException
from superset.security.guest_token import (
GuestToken,
GuestTokenResourceType,
GuestUser,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_guest_user(datasets: list[int] | None = None) -> GuestUser:
"""Build a GuestUser whose token optionally carries a datasets allowlist."""
token: GuestToken = {
"user": {},
"resources": [{"type": GuestTokenResourceType.DASHBOARD, "id": "dash-uuid"}],
"rls_rules": [],
"iat": 0,
"exp": 9999999999,
}
if datasets is not None:
token["datasets"] = datasets
return GuestUser(token=token, roles=[])
def _make_datasource(dataset_id: int) -> MagicMock:
"""Return a minimal datasource mock with a numeric id."""
ds = MagicMock()
ds.id = dataset_id
ds.perm = "datasource_access"
return ds
# ---------------------------------------------------------------------------
# create_guest_access_token — JWT claims
# ---------------------------------------------------------------------------
def test_create_guest_access_token_without_datasets_omits_claim() -> None:
"""When datasets=None the JWT must not contain a datasets key."""
from superset.security.manager import SupersetSecurityManager
sm = MagicMock(spec=SupersetSecurityManager)
sm._get_current_epoch_time.return_value = 0
sm._get_guest_token_jwt_audience.return_value = "superset"
sm.pyjwt_for_guest_token = MagicMock()
with patch(
"superset.security.manager.get_conf",
return_value={
"GUEST_TOKEN_JWT_SECRET": "secret",
"GUEST_TOKEN_JWT_ALGO": "HS256",
"GUEST_TOKEN_JWT_EXP_SECONDS": 300,
},
):
SupersetSecurityManager.create_guest_access_token(
sm, user={}, resources=[], rls=[], datasets=None
)
claims = sm.pyjwt_for_guest_token.encode.call_args[0][0]
assert "datasets" not in claims
def test_create_guest_access_token_with_datasets_includes_claim() -> None:
"""When datasets is provided the JWT must include the datasets claim."""
from superset.security.manager import SupersetSecurityManager
sm = MagicMock(spec=SupersetSecurityManager)
sm._get_current_epoch_time.return_value = 0
sm._get_guest_token_jwt_audience.return_value = "superset"
sm.pyjwt_for_guest_token = MagicMock()
with patch(
"superset.security.manager.get_conf",
return_value={
"GUEST_TOKEN_JWT_SECRET": "secret",
"GUEST_TOKEN_JWT_ALGO": "HS256",
"GUEST_TOKEN_JWT_EXP_SECONDS": 300,
},
):
SupersetSecurityManager.create_guest_access_token(
sm, user={}, resources=[], rls=[], datasets=[7, 8]
)
claims = sm.pyjwt_for_guest_token.encode.call_args[0][0]
assert claims["datasets"] == [7, 8]
# ---------------------------------------------------------------------------
# raise_for_access — dataset allowlist enforcement
# ---------------------------------------------------------------------------
def _sm_for_access_test(guest_user: GuestUser) -> MagicMock:
"""Build a security-manager mock wired to grant basic datasource access
through the guest path so only the allowlist gate is exercised."""
from superset.security.manager import SupersetSecurityManager
sm = MagicMock(spec=SupersetSecurityManager)
sm.is_guest_user.return_value = True
sm.get_current_guest_user_if_guest.return_value = guest_user
# Make every upstream access check pass so we isolate the allowlist check.
sm.can_access_schema.return_value = True
return sm
def test_raise_for_access_no_datasets_claim_allows_any_datasource() -> None:
"""A token without a datasets claim must allow all datasources (backward compat)."""
from superset.security.manager import SupersetSecurityManager
guest_user = _make_guest_user(datasets=None)
sm = _sm_for_access_test(guest_user)
datasource = _make_datasource(dataset_id=99)
# can_access_schema returns True so the main block does not raise,
# then we hit our allowlist check — with no claim it must not raise either.
SupersetSecurityManager.raise_for_access(sm, datasource=datasource) # no exception
def test_raise_for_access_datasets_claim_allows_listed_datasource() -> None:
"""A token with datasets=[7, 8] must allow datasource id=7."""
from superset.security.manager import SupersetSecurityManager
guest_user = _make_guest_user(datasets=[7, 8])
sm = _sm_for_access_test(guest_user)
datasource = _make_datasource(dataset_id=7)
SupersetSecurityManager.raise_for_access(sm, datasource=datasource) # no exception
def test_raise_for_access_datasets_claim_blocks_unlisted_datasource() -> None:
"""A token with datasets=[7, 8] must block datasource id=99."""
from superset.security.manager import SupersetSecurityManager
guest_user = _make_guest_user(datasets=[7, 8])
sm = _sm_for_access_test(guest_user)
datasource = _make_datasource(dataset_id=99)
with pytest.raises(SupersetSecurityException):
SupersetSecurityManager.raise_for_access(sm, datasource=datasource)
def test_raise_for_access_empty_datasets_list_blocks_all() -> None:
"""An explicit empty allowlist (datasets=[]) must block every datasource."""
from superset.security.manager import SupersetSecurityManager
guest_user = _make_guest_user(datasets=[])
sm = _sm_for_access_test(guest_user)
datasource = _make_datasource(dataset_id=7)
with pytest.raises(SupersetSecurityException):
SupersetSecurityManager.raise_for_access(sm, datasource=datasource)
def test_raise_for_access_malformed_datasets_claim_blocks_access() -> None:
"""A non-integer element in the datasets claim must be treated as a denial."""
from superset.security.manager import SupersetSecurityManager
# Simulate a token whose datasets claim was tampered to contain strings.
guest_user = _make_guest_user(datasets=None)
guest_user.guest_token["datasets"] = ["7", "8"] # type: ignore[list-item]
sm = _sm_for_access_test(guest_user)
datasource = _make_datasource(dataset_id=7)
with pytest.raises(SupersetSecurityException):
SupersetSecurityManager.raise_for_access(sm, datasource=datasource)
# ---------------------------------------------------------------------------
# GuestTokenCreateSchema — datasets field
# ---------------------------------------------------------------------------
def test_guest_token_create_schema_datasets_optional() -> None:
"""datasets is optional — a payload without it must load successfully."""
from superset.security.api import GuestTokenCreateSchema
schema = GuestTokenCreateSchema()
result = schema.load({"resources": [{"type": "dashboard", "id": "abc"}], "rls": []})
assert result.get("datasets") is None
def test_guest_token_create_schema_datasets_accepted() -> None:
"""datasets=[7, 8] must load and be present in the result."""
from superset.security.api import GuestTokenCreateSchema
schema = GuestTokenCreateSchema()
result = schema.load(
{
"resources": [{"type": "dashboard", "id": "abc"}],
"rls": [],
"datasets": [7, 8],
}
)
assert result["datasets"] == [7, 8]
# ---------------------------------------------------------------------------
# get_current_guest_user_if_guest — direct implementation coverage
# ---------------------------------------------------------------------------
def test_get_current_guest_user_if_guest_returns_guest_user() -> None:
"""Returns the GuestUser when g.user is a GuestUser instance."""
from unittest.mock import patch
from superset.security.manager import SupersetSecurityManager
guest_user = _make_guest_user()
sm = MagicMock(spec=SupersetSecurityManager)
with patch("superset.security.manager.g") as mock_g:
mock_g.user = guest_user
result = SupersetSecurityManager.get_current_guest_user_if_guest(sm)
assert result is guest_user
def test_get_current_guest_user_if_guest_returns_none_for_regular_user() -> None:
"""Returns None when g.user is a regular (non-guest) user."""
from unittest.mock import patch
from superset.security.manager import SupersetSecurityManager
regular_user = MagicMock() # not a GuestUser instance
sm = MagicMock(spec=SupersetSecurityManager)
with patch("superset.security.manager.g") as mock_g:
mock_g.user = regular_user
result = SupersetSecurityManager.get_current_guest_user_if_guest(sm)
assert result is None
def test_raise_for_access_non_guest_skips_allowlist_check() -> None:
"""Allowlist check is skipped when get_current_guest_user_if_guest returns None."""
from superset.security.manager import SupersetSecurityManager
sm = MagicMock(spec=SupersetSecurityManager)
sm.get_current_guest_user_if_guest.return_value = None
datasource = _make_datasource(dataset_id=99)
# Should not raise — allowlist block is not entered when there is no guest user.
SupersetSecurityManager.raise_for_access(sm, datasource=datasource)
-16
View File
@@ -1,16 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

Some files were not shown because too many files have changed in this diff Show More