Compare commits

..

3 Commits

Author SHA1 Message Date
Evan Rusackas
e4cc16a9d7 ci: add Kesin11/actions-timeline to the heavy CI workflows
Adds a per-job/step Gantt chart (rendered as a mermaid diagram in the run
summary) to the 15 substantive CI workflows -- the 13 setup-backend
consumers plus superset-frontend.yml and docker.yml, the other two
heaviest CI paths. Skips trivial bot/label/notification workflows that run
in seconds and have nothing worth visualizing.

For single-job (or single-heavy-job-in-a-linear-chain) workflows, the step
is registered first, before checkout, so its post-processing hook -- which
is what actually renders the timeline -- captures the full job including
other steps' own cleanup. For workflows with multiple independent parallel
jobs, added a dedicated `actions-timeline` terminal job (`needs: [...]`,
`if: always()`) instead of duplicating the step into each parallel job:
the action fetches every job of the whole run from the GitHub API
regardless of which job it executes in, so one copy that runs after
every sibling job completes produces one authoritative timeline, while N
copies dropped into N parallel jobs would each race to render an
incomplete gantt before their siblings finish.

`expand-composite-actions: true` is set everywhere so setup-backend's
internal steps (Python setup, uv install, apt package caching, dependency
install) show up as their own bars rather than one opaque blob -- directly
useful given the last two PRs' worth of composite-action changes.

`actions: read` is added wherever needed to read job/step timing from the
Actions API, either to the workflow's top-level `permissions:` (when the
job in question has no job-level override) or directly into the relevant
job's own `permissions:` block (when one already exists, since a
job-level block replaces rather than merges with the workflow-level one).
2026-07-27 22:10:27 -07:00
Evan Rusackas
82b609addb ci: cache apt packages with awalsh128/cache-apt-pkgs-action (#42499) 2026-07-27 21:46:23 -07:00
Evan Rusackas
10d2f0c585 ci: use astral-sh/setup-uv to speed up backend CI
setup-backend (used in 17 job steps across 13 workflows) bootstrapped uv
via `pip install --upgrade pip setuptools wheel uv` on every run, and only
cached pip's download dir via actions/setup-python's `cache: pip` -- pip
wasn't doing the actual installs (uv was), so that cache bought nothing.

astral-sh/setup-uv ships a prebuilt uv binary instead of installing it via
pip, and its own GitHub Actions cache (enable-cache: true) persists uv's
resolution/wheel cache across runs, keyed on requirements/*.txt and
pyproject.toml by default. Also drops the now-redundant pip/setuptools/wheel
upgrade -- uv's build isolation installs anything a package's build backend
declares (pyproject.toml's [build-system] already requires setuptools/wheel)
without needing them preinstalled in the target environment.

Also swaps the same `pip install uv` bootstrap in bump-python-package.yml.

The uv step is skipped when install-superset: false (helm-lint-test), since
otherwise there's nothing for it to install or cache and it fails the job.
2026-07-27 21:34:27 -07:00
18 changed files with 140 additions and 107 deletions

View File

@@ -5,10 +5,6 @@ inputs:
description: 'Python version to set up. Accepts a version number, "current", or "next".'
required: true
default: 'current'
cache:
description: 'Cache dependencies. Options: pip'
required: false
default: 'pip'
requirements-type:
description: 'Type of requirements to install. Options: base, development, default'
required: false
@@ -43,17 +39,24 @@ runs:
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ steps.set-python-version.outputs.python-version }}
cache: ${{ inputs.cache }}
- name: Install uv
if: inputs.install-superset == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
python-version: ${{ steps.set-python-version.outputs.python-version }}
enable-cache: true
- name: Install apt packages
if: inputs.install-superset == 'true'
uses: awalsh128/cache-apt-pkgs-action@553a35bb8ebd9fcabcb1c9451aa4c98e1b4ca8a9 # v1.6.3
with:
packages: libldap2-dev libsasl2-dev
version: 1.0
- name: Install dependencies
env:
INPUT_INSTALL_SUPERSET: ${{ inputs.install-superset }}
INPUT_REQUIREMENTS_TYPE: ${{ inputs.requirements-type }}
run: |
if [ "$INPUT_INSTALL_SUPERSET" = "true" ]; then
sudo apt-get update && sudo apt-get -y install libldap2-dev libsasl2-dev
pip install --upgrade pip setuptools wheel uv
if [ "$INPUT_REQUIREMENTS_TYPE" = "dev" ]; then
uv pip install --system -r requirements/development.txt
elif [ "$INPUT_REQUIREMENTS_TYPE" = "base" ]; then

View File

@@ -45,7 +45,10 @@ jobs:
python-version: "3.11"
- name: Install uv
run: pip install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
python-version: "3.11"
enable-cache: true
- name: supersetbot bump-python -p "${{ github.event.inputs.package }}"
env:

View File

@@ -11,6 +11,7 @@ on:
permissions:
contents: read
pull-requests: read
actions: read
# cancel previous workflow jobs for PRs
concurrency:
@@ -21,6 +22,10 @@ jobs:
check-python-deps:
runs-on: ubuntu-26.04
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -20,6 +20,7 @@ concurrency:
permissions:
contents: read
actions: read
jobs:
dependency-review:
@@ -50,6 +51,10 @@ jobs:
# You cannot use a liccheck.ini file in this workflow.
runs-on: ubuntu-26.04
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -213,3 +213,14 @@ jobs:
shell: bash
run: |
docker compose -f docker-compose-image-tag.yml up superset-init --exit-code-from superset-init
actions-timeline:
needs: [docker-build, docker-compose-image-tag]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true

View File

@@ -15,6 +15,7 @@ on:
permissions:
contents: read
actions: read
# cancel previous workflow jobs for PRs
concurrency:
@@ -32,6 +33,10 @@ jobs:
# rarely differ across patch versions, so 3x per PR is wasteful.
python-version: ${{ github.event_name == 'pull_request' && fromJSON('["current"]') || fromJSON('["current", "next"]') }}
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -11,6 +11,7 @@ on:
permissions:
contents: read
pull-requests: read
actions: read
# cancel previous workflow jobs for PRs
concurrency:
@@ -40,6 +41,10 @@ jobs:
ports:
- 16379:6379
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -30,6 +30,7 @@ concurrency:
permissions:
contents: read
actions: read
jobs:
config:
@@ -59,6 +60,10 @@ jobs:
name: Build & Deploy
runs-on: ubuntu-26.04
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.event.workflow_run.head_sha || github.sha }}"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
@@ -76,7 +81,10 @@ jobs:
distribution: "zulu"
java-version: "21"
- name: Install Graphviz
run: sudo apt-get install -y graphviz
uses: awalsh128/cache-apt-pkgs-action@553a35bb8ebd9fcabcb1c9451aa4c98e1b4ca8a9 # v1.6.3
with:
packages: graphviz
version: 1.0
- name: Compute Entity Relationship diagram (ERD)
env:
SUPERSET_SECRET_KEY: not-a-secret

View File

@@ -340,3 +340,14 @@ jobs:
exit 1
fi
echo "playwright-tests result: $RESULT (changes: $CHANGES)"
actions-timeline:
needs: [cypress-matrix, playwright-tests]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true

View File

@@ -11,6 +11,7 @@ on:
permissions:
contents: read
pull-requests: read
actions: read
# cancel previous workflow jobs for PRs
concurrency:
@@ -30,6 +31,10 @@ jobs:
run:
working-directory: superset-extensions-cli
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -195,3 +195,14 @@ jobs:
run: |
docker run --rm $TAG bash -c \
"npm run build-storybook && npx playwright install-deps && npx playwright install chromium && npm run test-storybook:ci"
actions-timeline:
needs: [report-coverage, lint-frontend, validate-frontend, test-storybook]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true

View File

@@ -8,6 +8,7 @@ on:
permissions:
contents: read
actions: read
# Serialize runs per PR without cancelling: when a first-time contributor's
# queued runs are approved together, cancel-in-progress lets an older run
@@ -21,6 +22,10 @@ jobs:
lint-test:
runs-on: ubuntu-26.04
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:

View File

@@ -54,6 +54,7 @@ jobs:
permissions:
contents: read
pull-requests: read
actions: read
strategy:
fail-fast: false
matrix:
@@ -79,6 +80,10 @@ jobs:
ports:
- 16379:6379
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event

View File

@@ -255,3 +255,14 @@ jobs:
exit 1
fi
echo "test-postgres result: $RESULT"
actions-timeline:
needs: [test-mysql, test-postgres, test-sqlite]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true

View File

@@ -158,3 +158,14 @@ jobs:
verbose: true
use_oidc: true
slug: apache/superset
actions-timeline:
needs: [test-postgres-presto, test-postgres-hive]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true

View File

@@ -1,65 +0,0 @@
name: Python Unit Test Results
on:
# zizmor: ignore[dangerous-triggers] - runs in base-branch context and only consumes artifacts uploaded by Python-Unit; never checks out PR code (see note below)
workflow_run:
workflows: ["Python-Unit"]
types: [completed]
# This workflow publishes a check run annotating failing Python unit tests
# inline on the PR diff, using JUnit XML uploaded by the Python-Unit workflow.
# It uses the workflow_run trigger so that it always runs in the base-branch
# context and can safely be granted write permissions, even for PRs from
# forks or Dependabot.
#
# IMPORTANT: This workflow must NEVER check out code from the PR branch. All
# data comes from artifacts uploaded by the Python-Unit workflow.
permissions:
contents: read
checks: write
issues: read
actions: read
jobs:
report:
runs-on: ubuntu-26.04
timeout-minutes: 10
if: >
github.event.workflow_run.conclusion == 'success' ||
github.event.workflow_run.conclusion == 'failure'
steps:
# Fails soft (continue-on-error) because the source unit-tests job is
# itself gated on change detection: a docs-only PR skips it entirely,
# so there is nothing to download or report on.
- name: Download JUnit results
id: download
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
pattern: "junit-results-*"
path: artifacts
merge-multiple: true
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Download event file
id: download-event
if: steps.download.outcome == 'success'
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: "Event File"
path: event
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Publish test results
if: steps.download.outcome == 'success' && steps.download-event.outcome == 'success'
uses: EnricoMi/publish-unit-test-result-action@d0a4676d0e0b938bc201470d88276b7c74c712b3 # v2.24.0
with:
commit: ${{ github.event.workflow_run.head_sha }}
event_file: event/event.json
event_name: ${{ github.event.workflow_run.event }}
files: "artifacts/**/*.xml"
check_name: "Python Unit Test Results"
comment_mode: "off"

View File

@@ -74,14 +74,14 @@ jobs:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear --maxfail=50 --junit-xml=test-results/junit-unit.xml
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear --maxfail=50
- name: Python 100% coverage unit tests
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100 --junit-xml=test-results/junit-sql-coverage.xml
pytest --durations-min=0.5 --cov=superset/semantic_layers/ ./tests/unit_tests/semantic_layers/ --cache-clear --cov-fail-under=100 --junit-xml=test-results/junit-semantic-layers-coverage.xml
pytest --durations-min=0.5 --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100
pytest --durations-min=0.5 --cov=superset/semantic_layers/ ./tests/unit_tests/semantic_layers/ --cache-clear --cov-fail-under=100
- name: Upload code coverage
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
@@ -89,33 +89,6 @@ jobs:
verbose: true
use_oidc: true
slug: apache/superset
# Uploaded even when a pytest step above fails, since that is exactly
# when the JUnit results are needed downstream, to annotate the PR with
# the failing tests. Consumed by the "Python Unit Test Results" workflow
# via workflow_run (see that workflow for why it can't just be a step
# here: it needs to run with write permissions, which this PR-triggered
# job can't safely have on a fork PR).
- name: Upload JUnit test results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: junit-results-${{ matrix.python-version }}
path: test-results/
retention-days: 7
# Uploads the raw pull_request event payload so the "Python Unit Test
# Results" workflow (running via workflow_run, in base-branch context) can
# look up which PR/commit to annotate without checking out untrusted code.
event-file:
runs-on: ubuntu-26.04
timeout-minutes: 5
steps:
- name: Upload event file
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: Event File
path: ${{ github.event_path }}
retention-days: 7
# Stable required-status-check anchor. `unit-tests` is a matrix job gated on
# change detection, so on non-Python PRs it is skipped and never produces its
@@ -128,7 +101,14 @@ jobs:
if: always()
runs-on: ubuntu-26.04
timeout-minutes: 5
permissions:
contents: read
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true
- name: Check unit-tests result
env:
RESULT: ${{ needs.unit-tests.result }}

View File

@@ -78,7 +78,10 @@ jobs:
- name: Install gettext tools
if: steps.check.outputs.python == 'true' || steps.check.outputs.frontend == 'true'
run: sudo apt-get update && sudo apt-get install -y gettext
uses: awalsh128/cache-apt-pkgs-action@553a35bb8ebd9fcabcb1c9451aa4c98e1b4ca8a9 # v1.6.3
with:
packages: gettext
version: 1.0
# Fetch the base ref so we can compare PR-introduced regressions
# against a fair baseline (also runs babel_update against the base
@@ -153,3 +156,14 @@ jobs:
- name: Fail if regression detected
if: steps.regression.outcome == 'failure'
run: exit 1
actions-timeline:
needs: [frontend-check-translations, babel-extract]
if: always()
runs-on: ubuntu-26.04
permissions:
actions: read
steps:
- uses: Kesin11/actions-timeline@7bf79990b7c09f5dfb570ac30b814ca597bd538e # v3.1.1
with:
expand-composite-actions: true