Compare commits

..
Author SHA1 Message Date
Superset DevandClaude Opus 4.7 4bca076e86 test(plugin-chart-country-map): playwright e2e for static assets + end-to-end render
Two narrowly-scoped Playwright tests covering the integration layers
unit tests can't reach:

1. **serves all required static GeoJSON assets** — directly fetches
   each well-known file (manifest, default + ukr admin0, sample
   admin1, regional aggregation, composite) and asserts the server
   returns 200 + a parsable payload. Catches packaging regressions
   like the .dockerignore-drop that produced 404s on the PR-preview
   build despite the files being committed.

2. **renders a France subdivisions chart end-to-end** — creates a
   country_map_v2 chart via the API against the built-in
   birth_france_by_region dataset, opens its explore page, asserts
   the renderer's error banner ("Error loading map" / "No GeoJSON
   URL resolved") never appears, and counts at least 50 SVG path
   elements (one per French department). This is the canary the
   admin_level string/number bug and the all-black-render theme bug
   would have tripped.

Cross-viz form_data migration stays unit-tested in
migrateFromLegacy.test.ts; per-control visibility/validation stays
unit-tested in controlPanel.test.ts. This spec is reserved for
glue-layer checks only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 20:30:52 -07:00
Superset DevandClaude Opus 4.7 7b5fce6f54 test(plugin-chart-country-map): unit tests for CountryMap renderer helpers
CountryMap.tsx had zero coverage despite being the file where most of
the user-reported bugs landed (all-black render, flying-islands toggle
no-op, tooltip language). Refactor the pure helpers out as named
exports and exercise them directly without rendering:

- featureKey: iso_3166_2 > adm0_a3 > '' resolution, missing-props
  tolerance
- featureName: language-specific lookup with lowercasing, generic-name
  fallback, empty fallback
- filterFeatures: includes-only mode, excludes mode, flying-islands
  toggle, composition (includes + excludes + flying islands together)
- resolveColors: every key returns a non-empty string even on a theme
  with no antd tokens (regression for the all-black bug); honors
  present tokens; treats empty-string tokens as falsy and falls back

resolveColors is also extracted from the component body so the
fallback contract is testable in isolation.

18 new tests; no behavior change to the component itself.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 18:58:19 -07:00
Superset DevandClaude Opus 4.7 610c435872 feat(plugin-chart-country-map): wire up the "Show flying islands" toggle end-to-end
The control was removed earlier because it was a documented no-op —
the build pipeline repositioned features per flying_islands.yaml but
didn't tag them, so the runtime drop logic had nothing to filter on.
Now actually functional:

- build.py: apply_flying_islands sets `_flying: true` on every feature
  it repositions (FRA's 5 DROMs, USA's Hawaii/Alaska/Puerto Rico, ESP
  Canary Islands, NOR Svalbard, PRT Madeira/Azores, etc.)
- controlPanel.tsx: re-add the `show_flying_islands` checkbox (default
  on, renderTrigger so the projection refits without re-querying)
- CountryMap.tsx: filterFeatures drops _flying-tagged features when the
  toggle is off; fit-to-selection projection automatically zooms to
  the remaining mainland
- types: declare optional `_flying` property
- Tests: assert control presence + default value; remove the
  intentionally-absent assertion
- Regenerated 5 admin1 files + the france_overseas composite to embed
  the new tags (only countries with flying_islands rules differ)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 23:32:34 -07:00
Superset DevandClaude Opus 4.7 df479e7e3c fix(plugin-chart-country-map): explicit color fallbacks so undefined theme tokens don't render black
Some Superset theme implementations don't expose every antd-style token
we were reading (colorFillTertiary, colorBgSpotlight, etc.). When a
token resolves to undefined, setAttribute('fill', undefined) on an SVG
path makes the browser fall back to the SVG default fill of black —
which manifested as a fully-black chart area with only the data-colored
features visible.

Add a literal hex fallback to every theme token the renderer reads so
the chart is always paintable, even on themes that don't ship a given
token.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 23:17:03 -07:00
Superset DevandClaude Opus 4.7 d9f04d6682 fix(plugin-chart-country-map): admin_level string/number coercion + lower row_limit default
Two user-visible bugs fixed:

1. **"No GeoJSON URL resolved" with admin_level + country set.** The
   SelectControl serializes admin_level as a string ('0' / '1' /
   'aggregated'), but transformProps was comparing against numbers
   (`adminLevel === 1`), so real-world charts silently fell through
   every branch and returned a null URL. Existing tests passed because
   they used number values directly. Normalize to string at the top
   of transformProps and compare against string constants; add two
   regression tests that pass the string form-data values reality
   actually sends.

2. **Default row_limit of 50000 exceeds many deployments' configured
   ROW_LIMIT ceiling, blocking Update Chart.** Choropleths key one row
   per region — even the densest country maps (France 101 departments,
   India 36 states, US 51 territories) are well under 10k rows.
   Override the shared default to 10000 via controlOverrides.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 17:51:14 -07:00
Superset DevandClaude Opus 4.7 ba4367d0ef chore(ci): make country-map drift comment sticky
Every push to a country-map PR retriggers the regen workflow, which
was stacking a fresh drift warning comment on every commit (15+ on
this branch alone). Switch to the sticky-comment pattern:

- Tag each drift comment with a hidden HTML marker
- On drift: look up the existing tagged comment by ID and edit it in
  place, only posting a new one if none exists
- On drift cleared: delete the existing tagged comment so the PR
  doesn't carry a stale warning

Cleaned up the existing 15 stacked comments out of band.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 16:43:29 -07:00
Superset DevandClaude Opus 4.7 900fc5da64 fix(plugin-chart-country-map): drop unused COMPOSITE_CHOICES constant
After the composite-scoping refactor moved composite choices to a
country-aware mapStateToProps, COMPOSITE_CHOICES was orphaned and
tripped lint-frontend / validate-frontend tsc with TS6133.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 16:40:58 -07:00
Superset DevandClaude Opus 4.7 d9f4367c09 fix(country-map): re-include country-maps GeoJSON in Docker build context
.dockerignore excludes superset/static/assets/ wholesale because the
rest of that directory is webpack-generated at image build time. Our
country-map plugin's GeoJSON outputs are committed (not webpack-
generated), so they were getting silently dropped from every Docker
build — ephemeral PR previews 404'd on every map file fetch.

Add a `!` re-include for the country-maps subdirectory so the assets
ride along while leaving the rest of the webpack-managed exclusion
intact.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 16:26:48 -07:00
Superset DevandClaude Opus 4.7 11f61092b9 feat(plugin-chart-country-map): scope composites by anchor country
Composites now declare their anchor country via base.adm0_a3 in
composite_maps.yaml; the build pipeline copies that into manifest.json
and the control panel uses it to:

- Hide the composite control entirely when the selected country has no
  scoped composite (and there's no country-agnostic composite either)
- Narrow the composite dropdown to entries that actually apply to the
  current country selection

Concretely: picking USA at the Country view no longer offers the
"France (with overseas territories)" composite — the control just
disappears. Picking FRA shows it.

Also updates the SIP's deprecation plan to reflect that no separate
"Switch to new Country Map" button is needed (the cross-viz form_data
hand-off via formDataOverrides + migrateFromLegacy already covers it),
and adds a Phase 2 DB migration that bulk-rewrites legacy charts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 11:21:31 -07:00
Superset DevandClaude Opus 4.7 6f882f215e feat(plugin-chart-country-map): friendlier labels + auto-migrate from legacy plugin
Two small UX wins on top of the previous control-panel fixes:

1. Rename admin_level options to "World" / "Country" / "Aggregated
   regions" (from the technical "Countries (Admin 0)" / "Subdivisions
   (Admin 1)"). The control's own label becomes "Map view" with an
   explanatory description. Underlying form_data values stay 0 / 1 /
   aggregated so saved charts don't break.

2. Auto-migrate form_data when a user switches a saved legacy
   country_map chart's viz type to country_map_v2:
     - select_country: 'france'         → admin_level=1, country=FRA
     - select_country: 'france_overseas' → composite=france_overseas
     - select_country: 'turkey_regions'  → admin_level=aggregated,
                                            country=TUR, region_set=nuts_1
     - …same pattern for france_regions / italy_regions /
       philippines_regions and ~180 other legacy country files.

   The mapping lives in src/plugin/migrateFromLegacy.ts (covered by
   12 unit tests). Migration only fills fields the user hasn't already
   set on the new chart, so explicit edits win over inferred values.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 09:44:01 -07:00
Superset DevandClaude Opus 4.7 d1a7c9e82f fix(plugin-chart-country-map): control-panel UX — validators, dynamic choices, visibility
Four UX bugs reported on the new country map chart:

1. **"Country: cannot be empty" stuck on the Data tab even at Admin 0.**
   The Country control had a hard `validators: [validateNonEmpty]`, but
   it's only rendered (and only meaningful) when admin_level != 0 and no
   composite is set. The validator was firing for the hidden empty value
   and trapped the user — the Update Chart button was permanently
   disabled with no visible control to satisfy. Switched to a dynamic
   validator via mapStateToProps so it only fires when the field is
   actually needed.

2. **"Aggregated regions" threw TypeError: t.map is not a function.**
   The region_set control used `choices: ({ controls }) => ...`, but
   SelectControl's `choices` must be a literal array. Moved the
   country-dependent choice computation into `mapStateToProps` (same
   pattern other plugins use).

3. **Composite map always visible and "sticky" overriding admin_level.**
   At Admin 0 (world choropleth) the composite override produced a map
   that didn't change when you toggled admin_level. Hidden composite
   control at Admin 0 entirely, and hidden it whenever the build
   pipeline didn't emit any composites — leaves room for future
   per-country scoping (e.g. only show france_overseas when country=FRA).

4. **Show flying islands checkbox was a no-op.** The build pipeline
   doesn't currently tag features as flying, so the runtime drop logic
   had nothing to act on. Removed from the control set rather than ship
   a misleading control; a single line brings it back once the build
   tags features.

Tests updated to cover all four behaviors.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 09:29:03 -07:00
Superset DevandClaude Opus 4.7 adbf5bcff8 feat(plugin-chart-country-map): ship all 33 NE worldviews at Admin 0
Previously the build only emitted the ukr (Ukraine) worldview, so the
worldview dropdown had a single option even though it claimed otherwise.
Build now produces Admin 0 GeoJSON for every NE-published editorial:
default, arg, bdg, bra, chn, deu, egy, esp, fra, gbr, grc, idn, ind, iso,
isr, ita, jpn, kor, mar, nep, nld, pak, pol, prt, pse, rus, sau, swe, tur,
twn, ukr, usa, vnm (33 total).

NE does not publish per-worldview Admin 1 variants, so subdivisions within
a country come from a single shared file. The frontend now always points
Admin 1, regional aggregation, and composite URLs at the ukr-prefixed
shared outputs regardless of the selected worldview — the worldview
control only affects the world (Admin 0) map.

- build.py: expand WORLDVIEWS_ADMIN_0 to 33 worldviews; main() builds
  Admin 0 for all of them, Admin 1 only for ukr
- transformProps.ts: introduce SHARED_ADMIN1_WORLDVIEW = 'ukr'; pin all
  non-Admin-0 URLs to it
- controlPanel.tsx: WORLDVIEW_LABELS now covers all 33 codes; unrecognized
  codes still fall back to raw code for forward-compat
- transformProps.test.ts: cover shared-Admin1 contract (admin1+chn still
  resolves to ukr_admin1_*)
- pre-commit: exclude .geo.json from check-added-large-files (existing
  rule only excluded .geojson and would block these ~2MB worldview files)
- README + SIP: document the worldview model and check off Phase 1 item

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 09:10:04 -07:00
Superset DevandClaude Opus 4.7 ae7a947bc0 fix: prettier on remaining country-map files
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 00:15:38 -07:00
Superset DevandClaude Opus 4.7 cc7ba360f8 fix(plugin-chart-country-map): add plugin reference + json include in root tsconfig
The plugin imports src/data/manifest.json. The plugin's own tsconfig
already had resolveJsonModule + json in include after the previous fix,
but the root superset-frontend/tsconfig.json (which tsc --build walks)
also needs to resolve the manifest.json across the lib/ and esm/ babel
outputs. Add the explicit json glob and register the plugin as a
project reference so tsc finds it in build mode.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 00:00:58 -07:00
Superset DevandClaude Opus 4.7 69afb7653b fix(plugin-chart-country-map): trailing newlines + correct theme/t imports
- Add trailing newline to all 220 geo.json outputs to satisfy
  end-of-file-fixer pre-commit hook
- build.py: post-process step ensures future regens emit trailing newlines
- CountryMap.tsx: import t/useTheme from @apache-superset/core
  (matches the rest of the codebase) and use antd theme tokens
  (colorBgSpotlight, colorTextLightSolid, colorErrorText, etc.)
  instead of legacy theme.colors.* paths

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 23:37:57 -07:00
Superset DevandClaude Opus 4.7 240bcad6bd fix(plugin-chart-country-map): trailing newline on static manifest + prettier format on transformProps.test
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 23:19:43 -07:00
Superset DevandClaude Opus 4.7 ae61f2f507 fix(plugin-chart-country-map): clear remaining CI issues
- transformProps: read snake_case via rawFormData (ChartProps.formData
  is camelCased), fixing 4 failing jest tests
- CountryMap.tsx: replace literal colors with theme tokens; wrap user
  strings with t() for i18n
- build.py: add proper dict[str, Any] type params, drop unused type:ignore,
  emit manifest.json with trailing newline for prettier/EOF parity
- test_build.py: top-of-file mypy ignore (unittest test scaffolding)
- pyproject.toml: per-file ruff ignores for the standalone build pipeline
  (TID251/S310/S603/S607/E501/C901/PT009 all intentional/inapplicable)
- regen workflow: surface drift via PR comment + step summary instead of
  failing — cross-platform mapshaper output reproducibility is still WIP

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 23:14:10 -07:00
Superset DevandClaude Opus 4.7 cbb8477d47 fix(plugin-chart-country-map): resolve TypeScript lint errors
- Remove unused CategoricalColorNamespace import in CountryMap.tsx
- Remove unused isAdminSubdivision helper in controlPanel.tsx
- Add resolveJsonModule + include manifest.json in tsconfig
- Cast chartProps.formData to CountryMapFormData in transformProps

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 23:03:58 -07:00
Evan RusackasandClaude Opus 4.7 9fd7fd441a fix(country-map): CI failures — license headers, lockfile sync, reproducible build
Three coordinated fixes for the 25 CI failures on the initial PR push:

1. **Lockfile sync.** Added @superset-ui/plugin-chart-country-map as
   a workspace dep in the previous commit but didn't update
   package-lock.json. CI's `npm ci` failed across frontend-build,
   cypress (12 jobs), playwright (4 jobs), docker (2 jobs), and
   frontend-check-translations. Re-ran `npm install --package-lock-only`
   to add the new workspace's 71 lock entries.

2. **License headers added** to 13 new files flagged by License Check:
   - 5 markdown READMEs / SIP_DRAFT (HTML-comment headers)
   - 5 YAML config files (`# Licensed ...`)
   - 2 Python files (`# Licensed ...`)
   - 1 shell script (preserves shebang)

3. **Reproducible build outputs.** The regen workflow detected drift
   on manifest.json + ukr_admin1_CAN.geo.json. Two root causes:
   - `build_timestamp_utc` field made manifest non-deterministic →
     dropped from the schema
   - Floating mapshaper version (`npx --yes mapshaper`) caused subtle
     simplification differences across runners → pinned to
     `mapshaper@0.7.15` via `npx --yes mapshaper@<version>`

Verified locally: rebuild from clean cache reproduces every output
byte-identically except the manifest (which now also matches once
the timestamp is gone).

Files changed:
  .gitignore                           — re-include rule for static dir
  superset-frontend/package-lock.json  — +71 lines for new workspace
  13 new files                         — ASF headers
  build.py                             — pin mapshaper, drop timestamp
  manifest.json (× 2)                  — regenerate w/o timestamp
  README.md (in static dir)            — header

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 22:32:51 -07:00
Evan RusackasandClaude Opus 4.7 cfdc7dce8e docs(country-map): SIP — phases 2/3/5 substantially done
Mark Phase 2 (plugin scaffolding) substantially complete: hosting
path wired, click-to-zoom ported. Phase 3 (controls) fully done with
manifest-driven choices replacing hardcoded tables. Phase 5
substantially done: 41 tests across 4 test files, CI workflow,
UPDATING.md entry.

Remaining items, all non-blocking for SIP filing:
- "Switch to new Country Map" button on legacy plugin renderer
- Cross-filter integration via setDataMask
- Composite projection support (geoAlbersUsa for USA)
- Real example images (need actual rendering capture)
- Update Superset docs site

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:39:15 -07:00
Evan RusackasandClaude Opus 4.7 9a4fff02bf test(country-map): controlPanel test (10 cases)
Completes the test trio (transformProps + buildQuery + controlPanel).
Verifies:

- All 9 new controls (worldview / admin_level / country / region_set /
  composite / region_includes / region_excludes / show_flying_islands /
  name_language) are present in the panel
- Worldview defaults to 'ukr' (Superset's editorial choice)
- show_flying_islands defaults to true
- name_language defaults to 'en'
- admin_level offers exactly the 3 expected codes
- Country selector visibility hides on Admin 0 OR when composite set
- Region-set selector only visible when admin_level === 'aggregated'
- Region-set choices key off the selected country (TUR → nuts_1,
  FRA → regions, USA → empty)
- Composite selector exposes france_overseas

These tests would fail loudly if anyone refactored the visibility
predicates or accidentally removed/renamed a control.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:38:26 -07:00
Evan RusackasandClaude Opus 4.7 8a06bbac1e test(country-map): unit tests for build script transforms (18 cases)
Covers the pure-Python helpers and YAML-applied transforms that the
build pipeline relies on. Subprocess calls (mapshaper, NE download)
are not exercised — those are integration concerns covered by the
regen workflow itself.

Test categories:
- _matches (4 tests): scalar equality, AND'd conditions, `in: [...]`
  list-membership matcher, missing property
- _bbox_center (2 tests): unit square, offset square
- _translate_and_scale (4 tests): pure translate, scale-around-centroid,
  combined transform, multipolygon handling
- _translate_and_scale_with_pivot (1 test): shared pivot preserves
  relative positions of grouped features (the Paris-petite-couronne case)
- _drop_parts (2 tests): drops specified indices, polygon unchanged
- _bbox_contains (2 tests): inside-bbox, outside-bbox-west
- apply_name_overrides (1 test): applies only to matching features,
  respects match conditions across countries (FRA "Seien" vs GBR "Seien"
  don't collide)
- apply_flying_islands (2 tests): repositions matched features,
  drop_outside_bbox guarded to Admin 1 only (the bug we fixed earlier)

Wired into the regen workflow as a step that runs BEFORE the build,
so a broken transform fails CI before producing potentially-bad output
files.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:37:27 -07:00
Evan RusackasandClaude Opus 4.7 cd85b2dc99 ci(country-map): regenerate GeoJSON outputs on config changes
New workflow `.github/workflows/country-map-build-regen.yml` that:

1. Triggers on PRs touching the build pipeline configs (or manually
   via workflow_dispatch).
2. Sets up Python 3.11 + Node 22 + PyYAML.
3. Runs `./scripts/build.sh` from a clean checkout.
4. Detects output drift (uncommitted changes in
   `superset/static/assets/country-maps/` or `src/data/manifest.json`).
5. If drift detected on a PR: leaves a comment telling the contributor
   to re-run the build locally and commit, then fails CI.

This forces "configs and outputs stay in sync" — a contributor can't
add a name override / region aggregation / composite without also
updating the committed GeoJSON outputs.

Tagged contents permissions only — does not auto-PR (which would
require deeper repo setup); contributors are expected to regenerate
locally and push.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:35:47 -07:00
Evan RusackasandClaude Opus 4.7 2bbbee6cec docs(country-map): UPDATING.md entry for plugin replacement
Add a "Country Map plugin redesigned (legacy plugin deprecated)"
section under the Next release covering:

- What changed (modern endpoint, worldview, Admin 0+1, aggregated
  regions, composite maps, region include/exclude, flying-islands
  toggle, name-language selector, build pipeline)
- Migration behavior (existing dashboards keep working, no DB
  migrations, deprecation badge in chart picker)
- For maintainers/power users (the notebook is no longer the source
  of truth; local touchups go in YAML configs or procedural/ scripts)

Pointer to SIP_DRAFT.md for full design rationale.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:34:49 -07:00
Evan RusackasandClaude Opus 4.7 f1ff71e849 feat(country-map): port click-to-zoom interaction from legacy plugin
Click a region: viewport zooms in (4x) centered on its centroid.
Click again or click the empty background: zoom back out.
600ms CSS transition on the SVG group.

Implementation note: zoom state lives on the DOM (zoomedFeature
local + group transform attribute) rather than React state. Two
reasons:
- Cleaner unmount (the transition completes naturally; no orphaned
  React state)
- Avoids a re-render cycle on every click that would re-fit the
  projection and undo the zoom

Background rect added to capture clicks outside any feature.

Cross-filter integration intentionally NOT added in this commit —
that needs the chart props' setDataMask path and a settings UI for
which feature property to use as the filter key. Separate concern,
follow-up commit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:33:40 -07:00
Evan RusackasandClaude Opus 4.7 5efb93b99b feat(country-map): manifest-driven control choices
Replace hardcoded WORLDVIEW / COUNTRY / REGION_SET / COMPOSITE choice
tables in controlPanel with options derived from the build pipeline's
manifest.json. Adding a new entry to a YAML config + re-running
./build.sh now populates the control automatically; no plugin code
change needed.

Build script: also writes manifest.json to the plugin's
src/data/manifest.json so controlPanel can `import` it synchronously
(no async fetch needed at chart-edit time). Data files (the actual
GeoJSONs) still live only at superset/static/assets/country-maps/ —
we don't want to bundle 17MB of choropleth data into the JS payload.

Lookup helpers preserve human-friendly labels:
- WORLDVIEW_LABELS — maps NE worldview codes (ukr, default, ind, ...)
  to friendly names ("Ukraine (default — Crimea as Ukrainian)" etc.);
  unmapped codes render as the raw code
- COUNTRY_LABELS — ISO_A3 → English country name (~85 entries);
  formatCountry renders as "France (FRA)"; unmapped codes render raw
- REGION_SET_LABELS / COMPOSITE_LABELS — same pattern

Manifest's regional_aggregations array is grouped by country into
the {country: [(set_id, label), ...]} shape the control panel needs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:32:52 -07:00
Evan RusackasandClaude Opus 4.7 1817e37b06 feat(country-map): wire build outputs to Flask static path + commit them
Move build pipeline outputs from a sibling `output/` directory into
Superset's Flask-served `superset/static/assets/country-maps/` so the
plugin can fetch them at runtime without webpack involvement (Flask
serves the tree at `/static/...` directly).

Commit the 220 generated GeoJSONs + manifest.json so a fresh
ephemeral environment can render the chart immediately, no build
step required at deploy time. Trade-off: ~17 MB of generated files
in-tree. (For comparison the legacy plugin commits ~34 MB of
GeoJSON; net change is -17 MB once we remove the legacy plugin in a
future major version.)

Files committed:
  superset/static/assets/country-maps/
    README.md                                         (humans, not data)
    manifest.json
    ukr_admin0.geo.json                          2.1 MB
    ukr_admin1_<adm0_a3>.geo.json    × 214      ~50 KB - 662 KB each
    regional_<country>_<set>_ukr.geo.json × 4   ~30 KB each
    composite_france_overseas_ukr.geo.json       322 KB

Build script changes:
- OUTPUT_DIR computed via SCRIPT_DIR.parents[3] / "superset" /
  "static" / "assets" / "country-maps"
- mkdir(parents=True) so a fresh checkout works first run
- Stale `output/` entry kept in scripts/.gitignore for safety
  (some local checkouts may have it from earlier iterations)

.gitignore: add re-include lines so superset/static/assets/country-maps/**
gets committed despite the broader superset/static/* exclusion.

SIP_DRAFT updated with the hosting-decision rationale.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:26:39 -07:00
Evan RusackasandClaude Opus 4.7 baf33093c2 docs(country-map): SIP — sync impl plan with renderer + controls + viz_type wiring
Mark Phase 2 substantially done (renderer ported, transformProps wired,
fit-to-selection works), Phase 3 done (full control panel committed),
Phase 4 mostly done (viz_type registered, legacy marked deprecated;
"switch to new" button still TODO), and Phase 5 partially done
(manifest + transformProps/buildQuery tests).

Remaining items for full POC: GeoJSON hosting path, click-to-zoom,
cross-filter, composite-projection render support, manifest-driven
controls, "Switch to new chart" button, CI workflow, example images,
UPDATING.md entry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:10:54 -07:00
Evan RusackasandClaude Opus 4.7 adc36ccf6d feat(country-map): wire new plugin into VizType registry + deprecate legacy
Three coordinated changes:

1. **VizType.CountryMapV2 = 'country_map_v2'** — new enum value for the
   modern plugin's chart type. Existing dashboards continue to use
   `country_map` (legacy plugin); new charts get `country_map_v2`.
   Naming follows the established pattern (LegacyBubble='bubble',
   Bubble='bubble_v2'). Eventual swap (in a major version) can rename
   so 'country_map' points to the new plugin.

2. **Legacy plugin marked deprecated**:
   - Display name changed to "Country Map (Legacy)" so the chart picker
     shows them side-by-side unambiguously
   - label: ChartLabel.Deprecated — surfaces the standard deprecation
     badge in the UI without hiding from the picker (existing dashboards
     keep working; users see "this is deprecated" when editing)
   - labelExplanation pointing users at the new chart

3. **MainPreset registers both**:
   - CountryMapChartPlugin (legacy) → VizType.CountryMap
   - CountryMapV2ChartPlugin (new)  → VizType.CountryMapV2

Plus added @superset-ui/plugin-chart-country-map as a workspace
dependency in superset-frontend/package.json so the symlink resolves
during npm install.

Not yet wired (next commits):
- "Switch to new Country Map" button on the legacy plugin's UI
  (the deprecation banner shows; the button to migrate form_data
  is a separate change in the explore controls)
- Hosting path for the new plugin's GeoJSON outputs (currently
  stubbed in transformProps — no static file serving wired yet)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:10:03 -07:00
Evan RusackasandClaude Opus 4.7 cb5bb69fa8 feat(country-map): build script — emit manifest.json describing outputs
Adds a manifest.json output the plugin can fetch at runtime to
populate worldview / country / region-set / composite dropdowns
dynamically. Adding a new entry to the YAML configs no longer requires
a plugin code change — re-run build.sh, manifest updates, plugin
controls reflect the new options.

Manifest schema:
  ne_pinned_tag, ne_pinned_sha, build_timestamp_utc
  worldviews: [<wv>, ...]
  admin_levels: [0, 1]
  countries_by_worldview: {<wv>: [<adm0_a3>, ...]}
  regional_aggregations: [{country, set_id, worldview, size_bytes}, ...]
  composites: [{id, worldview, size_bytes}, ...]

Sample current output:
  1 worldview (ukr), 211 countries with subdivisions, 4 regional sets,
  1 composite (france_overseas). Build pinned to NE v5.1.2.

Follow-up commit will replace the hardcoded choice tables in
controlPanel.tsx with manifest-driven options.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:06:30 -07:00
Evan RusackasandClaude Opus 4.7 52dfc853d2 test(country-map): transformProps + buildQuery tests
transformProps tests cover the URL-derivation logic — the core piece
that maps form_data into the right build-pipeline output:

- Admin 0 + worldview → world choropleth URL
- Admin 1 + country → per-country file
- Region set + country → regional aggregation
- Composite overrides admin level + country
- Worldview defaults to 'ukr' when not specified
- Different worldviews reflected in URL
- Admin 1 without country → null URL (chart UI prompts)
- Pass-through of metricName/numberFormat/linearColorScheme
- Pass-through of query data + width/height

buildQuery tests cover that we're producing a valid chart/data
QueryContext with the expected shape (one query, form_data preserved,
orderby normalized as array).

These are the units most likely to break silently if someone refactors
the form_data → URL mapping or the query layer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:01:25 -07:00
Evan RusackasandClaude Opus 4.7 f696fe1e05 feat(country-map): full control panel — worldview, admin level, region filters, language
Replace the placeholder control panel with the real form-data surface.
New controls (none of which exist on the legacy plugin):

- worldview: SelectControl, defaults to 'ukr' per documented editorial
  choice; configurable per-deployment + per-chart
- admin_level: SelectControl (Admin 0 / Admin 1 / Aggregated regions)
- country: SelectControl, visible only when admin_level !== 0 and no
  composite is selected
- region_set: SelectControl, visible only when admin_level === 'aggregated';
  choices depend on selected country (sourced from the regional_aggregations
  YAML's snapshot table)
- composite: SelectControl (france_overseas etc.); when set, overrides
  admin_level + country
- region_includes / region_excludes: free-form multi-select for client-side
  filtering at render time (these drive the projection refit too — the
  renderer's `filterFeatures` uses them)
- show_flying_islands: CheckboxControl, default true
- name_language: SelectControl with 20 NE-supported languages

The choice tables (WORLDVIEW_CHOICES, COUNTRY_CHOICES, REGION_SET_CHOICES_
BY_COUNTRY, COMPOSITE_CHOICES) are hardcoded snapshots for the POC.
Follow-up commit will replace these with options pulled from the build
pipeline's manifest.json so adding a new worldview or country only
requires a build script run, not a plugin code change.

Standard query controls (entity, metric, adhoc_filters, row_limit,
linear_color_scheme, number_format) preserved from the legacy plugin.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 17:00:08 -07:00
Evan RusackasandClaude Opus 4.7 b296345332 feat(country-map): port D3 renderer to modern modules + new features
Replace the placeholder renderer with a real D3-based choropleth.
Modernizes the legacy plugin's rendering in three ways:

1. **Modern D3 modules** (d3-geo, d3-array, d3-color) instead of the
   d3 v3 monolith. No more @ts-nocheck — fully typed.

2. **Built-in include/exclude filtering** (formData.region_includes /
   region_excludes) before render. Projection auto-fits the FILTERED
   feature set, so excluding Russia from a Europe view actually zooms
   to Europe (the "fit to selection" behavior).

3. **Multi-language display names** via name_language formData field.
   Falls back through name_<lang> → name → '' so any worldview/dataset
   degrades gracefully.

Other changes:
- Tooltip is React state instead of D3-driven HTML (cleaner unmount,
  no XSS surface from .html())
- Color scale uses sequential scheme registry from @superset-ui/core
  (same path as legacy)
- Cleanup on unmount via useEffect cleanup function

Stubbed for follow-ups:
- Click-to-zoom (legacy had this; revisit after deprecation banner)
- Cross-filter integration
- Composite projection support (geoAlbersUsa for USA, etc.) — falls
  back to plain Mercator + build-time repositioning for now
- Flying-islands toggle: feature-tagging in the build pipeline still
  needs to land before the toggle can drop them at render time

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:58:38 -07:00
Evan RusackasandClaude Opus 4.7 2165b95a08 docs(country-map): SIP — add test plan section for maintainer validation
Captures things the maintainer should manually verify before merge,
organized by category:
- Build pipeline (data correctness)
- Visual/cartographic quality (in-chart rendering)
- Controls UX
- Backward compatibility
- Per-deployment customization (config)
- Performance + ops

Items get added/checked off as the implementation progresses.
Includes specific spot-checks the agent can't validate alone:
Crimea-as-Ukrainian visual, Russia Chukchi connectedness, India
J&K under _ukr worldview, French composite layout, etc.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:57:03 -07:00
Evan RusackasandClaude Opus 4.7 fb31a5160d docs(country-map): SIP — sync implementation plan with current progress
Mark Phase 1 substantially complete (5/5 transforms working,
220 outputs verified, all schemas in place) and Phase 2 partially
complete (modern plugin scaffold compiles end-to-end, real renderer +
controls still TODO).

Add a "Current PR state" section showing the build script's actual
output so reviewers reading the SIP can see at a glance what the POC
produces today.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:45:49 -07:00
Evan RusackasandClaude Opus 4.7 07532ae1a7 feat(country-map): scaffold modern plugin shell
Bootstrap the new @superset-ui/plugin-chart-country-map package with
the modern ChartPlugin pattern. Mirrors plugin-chart-handlebars'
structure (the leanest existing modern plugin in the repo).

Files:
  package.json                     — workspace pkg with d3-* deps
  tsconfig.json                    — extends frontend root tsconfig,
                                     references core/chart-controls
  types/external.d.ts              — ambient types for png/jpg/geojson
  README.md                        — what this plugin is + status
  src/
    index.ts                       — package entry
    types.ts                       — CountryMapFormData + transform props
    CountryMap.tsx                 — placeholder renderer (real D3 port
                                     in a follow-up commit)
    plugin/
      index.ts                     — ChartPlugin class with metadata
      buildQuery.ts                — modern chart/data query (this is
                                     the core difference from the legacy
                                     plugin's explore_json path)
      controlPanel.tsx             — minimal placeholder; full control
                                     set (worldview / admin level /
                                     country / region include-exclude /
                                     fly-islands / language) ports next
      transformProps.ts            — derives geoJsonUrl from form_data
                                     using the build-script's output
                                     naming convention

Renderer scaffold fetches the resolved GeoJSON URL and shows a small
diagnostic block (URL + feature count + data row count + metric name)
so the plugin compiles end-to-end and can be wired to the chart-type
registry without a real D3 render. Real renderer + controls land in
the next commits.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:44:43 -07:00
Evan RusackasandClaude Opus 4.7 d7edbf747a feat(country-map): build script — split Admin 1 per country
The monolithic ukr_admin1.geo.json (15MB / 4595 features) was a single
file that any chart would have to download in full just to render one
country's subdivisions. Replace with per-country files keyed by
adm0_a3, each individually simplified.

Also drops single-subdivision countries (useless as choropleths) at
this stage, mirroring the notebook's auto-purge.

Output stats from full run:
  Files: 220 total
    1 × admin0 (world) ............ 2.1 MB
    4 × regional aggregations ..... 23-32 KB each
    1 × composite (france_overseas) 322 KB
  214 × per-country admin1 ........ 17 KB - 662 KB each (GBR largest)

Per-chart payload:
  world choropleth   → ukr_admin0.geo.json                  2.1 MB
  France departments → ukr_admin1_FRA.geo.json              308 KB
  US states          → ukr_admin1_USA.geo.json              ~250 KB
  Türkiye NUTS-1     → regional_TUR_nuts_1_ukr.geo.json     23 KB
  France w/ overseas → composite_france_overseas_ukr.geo.json 322 KB

All well within usable browser payload range. The plugin will lazy-load
only what's needed for the current chart's worldview/admin-level/country.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:39:47 -07:00
Evan RusackasandClaude Opus 4.7 989ed61f34 feat(country-map): build script — composite_maps transform (5/5 transforms)
Implements the fifth and final transform from the notebook audit.
A composite combines a base country's Admin 1 features with:
- base_repositions (with optional `group: true` for grouped transforms
  like Paris + petite couronne treated as one body)
- additions (features pulled from sibling countries' Admin 1, with
  optional dissolve, drop_parts, reposition, and attribute set)

Verified on France-with-Overseas:
  france_overseas: 108 features → composite_france_overseas_ukr.geo.json
                                  (322,058 bytes)

108 = 101 FRA admin1 departments + 7 additions (Polynésie française,
Terres australes et antarctiques françaises, Wallis-et-Futuna,
Nouvelle-Calédonie, Saint-Pierre-et-Miquelon, Saint-Martin,
Saint-Barthélémy).

Bug fix during implementation: composites pull additions from Admin 1
of sibling countries (Windward Islands is a PYF Admin 1 subdivision,
not an Admin 0 country), not from Admin 0. Initial implementation got
this wrong and warned 0 features. Fixed by sourcing from base_admin1
(the global Admin 1 dataset, which contains all countries'
subdivisions).

New helpers:
- _drop_parts(geom, indices) — drop sub-polygon indices from MultiPolygon
- _translate_and_scale_with_pivot — explicit pivot (vs feature centroid),
  used for `group: true` transforms

==== Build pipeline status ====

All 5 declarative transforms implemented and verified:
  ✓ name_overrides         (19 updates per Admin 1 build)
  ✓ flying_islands         (12 reposition + 5 bbox drop)
  ✓ territory_assignments  (4 features added: TWN/HKG/MAC/ALD)
  ✓ regional_aggregations  (4 region sets: TUR/FRA/ITA/PHL)
  ✓ composite_maps         (1 composite: france_overseas)

Current outputs (UA worldview):
  ukr_admin0.geo.json                       2.1 MB   249 features
  ukr_admin1.geo.json                        15 MB  4595 features
  regional_TUR_nuts_1_ukr.geo.json           23 KB    12 regions
  regional_FRA_regions_ukr.geo.json          32 KB    18 regions
  regional_ITA_regions_ukr.geo.json          32 KB    20 regions
  regional_PHL_regions_ukr.geo.json          32 KB    17 regions
  composite_france_overseas_ukr.geo.json    322 KB   108 features

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:34:43 -07:00
Evan RusackasandClaude Opus 4.7 1e77ff1706 feat(country-map): build script — regional_aggregations transform
Implements the fourth transform: dissolve Admin 1 subdivisions into
coarser administrative regions. Supports two mapping styles:
- explicit_mapping: per-region {name, members: [iso_3166_2, ...]}
- grouping_field: dissolve by an existing NE field (e.g. region_cod)

Verified counts match notebook expectations exactly:
  TUR/nuts_1:  81 subdivisions → 12 regions → 23 KB
  FRA/regions: 101 subdivisions → 18 regions → 32 KB
  ITA/regions: 110 subdivisions → 20 regions → 32 KB
  PHL/regions: 118 subdivisions → 17 regions → 32 KB

Per-region-set output is its own file (`regional_<country>_<set>_
<worldview>.geo.json`) so the plugin can lazy-load only what's needed
for the current chart.

Implementation:
- Filter base geo to country features
- Tag each with derived `_region_code` and `_region_name` (via reverse
  lookup of explicit_mapping, or via grouping_field value)
- mapshaper -dissolve handles the polygon merging in one pass
- Rename derived fields → standard `iso_3166_2` and `name` on output

Output sizes are tiny — each per-chart payload becomes ~30 KB instead
of pulling the full Admin 1 layer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:29:44 -07:00
Evan RusackasandClaude Opus 4.7 cb005a2ea5 feat(country-map): build script — territory_assignments transform
Implements the third transform: pull features from sibling Admin 0
records into a destination country's Admin 1 view. Used for:
- China + Taiwan/HK/Macau (NE keeps each as separate Admin 0)
- Finland + Åland (missing from FIN admin 1; NE keeps Åland as ALD
  admin 0)

Verified on real data:
  Building worldview=ukr admin_level=1
    territory_assignments: added 4 features from sibling Admin 0 records

(4 = TWN/HKG/MAC + ALD; ARMM-renamed BARMM region picks up correctly
because name_overrides ran first.)

Two bugs fixed along the way:

1. **Property name casing.** NE Admin 0 ships with uppercase property
   names (ADM0_A3, NAME_EN), Admin 1 with lowercase. All transforms
   downstream assume lowercase, so we now normalize to lowercase at
   shapefile-conversion time. Bonus: fixes a silent flying_islands
   bug where `adm0_a3` filters never matched at Admin 0 because the
   props were uppercase.

2. **drop_outside_bbox at Admin 0.** A country's multi-polygon often
   includes overseas territories (Netherlands → Caribbean), so bbox
   filtering at Admin 0 would drop entire countries. Now guarded to
   only run at Admin 1 where each feature is a single subdivision.

3. **Åland's NE code.** NE uses ALD, not the ISO 3166-1 ALA. Updated
   territory_assignments.yaml with comment noting the divergence.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:27:04 -07:00
Evan RusackasandClaude Opus 4.7 ae5e1132ba feat(country-map): build script — mapshaper -simplify final pass
Add mapshaper -simplify as a final-stage pass after the Python
transforms. Two-stage so the transforms operate on full-resolution
geometry (no risk of repositioning ghost features that got simplified
away) but the shipped output is browser-sized.

Default 5% with keep-shapes — preserves recognizable country shapes
while dropping the per-vertex bloat. keep-shapes prevents tiny features
(small island chains) from being collapsed away entirely.

Results on real outputs:
  Admin 0: 23.6 MB → 2.1 MB  (-91%, 249 features intact)
  Admin 1: 67.7 MB → 15 MB   (-78%, 4591 features, transforms preserved)

Per-feature simplify factors (the notebook had a size-based ladder)
can be added later if specific countries need tuning. 5% is the
mapshaper-recommended "good enough for web" default.

Open thought for future: split Admin 1 output by country
(`<worldview>_admin1_<adm0_a3>.geo.json`) so the per-chart payload is
~50KB-1MB instead of one 15MB global file. Will pair naturally with
the territory_assignments pass (which is per-country anyway).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:03:34 -07:00
Evan RusackasandClaude Opus 4.7 ea9b21b017 feat(country-map): build script — Admin 1 + flying_islands transform
Add Admin 1 build path and the second declarative transform. Exercises
the YAML config layer on real data:

  Building worldview=ukr admin_level=1
    loaded 4596 features
    name_overrides: applied 19 field updates across 10 entries
    flying_islands: repositioned 12 features, dropped 5 (outside-bbox)
    wrote ukr_admin1.geo.json (67,677,079 bytes, 4591 features)

Counts verified against expectations:
- 19 name_overrides = 2 France typos + 6 France ISO codes
  + 5 PHL Caraga renames + 6 PHL BARMM renames
- 12 repositions = 2 USA + 1 NOR + 2 PRT + 2 ESP + 5 FRA
- 5 drops = NLD Caribbean + GBR overseas territories

New: pure-Python translate/scale geometry transform (no shapely dep);
operates on Polygon/MultiPolygon coordinates. Scale pivot is the bbox
center of each matched feature — good enough for the visual layout
purposes we use it for. Output bbox correctness verified by counts.

Refactor: extract `build_one(worldview, admin_level, ...)` so the
target matrix can grow in subsequent commits.

What's stubbed (TODO inline): territory_assignments, composite_maps,
regional_aggregations, simplification, procedural/. Output is
uncompressed and unsimplified (67MB) — simplification will land with
the mapshaper -simplify pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:02:11 -07:00
Evan RusackasandClaude Opus 4.7 2db41bb2b2 feat(country-map): minimum-viable build pipeline (NE → GeoJSON)
End-to-end working pipeline replacing the legacy notebook for one
worldview / admin level. Verified locally:

  $ ./build.sh
  Country Map build — pinned to NE v5.1.2 (f1890d9f)
  Loaded 10 name override entries
  Building worldview=ukr admin_level=0
  Downloading NE ne_10m_admin_0_countries_ukr (worldview=ukr)…
    mapshaper: ne_10m_admin_0_countries_ukr.shp → _raw_ukr_admin0.geo.json
    loaded 249 features
    name_overrides: applied 0 field updates across 10 entries
    wrote .../output/ukr_admin0.geo.json (23,639,348 bytes)
  Done.

What's wired:
- NE download from pinned tag (v5.1.2 / SHA f1890d9f) with cache
- Shapefile → GeoJSON via mapshaper CLI
- YAML config loading (currently just name_overrides)
- name_overrides transform with {match, set} semantics, including
  the {in: [...]} list-membership matcher
- Output writes to scripts/output/ (gitignored)
- build.sh wrapper validates Python + Node + PyYAML are available

What's stubbed for future commits (TODO inline):
- Multiple worldviews (currently UA only)
- Admin 1 build (where name_overrides actually fire — currently no
  features in Admin 0 match the FRA/PHL admin1 entries)
- flying_islands, territory_assignments, regional_aggregations,
  composite_maps transforms
- Simplification (mapshaper -simplify)
- Procedural escape-hatch orchestration
- Manifest with NE SHA + build metadata

The 0 overrides applied is correct, not a bug: all current entries
target Admin 1 features.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 16:00:29 -07:00
Evan RusackasandClaude Opus 4.7 d2916b99ee feat(country-map): backfill France-with-Overseas composite from full notebook cell
The original draft was missing entries because notebook cell 63 was
truncated in the audit. Reading the full cell surfaced:

- Saint Martin (MAF) + Saint Barthélémy (BLM) as additional sister
  Admin 0 territories (small Caribbean islands, scaled up significantly
  for visibility — 5x and 8x respectively)
- Paris + petite couronne (Hauts-de-Seine, Seine-Saint-Denis,
  Val-de-Marne) as a metropolitan zoom-in (group + translate + scale 3x)
- Per-territory metadata renames (Polynésie française, Nouvelle-
  Calédonie, etc.) + ISO 3166-2 code assignments (FR-PF, FR-NC, etc.)

Schema additions:
- base_repositions[].group: true — when match yields multiple features,
  transform them as a single MultiPolygon then split back out
  (preserves per-feature attributes). Used for the Paris zoom-in.
- additions[].set: { name, iso_3166_2, ... } — override attributes on
  the added/dissolved feature

SPM offset placeholder is gone; composite definition now matches the
notebook's output exactly (modulo the build script implementing the
declarative schema).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:57:59 -07:00
Evan RusackasandClaude Opus 4.7 1eb48e94fc feat(country-map): scaffold scripts/ dir with YAML config schemas
First-pass schemas for the build pipeline's declarative config layer.
Each schema is documented inline + populated with concrete entries
ported from the legacy notebook's audited touchups (those that the
obsolescence check determined still need to ship).

scripts/
├── README.md                 — pipeline overview, layout, workflow
├── config/
│   ├── name_overrides.yaml         — France typos, ISO codes; PHL renames
│   ├── flying_islands.yaml         — USA/NOR/PRT/ESP/FRA repositions; NLD/GBR drops
│   ├── territory_assignments.yaml  — China + SARs; Finland + Åland
│   ├── regional_aggregations.yaml  — Turkey NUTS-1; FRA/ITA/PHL regions
│   └── composite_maps.yaml         — France-with-Overseas
└── procedural/
    └── README.md             — escape-hatch rules + skeleton (currently empty)

All five YAML files parse cleanly (validated with PyYAML).

Schema design choices:
- Every entry has a `description:` field. Forces honest documentation
  of why each fix exists; reviewers can scan rationale at a glance.
- Match semantics: simple AND-of-conditions; supports `{ in: [...] }`
  for value-set matching.
- composite_maps and territory_assignments share the "pull feature
  from sibling Admin 0" primitive; build script can implement once.
- composite_maps.yaml has a TODO marker for SPM offsets — notebook
  cell 63 was truncated in the audit; will backfill during build
  script implementation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:56:04 -07:00
Evan RusackasandClaude Opus 4.7 1fffd9d832 docs(country-map): SIP — add procedural/ escape hatch for edge cases
Most touchups fit declarative YAML cleanly (typos, ISO codes,
repositions, dissolves, composites). A few don't — e.g., the
France-with-Overseas notebook drops a specific sub-polygon by index
to avoid visual conflict with Corsica.

Add a `procedural/` directory of small, named, single-purpose Python
scripts as an escape hatch. Build orchestrator runs YAML transforms
first, then iterates procedural/*.py in numeric order.

Why this is much better than the notebook for the same content:
- Each fix is a separate file → conflicts localize to one fix at a
  time, not "the whole notebook touched the same cell"
- No kernel state, no output churn, pure functions on data
- Naming forces documentation; reviewers see add/remove at a glance
- Bounded growth: a 50-file procedural/ dir is annoying enough to
  signal "push back into YAML or upstream"; the notebook had no
  such signal and just bloated

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:50:05 -07:00
Evan RusackasandClaude Opus 4.7 1e349d26a4 docs(country-map): SIP — obsolescence check vs current NE 5.x
Verified each notebook touchup against current Natural Earth Admin 1
data. Headline wins:

OBSOLETE (NE caught up or worldview handles it):
- Vietnam diacritics: NE has correct values in name_vi field; manual
  rewrites collapse to zero by exposing name_language=vi
- Crimea/Sevastopol: handled by _ukr worldview (already confirmed)
- India Kashmir/Ladakh geometry: NE has UTs with correct ISO codes;
  worldview selection adjusts northern boundary
- Russia Chukchi antimeridian: NE has split geometry into properly-
  bounded polygons (x range [-180, 180]); D3 projection should handle
- Latvia third-party file: NE has 119 fine-grained admin1 features
- Philippines third-party file: NE has 118 admin1 features
- external_overrides.yaml DROPPED from the design

STILL NEEDED (NE still wrong / no upstream support):
- France typos (Seien-et-Marne, Haute-Rhin still in NE)
- France ISO codes (FR-75 → FR-75C, FR-GP → FR-971, etc. — 6 features)
- Philippines admin renames (Dinagat→Caraga, ARMM→BARMM, 11 features)
- China + SARs (Taiwan/HK/Macau still separate Admin 0 in NE)
- Finland + Åland (missing from FIN admin 1 in NE)
- Regional aggregations (Turkey NUTS-1, France/Italy/PHL regions)
- France-with-Overseas composite
- Flying-island repositions (USA, Norway, Portugal, Spain, France DOM)

Net: design surface shrinks, no third-party data dependencies in-tree,
config files stable enough that ongoing maintenance is small.

Two follow-up verifications still pending: Russia Chukchi renders
correctly under D3, India J&K geometry matches current Indian
boundary expectations under _ukr worldview.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:43:36 -07:00
Evan RusackasandClaude Opus 4.7 b12044cec7 docs(country-map): SIP — audit notebook touchups, enumerate config surface
Read all 96 cells of the legacy notebook; categorized every kind of
work it does into 14 buckets and mapped each to either:
-  already covered by the proposed design
- 🟡 covered but requires a YAML config we'll need to port
- 🟠 needs a NEW config file or pipeline feature beyond what was sketched
-  might be obsolete in current NE — verify per case

Surfaced four NEW config-driven build features beyond the original
sketch:
- territory_assignments.yaml (China + SARs, Finland + Åland)
- regional_aggregations.yaml (Turkey NUTS-1, France/Italy/Philippines
  regions; new "Aggregated regions" admin level)
- composite_maps.yaml (France-with-Overseas multi-country composite)
- external_overrides.yaml (India/Latvia/Philippines third-party
  GeoJSON; verify whether still needed against current NE)

Plus extended flying_islands.yaml with drop|reposition action modes
to subsume the bounds-clipping pattern (Netherlands, UK).

Updated open questions, smoke-test fixtures (now 5 cases that
exercise the full surface), and Phase 1 of the implementation plan
with the obsolescence check + external-GeoJSON check as concrete
follow-ups.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:33:54 -07:00
Evan RusackasandClaude Opus 4.7 a4b1ecafd1 docs(country-map): SIP draft for Country Map plugin redesign
Living document tracking the design for the Country Map rework. Will
be filed as a proper SIP issue when the POC matures; this file is
working scratch in the meantime.

Captured so far:
- Motivation: disputed-borders flashpoints, Jupyter notebook pain,
  legacy explore_json endpoint
- Goals: configurable worldview, Admin 0 + Admin 1, per-deployment
  + per-chart customization, mapshaper-based reproducible build
  pipeline, modern chart/data endpoint
- Spike findings: NE UA worldview vs Default — confirms Crimea-as-
  Ukrainian + several other commonly-expected positions (Kosovo,
  Western Sahara, Palestine, Cyprus, Kashmir)
- Deprecation plan: in-UI "switch to new Country Map" button + banner
  on legacy, modeled on existing deprecated-chart pattern

Open questions tracked inline; implementation phases sketched.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 15:16:03 -07:00
524 changed files with 23001 additions and 11404 deletions
+5
View File
@@ -43,6 +43,11 @@ superset-frontend/cypress-base/
superset-frontend/coverage/
superset-frontend/.temp_cache/
superset/static/assets/
# …but ship the committed Country Map GeoJSON outputs, which live under
# this otherwise webpack-managed path. Without this re-include the
# country-map plugin gets a 404 fetching every map file in dockerized
# deployments (including ephemeral PR previews).
!superset/static/assets/country-maps/
superset-websocket/dist/
venv
.venv
-4
View File
@@ -8,10 +8,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
pull-requests: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
+2 -2
View File
@@ -41,7 +41,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4
uses: github/codeql-action/init@68bde559dea0fdcac2102bfdf6230c5f70eb485e # v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -53,6 +53,6 @@ jobs:
- name: Perform CodeQL Analysis
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: github/codeql-action/analyze@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4
uses: github/codeql-action/analyze@68bde559dea0fdcac2102bfdf6230c5f70eb485e # v4
with:
category: "/language:${{matrix.language}}"
@@ -0,0 +1,126 @@
# Re-runs the Country Map plugin's build pipeline whenever its YAML
# configs change, then opens a PR with the updated GeoJSON outputs.
# Verifies the pipeline stays reproducible and surfaces cartographic
# diffs to maintainers as legible GeoJSON, not opaque notebook JSON.
name: country-map / regenerate outputs
on:
pull_request:
paths:
- 'superset-frontend/plugins/plugin-chart-country-map/scripts/**'
workflow_dispatch:
permissions:
contents: read
pull-requests: write
jobs:
regen:
name: Regenerate GeoJSON + manifest
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
- name: Setup Node
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Install build script deps
run: |
python -m pip install --upgrade pip
pip install pyyaml
- name: Run unit tests for build transforms
run: |
cd superset-frontend/plugins/plugin-chart-country-map/scripts
python -m unittest test_build -v
- name: Run build pipeline
run: |
cd superset-frontend/plugins/plugin-chart-country-map/scripts
./build.sh
- name: Detect output drift
id: drift
run: |
if [ -n "$(git status --porcelain superset/static/assets/country-maps/ \
superset-frontend/plugins/plugin-chart-country-map/src/data/manifest.json)" ]; then
echo "drift=true" >> "$GITHUB_OUTPUT"
echo "Outputs differ from committed snapshot:"
git status --short superset/static/assets/country-maps/
else
echo "drift=false" >> "$GITHUB_OUTPUT"
echo "Outputs match committed snapshot — nothing to do."
fi
# Sticky comment: every push to a PR retriggers this workflow, so
# we'd otherwise stack a fresh drift comment on every commit.
# Look for an existing comment carrying our marker and edit it in
# place; only post a new one the first time.
- name: Comment on PR if drift (sticky)
if: steps.drift.outputs.drift == 'true' && github.event_name == 'pull_request'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
COMMIT: ${{ github.event.pull_request.head.sha }}
run: |
MARKER="<!-- country-map-drift-bot -->"
BODY="$MARKER
⚠️ **Country Map**: drift detected between the locally committed GeoJSON outputs and what CI just regenerated for commit \`${COMMIT:0:7}\`. Either the YAML configs were updated without re-running \`./scripts/build.sh\`, OR mapshaper output differs cross-platform (macOS dev vs Linux CI). Investigate before merge.
_This comment is sticky and updated on every push; old drift comments are not stacked._"
# Find an existing sticky comment by marker
EXISTING=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
| head -n1)
if [ -n "$EXISTING" ]; then
gh api -X PATCH "repos/${REPO}/issues/comments/${EXISTING}" -f body="$BODY"
else
gh pr comment "$PR_NUMBER" --repo "$REPO" --body "$BODY"
fi
# If a previous push had drift but this one doesn't, remove the
# sticky comment so the PR doesn't carry a stale warning.
- name: Clear sticky comment if drift resolved
if: steps.drift.outputs.drift == 'false' && github.event_name == 'pull_request'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
MARKER="<!-- country-map-drift-bot -->"
EXISTING=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
| head -n1)
if [ -n "$EXISTING" ]; then
gh api -X DELETE "repos/${REPO}/issues/comments/${EXISTING}"
fi
# Informational only: cross-platform mapshaper output reproducibility
# is still being worked through. Don't block PRs on drift; surface
# via the comment above and a workflow summary.
- name: Summarize drift (informational)
if: steps.drift.outputs.drift == 'true' && github.event_name == 'pull_request'
run: |
{
echo "## Country Map: output drift detected (informational)";
echo "";
echo "Files differing from committed snapshot:";
echo '```';
git status --short superset/static/assets/country-maps/ \
superset-frontend/plugins/plugin-chart-country-map/src/data/manifest.json;
echo '```';
} >> "$GITHUB_STEP_SUMMARY"
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
- name: "Checkout Repository"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Dependency Review"
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4.9.0
continue-on-error: true
with:
fail-on-severity: critical
-4
View File
@@ -9,10 +9,6 @@ on:
branches:
- "master"
permissions:
contents: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
@@ -6,9 +6,6 @@ on:
- "master"
- "[0-9].[0-9]*"
permissions:
contents: read
jobs:
config:
runs-on: ubuntu-24.04
-3
View File
@@ -6,9 +6,6 @@ on:
- "superset-embedded-sdk/**"
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -6,9 +6,6 @@ on:
- "master"
- "[0-9].[0-9]*"
permissions:
contents: read
jobs:
config:
runs-on: ubuntu-24.04
@@ -8,9 +8,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
jobs:
validate-all-ghas:
-3
View File
@@ -4,9 +4,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
-4
View File
@@ -8,10 +8,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
pull-requests: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -78,13 +78,6 @@ jobs:
- name: yarn install
run: |
yarn install --check-cache
- name: Lint docs links
# Fast source-level check for bare relative internal links
# like `[Foo](../foo)` that Docusaurus's onBrokenLinks
# setting can't catch. Runs in seconds; fails fast before
# the expensive build step.
run: |
yarn lint:docs-links
- name: yarn typecheck
run: |
yarn typecheck
@@ -8,10 +8,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
pull-requests: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
-3
View File
@@ -6,9 +6,6 @@ on:
paths:
- "helm/**"
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -8,10 +8,6 @@ on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
pull-requests: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
-3
View File
@@ -11,9 +11,6 @@ on:
- "superset-websocket/**"
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
+6
View File
@@ -70,8 +70,14 @@ superset-websocket/config.json
node_modules
npm-debug.log*
superset/static/*
!superset/static/assets/
superset/static/assets/*
!superset/static/assets/.gitkeep
# Country Map plugin's generated GeoJSON outputs are committed so a
# fresh ephemeral env can render the chart without first running the
# build pipeline. See superset/static/assets/country-maps/README.md.
!superset/static/assets/country-maps/
!superset/static/assets/country-maps/**
superset/static/uploads/*
!superset/static/uploads/.gitkeep
yarn-error.log
+1 -1
View File
@@ -50,7 +50,7 @@ repos:
hooks:
- id: check-docstring-first
- id: check-added-large-files
exclude: ^.*\.(geojson)$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$|^superset/examples/.*/data\.parquet$
exclude: ^.*\.(geojson)$|^.*\.geo\.json$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$|^superset/examples/.*/data\.parquet$
- id: check-yaml
exclude: ^helm/superset/templates/
- id: debug-statements
-3
View File
@@ -43,9 +43,6 @@ _build/*
_static/*
.buildinfo
searchindex.js
# auto-generated by docs/scripts/convert-api-sidebar.mjs from openapi.json
sidebar.js
sidebar.ts
# auto generated
requirements/*
# vendorized
+23
View File
@@ -24,6 +24,29 @@ assists people when migrating to a new version.
## Next
### Country Map plugin redesigned (legacy plugin deprecated)
A new Country Map chart plugin (`@superset-ui/plugin-chart-country-map`, `viz_type='country_map_v2'`) replaces the legacy plugin (`@superset-ui/legacy-plugin-chart-country-map`, `viz_type='country_map'`). The legacy plugin is still installed and functional — existing dashboards continue to render unchanged — but is now badged as deprecated in the chart-type picker, with the display name "Country Map (Legacy)".
**What changed:**
- Modern `chart/data` endpoint instead of `explore_json` (full async / caching / semantic-layer integration)
- Configurable per-deployment + per-chart **worldview** for disputed regions (default ships Natural Earth's `_ukr` worldview, which shows Crimea as Ukrainian and aligns with broadly-expected positions on Kosovo, Western Sahara, Palestine, Cyprus, Kashmir; configurable via `superset_config.COUNTRY_MAP.default_worldview`)
- Both **Admin 0** (countries) **and Admin 1** (subdivisions) supported in one plugin (subsumes years of bespoke per-country submissions like French departments, Italian regions, Türkiye city map)
- New **Aggregated regions** admin level (Türkiye NUTS-1, France/Italy/Philippines administrative regions)
- New **composite maps** (e.g. France with overseas territories combining mainland + DROMs + 6 sister Admin 0 records)
- Per-chart **region include/exclude**, **flying-islands toggle**, **name-language selector**
- Build pipeline replaces the legacy Jupyter notebook with `mapshaper`-CLI-based reproducible scripts + 5 declarative YAML configs in `superset-frontend/plugins/plugin-chart-country-map/scripts/`
**Migration behavior:**
- Existing charts using `viz_type='country_map'` continue to render against the legacy plugin. No DB migrations needed.
- The legacy plugin's chart picker entry shows the standard Deprecation badge with an explanation pointing at the new chart type.
- Users can re-create existing charts using the modern plugin manually; an automated "Switch to new chart" button is planned.
- The legacy plugin (and its committed `src/countries/*.geojson` assets) will be removed in a future major release once existing dashboards have migrated.
**For maintainers / power users with custom modifications:**
- The legacy plugin's Jupyter notebook (`scripts/Country Map GeoJSON Generator.ipynb`) is no longer the source of truth. Local touchups should be ported to the new plugin's YAML configs (`scripts/config/{name_overrides,flying_islands,territory_assignments,regional_aggregations,composite_maps}.yaml`) or, for genuine edge cases that don't fit YAML, to the `scripts/procedural/` escape-hatch directory.
- See `superset-frontend/plugins/plugin-chart-country-map/SIP_DRAFT.md` for the full design rationale.
### Granular Export Controls
A new feature flag `GRANULAR_EXPORT_CONTROLS` introduces three fine-grained permissions that replace the legacy `can_csv` permission:
+22 -32
View File
@@ -31,9 +31,8 @@ You are currently in the `/docs` subdirectory of the Apache Superset repository.
├── superset-frontend/ # React/TypeScript frontend
└── docs/ # Documentation site (YOU ARE HERE)
├── docs/ # Main documentation content
├── admin_docs/ # Admin-focused guides
├── developer_docs/ # Developer guides
├── components/ # Component playground
├── developer_portal/ # Developer guides (currently disabled)
├── components/ # Component playground (currently disabled)
└── docusaurus.config.ts # Site configuration
```
@@ -47,19 +46,12 @@ yarn build # Build production site
yarn serve # Serve built site locally
# Version Management (USE THESE, NOT docusaurus commands)
# The add scripts auto-run `generate:smart` so auto-gen content (database
# pages, API reference, component pages) is fresh before snapshotting.
# For maximum-detail databases.json, drop the `database-diagnostics`
# artifact from Python-Integration CI at src/data/databases.json before
# cutting. See README.md "Before You Cut".
yarn version:add:user_docs <version> # Add new docs version
yarn version:add:admin_docs <version> # Add admin docs version
yarn version:add:developer_docs <version> # Add developer docs version
yarn version:add:docs <version> # Add new docs version
yarn version:add:developer_portal <version> # Add developer portal version
yarn version:add:components <version> # Add components version
yarn version:remove:user_docs <version> # Remove docs version
yarn version:remove:admin_docs <version> # Remove admin docs version
yarn version:remove:developer_docs <version> # Remove developer docs version
yarn version:remove:components <version> # Remove components version
yarn version:remove:docs <version> # Remove docs version
yarn version:remove:developer_portal <version> # Remove developer portal version
yarn version:remove:components <version> # Remove components version
# Quality Checks
yarn typecheck # TypeScript validation
@@ -103,14 +95,15 @@ docs/
└── [security guides]
```
### Admin Docs (`/admin_docs`)
Admin-focused content: installation, configuration, security.
### Developer Portal (`/developer_portal`) - Currently Disabled
When enabled, contains developer-focused content:
- API documentation
- Architecture guides
- CLI tools
- Code examples
### Developer Docs (`/developer_docs`)
Developer-focused content: API documentation, architecture guides, CLI tools, code examples.
### Component Playground (`/components`)
Interactive component examples for UI development.
### Component Playground (`/components`) - Currently Disabled
When enabled, provides interactive component examples for UI development.
## 📝 Documentation Standards
@@ -228,20 +221,17 @@ Versions are managed through `versions-config.json`:
```bash
# ✅ CORRECT - Updates both Docusaurus and versions-config.json
yarn version:add:user_docs 6.1.0
yarn version:add:docs 6.1.0
# ❌ WRONG - Only updates Docusaurus, breaks version dropdown
yarn docusaurus docs:version 6.1.0
```
### Version Files Created
When versioning, these files are created (per section, with the
section's plugin id as prefix):
- `<section>_versioned_docs/version-X.X.X/` - Snapshot of current docs
- `<section>_versioned_sidebars/version-X.X.X-sidebars.json` - Sidebar config
- `<section>_versions.json` - List of all versions
Section plugin ids: `user_docs`, `admin_docs`, `developer_docs`, `components`.
When versioning, these files are created:
- `versioned_docs/version-X.X.X/` - Snapshot of current docs
- `versioned_sidebars/version-X.X.X-sidebars.json` - Sidebar config
- `versions.json` - List of all versions
## 🎨 Styling and Theming
@@ -389,7 +379,7 @@ Docusaurus includes Algolia DocSearch integration configured in `docusaurus.conf
## 🚫 Common Pitfalls to Avoid
1. **Never use `yarn docusaurus docs:version`** - Use `yarn version:add:user_docs` instead
1. **Never use `yarn docusaurus docs:version`** - Use `yarn version:add:docs` instead
2. **Don't edit versioned docs directly** - Edit current docs and create new version
3. **Avoid absolute paths in links** - Use relative paths for maintainability
4. **Don't forget frontmatter** - Every doc needs title and description
@@ -419,7 +409,7 @@ yarn eslint
### Version Issues
If versions don't appear in dropdown:
1. Check `versions-config.json` includes the version
2. Verify version files exist in `<section>_versioned_docs/`
2. Verify version files exist in `versioned_docs/`
3. Restart dev server
## 📚 Resources
+22 -50
View File
@@ -37,45 +37,23 @@ Each section maintains its own version history and can be versioned independentl
To create a new version for any section, use the Docusaurus version command with the appropriate plugin ID or use our automated scripts:
#### Before You Cut
The cut snapshots whatever's on disk into a frozen historical version, including auto-generated content (database pages from `superset/db_engine_specs/`, API reference from `static/resources/openapi.json`, component pages from Storybook stories). The cut script refreshes these via `generate:smart` before snapshotting, but the **`databases.json` diagnostics file** needs special care to capture full detail:
1. **Canonical release cut**: download the `database-diagnostics` artifact from a green `Python-Integration` run on master, place it at `docs/src/data/databases.json`, then run the cut script with `--skip-generate` to preserve it. This is what the production deploy uses and includes full Flask-context diagnostics (driver versions, feature support matrix, etc.).
2. **Local dev cut**: just run the script normally. `generate:smart` will regenerate `databases.json` using your local Flask environment — accurate to whatever drivers/extras you have installed, but typically less complete than the CI artifact.
3. **No Flask available**: also fine — the database generator falls back to AST parsing of engine spec files. The MDX pages are still correct; only the diagnostics JSON is leaner.
Also: confirm `master` CI is green, and that your local checkout matches the SHA you intend to cut from.
#### Using Automated Scripts (Required)
**⚠️ Important:** Always use these custom commands instead of the native Docusaurus commands. These scripts ensure that both the Docusaurus versioning system AND the `versions-config.json` file are updated correctly, AND that auto-generated content is refreshed before snapshotting.
**⚠️ Important:** Always use these custom commands instead of the native Docusaurus commands. These scripts ensure that both the Docusaurus versioning system AND the `versions-config.json` file are updated correctly.
```bash
# Main Documentation
yarn version:add:user_docs 1.2.0
yarn version:add:docs 1.2.0
# Admin Docs
yarn version:add:admin_docs 1.2.0
# Developer Portal
yarn version:add:developer_portal 1.2.0
# Developer Docs
yarn version:add:developer_docs 1.2.0
# Component Playground
# Component Playground (when enabled)
yarn version:add:components 1.2.0
```
What the script does:
1. Refreshes auto-generated content via `generate:smart` (database pages, API reference, component pages).
2. Calls `yarn docusaurus docs:version` (or the per-section equivalent) to snapshot the section.
3. Freezes any data-file imports (`@site/static/*.json`, `../../data/*.json`) into a snapshot-local `_versioned_data/` dir so the historical version doesn't silently mutate when the source files change.
4. Adjusts relative import paths (`../../src/...``../../../src/...`) for files now one directory deeper.
5. Updates `versions-config.json` and `<section>_versions.json`.
**Do NOT use** the native Docusaurus commands directly (`yarn docusaurus docs:version`), as they will:
- ❌ Create version files but NOT update `versions-config.json`
- ❌ Skip auto-gen refresh, freezing whatever was on disk
- ❌ Skip data-import freezing, leaving the snapshot pointed at live data
- ❌ Cause versions to not appear in dropdown menus
- ❌ Require manual fixes to synchronize the configuration
@@ -98,7 +76,7 @@ If creating versions manually, you'll need to:
- **Versioned sidebars**: `[section]_versioned_sidebars/version-X.X.X-sidebars.json`
- **Versions list**: `[section]_versions.json`
All four sections (`user_docs`, `admin_docs`, `developer_docs`, `components`) follow this naming pattern uniformly.
Note: For main docs, the prefix is omitted (e.g., `versioned_docs/` instead of `docs_versioned_docs/`)
3. **Important**: After adding a version, restart the development server to see changes:
```bash
@@ -111,13 +89,10 @@ If creating versions manually, you'll need to:
#### Using Automated Scripts (Recommended)
```bash
# Main Documentation
yarn version:remove:user_docs 1.0.0
yarn version:remove:docs 1.0.0
# Admin Docs
yarn version:remove:admin_docs 1.0.0
# Developer Docs
yarn version:remove:developer_docs 1.0.0
# Developer Portal
yarn version:remove:developer_portal 1.0.0
# Component Playground
yarn version:remove:components 1.0.0
@@ -127,21 +102,18 @@ yarn version:remove:components 1.0.0
To manually remove a version:
1. **Delete the version folder** from the appropriate location:
- User Docs: `user_docs_versioned_docs/version-X.X.X/`
- Admin Docs: `admin_docs_versioned_docs/version-X.X.X/`
- Developer Docs: `developer_docs_versioned_docs/version-X.X.X/`
- Main docs: `versioned_docs/version-X.X.X/` (no prefix for main)
- Developer Portal: `developer_portal_versioned_docs/version-X.X.X/`
- Components: `components_versioned_docs/version-X.X.X/`
2. **Delete the version metadata file**:
- User Docs: `user_docs_versioned_sidebars/version-X.X.X-sidebars.json`
- Admin Docs: `admin_docs_versioned_sidebars/version-X.X.X-sidebars.json`
- Developer Docs: `developer_docs_versioned_sidebars/version-X.X.X-sidebars.json`
- Main docs: `versioned_sidebars/version-X.X.X-sidebars.json` (no prefix)
- Developer Portal: `developer_portal_versioned_sidebars/version-X.X.X-sidebars.json`
- Components: `components_versioned_sidebars/version-X.X.X-sidebars.json`
3. **Update the versions list file**:
- User Docs: `user_docs_versions.json`
- Admin Docs: `admin_docs_versions.json`
- Developer Docs: `developer_docs_versions.json`
- Main docs: `versions.json`
- Developer Portal: `developer_portal_versions.json`
- Components: `components_versions.json`
4. **Update configuration**:
@@ -173,12 +145,12 @@ docs: {
}
```
#### Developer Docs & Components (custom plugins)
#### Developer Portal & Components (custom plugins)
```typescript
{
id: 'developer_docs',
path: 'developer_docs',
routeBasePath: 'developer-docs',
id: 'developer_portal',
path: 'developer_portal',
routeBasePath: 'developer_portal',
includeCurrentVersion: true,
lastVersion: '1.1.0', // Default version
onlyIncludeVersions: ['current', '1.1.0', '1.0.0'],
@@ -212,8 +184,8 @@ docs: {
If you accidentally used `yarn docusaurus docs:version` instead of `yarn version:add`:
1. **Problem**: The version files were created but `versions-config.json` wasn't updated
2. **Solution**: Either:
- Revert the changes: `git restore user_docs_versions.json && rm -rf user_docs_versioned_docs/ user_docs_versioned_sidebars/`
- Then use the correct command: `yarn version:add:user_docs <version>`
- Revert the changes: `git restore versions.json && rm -rf versioned_docs/ versioned_sidebars/`
- Then use the correct command: `yarn version:add:docs <version>`
For other issues:
- **Restart the server**: Changes to version configuration require a server restart
@@ -222,7 +194,7 @@ For other issues:
#### Broken Links in Versioned Documentation
When creating a new version, links in the documentation are preserved as-is. Common issues:
- **Cross-section links**: Links between sections (e.g., from developer_docs to docs) need to be version-aware
- **Cross-section links**: Links between sections (e.g., from developer_portal to docs) need to be version-aware
- **Absolute vs relative paths**: Use relative paths within the same section
- **Version-specific URLs**: Update hardcoded URLs to use version variables
@@ -502,7 +502,6 @@ All MCP settings go in `superset_config.py`. Defaults are defined in `superset/m
| `MCP_DEBUG` | `False` | Enable debug logging |
| `MCP_DEV_USERNAME` | -- | Superset username for development mode (no auth) |
| `MCP_RBAC_ENABLED` | `True` | Enforce Superset's role-based access control on MCP tool calls. When `True`, each tool checks that the authenticated user has the required FAB permission before executing. Disable only for testing or trusted-network deployments. |
| `MCP_DISABLED_TOOLS` | `set()` | Set of tool names to remove from the MCP server at startup. Disabled tools are never advertised to AI clients during tool discovery. Useful when a custom extension tool should replace a built-in Superset tool. See [Disabling built-in tools](#disabling-built-in-tools). |
### Authentication
@@ -826,32 +825,6 @@ while True:
page += 1
```
## Disabling built-in tools
If you have deployed a custom tool via a Superset extension that supersedes one of the built-in Superset tools, you can suppress the built-in version so AI clients only discover your replacement. Disabled tools are removed from the server at startup and are never advertised during tool discovery.
Set `MCP_DISABLED_TOOLS` in your `superset_config.py` to a set of tool names:
```python
# superset_config.py
# Disable one tool
MCP_DISABLED_TOOLS = {"execute_sql"}
# Disable multiple tools
MCP_DISABLED_TOOLS = {"execute_sql", "health_check"}
```
Tool names match the function name used in the `@tool` decorator (e.g., `execute_sql`, `list_charts`, `health_check`). Extension-prefixed tools can also be disabled using their full prefixed name:
```python
MCP_DISABLED_TOOLS = {"extensions.myorg.myextension.some_tool"}
```
:::note
Specifying a tool name that does not exist logs a warning at startup and is otherwise ignored — it will not prevent the server from starting.
:::
## Security Best Practices
- **Use TLS** for all production MCP endpoints -- place the server behind a reverse proxy with HTTPS
+1
View File
@@ -0,0 +1 @@
[]
@@ -29,10 +29,10 @@ sidebar_position: 1
## Components
- [DropdownContainer](./dropdowncontainer.mdx)
- [Flex](./flex.mdx)
- [Grid](./grid.mdx)
- [Layout](./layout.mdx)
- [MetadataBar](./metadatabar.mdx)
- [Space](./space.mdx)
- [Table](./table.mdx)
- [DropdownContainer](./dropdowncontainer)
- [Flex](./flex)
- [Grid](./grid)
- [Layout](./layout)
- [MetadataBar](./metadatabar)
- [Space](./space)
- [Table](./table)
+1 -1
View File
@@ -62,7 +62,7 @@ This documentation is auto-generated from Storybook stories. To add or update co
4. Run `yarn generate:superset-components` in the `docs/` directory
:::info Work in Progress
This component library is actively being documented. See the [Components TODO](./TODO.md) page for a list of components awaiting documentation.
This component library is actively being documented. See the [Components TODO](./TODO) page for a list of components awaiting documentation.
:::
---
+46 -46
View File
@@ -29,49 +29,49 @@ sidebar_position: 1
## Components
- [AutoComplete](./autocomplete.mdx)
- [Avatar](./avatar.mdx)
- [Badge](./badge.mdx)
- [Breadcrumb](./breadcrumb.mdx)
- [Button](./button.mdx)
- [ButtonGroup](./buttongroup.mdx)
- [CachedLabel](./cachedlabel.mdx)
- [Card](./card.mdx)
- [Checkbox](./checkbox.mdx)
- [Collapse](./collapse.mdx)
- [DatePicker](./datepicker.mdx)
- [Divider](./divider.mdx)
- [EditableTitle](./editabletitle.mdx)
- [EmptyState](./emptystate.mdx)
- [FaveStar](./favestar.mdx)
- [IconButton](./iconbutton.mdx)
- [Icons](./icons.mdx)
- [IconTooltip](./icontooltip.mdx)
- [InfoTooltip](./infotooltip.mdx)
- [Input](./input.mdx)
- [Label](./label.mdx)
- [List](./list.mdx)
- [ListViewCard](./listviewcard.mdx)
- [Loading](./loading.mdx)
- [Menu](./menu.mdx)
- [Modal](./modal.mdx)
- [ModalTrigger](./modaltrigger.mdx)
- [Popover](./popover.mdx)
- [ProgressBar](./progressbar.mdx)
- [Radio](./radio.mdx)
- [SafeMarkdown](./safemarkdown.mdx)
- [Select](./select.mdx)
- [Skeleton](./skeleton.mdx)
- [Slider](./slider.mdx)
- [Steps](./steps.mdx)
- [Switch](./switch.mdx)
- [TableCollection](./tablecollection.mdx)
- [TableView](./tableview.mdx)
- [Tabs](./tabs.mdx)
- [Timer](./timer.mdx)
- [Tooltip](./tooltip.mdx)
- [Tree](./tree.mdx)
- [TreeSelect](./treeselect.mdx)
- [Typography](./typography.mdx)
- [UnsavedChangesModal](./unsavedchangesmodal.mdx)
- [Upload](./upload.mdx)
- [AutoComplete](./autocomplete)
- [Avatar](./avatar)
- [Badge](./badge)
- [Breadcrumb](./breadcrumb)
- [Button](./button)
- [ButtonGroup](./buttongroup)
- [CachedLabel](./cachedlabel)
- [Card](./card)
- [Checkbox](./checkbox)
- [Collapse](./collapse)
- [DatePicker](./datepicker)
- [Divider](./divider)
- [EditableTitle](./editabletitle)
- [EmptyState](./emptystate)
- [FaveStar](./favestar)
- [IconButton](./iconbutton)
- [Icons](./icons)
- [IconTooltip](./icontooltip)
- [InfoTooltip](./infotooltip)
- [Input](./input)
- [Label](./label)
- [List](./list)
- [ListViewCard](./listviewcard)
- [Loading](./loading)
- [Menu](./menu)
- [Modal](./modal)
- [ModalTrigger](./modaltrigger)
- [Popover](./popover)
- [ProgressBar](./progressbar)
- [Radio](./radio)
- [SafeMarkdown](./safemarkdown)
- [Select](./select)
- [Skeleton](./skeleton)
- [Slider](./slider)
- [Steps](./steps)
- [Switch](./switch)
- [TableCollection](./tablecollection)
- [TableView](./tableview)
- [Tabs](./tabs)
- [Timer](./timer)
- [Tooltip](./tooltip)
- [Tree](./tree)
- [TreeSelect](./treeselect)
- [Typography](./typography)
- [UnsavedChangesModal](./unsavedchangesmodal)
- [Upload](./upload)
@@ -327,13 +327,13 @@ stats.sort_stats('cumulative').print_stats(10)
## Resources
### Internal
- [Coding Guidelines](../guidelines/design-guidelines.md)
- [Testing Guide](../testing/overview.md)
- [Extension Architecture](../extensions/architecture.md)
- [Coding Guidelines](../guidelines/design-guidelines)
- [Testing Guide](../testing/overview)
- [Extension Architecture](../extensions/architecture)
### External
- [Google's Code Review Guide](https://google.github.io/eng-practices/review/)
- [Best Practices for Code Review](https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/)
- [The Art of Readable Code](https://www.oreilly.com/library/view/the-art-of/9781449318482/)
Next: [Reporting issues effectively](./issue-reporting.md)
Next: [Reporting issues effectively](./issue-reporting)
@@ -668,7 +668,7 @@ A series of checks will now run when you make a git commit.
## Linting
See [how tos](./howtos.md#linting)
See [how tos](./howtos#linting)
## GitHub Actions and `act`
@@ -77,7 +77,7 @@ Finally, never submit a PR that will put master branch in broken state. If the P
in `requirements.txt` pinned to a specific version which ensures that the application
build is deterministic.
- For TypeScript/JavaScript, include new libraries in `package.json`
- **Tests:** The pull request should include tests, either as doctests, unit tests, or both. Make sure to resolve all errors and test failures. See [Testing](./howtos.md#testing) for how to run tests.
- **Tests:** The pull request should include tests, either as doctests, unit tests, or both. Make sure to resolve all errors and test failures. See [Testing](./howtos#testing) for how to run tests.
- **Documentation:** If the pull request adds functionality, the docs should be updated as part of the same PR.
- **CI:** Reviewers will not review the code until all CI tests are passed. Sometimes there can be flaky tests. You can close and open PR to re-run CI test. Please report if the issue persists. After the CI fix has been deployed to `master`, please rebase your PR.
- **Code coverage:** Please ensure that code coverage does not decrease.
+1 -1
View File
@@ -282,7 +282,7 @@ You can now launch your VSCode debugger with the same config as above. VSCode wi
### Storybook
See the dedicated [Storybook documentation](../testing/storybook.md) for information on running Storybook locally and adding new stories.
See the dedicated [Storybook documentation](../testing/storybook) for information on running Storybook locally and adding new stories.
## Contributing Translations
@@ -413,6 +413,6 @@ Consider:
- **Feature Request**: Use feature request template
- **Question**: Use GitHub Discussions
- **Configuration Help**: Ask in Slack
- **Development Help**: See [Contributing Guide](./overview.md)
- **Development Help**: See [Contributing Guide](./overview)
Next: [Understanding the release process](./release-process.md)
Next: [Understanding the release process](./release-process)
+5 -5
View File
@@ -94,7 +94,7 @@ Look through the GitHub issues. Issues tagged with
Superset could always use better documentation,
whether as part of the official Superset docs,
in docstrings, `docs/*.rst` or even on the web as blog posts or
articles. See [Documentation](./howtos.md#contributing-to-documentation) for more details.
articles. See [Documentation](./howtos#contributing-to-documentation) for more details.
### Add Translations
@@ -103,7 +103,7 @@ text strings from Superset's UI. You can jump into the existing
language dictionaries at
`superset/translations/<language_code>/LC_MESSAGES/messages.po`, or
even create a dictionary for a new language altogether.
See [Translating](./howtos.md#contributing-translations) for more details.
See [Translating](./howtos#contributing-translations) for more details.
### Ask Questions
@@ -158,9 +158,9 @@ Security team members should also follow these general expectations:
Ready to contribute? Here's how to get started:
1. **[Set up your environment](./development-setup.md)** - Get Superset running locally
1. **[Set up your environment](./development-setup)** - Get Superset running locally
2. **[Find something to work on](#types-of-contributions)** - Pick an issue or feature
3. **[Submit your contribution](./submitting-pr.md)** - Create a pull request
4. **[Follow guidelines](./guidelines.md)** - Ensure code quality
3. **[Submit your contribution](./submitting-pr)** - Create a pull request
4. **[Follow guidelines](./guidelines)** - Ensure code quality
Welcome to the Apache Superset community! 🚀
@@ -466,4 +466,4 @@ Credit:
- [Release Scripts](https://github.com/apache/superset/tree/master/scripts/release)
- [Superset Repository Scripts](https://github.com/apache/superset/tree/master/scripts)
Next: Return to [Contributing Overview](./overview.md)
Next: Return to [Contributing Overview](./overview)
@@ -31,11 +31,11 @@ Learn how to create and submit high-quality pull requests to Apache Superset.
### Prerequisites
- [ ] Development environment is set up
- [ ] You've forked and cloned the repository
- [ ] You've read the [contributing overview](./overview.md)
- [ ] You've read the [contributing overview](./overview)
- [ ] You've found or created an issue to work on
### PR Readiness Checklist
- [ ] Code follows [coding guidelines](../guidelines/design-guidelines.md)
- [ ] Code follows [coding guidelines](../guidelines/design-guidelines)
- [ ] Tests are passing locally
- [ ] Linting passes (`pre-commit run --all-files`)
- [ ] Documentation is updated if needed
@@ -318,4 +318,4 @@ git push origin master
- **GitHub**: Tag @apache/superset-committers for attention
- **Mailing List**: dev@superset.apache.org
Next: [Understanding code review process](./code-review.md)
Next: [Understanding code review process](./code-review)
@@ -233,7 +233,7 @@ This architecture provides several key benefits:
Now that you understand the architecture, explore:
- **[Dependencies](./dependencies.md)** - Managing dependencies and understanding API stability
- **[Quick Start](./quick-start.md)** - Build your first extension
- **[Contribution Types](./contribution-types.md)** - What kinds of extensions you can build
- **[Development](./development.md)** - Project structure, APIs, and development workflow
- **[Dependencies](./dependencies)** - Managing dependencies and understanding API stability
- **[Quick Start](./quick-start)** - Build your first extension
- **[Contribution Types](./contribution-types)** - What kinds of extensions you can build
- **[Development](./development)** - Project structure, APIs, and development workflow
@@ -29,7 +29,7 @@ These UI components are available to Superset extension developers through the `
## Available Components
- [Alert](./alert.mdx)
- [Alert](./alert)
## Usage
@@ -90,4 +90,4 @@ InteractiveMyComponent.argTypes = {
## Interactive Documentation
For interactive examples with controls, run Storybook locally — see the [Storybook documentation](/developer-docs/testing/storybook).
For interactive examples with controls, visit the [Storybook](/storybook/?path=/docs/extension-components--docs).
@@ -110,7 +110,7 @@ editors.registerEditor(
);
```
See [Editors Extension Point](./extension-points/editors.md) for implementation details.
See [Editors Extension Point](./extension-points/editors) for implementation details.
## Backend
@@ -146,7 +146,7 @@ class MyExtensionAPI(RestApi):
from .api import MyExtensionAPI
```
**Note**: The [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects context and generates appropriate paths:
**Note**: The [`@api`](superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects context and generates appropriate paths:
- **Extension context**: `/extensions/{publisher}/{name}/` with ID prefixed as `extensions.{publisher}.{name}.{id}`
- **Host context**: `/api/v1/` with original ID
@@ -193,7 +193,7 @@ def get_summary() -> dict:
return {"status": "success", "result": {"queries_today": 42}}
```
See [MCP Integration](./mcp.md) for implementation details.
See [MCP Integration](./mcp) for implementation details.
### MCP Prompts
@@ -223,7 +223,7 @@ async def analysis_guide(ctx: Context) -> str:
"""
```
See [MCP Integration](./mcp.md) for implementation details.
See [MCP Integration](./mcp) for implementation details.
### Semantic Layers
@@ -161,6 +161,6 @@ Until then, monitor the Superset release notes and test your extensions with eac
## Next Steps
- **[Architecture](./architecture.md)** - Understand the extension system design
- **[Development](./development.md)** - Learn about APIs and development workflow
- **[Quick Start](./quick-start.md)** - Build your first extension
- **[Architecture](./architecture)** - Understand the extension system design
- **[Development](./development)** - Learn about APIs and development workflow
- **[Quick Start](./quick-start)** - Build your first extension
@@ -252,7 +252,7 @@ class DatasetReferencesAPI(RestApi):
### Automatic Context Detection
The [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects whether it's being used in host or extension code:
The [`@api`](superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects whether it's being used in host or extension code:
- **Extension APIs**: Registered under `/extensions/{publisher}/{name}/` with IDs prefixed as `extensions.{publisher}.{name}.{id}`
- **Host APIs**: Registered under `/api/v1/` with original IDs
@@ -217,6 +217,6 @@ const disposable = handle.registerCompletionProvider(provider);
## Next Steps
- **[SQL Lab Extension Points](./sqllab.md)** - Learn about other SQL Lab customizations
- **[Contribution Types](../contribution-types.md)** - Explore other contribution types
- **[Development](../development.md)** - Set up your development environment
- **[SQL Lab Extension Points](./sqllab)** - Learn about other SQL Lab customizations
- **[Contribution Types](../contribution-types)** - Explore other contribution types
- **[Development](../development)** - Set up your development environment
@@ -51,7 +51,7 @@ SQL Lab provides 4 extension points where extensions can contribute custom UI co
| **Right Sidebar** | `sqllab.rightSidebar` | ✓ | — | Custom panels (AI assistants, query analysis) |
| **Panels** | `sqllab.panels` | ✓ | ✓ | Custom tabs + toolbar actions (data profiling) |
\*Editor views are contributed via [Editor Contributions](./editors.md), not standard view contributions.
\*Editor views are contributed via [Editor Contributions](./editors), not standard view contributions.
## Customization Types
@@ -78,7 +78,7 @@ Extensions can add toolbar actions to **Left Sidebar**, **Editor**, and **Panels
### Custom Editors
Extensions can replace the default SQL editor with custom implementations (Monaco, CodeMirror, etc.). See [Editor Contributions](./editors.md) for details.
Extensions can replace the default SQL editor with custom implementations (Monaco, CodeMirror, etc.). See [Editor Contributions](./editors) for details.
## Examples
@@ -157,6 +157,6 @@ menus.registerMenuItem(
## Next Steps
- **[Contribution Types](../contribution-types.md)** - Learn about other contribution types (commands, menus)
- **[Development](../development.md)** - Set up your development environment
- **[Quick Start](../quick-start.md)** - Build a complete extension
- **[Contribution Types](../contribution-types)** - Learn about other contribution types (commands, menus)
- **[Development](../development)** - Set up your development environment
- **[Quick Start](../quick-start)** - Build a complete extension
+2 -2
View File
@@ -455,5 +455,5 @@ async def metrics_guide(ctx: Context) -> str:
## Next Steps
- **[Development](./development.md)** - Project structure, APIs, and dev workflow
- **[Security](./security.md)** - Security best practices for extensions
- **[Development](./development)** - Project structure, APIs, and dev workflow
- **[Security](./security)** - Security best practices for extensions
+10 -10
View File
@@ -47,13 +47,13 @@ Extension developers have access to pre-built UI components via `@apache-superse
## Next Steps
- **[Quick Start](./quick-start.md)** - Build your first extension with a complete walkthrough
- **[Architecture](./architecture.md)** - Design principles and system overview
- **[Dependencies](./dependencies.md)** - Managing dependencies and understanding API stability
- **[Contribution Types](./contribution-types.md)** - Available extension points
- **[Development](./development.md)** - Project structure, APIs, and development workflow
- **[Deployment](./deployment.md)** - Packaging and deploying extensions
- **[MCP Integration](./mcp.md)** - Adding AI agent capabilities using extensions
- **[Security](./security.md)** - Security considerations and best practices
- **[Tasks](./tasks.md)** - Framework for creating and managing long running tasks
- **[Community Extensions](./registry.md)** - Browse extensions shared by the community
- **[Quick Start](./quick-start)** - Build your first extension with a complete walkthrough
- **[Architecture](./architecture)** - Design principles and system overview
- **[Dependencies](./dependencies)** - Managing dependencies and understanding API stability
- **[Contribution Types](./contribution-types)** - Available extension points
- **[Development](./development)** - Project structure, APIs, and development workflow
- **[Deployment](./deployment)** - Packaging and deploying extensions
- **[MCP Integration](./mcp)** - Adding AI agent capabilities using extensions
- **[Security](./security)** - Security considerations and best practices
- **[Tasks](./tasks)** - Framework for creating and managing long running tasks
- **[Community Extensions](./registry)** - Browse extensions shared by the community
@@ -168,7 +168,7 @@ class HelloWorldAPI(RestApi):
**Key points:**
- Uses [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator with automatic context detection
- Uses [`@api`](superset-core/src/superset_core/rest_api/decorators.py) decorator with automatic context detection
- Extends `RestApi` from `superset_core.rest_api.api`
- Uses Flask-AppBuilder decorators (`@expose`, `@protect`, `@safe`)
- Returns responses using `self.response(status_code, result=data)`
@@ -184,7 +184,7 @@ Replace the generated print statement with API import to trigger registration:
from .api import HelloWorldAPI # noqa: F401
```
The [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects extension context and registers your API with proper namespacing.
The [`@api`](superset-core/src/superset_core/rest_api/decorators.py) decorator automatically detects extension context and registers your API with proper namespacing.
## Step 5: Create Frontend Component
@@ -225,7 +225,7 @@ The `@apache-superset/core` package must be listed in both `peerDependencies` (t
The webpack configuration requires specific settings for Module Federation. Key settings include `externalsType: "window"` and `externals` to map `@apache-superset/core` to `window.superset` at runtime, `import: false` for shared modules to use the host's React instead of bundling a separate copy, and `remoteEntry.[contenthash].js` for cache busting.
**Convention**: Superset always loads extensions by requesting the `./index` module from the Module Federation container. The `exposes` entry must be exactly `'./index': './src/index.tsx'` — do not rename or add additional entries. All API registrations must be reachable from that file. See [Architecture](./architecture.md#module-federation) for a full explanation.
**Convention**: Superset always loads extensions by requesting the `./index` module from the Module Federation container. The `exposes` entry must be exactly `'./index': './src/index.tsx'` — do not rename or add additional entries. All API registrations must be reachable from that file. See [Architecture](./architecture#module-federation) for a full explanation.
```javascript
const path = require('path');
@@ -496,7 +496,7 @@ Superset will extract and validate the extension metadata, load the assets, regi
Here's what happens when your extension loads:
1. **Superset starts**: Reads `manifest.json` from the `.supx` bundle and loads the backend entrypoint
2. **Backend registration**: `entrypoint.py` imports your API class, triggering the [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator to register it automatically
2. **Backend registration**: `entrypoint.py` imports your API class, triggering the [`@api`](superset-core/src/superset_core/rest_api/decorators.py) decorator to register it automatically
3. **Frontend loads**: When SQL Lab opens, Superset fetches the remote entry file
4. **Module Federation**: Webpack loads your extension module and resolves `@apache-superset/core` to `window.superset`
5. **Registration**: The module executes at load time, calling `views.registerView` to register your panel
@@ -509,9 +509,9 @@ Here's what happens when your extension loads:
Now that you have a working extension, explore:
- **[Development](./development.md)** - Project structure, APIs, and development workflow
- **[Contribution Types](./contribution-types.md)** - Other contribution points beyond panels
- **[Deployment](./deployment.md)** - Packaging and deploying your extension
- **[Security](./security.md)** - Security best practices for extensions
- **[Development](./development)** - Project structure, APIs, and development workflow
- **[Contribution Types](./contribution-types)** - Other contribution points beyond panels
- **[Deployment](./deployment)** - Packaging and deploying your extension
- **[Security](./security)** - Security best practices for extensions
For a complete real-world example, examine the query insights extension in the Superset codebase.
+1 -1
View File
@@ -28,7 +28,7 @@ By default, extensions are disabled and must be explicitly enabled by setting th
For external extensions, administrators are responsible for evaluating and verifying the security of any extensions they choose to install, just as they would when installing third-party NPM or PyPI packages. At this stage, all extensions run in the same context as the host application, without additional sandboxing. This means that external extensions can impact the security and performance of a Superset environment in the same way as any other installed dependency.
We plan to introduce an optional sandboxed execution model for extensions in the future (as part of an additional SIP). Until then, administrators should exercise caution and follow best practices when selecting and deploying third-party extensions. A directory of community extensions is available in the [Community Extensions](./registry.md) page. Note that these extensions are not vetted by the Apache Superset project—administrators must evaluate each extension before installation.
We plan to introduce an optional sandboxed execution model for extensions in the future (as part of an additional SIP). Until then, administrators should exercise caution and follow best practices when selecting and deploying third-party extensions. A directory of community extensions is available in the [Community Extensions](./registry) page. Note that these extensions are not vetted by the Apache Superset project—administrators must evaluate each extension before installation.
**Any performance or security vulnerabilities introduced by external extensions should be reported directly to the extension author, not as Superset vulnerabilities.**
@@ -114,7 +114,7 @@ class CreateDashboardCommand(BaseCommand):
### Data Access Objects (DAOs)
See: [DAO Style Guidelines and Best Practices](./backend/dao-style-guidelines.md)
See: [DAO Style Guidelines and Best Practices](./backend/dao-style-guidelines)
## Testing
@@ -29,16 +29,16 @@ This is a list of statements that describe how we do frontend development in Sup
- We develop using TypeScript.
- See: [SIP-36](https://github.com/apache/superset/issues/9101)
- We use React for building components, and Redux to manage app/global state.
- See: [Component Style Guidelines and Best Practices](./frontend/component-style-guidelines.md)
- See: [Component Style Guidelines and Best Practices](./frontend/component-style-guidelines)
- We prefer functional components to class components and use hooks for local component state.
- We use [Ant Design](https://ant.design/) components from our component library whenever possible, only building our own custom components when it's required.
- See: [SIP-48](https://github.com/apache/superset/issues/11283)
- We use [@emotion](https://emotion.sh/docs/introduction) to provide styling for our components, co-locating styling within component files.
- See: [SIP-37](https://github.com/apache/superset/issues/9145)
- See: [Emotion Styling Guidelines and Best Practices](./frontend/emotion-styling-guidelines.md)
- See: [Emotion Styling Guidelines and Best Practices](./frontend/emotion-styling-guidelines)
- We use Jest for unit tests, React Testing Library for component tests, and Cypress for end-to-end tests.
- See: [SIP-56](https://github.com/apache/superset/issues/11830)
- See: [Testing Guidelines and Best Practices](../testing/testing-guidelines.md)
- See: [Testing Guidelines and Best Practices](../testing/testing-guidelines)
- We add tests for every new component or file added to the frontend.
- We organize our repo so similar files live near each other, and tests are co-located with the files they test.
- See: [SIP-61](https://github.com/apache/superset/issues/12098)
@@ -46,6 +46,6 @@ This is a list of statements that describe how we do frontend development in Sup
- We use OXC (oxlint) and Prettier to automatically fix lint errors and format the code.
- We do not debate code formatting style in PRs, instead relying on automated tooling to enforce it.
- If there's not a linting rule, we don't have a rule!
- See: [Linting How-Tos](../contributing/howtos.md#typescript--javascript)
- See: [Linting How-Tos](../contributing/howtos#typescript--javascript)
- We use [React Storybook](https://storybook.js.org/) to help preview/test and stabilize our components
- A public Storybook with components from the `master` branch is available [here](https://apache-superset.github.io/superset-ui/?path=/story/*)
@@ -31,7 +31,7 @@ This guide is intended primarily for reusable components. Whenever possible, all
## General Guidelines
- We use [Ant Design](https://ant.design/) as our component library. Do not build a new component if Ant Design provides one but rather instead extend or customize what the library provides
- Always style your component using Emotion and always prefer the theme variables whenever applicable. See: [Emotion Styling Guidelines and Best Practices](./emotion-styling-guidelines.md)
- Always style your component using Emotion and always prefer the theme variables whenever applicable. See: [Emotion Styling Guidelines and Best Practices](./emotion-styling-guidelines)
- All components should be made to be reusable whenever possible
- All components should follow the structure and best practices as detailed below
@@ -53,7 +53,7 @@ superset-frontend/src/components
**Storybook:** Components should come with a storybook file whenever applicable, with the following naming convention `\{ComponentName\}.stories.tsx`. More details about Storybook below
**Unit and end-to-end tests:** All components should come with unit tests using Jest and React Testing Library. The file name should follow this naming convention `\{ComponentName\}.test.tsx`. Read the [Testing Guidelines and Best Practices](../../testing/testing-guidelines.md) for more details
**Unit and end-to-end tests:** All components should come with unit tests using Jest and React Testing Library. The file name should follow this naming convention `\{ComponentName\}.test.tsx`. Read the [Testing Guidelines and Best Practices](../../testing/testing-guidelines) for more details
**Reference naming:** Use `PascalCase` for React components and `camelCase` for component instances
+4 -4
View File
@@ -37,16 +37,16 @@ Superset embraces a testing pyramid approach:
## Testing Documentation
### Frontend Testing
- **[Frontend Testing](./frontend-testing.md)** - Jest, React Testing Library, and component testing strategies
- **[Frontend Testing](./frontend-testing)** - Jest, React Testing Library, and component testing strategies
### Backend Testing
- **[Backend Testing](./backend-testing.md)** - pytest, database testing, and API testing patterns
- **[Backend Testing](./backend-testing)** - pytest, database testing, and API testing patterns
### End-to-End Testing
- **[E2E Testing](./e2e-testing.md)** - Playwright testing for complete user workflows
- **[E2E Testing](./e2e-testing)** - Playwright testing for complete user workflows
### CI/CD Integration
- **[CI/CD](./ci-cd.md)** - Continuous integration, automated testing, and deployment pipelines
- **[CI/CD](./ci-cd)** - Continuous integration, automated testing, and deployment pipelines
## Testing Tools & Frameworks
+1
View File
@@ -0,0 +1 @@
[]
+1
View File
@@ -0,0 +1 @@
[]
+24 -43
View File
@@ -36,42 +36,6 @@ const versionsConfig = JSON.parse(fs.readFileSync(versionsConfigPath, 'utf8'));
// Build plugins array dynamically based on disabled flags
const dynamicPlugins = [];
// Add user_docs (formerly the preset-classic default docs instance) as an
// explicit plugin instance, so its versioned dirs follow the same
// `<id>_versioned_docs` / `<id>_versioned_sidebars` / `<id>_versions.json`
// naming as the other sections instead of the bare `versioned_*` prefix
// Docusaurus uses for the default plugin id.
if (!versionsConfig.user_docs.disabled) {
dynamicPlugins.push([
'@docusaurus/plugin-content-docs',
{
id: 'user_docs',
path: 'docs',
routeBasePath: 'user-docs',
sidebarPath: require.resolve('./sidebars.js'),
editUrl: ({ versionDocsDirPath, docPath }: { versionDocsDirPath: string; docPath: string }) => {
if (docPath === 'intro.md') {
return 'https://github.com/apache/superset/edit/master/README.md';
}
return `https://github.com/apache/superset/edit/master/docs/${versionDocsDirPath}/${docPath}`;
},
remarkPlugins: [remarkImportPartial, remarkLocalizeBadges, remarkTechArticleSchema],
admonitions: {
keywords: ['note', 'tip', 'info', 'warning', 'danger', 'resources'],
extendDefaults: true,
},
docItemComponent: '@theme/DocItem',
includeCurrentVersion: versionsConfig.user_docs.includeCurrentVersion,
lastVersion: versionsConfig.user_docs.lastVersion,
onlyIncludeVersions: versionsConfig.user_docs.onlyIncludeVersions,
versions: versionsConfig.user_docs.versions,
disableVersioning: false,
showLastUpdateAuthor: true,
showLastUpdateTime: true,
},
]);
}
// Add components plugin if not disabled
if (!versionsConfig.components.disabled) {
dynamicPlugins.push([
@@ -290,7 +254,7 @@ const config: Config = {
'Apache Superset is a modern data exploration and visualization platform',
url: 'https://superset.apache.org',
baseUrl: '/',
onBrokenLinks: 'throw',
onBrokenLinks: 'warn',
markdown: {
mermaid: true,
hooks: {
@@ -739,12 +703,29 @@ const config: Config = {
[
'@docusaurus/preset-classic',
{
// The user-docs section is configured as an explicit plugin
// instance above (id: 'user_docs') rather than the preset's
// default docs slot, so that all four sections use parallel
// `<id>_versioned_docs` / `<id>_versioned_sidebars` /
// `<id>_versions.json` naming on disk.
docs: false,
docs: {
routeBasePath: 'user-docs',
sidebarPath: require.resolve('./sidebars.js'),
editUrl: ({ versionDocsDirPath, docPath }) => {
if (docPath === 'intro.md') {
return 'https://github.com/apache/superset/edit/master/README.md';
}
return `https://github.com/apache/superset/edit/master/docs/${versionDocsDirPath}/${docPath}`;
},
remarkPlugins: [remarkImportPartial, remarkLocalizeBadges, remarkTechArticleSchema],
admonitions: {
keywords: ['note', 'tip', 'info', 'warning', 'danger', 'resources'],
extendDefaults: true,
},
includeCurrentVersion: versionsConfig.docs.includeCurrentVersion,
lastVersion: versionsConfig.docs.lastVersion, // Make 'next' the default
onlyIncludeVersions: versionsConfig.docs.onlyIncludeVersions,
versions: versionsConfig.docs.versions,
disableVersioning: false,
showLastUpdateAuthor: true,
showLastUpdateTime: true,
docItemComponent: '@theme/DocItem',
},
blog: {
showReadingTime: true,
// Please change this to your repo.
+9 -12
View File
@@ -30,20 +30,17 @@
"lint:db-metadata:report": "python3 ../superset/db_engine_specs/lint_metadata.py --markdown -o ../superset/db_engine_specs/METADATA_STATUS.md",
"update:readme-db-logos": "node scripts/generate-database-docs.mjs --update-readme",
"eslint": "eslint .",
"lint:docs-links": "node scripts/lint-docs-links.mjs",
"version:add": "node scripts/manage-versions.mjs add",
"version:remove": "node scripts/manage-versions.mjs remove",
"version:add:user_docs": "node scripts/manage-versions.mjs add user_docs",
"version:add:admin_docs": "node scripts/manage-versions.mjs add admin_docs",
"version:add:developer_docs": "node scripts/manage-versions.mjs add developer_docs",
"version:add:docs": "node scripts/manage-versions.mjs add docs",
"version:add:developer_portal": "node scripts/manage-versions.mjs add developer_portal",
"version:add:components": "node scripts/manage-versions.mjs add components",
"version:remove:user_docs": "node scripts/manage-versions.mjs remove user_docs",
"version:remove:admin_docs": "node scripts/manage-versions.mjs remove admin_docs",
"version:remove:developer_docs": "node scripts/manage-versions.mjs remove developer_docs",
"version:remove:docs": "node scripts/manage-versions.mjs remove docs",
"version:remove:developer_portal": "node scripts/manage-versions.mjs remove developer_portal",
"version:remove:components": "node scripts/manage-versions.mjs remove components"
},
"dependencies": {
"@ant-design/icons": "^6.2.3",
"@ant-design/icons": "^6.2.2",
"@docusaurus/core": "^3.10.1",
"@docusaurus/faster": "^3.10.1",
"@docusaurus/plugin-client-redirects": "^3.10.1",
@@ -71,9 +68,9 @@
"@storybook/theming": "^8.6.15",
"@superset-ui/core": "^0.20.4",
"@swc/core": "^1.15.33",
"antd": "^6.4.3",
"baseline-browser-mapping": "^2.10.30",
"caniuse-lite": "^1.0.30001793",
"antd": "^6.3.7",
"baseline-browser-mapping": "^2.10.29",
"caniuse-lite": "^1.0.30001792",
"docusaurus-plugin-openapi-docs": "^5.0.2",
"docusaurus-theme-openapi-docs": "^5.0.2",
"js-yaml": "^4.1.1",
@@ -87,7 +84,7 @@
"react-svg-pan-zoom": "^3.13.1",
"react-table": "^7.8.0",
"remark-import-partial": "^0.0.2",
"reselect": "^5.2.0",
"reselect": "^5.1.1",
"storybook": "^8.6.18",
"swagger-ui-react": "^5.32.5",
"swc-loader": "^0.2.7",
+2 -10
View File
@@ -1260,15 +1260,7 @@ function generateCategoryIndex(category, components) {
};
const componentList = components
.sort((a, b) => a.componentName.localeCompare(b.componentName))
// `.mdx` suffix matches the actual component page files emitted
// by this generator (see the MDX wrappers below). The extension
// is required: Docusaurus only validates and rewrites *file-based*
// references (.md/.mdx). Bare relative paths bypass the file
// resolver and get emitted as raw HTML hrefs that the browser
// resolves against the current URL — which gives the wrong
// directory for trailing-slash routes and breaks SPA navigation.
// See docs/scripts/lint-docs-links.mjs.
.map(c => `- [${c.componentName}](./${c.componentName.toLowerCase()}.mdx)`)
.map(c => `- [${c.componentName}](./${c.componentName.toLowerCase()})`)
.join('\n');
return `---
@@ -1374,7 +1366,7 @@ This documentation is auto-generated from Storybook stories. To add or update co
4. Run \`yarn generate:superset-components\` in the \`docs/\` directory
:::info Work in Progress
This component library is actively being documented. See the [Components TODO](./TODO.md) page for a list of components awaiting documentation.
This component library is actively being documented. See the [Components TODO](./TODO) page for a list of components awaiting documentation.
:::
---
-230
View File
@@ -1,230 +0,0 @@
#!/usr/bin/env node
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
/**
* lint-docs-links source-level checks for internal markdown links.
*
* Catches three failure modes that combine to break SPA navigation in
* a Docusaurus build:
*
* 1. BARE `[X](../foo)` with no extension. Skips
* Docusaurus's file resolver entirely. Emitted
* as a raw href and resolved by the browser
* against the current page URL usually the
* wrong directory for trailing-slash routes.
* `onBrokenLinks: 'throw'` cannot catch this.
*
* 2. MISSING_TARGET `[X](./gone.md)` with an extension, but no
* file at that path. The Docusaurus build
* catches this too (via
* `onBrokenMarkdownLinks: 'throw'`) but only
* after a multi-minute build. This script
* flags it in ~1s.
*
* 3. WRONG_EXTENSION `[X](./foo.md)` where the file is actually
* `foo.mdx` (or vice versa). Same end result
* as MISSING_TARGET, but the fix is one
* character so we report it as its own
* category with the actual extension on disk.
*
* Skips: fenced code blocks, asset-style targets (.png/.json/etc.),
* external URLs, in-page anchors, and the `versioned_docs/`
* snapshots (those are frozen historical content).
*
* Run from `docs/`:
* node scripts/lint-docs-links.mjs
*
* Exits 0 on clean, 1 on any finding.
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const docsRoot = path.join(__dirname, '..');
const ROOTS = ['docs', 'admin_docs', 'developer_docs', 'components'];
const NON_DOC_EXTENSIONS = new Set([
'.png', '.jpg', '.jpeg', '.gif', '.webp', '.svg', '.ico',
'.json', '.yaml', '.yml', '.txt', '.csv',
'.zip', '.tar', '.gz',
'.pdf',
'.mp4', '.webm', '.mov',
]);
const LINK_RE = /\[[^\]\n]+?\]\((?<url>\.{1,2}\/[^)\s]+?)\)/g;
/**
* Classify a single markdown link from a source file.
* Returns one of: ok / bare / asset / missing-target / wrong-extension.
*/
function classifyLink(sourceFile, url) {
const stripped = url.split('#', 1)[0].split('?', 1)[0];
const ext = path.extname(stripped).toLowerCase();
// Non-doc assets — legit bare extensions, leave alone.
if (ext && NON_DOC_EXTENSIONS.has(ext)) {
return { kind: 'asset' };
}
// Anything that doesn't end in .md/.mdx is a bare relative URL.
if (ext !== '.md' && ext !== '.mdx') {
return { kind: 'bare' };
}
// Has a .md/.mdx extension — make sure the target exists.
const target = path.normalize(path.join(path.dirname(sourceFile), stripped));
if (fs.existsSync(target)) {
return { kind: 'ok' };
}
// Target doesn't exist — check if the OTHER extension does.
const otherExt = ext === '.md' ? '.mdx' : '.md';
const otherTarget = target.slice(0, -ext.length) + otherExt;
if (fs.existsSync(otherTarget)) {
return { kind: 'wrong-extension', actualExt: otherExt };
}
return { kind: 'missing-target' };
}
function* walk(dir) {
const entries = fs.readdirSync(dir, { withFileTypes: true });
for (const entry of entries) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
if (
entry.name.startsWith('.') ||
entry.name === 'node_modules' ||
entry.name.endsWith('_versioned_docs') ||
entry.name === 'versioned_docs'
) {
continue;
}
yield* walk(full);
} else if (entry.isFile()) {
if (entry.name.endsWith('.md') || entry.name.endsWith('.mdx')) {
yield full;
}
}
}
}
function lintFile(file) {
const src = fs.readFileSync(file, 'utf8');
const findings = [];
let inFence = false;
const lines = src.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.trimStart().startsWith('```')) {
inFence = !inFence;
continue;
}
if (inFence) continue;
for (const m of line.matchAll(LINK_RE)) {
const url = m.groups.url;
const result = classifyLink(file, url);
if (result.kind !== 'ok' && result.kind !== 'asset') {
findings.push({ line: i + 1, url, ...result });
}
}
}
return findings;
}
const findings = [];
for (const root of ROOTS) {
const abs = path.join(docsRoot, root);
if (!fs.existsSync(abs)) continue;
for (const file of walk(abs)) {
for (const f of lintFile(file)) {
findings.push({ file: path.relative(docsRoot, file), ...f });
}
}
}
if (findings.length === 0) {
console.log('✓ lint-docs-links: no broken internal links found');
process.exit(0);
}
// Group by kind for readable output.
const groups = {
bare: [],
'wrong-extension': [],
'missing-target': [],
};
for (const f of findings) {
groups[f.kind].push(f);
}
console.error(
`✗ lint-docs-links: found ${findings.length} broken internal link(s)`
);
console.error('');
if (groups.bare.length) {
console.error(
` ${groups.bare.length} bare relative link(s) (no .md/.mdx extension)`
);
console.error(
" Docusaurus's file resolver skips these; the browser resolves them"
);
console.error(
' against the current page URL — wrong directory for trailing-slash routes.'
);
console.error(' Add the extension so the file resolver picks them up.');
console.error('');
for (const f of groups.bare) {
console.error(` ${f.file}:${f.line} ${f.url}`);
}
console.error('');
}
if (groups['wrong-extension'].length) {
console.error(
` ${groups['wrong-extension'].length} wrong-extension link(s) (.md vs .mdx mismatch)`
);
console.error(' The target file exists with the other extension on disk.');
console.error('');
for (const f of groups['wrong-extension']) {
console.error(
` ${f.file}:${f.line} ${f.url} → use ${f.actualExt}`
);
}
console.error('');
}
if (groups['missing-target'].length) {
console.error(
` ${groups['missing-target'].length} missing-target link(s) (file doesn't exist)`
);
console.error('');
for (const f of groups['missing-target']) {
console.error(` ${f.file}:${f.line} ${f.url}`);
}
console.error('');
}
process.exit(1);
+58 -211
View File
@@ -30,11 +30,9 @@ const __dirname = path.dirname(__filename);
const CONFIG_FILE = path.join(__dirname, '..', 'versions-config.json');
// Parse command line arguments
const rawArgs = process.argv.slice(2);
const skipGenerate = rawArgs.includes('--skip-generate');
const args = rawArgs.filter((a) => a !== '--skip-generate');
const args = process.argv.slice(2);
const command = args[0]; // 'add' or 'remove'
const section = args[1]; // 'user_docs', 'admin_docs', 'developer_docs', or 'components'
const section = args[1]; // 'docs', 'developer_portal', or 'components'
const version = args[2]; // version string like '1.2.0'
function loadConfig() {
@@ -45,156 +43,36 @@ function saveConfig(config) {
fs.writeFileSync(CONFIG_FILE, JSON.stringify(config, null, 2) + '\n');
}
function freezeDataImports(section, version) {
// MDX files can `import` JSON/YAML data from outside the section, either
// via escaping relative paths (e.g. country-map-tools.mdx imports
// `../../data/countries.json`) or via the `@site/` alias (e.g.
// feature-flags.mdx imports `@site/static/feature-flags.json`). Without
// intervention the snapshot keeps reading the live file, so the
// historical version's content silently changes whenever the data file
// is updated. Copy each escaping data import into a snapshot-local
// `_versioned_data/` dir and rewrite the import to point there.
// The user_docs section's source content lives in `docs/docs/` (the
// historical folder name), while admin_docs / developer_docs /
// components match their plugin id 1:1.
const sectionDir = section === 'user_docs' ? 'docs' : section;
const sectionRoot = path.join(__dirname, '..', sectionDir);
const docsRoot = path.join(__dirname, '..');
const versionedDocsDir = `${section}_versioned_docs/version-${version}`;
const versionedDocsPath = path.join(__dirname, '..', versionedDocsDir);
const frozenDataDir = path.join(versionedDocsPath, '_versioned_data');
function fixVersionedImports(version) {
const versionedDocsPath = path.join(__dirname, '..', 'versioned_docs', `version-${version}`);
if (!fs.existsSync(versionedDocsPath)) {
return;
}
// Files that need import path fixes
const filesToFix = [
'contributing/resources.mdx',
'configuration/country-map-tools.mdx'
];
console.log(` Freezing data imports in ${versionedDocsDir}...`);
console.log(` Fixing relative imports in versioned docs...`);
// Matches data file imports in two flavors:
// `from '../../foo/bar.json'` (relative, must escape one or more dirs)
// `from '@site/static/foo.json'` (Docusaurus site-root alias)
const dataImportRe = /(from\s+['"])((?:\.\.\/)+|@site\/)([^'"\s]+\.(?:json|ya?ml))(['"])/g;
filesToFix.forEach(filePath => {
const fullPath = path.join(versionedDocsPath, filePath);
if (fs.existsSync(fullPath)) {
let content = fs.readFileSync(fullPath, 'utf8');
function freezeOne(fullPath, depth, prefix, pathSpec, importPath, suffix) {
let resolvedSource;
if (pathSpec === '@site/') {
// `@site/...` always resolves relative to the docs root.
resolvedSource = path.join(docsRoot, importPath);
} else {
// Relative path — must escape the file's depth within the section
// to point at content outside the section. Imports that stay inside
// are copied wholesale by Docusaurus, so we leave them alone.
const upCount = pathSpec.match(/\.\.\//g).length;
if (upCount <= depth) return null;
const relativeFromVersioned = path.relative(versionedDocsPath, fullPath);
const originalDir = path.dirname(path.join(sectionRoot, relativeFromVersioned));
resolvedSource = path.resolve(originalDir, pathSpec + importPath);
// Fix imports that go up two directories to go up three instead
content = content.replace(
/from ['"]\.\.\/\.\.\/src\//g,
"from '../../../src/"
);
content = content.replace(
/from ['"]\.\.\/\.\.\/data\//g,
"from '../../../data/"
);
fs.writeFileSync(fullPath, content);
console.log(` Fixed imports in ${filePath}`);
}
// Skip imports that land inside the section root — those get copied
// with the section snapshot already.
const relFromSection = path.relative(sectionRoot, resolvedSource);
if (!relFromSection.startsWith('..')) return null;
const relFromDocsRoot = path.relative(docsRoot, resolvedSource);
if (relFromDocsRoot.startsWith('..') || !fs.existsSync(resolvedSource)) {
return null;
}
const destPath = path.join(frozenDataDir, relFromDocsRoot);
fs.mkdirSync(path.dirname(destPath), { recursive: true });
fs.copyFileSync(resolvedSource, destPath);
const rewritten = path
.relative(path.dirname(fullPath), destPath)
.split(path.sep)
.join('/');
const finalImport = rewritten.startsWith('.') ? rewritten : `./${rewritten}`;
return `${prefix}${finalImport}${suffix}`;
}
function walk(dir, depth) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name.startsWith('_')) continue;
walk(fullPath, depth + 1);
} else if (entry.isFile() && /\.(md|mdx)$/.test(entry.name)) {
const original = fs.readFileSync(fullPath, 'utf8');
let inFence = false;
let mutated = false;
const updated = original.split('\n').map(line => {
if (/^\s*(```|~~~)/.test(line)) {
inFence = !inFence;
return line;
}
if (inFence) return line;
return line.replace(dataImportRe, (match, prefix, pathSpec, importPath, suffix) => {
const rewritten = freezeOne(fullPath, depth, prefix, pathSpec, importPath, suffix);
if (rewritten === null) return match;
mutated = true;
return rewritten;
});
}).join('\n');
if (mutated) {
fs.writeFileSync(fullPath, updated);
const rel = path.relative(versionedDocsPath, fullPath);
console.log(` Froze data imports in ${rel}`);
}
}
}
}
walk(versionedDocsPath, 0);
}
function fixVersionedImports(section, version) {
// Versioned content lands one directory deeper than the source content,
// so any `../../src/` or `../../data/` imports in .md/.mdx files need
// an extra `../` to keep reaching docs/src and docs/data.
const versionedDocsDir = `${section}_versioned_docs/version-${version}`;
const versionedDocsPath = path.join(__dirname, '..', versionedDocsDir);
if (!fs.existsSync(versionedDocsPath)) {
return;
}
console.log(` Fixing relative imports in ${versionedDocsDir}...`);
// Imports whose `../` count exceeds the file's depth within the section
// escape the section root, so they need one extra `../` once the file
// lives one level deeper inside the snapshot dir. Imports that stay
// inside the section are unaffected (the section copies wholesale).
function walk(dir, depth) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
walk(fullPath, depth + 1);
} else if (entry.isFile() && /\.(md|mdx)$/.test(entry.name)) {
const original = fs.readFileSync(fullPath, 'utf8');
// Track fenced code blocks so we don't rewrite import samples inside
// ```ts / ```js (etc.) blocks that are documentation, not real imports.
let inFence = false;
const updated = original.split('\n').map(line => {
if (/^\s*(```|~~~)/.test(line)) {
inFence = !inFence;
return line;
}
if (inFence) return line;
return line.replace(
/(from\s+['"])((?:\.\.\/)+)/g,
(match, prefix, dots) => {
const upCount = dots.match(/\.\.\//g).length;
return upCount > depth ? `${prefix}../${dots}` : match;
},
);
}).join('\n');
if (updated !== original) {
fs.writeFileSync(fullPath, updated);
const rel = path.relative(versionedDocsPath, fullPath);
console.log(` Fixed imports in ${rel}`);
}
}
}
}
walk(versionedDocsPath, 0);
});
}
function addVersion(section, version) {
@@ -213,30 +91,10 @@ function addVersion(section, version) {
console.log(`Creating version ${version} for ${section}...`);
// Refresh auto-generated content (database pages, API reference,
// component playground) so the snapshot captures the current state of
// master rather than whatever happened to be on disk. `generate:smart`
// hashes its inputs and skips unchanged generators, so this is cheap
// when the dev already has fresh output.
//
// Use --skip-generate if you've placed a CI-artifact databases.json
// (the `database-diagnostics` artifact from Python-Integration) and
// want to preserve it instead of letting the local env regenerate it.
// See docs/README.md "Before You Cut" for the canonical release flow.
if (skipGenerate) {
console.log(` Skipping auto-gen refresh (--skip-generate set)`);
} else {
console.log(` Refreshing auto-generated docs...`);
try {
execSync('yarn run generate:smart', { stdio: 'inherit' });
} catch (error) {
console.error(`Failed to refresh auto-generated docs: ${error.message}`);
process.exit(1);
}
}
// Run Docusaurus version command
const docusaurusCommand = `yarn docusaurus docs:version:${section} ${version}`;
const docusaurusCommand = section === 'docs'
? `yarn docusaurus docs:version ${version}`
: `yarn docusaurus docs:version:${section} ${version}`;
try {
execSync(docusaurusCommand, { stdio: 'inherit' });
@@ -245,12 +103,10 @@ function addVersion(section, version) {
process.exit(1);
}
// Freeze data imports BEFORE adjusting paths, so the depth-aware rewriter
// doesn't process the now-local imports we just rewrote.
freezeDataImports(section, version);
// Fix relative imports in versioned content
fixVersionedImports(section, version);
// Fix relative imports in versioned docs (for main docs section only)
if (section === 'docs') {
fixVersionedImports(version);
}
// Update config
// Add to onlyIncludeVersions array (after 'current')
@@ -258,21 +114,17 @@ function addVersion(section, version) {
config[section].onlyIncludeVersions.splice(versionIndex, 0, version);
// Add version metadata
const versionPath = section === 'docs' ? version : version;
config[section].versions[version] = {
label: version,
path: version,
path: versionPath,
banner: 'none'
};
// Note: we deliberately do NOT auto-bump `lastVersion` to the new
// version. Superset's docs site keeps `lastVersion: 'current'` so
// the canonical URLs (`/user-docs/...`, `/admin-docs/...`,
// `/developer-docs/...`, `/components/...`) always render master
// content; cut versions are accessed only via their explicit version
// segment. (`/docs/...` paths are legacy and handled via per-page
// redirects in docusaurus.config.ts — not a current canonical
// form.) If you want a different policy, edit versions-config.json
// after cutting.
// Optionally update lastVersion if this is the first non-current version
if (config[section].onlyIncludeVersions.length === 2) {
config[section].lastVersion = version;
}
saveConfig(config);
console.log(`✅ Version ${version} added successfully to ${section}`);
@@ -300,8 +152,13 @@ function removeVersion(section, version) {
console.log(`Removing version ${version} from ${section}...`);
// Determine file paths based on section
const versionedDocsDir = `${section}_versioned_docs/version-${version}`;
const versionedSidebarsFile = `${section}_versioned_sidebars/version-${version}-sidebars.json`;
const versionedDocsDir = section === 'docs'
? `versioned_docs/version-${version}`
: `${section}_versioned_docs/version-${version}`;
const versionedSidebarsFile = section === 'docs'
? `versioned_sidebars/version-${version}-sidebars.json`
: `${section}_versioned_sidebars/version-${version}-sidebars.json`;
// Remove versioned files
const docsPath = path.join(__dirname, '..', versionedDocsDir);
@@ -318,7 +175,9 @@ function removeVersion(section, version) {
}
// Update versions.json file
const versionsJsonFile = `${section}_versions.json`;
const versionsJsonFile = section === 'docs'
? 'versions.json'
: `${section}_versions.json`;
const versionsJsonPath = path.join(__dirname, '..', versionsJsonFile);
if (fs.existsSync(versionsJsonPath)) {
@@ -326,17 +185,8 @@ function removeVersion(section, version) {
const versionIndex = versions.indexOf(version);
if (versionIndex > -1) {
versions.splice(versionIndex, 1);
if (versions.length === 0) {
// Sections with no versions shouldn't carry an empty versions file
// on disk — Docusaurus doesn't require it, and an empty `[]` file
// gets picked up by `docusaurus version` and snapshotted into the
// next cut.
fs.unlinkSync(versionsJsonPath);
console.log(` Removed empty ${versionsJsonFile}`);
} else {
fs.writeFileSync(versionsJsonPath, JSON.stringify(versions, null, 2) + '\n');
console.log(` Updated ${versionsJsonFile}`);
}
fs.writeFileSync(versionsJsonPath, JSON.stringify(versions, null, 2) + '\n');
console.log(` Updated ${versionsJsonFile}`);
}
}
@@ -361,20 +211,17 @@ function removeVersion(section, version) {
function printUsage() {
console.log(`
Usage:
node scripts/manage-versions.mjs add <section> <version> [--skip-generate]
node scripts/manage-versions.mjs remove <section> <version>
node scripts/manage-versions.js add <section> <version>
node scripts/manage-versions.js remove <section> <version>
Where:
- section: 'user_docs', 'admin_docs', 'developer_docs', or 'components'
- section: 'docs', 'developer_portal', or 'components'
- version: version string (e.g., '1.2.0', '2.0.0')
- --skip-generate: skip refreshing auto-generated docs before snapshotting
(use when you've already placed a fresh databases.json
from CI and want to preserve it)
Examples:
node scripts/manage-versions.mjs add user_docs 2.0.0
node scripts/manage-versions.mjs add developer_docs 1.3.0
node scripts/manage-versions.mjs remove components 1.0.0
node scripts/manage-versions.js add docs 2.0.0
node scripts/manage-versions.js add developer_portal 1.3.0
node scripts/manage-versions.js remove components 1.0.0
`);
}
+8 -18
View File
@@ -30,28 +30,19 @@ import { DownOutlined } from '@ant-design/icons';
import styles from './styles.module.css';
// Map each versioned plugin id to the URL prefix it actually serves
// content from. The plugin ids use underscores while several
// routeBasePath values use hyphens (and `user_docs` → `/user-docs`),
// so without this map the basePath derivation below would mis-split
// the pathname for those sections and the version dropdown would
// jump to the section root instead of preserving the current page.
//
// Keep in sync with the `routeBasePath` values in docusaurus.config.ts.
const PLUGIN_ID_TO_BASE_PATH = {
user_docs: '/user-docs',
components: '/components',
admin_docs: '/admin-docs',
developer_docs: '/developer-docs',
};
export default function DocVersionBadge() {
const activePlugin = useActivePlugin();
const { pathname } = useLocation();
const pluginId = activePlugin?.pluginId;
const [versionedPath, setVersionedPath] = React.useState('');
const isVersioned = pluginId && pluginId in PLUGIN_ID_TO_BASE_PATH;
// Show version selector for all versioned sections
const isVersioned = [
'default', // main docs
'components',
'tutorials',
'developer_portal',
].includes(pluginId);
const { preferredVersion } = useDocsPreferredVersion(pluginId);
const versions = useVersions(pluginId);
@@ -62,8 +53,7 @@ export default function DocVersionBadge() {
if (!pathname || !version || !pluginId) return;
let relativePath = '';
const basePath = PLUGIN_ID_TO_BASE_PATH[pluginId];
if (!basePath) return;
const basePath = pluginId === 'default' ? '/docs' : `/${pluginId}`;
// Handle different version path patterns
if (pathname.includes(basePath)) {
+121
View File
@@ -0,0 +1,121 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import React, { useState, useEffect } from 'react';
import DocVersionBanner from '@theme-original/DocVersionBanner';
import {
useActivePlugin,
useDocsVersion,
useVersions,
} from '@docusaurus/plugin-content-docs/client';
import { useLocation } from '@docusaurus/router';
import { useDocsPreferredVersion } from '@docusaurus/theme-common';
import { Dropdown } from 'antd';
import { DownOutlined } from '@ant-design/icons';
import styles from './styles.module.css';
export default function DocVersionBannerWrapper(props) {
const activePlugin = useActivePlugin();
const { pathname } = useLocation();
const pluginId = activePlugin?.pluginId;
const [versionedPath, setVersionedPath] = useState('');
// Only show version selector for tutorials
// Main docs, components, and developer_portal use the DocVersionBadge component instead
const isVersioned = pluginId && ['tutorials'].includes(pluginId);
const { preferredVersion } = useDocsPreferredVersion(pluginId);
const versions = useVersions(pluginId);
const version = useDocsVersion();
// Early return if required data is not available
if (!isVersioned || !versions || !version) {
return <DocVersionBanner {...props} />;
}
// Extract the current page path relative to the version
useEffect(() => {
if (!pathname || !version || !pluginId) return;
let relativePath = '';
// Handle different version path patterns
if (pathname.includes(`/${pluginId}/`)) {
// Extract the part after the version
// Example: /components/1.1.0/ui-components/button -> /ui-components/button
const parts = pathname.split(`/${pluginId}/`);
if (parts.length > 1) {
const afterPluginId = parts[1];
// Find where the version part ends
const versionParts = afterPluginId.split('/');
if (versionParts.length > 1) {
// Remove the version part and join the rest
relativePath = '/' + versionParts.slice(1).join('/');
}
}
}
setVersionedPath(relativePath);
}, [pathname, version, pluginId]);
// Create dropdown items for version selection
const items = versions.map(v => {
// Construct the URL for this version, preserving the current page
// v.path is the version-specific path like "1.0.0" or "next"
let versionUrl = v.path;
if (versionedPath) {
// Construct the full URL with the version and the current page path
versionUrl = v.path + versionedPath;
}
return {
key: v.name,
label: (
<a href={versionUrl}>
{v.label}
{v.name === version.name && ' (current)'}
{v.name === preferredVersion?.name && ' (preferred)'}
</a>
),
};
});
return (
<>
<DocVersionBanner {...props} />
{isVersioned && (
<div className={styles.versionBanner}>
<div className={styles.versionContainer}>
<span className={styles.versionLabel}>Version:</span>
<Dropdown menu={{ items }} trigger={['click']}>
<a
onClick={e => e.preventDefault()}
className={styles.versionSelector}
>
{version.label} <DownOutlined />
</a>
</Dropdown>
</div>
</div>
)}
</>
);
}
@@ -0,0 +1,49 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
.versionBanner {
background-color: var(--ifm-color-emphasis-100);
padding: 0.5rem 1rem;
margin-bottom: 1rem;
border-bottom: 1px solid var(--ifm-color-emphasis-200);
}
.versionContainer {
display: flex;
align-items: center;
max-width: var(--ifm-container-width);
margin: 0 auto;
padding: 0 var(--ifm-spacing-horizontal);
}
.versionLabel {
font-weight: bold;
margin-right: 0.5rem;
}
.versionSelector {
cursor: pointer;
color: var(--ifm-color-primary);
font-weight: 500;
}
.versionSelector:hover {
text-decoration: none;
color: var(--ifm-color-primary-darker);
}
+1 -1
View File
@@ -174,7 +174,7 @@
"default": false,
"lifecycle": "testing",
"description": "Allows users to add a superset:// DB that can query across databases. Experimental with potential security/performance risks. See SUPERSET_META_DB_LIMIT.",
"docs": "https://superset.apache.org/user-docs/databases/supported/superset-meta-database"
"docs": "https://superset.apache.org/docs/configuration/databases/#querying-across-databases"
},
{
"name": "ESTIMATE_QUERY_COST",
+3
View File
@@ -0,0 +1,3 @@
[
"1.0.0"
]
@@ -20,12 +20,12 @@ Alerts and reports are disabled by default. To turn them on, you need to do some
#### In your `superset_config.py` or `superset_config_docker.py`
- `"ALERT_REPORTS"` [feature flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) must be turned to True.
- `"ALERT_REPORTS"` [feature flag](/docs/6.0.0/configuration/configuring-superset#feature-flags) must be turned to True.
- `beat_schedule` in CeleryConfig must contain schedule for `reports.scheduler`.
- At least one of those must be configured, depending on what you want to use:
- emails: `SMTP_*` settings
- Slack messages: `SLACK_API_TOKEN`
- Users can customize the email subject by including date code placeholders, which will automatically be replaced with the corresponding UTC date when the email is sent. To enable this functionality, activate the `"DATE_FORMAT_IN_EMAIL_SUBJECT"` [feature flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags). This enables date formatting in email subjects, preventing all reporting emails from being grouped into the same thread (optional for the reporting feature).
- Users can customize the email subject by including date code placeholders, which will automatically be replaced with the corresponding UTC date when the email is sent. To enable this functionality, activate the `"DATE_FORMAT_IN_EMAIL_SUBJECT"` [feature flag](/docs/6.0.0/configuration/configuring-superset#feature-flags). This enables date formatting in email subjects, preventing all reporting emails from being grouped into the same thread (optional for the reporting feature).
- Use date codes from [strftime.org](https://strftime.org/) to create the email subject.
- If no date code is provided, the original string will be used as the email subject.
@@ -38,7 +38,7 @@ Screenshots will be taken but no messages actually sent as long as `ALERT_REPORT
- You must install a headless browser, for taking screenshots of the charts and dashboards. Only Firefox and Chrome are currently supported.
> If you choose Chrome, you must also change the value of `WEBDRIVER_TYPE` to `"chrome"` in your `superset_config.py`.
Note: All the components required (Firefox headless browser, Redis, Postgres db, celery worker and celery beat) are present in the *dev* docker image if you are following [Installing Superset Locally](/user-docs/6.0.0/installation/docker-compose/).
Note: All the components required (Firefox headless browser, Redis, Postgres db, celery worker and celery beat) are present in the *dev* docker image if you are following [Installing Superset Locally](/docs/6.0.0/installation/docker-compose/).
All you need to do is add the required config variables described in this guide (See `Detailed Config`).
If you are running a non-dev docker image, e.g., a stable release like `apache/superset:3.1.0`, that image does not include a headless browser. Only the `superset_worker` container needs this headless browser to browse to the target chart or dashboard.
@@ -70,7 +70,7 @@ Note: when you configure an alert or a report, the Slack channel list takes chan
### Kubernetes-specific
- You must have a `celery beat` pod running. If you're using the chart included in the GitHub repository under [helm/superset](https://github.com/apache/superset/tree/master/helm/superset), you need to put `supersetCeleryBeat.enabled = true` in your values override.
- You can see the dedicated docs about [Kubernetes installation](/user-docs/6.0.0/installation/kubernetes) for more details.
- You can see the dedicated docs about [Kubernetes installation](/docs/6.0.0/installation/kubernetes) for more details.
### Docker Compose specific
@@ -78,11 +78,11 @@ Caching for SQL Lab query results is used when async queries are enabled and is
Note that this configuration does not use a flask-caching dictionary for its configuration, but
instead requires a cachelib object.
See [Async Queries via Celery](/user-docs/6.0.0/configuration/async-queries-celery) for details.
See [Async Queries via Celery](/docs/6.0.0/configuration/async-queries-celery) for details.
## Caching Thumbnails
This is an optional feature that can be turned on by activating its [feature flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) on config:
This is an optional feature that can be turned on by activating its [feature flag](/docs/6.0.0/configuration/configuring-superset#feature-flags) on config:
```
FEATURE_FLAGS = {
@@ -37,7 +37,7 @@ ENV SUPERSET_CONFIG_PATH /app/superset_config.py
```
Docker compose deployments handle application configuration differently using specific conventions.
Refer to the [docker compose tips & configuration](/user-docs/6.0.0/installation/docker-compose#docker-compose-tips--configuration)
Refer to the [docker compose tips & configuration](/docs/6.0.0/installation/docker-compose#docker-compose-tips--configuration)
for details.
The following is an example of just a few of the parameters you can set in your `superset_config.py` file:
@@ -254,7 +254,7 @@ flask --app "superset.app:create_app(superset_app_root='/analytics')"
### Docker builds
The [docker compose](/user-docs/6.0.0/installation/docker-compose#configuring-further) developer
The [docker compose](/docs/6.0.0/installation/docker-compose#configuring-further) developer
configuration includes an additional environmental variable,
[`SUPERSET_APP_ROOT`](https://github.com/apache/superset/blob/master/docker/.env),
to simplify the process of setting up a non-default root path across the services.
@@ -449,4 +449,4 @@ FEATURE_FLAGS = {
}
```
A current list of feature flags can be found in the [Feature Flags](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) documentation.
A current list of feature flags can be found in the [Feature Flags](/docs/6.0.0/configuration/feature-flags) documentation.
@@ -14,7 +14,7 @@ in your environment.
Youll need to install the required packages for the database you want to use as your metadata database
as well as the packages needed to connect to the databases you want to access through Superset.
For information about setting up Superset's metadata database, please refer to
installation documentations ([Docker Compose](/user-docs/6.0.0/installation/docker-compose), [Kubernetes](/user-docs/6.0.0/installation/kubernetes))
installation documentations ([Docker Compose](/docs/6.0.0/installation/docker-compose), [Kubernetes](/docs/6.0.0/installation/kubernetes))
:::
This documentation tries to keep pointer to the different drivers for commonly used database
@@ -26,7 +26,7 @@ Superset requires a Python [DB-API database driver](https://peps.python.org/pep-
and a [SQLAlchemy dialect](https://docs.sqlalchemy.org/en/20/dialects/) to be installed for
each database engine you want to connect to.
You can read more [here](/user-docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) about how to
You can read more [here](/docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) about how to
install new database drivers into your Superset configuration.
### Supported Databases and Dependencies
@@ -37,53 +37,53 @@ are compatible with Superset.
| <div style={{width: '150px'}}>Database</div> | PyPI package | Connection String |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [AWS Athena](/user-docs/6.0.0/configuration/databases#aws-athena) | `pip install pyathena[pandas]` , `pip install PyAthenaJDBC` | `awsathena+rest://{access_key_id}:{access_key}@athena.{region}.amazonaws.com/{schema}?s3_staging_dir={s3_staging_dir}&...` |
| [AWS DynamoDB](/user-docs/6.0.0/configuration/databases#aws-dynamodb) | `pip install pydynamodb` | `dynamodb://{access_key_id}:{secret_access_key}@dynamodb.{region_name}.amazonaws.com?connector=superset` |
| [AWS Redshift](/user-docs/6.0.0/configuration/databases#aws-redshift) | `pip install sqlalchemy-redshift` | `redshift+psycopg2://<userName>:<DBPassword>@<AWS End Point>:5439/<Database Name>` |
| [Apache Doris](/user-docs/6.0.0/configuration/databases#apache-doris) | `pip install pydoris` | `doris://<User>:<Password>@<Host>:<Port>/<Catalog>.<Database>` |
| [Apache Drill](/user-docs/6.0.0/configuration/databases#apache-drill) | `pip install sqlalchemy-drill` | `drill+sadrill://<username>:<password>@<host>:<port>/<storage_plugin>`, often useful: `?use_ssl=True/False` |
| [Apache Druid](/user-docs/6.0.0/configuration/databases#apache-druid) | `pip install pydruid` | `druid://<User>:<password>@<Host>:<Port-default-9088>/druid/v2/sql` |
| [Apache Hive](/user-docs/6.0.0/configuration/databases#hive) | `pip install pyhive` | `hive://hive@{hostname}:{port}/{database}` |
| [Apache Impala](/user-docs/6.0.0/configuration/databases#apache-impala) | `pip install impyla` | `impala://{hostname}:{port}/{database}` |
| [Apache Kylin](/user-docs/6.0.0/configuration/databases#apache-kylin) | `pip install kylinpy` | `kylin://<username>:<password>@<hostname>:<port>/<project>?<param1>=<value1>&<param2>=<value2>` |
| [Apache Pinot](/user-docs/6.0.0/configuration/databases#apache-pinot) | `pip install pinotdb` | `pinot://BROKER:5436/query?server=http://CONTROLLER:5983/` |
| [Apache Solr](/user-docs/6.0.0/configuration/databases#apache-solr) | `pip install sqlalchemy-solr` | `solr://{username}:{password}@{hostname}:{port}/{server_path}/{collection}` |
| [Apache Spark SQL](/user-docs/6.0.0/configuration/databases#apache-spark-sql) | `pip install pyhive` | `hive://hive@{hostname}:{port}/{database}` |
| [Ascend.io](/user-docs/6.0.0/configuration/databases#ascendio) | `pip install impyla` | `ascend://{username}:{password}@{hostname}:{port}/{database}?auth_mechanism=PLAIN;use_ssl=true` |
| [Azure MS SQL](/user-docs/6.0.0/configuration/databases#sql-server) | `pip install pymssql` | `mssql+pymssql://UserName@presetSQL:TestPassword@presetSQL.database.windows.net:1433/TestSchema` |
| [ClickHouse](/user-docs/6.0.0/configuration/databases#clickhouse) | `pip install clickhouse-connect` | `clickhousedb://{username}:{password}@{hostname}:{port}/{database}` |
| [CockroachDB](/user-docs/6.0.0/configuration/databases#cockroachdb) | `pip install cockroachdb` | `cockroachdb://root@{hostname}:{port}/{database}?sslmode=disable` |
| [Couchbase](/user-docs/6.0.0/configuration/databases#couchbase) | `pip install couchbase-sqlalchemy` | `couchbase://{username}:{password}@{hostname}:{port}?truststorepath={ssl certificate path}` |
| [CrateDB](/user-docs/6.0.0/configuration/databases#cratedb) | `pip install sqlalchemy-cratedb` | `crate://{username}:{password}@{hostname}:{port}`, often useful: `?ssl=true/false` or `?schema=testdrive`. |
| [Denodo](/user-docs/6.0.0/configuration/databases#denodo) | `pip install denodo-sqlalchemy` | `denodo://{username}:{password}@{hostname}:{port}/{database}` |
| [Dremio](/user-docs/6.0.0/configuration/databases#dremio) | `pip install sqlalchemy_dremio` |`dremio+flight://{username}:{password}@{host}:32010`, often useful: `?UseEncryption=true/false`. For Legacy ODBC: `dremio+pyodbc://{username}:{password}@{host}:31010` |
| [Elasticsearch](/user-docs/6.0.0/configuration/databases#elasticsearch) | `pip install elasticsearch-dbapi` | `elasticsearch+http://{user}:{password}@{host}:9200/` |
| [Exasol](/user-docs/6.0.0/configuration/databases#exasol) | `pip install sqlalchemy-exasol` | `exa+pyodbc://{username}:{password}@{hostname}:{port}/my_schema?CONNECTIONLCALL=en_US.UTF-8&driver=EXAODBC` |
| [Google BigQuery](/user-docs/6.0.0/configuration/databases#google-bigquery) | `pip install sqlalchemy-bigquery` | `bigquery://{project_id}` |
| [Google Sheets](/user-docs/6.0.0/configuration/databases#google-sheets) | `pip install shillelagh[gsheetsapi]` | `gsheets://` |
| [Firebolt](/user-docs/6.0.0/configuration/databases#firebolt) | `pip install firebolt-sqlalchemy` | `firebolt://{client_id}:{client_secret}@{database}/{engine_name}?account_name={name}` |
| [Hologres](/user-docs/6.0.0/configuration/databases#hologres) | `pip install psycopg2` | `postgresql+psycopg2://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [IBM Db2](/user-docs/6.0.0/configuration/databases#ibm-db2) | `pip install ibm_db_sa` | `db2+ibm_db://` |
| [IBM Netezza Performance Server](/user-docs/6.0.0/configuration/databases#ibm-netezza-performance-server) | `pip install nzalchemy` | `netezza+nzpy://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [MySQL](/user-docs/6.0.0/configuration/databases#mysql) | `pip install mysqlclient` | `mysql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [OceanBase](/user-docs/6.0.0/configuration/databases#oceanbase) | `pip install oceanbase_py` | `oceanbase://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [Oracle](/user-docs/6.0.0/configuration/databases#oracle) | `pip install cx_Oracle` | `oracle://<username>:<password>@<hostname>:<port>` |
| [Parseable](/user-docs/6.0.0/configuration/databases#parseable) | `pip install sqlalchemy-parseable` | `parseable://<UserName>:<DBPassword>@<Database Host>/<Stream Name>` |
| [PostgreSQL](/user-docs/6.0.0/configuration/databases#postgres) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [Presto](/user-docs/6.0.0/configuration/databases#presto) | `pip install pyhive` | `presto://{username}:{password}@{hostname}:{port}/{database}` |
| [SAP Hana](/user-docs/6.0.0/configuration/databases#hana) | `pip install hdbcli sqlalchemy-hana` or `pip install apache_superset[hana]` | `hana://{username}:{password}@{host}:{port}` |
| [SingleStore](/user-docs/6.0.0/configuration/databases#singlestore) | `pip install sqlalchemy-singlestoredb` | `singlestoredb://{username}:{password}@{host}:{port}/{database}` |
| [StarRocks](/user-docs/6.0.0/configuration/databases#starrocks) | `pip install starrocks` | `starrocks://<User>:<Password>@<Host>:<Port>/<Catalog>.<Database>` |
| [Snowflake](/user-docs/6.0.0/configuration/databases#snowflake) | `pip install snowflake-sqlalchemy` | `snowflake://{user}:{password}@{account}.{region}/{database}?role={role}&warehouse={warehouse}` |
| [AWS Athena](/docs/6.0.0/configuration/databases#aws-athena) | `pip install pyathena[pandas]` , `pip install PyAthenaJDBC` | `awsathena+rest://{access_key_id}:{access_key}@athena.{region}.amazonaws.com/{schema}?s3_staging_dir={s3_staging_dir}&...` |
| [AWS DynamoDB](/docs/6.0.0/configuration/databases#aws-dynamodb) | `pip install pydynamodb` | `dynamodb://{access_key_id}:{secret_access_key}@dynamodb.{region_name}.amazonaws.com?connector=superset` |
| [AWS Redshift](/docs/6.0.0/configuration/databases#aws-redshift) | `pip install sqlalchemy-redshift` | `redshift+psycopg2://<userName>:<DBPassword>@<AWS End Point>:5439/<Database Name>` |
| [Apache Doris](/docs/6.0.0/configuration/databases#apache-doris) | `pip install pydoris` | `doris://<User>:<Password>@<Host>:<Port>/<Catalog>.<Database>` |
| [Apache Drill](/docs/6.0.0/configuration/databases#apache-drill) | `pip install sqlalchemy-drill` | `drill+sadrill://<username>:<password>@<host>:<port>/<storage_plugin>`, often useful: `?use_ssl=True/False` |
| [Apache Druid](/docs/6.0.0/configuration/databases#apache-druid) | `pip install pydruid` | `druid://<User>:<password>@<Host>:<Port-default-9088>/druid/v2/sql` |
| [Apache Hive](/docs/6.0.0/configuration/databases#hive) | `pip install pyhive` | `hive://hive@{hostname}:{port}/{database}` |
| [Apache Impala](/docs/6.0.0/configuration/databases#apache-impala) | `pip install impyla` | `impala://{hostname}:{port}/{database}` |
| [Apache Kylin](/docs/6.0.0/configuration/databases#apache-kylin) | `pip install kylinpy` | `kylin://<username>:<password>@<hostname>:<port>/<project>?<param1>=<value1>&<param2>=<value2>` |
| [Apache Pinot](/docs/6.0.0/configuration/databases#apache-pinot) | `pip install pinotdb` | `pinot://BROKER:5436/query?server=http://CONTROLLER:5983/` |
| [Apache Solr](/docs/6.0.0/configuration/databases#apache-solr) | `pip install sqlalchemy-solr` | `solr://{username}:{password}@{hostname}:{port}/{server_path}/{collection}` |
| [Apache Spark SQL](/docs/6.0.0/configuration/databases#apache-spark-sql) | `pip install pyhive` | `hive://hive@{hostname}:{port}/{database}` |
| [Ascend.io](/docs/6.0.0/configuration/databases#ascendio) | `pip install impyla` | `ascend://{username}:{password}@{hostname}:{port}/{database}?auth_mechanism=PLAIN;use_ssl=true` |
| [Azure MS SQL](/docs/6.0.0/configuration/databases#sql-server) | `pip install pymssql` | `mssql+pymssql://UserName@presetSQL:TestPassword@presetSQL.database.windows.net:1433/TestSchema` |
| [ClickHouse](/docs/6.0.0/configuration/databases#clickhouse) | `pip install clickhouse-connect` | `clickhousedb://{username}:{password}@{hostname}:{port}/{database}` |
| [CockroachDB](/docs/6.0.0/configuration/databases#cockroachdb) | `pip install cockroachdb` | `cockroachdb://root@{hostname}:{port}/{database}?sslmode=disable` |
| [Couchbase](/docs/6.0.0/configuration/databases#couchbase) | `pip install couchbase-sqlalchemy` | `couchbase://{username}:{password}@{hostname}:{port}?truststorepath={ssl certificate path}` |
| [CrateDB](/docs/6.0.0/configuration/databases#cratedb) | `pip install sqlalchemy-cratedb` | `crate://{username}:{password}@{hostname}:{port}`, often useful: `?ssl=true/false` or `?schema=testdrive`. |
| [Denodo](/docs/6.0.0/configuration/databases#denodo) | `pip install denodo-sqlalchemy` | `denodo://{username}:{password}@{hostname}:{port}/{database}` |
| [Dremio](/docs/6.0.0/configuration/databases#dremio) | `pip install sqlalchemy_dremio` |`dremio+flight://{username}:{password}@{host}:32010`, often useful: `?UseEncryption=true/false`. For Legacy ODBC: `dremio+pyodbc://{username}:{password}@{host}:31010` |
| [Elasticsearch](/docs/6.0.0/configuration/databases#elasticsearch) | `pip install elasticsearch-dbapi` | `elasticsearch+http://{user}:{password}@{host}:9200/` |
| [Exasol](/docs/6.0.0/configuration/databases#exasol) | `pip install sqlalchemy-exasol` | `exa+pyodbc://{username}:{password}@{hostname}:{port}/my_schema?CONNECTIONLCALL=en_US.UTF-8&driver=EXAODBC` |
| [Google BigQuery](/docs/6.0.0/configuration/databases#google-bigquery) | `pip install sqlalchemy-bigquery` | `bigquery://{project_id}` |
| [Google Sheets](/docs/6.0.0/configuration/databases#google-sheets) | `pip install shillelagh[gsheetsapi]` | `gsheets://` |
| [Firebolt](/docs/6.0.0/configuration/databases#firebolt) | `pip install firebolt-sqlalchemy` | `firebolt://{client_id}:{client_secret}@{database}/{engine_name}?account_name={name}` |
| [Hologres](/docs/6.0.0/configuration/databases#hologres) | `pip install psycopg2` | `postgresql+psycopg2://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [IBM Db2](/docs/6.0.0/configuration/databases#ibm-db2) | `pip install ibm_db_sa` | `db2+ibm_db://` |
| [IBM Netezza Performance Server](/docs/6.0.0/configuration/databases#ibm-netezza-performance-server) | `pip install nzalchemy` | `netezza+nzpy://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [MySQL](/docs/6.0.0/configuration/databases#mysql) | `pip install mysqlclient` | `mysql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [OceanBase](/docs/6.0.0/configuration/databases#oceanbase) | `pip install oceanbase_py` | `oceanbase://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [Oracle](/docs/6.0.0/configuration/databases#oracle) | `pip install cx_Oracle` | `oracle://<username>:<password>@<hostname>:<port>` |
| [Parseable](/docs/6.0.0/configuration/databases#parseable) | `pip install sqlalchemy-parseable` | `parseable://<UserName>:<DBPassword>@<Database Host>/<Stream Name>` |
| [PostgreSQL](/docs/6.0.0/configuration/databases#postgres) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [Presto](/docs/6.0.0/configuration/databases#presto) | `pip install pyhive` | `presto://{username}:{password}@{hostname}:{port}/{database}` |
| [SAP Hana](/docs/6.0.0/configuration/databases#hana) | `pip install hdbcli sqlalchemy-hana` or `pip install apache_superset[hana]` | `hana://{username}:{password}@{host}:{port}` |
| [SingleStore](/docs/6.0.0/configuration/databases#singlestore) | `pip install sqlalchemy-singlestoredb` | `singlestoredb://{username}:{password}@{host}:{port}/{database}` |
| [StarRocks](/docs/6.0.0/configuration/databases#starrocks) | `pip install starrocks` | `starrocks://<User>:<Password>@<Host>:<Port>/<Catalog>.<Database>` |
| [Snowflake](/docs/6.0.0/configuration/databases#snowflake) | `pip install snowflake-sqlalchemy` | `snowflake://{user}:{password}@{account}.{region}/{database}?role={role}&warehouse={warehouse}` |
| SQLite | No additional library needed | `sqlite://path/to/file.db?check_same_thread=false` |
| [SQL Server](/user-docs/6.0.0/configuration/databases#sql-server) | `pip install pymssql` | `mssql+pymssql://<Username>:<Password>@<Host>:<Port-default:1433>/<Database Name>` |
| [TDengine](/user-docs/6.0.0/configuration/databases#tdengine) | `pip install taospy` `pip install taos-ws-py` | `taosws://<user>:<password>@<host>:<port>` |
| [Teradata](/user-docs/6.0.0/configuration/databases#teradata) | `pip install teradatasqlalchemy` | `teradatasql://{user}:{password}@{host}` |
| [TimescaleDB](/user-docs/6.0.0/configuration/databases#timescaledb) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>:<Port>/<Database Name>` |
| [Trino](/user-docs/6.0.0/configuration/databases#trino) | `pip install trino` | `trino://{username}:{password}@{hostname}:{port}/{catalog}` |
| [Vertica](/user-docs/6.0.0/configuration/databases#vertica) | `pip install sqlalchemy-vertica-python` | `vertica+vertica_python://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [YDB](/user-docs/6.0.0/configuration/databases#ydb) | `pip install ydb-sqlalchemy` | `ydb://{host}:{port}/{database_name}` |
| [YugabyteDB](/user-docs/6.0.0/configuration/databases#yugabytedb) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [SQL Server](/docs/6.0.0/configuration/databases#sql-server) | `pip install pymssql` | `mssql+pymssql://<Username>:<Password>@<Host>:<Port-default:1433>/<Database Name>` |
| [TDengine](/docs/6.0.0/configuration/databases#tdengine) | `pip install taospy` `pip install taos-ws-py` | `taosws://<user>:<password>@<host>:<port>` |
| [Teradata](/docs/6.0.0/configuration/databases#teradata) | `pip install teradatasqlalchemy` | `teradatasql://{user}:{password}@{host}` |
| [TimescaleDB](/docs/6.0.0/configuration/databases#timescaledb) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>:<Port>/<Database Name>` |
| [Trino](/docs/6.0.0/configuration/databases#trino) | `pip install trino` | `trino://{username}:{password}@{hostname}:{port}/{catalog}` |
| [Vertica](/docs/6.0.0/configuration/databases#vertica) | `pip install sqlalchemy-vertica-python` | `vertica+vertica_python://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
| [YDB](/docs/6.0.0/configuration/databases#ydb) | `pip install ydb-sqlalchemy` | `ydb://{host}:{port}/{database_name}` |
| [YugabyteDB](/docs/6.0.0/configuration/databases#yugabytedb) | `pip install psycopg2` | `postgresql://<UserName>:<DBPassword>@<Database Host>/<Database Name>` |
---
@@ -109,7 +109,7 @@ The connector library installation process is the same for all additional librar
#### 1. Determine the driver you need
Consult the [list of database drivers](/user-docs/6.0.0/configuration/databases)
Consult the [list of database drivers](/docs/6.0.0/configuration/databases)
and find the PyPI package needed to connect to your database. In this example, we're connecting
to a MySQL database, so we'll need the `mysqlclient` connector library.
@@ -165,11 +165,11 @@ to your database via the Superset web UI.
As an admin user, go to Settings -> Data: Database Connections and click the +DATABASE button.
From there, follow the steps on the
[Using Database Connection UI page](/user-docs/6.0.0/configuration/databases#connecting-through-the-ui).
[Using Database Connection UI page](/docs/6.0.0/configuration/databases#connecting-through-the-ui).
Consult the page for your specific database type in the Superset documentation to determine
the connection string and any other parameters you need to input. For instance,
on the [MySQL page](/user-docs/6.0.0/configuration/databases#mysql), we see that the connection string
on the [MySQL page](/docs/6.0.0/configuration/databases#mysql), we see that the connection string
to a local MySQL database differs depending on whether the setup is running on Linux or Mac.
Click the “Test Connection” button, which should result in a popup message saying,
@@ -407,7 +407,7 @@ this:
crate://<username>:<password>@<clustername>.cratedb.net:4200/?ssl=true
```
Follow the steps [here](/user-docs/6.0.0/configuration/databases#installing-database-drivers)
Follow the steps [here](/docs/6.0.0/configuration/databases#installing-database-drivers)
to install the CrateDB connector package when setting up Superset locally using
Docker Compose.
@@ -782,7 +782,7 @@ The recommended connector library for BigQuery is
##### Install BigQuery Driver
Follow the steps [here](/user-docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) about how to
Follow the steps [here](/docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) about how to
install new database drivers when setting up Superset locally via docker compose.
```bash
@@ -1177,7 +1177,7 @@ risingwave://root@{hostname}:{port}/{database}?sslmode=disable
##### Install Snowflake Driver
Follow the steps [here](/user-docs/6.0.0/configuration/databases#installing-database-drivers) about how to
Follow the steps [here](/docs/6.0.0/configuration/databases#installing-database-drivers) about how to
install new database drivers when setting up Superset locally via docker compose.
```bash
@@ -51,7 +51,7 @@ Restart Superset for this configuration change to take effect.
#### Making a Dashboard Public
1. Add the `'DASHBOARD_RBAC': True` [Feature Flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) to `superset_config.py`
1. Add the `'DASHBOARD_RBAC': True` [Feature Flag](/docs/6.0.0/configuration/feature-flags) to `superset_config.py`
2. Add the `Public` role to your dashboard as described [here](https://superset.apache.org/docs/using-superset/creating-your-first-dashboard/#manage-access-to-dashboards)
#### Embedding a Public Dashboard
@@ -10,7 +10,7 @@ version: 1
## Jinja Templates
SQL Lab and Explore supports [Jinja templating](https://jinja.palletsprojects.com/en/2.11.x/) in queries.
To enable templating, the `ENABLE_TEMPLATE_PROCESSING` [feature flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) needs to be enabled in
To enable templating, the `ENABLE_TEMPLATE_PROCESSING` [feature flag](/docs/6.0.0/configuration/configuring-superset#feature-flags) needs to be enabled in
`superset_config.py`. When templating is enabled, python code can be embedded in virtual datasets and
in Custom SQL in the filter and metric controls in Explore. By default, the following variables are
made available in the Jinja context:
@@ -20,7 +20,7 @@ To help make the problem somewhat tractable—given that Apache Superset has no
To strive for data consistency (regardless of the timezone of the client) the Apache Superset backend tries to ensure that any timestamp sent to the client has an explicit (or semi-explicit as in the case with [Epoch time](https://en.wikipedia.org/wiki/Unix_time) which is always in reference to UTC) timezone encoded within.
The challenge however lies with the slew of [database engines](/user-docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) which Apache Superset supports and various inconsistencies between their [Python Database API (DB-API)](https://www.python.org/dev/peps/pep-0249/) implementations combined with the fact that we use [Pandas](https://pandas.pydata.org/) to read SQL into a DataFrame prior to serializing to JSON. Regrettably Pandas ignores the DB-API [type_code](https://www.python.org/dev/peps/pep-0249/#type-objects) relying by default on the underlying Python type returned by the DB-API. Currently only a subset of the supported database engines work correctly with Pandas, i.e., ensuring timestamps without an explicit timestamp are serializd to JSON with the server timezone, thus guaranteeing the client will display timestamps in a consistent manner irrespective of the client's timezone.
The challenge however lies with the slew of [database engines](/docs/6.0.0/configuration/databases#installing-drivers-in-docker-images) which Apache Superset supports and various inconsistencies between their [Python Database API (DB-API)](https://www.python.org/dev/peps/pep-0249/) implementations combined with the fact that we use [Pandas](https://pandas.pydata.org/) to read SQL into a DataFrame prior to serializing to JSON. Regrettably Pandas ignores the DB-API [type_code](https://www.python.org/dev/peps/pep-0249/#type-objects) relying by default on the underlying Python type returned by the DB-API. Currently only a subset of the supported database engines work correctly with Pandas, i.e., ensuring timestamps without an explicit timestamp are serializd to JSON with the server timezone, thus guaranteeing the client will display timestamps in a consistent manner irrespective of the client's timezone.
For example the following is a comparison of MySQL and Presto,
@@ -77,7 +77,7 @@ Look through the GitHub issues. Issues tagged with
Superset could always use better documentation,
whether as part of the official Superset docs,
in docstrings, `docs/*.rst` or even on the web as blog posts or
articles. See [Documentation](/user-docs/6.0.0/contributing/howtos#contributing-to-documentation) for more details.
articles. See [Documentation](/docs/6.0.0/contributing/howtos#contributing-to-documentation) for more details.
### Add Translations
@@ -599,7 +599,7 @@ export enum FeatureFlag {
those specified under FEATURE_FLAGS in `superset_config.py`. For example, `DEFAULT_FEATURE_FLAGS = { 'FOO': True, 'BAR': False }` in `superset/config.py` and `FEATURE_FLAGS = { 'BAR': True, 'BAZ': True }` in `superset_config.py` will result
in combined feature flags of `{ 'FOO': True, 'BAR': True, 'BAZ': True }`.
The current status of the usability of each flag (stable vs testing, etc) can be found in the [Feature Flags](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) documentation.
The current status of the usability of each flag (stable vs testing, etc) can be found in the [Feature Flags](/docs/6.0.0/configuration/feature-flags) documentation.
## Git Hooks
@@ -614,7 +614,7 @@ A series of checks will now run when you make a git commit.
## Linting
See [how tos](/user-docs/6.0.0/contributing/howtos#linting)
See [how tos](/docs/6.0.0/contributing/howtos#linting)
## GitHub Actions and `act`
@@ -57,7 +57,7 @@ Finally, never submit a PR that will put master branch in broken state. If the P
in `requirements.txt` pinned to a specific version which ensures that the application
build is deterministic.
- For TypeScript/JavaScript, include new libraries in `package.json`
- **Tests:** The pull request should include tests, either as doctests, unit tests, or both. Make sure to resolve all errors and test failures. See [Testing](/user-docs/6.0.0/contributing/howtos#testing) for how to run tests.
- **Tests:** The pull request should include tests, either as doctests, unit tests, or both. Make sure to resolve all errors and test failures. See [Testing](/docs/6.0.0/contributing/howtos#testing) for how to run tests.
- **Documentation:** If the pull request adds functionality, the docs should be updated as part of the same PR.
- **CI:** Reviewers will not review the code until all CI tests are passed. Sometimes there can be flaky tests. You can close and open PR to re-run CI test. Please report if the issue persists. After the CI fix has been deployed to `master`, please rebase your PR.
- **Code coverage:** Please ensure that code coverage does not decrease.
@@ -51,11 +51,11 @@ multiple tables as long as your database account has access to the tables.
## How do I create my own visualization?
We recommend reading the instructions in
[Creating Visualization Plugins](/user-docs/6.0.0/contributing/howtos#creating-visualization-plugins).
[Creating Visualization Plugins](/docs/6.0.0/contributing/howtos#creating-visualization-plugins).
## Can I upload and visualize CSV data?
Absolutely! Read the instructions [here](/user-docs/using-superset/exploring-data) to learn
Absolutely! Read the instructions [here](/docs/using-superset/exploring-data) to learn
how to enable and use CSV upload.
## Why are my queries timing out?
@@ -142,7 +142,7 @@ SQLALCHEMY_DATABASE_URI = 'sqlite:////new/location/superset.db?check_same_thread
```
You can read more about customizing Superset using the configuration file
[here](/user-docs/6.0.0/configuration/configuring-superset).
[here](/docs/6.0.0/configuration/configuring-superset).
## What if the table schema changed?
@@ -157,7 +157,7 @@ table afterwards to configure the Columns tab, check the appropriate boxes and s
To clarify, the database backend is an OLTP database used by Superset to store its internal
information like your list of users and dashboard definitions. While Superset supports a
[variety of databases as data _sources_](/user-docs/6.0.0/configuration/databases#installing-database-drivers),
[variety of databases as data _sources_](/docs/6.0.0/configuration/databases#installing-database-drivers),
only a few database engines are supported for use as the OLTP backend / metadata store.
Superset is tested using MySQL, PostgreSQL, and SQLite backends. Its recommended you install
@@ -190,7 +190,7 @@ second etc). Example:
## Does Superset work with [insert database engine here]?
The [Connecting to Databases section](/user-docs/6.0.0/configuration/databases) provides the best
The [Connecting to Databases section](/docs/6.0.0/configuration/databases) provides the best
overview for supported databases. Database engines not listed on that page may work too. We rely on
the community to contribute to this knowledge base.
@@ -226,7 +226,7 @@ are typical in basic SQL:
## Does Superset offer a public API?
Yes, a public REST API, and the surface of that API formal is expanding steadily. You can read more about this API and
interact with it using Swagger [here](/developer-docs/api).
interact with it using Swagger [here](/docs/api).
Some of the
original vision for the collection of endpoints under **/api/v1** was originally specified in
@@ -266,7 +266,7 @@ Superset uses [Scarf](https://about.scarf.sh/) by default to collect basic telem
We use the [Scarf Gateway](https://docs.scarf.sh/gateway/) to sit in front of container registries, the [scarf-js](https://about.scarf.sh/package-sdks) package to track `npm` installations, and a Scarf pixel to gather anonymous analytics on Superset page views.
Scarf purges PII and provides aggregated statistics. Superset users can easily opt out of analytics in various ways documented [here](https://docs.scarf.sh/gateway/#do-not-track) and [here](https://docs.scarf.sh/package-analytics/#as-a-user-of-a-package-using-scarf-js-how-can-i-opt-out-of-analytics).
Superset maintainers can also opt out of telemetry data collection by setting the `SCARF_ANALYTICS` environment variable to `false` in the Superset container (or anywhere Superset/webpack are run).
Additional opt-out instructions for Docker users are available on the [Docker Installation](/user-docs/6.0.0/installation/docker-compose) page.
Additional opt-out instructions for Docker users are available on the [Docker Installation](/docs/6.0.0/installation/docker-compose) page.
## Does Superset have an archive panel or trash bin from which a user can recover deleted assets?
@@ -24,10 +24,10 @@ A Superset installation is made up of these components:
The optional components above are necessary to enable these features:
- [Alerts and Reports](/user-docs/6.0.0/configuration/alerts-reports)
- [Caching](/user-docs/6.0.0/configuration/cache)
- [Async Queries](/user-docs/6.0.0/configuration/async-queries-celery/)
- [Dashboard Thumbnails](/user-docs/6.0.0/configuration/cache/#caching-thumbnails)
- [Alerts and Reports](/docs/6.0.0/configuration/alerts-reports)
- [Caching](/docs/6.0.0/configuration/cache)
- [Async Queries](/docs/6.0.0/configuration/async-queries-celery/)
- [Dashboard Thumbnails](/docs/6.0.0/configuration/cache/#caching-thumbnails)
If you install with Kubernetes or Docker Compose, all of these components will be created.
@@ -59,7 +59,7 @@ The caching layer serves two main functions:
- Store the results of queries to your data warehouse so that when a chart is loaded twice, it pulls from the cache the second time, speeding up the application and reducing load on your data warehouse.
- Act as a message broker for the worker, enabling the Alerts & Reports, async queries, and thumbnail caching features.
Most people use Redis for their cache, but Superset supports other options too. See the [cache docs](/user-docs/6.0.0/configuration/cache/) for more.
Most people use Redis for their cache, but Superset supports other options too. See the [cache docs](/docs/6.0.0/configuration/cache/) for more.
### Worker and Beat
@@ -67,6 +67,6 @@ This is one or more workers who execute tasks like run async queries or take sna
## Other components
Other components can be incorporated into Superset. The best place to learn about additional configurations is the [Configuration page](/user-docs/6.0.0/configuration/configuring-superset). For instance, you could set up a load balancer or reverse proxy to implement HTTPS in front of your Superset application, or specify a Mapbox URL to enable geospatial charts, etc.
Other components can be incorporated into Superset. The best place to learn about additional configurations is the [Configuration page](/docs/6.0.0/configuration/configuring-superset). For instance, you could set up a load balancer or reverse proxy to implement HTTPS in front of your Superset application, or specify a Mapbox URL to enable geospatial charts, etc.
Superset won't even start without certain configuration settings established, so it's essential to review that page.
@@ -21,7 +21,7 @@ with our [installing on k8s](https://superset.apache.org/docs/installation/runni
documentation.
:::
As mentioned in our [quickstart guide](/user-docs/quickstart), the fastest way to try
As mentioned in our [quickstart guide](/docs/quickstart), the fastest way to try
Superset locally is using Docker Compose on a Linux or Mac OSX
computer. Superset does not have official support for Windows. It's also the easiest
way to launch a fully functioning **development environment** quickly.
@@ -9,11 +9,11 @@ import useBaseUrl from "@docusaurus/useBaseUrl";
# Installation Methods
How should you install Superset? Here's a comparison of the different options. It will help if you've first read the [Architecture](/user-docs/6.0.0/installation/architecture) page to understand Superset's different components.
How should you install Superset? Here's a comparison of the different options. It will help if you've first read the [Architecture](/docs/6.0.0/installation/architecture page to understand Superset's different components.
The fundamental trade-off is between you needing to do more of the detail work yourself vs. using a more complex deployment route that handles those details.
## [Docker Compose](/user-docs/6.0.0/installation/docker-compose)
## [Docker Compose](/docs/6.0.0/installation/docker-compose
**Summary:** This takes advantage of containerization while remaining simpler than Kubernetes. This is the best way to try out Superset; it's also useful for developing & contributing back to Superset.
@@ -27,9 +27,9 @@ You will need to back up your metadata DB. That could mean backing up the servic
You will also need to extend the Superset docker image. The default `lean` images do not contain drivers needed to access your metadata database (Postgres or MySQL), nor to access your data warehouse, nor the headless browser needed for Alerts & Reports. You could run a `-dev` image while demoing Superset, which has some of this, but you'll still need to install the driver for your data warehouse. The `-dev` images run as root, which is not recommended for production.
Ideally you will build your own image of Superset that extends `lean`, adding what your deployment needs. See [Building your own production Docker image](/user-docs/6.0.0/installation/docker-builds/#building-your-own-production-docker-image).
Ideally you will build your own image of Superset that extends `lean`, adding what your deployment needs. See [Building your own production Docker image](/docs/6.0.0/installation/docker-builds/#building-your-own-production-docker-image).
## [Kubernetes (K8s)](/user-docs/6.0.0/installation/kubernetes)
## [Kubernetes (K8s)](/docs/6.0.0/installation/kubernetes
**Summary:** This is the best-practice way to deploy a production instance of Superset, but has the steepest skill requirement - someone who knows Kubernetes.
@@ -41,7 +41,7 @@ A K8s deployment can scale up and down based on usage and deploy rolling updates
You will need to build your own Docker image, and back up your metadata DB, both as described in Docker Compose above. You'll also need to customize your Helm chart values and deploy and maintain your Kubernetes cluster.
## [PyPI (Python)](/user-docs/6.0.0/installation/pypi)
## [PyPI (Python)](/docs/6.0.0/installation/pypi
**Summary:** This is the only method that requires no knowledge of containers. It requires the most hands-on work to deploy, connect, and maintain each component.
@@ -149,7 +149,7 @@ For production clusters it's recommended to build own image with this step done
Superset requires a Python DB-API database driver and a SQLAlchemy
dialect to be installed for each datastore you want to connect to.
See [Install Database Drivers](/user-docs/6.0.0/configuration/databases) for more information.
See [Install Database Drivers](/docs/6.0.0/configuration/databases) for more information.
It is recommended that you refer to versions listed in
[pyproject.toml](https://github.com/apache/superset/blob/master/pyproject.toml)
instead of hard-coding them in your bootstrap script, as seen below.
@@ -310,7 +310,7 @@ configOverrides:
### Enable Alerts and Reports
For this, as per the [Alerts and Reports doc](/user-docs/6.0.0/configuration/alerts-reports), you will need to:
For this, as per the [Alerts and Reports doc](/docs/6.0.0/configuration/alerts-reports), you will need to:
#### Install a supported webdriver in the Celery worker
@@ -172,7 +172,7 @@ how to set up a development environment.
## Resources
- [Superset "In the Wild"](https://github.com/apache/superset/blob/master/RESOURCES/INTHEWILD.md) - open a PR to add your org to the list!
- [Feature Flags](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) - the status of Superset's Feature Flags.
- [Feature Flags](/docs/6.0.0/configuration/feature-flags) - the status of Superset's Feature Flags.
- [Standard Roles](https://github.com/apache/superset/blob/master/RESOURCES/STANDARD_ROLES.md) - How RBAC permissions map to roles.
- [Superset Wiki](https://github.com/apache/superset/wiki) - Tons of additional community resources: best practices, community content and other information.
- [Superset SIPs](https://github.com/orgs/apache/projects/170) - The status of Superset's SIPs (Superset Improvement Proposals) for both consensus and implementation status.
@@ -15,7 +15,7 @@ Although we recommend using `Docker Compose` for a quick start in a sandbox-type
environment and for other development-type use cases, **we
do not recommend this setup for production**. For this purpose please
refer to our
[Installing on Kubernetes](/user-docs/6.0.0/installation/kubernetes/)
[Installing on Kubernetes](/docs/6.0.0/installation/kubernetes/)
page.
:::
@@ -73,10 +73,10 @@ processes by running Docker Compose `stop` command. By doing so, you can avoid d
From this point on, you can head on to:
- [Create your first Dashboard](/user-docs/6.0.0/using-superset/creating-your-first-dashboard)
- [Connect to a Database](/user-docs/6.0.0/configuration/databases)
- [Using Docker Compose](/user-docs/6.0.0/installation/docker-compose)
- [Configure Superset](/user-docs/6.0.0/configuration/configuring-superset/)
- [Installing on Kubernetes](/user-docs/6.0.0/installation/kubernetes/)
- [Create your first Dashboard](/docs/6.0.0/using-superset/creating-your-first-dashboard)
- [Connect to a Database](/docs/6.0.0/configuration/databases)
- [Using Docker Compose](/docs/6.0.0/installation/docker-compose)
- [Configure Superset](/docs/6.0.0/configuration/configuring-superset/)
- [Installing on Kubernetes](/docs/6.0.0/installation/kubernetes/)
Or just explore our [Documentation](https://superset.apache.org/docs/intro)!
@@ -31,7 +31,7 @@ your existing SQL-speaking database or data store.
First things first, we need to add the connection credentials to your database to be able
to query and visualize data from it. If you're using Superset locally via
[Docker compose](/user-docs/6.0.0/installation/docker-compose), you can
[Docker compose](/docs/6.0.0/installation/docker-compose), you can
skip this step because a Postgres database, named **examples**, is included and
pre-configured in Superset for you.
@@ -188,7 +188,7 @@ Access to dashboards is managed via owners (users that have edit permissions to
Non-owner users access can be managed in two different ways. The dashboard needs to be published to be visible to other users.
1. Dataset permissions - if you add to the relevant role permissions to datasets it automatically grants implicit access to all dashboards that uses those permitted datasets.
2. Dashboard roles - if you enable [**DASHBOARD_RBAC** feature flag](/user-docs/6.0.0/configuration/configuring-superset#feature-flags) then you will be able to manage which roles can access the dashboard
2. Dashboard roles - if you enable [**DASHBOARD_RBAC** feature flag](/docs/6.0.0/configuration/configuring-superset#feature-flags) then you will be able to manage which roles can access the dashboard
- Granting a role access to a dashboard will bypass dataset level checks. Having dashboard access implicitly grants read access to all the featured charts in the dashboard, and thereby also all the associated datasets.
- If no roles are specified for a dashboard, regular **Dataset permissions** will apply.
+1 -1
View File
@@ -1,5 +1,5 @@
{
"user_docs": {
"docs": {
"disabled": false,
"lastVersion": "current",
"includeCurrentVersion": true,

Some files were not shown because too many files have changed in this diff Show More