Two small UX wins on top of the previous control-panel fixes:
1. Rename admin_level options to "World" / "Country" / "Aggregated
regions" (from the technical "Countries (Admin 0)" / "Subdivisions
(Admin 1)"). The control's own label becomes "Map view" with an
explanatory description. Underlying form_data values stay 0 / 1 /
aggregated so saved charts don't break.
2. Auto-migrate form_data when a user switches a saved legacy
country_map chart's viz type to country_map_v2:
- select_country: 'france' → admin_level=1, country=FRA
- select_country: 'france_overseas' → composite=france_overseas
- select_country: 'turkey_regions' → admin_level=aggregated,
country=TUR, region_set=nuts_1
- …same pattern for france_regions / italy_regions /
philippines_regions and ~180 other legacy country files.
The mapping lives in src/plugin/migrateFromLegacy.ts (covered by
12 unit tests). Migration only fills fields the user hasn't already
set on the new chart, so explicit edits win over inferred values.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four UX bugs reported on the new country map chart:
1. **"Country: cannot be empty" stuck on the Data tab even at Admin 0.**
The Country control had a hard `validators: [validateNonEmpty]`, but
it's only rendered (and only meaningful) when admin_level != 0 and no
composite is set. The validator was firing for the hidden empty value
and trapped the user — the Update Chart button was permanently
disabled with no visible control to satisfy. Switched to a dynamic
validator via mapStateToProps so it only fires when the field is
actually needed.
2. **"Aggregated regions" threw TypeError: t.map is not a function.**
The region_set control used `choices: ({ controls }) => ...`, but
SelectControl's `choices` must be a literal array. Moved the
country-dependent choice computation into `mapStateToProps` (same
pattern other plugins use).
3. **Composite map always visible and "sticky" overriding admin_level.**
At Admin 0 (world choropleth) the composite override produced a map
that didn't change when you toggled admin_level. Hidden composite
control at Admin 0 entirely, and hidden it whenever the build
pipeline didn't emit any composites — leaves room for future
per-country scoping (e.g. only show france_overseas when country=FRA).
4. **Show flying islands checkbox was a no-op.** The build pipeline
doesn't currently tag features as flying, so the runtime drop logic
had nothing to act on. Removed from the control set rather than ship
a misleading control; a single line brings it back once the build
tags features.
Tests updated to cover all four behaviors.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously the build only emitted the ukr (Ukraine) worldview, so the
worldview dropdown had a single option even though it claimed otherwise.
Build now produces Admin 0 GeoJSON for every NE-published editorial:
default, arg, bdg, bra, chn, deu, egy, esp, fra, gbr, grc, idn, ind, iso,
isr, ita, jpn, kor, mar, nep, nld, pak, pol, prt, pse, rus, sau, swe, tur,
twn, ukr, usa, vnm (33 total).
NE does not publish per-worldview Admin 1 variants, so subdivisions within
a country come from a single shared file. The frontend now always points
Admin 1, regional aggregation, and composite URLs at the ukr-prefixed
shared outputs regardless of the selected worldview — the worldview
control only affects the world (Admin 0) map.
- build.py: expand WORLDVIEWS_ADMIN_0 to 33 worldviews; main() builds
Admin 0 for all of them, Admin 1 only for ukr
- transformProps.ts: introduce SHARED_ADMIN1_WORLDVIEW = 'ukr'; pin all
non-Admin-0 URLs to it
- controlPanel.tsx: WORLDVIEW_LABELS now covers all 33 codes; unrecognized
codes still fall back to raw code for forward-compat
- transformProps.test.ts: cover shared-Admin1 contract (admin1+chn still
resolves to ukr_admin1_*)
- pre-commit: exclude .geo.json from check-added-large-files (existing
rule only excluded .geojson and would block these ~2MB worldview files)
- README + SIP: document the worldview model and check off Phase 1 item
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The plugin imports src/data/manifest.json. The plugin's own tsconfig
already had resolveJsonModule + json in include after the previous fix,
but the root superset-frontend/tsconfig.json (which tsc --build walks)
also needs to resolve the manifest.json across the lib/ and esm/ babel
outputs. Add the explicit json glob and register the plugin as a
project reference so tsc finds it in build mode.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add trailing newline to all 220 geo.json outputs to satisfy
end-of-file-fixer pre-commit hook
- build.py: post-process step ensures future regens emit trailing newlines
- CountryMap.tsx: import t/useTheme from @apache-superset/core
(matches the rest of the codebase) and use antd theme tokens
(colorBgSpotlight, colorTextLightSolid, colorErrorText, etc.)
instead of legacy theme.colors.* paths
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- transformProps: read snake_case via rawFormData (ChartProps.formData
is camelCased), fixing 4 failing jest tests
- CountryMap.tsx: replace literal colors with theme tokens; wrap user
strings with t() for i18n
- build.py: add proper dict[str, Any] type params, drop unused type:ignore,
emit manifest.json with trailing newline for prettier/EOF parity
- test_build.py: top-of-file mypy ignore (unittest test scaffolding)
- pyproject.toml: per-file ruff ignores for the standalone build pipeline
(TID251/S310/S603/S607/E501/C901/PT009 all intentional/inapplicable)
- regen workflow: surface drift via PR comment + step summary instead of
failing — cross-platform mapshaper output reproducibility is still WIP
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Remove unused CategoricalColorNamespace import in CountryMap.tsx
- Remove unused isAdminSubdivision helper in controlPanel.tsx
- Add resolveJsonModule + include manifest.json in tsconfig
- Cast chartProps.formData to CountryMapFormData in transformProps
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three coordinated fixes for the 25 CI failures on the initial PR push:
1. **Lockfile sync.** Added @superset-ui/plugin-chart-country-map as
a workspace dep in the previous commit but didn't update
package-lock.json. CI's `npm ci` failed across frontend-build,
cypress (12 jobs), playwright (4 jobs), docker (2 jobs), and
frontend-check-translations. Re-ran `npm install --package-lock-only`
to add the new workspace's 71 lock entries.
2. **License headers added** to 13 new files flagged by License Check:
- 5 markdown READMEs / SIP_DRAFT (HTML-comment headers)
- 5 YAML config files (`# Licensed ...`)
- 2 Python files (`# Licensed ...`)
- 1 shell script (preserves shebang)
3. **Reproducible build outputs.** The regen workflow detected drift
on manifest.json + ukr_admin1_CAN.geo.json. Two root causes:
- `build_timestamp_utc` field made manifest non-deterministic →
dropped from the schema
- Floating mapshaper version (`npx --yes mapshaper`) caused subtle
simplification differences across runners → pinned to
`mapshaper@0.7.15` via `npx --yes mapshaper@<version>`
Verified locally: rebuild from clean cache reproduces every output
byte-identically except the manifest (which now also matches once
the timestamp is gone).
Files changed:
.gitignore — re-include rule for static dir
superset-frontend/package-lock.json — +71 lines for new workspace
13 new files — ASF headers
build.py — pin mapshaper, drop timestamp
manifest.json (× 2) — regenerate w/o timestamp
README.md (in static dir) — header
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mark Phase 2 (plugin scaffolding) substantially complete: hosting
path wired, click-to-zoom ported. Phase 3 (controls) fully done with
manifest-driven choices replacing hardcoded tables. Phase 5
substantially done: 41 tests across 4 test files, CI workflow,
UPDATING.md entry.
Remaining items, all non-blocking for SIP filing:
- "Switch to new Country Map" button on legacy plugin renderer
- Cross-filter integration via setDataMask
- Composite projection support (geoAlbersUsa for USA)
- Real example images (need actual rendering capture)
- Update Superset docs site
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the test trio (transformProps + buildQuery + controlPanel).
Verifies:
- All 9 new controls (worldview / admin_level / country / region_set /
composite / region_includes / region_excludes / show_flying_islands /
name_language) are present in the panel
- Worldview defaults to 'ukr' (Superset's editorial choice)
- show_flying_islands defaults to true
- name_language defaults to 'en'
- admin_level offers exactly the 3 expected codes
- Country selector visibility hides on Admin 0 OR when composite set
- Region-set selector only visible when admin_level === 'aggregated'
- Region-set choices key off the selected country (TUR → nuts_1,
FRA → regions, USA → empty)
- Composite selector exposes france_overseas
These tests would fail loudly if anyone refactored the visibility
predicates or accidentally removed/renamed a control.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Covers the pure-Python helpers and YAML-applied transforms that the
build pipeline relies on. Subprocess calls (mapshaper, NE download)
are not exercised — those are integration concerns covered by the
regen workflow itself.
Test categories:
- _matches (4 tests): scalar equality, AND'd conditions, `in: [...]`
list-membership matcher, missing property
- _bbox_center (2 tests): unit square, offset square
- _translate_and_scale (4 tests): pure translate, scale-around-centroid,
combined transform, multipolygon handling
- _translate_and_scale_with_pivot (1 test): shared pivot preserves
relative positions of grouped features (the Paris-petite-couronne case)
- _drop_parts (2 tests): drops specified indices, polygon unchanged
- _bbox_contains (2 tests): inside-bbox, outside-bbox-west
- apply_name_overrides (1 test): applies only to matching features,
respects match conditions across countries (FRA "Seien" vs GBR "Seien"
don't collide)
- apply_flying_islands (2 tests): repositions matched features,
drop_outside_bbox guarded to Admin 1 only (the bug we fixed earlier)
Wired into the regen workflow as a step that runs BEFORE the build,
so a broken transform fails CI before producing potentially-bad output
files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New workflow `.github/workflows/country-map-build-regen.yml` that:
1. Triggers on PRs touching the build pipeline configs (or manually
via workflow_dispatch).
2. Sets up Python 3.11 + Node 22 + PyYAML.
3. Runs `./scripts/build.sh` from a clean checkout.
4. Detects output drift (uncommitted changes in
`superset/static/assets/country-maps/` or `src/data/manifest.json`).
5. If drift detected on a PR: leaves a comment telling the contributor
to re-run the build locally and commit, then fails CI.
This forces "configs and outputs stay in sync" — a contributor can't
add a name override / region aggregation / composite without also
updating the committed GeoJSON outputs.
Tagged contents permissions only — does not auto-PR (which would
require deeper repo setup); contributors are expected to regenerate
locally and push.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a "Country Map plugin redesigned (legacy plugin deprecated)"
section under the Next release covering:
- What changed (modern endpoint, worldview, Admin 0+1, aggregated
regions, composite maps, region include/exclude, flying-islands
toggle, name-language selector, build pipeline)
- Migration behavior (existing dashboards keep working, no DB
migrations, deprecation badge in chart picker)
- For maintainers/power users (the notebook is no longer the source
of truth; local touchups go in YAML configs or procedural/ scripts)
Pointer to SIP_DRAFT.md for full design rationale.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Click a region: viewport zooms in (4x) centered on its centroid.
Click again or click the empty background: zoom back out.
600ms CSS transition on the SVG group.
Implementation note: zoom state lives on the DOM (zoomedFeature
local + group transform attribute) rather than React state. Two
reasons:
- Cleaner unmount (the transition completes naturally; no orphaned
React state)
- Avoids a re-render cycle on every click that would re-fit the
projection and undo the zoom
Background rect added to capture clicks outside any feature.
Cross-filter integration intentionally NOT added in this commit —
that needs the chart props' setDataMask path and a settings UI for
which feature property to use as the filter key. Separate concern,
follow-up commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace hardcoded WORLDVIEW / COUNTRY / REGION_SET / COMPOSITE choice
tables in controlPanel with options derived from the build pipeline's
manifest.json. Adding a new entry to a YAML config + re-running
./build.sh now populates the control automatically; no plugin code
change needed.
Build script: also writes manifest.json to the plugin's
src/data/manifest.json so controlPanel can `import` it synchronously
(no async fetch needed at chart-edit time). Data files (the actual
GeoJSONs) still live only at superset/static/assets/country-maps/ —
we don't want to bundle 17MB of choropleth data into the JS payload.
Lookup helpers preserve human-friendly labels:
- WORLDVIEW_LABELS — maps NE worldview codes (ukr, default, ind, ...)
to friendly names ("Ukraine (default — Crimea as Ukrainian)" etc.);
unmapped codes render as the raw code
- COUNTRY_LABELS — ISO_A3 → English country name (~85 entries);
formatCountry renders as "France (FRA)"; unmapped codes render raw
- REGION_SET_LABELS / COMPOSITE_LABELS — same pattern
Manifest's regional_aggregations array is grouped by country into
the {country: [(set_id, label), ...]} shape the control panel needs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move build pipeline outputs from a sibling `output/` directory into
Superset's Flask-served `superset/static/assets/country-maps/` so the
plugin can fetch them at runtime without webpack involvement (Flask
serves the tree at `/static/...` directly).
Commit the 220 generated GeoJSONs + manifest.json so a fresh
ephemeral environment can render the chart immediately, no build
step required at deploy time. Trade-off: ~17 MB of generated files
in-tree. (For comparison the legacy plugin commits ~34 MB of
GeoJSON; net change is -17 MB once we remove the legacy plugin in a
future major version.)
Files committed:
superset/static/assets/country-maps/
README.md (humans, not data)
manifest.json
ukr_admin0.geo.json 2.1 MB
ukr_admin1_<adm0_a3>.geo.json × 214 ~50 KB - 662 KB each
regional_<country>_<set>_ukr.geo.json × 4 ~30 KB each
composite_france_overseas_ukr.geo.json 322 KB
Build script changes:
- OUTPUT_DIR computed via SCRIPT_DIR.parents[3] / "superset" /
"static" / "assets" / "country-maps"
- mkdir(parents=True) so a fresh checkout works first run
- Stale `output/` entry kept in scripts/.gitignore for safety
(some local checkouts may have it from earlier iterations)
.gitignore: add re-include lines so superset/static/assets/country-maps/**
gets committed despite the broader superset/static/* exclusion.
SIP_DRAFT updated with the hosting-decision rationale.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three coordinated changes:
1. **VizType.CountryMapV2 = 'country_map_v2'** — new enum value for the
modern plugin's chart type. Existing dashboards continue to use
`country_map` (legacy plugin); new charts get `country_map_v2`.
Naming follows the established pattern (LegacyBubble='bubble',
Bubble='bubble_v2'). Eventual swap (in a major version) can rename
so 'country_map' points to the new plugin.
2. **Legacy plugin marked deprecated**:
- Display name changed to "Country Map (Legacy)" so the chart picker
shows them side-by-side unambiguously
- label: ChartLabel.Deprecated — surfaces the standard deprecation
badge in the UI without hiding from the picker (existing dashboards
keep working; users see "this is deprecated" when editing)
- labelExplanation pointing users at the new chart
3. **MainPreset registers both**:
- CountryMapChartPlugin (legacy) → VizType.CountryMap
- CountryMapV2ChartPlugin (new) → VizType.CountryMapV2
Plus added @superset-ui/plugin-chart-country-map as a workspace
dependency in superset-frontend/package.json so the symlink resolves
during npm install.
Not yet wired (next commits):
- "Switch to new Country Map" button on the legacy plugin's UI
(the deprecation banner shows; the button to migrate form_data
is a separate change in the explore controls)
- Hosting path for the new plugin's GeoJSON outputs (currently
stubbed in transformProps — no static file serving wired yet)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a manifest.json output the plugin can fetch at runtime to
populate worldview / country / region-set / composite dropdowns
dynamically. Adding a new entry to the YAML configs no longer requires
a plugin code change — re-run build.sh, manifest updates, plugin
controls reflect the new options.
Manifest schema:
ne_pinned_tag, ne_pinned_sha, build_timestamp_utc
worldviews: [<wv>, ...]
admin_levels: [0, 1]
countries_by_worldview: {<wv>: [<adm0_a3>, ...]}
regional_aggregations: [{country, set_id, worldview, size_bytes}, ...]
composites: [{id, worldview, size_bytes}, ...]
Sample current output:
1 worldview (ukr), 211 countries with subdivisions, 4 regional sets,
1 composite (france_overseas). Build pinned to NE v5.1.2.
Follow-up commit will replace the hardcoded choice tables in
controlPanel.tsx with manifest-driven options.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
transformProps tests cover the URL-derivation logic — the core piece
that maps form_data into the right build-pipeline output:
- Admin 0 + worldview → world choropleth URL
- Admin 1 + country → per-country file
- Region set + country → regional aggregation
- Composite overrides admin level + country
- Worldview defaults to 'ukr' when not specified
- Different worldviews reflected in URL
- Admin 1 without country → null URL (chart UI prompts)
- Pass-through of metricName/numberFormat/linearColorScheme
- Pass-through of query data + width/height
buildQuery tests cover that we're producing a valid chart/data
QueryContext with the expected shape (one query, form_data preserved,
orderby normalized as array).
These are the units most likely to break silently if someone refactors
the form_data → URL mapping or the query layer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the placeholder control panel with the real form-data surface.
New controls (none of which exist on the legacy plugin):
- worldview: SelectControl, defaults to 'ukr' per documented editorial
choice; configurable per-deployment + per-chart
- admin_level: SelectControl (Admin 0 / Admin 1 / Aggregated regions)
- country: SelectControl, visible only when admin_level !== 0 and no
composite is selected
- region_set: SelectControl, visible only when admin_level === 'aggregated';
choices depend on selected country (sourced from the regional_aggregations
YAML's snapshot table)
- composite: SelectControl (france_overseas etc.); when set, overrides
admin_level + country
- region_includes / region_excludes: free-form multi-select for client-side
filtering at render time (these drive the projection refit too — the
renderer's `filterFeatures` uses them)
- show_flying_islands: CheckboxControl, default true
- name_language: SelectControl with 20 NE-supported languages
The choice tables (WORLDVIEW_CHOICES, COUNTRY_CHOICES, REGION_SET_CHOICES_
BY_COUNTRY, COMPOSITE_CHOICES) are hardcoded snapshots for the POC.
Follow-up commit will replace these with options pulled from the build
pipeline's manifest.json so adding a new worldview or country only
requires a build script run, not a plugin code change.
Standard query controls (entity, metric, adhoc_filters, row_limit,
linear_color_scheme, number_format) preserved from the legacy plugin.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the placeholder renderer with a real D3-based choropleth.
Modernizes the legacy plugin's rendering in three ways:
1. **Modern D3 modules** (d3-geo, d3-array, d3-color) instead of the
d3 v3 monolith. No more @ts-nocheck — fully typed.
2. **Built-in include/exclude filtering** (formData.region_includes /
region_excludes) before render. Projection auto-fits the FILTERED
feature set, so excluding Russia from a Europe view actually zooms
to Europe (the "fit to selection" behavior).
3. **Multi-language display names** via name_language formData field.
Falls back through name_<lang> → name → '' so any worldview/dataset
degrades gracefully.
Other changes:
- Tooltip is React state instead of D3-driven HTML (cleaner unmount,
no XSS surface from .html())
- Color scale uses sequential scheme registry from @superset-ui/core
(same path as legacy)
- Cleanup on unmount via useEffect cleanup function
Stubbed for follow-ups:
- Click-to-zoom (legacy had this; revisit after deprecation banner)
- Cross-filter integration
- Composite projection support (geoAlbersUsa for USA, etc.) — falls
back to plain Mercator + build-time repositioning for now
- Flying-islands toggle: feature-tagging in the build pipeline still
needs to land before the toggle can drop them at render time
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Captures things the maintainer should manually verify before merge,
organized by category:
- Build pipeline (data correctness)
- Visual/cartographic quality (in-chart rendering)
- Controls UX
- Backward compatibility
- Per-deployment customization (config)
- Performance + ops
Items get added/checked off as the implementation progresses.
Includes specific spot-checks the agent can't validate alone:
Crimea-as-Ukrainian visual, Russia Chukchi connectedness, India
J&K under _ukr worldview, French composite layout, etc.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mark Phase 1 substantially complete (5/5 transforms working,
220 outputs verified, all schemas in place) and Phase 2 partially
complete (modern plugin scaffold compiles end-to-end, real renderer +
controls still TODO).
Add a "Current PR state" section showing the build script's actual
output so reviewers reading the SIP can see at a glance what the POC
produces today.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bootstrap the new @superset-ui/plugin-chart-country-map package with
the modern ChartPlugin pattern. Mirrors plugin-chart-handlebars'
structure (the leanest existing modern plugin in the repo).
Files:
package.json — workspace pkg with d3-* deps
tsconfig.json — extends frontend root tsconfig,
references core/chart-controls
types/external.d.ts — ambient types for png/jpg/geojson
README.md — what this plugin is + status
src/
index.ts — package entry
types.ts — CountryMapFormData + transform props
CountryMap.tsx — placeholder renderer (real D3 port
in a follow-up commit)
plugin/
index.ts — ChartPlugin class with metadata
buildQuery.ts — modern chart/data query (this is
the core difference from the legacy
plugin's explore_json path)
controlPanel.tsx — minimal placeholder; full control
set (worldview / admin level /
country / region include-exclude /
fly-islands / language) ports next
transformProps.ts — derives geoJsonUrl from form_data
using the build-script's output
naming convention
Renderer scaffold fetches the resolved GeoJSON URL and shows a small
diagnostic block (URL + feature count + data row count + metric name)
so the plugin compiles end-to-end and can be wired to the chart-type
registry without a real D3 render. Real renderer + controls land in
the next commits.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The monolithic ukr_admin1.geo.json (15MB / 4595 features) was a single
file that any chart would have to download in full just to render one
country's subdivisions. Replace with per-country files keyed by
adm0_a3, each individually simplified.
Also drops single-subdivision countries (useless as choropleths) at
this stage, mirroring the notebook's auto-purge.
Output stats from full run:
Files: 220 total
1 × admin0 (world) ............ 2.1 MB
4 × regional aggregations ..... 23-32 KB each
1 × composite (france_overseas) 322 KB
214 × per-country admin1 ........ 17 KB - 662 KB each (GBR largest)
Per-chart payload:
world choropleth → ukr_admin0.geo.json 2.1 MB
France departments → ukr_admin1_FRA.geo.json 308 KB
US states → ukr_admin1_USA.geo.json ~250 KB
Türkiye NUTS-1 → regional_TUR_nuts_1_ukr.geo.json 23 KB
France w/ overseas → composite_france_overseas_ukr.geo.json 322 KB
All well within usable browser payload range. The plugin will lazy-load
only what's needed for the current chart's worldview/admin-level/country.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the fifth and final transform from the notebook audit.
A composite combines a base country's Admin 1 features with:
- base_repositions (with optional `group: true` for grouped transforms
like Paris + petite couronne treated as one body)
- additions (features pulled from sibling countries' Admin 1, with
optional dissolve, drop_parts, reposition, and attribute set)
Verified on France-with-Overseas:
france_overseas: 108 features → composite_france_overseas_ukr.geo.json
(322,058 bytes)
108 = 101 FRA admin1 departments + 7 additions (Polynésie française,
Terres australes et antarctiques françaises, Wallis-et-Futuna,
Nouvelle-Calédonie, Saint-Pierre-et-Miquelon, Saint-Martin,
Saint-Barthélémy).
Bug fix during implementation: composites pull additions from Admin 1
of sibling countries (Windward Islands is a PYF Admin 1 subdivision,
not an Admin 0 country), not from Admin 0. Initial implementation got
this wrong and warned 0 features. Fixed by sourcing from base_admin1
(the global Admin 1 dataset, which contains all countries'
subdivisions).
New helpers:
- _drop_parts(geom, indices) — drop sub-polygon indices from MultiPolygon
- _translate_and_scale_with_pivot — explicit pivot (vs feature centroid),
used for `group: true` transforms
==== Build pipeline status ====
All 5 declarative transforms implemented and verified:
✓ name_overrides (19 updates per Admin 1 build)
✓ flying_islands (12 reposition + 5 bbox drop)
✓ territory_assignments (4 features added: TWN/HKG/MAC/ALD)
✓ regional_aggregations (4 region sets: TUR/FRA/ITA/PHL)
✓ composite_maps (1 composite: france_overseas)
Current outputs (UA worldview):
ukr_admin0.geo.json 2.1 MB 249 features
ukr_admin1.geo.json 15 MB 4595 features
regional_TUR_nuts_1_ukr.geo.json 23 KB 12 regions
regional_FRA_regions_ukr.geo.json 32 KB 18 regions
regional_ITA_regions_ukr.geo.json 32 KB 20 regions
regional_PHL_regions_ukr.geo.json 32 KB 17 regions
composite_france_overseas_ukr.geo.json 322 KB 108 features
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the fourth transform: dissolve Admin 1 subdivisions into
coarser administrative regions. Supports two mapping styles:
- explicit_mapping: per-region {name, members: [iso_3166_2, ...]}
- grouping_field: dissolve by an existing NE field (e.g. region_cod)
Verified counts match notebook expectations exactly:
TUR/nuts_1: 81 subdivisions → 12 regions → 23 KB
FRA/regions: 101 subdivisions → 18 regions → 32 KB
ITA/regions: 110 subdivisions → 20 regions → 32 KB
PHL/regions: 118 subdivisions → 17 regions → 32 KB
Per-region-set output is its own file (`regional_<country>_<set>_
<worldview>.geo.json`) so the plugin can lazy-load only what's needed
for the current chart.
Implementation:
- Filter base geo to country features
- Tag each with derived `_region_code` and `_region_name` (via reverse
lookup of explicit_mapping, or via grouping_field value)
- mapshaper -dissolve handles the polygon merging in one pass
- Rename derived fields → standard `iso_3166_2` and `name` on output
Output sizes are tiny — each per-chart payload becomes ~30 KB instead
of pulling the full Admin 1 layer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the third transform: pull features from sibling Admin 0
records into a destination country's Admin 1 view. Used for:
- China + Taiwan/HK/Macau (NE keeps each as separate Admin 0)
- Finland + Åland (missing from FIN admin 1; NE keeps Åland as ALD
admin 0)
Verified on real data:
Building worldview=ukr admin_level=1
territory_assignments: added 4 features from sibling Admin 0 records
(4 = TWN/HKG/MAC + ALD; ARMM-renamed BARMM region picks up correctly
because name_overrides ran first.)
Two bugs fixed along the way:
1. **Property name casing.** NE Admin 0 ships with uppercase property
names (ADM0_A3, NAME_EN), Admin 1 with lowercase. All transforms
downstream assume lowercase, so we now normalize to lowercase at
shapefile-conversion time. Bonus: fixes a silent flying_islands
bug where `adm0_a3` filters never matched at Admin 0 because the
props were uppercase.
2. **drop_outside_bbox at Admin 0.** A country's multi-polygon often
includes overseas territories (Netherlands → Caribbean), so bbox
filtering at Admin 0 would drop entire countries. Now guarded to
only run at Admin 1 where each feature is a single subdivision.
3. **Åland's NE code.** NE uses ALD, not the ISO 3166-1 ALA. Updated
territory_assignments.yaml with comment noting the divergence.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add mapshaper -simplify as a final-stage pass after the Python
transforms. Two-stage so the transforms operate on full-resolution
geometry (no risk of repositioning ghost features that got simplified
away) but the shipped output is browser-sized.
Default 5% with keep-shapes — preserves recognizable country shapes
while dropping the per-vertex bloat. keep-shapes prevents tiny features
(small island chains) from being collapsed away entirely.
Results on real outputs:
Admin 0: 23.6 MB → 2.1 MB (-91%, 249 features intact)
Admin 1: 67.7 MB → 15 MB (-78%, 4591 features, transforms preserved)
Per-feature simplify factors (the notebook had a size-based ladder)
can be added later if specific countries need tuning. 5% is the
mapshaper-recommended "good enough for web" default.
Open thought for future: split Admin 1 output by country
(`<worldview>_admin1_<adm0_a3>.geo.json`) so the per-chart payload is
~50KB-1MB instead of one 15MB global file. Will pair naturally with
the territory_assignments pass (which is per-country anyway).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add Admin 1 build path and the second declarative transform. Exercises
the YAML config layer on real data:
Building worldview=ukr admin_level=1
loaded 4596 features
name_overrides: applied 19 field updates across 10 entries
flying_islands: repositioned 12 features, dropped 5 (outside-bbox)
wrote ukr_admin1.geo.json (67,677,079 bytes, 4591 features)
Counts verified against expectations:
- 19 name_overrides = 2 France typos + 6 France ISO codes
+ 5 PHL Caraga renames + 6 PHL BARMM renames
- 12 repositions = 2 USA + 1 NOR + 2 PRT + 2 ESP + 5 FRA
- 5 drops = NLD Caribbean + GBR overseas territories
New: pure-Python translate/scale geometry transform (no shapely dep);
operates on Polygon/MultiPolygon coordinates. Scale pivot is the bbox
center of each matched feature — good enough for the visual layout
purposes we use it for. Output bbox correctness verified by counts.
Refactor: extract `build_one(worldview, admin_level, ...)` so the
target matrix can grow in subsequent commits.
What's stubbed (TODO inline): territory_assignments, composite_maps,
regional_aggregations, simplification, procedural/. Output is
uncompressed and unsimplified (67MB) — simplification will land with
the mapshaper -simplify pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
End-to-end working pipeline replacing the legacy notebook for one
worldview / admin level. Verified locally:
$ ./build.sh
Country Map build — pinned to NE v5.1.2 (f1890d9f)
Loaded 10 name override entries
Building worldview=ukr admin_level=0
Downloading NE ne_10m_admin_0_countries_ukr (worldview=ukr)…
mapshaper: ne_10m_admin_0_countries_ukr.shp → _raw_ukr_admin0.geo.json
loaded 249 features
name_overrides: applied 0 field updates across 10 entries
wrote .../output/ukr_admin0.geo.json (23,639,348 bytes)
Done.
What's wired:
- NE download from pinned tag (v5.1.2 / SHA f1890d9f) with cache
- Shapefile → GeoJSON via mapshaper CLI
- YAML config loading (currently just name_overrides)
- name_overrides transform with {match, set} semantics, including
the {in: [...]} list-membership matcher
- Output writes to scripts/output/ (gitignored)
- build.sh wrapper validates Python + Node + PyYAML are available
What's stubbed for future commits (TODO inline):
- Multiple worldviews (currently UA only)
- Admin 1 build (where name_overrides actually fire — currently no
features in Admin 0 match the FRA/PHL admin1 entries)
- flying_islands, territory_assignments, regional_aggregations,
composite_maps transforms
- Simplification (mapshaper -simplify)
- Procedural escape-hatch orchestration
- Manifest with NE SHA + build metadata
The 0 overrides applied is correct, not a bug: all current entries
target Admin 1 features.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The original draft was missing entries because notebook cell 63 was
truncated in the audit. Reading the full cell surfaced:
- Saint Martin (MAF) + Saint Barthélémy (BLM) as additional sister
Admin 0 territories (small Caribbean islands, scaled up significantly
for visibility — 5x and 8x respectively)
- Paris + petite couronne (Hauts-de-Seine, Seine-Saint-Denis,
Val-de-Marne) as a metropolitan zoom-in (group + translate + scale 3x)
- Per-territory metadata renames (Polynésie française, Nouvelle-
Calédonie, etc.) + ISO 3166-2 code assignments (FR-PF, FR-NC, etc.)
Schema additions:
- base_repositions[].group: true — when match yields multiple features,
transform them as a single MultiPolygon then split back out
(preserves per-feature attributes). Used for the Paris zoom-in.
- additions[].set: { name, iso_3166_2, ... } — override attributes on
the added/dissolved feature
SPM offset placeholder is gone; composite definition now matches the
notebook's output exactly (modulo the build script implementing the
declarative schema).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First-pass schemas for the build pipeline's declarative config layer.
Each schema is documented inline + populated with concrete entries
ported from the legacy notebook's audited touchups (those that the
obsolescence check determined still need to ship).
scripts/
├── README.md — pipeline overview, layout, workflow
├── config/
│ ├── name_overrides.yaml — France typos, ISO codes; PHL renames
│ ├── flying_islands.yaml — USA/NOR/PRT/ESP/FRA repositions; NLD/GBR drops
│ ├── territory_assignments.yaml — China + SARs; Finland + Åland
│ ├── regional_aggregations.yaml — Turkey NUTS-1; FRA/ITA/PHL regions
│ └── composite_maps.yaml — France-with-Overseas
└── procedural/
└── README.md — escape-hatch rules + skeleton (currently empty)
All five YAML files parse cleanly (validated with PyYAML).
Schema design choices:
- Every entry has a `description:` field. Forces honest documentation
of why each fix exists; reviewers can scan rationale at a glance.
- Match semantics: simple AND-of-conditions; supports `{ in: [...] }`
for value-set matching.
- composite_maps and territory_assignments share the "pull feature
from sibling Admin 0" primitive; build script can implement once.
- composite_maps.yaml has a TODO marker for SPM offsets — notebook
cell 63 was truncated in the audit; will backfill during build
script implementation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Most touchups fit declarative YAML cleanly (typos, ISO codes,
repositions, dissolves, composites). A few don't — e.g., the
France-with-Overseas notebook drops a specific sub-polygon by index
to avoid visual conflict with Corsica.
Add a `procedural/` directory of small, named, single-purpose Python
scripts as an escape hatch. Build orchestrator runs YAML transforms
first, then iterates procedural/*.py in numeric order.
Why this is much better than the notebook for the same content:
- Each fix is a separate file → conflicts localize to one fix at a
time, not "the whole notebook touched the same cell"
- No kernel state, no output churn, pure functions on data
- Naming forces documentation; reviewers see add/remove at a glance
- Bounded growth: a 50-file procedural/ dir is annoying enough to
signal "push back into YAML or upstream"; the notebook had no
such signal and just bloated
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Verified each notebook touchup against current Natural Earth Admin 1
data. Headline wins:
OBSOLETE (NE caught up or worldview handles it):
- Vietnam diacritics: NE has correct values in name_vi field; manual
rewrites collapse to zero by exposing name_language=vi
- Crimea/Sevastopol: handled by _ukr worldview (already confirmed)
- India Kashmir/Ladakh geometry: NE has UTs with correct ISO codes;
worldview selection adjusts northern boundary
- Russia Chukchi antimeridian: NE has split geometry into properly-
bounded polygons (x range [-180, 180]); D3 projection should handle
- Latvia third-party file: NE has 119 fine-grained admin1 features
- Philippines third-party file: NE has 118 admin1 features
- external_overrides.yaml DROPPED from the design
STILL NEEDED (NE still wrong / no upstream support):
- France typos (Seien-et-Marne, Haute-Rhin still in NE)
- France ISO codes (FR-75 → FR-75C, FR-GP → FR-971, etc. — 6 features)
- Philippines admin renames (Dinagat→Caraga, ARMM→BARMM, 11 features)
- China + SARs (Taiwan/HK/Macau still separate Admin 0 in NE)
- Finland + Åland (missing from FIN admin 1 in NE)
- Regional aggregations (Turkey NUTS-1, France/Italy/PHL regions)
- France-with-Overseas composite
- Flying-island repositions (USA, Norway, Portugal, Spain, France DOM)
Net: design surface shrinks, no third-party data dependencies in-tree,
config files stable enough that ongoing maintenance is small.
Two follow-up verifications still pending: Russia Chukchi renders
correctly under D3, India J&K geometry matches current Indian
boundary expectations under _ukr worldview.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Read all 96 cells of the legacy notebook; categorized every kind of
work it does into 14 buckets and mapped each to either:
- ✅ already covered by the proposed design
- 🟡 covered but requires a YAML config we'll need to port
- 🟠 needs a NEW config file or pipeline feature beyond what was sketched
- ⚪ might be obsolete in current NE — verify per case
Surfaced four NEW config-driven build features beyond the original
sketch:
- territory_assignments.yaml (China + SARs, Finland + Åland)
- regional_aggregations.yaml (Turkey NUTS-1, France/Italy/Philippines
regions; new "Aggregated regions" admin level)
- composite_maps.yaml (France-with-Overseas multi-country composite)
- external_overrides.yaml (India/Latvia/Philippines third-party
GeoJSON; verify whether still needed against current NE)
Plus extended flying_islands.yaml with drop|reposition action modes
to subsume the bounds-clipping pattern (Netherlands, UK).
Updated open questions, smoke-test fixtures (now 5 cases that
exercise the full surface), and Phase 1 of the implementation plan
with the obsolescence check + external-GeoJSON check as concrete
follow-ups.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Living document tracking the design for the Country Map rework. Will
be filed as a proper SIP issue when the POC matures; this file is
working scratch in the meantime.
Captured so far:
- Motivation: disputed-borders flashpoints, Jupyter notebook pain,
legacy explore_json endpoint
- Goals: configurable worldview, Admin 0 + Admin 1, per-deployment
+ per-chart customization, mapshaper-based reproducible build
pipeline, modern chart/data endpoint
- Spike findings: NE UA worldview vs Default — confirms Crimea-as-
Ukrainian + several other commonly-expected positions (Kosovo,
Western Sahara, Palestine, Cyprus, Kashmir)
- Deprecation plan: in-UI "switch to new Country Map" button + banner
on legacy, modeled on existing deprecated-chart pattern
Open questions tracked inline; implementation phases sketched.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
"⚠️ Country Map: drift detected between locally committed GeoJSON outputs and CI-regenerated outputs. Either the YAML configs were updated without re-running \`./scripts/build.sh\`, OR mapshaper output differs cross-platform (macOS dev vs Linux CI). Investigate before merge."
# Part 2: Verify RSA key - this is the same as running `gpg --verify {release}.asc {release}` and comparing the RSA key and email address against the KEYS file # noqa: E501
@@ -24,6 +24,43 @@ assists people when migrating to a new version.
## Next
### Country Map plugin redesigned (legacy plugin deprecated)
A new Country Map chart plugin (`@superset-ui/plugin-chart-country-map`, `viz_type='country_map_v2'`) replaces the legacy plugin (`@superset-ui/legacy-plugin-chart-country-map`, `viz_type='country_map'`). The legacy plugin is still installed and functional — existing dashboards continue to render unchanged — but is now badged as deprecated in the chart-type picker, with the display name "Country Map (Legacy)".
**What changed:**
- Modern `chart/data` endpoint instead of `explore_json` (full async / caching / semantic-layer integration)
- Configurable per-deployment + per-chart **worldview** for disputed regions (default ships Natural Earth's `_ukr` worldview, which shows Crimea as Ukrainian and aligns with broadly-expected positions on Kosovo, Western Sahara, Palestine, Cyprus, Kashmir; configurable via `superset_config.COUNTRY_MAP.default_worldview`)
- Both **Admin 0** (countries) **and Admin 1** (subdivisions) supported in one plugin (subsumes years of bespoke per-country submissions like French departments, Italian regions, Türkiye city map)
- Build pipeline replaces the legacy Jupyter notebook with `mapshaper`-CLI-based reproducible scripts + 5 declarative YAML configs in `superset-frontend/plugins/plugin-chart-country-map/scripts/`
**Migration behavior:**
- Existing charts using `viz_type='country_map'` continue to render against the legacy plugin. No DB migrations needed.
- The legacy plugin's chart picker entry shows the standard Deprecation badge with an explanation pointing at the new chart type.
- Users can re-create existing charts using the modern plugin manually; an automated "Switch to new chart" button is planned.
- The legacy plugin (and its committed `src/countries/*.geojson` assets) will be removed in a future major release once existing dashboards have migrated.
**For maintainers / power users with custom modifications:**
- The legacy plugin's Jupyter notebook (`scripts/Country Map GeoJSON Generator.ipynb`) is no longer the source of truth. Local touchups should be ported to the new plugin's YAML configs (`scripts/config/{name_overrides,flying_islands,territory_assignments,regional_aggregations,composite_maps}.yaml`) or, for genuine edge cases that don't fit YAML, to the `scripts/procedural/` escape-hatch directory.
- See `superset-frontend/plugins/plugin-chart-country-map/SIP_DRAFT.md` for the full design rationale.
### Granular Export Controls
A new feature flag `GRANULAR_EXPORT_CONTROLS` introduces three fine-grained permissions that replace the legacy `can_csv` permission:
When the feature flag is enabled, these permissions are enforced on both the frontend (disabled buttons with tooltips) and backend (403 responses from API endpoints). When disabled, legacy `can_csv` behavior is preserved.
**Migration behavior:** All three new permissions are granted to every role that currently has `can_csv`, preserving existing access. Admins can then selectively revoke individual export permissions from specific roles as needed.
### Deck.gl MapBox viewport and opacity controls are functional
The Deck.gl MapBox chart's **Opacity**, **Default longitude**, **Default latitude**, and **Zoom** controls were previously non-functional — changing them had no effect on the rendered map. These controls are now wired up correctly.
@@ -32,6 +69,13 @@ The Deck.gl MapBox chart's **Opacity**, **Default longitude**, **Default latitud
**To restore fit-to-data behavior:** Open the chart in Explore, clear the **Default longitude**, **Default latitude**, and **Zoom** fields in the Viewport section, and re-save the chart.
### Combined datasource list endpoint
Added a new combined datasource list endpoint at `GET /api/v1/datasource/` to serve datasets and semantic views in one response.
- The endpoint is available to users with at least one of `can_read` on `Dataset` or `SemanticView`.
- Semantic views are included only when the `SEMANTIC_LAYERS` feature flag is enabled.
- The endpoint enforces strict `order_column` validation and returns `400` for invalid sort columns.
### ClickHouse minimum driver version bump
The minimum required version of `clickhouse-connect` has been raised to `>=0.13.0`. If you are using the ClickHouse connector, please upgrade your `clickhouse-connect` package. The `_mutate_label` workaround that appended hash suffixes to column aliases has also been removed, as it is no longer needed with modern versions of the driver.
@@ -307,7 +351,7 @@ Note: Pillow is now a required dependency (previously optional) to support image
- [33116](https://github.com/apache/superset/pull/33116) In Echarts Series charts (e.g. Line, Area, Bar, etc.) charts, the `x_axis_sort_series` and `x_axis_sort_series_ascending` form data items have been renamed with `x_axis_sort` and `x_axis_sort_asc`.
There's a migration added that can potentially affect a significant number of existing charts.
- [32317](https://github.com/apache/superset/pull/32317) The horizontal filter bar feature is now out of testing/beta development and its feature flag `HORIZONTAL_FILTER_BAR` has been removed.
- [31590](https://github.com/apache/superset/pull/31590) Marks the begining of intricate work around supporting dynamic Theming, and breaks support for [THEME_OVERRIDES](https://github.com/apache/superset/blob/732de4ac7fae88e29b7f123b6cbb2d7cd411b0e4/superset/config.py#L671) in favor of a new theming system based on AntD V5. Likely this will be in disrepair until settling over the 5.x lifecycle.
- [31590](https://github.com/apache/superset/pull/31590) Marks the beginning of intricate work around supporting dynamic Theming, and breaks support for [THEME_OVERRIDES](https://github.com/apache/superset/blob/732de4ac7fae88e29b7f123b6cbb2d7cd411b0e4/superset/config.py#L671) in favor of a new theming system based on AntD V5. Likely this will be in disrepair until settling over the 5.x lifecycle.
- [32432](https://github.com/apache/superset/pull/32432) Moves the List Roles FAB view to the frontend and requires `FAB_ADD_SECURITY_API` to be enabled in the configuration and `superset init` to be executed.
- [34319](https://github.com/apache/superset/pull/34319) Drill to Detail and Drill By is now supported in Embedded mode, and also with the `DASHBOARD_RBAC` FF. If you don't want to expose these features in Embedded / `DASHBOARD_RBAC`, make sure the roles used for Embedded / `DASHBOARD_RBAC`don't have the required permissions to perform D2D actions.
@@ -322,7 +366,7 @@ Note: Pillow is now a required dependency (previously optional) to support image
- [31774](https://github.com/apache/superset/pull/31774): Fixes the spelling of the `USE-ANALAGOUS-COLORS` feature flag. Please update any scripts/configuration item to use the new/corrected `USE-ANALOGOUS-COLORS` flag spelling.
- [31582](https://github.com/apache/superset/pull/31582) Removed the legacy Area, Bar, Event Flow, Heatmap, Histogram, Line, Sankey, and Sankey Loop charts. They were all automatically migrated to their ECharts counterparts with the exception of the Event Flow and Sankey Loop charts which were removed as they were not actively maintained and not widely used. If you were using the Event Flow or Sankey Loop charts, you will need to find an alternative solution.
- [31198](https://github.com/apache/superset/pull/31198) Disallows by default the use of the following ClickHouse functions: "version", "currentDatabase", "hostName".
- [29798](https://github.com/apache/superset/pull/29798) Since 3.1.0, the intial schedule for an alert or report was mistakenly offset by the specified timezone's relation to UTC. The initial schedule should now begin at the correct time.
- [29798](https://github.com/apache/superset/pull/29798) Since 3.1.0, the initial schedule for an alert or report was mistakenly offset by the specified timezone's relation to UTC. The initial schedule should now begin at the correct time.
- [30021](https://github.com/apache/superset/pull/30021) The `dev` layer in our Dockerfile no long includes firefox binaries, only Chromium to reduce bloat/docker-build-time.
- [30099](https://github.com/apache/superset/pull/30099) Translations are no longer included in the default docker image builds. If your environment requires translations, you'll want to set the docker build arg `BUILD_TRANSLATIONS=true`.
- [31262](https://github.com/apache/superset/pull/31262) NOTE: deprecated `pylint` in favor of `ruff` as our only python linter. Only affect development workflows positively (not the release itself). It should cover most important rules, be much faster, but some things linting rules that were enforced before may not be enforce in the exact same way as before.
@@ -335,7 +379,7 @@ Note: Pillow is now a required dependency (previously optional) to support image
- [25166](https://github.com/apache/superset/pull/25166) Changed the default configuration of `UPLOAD_FOLDER` from `/app/static/uploads/` to `/static/uploads/`. It also removed the unused `IMG_UPLOAD_FOLDER` and `IMG_UPLOAD_URL` configuration options.
- [30284](https://github.com/apache/superset/pull/30284) Deprecated GLOBAL_ASYNC_QUERIES_REDIS_CONFIG in favor of the new GLOBAL_ASYNC_QUERIES_CACHE_BACKEND configuration. To leverage Redis Sentinel, set CACHE_TYPE to RedisSentinelCache, or use RedisCache for standalone Redis
- [31961](https://github.com/apache/superset/pull/31961) Upgraded React from version 16.13.1 to 17.0.2. If you are using custom frontend extensions or plugins, you may need to update them to be compatible with React 17.
- [31260](https://github.com/apache/superset/pull/31260) Docker images now use `uv pip install` instead of `pip install` to manage the python envrionment. Most docker-based deployments will be affected, whether you derive one of the published images, or have custom bootstrap script that install python libraries (drivers)
- [31260](https://github.com/apache/superset/pull/31260) Docker images now use `uv pip install` instead of `pip install` to manage the python environment. Most docker-based deployments will be affected, whether you derive one of the published images, or have custom bootstrap script that install python libraries (drivers)
### Potential Downtime
@@ -412,7 +456,7 @@ Note: Pillow is now a required dependency (previously optional) to support image
- [26462](https://github.com/apache/superset/issues/26462): Removes the Profile feature given that it's not actively maintained and not widely used.
- [26377](https://github.com/apache/superset/pull/26377): Removes the deprecated Redirect API that supported short URLs used before the permalink feature.
- [26329](https://github.com/apache/superset/issues/26329): Removes the deprecated `DASHBOARD_NATIVE_FILTERS` feature flag. The previous value of the feature flag was `True` and now the feature is permanently enabled.
- [25510](https://github.com/apache/superset/pull/25510): Reenforces that any newly defined Python data format (other than epoch) must adhere to the ISO 8601 standard (enforced by way of validation at the API and database level) after a previous relaxation to include slashes in addition to dashes. From now on when specifying new columns, dataset owners will need to use a SQL expression instead to convert their string columns of the form %Y/%m/%d etc. to a `DATE`, `DATETIME`, etc. type.
- [25510](https://github.com/apache/superset/pull/25510): Reinforces that any newly defined Python data format (other than epoch) must adhere to the ISO 8601 standard (enforced by way of validation at the API and database level) after a previous relaxation to include slashes in addition to dashes. From now on when specifying new columns, dataset owners will need to use a SQL expression instead to convert their string columns of the form %Y/%m/%d etc. to a `DATE`, `DATETIME`, etc. type.
- [26372](https://github.com/apache/superset/issues/26372): Removes the deprecated `GENERIC_CHART_AXES` feature flag. The previous value of the feature flag was `True` and now the feature is permanently enabled.
WEBDRIVER_BASEURL=f"http://superset_app{os.environ.get('SUPERSET_APP_ROOT','/')}/"# When using docker compose baseurl should be http://superset_nginx{ENV{BASEPATH}}/ # noqa: E501
When a report includes file attachments (CSV, PDF, or PNG screenshots), the request is sent as `multipart/form-data` instead. In that case, each top-level payload field (`name`, `text`, `description`, `url`) becomes its own form field, and nested structures like `header` are serialized as a JSON-encoded string in their own field. Every attachment is added as a repeated form field named `files`:
Webhook consumers should branch on `Content-Type`: parse the body as JSON when `application/json`, or read the individual form fields (decoding `header` as JSON) when `multipart/form-data`.
#### HTTPS Enforcement
To require HTTPS webhook URLs (recommended for production), set:
```python
ALERT_REPORTS_WEBHOOK_HTTPS_ONLY = True
```
When enabled, Superset rejects webhook configurations that use `http://` URLs.
#### Retry Behavior
Superset automatically retries webhook deliveries on `429 Too Many Requests` and `5xx` server errors using exponential backoff.
### Kubernetes-specific
- You must have a `celery beat` pod running. If you're using the chart included in the GitHub repository under [helm/superset](https://github.com/apache/superset/tree/master/helm/superset), you need to put `supersetCeleryBeat.enabled = true` in your values override.
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
*/}
---
title: AWS IAM Authentication
sidebar_label: AWS IAM Authentication
sidebar_position: 15
---
# AWS IAM Authentication for AWS Databases
Superset supports IAM-based authentication for **Amazon Aurora** (PostgreSQL and MySQL) and **Amazon Redshift**. IAM auth eliminates the need for database passwords — Superset generates a short-lived auth token using temporary AWS credentials instead.
Cross-account IAM role assumption via STS `AssumeRole` is supported, allowing a Superset deployment in one AWS account to connect to databases in a different account.
## Prerequisites
- Enable the `AWS_DATABASE_IAM_AUTH` feature flag in `superset_config.py`. IAM authentication is gated behind this flag; if it is disabled, connections using `aws_iam` fail with *"AWS IAM database authentication is not enabled."*
```python
FEATURE_FLAGS = {
"AWS_DATABASE_IAM_AUTH": True,
}
```
- `boto3` must be installed in your Superset environment:
```bash
pip install boto3
```
- The Superset server's IAM role (or static credentials) must have permission to call `sts:AssumeRole` (for cross-account) or the same-account permissions for the target service:
- **Redshift Serverless**: `redshift-serverless:GetCredentials` and `redshift-serverless:GetWorkgroup`
- SSL must be enabled on the Aurora / Redshift endpoint (required for IAM token auth).
## Configuration
IAM authentication is configured via the **encrypted_extra** field of the database connection. Access this field in the **Advanced** → **Security** section of the database connection form, under **Secure Extra**.
**3. Configure the database connection in Superset** using the `role_arn` and `external_id` from the trust policy (as shown in the configuration example above).
## Credential Caching
STS credentials are cached in memory keyed by `(role_arn, region, external_id)` with a 10-minute TTL. This reduces the number of STS API calls when multiple queries are executed with the same connection. Tokens are refreshed automatically before expiry.
Additional selenium web drive configuration can be set using `WEBDRIVER_CONFIGURATION`. You can
implement a custom function to authenticate selenium. The default function uses the `flask-login`
To control which user account is used for rendering thumbnails and warming up caches, configure
`THUMBNAIL_EXECUTORS` and `CACHE_WARMUP_EXECUTORS`. Each accepts a list of executor types (which
resolve to an owner, creator, modifier, or the currently-logged-in user) and/or a `FixedExecutor`
pinned to a specific username. By default, thumbnails render as the current user
(`ExecutorType.CURRENT_USER`) and cache warmup runs as the chart/dashboard owner
(`ExecutorType.OWNER`).
To force both to run as a dedicated service account (`admin` in this example):
```python
from superset.tasks.types import ExecutorType, FixedExecutor
THUMBNAIL_EXECUTORS = [FixedExecutor("admin")]
CACHE_WARMUP_EXECUTORS = [FixedExecutor("admin")]
```
Use a dedicated read-only service account here rather than a personal admin account, so that
thumbnail rendering and cache warmup tasks don't fail if a specific user's credentials change.
Additional Selenium WebDriver configuration can be set using `WEBDRIVER_CONFIGURATION`. You can
implement a custom function to authenticate Selenium. The default function uses the `flask-login`
session cookie. Here's an example of a custom function signature:
```python
@@ -159,6 +178,20 @@ Then on configuration:
WEBDRIVER_AUTH_FUNC = auth_driver
```
## ETag Support for Thumbnails
Thumbnail and screenshot endpoints return `ETag` response headers based on the cached content digest. Clients can use conditional requests to avoid downloading unchanged images:
```
GET /api/v1/chart/42/thumbnail/
If-None-Match: "abc123..."
→ 304 Not Modified (if unchanged)
→ 200 OK (with new image if changed)
```
This is particularly useful for embedded dashboards and external integrations that periodically poll for updated screenshots — unchanged thumbnails return immediately with no payload.
## Distributed Coordination Backend
Superset supports an optional distributed coordination (`DISTRIBUTED_COORDINATION_CONFIG`) for
For public OAuth2 clients that cannot securely store a client secret, enable Proof Key for Code Exchange (PKCE) by adding `code_challenge_method` to the `remote_app` configuration:
```python
OAUTH_PROVIDERS = [
{
'name': 'myProvider',
'remote_app': {
'client_id': 'myClientId',
'client_secret': 'mySecret', # may be empty for pure public clients
PKCE (`S256`) is recommended for all OAuth2 flows, even when a client secret is present, as it protects against authorization code interception attacks.
## LDAP Authentication
FAB supports authenticating user credentials against an LDAP server.
@@ -444,6 +472,38 @@ FEATURE_FLAGS = {
A current list of feature flags can be found in the [Feature Flags](/admin-docs/configuration/feature-flags) documentation.
## Security Configuration
### HASH_ALGORITHM
Controls the hashing algorithm used for internal checksums and cache keys (thumbnails, cache keys, etc.). The default is `sha256`, which satisfies environments with stricter compliance requirements (e.g., FedRAMP). Set it to `md5` to retain the legacy behavior from older Superset deployments:
```python
HASH_ALGORITHM = "sha256" # default; set to "md5" for legacy behavior
```
A companion `HASH_ALGORITHM_FALLBACKS` list (default: `["md5"]`) lets UUID lookups fall back to older algorithms, which enables gradual migration without breaking existing entries. Set it to `[]` for strict mode (use only `HASH_ALGORITHM`).
:::note
This setting affects internal Superset operations only, not user passwords or authentication tokens. Changing it in an existing deployment may invalidate cached values but does not require a database migration.
:::
## SQL Lab Query History Pruning
SQL Lab query history is stored in the metadata database and is **not** pruned by default. To trim older rows, enable the `prune_query` Celery beat task by uncommenting it in `CELERY_BEAT_SCHEDULE` and choosing a retention window:
Adjust `retention_period_days` to control how long query rows are kept. Companion opt-in tasks (`prune_logs`, `prune_tasks`) exist for pruning the logs and tasks tables; see the commented-out examples in `superset/config.py`. Without enabling these tasks, the metadata database will grow unbounded over time.
:::resources
- [Blog: Feature Flags in Apache Superset](https://preset.io/blog/feature-flags-in-apache-superset-and-preset/)
The superset cli allows you to import and export datasources from and to YAML. Datasources include
databases. The data is expected to be organized in the following hierarchy:
:::info
Superset's ZIP-based import/export also covers **dashboards**, **charts**, and **saved queries**, exercised through the UI and REST API. The [Dashboard Import Overwrite Behavior](#dashboard-import-overwrite-behavior) and [UUIDs in API Responses](#uuids-in-api-responses) sections below document the behavior shared across all asset types.
:::
```text
├──databases
| ├──database_1
@@ -26,6 +30,10 @@ databases. The data is expected to be organized in the following hierarchy:
| └── ... (more databases)
```
:::note
When you export a database connection, the `masked_encrypted_extra` field (used for sensitive connection parameters such as service account JSON, OAuth tokens, and other encrypted credentials) is included in the export. When importing on another instance, these values are decrypted and re-encrypted using the destination instance's `SECRET_KEY`. Ensure the receiving instance has a valid `SECRET_KEY` configured before importing.
:::
## Exporting Datasources to YAML
You can print your current datasources to stdout by running:
@@ -75,6 +83,29 @@ The optional username flag **-u** sets the user used for the datasource import.
When importing a dashboard ZIP with the **overwrite** option enabled, any existing charts that are part of the dashboard are **replaced** rather than duplicated. This applies to:
- Charts whose UUID matches a chart already present in the target instance
- The full chart configuration (query, visualization type, columns, metrics) is replaced by the imported version
If you import without the overwrite flag, existing charts with conflicting UUIDs are left unchanged and the import skips those objects. Use overwrite when you want to push a fully updated dashboard (including chart definitions) from a development or staging environment to production.
## UUIDs in API Responses
The REST API POST endpoints for **datasets**, **charts**, and **dashboards** include the auto-generated `uuid` field in the response body:
```json
{
"id": 42,
"uuid": "b8a8d5c3-1234-4abc-8def-0123456789ab",
...
}
```
UUIDs remain stable across import/export cycles and can be used for cross-environment workflows — for example, recording a UUID when creating a chart in development and using it to identify the matching chart after importing into production.
## Legacy Importing Datasources
### From older versions of Superset to current version
@@ -501,6 +501,7 @@ All MCP settings go in `superset_config.py`. Defaults are defined in `superset/m
| `MCP_SERVICE_URL` | `None` | Public base URL for MCP-generated links (set this when behind a reverse proxy) |
| `MCP_DEBUG` | `False` | Enable debug logging |
| `MCP_DEV_USERNAME` | -- | Superset username for development mode (no auth) |
| `MCP_RBAC_ENABLED` | `True` | Enforce Superset's role-based access control on MCP tool calls. When `True`, each tool checks that the authenticated user has the required FAB permission before executing. Disable only for testing or trusted-network deployments. |
### Authentication
@@ -516,6 +517,7 @@ All MCP settings go in `superset_config.py`. Defaults are defined in `superset/m
| `MCP_USER_RESOLVER` | `None` | Custom function `(app, access_token) -> username` to extract a Superset username from a validated JWT token. When `None`, the default resolver checks `preferred_username`, `username`, `email`, and `sub` claims in that order. |
### Response Size Guard
@@ -599,6 +601,43 @@ MCP_STORE_CONFIG = {
| `event_store_max_events` | `100` | Maximum events retained per session |
| `event_store_ttl` | `3600` | Event TTL in seconds |
### Tool Search
By default the MCP server exposes a lightweight tool-search interface instead of advertising every tool at once. This reduces the initial context sent to the LLM by ~70%, which lowers cost and latency. The AI client discovers tools on demand by calling `search_tools` and then invokes them via `call_tool`.
```python
MCP_TOOL_SEARCH_CONFIG = {
"enabled": True,
"strategy": "bm25", # "bm25" (natural language) or "regex"
| `max_results` | `5` | Maximum tools returned per search query |
| `always_visible` | See above | Tools that always appear in `list_tools`, regardless of search |
| `include_schemas` | `False` | When `False` (default, "summary mode"), search results omit `inputSchema` entirely and include a lightweight `parameters_hint` listing top-level parameter names. Set to `True` to include the full `inputSchema` in search results. Full schemas are always used when a tool is actually invoked via `call_tool`. |
| `compact_schemas` | `True` | Strip `$defs` / `$ref` and replace with `{"type": "object"}` in search results to reduce token cost. Only takes effect when `include_schemas=True` — ignored in summary mode. |
| `max_description_length` | `300` | Truncate tool descriptions in search results (0 = no truncation). Applies in both summary and full-schema modes. |
:::tip
Set `enabled: False` to revert to the traditional "show all tools at once" behavior, which some clients or workflows may prefer.
:::
Tool search reduces the initial token cost from ~15–20K tokens (full catalog) down to ~4–5K tokens (pinned tools + search interface) — roughly 85% savings at the start of each conversation.
### Session & CSRF
These values are flat-merged into the Flask app config used by the MCP server process:
@@ -620,6 +659,102 @@ MCP_CSRF_CONFIG = {
---
## Access Control
### RBAC Enforcement
The MCP server respects Superset's full role-based access control (RBAC). Every authenticated user can only access the data and operations their Superset roles permit — the same rules that apply in the Superset UI apply through MCP.
Each tool declares one or more required FAB permissions. The table below maps tool groups to their permission requirements:
| Tool group | Required FAB permission |
|------------|------------------------|
| `list_charts`, `get_chart_info`, `get_chart_data`, `get_chart_preview`, `generate_chart`, `update_chart` | `can_read` on `Chart` (read), `can_write` on `Chart` (mutate) |
| `list_dashboards`, `get_dashboard_info`, `generate_dashboard`, `add_chart_to_existing_dashboard` | `can_read` on `Dashboard` (read), `can_write` on `Dashboard` (mutate) |
| `list_datasets`, `get_dataset_info`, `create_virtual_dataset` | `can_read` on `Dataset` (read), `can_write` on `Dataset` (mutate) |
| `list_databases`, `get_database_info` | `can_read` on `Database` |
| `execute_sql` | `can_execute_sql_query` on `SQLLab` |
| `open_sql_lab_with_context` | `can_read` on `SQLLab` |
| `save_sql_query` | `can_write` on `SavedQuery` |
| `health_check` | None (public) |
To disable RBAC checking globally (for trusted-network deployments or testing), set:
```python
# superset_config.py
MCP_RBAC_ENABLED = False
```
:::warning
Disabling RBAC removes all permission checks from MCP tool calls. Only do this on isolated, internal deployments where all MCP users are trusted admins.
:::
### Audit Log
All MCP tool calls are recorded in Superset's action log. You can view them at **Settings → Action Log** (admin only). Each log entry records:
- The tool name (e.g., `mcp.generate_chart.db_write`)
- The authenticated user
- A timestamp
This makes MCP activity fully auditable alongside regular Superset activity. The action log uses the same event logger as the rest of Superset, so existing log ingestion pipelines (e.g., sending logs to Elasticsearch or a SIEM) capture MCP events automatically.
### Middleware Pipeline
Every MCP request passes through a middleware stack before reaching the tool function. The default stack (assembled in `build_middleware_list()` in `server.py`) is:
| Middleware | Purpose | Default |
|------------|---------|---------|
| `StructuredContentStripperMiddleware` | Strips `structuredContent` from responses for Claude.ai bridge compatibility | Enabled |
| `LoggingMiddleware` | Logs each tool call with user, parameters, and duration | Enabled |
| `GlobalErrorHandlerMiddleware` | Catches unhandled exceptions and sanitizes sensitive data before it reaches the client | Enabled |
| `ResponseSizeGuardMiddleware` | Estimates token count, warns at 80% of limit, blocks at limit | Enabled (configurable via `MCP_RESPONSE_SIZE_CONFIG`) |
| `ResponseCachingMiddleware` | Caches read-heavy tool responses (in-memory or Redis) | Disabled (enable via `MCP_CACHE_CONFIG`) |
Additional middleware classes (`RateLimitMiddleware`, `FieldPermissionsMiddleware`, `PrivateToolMiddleware`) are implemented in `superset/mcp_service/middleware.py` but are not added to the default pipeline. They are available for operators who want to layer them in via a custom startup path.
### Error Sanitization
The `GlobalErrorHandlerMiddleware` automatically redacts sensitive information from all error messages before they reach the LLM client. The following are replaced with generic messages:
- **Database connection strings** — replaced with a generic connection error message
- **API keys and tokens** — redacted from error traces
- **File system paths** — stripped to prevent information disclosure
- **IP addresses** — removed from error context
This ensures that a misconfigured database connection or an unexpected exception never leaks credentials or internal topology to the LLM or its users. All regex patterns used for redaction are bounded to prevent ReDoS attacks.
---
## Performance
### Connection Pooling
Each MCP server process maintains its own SQLAlchemy connection pool to the database. For multi-worker deployments, total open connections = **workers × pool size**.
```python
# superset_config.py
SQLALCHEMY_POOL_SIZE = 5
SQLALCHEMY_MAX_OVERFLOW = 10
SQLALCHEMY_POOL_TIMEOUT = 30
SQLALCHEMY_POOL_RECYCLE = 3600 # Recycle connections after 1 hour
```
For a 3-pod Kubernetes deployment with the defaults above, expect up to 3 × (5 + 10) = 45 connections. Size your database's `max_connections` accordingly.
### Response Caching
Enable response caching for read-heavy workloads (dashboards/datasets that don't change frequently). With the in-memory backend (default when `MCP_STORE_CONFIG` is disabled), caching is per-process. Use Redis-backed caching for consistent cache hits across multiple pods:
Mutating tools (`generate_chart`, `update_chart`, `execute_sql`, `generate_dashboard`) are always excluded from caching regardless of this setting.
---
## Troubleshooting
### Server won't start
@@ -664,6 +799,32 @@ MCP_CSRF_CONFIG = {
---
## Audit Events
All MCP tool calls are logged to Superset's event logger, the same system used by the web UI (viewable at **Settings → Action Log**). Each event captures:
- **User**: the resolved Superset username from the JWT or dev config
- **Timestamp**: when the operation ran
This means MCP activity is auditable alongside normal user activity. No additional configuration is required — logging is on by default whenever the event logger is enabled in your Superset deployment.
## Tool Pagination
MCP list tools (`list_datasets`, `list_charts`, `list_dashboards`, `list_databases`) use **offset pagination** via `page` (1-based) and `page_size` parameters. Responses include `page`, `page_size`, `total_count`, `total_pages`, `has_previous`, and `has_next`. To iterate through all results:
```python
# Example: fetch all charts across pages
all_charts = []
page = 1
while True:
result = mcp.list_charts(page=page, page_size=50)
all_charts.extend(result["charts"])
if not result.get("has_next"):
break
page += 1
```
## Security Best Practices
- **Use TLS** for all production MCP endpoints -- place the server behind a reverse proxy with HTTPS
@@ -64,7 +64,7 @@ There are two approaches to making dashboards publicly accessible:
3. Edit each dashboard's properties and add the "Public" role
4. Only dashboards with the Public role explicitly assigned are visible to anonymous users
See the [Public role documentation](/admin-docs/security/security#public) for more details.
See the [Public role documentation](/admin-docs/security/#public) for more details.
#### Embedding a Public Dashboard
@@ -111,7 +111,7 @@ FEATURE_FLAGS = {
This flag only hides the logout button when Superset detects it is running inside an iframe. Users accessing Superset directly (not embedded) will still see the logout button regardless of this setting.
:::note
When embedding with SSO, also set `SESSION_COOKIE_SAMESITE = 'None'` and `SESSION_COOKIE_SECURE = True`. See [Security documentation](/docs/security/securing_superset) for details.
When embedding with SSO, also set `SESSION_COOKIE_SAMESITE = 'None'` and `SESSION_COOKIE_SECURE = True`. See [Security documentation](/admin-docs/security/securing_superset) for details.
# - OS preference detection is automatically enabled
```
### App Branding
The application name shown in the browser title bar and navigation can be
set via the `brandAppName` theme token:
```python
THEME_DEFAULT = {
"token": {
"brandAppName": "Acme Analytics",
# ... other tokens
}
}
```
Or in the theme CRUD UI JSON editor:
```json
{
"token": {
"brandAppName": "Acme Analytics"
}
}
```
The existing `APP_NAME` Python config key continues to work for backward compatibility.
`brandAppName` takes precedence when both are set, and allows different themes to carry different brand names.
Email and alert/report notification subjects are driven by backend settings such as
`EMAIL_REPORTS_SUBJECT_PREFIX` and `APP_NAME`, not by this theme token.
### Migration from Configuration to UI
When `ENABLE_UI_THEME_ADMINISTRATION = True`:
@@ -93,6 +122,17 @@ When `ENABLE_UI_THEME_ADMINISTRATION = True`:
3. Administrators can change system themes without restarting Superset
4. Configuration file themes serve as fallbacks when no UI themes are set
### Theme Validation and Fallback
Superset validates theme JSON when it is saved, either through the UI or via configuration. If a theme contains invalid tokens or an unrecognized structure, Superset logs a warning and falls back to the built-in default theme rather than applying a broken configuration. This prevents a bad theme from rendering the application unusable.
The fallback order is:
1. **UI-configured system theme** (highest priority, if `ENABLE_UI_THEME_ADMINISTRATION = True`)
2. **`THEME_DEFAULT` / `THEME_DARK`** from `superset_config.py`
3. **Built-in Superset default theme** (always present as a safety net)
If you see unexpected styling after a config change, check the Superset server logs for theme validation warnings.
### Copying Themes Between Systems
To export a theme for use in configuration files or another instance:
@@ -114,7 +154,11 @@ Superset supports custom fonts through the theme configuration, allowing you to
### Default Fonts
By default, Superset uses Inter and Fira Code fonts which are bundled with the application via `@fontsource` packages. These fonts work offline and require no external network calls.
By default, Superset uses **Inter** for UI text and **IBM Plex Mono** for code (SQL editors, JSON fields, and other monospace contexts). Both fonts are bundled with the application via `@fontsource` packages and work offline without any external network calls.
:::note
IBM Plex Mono replaced Fira Code as the default code font in Superset 6.1. If you have an existing theme that explicitly sets `fontFamilyCode: "Fira Code, ..."`, you may want to update it.
:::
### Configuring Custom Fonts
@@ -312,11 +356,25 @@ Available chart types for `echartsOptionsOverridesByChartType`:
- `echarts_heatmap` - Heatmaps
- `echarts_mixed_timeseries` - Mixed time series
### Array Property Overrides
Array properties (such as color palettes) are fully supported in overrides. Arrays are **replaced entirely** rather than merged, so specify the complete array:
```python
THEME_DEFAULT = {
"token": { ... },
"echartsOptionsOverrides": {
# Replace the default color palette for all ECharts visualizations
@@ -20,7 +20,7 @@ To help make the problem somewhat tractable—given that Apache Superset has no
To strive for data consistency (regardless of the timezone of the client) the Apache Superset backend tries to ensure that any timestamp sent to the client has an explicit (or semi-explicit as in the case with [Epoch time](https://en.wikipedia.org/wiki/Unix_time) which is always in reference to UTC) timezone encoded within.
The challenge however lies with the slew of [database engines](/admin-docs/databases#installing-drivers-in-docker) which Apache Superset supports and various inconsistencies between their [Python Database API (DB-API)](https://www.python.org/dev/peps/pep-0249/) implementations combined with the fact that we use [Pandas](https://pandas.pydata.org/) to read SQL into a DataFrame prior to serializing to JSON. Regrettably Pandas ignores the DB-API [type_code](https://www.python.org/dev/peps/pep-0249/#type-objects) relying by default on the underlying Python type returned by the DB-API. Currently only a subset of the supported database engines work correctly with Pandas, i.e., ensuring timestamps without an explicit timestamp are serializd to JSON with the server timezone, thus guaranteeing the client will display timestamps in a consistent manner irrespective of the client's timezone.
The challenge however lies with the slew of [database engines](/user-docs/databases#installing-drivers-in-docker) which Apache Superset supports and various inconsistencies between their [Python Database API (DB-API)](https://www.python.org/dev/peps/pep-0249/) implementations combined with the fact that we use [Pandas](https://pandas.pydata.org/) to read SQL into a DataFrame prior to serializing to JSON. Regrettably Pandas ignores the DB-API [type_code](https://www.python.org/dev/peps/pep-0249/#type-objects) relying by default on the underlying Python type returned by the DB-API. Currently only a subset of the supported database engines work correctly with Pandas, i.e., ensuring timestamps without an explicit timestamp are serialized to JSON with the server timezone, thus guaranteeing the client will display timestamps in a consistent manner irrespective of the client's timezone.
For example the following is a comparison of MySQL and Presto,
@@ -52,6 +52,15 @@ only see the objects that they have access to.
The **sql_lab** role grants access to SQL Lab. Note that while **Admin** users have access
to all databases by default, both **Alpha** and **Gamma** users need to be given access on a per database basis.
Beyond the base `sql_lab` role, two additional SQL Lab permissions must be explicitly granted for users who need these capabilities:
| Permission | Feature |
|------------|---------|
| `can_estimate_query_cost` on `SQLLab` | Estimate query cost before running |
| `can_format_sql` on `SQLLab` | Format SQL using the database's dialect |
Grant these in **Security → List Roles** by adding the permissions to the relevant role.
### Public
The **Public** role is the most restrictive built-in role, designed specifically for anonymous/unauthenticated
@@ -182,6 +191,8 @@ However, it is crucial to understand the following:
By combining Superset's configurable safeguards with strong database-level security practices, you can achieve a more robust and layered security posture.
**Dataset Sample Access**: The `get_samples()` endpoint now enforces datasource-level access control. Users can only fetch sample rows from datasets they have been explicitly granted access to — the same permission check applied when running chart queries. This closes a prior gap where unauthenticated or under-privileged access could retrieve sample data.
### REST API for user & role management
Flask-AppBuilder supports a REST API for user CRUD,
@@ -194,6 +205,57 @@ FAB_ADD_SECURITY_API = True
Once configured, the documentation for additional "Security" endpoints will be visible in Swagger for you to explore.
### API Key Authentication
Superset supports long-lived API keys for service accounts, CI/CD pipelines, and programmatic integrations (including MCP clients).
#### Enabling API Key Authentication
API key authentication is **disabled by default**. To turn it on, set the Flask-AppBuilder config value in `superset_config.py` and also enable the matching feature flag so the management UI is exposed:
```python
FAB_API_KEY_ENABLED = True
FEATURE_FLAGS = {
"FAB_API_KEY_ENABLED": True,
}
```
The config value registers the `ApiKeyApi` blueprint on the backend; the feature flag controls whether the UI for managing keys appears for the user. See the [Feature Flags](/admin-docs/configuration/feature-flags) documentation for more on feature flag configuration.
#### Creating an API Key
Once enabled, each user manages their own keys from their profile page:
1. Open the user menu (top-right) and click **Info** to navigate to the User Info page
2. Expand the **API Keys** section
3. Click **+ API Key**
4. Enter a name and (optionally) an expiration date
5. Copy the generated token — it is shown only once
Only users with the `can_read` and `can_write` permissions on `ApiKey` (granted by default to Admins) can manage API keys.
#### Using an API Key
Pass the key as a Bearer token in the `Authorization` header:
```
Authorization: Bearer <your-api-key>
```
This works for all REST API endpoints and the MCP server. The request is executed with the permissions of the user who created the key.
#### Use Cases
- **CI/CD pipelines** — automated chart/dashboard exports and imports
- **MCP integrations** — connect AI assistants without interactive login
- **External services** — dashboards embedded in other applications
- **Service accounts** — long-lived credentials that don't expire with session cookies
:::caution
Store API keys securely. Anyone with a valid key can make requests on behalf of the creating user. Revoke keys promptly if they are compromised by deleting them from the **API Keys** section of your User Info page.
:::
### Customizing Permissions
The permissions exposed by FAB are very granular and allow for a great level of
@@ -239,26 +301,143 @@ based on the roles and permissions that were attributed.
### Row Level Security
Using Row Level Security filters (under the **Security** menu) you can create filters
that are assigned to a particular table, as well as a set of roles.
that are assigned to a particular dataset, as well as a set of roles.
If you want members of the Finance team to only have access to
rows where `department = "finance"`, you could:
- Create a Row Level Security filter with that clause (`department = "finance"`)
- Then assign the clause to the **Finance** role and the table it applies to
- Then assign the clause to the **Finance** role and the dataset it applies to
The **clause** field, which can contain arbitrary text, is then added to the generated
SQL statement’s WHERE clause. So you could even do something like create a filter
SQL statement's WHERE clause. So you could even do something like create a filter
for the last 30 days and apply it to a specific role, with a clause
like `date_field > DATE_SUB(NOW(), INTERVAL 30 DAY)`. It can also support
multiple conditions: `client_id = 6` AND `advertiser="foo"`, etc.
All relevant Row level security filters will be combined together (under the hood,
the different SQL clauses are combined using AND statements). This means it's
possible to create a situation where two roles conflict in such a way as to limit a table subset to empty.
RLS clauses also support **Jinja templating** when `ENABLE_TEMPLATE_PROCESSING` is enabled, so you can write dynamic filters such as
`user_id = '{{ current_username() }}'` to restrict rows based on the logged-in user.
For example, the filters `client_id=4` and `client_id=5`, applied to a role,
will result in users of that role having `client_id=4` AND `client_id=5`
added to their query, which can never be true.
#### Filter Types
There are two types of RLS filters:
- **Regular** — The filter clause is applied when the querying user belongs to one of the
roles assigned to the filter. Use this to restrict what specific roles can see.
- **Base** — The filter clause is applied to **all** users _except_ those in the assigned
roles. Use this to define a default restriction that privileged roles (e.g. Admin) are
exempt from. For example, a Base filter with clause `1 = 0` and the Admin role would
hide all rows from everyone except Admin — useful as a deny-by-default baseline.
#### Group Keys and Filter Combination
All applicable RLS filters are combined before being added to the query. The combination
rules are:
- Filters that share the **same group key** are combined with **OR** (any match within
the group is sufficient).
- Different filter groups (different group keys, or no group key) are combined with
**AND** (all groups must match).
- Filters with **no group key** are each treated as their own group and are always AND'd.
| `POST` | [Create a new dashboard](/developer-docs/api/create-a-new-dashboard) | `/api/v1/dashboard/` |
| `GET` | [Get metadata information about this API resource (dashboard--info)](/developer-docs/api/get-metadata-information-about-this-api-resource-dashboard-info) | `/api/v1/dashboard/_info` |
| `POST` | [Create a copy of an existing dashboard](/developer-docs/api/create-a-copy-of-an-existing-dashboard) | `/api/v1/dashboard/{id_or_slug}/copy/` |
| `DELETE` | [Delete a dashboard](/developer-docs/api/delete-a-dashboard) | `/api/v1/dashboard/{pk}` |
| `PUT` | [Update a dashboard](/developer-docs/api/update-a-dashboard) | `/api/v1/dashboard/{pk}` |
| `POST` | [Compute and cache a screenshot (dashboard-pk-cache-dashboard-screenshot)](/developer-docs/api/compute-and-cache-a-screenshot-dashboard-pk-cache-dashboard-screenshot) | `/api/v1/dashboard/{pk}/cache_dashboard_screenshot/` |
| `PUT` | [Update chart customizations configuration for a dashboard.](/developer-docs/api/update-chart-customizations-configuration-for-a-dashboard) | `/api/v1/dashboard/{pk}/chart_customizations` |
| `PUT` | [Update colors configuration for a dashboard.](/developer-docs/api/update-colors-configuration-for-a-dashboard) | `/api/v1/dashboard/{pk}/colors` |
| `GET` | [Export dashboard as example bundle](/developer-docs/api/export-dashboard-as-example-bundle) | `/api/v1/dashboard/{pk}/export_as_example/` |
| `DELETE` | [Remove the dashboard from the user favorite list](/developer-docs/api/remove-the-dashboard-from-the-user-favorite-list) | `/api/v1/dashboard/{pk}/favorites/` |
| `POST` | [Mark the dashboard as favorite for the current user](/developer-docs/api/mark-the-dashboard-as-favorite-for-the-current-user) | `/api/v1/dashboard/{pk}/favorites/` |
| `PUT` | [Update native filters configuration for a dashboard.](/developer-docs/api/update-native-filters-configuration-for-a-dashboard) | `/api/v1/dashboard/{pk}/filters` |
| `GET` | [Get a computed screenshot from cache (dashboard-pk-screenshot-digest)](/developer-docs/api/get-a-computed-screenshot-from-cache-dashboard-pk-screenshot-digest) | `/api/v1/dashboard/{pk}/screenshot/{digest}/` |
| `GET` | [Get a list of charts](/developer-docs/api/get-a-list-of-charts) | `/api/v1/chart/` |
| `POST` | [Create a new chart](/developer-docs/api/create-a-new-chart) | `/api/v1/chart/` |
| `GET` | [Get metadata information about this API resource (chart--info)](/developer-docs/api/get-metadata-information-about-this-api-resource-chart-info) | `/api/v1/chart/_info` |
| `GET` | [Get a list of datasets](/developer-docs/api/get-a-list-of-datasets) | `/api/v1/dataset/` |
| `POST` | [Create a new dataset](/developer-docs/api/create-a-new-dataset) | `/api/v1/dataset/` |
| `GET` | [Get metadata information about this API resource (dataset--info)](/developer-docs/api/get-metadata-information-about-this-api-resource-dataset-info) | `/api/v1/dataset/_info` |
| `GET` | [Get a dataset](/developer-docs/api/get-a-dataset) | `/api/v1/dataset/{id_or_uuid}` |
| `GET` | [Get charts and dashboards count associated to a dataset](/developer-docs/api/get-charts-and-dashboards-count-associated-to-a-dataset) | `/api/v1/dataset/{id_or_uuid}/related_objects` |
| `DELETE` | [Delete a dataset](/developer-docs/api/delete-a-dataset) | `/api/v1/dataset/{pk}` |
| `GET` | [Get a dataset](/developer-docs/api/get-a-dataset) | `/api/v1/dataset/{pk}` |
| `PUT` | [Update a dataset](/developer-docs/api/update-a-dataset) | `/api/v1/dataset/{pk}` |
| `DELETE` | [Delete a dataset column](/developer-docs/api/delete-a-dataset-column) | `/api/v1/dataset/{pk}/column/{column_id}` |
| `DELETE` | [Delete a dataset metric](/developer-docs/api/delete-a-dataset-metric) | `/api/v1/dataset/{pk}/metric/{metric_id}` |
| `PUT` | [Refresh and update columns of a dataset](/developer-docs/api/refresh-and-update-columns-of-a-dataset) | `/api/v1/dataset/{pk}/refresh` |
| `GET` | [Get charts and dashboards count associated to a dataset](/developer-docs/api/get-charts-and-dashboards-count-associated-to-a-dataset) | `/api/v1/dataset/{pk}/related_objects` |
| `GET` | [Get distinct values from field data (dataset-distinct-column-name)](/developer-docs/api/get-distinct-values-from-field-data-dataset-distinct-column-name) | `/api/v1/dataset/distinct/{column_name}` |
| `POST` | [Duplicate a dataset](/developer-docs/api/duplicate-a-dataset) | `/api/v1/dataset/duplicate` |
| `GET` | [Get all schemas from a database](/developer-docs/api/get-all-schemas-from-a-database) | `/api/v1/database/{pk}/schemas/` |
| `GET` | [Get database select star for table (database-pk-select-star-table-name)](/developer-docs/api/get-database-select-star-for-table-database-pk-select-star-table-name) | `/api/v1/database/{pk}/select_star/{table_name}/` |
| `GET` | [Get database select star for table (database-pk-select-star-table-name-schema-name)](/developer-docs/api/get-database-select-star-for-table-database-pk-select-star-table-name-schema-name) | `/api/v1/database/{pk}/select_star/{table_name}/{schema_name}/` |
| `DELETE` | [Delete a SSH tunnel](/developer-docs/api/delete-a-ssh-tunnel) | `/api/v1/database/{pk}/ssh_tunnel/` |
| `POST` | [Re-sync all permissions for a database connection](/developer-docs/api/re-sync-all-permissions-for-a-database-connection) | `/api/v1/database/{pk}/sync_permissions/` |
| `GET` | [Get names of databases currently available](/developer-docs/api/get-names-of-databases-currently-available) | `/api/v1/database/available/` |
| `GET` | [Download database(s) and associated dataset(s) as a zip file](/developer-docs/api/download-database-s-and-associated-dataset-s-as-a-zip-file) | `/api/v1/database/export/` |
<summary><strong>Datasources</strong> (1 endpoints) — Query datasource metadata and column values.</summary>
<summary><strong>Datasources</strong> (2 endpoints) — Query datasource metadata and column values.</summary>
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | [Get possible values for a datasource column](/developer-docs/api/get-possible-values-for-a-datasource-column) | `/api/v1/datasource/{datasource_type}/{datasource_id}/column/{column_name}/values/` |
| `POST` | [Validate a SQL expression against a datasource](/developer-docs/api/validate-a-sql-expression-against-a-datasource) | `/api/v1/datasource/{datasource_type}/{datasource_id}/validate_expression/` |
</details>
<details>
<summary><strong>Advanced Data Type</strong> (2 endpoints) — Endpoints for advanced data type operations and conversions.</summary>
<summary><strong>Advanced Data Type</strong> (2 endpoints) — Advanced data type operations and conversions.</summary>
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | [Return an AdvancedDataTypeResponse](/developer-docs/api/return-an-advanceddatatyperesponse) | `/api/v1/advanced_data_type/convert` |
| `GET` | [Return an AdvancedDataTypeResponse](/developer-docs/api/return-an-advanced-data-type-response) | `/api/v1/advanced_data_type/convert` |
| `GET` | [Return a list of available advanced data types](/developer-docs/api/return-a-list-of-available-advanced-data-types) | `/api/v1/advanced_data_type/types` |
| `POST` | [Create a new permanent link (explore-permalink)](/developer-docs/api/create-a-new-permanent-link-explore-permalink) | `/api/v1/explore/permalink` |
| `POST` | [Create a new permanent link (sqllab-permalink)](/developer-docs/api/create-a-new-permanent-link-sqllab-permalink) | `/api/v1/sqllab/permalink` |
| `GET` | [Get permanent link state for SQLLab editor.](/developer-docs/api/get-permanent-link-state-for-sqllab-editor) | `/api/v1/sqllab/permalink/{key}` |
| `GET` | [Get permanent link state for SQLLab editor.](/developer-docs/api/get-permanent-link-state-for-sql-lab-editor) | `/api/v1/sqllab/permalink/{key}` |
| `GET` | [Get a list of themes](/developer-docs/api/get-a-list-of-themes) | `/api/v1/theme/` |
| `POST` | [Create a theme](/developer-docs/api/create-a-theme) | `/api/v1/theme/` |
| `GET` | [Get metadata information about this API resource (theme--info)](/developer-docs/api/get-metadata-information-about-this-api-resource-theme-info) | `/api/v1/theme/_info` |
| `DELETE` | [Delete a theme](/developer-docs/api/delete-a-theme) | `/api/v1/theme/{pk}` |
| `GET` | [Get a theme](/developer-docs/api/get-a-theme) | `/api/v1/theme/{pk}` |
| `PUT` | [Update a theme](/developer-docs/api/update-a-theme) | `/api/v1/theme/{pk}` |
| `PUT` | [Set a theme as the system dark theme](/developer-docs/api/set-a-theme-as-the-system-dark-theme) | `/api/v1/theme/{pk}/set_system_dark` |
| `PUT` | [Set a theme as the system default theme](/developer-docs/api/set-a-theme-as-the-system-default-theme) | `/api/v1/theme/{pk}/set_system_default` |
| `DELETE` | [Delete security user registrations by pk](/developer-docs/api/delete-security-user-registrations-by-pk) | `/api/v1/security/user_registrations/{pk}` |
| `GET` | [Get security user registrations by pk](/developer-docs/api/get-security-user-registrations-by-pk) | `/api/v1/security/user_registrations/{pk}` |
| `PUT` | [Update security user registrations by pk](/developer-docs/api/update-security-user-registrations-by-pk) | `/api/v1/security/user_registrations/{pk}` |
| `GET` | [Get distinct values from field data (security-user-registrations-distinct-column-name)](/developer-docs/api/get-distinct-values-from-field-data-security-user-registrations-distinct-column-name) | `/api/v1/security/user_registrations/distinct/{column_name}` |
| `GET` | [Get related fields data (security-user-registrations-related-column-name)](/developer-docs/api/get-related-fields-data-security-user-registrations-related-column-name) | `/api/v1/security/user_registrations/related/{column_name}` |
import { Tabs } from '@superset-ui/core/components';
```
---
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
