mirror of
https://github.com/apache/superset.git
synced 2026-05-24 01:05:21 +00:00
9fd7fd441a
Three coordinated fixes for the 25 CI failures on the initial PR push:
1. **Lockfile sync.** Added @superset-ui/plugin-chart-country-map as
a workspace dep in the previous commit but didn't update
package-lock.json. CI's `npm ci` failed across frontend-build,
cypress (12 jobs), playwright (4 jobs), docker (2 jobs), and
frontend-check-translations. Re-ran `npm install --package-lock-only`
to add the new workspace's 71 lock entries.
2. **License headers added** to 13 new files flagged by License Check:
- 5 markdown READMEs / SIP_DRAFT (HTML-comment headers)
- 5 YAML config files (`# Licensed ...`)
- 2 Python files (`# Licensed ...`)
- 1 shell script (preserves shebang)
3. **Reproducible build outputs.** The regen workflow detected drift
on manifest.json + ukr_admin1_CAN.geo.json. Two root causes:
- `build_timestamp_utc` field made manifest non-deterministic →
dropped from the schema
- Floating mapshaper version (`npx --yes mapshaper`) caused subtle
simplification differences across runners → pinned to
`mapshaper@0.7.15` via `npx --yes mapshaper@<version>`
Verified locally: rebuild from clean cache reproduces every output
byte-identically except the manifest (which now also matches once
the timestamp is gone).
Files changed:
.gitignore — re-include rule for static dir
superset-frontend/package-lock.json — +71 lines for new workspace
13 new files — ASF headers
build.py — pin mapshaper, drop timestamp
manifest.json (× 2) — regenerate w/o timestamp
README.md (in static dir) — header
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
63 lines
3.4 KiB
Markdown
63 lines
3.4 KiB
Markdown
<!--
|
|
Licensed to the Apache Software Foundation (ASF) under one
|
|
or more contributor license agreements. See the NOTICE file
|
|
distributed with this work for additional information
|
|
regarding copyright ownership. The ASF licenses this file
|
|
to you under the Apache License, Version 2.0 (the
|
|
"License"); you may not use this file except in compliance
|
|
with the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing,
|
|
software distributed under the License is distributed on an
|
|
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations
|
|
under the License.
|
|
-->
|
|
|
|
# Country Map data pipeline
|
|
|
|
This directory contains the build pipeline that turns upstream Natural Earth data into the GeoJSON files consumed by `@superset-ui/plugin-chart-country-map`.
|
|
|
|
It replaces the legacy `scripts/Country Map GeoJSON Generator.ipynb` notebook. See `SIP_DRAFT.md` in the parent directory for the full design rationale.
|
|
|
|
## Layout
|
|
|
|
```
|
|
scripts/
|
|
build.sh # one-shot reproducible build
|
|
README.md # this file
|
|
config/ # declarative YAML — handles ~95% of fixes
|
|
name_overrides.yaml # typos, deprecated ISO codes, admin renames
|
|
flying_islands.yaml # repositioning + bbox drops for far-flung territories
|
|
territory_assignments.yaml # add features from sibling Admin 0 records
|
|
regional_aggregations.yaml # dissolve Admin 1 into administrative regions
|
|
composite_maps.yaml # multi-country composites (e.g. France-with-Overseas)
|
|
procedural/ # escape hatch — handles the rare 5%
|
|
README.md # when to use, when not
|
|
NN_<descriptive_name>.py # one focused script per genuine edge case
|
|
output/ # gitignored — build artifacts
|
|
```
|
|
|
|
## Operating principles
|
|
|
|
- **Default tool: declarative YAML.** Most touchups are renames, repositions, dissolves, or filters — all expressible in YAML. Diffs are small, conflicts localize cleanly to one entry, contributors can submit "fix typo X" as a one-line PR.
|
|
- **Escape hatch: `procedural/` directory** of small, named, single-purpose Python scripts for the rare cases YAML can't express cleanly. Each script has a header comment explaining *why* it's not in YAML. See `procedural/README.md` for the bar.
|
|
- **Build is reproducible from a pinned NE version.** `build.sh` records the NE git SHA it consumed; outputs are deterministic given inputs.
|
|
- **CI regenerates on schema change** and opens a PR if outputs differ. Maintainers review the cartographic diff in legible GeoJSON, not opaque notebook JSON.
|
|
|
|
## Workflow for adding a fix
|
|
|
|
1. Identify the upstream NE issue (wrong name, missing territory, etc.).
|
|
2. **Try YAML first.** Add the smallest possible entry to the appropriate config file with a `description` field explaining the fix.
|
|
3. If YAML can't express it cleanly, add a numbered script in `procedural/` with a header comment explaining why YAML didn't fit.
|
|
4. Run `build.sh` locally, verify the output GeoJSON looks right.
|
|
5. Open PR. Reviewer sees the YAML diff (or new procedural script) plus the regenerated GeoJSON.
|
|
|
|
## See also
|
|
|
|
- `SIP_DRAFT.md` (parent dir) — design rationale, notebook audit, obsolescence check
|
|
- `procedural/README.md` — when to use the escape hatch
|