Compare commits

..

15 Commits

Author SHA1 Message Date
Đỗ Trọng Hải
4556840410 Merge branch 'master' into feat/translation-backfill-tooling 2026-05-19 11:32:19 +07:00
Claude Code
de4c929e44 test(translations): add unit tests for build_translation_index
Adds 17 tests under tests/unit_tests/scripts/translations/ covering the
two pure helpers (_is_translated, _plural_key) and the build_index
workflow against a tmp_path fixture with three fake languages.

Locks in the cross-language index shape that the AI backfill prompt
depends on:
- fuzzy entries surface as null (so unreviewed drafts don't feed back
  into the prompt as trusted context, and don't inflate --min-context)
- every msgid has a slot for every non-en language (null when missing)
- plural entries are keyed by the msgid\x00msgid_plural composite
- the en source locale and directories without LC_MESSAGES/messages.po
  are excluded
- the .po header entry (empty msgid) is skipped

Addresses the reviewer question on whether the new tooling script
should have test coverage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 13:54:05 -07:00
Claude Code
7e2064037e chore(deps): add polib to development requirements
The translation backfill script and its unit tests rely on polib, but
the package wasn't declared in any requirements file. CI unit-tests
fails at collection time with ModuleNotFoundError when pytest tries to
import the test module.

Add polib to the development extra in pyproject.toml and pin polib==1.2.0
in requirements/development.txt so CI installs it. A full
./scripts/uv-pip-compile.sh regen would also touch many unrelated
pins; this targeted addition matches the pyproject.toml declaration
without that churn.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 14:33:32 -07:00
Evan Rusackas
6d2bf6b10c Merge branch 'master' into feat/translation-backfill-tooling 2026-05-11 19:55:42 -07:00
Superset Dev
fe3fa946c4 fix(i18n): handle JSON-list plural responses from the model
A fresh test run on French exposed a real bug in _apply_translation:
when the model returns a JSON list for a plural entry (e.g.
["form0", "form1"], which is a valid representation since plural forms
are ordered), the previous code took the else branch and broadcast
str(list) — Python list-repr like ['form0', 'form1'] — to every plural
form. Both msgstr[0] and msgstr[1] ended up containing the same
literal Python list-repr string, breaking gettext lookups for that
entry. Spanish dodged it by chance (the model returned dicts that
time); the failure mode is reproducible on French.

Changes:
- Extract _apply_plural_translation helper. Handles dict, list,
  scalar, and non-JSON-string responses. List path distributes forms
  by index and repeats the last form if the model returned fewer
  forms than the language requires (better than leaving slots blank,
  which falls back to displaying the raw English msgid).
- The split also drops _apply_translation's cyclomatic complexity
  back below the C901 threshold.
- Adds 4 regression tests covering: list response, list response
  round-tripped through parse_response, list shorter than required
  forms (last-form-repeats), and empty list (falls back to raw-string
  broadcast).

Verified end-to-end on French: the previously-broken plural entry
"Added 1 new column to the virtual dataset" / "Added %s new columns
to the virtual dataset" now writes msgstr[0] and msgstr[1] correctly
on a fresh run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 17:47:24 -07:00
Superset Dev
795d6e67df test(i18n): cover backfill_po + address review feedback
Addresses sadpandajoe's review on #39448:

1. Adds tests/unit_tests/scripts/translations/backfill_po_test.py with
   19 cases covering parse_response (singular/plural/markdown-fence
   stripping/non-ASCII/non-numeric keys/list-and-scalar rejection/JSON
   errors) and _apply_translation (singular path, plural-dict path,
   plural-scalar fallback, plural invalid-JSON fallback, fuzzy flag,
   attribution append/dedup, end-to-end round-trip from parse_response
   into _apply_translation). The script is loaded via importlib since
   it lives outside the package tree.

2. translate_batch now pipes the prompt over stdin instead of passing
   it as argv. With --batch-size 50 and many reference languages a
   single batch can grow into the tens of KB and approach ARG_MAX on
   some platforms; stdin removes that ceiling.

3. _process_batches now saves the catalog after each batch that wrote
   at least one translation (when not in --dry-run). For sparse
   languages with thousands of missing strings, a crash mid-run now
   only loses the in-flight batch rather than every batch translated
   so far. The full save at end of backfill() is removed since the
   per-batch save covers it.

4. Module docstring referenced --fuzzy/--no-fuzzy but argparse only
   registers --no-fuzzy; doc updated to match the actual flag.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 16:59:40 -07:00
Claude Code
73a042ed5c fix(i18n): validate parse_response JSON is a dict before .items()
A non-object JSON response (list, scalar, null) would raise AttributeError
from .items(). _process_batches only catches (ValueError, RuntimeError),
so the crash would abort the entire run instead of being handled per-batch.
Surface the type error as ValueError so it's caught gracefully.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-20 08:52:54 -07:00
Evan Rusackas
8e899d1497 Merge branch 'master' into feat/translation-backfill-tooling 2026-04-19 15:29:55 -04:00
Claude Code
1ac81f7a31 docs(i18n): add docstrings to backfill() and main()
Addresses two codeant-ai review comments about missing docstrings on
newly added top-level functions.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 16:14:04 -07:00
Claude Code
37af05c292 fix(i18n): restore clobbered code, fix plural coercion & fuzzy-as-context
Four "Apply suggestion" commits from codeant-ai replaced real code bodies
with docstring-only lines, breaking syntax (syntax errors at
build_translation_index.py:119). Restore the bodies while keeping the
suggested docstrings:

- build_translation_index.py: _plural_key, main()
- backfill_po.py: _lang_name, _plural_key

Also addresses two major issues raised in review:

1. parse_response() in backfill_po.py used str(v) on values, which
   converted dict responses (from plural entries) into Python repr
   like "{'0': 'x'}" that json.loads could not later parse in
   _apply_translation. Serialize dict/list values with json.dumps.

2. build_index() wrote fuzzy entries as trusted context in the
   cross-language index, letting AI-generated drafts propagate back
   into future backfill runs as if reviewed. Gate index values via
   _is_translated so fuzzy entries become null.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 16:12:08 -07:00
Evan Rusackas
88c7389ee5 Apply suggestion from @codeant-ai-for-open-source[bot]
Co-authored-by: codeant-ai-for-open-source[bot] <244253245+codeant-ai-for-open-source[bot]@users.noreply.github.com>
2026-04-17 18:18:10 -07:00
Evan Rusackas
ee6779c84a Apply suggestion from @codeant-ai-for-open-source[bot]
Co-authored-by: codeant-ai-for-open-source[bot] <244253245+codeant-ai-for-open-source[bot]@users.noreply.github.com>
2026-04-17 18:17:53 -07:00
Evan Rusackas
7bc0e385b8 Apply suggestion from @codeant-ai-for-open-source[bot]
Co-authored-by: codeant-ai-for-open-source[bot] <244253245+codeant-ai-for-open-source[bot]@users.noreply.github.com>
2026-04-17 18:17:42 -07:00
Evan Rusackas
f684eccd94 Apply suggestion from @codeant-ai-for-open-source[bot]
Co-authored-by: codeant-ai-for-open-source[bot] <244253245+codeant-ai-for-open-source[bot]@users.noreply.github.com>
2026-04-17 18:17:12 -07:00
Claude Code
0a91bce9ea feat(i18n): add AI-assisted translation backfill tooling + Spanish translations
Adds two scripts to help maintainers fill in missing .po translations
using Claude AI, and applies them to backfill all 184 missing Spanish strings.

New scripts:
- scripts/translations/build_translation_index.py — reads every .po file
  and outputs a cross-language JSON index {msgid: {lang: translation}}
  used to provide reference context to the AI
- scripts/translations/backfill_po.py — for a target language, finds all
  untranslated entries, batches them, and calls claude -p with cross-language
  context to generate draft translations marked #, fuzzy for human review

Design highlights:
- Cross-language translations are passed per-string so the AI can disambiguate
  ambiguous English (e.g. "Scale", "Table") from how other translators handled it
- --min-context N skips strings with fewer than N reference translations
- Each generated entry is tagged with a translator comment listing the model
  and which languages provided context (e.g. [refs: fr, ru])
- translation_index.json added to .gitignore (regenerated locally)

Spanish translations:
- Backfilled all 184 previously untranslated strings in es/LC_MESSAGES/messages.po
- All entries marked #, fuzzy pending human review

Docs: added "Backfilling missing translations with AI" section to
docs/developer_docs/contributing/howtos.md

npm shortcuts added to superset-frontend/package.json:
- translations:build-index
- translations:backfill

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 10:04:22 -07:00
36 changed files with 10929 additions and 4122 deletions

2
.gitignore vendored
View File

@@ -115,6 +115,8 @@ release.json
superset/translations/**/messages.json
# these mo binary files are generated by `pybabel compile`
superset/translations/**/messages.mo
# cross-language index generated by scripts/translations/build_translation_index.py
superset/translations/translation_index.json
docker/requirements-local.txt

View File

@@ -1,168 +0,0 @@
---
title: Dashboard Performance
hide_title: true
sidebar_position: 5
version: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Dashboard Performance
A dashboard's perceived speed is determined by three independent things: how
many charts have to render, how many queries the backend can execute
concurrently, and how quickly the underlying data warehouse can return
results. Superset gives you levers for the first two; the third belongs to
your warehouse. This page covers the dashboard-side levers and the practical
guidance around them.
## Is there a maximum chart count per dashboard?
**No hard limit is enforced** — Superset has no configuration key that
caps the number of charts on a dashboard. In practice, dashboards behave
well up to a few dozen charts. Beyond that, you'll typically feel friction
on the initial load and during cross-filter / time-range updates, even with
the lazy-loading optimizations described below.
Rough thresholds to keep in mind:
- **Under ~25 charts**: usually no perceptible problem.
- **2550 charts**: still fine, but you start to want tabs to break the
page into chunks the user actually looks at.
- **Over ~50 charts**: split into multiple dashboards or use tabs
aggressively. The bottleneck is rarely Superset itself — it's the
warehouse executing dozens of queries in parallel and the browser
rendering dozens of chart frames.
These are guidelines, not guarantees. A dashboard of 100 sparkline-style
charts hitting a fast cache behaves very differently from a dashboard of
20 heavy aggregations against a cold warehouse.
## Lazy rendering — `DASHBOARD_VIRTUALIZATION`
Superset's dashboard layout is virtualized at the row level. Charts that
are far below the user's current scroll position are not rendered (and
therefore don't fetch data) until the user scrolls them into view, and they
are unmounted again if scrolled well past. This is on by default.
**Feature flag**: `DASHBOARD_VIRTUALIZATION` (default: `True`)
The flag is `stable` and marked for path-to-deprecation — meaning the
behavior will eventually be non-optional, but the flag still exists so
operators can disable it if a specific layout misbehaves.
**Behavior** (from `superset-frontend/src/dashboard/components/gridComponents/Row/Row.tsx`):
- A chart is rendered when its row scrolls within **1 viewport height** of
the visible area.
- A chart is unmounted when its row scrolls more than **4 viewport
heights** away from the visible area.
- Tabs that aren't currently selected don't render their content at all
(see below).
- The unmounting half is skipped in **embedded** mode (so an embedded
dashboard keeps its charts mounted once they've been seen, which avoids
re-fetching on scroll-up). Both halves are skipped for **headless /
bot** rendering (so screenshot / report jobs load every chart).
## Deferred data fetch — `DASHBOARD_VIRTUALIZATION_DEFER_DATA`
By default, `DASHBOARD_VIRTUALIZATION` controls *rendering* — but charts
that don't render also don't fetch data, because Superset's chart
components issue their data request on mount. `DASHBOARD_VIRTUALIZATION_DEFER_DATA`
is a supplementary flag that further defers the data request itself, useful
for backends where opening a connection or compiling a query is expensive
even if the result is later thrown away.
**Feature flag**: `DASHBOARD_VIRTUALIZATION_DEFER_DATA` (default: `False`)
Enable this if you see warehouse load spike on dashboard *open* even
though most charts are off-screen.
## Per-tab lazy loading
**This is on by default and has no flag.** A tab's content is not rendered
until the user activates that tab, so charts inside an unselected tab do
not fetch data on dashboard open. When the user clicks the tab, that
tab's charts mount and fetch in the normal way.
Practically: tabs are the single most effective tool for a large
dashboard. Splitting 60 charts across 4 tabs effectively turns dashboard
open into "load ~15 charts," and the remaining ones lazy-load only if the
user goes looking.
## Is there a switch to cap concurrent chart queries?
**No.** Superset does not implement a frontend-side concurrent-request
limiter. Each chart issues its own data request when it mounts, and the
browser handles parallelism (typically ~6 in-flight HTTP requests per
origin, then the rest queue). Backend throughput is bounded by your
Gunicorn worker count for synchronous query execution, or by your Celery
worker pool when [async queries](./async-queries-celery.mdx) are enabled.
If you need to throttle warehouse load, the right place is:
1. The warehouse itself (connection pool / concurrency limits).
2. Superset's Celery configuration (smaller worker pool when async
queries are on).
3. Splitting heavy charts across tabs or separate dashboards (each
dashboard load only fetches what's visible).
## Splitting strategies
When a dashboard outgrows comfortable performance, the options in order
of effort:
**1. Move sections into tabs.** Same dashboard, but only the active tab's
charts fetch. This is the cheapest change and often the only one needed.
**2. Cache aggressively.** A Redis cache backend (see
[Caching](./cache.mdx)) means repeat dashboard loads serve from cache
rather than re-hitting the warehouse. This is especially impactful for
dashboards opened by many users in close succession.
**3. Enable async queries.** [Async query execution](./async-queries-celery.mdx)
via Celery decouples query duration from request lifetime, so a slow
chart doesn't block the page. The user sees other charts come in as
their queries complete.
**4. Split into multiple dashboards.** Group related charts into purpose-
specific dashboards rather than one mega-dashboard. Link them from a
landing dashboard or a navigation menu.
**5. Pre-aggregate at the warehouse level.** If the same expensive
aggregation appears across many charts, materialize it as a view or
scheduled table in the warehouse so each chart query is a cheap lookup.
## Operational notes
- The feature flags above are set in `superset_config.py`, e.g.:
```python
FEATURE_FLAGS = {
"DASHBOARD_VIRTUALIZATION": True,
"DASHBOARD_VIRTUALIZATION_DEFER_DATA": True,
}
```
- See [Feature Flags](./feature-flags.mdx) for the full list of supported
flags and their lifecycle stages.
- Report and screenshot jobs (alerts, scheduled reports, dashboard
exports) intentionally bypass row virtualization so the rendered
artifact includes every chart, not just the ones above the fold.

View File

@@ -335,6 +335,92 @@ npm run build-translation
pybabel compile -d superset/translations
```
### Backfilling missing translations with AI
For languages with many untranslated strings, the repo includes a script that
uses Claude AI to generate draft translations for any missing entries. All
AI-generated strings are marked `#, fuzzy` and tagged with an attribution
comment so that human reviewers know they need to be checked before merging.
#### Prerequisites
```bash
pip install -r superset/translations/requirements.txt
```
Claude Code must be installed and authenticated (`claude --version` should
work). The script calls `claude -p` internally — no separate API key is needed.
#### Step 1 — Build the translation index
The index captures every already-translated string in every language and
serves as cross-language context for the AI. Rebuild it whenever `.po` files
change significantly:
```bash
python scripts/translations/build_translation_index.py
# Writes: superset/translations/translation_index.json
```
#### Step 2 — Preview with a dry run
Check what would be translated without writing anything:
```bash
python scripts/translations/backfill_po.py --lang fr --limit 20 --dry-run
```
Output shows each string, its translation, and a context tag:
- No tag — 3+ reference languages available (high confidence)
- `[ctx:N]` — only N other languages have this string (lower confidence)
- `[ctx:0]` — no other language has this string yet; English alone used
#### Step 3 — Run the backfill
```bash
python scripts/translations/backfill_po.py --lang fr
```
Options:
| Flag | Default | Description |
|------|---------|-------------|
| `--lang LANG` | required | ISO language code (`fr`, `de`, `ja`, …) |
| `--batch-size N` | 50 | Strings per Claude request |
| `--limit N` | unlimited | Stop after N entries |
| `--min-context N` | 0 | Skip entries with fewer than N reference translations |
| `--model MODEL` | `claude-sonnet-4-6` | Claude model to use |
| `--dry-run` | off | Print without writing |
| `--no-fuzzy` | off | Don't mark entries as fuzzy |
Use `--min-context 2` to skip strings that have fewer than 2 reference
translations in other languages. Those strings are more likely to be ambiguous
(short labels, UI fragments) where the correct meaning can't be inferred
without additional context.
#### Step 4 — Review and commit
Open the target `.po` file and search for `fuzzy`. For each generated entry:
1. Verify the translation is correct for the UI context.
2. Remove the `# Machine-translated via backfill_po.py` comment and the
`#, fuzzy` flag line once you are satisfied.
3. If the translation is wrong, correct the `msgstr` before removing the flag.
4. Commit the `.po` file — do **not** commit `translation_index.json` (it is
gitignored and regenerated locally).
#### Running via npm
From `superset-frontend/`:
```bash
# Rebuild index
npm run translations:build-index
# Backfill (pass arguments after --)
npm run translations:backfill -- --lang fr --dry-run
```
## Linting
### Python

View File

@@ -89,7 +89,7 @@
"remark-import-partial": "^0.0.2",
"reselect": "^5.2.0",
"storybook": "^8.6.18",
"swagger-ui-react": "^5.32.6",
"swagger-ui-react": "^5.32.5",
"swc-loader": "^0.2.7",
"tinycolor2": "^1.4.2",
"unist-util-visit": "^5.1.0"
@@ -127,8 +127,7 @@
"resolutions": {
"react-redux": "^9.2.0",
"@reduxjs/toolkit": "^2.5.0",
"baseline-browser-mapping": "^2.9.19",
"swagger-client": "3.37.3"
"baseline-browser-mapping": "^2.9.19"
},
"packageManager": "yarn@1.22.22+sha1.ac34549e6aa8e7ead463a7407e1c7390f61a6610"
}

View File

@@ -3898,6 +3898,59 @@
"@swagger-api/apidom-error" "^1.11.0"
"@swaggerexpert/json-pointer" "^2.10.1"
"@swagger-api/apidom-ns-api-design-systems@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-api-design-systems/-/apidom-ns-api-design-systems-1.11.0.tgz#c9bee1def674b6b12559a97da243c9358e4f5d13"
integrity sha512-IskDsUkUtNas4guoChRKKkw0wOst64nRA24WuIjLf8ztfBdcl/oqx/cgy8pwWCUqNYvL9L3+sD5HeuokqMrySw==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-error" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-1" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-ns-arazzo-1@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-arazzo-1/-/apidom-ns-arazzo-1-1.11.0.tgz#ceda21f89fe970d8c6e2504e0ea644f864eabae1"
integrity sha512-n+aGSlLHyrpmCaBa9DBZkIqnNVzYAYSa010MvAwhlwtW3EbFYNwYWinbTwLqCd3leN6XWTvQYCvk0/k7/9Cq4A==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-json-schema-2020-12" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-ns-asyncapi-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-asyncapi-2/-/apidom-ns-asyncapi-2-1.11.0.tgz#9e7dcb40c6de940c6b4ad23c8413864aa3286127"
integrity sha512-SHh3naFZlXFI0gG36tNYvJ/VO8aZsjnXIQAqJHfOE6rrpl5msJrdDatmNczh+57WPZxEZA+KTXWCqNKdeu3G3Q==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-json-schema-draft-7" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-ns-asyncapi-3@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-asyncapi-3/-/apidom-ns-asyncapi-3-1.11.0.tgz#92a3f5b78f795f651114a5d7e572dadd051e62b0"
integrity sha512-4vrgNYDj68hgmgZj1eGBaBr5xqIETWn4jAioiRHek4jV1FLvmxCs3nC2nYs8CzQqqJ1bqirdiirrUpqhaQvTEA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-ns-json-schema-2019-09@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-json-schema-2019-09/-/apidom-ns-json-schema-2019-09-1.11.0.tgz#18531aa5a09192d2f296474f458b493e64f40d5c"
@@ -3967,6 +4020,20 @@
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.4"
"@swagger-api/apidom-ns-openapi-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-openapi-2/-/apidom-ns-openapi-2-1.11.0.tgz#89b97db3173589cab4b9c57650a6d75638f870bb"
integrity sha512-cAIPJhLxm/nj1kzneNySeaTahY+hH5gkGNsgbmifGnLPsC5YOOfEVMKLj18IREdXqdnxJgRbsI9Azl4g09TPkg==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-error" "^1.11.0"
"@swagger-api/apidom-ns-json-schema-draft-4" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-ns-openapi-3-0@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-ns-openapi-3-0/-/apidom-ns-openapi-3-0-1.11.0.tgz#4b44fb8a292439f4966bedef7d0c484926ece0e5"
@@ -4014,10 +4081,285 @@
ramda-adjunct "^5.0.0"
ts-mixer "^6.0.3"
"@swagger-api/apidom-parser-adapter-api-design-systems-json@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-api-design-systems-json/-/apidom-parser-adapter-api-design-systems-json-1.11.0.tgz#b7c757fadafe6bbcfb12f897b9f7432795de9460"
integrity sha512-0OdwcnV/QF+Vs3Vj0dTmlRHEp9WQg9aBvWWl8Fq25OviyDhGGRpqgkEAOjtVYCH3XyZ1Xz+jhIDOdd5pxBajsA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-api-design-systems" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-api-design-systems-yaml@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-api-design-systems-yaml/-/apidom-parser-adapter-api-design-systems-yaml-1.11.0.tgz#7dd524f0241fd9997e8959a071a3e8b2cf58f1f6"
integrity sha512-K714DT6nFW+ZM9LTo+c120zkUjsEcIFO2DU+0cnzReRyenb1x6RZe+uOqTt7iWohnnWp2FV/j0exd/mCsxW65Q==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-api-design-systems" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-arazzo-json-1@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-arazzo-json-1/-/apidom-parser-adapter-arazzo-json-1-1.11.0.tgz#8b8bafdab34b4a917a2cb68aeb0e9e94ad511fc2"
integrity sha512-z9K6XEr3AafV2EA+1pfW+8VoMCCSSpm2IU7oUTjSnhxRb5t/DZR4Qg8FEK8tRKdS2BO2kFFLb2xikrY3Qx8B+g==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-arazzo-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-arazzo-yaml-1@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-arazzo-yaml-1/-/apidom-parser-adapter-arazzo-yaml-1-1.11.0.tgz#2a21d25375dae5bfd76ef331a581ad84fd00ccdd"
integrity sha512-HPb7Wzr+cj0IJkRRlqsK1tNCQXivuGRP4iB2yek16sQZXo2eqSUZ3j3Lz/WwWgnN/FWGAODm4bj9+EhGQ11TnA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-arazzo-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-asyncapi-json-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-asyncapi-json-2/-/apidom-parser-adapter-asyncapi-json-2-1.11.0.tgz#b9c6efff4ece06882a3aba91ed787b307618d6fe"
integrity sha512-sQenLXZRmTDQehe3JCSQpz6jpE3DhMQ0aoe2gpNqo23Gt/4oeW6nAP2h49q9Ne+CHPp0ApFUUyIXF7UTmbUWqA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-asyncapi-json-3@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-asyncapi-json-3/-/apidom-parser-adapter-asyncapi-json-3-1.11.0.tgz#af21567e11a9e2eaf84748d5a5a93436c19bc71a"
integrity sha512-aGnG3AYp4Qsimn1FOP0B9leYCJAQVockzHqyJj30xiNAXquBMXr6lq3L2/AEsmpDGv/x/++YJ4p2ggSxy12QNw==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-3" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-asyncapi-yaml-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-asyncapi-yaml-2/-/apidom-parser-adapter-asyncapi-yaml-2-1.11.0.tgz#c8d2670275331f075714bdf6afb99a00aa53d3e4"
integrity sha512-iIRlB8B46UPiu0EkKhq1TvwloBgObASJ5ROx8rhT5+Pj+BBegE+KIY02EUKwcz5FgXJrH3XcltLiI7ZA68347Q==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-asyncapi-yaml-3@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-asyncapi-yaml-3/-/apidom-parser-adapter-asyncapi-yaml-3-1.11.0.tgz#aaaf47cd034228d453d6426b3afcf2b429a3a497"
integrity sha512-BF2ZyQYMUNrjP1nMneX6ZD2IWBLycWpxg3yllXDCJtfdQT/IMzldIPKCNI9qoBE57lM6j2hpy+Jd86QJk20t2w==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-3" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-json@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-json/-/apidom-parser-adapter-json-1.11.0.tgz#81e5919a2e4139492c0f910f8413557eb059de8a"
integrity sha512-DObW0LxYwif0erzGoXiEAZ6ecc/18LIEKxjEAc5Bw2M5I0C/iGW4y/UxAywihGvhMEo1gOvdO6w9Jh6UnuPVmA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-ast" "^1.11.0"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-error" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
tree-sitter "=0.21.1"
tree-sitter-json "=0.24.8"
web-tree-sitter "=0.24.5"
"@swagger-api/apidom-parser-adapter-openapi-json-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-json-2/-/apidom-parser-adapter-openapi-json-2-1.11.0.tgz#c48a0a51b4381ebf5b7c3406b90d9a25dfa48a49"
integrity sha512-dREUHAEHVry9aSGjqDpYF9Wzm1lgUkV6EgoYDflyQ9HxgCwhucDPFmUgI7UaR0G6bplnJumMcZXh1I1TGn1v7Q==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-0@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-json-3-0/-/apidom-parser-adapter-openapi-json-3-0-1.11.0.tgz#e432f87615bdd5652aa7c299454ede2cbec46f4a"
integrity sha512-U/NZpvuj9IpUS48zF2tYbgW2AtTw6Yi6kXNiHUtgUEomxYdb6XQeKLDGvgeWjgAgfUROohakcH+wx713VCGxfQ==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-0" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-1@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-json-3-1/-/apidom-parser-adapter-openapi-json-3-1-1.11.0.tgz#1c5043a00620235c0b175d03a1da4a17d41294fd"
integrity sha512-fYarNeaz39oKZ6VwqwON+IeJszidZGPvUYDfggLaar81NGimrz07y1U+DhAf96IX3qgUa2J6Fu3Bv1r57hs6Ng==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-json-3-2/-/apidom-parser-adapter-openapi-json-3-2-1.11.0.tgz#29bcb3388b9a452a78a01d2491e5891009755c2d"
integrity sha512-jtMoAH3R73bQUc4D2cJTUUvO4iJz9CV1W4+zoU/gT2l6h8Ji5EhZH0/VyynUk4J6mW/GdwxUN/q5z2P/DtSmfA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-yaml-2/-/apidom-parser-adapter-openapi-yaml-2-1.11.0.tgz#08a94606d3b636075aacb9af80b5fd25b4b15782"
integrity sha512-e8L4kHahgkOIzCCSGs5jTahXLInERNr37teSLS4SuqYgSVWr9AVXuNvpHNYGeMECD8briGIGfAAtnZChCGYrEA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-0@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-yaml-3-0/-/apidom-parser-adapter-openapi-yaml-3-0-1.11.0.tgz#55c35ac58764b20890456707c9176ef48f20d741"
integrity sha512-s+AXnNzLeAk28jUAeXwTSR1AlX+TXIAt2GfFgWUAV+SFw2OhRpoKYLzItN3n2UsHselqHvfyUL9xNCJBZleQtQ==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-0" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-1@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-yaml-3-1/-/apidom-parser-adapter-openapi-yaml-3-1-1.11.0.tgz#a6dc3316738f3175d896a832701c349ec416c11a"
integrity sha512-xyUyehHhB+BSOAT7mYGqmcEozuLKxmx1Hug97O9SVgNU8QTClc95+VWrAHhJbn8juPR6y2vSwm/wrQDwb4yq7w==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-openapi-yaml-3-2/-/apidom-parser-adapter-openapi-yaml-3-2-1.11.0.tgz#bc83e0d8ec0a54c67c8a77109be56c5eef5be169"
integrity sha512-u7Y98zdjEs+0Upa8TdxOsb7z8hYJmLz9lVleRiB7rqysVga6oSDI5NAFdLVqMB6uAUuFi/tyiuiFT4Qosfd6Vw==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-parser-adapter-yaml-1-2/-/apidom-parser-adapter-yaml-1-2-1.11.0.tgz#6bca1a605127d904bde0419bfd2df880c6051757"
integrity sha512-FZK9KfwiTnNc+imxg7Wu2ktKhXCYPeFQZ1uZJzJL/hk1n+zyPfRY/4Aue4HzDcG8+wbItd3dRjKClFanVZAXoA==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-ast" "^1.11.0"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-error" "^1.11.0"
"@tree-sitter-grammars/tree-sitter-yaml" "=0.7.1"
"@types/ramda" "~0.30.0"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
tree-sitter "=0.22.4"
web-tree-sitter "=0.24.5"
"@swagger-api/apidom-reference@^1.11.0":
version "1.11.0"
resolved "https://registry.yarnpkg.com/@swagger-api/apidom-reference/-/apidom-reference-1.11.0.tgz#9deaff0a93e46058c090946394dbb83975a86a51"
integrity sha512-ftqegYrxxl9UwQFbdVOtXIqNolVd25M5u53X8fP96Wx6lEVr5Ed7B6+dzch8ttCUmKeoLIeagvt76b6BoYtnLw==
dependencies:
"@babel/runtime-corejs3" "^7.26.10"
"@swagger-api/apidom-core" "^1.11.0"
"@swagger-api/apidom-error" "^1.11.0"
"@types/ramda" "~0.30.0"
axios "^1.15.0"
minimatch "^10.2.1"
ramda "~0.30.0"
ramda-adjunct "^5.0.0"
optionalDependencies:
"@swagger-api/apidom-json-pointer" "^1.11.0"
"@swagger-api/apidom-ns-arazzo-1" "^1.11.0"
"@swagger-api/apidom-ns-asyncapi-2" "^1.11.0"
"@swagger-api/apidom-ns-openapi-2" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-0" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-1" "^1.11.0"
"@swagger-api/apidom-ns-openapi-3-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-api-design-systems-json" "^1.11.0"
"@swagger-api/apidom-parser-adapter-api-design-systems-yaml" "^1.11.0"
"@swagger-api/apidom-parser-adapter-arazzo-json-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-arazzo-yaml-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-asyncapi-json-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-asyncapi-json-3" "^1.11.0"
"@swagger-api/apidom-parser-adapter-asyncapi-yaml-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-asyncapi-yaml-3" "^1.11.0"
"@swagger-api/apidom-parser-adapter-json" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-json-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-0" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-json-3-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-0" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-1" "^1.11.0"
"@swagger-api/apidom-parser-adapter-openapi-yaml-3-2" "^1.11.0"
"@swagger-api/apidom-parser-adapter-yaml-1-2" "^1.11.0"
"@swaggerexpert/cookie@^2.0.2":
version "2.0.2"
@@ -4201,6 +4543,14 @@
dependencies:
defer-to-connect "^2.0.1"
"@tree-sitter-grammars/tree-sitter-yaml@=0.7.1":
version "0.7.1"
resolved "https://registry.npmjs.org/@tree-sitter-grammars/tree-sitter-yaml/-/tree-sitter-yaml-0.7.1.tgz"
integrity sha512-AynBwkIoQCTgjDR33bDUp9Mqq+YTco0is3n5hRApMqG9of/6A4eQsfC1/uSEeHSUyMQSYawcAWamsexnVpIP4Q==
dependencies:
node-addon-api "^8.3.1"
node-gyp-build "^4.8.4"
"@tybys/wasm-util@^0.10.1":
version "0.10.1"
resolved "https://registry.yarnpkg.com/@tybys/wasm-util/-/wasm-util-0.10.1.tgz#ecddd3205cf1e2d5274649ff0eedd2991ed7f414"
@@ -5129,6 +5479,13 @@ address@^1.0.1:
resolved "https://registry.npmjs.org/address/-/address-1.2.2.tgz"
integrity sha512-4B/qKCfeE/ODUaAUpSwfzazo5x29WD4r3vXiWsB7I2mSDAihwEqKO+g8GELZUQSSAo5e1XTYh3ZVfLyxBc12nA==
agent-base@6:
version "6.0.2"
resolved "https://registry.yarnpkg.com/agent-base/-/agent-base-6.0.2.tgz#49fff58577cfee3f37176feab4c22e00f86d7f77"
integrity sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==
dependencies:
debug "4"
aggregate-error@^3.0.0:
version "3.1.0"
resolved "https://registry.npmjs.org/aggregate-error/-/aggregate-error-3.1.0.tgz"
@@ -5490,6 +5847,11 @@ async@3.2.6:
resolved "https://registry.npmjs.org/async/-/async-3.2.6.tgz"
integrity sha512-htCUDlxyyCLMgaM3xXg0C0LW2xqfuQ6p05pCEIsXuyQ+a1koYKTuBMzRNwmybfLgvJDMd0r1LTn4+E0Ti6C2AA==
asynckit@^0.4.0:
version "0.4.0"
resolved "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz"
integrity sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==
autolinker@^3.11.0:
version "3.16.2"
resolved "https://registry.npmjs.org/autolinker/-/autolinker-3.16.2.tgz"
@@ -5516,6 +5878,16 @@ available-typed-arrays@^1.0.7:
dependencies:
possible-typed-array-names "^1.0.0"
axios@^1.15.0:
version "1.16.1"
resolved "https://registry.yarnpkg.com/axios/-/axios-1.16.1.tgz#517e29291d19d6e8cf919ff264f4fe157261ba12"
integrity sha512-caYkukvroVPO8KrzuJEb50Hm07KwfBZPEC3VeFHTsqWHvKTsy54hjJz9BS/cdaypROE2rH6xvm9mHX4fgWkr3A==
dependencies:
follow-redirects "^1.16.0"
form-data "^4.0.5"
https-proxy-agent "^5.0.1"
proxy-from-env "^2.1.0"
babel-loader@^9.2.1:
version "9.2.1"
resolved "https://registry.npmjs.org/babel-loader/-/babel-loader-9.2.1.tgz"
@@ -6052,6 +6424,13 @@ combine-promises@^1.1.0:
resolved "https://registry.npmjs.org/combine-promises/-/combine-promises-1.2.0.tgz"
integrity sha512-VcQB1ziGD0NXrhKxiwyNbCDmRzs/OShMs2GqW2DlU2A/Sd0nQxE1oWDAE5O0ygSx5mgQOn9eIFh7yKPgFRVkPQ==
combined-stream@^1.0.8:
version "1.0.8"
resolved "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz"
integrity sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==
dependencies:
delayed-stream "~1.0.0"
comma-separated-tokens@^2.0.0:
version "2.0.3"
resolved "https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-2.0.3.tgz"
@@ -6976,6 +7355,11 @@ delaunator@5:
dependencies:
robust-predicates "^3.0.2"
delayed-stream@~1.0.0:
version "1.0.0"
resolved "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz"
integrity sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==
depd@2.0.0, depd@~2.0.0:
version "2.0.0"
resolved "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz"
@@ -8020,7 +8404,7 @@ flatted@^3.2.9:
resolved "https://registry.yarnpkg.com/flatted/-/flatted-3.4.2.tgz#f5c23c107f0f37de8dbdf24f13722b3b98d52726"
integrity sha512-PjDse7RzhcPkIJwy5t7KPWQSZ9cAbzQXcafsetQoD7sOJRQlGikNbx7yZp2OotDnJyrDcbyRq3Ttb18iYOqkxA==
follow-redirects@^1.0.0:
follow-redirects@^1.0.0, follow-redirects@^1.16.0:
version "1.16.0"
resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.16.0.tgz#28474a159d3b9d11ef62050a14ed60e4df6d61bc"
integrity sha512-y5rN/uOsadFT/JfYwhxRS5R7Qce+g3zG97+JrtFZlC9klX/W5hD7iiLzScI4nZqUS7DNUdhPgw4xI8W2LuXlUw==
@@ -8042,6 +8426,17 @@ form-data-encoder@^2.1.2:
resolved "https://registry.npmjs.org/form-data-encoder/-/form-data-encoder-2.1.4.tgz"
integrity sha512-yDYSgNMraqvnxiEXO4hi88+YZxaHC6QKzb5N84iRCTDeRO7ZALpir/lVmf/uXUhnwUr2O4HU8s/n6x+yNjQkHw==
form-data@^4.0.5:
version "4.0.5"
resolved "https://registry.yarnpkg.com/form-data/-/form-data-4.0.5.tgz#b49e48858045ff4cbf6b03e1805cebcad3679053"
integrity sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==
dependencies:
asynckit "^0.4.0"
combined-stream "^1.0.8"
es-set-tostringtag "^2.1.0"
hasown "^2.0.2"
mime-types "^2.1.12"
format@^0.2.0:
version "0.2.2"
resolved "https://registry.npmjs.org/format/-/format-0.2.2.tgz"
@@ -8703,6 +9098,14 @@ http2-wrapper@^2.1.10:
quick-lru "^5.1.1"
resolve-alpn "^1.2.0"
https-proxy-agent@^5.0.1:
version "5.0.1"
resolved "https://registry.yarnpkg.com/https-proxy-agent/-/https-proxy-agent-5.0.1.tgz#c59ef224a04fe8b754f3db0063a25ea30d0005d6"
integrity sha512-dFcAjpTQFgoLMzC2VwU+C/CbS7uRL0lWmxDITmqm7C+7F0Odmj6s9l6alZc6AELXhrnggM2CeWSXHGOdX2YtwA==
dependencies:
agent-base "6"
debug "4"
human-signals@^2.1.0:
version "2.1.0"
resolved "https://registry.npmjs.org/human-signals/-/human-signals-2.1.0.tgz"
@@ -10759,7 +11162,7 @@ mime-types@2.1.18:
dependencies:
mime-db "~1.33.0"
mime-types@^2.1.27, mime-types@~2.1.17, mime-types@~2.1.24, mime-types@~2.1.34:
mime-types@^2.1.12, mime-types@^2.1.27, mime-types@~2.1.17, mime-types@~2.1.24, mime-types@~2.1.34:
version "2.1.35"
resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz"
integrity sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==
@@ -10825,7 +11228,7 @@ minimatch@3.1.5, minimatch@^3.1.1, minimatch@^3.1.2:
dependencies:
brace-expansion "^1.1.7"
minimatch@^10.2.2:
minimatch@^10.2.1, minimatch@^10.2.2:
version "10.2.5"
resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-10.2.5.tgz#bd48687a0be38ed2961399105600f832095861d1"
integrity sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==
@@ -10932,9 +11335,14 @@ node-addon-api@^7.0.0:
resolved "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.1.tgz"
integrity sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==
node-addon-api@^8.0.0, node-addon-api@^8.2.2, node-addon-api@^8.3.0, node-addon-api@^8.3.1:
version "8.5.0"
resolved "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.5.0.tgz"
integrity sha512-/bRZty2mXUIFY/xU5HLvveNHlswNJej+RnxBjOMkidWfwZzgTbPG1E3K5TOxRLOR+5hX7bSofy8yf1hZevMS8A==
node-domexception@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/node-domexception/-/node-domexception-1.0.0.tgz#6888db46a1f71c0b76b3f7555016b63fe64766e5"
resolved "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz"
integrity sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==
node-emoji@^2.1.0:
@@ -10949,7 +11357,7 @@ node-emoji@^2.1.0:
node-fetch-commonjs@^3.3.2:
version "3.3.2"
resolved "https://registry.yarnpkg.com/node-fetch-commonjs/-/node-fetch-commonjs-3.3.2.tgz#0dd0fd4c4a314c5234f496ff7b5d9ce5a6c8feaa"
resolved "https://registry.npmjs.org/node-fetch-commonjs/-/node-fetch-commonjs-3.3.2.tgz"
integrity sha512-VBlAiynj3VMLrotgwOS3OyECFxas5y7ltLcK4t41lMUZeaK15Ym4QRkqN0EQKAFL42q9i21EPKjzLUPfltR72A==
dependencies:
node-domexception "^1.0.0"
@@ -10969,6 +11377,11 @@ node-fetch@^2.6.1:
dependencies:
whatwg-url "^5.0.0"
node-gyp-build@^4.8.0, node-gyp-build@^4.8.2, node-gyp-build@^4.8.4:
version "4.8.4"
resolved "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz"
integrity sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==
node-readfiles@^0.2.0:
version "0.2.0"
resolved "https://registry.npmjs.org/node-readfiles/-/node-readfiles-0.2.0.tgz"
@@ -12257,6 +12670,11 @@ proxy-addr@~2.0.7:
forwarded "0.2.0"
ipaddr.js "1.9.1"
proxy-from-env@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/proxy-from-env/-/proxy-from-env-2.1.0.tgz#a7487568adad577cfaaa7e88c49cab3ab3081aba"
integrity sha512-cJ+oHTW1VAEa8cJslgmUZrc+sjRKgAKl3Zyse6+PV38hZe/V6Z14TbCuXcan9F9ghlz4QrFr2c92TNF82UkYHA==
punycode@^1.4.1:
version "1.4.1"
resolved "https://registry.npmjs.org/punycode/-/punycode-1.4.1.tgz"
@@ -14032,7 +14450,7 @@ svgo@^3.0.2, svgo@^3.2.0:
picocolors "^1.0.0"
sax "^1.5.0"
swagger-client@3.37.3, swagger-client@^3.37.4:
swagger-client@^3.37.3:
version "3.37.3"
resolved "https://registry.yarnpkg.com/swagger-client/-/swagger-client-3.37.3.tgz#dbe3f0d22c367d4bc04cc7ddaf5224e116d46074"
integrity sha512-PZv5smQPnPwfP6mnkq96fOp/RNDKBqd8vfwE4UuwA229wsesj20yd7RadXx+9uLBC3c0H6cu/H+bnbMTWG6oUQ==
@@ -14057,10 +14475,10 @@ swagger-client@3.37.3, swagger-client@^3.37.4:
ramda "^0.30.1"
ramda-adjunct "^5.1.0"
swagger-ui-react@^5.32.6:
version "5.32.6"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.32.6.tgz#00c3f99a5b1f6c0debb2ee0589018dfc79ba1d4a"
integrity sha512-2q2kXd6eDR+syyWV5HE2CkWANyr2MHPkNezG4M7fC0FPlBUZEsNgyA/2dcb9dIwgE5xd995dO42h89fNMF5/ng==
swagger-ui-react@^5.32.5:
version "5.32.5"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.32.5.tgz#c4650972c71aaf4107a5f742e9c45615eba6857f"
integrity sha512-u86Qx36C5FvmJFVGMF3s62dxR3l0EfUmlylJVqCJ4vL0tvfd38kNdCQan9app6Y+C25uqVAjGLYu2w87UMD35Q==
dependencies:
"@babel/runtime-corejs3" "^7.27.1"
"@scarf/scarf" "=1.4.0"
@@ -14091,7 +14509,7 @@ swagger-ui-react@^5.32.6:
reselect "^5.1.1"
serialize-error "^8.1.0"
sha.js "^2.4.12"
swagger-client "^3.37.4"
swagger-client "^3.37.3"
url-parse "^1.5.10"
xml "=1.0.1"
xml-but-prettier "^1.0.1"
@@ -14261,6 +14679,30 @@ tree-dump@^1.0.3:
resolved "https://registry.npmjs.org/tree-dump/-/tree-dump-1.1.0.tgz"
integrity sha512-rMuvhU4MCDbcbnleZTFezWsaZXRFemSqAM+7jPnzUl1fo9w3YEKOxAeui0fz3OI4EU4hf23iyA7uQRVko+UaBA==
tree-sitter-json@=0.24.8:
version "0.24.8"
resolved "https://registry.npmjs.org/tree-sitter-json/-/tree-sitter-json-0.24.8.tgz"
integrity sha512-Tc9ZZYwHyWZ3Tt1VEw7Pa2scu1YO7/d2BCBbKTx5hXwig3UfdQjsOPkPyLpDJOn/m1UBEWYAtSdGAwCSyagBqQ==
dependencies:
node-addon-api "^8.2.2"
node-gyp-build "^4.8.2"
tree-sitter@=0.21.1:
version "0.21.1"
resolved "https://registry.yarnpkg.com/tree-sitter/-/tree-sitter-0.21.1.tgz#fbb34c09056700814af0e1e37688e06463ba04c4"
integrity sha512-7dxoA6kYvtgWw80265MyqJlkRl4yawIjO7S5MigytjELkX43fV2WsAXzsNfO7sBpPPCF5Gp0+XzHk0DwLCq3xQ==
dependencies:
node-addon-api "^8.0.0"
node-gyp-build "^4.8.0"
tree-sitter@=0.22.4:
version "0.22.4"
resolved "https://registry.npmjs.org/tree-sitter/-/tree-sitter-0.22.4.tgz"
integrity sha512-usbHZP9/oxNsUY65MQUsduGRqDHQOou1cagUSwjhoSYAmSahjQDAVsh9s+SlZkn8X8+O1FULRGwHu7AFP3kjzg==
dependencies:
node-addon-api "^8.3.0"
node-gyp-build "^4.8.4"
trim-lines@^3.0.0:
version "3.0.1"
resolved "https://registry.npmjs.org/trim-lines/-/trim-lines-3.0.1.tgz"
@@ -14880,9 +15322,14 @@ web-namespaces@^2.0.0:
web-streams-polyfill@^3.0.3:
version "3.3.3"
resolved "https://registry.yarnpkg.com/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz#2073b91a2fdb1fbfbd401e7de0ac9f8214cecb4b"
resolved "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz"
integrity sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==
web-tree-sitter@=0.24.5:
version "0.24.5"
resolved "https://registry.npmjs.org/web-tree-sitter/-/web-tree-sitter-0.24.5.tgz"
integrity sha512-+J/2VSHN8J47gQUAvF8KDadrfz6uFYVjxoxbKWDoXVsH2u7yLdarCnIURnrMA6uSRkgX3SdmqM5BOoQjPdSh5w==
webidl-conversions@^3.0.0:
version "3.0.1"
resolved "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz"

View File

@@ -220,6 +220,7 @@ development = [
"openapi-spec-validator",
"parameterized",
"pip",
"polib", # used by scripts/translations/ and their unit tests
"pre-commit",
"progress>=1.5,<2",
"psutil",

View File

@@ -677,6 +677,8 @@ ply==3.11
# via
# -c requirements/base-constraint.txt
# jsonpath-ng
polib==1.2.0
# via apache-superset
polyline==2.0.2
# via
# -c requirements/base-constraint.txt

View File

@@ -0,0 +1,632 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Backfill missing translations in a .po file using Claude AI.
For each untranslated (empty msgstr) entry in the target language, the script
sends the English source string along with all available translations in other
languages to Claude as context, then writes the AI-generated translation back
into the .po file marked as #, fuzzy for human review.
Usage:
# Build the translation index first (one-time or when .po files change)
python scripts/translations/build_translation_index.py
# Backfill French translations
python scripts/translations/backfill_po.py --lang fr
# Dry run (print what would be translated without writing)
python scripts/translations/backfill_po.py --lang de --dry-run
# Limit to 100 entries and use a specific model
python scripts/translations/backfill_po.py --lang es --limit 100 \
--model claude-opus-4-6
Options:
--lang LANG ISO language code to backfill (required)
--batch-size N Number of strings per Claude request (default: 50)
--limit N Stop after translating N entries (default: unlimited)
--model MODEL Claude model ID (default: claude-sonnet-4-6)
--index PATH Path to translation_index.json (default: auto-detect)
--dry-run Print translations without writing to .po file
--no-fuzzy Do not mark generated translations as fuzzy (default: mark fuzzy)
"""
from __future__ import annotations
import argparse
import json
import re
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Any
try:
import polib # type: ignore[import-untyped]
except ImportError:
print("polib is required. Run: pip install polib", file=sys.stderr)
sys.exit(1)
TRANSLATIONS_DIR = Path(__file__).parent.parent.parent / "superset" / "translations"
DEFAULT_INDEX = TRANSLATIONS_DIR / "translation_index.json"
DEFAULT_MODEL = "claude-sonnet-4-6"
DEFAULT_BATCH_SIZE = 50
# Language names for the prompt, keyed by ISO code
LANGUAGE_NAMES: dict[str, str] = {
"ar": "Arabic",
"ca": "Catalan",
"de": "German",
"es": "Spanish",
"fa": "Persian (Farsi)",
"fr": "French",
"it": "Italian",
"ja": "Japanese",
"ko": "Korean",
"mi": "Māori",
"nl": "Dutch",
"pl": "Polish",
"pt": "Portuguese",
"pt_BR": "Brazilian Portuguese",
"ru": "Russian",
"sk": "Slovak",
"sl": "Slovenian",
"tr": "Turkish",
"uk": "Ukrainian",
"zh": "Chinese (Simplified)",
"zh_TW": "Chinese (Traditional)",
}
def _lang_name(code: str) -> str:
"""Return a human-readable language name for an ISO language code."""
return LANGUAGE_NAMES.get(code, code)
def _plural_key(msgid: str, msgid_plural: str) -> str:
"""Build the translation index key used for pluralized entries."""
return f"{msgid}\x00{msgid_plural}"
def _is_missing(entry: polib.POEntry) -> bool:
"""Return True for entries that need a translation."""
if entry.obsolete:
return False
if entry.msgid_plural:
return not any(v for v in entry.msgstr_plural.values())
return not entry.msgstr
def _context_langs(
item: dict[str, Any], index: dict[str, Any], target_lang: str
) -> list[str]:
"""Return sorted list of language codes that have translations for this entry."""
key = item["index_key"]
if key not in index:
return []
return sorted(
lang for lang, val in index[key].items() if lang != target_lang and val
)
def _context_count(
item: dict[str, Any], index: dict[str, Any], target_lang: str
) -> int:
"""Return the number of other-language translations available for this entry."""
return len(_context_langs(item, index, target_lang))
def _render_item(
i: int,
item: dict[str, Any],
index: dict[str, Any],
target_lang: str,
reference_langs_sorted: list[str],
) -> list[str]:
"""Render one batch entry as prompt lines."""
lines: list[str] = []
ctx = _context_count(item, index, target_lang)
if ctx == 0:
lines.append(
f"--- [{i}] (no reference translations — translate conservatively) ---"
)
else:
plural = "s" if ctx != 1 else ""
lines.append(f"--- [{i}] ({ctx} reference translation{plural}) ---")
lines.append(f"English: {json.dumps(item['msgid'], ensure_ascii=False)}")
if item.get("msgid_plural"):
plural_json = json.dumps(item["msgid_plural"], ensure_ascii=False)
lines.append(f"English plural: {plural_json}")
key = item["index_key"]
if key in index and reference_langs_sorted:
for lang in reference_langs_sorted:
val = index[key].get(lang)
if val is None:
continue
if isinstance(val, dict):
forms = "; ".join(
f"[{k}] {json.dumps(v, ensure_ascii=False)}" for k, v in val.items()
)
lines.append(f"{_lang_name(lang)}: {forms}")
else:
lines.append(
f"{_lang_name(lang)}: {json.dumps(val, ensure_ascii=False)}"
)
lines.append("")
return lines
def build_prompt(
target_lang: str,
batch: list[dict[str, Any]],
index: dict[str, Any],
) -> str:
"""Build the Claude prompt for a batch of entries."""
lang_name = _lang_name(target_lang)
# Collect which other languages actually have translations for this batch
reference_langs: set[str] = set()
for item in batch:
key = item["index_key"]
if key in index:
reference_langs.update(
lang for lang, val in index[key].items() if lang != target_lang and val
)
reference_langs_sorted = sorted(reference_langs)
lines: list[str] = [
"You are a professional translator specializing in software UI strings.",
f"Translate the following English strings into {lang_name} ({target_lang}).",
"",
"Rules:",
"- Preserve all format placeholders exactly: %(name)s, {name}, %s, %d, etc.",
"- Preserve HTML tags if present.",
"- Keep the same tone and register as the reference translations.",
"- For plural forms, provide translations for all plural forms"
" required by the language.",
"- Return ONLY a JSON object mapping each numeric index (as a string)"
" to its translation.",
"- Do not add any explanation, preamble, or markdown fences.",
"",
"Important: Many strings are short fragments or single words that are"
" ambiguous in English (e.g. 'Scale' could mean a measurement scale,"
" to scale an image, or fish scales). Use the translations in other"
" languages as your primary signal for which meaning is intended —"
" they collectively disambiguate the intended sense. When no"
" other-language translations are available for an entry, translate"
" conservatively based on the most common meaning in a data"
" visualization UI context.",
"",
]
if reference_langs_sorted:
lines.append(
f"Reference translations are provided per string where available "
f"({', '.join(_lang_name(lc) for lc in reference_langs_sorted)})."
)
lines.append("")
lines.append("Strings to translate:")
lines.append("")
for i, item in enumerate(batch):
lines.extend(_render_item(i, item, index, target_lang, reference_langs_sorted))
if batch and batch[0].get("msgid_plural"):
# Add guidance on plural form counts per language
lines.append(
"Note: provide ALL plural forms required by the target language "
"(e.g. French needs 2, Russian needs 3, Arabic needs 6)."
)
lines.append("")
lines.append(
'Expected output format: {"0": "<translation>", "1": "<translation>", ...}'
)
lines.append("(keys are the numeric indices of the strings above)")
return "\n".join(lines)
def parse_response(text: str, batch_size: int) -> dict[int, str]:
"""Parse the JSON object from Claude's response."""
# Strip any accidental markdown fences
text = re.sub(r"^```[^\n]*\n", "", text.strip())
text = re.sub(r"\n```$", "", text)
try:
raw = json.loads(text)
except json.JSONDecodeError as exc:
raise ValueError(
f"Could not parse response as JSON: {exc}\n\nResponse:\n{text}"
) from exc
# _process_batches only catches ValueError/RuntimeError, so a non-object
# response (list, scalar, null) must surface as ValueError rather than
# bubbling up an AttributeError from .items() and aborting the whole run.
if not isinstance(raw, dict):
raise ValueError(
f"Expected a JSON object mapping indices to translations, "
f"got {type(raw).__name__}.\n\nResponse:\n{text}"
)
# Preserve dict/list values as JSON strings so plural responses (where
# v is a dict of plural forms) can be re-parsed downstream by
# _apply_translation's json.loads. str(v) on a dict produces Python
# repr ({'0': 'x'}) which is not valid JSON.
return {
int(k): (
json.dumps(v, ensure_ascii=False) if isinstance(v, (dict, list)) else str(v)
)
for k, v in raw.items()
if str(k).isdigit()
}
def translate_batch(
model: str,
target_lang: str,
batch: list[dict[str, Any]],
index: dict[str, Any],
) -> dict[int, str]:
"""Send a batch of strings to Claude via `claude -p`.
Returns a dict mapping batch index to translated string.
"""
claude_bin = shutil.which("claude")
if not claude_bin:
raise RuntimeError(
"claude CLI not found. Install Claude Code or add it to PATH."
)
prompt = build_prompt(target_lang, batch, index)
# Pipe the prompt over stdin rather than passing it as argv: a single batch
# with many reference languages can grow into the tens of KB and approach
# ARG_MAX on some platforms.
# claude_bin is resolved via shutil.which — not user-controlled input
result = subprocess.run( # noqa: S603
[claude_bin, "--model", model, "-p"],
input=prompt,
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise RuntimeError(
f"claude exited with code {result.returncode}:\n{result.stderr}"
)
return parse_response(result.stdout.strip(), len(batch))
def _apply_plural_translation(entry: polib.POEntry, translation: str) -> None:
"""Distribute a model response across the entry's plural forms.
Model may return a JSON dict ({"0": "form0", "1": "form1"}), a JSON list
(["form0", "form1"], also valid since plural forms are ordered), a JSON
scalar (a single translation that fills every form), or a plain non-JSON
string (older models that ignore the JSON instruction).
"""
try:
plural_value = json.loads(translation)
except (json.JSONDecodeError, ValueError):
for k in entry.msgstr_plural:
entry.msgstr_plural[k] = translation
return
if isinstance(plural_value, dict):
entry.msgstr_plural = {int(k): str(v) for k, v in plural_value.items()}
return
if isinstance(plural_value, list) and plural_value:
# Distribute list items across plural form indices in order; if the
# model returned fewer forms than the language requires, repeat the
# last form rather than leaving slots blank.
forms = [str(v) for v in plural_value]
for k in sorted(entry.msgstr_plural):
entry.msgstr_plural[k] = forms[k] if k < len(forms) else forms[-1]
return
# Scalar (or empty list) — broadcast to every form.
fill = str(plural_value) if plural_value not in (None, []) else translation
for k in entry.msgstr_plural:
entry.msgstr_plural[k] = fill
def _apply_translation(
entry: polib.POEntry,
translation: str,
item: dict[str, Any],
model: str,
mark_fuzzy: bool,
) -> None:
"""Write a translation string into a POEntry and add attribution."""
if entry.msgid_plural:
_apply_plural_translation(entry, translation)
else:
entry.msgstr = translation
if mark_fuzzy and "fuzzy" not in entry.flags:
entry.flags.append("fuzzy")
refs = item["context_langs"]
refs_tag = f" [refs: {', '.join(refs)}]" if refs else " [no refs]"
attribution = f"Machine-translated via backfill_po.py ({model}){refs_tag}"
if entry.tcomment:
if attribution not in entry.tcomment:
entry.tcomment = f"{entry.tcomment}\n{attribution}"
else:
entry.tcomment = attribution
def _build_batch_items(
entries: list[polib.POEntry],
index: dict[str, Any],
lang: str,
) -> list[dict[str, Any]]:
"""Convert a list of POEntries into the dict format used by translate_batch."""
items: list[dict[str, Any]] = []
for entry in entries:
if entry.msgid_plural:
item: dict[str, Any] = {
"msgid": entry.msgid,
"msgid_plural": entry.msgid_plural,
"index_key": _plural_key(entry.msgid, entry.msgid_plural),
"is_plural": True,
}
else:
item = {
"msgid": entry.msgid,
"index_key": entry.msgid,
"is_plural": False,
}
item["context_langs"] = _context_langs(item, index, lang)
item["context_count"] = len(item["context_langs"])
items.append(item)
return items
def _process_batches(
missing: list[polib.POEntry],
index: dict[str, Any],
lang: str,
batch_size: int,
model: str,
dry_run: bool,
mark_fuzzy: bool,
cat: polib.POFile | None = None,
po_path: Path | None = None,
) -> tuple[int, int]:
"""Translate missing entries in batches. Returns (translated, failed) counts.
When ``cat`` and ``po_path`` are provided and ``dry_run`` is False, the
catalog is saved to disk after each batch that produced at least one
successful translation. This means a crash mid-run only loses the in-flight
batch rather than every batch translated so far.
"""
translated_count = 0
failed_count = 0
for batch_start in range(0, len(missing), batch_size):
batch_entries = missing[batch_start : batch_start + batch_size]
batch_items = _build_batch_items(batch_entries, index, lang)
end = min(batch_start + batch_size, len(missing))
print(
f" Translating entries {batch_start + 1}{end} of {len(missing)}",
file=sys.stderr,
)
try:
translations = translate_batch(model, lang, batch_items, index)
except (ValueError, RuntimeError) as exc:
print(f" ERROR in batch starting at {batch_start}: {exc}", file=sys.stderr)
failed_count += len(batch_entries)
continue
batch_applied = 0
for i, entry in enumerate(batch_entries):
translation = translations.get(i)
if translation is None:
print(
f" WARNING: no translation returned for index {i} "
f"(msgid: {entry.msgid[:60]!r})",
file=sys.stderr,
)
failed_count += 1
continue
if dry_run:
ctx = batch_items[i]["context_count"]
ctx_tag = f" [ctx:{ctx}]" if ctx < 3 else ""
print(
f" [{lang}]{ctx_tag} {entry.msgid[:60]!r}{translation[:60]!r}"
)
else:
_apply_translation(
entry, translation, batch_items[i], model, mark_fuzzy
)
batch_applied += 1
translated_count += 1
if (
not dry_run
and batch_applied > 0
and cat is not None
and po_path is not None
):
cat.save()
print(
f" Saved {po_path} ({batch_applied} entry(ies) in this batch).",
file=sys.stderr,
)
return translated_count, failed_count
def backfill(
lang: str,
*,
batch_size: int = DEFAULT_BATCH_SIZE,
limit: int | None = None,
min_context: int = 0,
model: str = DEFAULT_MODEL,
index_path: Path = DEFAULT_INDEX,
dry_run: bool = False,
mark_fuzzy: bool = True,
) -> None:
"""Backfill missing translations in the target language's .po file."""
po_path = TRANSLATIONS_DIR / lang / "LC_MESSAGES" / "messages.po"
if not po_path.exists():
print(f"No .po file found for language '{lang}': {po_path}", file=sys.stderr)
sys.exit(1)
if not index_path.exists():
print(
f"Translation index not found at {index_path}.\n"
"Run: python scripts/translations/build_translation_index.py",
file=sys.stderr,
)
sys.exit(1)
print("Loading translation index …", file=sys.stderr)
with open(index_path, encoding="utf-8") as f:
index: dict[str, Any] = json.load(f)
print(f"Loading {po_path}", file=sys.stderr)
cat = polib.pofile(str(po_path))
missing: list[polib.POEntry] = [e for e in cat if e.msgid and _is_missing(e)]
print(f"Found {len(missing)} untranslated entries for '{lang}'.", file=sys.stderr)
if min_context > 0:
before = len(missing)
missing = [
e
for e in missing
if _context_count(
{
"index_key": (
_plural_key(e.msgid, e.msgid_plural)
if e.msgid_plural
else e.msgid
)
},
index,
lang,
)
>= min_context
]
skipped = before - len(missing)
print(
f"Skipping {skipped} entries with fewer than {min_context} reference "
f"translation(s) (use --min-context 0 to include them).",
file=sys.stderr,
)
if limit is not None:
missing = missing[:limit]
print(f"Limiting to {limit} entries.", file=sys.stderr)
if not missing:
print("Nothing to do.", file=sys.stderr)
return
translated_count, failed_count = _process_batches(
missing,
index,
lang,
batch_size,
model,
dry_run,
mark_fuzzy,
cat=cat,
po_path=po_path,
)
print(
f"\nDone. Translated: {translated_count}, Failed/skipped: {failed_count}.",
file=sys.stderr,
)
if not dry_run and translated_count > 0:
print(
f"Translations written to {po_path} (marked #, fuzzy for review).",
file=sys.stderr,
)
def main() -> None:
"""Parse CLI arguments and run translation backfill."""
parser = argparse.ArgumentParser(
description="Backfill missing .po translations using Claude AI",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument(
"--lang", required=True, help="ISO language code (e.g. fr, de, ja)"
)
parser.add_argument(
"--batch-size",
type=int,
default=DEFAULT_BATCH_SIZE,
help=f"Strings per Claude request (default: {DEFAULT_BATCH_SIZE})",
)
parser.add_argument(
"--limit",
type=int,
default=None,
help="Maximum number of entries to translate (default: unlimited)",
)
parser.add_argument(
"--model",
default=DEFAULT_MODEL,
help=f"Claude model ID (default: {DEFAULT_MODEL})",
)
parser.add_argument(
"--index",
type=Path,
default=DEFAULT_INDEX,
help=f"Path to translation_index.json (default: {DEFAULT_INDEX})",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Print translations without modifying the .po file",
)
parser.add_argument(
"--min-context",
type=int,
default=0,
metavar="N",
help=(
"Skip entries with fewer than N reference translations in other languages "
"(default: 0 = translate everything). Strings with low context are more "
"likely to be ambiguous single words or fragments — set to e.g. 2 to only "
"translate strings that have been confirmed in at least 2 other languages."
),
)
parser.add_argument(
"--no-fuzzy",
dest="mark_fuzzy",
action="store_false",
default=True,
help="Do not mark generated translations as #, fuzzy",
)
args = parser.parse_args()
backfill(
lang=args.lang,
batch_size=args.batch_size,
limit=args.limit,
min_context=args.min_context,
model=args.model,
index_path=args.index,
dry_run=args.dry_run,
mark_fuzzy=args.mark_fuzzy,
)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,153 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Build a cross-language translation index from all .po files.
Outputs a JSON file structured as:
{
"<msgid>": {
"<lang>": "<translated string or null>",
...
},
...
}
For plural entries the key is "<msgid>\x00<msgid_plural>" and the value
is a dict mapping lang -> {0: "...", 1: "..."} (or null if untranslated).
Usage:
python scripts/translations/build_translation_index.py
python scripts/translations/build_translation_index.py \
--translations-dir superset/translations \
--output /tmp/translation_index.json
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
from typing import Any
try:
import polib # type: ignore[import-untyped]
except ImportError:
print("polib is required. Install with: pip install polib", file=sys.stderr)
sys.exit(1)
TRANSLATIONS_DIR = Path(__file__).parent.parent.parent / "superset" / "translations"
DEFAULT_OUTPUT = (
Path(__file__).parent.parent.parent
/ "superset"
/ "translations"
/ "translation_index.json"
)
def _is_translated(entry: polib.POEntry) -> bool:
"""Return True if the entry has a non-empty, non-fuzzy translation."""
if "fuzzy" in entry.flags:
return False
if entry.msgid_plural:
return any(v for v in entry.msgstr_plural.values())
return bool(entry.msgstr)
def _plural_key(entry: polib.POEntry) -> str:
"""Build the combined key used for plural translation entries."""
return f"{entry.msgid}\x00{entry.msgid_plural}"
def build_index(translations_dir: Path) -> dict[str, Any]:
"""Read all .po files and build a combined translation index."""
index: dict[str, dict[str, Any]] = {}
langs = sorted(
d
for d in os.listdir(translations_dir)
if (translations_dir / d / "LC_MESSAGES" / "messages.po").exists()
and d != "en" # en has empty msgstr by convention (source = target)
)
for lang in langs:
po_path = translations_dir / lang / "LC_MESSAGES" / "messages.po"
cat = polib.pofile(str(po_path))
for entry in cat:
if not entry.msgid:
continue # skip header entry
if entry.msgid_plural:
key = _plural_key(entry)
if key not in index:
index[key] = {}
# Fuzzy entries are unreviewed (often machine-generated drafts),
# so excluding them prevents feeding unverified translations
# back into the AI backfill prompt as trusted context.
index[key][lang] = (
dict(entry.msgstr_plural) if _is_translated(entry) else None
)
else:
key = entry.msgid
if key not in index:
index[key] = {}
index[key][lang] = entry.msgstr if _is_translated(entry) else None
# Ensure every entry has a slot for every language (null if missing)
for key in index:
for lang in langs:
index[key].setdefault(lang, None)
return index
def main() -> None:
"""Parse arguments, build the translation index, and write it to disk."""
parser = argparse.ArgumentParser(
description="Build cross-language translation index"
)
parser.add_argument(
"--translations-dir",
type=Path,
default=TRANSLATIONS_DIR,
help="Path to the translations directory (default: superset/translations)",
)
parser.add_argument(
"--output",
"-o",
type=Path,
default=DEFAULT_OUTPUT,
help=(
"Output JSON file path"
" (default: superset/translations/translation_index.json)"
),
)
args = parser.parse_args()
print(f"Reading .po files from {args.translations_dir}", file=sys.stderr)
index = build_index(args.translations_dir)
print(f"Indexed {len(index)} message IDs.", file=sys.stderr)
args.output.parent.mkdir(parents=True, exist_ok=True)
with open(args.output, "w", encoding="utf-8") as f:
json.dump(index, f, ensure_ascii=False, indent=2)
print(f"Written to {args.output}", file=sys.stderr)
if __name__ == "__main__":
main()

File diff suppressed because it is too large Load Diff

View File

@@ -43,6 +43,8 @@
"build-instrumented": "cross-env NODE_ENV=production BABEL_ENV=instrumented webpack --mode=production --color",
"build-storybook": "storybook build",
"build-translation": "scripts/po2json.sh",
"translations:build-index": "python3 ../scripts/translations/build_translation_index.py",
"translations:backfill": "python3 ../scripts/translations/backfill_po.py",
"bundle-stats": "cross-env BUNDLE_ANALYZER=true npm run build && npx open-cli ../superset/static/stats/statistics.html",
"clear-npm": "mkdir -p /tmp/empty && rsync -a --delete /tmp/empty/ node_modules/ && rmdir node_modules /tmp/empty",
"core:cover": "cross-env NODE_ENV=test NODE_OPTIONS=\"--max-old-space-size=4096\" jest --coverage --coverageThreshold='{\"global\":{\"statements\":100,\"branches\":100,\"functions\":100,\"lines\":100}}' --collectCoverageFrom='[\"packages/**/src/**/*.{js,ts}\", \"!packages/superset-core/**/*\"]' packages",
@@ -340,8 +342,8 @@
"html-webpack-plugin": "^5.6.7",
"http-server": "^14.1.1",
"imports-loader": "^5.0.0",
"jest": "^30.4.2",
"jest-environment-jsdom": "^30.4.1",
"jest": "^30.3.0",
"jest-environment-jsdom": "^29.7.0",
"jest-html-reporter": "^4.4.0",
"jest-websocket-mock": "^2.5.0",
"js-yaml-loader": "^1.2.2",
@@ -412,14 +414,7 @@
"@luma.gl/gltf": "~9.2.5",
"@luma.gl/shadertools": "~9.2.5",
"@luma.gl/webgl": "~9.2.5",
"fast-xml-parser": "^5.8.0",
"jest-mock": "^30.4.0",
"jest-runtime": "^30.4.0",
"@jest/globals": "^30.4.0",
"@jest/types": "^30.4.0",
"jest-util": "^30.4.0",
"jest-circus": "^30.4.0",
"jest-environment-node": "^30.4.0"
"fast-xml-parser": "^5.8.0"
},
"readme": "ERROR: No README data found!",
"scarfSettings": {

View File

@@ -154,7 +154,7 @@ test('accepts custom style props', () => {
render(<DropdownContainer items={generateItems(2)} style={customStyle} />);
const container = screen.getByTestId('container');
expect(container).toHaveStyle('background-color: rgb(255, 0, 0)');
expect(container).toHaveStyle('background-color: red');
expect(container).toHaveStyle('padding: 10px');
});

View File

@@ -553,26 +553,22 @@ describe('SupersetClientClass', () => {
});
describe('when unauthorized', () => {
let originalLocation: any;
let authSpy: jest.SpyInstance;
let locationSpy: jest.SpyInstance;
let mockHrefValue: string;
const mockRequestUrl = 'https://host/get/url';
const mockRequestPath = '/get/url';
const mockRequestSearch = '?param=1&param=2';
const mockHref = mockRequestUrl + mockRequestSearch;
beforeEach(() => {
mockHrefValue = mockHref;
locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
originalLocation = window.location;
// @ts-expect-error
delete window.location;
window.location = {
pathname: mockRequestPath,
search: mockRequestSearch,
get href() {
return mockHrefValue;
},
set href(v: string) {
mockHrefValue = v;
},
} as unknown as Location);
href: mockHref,
} as unknown as Location;
authSpy = jest
.spyOn(SupersetClientClass.prototype, 'ensureAuth')
.mockImplementation();
@@ -582,7 +578,7 @@ describe('SupersetClientClass', () => {
afterEach(() => {
authSpy.mockReset();
locationSpy.mockRestore();
window.location = originalLocation;
});
test('should redirect', async () => {
@@ -603,17 +599,11 @@ describe('SupersetClientClass', () => {
test('should not redirect again if already on login page', async () => {
const client = new SupersetClientClass({});
mockHrefValue = '/login?next=something';
locationSpy.mockReturnValue({
get href() {
return mockHrefValue;
},
set href(v: string) {
mockHrefValue = v;
},
window.location = {
href: '/login?next=something',
pathname: '/login',
search: '?next=something',
} as unknown as Location);
} as unknown as Location;
let error;
try {

View File

@@ -360,26 +360,18 @@ describe('callApi()', () => {
});
describe('caching', () => {
let locationSpy: jest.SpyInstance;
let mockProtocol = 'https:';
const origLocation = window.location;
beforeAll(() => {
locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
get protocol() {
return mockProtocol;
},
set protocol(v: string) {
mockProtocol = v;
},
} as Location);
Object.defineProperty(window, 'location', { value: {} });
});
afterAll(() => {
locationSpy.mockRestore();
Object.defineProperty(window, 'location', { value: origLocation });
});
beforeEach(async () => {
mockProtocol = 'https:';
window.location.protocol = 'https:';
await caches.delete(constants.CACHE_KEY);
});

View File

@@ -65,7 +65,7 @@ describe('updateTextNode(node, options)', () => {
test('handles setting font', () => {
const node = updateTextNode(createTextNode(), {
style: {
font: 'italic 700 30px Lobster',
font: 'italic 30px Lobster 700',
},
});
expect(node.getAttribute('class')).toEqual('');

View File

@@ -630,11 +630,9 @@ describe('plugin-chart-table', () => {
);
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe('');
});
test('render cell without color', () => {
@@ -671,14 +669,12 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe(
'rgba(172, 225, 196, 0.81)',
'rgba(172, 225, 196, 0.812)',
);
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgba(0, 0, 0, 0)',
);
expect(getComputedStyle(screen.getByText('N/A')).background).toBe(
'rgba(0, 0, 0, 0)',
'',
);
expect(getComputedStyle(screen.getByText('N/A')).background).toBe('');
});
test('preserves muted null styling when no formatter resolves text color', () => {
@@ -1063,10 +1059,10 @@ describe('plugin-chart-table', () => {
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgba(0, 0, 0, 0)',
'',
);
});
@@ -1094,11 +1090,9 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('Maria')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe('');
});
test('render color with string column color formatter (operator containing)', () => {
@@ -1125,11 +1119,9 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe('');
});
test('render color with string column color formatter (operator not containing)', () => {
@@ -1156,10 +1148,10 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgba(0, 0, 0, 0)',
'',
);
});
@@ -1187,10 +1179,10 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgba(0, 0, 0, 0)',
'',
);
});
@@ -1217,13 +1209,13 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('Joe')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('Maria')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
});
@@ -1251,11 +1243,9 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('true')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('false')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('false')).background).toBe('');
});
test('render color with boolean column color formatter (operator is false)', () => {
@@ -1282,11 +1272,9 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('false')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('true')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('true')).background).toBe('');
});
test('render color with boolean column color formatter (operator is null)', () => {
@@ -1313,14 +1301,10 @@ describe('plugin-chart-table', () => {
}),
);
expect(getComputedStyle(screen.getByText('N/A')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('true')).background).toBe(
'rgba(0, 0, 0, 0)',
);
expect(getComputedStyle(screen.getByText('false')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('true')).background).toBe('');
expect(getComputedStyle(screen.getByText('false')).background).toBe('');
});
test('render color with boolean column color formatter (operator is not null)', () => {
@@ -1349,14 +1333,12 @@ describe('plugin-chart-table', () => {
const trueElements = screen.getAllByText('true');
const falseElements = screen.getAllByText('false');
expect(getComputedStyle(trueElements[0]).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(falseElements[0]).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByText('N/A')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByText('N/A')).background).toBe('');
});
test('render color with column color formatter to entire row', () => {
@@ -1385,13 +1367,13 @@ describe('plugin-chart-table', () => {
);
expect(getComputedStyle(screen.getByText('Michael')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByTitle('0.123456')).background).toBe(
'rgb(172, 225, 196)',
'rgba(172, 225, 196, 1)',
);
});
@@ -1662,9 +1644,7 @@ describe('plugin-chart-table', () => {
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe(
'rgba(0, 0, 0, 0)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe('');
});
test('render color with useGradient true returns gradient color', () => {
@@ -1694,11 +1674,9 @@ describe('plugin-chart-table', () => {
// When useGradient is true, should return gradient color with opacity
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe('');
});
test('render color with useGradient undefined defaults to gradient (backward compatibility)', () => {
@@ -1727,11 +1705,9 @@ describe('plugin-chart-table', () => {
// When useGradient is undefined, should default to gradient for backward compatibility
expect(getComputedStyle(screen.getByTitle('2467063')).background).toBe(
'rgb(172, 225, 196)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe(
'rgba(0, 0, 0, 0)',
'rgba(172, 225, 196, 1)',
);
expect(getComputedStyle(screen.getByTitle('2467')).background).toBe('');
});
test('render color with useGradient false and None operator returns solid color', () => {

View File

@@ -19,38 +19,6 @@
import JSDOMEnvironment from 'jest-environment-jsdom';
// jest-environment-jsdom 30 bundles jsdom 26, which marks window.location as
// [LegacyUnforgeable] (configurable: false). jest 30's spyOn now strictly
// checks the configurable flag and throws when it's false, breaking every test
// that uses jest.spyOn(window, 'location', 'get').
//
// We intercept Object.defineProperties at module-load time (before any JSDOM
// instance is created). The interceptor makes window.location configurable
// every time jsdom creates a new Window, restoring the ability to spy on it.
// This file is only required by Jest in the test environment so the
// monkey-patch is safe.
const _originalDefineProperties = Object.defineProperties.bind(Object);
(Object as any).defineProperties = function (
obj: object,
props: PropertyDescriptorMap,
) {
if (
props !== null &&
typeof props === 'object' &&
Object.prototype.hasOwnProperty.call(props, 'location') &&
(props as any).location?.configurable === false
) {
// Allow jest.spyOn(window, 'location', 'get') to work in tests by making
// the property configurable. This deviates from the browser spec's
// [LegacyUnforgeable] requirement but is acceptable in a test environment.
props = {
...props,
location: { ...(props as any).location, configurable: true },
};
}
return _originalDefineProperties(obj, props);
};
// https://github.com/facebook/jest/blob/v29.4.3/website/versioned_docs/version-29.4/Configuration.md#testenvironment-string
export default class FixJSDOMEnvironment extends JSDOMEnvironment {
constructor(...args: ConstructorParameters<typeof JSDOMEnvironment>) {

View File

@@ -582,25 +582,16 @@ describe('async actions', () => {
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
describe('runQuery with query params', () => {
let locationSpy: jest.SpyInstance;
const { location } = window;
beforeAll(() => {
const u = new URL('http://localhost/sqllab/?foo=bar');
locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
href: u.href,
pathname: u.pathname,
search: u.search,
hash: u.hash,
origin: u.origin,
host: u.host,
hostname: u.hostname,
port: u.port,
protocol: u.protocol,
} as Location);
delete (window as any).location;
(window as any).location = new URL('http://localhost/sqllab/?foo=bar');
});
afterAll(() => {
locationSpy.mockRestore();
delete (window as any).location;
window.location = location;
});
const makeRequest = () => {

View File

@@ -244,9 +244,11 @@ test('reads database from localStorage when URL has db param', () => {
jest.spyOn(localStorageHelpers, 'getItem').mockReturnValue(localStorageDb);
const locationSpy = jest
.spyOn(window, 'location', 'get')
.mockReturnValue({ ...window.location, search: '?db=true' } as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { search: '?db=true' },
writable: true,
});
const store = mockStore(createInitialState());
const { result, rerender } = renderHook(
@@ -267,7 +269,10 @@ test('reads database from localStorage when URL has db param', () => {
null,
);
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
});
});
test('returns null db when dbId does not exist in databases', () => {

View File

@@ -45,23 +45,15 @@ const createProps = () => ({
submenuKey: 'share',
});
const postDashboardPermalinkMockUrl = `http://localhost/api/v1/dashboard/${DASHBOARD_ID}/permalink`;
const originalLocation = window.location;
let hrefValue = '';
let locationSpy: jest.SpyInstance;
const postDashboardPermalinkMockUrl = `http://localhost/api/v1/dashboard/${DASHBOARD_ID}/permalink`;
beforeEach(() => {
jest.clearAllMocks();
hrefValue = '';
locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
get href() {
return hrefValue;
},
set href(v: string) {
hrefValue = v;
},
} as Location);
// @ts-expect-error
delete window.location;
window.location = { href: '' } as any;
fetchMock.clearHistory().removeRoutes();
fetchMock.post(
postDashboardPermalinkMockUrl,
@@ -71,7 +63,7 @@ beforeEach(() => {
});
afterEach(() => {
locationSpy.mockRestore();
window.location = originalLocation;
window.featureFlags = {};
fetchMock.clearHistory().removeRoutes();
});

View File

@@ -18,18 +18,19 @@
*/
import extractUrlParams from './extractUrlParams';
const originalWindowLocation = window.location;
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
describe('extractUrlParams', () => {
let locationSpy: jest.SpyInstance;
beforeAll(() => {
locationSpy = jest
.spyOn(window, 'location', 'get')
.mockReturnValue({ search: '?edit=true&abc=123' } as Location);
// @ts-expect-error
delete window.location;
// @ts-expect-error
window.location = { search: '?edit=true&abc=123' };
});
afterAll(() => {
locationSpy.mockRestore();
window.location = originalWindowLocation;
});
test('returns all urlParams', () => {

View File

@@ -29,8 +29,9 @@ describe('getChartIdsFromLayout', () => {
},
};
const globalLocation = window.location;
afterEach(() => {
jest.restoreAllMocks();
window.location = globalLocation;
});
test('should encode filters', () => {
@@ -92,11 +93,16 @@ describe('getChartIdsFromLayout', () => {
});
test('should preserve unknown filters', () => {
jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
origin: 'https://localhost',
search: '?unknown_param=value',
} as Location);
const windowSpy = jest.spyOn(window, 'window', 'get');
windowSpy.mockImplementation(
() =>
({
location: {
origin: 'https://localhost',
search: '?unknown_param=value',
},
}) as unknown as Window & typeof globalThis,
);
const urlWithStandalone = getDashboardUrl({
pathname: 'path',
filters: {},
@@ -106,6 +112,7 @@ describe('getChartIdsFromLayout', () => {
expect(urlWithStandalone).toBe(
`path?unknown_param=value&standalone=${DashboardStandaloneMode.HideNav}`,
);
windowSpy.mockRestore();
});
test('should pass through a router-relative pathname unchanged', () => {
@@ -115,18 +122,21 @@ describe('getChartIdsFromLayout', () => {
hash: '',
standalone: DashboardStandaloneMode.HideNav,
});
expect(url).toBe(
`/dashboard/1/?standalone=${DashboardStandaloneMode.HideNav}`,
);
expect(url).toBe(`/dashboard/1/?standalone=${DashboardStandaloneMode.HideNav}`);
});
test('should process native filters key', () => {
jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
origin: 'https://localhost',
search:
'?preselect_filters=%7B%7D&native_filters_key=024380498jdkjf-2094838',
} as Location);
const windowSpy = jest.spyOn(window, 'window', 'get');
windowSpy.mockImplementation(
() =>
({
location: {
origin: 'https://localhost',
search:
'?preselect_filters=%7B%7D&native_filters_key=024380498jdkjf-2094838',
},
}) as unknown as Window & typeof globalThis,
);
const urlWithNativeFilters = getDashboardUrl({
pathname: 'path',
@@ -136,5 +146,6 @@ describe('getChartIdsFromLayout', () => {
expect(urlWithNativeFilters).toBe(
'path?preselect_filters=%7B%7D&native_filters_key=024380498jdkjf-2094838',
);
windowSpy.mockRestore();
});
});

View File

@@ -120,11 +120,6 @@ fetchMock.get('glob:*/api/v1/chart/*', {
});
const defaultPath = '/explore/';
afterEach(() => {
jest.restoreAllMocks();
});
const renderWithRouter = ({
search = '',
overridePathname,
@@ -139,10 +134,11 @@ const renderWithRouter = ({
history?: ReturnType<typeof createMemoryHistory>;
} = {}) => {
const path = overridePathname ?? defaultPath;
jest.spyOn(window, 'location', 'get').mockReturnValue({
pathname: path,
search,
} as Location);
Object.defineProperty(window, 'location', {
get() {
return { pathname: path, search };
},
});
const history =
existingHistory ??
createMemoryHistory({ initialEntries: [`${path}${search}`] });

View File

@@ -46,8 +46,14 @@ jest.mock('src/components/Datasource/components/DatasourceEditor', () => ({
),
}));
let originalLocation: Location;
beforeEach(() => {
originalLocation = window.location;
});
afterEach(() => {
jest.restoreAllMocks();
window.location = originalLocation;
try {
const unmatched = fetchMock.callHistory.calls('unmatched');
@@ -533,10 +539,10 @@ test('should show missing params state', () => {
});
test('should show missing dataset state', () => {
jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?slice_id=152',
} as Location);
// @ts-expect-error - overriding window.location for test
delete window.location;
// @ts-expect-error - overriding window.location for test
window.location = { search: '?slice_id=152' };
const props = createProps({ datasource: fallbackExploreInitialData.dataset });
render(<DatasourceControl {...props} />, { useRedux: true, useRouter: true });
expect(screen.getAllByText(/missing dataset/i)).toHaveLength(2);
@@ -548,10 +554,10 @@ test('should show missing dataset state', () => {
});
test('should show forbidden dataset state', () => {
jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?slice_id=152',
} as Location);
// @ts-expect-error - overriding window.location for test
delete window.location;
// @ts-expect-error - overriding window.location for test
window.location = { search: '?slice_id=152' };
const error = {
error_type: 'TABLE_SECURITY_ACCESS_ERROR',
statusText: 'FORBIDDEN',

View File

@@ -21,24 +21,10 @@ import { VizType } from '@superset-ui/core';
import { getParsedExploreURLParams } from './getParsedExploreURLParams';
const EXPLORE_BASE_URL = 'http://localhost:9000/explore/';
afterEach(() => {
jest.restoreAllMocks();
});
const setupLocation = (newUrl: string) => {
const u = new URL(newUrl);
jest.spyOn(window, 'location', 'get').mockReturnValue({
href: u.href,
pathname: u.pathname,
search: u.search,
hash: u.hash,
origin: u.origin,
host: u.host,
hostname: u.hostname,
port: u.port,
protocol: u.protocol,
} as Location);
delete (window as any).location;
// @ts-expect-error
window.location = new URL(newUrl);
};
test('get form_data_key and slice_id from search params - url when moving from dashboard to explore', () => {

View File

@@ -229,10 +229,10 @@ test('switches to Mine tab correctly', async () => {
test('handles create dashboard button click', async () => {
const assignMock = jest.fn();
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
assign: assignMock,
} as Location);
Object.defineProperty(window, 'location', {
value: { assign: assignMock },
writable: true,
});
render(
<Router history={history}>
@@ -244,7 +244,6 @@ test('handles create dashboard button click', async () => {
const createButton = screen.getByRole('button', { name: /dashboard$/i });
await userEvent.click(createButton);
expect(assignMock).toHaveBeenCalledWith('/dashboard/new');
locationSpy.mockRestore();
});
test('switches to Other tab when available', async () => {

View File

@@ -99,10 +99,11 @@ describe('logger middleware', () => {
test('should include ts, start_offset, event_name, impression_id, source, and source_id in every event', () => {
// Set window.location to include /dashboard/ so the middleware adds dashboard context
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
href: `http://localhost/dashboard/${dashboardId}/`,
} as Location);
const originalHref = window.location.href;
Object.defineProperty(window, 'location', {
value: { href: `http://localhost/dashboard/${dashboardId}/` },
writable: true,
});
try {
const fetchLog = (logger as Function)(mockStore)(next);
@@ -133,7 +134,11 @@ describe('logger middleware', () => {
expect(typeof events[0].ts).toBe('number');
expect(typeof events[0].start_offset).toBe('number');
} finally {
locationSpy.mockRestore();
// Restore original location
Object.defineProperty(window, 'location', {
value: { href: originalHref },
writable: true,
});
}
});

View File

@@ -251,16 +251,23 @@ test('handles special characters in dataset name from URL parameter', async () =
status: 200,
});
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?dataset=flights%C3%86%20test',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: {
...originalLocation,
search: '?dataset=flights%C3%86%20test',
},
writable: true,
});
await renderComponent();
expect(await screen.findByText('flightsÆ test')).toBeInTheDocument();
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
});
});
test('pre-selects the dataset from URL parameter and shows it in dropdown', async () => {
@@ -281,16 +288,20 @@ test('pre-selects the dataset from URL parameter and shows it in dropdown', asyn
status: 200,
});
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?dataset=flights',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { ...originalLocation, search: '?dataset=flights' },
writable: true,
});
await renderComponent();
expect(await screen.findByText('flights')).toBeInTheDocument();
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
});
});
test('shows loading spinner when dataset parameter is present in URL', async () => {
@@ -318,10 +329,11 @@ test('shows loading spinner when dataset parameter is present in URL', async ()
})),
);
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?dataset=flights',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { ...originalLocation, search: '?dataset=flights' },
writable: true,
});
render(
<ChartCreation
@@ -344,7 +356,10 @@ test('shows loading spinner when dataset parameter is present in URL', async ()
expect(screen.queryByRole('status')).not.toBeInTheDocument();
});
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
});
});
test('shows only exact match when loading dataset from URL, not partial matches', async () => {
@@ -391,15 +406,19 @@ test('shows only exact match when loading dataset from URL, not partial matches'
};
});
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?dataset=flights',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { ...originalLocation, search: '?dataset=flights' },
writable: true,
});
await renderComponent();
await screen.findByText('flights');
expect(screen.queryByText('flights_delayed')).not.toBeInTheDocument();
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
});
});

View File

@@ -57,28 +57,28 @@ test('isAllowedScheme allows relative URLs (unparseable as absolute)', () => {
});
test('getTargetUrl reads the url query parameter', () => {
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
search: '?url=https%3A%2F%2Fexample.com%2Fpage',
} as Location);
Object.defineProperty(window, 'location', {
value: { search: '?url=https%3A%2F%2Fexample.com%2Fpage' },
writable: true,
});
expect(getTargetUrl()).toBe('https://example.com/page');
locationSpy.mockRestore();
});
test('getTargetUrl returns empty string when url param is missing', () => {
const locationSpy = jest
.spyOn(window, 'location', 'get')
.mockReturnValue({ search: '' } as Location);
Object.defineProperty(window, 'location', {
value: { search: '' },
writable: true,
});
expect(getTargetUrl()).toBe('');
locationSpy.mockRestore();
});
test('getTargetUrl does not double-decode percent-encoded values', () => {
// %253A is the double-encoding of ":" — after one decode it should remain %3A
const locationSpy = jest
.spyOn(window, 'location', 'get')
.mockReturnValue({ search: '?url=javascript%253Aalert(1)' } as Location);
Object.defineProperty(window, 'location', {
value: { search: '?url=javascript%253Aalert(1)' },
writable: true,
});
expect(getTargetUrl()).toBe('javascript%3Aalert(1)');
locationSpy.mockRestore();
});
test('trustUrl stores and isUrlTrusted retrieves a URL', () => {

View File

@@ -21,21 +21,18 @@ import { availableDomains, allowCrossDomain } from './hostNamesConfig';
// eslint-disable-next-line no-restricted-globals -- TODO: Migrate from describe blocks
describe('hostNamesConfig', () => {
let locationSpy: jest.SpyInstance;
beforeEach(() => {
// Reset DOM
document.body.innerHTML = '';
// Mock window.location
locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
hostname: 'localhost',
search: '',
} as Location);
});
afterEach(() => {
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: {
hostname: 'localhost',
search: '',
},
writable: true,
});
});
test('should export availableDomains as array of strings', () => {

View File

@@ -104,10 +104,15 @@ test('toQueryString should handle special characters in keys and values', () =>
});
test('getDashboardUrlParams should exclude edit parameter by default', () => {
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?edit=true&standalone=false&expand_filters=1',
} as Location);
// Mock window.location.search to include edit parameter
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: {
...originalLocation,
search: '?edit=true&standalone=false&expand_filters=1',
},
writable: true,
});
const urlParams = getDashboardUrlParams(['edit']);
const paramNames = urlParams.map(([key]) => key);
@@ -116,14 +121,20 @@ test('getDashboardUrlParams should exclude edit parameter by default', () => {
expect(paramNames).toContain('standalone');
expect(paramNames).toContain('expand_filters');
locationSpy.mockRestore();
// Restore original location
window.location = originalLocation;
});
test('getDashboardUrlParams should exclude multiple parameters when provided', () => {
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?edit=true&standalone=false&debug=true&test=value',
} as Location);
// Mock window.location.search with multiple parameters
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: {
...originalLocation,
search: '?edit=true&standalone=false&debug=true&test=value',
},
writable: true,
});
const urlParams = getDashboardUrlParams(['edit', 'debug']);
const paramNames = urlParams.map(([key]) => key);
@@ -133,27 +144,36 @@ test('getDashboardUrlParams should exclude multiple parameters when provided', (
expect(paramNames).toContain('standalone');
expect(paramNames).toContain('test');
locationSpy.mockRestore();
// Restore original location
window.location = originalLocation;
});
test('getUrlParam reads from window.location.search by default', () => {
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '?dashboard_page_id=from-window',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { ...originalLocation, search: '?dashboard_page_id=from-window' },
writable: true,
configurable: true,
});
expect(getUrlParam(URL_PARAMS.dashboardPageId)).toBe('from-window');
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
configurable: true,
});
});
test('getUrlParam uses provided search string instead of window.location.search (Safari race condition fix)', () => {
// Simulate Safari race condition: window.location.search is stale (empty),
// but the correct search string is passed in from React Router's useLocation()
const locationSpy = jest.spyOn(window, 'location', 'get').mockReturnValue({
...window.location,
search: '',
} as Location);
const originalLocation = window.location;
Object.defineProperty(window, 'location', {
value: { ...originalLocation, search: '' },
writable: true,
configurable: true,
});
// Without the search override, window.location.search is stale — returns null (the bug)
expect(getUrlParam(URL_PARAMS.dashboardPageId)).toBeNull();
@@ -163,5 +183,9 @@ test('getUrlParam uses provided search string instead of window.location.search
getUrlParam(URL_PARAMS.dashboardPageId, '?dashboard_page_id=correct-id'),
).toBe('correct-id');
locationSpy.mockRestore();
Object.defineProperty(window, 'location', {
value: originalLocation,
writable: true,
configurable: true,
});
});

File diff suppressed because it is too large Load Diff

View File

@@ -16,3 +16,4 @@
# under the License.
Babel==2.9.1
jinja2==3.1.6
polib>=1.2.0

View File

@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

View File

@@ -0,0 +1,290 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Tests for ``scripts/translations/backfill_po.py``.
The script is not installed as a package, so it is loaded via importlib from
its filesystem path. The two units exercised here — ``parse_response`` and
``_apply_translation`` — have enough edge cases (dict/list/scalar responses,
plural vs singular entries, fuzzy flag, attribution comments) to be worth
pinning against regressions.
"""
import importlib.util
import json # noqa: TID251 - testing a standalone script that uses stdlib json
from pathlib import Path
import polib # type: ignore[import-untyped]
import pytest
_SCRIPT_PATH = (
Path(__file__).resolve().parents[4] / "scripts" / "translations" / "backfill_po.py"
)
_spec = importlib.util.spec_from_file_location("backfill_po", _SCRIPT_PATH)
assert _spec is not None, f"Could not load {_SCRIPT_PATH}"
assert _spec.loader is not None, f"No loader on spec for {_SCRIPT_PATH}"
backfill_po = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(backfill_po)
def test_parse_response_singular_strings() -> None:
"""A flat object of int-keyed strings is returned as-is."""
text = '{"0": "hola", "1": "mundo"}'
assert backfill_po.parse_response(text, batch_size=2) == {
0: "hola",
1: "mundo",
}
def test_parse_response_strips_markdown_fences() -> None:
"""Models sometimes wrap JSON in ```json fences; those must be stripped."""
text = '```json\n{"0": "hola"}\n```'
assert backfill_po.parse_response(text, batch_size=1) == {0: "hola"}
def test_parse_response_preserves_plural_dict_as_json() -> None:
"""
Plural entries arrive as nested dicts and must round-trip through
json.loads downstream — str(dict) would emit Python repr (single quotes)
and break parsing in _apply_translation. The serialized form must be
valid JSON.
"""
text = '{"0": {"0": "manzana", "1": "manzanas"}}'
parsed = backfill_po.parse_response(text, batch_size=1)
assert set(parsed.keys()) == {0}
# Must be valid JSON (double-quoted), not Python repr (single-quoted).
assert json.loads(parsed[0]) == {"0": "manzana", "1": "manzanas"}
def test_parse_response_preserves_non_ascii() -> None:
"""ensure_ascii=False keeps non-ASCII characters readable in the .po file."""
text = '{"0": {"0": "日本語", "1": "日本語s"}}'
parsed = backfill_po.parse_response(text, batch_size=1)
assert "日本語" in parsed[0]
def test_parse_response_skips_non_numeric_keys() -> None:
"""Keys that are not numeric strings are silently skipped."""
text = '{"0": "ok", "comment": "ignored", "2": "kept"}'
assert backfill_po.parse_response(text, batch_size=3) == {
0: "ok",
2: "kept",
}
@pytest.mark.parametrize(
"raw",
['["hola", "mundo"]', '"just a string"', "null", "42"],
)
def test_parse_response_rejects_non_object(raw: str) -> None:
"""
Non-object JSON (list, string, null, number) must raise ValueError so
_process_batches catches it instead of crashing on AttributeError from
.items().
"""
with pytest.raises(ValueError, match="Expected a JSON object"):
backfill_po.parse_response(raw, batch_size=1)
def test_parse_response_rejects_invalid_json() -> None:
"""Garbage input surfaces as ValueError, not the underlying JSONDecodeError."""
with pytest.raises(ValueError, match="Could not parse response as JSON"):
backfill_po.parse_response("not even close to json", batch_size=1)
# ---------------------------------------------------------------------------
# _apply_translation
# ---------------------------------------------------------------------------
def _make_singular_entry(msgid: str = "Hello") -> polib.POEntry:
return polib.POEntry(msgid=msgid, msgstr="")
def _make_plural_entry(
msgid: str = "%(n)s apple",
msgid_plural: str = "%(n)s apples",
) -> polib.POEntry:
entry = polib.POEntry(msgid=msgid, msgid_plural=msgid_plural)
entry.msgstr_plural = {0: "", 1: ""}
return entry
def _item(refs: list[str] | None = None) -> dict[str, list[str]]:
return {"context_langs": refs if refs is not None else ["fr", "de"]}
def test_apply_translation_singular_writes_msgstr_and_marks_fuzzy() -> None:
entry = _make_singular_entry()
backfill_po._apply_translation(
entry, "Hola", _item(["fr", "de"]), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr == "Hola"
assert "fuzzy" in entry.flags
def test_apply_translation_singular_no_fuzzy_when_disabled() -> None:
entry = _make_singular_entry()
backfill_po._apply_translation(
entry, "Hola", _item(), model="claude-test", mark_fuzzy=False
)
assert "fuzzy" not in entry.flags
def test_apply_translation_attribution_includes_refs() -> None:
entry = _make_singular_entry()
backfill_po._apply_translation(
entry, "Hola", _item(["fr", "de"]), model="claude-test", mark_fuzzy=True
)
assert "Machine-translated via backfill_po.py (claude-test)" in entry.tcomment
assert "[refs: fr, de]" in entry.tcomment
def test_apply_translation_attribution_marks_no_refs() -> None:
entry = _make_singular_entry()
backfill_po._apply_translation(
entry, "Hola", _item([]), model="claude-test", mark_fuzzy=True
)
assert "[no refs]" in entry.tcomment
def test_apply_translation_attribution_appended_not_duplicated() -> None:
"""Re-running on an already-translated entry must not duplicate attribution."""
entry = _make_singular_entry()
entry.tcomment = "Existing maintainer note"
backfill_po._apply_translation(
entry, "Hola", _item(["fr"]), model="claude-test", mark_fuzzy=True
)
# Existing comment preserved, attribution appended.
assert entry.tcomment.startswith("Existing maintainer note\n")
assert "Machine-translated via backfill_po.py" in entry.tcomment
# Apply again — attribution must not duplicate.
backfill_po._apply_translation(
entry, "Hola", _item(["fr"]), model="claude-test", mark_fuzzy=True
)
assert entry.tcomment.count("Machine-translated via backfill_po.py") == 1
def test_apply_translation_plural_dict_response() -> None:
"""A JSON-dict response writes each plural form to msgstr_plural."""
entry = _make_plural_entry()
translation = json.dumps({"0": "manzana", "1": "manzanas"})
backfill_po._apply_translation(
entry, translation, _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzanas"}
assert "fuzzy" in entry.flags
def test_apply_translation_plural_scalar_json_fills_all_forms() -> None:
"""
A JSON-scalar response (e.g. ``"hola"``) is broadcast to every plural form.
This is the documented fallback when the model returns a single string for
a plural entry.
"""
entry = _make_plural_entry()
backfill_po._apply_translation(
entry, '"manzana"', _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzana"}
def test_apply_translation_plural_invalid_json_fills_all_forms() -> None:
"""
A non-JSON string also broadcasts to every plural form (rather than
crashing). This handles older models that ignore the JSON instruction.
"""
entry = _make_plural_entry()
backfill_po._apply_translation(
entry, "manzana", _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzana"}
def test_apply_translation_plural_round_trip_from_parse_response() -> None:
"""
End-to-end guard: the JSON string produced by parse_response for a plural
entry must be consumable by _apply_translation without losing forms. This
is the regression that #39448 fixed (str(dict) → Python repr broke the
round-trip).
"""
raw = '{"0": {"0": "manzana", "1": "manzanas"}}'
parsed = backfill_po.parse_response(raw, batch_size=1)
entry = _make_plural_entry()
backfill_po._apply_translation(
entry, parsed[0], _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzanas"}
def test_apply_translation_plural_list_response() -> None:
"""
Models sometimes return a JSON array for plural forms (forms are ordered,
so a list is a valid representation). Each element must map to the
corresponding plural index. Without this branch, ``str(list)`` would emit
Python list-repr and broadcast it to every form — observed in the wild
on a fresh run for French.
"""
entry = _make_plural_entry()
translation = json.dumps(["manzana", "manzanas"])
backfill_po._apply_translation(
entry, translation, _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzanas"}
def test_apply_translation_plural_list_round_trip_from_parse_response() -> None:
"""
The list-of-forms response must also survive parse_response → _apply
round-trip. parse_response JSON-serializes lists; _apply_translation
must json.loads them back into a list and distribute across forms.
"""
raw = '{"0": ["manzana", "manzanas"]}'
parsed = backfill_po.parse_response(raw, batch_size=1)
entry = _make_plural_entry()
backfill_po._apply_translation(
entry, parsed[0], _item(), model="claude-test", mark_fuzzy=True
)
assert entry.msgstr_plural == {0: "manzana", 1: "manzanas"}
def test_apply_translation_plural_list_shorter_repeats_last_form() -> None:
"""
If the model returns fewer forms than the language requires, repeat the
last form rather than leaving slots empty (which would render as the
literal English msgid via gettext fallback).
"""
entry = polib.POEntry(msgid="apple", msgid_plural="apples")
entry.msgstr_plural = {0: "", 1: "", 2: ""}
backfill_po._apply_translation(
entry,
json.dumps(["uno", "dos"]),
_item(),
model="claude-test",
mark_fuzzy=True,
)
assert entry.msgstr_plural == {0: "uno", 1: "dos", 2: "dos"}
def test_apply_translation_plural_empty_list_falls_back_to_string_broadcast() -> None:
"""An empty JSON list isn't usable; fall back to writing the raw string."""
entry = _make_plural_entry()
backfill_po._apply_translation(
entry, "[]", _item(), model="claude-test", mark_fuzzy=True
)
# Falls into the JSONDecodeError/ValueError branch → broadcast raw string.
assert entry.msgstr_plural == {0: "[]", 1: "[]"}

View File

@@ -0,0 +1,256 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Tests for ``scripts/translations/build_translation_index.py``.
The script is not installed as a package, so it is loaded via importlib
from its filesystem path. The units exercised here pin the cross-language
index shape that the AI backfill prompt depends on: fuzzy entries must be
excluded (so unreviewed drafts don't feed back as trusted context), every
entry must have a slot for every language (null when missing), and plural
entries must be keyed by the ``msgid\\x00msgid_plural`` composite.
"""
import importlib.util
from pathlib import Path
import polib # type: ignore[import-untyped]
import pytest
_SCRIPT_PATH = (
Path(__file__).resolve().parents[4]
/ "scripts"
/ "translations"
/ "build_translation_index.py"
)
_spec = importlib.util.spec_from_file_location("build_translation_index", _SCRIPT_PATH)
assert _spec is not None, f"Could not load {_SCRIPT_PATH}"
assert _spec.loader is not None, f"No loader on spec for {_SCRIPT_PATH}"
build_translation_index = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(build_translation_index)
# ---------------------------------------------------------------------------
# _is_translated
# ---------------------------------------------------------------------------
def test_is_translated_empty_singular() -> None:
entry = polib.POEntry(msgid="Hello", msgstr="")
assert build_translation_index._is_translated(entry) is False
def test_is_translated_populated_singular() -> None:
entry = polib.POEntry(msgid="Hello", msgstr="Hola")
assert build_translation_index._is_translated(entry) is True
def test_is_translated_fuzzy_entry_is_not_trusted() -> None:
"""
Fuzzy entries are unreviewed (often AI-generated drafts). They must not
count as translated, or backfill runs will feed their own prior output
back into the prompt as trusted context.
"""
entry = polib.POEntry(msgid="Hello", msgstr="Hola", flags=["fuzzy"])
assert build_translation_index._is_translated(entry) is False
def test_is_translated_plural_any_form_counts() -> None:
entry = polib.POEntry(msgid="apple", msgid_plural="apples")
entry.msgstr_plural = {0: "manzana", 1: ""}
assert build_translation_index._is_translated(entry) is True
def test_is_translated_plural_all_empty() -> None:
entry = polib.POEntry(msgid="apple", msgid_plural="apples")
entry.msgstr_plural = {0: "", 1: ""}
assert build_translation_index._is_translated(entry) is False
def test_is_translated_plural_fuzzy_is_not_trusted() -> None:
entry = polib.POEntry(msgid="apple", msgid_plural="apples", flags=["fuzzy"])
entry.msgstr_plural = {0: "manzana", 1: "manzanas"}
assert build_translation_index._is_translated(entry) is False
# ---------------------------------------------------------------------------
# _plural_key
# ---------------------------------------------------------------------------
def test_plural_key_uses_null_byte_separator() -> None:
"""The composite key must use \\x00 so it cannot collide with any msgid."""
entry = polib.POEntry(msgid="apple", msgid_plural="apples")
assert build_translation_index._plural_key(entry) == "apple\x00apples"
# ---------------------------------------------------------------------------
# build_index
# ---------------------------------------------------------------------------
def _write_po(path: Path, entries: list[polib.POEntry]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
po = polib.POFile()
po.metadata = {
"Content-Type": "text/plain; charset=UTF-8",
"Content-Transfer-Encoding": "8bit",
}
for entry in entries:
po.append(entry)
po.save(str(path))
@pytest.fixture
def translations_dir(tmp_path: Path) -> Path:
"""
Build a minimal translations directory with three languages:
- es: has "Hello" translated, "World" missing, plural translated
- fr: has "Hello" fuzzy (must be treated as missing), "World" translated
- en: source locale — must be excluded from the index
"""
root = tmp_path / "translations"
_write_po(
root / "es" / "LC_MESSAGES" / "messages.po",
[
polib.POEntry(msgid="Hello", msgstr="Hola"),
polib.POEntry(msgid="World", msgstr=""),
_plural_entry("apple", "apples", {0: "manzana", 1: "manzanas"}),
],
)
_write_po(
root / "fr" / "LC_MESSAGES" / "messages.po",
[
polib.POEntry(msgid="Hello", msgstr="Bonjour", flags=["fuzzy"]),
polib.POEntry(msgid="World", msgstr="Monde"),
_plural_entry("apple", "apples", {0: "", 1: ""}),
],
)
_write_po(
root / "en" / "LC_MESSAGES" / "messages.po",
[polib.POEntry(msgid="Hello", msgstr="")],
)
return root
def _plural_entry(
msgid: str, msgid_plural: str, plurals: dict[int, str]
) -> polib.POEntry:
entry = polib.POEntry(msgid=msgid, msgid_plural=msgid_plural)
entry.msgstr_plural = plurals
return entry
def test_build_index_excludes_en(translations_dir: Path) -> None:
"""``en`` is the source locale and must never appear in the index."""
index = build_translation_index.build_index(translations_dir)
for value in index.values():
assert "en" not in value
def test_build_index_records_singular_translations(translations_dir: Path) -> None:
index = build_translation_index.build_index(translations_dir)
assert index["Hello"]["es"] == "Hola"
assert index["World"]["fr"] == "Monde"
def test_build_index_fuzzy_entries_become_null(translations_dir: Path) -> None:
"""
Fuzzy translations must surface as null. If they leaked through as text,
they would (a) feed unreviewed AI output back into the backfill prompt
as trusted context, and (b) inflate the --min-context count past the
threshold of real reviewed translations.
"""
index = build_translation_index.build_index(translations_dir)
assert index["Hello"]["fr"] is None
def test_build_index_missing_translations_become_null(
translations_dir: Path,
) -> None:
"""Empty msgstr → null (not empty string)."""
index = build_translation_index.build_index(translations_dir)
assert index["World"]["es"] is None
def test_build_index_fills_every_language_slot(translations_dir: Path) -> None:
"""
Every msgid must have a slot for every non-en language, even if that
language's .po file did not contain the entry. Defaults to null.
"""
index = build_translation_index.build_index(translations_dir)
expected_langs = {"es", "fr"}
for key, value in index.items():
assert set(value.keys()) == expected_langs, (
f"{key!r} missing language slots: {set(value.keys())}"
)
def test_build_index_plural_uses_composite_key(translations_dir: Path) -> None:
"""Plural entries must be keyed by ``msgid\\x00msgid_plural``."""
index = build_translation_index.build_index(translations_dir)
assert "apple\x00apples" in index
assert "apple" not in index # not stored under bare msgid
def test_build_index_plural_translated_stored_as_dict(
translations_dir: Path,
) -> None:
index = build_translation_index.build_index(translations_dir)
plural = index["apple\x00apples"]
assert plural["es"] == {0: "manzana", 1: "manzanas"}
def test_build_index_plural_untranslated_stored_as_null(
translations_dir: Path,
) -> None:
"""Empty plural forms across the board → null, not an empty dict."""
index = build_translation_index.build_index(translations_dir)
plural = index["apple\x00apples"]
assert plural["fr"] is None
def test_build_index_skips_languages_without_messages_po(tmp_path: Path) -> None:
"""
A subdirectory that doesn't contain ``LC_MESSAGES/messages.po`` (e.g.
leftover scratch dirs, dotfiles) must not be picked up as a language.
"""
root = tmp_path / "translations"
_write_po(
root / "es" / "LC_MESSAGES" / "messages.po",
[polib.POEntry(msgid="Hello", msgstr="Hola")],
)
(root / "scratch").mkdir() # no LC_MESSAGES/messages.po
(root / ".DS_Store").touch()
index = build_translation_index.build_index(root)
assert index == {"Hello": {"es": "Hola"}}
def test_build_index_skips_header_entry(tmp_path: Path) -> None:
"""
The .po header entry has an empty msgid by convention. It must not be
included as a translation key.
"""
root = tmp_path / "translations"
_write_po(
root / "es" / "LC_MESSAGES" / "messages.po",
[polib.POEntry(msgid="Hello", msgstr="Hola")],
)
index = build_translation_index.build_index(root)
assert "" not in index