Compare commits

..

276 Commits

Author SHA1 Message Date
Michael S. Molina
0c05300ce5 fix: Missing argument 2022-09-14 18:02:00 -03:00
Michael S. Molina
67e8b9912c Updates CHANGELOG.md 2022-09-14 17:50:14 -03:00
Mayur
7d70ea1235 fix: dashboard filter value is cleared when 2 similar dashboards opened in succession (#21461)
(cherry picked from commit 59ca7861c0)
2022-09-14 15:09:12 -03:00
Bogdan
f3153760cd perf: Memoize the common_bootstrap_payload and include user param (#21018) (#21439)
Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
2022-09-14 15:08:49 -03:00
Daniel Vaz Gaspar
93611064b4 fix: cached common bootstrap Revert (#21018) (#21419)
(cherry picked from commit 094400c308)
2022-09-14 14:55:23 -03:00
Michael S. Molina
79cb268279 fix: Test host reference 2022-09-07 09:08:09 -03:00
Michael S. Molina
cca31299d3 Updates CHANGELOG.md 2022-09-06 14:36:34 -03:00
Michael S. Molina
37bdf434bf fix: Connect database from top right corner button doesn't work 2022-09-06 14:13:32 -03:00
Daniel Vaz Gaspar
8e7fb96f06 fix: disallow users from viewing other user's profile on config (#21302) 2022-09-06 10:24:03 -03:00
Daniel Vaz Gaspar
85fb2433ab feat: adds TLS certificate validation option for SMTP (#21272)
(cherry picked from commit 9fd752057e)
2022-09-06 10:12:01 -03:00
ʈᵃᵢ
9861c2b9e7 fix(celery cache warmup): add auth and use warm_up_cache endpoint (#21076)
(cherry picked from commit 04dd8d414d)
2022-09-06 10:12:01 -03:00
Michael S. Molina
7ad097371e fix: Adds yeoman-environment to dev dependencies 2022-08-31 09:44:07 -03:00
Michael S. Molina
44e3d60c35 fix: Overrides omnibar React dependency 2022-08-31 08:47:48 -03:00
Jesse Yang
52e1349acc chore(build): upgrade less-loader (#19703) 2022-08-30 18:36:39 -03:00
Michael S. Molina
5a3bb82b8d Fixes lint errors 2022-08-30 15:28:07 -03:00
Michael S. Molina
a50b0a91fc chore: Updates CHANGELOG.md and package.json 2022-08-30 09:53:15 -03:00
nisheldo
fb0d6d57d6 chore: updating python docker image to 3.8.13 (#20550)
(cherry picked from commit 7275805e95)
2022-08-30 09:36:59 -03:00
Ilyas
5f3301aadb fix: exporting CSV can't apply pagination #17861 (#20178) 2022-08-30 09:35:41 -03:00
Diego Medina
e5de0909c3 fix: Support the Clipboard API in modern browsers (#20058)
* fix: Support the Clipboard API in modern browsers

* fix tests

* PR comment

* Improvements
2022-08-29 15:28:09 -03:00
Bogdan
6bcb9674da Memoize the common_bootstrap_payload (#21018)
Try patch

Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
(cherry picked from commit 495a205dec)
2022-08-29 14:26:51 -03:00
Bogdan
1e8259a410 perf: Implement model specific lookups by id to improve performance (#20974)
* Implement model specific lookups by id to improve performance

* Address comments e.g. better variable names and test cleanup

* commit after cleanup

* even better name and test cleanup via rollback

Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
2022-08-29 14:26:18 -03:00
Simon Thelin
094b17e8cc fix(20428): Address-Presto/Trino-Poll-Issue-Refactor (#20434)
* fix(20428)-Address-Presto/Trino-Poll-Issue-Refacto
r

Update linter

* Update to only use BaseEngineSpec handle_cursor

* Fix CI

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>
2022-08-29 11:24:07 -03:00
Multazim Deshmukh
991a453ff2 fix(docker): Make Gunicorn Keepalive Adjustable (#20348)
Co-authored-by: Multazim Deshmukh <multazim.deshmukh@morningstar.com>
(cherry picked from commit 86368dd406)
2022-06-24 17:43:14 -03:00
Stephen Liu
8e1b2a1354 fix(chart): chart gets cut off on the dashboard (#20315)
* fix(chart): chart gets cut off on the dashboard

* add some failsafe

* address comment

(cherry picked from commit 07b4a7159d)
2022-06-24 17:43:14 -03:00
Ensky
6dee53682e Fixes #20155 (#20273)
(cherry picked from commit 77e326fd95)
2022-06-24 17:43:14 -03:00
Diego Medina
4073b58aa2 fix: Box Plot Chart throws an error when the average (AVG) / SUM is being calculated on the Metrics (#20235)
* fix: Box Plot Chart throws an error when the average (AVG) / SUM is being calculated on the Metrics

* add test

(cherry picked from commit 8638f59b4c)
2022-06-24 15:19:33 -03:00
John Bodley
d512e89aa9 fix(csv): Ensure df_to_escaped_csv handles NULL (#20151)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
(cherry picked from commit 97ce920d49)
2022-06-24 15:19:33 -03:00
Diego Medina
f8a369d233 fix: avoid while cycle in computeMaxFontSize for big Number run forever when css rule applied (#20173)
(cherry picked from commit 365acee663)
2022-06-24 15:19:33 -03:00
Beto Dealmeida
af5ded3fcb fix: sqloxide optional (#19570) 2022-06-24 15:18:21 -03:00
Michael S. Molina
48f3eb4273 Updates CHANGELOG 2022-05-27 12:22:43 -03:00
Ville Brofeldt
c3604dc4de fix: login button does not render (#19685)
* fix: login button does not render

* add type guard
2022-05-27 10:14:21 -03:00
Michael S. Molina
17e10b77ca Updates version and CHANGELOG 2022-05-26 10:36:35 -03:00
Ville Brofeldt
8089afce1e fix(temporary-cache): when user is anonymous (#20181)
* fix(temporary-cache): fail on anonymous user

* make exceptions generic

* fix test

* remove redundant bool return

* fix unit tests

(cherry picked from commit 64c4226817)
2022-05-26 10:28:14 -03:00
Ramunas Balukonis
d652e0cab2 fix: "Week Staring Monday" time grain for BigQuery (#20091)
* Week Staring From Monday added for BQ

* lint fix

(cherry picked from commit 694f75d376)
2022-05-26 10:28:14 -03:00
cccs-tom
562e3a769b fix: Allow dataset owners to see their datasets (#20135)
(cherry picked from commit b0c6935f06)
2022-05-26 10:28:14 -03:00
Ville Brofeldt
c5e2809c2c fix(presto,trino): use correct literal dttm separator (#20123)
* fix(presto,trino): use correct literal dttm separator

* remove redundant tests

(cherry picked from commit e2f11d3680)
2022-05-26 10:28:14 -03:00
Ville Brofeldt
97c6af196e fix(generic-axes): apply contribution before flatten (#20077)
(cherry picked from commit d5802f7896)
2022-05-26 10:28:14 -03:00
thinhnd2104
bda9044bfd fix: Athena timestamp literal format (#19970)
* Fix Athena timestamp literal format

related: [#19969]

* Update test_athena.py

Update test

* Update athena.py

* Resolving CI/CD

* lint

Co-authored-by: Thinh Nguyen Duc <thinhnd.it@tripi.vn>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
(cherry picked from commit 85315460cb)
2022-05-26 10:28:13 -03:00
Ville Brofeldt
4e44f4cc11 fix(plugin-chart-echarts): support adhoc x-axis (#20055)
* fix(plugin-chart-echarts): support adhoc x-axis

* simplify code

(cherry picked from commit b53daa91ec)
2022-05-26 10:28:13 -03:00
Yeachan Park
8becd3e080 fix(sqllab/charts): casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp (#18873)
* fix casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp from sqllab and charts

* Add unittests

* Lint changes and parameter variable rename

* Fix linting

(cherry picked from commit 8b72354654)
2022-05-26 10:28:13 -03:00
Ville Brofeldt
9ca53b8905 fix(sqla): replace custom dttm type with literal_column (#19917)
(cherry picked from commit 99f1f9ec61)
2022-05-26 10:28:13 -03:00
Hugh A. Miles II
2bd89d1705 fix: Alpha should not be able to edit datasets that they don't own (#19854)
* fix api for checking owners

* fix styles for disabling

* fix styles for disabling

* fix lint

* fix lint

* add owners key

* plzz

* remove

* update test

* add tooltip

* add type

* fix test

* fix user reference

* lit

* fix test

* work

(cherry picked from commit 8b15b68979)
2022-05-26 10:28:13 -03:00
Ville Brofeldt
94d4179045 update changelog entries 2022-05-26 14:47:45 +03:00
Ville Brofeldt
054e6b2334 update changelog 2022-04-25 10:55:40 +03:00
Diego Medina
397a182c2f fix(sql lab): when editing a saved query, the status is lost when switching tabs (#19448)
(cherry picked from commit 800ced5e25)
2022-04-25 10:46:54 +03:00
Ville Brofeldt
1050fcbd3c fix(key_value): use longblob on mysql (#19805)
* fix(key_value): use longblob on mysql

* set length

(cherry picked from commit a1bd5b283c)
2022-04-25 10:46:54 +03:00
Kamil Gabryjelski
387138eb9e Fix display of column config in table chart (#19806)
(cherry picked from commit 12bc30e2c7)
2022-04-25 10:46:53 +03:00
Yongjie Zhao
4fa96d3a3b fix: lost renameOperator in mixed timeseries chart (#19802)
(cherry picked from commit 108a2a4eaf)
2022-04-25 10:46:53 +03:00
smileydev
5567828ee8 fix(chart & explore): Show labels for SliderControl (#19765)
* fix(chart & explore): make to show label of slidercontrol

* fix(chart & explore): make to update SliderControl props

* fix(chart & explore): make to fix lint

(cherry picked from commit dfbaba97c6)
2022-04-25 10:46:53 +03:00
Yongjie Zhao
8c8bbfb89f feat: add renameOperator (#19776)
(cherry picked from commit 3c28cd4625)
2022-04-25 10:46:53 +03:00
Cedric Gampert
2b6e35e039 fix: dashboard standalone class not added when parameter set (#16619)
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
(cherry picked from commit 5134c63ae2)
2022-04-25 10:46:53 +03:00
Diego Medina
1d141be463 fix: SQL Lab UI Error: Objects are not valid as a React child (#19783)
(cherry picked from commit dfba9ea596)
2022-04-25 10:46:52 +03:00
Diego Medina
87b51c26f4 fix(sql lab): replace the output column in the query history table (#19370)
* fix(sql lab): replace the output column in the query history table

* improvements

(cherry picked from commit d4e7c65f6a)
2022-04-25 10:46:52 +03:00
Ville Brofeldt
c9e3ca11e2 fix(dashboard): copy permalink to dashboard chart (#19772)
* fix(dashboard): copy permalink to dashboard chart

* lint

* address comments

(cherry picked from commit e061955fd0)
2022-04-25 10:43:09 +03:00
smileydev
5df14ed8f9 fix(explore): make to show the null value as N/A in view result (#19603)
* fix(explore): make to show the null value as N/A in view result

* fix(explore): make to remove console

* fix(explore): make to remove console in Cell

* fix(explore): make to translate N/A

(cherry picked from commit 34323f9b5f)
2022-04-25 10:43:09 +03:00
Ville Brofeldt
2d7d2dd373 fix(permalink): remove memoize on get salt func (#19749)
(cherry picked from commit cf5145918b)
2022-04-25 10:43:09 +03:00
Kamil Gabryjelski
28aa69628f chore: Clean redundant dependency from useMemo dep array (#19732)
(cherry picked from commit b7759e6fd1)
2022-04-25 10:35:31 +03:00
Michael S. Molina
f8b202f670 fix: Filter dependencies are not being applied in default values (#19698)
(cherry picked from commit 7bc9123fe5)
2022-04-25 10:35:31 +03:00
Ville Brofeldt
ccf296b786 update changelog 2022-04-15 15:11:31 +03:00
Jesse Yang
90b08fa095 chore: skip SIP-68 shadow writing for LTS 2022-04-15 14:52:25 +03:00
smileydev
cdc136c70a fix(dnd&column): make to fix the blank state issue when only one column select (#19651)
(cherry picked from commit c320c295dc)
2022-04-15 14:52:11 +03:00
Yongjie Zhao
98c4d943da fix: drop the first level of MultiIndex (#19716)
(cherry picked from commit 9425dd2cac)
2022-04-15 14:52:10 +03:00
Kamil Gabryjelski
3c8e65960f fix(dashboard): Fix BigNumber causing dashboard to crash when overflowing (#19688)
* fix(dashboard): fix(plugin-chart-echarts): Fix BigNumber causing dashboard to crash when overflowing

* Add tooltips for truncated titles

* Fix type

(cherry picked from commit ee85466f2e)
2022-04-15 14:52:10 +03:00
Ville Brofeldt
d657d813e3 fix(sql-lab): do not replace undefined schema with empty object (#19664)
(cherry picked from commit 87d47987b7)
2022-04-15 14:52:10 +03:00
Yongjie Zhao
95e5b59c38 fix: time comparision (#19659)
(cherry picked from commit d7dd4119d4)
2022-04-15 14:52:10 +03:00
Ville Brofeldt
419316e84a fix tests 2022-04-11 09:49:29 +03:00
Ville Brofeldt
13e81c683e update changelog 2022-04-08 18:19:18 +03:00
Ville Brofeldt
77d6207bed docs: release notes for 1.5 (#19530)
* docs: release notes for 1.5

* rephrase and remove secret key from 1.4

* address review comments

(cherry picked from commit 1c1beb653a)
2022-04-08 16:47:07 +03:00
Ville Brofeldt
ecaecf0b6c fix(select): render when empty multiselect (#19612)
* fix(select): render when empty multiselect

* disable flaky test

(cherry picked from commit 1ad82af058)
2022-04-08 16:41:34 +03:00
Ville Brofeldt
280ecab0e6 chore: remove redundant adodbapi warning (#19557)
(cherry picked from commit 0d331f5bd8)
2022-04-08 16:41:33 +03:00
Ville Brofeldt
5ca126698a fix(dataset): handle missing python_type gracefully (#19553)
* fix(dataset): handle missing python_type gracefully

* refactor TEMPORAL_TYPES

(cherry picked from commit d9343a4639)
2022-04-08 16:41:33 +03:00
Ville Brofeldt
44eb81e35f fix(sqla): apply jinja to metrics (#19565)
(cherry picked from commit 34b55765c4)
2022-04-08 16:41:33 +03:00
Ville Brofeldt
1e567999d9 fix(dataset): avoid crash if database missing (#19582)
(cherry picked from commit db2135109a)
2022-04-08 16:41:33 +03:00
Yongjie Zhao
d99a9a4134 fix: big number with trendline can't calculate cumsum (#19542)
(cherry picked from commit 2daa071633)
2022-04-08 16:41:32 +03:00
Ville Brofeldt
c4d24a09a8 fix(sqllab): null database with backend persistence (#19548)
(cherry picked from commit 2d81c4c79f)
2022-04-08 16:41:32 +03:00
Ville Brofeldt
34e3119c40 fix(migrations): reorder skipped 1.4 migrations (#19543)
(cherry picked from commit e1ef2baad7)
2022-04-08 16:41:32 +03:00
Ville Brofeldt
031fadbed8 chore(releasing): use node 16 for testing and remove redundant updating note (#19505)
* chore(releasing): use node 16 for testing

* remove note about required cache
2022-04-04 13:43:26 +03:00
Ville Brofeldt
cc51a84c26 fix bad conflict resolution 2022-04-04 11:58:52 +03:00
Ville Brofeldt
2001fb5037 add noop endpoint migration 2022-04-04 11:20:29 +03:00
Ville Brofeldt
96901d6c46 update changelog, updating, version 2022-04-04 11:11:19 +03:00
dudasaron
dba4610f9b fix(dashboard list): do not show favorite star for anonymous users #18210 (#19409)
* fix: Only show favorite star on dashboard list if user is logged in #18210

* Fix linter errors

(cherry picked from commit b8891acf4a)
2022-04-04 11:05:38 +03:00
Ville Brofeldt
f44ed063e8 chore(plugin-chart-echarts): upgrade echarts 5.3.2 (#19481)
(cherry picked from commit 11bf0d09cb)
2022-04-04 11:00:48 +03:00
Yongjie Zhao
90f49e261e fix: flaky test for dashboard changed on (#19483)
(cherry picked from commit a2e921bd03)
2022-04-04 11:00:48 +03:00
Yongjie Zhao
3ff9cdeb65 fix: can't sync temporal flag on virtual table (#19366)
(cherry picked from commit d954c3df86)
2022-04-04 11:00:48 +03:00
Ville Brofeldt
3c627bdc61 fix: request samples with default row limit (#19456)
* fix: request samples with default row limit

* lodashLint

* fix cypress test

(cherry picked from commit d684ad073d)
2022-04-04 11:00:47 +03:00
Jinghuayao
3eece91378 test(native filter): add new test for dependent filter (#19392)
* add new test for dependent filter

(cherry picked from commit 2a75e4c3c3)
2022-04-04 11:00:47 +03:00
Diego Medina
af0dc75207 feat(sql lab): enable ACE editor search in SQL editors (#19328)
(cherry picked from commit eab9388f7c)
2022-04-04 11:00:47 +03:00
Beto Dealmeida
840be9972f feat: improve adhoc SQL validation (#19454)
* feat: improve adhoc SQL validation

* Small changes

* Add more unit tests

(cherry picked from commit 6828624f61)
2022-04-04 11:00:47 +03:00
James Turton
e4cbbdc653 fix(drill): specify an SA URL parm of impersonation_target for drill+sadrill (#19252)
* Update drill+sadrill to specify an SA URL parm of "impersonation_target".

Sqlalchemy-drill is being updated to support impersonation with the
drill+sadrill driver, where previously it did not.  The way that callers
should specify impersonation matches that for the drill+jdbc driver in that
a SA URL parameter of impersonation_target should be set to the username
of the user to be impersonated, while the stadard SA username and password
should be those of the proxy user.

* Remove lint.

* Address review comments.

* Use idiomatic pytest to test for a raised exception.

* Fix import statement order in drill.py.

(cherry picked from commit 85e330e94b)
2022-04-04 11:00:46 +03:00
Michael S. Molina
a96ff005fd refactor: Removes the CSS files from the Heatmap plugin (#19417)
* refactor: Removes the CSS files from the Heatmap plugin

* Adds peer dependencies

(cherry picked from commit ac6b2f2d93)
2022-04-04 11:00:46 +03:00
Michael S. Molina
70d800dc27 refactor: Removes the CSS files from the Calendar Heatmap plugin (#19436)
(cherry picked from commit 13467beb53)
2022-04-04 11:00:46 +03:00
Michael S. Molina
99c8f9bd13 chore: Removes plugin-chart-pivot-table hard-coded colors (#19439)
(cherry picked from commit 31095a523d)
2022-04-04 11:00:46 +03:00
Michael S. Molina
91bf9bd68b chore: Removes hard-coded colors from legacy-preset-chart-nvd3 (#19443)
(cherry picked from commit 47abad8673)
2022-04-04 11:00:45 +03:00
smileydev
e93d64d58e fix(dashboard): make to show the correct owned objects (#19372)
* fix(dashboard): make to show the correct owned objects

* fix(dashboard): make to filter is reusable

* fix(homepage): make sure the type Array<Filters>

* fix(homepage): make to display dashboard correctly by owners

(cherry picked from commit 6d89ffbcb7)
2022-04-04 11:00:45 +03:00
Yongjie Zhao
3c09690ed2 fix: can't save dataset modal (#19452)
(cherry picked from commit 6b136c2bc9)
2022-04-04 11:00:45 +03:00
David Aaron Suddjian
795ed3c719 feat: Embedded dashboard configuration (#19364)
* embedded dashboard model

* embedded dashboard endpoints

* DRY up using the with_dashboard decorator elsewhere

* wip

* check feature flags and permissions

* wip

* sdk

* urls

* dao option for id column

* got it working

* Update superset/embedded/view.py

* use the curator check

* put back old endpoint, for now

* allow access by either embedded.uuid or dashboard.id

* keep the old endpoint around, for the time being

* openapi

* lint

* lint

* lint

* test stuff

* lint, test

* typo

* Update superset-frontend/src/embedded/index.tsx

* Update superset-frontend/src/embedded/index.tsx

* fix tests

* bump sdk

(cherry picked from commit 8e29ec5a66)
2022-04-04 11:00:45 +03:00
Ville Brofeldt
a7ee677154 docs: add 1.4.2 changelog entries (#19411)
(cherry picked from commit a4c261d72c)
2022-04-04 11:00:44 +03:00
Michael S. Molina
0c78522bfe chore: Disables no literal colors for the theme configuration (#19437)
(cherry picked from commit 193744842b)
2022-04-04 11:00:44 +03:00
Daniel Vaz Gaspar
bfa203aee0 fix: missing init on importexport pkg (#19435)
(cherry picked from commit 2451937097)
2022-04-04 11:00:44 +03:00
Daniel Vaz Gaspar
7aba89c486 fix: add missing init on python pkg key_value (#19428)
* fix: add missing init on python pkg key_value

* fix lint issues

* fix lint issues

(cherry picked from commit fa35109bf2)
2022-04-04 11:00:44 +03:00
Srini Kadamati
625555ac7e chore: attempt to force docs deploy (#19424)
(cherry picked from commit a8e7624eb5)
2022-04-04 11:00:44 +03:00
Beto Dealmeida
038d114b07 perf: improve perf in SIP-68 migration (#19416)
* chore: improve perf in SIP-68 migration

* Small fixes

* Create tables referenced in SQL

* Update logic in SqlaTable as well

* Fix unit tests

(cherry picked from commit 63b5e2e4fa)
2022-04-04 11:00:43 +03:00
Srini Kadamati
ba22905610 chore: update slack invite url (#19412)
* chore: update slack invite url

* small fix in CoC

(cherry picked from commit 9d71f33d62)
2022-04-03 19:16:14 +03:00
Ville Brofeldt
fb929ab649 run black on remaining files 2022-04-03 19:15:25 +03:00
Ville Brofeldt
a70f4dc52f chore: upgrade black (#19410) 2022-04-03 19:13:17 +03:00
Michael S. Molina
1edd5f1343 chore: Removes direct theme imports (#19368) 2022-04-03 19:08:52 +03:00
Beto Dealmeida
cccec50454 feat: disable edits on external assets (#19344)
* feat: disable edits on external assets

* Update tests

(cherry picked from commit d304849b46)
2022-04-03 19:04:00 +03:00
Evan Rusackas
495b29a4eb chore: bump storybook to 6.4.19, migrating config, and linting storybook config (#19245)
* chore: bump storybook to 6.4.19 and reconfigure accordingly

* now linting the storybook directory, since it gets validated by CI

* linting 

(cherry picked from commit b689ac2d11)
2022-04-03 19:04:00 +03:00
Yongjie Zhao
5f2ffb3ba4 feat: linear imputation in Resample (#19393)
(cherry picked from commit a39dd4493e)
2022-04-03 19:03:59 +03:00
Geido
56e78b9ef7 chore: Eslint custom plugin to warn about hex and literal colors (#19239)
* wip

* Add eslint custom plugin

* Refactor

* Clean up

* Update superset-frontend/buildtools/eslint-plugin-theme-colors/index.js

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* Refactor

* Update superset-frontend/buildtools/eslint-plugin-theme-colors/index.js

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* Clean up

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
(cherry picked from commit 6b9113a17b)
2022-04-03 19:03:59 +03:00
Sujith Kumar S
ae2763af53 docs: Correcting the document font size (#19390)
* Correcting with doc font size

Correcting with doc font size

* Correcting with doc font size

Correcting with doc font size

* Update docs/docs/installation/configuring-superset.mdx

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
(cherry picked from commit d46a550774)
2022-04-03 19:03:59 +03:00
Stephen Liu
fb0ae24c6b chore: add type checking in plugin test directory (#19387)
(cherry picked from commit 6f5778273e)
2022-04-03 19:03:58 +03:00
smileydev
0c75d9ede0 fix(database): make to allow the expose db as default in sqlalchemy form db (#19337)
(cherry picked from commit 5ae7e54999)
2022-04-03 19:03:58 +03:00
Jean-Louis Queguiner
029cf73b75 Fix add-driver-docker in Readme (#19321)
(cherry picked from commit 2b53578ad7)
2022-04-03 19:03:58 +03:00
Ville Brofeldt
dead7c253c fix(caching): support metastore cache for non-required caches (#19369)
* fix(caching): support metastore cache for non-required caches

* lint

(cherry picked from commit 02308a6f78)
2022-04-03 19:03:58 +03:00
Ville Brofeldt
25c5e2b4c7 chore: add support for npm8 (#19365)
(cherry picked from commit 6141a9aa4a)
2022-04-03 19:03:58 +03:00
Kamil Gabryjelski
2f497190ed feat(explore): Dataset panel option tooltips (#19259)
* feat(explore): Add description to column/metric tooltips in dataset panel

* Fix tests

* Address code review comments

(cherry picked from commit 45c28c8046)
2022-04-03 19:03:57 +03:00
smileydev
7af92ed337 fix(explore-sqllab): make that Timestamp column keep the Is temporal flagged when overwriting (#19010)
(cherry picked from commit 446358642f)
2022-04-03 19:03:57 +03:00
Yongjie Zhao
da4a79f276 fix: number format should editable when AA in time comparison (#19351)
(cherry picked from commit e15573d445)
2022-04-03 19:03:57 +03:00
smileydev
d65954994c fix(report-add): make to add error toast message when already attached report is added again into dashboard or chart (#19122)
* fix(report-add): make to add error toast message when already attached report is added again into dashboard or chart

* fix(export-add): make to setup the default error message

* fix(report-add): make to allow the errorMessage undefined in LabeledErrorBoundInput component

(cherry picked from commit cb0357005e)
2022-04-03 19:03:57 +03:00
Smart-Codi
0439d8db40 fix: Remove gap from SQLLab results bottom (#19138)
* Remove gap from SQLLab results bottom

* resolve comment

(cherry picked from commit 8947eb9680)
2022-04-03 19:03:56 +03:00
Geido
549b475a94 Fix NoAuthorizationError (#19355)
(cherry picked from commit a2bb91243a)
2022-04-03 19:03:56 +03:00
Ville Brofeldt
a6a2def6d3 feat: introduce hashids permalink keys (#19324)
* feat: introduce hashids permalink keys

* implement dashboard permalinks

* remove shorturl notice from UPDATING.md

* lint

* fix test

* introduce KeyValueResource

* make filterState optional

* fix test

* fix resource names

(cherry picked from commit f4b71abb22)
2022-04-03 19:03:56 +03:00
Kasia
18f82411c9 fix: Adaptive formatting spelling (#19359)
Co-authored-by: Kasia Zajac <kasiazajac@Kasias-MacBook-Pro-2.local>
(cherry picked from commit dc769a9a34)
2022-04-03 19:03:56 +03:00
Beto Dealmeida
d9377559f6 feat: import external management columns (#19315)
* feat: import flags

* Add tests

(cherry picked from commit c7f9060a2f)
2022-04-03 19:03:55 +03:00
Ville Brofeldt
a591eccfc2 docs: update lock + general cleanup (#19350)
* docs: update lock file

* remove redundant npm lock file

* add missing peer deps

* fix TODO and bump antd

* fix API spec script

(cherry picked from commit 3313530f4d)
2022-04-03 19:03:55 +03:00
Erik Ritter
069d42ed14 fix: typo on doc string (#19346)
(cherry picked from commit 2af2d00e85)
2022-04-03 19:03:55 +03:00
Richard Whaling
89b7b3784a feat: add duckdb as DataSource - Fixes #14563 (#19317)
* + duckdb support

needs the forked version of [duckdb-engine](https://github.com/alitrack/duckdb_engine)

* Update duckdb.py

update  _time_grain_expressions

* removed superfluous get_all_datasource_names def in duckdb engine spec

* added exception handling for duckdb single-threaded RuntimeError

* fixed linter blips and other stylistic cleanup in duckdb.py

* one last round of linter tweaks in test_connection.py for duckdb support

Co-authored-by: Steven Lee <admin@alitrack.com>
Co-authored-by: Richard Whaling <richardwhaling@Richards-MacBook-Pro.local>
(cherry picked from commit 202e34a259)
2022-04-03 19:03:55 +03:00
Beto Dealmeida
3b0b60c57f feat: external management flags in CRUD (#19318)
(cherry picked from commit 9766726b26)
2022-04-03 19:03:55 +03:00
Michael S. Molina
a81b1ab374 fix: Null values on Explore filter (#19341)
(cherry picked from commit 65c204b288)
2022-04-03 19:03:54 +03:00
Norman Hooper
6ccc1c86fa Use three backticks for code blocks (#19331)
(cherry picked from commit 3340f05dd3)
2022-04-03 19:03:54 +03:00
ND
99f36f73d2 Correcting some typos (#19338)
Co-authored-by: Andreas Burner <andreas.burner@smartstream-stp.com>
(cherry picked from commit 7ba06ee944)
2022-04-03 19:03:54 +03:00
Lily Kuang
b21d4f841c fix(embedded): download chart as image (#19339)
(cherry picked from commit 0ad692511e)
2022-04-03 19:03:54 +03:00
Kamil Gabryjelski
158d442f0c fix(dashboard): Chart stuck in loading state when when datasets request and chart request fail (#19327)
(cherry picked from commit a08f83bc60)
2022-04-03 19:03:53 +03:00
smileydev
baeb36f8c6 fix(explore): make to convert null to N/A in view results (#19316)
* fix(explore): make to convert null to N/A in view results

* fix(explore): make to null formatter move before timeFormatter

(cherry picked from commit 468c5ca29a)
2022-04-03 19:03:53 +03:00
Diego Medina
52f9b718f2 fix: regression on Select component when handling null values (#19326)
(cherry picked from commit 9e58916d93)
2022-04-03 19:03:53 +03:00
Stephen Liu
bf92b2067e chore: add type checking in package tests directory (#19320)
(cherry picked from commit b0397beb8e)
2022-04-03 19:03:53 +03:00
Sujith Kumar S
d429bbd3b2 docs: SECRET_KEY Rotation Documentation (#19233)
* SECRET_KEY Rotation

Additional documentation for SECRET_KEY rotation and SECRET_KEY setting up.

* Bumped the helm chart version to 0.5.11

Bumped the helm chart version for the new changes.

* Removed the default secret key value from the configuration docs.

Removed the default secret key value from the configuration docs.
2022-04-03 19:02:44 +03:00
Daniel Vaz Gaspar
b51eadd657 chore: bump FAB to 3.4.5 (#19323)
(cherry picked from commit 87e36d6abb)
2022-04-03 19:01:40 +03:00
Daniel Vaz Gaspar
0664a05afb ci: add more code owners to helm chart source (#19322)
(cherry picked from commit ff6ee3766c)
2022-04-03 19:01:40 +03:00
Ville Brofeldt
3a9435df72 fix(dashboard): fix default filter bar visibility + add docs (#18741)
* fix(dashboard): fix default filter tab visibility + add tests

* fix types

* lint

* rename docs + add double bang to length

(cherry picked from commit b7ecb14230)
2022-04-03 19:01:40 +03:00
Yongjie Zhao
9bc76337cf feat(advanced analysis): support MultiIndex column in post processing stage (#19116) 2022-04-03 19:00:01 +03:00
AAfghahi
f8a92de75c docusearch (#19289)
(cherry picked from commit 6083545e86)
2022-04-03 18:57:31 +03:00
Evan Rusackas
12759ec264 chore: fixing a pluralization typo (#19295)
(cherry picked from commit e45235e482)
2022-04-03 18:57:31 +03:00
Ville Brofeldt
f0630a6ea7 fix(cache): only warn about fallback cache for non-debug mode (#19305)
(cherry picked from commit 7ca10fb770)
2022-04-03 18:57:31 +03:00
Beto Dealmeida
d4223d7dc4 feat: API for asset sync (#19220)
* feat: API for asset sync

* Add unit tests.

* Improve tests

* Move files

* Add more tests

(cherry picked from commit b05e7dbf2a)
2022-04-03 18:57:30 +03:00
Michael S. Molina
46dbf6c50c fix: Don't allow duplicated tag values in the Select (#19283)
* fix: Don't allow duplicated tag values in the Select

* Addresses comments and adds test

(cherry picked from commit d3ce398448)
2022-04-03 18:57:30 +03:00
Kamil Gabryjelski
19ee561092 chore: Make font-weights themable, fix font faces (#19236)
* fix(fonts): Import all necessary font packages

* Make html tags themable

* Set bold font weight to 600, add medium font weight to theme

* Replace hard coded font weights with theme variables

* Change some font weight light elements to normal

* Fix tests

* Fix bug in pivot table

* Address code review comments
2022-04-03 18:56:12 +03:00
David Aaron Suddjian
c6a4d75954 Update ToastPresenter.tsx (#19296)
(cherry picked from commit 35b8a72dae)
2022-04-03 14:10:50 +03:00
David Aaron Suddjian
55aac5a3cb allow overriding the guest token PyJWT instance (#19293)
(cherry picked from commit f9feb1b7f3)
2022-04-03 14:10:50 +03:00
Diego Medina
712212be6d fix: most Recently Selected Table Should Appear at the Top of the List on the Left Panel (#19258)
(cherry picked from commit 4669b6ce11)
2022-04-03 14:10:50 +03:00
smileydev
792473f6db fix(dashboard): make to show the correct owned objects (#19223)
* fix(dashboard): make to show the correct owned objects

* fix(dashboard): make to filter is reusable

* fix(homepage): make sure the type Array<Filters>

(cherry picked from commit 29cba2b00c)
2022-04-03 14:10:49 +03:00
Diego Medina
2a2105c8c8 chore: add missing keys to components inside lists (#19161)
(cherry picked from commit 54f1b35161)
2022-04-03 14:10:49 +03:00
Phillip Kelley-Dotson
e1964a8dfe feat: allow uploads in crud view (#18953)
* feat: allow uploads in crud view

* fix merge conflict and fix ts

* fix import

* fix tests

* fix lint

* remove unused var

* fix underline flash and alignment

* fix offset

* fix icon alignment

* fix labels and css issues

* make drowdown primary all the time

* make global

* fix lables

* add upload perms to utils

* remove unused code

* add suggested changes

* update menuright

(cherry picked from commit d771ddbb94)
2022-04-03 14:10:49 +03:00
Ville Brofeldt
f3172010d5 feat(key-value): add superset metastore cache (#19232)
(cherry picked from commit 72b9a7fa5b)
2022-04-03 14:10:49 +03:00
John Bodley
81a1abfab4 fix(presto/trino): Add TIME/TIMESTAMP WITH TIME ZONE (#19263)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
(cherry picked from commit 82a6811e7e)
2022-04-03 14:10:48 +03:00
David Aaron Suddjian
dc0153151c feat: SupersetClient config to override 401 behavior (#19144)
* wip

* feat: make 401 responses configurable in SupersetClient

* sort

* guest unauthorized handler

* add toast container to embedded app

* add option for toast presenter to go at the top

* remove confusing comms logging

* lint

* Update superset-frontend/src/embedded/index.tsx

* type correction

(cherry picked from commit 96a123f553)
2022-04-03 14:10:48 +03:00
Elizabeth Thompson
910679aad5 fix dataset update table (#19269)
(cherry picked from commit 88029e21b6)
2022-04-03 14:10:48 +03:00
PApostol
70facad1b2 Various docstring fixes (#18221)
(cherry picked from commit c07a707eab)
2022-04-03 14:10:48 +03:00
prassanna-helixsense-com
565d83f9ec Update README.md (#19270)
(cherry picked from commit d215cbcdc8)
2022-04-03 14:10:48 +03:00
Stephen Liu
7d239b8958 feat: improve color consistency (save all labels) (#19038)
(cherry picked from commit dc575080d7)
2022-04-03 14:10:47 +03:00
Ville Brofeldt
408573d4d6 feat: add support for comments in adhoc clauses (#19248)
* feat: add support for comments in adhoc clauses

* sanitize remaining freeform clauses

* sanitize adhoc having in frontend

* address review comment

(cherry picked from commit f341025d80)
2022-04-03 14:10:47 +03:00
Diego Medina
f6346d627b fix: custom SQL in Sort By Breaks Bar Chart (#19069)
(cherry picked from commit 4f0074a4ae)
2022-04-03 14:10:47 +03:00
smileydev
fed1c24252 fix(chart-crud): make to update Viz type Filter label to Chart type (#19140)
(cherry picked from commit b888341090)
2022-04-03 14:10:47 +03:00
Lily Kuang
c793b7bfe9 fix: allow subquery in ad-hoc SQL (WIP) (#19242)
* allow adhoc subquery

* add config for allow ad hoc subquery

* default to true allow adhoc subquery

* fix test

* Update superset/errors.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/connectors/sqla/utils.py

Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>

* rename and add doc string

* fix for big query test

* Update superset/connectors/sqla/utils.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Apply suggestions from code review

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* add test

* update validate adhoc subquery

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>
(cherry picked from commit 50902d51f5)
2022-04-03 14:10:46 +03:00
Elizabeth Thompson
1953233b68 update changelog and updating files from 1.4.1 (#18648) 2022-04-03 14:09:25 +03:00
smileydev
1834a268a1 fix(explore comma): make that the comma can be added by removing it from token separators… (#18926)
* make that the comma can be added by removing it from token separators in select component.

* fix(explore comma): add the allowTokenSeperators props into Select

* fix(explore comma): make to allow to customize the token separators in Select.

* fix(explore comma): make to add the unit test and story book.

* fix(explore comma): make to fix the lint

* fix(explore comma): make to fix the spell  & add tokenSeparatprs props to PickedSelectProps

* Update Select.tsx

* fix(explore comma): make to run lint fix

(cherry picked from commit e7355b9610)
2022-04-03 14:04:53 +03:00
Venu Vardhan Reddy Tekula
1c53a76bcc docs: fix broken links in the documentation (#19235)
(cherry picked from commit 2a89da2ef1)
2022-04-03 14:04:53 +03:00
Stephen Liu
33beba96e3 chore: use order_desc shared control consistently (#19172)
(cherry picked from commit 10eb6c77a4)
2022-04-03 14:04:53 +03:00
Yongjie Zhao
f1ea0ad56c fix: adhoc column in legacy chart (#19234)
(cherry picked from commit b5e9fad11a)
2022-04-03 14:04:52 +03:00
Diego Medina
f2541429cd fix(sql lab): deleting the last saved query or the last executed from history (#19225)
* fix: fix issue when deleting the last saved query or the last executed query

* merge migration

(cherry picked from commit aa5c80bda6)
2022-04-03 14:04:52 +03:00
Beto Dealmeida
3de8370ead feat: import/export assets commands (#19217)
* feat: import/export assets commands

* Add overwrite test

* Fix tests

(cherry picked from commit 51061f0d67)
2022-04-03 14:04:52 +03:00
Jesse Yang
2651c1d925 chore: upgrade mypy (#19227)
(cherry picked from commit 92cd0a18e6)
2022-04-03 14:04:52 +03:00
Diego Medina
0522296607 fix: allow to select <NULL> in a native filter single mode (#19076)
* fix: allow to select <NULL> in a native filter single mode

* fix lint issue

* Update superset-frontend/src/components/Select/utils.ts

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* fix

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
(cherry picked from commit 19fcd03c89)
2022-04-03 14:04:52 +03:00
smileydev
823b9b2ff2 fix(select): make to consider the case sensitive in case of d3 format selector (#19159)
(cherry picked from commit d099f5ed4a)
2022-04-03 14:04:51 +03:00
Craig Rueda
f91f9f5aae feat(ui): Adding manifest prefix config (#19141)
* Adding manifest prefix config

* Fixing broken tests

* Fixing import

* Adding prefix for remaining assets

* Changing static prefix strategy

* Fixing DST test

* Fixing up formatting

* Fixing up async_query_manager.py types
2022-04-03 14:03:35 +03:00
cccs-Dustin
a1b9b2946d feat(sqllab): Add a configuration option to disable data preview (#19104)
(cherry picked from commit 02ef9ca4cd)
2022-04-03 14:02:24 +03:00
Diego Medina
71338cab57 feat: scroll to bottom when adding a new native filter and the page is filled (#19053)
* feat: Scroll to bottom when adding a new native filter and the page is filled

* Add test

(cherry picked from commit cfb967f430)
2022-04-03 14:02:24 +03:00
Diego Medina
28be2311a0 fix: Popovers in Explore not attached to the fields they are triggered by (#19139)
* fix: Popovers in Explore not attached to the fields they are triggered by

* fix

* PR comment

* remove unused import

(cherry picked from commit 0277ebc225)
2022-04-03 14:02:24 +03:00
Diego Medina
ff277e0517 fix: auto-complete of tables and names are not working in SQL lab (#19152)
(cherry picked from commit 3b427b2029)
2022-04-03 14:02:24 +03:00
Beto Dealmeida
7bc6e14151 chore: refactor import command (#19216)
(cherry picked from commit a4848a2f46)
2022-04-03 14:02:24 +03:00
Ville Brofeldt
8c102174b8 feat: add permalink to dashboard and explore (#19078)
* rename key_value to temporary_cache

* add migration

* create new key_value package

* add commands

* lots of new stuff

* fix schema reference

* remove redundant filter state from bootstrap data

* add missing license headers

* fix pylint

* fix dashboard permalink access

* use valid json mocks for filter state tests

* fix temporary cache tests

* add anchors to dashboard state

* lint

* fix util test

* fix url shortlink button tests

* remove legacy shortner

* remove unused imports

* fix js tests

* fix test

* add native filter state to anchor link

* add UPDATING.md section

* address comments

* address comments

* lint

* fix test

* add utils tests + other test stubs

* add key_value integration tests

* add filter box state to permalink state

* fully support persisting url parameters

* lint, add redirects and a few integration tests

* fix test + clean up trailing comma

* fix anchor bug

* change value to LargeBinary to support persisting binary values

* fix urlParams type and simplify urlencode

* lint

* add optional entry expiration

* fix incorrect chart id + add test

(cherry picked from commit b7a0559aaf)
2022-04-03 14:02:23 +03:00
Beto Dealmeida
90f4d77422 feat: add export_related flag (#19215)
* feat: add export_related flag

* Fix lint

(cherry picked from commit d01fdad1d8)
2022-04-03 14:02:23 +03:00
Hugh A. Miles II
2979ff26cc fix: Revert "refactor: converted QueryAutoRefresh to functional component … (#19226)
* Revert "refactor: converted QueryAutoRefresh to functional component (#18179)"

This reverts commit f497c1900e.

* lint

(cherry picked from commit fc8721800b)
2022-04-03 14:02:23 +03:00
Jesse Yang
a68b1fd686 test: fix TimezoneSelector tests on daylight saving time (#19156)
(cherry picked from commit 8d53db1db6)
2022-04-03 14:02:23 +03:00
Hugh A. Miles II
ca0c12f2b3 fix: Logic for showing extension in Global Nav (#19158)
* fix logic for checking extensions

* add specific types

* fix lint

(cherry picked from commit 181ecf4509)
2022-04-03 14:02:22 +03:00
Lyndsi Kay Williams
3f1074e6a3 fix(sqllab): Updated blank states for query results and query history (#19111)
* Empty states updated on result tab and query history tab

* Testing on query history blank state

* Testing on result tab with empty state

* Forgot to remove a comment

* Corrected empty state image size and centered with drag bar

* Centered blank states vertically

(cherry picked from commit bb618a47ff)
2022-04-03 14:02:22 +03:00
David Aaron Suddjian
337ea4f59d fix: clean up chart metadata config (#19143)
* fix: clean up chart metadata config

* missed a spot

* missed another spot

* fix failing time-specific test

* can't call translation functions here

* Revert "fix failing time-specific test"

This reverts commit 3eeb8ab9cc.

* skip problematic test

* extra import

(cherry picked from commit 3d66912d89)
2022-04-03 14:02:22 +03:00
smileydev
db8b6af7c7 fix(dashboard-css): make to stay custom css when reload (#19084)
* fix(dashboard-css): make to stay custome css when reload

* fix(dashboard-css): make to add injectCustomCSS into HeaderActionsDropdown.test.tsx

(cherry picked from commit 30c97ad5bf)
2022-04-03 14:02:22 +03:00
smileydev
a0138af7f8 fix(dbs-dropdown): make to allow the search in supported db dropdown (#19125)
(cherry picked from commit 8e7af79aa9)
2022-04-03 14:02:22 +03:00
Diego Medina
ba57cb410e fix(dashboard): scrolling table viz overlaps next chart (#19121)
(cherry picked from commit 74910f99d8)
2022-04-03 14:02:21 +03:00
Jinghuayao
37162a923b test(native filter): add new native filter test (#19115)
* add new native filter test

(cherry picked from commit 981f09b5db)
2022-04-03 14:02:21 +03:00
David Aaron Suddjian
0598e95b1b chore(embedded): refresh the guest token (#19132)
* refresh the guest token

* put back the date logic

* version

* fix time hijinks

* test

* Update superset-embedded-sdk/src/guestTokenRefresh.ts

(cherry picked from commit 54b60ded8e)
2022-04-03 14:02:21 +03:00
Beto Dealmeida
2e6bd1590d feat: helper functions for RLS (#19055)
* feat: helper functions for RLS

* Add function to inject RLS

* Add UNION tests

* Add tests for schema

* Add more tests; cleanup

* has_table_query via tree traversal

* Wrap existing predicate in parenthesis

* Clean up logic

* Improve table matching

(cherry picked from commit 8234395466)
2022-04-03 14:02:21 +03:00
Hugh A. Miles II
20fd7a7cd5 fix: Add perm for showing DBC-UI in Global Nav (#19023)
* add perm for global db add

* fix permissions

* missing roles params

(cherry picked from commit c337491d0e)
2022-04-03 14:02:21 +03:00
Jesse Yang
fcca027b88 feat(select): keep options order when in single mode (#19085)
(cherry picked from commit ae13d8313b)
2022-04-03 14:02:20 +03:00
Lily Kuang
2b8b9b366a feat(embedded): add optional dashboard ui configuration (#19031)
* feat: add optional dashboard ui configuration

* change all flags to boolean

* update README and lint

(cherry picked from commit 124cb0dc66)
2022-04-03 14:02:20 +03:00
Stephen Liu
877ee42a1c feat(viz-gallery): add 'feature' tag and fuzzy search weighting (#18662)
* feat(viz-gallery): add 'feature' tag and fuzzy search weighting

* add search weight

* Update superset-frontend/src/explore/components/controls/VizTypeControl/VizTypeGallery.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/explore/components/controls/VizTypeControl/VizTypeGallery.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/explore/components/controls/VizTypeControl/VizTypeGallery.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/explore/components/controls/VizTypeControl/VizTypeGallery.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* some improvements

* take metadata out

* use chartLabel enum to unify

* add test chart

* fix cache

* Resolving TS Lint issue

* Appeasing the linter

* Removing one example implementation

* Removing another example label implementation

* Removing the third example label implementation

Co-authored-by: Evan Rusackas <evan@preset.io>
(cherry picked from commit 7524e1e3c8)
2022-04-03 14:02:20 +03:00
Jesse Yang
a2fc96a9c3 refactor(TimezoneSelector): simplify override logics and tests (#19090)
(cherry picked from commit 3a78165d13)
2022-04-03 14:02:20 +03:00
smileydev
891798b871 fix(dashobard-edge-cutting): make to be not cut without Filter (#19080)
(cherry picked from commit 158396fb6c)
2022-04-03 14:02:20 +03:00
Lily Kuang
cb5d814fd7 fix: cache key with guest token rls (#19110)
* add guest rls clause to cache key

* lint

* pylint

* add app back

(cherry picked from commit 27268169e4)
2022-04-03 14:02:19 +03:00
7vikpeculiar
7d0ee284fa Modified alerts and reports docs for clarity (#19091)
(cherry picked from commit a37a4ed35f)
2022-04-03 14:02:19 +03:00
Kamil Gabryjelski
89edf8a7a9 fix(dashboard): Empty states overflowing small chart containers (#19095)
* fix(dashboard): Empty states overflowing small chart containers

* Fix test

(cherry picked from commit 70081a698f)
2022-04-03 14:02:19 +03:00
Kamil Gabryjelski
088f6f75ff perf(dashboard): Send chart requests before native filter requests (#19077)
(cherry picked from commit b8091e33a9)
2022-04-03 14:02:19 +03:00
smileydev
d0c9df4d7b fix(plugin-chart-echarts): make to allow the custome of x & y axis title margin i… (#18947)
* fix(chartviz): make to allow the custome of x & y axis title margin in  chart

* fix(chartviz): add eslint radix error in chart.js

* fix(chartviz): change the transformProps in chart plugin & creat helper.

* fix(chartviz): lint fix & chart.js back

* fix(plugin-chart-echarts): make to allow the custom margin of X & y axis in BoxPlot & Mixedtimeseries charts

* fix(plugin-chart-echarts): make to change changeNumber to changeInteger

* fix(plugin-chart-echarts): make to add license & change file name

(cherry picked from commit c79ee56884)
2022-04-03 14:02:19 +03:00
Ville Brofeldt
36561b7943 fix(dashboard): import handle missing excluded charts (#19088)
(cherry picked from commit 999c2c6826)
2022-04-03 14:02:18 +03:00
Smart-Codi
2193e17940 fix: Show Totals error with sort and group by together (#19072)
(cherry picked from commit bc6aad0a88)
2022-04-03 14:02:18 +03:00
Smart-Codi
dae9e7c020 fix database import error (#19037)
(cherry picked from commit e97cdba092)
2022-04-03 14:02:18 +03:00
Hugh A. Miles II
2c151f352e fix autocomplete (#19047)
(cherry picked from commit a21d8a0bf4)
2022-04-03 14:02:18 +03:00
Diego Medina
0aba34a807 fix: Pivot Table Conditional Formatting Doesn't Show All Options (#19071)
* fix: Pivot Table Conditional Formatting Doesn't Show All Options

* PR comments

* PR comments

(cherry picked from commit 0e0beceac1)
2022-04-03 14:02:18 +03:00
smileydev
67d7a7b115 fix(allow-db-explore): make that the set the allow-db-explore option (#19030)
(cherry picked from commit 62ad574c24)
2022-04-03 14:02:17 +03:00
Kamil Gabryjelski
cd2a958ce3 perf(dashboard): Improve performance of complex dashboards (#19064)
* perf(dashboard): Improve performance of filter indicators

* Improve perf of cross filters

* Rename old function

* Fix undefined

* fix type

* fix tests

* fix undefined

* Address code review comments

* Address code review comments

(cherry picked from commit 3c1fb944c1)
2022-04-03 14:02:17 +03:00
smileydev
4462cfeb0f fix(altered-modal): make specified text fields wrap in table (#18822)
* fix(altered-modal): make all text fields wrap

* fix(altered-modal): limit the wrap text in particular column

* fix(altered-modal): make to update the unit test

* fix(altered-modal): make to fix the type of columnsForWrapText

* fix(alerted-modal): make to fix the type of columnsForWrapTest with string type

(cherry picked from commit 220c46131e)
2022-04-03 14:02:17 +03:00
Diego Medina
4c9785f3e3 fix: enable find text browser functionality inside SQL Lab editor (#19061)
(cherry picked from commit fd757c4aa4)
2022-04-03 14:02:17 +03:00
Diego Medina
005949f6b9 fix: show the total row count in the SQL Lab Query History tab when limited by DISPLAY_MAX_ROW (#19054)
(cherry picked from commit bd76648e4e)
2022-04-03 14:02:17 +03:00
Geido
4bff8fe95e Control sortby based on series limit (#18950)
(cherry picked from commit f53f86f796)
2022-04-03 14:02:16 +03:00
Michael S. Molina
3729df2653 fix: Select clear and loading icons overlap (#19070)
(cherry picked from commit 79a7a5d1b1)
2022-04-03 14:02:16 +03:00
Michael S. Molina
56b1144abd fix: Dataset search when creating a chart (#19065)
(cherry picked from commit fd154f7ba6)
2022-04-03 14:02:16 +03:00
Elizabeth Thompson
253f80ab6d chore: log multiple errors (#14064)
* log all errors from db create

* return unique set of errors

* sort set for exceptions list

* run black

(cherry picked from commit c143b37128)
2022-04-03 14:02:16 +03:00
Grace Guo
e97b123961 fix(time-series table): display null values in time-series table and sortable (#19024)
* fix: display null values in time-series table and sortable

* add unit test

* fix unit test

* Add sortNumericValues with different nan treatment

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
(cherry picked from commit d539fc217a)
2022-04-03 14:02:16 +03:00
Phillip Kelley-Dotson
bc65cf4509 fix: ensure validation for db modal for googlesheets (#19018)
* fix: ensure validation for db modal for googlesheets

* chain async function

(cherry picked from commit bb17decb06)
2022-04-03 14:02:15 +03:00
Srini Kadamati
3ce4f051d5 docs: renamed yugabyte to yugabytedb (#19068)
* docs: renamed yugabyte to yugabytedb

* fixed redirect error

* pass that test!

(cherry picked from commit 7af26a0492)
2022-04-03 14:02:15 +03:00
Diego Medina
8bef059624 Add previous line hotkey to SQL editor for macOS (#19052)
(cherry picked from commit aa0ec717a2)
2022-04-03 14:02:15 +03:00
Thomas Desrosiers
4b8fc06e5f fix: Update time grain expressions for Spark >= 3.x (#18690)
* Fix the time grain expressions for Spark >= 2.3.0

Spark removed date format string 'u' in Spark 3.0. Switch to using date_trunc which has been around since 2.3

* Review: Pull out time_grain_expressoins into its own thing

(cherry picked from commit 03b2b06e90)
2022-04-03 14:02:15 +03:00
Phillip Kelley-Dotson
29b46d00f9 fix: remove unnecessary divider (#19048)
* fix: remove unnecessary divider

* remove console

(cherry picked from commit 4d96393faf)
2022-04-03 14:02:15 +03:00
Jesse Yang
032f560c2c feat(select): sort exact and startsWith match to first (#18856)
(cherry picked from commit c75f233109)
2022-04-03 14:02:14 +03:00
Michael S. Molina
219fa570b0 chore: Moves Chart to the components folder (#19029)
(cherry picked from commit 04a36d5c92)
2022-04-03 14:02:14 +03:00
chanyou0311
05f25d350c Install and use vm-browserify for sandboxedEval (#19059)
(cherry picked from commit 2cc5678dfc)
2022-04-03 14:02:14 +03:00
Alex Rothberg
1658a9f6b4 docs(building): link frontend asset instructions correctly (#19050)
(cherry picked from commit e89f0abf95)
2022-04-03 14:02:14 +03:00
Erik Ritter
379676e1f6 fix: improve cypress flakiness (#19044)
(cherry picked from commit 002bd6ed0a)
2022-04-03 14:02:13 +03:00
Ville Brofeldt
02d9825b11 chore(plugin-chart-echarts): bump echarts 5.3.1 (#19041)
(cherry picked from commit 1b7d056126)
2022-04-03 14:02:13 +03:00
pablo-guerra
45dc7b5984 chore: adding Dutch language translations (NL) (#18965)
* added NL translations

* added Dutch to config

* fixed end of line on .po file

Co-authored-by: pablo.guerra-at-026399402087 <pablo.guerra@026399402087>
(cherry picked from commit 44675037bd)
2022-04-03 14:02:13 +03:00
Michael S. Molina
4e78efcf07 chore: Moves CRUD components to the Datasource component (#19032)
* chore: Moves CRUD components to the Datasource component

* Fixes imports

* Fixes less import

(cherry picked from commit 71b8d31e77)
2022-04-03 14:02:13 +03:00
Stephen Liu
1eac8712a4 fix(big-number): Big Number with Trendline Chart is not working if Time Grain is set to Month (#19043)
* fix(big-number): Big Number with Trendline Chart is not working if Time Grain is set to Month

* use start frequency.

(cherry picked from commit c32eaf47e5)
2022-04-03 14:02:12 +03:00
Duc Le Tu
9777e6d148 Update impala.mdx (#19040)
(cherry picked from commit b2ee13f6f8)
2022-04-03 14:02:12 +03:00
Karthikeyan Singaravelan
585b032a1a chore: Fix deprecated unittest aliases. (#19042)
(cherry picked from commit 864bafc655)
2022-04-03 14:02:12 +03:00
Hugh A. Miles II
fb5d77e404 feat: Allow users to bust cache in report dashboard + alerts charts + alert dashboards (#18795)
* wip

* add force cahce bypass option to alerts

* remove default for alerts to bypass cache

* save for now

* save for now

* fix

* commenting out for now

* fix linting

* remove link

* add back force id test

* add frontend test

* address

(cherry picked from commit 8c52fe3476)
2022-04-03 14:02:12 +03:00
Michael S. Molina
6178f0515a refactor: Deletes the common/components folder (#18999)
(cherry picked from commit 329855170e)
2022-04-03 14:02:12 +03:00
Kamil Gabryjelski
8640814a92 fix(plugin-chart-echarts): Apply temporary filters to Query B in explore (#18998)
* fix(explore): Extra filters not applied to query b in mixed timeseries

* Add return type

* Apply review comment

* Fix non-dnd filters

(cherry picked from commit 9f834e8317)
2022-04-03 14:02:11 +03:00
cccs-Dustin
00a53dae73 fix(SQL Editor): names new query tabs correctly (#18951)
* Added in code changes that now properly increment the Untitled Query SQL Lab tab names. All that is left is to add tests to make sure that the function works correctly

* Updated the code so that it adds to the untitled_query_numbers variable only if the character after the string 'Untitled Query ' is a number. This prevents any issues when trying to get the maximum value in the list.

* Refactored part of the mapping code, to make it shorter and easier to read/understand

* Fixed issues in the code that were causing some of the CI tests to fail

* Made code changes based on comments within the PR. Also added a unit test to make sure that the newQueryEditor function in the TabbedSqlEditors component works as intended

* Fixed the failing cypress test in tabs.test.js

(cherry picked from commit 5a5ff99c37)
2022-04-03 14:02:11 +03:00
Michael S. Molina
ea534e2386 feat: Adds support to multiple dependencies to the native filters (#18793)
* chore(native-filters): Remove cascading popovers from filter bar

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
2022-04-03 14:00:26 +03:00
Phillip Kelley-Dotson
073be5d74f chore: change case for upload selection (#19021)
* chore: change case for upload selection

* fix label

(cherry picked from commit 299b5dc644)
2022-04-03 13:52:39 +03:00
Beto Dealmeida
8951e23ac6 fix: remove unneeded complexity in migration (#19022)
(cherry picked from commit 50bb86d666)
2022-04-03 13:52:39 +03:00
smileydev
3500b31551 fix(nested-tab-available): make the another tabs into one tabs available (#18877)
* fix(nested-tab-available): make the another tabs into one tabs available

* fix(netsted-tab): remove code of disable nested tab

(cherry picked from commit 6fe2431676)
2022-04-03 13:52:39 +03:00
dependabot[bot]
fcf98ec889 chore(deps): bump url-parse from 1.5.7 to 1.5.10 in /docs (#19019)
Bumps [url-parse](https://github.com/unshiftio/url-parse) from 1.5.7 to 1.5.10.
- [Release notes](https://github.com/unshiftio/url-parse/releases)
- [Commits](https://github.com/unshiftio/url-parse/compare/1.5.7...1.5.10)

---
updated-dependencies:
- dependency-name: url-parse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 6becd38e7f)
2022-04-03 13:52:39 +03:00
Jesse Yang
78e85aed02 feat: remove loading indicator when typing in select (#18799)
(cherry picked from commit 5a8eb09afb)
2022-04-03 13:52:39 +03:00
Beto Dealmeida
75afb3a7b6 feat: show user email in dashboard API (#19004)
* Feat: show user email in dashboard API

* Fix test

(cherry picked from commit be88cb9ba0)
2022-04-03 13:52:38 +03:00
John Bodley
3b2b6674f6 fix(sqllab): Replace stringified 'null' schema column values with NULL (#18992)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
(cherry picked from commit 19eb73b316)
2022-04-03 13:52:38 +03:00
Ville Brofeldt
5a8cd3477b chore(cache): default to SimpleCache in debug mode (#18976)
* chore(cache): default to SimpleCache in debug mode

* lint

* clean up type

* use util

* fix integration test cache configs

* remove util from cache manager

* remove trailing comma

* fix more tests

* fix truthiness check

* fix tests and improve deprecation notice

* fix default cache threshold

* move debug check to cache_manager

* remove separate getter

* update docs

* remove default cache config

(cherry picked from commit a04f1d4c87)
2022-04-03 13:52:38 +03:00
Ville Brofeldt
6d2b583fee fix(chart): deprecate persisting url_params (#18960)
* fix(chart): deprecate peristing url_params

* remove duplicated backend logic

* use omitBy

* simplify omit

(cherry picked from commit bd63a1bd98)
2022-04-03 13:52:38 +03:00
Craig Rueda
3c90220cf6 fix(packages): Fixing users packages (#18973)
* Fixing users packages

* Linting me() fn

* Fixing tests / moving packages

* Fixing imports

(cherry picked from commit 5fc7adb55b)
2022-04-03 13:52:38 +03:00
Srini Kadamati
e8cf9446a6 docs: renamed yugabytedb to yugabyte as per request by Yugabyte team (#18983)
Co-authored-by: Srinivasa Kadamati <srinik@Srinivasas-MacBook-Air.local>
(cherry picked from commit 563cce1e29)
2022-04-03 13:52:37 +03:00
Brian Nguyen
4a91f76798 refactor: converted QueryAutoRefresh to functional component (#18179)
* converted file to functional component

* refactor: convert class component to functional component

* refactor: convert class component to functional"

* Working on converting the shouldCheckForQueries test to RTL

* Working on converting first test to RTL

* Working on first test for queryAutoRefresh

* Finished Tests, pushing for review

* Cleaned up comments and console logs

Co-authored-by: Josue Lugaro <josuelugaro15@gmail.com>
(cherry picked from commit f497c1900e)
2022-04-03 13:52:37 +03:00
Ville Brofeldt
603f43bd95 bump and harmonize urijs and xss (#18922)
(cherry picked from commit 834686a3d9)
2022-04-03 13:52:37 +03:00
Kamil Gabryjelski
8bfb2f854e chore(native-filters): Add unit tests for filter cards (#18967)
* chore(native-filters): Add unit tests for filter cards

* Fix test

(cherry picked from commit ec746c2a10)
2022-04-03 13:52:37 +03:00
smileydev
178736ad27 fix(tooltip-not-dissapeared): make the tooltip disappear when mouse away (#18892)
* fix(tooltip-not-dissapeared): make the tooltip disappear when mouse away

* fix(tooltip-not-dissapeared): change css props instead of sytles props

(cherry picked from commit ca93d63bbb)
2022-04-03 13:52:37 +03:00
smileydev
fe0f1de64b fix(external-link): change the external link in creating chart dashboard (#18982)
(cherry picked from commit 37752cea9b)
2022-04-03 13:52:36 +03:00
Kamil Gabryjelski
f7b72d9486 fix(native-filters): Default value not shown in native filters (#18963)
* fix(native-filters): Default value not shown in native filters

* Reuse stringified datamask

(cherry picked from commit 2072225a86)
2022-04-03 13:52:36 +03:00
Stephen Liu
1f4ff51322 fix(plugin-chart-echarts): fix customize margin (#18958)
(cherry picked from commit c4e3c45b3c)
2022-04-03 13:52:36 +03:00
Grace Guo
928309841c fix: add/find owners from report/alert modal (#18971)
(cherry picked from commit 7e336d1230)
2022-04-03 13:52:36 +03:00
John Bodley
10d382de56 chore(security): Remove obsolete FAB metric_access permission (#16860)
* Update manager.py

* Update manager.py

* Update manager.py

* Update manager.py

(cherry picked from commit 85f0715644)
2022-04-03 13:52:36 +03:00
serenajiang
46015e2c66 feat(TimeTableViz): sort by first metric (#18896)
(cherry picked from commit 760dab9abd)
2022-04-03 13:52:36 +03:00
Geido
da64e7966f fix(Explore): Pivot table V2 sort by failure with D&D enabled (#18835)
* wip

* Add tests and clean up

* Clean up

* Remove unused import

(cherry picked from commit eafe0cfc6f)
2022-04-03 13:52:35 +03:00
Kamil Gabryjelski
335a90a5ac feat(legacy-preset-chart-deckgl): Add ,.1f and ,.2f value formats to deckgl charts (#18945)
* Add ,.1f and ,.2f value formats to deckgl charts

* Remove duplicated code

(cherry picked from commit c56dc8eace)
2022-04-03 13:52:35 +03:00
GaryNg
ee3b3aea5d docs: fix sql templating typos (#18955)
(cherry picked from commit 4775d3900f)
2022-04-03 13:52:35 +03:00
Ville Brofeldt
268ce0ae8b fix(dataset): handle missing database in migration (#18948)
(cherry picked from commit 2baceddbf1)
2022-04-03 13:52:35 +03:00
AAfghahi
9375ebfbff fix(sql Lab tabs): Empty SQL Lab tabs (#18817)
* Empty SQL table message on zero tabs

* sql editor no editor tab bug fix

* Revert Error message

* empty state tab state

* added a unit test

* addressed reviews

* kasia feedback

Co-authored-by: Yahya Kayani <yahyakiani1@gmail.com>
(cherry picked from commit 147dc5af71)
2022-04-03 13:52:35 +03:00
John Bodley
5b465c2ff1 fix: bump Helm chart release version (#18751) (#18758)
Co-authored-by: wiktor2200 <wiktor2200@users.noreply.github.com>
(cherry picked from commit 099421770c)
2022-04-03 13:52:35 +03:00
Smart-Codi
a76ae2edb6 fix: null schema issue in saved queries (#18849)
(cherry picked from commit 79633ce673)
2022-04-03 13:52:34 +03:00
Michael S. Molina
419ca75412 chore: Moves components out of the commons folder (#18931)
* refactor: Moves components out of the commons folder

* Fixes tests

(cherry picked from commit fd4bd1edbc)
2022-04-03 13:52:34 +03:00
Stephen Liu
2fc2e275d4 fix(deck.gl): multiple layers map size is shrunk (#18939)
(cherry picked from commit 2cb3635256)
2022-04-03 13:52:34 +03:00
Stephen Liu
ce61dbc2bc fix(native-filter): fix required filters of a dashboard won't load properly (#18695)
(cherry picked from commit 0b1bc5b5de)
2022-04-03 13:52:34 +03:00
Jinghuayao
77c8987bbe test(Native Filter): User can create parent filters using "Filter is hierarchical (#18915)
* add new native test for parent filter

(cherry picked from commit 2bd8d3b349)
2022-04-03 13:52:33 +03:00
Smart-Codi
b78bced141 fix: Monospacing errors in dashboards & charts (#18796)
* fix: Monospacing errors in dashboards & charts

* removed unnecessary styling

(cherry picked from commit 4923256b01)
2022-04-03 13:52:33 +03:00
Smart-Codi
4167cfa30e fix Metadata browser in SQL not render nicely in Safari (#18855)
(cherry picked from commit 0db49b1a7c)
2022-04-03 13:52:33 +03:00
Phillip Kelley-Dotson
e20788c0e1 chore: add intersect method from backend to fontend for upload extensions (#18811)
* chore: add intersect method from be to font end

* fix lint

* add suggestion

* fix python test

* run precommit

* fix pytlint

* update changes from masters

(cherry picked from commit 8d38675cbc)
2022-04-03 13:52:33 +03:00
7726 changed files with 562487 additions and 1018499 deletions

View File

@@ -17,14 +17,7 @@
# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
---
notifications:
commits: commits@superset.apache.org
issues: notifications@superset.apache.org
pullrequests: notifications@superset.apache.org
discussions: notifications@superset.apache.org
github:
del_branch_on_merge: true
description: "Apache Superset is a Data Visualization and Data Exploration Platform"
homepage: https://superset.apache.org/
labels:
@@ -54,44 +47,31 @@ github:
projects: true
# Enable wiki for documentation
wiki: true
# Enable discussions
discussions: true
enabled_merge_buttons:
squash: true
merge: false
rebase: false
ghp_branch: gh-pages
ghp_path: /
protected_branches:
master:
required_status_checks:
# strict means "Require branches to be up to date before merging".
strict: false
# contexts are the names of checks that must pass
# unfortunately AFAICT for `matrix:` jobs, we have to itemize every
# combination here.
contexts:
- lint-check
- cypress-matrix (0, chrome)
- check
- cypress-matrix (1, chrome)
- cypress-matrix (2, chrome)
- cypress-matrix (3, chrome)
- cypress-matrix (4, chrome)
- cypress-matrix (5, chrome)
- dependency-review
- docker-build
- frontend-build
- playwright-tests (chromium)
- pre-commit (current)
- pre-commit (previous)
- test-mysql
- test-postgres (current)
- test-postgres-hive
- test-postgres-presto
- test-sqlite
- unit-tests (current)
- pre-commit (3.8)
- python-lint (3.8)
- test-mysql (3.8)
- test-postgres (3.8)
- test-postgres (3.9)
- test-sqlite (3.8)
required_pull_request_reviews:
dismiss_stale_reviews: false
@@ -99,10 +79,3 @@ github:
required_approving_review_count: 1
required_signatures: false
gh-pages:
required_pull_request_reviews:
dismiss_stale_reviews: false
require_code_owner_reviews: true
required_approving_review_count: 1
required_signatures: false

View File

@@ -1,10 +0,0 @@
# JavaScript to TypeScript Migration Command
## Usage
```
/js-to-ts <core-filename>
```
- `<core-filename>` - Path to CORE file relative to `superset-frontend/` (e.g., `src/utils/common.js`, `src/middleware/loggerMiddleware.js`)
## Agent Instructions
**See:** [../projects/js-to-ts/AGENT.md](../projects/js-to-ts/AGENT.md) for complete migration guide.

View File

@@ -1,684 +0,0 @@
# JavaScript to TypeScript Migration Agent Guide
**Complete technical reference for converting JavaScript/JSX files to TypeScript/TSX in Apache Superset frontend.**
**Agent Role:** Atomic migration unit - migrate the core file + ALL related tests/mocks as one cohesive unit. Use `git mv` to preserve history, NO `git commit`. NO global import changes. Report results upon completion.
---
## 🎯 Migration Principles
1. **Atomic migration units** - Core file + all related tests/mocks migrate together
2. **Zero `any` types** - Use proper TypeScript throughout
3. **Leverage existing types** - Reuse established definitions
4. **Type inheritance** - Derivatives extend base component types
5. **Strategic placement** - File types for maximum discoverability
6. **Surgical improvements** - Enhance existing types during migration
---
## Step 0: Dependency Check (MANDATORY)
**Command:**
```bash
grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" superset-frontend/{filename}
```
**Decision:**
- ✅ No matches → Proceed with atomic migration (core + tests + mocks)
- ❌ Matches found → EXIT with dependency report (see format below)
---
## Step 1: Identify Related Files (REQUIRED)
**Atomic Migration Scope:**
For core file `src/utils/example.js`, also migrate:
- `src/utils/example.test.js` / `src/utils/example.test.jsx`
- `src/utils/example.spec.js` / `src/utils/example.spec.jsx`
- `src/utils/__mocks__/example.js`
- Any other related test/mock files found by pattern matching
**Find all related test and mock files:**
```bash
# Pattern-based search for related files
basename=$(basename {filename} .js)
dirname=$(dirname superset-frontend/{filename})
# Find test files
find "$dirname" -name "${basename}.test.js" -o -name "${basename}.test.jsx"
find "$dirname" -name "${basename}.spec.js" -o -name "${basename}.spec.jsx"
# Find mock files
find "$dirname" -name "__mocks__/${basename}.js"
find "$dirname" -name "${basename}.mock.js"
```
**Migration Requirement:** All discovered related files MUST be migrated together as one atomic unit.
**Test File Creation:** If NO test files exist for the core file, CREATE a minimal test file using the following pattern:
- Location: Same directory as core file
- Name: `{basename}.test.ts` (e.g., `DebouncedMessageQueue.test.ts`)
- Content: Basic test structure importing and testing the main functionality
- Use proper TypeScript types in test file
---
## 🗺️ Type Reference Map
### From `@superset-ui/core`
```typescript
// Data & Query
QueryFormData, QueryData, JsonObject, AnnotationData, AdhocMetric
LatestQueryFormData, GenericDataType, DatasourceType, ExtraFormData
DataMaskStateWithId, NativeFilterScope, NativeFiltersState, NativeFilterTarget
// UI & Theme
FeatureFlagMap, LanguagePack, ColorSchemeConfig, SequentialSchemeConfig
```
### From `@superset-ui/chart-controls`
```typescript
Dataset, ColumnMeta, ControlStateMapping
```
### From Local Types (`src/types/`)
```typescript
// Authentication
User, UserWithPermissionsAndRoles, BootstrapUser, PermissionsAndRoles
// Dashboard
Dashboard, DashboardState, DashboardInfo, DashboardLayout, LayoutItem
ComponentType, ChartConfiguration, ActiveFilters
// Charts
Chart, ChartState, ChartStatus, ChartLinkedDashboard, Slice, SaveActionType
// Data
Datasource, Database, Owner, Role
// UI Components
TagType, FavoriteStatus, Filter, ImportResourceName
```
### From Domain Types
```typescript
// src/dashboard/types.ts
RootState, ChartsState, DatasourcesState, FilterBarOrientation
ChartCrossFiltersConfig, ActiveTabs, MenuKeys
// src/explore/types.ts
ExplorePageInitialData, ExplorePageState, ExploreResponsePayload, OptionSortType
// src/SqlLab/types.ts
[SQL Lab specific types]
```
---
## 🏗️ Type Organization Strategy
### Type Placement Hierarchy
1. **Component-Colocated** (90% of cases)
```typescript
// Same file as component
interface MyComponentProps {
title: string;
onClick: () => void;
}
```
2. **Feature-Shared**
```typescript
// src/[domain]/components/[Feature]/types.ts
export interface FilterConfiguration {
filterId: string;
targets: NativeFilterTarget[];
}
```
3. **Domain-Wide**
```typescript
// src/[domain]/types.ts
export interface ExploreFormData extends QueryFormData {
viz_type: string;
}
```
4. **Global**
```typescript
// src/types/[TypeName].ts
export interface ApiResponse<T> {
result: T;
count?: number;
}
```
### Type Discovery Commands
```bash
# Search existing types before creating
find superset-frontend/src -name "types.ts" -exec grep -l "[TypeConcept]" {} \;
grep -r "interface.*Props\|type.*Props" superset-frontend/src/
```
### Derivative Component Patterns
**Rule:** Components that extend others should extend their type interfaces.
```typescript
// ✅ Base component type
interface SelectProps {
value: string | number;
options: SelectOption[];
onChange: (value: string | number) => void;
disabled?: boolean;
}
// ✅ Derivative extends base
interface ChartSelectProps extends SelectProps {
charts: Chart[];
onChartSelect: (chart: Chart) => void;
}
// ✅ Derivative with modified props
interface DatabaseSelectProps extends Omit<SelectProps, 'value' | 'onChange'> {
value: number; // Narrowed type
onChange: (databaseId: number) => void; // Specific signature
}
```
**Common Patterns:**
- **Extension:** `extends BaseProps` - adds new props
- **Omission:** `Omit<BaseProps, 'prop'>` - removes props
- **Modification:** `Omit<BaseProps, 'prop'> & { prop: NewType }` - changes prop type
- **Restriction:** Override with narrower types (union → specific)
---
## 📋 Migration Recipe
### Step 2: File Conversion
```bash
# Use git mv to preserve history
git mv component.js component.ts
git mv Component.jsx Component.tsx
```
### Step 3: Import & Type Setup
```typescript
// Import order (enforced by linting)
import { FC, ReactNode } from 'react';
import { JsonObject, QueryFormData } from '@superset-ui/core';
import { Dataset } from '@superset-ui/chart-controls';
import type { Dashboard } from 'src/types/Dashboard';
```
### Step 4: Function & Component Typing
```typescript
// Functions with proper parameter/return types
export function processData(
data: Dataset[],
config: JsonObject
): ProcessedData[] {
// implementation
}
// Component props with inheritance
interface ComponentProps extends BaseProps {
data: Chart[];
onSelect: (id: number) => void;
}
const Component: FC<ComponentProps> = ({ data, onSelect }) => {
// implementation
};
```
### Step 5: State & Redux Typing
```typescript
// Hooks with specific types
const [data, setData] = useState<Chart[]>([]);
const [selected, setSelected] = useState<number | null>(null);
// Redux with existing RootState
const mapStateToProps = (state: RootState) => ({
charts: state.charts,
user: state.user,
});
```
---
## 🧠 Type Debugging Strategies (Real-World Learnings)
### The Evolution of Type Approaches
When you hit type errors, follow this debugging evolution:
#### 1. ❌ Idealized Union Types (First Attempt)
```typescript
// Looks clean but doesn't match reality
type DatasourceInput = Datasource | QueryEditor;
```
**Problem**: Real calling sites pass variations, not exact types.
#### 2. ❌ Overly Precise Types (Second Attempt)
```typescript
// Tried to match exact calling signatures
type DatasourceInput =
| IDatasource // From DatasourcePanel
| (QueryEditor & { columns: ColumnMeta[] }); // From SaveQuery
```
**Problem**: Too rigid, doesn't handle legacy variations.
#### 3. ✅ Flexible Interface (Final Solution)
```typescript
// Captures what the function actually needs
interface DatasourceInput {
name?: string | null; // Allow null for compatibility
datasource_name?: string | null; // Legacy variations
columns?: any[]; // Multiple column types accepted
database?: { id?: number };
// ... other optional properties
}
```
**Success**: Works with all calling sites, focuses on function needs.
### Type Debugging Process
1. **Start with compilation errors** - they show exact mismatches
2. **Examine actual usage** - look at calling sites, not idealized types
3. **Build flexible interfaces** - capture what functions need, not rigid contracts
4. **Iterate based on downstream validation** - let calling sites guide your types
---
## 🚨 Anti-Patterns to Avoid
```typescript
// ❌ Never use any
const obj: any = {};
// ✅ Use proper types
const obj: Record<string, JsonObject> = {};
// ❌ Don't recreate base component props
interface ChartSelectProps {
value: string; // Duplicated from SelectProps
onChange: () => void; // Duplicated from SelectProps
charts: Chart[]; // New prop
}
// ✅ Inherit and extend
interface ChartSelectProps extends SelectProps {
charts: Chart[]; // Only new props
}
// ❌ Don't create ad-hoc type variations
interface UserInfo {
name: string;
email: string;
}
// ✅ Extend existing types (DRY principle)
import { User } from 'src/types/bootstrapTypes';
type UserDisplayInfo = Pick<User, 'firstName' | 'lastName' | 'email'>;
// ❌ Don't create overly rigid unions
type StrictInput = ExactTypeA | ExactTypeB;
// ✅ Create flexible interfaces for function parameters
interface FlexibleInput {
// Focus on what the function actually needs
commonProperty: string;
optionalVariations?: any; // Allow for legacy variations
}
```
## 📍 DRY Type Guidelines (WHERE TYPES BELONG)
### Type Placement Rules
**CRITICAL**: Type variations must live close to where they belong, not scattered across files.
#### ✅ Proper Type Organization
```typescript
// ❌ Don't create one-off interfaces in utility files
// src/utils/datasourceUtils.ts
interface DatasourceInput { /* custom interface */ } // Wrong!
// ✅ Use existing types or extend them in their proper domain
// src/utils/datasourceUtils.ts
import { IDatasource } from 'src/explore/components/DatasourcePanel';
import { QueryEditor } from 'src/SqlLab/types';
// Create flexible interface that references existing types
interface FlexibleDatasourceInput {
// Properties that actually exist across variations
}
```
#### Type Location Hierarchy
1. **Domain Types**: `src/{domain}/types.ts` (dashboard, explore, SqlLab)
2. **Component Types**: Co-located with components
3. **Global Types**: `src/types/` directory
4. **Utility Types**: Only when they truly don't belong elsewhere
#### ✅ DRY Type Patterns
```typescript
// ✅ Extend existing domain types
interface SaveQueryData extends Pick<QueryEditor, 'sql' | 'dbId' | 'catalog'> {
columns: ColumnMeta[]; // Add what's needed
}
// ✅ Create flexible interfaces for cross-domain utilities
interface CrossDomainInput {
// Common properties that exist across different source types
name?: string | null; // Accommodate legacy null values
// Only include properties the function actually uses
}
```
---
## 🎯 PropTypes Auto-Generation (Elegant Approach)
**IMPORTANT**: Superset has `babel-plugin-typescript-to-proptypes` configured to automatically generate PropTypes from TypeScript interfaces. Use this instead of manual PropTypes duplication!
### ❌ Manual PropTypes Duplication (Avoid This)
```typescript
export interface MyComponentProps {
title: string;
count?: number;
}
// 8+ lines of manual PropTypes duplication 😱
const propTypes = PropTypes.shape({
title: PropTypes.string.isRequired,
count: PropTypes.number,
});
export default propTypes;
```
### ✅ Auto-Generated PropTypes (Use This)
```typescript
import { InferProps } from 'prop-types';
export interface MyComponentProps {
title: string;
count?: number;
}
// Single validator function - babel plugin auto-generates PropTypes! ✨
export default function MyComponentValidator(props: MyComponentProps) {
return null; // PropTypes auto-assigned by babel-plugin-typescript-to-proptypes
}
// Optional: For consumers needing PropTypes type inference
export type MyComponentPropsInferred = InferProps<typeof MyComponentValidator>;
```
### Migration Pattern for Type-Only Files
**When migrating type-only files with manual PropTypes:**
1. **Keep the TypeScript interfaces** (single source of truth)
2. **Replace manual PropTypes** with validator function
3. **Remove PropTypes imports** and manual shape definitions
4. **Add InferProps import** if type inference needed
**Example Migration:**
```typescript
// Before: 25+ lines with manual PropTypes duplication
export interface AdhocFilterType { /* ... */ }
const adhocFilterTypePropTypes = PropTypes.oneOfType([...]);
// After: 3 lines with auto-generation
export interface AdhocFilterType { /* ... */ }
export default function AdhocFilterValidator(props: { filter: AdhocFilterType }) {
return null; // Auto-generated PropTypes by babel plugin
}
```
### Component PropTypes Pattern
**For React components, the babel plugin works automatically:**
```typescript
interface ComponentProps {
title: string;
onClick: () => void;
}
const MyComponent: FC<ComponentProps> = ({ title, onClick }) => {
// Component implementation
};
// PropTypes automatically generated by babel plugin - no manual work needed!
export default MyComponent;
```
### Auto-Generation Benefits
- ✅ **Single source of truth**: TypeScript interfaces drive PropTypes
- ✅ **No duplication**: Eliminate 15-20 lines of manual PropTypes code
- ✅ **Automatic updates**: Changes to TypeScript automatically update PropTypes
- ✅ **Type safety**: Compile-time checking ensures PropTypes match interfaces
- ✅ **Backward compatibility**: Existing JavaScript components continue working
### Babel Plugin Configuration
The plugin is already configured in `babel.config.js`:
```javascript
['babel-plugin-typescript-to-proptypes', { loose: true }]
```
**No additional setup required** - just use TypeScript interfaces and the plugin handles the rest!
---
## 🧪 Test File Migration Patterns
### Test File Priority
- **Always migrate test files** alongside production files
- **Test files are often leaf nodes** - good starting candidates
- **Create tests if missing** - Leverage new TypeScript types for better test coverage
### Test-Specific Type Patterns
```typescript
// Mock interfaces for testing
interface MockStore {
getState: () => Partial<RootState>; // Partial allows minimal mocking
}
// Type-safe mocking for complex objects
const mockDashboardInfo: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
};
// Sinon stub typing
let postStub: sinon.SinonStub;
beforeEach(() => {
postStub = sinon.stub(SupersetClient, 'post');
});
// Use stub reference instead of original method
expect(postStub.callCount).toBe(1);
expect(postStub.getCall(0).args[0].endpoint).toMatch('/api/');
```
### Test Migration Recipe
1. **Migrate production file first** (if both need migration)
2. **Update test imports** to point to `.ts/.tsx` files
3. **Add proper mock typing** using `Partial<T> as T` pattern
4. **Fix stub typing** - Use stub references, not original methods
5. **Verify all tests pass** with TypeScript compilation
---
## 🔧 Type Conflict Resolution
### Multiple Type Definitions Issue
**Problem**: Same type name defined in multiple files causes compilation errors.
**Example**: `DashboardInfo` defined in both:
- `src/dashboard/reducers/types.ts` (minimal)
- `src/dashboard/components/Header/types.ts` (different shape)
- `src/dashboard/types.ts` (complete - used by RootState)
### Resolution Strategy
1. **Identify the authoritative type**:
```bash
# Find which type is used by RootState/main interfaces
grep -r "DashboardInfo" src/dashboard/types.ts
```
2. **Use import from authoritative source**:
```typescript
// ✅ Import from main domain types
import { RootState, DashboardInfo } from 'src/dashboard/types';
// ❌ Don't import from component-specific files
import { DashboardInfo } from 'src/dashboard/components/Header/types';
```
3. **Mock complex types in tests**:
```typescript
// For testing - provide minimal required fields
const mockInfo: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
// Only provide fields actually used in test
};
```
### Type Hierarchy Discovery Commands
```bash
# Find all definitions of a type
grep -r "interface.*TypeName\|type.*TypeName" src/
# Find import usage patterns
grep -r "import.*TypeName" src/
# Check what RootState uses
grep -A 10 -B 10 "TypeName" src/*/types.ts
```
---
## Agent Constraints (CRITICAL)
1. **Use git mv** - Run `git mv file.js file.ts` to preserve git history, but NO `git commit`
2. **NO global import changes** - Don't update imports across codebase
3. **Type files OK** - Can modify existing type files to improve/align types
4. **Single-File TypeScript Validation** (CRITICAL) - tsc has known issues with multi-file compilation:
- **Core Issue**: TypeScript's `tsc` has documented problems validating multiple files simultaneously in complex projects
- **Solution**: ALWAYS validate files one at a time using individual `tsc` calls
- **Command Pattern**: `cd superset-frontend && npx tscw --noEmit --allowJs --composite false --project tsconfig.json {single-file-path}`
- **Why**: Multi-file validation can produce false positives, miss real errors, and conflict during parallel agent execution
5. **Downstream Impact Validation** (CRITICAL) - Your migration affects calling sites:
- **Find downstream files**: `find superset-frontend/src -name "*.tsx" -o -name "*.ts" | xargs grep -l "your-core-filename" 2>/dev/null || echo "No files found"`
- **Validate each downstream file individually**: `cd superset-frontend && npx tscw --noEmit --allowJs --composite false --project tsconfig.json {each-downstream-file}`
- **Fix type mismatches** you introduced in calling sites
- **NEVER ignore downstream errors** - they indicate your types don't match reality
6. **Avoid Project-Wide Validation During Migration**:
- **NEVER use `npm run type`** during parallel agent execution - produces unreliable results
- **Single-file validation is authoritative** - trust individual file checks over project-wide scans
6. **ESLint validation** - Run `npm run eslint -- --fix {file}` for each migrated file to auto-fix formatting/linting issues
6. Zero `any` types - use proper TypeScript types
7. Search existing types before creating new ones
8. Follow patterns from this guide
---
## Success Report Format
```
SUCCESS: Atomic Migration of {core-filename}
## Files Migrated (Atomic Unit)
- Core: {core-filename} → {core-filename.ts/tsx}
- Tests: {list-of-test-files} → {list-of-test-files.ts/tsx} OR "CREATED: {basename}.test.ts"
- Mocks: {list-of-mock-files} → {list-of-mock-files.ts}
- Type files modified: {list-of-type-files}
## Types Created/Improved
- {TypeName}: {location} ({scope}) - {rationale}
- {ExistingType}: enhanced in {location} - {improvement-description}
## Documentation Recommendations
- ADD_TO_DIRECTORY: {TypeName} - {reason}
- NO_DOCUMENTATION: {TypeName} - {reason}
## Quality Validation
- **Single-File TypeScript Validation**: ✅ PASS - Core files individually validated
- Core file: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {core-file}`
- Test files: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {test-file}` (if exists)
- **Downstream Impact Check**: ✅ PASS - Found {N} files importing this module, all validate individually
- Downstream files: {list-of-files-that-import-your-module}
- Individual validation: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {each-downstream-file}`
- **ESLint validation**: ✅ PASS (using `npm run eslint -- --fix {files}` to auto-fix formatting)
- **Zero any types**: ✅ PASS
- **Local imports resolved**: ✅ PASS
- **Functionality preserved**: ✅ PASS
- **Tests pass** (if test file): ✅ PASS
- **Follow-up action required**: {YES/NO}
## Validation Strategy Notes
- **Single-file approach used**: Avoided multi-file tsc validation due to known TypeScript compilation issues
- **Project-wide validation skipped**: `npm run type` not used during parallel migration to prevent false positives
## Migration Learnings
- Type conflicts encountered: {describe any multiple type definitions}
- Mock patterns used: {describe test mocking approaches}
- Import hierarchy decisions: {note authoritative type sources used}
- PropTypes strategy: {AUTO_GENERATED via babel plugin | MANUAL_DUPLICATION_REMOVED | N/A}
## Improvement Suggestions for Documentation
- AGENT.md enhancement: {suggest additions to migration guide}
- Common pattern identified: {note reusable patterns for future migrations}
```
---
## Dependency Block Report Format
```
DEPENDENCY_BLOCK: Cannot migrate {filename}
## Blocking Dependencies
- {path}: {type} - {usage} - {priority}
## Impact Analysis
- Estimated types: {number}
- Expected locations: {list}
- Cross-domain: {YES/NO}
## Recommended Order
{ordered-list}
```
---
## 📚 Quick Reference
**Type Utilities:**
- `Record<K, V>` - Object with specific key/value types
- `Partial<T>` - All properties optional
- `Pick<T, K>` - Subset of properties
- `Omit<T, K>` - Exclude specific properties
- `NonNullable<T>` - Exclude null/undefined
**Event Types:**
- `MouseEvent<HTMLButtonElement>`
- `ChangeEvent<HTMLInputElement>`
- `FormEvent<HTMLFormElement>`
**React Types:**
- `FC<Props>` - Functional component
- `ReactNode` - Any renderable content
- `CSSProperties` - Style objects
---
**Remember:** Every type should add value and clarity. The goal is meaningful type safety that catches bugs and improves developer experience.

View File

@@ -1,199 +0,0 @@
# JS-to-TS Coordinator Workflow
**Role:** Strategic migration coordination - select leaf-node files, trigger agents, review results, handle integration, manage dependencies.
---
## 1. Core File Selection Strategy
**Target ONLY Core Files**: Coordinators identify core files (production code), agents handle related tests/mocks atomically.
**File Analysis Commands**:
```bash
# Find CORE files with no JS/JSX dependencies (exclude tests/mocks) - SIZE PRIORITIZED
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | xargs wc -l | sort -n | head -20
# Alternative: Get file sizes in lines with paths
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | while read file; do
lines=$(wc -l < "$file")
echo "$lines $file"
done | sort -n | head -20
# Check dependencies for core files only (start with smallest)
for file in <core-files-sorted-by-size>; do
echo "=== $file ($(wc -l < "$file") lines) ==="
grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" "$file" || echo "✅ LEAF CANDIDATE"
done
# Identify heavily imported files (migrate last)
grep -r "from.*utils/common" superset-frontend/src/ | wc -l
# Quick leaf analysis with size priority
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | head -30 | while read file; do
deps=$(grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" "$file" | wc -l)
lines=$(wc -l < "$file")
if [ "$deps" -eq 0 ]; then
echo "✅ LEAF: $lines lines - $file"
fi
done | sort -n
```
**Priority Order** (Smallest files first for easier wins):
1. **Small leaf files** (<50 lines) - No JS/JSX imports, quick TypeScript conversion
2. **Medium leaf files** (50-200 lines) - Self-contained utilities and helpers
3. **Small dependency files** (<100 lines) - Import only already-migrated files
4. **Larger components** (200+ lines) - Complex but well-contained functionality
5. **Core foundational files** (utils/common.js, controls.jsx) - migrate last regardless of size
**Size-First Benefits**:
- Faster completion builds momentum
- Earlier validation of migration patterns
- Easier rollback if issues arise
- Better success rate for agent learning
**Migration Unit**: Each agent call migrates:
- 1 core file (primary target)
- All related `*.test.js/jsx` files
- All related `*.mock.js` files
- All related `__mocks__/` files
---
## 2. Task Creation & Agent Control
### Task Triggering
When triggering the `/js-to-ts` command:
- **Task Title**: Use the core filename as the task title (e.g., "DebouncedMessageQueue.js migration", "hostNamesConfig.js migration")
- **Task Description**: Include the full relative path to help agent locate the file
- **Reference**: Point agent to [AGENT.md](./AGENT.md) for technical instructions
### Post-Processing Workflow
After each agent completes:
1. **Review Agent Report**: Always read and analyze the complete agent report
2. **Share Summary**: Provide user with key highlights from agent's work:
- Files migrated (core + tests/mocks)
- Types created or improved
- Any validation issues or coordinator actions needed
3. **Quality Assessment**: Evaluate agent's TypeScript implementation against criteria:
-**Type Usage**: Proper types used, no `any` types
-**Type Filing**: Types placed in correct hierarchy (component → feature → domain → global)
-**Side Effects**: No unintended changes to other files
-**Import Alignment**: Proper .ts/.tsx import extensions
4. **Integration Decision**:
- **COMMIT**: If agent work is complete and high quality
- **FIX & COMMIT**: If minor issues need coordinator fixes
- **ROLLBACK**: If major issues require complete rework
5. **Next Action**: Ask user preference - commit this work or trigger next migration
---
## 3. Integration Decision Framework
**Automatic Integration** ✅:
- `npm run type` passes without errors
- Agent created clean TypeScript with proper types
- Types appropriately filed in hierarchy
**Coordinator Integration** (Fix Side-Effects) 🔧:
- `npm run type` fails BUT agent's work is high quality
- Good type usage, proper patterns, well-organized
- Side-effects are manageable TypeScript compilation errors
- **Coordinator Action**: Integrate the change, then fix global compilation issues
**Rollback Only** ❌:
- Agent introduced `any` types or poor type choices
- Types poorly organized or conflicting with existing patterns
- Fundamental approach issues requiring complete rework
**Integration Process**:
1. **Review**: Agent already used `git mv` to preserve history
2. **Fix Side-Effects**: Update dependent files with proper import extensions
3. **Resolve Types**: Fix any cascading type issues across codebase
4. **Validate**: Ensure `npm run type` passes after fixes
---
## 4. Common Integration Patterns
**Common Side-Effects (Expect These)**:
- **Type import conflicts**: Multiple definitions of same type name
- **Mock object typing**: Tests need complete type satisfaction
- **Stub method references**: Use stub vars instead of original methods
**Coordinator Fixes (Standard Process)**:
1. **Import Resolution**:
```bash
# Find authoritative type source
grep -r "TypeName" src/*/types.ts
# Import from domain types (src/dashboard/types.ts) not component types
```
2. **Test Mock Completion**:
```typescript
// Use Partial<T> as T pattern for minimal mocking
const mockDashboard: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
};
```
3. **Stub Reference Fixes**:
```typescript
// ✅ Use stub variable
expect(postStub.callCount).toBe(1);
// ❌ Don't use original method
expect(SupersetClient.post.callCount).toBe(1);
```
4. **Validation Commands**:
```bash
npm run type # TypeScript compilation
npm test -- filename # Test functionality
git status # Should show rename, not add/delete
```
---
## 5. File Categories for Planning
### Leaf Files (Start Here)
**Self-contained files with minimal JS/JSX dependencies**:
- Test files (80 files) - Usually only import the file being tested
- Utility files without internal dependencies
- Components importing only external libraries
### Heavily Imported Files (Migrate Last)
**Core files that many others depend on**:
- `utils/common.js` - Core utility functions
- `utils/reducerUtils.js` - Redux helpers
- `@superset-ui/core` equivalent files
- Major state management files (`explore/store.js`, `dashboard/actions/`)
### Complex Components (Middle Priority)
**Large files requiring careful type analysis**:
- `components/Datasource/DatasourceEditor.jsx` (1,809 lines)
- `explore/components/controls/AnnotationLayerControl/AnnotationLayer.jsx` (1,031 lines)
- `explore/components/ExploreViewContainer/index.jsx` (911 lines)
---
## 6. Success Metrics & Continuous Improvement
**Per-File Gates**:
- ✅ `npm run type` passes after each migration
- ✅ Zero `any` types introduced
- ✅ All imports properly typed
- ✅ Types filed in correct hierarchy
**Linear Scheduling**:
When agents report `DEPENDENCY_BLOCK`:
- Queue dependencies in linear order
- Process one file at a time to avoid conflicts
- Handle cascading type changes between files
**After Each Migration**:
1. **Update guides** with new patterns discovered
2. **Document coordinator fixes** that become common
3. **Enhance agent instructions** based on recurring issues
4. **Track success metrics** - automatic vs coordinator integration rates

View File

@@ -1,76 +0,0 @@
# JavaScript to TypeScript Migration Project
Progressive migration of 219 JS/JSX files to TypeScript in Apache Superset frontend.
## 📁 Project Documentation
- **[AGENT.md](./AGENT.md)** - Complete technical migration guide for agents (includes type reference, patterns, validation)
- **[COORDINATOR.md](./COORDINATOR.md)** - Strategic workflow for coordinators (file selection, task management, integration)
## 🎯 Quick Start
**For Agents:** Read [AGENT.md](./AGENT.md) for complete migration instructions
**For Coordinators:** Read [COORDINATOR.md](./COORDINATOR.md) for workflow and [AGENT.md](./AGENT.md) for supervision
**Command:** `/js-to-ts <filename>` - See [../../commands/js-to-ts.md](../../commands/js-to-ts.md)
## 📊 Migration Progress
**Scope**: 219 files total (112 JS + 107 JSX)
- Production files: 139 (63%)
- Test files: 80 (37%)
**Strategy**: Leaf-first migration with dependency-aware coordination
### Completed Migrations ✅
1. **roundDecimal** - `plugins/legacy-plugin-chart-map-box/src/utils/roundDecimal.js`
- Migrated core + test files
- Added proper TypeScript function signature with optional precision parameter
- All tests pass
2. **timeGrainSqlaAnimationOverrides** - `src/explore/controlPanels/timeGrainSqlaAnimationOverrides.js`
- Migrated to TypeScript with ControlPanelState and Dataset types
- Added TimeGrainOverrideState interface for return type
- Used type guards for safe property access
3. **DebouncedMessageQueue** - `src/utils/DebouncedMessageQueue.js`
- Migrated to TypeScript with proper generics
- Created DebouncedMessageQueueOptions interface
- **CREATED test file** with 4 comprehensive test cases
- Excellent class property typing with private/readonly modifiers
**Files Migrated**: 3/219 (1.4%)
**Tests Created**: 2 (roundDecimal had existing, DebouncedMessageQueue created)
### Next Candidates (Leaf Nodes) 🎯
**Identified leaf files with no JS/JSX dependencies:**
- `src/utils/hostNamesConfig.js` - Domain configuration utility
- `src/explore/controlPanels/Separator.js` - Control panel configuration
- `src/middleware/loggerMiddleware.js` - Logging middleware
**Migration Quality**: All completed migrations have:
- ✅ Zero `any` types
- ✅ Proper TypeScript compilation
- ✅ ESLint validation passed
- ✅ Test coverage (created where missing)
---
## 📈 Success Metrics
**Per-File Gates**:
-`npm run type` passes after each migration
- ✅ Zero `any` types introduced
- ✅ All imports properly typed
- ✅ Types filed in correct hierarchy
**Overall Progress**:
- **Automatic Integration Rate**: 100% (3/3 migrations required no coordinator fixes)
- **Test Coverage**: Improved (1 new test file created)
- **Type Safety**: Enhanced with proper interfaces and generics
---
*This is a claudette-managed progressive refactor. All documentation and coordination resources are organized under `.claude/projects/js-to-ts/`*

View File

@@ -1,36 +0,0 @@
# .coveragerc to control coverage.py
[run]
branch = True
source = superset
# omit = bad_file.py
[paths]
source =
superset/
*/site-packages/
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain about missing debug-only code:
def __repr__
if self\.debug
# Don't complain if tests don't hit defensive assertion code:
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
# Ignore importlib backport
from importlib
if TYPE_CHECKING:
#fail_under = 100
show_missing = True

View File

@@ -1,125 +0,0 @@
---
description: Apache Superset development standards and guidelines for Cursor IDE
globs: ["**/*.py", "**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.sql", "**/*.md"]
alwaysApply: true
---
# Apache Superset Development Standards for Cursor IDE
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
**These migrations are actively happening - avoid deprecated patterns:**
### Frontend Modernization
- **NO `any` types** - Use proper TypeScript types
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
- **NO Enzyme** - Use React Testing Library/Jest (Enzyme fully removed)
- **Use @superset-ui/core** - Don't import Ant Design directly
### Testing Strategy Migration
- **Prefer unit tests** over integration tests
- **Prefer integration tests** over Cypress end-to-end tests
- **Cypress is last resort** - Actively moving away from Cypress
- **Use Jest + React Testing Library** for component testing
### Backend Type Safety
- **Add type hints** - All new Python code needs proper typing
- **MyPy compliance** - Run `pre-commit run mypy` to validate
- **SQLAlchemy typing** - Use proper model annotations
## Code Standards
### TypeScript Frontend
- **NO `any` types** - Use proper TypeScript
- **Functional components** with hooks
- **@superset-ui/core** for UI components (not direct antd)
- **Jest** for testing (NO Enzyme)
- **Redux** for global state, hooks for local
### Python Backend
- **Type hints required** for all new code
- **MyPy compliant** - run `pre-commit run mypy`
- **SQLAlchemy models** with proper typing
- **pytest** for testing
### Apache License Headers
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
- **LLM instruction files are excluded** - Files like LLMS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
## Key Directory Structure
```
superset/
├── superset/ # Python backend (Flask, SQLAlchemy)
│ ├── views/api/ # REST API endpoints
│ ├── models/ # Database models
│ └── connectors/ # Database connections
├── superset-frontend/src/ # React TypeScript frontend
│ ├── components/ # Reusable components
│ ├── explore/ # Chart builder
│ ├── dashboard/ # Dashboard interface
│ └── SqlLab/ # SQL editor
├── superset-frontend/packages/
│ └── superset-ui-core/ # UI component library (USE THIS)
├── tests/ # Python/integration tests
├── docs/ # Documentation (UPDATE FOR CHANGES)
└── UPDATING.md # Breaking changes log
```
## Architecture Patterns
### Dataset-Centric Approach
Charts built from enriched datasets containing:
- Dimension columns with labels/descriptions
- Predefined metrics as SQL expressions
- Self-service analytics within defined contexts
### Security & Features
- **RBAC**: Role-based access via Flask-AppBuilder
- **Feature flags**: Control feature rollouts
- **Row-level security**: SQL-based data access control
## Test Utilities
### Python Test Helpers
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
- **`@with_config`** - Config mocking decorator
- **`@with_feature_flags`** - Feature flag testing
- **`login_as()`, `login_as_admin()`** - Authentication helpers
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
### TypeScript Test Helpers
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
- **`createWrapper()`** - Redux/Router/Theme wrapper
- **`selectOption()`** - Select component helper
- **React Testing Library** - NO Enzyme (removed)
## Pre-commit Validation
**Use pre-commit hooks for quality validation:**
```bash
# Install hooks
pre-commit install
# Quick validation (faster than --all-files)
pre-commit run # Staged files only
pre-commit run mypy # Python type checking
pre-commit run prettier # Code formatting
pre-commit run eslint # Frontend linting
```
## Development Guidelines
- **Documentation**: Update docs/ for any user-facing changes
- **Breaking Changes**: Add to UPDATING.md
- **Docstrings**: Required for new functions/classes
- **Follow existing patterns**: Mimic code style, use existing libraries and utilities
- **Type Safety**: This codebase is actively modernizing toward full TypeScript and type safety
- **Always run `pre-commit run`** to validate changes before committing
---
**Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

View File

@@ -1,20 +0,0 @@
# Keep this in sync with the base image in the main Dockerfile (ARG PY_VER)
FROM python:3.11.13-trixie AS base
# Install system dependencies that Superset needs
# This layer will be cached across Codespace sessions
RUN apt-get update && apt-get install -y \
libsasl2-dev \
libldap2-dev \
libpq-dev \
tmux \
gh \
&& rm -rf /var/lib/apt/lists/*
# Install uv for fast Python package management
# This will also be cached in the image
RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \
echo 'export PATH="/root/.cargo/bin:$PATH"' >> /etc/bash.bashrc
# Set the cargo/bin directory in PATH for all users
ENV PATH="/root/.cargo/bin:${PATH}"

View File

@@ -1,5 +0,0 @@
# Superset Development with GitHub Codespaces
For complete documentation on using GitHub Codespaces with Apache Superset, please see:
**[Setting up a Development Environment - GitHub Codespaces](https://superset.apache.org/docs/contributing/development#github-codespaces-cloud-development)**

View File

@@ -1,62 +0,0 @@
# Superset Codespaces environment setup
# This file is appended to ~/.bashrc during Codespace setup
# Find the workspace directory (handles both 'superset' and 'superset-2' names)
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
if [ -n "$WORKSPACE_DIR" ]; then
# Check if virtual environment exists
if [ -d "$WORKSPACE_DIR/.venv" ]; then
# Activate the virtual environment
source "$WORKSPACE_DIR/.venv/bin/activate"
echo "✅ Python virtual environment activated"
# Verify pre-commit is installed and set up
if command -v pre-commit &> /dev/null; then
echo "✅ pre-commit is available ($(pre-commit --version))"
# Install git hooks if not already installed
if [ -d "$WORKSPACE_DIR/.git" ] && [ ! -f "$WORKSPACE_DIR/.git/hooks/pre-commit" ]; then
echo "🪝 Installing pre-commit hooks..."
cd "$WORKSPACE_DIR" && pre-commit install
fi
else
echo "⚠️ pre-commit not found. Run: pip install pre-commit"
fi
else
echo "⚠️ Python virtual environment not found at $WORKSPACE_DIR/.venv"
echo " Run: cd $WORKSPACE_DIR && .devcontainer/setup-dev.sh"
fi
# Always cd to the workspace directory for convenience
cd "$WORKSPACE_DIR"
fi
# Add helpful aliases for Superset development
alias start-superset="$WORKSPACE_DIR/.devcontainer/start-superset.sh"
alias setup-dev="$WORKSPACE_DIR/.devcontainer/setup-dev.sh"
# Show helpful message on login
echo ""
echo "🚀 Superset Codespaces Environment"
echo "=================================="
# Check if Superset is running
if docker ps 2>/dev/null | grep -q "superset"; then
echo "✅ Superset is running!"
echo " - Check the 'Ports' tab for your live Superset URL"
echo " - Initial startup takes 10-20 minutes"
echo " - Login: admin/admin"
else
echo "⚠️ Superset is not running. Use: start-superset"
# Check if there's a startup log
if [ -f "/tmp/superset-startup.log" ]; then
echo " 📋 Startup log found: cat /tmp/superset-startup.log"
fi
fi
echo ""
echo "Quick commands:"
echo " start-superset - Start Superset with Docker Compose"
echo " setup-dev - Set up Python environment (if not already done)"
echo " pre-commit run - Run pre-commit checks on staged files"
echo ""

View File

@@ -1,20 +0,0 @@
#!/bin/bash
# Script to build and push the devcontainer image to GitHub Container Registry
# This allows caching the image between Codespace sessions
# You'll need to run this with appropriate GitHub permissions
# gh auth login --scopes write:packages
REGISTRY="ghcr.io"
OWNER="apache"
REPO="superset"
TAG="devcontainer-base"
echo "Building devcontainer image..."
docker build -t $REGISTRY/$OWNER/$REPO:$TAG .devcontainer/
echo "Pushing to GitHub Container Registry..."
docker push $REGISTRY/$OWNER/$REPO:$TAG
echo "Done! Update .devcontainer/devcontainer.json to use:"
echo " \"image\": \"$REGISTRY/$OWNER/$REPO:$TAG\""

View File

@@ -1,19 +0,0 @@
{
// Extend the base configuration
"extends": "../devcontainer-base.json",
"name": "Apache Superset Development (Default)",
// Forward ports for development
"forwardPorts": [9001],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
}
},
// Auto-start Superset on Codespace resume
"postStartCommand": ".devcontainer/start-superset.sh"
}

View File

@@ -1,39 +0,0 @@
{
"name": "Apache Superset Development",
// Keep this in sync with the base image in Dockerfile (ARG PY_VER)
// Using the same base as Dockerfile, but non-slim for dev tools
"image": "python:3.11.13-bookworm",
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {
"moby": true,
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
"ghcr.io/devcontainers/features/sshd:1": {
"version": "latest"
}
},
// Run commands after container is created
"postCreateCommand": "chmod +x .devcontainer/setup-dev.sh && .devcontainer/setup-dev.sh",
// VS Code customizations
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}
}
}

View File

@@ -1,66 +0,0 @@
{
"name": "Apache Superset Development",
// Option 1: Use pre-built image directly
// "image": "ghcr.io/apache/superset:devcontainer-base",
// Option 2: Build from Dockerfile with cache (current approach)
"build": {
"dockerfile": "Dockerfile",
"context": ".",
// Cache from the Apache registry image
"cacheFrom": ["ghcr.io/apache/superset:devcontainer-base"]
},
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {
"moby": true,
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
"ghcr.io/devcontainers/features/sshd:1": {
"version": "latest"
}
},
// Forward ports for development
"forwardPorts": [9001],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
}
},
// Run commands after container is created
"postCreateCommand": "bash .devcontainer/setup-dev.sh || echo '⚠️ Setup had issues - run .devcontainer/setup-dev.sh manually'",
// Auto-start Superset after ensuring Docker is ready
// Run in foreground to see any errors, but don't block on failures
"postStartCommand": "bash -c 'echo \"Waiting 30s for services to initialize...\"; sleep 30; .devcontainer/start-superset.sh || echo \"⚠️ Auto-start failed - run start-superset manually\"'",
// Set environment variables
"remoteEnv": {
// Removed automatic venv activation to prevent startup issues
// The setup script will handle this
},
// VS Code customizations
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}
}
}

View File

@@ -1,32 +0,0 @@
#!/bin/bash
# Setup script for Superset Codespaces development environment
echo "🔧 Setting up Superset development environment..."
# The universal image has most tools, just need Superset-specific libs
echo "📦 Installing Superset-specific dependencies..."
sudo apt-get update
sudo apt-get install -y \
libsasl2-dev \
libldap2-dev \
libpq-dev \
tmux \
gh
# Install uv for fast Python package management
echo "📦 Installing uv..."
curl -LsSf https://astral.sh/uv/install.sh | sh
# Add cargo/bin to PATH for uv
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.zshrc
# Install Claude Code CLI via npm
echo "🤖 Installing Claude Code..."
npm install -g @anthropic-ai/claude-code
# Make the start script executable
chmod +x .devcontainer/start-superset.sh
echo "✅ Development environment setup complete!"
echo "🚀 Run '.devcontainer/start-superset.sh' to start Superset"

View File

@@ -1,69 +0,0 @@
#!/bin/bash
# Startup script for Superset in Codespaces
echo "🚀 Starting Superset in Codespaces..."
echo "🌐 Frontend will be available at port 9001"
# Check if MCP is enabled
if [ "$ENABLE_MCP" = "true" ]; then
echo "🤖 MCP Service will be available at port 5008"
fi
# Find the workspace directory (Codespaces clones as 'superset', not 'superset-2')
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
if [ -n "$WORKSPACE_DIR" ]; then
cd "$WORKSPACE_DIR"
echo "📁 Working in: $WORKSPACE_DIR"
else
echo "📁 Using current directory: $(pwd)"
fi
# Check if docker is running
if ! docker info > /dev/null 2>&1; then
echo "⏳ Waiting for Docker to start..."
sleep 5
fi
# Clean up any existing containers
echo "🧹 Cleaning up existing containers..."
docker-compose -f docker-compose-light.yml --profile mcp down
# Start services
echo "🏗️ Building and starting services..."
echo ""
echo "📝 Once started, login with:"
echo " Username: admin"
echo " Password: admin"
echo ""
echo "📋 Running in foreground with live logs (Ctrl+C to stop)..."
# Run docker-compose and capture exit code
if [ "$ENABLE_MCP" = "true" ]; then
echo "🤖 Starting with MCP Service enabled..."
docker-compose -f docker-compose-light.yml --profile mcp up
else
docker-compose -f docker-compose-light.yml up
fi
EXIT_CODE=$?
# If it failed, provide helpful instructions
if [ $EXIT_CODE -ne 0 ] && [ $EXIT_CODE -ne 130 ]; then # 130 is Ctrl+C
echo ""
echo "❌ Superset startup failed (exit code: $EXIT_CODE)"
echo ""
echo "🔄 To restart Superset, run:"
echo " .devcontainer/start-superset.sh"
echo ""
echo "🔧 For troubleshooting:"
echo " # View logs:"
echo " docker-compose -f docker-compose-light.yml logs"
echo ""
echo " # Clean restart (removes volumes):"
echo " docker-compose -f docker-compose-light.yml down -v"
echo " .devcontainer/start-superset.sh"
echo ""
echo " # Common issues:"
echo " - Network timeouts: Just retry, often transient"
echo " - Port conflicts: Check 'docker ps'"
echo " - Database issues: Try clean restart with -v"
fi

View File

@@ -1,29 +0,0 @@
{
// Extend the base configuration
"extends": "../devcontainer-base.json",
"name": "Apache Superset Development with MCP",
// Forward ports for development
"forwardPorts": [9001, 5008],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
},
"5008": {
"label": "MCP Service (Model Context Protocol)",
"onAutoForward": "notify",
"visibility": "private"
}
},
// Auto-start Superset with MCP on Codespace resume
"postStartCommand": "ENABLE_MCP=true .devcontainer/start-superset.sh",
// Environment variables
"containerEnv": {
"ENABLE_MCP": "true"
}
}

View File

@@ -34,6 +34,7 @@
**/*.sqllite
**/*.swp
**/.terser-plugin-cache/
**/.storybook/
**/node_modules/
tests/
@@ -41,8 +42,6 @@ docs/
install/
superset-frontend/cypress-base/
superset-frontend/coverage/
superset-frontend/.temp_cache/
superset/static/assets/
superset-websocket/dist/
venv
.venv

View File

@@ -15,4 +15,4 @@
# limitations under the License.
#
FLASK_APP="superset.app:create_app()"
FLASK_DEBUG=true
FLASK_ENV="development"

3
.gitattributes vendored
View File

@@ -1,4 +1 @@
docker/**/*.sh text eol=lf
*.svg binary
*.ipynb binary
*.geojson binary

44
.github/CODEOWNERS vendored
View File

@@ -1,42 +1,14 @@
# Notify all committers of DB migration changes, per SIP-59
# https://github.com/apache/superset/issues/13351
/superset/migrations/ @apache/superset-committers
/superset/migrations/ @mistercrunch @michael-s-molina @betodealmeida @eschutho @sadpandajoe
# Notify Preset team when ephemeral env settings are changed
.github/workflows/ecs-task-definition.json @robdiciuccio @craig-rueda @rusackas @eschutho @dpgaspar @nytai @mistercrunch
.github/workflows/docker-ephemeral-env.yml @robdiciuccio @craig-rueda @rusackas @eschutho @dpgaspar @nytai @mistercrunch
.github/workflows/ephemeral*.yml @robdiciuccio @craig-rueda @rusackas @eschutho @dpgaspar @nytai @mistercrunch
# Notify some committers of changes in the components
/superset-frontend/src/components/Select/ @michael-s-molina @geido @kgabryje
/superset-frontend/src/components/MetadataBar/ @michael-s-molina @geido @kgabryje
/superset-frontend/src/components/DropdownContainer/ @michael-s-molina @geido @kgabryje
# Notify some committers of changes in the Select component
/superset-frontend/src/components/Select/ @michael-s-molina @geido
# Notify Helm Chart maintainers about changes in it
/helm/superset/ @craig-rueda @dpgaspar @villebro @nytai @michael-s-molina @mistercrunch @rusackas @Antonio-RiveroMartnez
# Notify E2E test maintainers of changes
/superset-frontend/cypress-base/ @sadpandajoe @geido @eschutho @rusackas @betodealmeida @mistercrunch
# Notify PMC members of changes to GitHub Actions
/.github/ @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @kgabryje @dpgaspar @sadpandajoe
# Notify PMC members of changes to required GitHub Actions
/.asf.yaml @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @kgabryje @dpgaspar @Antonio-RiveroMartnez
# Maps are a finicky contribution process we care about
**/*.geojson @villebro @rusackas
/superset-frontend/plugins/legacy-plugin-chart-country-map/ @villebro @rusackas
# Notify PMC members of changes to extension-related files
/superset-core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-extensions-cli/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset/core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset/extensions/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/packages/superset-core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/extensions/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/helm/superset/ @craig-rueda @dpgaspar @villebro

View File

@@ -1,99 +0,0 @@
name: Bug report
description: Report a bug to improve Superset's stability
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Hello Superset Community member! Please keep things tidy by putting your post in the proper place:
🚨 Reporting a security issue: send an email to security@superset.apache.org. DO NOT USE GITHUB ISSUES TO REPORT SECURITY PROBLEMS.
🐛 Reporting a bug: use this form.
🙏 Asking a question or getting help: post in the [Superset Slack chat](http://bit.ly/join-superset-slack) or [GitHub Discussions](https://github.com/apache/superset/discussions) under "Q&A / Help".
💡 Requesting a new feature: Search [GitHub Discussions](https://github.com/apache/superset/discussions) to see if it exists already. If not, add a new post there under "Ideas".
- type: textarea
id: bug-description
attributes:
label: Bug description
description: A clear description of what the bug is, including reproduction steps and expected behavior.
placeholder: |
The bug is that...
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
validations:
required: true
- type: textarea
id: screenshots-recordings
attributes:
label: Screenshots/recordings
description: If applicable, add screenshots or recordings to help explain your problem.
- type: markdown
attributes:
value: |
### Environment
Please specify your environment. If your environment does not match the alternatives, you need to upgrade your environment before submitting the issue as it may have already been fixed. For additional information about the releases, see [Release Process](https://github.com/apache/superset/wiki/Release-Process).
- type: dropdown
id: superset-version
attributes:
label: Superset version
options:
- master / latest-dev
- "5.0.0"
- "4.1.3"
validations:
required: true
- type: dropdown
id: python-version
attributes:
label: Python version
options:
- "3.9"
- "3.10"
- "3.11"
- Not applicable
- I don't know
validations:
required: true
- type: dropdown
id: node-version
attributes:
label: Node version
options:
- "16"
- "17"
- "18 or greater"
- Not applicable
- I don't know
validations:
required: true
- type: dropdown
id: browser
attributes:
label: Browser
options:
- Chrome
- Firefox
- Safari
- Not applicable
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: |
Add any other context about the problem here such as the feature flags that you have enabled, any customizations you have made, the data source you are querying, etc.
- type: checkboxes
id: checklist
attributes:
label: Checklist
description: Make sure to follow these steps before submitting your issue - thank you!
options:
- label: I have searched Superset docs and Slack and didn't find a solution to my problem.
- label: I have searched the GitHub issue tracker and didn't find a similar bug report.
- label: I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
validations:
required: true

50
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,50 @@
---
name: Bug report
about: Create a report to help us improve
labels: "#bug"
---
A clear and concise description of what the bug is.
#### How to reproduce the bug
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
### Expected results
what you expected to happen.
### Actual results
what actually happens.
#### Screenshots
If applicable, add screenshots to help explain your problem.
### Environment
(please complete the following information):
- browser type and version:
- superset version: `superset version`
- python version: `python --version`
- node.js version: `node -v`
- any feature flags active:
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ ] I have reproduced the issue with at least the latest released version of superset.
- [ ] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Add any other context about the problem here.

View File

@@ -1,12 +0,0 @@
---
blank_issues_enabled: false
contact_links:
- name: Feature Request
url: https://github.com/apache/superset/discussions/new?category=ideas
about: Propose a feature request to the Superset community
- name: Q&A
url: https://github.com/apache/superset/discussions/new?category=q-a-help
about: Open a community Q&A thread on GitHub Discussions
- name: Slack
url: https://bit.ly/join-superset-slack
about: Join the Superset Community on Slack for other discussions and assistance

View File

@@ -2,6 +2,7 @@
name: Cosmetic Issue
about: Describe a cosmetic issue with CSS, positioning, layout, labeling, or similar
labels: "cosmetic-issue"
---
## Screenshot

View File

@@ -0,0 +1,18 @@
---
name: Feature request
about: Suggest an idea for this project
labels: "#enhancement"
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -0,0 +1,12 @@
---
name: Security vulnerability
about: Report a security vulnerability or issue
labels: "#security"
---
## DO NOT REPORT SECURITY VULNERABILITIES HERE
Please report security vulnerabilities to private@superset.apache.org.
In the event a community member discovers a security flaw in Superset, it is important to follow the [Apache Security Guidelines](https://www.apache.org/security/committers.html) and release a fix as quickly as possible before public disclosure. Reporting security vulnerabilities through the usual GitHub Issues channel is not ideal as it will publicize the flaw before a fix can be applied.

View File

@@ -1,15 +1,14 @@
---
name: SIP
about: "Superset Improvement Proposal. See SIP-0 (https://github.com/apache/superset/issues/5602) for details. A SIP introduces any major change into Apache Superset's code or process."
labels: sip
title: "[SIP] Your Title Here (do not add SIP number)"
assignees: "apache/superset-committers"
about: Superset Improvement Proposal
labels: "#SIP"
---
*Please make sure you are familiar with the SIP process documented*
[here](https://github.com/apache/superset/issues/5602). The SIP will be numbered by a committer upon acceptance.
(here)[https://github.com/apache/superset/issues/5602]. The SIP number should be the next number after the latest SIP listed [here](https://github.com/apache/superset/issues?q=is%3Aissue+label%3Asip).
## [SIP] Proposal for ...<title>
## [SIP-\<number>] Proposal for <title>
### Motivation

38
.github/SECURITY.md vendored
View File

@@ -1,38 +0,0 @@
# Security Policy
This is a project of the [Apache Software Foundation](https://apache.org) and follows the
ASF [vulnerability handling process](https://apache.org/security/#vulnerability-handling).
## Reporting Vulnerabilities
**⚠️ Please do not file GitHub issues for security vulnerabilities as they are public! ⚠️**
Apache Software Foundation takes a rigorous standpoint in annihilating the security issues
in its software projects. Apache Superset is highly sensitive and forthcoming to issues
pertaining to its features and functionality.
If you have any concern or believe you have found a vulnerability in Apache Superset,
please get in touch with the Apache Superset Security Team privately at
e-mail address [security@superset.apache.org](mailto:security@superset.apache.org).
More details can be found on the ASF website at
[ASF vulnerability reporting process](https://apache.org/security/#reporting-a-vulnerability)
We kindly ask you to include the following information in your report:
- Apache Superset version that you are using
- A sanitized copy of your `superset_config.py` file or any config overrides
- Detailed steps to reproduce the vulnerability
Note that Apache Superset is not responsible for any third-party dependencies that may
have security issues. Any vulnerabilities found in third-party dependencies should be
reported to the maintainers of those projects. Results from security scans of Apache
Superset dependencies found on its official Docker image can be remediated at release time
by extending the image itself.
**Your responsible disclosure and collaboration are invaluable.**
## Extra Information
- [Apache Superset documentation](https://superset.apache.org/docs/security)
- [Common Vulnerabilities and Exposures by release](https://superset.apache.org/docs/security/cves)
- [How Security Vulnerabilities are Reported & Handled in Apache Superset (Blog)](https://preset.io/blog/how-security-vulnerabilities-are-reported-and-handled-in-apache-superset/)

View File

@@ -1,34 +0,0 @@
name: Change Detector
description: Detects file changes for pull request and push events
inputs:
token:
description: GitHub token for authentication
required: true
outputs:
python:
description: Whether Python-related files were changed
value: ${{ steps.change-detector.outputs.python }}
frontend:
description: Whether frontend-related files were changed
value: ${{ steps.change-detector.outputs.frontend }}
docker:
description: Whether docker-related files were changed
value: ${{ steps.change-detector.outputs.docker }}
docs:
description: Whether docs-related files were changed
value: ${{ steps.change-detector.outputs.docs }}
superset-extensions-cli:
description: Whether superset-extensions-cli package-related files were changed
value: ${{ steps.change-detector.outputs.superset-extensions-cli }}
runs:
using: composite
steps:
- name: Detect file changes
id: change-detector
run: |
python --version
python scripts/change_detector.py
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.token }}
GITHUB_OUTPUT: ${{ github.output }}

View File

@@ -1,23 +0,0 @@
name: Label Draft PRs
on:
pull_request:
types:
- opened
- converted_to_draft
jobs:
label-draft:
runs-on: ubuntu-latest
steps:
- name: Check if the PR is a draft
id: check-draft
uses: actions/github-script@v6
with:
script: |
const isDraft = context.payload.pull_request.draft;
core.setOutput('isDraft', isDraft);
- name: Add `review:draft` Label
if: steps.check-draft.outputs.isDraft == 'true'
uses: actions-ecosystem/action-add-labels@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
labels: "review:draft"

View File

@@ -1,58 +0,0 @@
name: 'Setup Python Environment'
description: 'Set up Python and install dependencies with optional configurations.'
inputs:
python-version:
description: 'Python version to set up. Accepts a version number, "current", or "next".'
required: true
default: 'current'
cache:
description: 'Cache dependencies. Options: pip'
required: false
default: 'pip'
requirements-type:
description: 'Type of requirements to install. Options: base, development, default'
required: false
default: 'dev'
install-superset:
description: 'Whether to install Superset itself. If false, only python is installed'
required: false
default: 'true'
runs:
using: 'composite'
steps:
- name: Interpret Python Version
id: set-python-version
shell: bash
run: |
if [ "${{ inputs.python-version }}" = "current" ]; then
echo "PYTHON_VERSION=3.11" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "next" ]; then
# currently disabled in GHA matrixes because of library compatibility issues
echo "PYTHON_VERSION=3.12" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "previous" ]; then
echo "PYTHON_VERSION=3.10" >> $GITHUB_ENV
else
echo "PYTHON_VERSION=${{ inputs.python-version }}" >> $GITHUB_ENV
fi
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: ${{ inputs.cache }}
- name: Install dependencies
run: |
if [ "${{ inputs.install-superset }}" = "true" ]; then
sudo apt-get update && sudo apt-get -y install libldap2-dev libsasl2-dev
pip install --upgrade pip setuptools wheel uv
if [ "${{ inputs.requirements-type }}" = "dev" ]; then
uv pip install --system -r requirements/development.txt
elif [ "${{ inputs.requirements-type }}" = "base" ]; then
uv pip install --system -r requirements/base.txt
fi
uv pip install --system -e .
fi
shell: bash

View File

@@ -1,69 +0,0 @@
name: "Setup Docker Environment"
description: "Reusable steps for setting up QEMU, Docker Buildx, DockerHub login, Supersetbot, and optionally Docker Compose"
inputs:
build:
description: "Used for building?"
required: false
default: "false"
dockerhub-user:
description: "DockerHub username"
required: false
dockerhub-token:
description: "DockerHub token"
required: false
install-docker-compose:
description: "Flag to install Docker Compose"
required: false
default: "true"
login-to-dockerhub:
description: "Whether you want to log into dockerhub"
required: false
default: "true"
outputs: {}
runs:
using: "composite"
steps:
- name: Set up QEMU
if: ${{ inputs.build == 'true' }}
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
if: ${{ inputs.build == 'true' }}
uses: docker/setup-buildx-action@v3
- name: Try to login to DockerHub
if: ${{ inputs.login-to-dockerhub == 'true' }}
continue-on-error: true
uses: docker/login-action@v3
with:
username: ${{ inputs.dockerhub-user }}
password: ${{ inputs.dockerhub-token }}
- name: Install Docker Compose
if: ${{ inputs.install-docker-compose == 'true' }}
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
# Download and save the Docker GPG key in the correct format
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
# Ensure the key file is readable
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the Docker repository using the correct key
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Update package lists and install Docker Compose plugin
sudo apt update
sudo apt install -y docker-compose-plugin
- name: Docker Version Info
shell: bash
run: docker info

View File

@@ -1,40 +0,0 @@
name: 'Setup supersetbot'
description: 'Sets up supersetbot npm lib from the repo or npm'
inputs:
from-npm:
description: 'Install from npm instead of local setup'
required: false
default: 'true' # Defaults to using the local setup
runs:
using: 'composite'
steps:
- name: Setup Node Env
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install supersetbot from npm
if: ${{ inputs.from-npm == 'true' }}
shell: bash
run: npm install -g supersetbot
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
if: ${{ inputs.from-npm == 'false' }}
uses: actions/checkout@v4
with:
repository: apache-superset/supersetbot
path: supersetbot
- name: Setup supersetbot from repo
if: ${{ inputs.from-npm == 'false' }}
shell: bash
working-directory: supersetbot
run: |
# simple trick to install globally with dependencies
npm pack
npm install -g ./supersetbot*.tgz
- name: echo supersetbot version
shell: bash
run: supersetbot version

View File

@@ -1 +0,0 @@
../AGENTS.md

350
.github/dependabot.yml vendored
View File

@@ -1,367 +1,29 @@
version: 2
enable-beta-ecosystems: true
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
- package-ecosystem: "npm"
ignore:
# not until React >= 18.0.0
- dependency-name: "storybook"
- dependency-name: "@storybook*"
# JSDOM v30 doesn't play well with Jest v30
# Source: https://jestjs.io/blog#known-issues
# GH thread: https://github.com/jsdom/jsdom/issues/3492
- dependency-name: "jest-environment-jsdom"
directory: "/superset-frontend/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 30
versioning-strategy: increase
# NOTE: `uv` support is in beta, more details here:
# https://github.com/dependabot/dependabot-core/pull/10040#issuecomment-2696978430
- package-ecosystem: "uv"
directory: "requirements/"
open-pull-requests-limit: 10
- package-ecosystem: "pip"
directory: "/requirements/"
schedule:
interval: "weekly"
interval: "daily"
labels:
- uv
- pip
- dependabot
- package-ecosystem: "npm"
directory: ".github/actions"
schedule:
interval: "daily"
open-pull-requests-limit: 10
versioning-strategy: increase
open-pull-requests-limit: 0
- package-ecosystem: "npm"
directory: "/docs/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-websocket/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-websocket/utils/client-ws-app/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 10
versioning-strategy: increase
# Now for all of our plugins and packages!
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-calendar/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-histogram/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-partition/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-world-map/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-pivot-table/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-chord/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-horizon/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-rose/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-deckgl/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-table/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-country-map/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-map-box/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sankey/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-nvd3/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-word-cloud/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-event-flow/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-paired-t-test/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sankey-loop/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-echarts/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/preset-chart-xy/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-heatmap/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-parallel-coordinates/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sunburst/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-handlebars/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/generator-superset/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-chart-controls/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-core/"
ignore:
# not until React >= 18.0.0
- dependency-name: "react-markdown"
- dependency-name: "remark-gfm"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-demo/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-switchboard/"
schedule:
interval: "daily"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
open-pull-requests-limit: 0

158
.github/labeler.yml vendored
View File

@@ -1,158 +0,0 @@
# TODO (if we can)
# - Label PRs in need of codeowner review
# - viz:charts:xyz labels
# component/design system areas
# - storybook(s)
# - f/e and b/e test changes?
# - product areas (SQL Lab, Explore, Dashboard, etc.)
# - database areas (SQLAlchemy, labelind DBs by driver, etc.)
############################################
# General workflow warnings
# full list of labels is here: https://github.com/apache/superset/labels
############################################
"risk:db-migration":
- changed-files:
- any-glob-to-any-file:
- 'superset/migrations/**'
############################################
# Dependencies
############################################
"dependencies:python":
- changed-files:
- any-glob-to-any-file:
- 'superset/requirements/**'
- 'superset/translations/requirements.txt'
- 'RELEASING/requirements.txt'
"dependencies:npm":
- changed-files:
- any-glob-to-any-file:
- 'superset-frontend/package.json'
- 'superset-frontend/package-lock.json'
- 'superset-embedded-sdk/package.json'
- 'superset-embedded-sdk/package-lock.json'
- 'superset-websocket/package.json'
- 'superset-websocket/package-lock.json'
- 'superset-frontend/cypress-base/package.json'
- 'superset-frontend/cypress-base/package-lock.json'
- 'superset-frontend/packages/**/package.json'
- 'superset-frontend/plugins/**/package.json'
############################################
# Areas of the main codebase
############################################
"doc":
- changed-files:
- any-glob-to-any-file:
- 'docs/**'
"api":
- changed-files:
- any-glob-to-any-file:
- 'superset/**/api.py'
- 'superset/views/core.py'
"i18n":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/**'
"i18n:brazilian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/pt_BR/**'
"i18n:chinese":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/zh/**'
"i18n:traditional-chinese":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/zh_TW/**'
"i18n:dutch":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/nl/**'
"i18n:french":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/fr/**'
"i18n:italian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/it/**'
"i18n:japanese":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/ja/**'
"i18n:korean":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/ko/**'
"i18n:portuguese":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/pt/**'
"i18n:russian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/ru/**'
"i18n:slovak":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/sk/**'
"i18n:ukrainian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/uk/**'
"i18n:spanish":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/es/**'
"i18n:persian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/fa/**'
############################################
# Sub-projects and monorepo packages
############################################
"plugins":
- changed-files:
- any-glob-to-any-file:
- 'superset-frontend/plugins/**'
"packages":
- changed-files:
- any-glob-to-any-file:
- 'superset-frontend/packages/**'
"embedded":
- changed-files:
- any-glob-to-any-file:
- 'superset-embedded-sdk/**'
"github_actions":
- changed-files:
- any-glob-to-any-file:
- '.github/actions/**'
- '.github/workflows/**'

View File

@@ -31,6 +31,19 @@ say() {
fi
}
# default command to run when the `run` input is empty
default-setup-command() {
apt-get-install
pip-upgrade
}
apt-get-install() {
say "::group::apt-get install dependencies"
sudo apt-get update && sudo apt-get install --yes \
libsasl2-dev
say "::endgroup::"
}
pip-upgrade() {
say "::group::Upgrade pip"
pip install --upgrade pip
@@ -89,8 +102,6 @@ EOF
setup-mysql() {
say "::group::Initialize database"
mysql -h 127.0.0.1 -P 13306 -u root --password=root <<-EOF
SET GLOBAL transaction_isolation='READ-COMMITTED';
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
DROP DATABASE IF EXISTS superset;
CREATE DATABASE superset DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;
DROP DATABASE IF EXISTS sqllab_test_db;
@@ -117,17 +128,9 @@ testdata() {
say "::endgroup::"
}
celery-worker() {
cd "$GITHUB_WORKSPACE"
say "::group::Start Celery worker"
# must specify PYTHONPATH to make `tests.superset_test_config` importable
export PYTHONPATH="$GITHUB_WORKSPACE"
celery \
--app=superset.tasks.celery_app:app \
worker \
--concurrency=2 \
--detach \
--optimization=fair
codecov() {
say "::group::Upload code coverage"
bash ".github/workflows/codecov.sh" "$@"
say "::endgroup::"
}
@@ -143,174 +146,69 @@ cypress-install() {
cache-save cypress
}
cypress-run-all() {
local USE_DASHBOARD=$1
local APP_ROOT=$2
# Run Cypress and upload coverage reports
cypress-run() {
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"
local page=$1
local group=${2:-Default}
local cypress="./node_modules/.bin/cypress run"
local browser=${CYPRESS_BROWSER:-chrome}
export TERM="xterm"
say "::group::Run Cypress for [$page]"
if [[ -z $CYPRESS_KEY ]]; then
$cypress --spec "cypress/integration/$page" --browser "$browser"
else
export CYPRESS_RECORD_KEY=`echo $CYPRESS_KEY | base64 --decode`
# additional flags for Cypress dashboard recording
$cypress --spec "cypress/integration/$page" --browser "$browser" \
--record --group "$group" --tag "${GITHUB_REPOSITORY},${GITHUB_EVENT_NAME}" \
--parallel --ci-build-id "${GITHUB_SHA:0:8}-${NONCE}"
fi
# don't add quotes to $record because we do want word splitting
say "::endgroup::"
}
cypress-run-all() {
# Start Flask and run it in background
# --no-debugger means disable the interactive debugger on the 500 page
# so errors can print to stderr.
local flasklog="${HOME}/flask.log"
local port=8081
CYPRESS_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
export SUPERSET_APP_ROOT=$APP_ROOT
CYPRESS_BASE_URL=${CYPRESS_BASE_URL}${APP_ROOT}
fi
export CYPRESS_BASE_URL
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
USE_DASHBOARD_FLAG=''
if [ "$USE_DASHBOARD" = "true" ]; then
USE_DASHBOARD_FLAG='--use-dashboard'
fi
# UNCOMMENT the next few commands to monitor memory usage
# monitor_memory & # Start memory monitoring in the background
# memoryMonitorPid=$!
python ../../scripts/cypress_run.py --parallelism $PARALLELISM --parallelism-id $PARALLEL_ID --group $PARALLEL_ID --retries 5 $USE_DASHBOARD_FLAG
# kill $memoryMonitorPid
# After job is done, print out Flask log for debugging
echo "::group::Flask log for default run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
}
playwright-install() {
cd "$GITHUB_WORKSPACE/superset-frontend"
say "::group::Install Playwright browsers"
npx playwright install --with-deps chromium
# Create output directories for test results and debugging
mkdir -p playwright-results
mkdir -p test-results
say "::endgroup::"
}
playwright-run() {
local APP_ROOT=$1
local TEST_PATH=$2
# Start Flask from the project root (same as Cypress)
cd "$GITHUB_WORKSPACE"
local flasklog="${HOME}/flask-playwright.log"
local port=8081
PLAYWRIGHT_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
export SUPERSET_APP_ROOT=$APP_ROOT
PLAYWRIGHT_BASE_URL=${PLAYWRIGHT_BASE_URL}${APP_ROOT}/
fi
export PLAYWRIGHT_BASE_URL
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
# Ensure cleanup on exit
trap "kill $flaskProcessId 2>/dev/null || true" EXIT
# Wait for server to be ready with health check
local timeout=60
say "Waiting for Flask server to start on port $port..."
while [ $timeout -gt 0 ]; do
if curl -f ${PLAYWRIGHT_BASE_URL}/health >/dev/null 2>&1; then
say "Flask server is ready"
break
fi
sleep 1
timeout=$((timeout - 1))
done
if [ $timeout -eq 0 ]; then
echo "::error::Flask server failed to start within 60 seconds"
echo "::group::Flask startup log"
cat "$flasklog"
echo "::endgroup::"
return 1
fi
# Change to frontend directory for Playwright execution
cd "$GITHUB_WORKSPACE/superset-frontend"
say "::group::Run Playwright tests"
echo "Running Playwright with baseURL: ${PLAYWRIGHT_BASE_URL}"
if [ -n "$TEST_PATH" ]; then
# Check if there are any test files in the specified path
if ! find "playwright/tests/${TEST_PATH}" -name "*.spec.ts" -type f 2>/dev/null | grep -q .; then
echo "No test files found in ${TEST_PATH} - skipping test run"
say "::endgroup::"
kill $flaskProcessId
return 0
fi
echo "Running tests: ${TEST_PATH}"
# Set INCLUDE_EXPERIMENTAL=true to allow experimental tests to run
export INCLUDE_EXPERIMENTAL=true
npx playwright test "${TEST_PATH}" --output=playwright-results
local status=$?
# Unset to prevent leaking into subsequent commands
unset INCLUDE_EXPERIMENTAL
else
echo "Running all required tests (experimental/ excluded via playwright.config.ts)"
npx playwright test --output=playwright-results
local status=$?
fi
say "::endgroup::"
# After job is done, print out Flask log for debugging
echo "::group::Flask log for Playwright run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
return $status
}
eyes-storybook-dependencies() {
say "::group::install eyes-storyook dependencies"
sudo apt-get update -y && sudo apt-get -y install gconf-service ca-certificates libxshmfence-dev fonts-liberation libappindicator3-1 libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libgconf-2-4 libglib2.0-0 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libnss3 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release xdg-utils libappindicator1
say "::endgroup::"
}
monitor_memory() {
# This is a small utility to monitor memory usage. Useful for debugging memory in GHA.
# To use wrap your command as follows
#
# monitor_memory & # Start memory monitoring in the background
# memoryMonitorPid=$!
# YOUR_COMMAND_HERE
# kill $memoryMonitorPid
while true; do
echo "$(date) - Top 5 memory-consuming processes:"
ps -eo pid,comm,%mem --sort=-%mem | head -n 6 # First line is the header, next 5 are top processes
sleep 2
done
}
cypress-run-applitools() {
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"
local flasklog="${HOME}/flask.log"
local port=8081
local cypress="./node_modules/.bin/cypress run"
local browser=${CYPRESS_BROWSER:-chrome}
export CYPRESS_BASE_URL="http://localhost:${port}"
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
$cypress --spec "cypress/applitools/**/*" --browser "$browser" --headless
cypress-run "*/**/*"
# After job is done, print out Flask log for debugging
say "::group::Flask log for default run"
cat "$flasklog"
say "::endgroup::"
# Rerun SQL Lab tests with backend persist enabled
export SUPERSET_CONFIG=tests.integration_tests.superset_test_config_sqllab_backend_persist
# Restart Flask with new configs
kill $flaskProcessId
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
cypress-run "sqllab/*" "Backend persist"
# Upload code coverage separately so each page can have separate flags
# -c will clean existing coverage reports, -F means add flags
# || true to prevent CI failure on codecov upload
codecov -c -F "cypress" || true
say "::group::Flask log for backend persist"
cat "$flasklog"
say "::endgroup::"
# make sure the program exits
kill $flaskProcessId
}

View File

@@ -1,77 +0,0 @@
name: Bump Python Package
on:
# Can be triggered manually
workflow_dispatch:
inputs:
package:
required: false
description: The python package to bump (all if empty)
group:
required: false
description: The optional dependency group to bump (as defined in pyproject.toml)
limit:
required: true
description: Max number of PRs to open (0 for no limit)
default: 5
extra-flags:
required: false
default: --only-base
description: Additional flags to pass to the bump-python command
#schedule:
# - cron: '0 0 * * *' # Runs daily at midnight UTC
jobs:
bump-python-package:
runs-on: ubuntu-24.04
permissions:
actions: write
contents: write
pull-requests: write
checks: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: true
ref: master
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v6
with:
python-version: "3.10"
- name: Install uv
run: pip install uv
- name: supersetbot bump-python -p "${{ github.event.inputs.package }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --global user.email "action@github.com"
git config --global user.name "GitHub Action"
PACKAGE_OPT=""
if [ -n "${{ github.event.inputs.package }}" ]; then
PACKAGE_OPT="-p ${{ github.event.inputs.package }}"
fi
GROUP_OPT=""
if [ -n "${{ github.event.inputs.group }}" ]; then
GROUP_OPT="-g ${{ github.event.inputs.group }}"
fi
EXTRA_FLAGS="${{ github.event.inputs.extra-flags }}"
supersetbot bump-python \
--verbose \
--use-current-repo \
--include-subpackages \
--limit ${{ github.event.inputs.limit }} \
$PACKAGE_OPT \
$GROUP_OPT \
$EXTRA_FLAGS

View File

@@ -25,8 +25,6 @@ const assetsConfig = {
path: [`${workspaceDirectory}/superset/static/assets`],
hashFiles: [
`${workspaceDirectory}/superset-frontend/src/**/*`,
`${workspaceDirectory}/superset-frontend/packages/**/*`,
`${workspaceDirectory}/superset-frontend/plugins/**/*`,
`${workspaceDirectory}/superset-frontend/*.js`,
`${workspaceDirectory}/superset-frontend/*.json`,
],

View File

@@ -9,15 +9,12 @@ on:
jobs:
cancel-duplicate-runs:
name: Cancel duplicate workflow runs
runs-on: ubuntu-24.04
permissions:
actions: write
contents: read
runs-on: ubuntu-20.04
steps:
- name: Check number of queued tasks
id: check_queued
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPO: ${{ github.repository }}
run: |
get_count() {
@@ -27,16 +24,16 @@ jobs:
}
count=$(( `get_count queued` + `get_count in_progress` ))
echo "Found $count unfinished jobs."
echo "count=$count" >> $GITHUB_OUTPUT
echo "::set-output name=count::$count"
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
if: steps.check_queued.outputs.count >= 20
uses: actions/checkout@v5
uses: actions/checkout@v2
- name: Cancel duplicate workflow runs
if: steps.check_queued.outputs.count >= 20
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: |
pip install click requests typing_extensions python-dateutil

View File

@@ -1,59 +0,0 @@
name: Check python dependencies
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
check-python-deps:
runs-on: ubuntu-22.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
fetch-depth: 1
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
- name: Run uv
if: steps.check.outputs.python
run: ./scripts/uv-pip-compile.sh
- name: Check for uncommitted changes
if: steps.check.outputs.python
run: |
echo "Full diff (for logging/debugging):"
git diff
echo "Filtered diff (excluding comments and whitespace):"
filtered_diff=$(git diff -U0 | grep '^[-+]' | grep -vE '^[-+]{3}' | grep -vE '^[-+][[:space:]]*#' | grep -vE '^[-+][[:space:]]*$' || true)
echo "$filtered_diff"
if [[ -n "$filtered_diff" ]]; then
echo
echo "ERROR: The pinned dependencies are not up-to-date."
echo "Please run './scripts/uv-pip-compile.sh' and commit the changes."
echo "More info: https://github.com/apache/superset/tree/master/requirements"
exit 1
else
echo "Pinned dependencies are up-to-date."
fi

View File

@@ -3,39 +3,24 @@ on:
push:
paths:
- "superset/migrations/**"
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
paths:
- "superset/migrations/**"
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
check_db_migration_conflict:
name: Check DB migration conflict
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
- name: Check and notify
uses: actions/github-script@v8
uses: actions/github-script@v3
with:
github-token: ${{ github.token }}
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
// API reference: https://octokit.github.io/rest.js
const currentBranch = context.ref.replace('refs/heads/', '');
// Find all pull requests to current branch
const opts = github.rest.pulls.list.endpoint.merge({
const opts = github.pulls.list.endpoint.merge({
owner: context.repo.owner,
repo: context.repo.repo,
base: context.ref,
@@ -49,7 +34,7 @@ jobs:
}
for (const pull of pulls) {
const listFilesOpts = await github.rest.pulls.listFiles.endpoint.merge({
const listFilesOpts = await github.pulls.listFiles.endpoint.merge({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pull.number,
@@ -59,17 +44,16 @@ jobs:
files.some(x => x.contents_url.includes('/contents/superset/migrations'))
) {
console.log(`PR #${pull.number} "${pull.title}" also added db migration`)
await github.rest.issues.createComment({
await github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pull.number,
body:
`# 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️ 🙅‍♂️` +
`❗ @${pull.user.login} Your base branch \`${currentBranch}\` has ` +
`⚠️ @${pull.user.login} Your base branch \`${currentBranch}\` has just ` +
'also updated `superset/migrations`.\n' +
'\n' +
'**Please consider rebasing your branch and [resolving potential db migration conflicts](https://github.com/apache/superset/blob/master/CONTRIBUTING.md#merging-db-migrations).**',
'**Please consider rebasing your branch to avoid db migration conflicts.**',
});
}
}

View File

@@ -1,82 +0,0 @@
name: Claude PR Assistant
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
jobs:
check-permissions:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
runs-on: ubuntu-latest
outputs:
allowed: ${{ steps.check.outputs.allowed }}
steps:
- name: Check if user is allowed
id: check
run: |
# List of allowed users
ALLOWED_USERS="mistercrunch,rusackas"
# Get the commenter's username
COMMENTER="${{ github.event.comment.user.login }}"
echo "Checking permissions for user: $COMMENTER"
# Check if user is in allowed list
if [[ ",$ALLOWED_USERS," == *",$COMMENTER,"* ]]; then
echo "allowed=true" >> $GITHUB_OUTPUT
echo "✅ User $COMMENTER is allowed to use Claude"
else
echo "allowed=false" >> $GITHUB_OUTPUT
echo "❌ User $COMMENTER is not allowed to use Claude"
fi
deny-access:
needs: check-permissions
if: needs.check-permissions.outputs.allowed == 'false'
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Comment access denied
uses: actions/github-script@v8
with:
script: |
const message = `👋 Hi @${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}!
Thanks for trying to use Claude Code, but currently only certain team members have access to this feature.
If you believe you should have access, please contact a project maintainer.`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: message
});
claude-code-action:
needs: check-permissions
if: needs.check-permissions.outputs.allowed == 'true'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 1
- name: Run Claude PR Action
uses: anthropics/claude-code-action@beta
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
timeout_minutes: "60"

1903
.github/workflows/codecov.sh vendored Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -1,58 +0,0 @@
name: "CodeQL"
on:
push:
branches: ["master", "[0-9].[0-9]*"]
pull_request:
# The branches below must be a subset of the branches above
branches: ["master"]
schedule:
- cron: "0 4 * * *"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
analyze:
name: Analyze
runs-on: ubuntu-24.04
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: ["python", "javascript"]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
- name: Perform CodeQL Analysis
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: github/codeql-action/analyze@v4
with:
category: "/language:${{matrix.language}}"

View File

@@ -1,71 +0,0 @@
# Dependency Review Action
#
# This Action will scan dependency manifest files that change as part of a Pull Request, surfacing known-vulnerable versions of the packages declared or updated in the PR. Once installed, if the workflow run is marked as required, PRs introducing known-vulnerable packages will be blocked from merging.
#
# Source repository: https://github.com/actions/dependency-review-action
# Public documentation: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement
name: "Dependency Review"
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
jobs:
dependency-review:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
steps:
- name: "Checkout Repository"
uses: actions/checkout@v5
- name: "Dependency Review"
uses: actions/dependency-review-action@v4
continue-on-error: true
with:
fail-on-severity: critical
# compatible/incompatible licenses addressed here: https://www.apache.org/legal/resolved.html
# find SPDX identifiers here: https://spdx.org/licenses/
deny-licenses: MS-LPL, BUSL-1.1, QPL-1.0, Sleepycat, SSPL-1.0, CPOL-1.02, AGPL-3.0, GPL-1.0+, BSD-4-Clause-UC, NPL-1.0, NPL-1.1, JSON
# pkg:npm/store2@2.14.2
# adding an exception for an ambigious license on store2, which has been resolved in
# the latest version. It's MIT: https://github.com/nbubna/store/blob/master/LICENSE-MIT
# pkg:npm/applitools/*
# adding exception for all applitools modules (eyes-cypress and its dependencies),
# which has an explicit OSS license approved by ASF
# license: https://applitools.com/legal/open-source-terms-of-use/
# pkg:npm/node-forge@1.3.1
# selecting BSD-3-Clause licensing terms for node-forge to ensure compatibility with Apache
allow-dependencies-licenses: pkg:npm/store2@2.14.2, pkg:npm/applitools/core, pkg:npm/applitools/core-base, pkg:npm/applitools/css-tree, pkg:npm/applitools/ec-client, pkg:npm/applitools/eg-socks5-proxy-server, pkg:npm/applitools/eyes, pkg:npm/applitools/eyes-cypress, pkg:npm/applitools/nml-client, pkg:npm/applitools/tunnel-client, pkg:npm/applitools/utils, pkg:npm/node-forge@1.3.1, pkg:npm/rgbcolor, pkg:npm/jszip@3.10.1
python-dependency-liccheck:
# NOTE: Configuration for liccheck lives in our pyproject.yml.
# You cannot use a liccheck.ini file in this workflow.
runs-on: ubuntu-22.04
steps:
- name: "Checkout Repository"
uses: actions/checkout@v5
- name: Setup Python
uses: ./.github/actions/setup-backend/
with:
requirements-type: base
- name: "Set up liccheck"
run: |
uv pip install --system liccheck
- name: "Run liccheck"
run: |
# run the checks
liccheck -R output.txt
# Print the report
cat output.txt

View File

@@ -0,0 +1,78 @@
name: Push ephmereral env image
on:
workflow_run:
workflows: ["Docker"]
types:
- completed
jobs:
docker_ephemeral_env:
name: Push ephemeral env Docker image to ECR
if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
steps:
- name: 'Download artifact'
uses: actions/github-script@v3.1.0
with:
script: |
const artifacts = await github.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
core.info('*** artifacts')
core.info(JSON.stringify(artifacts))
const matchArtifact = artifacts.data.artifacts.filter((artifact) => {
return artifact.name == "build"
})[0];
if(!matchArtifact) return core.setFailed("Build artifacts not found")
const download = await github.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
var fs = require('fs');
fs.writeFileSync('${{github.workspace}}/build.zip', Buffer.from(download.data));
- run: unzip build.zip
- name: Display downloaded files (debug)
run: ls -la
- name: Get SHA
id: get-sha
run: echo "::set-output name=sha::$(cat ./SHA)"
- name: Get PR
id: get-pr
run: echo "::set-output name=num::$(cat ./PR-NUM)"
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Load, tag and push image to ECR
id: push-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: superset-ci
SHA: ${{ steps.get-sha.outputs.sha }}
IMAGE_TAG: pr-${{ steps.get-pr.outputs.num }}
run: |
docker load < $SHA.tar.gz
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$SHA
docker push -a $ECR_REGISTRY/$ECR_REPOSITORY

22
.github/workflows/docker-release.yml vendored Normal file
View File

@@ -0,0 +1,22 @@
name: Docker
on:
release:
types: [published]
jobs:
docker-release:
name: docker-release
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.ref }}
- shell: bash
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
.github/workflows/docker_build_push.sh

View File

@@ -1,140 +1,42 @@
name: Build & publish docker images
name: Docker
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
branches:
- "master"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
types: [synchronize, opened, reopened, ready_for_review]
jobs:
setup_matrix:
runs-on: ubuntu-24.04
outputs:
matrix_config: ${{ steps.set_matrix.outputs.matrix_config }}
steps:
- id: set_matrix
run: |
MATRIX_CONFIG=$(if [ "${{ github.event_name }}" == "pull_request" ]; then echo '["dev", "lean"]'; else echo '["dev", "lean", "py310", "websocket", "dockerize", "py311", "py312"]'; fi)
echo "matrix_config=${MATRIX_CONFIG}" >> $GITHUB_OUTPUT
echo $GITHUB_OUTPUT
docker-build:
if: github.event.pull_request.draft == false
name: docker-build
needs: setup_matrix
runs-on: ubuntu-24.04
strategy:
matrix:
build_preset: ${{fromJson(needs.setup_matrix.outputs.matrix_config)}}
fail-fast: false
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
IMAGE_TAG: apache/superset:GHA-${{ matrix.build_preset }}-${{ github.run_id }}
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Docker Environment
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
- name: Setup supersetbot
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
uses: ./.github/actions/setup-supersetbot/
- name: Build Docker Image
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
shell: bash
- shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
# Single platform builds in pull_request context to speed things up
if [ "${{ github.event_name }}" = "push" ]; then
PLATFORM_ARG="--platform linux/arm64 --platform linux/amd64"
# can only --load images in single-platform builds
PUSH_OR_LOAD="--push"
elif [ "${{ github.event_name }}" = "pull_request" ]; then
PLATFORM_ARG="--platform linux/amd64"
PUSH_OR_LOAD="--load"
fi
.github/workflows/docker_build_push.sh
supersetbot docker \
$PUSH_OR_LOAD \
--preset ${{ matrix.build_preset }} \
--context "$EVENT" \
--context-ref "$RELEASE" $FORCE_LATEST \
--extra-flags "--build-arg INCLUDE_CHROMIUM=false --tag $IMAGE_TAG" \
$PLATFORM_ARG
# in the context of push (using multi-platform build), we need to pull the image locally
- name: Docker pull
if: github.event_name == 'push' && (steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker)
run: docker pull $IMAGE_TAG
- name: Print docker stats
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
- name: Build ephemeral env image
if: github.event_name == 'pull_request'
run: |
echo "SHA: ${{ github.sha }}"
echo "IMAGE: $IMAGE_TAG"
docker images $IMAGE_TAG
docker history $IMAGE_TAG
mkdir -p ./build
echo ${{ github.sha }} > ./build/SHA
echo ${{ github.event.pull_request.number }} > ./build/PR-NUM
docker build --target ci -t ${{ github.sha }} -t "pr-${{ github.event.pull_request.number }}" .
docker save ${{ github.sha }} | gzip > ./build/${{ github.sha }}.tar.gz
- name: docker-compose sanity check
if: (steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker) && matrix.build_preset == 'dev'
shell: bash
run: |
export SUPERSET_BUILD_TARGET=${{ matrix.build_preset }}
# This should reuse the CACHED image built in the previous steps
docker compose build superset-init --build-arg DEV_MODE=false --build-arg INCLUDE_CHROMIUM=false
docker compose up superset-init --exit-code-from superset-init
docker-compose-image-tag:
# Run this job only on pushes to master (not for PRs)
# goal is to check that building the latest image works, not required for all PR pushes
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
- name: Upload build artifacts
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v2
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Docker Environment
if: steps.check.outputs.docker
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "false"
install-docker-compose: "true"
- name: docker-compose sanity check
if: steps.check.outputs.docker
shell: bash
run: |
docker compose -f docker-compose-image-tag.yml up superset-init --exit-code-from superset-init
name: build
path: build/

80
.github/workflows/docker_build_push.sh vendored Executable file
View File

@@ -0,0 +1,80 @@
#!/usr/bin/env bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -eo pipefail
SHA=$(git rev-parse HEAD)
REPO_NAME="apache/superset"
if [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
REFSPEC=$(echo "${GITHUB_HEAD_REF}" | sed 's/[^a-zA-Z0-9]/-/g' | head -c 40)
PR_NUM=$(echo "${GITHUB_REF}" | sed 's:refs/pull/::' | sed 's:/merge::')
LATEST_TAG="pr-${PR_NUM}"
elif [[ "${GITHUB_EVENT_NAME}" == "release" ]]; then
REFSPEC=$(echo "${GITHUB_REF}" | sed 's:refs/tags/::' | head -c 40)
LATEST_TAG="${REFSPEC}"
else
REFSPEC=$(echo "${GITHUB_REF}" | sed 's:refs/heads/::' | sed 's/[^a-zA-Z0-9]/-/g' | head -c 40)
LATEST_TAG="${REFSPEC}"
fi
if [[ "${REFSPEC}" == "master" ]]; then
LATEST_TAG="latest"
fi
cat<<EOF
Rolling with tags:
- ${REPO_NAME}:${SHA}
- ${REPO_NAME}:${REFSPEC}
- ${REPO_NAME}:${LATEST_TAG}
EOF
#
# Build the "lean" image
#
docker build --target lean \
-t "${REPO_NAME}:${SHA}" \
-t "${REPO_NAME}:${REFSPEC}" \
-t "${REPO_NAME}:${LATEST_TAG}" \
--label "sha=${SHA}" \
--label "built_at=$(date)" \
--label "target=lean" \
--label "build_actor=${GITHUB_ACTOR}" \
.
#
# Build the dev image
#
docker build --target dev \
-t "${REPO_NAME}:${SHA}-dev" \
-t "${REPO_NAME}:${REFSPEC}-dev" \
-t "${REPO_NAME}:${LATEST_TAG}-dev" \
--label "sha=${SHA}" \
--label "built_at=$(date)" \
--label "target=dev" \
--label "build_actor=${GITHUB_ACTOR}" \
.
if [ -z "${DOCKERHUB_TOKEN}" ]; then
# Skip if secrets aren't populated -- they're only visible for actions running in the repo (not on forks)
echo "Skipping Docker push"
else
# Login and push
docker logout
docker login --username "${DOCKERHUB_USER}" --password "${DOCKERHUB_TOKEN}"
docker push --all-tags "${REPO_NAME}"
fi

View File

@@ -23,14 +23,6 @@
{
"name": "SUPERSET_PORT",
"value": "8080"
},
{
"name": "SUPERSET_SECRET_KEY",
"value": "super-secret-for-ephemerals"
},
{
"name": "TALISMAN_ENABLED",
"value": "False"
}
],
"mountPoints": [],

View File

@@ -3,35 +3,19 @@ name: Embedded SDK Release
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.NPM_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
build:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
defaults:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version-file: './superset-embedded-sdk/.nvmrc'
node-version: "16"
registry-url: 'https://registry.npmjs.org'
- run: npm ci
- run: npm run ci:release

View File

@@ -6,22 +6,18 @@ on:
- "superset-embedded-sdk/**"
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
embedded-sdk-test:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
defaults:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version-file: './superset-embedded-sdk/.nvmrc'
node-version: "16"
registry-url: 'https://registry.npmjs.org'
- run: npm ci
- run: npm test

View File

@@ -1,39 +1,16 @@
name: Cleanup ephemeral envs (PR close) [DEPRECATED]
# ⚠️ DEPRECATION NOTICE ⚠️
# This workflow is deprecated and will be removed in a future version.
# The new Superset Showtime workflow handles cleanup automatically.
# See .github/workflows/showtime.yml and showtime-cleanup.yml for replacements.
# Migration guide: https://github.com/mistercrunch/superset-showtime
name: Cleanup ephemeral envs (PR close)
on:
pull_request_target:
types: [closed]
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.AWS_ACCESS_KEY_ID != '' && secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
ephemeral-env-cleanup:
needs: config
if: needs.config.outputs.has-secrets
name: Cleanup ephemeral envs
runs-on: ubuntu-24.04
permissions:
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v5
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
@@ -42,7 +19,7 @@ jobs:
- name: Describe ECS service
id: describe-services
run: |
echo "active=$(aws ecs describe-services --cluster superset-ci --services pr-${{ github.event.number }}-service | jq '.services[] | select(.status == "ACTIVE") | any')" >> $GITHUB_OUTPUT
echo "::set-output name=active::$(aws ecs describe-services --cluster superset-ci --services pr-${{ github.event.number }}-service | jq '.services[] | select(.status == "ACTIVE") | any')"
- name: Delete ECS service
if: steps.describe-services.outputs.active == 'true'
@@ -56,7 +33,7 @@ jobs:
- name: Login to Amazon ECR
if: steps.describe-services.outputs.active == 'true'
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
uses: aws-actions/amazon-ecr-login@v1
- name: Delete ECR image tag
if: steps.describe-services.outputs.active == 'true'
@@ -69,13 +46,13 @@ jobs:
- name: Comment (success)
if: steps.describe-services.outputs.active == 'true'
uses: actions/github-script@v8
uses: actions/github-script@v3
with:
github-token: ${{github.token}}
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.rest.issues.createComment({
github.issues.createComment({
issue_number: ${{ github.event.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '⚠️ **DEPRECATED WORKFLOW** - Ephemeral environment shutdown and build artifacts deleted. Please migrate to the new Superset Showtime system for future PRs.'
body: 'Ephemeral environment shutdown and build artifacts deleted.'
})

View File

@@ -1,344 +1,194 @@
name: Ephemeral env workflow [DEPRECATED]
# ⚠️ DEPRECATION NOTICE ⚠️
# This workflow is deprecated and will be removed in a future version.
# Please use the new Superset Showtime workflow instead:
# - Use label "🎪 trigger-start" instead of "testenv-up"
# - Showtime provides better reliability and easier management
# - See .github/workflows/showtime.yml for the replacement
# - Migration guide: https://github.com/mistercrunch/superset-showtime
# Example manual trigger:
# gh workflow run ephemeral-env.yml --ref fix_ephemerals --field label_name="testenv-up" --field issue_number=666
name: Ephemeral env workflow
on:
pull_request_target:
types:
- labeled
workflow_dispatch:
inputs:
label_name:
description: 'Label name to simulate label-based /testenv trigger'
required: true
default: 'testenv-up'
issue_number:
description: 'Issue or PR number'
required: true
issue_comment:
types: [created]
jobs:
ephemeral-env-label:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-label
cancel-in-progress: true
name: Evaluate ephemeral env label trigger
runs-on: ubuntu-24.04
permissions:
pull-requests: write
ephemeral_env_comment:
if: github.event.issue.pull_request
name: Evaluate ephemeral env comment trigger (/testenv)
runs-on: ubuntu-latest
outputs:
slash-command: ${{ steps.eval-label.outputs.result }}
slash-command: ${{ steps.eval-body.outputs.result }}
feature-flags: ${{ steps.eval-feature-flags.outputs.result }}
sha: ${{ steps.get-sha.outputs.sha }}
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
steps:
- name: Check for the "testenv-up" label
id: eval-label
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
LABEL_NAME="${{ github.event.inputs.label_name }}"
else
LABEL_NAME="${{ github.event.label.name }}"
fi
- name: Debug
run: |
echo "Comment on PR #${{ github.event.issue.number }} by ${{ github.event.issue.user.login }}, ${{ github.event.comment.author_association }}"
echo "Evaluating label: $LABEL_NAME"
- name: Eval comment body for /testenv slash command
uses: actions/github-script@v3
id: eval-body
with:
result-encoding: string
script: |
const pattern = /^\/testenv (up|down)/
const result = pattern.exec(context.payload.comment.body)
return result === null ? 'noop' : result[1]
if [[ "$LABEL_NAME" == "testenv-up" ]]; then
echo "result=up" >> $GITHUB_OUTPUT
else
echo "result=noop" >> $GITHUB_OUTPUT
fi
- name: Eval comment body for feature flags
uses: actions/github-script@v3
id: eval-feature-flags
with:
script: |
const pattern = /FEATURE_(\w+)=(\w+)/g;
let results = [];
[...context.payload.comment.body.matchAll(pattern)].forEach(match => {
const config = {
name: `SUPERSET_FEATURE_${match[1]}`,
value: match[2],
};
results.push(config);
});
return results;
- name: Get event SHA
id: get-sha
if: steps.eval-label.outputs.result == 'up'
uses: actions/github-script@v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
let prSha;
- name: Limit to committers
if: >
steps.eval-body.outputs.result != 'noop' &&
github.event.comment.author_association != 'MEMBER' &&
github.event.comment.author_association != 'OWNER'
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Ephemeral environment creation is currently limited to committers.'
github.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
})
core.setFailed(errMsg)
// If event is workflow_dispatch, use the issue_number from inputs
if (context.eventName === "workflow_dispatch") {
const prNumber = "${{ github.event.inputs.issue_number }}";
if (!prNumber) {
console.log("No PR number found.");
return;
}
// Fetch PR details using the provided issue_number
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber
});
prSha = pr.head.sha;
} else {
// If it's not workflow_dispatch, use the PR head sha from the event
prSha = context.payload.pull_request.head.sha;
}
console.log(`PR SHA: ${prSha}`);
core.setOutput("sha", prSha);
- name: Looking for feature flags in PR description
uses: actions/github-script@v8
id: eval-feature-flags
if: steps.eval-label.outputs.result == 'up'
with:
script: |
const description = context.payload.pull_request
? context.payload.pull_request.body || ''
: context.payload.inputs.pr_description || '';
const pattern = /FEATURE_(\w+)=(\w+)/g;
let results = [];
[...description.matchAll(pattern)].forEach(match => {
const config = {
name: `SUPERSET_FEATURE_${match[1]}`,
value: match[2],
};
results.push(config);
});
return results;
- name: Reply with confirmation comment
uses: actions/github-script@v8
if: steps.eval-label.outputs.result == 'up'
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const action = '${{ steps.eval-label.outputs.result }}';
const user = context.actor;
const runId = context.runId;
const workflowUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${runId}`;
const issueNumber = context.payload.pull_request
? context.payload.pull_request.number
: context.payload.inputs.issue_number;
if (!issueNumber) {
throw new Error("Issue number is not available.");
}
const body = `⚠️ **DEPRECATED WORKFLOW** ⚠️\n\n@${user} This workflow is deprecated! Please use the new **Superset Showtime** system instead:\n\n` +
`- Replace "testenv-up" label with "🎪 trigger-start"\n` +
`- Better reliability and easier management\n` +
`- See https://github.com/mistercrunch/superset-showtime for details\n\n` +
`Processing your ephemeral environment request [here](${workflowUrl}). Action: **${action}**.` +
` More information on [how to use or configure ephemeral environments]` +
`(https://superset.apache.org/docs/contributing/howtos/#github-ephemeral-environments)`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
body,
});
ephemeral-docker-build:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-build
cancel-in-progress: true
needs: ephemeral-env-label
if: needs.ephemeral-env-label.outputs.slash-command == 'up'
name: ephemeral-docker-build
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ needs.ephemeral-env-label.outputs.sha }} : ${{steps.get-sha.outputs.sha}} )"
uses: actions/checkout@v5
with:
ref: ${{ needs.ephemeral-env-label.outputs.sha }}
persist-credentials: false
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
install-docker-compose: "false"
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Build ephemeral env image
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
supersetbot docker \
--push \
--load \
--preset ci \
--platform linux/amd64 \
--context-ref "$RELEASE" \
--extra-flags "--build-arg INCLUDE_CHROMIUM=false"
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v5
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Load, tag and push image to ECR
id: push-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: superset-ci
IMAGE_TAG: apache/superset:${{ needs.ephemeral-env-label.outputs.sha }}-ci
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
run: |
docker tag $IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:pr-$PR_NUMBER-ci
docker push -a $ECR_REGISTRY/$ECR_REPOSITORY
ephemeral-env-up:
needs: [ephemeral-env-label, ephemeral-docker-build]
if: needs.ephemeral-env-label.outputs.slash-command == 'up'
ephemeral_env_up:
needs: ephemeral_env_comment
if: needs.ephemeral_env_comment.outputs.slash-command == 'up'
name: Spin up an ephemeral environment
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/checkout@v2
with:
persist-credentials: false
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v5
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Check target image exists in ECR
id: check-image
continue-on-error: true
env:
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
run: |
aws ecr describe-images \
--registry-id $(echo "${{ steps.login-ecr.outputs.registry }}" | grep -Eo "^[0-9]+") \
--repository-name superset-ci \
--image-ids imageTag=pr-$PR_NUMBER-ci
- name: Check target image exists in ECR
id: check-image
continue-on-error: true
run: |
aws ecr describe-images \
--registry-id $(echo "${{ steps.login-ecr.outputs.registry }}" | grep -Eo "^[0-9]+") \
--repository-name superset-ci \
--image-ids imageTag=pr-${{ github.event.issue.number }}
- name: Fail on missing container image
if: steps.check-image.outcome == 'failure'
uses: actions/github-script@v8
with:
github-token: ${{ github.token }}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Container image not yet published for this PR. Please try again when build is complete.';
github.rest.issues.createComment({
issue_number: ${{ github.event.inputs.issue_number || github.event.pull_request.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
});
core.setFailed(errMsg);
- name: Fail on missing container image
if: steps.check-image.outcome == 'failure'
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Container image not yet published for this PR. Please try again when build is complete.'
github.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
})
core.setFailed(errMsg)
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: .github/workflows/ecs-task-definition.json
container-name: superset-ci
image: ${{ steps.login-ecr.outputs.registry }}/superset-ci:pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-ci
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: .github/workflows/ecs-task-definition.json
container-name: superset-ci
image: ${{ steps.login-ecr.outputs.registry }}/superset-ci:pr-${{ github.event.issue.number }}
- name: Update env vars in the Amazon ECS task definition
run: |
cat <<< "$(jq '.containerDefinitions[0].environment += ${{ needs.ephemeral-env-label.outputs.feature-flags }}' < ${{ steps.task-def.outputs.task-definition }})" > ${{ steps.task-def.outputs.task-definition }}
- name: Update env vars in the Amazon ECS task definition
run: |
cat <<< "$(jq '.containerDefinitions[0].environment += ${{ needs.ephemeral_env_comment.outputs.feature-flags }}' < ${{ steps.task-def.outputs.task-definition }})" > ${{ steps.task-def.outputs.task-definition }}
- name: Describe ECS service
id: describe-services
run: |
echo "active=$(aws ecs describe-services --cluster superset-ci --services pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-service | jq '.services[] | select(.status == "ACTIVE") | any')" >> $GITHUB_OUTPUT
- name: Create ECS service
id: create-service
if: steps.describe-services.outputs.active != 'true'
env:
ECR_SUBNETS: subnet-0e15a5034b4121710,subnet-0e8efef4a72224974
ECR_SECURITY_GROUP: sg-092ff3a6ae0574d91
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
run: |
aws ecs create-service \
--cluster superset-ci \
--service-name pr-$PR_NUMBER-service \
--task-definition superset-ci \
--launch-type FARGATE \
--desired-count 1 \
--platform-version LATEST \
--network-configuration "awsvpcConfiguration={subnets=[$ECR_SUBNETS],securityGroups=[$ECR_SECURITY_GROUP],assignPublicIp=ENABLED}" \
--tags key=pr,value=$PR_NUMBER key=github_user,value=${{ github.actor }}
- name: Deploy Amazon ECS task definition
id: deploy-task
uses: aws-actions/amazon-ecs-deploy-task-definition@v2
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-service
cluster: superset-ci
wait-for-service-stability: true
wait-for-minutes: 10
- name: Describe ECS service
id: describe-services
run: |
echo "::set-output name=active::$(aws ecs describe-services --cluster superset-ci --services pr-${{ github.event.issue.number }}-service | jq '.services[] | select(.status == "ACTIVE") | any')"
- name: List tasks
id: list-tasks
run: |
echo "task=$(aws ecs list-tasks --cluster superset-ci --service-name pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-service | jq '.taskArns | first')" >> $GITHUB_OUTPUT
- name: Get network interface
id: get-eni
run: |
echo "eni=$(aws ecs describe-tasks --cluster superset-ci --tasks ${{ steps.list-tasks.outputs.task }} | jq '.tasks[0].attachments[0].details | map(select(.name=="networkInterfaceId"))[0].value')" >> $GITHUB_OUTPUT
- name: Get public IP
id: get-ip
run: |
echo "ip=$(aws ec2 describe-network-interfaces --network-interface-ids ${{ steps.get-eni.outputs.eni }} | jq -r '.NetworkInterfaces | first | .Association.PublicIp')" >> $GITHUB_OUTPUT
- name: Comment (success)
if: ${{ success() }}
uses: actions/github-script@v8
with:
github-token: ${{github.token}}
script: |
const issue_number = context.payload.inputs?.issue_number || context.issue.number;
github.rest.issues.createComment({
issue_number: issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `@${{ github.actor }} Ephemeral environment spinning up at http://${{ steps.get-ip.outputs.ip }}:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.`
});
- name: Comment (failure)
if: ${{ failure() }}
uses: actions/github-script@v8
with:
github-token: ${{github.token}}
script: |
const issue_number = context.payload.inputs?.issue_number || context.issue.number;
github.rest.issues.createComment({
issue_number: issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.inputs.user_login || github.event.comment.user.login }} Ephemeral environment creation failed. Please check the Actions logs for details.'
})
- name: Create ECS service
if: steps.describe-services.outputs.active != 'true'
id: create-service
env:
ECR_SUBNETS: subnet-0e15a5034b4121710,subnet-0e8efef4a72224974
ECR_SECURITY_GROUP: sg-092ff3a6ae0574d91
run: |
aws ecs create-service \
--cluster superset-ci \
--service-name pr-${{ github.event.issue.number }}-service \
--task-definition superset-ci \
--launch-type FARGATE \
--desired-count 1 \
--platform-version LATEST \
--network-configuration "awsvpcConfiguration={subnets=[$ECR_SUBNETS],securityGroups=[$ECR_SECURITY_GROUP],assignPublicIp=ENABLED}" \
--tags key=pr,value=${{ github.event.issue.number }} key=github_user,value=${{ github.actor }}
- name: Deploy Amazon ECS task definition
id: deploy-task
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: pr-${{ github.event.issue.number }}-service
cluster: superset-ci
wait-for-service-stability: true
wait-for-minutes: 10
- name: List tasks
id: list-tasks
run: |
echo "::set-output name=task::$(aws ecs list-tasks --cluster superset-ci --service-name pr-${{ github.event.issue.number }}-service | jq '.taskArns | first')"
- name: Get network interface
id: get-eni
run: |
echo "::set-output name=eni::$(aws ecs describe-tasks --cluster superset-ci --tasks ${{ steps.list-tasks.outputs.task }} | jq '.tasks | .[0] | .attachments | .[0] | .details | map(select(.name=="networkInterfaceId")) | .[0] | .value')"
- name: Get public IP
id: get-ip
run: |
echo "::set-output name=ip::$(aws ec2 describe-network-interfaces --network-interface-ids ${{ steps.get-eni.outputs.eni }} | jq -r '.NetworkInterfaces | first | .Association.PublicIp')"
- name: Comment (success)
if: ${{ success() }}
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.comment.user.login }} Ephemeral environment spinning up at http://${{ steps.get-ip.outputs.ip }}:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.'
})
- name: Comment (failure)
if: ${{ failure() }}
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.comment.user.login }} Ephemeral environment creation failed. Please check the Actions logs for details.'
})

View File

@@ -1,65 +0,0 @@
name: Generate FOSSA report
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.FOSSA_API_KEY != '' ) || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
license_check:
needs: config
if: needs.config.outputs.has-secrets
name: Generate Report
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Setup Java
uses: actions/setup-java@v5
with:
distribution: "temurin"
java-version: "11"
- name: Generate fossa report
env:
FOSSA_API_KEY: ${{ secrets.FOSSA_API_KEY }}
run: |
set -eo pipefail
if [[ "${{github.event_name}}" != "pull_request" ]]; then
./scripts/fossa.sh
exit 0
fi
URL="https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files"
FILES=$(curl -s -X GET -G $URL | jq -r '.[] | .filename')
cat<<EOF
CHANGED FILES:
$FILES
EOF
if [[ "${FILES}" =~ (.*package*\.json|requirements\/[a-z_-]+\.txt|setup\.py) ]]; then
echo "Detected dependency changes... running fossa check"
./scripts/fossa.sh
else
echo "No dependency changes... skiping fossa check"
fi
shell: bash

View File

@@ -1,28 +0,0 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Inspired from https://github.com/mpalmer/action-validator?tab=readme-ov-file#pre-commit-hook-example
echo "Running pre-commit hook for GitHub Actions: https://github.com/mpalmer/action-validator"
for action in $(git ls-files .github/ | grep -E '^\.github/(workflows|actions)/.*\.ya?ml$'); do
if action-validator "$action"; then
echo "$action"
else
echo "$action"
exit 1
fi
done

View File

@@ -1,28 +0,0 @@
name: Validate All GitHub Actions
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
jobs:
validate-all-ghas:
runs-on: ubuntu-24.04
steps:
- name: Checkout Repository
uses: actions/checkout@v5
- name: Set up Node.js
uses: actions/setup-node@v5
with:
node-version: '20'
- name: Install Dependencies
run: npm install -g @action-validator/core @action-validator/cli --save-dev
- name: Run Script
run: bash .github/workflows/github-action-validator.sh

View File

@@ -1,34 +0,0 @@
name: supersetbot orglabel based on author
on:
issues:
types: [created, edited]
pull_request:
types: [created, edited]
jobs:
superbot-orglabel:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute supersetbot orglabel command
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Label the issue with the appropriate org using supersetbot
# - this requires for the author to be publicly associated with their org
# - and for the org to be listed in `supersetbot/src/metadata.js`
supersetbot orglabel --issue ${{ github.event.number }} --repo ${{ github.repository }} || true

View File

@@ -1,21 +0,0 @@
name: "Pull Request Labeler"
on:
- pull_request_target
jobs:
labeler:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-24.04
steps:
- uses: actions/labeler@v6
with:
sync-labels: true
# TODO: run scripts based on labels!
# - id: run-translation-scripts
# if: contains(steps.label-the-PR.outputs.all-labels, 'i18n')
# run: |
# echo "Running translation scripts"
# # Generate .pot -> .po -> .json files

View File

@@ -6,13 +6,11 @@ on:
jobs:
latest-release:
name: Add/update tag to new release
runs-on: ubuntu-24.04
permissions:
contents: write
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
@@ -22,11 +20,6 @@ jobs:
run: |
source ./scripts/tag_latest_release.sh $(echo ${{ github.event.release.tag_name }}) --dry-run
- name: Configure Git
run: |
git config user.name "$GITHUB_ACTOR"
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Run latest-tag
uses: ./.github/actions/latest-tag
if: (! ${{ steps.latest-tag.outputs.SKIP_TAG }} )
@@ -34,4 +27,4 @@ jobs:
description: Superset latest release
tag-name: latest
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,28 +0,0 @@
name: License Template Check
on:
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
license_check:
name: License Check
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Setup Java
uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: '11'
- name: Run license check
run: ./scripts/check_license.sh

99
.github/workflows/misc.yml vendored Normal file
View File

@@ -0,0 +1,99 @@
name: Miscellaneous
on:
push:
branches-ignore:
- "dependabot/**"
pull_request:
jobs:
license_check:
name: License Check
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Setup Java
uses: actions/setup-java@v1
with:
java-version: 8
- name: Generate fossa report
env:
FOSSA_API_KEY: ${{ secrets.FOSSA_API_KEY }}
run: |
set -eo pipefail
if [[ "${{github.event_name}}" != "pull_request" ]]; then
./scripts/fossa.sh
exit 0
fi
URL="https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files"
FILES=$(curl -s -X GET -G $URL | jq -r '.[] | .filename')
cat<<EOF
CHANGED FILES:
$FILES
EOF
if [[ "${FILES}" =~ (.*package*\.json|requirements\/[a-z_-]+\.txt|setup\.py) ]]; then
echo "Detected dependency changes... running fossa check"
./scripts/fossa.sh
else
echo "No dependency changes... skiping fossa check"
fi
shell: bash
- name: Run license check
run: ./scripts/check_license.sh
prefer_typescript:
if: github.ref == 'ref/heads/master' && github.event_name == 'pull_request'
name: Prefer Typescript
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Get changed files
id: changed
uses: ./.github/actions/file-changes-action
with:
githubToken: ${{ secrets.GITHUB_TOKEN }}
- name: Determine if a .js or .jsx file was added
id: check
run: |
js_files_added() {
jq -r '
map(
select(
endswith(".js") or endswith(".jsx")
)
) | join("\n")
' ${HOME}/files_added.json
}
echo ::set-output name=js_files_added::$(js_files_added)
- if: steps.check.outputs.js_files_added
name: Add Comment to PR
uses: ./.github/actions/comment-on-pr
continue-on-error: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
msg: |
### WARNING: Prefer TypeScript
Looks like your PR contains new `.js` or `.jsx` files:
```
${{steps.check.outputs.js_files_added}}
```
As decided in [SIP-36](https://github.com/apache/superset/issues/9101), all new frontend code should be written in TypeScript. Please convert above files to TypeScript then re-request review.

View File

@@ -1,25 +0,0 @@
name: Hold Label Check
on:
pull_request:
types: [labeled, unlabeled, opened, reopened, synchronize]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
check-hold-label:
runs-on: ubuntu-24.04
steps:
- name: Check for 'hold' label
uses: actions/github-script@v8
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
const payload = context.payload.pull_request
const holdLabelPresent = !!payload.labels.find(label => label.name.includes('hold'))
if (holdLabelPresent) {
core.setFailed('Hold label is present, merge is blocked.')
}

View File

@@ -9,14 +9,11 @@ on:
types: [opened, edited, reopened, synchronize]
jobs:
lint-check:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
check:
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
@@ -28,4 +25,4 @@ jobs:
on-failed-regex-create-review: false
on-failed-regex-comment:
"Please format your PR title to match: `%regex%`!"
repo-token: "${{ github.token }}"
repo-token: "${{ secrets.GITHUB_TOKEN }}"

View File

@@ -1,85 +0,0 @@
name: pre-commit checks
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
pre-commit:
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: ["current", "previous", "next"]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: ./.github/actions/setup-backend/
with:
python-version: ${{ matrix.python-version }}
- name: Enable brew and helm-docs
# Add brew to the path - see https://github.com/actions/runner-images/issues/6283
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
echo "HOMEBREW_PREFIX=$HOMEBREW_PREFIX" >>"${GITHUB_ENV}"
echo "HOMEBREW_CELLAR=$HOMEBREW_CELLAR" >>"${GITHUB_ENV}"
echo "HOMEBREW_REPOSITORY=$HOMEBREW_REPOSITORY" >>"${GITHUB_ENV}"
brew install norwoodj/tap/helm-docs
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: '20'
- name: Install Frontend Dependencies
run: |
cd superset-frontend
npm ci
- name: Install Docs Dependencies
run: |
cd docs
yarn install --immutable
- name: Cache pre-commit environments
uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit-v2-${{ runner.os }}-py${{ matrix.python-version }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: |
pre-commit-v2-${{ runner.os }}-py${{ matrix.python-version }}-
- name: pre-commit
run: |
set +e # Don't exit immediately on failure
export SKIP=eslint-frontend,type-checking-frontend
pre-commit run --all-files
PRE_COMMIT_EXIT_CODE=$?
git diff --quiet --exit-code
GIT_DIFF_EXIT_CODE=$?
if [ "${PRE_COMMIT_EXIT_CODE}" -ne 0 ] || [ "${GIT_DIFF_EXIT_CODE}" -ne 0 ]; then
if [ "${PRE_COMMIT_EXIT_CODE}" -ne 0 ]; then
echo "❌ Pre-commit check failed (exit code: ${EXIT_CODE})."
else
echo "❌ Git working directory is dirty."
echo "📌 This likely means that pre-commit made changes that were not committed."
echo "🔍 Modified files:"
git diff --name-only
fi
echo "🚒 To prevent/address this CI issue, please install/use pre-commit locally."
echo "📖 More details here: https://superset.apache.org/docs/contributing/development#git-hooks"
exit 1
fi

View File

@@ -1,70 +0,0 @@
name: Prefer TypeScript
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
paths:
- "superset-frontend/src/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset-frontend/src/**"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
prefer_typescript:
if: github.ref == 'ref/heads/master' && github.event_name == 'pull_request'
name: Prefer TypeScript
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Get changed files
id: changed
uses: ./.github/actions/file-changes-action
with:
githubToken: ${{ github.token }}
- name: Determine if a .js or .jsx file was added
id: check
run: |
js_files_added() {
jq -r '
map(
select(
endswith(".js") or endswith(".jsx")
)
) | join("\n")
' ${HOME}/files_added.json
}
echo "js_files_added=$(js_files_added)" >> $GITHUB_OUTPUT
- if: steps.check.outputs.js_files_added
name: Add Comment to PR
uses: ./.github/actions/comment-on-pr
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
with:
msg: |
### WARNING: Prefer TypeScript
Looks like your PR contains new `.js` or `.jsx` files:
```
${{steps.check.outputs.js_files_added}}
```
As decided in [SIP-36](https://github.com/apache/superset/issues/9101), all new frontend code should be written in TypeScript. Please convert above files to TypeScript then re-request review.

View File

@@ -3,52 +3,36 @@ name: release-workflow
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.NPM_TOKEN != '' && secrets.GH_PERSONAL_ACCESS_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
build:
needs: config
if: needs.config.outputs.has-secrets
name: Bump version and publish package(s)
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
strategy:
matrix:
node-version: [16]
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v2
with:
# pulls all commits (needed for lerna / semantic release to correctly version)
fetch-depth: 0
- name: Get tags and filter trigger tags
run: |
if ! git fetch --depth=1 origin "+refs/tags/*:refs/tags/*"; then
echo "::notice title=Workflow skipped::No tags present in repository"
exit
fi
echo "HAS_TAGS=1" >> $GITHUB_ENV"
git fetch --depth=1 origin "+refs/tags/*:refs/tags/*"
git fetch --prune --unshallow
git tag -d `git tag | grep -E '^trigger-'`
- name: Install Node.js
if: env.HAS_TAGS
uses: actions/setup-node@v5
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: ${{ matrix.node-version }}
- name: Cache npm
if: env.HAS_TAGS
uses: actions/cache@v4
uses: actions/cache@v1
with:
path: ~/.npm # npm cache files are stored in `~/.npm` on Linux/macOS
key: ${{ runner.OS }}-node-${{ hashFiles('**/package-lock.json') }}
@@ -57,12 +41,10 @@ jobs:
${{ runner.OS }}-
- name: Get npm cache directory path
if: env.HAS_TAGS
id: npm-cache-dir-path
run: echo "dir=$(npm config get cache)" >> $GITHUB_OUTPUT
run: echo "::set-output name=dir::$(npm config get cache)"
- name: Cache npm
if: env.HAS_TAGS
uses: actions/cache@v4
uses: actions/cache@v1
id: npm-cache # use this to check for `cache-hit` (`steps.npm-cache.outputs.cache-hit != 'true'`)
with:
path: ${{ steps.npm-cache-dir-path.outputs.dir }}
@@ -71,20 +53,16 @@ jobs:
${{ runner.os }}-npm-
- name: Install dependencies
if: env.HAS_TAGS
working-directory: ./superset-frontend
run: npm ci
- name: Run unit tests
if: env.HAS_TAGS
working-directory: ./superset-frontend
run: npm run test -- plugins packages
- name: Build packages
if: env.HAS_TAGS
working-directory: ./superset-frontend
run: npm run plugins:build
- name: Configure npm and git
if: env.HAS_TAGS
run: |
echo "@superset-ui:registry=https://registry.npmjs.org/" > .npmrc
echo "registry=https://registry.npmjs.org/" >> .npmrc
@@ -92,17 +70,17 @@ jobs:
npm whoami
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git remote set-url origin "https://${GITHUB_TOKEN}@github.com/apache-superset/superset-ui.git" > /dev/null 2>&1
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Bump version and publish package(s)
if: env.HAS_TAGS
working-directory: ./superset-frontend
run: |
git tag -d `git tag | grep -E '^trigger-'`
npm run plugins:release-from-tag
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}

View File

@@ -1,50 +0,0 @@
name: 🎪 Showtime Cleanup
# Scheduled cleanup of expired environments
on:
schedule:
- cron: '0 */6 * * *' # Every 6 hours
# Manual trigger for testing
workflow_dispatch:
inputs:
max_age_hours:
description: 'Maximum age in hours before cleanup'
required: false
default: '48'
type: string
# Common environment variables
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ vars.AWS_REGION || 'us-west-2' }}
GITHUB_ORG: ${{ github.repository_owner }}
GITHUB_REPO: ${{ github.event.repository.name }}
jobs:
cleanup-expired:
name: Clean up expired showtime environments
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Install Superset Showtime
run: pip install superset-showtime
- name: Cleanup expired environments
run: |
MAX_AGE="${{ github.event.inputs.max_age_hours || '48' }}"
# Validate max_age is numeric
if [[ ! "$MAX_AGE" =~ ^[0-9]+$ ]]; then
echo "❌ Invalid max_age_hours format: $MAX_AGE (must be numeric)"
exit 1
fi
echo "Cleaning up environments older than ${MAX_AGE}h"
python -m showtime cleanup --older-than "${MAX_AGE}h"

View File

@@ -1,179 +0,0 @@
name: 🎪 Superset Showtime
# Ultra-simple: just sync on any PR state change
on:
pull_request_target:
types: [labeled, unlabeled, synchronize, closed]
# Manual testing
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to sync'
required: true
type: number
sha:
description: 'Specific SHA to deploy (optional, defaults to latest)'
required: false
type: string
# Common environment variables for all jobs (non-sensitive only)
env:
AWS_REGION: us-west-2
GITHUB_ORG: ${{ github.repository_owner }}
GITHUB_REPO: ${{ github.event.repository.name }}
GITHUB_ACTOR: ${{ github.actor }}
jobs:
sync:
name: 🎪 Sync PR to desired state
runs-on: ubuntu-latest
timeout-minutes: 90
permissions:
contents: read
pull-requests: write
steps:
- name: Security Check - Authorize Maintainers Only
id: auth
uses: actions/github-script@v8
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
script: |
const actor = context.actor;
console.log(`🔍 Checking authorization for ${actor}`);
// Early exit for workflow_dispatch - assume authorized since it's manually triggered
if (context.eventName === 'workflow_dispatch') {
console.log(`✅ Workflow dispatch event - assuming authorized for ${actor}`);
core.setOutput('authorized', 'true');
return;
}
const { data: permission } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: actor
});
console.log(`📊 Permission level for ${actor}: ${permission.permission}`);
const authorized = ['write', 'admin'].includes(permission.permission);
// If this is a synchronize event from unauthorized user, check if Showtime is active and set blocked label
if (!authorized && context.eventName === 'pull_request_target' && context.payload.action === 'synchronize') {
console.log(`🔒 Synchronize event detected - checking if Showtime is active`);
// Check if PR has any circus tent labels (Showtime is in use)
const { data: issue } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number
});
const hasCircusLabels = issue.labels.some(label => label.name.startsWith('🎪 '));
if (hasCircusLabels) {
console.log(`🎪 Circus labels found - setting blocked label to prevent auto-deployment`);
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
labels: ['🎪 🔒 showtime-blocked']
});
console.log(`✅ Blocked label set - Showtime will detect and skip operations`);
} else {
console.log(` No circus labels found - Showtime not in use, skipping block`);
}
}
if (!authorized) {
console.log(`🚨 Unauthorized user ${actor} - skipping all operations`);
core.setOutput('authorized', 'false');
return;
}
console.log(`✅ Authorized maintainer: ${actor}`);
core.setOutput('authorized', 'true');
- name: Install Superset Showtime
if: steps.auth.outputs.authorized == 'true'
run: |
echo "::notice::Maintainer ${{ github.actor }} triggered deploy for PR ${{ github.event.pull_request.number || github.event.inputs.pr_number }}"
pip install --upgrade superset-showtime
showtime version
- name: Check what actions are needed
if: steps.auth.outputs.authorized == 'true'
id: check
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Bulletproof PR number extraction
if [[ -n "${{ github.event.pull_request.number }}" ]]; then
PR_NUM="${{ github.event.pull_request.number }}"
elif [[ -n "${{ github.event.inputs.pr_number }}" ]]; then
PR_NUM="${{ github.event.inputs.pr_number }}"
else
echo "❌ No PR number found in event or inputs"
exit 1
fi
echo "Using PR number: $PR_NUM"
# Run sync check-only with optional SHA override
if [[ -n "${{ github.event.inputs.sha }}" ]]; then
OUTPUT=$(python -m showtime sync $PR_NUM --check-only --sha "${{ github.event.inputs.sha }}")
else
OUTPUT=$(python -m showtime sync $PR_NUM --check-only)
fi
echo "$OUTPUT"
# Extract the outputs we need for conditional steps
BUILD=$(echo "$OUTPUT" | grep "build_needed=" | cut -d'=' -f2)
SYNC=$(echo "$OUTPUT" | grep "sync_needed=" | cut -d'=' -f2)
PR_NUM_OUT=$(echo "$OUTPUT" | grep "pr_number=" | cut -d'=' -f2)
TARGET_SHA=$(echo "$OUTPUT" | grep "target_sha=" | cut -d'=' -f2)
echo "build_needed=$BUILD" >> $GITHUB_OUTPUT
echo "sync_needed=$SYNC" >> $GITHUB_OUTPUT
echo "pr_number=$PR_NUM_OUT" >> $GITHUB_OUTPUT
echo "target_sha=$TARGET_SHA" >> $GITHUB_OUTPUT
- name: Checkout PR code (only if build needed)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.build_needed == 'true'
uses: actions/checkout@v5
with:
ref: ${{ steps.check.outputs.target_sha }}
persist-credentials: false
- name: Setup Docker Environment (only if build needed)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.build_needed == 'true'
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
install-docker-compose: "false"
- name: Execute sync (handles everything)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.sync_needed == 'true'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
PR_NUM="${{ steps.check.outputs.pr_number }}"
TARGET_SHA="${{ steps.check.outputs.target_sha }}"
if [[ -n "$TARGET_SHA" ]]; then
python -m showtime sync $PR_NUM --sha "$TARGET_SHA"
else
python -m showtime sync $PR_NUM
fi

View File

@@ -1,67 +0,0 @@
name: Superset App CLI tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-load-examples:
runs-on: ubuntu-24.04
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
REDIS_PORT: 16379
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
# Use custom ports for services to avoid accidentally connecting to
# GitHub action runner's default installations
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
- name: Setup Postgres
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: superset init
if: steps.check.outputs.python
run: |
pip install -e .
superset db upgrade
superset load_test_users
- name: superset load_examples
if: steps.check.outputs.python
run: |
# load examples without test data
superset load_examples --load-big-data

View File

@@ -1,91 +0,0 @@
name: Applitools Cypress
on:
schedule:
- cron: "0 1 * * *"
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.APPLITOOLS_API_KEY != '' && secrets.APPLITOOLS_API_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
cypress-applitools:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
browser: ["chrome"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
APPLITOOLS_APP_NAME: Superset
APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}
APPLITOOLS_BATCH_ID: ${{ github.sha }}
APPLITOOLS_BATCH_NAME: Superset Cypress
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
ref: master
- name: Setup Python
uses: ./.github/actions/setup-backend/
- name: Import test data
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Setup Postgres
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Install cypress
uses: ./.github/actions/cached-dependencies
with:
run: cypress-install
- name: Run Cypress
uses: ./.github/actions/cached-dependencies
env:
CYPRESS_BROWSER: ${{ matrix.browser }}
with:
run: cypress-run-applitools

View File

@@ -1,52 +0,0 @@
name: Applitools Storybook
on:
schedule:
- cron: "0 0 * * *"
env:
APPLITOOLS_APP_NAME: Superset
APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}
APPLITOOLS_BATCH_ID: ${{ github.sha }}
APPLITOOLS_BATCH_NAME: Superset Storybook
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.APPLITOOLS_API_KEY != '' && secrets.APPLITOOLS_API_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
cron:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
ref: master
- name: Set up Node.js
uses: actions/setup-node@v5
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install eyes-storybook dependencies
uses: ./.github/actions/cached-dependencies
with:
run: eyes-storybook-dependencies
- name: Install NPM dependencies
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Run Applitools Eyes-Storybook
working-directory: ./superset-frontend
run: npx eyes-storybook -u https://superset-storybook.netlify.app/

View File

@@ -1,75 +0,0 @@
name: Docs Deployment
on:
push:
paths:
- "docs/**"
- "README.md"
branches:
- "master"
workflow_dispatch: {}
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.SUPERSET_SITE_BUILD != '' && secrets.SUPERSET_SITE_BUILD != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
build-deploy:
needs: config
if: needs.config.outputs.has-secrets
name: Build & Deploy
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Set up Node.js
uses: actions/setup-node@v5
with:
node-version-file: './docs/.nvmrc'
- name: Setup Python
uses: ./.github/actions/setup-backend/
- uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: '21'
- name: Install Graphviz
run: sudo apt-get install -y graphviz
- name: Compute Entity Relationship diagram (ERD)
env:
SUPERSET_SECRET_KEY: not-a-secret
run: |
python scripts/erd/erd.py
curl -L http://sourceforge.net/projects/plantuml/files/1.2023.7/plantuml.1.2023.7.jar/download > ~/plantuml.jar
java -jar ~/plantuml.jar -v -tsvg -r -o "${{ github.workspace }}/docs/static/img/" "${{ github.workspace }}/scripts/erd/erd.puml"
- name: yarn install
working-directory: docs
run: |
yarn install --check-cache
- name: yarn build
working-directory: docs
run: |
yarn build
- name: deploy docs
uses: ./.github/actions/github-action-push-to-another-repository
env:
API_TOKEN_GITHUB: ${{ secrets.SUPERSET_SITE_BUILD }}
with:
source-directory: "./docs/build"
destination-github-username: "apache"
destination-repository-name: "superset-site"
target-branch: "asf-site"
commit-message: "deploying docs: ${{ github.event.head_commit.message }} (apache/superset@${{ github.sha }})"
user-email: dev@superset.apache.org

View File

@@ -1,77 +0,0 @@
name: Docs Testing
on:
pull_request:
paths:
- "docs/**"
- ".github/workflows/superset-docs-verify.yml"
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
linkinator:
# See docs here: https://github.com/marketplace/actions/linkinator
name: Link Checking
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
# Do not bump this linkinator-action version without opening
# an ASF Infra ticket to allow the new version first!
- uses: JustinBeckwith/linkinator-action@3d5ba091319fa7b0ac14703761eebb7d100e6f6d # v1.11.0
continue-on-error: true # This will make the job advisory (non-blocking, no red X)
with:
paths: "**/*.md, **/*.mdx"
linksToSkip: >-
^https://github.com/apache/(superset|incubator-superset)/(pull|issues)/\d+,
^https://github.com/apache/(superset|incubator-superset)/commit/[a-f0-9]+,
superset-frontend/.*CHANGELOG\.md,
http://localhost:8088/,
http://127.0.0.1:3000/,
http://localhost:9001/,
https://charts.bitnami.com/bitnami,
https://www.li.me/,
https://www.fanatics.com/,
https://tails.com/gb/,
https://www.techaudit.info/,
https://avetilearning.com/,
https://www.udemy.com/,
https://trustmedis.com/,
http://theiconic.com.au/,
https://dev.mysql.com/doc/refman/5.7/en/innodb-limits.html,
^https://img\.shields\.io/.*,
https://vkusvill.ru/,
https://www.linkedin.com/in/mark-thomas-b16751158/,
https://theiconic.com.au/,
https://wattbewerb.de/,
https://timbr.ai/,
https://opensource.org/license/apache-2-0,
https://www.plaidcloud.com/
build-deploy:
name: Build & Deploy
runs-on: ubuntu-24.04
defaults:
run:
working-directory: docs
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Set up Node.js
uses: actions/setup-node@v5
with:
node-version-file: './docs/.nvmrc'
- name: yarn install
run: |
yarn install --check-cache
- name: yarn typecheck
run: |
yarn typecheck
- name: yarn build
run: |
yarn build

41
.github/workflows/superset-docs.yml vendored Normal file
View File

@@ -0,0 +1,41 @@
name: Docs
on:
push:
paths:
- "docs/**"
pull_request:
paths:
- "docs/**"
jobs:
build-deploy:
name: Build & Deploy
runs-on: ubuntu-20.04
defaults:
run:
working-directory: docs
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: yarn install
run: |
yarn install --check-cache
- name: yarn build
run: |
yarn build
- name: deploy docs
if: github.ref == 'refs/heads/master'
uses: ./.github/actions/github-action-push-to-another-repository
env:
API_TOKEN_GITHUB: ${{ secrets.SUPERSET_SITE_BUILD }}
with:
source-directory: './docs/build'
destination-github-username: 'apache'
destination-repository-name: 'superset-site'
target-branch: 'asf-site'
commit-message: "deploying docs: ${{ github.event.head_commit.message }} (apache/superset@${{ github.sha }})"
user-email: dev@superset.apache.org

View File

@@ -2,267 +2,124 @@ name: E2E
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/**/docs/**"
paths-ignore:
- "docs/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
workflow_dispatch:
inputs:
use_dashboard:
description: 'Use Cypress Dashboard (true/false) [paid service - trigger manually when needed]. You MUST provide a branch and/or PR number below for this to work.'
required: false
default: 'false'
ref:
description: 'The branch or tag to checkout'
required: false
default: ''
pr_id:
description: 'The pull request ID to checkout'
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
cypress-matrix:
# Somehow one test flakes on 24.04 for unknown reasons, this is the only GHA left on 22.04
runs-on: ubuntu-22.04
permissions:
contents: read
pull-requests: read
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
# when one test fails, DO NOT cancel the other
# parallel_id, because this will kill Cypress processes
# containers, because this will kill Cypress processes
# leaving the Dashboard hanging ...
# https://github.com/cypress-io/github-action/issues/48
fail-fast: false
matrix:
parallel_id: [0, 1, 2, 3, 4, 5]
containers: [1, 2, 3]
browser: ["chrome"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
FLASK_ENV: development
ENABLE_REACT_CRUD_VIEWS: true
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
# Only use dashboard when explicitly requested via workflow_dispatch
USE_DASHBOARD: ${{ github.event.inputs.use_dashboard == 'true' || 'false' }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
services:
postgres:
image: postgres:16-alpine
image: postgres:10-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
image: redis:5-alpine
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@v5
- name: "Checkout (pull) ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
if: github.event_name == 'push'
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@v5
- name: "Checkout (pull_request) ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
if: github.event_name == 'pull_request' || github.event_name == 'pull_request_target'
with:
ref: "refs/pull/${{ github.event.number }}/merge"
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@v5
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
- name: Check if python or frontend changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python frontend
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: "3.8"
- name: OS dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: apt-get-install
- name: Install python dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
pip-upgrade
pip install -r requirements/testing.txt
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@v5
uses: actions/setup-node@v2
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: '16'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: cypress-install
- name: Run Cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
env:
CYPRESS_BROWSER: ${{ matrix.browser }}
PARALLEL_ID: ${{ matrix.parallel_id }}
PARALLELISM: 6
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
NODE_OPTIONS: "--max-old-space-size=4096"
CYPRESS_KEY: YjljODE2MzAtODcwOC00NTA3LWE4NmMtMTU3YmFmMjIzOTRhCg==
with:
run: cypress-run-all ${{ env.USE_DASHBOARD }} ${{ matrix.app_root }}
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
run: cypress-run-all
- name: Upload Artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v2
if: failure()
with:
name: screenshots
path: ${{ github.workspace }}/superset-frontend/cypress-base/cypress/screenshots
name: cypress-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}-${{ matrix.parallel_id }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}
playwright-tests:
runs-on: ubuntu-22.04
permissions:
contents: read
pull-requests: read
strategy:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@v5
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@v5
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@v5
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Required Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
with:
run: playwright-run "${{ matrix.app_root }}"
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
- name: Upload Playwright Artifacts
uses: actions/upload-artifact@v4
if: failure()
with:
path: |
${{ github.workspace }}/superset-frontend/playwright-results/
${{ github.workspace }}/superset-frontend/test-results/
name: playwright-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}

View File

@@ -1,64 +0,0 @@
name: Superset Extensions CLI Package Tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-superset-extensions-cli-package:
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: ["previous", "current", "next"]
defaults:
run:
working-directory: superset-extensions-cli
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.superset-extensions-cli
uses: ./.github/actions/setup-backend/
with:
python-version: ${{ matrix.python-version }}
requirements-type: dev
- name: Run pytest with coverage
if: steps.check.outputs.superset-extensions-cli
run: |
pytest --cov=superset_extensions_cli --cov-report=xml --cov-report=term-missing --cov-report=html -v --tb=short
- name: Upload coverage reports to Codecov
if: steps.check.outputs.superset-extensions-cli
uses: codecov/codecov-action@v5
with:
file: ./coverage.xml
flags: superset-extensions-cli
name: superset-extensions-cli-coverage
fail_ci_if_error: false
- name: Upload HTML coverage report
if: steps.check.outputs.superset-extensions-cli
uses: actions/upload-artifact@v4
with:
name: superset-extensions-cli-coverage-html
path: htmlcov/

View File

@@ -1,169 +1,67 @@
name: "Frontend Build CI (unit tests, linting & sanity checks)"
name: Frontend
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/**/docs/**"
- "dependabot/**/cypress-base/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
env:
TAG: apache/superset:GHA-${{ github.run_id }}
jobs:
frontend-build:
runs-on: ubuntu-24.04
outputs:
should-run: ${{ steps.check.outputs.frontend }}
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
steps:
- name: Checkout Code
uses: actions/checkout@v5
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
fetch-depth: 0
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Check for File Changes
submodules: recursive
- name: Check npm lock file version
run: ./scripts/ci_check_npm_lock_version.sh ./superset-frontend/package-lock.json
- name: Check if frontend changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Build Docker Image
if: steps.check.outputs.frontend
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "git rev-parse --short HEAD"
git rev-parse --short HEAD
echo "git show -s --format=raw HEAD"
git show -s --format=raw HEAD
docker buildx build \
-t $TAG \
--cache-from=type=registry,ref=apache/superset-cache:3.10-slim-trixie \
--target superset-node-ci \
.
- name: Save Docker Image as Artifact
if: steps.check.outputs.frontend
run: |
docker save $TAG | gzip > docker-image.tar.gz
- name: Upload Docker Image Artifact
if: steps.check.outputs.frontend
uses: actions/upload-artifact@v4
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh frontend
- name: Setup Node.js
if: steps.check.outcome == 'failure'
uses: actions/setup-node@v2
with:
name: docker-image
path: docker-image.tar.gz
sharded-jest-tests:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
strategy:
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8]
fail-fast: false
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@v5
node-version: "16"
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
name: docker-image
- name: Load Docker Image
run: docker load < docker-image.tar.gz
- name: npm run test with coverage
run: |
mkdir -p ${{ github.workspace }}/superset-frontend/coverage
docker run \
-v ${{ github.workspace }}/superset-frontend/coverage:/app/superset-frontend/coverage \
--rm $TAG \
bash -c \
"npm run test -- --coverage --shard=${{ matrix.shard }}/8 --coverageReporters=json-summary"
- name: Upload Coverage Artifact
uses: actions/upload-artifact@v4
with:
name: coverage-artifacts-${{ matrix.shard }}
path: superset-frontend/coverage
report-coverage:
needs: [sharded-jest-tests]
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Coverage Artifacts
uses: actions/download-artifact@v5
with:
pattern: coverage-artifacts-*
path: coverage/
- name: Show Files
run: find coverage/
- name: Merge Code Coverage
run: npx nyc merge coverage/ merged-output/coverage-summary.json
- name: Upload Code Coverage
uses: codecov/codecov-action@v5
with:
flags: javascript
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
files: merged-output/coverage-summary.json
slug: apache/superset
lint-frontend:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@v5
with:
name: docker-image
- name: Load Docker Image
run: |
docker load < docker-image.tar.gz
run: npm-install
- name: lint
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
docker run --rm $TAG bash -c \
"npm i && npm run lint"
- name: tsc
npm run lint
npm run prettier-check
- name: Build plugins packages
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: npm run plugins:build
- name: Build plugins Storybook
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: npm run plugins:build-storybook
- name: unit tests
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
docker run --rm $TAG bash -c \
"npm i && npm run plugins:build && npm run type"
validate-frontend:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@v5
with:
name: docker-image
- name: Load Docker Image
run: docker load < docker-image.tar.gz
- name: Build Plugins Packages
run: |
docker run --rm $TAG bash -c \
"npm run plugins:build"
- name: Build Plugins Storybook
run: |
docker run --rm $TAG bash -c \
"npm run plugins:build-storybook"
npm run test -- --coverage
# todo: remove this step when fix generator as a project in root jest.config.js
- name: generator-superset unit tests
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend/packages/generator-superset
run: npx jest
- name: Upload code coverage
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: ../.github/workflows/codecov.sh -c -F javascript

View File

@@ -1,36 +1,28 @@
name: "Helm: lint and test charts"
name: Lint and Test Charts
on:
pull_request:
types: [opened, edited, reopened, synchronize]
paths:
- "helm/**"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
lint-test:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
fetch-depth: 0
- name: Set up Helm
uses: azure/setup-helm@v4
uses: azure/setup-helm@v1
with:
version: v3.16.4
version: v3.5.4
- name: Setup Python
uses: ./.github/actions/setup-backend/
- uses: actions/setup-python@v2
with:
install-superset: 'false'
python-version: 3.8
- name: Set up chart-testing
uses: ./.github/actions/chart-testing-action
@@ -40,7 +32,7 @@ jobs:
run: |
changed=$(ct list-changed --print-config)
if [[ -n "$changed" ]]; then
echo "changed=true" >> $GITHUB_OUTPUT
echo "::set-output name=changed::true"
fi
env:
CT_CHART_DIRS: helm
@@ -52,4 +44,3 @@ jobs:
CT_CHART_DIRS: helm
CT_LINT_CONF: lintconf.yaml
CT_SINCE: HEAD
CT_CHART_REPOS: bitnami=https://charts.bitnami.com/bitnami

View File

@@ -1,38 +1,20 @@
# This workflow automates the release process for Helm charts.
# The workflow creates a new branch for the release and opens a pull request against the 'gh-pages' branch,
# allowing the changes to be reviewed and merged manually.
name: "Helm: release charts"
name: Release Charts
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
paths:
- "helm/**"
workflow_dispatch:
inputs:
ref:
description: "The branch, tag, or commit SHA to check out"
required: false
default: "master"
- 'helm/**'
jobs:
release:
runs-on: ubuntu-24.04
permissions:
contents: write
pull-requests: write
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
ref: ${{ inputs.ref || github.ref_name }}
persist-credentials: true
persist-credentials: false
submodules: recursive
fetch-depth: 0
@@ -42,84 +24,14 @@ jobs:
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Install Helm
uses: azure/setup-helm@v4
uses: azure/setup-helm@v1
with:
version: v3.5.4
- name: Add bitnami repo dependency
run: helm repo add bitnami https://charts.bitnami.com/bitnami
- name: Fetch/list all tags
run: |
# Debugging tags
git fetch --tags --force
git tag -d superset-helm-chart-0.13.4 || true
echo "DEBUG TAGS"
git show-ref --tags
- name: Create unique pages branch name
id: vars
run: echo "branch_name=helm-publish-${GITHUB_SHA:0:7}" >> $GITHUB_ENV
- name: Force recreate branch from gh-pages
run: |
# Ensure a clean working directory
git reset --hard
git clean -fdx
git checkout -b local_gha_temp
git submodule update
# Fetch the latest gh-pages branch
git fetch origin gh-pages
# Check out and reset the target branch based on gh-pages
git checkout -B ${{ env.branch_name }} origin/gh-pages
# Remove submodules from the branch
git submodule deinit -f --all
# Force push to the remote branch
git push origin ${{ env.branch_name }} --force
# Return to the original branch
git checkout local_gha_temp
- name: Fetch/list all tags
run: |
git submodule update
cat .github/actions/chart-releaser-action/action.yml
- name: Run chart-releaser
uses: ./.github/actions/chart-releaser-action
with:
version: v1.6.0
charts_dir: helm
mark_as_latest: false
pages_branch: ${{ env.branch_name }}
env:
CR_TOKEN: "${{ github.token }}"
CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
CR_RELEASE_NAME_TEMPLATE: "superset-helm-chart-{{ .Version }}"
- name: Open Pull Request
uses: actions/github-script@v8
with:
script: |
const branchName = '${{ env.branch_name }}';
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/');
if (!branchName) {
throw new Error("Branch name is not defined.");
}
const pr = await github.rest.pulls.create({
owner,
repo,
title: `Helm chart release for ${branchName}`,
head: branchName,
base: "gh-pages", // Adjust if the target branch is different
body: `This PR releases Helm charts to the gh-pages branch.`,
});
core.info(`Pull request created: ${pr.data.html_url}`);
env:
BRANCH_NAME: ${{ env.branch_name }}

View File

@@ -1,142 +0,0 @@
name: Playwright Experimental Tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
workflow_dispatch:
inputs:
ref:
description: 'The branch or tag to checkout'
required: false
default: ''
pr_id:
description: 'The pull request ID to checkout'
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
# NOTE: Required Playwright tests are in superset-e2e.yml (E2E / playwright-tests)
# This workflow contains only experimental tests that run in shadow mode
playwright-tests-experimental:
runs-on: ubuntu-22.04
continue-on-error: true
permissions:
contents: read
pull-requests: read
strategy:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@v5
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@v5
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@v5
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@v5
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Experimental Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
with:
run: playwright-run "${{ matrix.app_root }}" experimental/
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
- name: Upload Playwright Artifacts
uses: actions/upload-artifact@v4
if: failure()
with:
path: |
${{ github.workspace }}/superset-frontend/playwright-results/
${{ github.workspace }}/superset-frontend/test-results/
name: playwright-experimental-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}

View File

@@ -3,19 +3,18 @@ name: Python-Integration
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-mysql:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -24,60 +23,64 @@ jobs:
mysql+mysqldb://superset:superset@127.0.0.1:13306/superset?charset=utf8mb4&binary_prefix=true
services:
mysql:
image: mysql:8.0
image: mysql:5.7
env:
MYSQL_ROOT_PASSWORD: root
ports:
- 13306:3306
options: >-
--health-cmd="mysqladmin ping --silent"
--health-interval=10s
--health-timeout=5s
--health-retries=5
redis:
image: redis:7-alpine
image: redis:5-alpine
options: --entrypoint redis-server
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup MySQL
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-mysql
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-mysql
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (MySQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@v5
with:
flags: python,mysql
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F mysql
test-postgres:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["current", "previous", "next"]
python-version: [3.8, 3.9]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -85,7 +88,7 @@ jobs:
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
services:
postgres:
image: postgres:16-alpine
image: postgres:10-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -94,95 +97,106 @@ jobs:
# GitHub action runner's default installations
- 15432:5432
redis:
image: redis:7-alpine
image: redis:5-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Setup Postgres
if: steps.check.outputs.python
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@v5
with:
flags: python,postgres
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F postgres
test-sqlite:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
REDIS_PORT: 16379
SUPERSET__SQLALCHEMY_DATABASE_URI: |
sqlite:///${{ github.workspace }}/.temp/superset.db?check_same_thread=true
SUPERSET__SQLALCHEMY_EXAMPLES_URI: |
sqlite:///${{ github.workspace }}/.temp/examples.db?check_same_thread=true
sqlite:///${{ github.workspace }}/.temp/unittest.db
services:
redis:
image: redis:7-alpine
image: redis:5-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
# sqlite needs this working directory
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
mkdir ${{ github.workspace }}/.temp
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (SQLite)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@v5
with:
flags: python,sqlite
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F sqlite

View File

@@ -0,0 +1,111 @@
# Python Misc unit tests
name: Python Misc
on:
push:
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
jobs:
python-lint:
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check if python changes are present
id: check
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
- name: pylint
if: steps.check.outcome == 'failure'
# `-j 0` run Pylint in parallel
run: pylint -j 0 superset
pre-commit:
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: |
requirements/base.txt
requirements/integration.txt
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/base.txt
pip install -r requirements/integration.txt
- name: pre-commit
run: pre-commit run --all-files
babel-extract:
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/base.txt'
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/base.txt
- name: Test babel extraction
run: flask fab babel-extract --target superset/translations --output superset/translations/messages.pot --config superset/translations/babel.cfg -k _,__,t,tn,tct

View File

@@ -3,20 +3,18 @@ name: Python Presto/Hive
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-postgres-presto:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -25,7 +23,7 @@ jobs:
SUPERSET__SQLALCHEMY_EXAMPLES_URI: presto://localhost:15433/memory/default
services:
postgres:
image: postgres:16-alpine
image: postgres:10-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -43,48 +41,57 @@ jobs:
# GitHub action runner's default installations
- 15433:8080
redis:
image: redis:7-alpine
image: redis:5-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python == 'true'
- name: Setup Postgres
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
echo "${{ steps.check.outputs.python }}"
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
- name: Upload code coverage
uses: codecov/codecov-action@v5
with:
flags: python,presto
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F presto
test-postgres-hive:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -94,7 +101,7 @@ jobs:
UPLOAD_FOLDER: /tmp/.superset/uploads/
services:
postgres:
image: postgres:16-alpine
image: postgres:10-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -103,50 +110,56 @@ jobs:
# GitHub action runner's default installations
- 15432:5432
redis:
image: redis:7-alpine
image: redis:5-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Create csv upload directory
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: sudo mkdir -p /tmp/.superset/uploads
- name: Give write access to the csv upload directory
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: sudo chown -R $USER:$USER /tmp/.superset
- name: Start hadoop and hive
if: steps.check.outputs.python
run: docker compose -f scripts/databases/hive/docker-compose.yml up -d
if: steps.check.outcome == 'failure'
run: docker-compose -f scripts/databases/hive/docker-compose.yml up -d
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup Postgres
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
pip install -e .[hive]
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
- name: Upload code coverage
uses: codecov/codecov-action@v5
with:
flags: python,hive
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F hive

View File

@@ -3,58 +3,56 @@ name: Python-Unit
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
unit-tests:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["previous", "current", "next"]
python-version: [3.8, 3.9]
env:
PYTHONPATH: ${{ github.workspace }}
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Python unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear --maxfail=50
- name: Python 100% coverage unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100
- name: Upload code coverage
uses: codecov/codecov-action@v5
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
# TODO: separated requiermentes.txt file just for unit tests
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
flags: python,unit
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
mkdir ${{ github.workspace }}/.temp
- name: Python unit tests
if: steps.check.outcome == 'failure'
run: |
pytest --durations=0 ./tests/common ./tests/unit_tests --cache-clear
- name: Upload code coverage
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F unit

View File

@@ -2,66 +2,56 @@ name: Translations
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
branches-ignore:
- "dependabot/npm_and_yarn/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
frontend-check-translations:
runs-on: ubuntu-24.04
frontend-check:
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Node.js
if: steps.check.outputs.frontend
uses: actions/setup-node@v5
uses: actions/setup-node@v2
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: '16'
- name: Install dependencies
if: steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: lint
if: steps.check.outputs.frontend
working-directory: ./superset-frontend
run: |
npm run build-translation
npm run check-translation
babel-extract:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install -r requirements/base.txt
- name: Test babel extraction
if: steps.check.outputs.python
run: ./scripts/translations/babel_update.sh
run: ./scripts/babel_update.sh

View File

@@ -1,38 +1,27 @@
name: WebSocket server
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
paths:
- "superset-websocket/**"
pull_request:
paths:
- "superset-websocket/**"
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
app-checks:
runs-on: ubuntu-24.04
if: github.event.pull_request.draft == false
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
uses: actions/checkout@v2
with:
persist-credentials: false
- name: Install dependencies
working-directory: ./superset-websocket
run: npm ci
- name: eslint
run: npm install
- name: lint
working-directory: ./superset-websocket
run: npm run eslint -- . --quiet
- name: typescript checks
working-directory: ./superset-websocket
run: npm run type
run: npm run lint
- name: prettier
working-directory: ./superset-websocket
run: npm run prettier-check

View File

@@ -1,56 +0,0 @@
name: SupersetBot Workflow
on:
issue_comment:
types: [created, edited]
# Making the workflow testable since `issue_comment` only triggers on
# the default branch
workflow_dispatch:
inputs:
comment_body:
description: 'Comment Body'
required: true
type: string
jobs:
supersetbot:
runs-on: ubuntu-24.04
if: >
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@supersetbot'))
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Quickly add thumbs up!
if: github.event_name == 'issue_comment' && contains(github.event.comment.body, '@supersetbot')
uses: actions/github-script@v8
with:
script: |
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/')
await github.rest.reactions.createForIssueComment({
owner,
repo,
comment_id: context.payload.comment.id,
content: '+1'
});
- name: "Checkout ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
persist-credentials: false
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute custom Node.js script
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_ISSUE_NUMBER: ${{ github.event.issue.number }}
COMMENT_BODY: ${{ github.event.comment.body }}${{ github.event.inputs.comment_body }}
run: |
supersetbot run "$COMMENT_BODY"

View File

@@ -1,134 +0,0 @@
name: Publish a Release
on:
release:
types: [published, edited]
# Can be triggered manually
workflow_dispatch:
inputs:
release:
required: true
description: The version to generate
git-ref:
required: true
description: The git reference to checkout prior to running the docker build
force-latest:
required: true
type: choice
default: 'false'
description: Whether to force a latest tag on the release
options:
- 'true'
- 'false'
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.DOCKERHUB_USER != '' && secrets.DOCKERHUB_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
docker-release:
needs: config
if: needs.config.outputs.has-secrets
name: docker-release
runs-on: ubuntu-24.04
strategy:
matrix:
build_preset: ["dev", "lean", "py310", "websocket", "dockerize", "py311", "py312"]
fail-fast: false
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
install-docker-compose: "false"
build: "true"
- name: Use Node.js 20
uses: actions/setup-node@v5
with:
node-version: 20
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute custom Node.js script
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
RELEASE="${{ github.event.release.tag_name }}"
FORCE_LATEST=""
EVENT="${{github.event_name}}"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# in the case of a manually-triggered run, read release from input
RELEASE="${{ github.event.inputs.release }}"
if [ "${{ github.event.inputs.force-latest }}" = "true" ]; then
FORCE_LATEST="--force-latest"
fi
git checkout "${{ github.event.inputs.git-ref }}"
EVENT="release"
fi
supersetbot docker \
--push \
--preset ${{ matrix.build_preset }} \
--context "$EVENT" \
--context-ref "$RELEASE" $FORCE_LATEST \
--platform "linux/arm64" \
--platform "linux/amd64"
# Returning to master to support closing setup-supersetbot
git checkout master
update-prs-with-release-info:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Use Node.js 20
uses: actions/setup-node@v5
with:
node-version: 20
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Label the PRs with the right release-related labels
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
export GITHUB_ACTOR=""
git fetch --all --tags
git checkout master
RELEASE="${{ github.event.release.tag_name }}"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# in the case of a manually-triggered run, read release from input
RELEASE="${{ github.event.inputs.release }}"
fi
supersetbot release-label $RELEASE

View File

@@ -1,47 +0,0 @@
name: Upload Technical Debt Metrics to Google Sheets
on:
push:
branches:
- master
- "[0-9].[0-9]*"
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.GSHEET_KEY != '' ) || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
process-and-upload:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
name: Generate Reports
steps:
- name: Checkout Repository
uses: actions/checkout@v5
- name: Set up Node.js
uses: actions/setup-node@v5
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install Dependencies
run: npm ci
working-directory: ./superset-frontend
- name: Run Script
env:
SPREADSHEET_ID: "1oABNnzxJYzwUrHjr_c9wfYEq9dFL1ScVof9LlaAdxvo"
SERVICE_ACCOUNT_KEY: ${{ secrets.GSHEET_KEY }}
run: npm run lint-stats
continue-on-error: true
working-directory: ./superset-frontend

View File

@@ -1,22 +0,0 @@
name: Welcome New Contributor
on:
pull_request_target:
types: [opened]
jobs:
welcome:
runs-on: ubuntu-24.04
permissions:
pull-requests: write
steps:
- name: Welcome Message
uses: actions/first-interaction@v3
continue-on-error: true
with:
repo-token: ${{ github.token }}
pr-message: |-
Congrats on making your first PR and thank you for contributing to Superset! :tada: :heart:
We hope to see you in our [Slack](https://apache-superset.slack.com/) community too! Not signed up? Use our [Slack App](http://bit.ly/join-superset-slack) to self-register.

38
.gitignore vendored
View File

@@ -21,7 +21,6 @@
*.swp
__pycache__
.aider*
.local
.cache
.bento*
@@ -33,7 +32,6 @@ cover
.env
.envrc
.idea
.roo
.mypy_cache
.python-version
.tox
@@ -44,7 +42,7 @@ _modules
_static
build
app.db
*.egg-info/
apache_superset.egg-info/
changelog.sh
dist
dump.rdb
@@ -52,6 +50,7 @@ env
venv*
env_py3
envpy3
env36
local_config.py
/superset_config.py
/superset_text.yml
@@ -60,22 +59,17 @@ superset/bin/supersetc
tmp
rat-results.txt
superset/app/
superset-websocket/config.json
# Node.js, webpack artifacts, storybook
*.entry.js
*.js.map
node_modules
npm-debug.log*
superset/static/assets/*
!superset/static/assets/.gitkeep
superset/static/uploads/*
!superset/static/uploads/.gitkeep
superset/static/assets
superset/static/version_info.json
superset-frontend/**/esm/*
superset-frontend/**/lib/*
superset-frontend/**/storybook-static/*
superset-frontend/migration-storybook.log
yarn-error.log
*.map
*.min.js
@@ -93,7 +87,6 @@ scripts/*.zip
# IntelliJ
*.iml
venv
.venv
@eaDir/
# PyCharm
@@ -109,32 +102,11 @@ ghostdriver.log
testCSV.csv
.terser-plugin-cache/
apache-superset-*.tar.gz*
apache_superset-*.tar.gz*
release.json
# Translation-related files
# these json files are generated by ./scripts/po2json.sh
superset/translations/**/messages.json
# these mo binary files are generated by `pybabel compile`
superset/translations/**/messages.mo
# Translation binaries
messages.mo
docker/requirements-local.txt
cache/
docker/*local*
docker/superset-websocket/config.json
docker-compose.override.yml
.temp_cache
# Jest test report
test-report.html
superset/static/stats/statistics.html
# LLM-related
CLAUDE.local.md
PROJECT.md
.aider*
.claude_rc*
.env.local
oxc-custom-build/

View File

@@ -1,4 +0,0 @@
{
"no-bare-urls": false,
"line-length": false
}

View File

@@ -15,123 +15,39 @@
# limitations under the License.
#
repos:
- repo: https://github.com/MarcoGorelli/auto-walrus
rev: 0.3.4
- repo: https://github.com/PyCQA/isort
rev: 5.9.3
hooks:
- id: auto-walrus
- id: isort
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
rev: v0.941
hooks:
- id: mypy
name: mypy (main)
args: [--check-untyped-defs]
exclude: ^superset-extensions-cli/
additional_dependencies: [
types-simplejson,
types-python-dateutil,
types-requests,
# types-redis 4.6.0.5 is failing mypy
# because of https://github.com/python/typeshed/pull/10531
types-redis==4.6.0.4,
types-pytz,
types-croniter,
types-PyYAML,
types-setuptools,
types-paramiko,
types-Markdown,
]
- id: mypy
name: mypy (superset-extensions-cli)
args: [--check-untyped-defs]
files: ^superset-extensions-cli/
additional_dependencies: [types-all]
- repo: https://github.com/peterdemin/pip-compile-multi
rev: v2.4.1
hooks:
- id: pip-compile-multi-verify
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
rev: v3.2.0
hooks:
- id: check-docstring-first
- id: check-added-large-files
exclude: ^.*\.(geojson)$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$
exclude: \.(geojson)$
- id: check-yaml
exclude: ^helm/superset/templates/
- id: debug-statements
- id: end-of-file-fixer
exclude: .*/lerna\.json$
- id: trailing-whitespace
exclude: ^.*\.(snap)
args: ["--markdown-linebreak-ext=md"]
- repo: local
- repo: https://github.com/psf/black
rev: 22.3.0
hooks:
- id: prettier-frontend
name: prettier (frontend)
entry: bash -c 'cd superset-frontend && for file in "$@"; do npx prettier --write "${file#superset-frontend/}"; done'
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx|css|scss|sass|json)$
- repo: local
- id: black
language_version: python3
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v2.4.1 # Use the sha or tag you want to point at
hooks:
- id: oxlint-frontend
name: oxlint (frontend)
entry: ./scripts/oxlint.sh
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx)$
- id: custom-rules-frontend
name: custom rules (frontend)
entry: ./scripts/check-custom-rules.sh
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx)$
- id: eslint-docs
name: eslint (docs)
entry: bash -c 'cd docs && FILES=$(echo "$@" | sed "s|docs/||g") && yarn eslint --fix --quiet $FILES'
language: system
pass_filenames: true
files: ^docs/.*\.(js|jsx|ts|tsx)$
- id: type-checking-frontend
name: Type-Checking (Frontend)
entry: ./scripts/check-type.js package=superset-frontend excludeDeclarationDir=cypress-base
language: system
files: ^superset-frontend\/.*\.(js|jsx|ts|tsx)$
exclude: ^superset-frontend/cypress-base\/
require_serial: true
# blacklist unsafe functions like make_url (see #19526)
- repo: https://github.com/skorokithakis/blacklist-pre-commit-hook
rev: e2f070289d8eddcaec0b580d3bde29437e7c8221
hooks:
- id: blacklist
args: ["--blacklisted-names=make_url", "--ignore=tests/"]
- repo: https://github.com/norwoodj/helm-docs
rev: v1.14.2
hooks:
- id: helm-docs
files: helm
verbose: false
args: ["--log-level", "error"]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.7
hooks:
- id: ruff-format
- id: ruff
args: [--fix]
- repo: local
hooks:
- id: pylint
name: pylint with custom Superset plugins
entry: bash
language: system
types: [python]
exclude: ^(tests/|superset/migrations/|scripts/|RELEASING/|docker/)
args:
- -c
- |
TARGET_BRANCH=${GITHUB_BASE_REF:-master}
# Only fetch if we're not in CI (CI already has all refs)
if [ -z "$CI" ]; then
git fetch --no-recurse-submodules origin "$TARGET_BRANCH" 2>/dev/null || true
fi
BASE=$(git merge-base origin/"$TARGET_BRANCH" HEAD 2>/dev/null) || BASE="HEAD"
files=$(git diff --name-only --diff-filter=ACM "$BASE"..HEAD 2>/dev/null | grep '^superset/.*\.py$' || true)
if [ -n "$files" ]; then
pylint --rcfile=.pylintrc --load-plugins=superset.extensions.pylint --reports=no $files
else
echo "No Python files to lint."
fi
- id: prettier
args: ['--ignore-path=./superset-frontend/.prettierignore']
files: 'superset-frontend'

View File

@@ -36,7 +36,7 @@ persistent=yes
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=superset.extensions.pylint
load-plugins=
# Use multiple processes to speed up Pylint.
jobs=2
@@ -50,11 +50,44 @@ unsafe-load-any-extension=no
# run arbitrary code
extension-pkg-whitelist=pyarrow
# Allow optimization of some AST trees. This will activate a peephole AST
# optimizer, which will apply various small optimizations. For instance, it can
# be used to obtain the result of joining multiple strings with the addition
# operator. Joining a lot of strings can lead to a maximum recursion error in
# Pylint and this flag can prevent that. It has one side effect, the resulting
# AST will be different than the one from reality. This option is deprecated
# and it will be removed in Pylint 2.0.
optimize-ast=no
[MESSAGES CONTROL]
disable=all
enable=json-import,disallowed-sql-import,consider-using-transaction
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=
useless-suppression,
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=
missing-docstring,
duplicate-code,
unspecified-encoding,
# re-enable once this no longer raises false positives
too-many-instance-attributes
[REPORTS]
@@ -63,6 +96,12 @@ enable=json-import,disallowed-sql-import,consider-using-transaction
# mypackage.mymodule.MyReporterClass.
output-format=text
# Put messages in a separate file for each module / package specified on the
# command line instead of printing them on stdout. Reports (if any) will be
# written in a file name "pylint_global.[txt|html]". This option is deprecated
# and it will be removed in Pylint 2.0.
files-output=no
# Tells whether to display a full report or only the messages
reports=yes
@@ -84,7 +123,7 @@ evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / stateme
good-names=_,df,ex,f,i,id,j,k,l,o,pk,Run,ts,v,x,y
# Bad variable names which should always be refused, separated by a comma
bad-names=bar,baz,db,fd,foo,sesh,session,tata,toto,tutu
bad-names=fd,foo,bar,baz,toto,tutu,tata
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
@@ -95,40 +134,68 @@ include-naming-hint=no
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=
abc.abstractproperty,
sqlalchemy.ext.hybrid.hybrid_property
property-classes=abc.abstractproperty
# Regular expression matching correct argument names
argument-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for argument names
argument-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct method names
method-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for method names
method-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct variable names
variable-rgx=[a-z_][a-z0-9_]{1,30}$
# Naming hint for variable names
variable-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct inline iteration names
inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
# Naming hint for inline iteration names
inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
# Regular expression matching correct constant names
const-rgx=(([A-Za-z_][A-Za-z0-9_]*)|(__.*__))$
# Naming hint for constant names
const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Regular expression matching correct class names
class-rgx=[A-Z_][a-zA-Z0-9]+$
# Naming hint for class names
class-name-hint=[A-Z_][a-zA-Z0-9]+$
# Regular expression matching correct class attribute names
class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Naming hint for class attribute names
class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Regular expression matching correct module names
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Naming hint for module names
module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Regular expression matching correct attribute names
attr-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for attribute names
attr-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct function names
function-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for function names
function-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
@@ -147,7 +214,7 @@ max-nested-blocks=5
[FORMAT]
# Maximum number of characters on a single line.
max-line-length=100
max-line-length=90
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
@@ -156,6 +223,12 @@ ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# else.
single-line-if-stmt=no
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,dict-separator
# Maximum number of lines in a module
max-module-lines=1000
@@ -225,12 +298,12 @@ ignore-mixin-members=yes
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=numpy,pandas,alembic.op,sqlalchemy,alembic.context,flask_appbuilder.security.sqla.PermissionView.role,flask_appbuilder.Model.metadata,flask_appbuilder.Base.metadata
ignored-modules=numpy,pandas,alembic.op,sqlalchemy,alembic.context,flask_appbuilder.security.sqla.PermissionView.role,flask_appbuilder.Model.metadata,flask_appbuilder.Base.metadata,distutils
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=contextlib.closing,optparse.Values,thread._local,_thread._local
ignored-classes=contextlib.closing,optparse.Values,thread._local,_thread._local,sqlalchemy.orm.scoping.scoped_session
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
@@ -352,4 +425,4 @@ analyse-fallback-blocks=no
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=builtins.Exception
overgeneral-exceptions=Exception

View File

@@ -1,5 +1,6 @@
# Note: these patterns are applied to single files or directories, not full paths
.gitignore
docs/README.md
.gitattributes
.gitkeep
.coverage
@@ -11,7 +12,6 @@
.nvmrc
.prettierrc
.rat-excludes
.swcrc
.*log
.*pyc
.*lock
@@ -33,11 +33,10 @@ apache_superset.egg-info
# json and csv in general cannot have comments
.*json
.*csv
# jinja templates often need to be as-is
.*j2
# Generated doc files
env/*
docs/.htaccess*
docs-v2/.htaccess*
.nojekyll
_build/*
_static/*
@@ -63,29 +62,3 @@ tsconfig.tsbuildinfo
.prettierignore
generator-superset/*
temporary_superset_ui/*
# skip license checks for auto-generated test snapshots
.*snap
# docs overrides for third party logos we don't have the rights to
google-big-query.svg
google-sheets.svg
ibm-db2.svg
postgresql.svg
snowflake.svg
ydb.svg
loading.svg
# docs-related
erd.puml
erd.svg
intro_header.txt
# for LLMs
llm-context.md
AGENTS.md
LLMS.md
CLAUDE.md
CURSOR.md
GEMINI.md
GPT.md

Some files were not shown because too many files have changed in this diff Show More