Compare commits

...
Author SHA1 Message Date
Beto Dealmeida 99797911bd Update CHANELOG 2021-04-05 10:45:12 -07:00
Maxime BeaucheminandBeto Dealmeida a4e8c0a5a1 fix: 'Run in SQL Lab' redirect bug (#13657) 2021-04-05 10:42:35 -07:00
Beto Dealmeida 69dae2ff52 Fix release 2021-03-26 16:56:19 -07:00
Beto Dealmeida bc7315ea0c Update version in superset-frontend/package.json 2021-03-26 16:32:30 -07:00
Lily KuangandBeto Dealmeida a6e15e0353 chore: update change log with cherry-picks for release 1.1 (#13824)
* add 1.1 cherries to change log

* one more cherry
2021-03-26 15:58:55 -07:00
Ben ReinhartandBeto Dealmeida ed88c9bd47 fix(#13734): Properly escape special characters in CSV output (#13735)
* fix: Escape csv content during downloads

* Reuse CsvResponse object

* Use correct mimetype for csv responses

* Ensure that headers are also escaped

* Update escaping logic
2021-03-26 15:23:04 -07:00
Beto DealmeidaandMichael S. Molina d72c3b0686 chore: Update docs for 1.1 (#13732)
* Update docs for 1.1

* Update CONTRIBUTING.md

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* Update UPDATING.md

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* Update UPDATING.md

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

* Update UPDATING.md

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>

Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
2021-03-24 19:25:20 -07:00
Srini KadamatiandBeto Dealmeida 531729a11a docs: added release notes for 1.1 (#13777)
* docs: added release notes for 1.1

* fixed link

* incorporated villes excellent feedback
2021-03-24 10:15:41 -07:00
Lily KuangandBeto Dealmeida 2cb9c97ee7 chore: script for changelog (#13588)
* change log with section

* add risk flag for showing risk pull request

* update changlog.md

* lint mypy

* small fixes
2021-03-19 14:08:54 -07:00
Elizabeth ThompsonandBeto Dealmeida e054f17f73 docs: add file for users to see status of feature flags (#13402)
* add file for users to see status of feature flags

* move comment

* update flags per feedback

* move to subfolder and improve docs

* move CODE_OF_CONDUCT to root
2021-03-19 14:07:59 -07:00
Daniel Vaz GasparandBeto Dealmeida eb35b804ac fix: url shortener invalid input (#13461)
* fix: url shortner invalid input

* fix lint
2021-03-12 13:32:19 -08:00
MayurandBeto Dealmeida e8162cb24f changes added (#13431) 2021-03-12 13:31:45 -08:00
Ville BrofeldtandBeto Dealmeida d5ef4897eb chore: bump pyarrow and pandas (#12882)
* bump pyarrow and pandas

* remove df copy
2021-03-12 13:29:17 -08:00
c9f5e1c10a fix(alerts&reports): Alerts & Reports will use values from WEBDRIVER_WINDOW option (#13157)
* fix: thumbnails and reports will be use WEBDRIVER_WINDOW option

* changes reformatted

* config change reverted. thumbnails sizes changed to original

* typo fix

* bugfix

defining defaults in thumbnails.py caused thumbnail caches invalidated.
they moved to init.

Co-authored-by: Ibrahim Ercan <ibrahim.ercan@vlmedia.com.tr>
2021-03-12 13:28:20 -08:00
Rob DiCiuccioandBeto Dealmeida e98151d529 fix(celery): Reset DB connection pools for forked worker processes (#13350)
* Reset sqlalchemy connection pool on celery process fork

* Fix race condition with async chart loading state

* pylint: ignore

* prettier
2021-03-12 13:26:19 -08:00
Beto Dealmeida c881c06b7c feat: add Firebird DB engine spec (#13353)
* feat: add Firebird DB engine spec

* Add dep to setup.py

* Fix lint

* Add tests

* Remove uneeded code

* Fix old bug
2021-03-12 13:25:53 -08:00
Beto Dealmeida 2facfd82d0 WIP 2021-02-26 16:50:10 -08:00
44 changed files with 1469 additions and 235 deletions
+316
View File
@@ -18,6 +18,322 @@ under the License.
-->
## Change Log
### 1.1 (Wed Feb 24 12:08:28 2021 +0200)
**Database Migrations**
- [#13111](https://github.com/apache/superset/pull/13111) feat: add Echarts Graph chart (@mayurnewase)
- [#13262](https://github.com/apache/superset/pull/13262) fix: migration script message formatting issue (@ktmud)
- [#13216](https://github.com/apache/superset/pull/13216) fix: add missing columns (@betodealmeida)
- [#12960](https://github.com/apache/superset/pull/12960) fix: Retroactively add granularity param to charts (@etr2460)
- [#13052](https://github.com/apache/superset/pull/13052) fix(explore): pie chart label bugs (@villebro)
- [#13015](https://github.com/apache/superset/pull/13015) fix: time filter db migration optimization (@ktmud)
- [#12950](https://github.com/apache/superset/pull/12950) chore: rename revision to match naming scheme (@etr2460)
- [#12680](https://github.com/apache/superset/pull/12680) feat(dashboard_rbac): dashboard lists (@amitmiran137)
- [#12552](https://github.com/apache/superset/pull/12552) fix(explore): deprecated x periods pattern in new time picker value (@zhaoyongjie)
- [#12336](https://github.com/apache/superset/pull/12336) chore: remove additional incubator references (@amitmiran137)
**Features**
- [#13353](https://github.com/apache/superset/pull/13353) feat: add Firebird DB engine spec (@betodealmeida)
- [#13271](https://github.com/apache/superset/pull/13271) feat(chart-data): add rowcount, timegrain and column result types (@villebro)
- [#13214](https://github.com/apache/superset/pull/13214) feat(presto): add support for user impersonation (@rijojoseph07)
- [#13049](https://github.com/apache/superset/pull/13049) feat: Add sort by for dist bar chart (@maloun96)
- [#13057](https://github.com/apache/superset/pull/13057) feat: Add sort by metric for charts with multiple metrics (@maloun96)
- [#13182](https://github.com/apache/superset/pull/13182) feat: Move SQLAlchemy url reference to config (@hughhhh)
- [#13130](https://github.com/apache/superset/pull/13130) feat(helm): Evaluate configOverrides as templates (@Yann-J)
- [#13205](https://github.com/apache/superset/pull/13205) feat(filter-sets): Saving filter sets in metadata (@simcha90)
- [#13037](https://github.com/apache/superset/pull/13037) feat: add autocopy functionality to CopyToClipboard (#10328) (@michael-s-molina)
- [#13031](https://github.com/apache/superset/pull/13031) feat(filters-set): basic implementation for managing user filter sets (@simcha90)
- [#13150](https://github.com/apache/superset/pull/13150) feat(alerts): apply SQL limit to all alerts (@dpgaspar)
- [#13116](https://github.com/apache/superset/pull/13116) feat(helm): Helm template for Celery beat (for reporting and alerting) (@Yann-J)
- [#13114](https://github.com/apache/superset/pull/13114) feat: Added annotations and loadBalancerIP option in helm service.yaml (@cheunhong)
- [#13148](https://github.com/apache/superset/pull/13148) feat(native-filters): enable filter indicator and make datasource optional (@simcha90)
- [#12978](https://github.com/apache/superset/pull/12978) feat(dashboard): API to get a dashboard's charts (@suddjian)
- [#13105](https://github.com/apache/superset/pull/13105) feat: first step native support Trino (@dungdm93)
- [#13058](https://github.com/apache/superset/pull/13058) feat: add sort_by_metric for charts with single metric (@maloun96)
- [#13035](https://github.com/apache/superset/pull/13035) feat(dashboard_rbac): add support for related roles (@amitmiran137)
- [#13108](https://github.com/apache/superset/pull/13108) feat(native-filters): hide filterBar and toggle icon when in editMode (@amitmiran137)
- [#13096](https://github.com/apache/superset/pull/13096) feat: Custom superset_config.py + secret envs (@Yann-J)
- [#13005](https://github.com/apache/superset/pull/13005) feat(native-filters): add storybook entry for select filter (@simcha90)
- [#12992](https://github.com/apache/superset/pull/12992) feat(native-filters): Time native filter (@simcha90)
- [#13056](https://github.com/apache/superset/pull/13056) feat: Force pod restart on config changes (@Yann-J)
- [#12662](https://github.com/apache/superset/pull/12662) feat(cross-filters): add cross filters (@simcha90)
- [#12918](https://github.com/apache/superset/pull/12918) feat(style): hide dashboard header by url parameter (@simcha90)
- [#12602](https://github.com/apache/superset/pull/12602) feat(db engines): add support for Opendistro Elasticsearch (AWS ES) (@dpgaspar)
- [#12996](https://github.com/apache/superset/pull/12996) feat: Save datapanel state in local storage (@kgabryje)
- [#12865](https://github.com/apache/superset/pull/12865) feat(dashboard_rbac): dashboards API support for roles create/update + roles validation (@amitmiran137)
- [#12782](https://github.com/apache/superset/pull/12782) feat: reset metrics on dataset change (@pkdotson)
- [#12875](https://github.com/apache/superset/pull/12875) feat(dashboard_rbac): dashboard_view access enforcement (@amitmiran137)
- [#12887](https://github.com/apache/superset/pull/12887) feat(time-table): add row limit control for timeseries table (@maloun96)
- [#12909](https://github.com/apache/superset/pull/12909) feat: clear search on dataset change (@pkdotson)
- [#12840](https://github.com/apache/superset/pull/12840) feat: add separate endpoint to fetch function names for autocomplete (@betodealmeida)
- [#12858](https://github.com/apache/superset/pull/12858) feat(style-theme): add support for custom superset themes (@simcha90)
- [#12199](https://github.com/apache/superset/pull/12199) feat(native-filters): Add defaultValue for Native filters modal (@simcha90)
- [#12663](https://github.com/apache/superset/pull/12663) feat: request ids on API related endpoints (@dpgaspar)
- [#12716](https://github.com/apache/superset/pull/12716) feat(native-filters): apply scoping of native filters to dashboard (@simcha90)
- [#12609](https://github.com/apache/superset/pull/12609) feat: default timepicker to last week when dataset is changed (@pkdotson)
- [#12740](https://github.com/apache/superset/pull/12740) feat: Adding option to set_database_uri CLI command (@craig-rueda)
- [#12635](https://github.com/apache/superset/pull/12635) feat: add decorator to guard public APIs (@betodealmeida)
- [#12581](https://github.com/apache/superset/pull/12581) feat(bigquery): implement custom minute time grains (@villebro)
- [#11720](https://github.com/apache/superset/pull/11720) feat(db-engine-specs): add support for Postgres root cert (@villebro)
- [#12415](https://github.com/apache/superset/pull/12415) feat: new Slack dashboard (@betodealmeida)
**Fixes**
- [#13657](https://github.com/apache/superset/pull/13657) fix: 'Run in SQL Lab' redirect bug (@mistercrunch)
- [#13735](https://github.com/apache/superset/pull/13735) fix: Properly escape special characters in CSV output (@benjreinhart)
- [#13461](https://github.com/apache/superset/pull/13461) fix: url shortener invalid input (@dpgaspar)
- [#13157](https://github.com/apache/superset/pull/13157) fix(alerts&reports): Alerts & Reports will use values from WEBDRIVER_WINDOW option (@iercan)
- [#13350](https://github.com/apache/superset/pull/13350) fix(celery): Reset DB connection pools for forked worker processes (@robdiciuccio)
- [#13300](https://github.com/apache/superset/pull/13300) fix: exception when saving dash (@amitmiran137)
- [#13284](https://github.com/apache/superset/pull/13284) fix(listview): preserve bulk selections when view mode is changed (@nytai)
- [#13289](https://github.com/apache/superset/pull/13289) fix(alerts): Handle None on results (@dpgaspar)
- [#12937](https://github.com/apache/superset/pull/12937) fix: chart keys in MultiLineViz (@rwspielman)
- [#13254](https://github.com/apache/superset/pull/13254) fix: style tweaks for inconsistent button spacing (@rusackas)
- [#13291](https://github.com/apache/superset/pull/13291) fix(home): missing key and invalid dates in Recents cards (@ktmud)
- [#12926](https://github.com/apache/superset/pull/12926) fix: add alert report timeout limits (@riahk)
- [#13283](https://github.com/apache/superset/pull/13283) fix(menu): links that result in 404s (@nytai)
- [#13245](https://github.com/apache/superset/pull/13245) fix: bulk select on card view (@lilykuang)
- [#13233](https://github.com/apache/superset/pull/13233) fix(viz): calendar heatmap granularity (@maloun96)
- [#13186](https://github.com/apache/superset/pull/13186) fix: SqlEditor_spec tests (@michael-s-molina)
- [#13222](https://github.com/apache/superset/pull/13222) fix(alert/report): alert modal loading dropdown options (@lilykuang)
- [#13122](https://github.com/apache/superset/pull/13122) fix: TimeTable average with nulls calculations (@pavlozt)
- [#13181](https://github.com/apache/superset/pull/13181) fix(explore): Fix downloading as image charts which use Mapbox (@kgabryje)
- [#13163](https://github.com/apache/superset/pull/13163) fix: Fixing pinot query generation for date format conversion from python datetime format to java simple date format (@fx19880617)
- [#13123](https://github.com/apache/superset/pull/13123) fix: add the untranslated text content of Chart page, Sql Editor page and some dashboard label in messages.pot (@cdmikechen)
- [#13211](https://github.com/apache/superset/pull/13211) fix(explore): Reset values in TextControl only when datasource changes (@kgabryje)
- [#13219](https://github.com/apache/superset/pull/13219) fix: Update display string when sqlalchemy doc config us changed (@hughhhh)
- [#13217](https://github.com/apache/superset/pull/13217) fix(native-filters): uncapitalize filter name (@amitmiran137)
- [#13160](https://github.com/apache/superset/pull/13160) fix: Grab datasets initially onMount (@hughhhh)
- [#13086](https://github.com/apache/superset/pull/13086) fix: accept null groupby in form data for timeseries table viz (@michellethomas)
- [#13173](https://github.com/apache/superset/pull/13173) fix(api): apply dashboard filter to get dash charts API (@dpgaspar)
- [#13178](https://github.com/apache/superset/pull/13178) fix(cross-filters): apply cross filters for all charts (@simcha90)
- [#13141](https://github.com/apache/superset/pull/13141) fix: handle lack of dynamic plugins (@suddjian)
- [#13153](https://github.com/apache/superset/pull/13153) fix: engines that don't support comments (@dpgaspar)
- [#12977](https://github.com/apache/superset/pull/12977) fix(alert/report): bulk select mode should be deactivated (@lilykuang)
- [#13180](https://github.com/apache/superset/pull/13180) fix(sqllab): menu link results in 404 (@nytai)
- [#13164](https://github.com/apache/superset/pull/13164) fix(native-filters): Apply range filter (@simcha90)
- [#13165](https://github.com/apache/superset/pull/13165) fix(native-filters): correct apply of native filter scopes (@simcha90)
- [#13154](https://github.com/apache/superset/pull/13154) fix(explore): Chart panel overflowing (@kgabryje)
- [#13120](https://github.com/apache/superset/pull/13120) fix: misspelling Erik Ritter (@etr2460)
- [#12955](https://github.com/apache/superset/pull/12955) fix(alert/report): add/edit modal issue (@lilykuang)
- [#13133](https://github.com/apache/superset/pull/13133) fix(explore): Certificate icon not displaying for certified metrics (@kgabryje)
- [#13147](https://github.com/apache/superset/pull/13147) fix(database): ensure pk_constraint is JSON serializable (@villebro)
- [#13138](https://github.com/apache/superset/pull/13138) fix(chart-data-api): support numeric temporal columns (@villebro)
- [#13090](https://github.com/apache/superset/pull/13090) fix(alerts): void query with numeric comparison (@dpgaspar)
- [#13109](https://github.com/apache/superset/pull/13109) fix(FilterBox): treat empty defaultValues as null (@ktmud)
- [#13094](https://github.com/apache/superset/pull/13094) fix: Fix switching viz type to and from Filter box (@kgabryje)
- [#13022](https://github.com/apache/superset/pull/13022) fix: Add some missing Chinese translation content, including superset-ui project (@cdmikechen)
- [#13000](https://github.com/apache/superset/pull/13000) fix(native-filters): set currentValue null when empty (@villebro)
- [#13061](https://github.com/apache/superset/pull/13061) fix: Update http error code from 400 to 403 for permission error (@duynguyenhoang)
- [#13029](https://github.com/apache/superset/pull/13029) fix(explore): Enable selecting an option not included in suggestions (@kgabryje)
- [#13076](https://github.com/apache/superset/pull/13076) fix: API tests, make them possible to run independently again (@dpgaspar)
- [#13051](https://github.com/apache/superset/pull/13051) fix: add config to disable dataset ownership on the old api (@dpgaspar)
- [#12931](https://github.com/apache/superset/pull/12931) fix: add required \* indicator to message content/notif method in alert/report modal (@riahk)
- [#13075](https://github.com/apache/superset/pull/13075) fix(ci): multiline regex in change detection (@villebro)
- [#13008](https://github.com/apache/superset/pull/13008) fix: Disabled state button transition time (@AAfghahi)
- [#13059](https://github.com/apache/superset/pull/13059) fix: sorting by saved metric (@villebro)
- [#13041](https://github.com/apache/superset/pull/13041) fix: timeout context manager on Windows (@betodealmeida)
- [#13042](https://github.com/apache/superset/pull/13042) fix: vertical scroll in query history (@betodealmeida)
- [#13026](https://github.com/apache/superset/pull/13026) fix: UI toast typo (@dpgaspar)
- [#13039](https://github.com/apache/superset/pull/13039) fix(build): black failing on master, add to required checks (@robdiciuccio)
- [#13024](https://github.com/apache/superset/pull/13024) fix: fix the untranslated text content of Dashboard page (@cdmikechen)
- [#13034](https://github.com/apache/superset/pull/13034) fix(ci): remove signature requirements for commits to master (@nytai)
- [#12999](https://github.com/apache/superset/pull/12999) fix: add alerts and report to default config (@dpgaspar)
- [#12945](https://github.com/apache/superset/pull/12945) fix: refresh dashboard list after bulk delete (@michael-s-molina)
- [#12923](https://github.com/apache/superset/pull/12923) fix: added text and changed margins (@AAfghahi)
- [#12966](https://github.com/apache/superset/pull/12966) fix: typo to apache/superset (@AAfghahi)
- [#12973](https://github.com/apache/superset/pull/12973) fix: SQL Lab editor height in Safari (@betodealmeida)
- [#12956](https://github.com/apache/superset/pull/12956) fix: SqlLab Run button fix (@AAfghahi)
- [#12953](https://github.com/apache/superset/pull/12953) fix: Detecting top scroll on Dashboard header (@kkucharc)
- [#12933](https://github.com/apache/superset/pull/12933) fix(viz): improve dtype inference logic (@villebro)
- [#12859](https://github.com/apache/superset/pull/12859) fix: Add inclusive and exclusive text for start date and end date (@maloun96)
- [#12930](https://github.com/apache/superset/pull/12930) fix(viz): bump table viz to fix ordering bug (@villebro)
- [#12906](https://github.com/apache/superset/pull/12906) fix: Config for dataset health check (@graceguo-supercat)
- [#12905](https://github.com/apache/superset/pull/12905) fix(chart): allow `null` for optional query object props (@ktmud)
- [#12898](https://github.com/apache/superset/pull/12898) fix: comment typo (@etr2460)
- [#12890](https://github.com/apache/superset/pull/12890) fix(explore): missing select when groupby without metrics (@villebro)
- [#12579](https://github.com/apache/superset/pull/12579) fix: Installing Superset Locally Using Docker Compose (@mephinet)
- [#12841](https://github.com/apache/superset/pull/12841) fix: correct title weights in UPDATING.md (@eschutho)
- [#12836](https://github.com/apache/superset/pull/12836) fix: remove explicit cursor closing for BQ (@betodealmeida)
- [#12625](https://github.com/apache/superset/pull/12625) fix(CI): properly configure cancel duplicates (@ktmud)
- [#12736](https://github.com/apache/superset/pull/12736) fix: Remove double tooltip from metric label (@kgabryje)
- [#12800](https://github.com/apache/superset/pull/12800) fix: rm grace period field for reports modal (@riahk)
- [#12821](https://github.com/apache/superset/pull/12821) fix: don't close cursor before closing connection (@betodealmeida)
- [#12811](https://github.com/apache/superset/pull/12811) fix(reports): handle exceptions properly in scope (@dpgaspar)
- [#12742](https://github.com/apache/superset/pull/12742) fix: Prevent dashboard with filter_values template cause incompatible indicator (@graceguo-supercat)
- [#12219](https://github.com/apache/superset/pull/12219) fix: tests errors and warnings - iteration 3 (#12212) (@michael-s-molina)
- [#12224](https://github.com/apache/superset/pull/12224) fix: tests errors and warnings - iteration 5 (#12212) (@michael-s-molina)
- [#12227](https://github.com/apache/superset/pull/12227) fix: tests errors and warnings - iteration 6 (#12212) (@michael-s-molina)
- [#12223](https://github.com/apache/superset/pull/12223) fix: tests errors and warnings - iteration 4 (#12212) (@michael-s-molina)
- [#12245](https://github.com/apache/superset/pull/12245) fix: tests errors and warnings - iteration 7 (#12212) (@michael-s-molina)
- [#12214](https://github.com/apache/superset/pull/12214) fix: test errors and warnings - iteration 2 (#12212) (@michael-s-molina)
- [#12706](https://github.com/apache/superset/pull/12706) fix: update latest tag action to use set-output (@eschutho)
- [#12658](https://github.com/apache/superset/pull/12658) fix(dashboard): top dropdown indicator in tab moved closer to the bottom edge (@kkucharc)
- [#12519](https://github.com/apache/superset/pull/12519) fix(native-filters): Reset column field for removed dataset (@agatapst)
- [#12524](https://github.com/apache/superset/pull/12524) fix(native-filters): Improve UI for long native filters names (@agatapst)
- [#12687](https://github.com/apache/superset/pull/12687) fix(native-filters): Show incompatible native filters indicator (@agatapst)
- [#12702](https://github.com/apache/superset/pull/12702) fix(load_examples): better fix for load_data (@betodealmeida)
- [#12692](https://github.com/apache/superset/pull/12692) fix: Adding gatsby remove serviceworker (@craig-rueda)
- [#12659](https://github.com/apache/superset/pull/12659) fix(explore): Error message not displayed correctly in Timeseries table chart (@kgabryje)
- [#12627](https://github.com/apache/superset/pull/12627) fix: Stabilize and deprecate legacy alerts module (@robdiciuccio)
- [#12638](https://github.com/apache/superset/pull/12638) fix(CI): remove duplicate hive and presto tests (@ktmud)
- [#12607](https://github.com/apache/superset/pull/12607) fix: save tabs when saving the query bug (@hughhhh)
- [#12240](https://github.com/apache/superset/pull/12240) fix: helm chart to use Bitnami chart (@liamnv)
- [#12567](https://github.com/apache/superset/pull/12567) fix: update helm chart repository (@palindrom615)
- [#12526](https://github.com/apache/superset/pull/12526) fix: ignore system files on import (@betodealmeida)
- [#12182](https://github.com/apache/superset/pull/12182) fix: Add Omnibar Feature Flag (@hughhhh)
- [#12252](https://github.com/apache/superset/pull/12252) fix: Displaying filter indicators (@agatapst)
- [#12360](https://github.com/apache/superset/pull/12360) fix: css templates modal cleanup: keep modal open on error, utilize useEffect (@riahk)
- [#12467](https://github.com/apache/superset/pull/12467) fix(alert/report): remove crontab guru link from add/edit modal (@lilykuang)
- [#12461](https://github.com/apache/superset/pull/12461) fix(releasing): svn dockerfile requirements path (@villebro)
- [#12412](https://github.com/apache/superset/pull/12412) fix: uneven list header padding (#12029) (@michael-s-molina)
- [#12432](https://github.com/apache/superset/pull/12432) fix(explore): fixed table header in chrome (@junlincc)
- [#12433](https://github.com/apache/superset/pull/12433) fix: update savedatasetmodal message (@hughhhh)
- [#12354](https://github.com/apache/superset/pull/12354) fix(api): prefix class names on events actions (@dpgaspar)
- [#12404](https://github.com/apache/superset/pull/12404) fix(explore): Prevent duplicated query by data table (@graceguo-supercat)
**Others**
- [#13777](https://github.com/apache/superset/pull/13777) docs: added release notes for 1.1 (@srinify)
- [#13588](https://github.com/apache/superset/pull/13588) chore: script for changelog (@lilykuang)
- [#13402](https://github.com/apache/superset/pull/13402) docs: add file for users to see status of feature flags (@eschutho)
- [#13431](https://github.com/apache/superset/pull/13431) chore: clean remaining old directed force graph code (@mayurnewase)
- [#12882](https://github.com/apache/superset/pull/12882) chore: bump pyarrow and pandas (@villebro)
- [#13304](https://github.com/apache/superset/pull/13304) test: change data-test-id to data-test in FlashProvider (@yardz)
- [#13171](https://github.com/apache/superset/pull/13171) docs: Connecting to Trino Databases (@dungdm93)
- [#12238](https://github.com/apache/superset/pull/12238) chore: update instructions for Pinot in connecting to databases (@pablo-tech)
- [#12920](https://github.com/apache/superset/pull/12920) refactor: Bootstrap to AntD - Collapse (@michael-s-molina)
- [#13292](https://github.com/apache/superset/pull/13292) test: replace data-test-id with data-test (@ktmud)
- [#13220](https://github.com/apache/superset/pull/13220) test: Switching FlashProvider to function component & creating tests (@yardz)
- [#13235](https://github.com/apache/superset/pull/13235) refactor: Move raise_for_dashboard_access to security manager (@bolkedebruin)
- [#13212](https://github.com/apache/superset/pull/13212) refactor(api): csrf token on the new REST API (@dpgaspar)
- [#12870](https://github.com/apache/superset/pull/12870) test: fix schedules tests (@kstrz)
- [#12864](https://github.com/apache/superset/pull/12864) test: reports/commands_tests (@kstrz)
- [#13241](https://github.com/apache/superset/pull/13241) test: add Tests and Storybook for AsyncAceEditor (@michael-s-molina)
- [#13268](https://github.com/apache/superset/pull/13268) refactor(native-filters): Refactor filters config modal module (@simcha90)
- [#13258](https://github.com/apache/superset/pull/13258) chore(viz): bump superset-ui 0.17.10 (@villebro)
- [#13260](https://github.com/apache/superset/pull/13260) ci: always execute tests on non-PRs (@villebro)
- [#12122](https://github.com/apache/superset/pull/12122) refactor: Bootstrap to AntD - Alert (@michael-s-molina)
- [#13215](https://github.com/apache/superset/pull/13215) chore: bump node.js to v14 (@kgabryje)
- [#13250](https://github.com/apache/superset/pull/13250) refactor: add frontend debounce constants (@ktmud)
- [#13213](https://github.com/apache/superset/pull/13213) test: Tests and Storybook entry for the ModalTrigger Component (@geido)
- [#13176](https://github.com/apache/superset/pull/13176) test: Tests and Storybook entry for the Popover component (@geido)
- [#12773](https://github.com/apache/superset/pull/12773) chore: Tab title to be empty when creating a new tab (@geido)
- [#13002](https://github.com/apache/superset/pull/13002) refactor: Bootstrap to AntD - DropdownButton (@michael-s-molina)
- [#13098](https://github.com/apache/superset/pull/13098) refactor: HighligthedSQL into functional component with Typescript (@AAfghahi)
- [#13143](https://github.com/apache/superset/pull/13143) chore: add non-dev docker-compose workflow (@nytai)
- [#13185](https://github.com/apache/superset/pull/13185) chore: use shillelagh instead of gsheetsdb (@betodealmeida)
- [#13201](https://github.com/apache/superset/pull/13201) docs: Fix typo in pyathena connection string (@ctSkennerton)
- [#13179](https://github.com/apache/superset/pull/13179) test: Adding test and storybook to "Loading" component. (@yardz)
- [#13184](https://github.com/apache/superset/pull/13184) test: creates storybook and tests for Tooltip component (@michael-s-molina)
- [#13136](https://github.com/apache/superset/pull/13136) test(coverage): ignore database migrations on coverage reports (@willbarrett)
- [#13142](https://github.com/apache/superset/pull/13142) refactor: remove deprecated field from frontend usage (@suddjian)
- [#13087](https://github.com/apache/superset/pull/13087) perf: add lazy loading along react-router routes and router links in menu (@nytai)
- [#13139](https://github.com/apache/superset/pull/13139) style(list view): table sort icon alignment (@nytai)
- [#13149](https://github.com/apache/superset/pull/13149) chore: change test environment to jsdom (@michael-s-molina)
- [#13100](https://github.com/apache/superset/pull/13100) chore: upgrade to npm 7 and superset-ui 0.17.9 (@ktmud)
- [#13113](https://github.com/apache/superset/pull/13113) build: add more files to .dockerignore (@ktmud)
- [#13021](https://github.com/apache/superset/pull/13021) refactor(native-filters): decouple params from filter config modal (first phase) (@simcha90)
- [#13079](https://github.com/apache/superset/pull/13079) test(frontend): improves react-testing-library configuration (@michael-s-molina)
- [#13083](https://github.com/apache/superset/pull/13083) docs: Added a note about the ! prefix for breaking changes to CONTRIBUTING.md (@garden-of-delete)
- [#13069](https://github.com/apache/superset/pull/13069) chore: lock down npm to v6 (@ktmud)
- [#13063](https://github.com/apache/superset/pull/13063) style: fix typo in documentation (@dbz10)
- [#12981](https://github.com/apache/superset/pull/12981) docs(changelog): add entries for 1.0.1 (@villebro)
- [#12982](https://github.com/apache/superset/pull/12982) ci: skip cypress if no code changes (@villebro)
- [#12970](https://github.com/apache/superset/pull/12970) chore: add cypress required checks for branch protection (@eschutho)
- [#13014](https://github.com/apache/superset/pull/13014) chore: updates storybook to version 6.1.17 (@michael-s-molina)
- [#13004](https://github.com/apache/superset/pull/13004) chore: Swap Slack Url 2 more places (@srinify)
- [#12889](https://github.com/apache/superset/pull/12889) refactor(native-filters): refactor code of native filters (@simcha90)
- [#12991](https://github.com/apache/superset/pull/12991) docs: typo fix in release notes (@iercan)
- [#12873](https://github.com/apache/superset/pull/12873) refactor: TemplateParamsEditor.jsx converted from class to functional component (@lyndsiWilliams)
- [#12806](https://github.com/apache/superset/pull/12806) refactor: speed up conversion from dataframe to list of records (@TColl)
- [#12980](https://github.com/apache/superset/pull/12980) chore: rename action to docker-build (@eschutho)
- [#12969](https://github.com/apache/superset/pull/12969) chore: add more required checks for branch protection (@eschutho)
- [#12974](https://github.com/apache/superset/pull/12974) chore: swap slack url on ASF docs (@srinify)
- [#12972](https://github.com/apache/superset/pull/12972) chore: cleanup no-op Cypress (chrome) job (@ktmud)
- [#12968](https://github.com/apache/superset/pull/12968) chore: change name of lint action (@eschutho)
- [#12967](https://github.com/apache/superset/pull/12967) chore: remove example token to avoid confusion (@ktmud)
- [#12928](https://github.com/apache/superset/pull/12928) chore: add more required checks (@eschutho)
- [#12583](https://github.com/apache/superset/pull/12583) ci: conditionally skip python and frontend tests (@villebro)
- [#12180](https://github.com/apache/superset/pull/12180) test: Added some Cypress tests for Native Filters (@adam-stasiak)
- [#12936](https://github.com/apache/superset/pull/12936) test: fix table chart sort order E2E tests (@ktmud)
- [#12921](https://github.com/apache/superset/pull/12921) chore: rename duplicate build jobs (@eschutho)
- [#12885](https://github.com/apache/superset/pull/12885) test: speedup celery tests (@kstrz)
- [#12901](https://github.com/apache/superset/pull/12901) build(deps): Bump jinja2 to 2.11.3 (@robdiciuccio)
- [#12848](https://github.com/apache/superset/pull/12848) chore(dashboard_rbac): remove overlapping security tests (@amitmiran137)
- [#12894](https://github.com/apache/superset/pull/12894) chore: remove SIP-38 feature flag (@villebro)
- [#12694](https://github.com/apache/superset/pull/12694) chore: add required review on master (@eschutho)
- [#12738](https://github.com/apache/superset/pull/12738) refactor: Bootstrap to AntD - Radio (@michael-s-molina)
- [#12869](https://github.com/apache/superset/pull/12869) refactor: dbapi exception mapping for dbapi's (@dpgaspar)
- [#12884](https://github.com/apache/superset/pull/12884) chore(lint): fix pre-commit error (@villebro)
- [#12857](https://github.com/apache/superset/pull/12857) refactor(color-schemes): refactor setting of color schemes (@simcha90)
- [#12774](https://github.com/apache/superset/pull/12774) refactor: Bootstrap to AntD - Label (@michael-s-molina)
- [#10771](https://github.com/apache/superset/pull/10771) [Helm] Automate datasource import (@ejianu)
- [#12610](https://github.com/apache/superset/pull/12610) build: Skip loading example data from configs in CI (@robdiciuccio)
- [#12832](https://github.com/apache/superset/pull/12832) refactor: Bootstrap to AntD - Button (@michael-s-molina)
- [#12854](https://github.com/apache/superset/pull/12854) chore: bump superset-ui packages to v0.17.2 and v0.17.3 (@ktmud)
- [#12846](https://github.com/apache/superset/pull/12846) chore: Slight formatting fixes in exploring-data.mdx (@neilsmiller)
- [#12812](https://github.com/apache/superset/pull/12812) chore(native-filters): Connect indicator magnifier with Filter Bar (@agatapst)
- [#12828](https://github.com/apache/superset/pull/12828) chore: update capitalization guideline (@ktmud)
- [#12831](https://github.com/apache/superset/pull/12831) docs: fix config comment (@zsellami)
- [#12778](https://github.com/apache/superset/pull/12778) chore: Added documentation for 1.0 flags (@srinify)
- [#12743](https://github.com/apache/superset/pull/12743) docs: wrong port name in NOTES.txt (@maddyobrienjones)
- [#12816](https://github.com/apache/superset/pull/12816) docs: add Cape Crypto to users list (@leocape)
- [#12833](https://github.com/apache/superset/pull/12833) docs: fix broken link to spark. (@pr-lawrence)
- [#12772](https://github.com/apache/superset/pull/12772) refactor: change Windows new tab shortcut to ctrl + q (@lyndsiWilliams)
- [#10270](https://github.com/apache/superset/pull/10270) refactor: migrate table chart to new API (@ktmud)
- [#12815](https://github.com/apache/superset/pull/12815) chore: Upgrade react-dnd to 11.1.3 (@kgabryje)
- [#12710](https://github.com/apache/superset/pull/12710) docs: Add support for Hologres (@TimothyDing)
- [#12818](https://github.com/apache/superset/pull/12818) docs: add A3Data to users list (@neylsoncrepalde)
- [#12795](https://github.com/apache/superset/pull/12795) refactor: Updated tooltip help text in edit database > SQL Lab settings (@lyndsiWilliams)
- [#12813](https://github.com/apache/superset/pull/12813) chore: adding feature flags v1 documentation in Release Notes (@srinify)
- [#12748](https://github.com/apache/superset/pull/12748) chore(docs): update tox testing run to relevant python env (@amitmiran137)
- [#12754](https://github.com/apache/superset/pull/12754) chore: add breaking change to title regex (@eschutho)
- [#12749](https://github.com/apache/superset/pull/12749) docs: Adding design guidelines to CONTRIBUTING.md (@mihir174)
- [#12792](https://github.com/apache/superset/pull/12792) docs: fix 1.0.0 README images (v2) (@dpgaspar)
- [#12781](https://github.com/apache/superset/pull/12781) docs: 1.0.0 release notes update: media + feature flags (@garden-of-delete)
- [#12777](https://github.com/apache/superset/pull/12777) docs: update changelog for 1.0.0 (@eschutho)
- [#12626](https://github.com/apache/superset/pull/12626) docs: Updates to Superset Site for 1.0 (@srinify)
- [#12655](https://github.com/apache/superset/pull/12655) test(native-filters): scoping tree in native filters modal (@simcha90)
- [#12605](https://github.com/apache/superset/pull/12605) chore: upgrade Cypress to 6.2.1 (@ktmud)
- [#12218](https://github.com/apache/superset/pull/12218) refactor(explore): Enhance Dataset and Control panel Collapse components (@geido)
- [#12739](https://github.com/apache/superset/pull/12739) docs: Fixed typo on line 348 (@lyndsiWilliams)
- [#12693](https://github.com/apache/superset/pull/12693) docs: Remove gatsby-plugin-offline (@robdiciuccio)
- [#12615](https://github.com/apache/superset/pull/12615) test: oracle engine spec (@kstrz)
- [#12520](https://github.com/apache/superset/pull/12520) test: hive db engine spec (@kstrz)
- [#12737](https://github.com/apache/superset/pull/12737) chore: Replace Expired Join Slack Community URL With New One (@srinify)
- [#12718](https://github.com/apache/superset/pull/12718) docs: link fix in CONTRIBUTING.md (@ayanginet)
- [#12709](https://github.com/apache/superset/pull/12709) ci: use git submodules for (securely) using third party Github Actions (@TobKed)
- [#12678](https://github.com/apache/superset/pull/12678) chore: add a note about using Draft PR's importance (@amitmiran137)
- [#12673](https://github.com/apache/superset/pull/12673) docs(db engines): Updade DB2 connection prefix (@cabo40)
- [#12682](https://github.com/apache/superset/pull/12682) docs: updates updating.md with 1.0 (@dpgaspar)
- [#12455](https://github.com/apache/superset/pull/12455) chore: apply capitalization guidelines - iteration 9 (#12343) (@michael-s-molina)
- [#12666](https://github.com/apache/superset/pull/12666) chore: skip CI on draft PR (@amitmiran137)
- [#12594](https://github.com/apache/superset/pull/12594) test: presto engine spec tests (@kstrz)
- [#12539](https://github.com/apache/superset/pull/12539) test(hana): add test_convert_dttm (@kstrz)
- [#12633](https://github.com/apache/superset/pull/12633) docs(release): add some improvements from 1.0.0 release round (@villebro)
- [#12616](https://github.com/apache/superset/pull/12616) test: sqlite db engine spec (@kstrz)
- [#12642](https://github.com/apache/superset/pull/12642) docs: fix redirect from bad find/replace (@mistercrunch)
- [#12637](https://github.com/apache/superset/pull/12637) docs: README.md navigation changes for 1.0.0 (@garden-of-delete)
- [#12617](https://github.com/apache/superset/pull/12617) refactor: rename DbColumnType to GenericDataType (@ktmud)
- [#12171](https://github.com/apache/superset/pull/12171) refactor: Sync Scoping tree with Forms data (@simcha90)
- [#12413](https://github.com/apache/superset/pull/12413) chore(chart): Save modal select placeholder value (@nikolagigic)
- [#12591](https://github.com/apache/superset/pull/12591) docs(release): add vote thread to result template (@villebro)
- [#12570](https://github.com/apache/superset/pull/12570) refactor: extract to method is_user_admin (@amitmiran137)
- [#12571](https://github.com/apache/superset/pull/12571) chore: update contribution guide (@amitmiran137)
- [#12511](https://github.com/apache/superset/pull/12511) docs : release notes 1.0 (@garden-of-delete)
- [#12563](https://github.com/apache/superset/pull/12563) chore: upgrade CI runner to Ubuntu 20.04 (@ktmud)
- [#12534](https://github.com/apache/superset/pull/12534) build(cypress): Add Cypress parallelization with obfuscated record key (@robdiciuccio)
- [#12490](https://github.com/apache/superset/pull/12490) test: /postgres engine spec test coverage (@kstrz)
- [#12548](https://github.com/apache/superset/pull/12548) test: mysql engine spec (@kstrz)
- [#12510](https://github.com/apache/superset/pull/12510) build: fix emotion theme errors when npm link @superset-ui/core (@ktmud)
- [#12523](https://github.com/apache/superset/pull/12523) ci(cypress): disable failing test (@villebro)
- [#12405](https://github.com/apache/superset/pull/12405) chore: remove duplicate DashboardFilter class and change imports accordingly (@amitmiran137)
- [#12420](https://github.com/apache/superset/pull/12420) chore: Add owner field to the alerts list (@bkyryliuk)
- [#12394](https://github.com/apache/superset/pull/12394) build: add cancel-workflow-runs action locally (@ktmud)
- [#12267](https://github.com/apache/superset/pull/12267) docs: fix styles and add algolia search classes (@pkdotson)
- [#12020](https://github.com/apache/superset/pull/12020) build(deps): bump ini from 1.3.5 to 1.3.8 in /docs (@dependabot[bot])
- [#12232](https://github.com/apache/superset/pull/12232) build(deps): bump urijs from 1.19.1 to 1.19.4 in /superset-frontend (@dependabot[bot])
- [#11869](https://github.com/apache/superset/pull/11869) docs: reword and cleanup docs for local dev server (@ktmud)
### 1.0.1 (Tue Feb 2 09:24:40 2021 +0000)
- [#12872](https://github.com/apache/superset/pull/12872) feat(release): add github token to changelog script (#12872) (@dpgaspar)
- [#12886](https://github.com/apache/superset/pull/12886) fix: allow users to reset their passwords (#12886) (@dpgaspar)
+4
View File
@@ -343,6 +343,8 @@ If the PR passes CI tests and does not have any `need:` labels, it is ready for
If an issue/PR has been inactive for >=30 days, it will be closed. If it does not have any status label, add `inactive`.
When creating a PR, if you're aiming to have it included in a specific release, please tag it with the version label. For example, to have a PR considered for inclusion in Superset 1.1 use the label `v1.1`.
## Reporting a Security Vulnerability
Please report security vulnerabilities to private@superset.apache.org.
@@ -570,6 +572,8 @@ export enum FeatureFlag {
those specified under FEATURE_FLAGS in `superset_config.py`. For example, `DEFAULT_FEATURE_FLAGS = { 'FOO': True, 'BAR': False }` in `superset/config.py` and `FEATURE_FLAGS = { 'BAR': True, 'BAZ': True }` in `superset_config.py` will result
in combined feature flags of `{ 'FOO': True, 'BAR': True, 'BAZ': True }`.
The current status of the usability of each flag (stable vs testing, etc) can be found in `RESOURCES/FEATURE_FLAGS.md`.
## Git Hooks
Superset uses Git pre-commit hooks courtesy of [pre-commit](https://pre-commit.com/). To install run the following:
+2 -2
View File
@@ -45,7 +45,7 @@ A modern, enterprise-ready business intelligence web application.
[**Get Involved**](#get-involved) |
[**Contributor Guide**](#contributor-guide) |
[**Resources**](#resources) |
[**Organizations Using Superset**](INTHEWILD.md)
[**Organizations Using Superset**](RESOURCES/INTHEWILD.md)
## Screenshots & Gifs
@@ -140,7 +140,7 @@ Want to add support for your datastore or data engine? Read more [here](https://
* Ask and answer questions on [StackOverflow](https://stackoverflow.com/questions/tagged/apache-superset) using the **apache-superset** tag
* [Join our community's Slack](https://join.slack.com/t/apache-superset/shared_invite/zt-l5f5e0av-fyYu8tlfdqbMdz_sPLwUqQ)
and please read our [Slack Community Guidelines](CODE_OF_CONDUCT.md#slack-community-guidelines)
and please read our [Slack Community Guidelines](https://github.com/apache/superset/blob/master/CODE_OF_CONDUCT.md#slack-community-guidelines)
* [Join our dev@superset.apache.org Mailing list](https://lists.apache.org/list.html?dev@superset.apache.org)
+112 -11
View File
@@ -26,12 +26,14 @@ from typing import Any, Dict, Iterator, List, Optional, Union
import click
try:
from github import BadCredentialsException, Github, PullRequest
from github import BadCredentialsException, Github, PullRequest, Repository
except ModuleNotFoundError:
print("PyGithub is a required package for this script")
exit(1)
SUPERSET_REPO = "apache/superset"
SUPERSET_PULL_REQUEST_TYPES = r"^(fix|feat|chore|refactor|docs|build|ci|/gmi)"
SUPERSET_RISKY_LABELS = r"^(blocking|risk|hold|revert|security vulnerability)"
@dataclass
@@ -65,15 +67,22 @@ class GitChangeLog:
"""
def __init__(
self, version: str, logs: List[GitLog], access_token: Optional[str] = None
self,
version: str,
logs: List[GitLog],
access_token: Optional[str] = None,
risk: Optional[bool] = False,
) -> None:
self._version = version
self._logs = logs
self._pr_logs_with_details: Dict[int, Dict[str, Any]] = {}
self._github_login_cache: Dict[str, Optional[str]] = {}
self._github_prs: Dict[int, Any] = {}
self._wait = 10
github_token = access_token or os.environ.get("GITHUB_TOKEN")
self._github = Github(github_token)
self._superset_repo = ""
self._show_risk = risk
self._superset_repo: Repository = None
def _fetch_github_pr(self, pr_number: int) -> PullRequest:
"""
@@ -81,6 +90,11 @@ class GitChangeLog:
"""
try:
github_repo = self._github.get_repo(SUPERSET_REPO)
self._superset_repo = github_repo
pull_request = self._github_prs.get(pr_number)
if not pull_request:
pull_request = github_repo.get_pull(pr_number)
self._github_prs[pr_number] = pull_request
except BadCredentialsException as ex:
print(
f"Bad credentials to github provided"
@@ -88,7 +102,7 @@ class GitChangeLog:
)
sys.exit(1)
return github_repo.get_pull(pr_number)
return pull_request
def _get_github_login(self, git_log: GitLog) -> Optional[str]:
"""
@@ -108,21 +122,102 @@ class GitChangeLog:
self._github_login_cache[author_name] = github_login
return github_login
def _has_commit_migrations(self, git_sha: str) -> bool:
commit = self._superset_repo.get_commit(sha=git_sha)
return any(
"superset/migrations/versions/" in file.filename for file in commit.files
)
def _get_pull_request_details(self, git_log: GitLog) -> Dict[str, Any]:
pr_number = git_log.pr_number
if pr_number:
detail = self._pr_logs_with_details.get(pr_number)
if detail:
return detail
pr_info = self._fetch_github_pr(pr_number)
has_migrations = self._has_commit_migrations(git_log.sha)
title = pr_info.title if pr_info else git_log.message
pr_type = re.match(SUPERSET_PULL_REQUEST_TYPES, title)
if pr_type:
pr_type = pr_type.group().strip('"')
labels = (" | ").join([label.name for label in pr_info.labels])
is_risky = self._is_risk_pull_request(pr_info.labels)
detail = {
"id": pr_number,
"has_migrations": has_migrations,
"labels": labels,
"title": title,
"type": pr_type,
"is_risky": is_risky or has_migrations,
}
if pr_number:
self._pr_logs_with_details[pr_number] = detail
return detail
def _is_risk_pull_request(self, labels: List[Any]) -> bool:
for label in labels:
risk_label = re.match(SUPERSET_RISKY_LABELS, label.name)
if risk_label is not None:
return True
return False
def _get_changelog_version_head(self) -> str:
return f"### {self._version} ({self._logs[0].time})"
def _parse_change_log(
self, changelog: Dict[str, str], pr_info: Dict[str, str], github_login: str,
):
formatted_pr = (
f"- [#{pr_info.get('id')}]"
f"(https://github.com/{SUPERSET_REPO}/pull/{pr_info.get('id')}) "
f"{pr_info.get('title')} (@{github_login})\n"
)
if pr_info.get("has_migrations"):
changelog["Database Migrations"] += formatted_pr
elif pr_info.get("type") == "fix":
changelog["Fixes"] += formatted_pr
elif pr_info.get("type") == "feat":
changelog["Features"] += formatted_pr
else:
changelog["Others"] += formatted_pr
def __repr__(self) -> str:
result = f"\n{self._get_changelog_version_head()}\n"
changelog = {
"Database Migrations": "\n",
"Features": "\n",
"Fixes": "\n",
"Others": "\n",
}
for i, log in enumerate(self._logs):
github_login = self._get_github_login(log)
pr_info = self._get_pull_request_details(log)
if not github_login:
github_login = log.author
result = result + (
f"- [#{log.pr_number}]"
f"(https://github.com/{SUPERSET_REPO}/pull/{log.pr_number}) "
f"{log.message} (@{github_login})\n"
)
if self._show_risk:
if pr_info.get("is_risky"):
result += (
f"- [#{log.pr_number}]"
f"(https://github.com/{SUPERSET_REPO}/pull/{log.pr_number}) "
f"{pr_info.get('title')} (@{github_login}) "
f"{pr_info.get('labels')} \n"
)
else:
self._parse_change_log(changelog, pr_info, github_login)
print(f"\r {i}/{len(self._logs)}", end="", flush=True)
if self._show_risk:
return result
for key in changelog:
result += f"**{key}** {changelog[key]}\n"
return result
def __iter__(self) -> Iterator[Dict[str, Any]]:
@@ -269,14 +364,20 @@ def compare(base_parameters: BaseParameters) -> None:
help="The github access token,"
" if not provided will try to fetch from GITHUB_TOKEN env var",
)
@click.option("--risk", is_flag=True, help="show all pull requests with risky labels")
@click.pass_obj
def change_log(base_parameters: BaseParameters, csv: str, access_token: str) -> None:
def change_log(
base_parameters: BaseParameters, csv: str, access_token: str, risk: bool
) -> None:
""" Outputs a changelog (by PR) """
previous_logs = base_parameters.previous_logs
current_logs = base_parameters.current_logs
previous_diff_logs = previous_logs.diff(current_logs)
logs = GitChangeLog(
current_logs.git_ref, previous_diff_logs[::-1], access_token=access_token
current_logs.git_ref,
previous_diff_logs[::-1],
access_token=access_token,
risk=risk,
)
if csv:
with open(csv, "w") as csv_file:
+114
View File
@@ -0,0 +1,114 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Release Notes for Superset 1.1.0
Superset 1.1.0 continues to build on the [1.0](https://github.com/apache/superset/blob/master/RELEASING/release-notes-1-0/README.md) release with big improvements to user experience, security, dashboard level access, and database connectivity.
- [**User Experience**](#user-experience)
- [**Dashboard Level Security**](#dashboard-level-security)
- [**Database Connectivity**](#database-connectivity)
- [**PR Highlights**](#pr-highlights)
- [**Breaking Changes and Full Changelog**](#breaking-changes-and-full-changelog)
# User Experience
In general, the quality, usability, and aesthetics of the Superset user experience continues to develop in the direction of [SIP-34](https://github.com/apache/superset/issues/8976) and more tests have been added to ui components to ensure usability is maintained.
The migration to Apache ECharts continues with the addition of a [new force-directed graph](https://github.com/apache/superset/pull/13111).
![Force Directed Graph](media/force_directed_graph.jpg)
The ECharts library included with Superset was also bumped to 5.0.2, which includes some [fixes to pie charts.](https://github.com/apache/superset/pull/13052) The overall improvement of chart options and quality can be expected to continue in future releases. New 'sort by' controls have also been [added](https://github.com/apache/superset/pull/13049) to [many charts](https://github.com/apache/superset/pull/13057).
The dashboard native filter feature, [while still behind a feature flag in this release,](https://github.com/apache/superset/blob/master/RELEASING/release-notes-1-0/README.md#feature-flags) has been improved and is ready to test out.
![Native Filter](media/native_filters.jpg)
Since the 1.0 release, we have seen a surge of support from the community around updating Superset's [documentation](https://superset.apache.org/docs/intro) and adding more tests to the UI. Thanks to all who contributed in this area. This is what open-source software is all about!
# Dashboard Level Security
Dashboard providers in an organization with many subgroups need the ability manage user access to dashboards and different levels of permissions (read, write, granter, owner). Improving dashboard level access was proposed (and approved) in [SIP-51](https://github.com/apache/superset/issues/10408).
![SIP 51](media/sip_51.jpg)
In 1.1, some key steps were taken towards the vision laid out in SIP-51. **Note that this functionality is still hidden behind feature flags and is an active area of development.** You can view a list of relevant PR's in [PR Highlights](#pr-highlights)
# Database Connectivity
Superset is only as good as the databases it can query. This release saw the expanded support of existing databases and foundational support for new databases.
- [Opendistro](https://github.com/apache/superset/pull/12602)
- [Trino](https://github.com/apache/superset/pull/13105)
- [CrateDB](https://github.com/apache/superset/pull/13152/files)
- [Apache Pinot](https://github.com/apache/superset/pull/13163)
- [Presto](https://github.com/apache/superset/pull/13214)
- [BigQuery](https://github.com/apache/superset/pull/12581)
- [Postgres](https://github.com/apache/superset/pull/11720)
- [Google Sheets](https://github.com/apache/superset/pull/13185)
- [Athena](https://github.com/apache/superset/pull/13201)
# PR Highlights
**Progress On Dashboard Native Filters**
- feat(chart-data): add rowcount, timegrain and column result types (#[13271](https://github.com/apache/superset/pull/13271))
- feat(native-filters): enable filter indicator and make datasource optional (#[13148](https://github.com/apache/superset/pull/13148))
- feat(native-filters): hide filterBar and toggle icon when in editMode (#[13108](https://github.com/apache/superset/pull/13108))
- feat(native-filters): add storybook entry for select filter (#[13005](https://github.com/apache/superset/pull/13005))
- feat(native-filters): Time native filter (#[12992](https://github.com/apache/superset/pull/12992))
- feat(native-filters): Add defaultValue for Native filters modal (#[12199](https://github.com/apache/superset/pull/12199))
- feat(native-filters): apply scoping of native filters to dashboard (#[12716](https://github.com/apache/superset/pull/12716))
**Progress On Dashboard Level Access**
- feat(dashboard-rbac): dashboard lists (#[12680](https://github.com/apache/superset/pull/12680))
- feat(dashboard-rbac): add support for related roles (#[13035](https://github.com/apache/superset/pull/13035))
- feat(dashboard-rbac): dashboards API support for roles create/update + roles validation (#[12865](https://github.com/apache/superset/pull/12865))
- feat(dashboard-rbac): `dashboard_view` access enforcement (#[12875](https://github.com/apache/superset/pull/12875))
**Improvements to Explore**
- feat: Add sort by metric for charts with multiple metrics (#[13057](https://github.com/apache/superset/pull/13057))
- feat: add sort_by_metric for charts with single metric (#[13058](https://github.com/apache/superset/pull/13058))
- feat: Add sort by for dist bar chart (#[13049](https://github.com/apache/superset/pull/13049))
- feat: reset metrics on dataset change (#[12782](https://github.com/apache/superset/pull/12782))
- feat: clear search on dataset change (#[12909](https://github.com/apache/superset/pull/12909))
- feat: default timepicker to last week when dataset is changed (#[12609](https://github.com/apache/superset/pull/12609))
**Improvements to Developer Experience**
- chore: add non-dev docker-compose workflow (#[13143](https://github.com/apache/superset/pull/13143))
- feat(style-theme): add support for custom superset themes (#[12858](https://github.com/apache/superset/pull/12858))
- feat: Move SQLAlchemy url reference to config (#[13182](https://github.com/apache/superset/pull/13182))
- feat(helm): Evaluate configOverrides as templates (#[13130](https://github.com/apache/superset/pull/13130))
- feat(helm): Helm template for Celery beat (for reporting and alerting) (#[13116](https://github.com/apache/superset/pull/13116))
- feat: Custom superset_config.py + secret envs (#[13096](https://github.com/apache/superset/pull/13096))
- feat: Force pod restart on config changes (#[13056](https://github.com/apache/superset/pull/13056))
- feat: add separate endpoint to fetch function names for autocomplete (#[12840](https://github.com/apache/superset/pull/12840))
- feat: request ids on API related endpoints (#[12663](https://github.com/apache/superset/pull/12663))
- feat: Adding option to `set_database_uri` CLI command (#[12740](https://github.com/apache/superset/pull/12740))
- feat: add decorator to guard public APIs (#[12635](https://github.com/apache/superset/pull/12635))
## Breaking Changes and Full Changelog
- To see the complete changelog in this release, head to [CHANGELOG.MD](https://github.com/apache/superset/blob/master/CHANGELOG.md).
- You can find a list of backwards incompatible changes [here](https://github.com/apache/superset/blob/3d103e66fcaee42a6b4a42b2638e13d5e2208c3b/UPDATING.md).
Binary file not shown.

After

Width:  |  Height:  |  Size: 329 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 231 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 312 KiB

+65
View File
@@ -0,0 +1,65 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Superset Feature Flags
This is a list of the current Superset optional features. See config.py for default values. These features can be turned on/off by setting your preferred values in superset_config.py to True/False respectively
## In Development
These features are considered **unfinished** and should only be used on development environments.
- CLIENT_CACHE
- DASHBOARD_CACHE
- DASHBOARD_NATIVE_FILTERS_SET
- DASHBOARD_RBAC
- DISABLE_DATASET_SOURCE_EDIT
- ENABLE_EXPLORE_JSON_CSRF_PROTECTION
- KV_STORE
- PRESTO_EXPAND_DATA
- REMOVE_SLICE_LEVEL_LABEL_COLORS
- SHARE_QUERIES_VIA_KV_STORE
- TAGGING_SYSTEM
## In Testing
These features are **finished** but currently being tested. They are usable, but may still contain some bugs.
- ALERT_REPORTS: [(docs)](https://superset.apache.org/docs/installation/alerts-reports)
- DYNAMIC_PLUGINS: [(docs)](https://superset.apache.org/docs/installation/running-on-kubernetes)
- DASHBOARD_NATIVE_FILTERS
- GLOBAL_ASYNC_QUERIES [(docs)](https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries)
- OMNIBAR
- VERSIONED_EXPORT
## Stable
These features flags are **safe for production** and have been tested.
- DASHBOARD_CROSS_FILTERS
- ESCAPE_MARKDOWN_HTML
- ENABLE_TEMPLATE_PROCESSING
- LISTVIEWS_DEFAULT_CARD_VIEW
- ROW_LEVEL_SECURITY
- SCHEDULED_QUERIES [(docs)](https://superset.apache.org/docs/installation/alerts-reports)
- SQL_VALIDATORS_BY_ENGINE [(docs)](https://superset.apache.org/docs/installation/sql-templating)
- SQLLAB_BACKEND_PERSISTENCE
- THUMBNAILS [(docs)](https://superset.apache.org/docs/installation/cache)
## Deprecated Flags
These features flags currently default to True and **will be removed in a future major release**. For this current release you can turn them off by setting your config to False, but it is advised to remove or set these flags in your local configuration to **True** so that you do not experience any unexpected changes in a future release.
- ALLOW_DASHBOARD_DOMAIN_SHARDING
- DISPLAY_MARKDOWN_HTML
- ENABLE_REACT_CRUD_VIEWS
+23 -1
View File
@@ -28,7 +28,29 @@ assists people when migrating to a new version.
### Deprecations
### Other
[shillelagh](https://github.com/betodealmeida/shillelagh/) is now the recommended module to connect Superset to Google Spreadsheets, since it's more robust and has extensive test coverage. You should uninstall the `gsheetsdb` module and install the `shillelagh` module in its place. Shillelagh is a drop-in replacement, so no modifications are needed to be done on existing queries, datasets or charts.
## 1.1.0
### Breaking Changes
- This is the first release since we adopted semantic versioning ([SIP-57](https://github.com/apache/superset/issues/12566)). There are no breaking changes in 1.1.0 since this is a minor release.
### Potential Downtime
- [13111](https://github.com/apache/superset/pull/13111) has a database migration that replaces `directed_force` charts with newer `graph_chart` charts based on Apache ECharts.
- [13216](https://github.com/apache/superset/pull/13216) adds a UUID column to models that are missing it. The original migration script that added the column would incorrectly complete when the column couldn't be added, resulting in a broken schema. The script is optimized for MySQL and Postgres, so depending on the database and the number of objects this migration might take considerable time.
- [12960](https://github.com/apache/superset/pull/12960) populates the granularity parameter in existing charts. Depending on the number of charts without a `granularity` or `granularity_sqla param` this might take considerable time.
- [13052](https://github.com/apache/superset/pull/13052) updates the label in existing pie charts, setting `label_type` from `pie_label_type`. Depending on the number of pie charts this might take considerable time.
- [12680](https://github.com/apache/superset/pull/12680) creates a new table, `dashboard_roles`, for role based dashboard level access.
- [12552](https://github.com/apache/superset/pull/12552) updates charts that have the time range defined using "until" and "since". Depending on the number of charts this might take considerable time.
### Deprecations
- [12552](https://github.com/apache/superset/pull/12552) removes the use of unclear time offsets, eg, "30 days". An error message is displayed if the user doesn't specify "ago" or "later", instructing the user of the correct format.
- [12627](https://github.com/apache/superset/pull/12627) deprecates the legacy alerts module.
### Other
- [shillelagh](https://github.com/betodealmeida/shillelagh/) is now the recommended module to connect Superset to Google Spreadsheets since it's more robust and has extensive test coverage. You should uninstall the `gsheetsdb` module and install the `shillelagh` module in its place. Shillelagh is a drop-in replacement, so no modifications are needed to be done on existing queries, datasets, or charts.
## 1.0.0
+1 -11
View File
@@ -1575,17 +1575,7 @@ You can enable or disable features with flag from ``superset_config.py``:
'PRESTO_EXPAND_DATA': False,
}
Here is a list of flags and descriptions:
* ENABLE_EXPLORE_JSON_CSRF_PROTECTION
* For some security concerns, you may need to enforce CSRF protection on all query request to explore_json endpoint. In Superset, we use `flask-csrf <https://sjl.bitbucket.io/flask-csrf/>`_ add csrf protection for all POST requests, but this protection doesn't apply to GET method.
* When ENABLE_EXPLORE_JSON_CSRF_PROTECTION is set to true, your users cannot make GET request to explore_json. The default value for this feature False (current behavior), explore_json accepts both GET and POST request. See `PR 7935 <https://github.com/apache/superset/pull/7935>`_ for more details.
* PRESTO_EXPAND_DATA
* When this feature is enabled, nested types in Presto will be expanded into extra columns and/or arrays. This is experimental, and doesn't work with all nested types.
A current list of feature flags can be found in `RESOURCES/FEATURE_FLAGS.md`
SIP-15
+1 -1
View File
@@ -51,7 +51,7 @@ const links = [
'join our monthly virtual meetups and register for any upcoming events',
],
[
'https://github.com/apache/superset/blob/master/INTHEWILD.md',
'https://github.com/apache/superset/blob/master/RESOURCES/INTHEWILD.md',
'Organizations',
'a list of some of the organizations using Superset in production',
],
@@ -192,17 +192,7 @@ DEFAULT_FEATURE_FLAGS = {
}
```
Here is a list of flags and descriptions:
- `ENABLE_EXPLORE_JSON_CSRF_PROTECTION`: For some security concerns, you may need to enforce CSRF
protection on all query request to the `explore_json` endpoint. When
`ENABLE_EXPLORE_JSON_CSRF_PROTECTION` is set to true, your users cannot make GET request to
`explore_json`. The default value for this feature is `False` and `explore_json` will accept both
GET and POST request. See [PR 7935](https://github.com/apache/superset/pull/7935) for
more details.
- `PRESTO_EXPAND_DATA`: When this feature is enabled, nested types in Presto will be expanded into
extra columns and / or arrays. This is experimental, and doesnt work with all nested types.
A current list of feature flags can be found in `RESOURCES/FEATURE_FLAGS.md`
### SIP 15
+1 -1
View File
@@ -56,4 +56,4 @@ production environment inside Kubernetes and serves 600+ daily active users view
a day.
You can find a partial list of industries and companies embracing Superset
[on this page in GitHub](https://github.com/apache/superset/blob/master/INTHEWILD.md).
[on this page in GitHub](https://github.com/apache/superset/blob/master/RESOURCES/INTHEWILD.md).
+295 -101
View File
@@ -5,107 +5,301 @@
#
# pip-compile-multi
#
-e file:. # via -r requirements/base.in
aiohttp==3.7.2 # via slackclient
alembic==1.4.3 # via flask-migrate
amqp==2.6.1 # via kombu
apispec[yaml]==3.3.2 # via flask-appbuilder
async-timeout==3.0.1 # via aiohttp
attrs==20.2.0 # via aiohttp, jsonschema
babel==2.8.0 # via flask-babel
backoff==1.10.0 # via apache-superset
billiard==3.6.3.0 # via celery
bleach==3.2.1 # via apache-superset
brotli==1.0.9 # via flask-compress
cachelib==0.1.1 # via apache-superset
celery==4.4.7 # via apache-superset
cffi==1.14.3 # via cryptography
chardet==3.0.4 # via aiohttp
click==7.1.2 # via apache-superset, flask, flask-appbuilder
colorama==0.4.4 # via apache-superset, flask-appbuilder
contextlib2==0.6.0.post1 # via apache-superset
convertdate==2.3.0 # via holidays
cron-descriptor==1.2.24 # via apache-superset
croniter==0.3.36 # via apache-superset
cryptography==3.2.1 # via apache-superset
decorator==4.4.2 # via retry
defusedxml==0.6.0 # via python3-openid
dnspython==2.0.0 # via email-validator
email-validator==1.1.1 # via flask-appbuilder
flask-appbuilder==3.1.1 # via apache-superset
flask-babel==1.0.0 # via flask-appbuilder
flask-caching==1.9.0 # via apache-superset
flask-compress==1.8.0 # via apache-superset
flask-jwt-extended==3.24.1 # via flask-appbuilder
flask-login==0.4.1 # via flask-appbuilder
flask-migrate==2.5.3 # via apache-superset
flask-openid==1.2.5 # via flask-appbuilder
flask-sqlalchemy==2.4.4 # via flask-appbuilder, flask-migrate
flask-talisman==0.7.0 # via apache-superset
flask-wtf==0.14.3 # via apache-superset, flask-appbuilder
flask==1.1.2 # via apache-superset, flask-appbuilder, flask-babel, flask-caching, flask-compress, flask-jwt-extended, flask-login, flask-migrate, flask-openid, flask-sqlalchemy, flask-wtf
geographiclib==1.50 # via geopy
geopy==2.0.0 # via apache-superset
gunicorn==20.0.4 # via apache-superset
holidays==0.10.3 # via apache-superset
humanize==3.1.0 # via apache-superset
idna==2.10 # via email-validator, yarl
importlib-metadata==2.1.1 # via -r requirements/base.in, jsonschema, kombu, markdown
isodate==0.6.0 # via apache-superset
itsdangerous==1.1.0 # via flask, flask-wtf
jinja2==2.11.3 # via flask, flask-babel
jsonschema==3.2.0 # via flask-appbuilder
kombu==4.6.11 # via celery
korean-lunar-calendar==0.2.1 # via holidays
mako==1.1.3 # via alembic
markdown==3.3.3 # via apache-superset
markupsafe==1.1.1 # via jinja2, mako, wtforms
marshmallow-enum==1.5.1 # via flask-appbuilder
marshmallow-sqlalchemy==0.23.1 # via flask-appbuilder
marshmallow==3.9.0 # via flask-appbuilder, marshmallow-enum, marshmallow-sqlalchemy
msgpack==1.0.0 # via apache-superset
multidict==5.0.0 # via aiohttp, yarl
natsort==7.0.1 # via croniter
numpy==1.19.4 # via pandas, pyarrow
packaging==20.4 # via bleach
pandas==1.1.4 # via apache-superset
parsedatetime==2.6 # via apache-superset
pathlib2==2.3.5 # via apache-superset
pgsanity==0.2.9 # via apache-superset
polyline==1.4.0 # via apache-superset
prison==0.1.3 # via flask-appbuilder
py==1.9.0 # via retry
pyarrow==1.0.1 # via apache-superset
pycparser==2.20 # via cffi
pyjwt==1.7.1 # via apache-superset, flask-appbuilder, flask-jwt-extended
pymeeus==0.3.7 # via convertdate
pyparsing==2.4.7 # via apache-superset, packaging
pyrsistent==0.16.1 # via -r requirements/base.in, jsonschema
python-dateutil==2.8.1 # via alembic, apache-superset, croniter, flask-appbuilder, holidays, pandas
python-dotenv==0.15.0 # via apache-superset
python-editor==1.0.4 # via alembic
python-geohash==0.8.5 # via apache-superset
python3-openid==3.2.0 # via flask-openid
pytz==2020.4 # via babel, celery, convertdate, flask-babel, pandas
pyyaml==5.3.1 # via apache-superset, apispec
redis==3.5.3 # via apache-superset
retry==0.9.2 # via apache-superset
selenium==3.141.0 # via apache-superset
simplejson==3.17.2 # via apache-superset
six==1.15.0 # via bleach, cryptography, flask-jwt-extended, flask-talisman, holidays, isodate, jsonschema, packaging, pathlib2, polyline, prison, pyrsistent, python-dateutil, sqlalchemy-utils, wtforms-json
slackclient==2.5.0 # via apache-superset
sqlalchemy-utils==0.36.8 # via apache-superset, flask-appbuilder
sqlalchemy==1.3.20 # via alembic, apache-superset, flask-sqlalchemy, marshmallow-sqlalchemy, sqlalchemy-utils
sqlparse==0.3.0 # via apache-superset
typing-extensions==3.7.4.3 # via aiohttp, apache-superset, yarl
urllib3==1.25.11 # via selenium
vine==1.3.0 # via amqp, celery
webencodings==0.5.1 # via bleach
werkzeug==1.0.1 # via flask, flask-jwt-extended
wtforms-json==0.3.3 # via apache-superset
wtforms==2.3.3 # via flask-wtf, wtforms-json
yarl==1.6.2 # via aiohttp
zipp==3.4.0 # via importlib-metadata
-e file:.
# via -r requirements/base.in
aiohttp==3.7.2
# via slackclient
alembic==1.4.3
# via flask-migrate
amqp==2.6.1
# via kombu
apispec[yaml]==3.3.2
# via flask-appbuilder
async-timeout==3.0.1
# via aiohttp
attrs==20.2.0
# via
# aiohttp
# jsonschema
babel==2.8.0
# via flask-babel
backoff==1.10.0
# via apache-superset
billiard==3.6.3.0
# via celery
bleach==3.2.1
# via apache-superset
brotli==1.0.9
# via flask-compress
cachelib==0.1.1
# via apache-superset
celery==4.4.7
# via apache-superset
cffi==1.14.3
# via cryptography
chardet==3.0.4
# via aiohttp
click==7.1.2
# via
# apache-superset
# flask
# flask-appbuilder
colorama==0.4.4
# via
# apache-superset
# flask-appbuilder
contextlib2==0.6.0.post1
# via apache-superset
convertdate==2.3.0
# via holidays
cron-descriptor==1.2.24
# via apache-superset
croniter==0.3.36
# via apache-superset
cryptography==3.2.1
# via apache-superset
decorator==4.4.2
# via retry
defusedxml==0.6.0
# via python3-openid
dnspython==2.0.0
# via email-validator
email-validator==1.1.1
# via flask-appbuilder
flask-appbuilder==3.1.1
# via apache-superset
flask-babel==1.0.0
# via flask-appbuilder
flask-caching==1.9.0
# via apache-superset
flask-compress==1.8.0
# via apache-superset
flask-jwt-extended==3.24.1
# via flask-appbuilder
flask-login==0.4.1
# via flask-appbuilder
flask-migrate==2.5.3
# via apache-superset
flask-openid==1.2.5
# via flask-appbuilder
flask-sqlalchemy==2.4.4
# via
# flask-appbuilder
# flask-migrate
flask-talisman==0.7.0
# via apache-superset
flask-wtf==0.14.3
# via
# apache-superset
# flask-appbuilder
flask==1.1.2
# via
# apache-superset
# flask-appbuilder
# flask-babel
# flask-caching
# flask-compress
# flask-jwt-extended
# flask-login
# flask-migrate
# flask-openid
# flask-sqlalchemy
# flask-wtf
geographiclib==1.50
# via geopy
geopy==2.0.0
# via apache-superset
gunicorn==20.0.4
# via apache-superset
holidays==0.10.3
# via apache-superset
humanize==3.1.0
# via apache-superset
idna==2.10
# via
# email-validator
# yarl
importlib-metadata==2.1.1
# via
# -r requirements/base.in
# jsonschema
# kombu
# markdown
isodate==0.6.0
# via apache-superset
itsdangerous==1.1.0
# via
# flask
# flask-wtf
jinja2==2.11.3
# via
# flask
# flask-babel
jsonschema==3.2.0
# via flask-appbuilder
kombu==4.6.11
# via celery
korean-lunar-calendar==0.2.1
# via holidays
mako==1.1.3
# via alembic
markdown==3.3.3
# via apache-superset
markupsafe==1.1.1
# via
# jinja2
# mako
# wtforms
marshmallow-enum==1.5.1
# via flask-appbuilder
marshmallow-sqlalchemy==0.23.1
# via flask-appbuilder
marshmallow==3.9.0
# via
# flask-appbuilder
# marshmallow-enum
# marshmallow-sqlalchemy
msgpack==1.0.0
# via apache-superset
multidict==5.0.0
# via
# aiohttp
# yarl
natsort==7.0.1
# via croniter
numpy==1.19.4
# via
# pandas
# pyarrow
packaging==20.4
# via bleach
pandas==1.2.2
# via apache-superset
parsedatetime==2.6
# via apache-superset
pathlib2==2.3.5
# via apache-superset
pgsanity==0.2.9
# via apache-superset
polyline==1.4.0
# via apache-superset
prison==0.1.3
# via flask-appbuilder
py==1.9.0
# via retry
pyarrow==3.0.0
# via apache-superset
pycparser==2.20
# via cffi
pyjwt==1.7.1
# via
# apache-superset
# flask-appbuilder
# flask-jwt-extended
pymeeus==0.3.7
# via convertdate
pyparsing==2.4.7
# via
# apache-superset
# packaging
pyrsistent==0.16.1
# via
# -r requirements/base.in
# jsonschema
python-dateutil==2.8.1
# via
# alembic
# apache-superset
# croniter
# flask-appbuilder
# holidays
# pandas
python-dotenv==0.15.0
# via apache-superset
python-editor==1.0.4
# via alembic
python-geohash==0.8.5
# via apache-superset
python3-openid==3.2.0
# via flask-openid
pytz==2020.4
# via
# babel
# celery
# convertdate
# flask-babel
# pandas
pyyaml==5.3.1
# via
# apache-superset
# apispec
redis==3.5.3
# via apache-superset
retry==0.9.2
# via apache-superset
selenium==3.141.0
# via apache-superset
simplejson==3.17.2
# via apache-superset
six==1.15.0
# via
# bleach
# cryptography
# flask-jwt-extended
# flask-talisman
# holidays
# isodate
# jsonschema
# packaging
# pathlib2
# polyline
# prison
# pyrsistent
# python-dateutil
# sqlalchemy-utils
# wtforms-json
slackclient==2.5.0
# via apache-superset
sqlalchemy-utils==0.36.8
# via
# apache-superset
# flask-appbuilder
sqlalchemy==1.3.20
# via
# alembic
# apache-superset
# flask-sqlalchemy
# marshmallow-sqlalchemy
# sqlalchemy-utils
sqlparse==0.3.0
# via apache-superset
typing-extensions==3.7.4.3
# via
# aiohttp
# apache-superset
# yarl
urllib3==1.25.11
# via selenium
vine==1.3.0
# via
# amqp
# celery
webencodings==0.5.1
# via bleach
werkzeug==1.0.1
# via
# flask
# flask-jwt-extended
wtforms-json==0.3.3
# via apache-superset
wtforms==2.3.3
# via
# flask-wtf
# wtforms-json
yarl==1.6.2
# via aiohttp
zipp==3.4.0
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
# setuptools
+3 -2
View File
@@ -87,7 +87,7 @@ setup(
"isodate",
"markdown>=3.0",
"msgpack>=1.0.0, <1.1",
"pandas>=1.1.2, <1.2",
"pandas>=1.2.2, <1.3",
"parsedatetime",
"pathlib2",
"pgsanity",
@@ -95,7 +95,7 @@ setup(
"python-dateutil",
"python-dotenv",
"python-geohash",
"pyarrow>=1.0.1, <1.1",
"pyarrow>=3.0.0, <3.1",
"pyyaml>=5.1",
"PyJWT>=1.7.1, <2",
"redis",
@@ -129,6 +129,7 @@ setup(
"elasticsearch": ["elasticsearch-dbapi>=0.2.0, <0.3.0"],
"exasol": ["sqlalchemy-exasol>=2.1.0, <2.2"],
"excel": ["xlrd>=1.2.0, <1.3"],
"firebird": ["sqlalchemy-firebird>=0.7.0, <0.8"],
"gsheets": ["shillelagh>=0.2, <0.3"],
"hana": ["hdbcli==2.4.162", "sqlalchemy_hana==0.4.0"],
"hive": ["pyhive[hive]>=0.6.1", "tableschema", "thrift>=0.11.0, <1.0.0"],
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "superset",
"version": "0.999.0dev",
"version": "1.1.0",
"description": "Superset is a data exploration platform designed to be visual, intuitive, and interactive.",
"license": "Apache-2.0",
"directories": {
+1 -1
View File
@@ -494,7 +494,7 @@ export function redirectSQLLab(formData) {
return dispatch => {
getChartDataRequest({ formData, resultFormat: 'json', resultType: 'query' })
.then(({ result }) => {
const redirectUrl = '/superset/sqllab';
const redirectUrl = '/superset/sqllab/';
const payload = {
datasourceKey: formData.datasource,
sql: result[0].query,
+3 -1
View File
@@ -38,7 +38,9 @@ const initState = getInitialState(bootstrapData);
const asyncEventMiddleware = initAsyncEvents({
config: bootstrapData.common.conf,
getPendingComponents: ({ charts }) =>
Object.values(charts).filter(c => c.chartStatus === 'loading'),
Object.values(charts).filter(
c => c.chartStatus === 'loading' && c.asyncJobId !== undefined,
),
successAction: (componentId, componentData) =>
actions.chartUpdateSucceeded(componentData, componentId),
errorAction: (componentId, response) =>
@@ -85,7 +85,7 @@ const DEFAULT_ORDER = [
'partition',
'event_flow',
'deck_path',
'directed_force',
'graph_chart',
'world_map',
'paired_ttest',
'para',
+3 -1
View File
@@ -40,7 +40,9 @@ const initState = getInitialState(bootstrapData);
const asyncEventMiddleware = initAsyncEvents({
config: bootstrapData.common.conf,
getPendingComponents: ({ charts }) =>
Object.values(charts).filter(c => c.chartStatus === 'loading'),
Object.values(charts).filter(
c => c.chartStatus === 'loading' && c.asyncJobId !== undefined,
),
successAction: (componentId, componentData) =>
actions.chartUpdateSucceeded(componentData, componentId),
errorAction: (componentId, response) =>
@@ -119,8 +119,7 @@ const initAsyncEvents = (options: AsyncEventOptions) => {
};
const processEvents = async () => {
const state = store.getState();
const queuedComponents = getPendingComponents(state);
let queuedComponents = getPendingComponents(store.getState());
const eventArgs = lastReceivedEventId
? { last_id: lastReceivedEventId }
: {};
@@ -128,6 +127,9 @@ const initAsyncEvents = (options: AsyncEventOptions) => {
if (queuedComponents && queuedComponents.length) {
try {
const { result: events } = await fetchEvents(eventArgs);
// refetch queuedComponents due to race condition where results are available
// before component state is updated with asyncJobId
queuedComponents = getPendingComponents(store.getState());
if (events && events.length) {
const componentsByJobId = queuedComponents.reduce((acc, item) => {
acc[item.asyncJobId] = item;
+1 -6
View File
@@ -484,12 +484,7 @@ class ChartRestApi(BaseSupersetModelRestApi):
if result_format == ChartDataResultFormat.CSV:
# return the first result
data = result["queries"][0]["data"]
return CsvResponse(
data,
status=200,
headers=generate_download_headers("csv"),
mimetype="application/csv",
)
return CsvResponse(data, headers=generate_download_headers("csv"))
if result_format == ChartDataResultFormat.JSON:
response_data = simplejson.dumps(
+5 -2
View File
@@ -35,6 +35,7 @@ from superset.exceptions import (
)
from superset.extensions import cache_manager, security_manager
from superset.stats_logger import BaseStatsLogger
from superset.utils import csv
from superset.utils.cache import generate_cache_key, set_and_log_cache
from superset.utils.core import (
ChartDataResultFormat,
@@ -119,7 +120,7 @@ class QueryContext:
# If the datetime format is unix, the parse will use the corresponding
# parsing logic
if not df.empty:
df = normalize_dttm_col(
normalize_dttm_col(
df=df,
timestamp_format=timestamp_format,
offset=self.datasource.offset,
@@ -151,7 +152,9 @@ class QueryContext:
def get_data(self, df: pd.DataFrame,) -> Union[str, List[Dict[str, Any]]]:
if self.result_format == ChartDataResultFormat.CSV:
include_index = not isinstance(df.index, pd.RangeIndex)
result = df.to_csv(index=include_index, **config["CSV_EXPORT"])
result = csv.df_to_escaped_csv(
df, index=include_index, **config["CSV_EXPORT"]
)
return result or ""
return df.to_dict(orient="records")
+10
View File
@@ -306,9 +306,19 @@ DEFAULT_FEATURE_FLAGS: Dict[str, bool] = {
"CLIENT_CACHE": False,
"DISABLE_DATASET_SOURCE_EDIT": False,
"DYNAMIC_PLUGINS": False,
# For some security concerns, you may need to enforce CSRF protection on
# all query request to explore_json endpoint. In Superset, we use
# `flask-csrf <https://sjl.bitbucket.io/flask-csrf/>`_ add csrf protection
# for all POST requests, but this protection doesn't apply to GET method.
# When ENABLE_EXPLORE_JSON_CSRF_PROTECTION is set to true, your users cannot
# make GET request to explore_json. explore_json accepts both GET and POST request.
# See `PR 7935 <https://github.com/apache/superset/pull/7935>`_ for more details.
"ENABLE_EXPLORE_JSON_CSRF_PROTECTION": False,
"ENABLE_TEMPLATE_PROCESSING": False,
"KV_STORE": False,
# When this feature is enabled, nested types in Presto will be
# expanded into extra columns and/or arrays. This is experimental,
# and doesn't work with all nested types.
"PRESTO_EXPAND_DATA": False,
# Exposes API endpoint to compute thumbnails
"THUMBNAILS": False,
+1 -1
View File
@@ -456,7 +456,7 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
)
return database.compile_sqla_query(qry)
if LimitMethod.FORCE_LIMIT:
if cls.limit_method == LimitMethod.FORCE_LIMIT:
parsed_query = sql_parse.ParsedQuery(sql)
sql = parsed_query.set_or_update_query_limit(limit)
+83
View File
@@ -0,0 +1,83 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from datetime import datetime
from typing import Optional
from superset.db_engine_specs.base import BaseEngineSpec, LimitMethod
from superset.utils import core as utils
class FirebirdEngineSpec(BaseEngineSpec):
"""Engine for Firebird"""
engine = "firebird"
engine_name = "Firebird"
# Firebird uses FIRST to limit: `SELECT FIRST 10 * FROM table`
limit_method = LimitMethod.FETCH_MANY
_time_grain_expressions = {
None: "{col}",
"PT1S": (
"CAST(CAST({col} AS DATE) "
"|| ' ' "
"|| EXTRACT(HOUR FROM {col}) "
"|| ':' "
"|| EXTRACT(MINUTE FROM {col}) "
"|| ':' "
"|| FLOOR(EXTRACT(SECOND FROM {col})) AS TIMESTAMP)"
),
"PT1M": (
"CAST(CAST({col} AS DATE) "
"|| ' ' "
"|| EXTRACT(HOUR FROM {col}) "
"|| ':' "
"|| EXTRACT(MINUTE FROM {col}) "
"|| ':00' AS TIMESTAMP)"
),
"PT1H": (
"CAST(CAST({col} AS DATE) "
"|| ' ' "
"|| EXTRACT(HOUR FROM {col}) "
"|| ':00:00' AS TIMESTAMP)"
),
"P1D": "CAST({col} AS DATE)",
"P1M": (
"CAST(EXTRACT(YEAR FROM {col}) "
"|| '-' "
"|| EXTRACT(MONTH FROM {col}) "
"|| '-01' AS DATE)"
),
"P1Y": "CAST(EXTRACT(YEAR FROM {col}) || '-01-01' AS DATE)",
}
@classmethod
def epoch_to_dttm(cls) -> str:
return "DATEADD(second, {col}, CAST('00:00:00' AS TIMESTAMP))"
@classmethod
def convert_dttm(cls, target_type: str, dttm: datetime) -> Optional[str]:
tt = target_type.upper()
if tt == utils.TemporalType.TIMESTAMP:
dttm_formatted = dttm.isoformat(sep=" ")
dttm_valid_precision = dttm_formatted[: len("YYYY-MM-DD HH:MM:SS.MMMM")]
return f"CAST('{dttm_valid_precision}' AS TIMESTAMP)"
if tt == utils.TemporalType.DATE:
return f"CAST('{dttm.date().isoformat()}' AS DATE)"
if tt == utils.TemporalType.TIME:
return f"CAST('{dttm.time().isoformat()}' AS TIME)"
return None
+7 -9
View File
@@ -94,23 +94,21 @@ def load_energy(
slc = Slice(
slice_name="Energy Force Layout",
viz_type="directed_force",
viz_type="graph_chart",
datasource_type="table",
datasource_id=tbl.id,
params=textwrap.dedent(
"""\
{
"charge": "-500",
"collapsed_fieldsets": "",
"groupby": [
"source",
"target"
],
"link_length": "200",
"source": "source",
"target": "target",
"edgeLength": 400,
"repulsion": 1000,
"layout": "force",
"metric": "sum__value",
"row_limit": "5000",
"slice_name": "Force",
"viz_type": "directed_force"
"viz_type": "graph_chart"
}
"""
),
+10 -2
View File
@@ -152,11 +152,19 @@ class BaseReportState:
screenshot: Optional[BaseScreenshot] = None
if self._report_schedule.chart:
url = self._get_url(standalone="true")
screenshot = ChartScreenshot(url, self._report_schedule.chart.digest)
screenshot = ChartScreenshot(
url,
self._report_schedule.chart.digest,
window_size=app.config["WEBDRIVER_WINDOW"]["slice"],
thumb_size=app.config["WEBDRIVER_WINDOW"]["slice"],
)
else:
url = self._get_url()
screenshot = DashboardScreenshot(
url, self._report_schedule.dashboard.digest
url,
self._report_schedule.dashboard.digest,
window_size=app.config["WEBDRIVER_WINDOW"]["dashboard"],
thumb_size=app.config["WEBDRIVER_WINDOW"]["dashboard"],
)
image_url = self._get_url(user_friendly=True)
user = self._get_screenshot_user()
+12 -2
View File
@@ -19,13 +19,16 @@
This is the main entrypoint used by Celery workers. As such,
it needs to call create_app() in order to initialize things properly
"""
from typing import Any
from celery.signals import worker_process_init
# Superset framework imports
from superset import create_app
from superset.extensions import celery_app
from superset.extensions import celery_app, db
# Init the Flask app / configure everything
create_app()
flask_app = create_app()
# Need to import late, as the celery_app will have been setup by "create_app()"
# pylint: disable=wrong-import-position, unused-import
@@ -33,3 +36,10 @@ from . import cache, schedules, scheduler # isort:skip
# Export the celery app globally for Celery (as run on the cmd line) to find
app = celery_app
@worker_process_init.connect
def reset_db_connection_pool(**kwargs: Any) -> None: # pylint: disable=unused-argument
with flask_app.app_context():
# https://docs.sqlalchemy.org/en/14/core/connections.html#engine-disposal
db.engine.dispose()
+2 -4
View File
@@ -1604,10 +1604,9 @@ def normalize_dttm_col(
timestamp_format: Optional[str],
offset: int,
time_shift: Optional[timedelta],
) -> pd.DataFrame:
) -> None:
if DTTM_ALIAS not in df.columns:
return df
df = df.copy()
return
if timestamp_format in ("epoch_s", "epoch_ms"):
dttm_col = df[DTTM_ALIAS]
if is_numeric_dtype(dttm_col):
@@ -1627,4 +1626,3 @@ def normalize_dttm_col(
df[DTTM_ALIAS] += timedelta(hours=offset)
if time_shift is not None:
df[DTTM_ALIAS] += time_shift
return df
+67
View File
@@ -0,0 +1,67 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import re
from typing import Any
import pandas as pd
negative_number_re = re.compile(r"^-[0-9.]+$")
# This regex will match if the string starts with:
#
# 1. one of -, @, +, |, =, %
# 2. two double quotes immediately followed by one of -, @, +, |, =, %
# 3. one or more spaces immediately followed by one of -, @, +, |, =, %
#
problematic_chars_re = re.compile(r'^(?:"{2}|\s{1,})(?=[\-@+|=%])|^[\-@+|=%]')
def escape_value(value: str) -> str:
"""
Escapes a set of special characters.
http://georgemauer.net/2017/10/07/csv-injection.html
"""
needs_escaping = problematic_chars_re.match(value) is not None
is_negative_number = negative_number_re.match(value) is not None
if needs_escaping and not is_negative_number:
# Escape pipe to be extra safe as this
# can lead to remote code execution
value = value.replace("|", "\\|")
# Precede the line with a single quote. This prevents
# evaluation of commands and some spreadsheet software
# will hide this visually from the user. Many articles
# claim a preceding space will work here too, however,
# when uploading a csv file in Google sheets, a leading
# space was ignored and code was still evaluated.
value = "'" + value
return value
def df_to_escaped_csv(df: pd.DataFrame, **kwargs: Any) -> Any:
escape_values = lambda v: escape_value(v) if isinstance(v, str) else v
# Escape csv headers
df = df.rename(columns=escape_values)
# Escape csv rows
df = df.applymap(escape_values)
return df.to_csv(**kwargs)
+61 -1
View File
@@ -22,6 +22,7 @@ import textwrap
import time
from abc import ABC, abstractmethod
from contextlib import contextmanager
from datetime import timedelta
from typing import Any, Callable, cast, Dict, Iterator, Optional, Type, Union
from flask import current_app, g, request
@@ -72,9 +73,68 @@ class AbstractEventLogger(ABC):
) -> None:
pass
def log_with_context(
self,
action: str,
duration: Optional[timedelta] = None,
object_ref: Optional[str] = None,
log_to_statsd: bool = True,
) -> None:
from superset.views.core import get_form_data
referrer = request.referrer[:1000] if request.referrer else None
user_id = g.user.get_id() if hasattr(g, "user") and g.user else None
payload = collect_request_payload()
if object_ref:
payload["object_ref"] = object_ref
dashboard_id: Optional[int] = None
try:
dashboard_id = int(payload.get("dashboard_id")) # type: ignore
except (TypeError, ValueError):
dashboard_id = None
if "form_data" in payload:
form_data, _ = get_form_data()
payload["form_data"] = form_data
slice_id = form_data.get("slice_id")
else:
slice_id = payload.get("slice_id")
try:
slice_id = int(slice_id) # type: ignore
except (TypeError, ValueError):
slice_id = 0
if log_to_statsd:
self.stats_logger.incr(action)
try:
# bulk insert
explode_by = payload.get("explode")
records = json.loads(payload.get(explode_by)) # type: ignore
except Exception: # pylint: disable=broad-except
records = [payload]
duration_ms = duration.total_seconds() * 1000 if duration else None
self.log(
user_id,
action,
records=records,
dashboard_id=dashboard_id,
slice_id=slice_id,
duration_ms=duration_ms,
referrer=referrer,
)
@contextmanager
def log_context( # pylint: disable=too-many-locals
self, action: str, object_ref: Optional[str] = None, log_to_statsd: bool = True,
self,
action: str,
object_ref: Optional[str] = None,
log_to_statsd: bool = True,
) -> Iterator[Callable[..., None]]:
"""
Log an event with additional information from the request context.
+22 -4
View File
@@ -195,12 +195,30 @@ class BaseScreenshot:
class ChartScreenshot(BaseScreenshot):
thumbnail_type: str = "chart"
element: str = "chart-container"
window_size: WindowSize = (800, 600)
thumb_size: WindowSize = (800, 600)
def __init__(
self,
url: str,
digest: str,
window_size: Optional[WindowSize] = None,
thumb_size: Optional[WindowSize] = None,
):
super().__init__(url, digest)
self.window_size = window_size or (800, 600)
self.thumb_size = thumb_size or (800, 600)
class DashboardScreenshot(BaseScreenshot):
thumbnail_type: str = "dashboard"
element: str = "grid-container"
window_size: WindowSize = (1600, int(1600 * 0.75))
thumb_size: WindowSize = (800, int(800 * 0.75))
def __init__(
self,
url: str,
digest: str,
window_size: Optional[WindowSize] = None,
thumb_size: Optional[WindowSize] = None,
):
super().__init__(url, digest)
self.window_size = window_size or (1600, 1200)
self.thumb_size = thumb_size or (800, 600)
+1
View File
@@ -479,6 +479,7 @@ class CsvResponse(Response): # pylint: disable=too-many-ancestors
"""
charset = conf["CSV_EXPORT"].get("encoding", "utf-8")
default_mimetype = "text/csv"
def check_ownership(obj: Any, raise_if_false: bool = True) -> bool:
+5 -12
View File
@@ -98,7 +98,7 @@ from superset.sql_parse import CtasMethod, ParsedQuery, Table
from superset.sql_validators import get_validator_by_name
from superset.tasks.async_queries import load_explore_json_into_cache
from superset.typing import FlaskResponse
from superset.utils import core as utils
from superset.utils import core as utils, csv
from superset.utils.async_query_manager import AsyncQueryTokenException
from superset.utils.cache import etag_cache
from superset.utils.core import ReservedUrlParameters
@@ -437,10 +437,7 @@ class Superset(BaseSupersetView): # pylint: disable=too-many-public-methods
) -> FlaskResponse:
if response_type == utils.ChartDataResultFormat.CSV:
return CsvResponse(
viz_obj.get_csv(),
status=200,
headers=generate_download_headers("csv"),
mimetype="application/csv",
viz_obj.get_csv(), headers=generate_download_headers("csv")
)
if response_type == utils.ChartDataResultType.QUERY:
@@ -2622,18 +2619,14 @@ class Superset(BaseSupersetView): # pylint: disable=too-many-public-methods
columns = [c["name"] for c in obj["columns"]]
df = pd.DataFrame.from_records(obj["data"], columns=columns)
logger.info("Using pandas to convert to CSV")
csv = df.to_csv(index=False, **config["CSV_EXPORT"])
else:
logger.info("Running a query to turn into CSV")
sql = query.select_sql or query.executed_sql
df = query.database.get_df(sql, query.schema)
# TODO(bkyryliuk): add compression=gzip for big files.
csv = df.to_csv(index=False, **config["CSV_EXPORT"])
response = Response(csv, mimetype="text/csv")
csv_data = csv.df_to_escaped_csv(df, index=False, **config["CSV_EXPORT"])
quoted_csv_name = parse.quote(query.name)
response.headers["Content-Disposition"] = (
f'attachment; filename="{quoted_csv_name}.csv"; '
f"filename*=UTF-8''{quoted_csv_name}.csv"
response = CsvResponse(
csv_data, headers=generate_download_headers("csv", quoted_csv_name)
)
event_info = {
"event_type": "data_export",
+20 -2
View File
@@ -14,6 +14,9 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import logging
from typing import Optional
from flask import flash, request, Response
from flask_appbuilder import expose
from flask_appbuilder.security.decorators import has_access_api
@@ -24,11 +27,22 @@ from superset.models import core as models
from superset.typing import FlaskResponse
from superset.views.base import BaseSupersetView
logger = logging.getLogger(__name__)
class R(BaseSupersetView): # pylint: disable=invalid-name
"""used for short urls"""
@staticmethod
def _validate_url(url: Optional[str] = None) -> bool:
if url and (
url.startswith("//superset/dashboard/")
or url.startswith("//superset/explore/")
):
return True
return False
@event_logger.log_this
@expose("/<int:url_id>")
def index(self, url_id: int) -> FlaskResponse: # pylint: disable=no-self-use
@@ -38,8 +52,9 @@ class R(BaseSupersetView): # pylint: disable=invalid-name
if url.url.startswith(explore_url):
explore_url += f"r={url_id}"
return redirect(explore_url[1:])
return redirect(url.url[1:])
if self._validate_url(url.url):
return redirect(url.url[1:])
return redirect("/")
flash("URL to nowhere...", "danger")
return redirect("/")
@@ -49,6 +64,9 @@ class R(BaseSupersetView): # pylint: disable=invalid-name
@expose("/shortner/", methods=["POST"])
def shortner(self) -> FlaskResponse: # pylint: disable=no-self-use
url = request.form.get("data")
if not self._validate_url(url):
logger.warning("Invalid URL: %s", url)
return Response(f"Invalid URL: {url}", 400)
obj = models.Url(url=url)
db.session.add(obj)
db.session.commit()
+3 -28
View File
@@ -66,7 +66,7 @@ from superset.extensions import cache_manager, security_manager
from superset.models.cache import CacheKey
from superset.models.helpers import QueryResult
from superset.typing import QueryObjectDict, VizData, VizPayload
from superset.utils import core as utils
from superset.utils import core as utils, csv
from superset.utils.cache import set_and_log_cache
from superset.utils.core import (
DTTM_ALIAS,
@@ -284,7 +284,7 @@ class BaseViz:
# If the datetime format is unix, the parse will use the corresponding
# parsing logic.
if not df.empty:
df = utils.normalize_dttm_col(
utils.normalize_dttm_col(
df=df,
timestamp_format=timestamp_format,
offset=self.datasource.offset,
@@ -614,7 +614,7 @@ class BaseViz:
def get_csv(self) -> Optional[str]:
df = self.get_df_payload()["df"] # leverage caching logic
include_index = not isinstance(df.index, pd.RangeIndex)
return df.to_csv(index=include_index, **config["CSV_EXPORT"])
return csv.df_to_escaped_csv(df, index=include_index, **config["CSV_EXPORT"])
def get_data(self, df: pd.DataFrame) -> VizData:
return df.to_dict(orient="records")
@@ -1898,31 +1898,6 @@ class SankeyViz(BaseViz):
return recs
class DirectedForceViz(BaseViz):
"""An animated directed force layout graph visualization"""
viz_type = "directed_force"
verbose_name = _("Directed Force Layout")
credits = 'd3noob @<a href="http://bl.ocks.org/d3noob/5141278">bl.ocks.org</a>'
is_timeseries = False
def query_obj(self) -> QueryObjectDict:
qry = super().query_obj()
if len(self.form_data["groupby"]) != 2:
raise QueryObjectValidationError(_("Pick exactly 2 columns to 'Group By'"))
qry["metrics"] = [self.form_data["metric"]]
if self.form_data.get("sort_by_metric", False):
qry["orderby"] = [(qry["metrics"][0], False)]
return qry
def get_data(self, df: pd.DataFrame) -> VizData:
if df.empty:
return None
df.columns = ["source", "target", "value"]
return df.to_dict(orient="records")
class ChordViz(BaseViz):
"""A Chord diagram"""
+22
View File
@@ -634,6 +634,28 @@ class TestCore(SupersetTestCase):
resp = self.client.post("/r/shortner/", data=dict(data=data))
assert re.search(r"\/r\/[0-9]+", resp.data.decode("utf-8"))
def test_shortner_invalid(self):
self.login(username="admin")
invalid_urls = [
"hhttp://invalid.com",
"hhttps://invalid.com",
"www.invalid.com",
]
for invalid_url in invalid_urls:
resp = self.client.post("/r/shortner/", data=dict(data=invalid_url))
assert resp.status_code == 400
def test_redirect_invalid(self):
model_url = models.Url(url="hhttp://invalid.com")
db.session.add(model_url)
db.session.commit()
self.login(username="admin")
response = self.client.get(f"/r/{model_url.id}")
assert response.headers["Location"] == "http://localhost/"
db.session.delete(model_url)
db.session.commit()
@skipUnless(
(is_feature_enabled("KV_STORE")), "skipping as /kv/ endpoints are not enabled"
)
+81
View File
@@ -0,0 +1,81 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from datetime import datetime
from unittest import mock
import pytest
from superset.db_engine_specs.firebird import FirebirdEngineSpec
grain_expressions = {
None: "timestamp_column",
"PT1S": (
"CAST(CAST(timestamp_column AS DATE) "
"|| ' ' "
"|| EXTRACT(HOUR FROM timestamp_column) "
"|| ':' "
"|| EXTRACT(MINUTE FROM timestamp_column) "
"|| ':' "
"|| FLOOR(EXTRACT(SECOND FROM timestamp_column)) AS TIMESTAMP)"
),
"PT1M": (
"CAST(CAST(timestamp_column AS DATE) "
"|| ' ' "
"|| EXTRACT(HOUR FROM timestamp_column) "
"|| ':' "
"|| EXTRACT(MINUTE FROM timestamp_column) "
"|| ':00' AS TIMESTAMP)"
),
"P1D": "CAST(timestamp_column AS DATE)",
"P1M": (
"CAST(EXTRACT(YEAR FROM timestamp_column) "
"|| '-' "
"|| EXTRACT(MONTH FROM timestamp_column) "
"|| '-01' AS DATE)"
),
"P1Y": "CAST(EXTRACT(YEAR FROM timestamp_column) || '-01-01' AS DATE)",
}
@pytest.mark.parametrize("grain,expected", grain_expressions.items())
def test_time_grain_expressions(grain, expected):
assert (
FirebirdEngineSpec._time_grain_expressions[grain].format(col="timestamp_column")
== expected
)
def test_epoch_to_dttm():
assert (
FirebirdEngineSpec.epoch_to_dttm().format(col="timestamp_column")
== "DATEADD(second, timestamp_column, CAST('00:00:00' AS TIMESTAMP))"
)
def test_convert_dttm():
dttm = datetime(2021, 1, 1)
assert (
FirebirdEngineSpec.convert_dttm("timestamp", dttm)
== "CAST('2021-01-01 00:00:00' AS TIMESTAMP)"
)
assert (
FirebirdEngineSpec.convert_dttm("TIMESTAMP", dttm)
== "CAST('2021-01-01 00:00:00' AS TIMESTAMP)"
)
assert FirebirdEngineSpec.convert_dttm("TIME", dttm) == "CAST('00:00:00' AS TIME)"
assert FirebirdEngineSpec.convert_dttm("DATE", dttm) == "CAST('2021-01-01' AS DATE)"
assert FirebirdEngineSpec.convert_dttm("STRING", dttm) is None
+7 -6
View File
@@ -141,16 +141,17 @@ def _get_energy_slices():
},
{
"slice_title": "Energy Force Layout",
"viz_type": "directed_force",
"viz_type": "graph_chart",
"params": {
"charge": "-500",
"collapsed_fieldsets": "",
"groupby": ["source", "target"],
"link_length": "200",
"source": "source",
"target": "target",
"edgeLength": 400,
"repulsion": 1000,
"layout": "force",
"metric": "sum__value",
"row_limit": "5000",
"slice_name": "Force",
"viz_type": "directed_force",
"viz_type": "graph_chart",
},
},
{
+80
View File
@@ -0,0 +1,80 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# pylint: disable=no-self-use
import io
import pandas as pd
import pytest
from superset.utils import csv
def test_escape_value():
result = csv.escape_value("value")
assert result == "value"
result = csv.escape_value("-10")
assert result == "-10"
result = csv.escape_value("@value")
assert result == "'@value"
result = csv.escape_value("+value")
assert result == "'+value"
result = csv.escape_value("-value")
assert result == "'-value"
result = csv.escape_value("=value")
assert result == "'=value"
result = csv.escape_value("|value")
assert result == "'\|value"
result = csv.escape_value("%value")
assert result == "'%value"
result = csv.escape_value("=cmd|' /C calc'!A0")
assert result == "'=cmd\|' /C calc'!A0"
result = csv.escape_value('""=10+2')
assert result == '\'""=10+2'
result = csv.escape_value(" =10+2")
assert result == "' =10+2"
def test_df_to_escaped_csv():
csv_rows = [
["col_a", "=func()"],
["-10", "=cmd|' /C calc'!A0"],
["a", '""=b'],
[" =a", "b"],
]
csv_str = "\n".join([",".join(row) for row in csv_rows])
df = pd.read_csv(io.StringIO(csv_str))
escaped_csv_str = csv.df_to_escaped_csv(df, encoding="utf8", index=False)
escaped_csv_rows = [row.split(",") for row in escaped_csv_str.strip().split("\n")]
assert escaped_csv_rows == [
["col_a", "'=func()"],
["-10", "'=cmd\|' /C calc'!A0"],
["a", "'=b"], # pandas seems to be removing the leading ""
["' =a", "b"],
]
+18 -8
View File
@@ -23,7 +23,7 @@ import hashlib
import json
import os
import re
from typing import Any, Tuple, List
from typing import Any, Tuple, List, Optional
from unittest.mock import Mock, patch
from tests.fixtures.birth_names_dashboard import load_birth_names_dashboard_with_slices
@@ -1135,28 +1135,38 @@ class TestUtils(SupersetTestCase):
assert extract_dataframe_dtypes(df) == [col[1] for col in cols]
def test_normalize_dttm_col(self):
def normalize_col(
df: pd.DataFrame,
timestamp_format: Optional[str],
offset: int,
time_shift: Optional[timedelta],
) -> pd.DataFrame:
df = df.copy()
normalize_dttm_col(df, timestamp_format, offset, time_shift)
return df
ts = pd.Timestamp(2021, 2, 15, 19, 0, 0, 0)
df = pd.DataFrame([{"__timestamp": ts, "a": 1}])
# test regular (non-numeric) format
assert normalize_dttm_col(df, None, 0, None)[DTTM_ALIAS][0] == ts
assert normalize_dttm_col(df, "epoch_ms", 0, None)[DTTM_ALIAS][0] == ts
assert normalize_dttm_col(df, "epoch_s", 0, None)[DTTM_ALIAS][0] == ts
assert normalize_col(df, None, 0, None)[DTTM_ALIAS][0] == ts
assert normalize_col(df, "epoch_ms", 0, None)[DTTM_ALIAS][0] == ts
assert normalize_col(df, "epoch_s", 0, None)[DTTM_ALIAS][0] == ts
# test offset
assert normalize_dttm_col(df, None, 1, None)[DTTM_ALIAS][0] == pd.Timestamp(
assert normalize_col(df, None, 1, None)[DTTM_ALIAS][0] == pd.Timestamp(
2021, 2, 15, 20, 0, 0, 0
)
# test offset and timedelta
assert normalize_dttm_col(df, None, 1, timedelta(minutes=30))[DTTM_ALIAS][
assert normalize_col(df, None, 1, timedelta(minutes=30))[DTTM_ALIAS][
0
] == pd.Timestamp(2021, 2, 15, 20, 30, 0, 0)
# test numeric epoch_s format
df = pd.DataFrame([{"__timestamp": ts.timestamp(), "a": 1}])
assert normalize_dttm_col(df, "epoch_s", 0, None)[DTTM_ALIAS][0] == ts
assert normalize_col(df, "epoch_s", 0, None)[DTTM_ALIAS][0] == ts
# test numeric epoch_ms format
df = pd.DataFrame([{"__timestamp": ts.timestamp() * 1000, "a": 1}])
assert normalize_dttm_col(df, "epoch_ms", 0, None)[DTTM_ALIAS][0] == ts
assert normalize_col(df, "epoch_ms", 0, None)[DTTM_ALIAS][0] == ts