mirror of
https://github.com/apache/superset.git
synced 2026-08-12 11:11:01 +00:00
Compare commits
39
Commits
+168
@@ -18,6 +18,174 @@ under the License.
|
||||
-->
|
||||
## Change Log
|
||||
|
||||
### 0.35.2 (2020/01/03 16:42 +00:00)
|
||||
- [#8918](https://github.com/apache/incubator-superset/pull/8918) [database] [log] Fix, Limit the amount of info on response (#8918) (@dpgaspar)
|
||||
- [#8759](https://github.com/apache/incubator-superset/pull/8759) Bump viz plugins for bug bash (#8759) (@etr2460)
|
||||
- [#8729](https://github.com/apache/incubator-superset/pull/8729) fix: don't show filter popover on explore view load (#8729) (@mistercrunch)
|
||||
- [#8718](https://github.com/apache/incubator-superset/pull/8718) Migrate filter_immune_slice_fields (#8718) (@betodealmeida)
|
||||
- [#8714](https://github.com/apache/incubator-superset/pull/8714) [fix] Druid IS NULL/IS NOT NULL filters (#8714) (@john-bodley)
|
||||
- [#8678](https://github.com/apache/incubator-superset/pull/8678) fix: default missing values to zero on area chart (#8678) (@villebro)
|
||||
- [#8573](https://github.com/apache/incubator-superset/pull/8573) [SECURITY] bump packages with security vulnerabilities (#8573) (@nytai)
|
||||
- [#8583](https://github.com/apache/incubator-superset/pull/8583) Bump pyarrow to 0.15.1 due to CVE (#8583) (@robdiciuccio)
|
||||
- [#8581](https://github.com/apache/incubator-superset/pull/8581) Fix error when templateParams is undefined (#8581) (@betodealmeida)
|
||||
- [#8575](https://github.com/apache/incubator-superset/pull/8575) Math.max(...array) considered harmful (#8575) (@betodealmeida)
|
||||
- [#8550](https://github.com/apache/incubator-superset/pull/8550) [Dashboard Import] Log error on dashboard import failure (#8550) (@etr2460)
|
||||
- [#8427](https://github.com/apache/incubator-superset/pull/8427) Default page: remove title attribute to fall back on appbuilder.app_name (#8427) (@Zanoellia)
|
||||
- [#8495](https://github.com/apache/incubator-superset/pull/8495) [datasource editor] Only one click target for edit action (#8495) (@graceguo-supercat)
|
||||
- [#8465](https://github.com/apache/incubator-superset/pull/8465) Update UPDATING.md (#8465) (@john-bodley)
|
||||
- [#8471](https://github.com/apache/incubator-superset/pull/8471) [Datasource Editor] A few small UI changes in modal to prevent accidental edits (#8471) (@graceguo-supercat)
|
||||
- [#8439](https://github.com/apache/incubator-superset/pull/8439) [setup] Fix, download_url (#8439) (@dpgaspar)
|
||||
- [#8449](https://github.com/apache/incubator-superset/pull/8449) Split up tests/db_engine_test.py (#8449) (@willbarrett)
|
||||
- [#8438](https://github.com/apache/incubator-superset/pull/8438) introduce a space in command line option (#8438) (@christophlingg)
|
||||
- [#8444](https://github.com/apache/incubator-superset/pull/8444) explain the need to enable async queries (#8444) (@christophlingg)
|
||||
- [#8442](https://github.com/apache/incubator-superset/pull/8442) spelling fix (#8442) (@austinpray)
|
||||
- [#8419](https://github.com/apache/incubator-superset/pull/8419) docs: add a note to RELEASING about Slack Channel (#8419) (@mistercrunch)
|
||||
|
||||
### 0.35.1 (2019/11/15 13:47 +00:00)
|
||||
- [#8457](https://github.com/apache/incubator-superset/pull/8457) [fix] Improve csv upload functionality (#8457) (@villebro)
|
||||
- [#8566](https://github.com/apache/incubator-superset/pull/8566) [druid] Fix, pydruid forced dependency (#8566) (@dpgaspar)
|
||||
- [#8558](https://github.com/apache/incubator-superset/pull/8558) bump legacy-preset-chart-nvd3 to 0.11.5 (#8558) (@nytai)
|
||||
- [#8498](https://github.com/apache/incubator-superset/pull/8498) build: bump dompurify version because of nasty xss bypass. (#8498) (@MarcusSorealheis)
|
||||
- [#8477](https://github.com/apache/incubator-superset/pull/8477) [cli] Fix, import datasources exported by UI (#8477) (@dpgaspar)
|
||||
- [#8487](https://github.com/apache/incubator-superset/pull/8487) fixing typo. (#8487) (@MarcusSorealheis)
|
||||
- [#8430](https://github.com/apache/incubator-superset/pull/8430) [pydruid] Bumping the pydruid version (#8430) (@john-bodley)
|
||||
|
||||
### 0.35.0 (2019/10/31 11:12 +00:00)
|
||||
- [#8436](https://github.com/apache/incubator-superset/pull/8436) [fix] Updating parse_human_timedelta typing (#8436) (@john-bodley)
|
||||
- [#8423](https://github.com/apache/incubator-superset/pull/8423) chore: disable another flaky cypress test (#8423) (@mistercrunch)
|
||||
- [#8331](https://github.com/apache/incubator-superset/pull/8331) Add improved typed casting to BigQuery (#8331) (@betodealmeida)
|
||||
- [#8353](https://github.com/apache/incubator-superset/pull/8353) [tests] Fix, removing deprecated function for future Python release compatibility. (#8353) (@MarcusSorealheis)
|
||||
- [#8413](https://github.com/apache/incubator-superset/pull/8413) fixed typo in installation instructions (#8413) (@francishemingway)
|
||||
- [#8409](https://github.com/apache/incubator-superset/pull/8409) [ci] Deprecate flake8 (#8409) (@john-bodley)
|
||||
- [#7702](https://github.com/apache/incubator-superset/pull/7702) [sqla] Enforcing ISO 8601 date/timestamp formats (#7702) (@john-bodley)
|
||||
- [#8370](https://github.com/apache/incubator-superset/pull/8370) [Config] Cache static resources (#8370) (@etr2460)
|
||||
- [#8367](https://github.com/apache/incubator-superset/pull/8367) [docs] CTAS on PostgreSQL needs commit to apply (#8367) (@dpgaspar)
|
||||
- [#8399](https://github.com/apache/incubator-superset/pull/8399) Update Slack invitation link (#8399) (@robdiciuccio)
|
||||
- [#8371](https://github.com/apache/incubator-superset/pull/8371) [fix] make names non-nullable (#8371) (@serenajiang)
|
||||
- [#8390](https://github.com/apache/incubator-superset/pull/8390) [form-data] Deleting obsolete Superset.get_form_data method (#8390) (@john-bodley)
|
||||
- [#8376](https://github.com/apache/incubator-superset/pull/8376) Add 0.34.1 PRs to CHANGELOG (#8376) (@villebro)
|
||||
- [#8372](https://github.com/apache/incubator-superset/pull/8372) [migration] Migrating since/until to time_range (#8372) (@john-bodley)
|
||||
- [#8374](https://github.com/apache/incubator-superset/pull/8374) [Cypress] Skip another flaky sqllab test (#8374) (@etr2460)
|
||||
- [#8369](https://github.com/apache/incubator-superset/pull/8369) Add incubating to NOTICE (#8369) (@villebro)
|
||||
- [#8368](https://github.com/apache/incubator-superset/pull/8368) [Docs] Put superset users in abc order (#8368) (@etr2460)
|
||||
- [#8365](https://github.com/apache/incubator-superset/pull/8365) Update README.md (#8365) (@christophlingg)
|
||||
- [#8332](https://github.com/apache/incubator-superset/pull/8332) Revert "[fix] make datasource names non-nullable (#8332)" (#8363) (@serenajiang)
|
||||
- [#8345](https://github.com/apache/incubator-superset/pull/8345) [dashboard] show less filter indicators (#8345) (@graceguo-supercat)
|
||||
- [#8330](https://github.com/apache/incubator-superset/pull/8330) Fix pypi link and replace gitter badge with slack (#8330) (@villebro)
|
||||
- [#7992](https://github.com/apache/incubator-superset/pull/7992) Revert "[init] Setting up cache before registering blueprints (#7992)" (#8005) (@john-bodley)
|
||||
- [#8343](https://github.com/apache/incubator-superset/pull/8343) Add support for Exasol (#8343) (@kevang)
|
||||
- [#8351](https://github.com/apache/incubator-superset/pull/8351) fix saving new sql lab queries (#8351) (@suddjian)
|
||||
- [#8350](https://github.com/apache/incubator-superset/pull/8350) Remove unused methods from Presto (#8350) (@betodealmeida)
|
||||
- [#8338](https://github.com/apache/incubator-superset/pull/8338) Fix lint in `superset/db_engine_spec` (#8338) (@willbarrett)
|
||||
- [#8332](https://github.com/apache/incubator-superset/pull/8332) [fix] make datasource names non-nullable (#8332) (@serenajiang)
|
||||
- [#8344](https://github.com/apache/incubator-superset/pull/8344) Add Fanatics to companies that use Superset (#8344) (@coderfender)
|
||||
- [#8347](https://github.com/apache/incubator-superset/pull/8347) [docs] Add Deepomatic in organization list (#8347) (@Zanoellia)
|
||||
- [#8296](https://github.com/apache/incubator-superset/pull/8296) feat: Set moment library language during app setup (#8296) (@oskarhinc)
|
||||
- [#8299](https://github.com/apache/incubator-superset/pull/8299) fix: npm audit fix bumps handlebars 4.1.2->4.3.1 (#8299) (@mistercrunch)
|
||||
- [#8175](https://github.com/apache/incubator-superset/pull/8175) [metric] Adding security for restricted metrics (#8175) (@john-bodley)
|
||||
- [#8304](https://github.com/apache/incubator-superset/pull/8304) [Presto] Handle uncaught exception in get_create_view (#8304) (@etr2460)
|
||||
- [#8305](https://github.com/apache/incubator-superset/pull/8305) [SQL Lab] Improve autocomplete (#8305) (@etr2460)
|
||||
- [#8298](https://github.com/apache/incubator-superset/pull/8298) Allow overwriting a SQLLab query that has previously been saved (#8298) (@suddjian)
|
||||
- [#8300](https://github.com/apache/incubator-superset/pull/8300) refactor: remove unused code (#8300) (@kristw)
|
||||
- [#8320](https://github.com/apache/incubator-superset/pull/8320) Update RunQueryActionButton.jsx (#8320) (@nabinkhadka)
|
||||
- [#8317](https://github.com/apache/incubator-superset/pull/8317) Adding explicit Flask session cookie options to default config (#8317) (@craig-rueda)
|
||||
- [#8239](https://github.com/apache/incubator-superset/pull/8239) [superset] Fix, missing return on error and improved query (#8239) (@dpgaspar)
|
||||
- [#8307](https://github.com/apache/incubator-superset/pull/8307) allow non-relative LOGO_TARGET_PATH (#8307) (@nytai)
|
||||
- [#8311](https://github.com/apache/incubator-superset/pull/8311) Add safaricom to companies using superset (#8311) (@mmutiso)
|
||||
- [#8309](https://github.com/apache/incubator-superset/pull/8309) Add explicit support for python 3.7 (#8309) (@villebro)
|
||||
- [#8272](https://github.com/apache/incubator-superset/pull/8272) Refine release instructions (#8272) (@villebro)
|
||||
- [#8262](https://github.com/apache/incubator-superset/pull/8262) Add commit to attempt to resolve query table lock (#8262) (@etr2460)
|
||||
- [#8282](https://github.com/apache/incubator-superset/pull/8282) chore: skip flaky SQL Lab test (#8282) (@mistercrunch)
|
||||
- [#8261](https://github.com/apache/incubator-superset/pull/8261) [migration] Stripping leading and trailing whitespace (#8261) (@john-bodley)
|
||||
- [#8256](https://github.com/apache/incubator-superset/pull/8256) [sql_json] Ensuring the request body is JSON encoded (#8256) (@john-bodley)
|
||||
- [#8240](https://github.com/apache/incubator-superset/pull/8240) [database] Improve form and API validation for invalid URI (#8240) (@dpgaspar)
|
||||
- [#8274](https://github.com/apache/incubator-superset/pull/8274) More intuitive display of negative time deltas (#8274) (#8276) (@benvdh)
|
||||
- [#8268](https://github.com/apache/incubator-superset/pull/8268) Fix no data in Presto (#8268) (@betodealmeida)
|
||||
- [#8258](https://github.com/apache/incubator-superset/pull/8258) refactor: use deck.gl plugins from npm package (#8258) (@kristw)
|
||||
- [#8138](https://github.com/apache/incubator-superset/pull/8138) [typing] add typing for superset/connectors and superset/common (#8138) (@serenajiang)
|
||||
- [#8163](https://github.com/apache/incubator-superset/pull/8163) [SQLLab] Refactor sql json endpoint (#8163) (@dpgaspar)
|
||||
- [#8257](https://github.com/apache/incubator-superset/pull/8257) Remove __row_id (#8257) (@betodealmeida)
|
||||
- [#8250](https://github.com/apache/incubator-superset/pull/8250) Update another pypi case from superset to apache-superset (#8250) (@etr2460)
|
||||
- [#8244](https://github.com/apache/incubator-superset/pull/8244) docs: reflect the pypi move from superset to apache-superset (#8244) (@mistercrunch)
|
||||
- [#8224](https://github.com/apache/incubator-superset/pull/8224) [sqllab] create query slightly earlier (#8224) (@serenajiang)
|
||||
- [#8253](https://github.com/apache/incubator-superset/pull/8253) Fix array casting (#8253) (@betodealmeida)
|
||||
- [#8243](https://github.com/apache/incubator-superset/pull/8243) Show Presto views as views, not tables (#8243) (@betodealmeida)
|
||||
- [#8251](https://github.com/apache/incubator-superset/pull/8251) Small fix for Presto dtype map (#8251) (@betodealmeida)
|
||||
- [#8246](https://github.com/apache/incubator-superset/pull/8246) Update README.md (#8246) (@MaiTiano)
|
||||
- [#8185](https://github.com/apache/incubator-superset/pull/8185) Add RegisteredLookupExtraction support to extraction function (#8185) (@1AB9502)
|
||||
- [#8213](https://github.com/apache/incubator-superset/pull/8213) Better distinction between tables and views, and show CREATE VIEW (#8213) (@betodealmeida)
|
||||
- [#8233](https://github.com/apache/incubator-superset/pull/8233) Simplify and improve `expand_data` in Presto (#8233) (@betodealmeida)
|
||||
- [#8172](https://github.com/apache/incubator-superset/pull/8172) Allow users to estimate query cost before executing it (#8172) (@betodealmeida)
|
||||
- [#8226](https://github.com/apache/incubator-superset/pull/8226) Handle int64 columns with missing data in SQL Lab (#8226) (@betodealmeida)
|
||||
- [#8222](https://github.com/apache/incubator-superset/pull/8222) refactor: prepare control panel configs for separation into plugins (#8222) (@kristw)
|
||||
- [#8221](https://github.com/apache/incubator-superset/pull/8221) fix: initialize control state for inline control config object (#8221) (@kristw)
|
||||
- [#8214](https://github.com/apache/incubator-superset/pull/8214) [talisman] Enforcing HTTP for status checks (#8214) (@john-bodley)
|
||||
- [#8173](https://github.com/apache/incubator-superset/pull/8173) feat: read control panel configs from registry (#8173) (@kristw)
|
||||
- [#8196](https://github.com/apache/incubator-superset/pull/8196) chore: fix version info (#8196) (@mistercrunch)
|
||||
- [#8212](https://github.com/apache/incubator-superset/pull/8212) fix: attempt to fix the flaky SQL Lab test (#8212) (@mistercrunch)
|
||||
- [#8210](https://github.com/apache/incubator-superset/pull/8210) Update messages.json (#8210) (@klxiang)
|
||||
- [#8166](https://github.com/apache/incubator-superset/pull/8166) [feature][dashboard] Show/hide filter indicator on the applicable charts when filter options are open/close (#8166) (@graceguo-supercat)
|
||||
- [#8206](https://github.com/apache/incubator-superset/pull/8206) Update README.md (#8206) (@garnservo)
|
||||
- [#8208](https://github.com/apache/incubator-superset/pull/8208) chore: update UPDATING.md with a 0.34.0 section (#8208) (@mistercrunch)
|
||||
- [#8034](https://github.com/apache/incubator-superset/pull/8034) [viz] Adding get_def typing (#8034) (@john-bodley)
|
||||
- [#8197](https://github.com/apache/incubator-superset/pull/8197) chore: deprecate restricted metrics (#8197) (@mistercrunch)
|
||||
- [#8198](https://github.com/apache/incubator-superset/pull/8198) Update messages.json (#8198) (@klxiang)
|
||||
- [#8204](https://github.com/apache/incubator-superset/pull/8204) change log from csv_export to data_export; fix db name in log (#8204) (@DiggidyDave)
|
||||
- [#8164](https://github.com/apache/incubator-superset/pull/8164) more detailed async error messages (#8164) (@serenajiang)
|
||||
- [#8182](https://github.com/apache/incubator-superset/pull/8182) [Config] Disable celery acks late (#8182) (@etr2460)
|
||||
- [#8038](https://github.com/apache/incubator-superset/pull/8038) Prevent 'main' database connection creation (#8038) (@mistercrunch)
|
||||
- [#8187](https://github.com/apache/incubator-superset/pull/8187) ignore formatting for <Select> (#8187) (@gbates101)
|
||||
- [#8184](https://github.com/apache/incubator-superset/pull/8184) [fix] retry stop_query on all exceptions (#8184) (@serenajiang)
|
||||
- [#8179](https://github.com/apache/incubator-superset/pull/8179) Update messages.json (#8179) (@klxiang)
|
||||
- [#8158](https://github.com/apache/incubator-superset/pull/8158) Bump cypress from 3.1.5 to 3.4.1 (#8158) (@mistercrunch)
|
||||
- [#8139](https://github.com/apache/incubator-superset/pull/8139) [sqllab] add retries for stop_query (#8139) (@serenajiang)
|
||||
- [#8143](https://github.com/apache/incubator-superset/pull/8143) [Codemod] Rename react unsafe lifecycles (#8143) (@etr2460)
|
||||
- [#8149](https://github.com/apache/incubator-superset/pull/8149) chore: remove PY3 conditional logic (#8149) (@mistercrunch)
|
||||
- [#8156](https://github.com/apache/incubator-superset/pull/8156) Forward SIGTERM signal to gunicorn (#8156) (@smacker)
|
||||
- [#8148](https://github.com/apache/incubator-superset/pull/8148) Adding explicit include for version_info.json (#8148) (@craig-rueda)
|
||||
- [#7822](https://github.com/apache/incubator-superset/pull/7822) Revert "Fix tooltips not visible for NVD3 charts on Firefox (#7822) (#7929)" (#8147) (@etr2460)
|
||||
- [#8099](https://github.com/apache/incubator-superset/pull/8099) Revert "Allow resizing width of SQL Lab left bar / editor (#8099)" (#8146) (@etr2460)
|
||||
- [#8130](https://github.com/apache/incubator-superset/pull/8130) removes unused state values and redundant presence checks (#8130) (@nytai)
|
||||
- [#8136](https://github.com/apache/incubator-superset/pull/8136) Add duration formatter for numeric data (#8136) (@villebro)
|
||||
- [#8140](https://github.com/apache/incubator-superset/pull/8140) remove unused files (#8140) (@kristw)
|
||||
- [#8135](https://github.com/apache/incubator-superset/pull/8135) [setup] use new python requires metadata key (#8135) (@dpgaspar)
|
||||
- [#8109](https://github.com/apache/incubator-superset/pull/8109) Moving towards a whitelist approach for MANIFEST.in (#8109) (@craig-rueda)
|
||||
- [#8069](https://github.com/apache/incubator-superset/pull/8069) [SQL Lab] Async query results serialization with MessagePack and PyArrow (#8069) (@robdiciuccio)
|
||||
- [#8115](https://github.com/apache/incubator-superset/pull/8115) feat: bump superset-ui dependencies to v0.12 and plugins to v0.11 (#8115) (@kristw)
|
||||
- [#8085](https://github.com/apache/incubator-superset/pull/8085) Add LogConfigurator, and add logging for CSV export (#8085) (@DiggidyDave)
|
||||
- [#8111](https://github.com/apache/incubator-superset/pull/8111) removed database deps from reqs-dev.txt and typo. (#8111) (@MarcusSorealheis)
|
||||
- [#8104](https://github.com/apache/incubator-superset/pull/8104) [logging] add query id to SQL Lab logs (#8104) (@serenajiang)
|
||||
- [#8098](https://github.com/apache/incubator-superset/pull/8098) [Hive Engine Spec] Fix latest partition logic (#8098) (@etr2460)
|
||||
- [#8099](https://github.com/apache/incubator-superset/pull/8099) Allow resizing width of SQL Lab left bar / editor (#8099) (@betodealmeida)
|
||||
- [#8106](https://github.com/apache/incubator-superset/pull/8106) Fix cache set kwargs (#8106) (@etr2460)
|
||||
- [#8105](https://github.com/apache/incubator-superset/pull/8105) Fix sqla query cache keys function (#8105) (@etr2460)
|
||||
- [#7569](https://github.com/apache/incubator-superset/pull/7569) refactor: separate vis-specific controls from centralized controls (#7569) (@felixcodes)
|
||||
- [#8100](https://github.com/apache/incubator-superset/pull/8100) docs: remove unused release instruction (#8100) (@mistercrunch)
|
||||
- [#8093](https://github.com/apache/incubator-superset/pull/8093) Move Darwin-specific JS libs to optional (#8093) (@friendzis)
|
||||
- [#8097](https://github.com/apache/incubator-superset/pull/8097) [fix] checks for stopped queries (#8097) (@serenajiang)
|
||||
- [#8087](https://github.com/apache/incubator-superset/pull/8087) fix: minor release/license related issues (#8087) (@mistercrunch)
|
||||
- [#8083](https://github.com/apache/incubator-superset/pull/8083) feat: replace table and box plot vis with new plugins (#8083) (@kristw)
|
||||
- [#8058](https://github.com/apache/incubator-superset/pull/8058) Add docstrings and typing to db_engine_specs and sql_parse (#8058) (@villebro)
|
||||
- [#7908](https://github.com/apache/incubator-superset/pull/7908) [Feature] Dashboard filter indicators (#7908) (@graceguo-supercat)
|
||||
- [#8078](https://github.com/apache/incubator-superset/pull/8078) Remove gevent installation as a separate layer (#8078) (@aspedrosa)
|
||||
- [#8067](https://github.com/apache/incubator-superset/pull/8067) remove idna req from setup.py (#8067) (@DiggidyDave)
|
||||
- [#8075](https://github.com/apache/incubator-superset/pull/8075) Move more Presto nested types behind feature flag (#8075) (@betodealmeida)
|
||||
- [#8051](https://github.com/apache/incubator-superset/pull/8051) bump a set of test-related JS libs (#8051) (@mistercrunch)
|
||||
- [#8035](https://github.com/apache/incubator-superset/pull/8035) Make package.json cross platform (#8035) (@fzzylogic)
|
||||
- [#8057](https://github.com/apache/incubator-superset/pull/8057) Adding support for filter box in annotations (#8057) (@KKcorps)
|
||||
- [#8042](https://github.com/apache/incubator-superset/pull/8042) Improve birth example dashboard (#8042) (@mistercrunch)
|
||||
- [#8040](https://github.com/apache/incubator-superset/pull/8040) [viz] Revert dropna logic for pivot tables (#8040) (@john-bodley)
|
||||
- [#8056](https://github.com/apache/incubator-superset/pull/8056) Add feature flag for Presto expand data (#8056) (@betodealmeida)
|
||||
- [#8039](https://github.com/apache/incubator-superset/pull/8039) Disable flaky cypress test (#8039) (@mistercrunch)
|
||||
- [#8045](https://github.com/apache/incubator-superset/pull/8045) Added WPSemantix to list of companies using Apache Superset in README.md file (#8045) (@semantiDan)
|
||||
- [#8032](https://github.com/apache/incubator-superset/pull/8032) Fix bug where lists in queries cannot contain multiple types (#8032) (@serenajiang)
|
||||
- [#8026](https://github.com/apache/incubator-superset/pull/8026) Two more organisations using Superset - Zalora, GFG (#8026) (@ksaagariconic)
|
||||
- [#8018](https://github.com/apache/incubator-superset/pull/8018) [Dashboard] Hide slice titles when they're empty (#8018) (@etr2460)
|
||||
- [#8016](https://github.com/apache/incubator-superset/pull/8016) Improve release instructions (#8016) (@mistercrunch)
|
||||
- [#8011](https://github.com/apache/incubator-superset/pull/8011) [SQL Lab] Improve perf for filterable table rendering (#8011) (@etr2460)
|
||||
- [#7822](https://github.com/apache/incubator-superset/pull/7822) Fix tooltips not visible for NVD3 charts on Firefox (#7822) (#7929) (@schoel-bis)
|
||||
- [#8003](https://github.com/apache/incubator-superset/pull/8003) Added better display of NULL values in FilterableTable (as in SQL Lab Results) (#8003) (@semantiDan)
|
||||
|
||||
### 0.34.1 (2019/09/27 18:22 +00:00)
|
||||
- [#8271](https://github.com/apache/incubator-superset/pull/8271) Fix: Missing time series datapoints in druid show up as absurd values in line chart (#8271) (@KKcorps)
|
||||
- [#8265](https://github.com/apache/incubator-superset/pull/8265) Fixed korean translation error. (#8265) (@kuckjwi0928)
|
||||
|
||||
@@ -22,6 +22,11 @@ under the License.
|
||||
You'll probably want to run these commands manually and understand what
|
||||
they do prior to doing so.
|
||||
|
||||
For coordinating on releases, on more operational topics that require more
|
||||
synchronous communications, we tend to use the `#apache-releases` channel
|
||||
on the Superset Slack. People crafting releases and those interested in
|
||||
partaking in the process should join the channel.
|
||||
|
||||
## Release setup
|
||||
|
||||
First you need to setup a few things. This is a one-off and doesn't
|
||||
|
||||
+6
-4
@@ -69,16 +69,18 @@ which adds missing non-nullable fields to the `datasources` table. Depending on
|
||||
the integrity of the data, manual intervention may be required.
|
||||
|
||||
* [5452](https://github.com/apache/incubator-superset/pull/5452): a change
|
||||
which adds missing non-nullable fields and uniqueness constraints to the
|
||||
`columns`and `table_columns` tables. Depending on the integrity of the data,
|
||||
manual intervention may be required.
|
||||
which adds missing non-nullable fields and uniqueness constraints (which may be
|
||||
case insensitive depending on your database configuration) to the `columns`and
|
||||
`table_columns` tables. Depending on the integrity of the data, manual
|
||||
intervention may be required.
|
||||
* `fabmanager` command line is deprecated since Flask-AppBuilder 2.0.0, use
|
||||
the new `flask fab <command>` integrated with *Flask cli*.
|
||||
* `SUPERSET_UPDATE_PERMS` environment variable was replaced by
|
||||
`FAB_UPDATE_PERMS` config boolean key. To disable automatic
|
||||
creation of permissions set `FAB_UPDATE_PERMS = False` on config.
|
||||
* [5453](https://github.com/apache/incubator-superset/pull/5453): a change
|
||||
which adds missing non-nullable fields and uniqueness constraints to the metrics
|
||||
which adds missing non-nullable fields and uniqueness constraints (which may be
|
||||
case insensitive depending on your database configuration) to the metrics
|
||||
and sql_metrics tables. Depending on the integrity of the data, manual
|
||||
intervention may be required.
|
||||
* [7616](https://github.com/apache/incubator-superset/pull/7616): this bug fix
|
||||
|
||||
+1
-1
@@ -96,7 +96,7 @@ Features
|
||||
Databases
|
||||
---------
|
||||
|
||||
The following RDBMS are currently suppored:
|
||||
The following RDBMS are currently supported:
|
||||
|
||||
- `Amazon Athena <https://aws.amazon.com/athena/>`_
|
||||
- `Amazon Redshift <https://aws.amazon.com/redshift/>`_
|
||||
|
||||
@@ -843,7 +843,7 @@ have the same configuration.
|
||||
|
||||
* To start a Celery worker to leverage the configuration run: ::
|
||||
|
||||
celery worker --app=superset.tasks.celery_app:app --pool=prefork -Ofair -c 4
|
||||
celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4
|
||||
|
||||
* To start a job which schedules periodic background jobs, run ::
|
||||
|
||||
@@ -888,6 +888,9 @@ cache store when upgrading an existing environment.
|
||||
entire setup. If not, background jobs can get scheduled multiple times
|
||||
resulting in weird behaviors like duplicate delivery of reports,
|
||||
higher than expected load / traffic etc.
|
||||
|
||||
* SQL Lab will only run your queries asynchronously if you enable
|
||||
"Asynchronous Query Execution" in your database settings.
|
||||
|
||||
|
||||
Email Reports
|
||||
|
||||
+1
-1
@@ -58,7 +58,7 @@ pathlib2==2.3.4
|
||||
polyline==1.4.0
|
||||
prison==0.1.2 # via flask-appbuilder
|
||||
py==1.8.0 # via retry
|
||||
pyarrow==0.14.1
|
||||
pyarrow==0.15.1
|
||||
pycparser==2.19 # via cffi
|
||||
pyjwt==1.7.1 # via flask-appbuilder, flask-jwt-extended
|
||||
pyrsistent==0.15.4 # via jsonschema
|
||||
|
||||
@@ -97,7 +97,7 @@ setup(
|
||||
"python-dateutil",
|
||||
"python-dotenv",
|
||||
"python-geohash",
|
||||
"pyarrow>=0.14.1, <0.15.0",
|
||||
"pyarrow>=0.15.1, <0.16.0",
|
||||
"pyyaml>=5.1",
|
||||
"retry>=0.9.2",
|
||||
"selenium>=3.141.0",
|
||||
@@ -115,15 +115,13 @@ setup(
|
||||
"mysql": ["mysqlclient==1.4.2.post1"],
|
||||
"postgres": ["psycopg2-binary==2.7.5"],
|
||||
"presto": ["pyhive[presto]>=0.4.0"],
|
||||
"druid": ["pydruid==0.5.2", "requests==2.22.0"],
|
||||
"druid": ["pydruid==0.5.7", "requests==2.22.0"],
|
||||
},
|
||||
python_requires="~=3.6",
|
||||
author="Apache Software Foundation",
|
||||
author_email="dev@superset.incubator.apache.org",
|
||||
url="https://superset.apache.org/",
|
||||
download_url=(
|
||||
"https://dist.apache.org/repos/dist/release/superset/" + version_string
|
||||
),
|
||||
download_url="https://www.apache.org/dist/incubator/superset/" + version_string,
|
||||
classifiers=[
|
||||
"Programming Language :: Python :: 3.6",
|
||||
"Programming Language :: Python :: 3.7",
|
||||
|
||||
@@ -164,7 +164,7 @@ describe('AdhocFilters', () => {
|
||||
.trigger('mousedown')
|
||||
.click();
|
||||
});
|
||||
|
||||
cy.get('.adhoc-filter-option').click({ force: true });
|
||||
cy.get('#filter-edit-popover').within(() => {
|
||||
cy.get('[data-test=adhoc-filter-simple-value]').within(() => {
|
||||
cy.get('div.select-input').click({ force: true });
|
||||
@@ -197,6 +197,7 @@ describe('AdhocFilters', () => {
|
||||
.click();
|
||||
});
|
||||
|
||||
cy.get('.adhoc-filter-option').click({ force: true });
|
||||
cy.get('#filter-edit-popover').within(() => {
|
||||
cy.get('#adhoc-filter-edit-tabs-tab-SQL').click();
|
||||
cy.get('.ace_content').click();
|
||||
|
||||
Generated
+315
-31
@@ -3800,9 +3800,9 @@
|
||||
}
|
||||
},
|
||||
"@superset-ui/legacy-plugin-chart-paired-t-test": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-plugin-chart-paired-t-test/-/legacy-plugin-chart-paired-t-test-0.11.0.tgz",
|
||||
"integrity": "sha512-XQjObNWy0lKQAwRP6hO6BrvAoaWZHe1IxfsAmELB7ikCYf7HSwWjWIR86JtLWQrz2fGq/42OH2A/pzuPATogkw==",
|
||||
"version": "0.11.11",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-plugin-chart-paired-t-test/-/legacy-plugin-chart-paired-t-test-0.11.11.tgz",
|
||||
"integrity": "sha512-mBi/3/AGQ6HM2tv+0h/NrcBDvN6/PV3OfZcxOGDZmxHFbkdjwX2obxf8MgVkvTlH6GcHEn2jfvtoIXOOwmQodA==",
|
||||
"requires": {
|
||||
"distributions": "^1.0.0",
|
||||
"prop-types": "^15.6.2",
|
||||
@@ -3875,13 +3875,13 @@
|
||||
}
|
||||
},
|
||||
"@superset-ui/legacy-plugin-chart-table": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-plugin-chart-table/-/legacy-plugin-chart-table-0.11.0.tgz",
|
||||
"integrity": "sha512-R9LuoWzR9YL/fkBq9iyFnzv3zwcNpU4D22zbtV1sx4EE/Rj2rN6u9glVruKcI4iwUwhakqFiTNMGlBph3A26xw==",
|
||||
"version": "0.11.4",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-plugin-chart-table/-/legacy-plugin-chart-table-0.11.4.tgz",
|
||||
"integrity": "sha512-yhzlBSVXNkiC4kkItDUPAYt1ZNDG1de3NhAr8m/ueN39RAsku0+O/3ZW3o2mQC0vjnjhREejUOqC5iaLAi3WQA==",
|
||||
"requires": {
|
||||
"d3": "^3.5.17",
|
||||
"datatables.net-bs": "^1.10.15",
|
||||
"dompurify": "^1.0.3",
|
||||
"dompurify": "^2.0.6",
|
||||
"prop-types": "^15.6.2"
|
||||
}
|
||||
},
|
||||
@@ -3950,14 +3950,14 @@
|
||||
}
|
||||
},
|
||||
"@superset-ui/legacy-preset-chart-nvd3": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-preset-chart-nvd3/-/legacy-preset-chart-nvd3-0.11.0.tgz",
|
||||
"integrity": "sha512-qZKQY/5myO0WKNm3e/8aa0Cp30e6t8fMTNBM+StD8CXfMhH9YEXLLGDyxLMVqY2Ytb2DszsmMLeldMAsdpneNA==",
|
||||
"version": "0.11.11",
|
||||
"resolved": "https://registry.npmjs.org/@superset-ui/legacy-preset-chart-nvd3/-/legacy-preset-chart-nvd3-0.11.11.tgz",
|
||||
"integrity": "sha512-IWNNHZDQ8l2SAOfGupDpOYok88w5bdwBEDUjXuMfM7usgWUCLeM642XRX6hYYCnnVtxdX2kfyGYnvdbC9VXU6w==",
|
||||
"requires": {
|
||||
"@data-ui/xy-chart": "^0.0.80",
|
||||
"@data-ui/xy-chart": "^0.0.84",
|
||||
"d3": "^3.5.17",
|
||||
"d3-tip": "^0.9.1",
|
||||
"dompurify": "^1.0.3",
|
||||
"dompurify": "^2.0.6",
|
||||
"fast-safe-stringify": "^2.0.6",
|
||||
"lodash": "^4.17.11",
|
||||
"mathjs": "^3.20.2",
|
||||
@@ -3965,6 +3965,290 @@
|
||||
"nvd3": "1.8.6",
|
||||
"prop-types": "^15.6.2",
|
||||
"urijs": "^1.18.10"
|
||||
},
|
||||
"dependencies": {
|
||||
"@data-ui/shared": {
|
||||
"version": "0.0.84",
|
||||
"resolved": "https://registry.npmjs.org/@data-ui/shared/-/shared-0.0.84.tgz",
|
||||
"integrity": "sha512-MsDLsFzBHFEREr/eF2/RX1o/cXioEg+VQTsM8gViW5ywGQ7Xo5+EqUOaBSrwqKAkvp3e8PaEZVkchPC54IBhrA==",
|
||||
"requires": {
|
||||
"@data-ui/theme": "^0.0.84",
|
||||
"@vx/event": "^0.0.165",
|
||||
"@vx/group": "^0.0.165",
|
||||
"@vx/shape": "^0.0.168",
|
||||
"@vx/tooltip": "0.0.165",
|
||||
"d3-array": "^1.2.1",
|
||||
"prop-types": "^15.5.10"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vx/shape": {
|
||||
"version": "0.0.168",
|
||||
"resolved": "https://registry.npmjs.org/@vx/shape/-/shape-0.0.168.tgz",
|
||||
"integrity": "sha512-urKZkwSafMpPQ0wI/L5FJmufRiAR4UsgYUCKxROjfE1Cf4jWNlK6mlVIIASxCdHlh9CGBbIrRMdl5Yv5lzqhjA==",
|
||||
"requires": {
|
||||
"@vx/curve": "0.0.165",
|
||||
"@vx/group": "0.0.165",
|
||||
"@vx/point": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"d3-path": "^1.0.5",
|
||||
"d3-shape": "^1.2.0",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"@data-ui/theme": {
|
||||
"version": "0.0.84",
|
||||
"resolved": "https://registry.npmjs.org/@data-ui/theme/-/theme-0.0.84.tgz",
|
||||
"integrity": "sha512-jIoHftC/5c/LVJYF4VSBjjVjrjc0yj4mLkGe8p0eVO7qUYKVvlWx7PrpM7ucyefvuAaKIwlr+Nh2xPGPdADjaA=="
|
||||
},
|
||||
"@data-ui/xy-chart": {
|
||||
"version": "0.0.84",
|
||||
"resolved": "https://registry.npmjs.org/@data-ui/xy-chart/-/xy-chart-0.0.84.tgz",
|
||||
"integrity": "sha512-4mRWEGfeQJ2kFXmQ81k1gDPx2zdkty6lt0+srui4zleSyhnBv1dmm9J03dq+qwr7+bpzjfq77nINV5HXWb31Bg==",
|
||||
"requires": {
|
||||
"@data-ui/shared": "^0.0.84",
|
||||
"@data-ui/theme": "^0.0.84",
|
||||
"@vx/axis": "^0.0.175",
|
||||
"@vx/curve": "^0.0.165",
|
||||
"@vx/event": "^0.0.165",
|
||||
"@vx/glyph": "^0.0.165",
|
||||
"@vx/gradient": "^0.0.165",
|
||||
"@vx/grid": "^0.0.180",
|
||||
"@vx/group": "^0.0.165",
|
||||
"@vx/pattern": "^0.0.165",
|
||||
"@vx/point": "^0.0.165",
|
||||
"@vx/responsive": "^0.0.192",
|
||||
"@vx/scale": "^0.0.165",
|
||||
"@vx/shape": "^0.0.165",
|
||||
"@vx/stats": "^0.0.165",
|
||||
"@vx/text": "^0.0.192",
|
||||
"@vx/threshold": "0.0.170",
|
||||
"@vx/tooltip": "^0.0.165",
|
||||
"@vx/voronoi": "^0.0.165",
|
||||
"d3-array": "^1.2.0",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"@vx/axis": {
|
||||
"version": "0.0.175",
|
||||
"resolved": "https://registry.npmjs.org/@vx/axis/-/axis-0.0.175.tgz",
|
||||
"integrity": "sha512-qVRIHurnbPnRF4p0KQITArOUSF564tWW1pc48giLz+DJGlcJ4H9RfOSTpV6rnnP15xto6pQdQehBgBAvFRmoig==",
|
||||
"requires": {
|
||||
"@vx/group": "0.0.170",
|
||||
"@vx/point": "0.0.165",
|
||||
"@vx/shape": "0.0.175",
|
||||
"@vx/text": "0.0.175",
|
||||
"classnames": "^2.2.5",
|
||||
"prop-types": "^15.6.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vx/group": {
|
||||
"version": "0.0.170",
|
||||
"resolved": "https://registry.npmjs.org/@vx/group/-/group-0.0.170.tgz",
|
||||
"integrity": "sha512-RnDdRoy0YI5hokk+YWXc8t39Kp51i4BdCpiwkDJU4YypGycTYnDFjicam6jigUmZ/6wyMirDf/aQboWviFLt2Q==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5"
|
||||
}
|
||||
},
|
||||
"@vx/shape": {
|
||||
"version": "0.0.175",
|
||||
"resolved": "https://registry.npmjs.org/@vx/shape/-/shape-0.0.175.tgz",
|
||||
"integrity": "sha512-bjAJoIIpKjUEPDV2xmTYGUvSvwRztv+6rd1c6NPZG/nIuqsMHFnFig/2xTcQJEQhRg6aKzvxIUo43zPSSq3fWA==",
|
||||
"requires": {
|
||||
"@vx/curve": "0.0.165",
|
||||
"@vx/group": "0.0.170",
|
||||
"@vx/point": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"d3-path": "^1.0.5",
|
||||
"d3-shape": "^1.2.0",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"@vx/text": {
|
||||
"version": "0.0.175",
|
||||
"resolved": "https://registry.npmjs.org/@vx/text/-/text-0.0.175.tgz",
|
||||
"integrity": "sha512-SOBhctXXAGhhpCOiTjxOM/8NDaDqGRk3OGfsJ714Mt1UJX6VQaKxFocZJwn6IMw3mNG6/p7O4Eao/gGDcoM6+A==",
|
||||
"requires": {
|
||||
"babel-plugin-lodash": "^3.3.2",
|
||||
"classnames": "^2.2.5",
|
||||
"lodash": "^4.17.4",
|
||||
"reduce-css-calc": "^1.3.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"@vx/bounds": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/bounds/-/bounds-0.0.165.tgz",
|
||||
"integrity": "sha512-ZvRb72/4QNs1ZrytZTZxd0hfAb/KKfhsdkcYtIQkmdF6dTsjigMQZ+h2bLvLnbZb/RxyCCoxdiZSGXd+T1c//Q==",
|
||||
"requires": {
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"@vx/curve": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/curve/-/curve-0.0.165.tgz",
|
||||
"integrity": "sha512-fiQAGrKNGjJbL+eixUckJqIZDWXH/1NtIyyDbSz3J7ksk0QpYr5BgWcNJN76HLNt7wfcLwNzCHeNs4iVYyFGTg==",
|
||||
"requires": {
|
||||
"d3-shape": "^1.0.6"
|
||||
}
|
||||
},
|
||||
"@vx/event": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/event/-/event-0.0.165.tgz",
|
||||
"integrity": "sha512-FsQiw0f3s5DQB6aBQmBcoWk9e4q65LcDobHIyV8qrmpW2QgV2NvQFM1w0Q300ohpRMgJDzGk68HHHQgFOJvApw==",
|
||||
"requires": {
|
||||
"@vx/point": "0.0.165"
|
||||
}
|
||||
},
|
||||
"@vx/glyph": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/glyph/-/glyph-0.0.165.tgz",
|
||||
"integrity": "sha512-kccUm40e/VCtayxqvcwc2K2M6oNXO7IafwIfw1RRv6Fj4Iutto9ZpI+PGOf/zPnYVueoLnWBXT/HE7IRS+C2gw==",
|
||||
"requires": {
|
||||
"@vx/group": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"d3-shape": "^1.2.0"
|
||||
}
|
||||
},
|
||||
"@vx/gradient": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/gradient/-/gradient-0.0.165.tgz",
|
||||
"integrity": "sha512-FjRXMTmcy7k0TWsfDzWWXw6T9WXKP+6LS/GRgnguq271pab/P+AdOJThsVxtBgUc8ZOAPbub3/2Gggz9d8tocg==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5",
|
||||
"prop-types": "^15.5.7"
|
||||
}
|
||||
},
|
||||
"@vx/grid": {
|
||||
"version": "0.0.180",
|
||||
"resolved": "https://registry.npmjs.org/@vx/grid/-/grid-0.0.180.tgz",
|
||||
"integrity": "sha512-+ugS0c6GbwHr6pFU0znnOG3/zTwRRadvWwj3E4ZOHmKUSz6ZEN6JNo+rD3WSZckYwLis6UivmYfJ5cV6AM4ufg==",
|
||||
"requires": {
|
||||
"@vx/group": "0.0.170",
|
||||
"@vx/point": "0.0.165",
|
||||
"@vx/shape": "0.0.179",
|
||||
"classnames": "^2.2.5",
|
||||
"prop-types": "^15.6.2"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vx/group": {
|
||||
"version": "0.0.170",
|
||||
"resolved": "https://registry.npmjs.org/@vx/group/-/group-0.0.170.tgz",
|
||||
"integrity": "sha512-RnDdRoy0YI5hokk+YWXc8t39Kp51i4BdCpiwkDJU4YypGycTYnDFjicam6jigUmZ/6wyMirDf/aQboWviFLt2Q==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5"
|
||||
}
|
||||
},
|
||||
"@vx/shape": {
|
||||
"version": "0.0.179",
|
||||
"resolved": "https://registry.npmjs.org/@vx/shape/-/shape-0.0.179.tgz",
|
||||
"integrity": "sha512-YHVNx4xGpbjolkW3Lb5pEgJB0+u349vfnLI976DJlinY0hRNa4TZbWXOB4ywLIrYzQEXXPMUR8WtdubNxg6g0w==",
|
||||
"requires": {
|
||||
"@vx/curve": "0.0.165",
|
||||
"@vx/group": "0.0.170",
|
||||
"@vx/point": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"d3-path": "^1.0.5",
|
||||
"d3-shape": "^1.2.0",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"@vx/group": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/group/-/group-0.0.165.tgz",
|
||||
"integrity": "sha512-gi1DSg8AAaVRseyWiq8y4bzyvKiQIXT6vDUYBVRmv2LBcpHocBGaxNiNK0X602RgLG0XmNyRv6qSCWLOaBs3Mg==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5"
|
||||
}
|
||||
},
|
||||
"@vx/pattern": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/pattern/-/pattern-0.0.165.tgz",
|
||||
"integrity": "sha512-h5nmfcYlQYYzNhlhqaYUvVnkmGnC0yWv5yU1snjHweGmIHTovV3RAbKgVFAP7kB3i2rbEtC3O8WkJN++cZdLzA==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"@vx/point": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/point/-/point-0.0.165.tgz",
|
||||
"integrity": "sha512-spoHilhjcWNgccrSzBUPw+PXV81tYxeyEWBkgr35aGVU4m7YT86Ywvfemwp7AVVGPn+XJHrhB0ujAhDoyqFPoA=="
|
||||
},
|
||||
"@vx/responsive": {
|
||||
"version": "0.0.192",
|
||||
"resolved": "https://registry.npmjs.org/@vx/responsive/-/responsive-0.0.192.tgz",
|
||||
"integrity": "sha512-HaXVwhSJXUfRbzRV+glxsX0ki2Hi1mdpz42iuGArVQgDPJEmBHjkXyoiXU8U6v66M7FAH+OyKgtc5j2bfhyYzA==",
|
||||
"requires": {
|
||||
"lodash": "^4.17.10",
|
||||
"prop-types": "^15.6.1",
|
||||
"resize-observer-polyfill": "1.5.0"
|
||||
}
|
||||
},
|
||||
"@vx/scale": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/scale/-/scale-0.0.165.tgz",
|
||||
"integrity": "sha512-5jSgXJDU6J/KWIyCbpjHqysPCddp7tG3LbTV7UmtB1Qleb4m4slShTVSE7+EKU+zgiQPDGm0+E2ht4cet+7F7A==",
|
||||
"requires": {
|
||||
"d3-scale": "^2.0.0"
|
||||
}
|
||||
},
|
||||
"@vx/shape": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/shape/-/shape-0.0.165.tgz",
|
||||
"integrity": "sha512-D9naH/glDtw8J8IcdumpRz1ihaoCAYMwFNh2KTv73HiTKrLQSXvIjwYFv9C0b8BCPNOXkDZS8s+AlgMSqGlZNQ==",
|
||||
"requires": {
|
||||
"@vx/curve": "0.0.165",
|
||||
"@vx/group": "0.0.165",
|
||||
"@vx/point": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"d3-path": "^1.0.5",
|
||||
"d3-shape": "^1.2.0",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"@vx/text": {
|
||||
"version": "0.0.192",
|
||||
"resolved": "https://registry.npmjs.org/@vx/text/-/text-0.0.192.tgz",
|
||||
"integrity": "sha512-lyy7eXfmQ8SJF7Qx+bCRcaEgvVSa18Lp6eRMo3GMANumUh9kSe7LwgqRFSdBJ85WkPqX+UOkJVyCH7AOlt0IWA==",
|
||||
"requires": {
|
||||
"classnames": "^2.2.5",
|
||||
"lodash": "^4.17.15",
|
||||
"prop-types": "^15.7.2",
|
||||
"reduce-css-calc": "^1.3.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"prop-types": {
|
||||
"version": "15.7.2",
|
||||
"resolved": "https://registry.npmjs.org/prop-types/-/prop-types-15.7.2.tgz",
|
||||
"integrity": "sha512-8QQikdH7//R2vurIJSutZ1smHYTcLpRWEOlHnzcWHmBYrOGUysKwSsrC89BCiFj3CbrfJ/nXFdJepOVrY1GCHQ==",
|
||||
"requires": {
|
||||
"loose-envify": "^1.4.0",
|
||||
"object-assign": "^4.1.1",
|
||||
"react-is": "^16.8.1"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"@vx/tooltip": {
|
||||
"version": "0.0.165",
|
||||
"resolved": "https://registry.npmjs.org/@vx/tooltip/-/tooltip-0.0.165.tgz",
|
||||
"integrity": "sha512-/x1NZc67QGQ4e/WNT7Ks5LYRyeLSqp8lG04gX5J6leUS0zscAVzo3aE5u65Qqbc0cnMyMPRZ2Qtb4klWTLg+eQ==",
|
||||
"requires": {
|
||||
"@vx/bounds": "0.0.165",
|
||||
"classnames": "^2.2.5",
|
||||
"prop-types": "^15.5.10"
|
||||
}
|
||||
},
|
||||
"react-is": {
|
||||
"version": "16.12.0",
|
||||
"resolved": "https://registry.npmjs.org/react-is/-/react-is-16.12.0.tgz",
|
||||
"integrity": "sha512-rPCkf/mWBtKc97aLL9/txD8DZdemK0vkA3JMLShjlJB3Pj3s+lpf1KaBzMfQrAmhMQB0n1cU/SUGgKKBCe837Q=="
|
||||
}
|
||||
}
|
||||
},
|
||||
"@superset-ui/number-format": {
|
||||
@@ -9219,9 +9503,9 @@
|
||||
}
|
||||
},
|
||||
"dompurify": {
|
||||
"version": "1.0.8",
|
||||
"resolved": "https://registry.npmjs.org/dompurify/-/dompurify-1.0.8.tgz",
|
||||
"integrity": "sha512-vetRFbN1SXSPfP3ClIiYnxTrXquSqakBEOoB5JESn0SVcSYzpu6ougjakpKnskGctYdlNpwf+riUHSkG7d4XUw=="
|
||||
"version": "2.0.7",
|
||||
"resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.0.7.tgz",
|
||||
"integrity": "sha512-S3O0lk6rFJtO01ZTzMollCOGg+WAtCwS3U5E2WSDY/x/sy7q70RjEC4Dmrih5/UqzLLB9XoKJ8KqwBxaNvBu4A=="
|
||||
},
|
||||
"domutils": {
|
||||
"version": "1.5.1",
|
||||
@@ -10399,9 +10683,9 @@
|
||||
"integrity": "sha1-PYpcZog6FqMMqGQ+hR8Zuqd5eRc="
|
||||
},
|
||||
"fast-safe-stringify": {
|
||||
"version": "2.0.6",
|
||||
"resolved": "https://registry.npmjs.org/fast-safe-stringify/-/fast-safe-stringify-2.0.6.tgz",
|
||||
"integrity": "sha512-q8BZ89jjc+mz08rSxROs8VsrBBcn1SIw1kq9NjolL509tkABRk9io01RAjSaEv1Xb2uFLt8VtRiZbGp5H8iDtg=="
|
||||
"version": "2.0.7",
|
||||
"resolved": "https://registry.npmjs.org/fast-safe-stringify/-/fast-safe-stringify-2.0.7.tgz",
|
||||
"integrity": "sha512-Utm6CdzT+6xsDk2m8S6uL8VHxNwI6Jub+e9NYTcAms28T84pTa25GJQV9j0CY0N1rM8hK4x6grpF2BQf+2qwVA=="
|
||||
},
|
||||
"fastparse": {
|
||||
"version": "1.1.2",
|
||||
@@ -11984,9 +12268,9 @@
|
||||
"dev": true
|
||||
},
|
||||
"handlebars": {
|
||||
"version": "4.3.1",
|
||||
"resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.3.1.tgz",
|
||||
"integrity": "sha512-c0HoNHzDiHpBt4Kqe99N8tdLPKAnGCQ73gYMPWtAYM4PwGnf7xl8PBUHJqh9ijlzt2uQKaSRxbXRt+rZ7M2/kA==",
|
||||
"version": "4.5.2",
|
||||
"resolved": "https://registry.npmjs.org/handlebars/-/handlebars-4.5.2.tgz",
|
||||
"integrity": "sha512-29Zxv/cynYB7mkT1rVWQnV7mGX6v7H/miQ6dbEpYTKq5eJBN7PsRB+ViYJlcT6JINTSu4dVB9kOqEun78h6Exg==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"neo-async": "^2.6.0",
|
||||
@@ -15742,9 +16026,9 @@
|
||||
"integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A=="
|
||||
},
|
||||
"lodash-es": {
|
||||
"version": "4.17.11",
|
||||
"resolved": "https://registry.npmjs.org/lodash-es/-/lodash-es-4.17.11.tgz",
|
||||
"integrity": "sha512-DHb1ub+rMjjrxqlB3H56/6MXtm1lSksDp2rA2cNWjG8mlDUYFhUj3Di2Zn5IwSU87xLv8tNIQ7sSwE/YOX/D/Q=="
|
||||
"version": "4.17.15",
|
||||
"resolved": "https://registry.npmjs.org/lodash-es/-/lodash-es-4.17.15.tgz",
|
||||
"integrity": "sha512-rlrc3yU3+JNOpZ9zj5pQtxnx2THmvRykwL4Xlxoa8I9lHBlVbbyPhgyPMioxVZ4NqyxaVVtaJnzsyOidQIhyyQ=="
|
||||
},
|
||||
"lodash.curry": {
|
||||
"version": "4.1.1",
|
||||
@@ -23738,20 +24022,20 @@
|
||||
"integrity": "sha512-T3PVJ6uz8i0HzPxOF9SWzWAlfN/DavlpQqepn22xgve/5QecC+XMCAtmUNnY7C9StehaV6exjUCI801lOI7QlQ=="
|
||||
},
|
||||
"uglify-js": {
|
||||
"version": "3.6.0",
|
||||
"resolved": "https://registry.npmjs.org/uglify-js/-/uglify-js-3.6.0.tgz",
|
||||
"integrity": "sha512-W+jrUHJr3DXKhrsS7NUVxn3zqMOFn0hL/Ei6v0anCIMoKC93TjcflTagwIHLW7SfMFfiQuktQyFVCFHGUE0+yg==",
|
||||
"version": "3.6.9",
|
||||
"resolved": "https://registry.npmjs.org/uglify-js/-/uglify-js-3.6.9.tgz",
|
||||
"integrity": "sha512-pcnnhaoG6RtrvHJ1dFncAe8Od6Nuy30oaJ82ts6//sGSXOP5UjBMEthiProjXmMNHOfd93sqlkztifFMcb+4yw==",
|
||||
"dev": true,
|
||||
"optional": true,
|
||||
"requires": {
|
||||
"commander": "~2.20.0",
|
||||
"commander": "~2.20.3",
|
||||
"source-map": "~0.6.1"
|
||||
},
|
||||
"dependencies": {
|
||||
"commander": {
|
||||
"version": "2.20.0",
|
||||
"resolved": "https://registry.npmjs.org/commander/-/commander-2.20.0.tgz",
|
||||
"integrity": "sha512-7j2y+40w61zy6YC2iRNpUe/NwhNyoXrYpHMrSunaMG64nRnaf96zO/KMQR4OyN/UnE5KLyEBnKHd4aG3rskjpQ==",
|
||||
"version": "2.20.3",
|
||||
"resolved": "https://registry.npmjs.org/commander/-/commander-2.20.3.tgz",
|
||||
"integrity": "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==",
|
||||
"dev": true,
|
||||
"optional": true
|
||||
},
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superset",
|
||||
"version": "0.999.0dev",
|
||||
"version": "0.35.2",
|
||||
"description": "Superset is a data exploration platform designed to be visual, intuitive, and interactive.",
|
||||
"license": "Apache-2.0",
|
||||
"directories": {
|
||||
@@ -64,20 +64,20 @@
|
||||
"@superset-ui/legacy-plugin-chart-iframe": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-map-box": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-markup": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-paired-t-test": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-paired-t-test": "^0.11.11",
|
||||
"@superset-ui/legacy-plugin-chart-parallel-coordinates": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-partition": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-pivot-table": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-rose": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-sankey": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-sunburst": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-table": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-table": "^0.11.4",
|
||||
"@superset-ui/legacy-plugin-chart-treemap": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-word-cloud": "^0.11.0",
|
||||
"@superset-ui/legacy-plugin-chart-world-map": "^0.11.0",
|
||||
"@superset-ui/legacy-preset-chart-big-number": "^0.11.0",
|
||||
"@superset-ui/legacy-preset-chart-deckgl": "^0.1.0",
|
||||
"@superset-ui/legacy-preset-chart-nvd3": "^0.11.0",
|
||||
"@superset-ui/legacy-preset-chart-nvd3": "^0.11.11",
|
||||
"@superset-ui/number-format": "^0.12.1",
|
||||
"@superset-ui/plugin-chart-table": "^0.11.0",
|
||||
"@superset-ui/preset-chart-xy": "^0.11.0",
|
||||
@@ -98,12 +98,13 @@
|
||||
"d3-color": "^1.2.0",
|
||||
"d3-scale": "^2.1.2",
|
||||
"dnd-core": "^2.6.0",
|
||||
"dompurify": "^1.0.3",
|
||||
"dompurify": "^2.0.7",
|
||||
"geolib": "^2.0.24",
|
||||
"immutable": "^3.8.2",
|
||||
"jquery": "^3.4.1",
|
||||
"json-bigint": "^0.3.0",
|
||||
"lodash": "^4.17.15",
|
||||
"lodash-es": "^4.17.14",
|
||||
"mathjs": "^3.20.2",
|
||||
"moment": "^2.20.1",
|
||||
"mousetrap": "^1.6.1",
|
||||
|
||||
@@ -91,6 +91,7 @@ describe('DatasourceEditor', () => {
|
||||
});
|
||||
|
||||
it('renders isSqla fields', () => {
|
||||
wrapper.setState({ activeTabKey: 4 });
|
||||
expect(wrapper.state('isSqla')).toBe(true);
|
||||
expect(wrapper.find(Field).find({ fieldKey: 'fetch_values_predicate' }).exists()).toBe(true);
|
||||
});
|
||||
|
||||
@@ -177,7 +177,7 @@ export function estimateQueryCost(query) {
|
||||
dispatch({ type: COST_ESTIMATE_STARTED, query }),
|
||||
SupersetClient.post({
|
||||
endpoint,
|
||||
postPayload: { sql, templateParams: JSON.parse(templateParams) },
|
||||
postPayload: { sql, templateParams: JSON.parse(templateParams || '{}') },
|
||||
})
|
||||
.then(({ json }) => dispatch({ type: COST_ESTIMATE_RETURNED, query, json }))
|
||||
.catch(response =>
|
||||
|
||||
@@ -170,10 +170,13 @@ export default class FilterableTable extends PureComponent {
|
||||
).map(dimension => dimension.width);
|
||||
|
||||
this.props.orderedColumnKeys.forEach((key, index) => {
|
||||
widthsByColumnKey[key] = Math.max(...colWidths.slice(
|
||||
// we can't use Math.max(...colWidths.slice(...)) here since the number
|
||||
// of elements might be bigger than the number of allowed arguments in a
|
||||
// Javascript function
|
||||
widthsByColumnKey[key] = colWidths.slice(
|
||||
index * (this.list.size + 1),
|
||||
(index + 1) * (this.list.size + 1),
|
||||
)) + PADDING;
|
||||
).reduce((a, b) => Math.max(a, b)) + PADDING;
|
||||
});
|
||||
|
||||
return widthsByColumnKey;
|
||||
|
||||
@@ -565,30 +565,20 @@ export class DatasourceEditor extends React.PureComponent {
|
||||
}
|
||||
|
||||
render() {
|
||||
const datasource = this.state.datasource;
|
||||
const { datasource, activeTabKey } = this.state;
|
||||
return (
|
||||
<div className="Datasource">
|
||||
{this.renderErrors()}
|
||||
<Tabs
|
||||
id="table-tabs"
|
||||
onSelect={this.handleTabSelect}
|
||||
defaultActiveKey={1}
|
||||
defaultActiveKey={activeTabKey}
|
||||
>
|
||||
<Tab eventKey={1} title={t('Settings')}>
|
||||
{this.state.activeTabKey === 1 &&
|
||||
<div>
|
||||
<Col md={6}>
|
||||
<FormContainer>
|
||||
{this.renderSettingsFieldset()}
|
||||
</FormContainer>
|
||||
</Col>
|
||||
<Col md={6}>
|
||||
<FormContainer>
|
||||
{this.renderAdvancedFieldset()}
|
||||
</FormContainer>
|
||||
</Col>
|
||||
</div>
|
||||
}
|
||||
<Tab
|
||||
title={<CollectionTabTitle collection={datasource.metrics} title={t('Metrics')} />}
|
||||
eventKey={1}
|
||||
>
|
||||
{activeTabKey === 1 && this.renderMetricCollection()}
|
||||
</Tab>
|
||||
<Tab
|
||||
title={
|
||||
@@ -596,7 +586,7 @@ export class DatasourceEditor extends React.PureComponent {
|
||||
}
|
||||
eventKey={2}
|
||||
>
|
||||
{this.state.activeTabKey === 2 &&
|
||||
{activeTabKey === 2 &&
|
||||
<div>
|
||||
<ColumnCollectionTable
|
||||
columns={this.state.databaseColumns}
|
||||
@@ -623,7 +613,7 @@ export class DatasourceEditor extends React.PureComponent {
|
||||
/>}
|
||||
eventKey={3}
|
||||
>
|
||||
{this.state.activeTabKey === 3 &&
|
||||
{activeTabKey === 3 &&
|
||||
<ColumnCollectionTable
|
||||
columns={this.state.calculatedColumns}
|
||||
onChange={calculatedColumns => this.setColumns({ calculatedColumns })}
|
||||
@@ -641,11 +631,25 @@ export class DatasourceEditor extends React.PureComponent {
|
||||
/>
|
||||
}
|
||||
</Tab>
|
||||
<Tab
|
||||
title={<CollectionTabTitle collection={datasource.metrics} title={t('Metrics')} />}
|
||||
eventKey={4}
|
||||
>
|
||||
{this.state.activeTabKey === 4 && this.renderMetricCollection()}
|
||||
<Tab eventKey={4} title={t('Settings')}>
|
||||
{activeTabKey === 4 &&
|
||||
<div>
|
||||
<div className="change-warning well">
|
||||
<span className="bold">{t('Be careful.')} </span>
|
||||
{t('Changing these settings will affect all charts using this datasource, including charts owned by other people.')}
|
||||
</div>
|
||||
<Col md={6}>
|
||||
<FormContainer>
|
||||
{this.renderSettingsFieldset()}
|
||||
</FormContainer>
|
||||
</Col>
|
||||
<Col md={6}>
|
||||
<FormContainer>
|
||||
{this.renderAdvancedFieldset()}
|
||||
</FormContainer>
|
||||
</Col>
|
||||
</div>
|
||||
}
|
||||
</Tab>
|
||||
</Tabs>
|
||||
</div>
|
||||
|
||||
@@ -20,3 +20,12 @@
|
||||
height: 600px;
|
||||
overflow: auto;
|
||||
}
|
||||
|
||||
.Datasource .change-warning {
|
||||
margin: 16px 10px 0;
|
||||
color: #FE4A49;
|
||||
}
|
||||
|
||||
.Datasource .change-warning .bold {
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
@@ -43,7 +43,7 @@ export default class AdhocFilterOption extends React.PureComponent {
|
||||
this.onPopoverResize = this.onPopoverResize.bind(this);
|
||||
this.onOverlayEntered = this.onOverlayEntered.bind(this);
|
||||
this.onOverlayExited = this.onOverlayExited.bind(this);
|
||||
this.state = { overlayShown: !this.props.adhocFilter.fromFormData };
|
||||
this.state = { overlayShown: false };
|
||||
}
|
||||
|
||||
onPopoverResize() {
|
||||
@@ -88,7 +88,6 @@ export default class AdhocFilterOption extends React.PureComponent {
|
||||
overlay={overlay}
|
||||
rootClose
|
||||
shouldUpdatePosition
|
||||
defaultOverlayShown={!adhocFilter.fromFormData}
|
||||
onEntered={this.onOverlayEntered}
|
||||
onExited={this.onOverlayExited}
|
||||
>
|
||||
|
||||
@@ -39,7 +39,7 @@ export default class AdhocMetricOption extends React.PureComponent {
|
||||
this.onOverlayEntered = this.onOverlayEntered.bind(this);
|
||||
this.onOverlayExited = this.onOverlayExited.bind(this);
|
||||
this.onPopoverResize = this.onPopoverResize.bind(this);
|
||||
this.state = { overlayShown: !this.props.adhocMetric.fromFormData };
|
||||
this.state = { overlayShown: false };
|
||||
}
|
||||
|
||||
onPopoverResize() {
|
||||
@@ -47,7 +47,7 @@ export default class AdhocMetricOption extends React.PureComponent {
|
||||
}
|
||||
|
||||
onOverlayEntered() {
|
||||
this.setState({ overlayShown: true });
|
||||
this.setState({ overlayShown: false });
|
||||
}
|
||||
|
||||
onOverlayExited() {
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
#datasource_menu {
|
||||
border-radius: 2px;
|
||||
padding-left: 8px;
|
||||
padding-right: 8px;
|
||||
}
|
||||
|
||||
#datasource_menu .caret {
|
||||
position: relative;
|
||||
padding-right: 8px;
|
||||
margin-left: 4px;
|
||||
color: #fff;
|
||||
top: -8px;
|
||||
}
|
||||
|
||||
#datasource_menu + ul {
|
||||
margin-top: 26px;
|
||||
}
|
||||
@@ -36,6 +36,8 @@ import ColumnOption from '../../../components/ColumnOption';
|
||||
import MetricOption from '../../../components/MetricOption';
|
||||
import DatasourceModal from '../../../datasource/DatasourceModal';
|
||||
import ChangeDatasourceModal from '../../../datasource/ChangeDatasourceModal';
|
||||
import TooltipWrapper from '../../../components/TooltipWrapper';
|
||||
import './DatasourceControl.css';
|
||||
|
||||
const propTypes = {
|
||||
onChange: PropTypes.func,
|
||||
@@ -115,56 +117,45 @@ class DatasourceControl extends React.PureComponent {
|
||||
}
|
||||
|
||||
render() {
|
||||
const { menuExpanded, showChangeDatasourceModal, showEditDatasourceModal } = this.state;
|
||||
const { showChangeDatasourceModal, showEditDatasourceModal } = this.state;
|
||||
const { datasource, onChange, onDatasourceSave, value } = this.props;
|
||||
return (
|
||||
<div>
|
||||
<ControlHeader {...this.props} />
|
||||
<div className="btn-group label-dropdown">
|
||||
<OverlayTrigger
|
||||
placement="right"
|
||||
overlay={
|
||||
<Tooltip id={'error-tooltip'}>{t('Click to edit the datasource')}</Tooltip>
|
||||
}
|
||||
<TooltipWrapper
|
||||
label="change-datasource"
|
||||
tooltip={t('Click to change the datasource')}
|
||||
>
|
||||
<div className="btn-group">
|
||||
<Label onClick={this.toggleEditDatasourceModal} className="label-btn-label">
|
||||
{datasource.name}
|
||||
</Label>
|
||||
</div>
|
||||
</OverlayTrigger>
|
||||
<DropdownButton
|
||||
noCaret
|
||||
title={
|
||||
<span>
|
||||
<i className={`float-right expander fa fa-angle-${menuExpanded ? 'up' : 'down'}`} />
|
||||
</span>}
|
||||
className="label label-btn m-r-5"
|
||||
bsSize="sm"
|
||||
id="datasource_menu"
|
||||
>
|
||||
<MenuItem
|
||||
eventKey="3"
|
||||
onClick={this.toggleEditDatasourceModal}
|
||||
<DropdownButton
|
||||
title={datasource.name}
|
||||
className="label label-default label-btn m-r-5"
|
||||
bsSize="sm"
|
||||
id="datasource_menu"
|
||||
>
|
||||
{t('Edit Datasource')}
|
||||
</MenuItem>
|
||||
{datasource.type === 'table' &&
|
||||
<MenuItem
|
||||
eventKey="3"
|
||||
href={`/superset/sqllab?datasourceKey=${value}`}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
onClick={this.toggleChangeDatasourceModal}
|
||||
>
|
||||
{t('Explore in SQL Lab')}
|
||||
</MenuItem>}
|
||||
<MenuItem
|
||||
eventKey="3"
|
||||
onClick={this.toggleChangeDatasourceModal}
|
||||
>
|
||||
{t('Change Datasource')}
|
||||
</MenuItem>
|
||||
</DropdownButton>
|
||||
{t('Change Datasource')}
|
||||
</MenuItem>
|
||||
{datasource.type === 'table' &&
|
||||
<MenuItem
|
||||
eventKey="3"
|
||||
href={`/superset/sqllab?datasourceKey=${value}`}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
>
|
||||
{t('Explore in SQL Lab')}
|
||||
</MenuItem>}
|
||||
<MenuItem
|
||||
eventKey="3"
|
||||
onClick={this.toggleEditDatasourceModal}
|
||||
>
|
||||
{t('Edit Datasource')}
|
||||
</MenuItem>
|
||||
</DropdownButton>
|
||||
</TooltipWrapper>
|
||||
<OverlayTrigger
|
||||
placement="right"
|
||||
overlay={
|
||||
|
||||
@@ -78,9 +78,7 @@ class QueryObject:
|
||||
)
|
||||
self.is_timeseries = is_timeseries
|
||||
self.time_range = time_range
|
||||
self.time_shift = (
|
||||
utils.parse_human_timedelta(time_shift) if time_shift else None
|
||||
)
|
||||
self.time_shift = utils.parse_human_timedelta(time_shift)
|
||||
self.groupby = groupby or []
|
||||
|
||||
# Temporal solution for backward compatability issue
|
||||
|
||||
+1
-1
@@ -279,7 +279,7 @@ SUPERSET_WEBSERVER_DOMAINS = None
|
||||
|
||||
# Allowed format types for upload on Database view
|
||||
# TODO: Add processing of other spreadsheet formats (xls, xlsx etc)
|
||||
ALLOWED_EXTENSIONS = set(["csv"])
|
||||
ALLOWED_EXTENSIONS = {"csv", "tsv"}
|
||||
|
||||
# CSV Options: key/value pairs that will be passed as argument to DataFrame.to_csv
|
||||
# method.
|
||||
|
||||
@@ -167,7 +167,7 @@ class DruidCluster(Model, AuditMixinNullable, ImportMixin):
|
||||
base_url = self.get_base_url(self.broker_host, self.broker_port)
|
||||
return f"{base_url}/{self.broker_endpoint}"
|
||||
|
||||
def get_pydruid_client(self) -> PyDruid:
|
||||
def get_pydruid_client(self) -> "PyDruid":
|
||||
cli = PyDruid(
|
||||
self.get_base_url(self.broker_host, self.broker_port), self.broker_endpoint
|
||||
)
|
||||
@@ -829,7 +829,7 @@ class DruidDatasource(Model, BaseDatasource):
|
||||
return granularity
|
||||
|
||||
@staticmethod
|
||||
def get_post_agg(mconf: Dict) -> Postaggregator:
|
||||
def get_post_agg(mconf: Dict) -> "Postaggregator":
|
||||
"""
|
||||
For a metric specified as `postagg` returns the
|
||||
kind of post aggregation for pydruid.
|
||||
@@ -1441,7 +1441,7 @@ class DruidDatasource(Model, BaseDatasource):
|
||||
return (col, extraction_fn)
|
||||
|
||||
@classmethod
|
||||
def get_filters(cls, raw_filters, num_cols, columns_dict) -> Filter:
|
||||
def get_filters(cls, raw_filters, num_cols, columns_dict) -> "Filter":
|
||||
"""Given Superset filter data structure, returns pydruid Filter(s)"""
|
||||
filters = None
|
||||
for flt in raw_filters:
|
||||
@@ -1559,9 +1559,9 @@ class DruidDatasource(Model, BaseDatasource):
|
||||
alphaNumeric=is_numeric_col,
|
||||
)
|
||||
elif op == "IS NULL":
|
||||
cond = Dimension(col) is None
|
||||
cond = Filter(dimension=col, value="")
|
||||
elif op == "IS NOT NULL":
|
||||
cond = Dimension(col) is not None
|
||||
cond = ~Filter(dimension=col, value="")
|
||||
|
||||
if filters:
|
||||
filters = Filter(type="and", fields=[cond, filters])
|
||||
@@ -1570,7 +1570,7 @@ class DruidDatasource(Model, BaseDatasource):
|
||||
|
||||
return filters
|
||||
|
||||
def _get_having_obj(self, col: str, op: str, eq: str) -> Having:
|
||||
def _get_having_obj(self, col: str, op: str, eq: str) -> "Having":
|
||||
cond = None
|
||||
if op == "==":
|
||||
if col in self.column_names:
|
||||
@@ -1584,7 +1584,7 @@ class DruidDatasource(Model, BaseDatasource):
|
||||
|
||||
return cond
|
||||
|
||||
def get_having_filters(self, raw_filters: List[Dict]) -> Having:
|
||||
def get_having_filters(self, raw_filters: List[Dict]) -> "Having":
|
||||
filters = None
|
||||
reversed_op_map = {"!=": "==", ">=": "<", "<=": ">"}
|
||||
|
||||
|
||||
@@ -239,6 +239,8 @@ class DruidClusterModelView(SupersetModelView, DeleteMixin, YamlExportMixin):
|
||||
),
|
||||
}
|
||||
|
||||
yaml_dict_key = "databases"
|
||||
|
||||
edit_form_extra_fields = {
|
||||
"cluster_name": QuerySelectField(
|
||||
"Cluster",
|
||||
|
||||
@@ -37,7 +37,7 @@ from sqlalchemy.sql.expression import ColumnClause, ColumnElement, Select, TextA
|
||||
from sqlalchemy.types import TypeEngine
|
||||
from werkzeug.utils import secure_filename
|
||||
|
||||
from superset import app, db, sql_parse
|
||||
from superset import app, sql_parse
|
||||
from superset.utils import core as utils
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -388,15 +388,17 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
df.to_sql(**kwargs)
|
||||
|
||||
@classmethod
|
||||
def create_table_from_csv(cls, form, table):
|
||||
""" Create table (including metadata in backend) from contents of a csv.
|
||||
def create_table_from_csv(cls, form) -> None:
|
||||
"""
|
||||
Create table from contents of a csv. Note: this method does not create
|
||||
metadata for the table.
|
||||
|
||||
:param form: Parameters defining how to process data
|
||||
:param table: Metadata of new table to be created
|
||||
"""
|
||||
|
||||
def _allowed_file(filename: str) -> bool:
|
||||
# Only allow specific file extensions as specified in the config
|
||||
extension = os.path.splitext(filename)[1]
|
||||
extension = os.path.splitext(filename)[1].lower()
|
||||
return (
|
||||
extension is not None and extension[1:] in config["ALLOWED_EXTENSIONS"]
|
||||
)
|
||||
@@ -432,12 +434,6 @@ class BaseEngineSpec: # pylint: disable=too-many-public-methods
|
||||
}
|
||||
cls.df_to_sql(**df_to_sql_kwargs)
|
||||
|
||||
table.user_id = g.user.id
|
||||
table.schema = form.schema.data
|
||||
table.fetch_metadata()
|
||||
db.session.add(table)
|
||||
db.session.commit()
|
||||
|
||||
@classmethod
|
||||
def convert_dttm(cls, target_type: str, dttm: datetime) -> str:
|
||||
"""
|
||||
|
||||
@@ -98,7 +98,7 @@ class HiveEngineSpec(PrestoEngineSpec):
|
||||
return []
|
||||
|
||||
@classmethod
|
||||
def create_table_from_csv(cls, form, table): # pylint: disable=too-many-locals
|
||||
def create_table_from_csv(cls, form) -> None: # pylint: disable=too-many-locals
|
||||
"""Uploads a csv file and creates a superset datasource in Hive."""
|
||||
|
||||
def convert_to_hive_type(col_type):
|
||||
|
||||
@@ -327,7 +327,7 @@ def load_world_bank_health_n_pop(only_metadata=False, force=False):
|
||||
merge_slice(slc)
|
||||
|
||||
print("Creating a World's Health Bank dashboard")
|
||||
dash_name = "World's Bank Data"
|
||||
dash_name = "World Bank's Data"
|
||||
slug = "world_health"
|
||||
dash = db.session.query(Dash).filter_by(slug=slug).first()
|
||||
|
||||
|
||||
@@ -571,6 +571,7 @@ class Dashboard(Model, AuditMixinNullable, ImportMixin):
|
||||
slices = copy(dashboard_to_import.slices)
|
||||
old_to_new_slc_id_dict = {}
|
||||
new_filter_immune_slices = []
|
||||
new_filter_immune_slice_fields = {}
|
||||
new_timed_refresh_immune_slices = []
|
||||
new_expanded_slices = {}
|
||||
i_params_dict = dashboard_to_import.params_dict
|
||||
@@ -596,6 +597,13 @@ class Dashboard(Model, AuditMixinNullable, ImportMixin):
|
||||
and old_slc_id_str in i_params_dict["filter_immune_slices"]
|
||||
):
|
||||
new_filter_immune_slices.append(new_slc_id_str)
|
||||
if (
|
||||
"filter_immune_slice_fields" in i_params_dict
|
||||
and old_slc_id_str in i_params_dict["filter_immune_slice_fields"]
|
||||
):
|
||||
new_filter_immune_slice_fields[new_slc_id_str] = i_params_dict[
|
||||
"filter_immune_slice_fields"
|
||||
][old_slc_id_str]
|
||||
if (
|
||||
"timed_refresh_immune_slices" in i_params_dict
|
||||
and old_slc_id_str in i_params_dict["timed_refresh_immune_slices"]
|
||||
@@ -632,6 +640,10 @@ class Dashboard(Model, AuditMixinNullable, ImportMixin):
|
||||
dashboard_to_import.alter_params(
|
||||
filter_immune_slices=new_filter_immune_slices
|
||||
)
|
||||
if new_filter_immune_slice_fields:
|
||||
dashboard_to_import.alter_params(
|
||||
filter_immune_slice_fields=new_filter_immune_slice_fields
|
||||
)
|
||||
if new_timed_refresh_immune_slices:
|
||||
dashboard_to_import.alter_params(
|
||||
timed_refresh_immune_slices=new_timed_refresh_immune_slices
|
||||
|
||||
@@ -4669,7 +4669,7 @@ msgid "CSV to Database configuration"
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:375
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:401 superset/views/core.py:667
|
||||
@@ -7765,4 +7765,3 @@ msgstr ""
|
||||
|
||||
#~ msgid "Saved Queries"
|
||||
#~ msgstr ""
|
||||
|
||||
|
||||
@@ -3124,7 +3124,7 @@
|
||||
"CSV to Database configuration": [
|
||||
""
|
||||
],
|
||||
"CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\"": [
|
||||
"CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\"": [
|
||||
""
|
||||
],
|
||||
"User": [
|
||||
|
||||
@@ -4880,8 +4880,8 @@ msgid "CSV to Database configuration"
|
||||
msgstr "CSV vers la configuration de la base de données"
|
||||
|
||||
#: superset/views/core.py:375
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgstr "Fichier CSV \"{0}\" chargé dans la table \"{1}\" de la base de données \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr "Fichier CSV \"%(csv_filename)s\" chargé dans la table \"%(table_name)s\" de la base de données \"%(db_name)s\""
|
||||
|
||||
#: superset/views/core.py:401 superset/views/core.py:667
|
||||
#: superset/views/sql_lab.py:23 superset/views/sql_lab.py:60
|
||||
|
||||
@@ -2923,7 +2923,7 @@
|
||||
"CSV to Database configuration": [
|
||||
""
|
||||
],
|
||||
"CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\"": [
|
||||
"CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\"": [
|
||||
""
|
||||
],
|
||||
"User": [
|
||||
@@ -3132,4 +3132,4 @@
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4432,7 +4432,7 @@ msgid "CSV to Database configuration"
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:363
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:389 superset/views/core.py:655
|
||||
@@ -6461,4 +6461,3 @@ msgstr "Query salvate"
|
||||
|
||||
#~ msgid "Slice"
|
||||
#~ msgstr "Slice"
|
||||
|
||||
|
||||
@@ -3788,7 +3788,7 @@ msgstr "대시보드 가져오기"
|
||||
msgid "CSV to Database configuration"
|
||||
msgstr ""
|
||||
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr ""
|
||||
|
||||
msgid "User"
|
||||
|
||||
@@ -4669,7 +4669,7 @@ msgid "CSV to Database configuration"
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:375
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr ""
|
||||
|
||||
#: superset/views/core.py:401 superset/views/core.py:667
|
||||
@@ -4967,4 +4967,3 @@ msgstr ""
|
||||
#: superset/views/sql_lab.py:86
|
||||
msgid "Saved Queries"
|
||||
msgstr ""
|
||||
|
||||
|
||||
@@ -2923,8 +2923,8 @@
|
||||
"CSV to Database configuration": [
|
||||
"Настройка CSV для БД"
|
||||
],
|
||||
"CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\"": [
|
||||
"CSV-файл \"{0}\" загружен в таблицу \"{1}\" базы данных \"{2}\""
|
||||
"CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\"": [
|
||||
"CSV-файл \"%(csv_filename)s\" загружен в таблицу \"%(table_name)s\" базы данных \"%(db_name)s\""
|
||||
],
|
||||
"User": [
|
||||
"Пользователь"
|
||||
@@ -3132,4 +3132,4 @@
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4544,8 +4544,8 @@ msgid "CSV to Database configuration"
|
||||
msgstr "Настройка CSV для БД"
|
||||
|
||||
#: superset/views/core.py:363
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgstr "CSV-файл \"{0}\" загружен в таблицу \"{1}\" базы данных \"{2}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr "CSV-файл \"%(csv_filename)s\" загружен в таблицу \"%(table_name)s\" базы данных \"%(db_name)s\""
|
||||
|
||||
#: superset/views/core.py:389 superset/views/core.py:655
|
||||
#: superset/views/sql_lab.py:16 superset/views/sql_lab.py:53
|
||||
|
||||
@@ -3124,8 +3124,8 @@
|
||||
"CSV to Database configuration": [
|
||||
"csv 到数据库配置"
|
||||
],
|
||||
"CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\"": [
|
||||
"csv 文件 \"{0}\" 上传到数据库 \"{2}\" 中的表 \"{1}\""
|
||||
"CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\"": [
|
||||
"csv 文件 \"%(csv_filename)s\" 上传到数据库 \"%(db_name)s\" 中的表 \"%(table_name)s\""
|
||||
],
|
||||
"User": [
|
||||
"用户"
|
||||
|
||||
@@ -4746,8 +4746,8 @@ msgid "CSV to Database configuration"
|
||||
msgstr "csv 到数据库配置"
|
||||
|
||||
#: superset/views/core.py:375
|
||||
msgid "CSV file \"{0}\" uploaded to table \"{1}\" in database \"{2}\""
|
||||
msgstr "csv 文件 \"{0}\" 上传到数据库 \"{2}\" 中的表 \"{1}\""
|
||||
msgid "CSV file \"%(csv_filename)s\" uploaded to table \"%(table_name)s\" in database \"%(db_name)s\""
|
||||
msgstr "csv 文件 \"%(csv_filename)s\" 上传到数据库 \"%(db_name)s\" 中的表 \"%(table_name)s\""
|
||||
|
||||
#: superset/views/core.py:401 superset/views/core.py:667
|
||||
#: superset/views/sql_lab.py:23 superset/views/sql_lab.py:60
|
||||
|
||||
@@ -291,7 +291,7 @@ class DashboardEncoder(json.JSONEncoder):
|
||||
return json.JSONEncoder(sort_keys=True).default(self, o)
|
||||
|
||||
|
||||
def parse_human_timedelta(s: str) -> timedelta:
|
||||
def parse_human_timedelta(s: Optional[str]) -> timedelta:
|
||||
"""
|
||||
Returns ``datetime.datetime`` from natural language time deltas
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ def import_datasource(
|
||||
"""Imports the datasource from the object to the database.
|
||||
|
||||
Metrics and columns and datasource will be overrided if exists.
|
||||
This function can be used to import/export dashboards between multiple
|
||||
This function can be used to import/export datasources between multiple
|
||||
superset instances. Audit metadata isn't copies over.
|
||||
"""
|
||||
make_transient(i_datasource)
|
||||
|
||||
@@ -19,7 +19,7 @@ import functools
|
||||
import logging
|
||||
import traceback
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
import simplejson as json
|
||||
import yaml
|
||||
@@ -207,12 +207,20 @@ def validate_json(form, field):
|
||||
|
||||
|
||||
class YamlExportMixin(object):
|
||||
yaml_dict_key: Optional[str] = None
|
||||
"""
|
||||
Override this if you want a dict response instead, with a certain key.
|
||||
Used on DatabaseView for cli compatibility
|
||||
"""
|
||||
|
||||
@action("yaml_export", __("Export to YAML"), __("Export to YAML?"), "fa-download")
|
||||
def yaml_export(self, items):
|
||||
if not isinstance(items, list):
|
||||
items = [items]
|
||||
|
||||
data = [t.export_to_dict() for t in items]
|
||||
if self.yaml_dict_key:
|
||||
data = {self.yaml_dict_key: data}
|
||||
return Response(
|
||||
yaml.safe_dump(data),
|
||||
headers=generate_download_headers("yaml"),
|
||||
|
||||
@@ -1111,7 +1111,8 @@ class Superset(BaseSupersetView):
|
||||
),
|
||||
"danger",
|
||||
)
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
logging.exception(e)
|
||||
flash(
|
||||
_(
|
||||
"An unknown error occurred. "
|
||||
@@ -2911,7 +2912,6 @@ class Superset(BaseSupersetView):
|
||||
return self.render_template(
|
||||
"superset/basic.html",
|
||||
entry="welcome",
|
||||
title="Superset",
|
||||
bootstrap_data=json.dumps(payload, default=utils.json_iso_dttm_ser),
|
||||
)
|
||||
|
||||
@@ -2999,7 +2999,7 @@ class Superset(BaseSupersetView):
|
||||
except Exception:
|
||||
return json_error_response(
|
||||
"Failed to fetch schemas allowed for csv upload in this database! "
|
||||
"Please contact Superset Admin!",
|
||||
"Please contact your Superset Admin!",
|
||||
stacktrace=utils.get_stacktrace(),
|
||||
)
|
||||
|
||||
|
||||
@@ -52,6 +52,8 @@ class DatabaseRestApi(DatabaseMixin, ModelRestApi):
|
||||
"allows_cost_estimate",
|
||||
"backend",
|
||||
]
|
||||
show_columns = list_columns
|
||||
|
||||
# Removes the local limit for the page size
|
||||
max_page_size = -1
|
||||
validators_columns = {"sqlalchemy_uri": sqlalchemy_uri_validator}
|
||||
|
||||
@@ -90,7 +90,17 @@ class CsvToDatabaseForm(DynamicForm):
|
||||
csv_file = FileField(
|
||||
_("CSV File"),
|
||||
description=_("Select a CSV file to be uploaded to a database."),
|
||||
validators=[FileRequired(), FileAllowed(["csv"], _("CSV Files Only!"))],
|
||||
validators=[
|
||||
FileRequired(),
|
||||
FileAllowed(
|
||||
config["ALLOWED_EXTENSIONS"],
|
||||
_(
|
||||
"Only the following file extensions are allowed: "
|
||||
"%(allowed_extensions)s",
|
||||
allowed_extensions=", ".join(config["ALLOWED_EXTENSIONS"]),
|
||||
),
|
||||
),
|
||||
],
|
||||
)
|
||||
con = QuerySelectField(
|
||||
_("Database"),
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
# pylint: disable=C,R,W
|
||||
import os
|
||||
|
||||
from flask import flash, redirect
|
||||
from flask import flash, g, redirect
|
||||
from flask_appbuilder import SimpleFormView
|
||||
from flask_appbuilder.forms import DynamicForm
|
||||
from flask_appbuilder.models.sqla.interface import SQLAInterface
|
||||
@@ -28,7 +28,7 @@ from wtforms.fields import StringField
|
||||
from wtforms.validators import ValidationError
|
||||
|
||||
import superset.models.core as models
|
||||
from superset import app, appbuilder, security_manager
|
||||
from superset import app, appbuilder, db, security_manager
|
||||
from superset.connectors.sqla.models import SqlaTable
|
||||
from superset.utils import core as utils
|
||||
from superset.views.base import DeleteMixin, SupersetModelView, YamlExportMixin
|
||||
@@ -54,6 +54,8 @@ class DatabaseView(DatabaseMixin, SupersetModelView, DeleteMixin, YamlExportMixi
|
||||
edit_template = "superset/models/database/edit.html"
|
||||
validators_columns = {"sqlalchemy_uri": [sqlalchemy_uri_form_validator]}
|
||||
|
||||
yaml_dict_key = "databases"
|
||||
|
||||
def _delete(self, pk):
|
||||
DeleteMixin._delete(self, pk)
|
||||
|
||||
@@ -102,10 +104,10 @@ class CsvToDatabaseView(SimpleFormView):
|
||||
|
||||
if not self.is_schema_allowed(database, schema_name):
|
||||
message = _(
|
||||
'Database "{0}" Schema "{1}" is not allowed for csv uploads. '
|
||||
"Please contact Superset Admin".format(
|
||||
database.database_name, schema_name
|
||||
)
|
||||
'Database "%(database_name)s" schema "%(schema_name)s" '
|
||||
"is not allowed for csv uploads. Please contact your Superset Admin.",
|
||||
database_name=database.database_name,
|
||||
schema_name=schema_name,
|
||||
)
|
||||
flash(message, "danger")
|
||||
return redirect("/csvtodatabaseview/form")
|
||||
@@ -117,32 +119,58 @@ class CsvToDatabaseView(SimpleFormView):
|
||||
try:
|
||||
utils.ensure_path_exists(config["UPLOAD_FOLDER"])
|
||||
csv_file.save(path)
|
||||
table = SqlaTable(table_name=form.name.data)
|
||||
table.database = form.data.get("con")
|
||||
table.database_id = table.database.id
|
||||
table.database.db_engine_spec.create_table_from_csv(form, table)
|
||||
table_name = form.name.data
|
||||
database = form.data.get("con")
|
||||
database.db_engine_spec.create_table_from_csv(form)
|
||||
|
||||
table = (
|
||||
db.session.query(SqlaTable)
|
||||
.filter_by(
|
||||
table_name=table_name,
|
||||
schema=form.schema.data,
|
||||
database_id=database.id,
|
||||
)
|
||||
.one_or_none()
|
||||
)
|
||||
if table:
|
||||
table.fetch_metadata()
|
||||
if not table:
|
||||
table = SqlaTable(table_name=table_name)
|
||||
table.database = database
|
||||
table.database_id = database.id
|
||||
table.user_id = g.user.id
|
||||
table.schema = form.schema.data
|
||||
table.fetch_metadata()
|
||||
db.session.add(table)
|
||||
db.session.commit()
|
||||
except Exception as e:
|
||||
db.session.rollback()
|
||||
try:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
pass
|
||||
message = (
|
||||
"Table name {} already exists. Please pick another".format(
|
||||
form.name.data
|
||||
)
|
||||
if isinstance(e, IntegrityError)
|
||||
else str(e)
|
||||
message = _(
|
||||
'Unable to upload CSV file "%(filename)s" to table '
|
||||
'"%(table_name)s" in database "%(db_name)s". '
|
||||
"Error message: %(error_msg)s",
|
||||
filename=csv_filename,
|
||||
table_name=form.name.data,
|
||||
db_name=database.database_name,
|
||||
error_msg=str(e),
|
||||
)
|
||||
|
||||
flash(message, "danger")
|
||||
stats_logger.incr("failed_csv_upload")
|
||||
return redirect("/csvtodatabaseview/form")
|
||||
|
||||
os.remove(path)
|
||||
# Go back to welcome page / splash screen
|
||||
db_name = table.database.database_name
|
||||
message = _(
|
||||
'CSV file "{0}" uploaded to table "{1}" in '
|
||||
'database "{2}"'.format(csv_filename, form.name.data, db_name)
|
||||
'CSV file "%(csv_filename)s" uploaded to table "%(table_name)s" in '
|
||||
'database "%(db_name)s"',
|
||||
csv_filename=csv_filename,
|
||||
table_name=form.name.data,
|
||||
db_name=table.database.database_name,
|
||||
)
|
||||
flash(message, "info")
|
||||
stats_logger.incr("successful_csv_upload")
|
||||
|
||||
@@ -38,6 +38,7 @@ class LogRestApi(LogMixin, ModelRestApi):
|
||||
resource_name = "log"
|
||||
allow_browser_login = True
|
||||
list_columns = ("user.username", "action", "dttm")
|
||||
show_columns = list_columns
|
||||
|
||||
|
||||
if (
|
||||
|
||||
+6
-1
@@ -1085,6 +1085,7 @@ class NVD3TimeSeriesViz(NVD3Viz):
|
||||
verbose_name = _("Time Series - Line Chart")
|
||||
sort_series = False
|
||||
is_timeseries = True
|
||||
pivot_fill_value: Optional[int] = None
|
||||
|
||||
def to_series(self, df, classed="", title_suffix=""):
|
||||
cols = []
|
||||
@@ -1157,7 +1158,10 @@ class NVD3TimeSeriesViz(NVD3Viz):
|
||||
)
|
||||
else:
|
||||
df = df.pivot_table(
|
||||
index=DTTM_ALIAS, columns=fd.get("groupby"), values=self.metric_labels
|
||||
index=DTTM_ALIAS,
|
||||
columns=fd.get("groupby"),
|
||||
values=self.metric_labels,
|
||||
fill_value=self.pivot_fill_value,
|
||||
)
|
||||
|
||||
rule = fd.get("resample_rule")
|
||||
@@ -1443,6 +1447,7 @@ class NVD3TimeSeriesStackedViz(NVD3TimeSeriesViz):
|
||||
viz_type = "area"
|
||||
verbose_name = _("Time Series - Stacked")
|
||||
sort_series = True
|
||||
pivot_fill_value = 0
|
||||
|
||||
|
||||
class DistributionPieViz(NVD3Viz):
|
||||
|
||||
+72
-14
@@ -598,42 +598,100 @@ class CoreTests(SupersetTestCase):
|
||||
|
||||
def test_import_csv(self):
|
||||
self.login(username="admin")
|
||||
filename = "testCSV.csv"
|
||||
table_name = "".join(random.choice(string.ascii_uppercase) for _ in range(5))
|
||||
|
||||
test_file = open(filename, "w+")
|
||||
test_file.write("a,b\n")
|
||||
test_file.write("john,1\n")
|
||||
test_file.write("paul,2\n")
|
||||
test_file.close()
|
||||
filename_1 = "testCSV.csv"
|
||||
test_file_1 = open(filename_1, "w+")
|
||||
test_file_1.write("a,b\n")
|
||||
test_file_1.write("john,1\n")
|
||||
test_file_1.write("paul,2\n")
|
||||
test_file_1.close()
|
||||
|
||||
filename_2 = "testCSV2.csv"
|
||||
test_file_2 = open(filename_2, "w+")
|
||||
test_file_2.write("b,c,d\n")
|
||||
test_file_2.write("john,1,x\n")
|
||||
test_file_2.write("paul,2,y\n")
|
||||
test_file_2.close()
|
||||
|
||||
example_db = utils.get_example_database()
|
||||
example_db.allow_csv_upload = True
|
||||
db_id = example_db.id
|
||||
db.session.commit()
|
||||
test_file = open(filename, "rb")
|
||||
form_data = {
|
||||
"csv_file": test_file,
|
||||
"csv_file": open(filename_1, "rb"),
|
||||
"sep": ",",
|
||||
"name": table_name,
|
||||
"con": db_id,
|
||||
"if_exists": "append",
|
||||
"if_exists": "fail",
|
||||
"index_label": "test_label",
|
||||
"mangle_dupe_cols": False,
|
||||
}
|
||||
url = "/databaseview/list/"
|
||||
add_datasource_page = self.get_resp(url)
|
||||
assert "Upload a CSV" in add_datasource_page
|
||||
self.assertIn("Upload a CSV", add_datasource_page)
|
||||
|
||||
url = "/csvtodatabaseview/form"
|
||||
form_get = self.get_resp(url)
|
||||
assert "CSV to Database configuration" in form_get
|
||||
self.assertIn("CSV to Database configuration", form_get)
|
||||
|
||||
try:
|
||||
# ensure uploaded successfully
|
||||
# initial upload with fail mode
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
assert 'CSV file "testCSV.csv" uploaded to table' in resp
|
||||
self.assertIn(
|
||||
f'CSV file "{filename_1}" uploaded to table "{table_name}"', resp
|
||||
)
|
||||
|
||||
# upload again with fail mode; should fail
|
||||
form_data["csv_file"] = open(filename_1, "rb")
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
self.assertIn(
|
||||
f'Unable to upload CSV file "{filename_1}" to table "{table_name}"',
|
||||
resp,
|
||||
)
|
||||
|
||||
# upload again with append mode
|
||||
form_data["csv_file"] = open(filename_1, "rb")
|
||||
form_data["if_exists"] = "append"
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
self.assertIn(
|
||||
f'CSV file "{filename_1}" uploaded to table "{table_name}"', resp
|
||||
)
|
||||
|
||||
# upload again with replace mode
|
||||
form_data["csv_file"] = open(filename_1, "rb")
|
||||
form_data["if_exists"] = "replace"
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
self.assertIn(
|
||||
f'CSV file "{filename_1}" uploaded to table "{table_name}"', resp
|
||||
)
|
||||
|
||||
# try to append to table from file with different schema
|
||||
form_data["csv_file"] = open(filename_2, "rb")
|
||||
form_data["if_exists"] = "append"
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
self.assertIn(
|
||||
f'Unable to upload CSV file "{filename_2}" to table "{table_name}"',
|
||||
resp,
|
||||
)
|
||||
|
||||
# replace table from file with different schema
|
||||
form_data["csv_file"] = open(filename_2, "rb")
|
||||
form_data["if_exists"] = "replace"
|
||||
resp = self.get_resp(url, data=form_data)
|
||||
self.assertIn(
|
||||
f'CSV file "{filename_2}" uploaded to table "{table_name}"', resp
|
||||
)
|
||||
table = (
|
||||
db.session.query(SqlaTable)
|
||||
.filter_by(table_name=table_name, database_id=db_id)
|
||||
.first()
|
||||
)
|
||||
# make sure the new column name is reflected in the table metadata
|
||||
self.assertIn("d", table.column_names)
|
||||
finally:
|
||||
os.remove(filename)
|
||||
os.remove(filename_1)
|
||||
os.remove(filename_2)
|
||||
|
||||
def test_dataframe_timezone(self):
|
||||
tz = psycopg2.tz.FixedOffsetTimezone(offset=60, name=None)
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from unittest import mock
|
||||
|
||||
from superset import app
|
||||
from superset.db_engine_specs import engines
|
||||
from superset.db_engine_specs.base import BaseEngineSpec, builtin_time_grains
|
||||
from superset.db_engine_specs.sqlite import SqliteEngineSpec
|
||||
from superset.utils.core import get_example_database
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class DbEngineSpecsTests(DbEngineSpecTestCase):
|
||||
def test_extract_limit_from_query(self, engine_spec_class=BaseEngineSpec):
|
||||
q0 = "select * from table"
|
||||
q1 = "select * from mytable limit 10"
|
||||
q2 = "select * from (select * from my_subquery limit 10) where col=1 limit 20"
|
||||
q3 = "select * from (select * from my_subquery limit 10);"
|
||||
q4 = "select * from (select * from my_subquery limit 10) where col=1 limit 20;"
|
||||
q5 = "select * from mytable limit 20, 10"
|
||||
q6 = "select * from mytable limit 10 offset 20"
|
||||
q7 = "select * from mytable limit"
|
||||
q8 = "select * from mytable limit 10.0"
|
||||
q9 = "select * from mytable limit x"
|
||||
q10 = "select * from mytable limit 20, x"
|
||||
q11 = "select * from mytable limit x offset 20"
|
||||
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q0), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q1), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q2), 20)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q3), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q4), 20)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q5), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q6), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q7), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q8), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q9), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q10), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q11), None)
|
||||
|
||||
def test_wrapped_semi_tabs(self):
|
||||
self.sql_limit_regex(
|
||||
"SELECT * FROM a \t \n ; \t \n ", "SELECT * FROM a\nLIMIT 1000"
|
||||
)
|
||||
|
||||
def test_simple_limit_query(self):
|
||||
self.sql_limit_regex("SELECT * FROM a", "SELECT * FROM a\nLIMIT 1000")
|
||||
|
||||
def test_modify_limit_query(self):
|
||||
self.sql_limit_regex("SELECT * FROM a LIMIT 9999", "SELECT * FROM a LIMIT 1000")
|
||||
|
||||
def test_limit_query_with_limit_subquery(self): # pylint: disable=invalid-name
|
||||
self.sql_limit_regex(
|
||||
"SELECT * FROM (SELECT * FROM a LIMIT 10) LIMIT 9999",
|
||||
"SELECT * FROM (SELECT * FROM a LIMIT 10) LIMIT 1000",
|
||||
)
|
||||
|
||||
def test_limit_with_expr(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000""",
|
||||
)
|
||||
|
||||
def test_limit_expr_and_semicolon(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990 ;""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000""",
|
||||
)
|
||||
|
||||
def test_get_datatype(self):
|
||||
self.assertEqual("VARCHAR", BaseEngineSpec.get_datatype("VARCHAR"))
|
||||
|
||||
def test_limit_with_implicit_offset(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990, 999999""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990, 1000""",
|
||||
)
|
||||
|
||||
def test_limit_with_explicit_offset(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990
|
||||
OFFSET 999999""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000
|
||||
OFFSET 999999""",
|
||||
)
|
||||
|
||||
def test_limit_with_non_token_limit(self):
|
||||
self.sql_limit_regex(
|
||||
"""SELECT 'LIMIT 777'""", """SELECT 'LIMIT 777'\nLIMIT 1000"""
|
||||
)
|
||||
|
||||
def test_time_grain_blacklist(self):
|
||||
with app.app_context():
|
||||
app.config["TIME_GRAIN_BLACKLIST"] = ["PT1M"]
|
||||
time_grain_functions = SqliteEngineSpec.get_time_grain_functions()
|
||||
self.assertNotIn("PT1M", time_grain_functions)
|
||||
|
||||
def test_time_grain_addons(self):
|
||||
with app.app_context():
|
||||
app.config["TIME_GRAIN_ADDONS"] = {"PTXM": "x seconds"}
|
||||
app.config["TIME_GRAIN_ADDON_FUNCTIONS"] = {
|
||||
"sqlite": {"PTXM": "ABC({col})"}
|
||||
}
|
||||
time_grains = SqliteEngineSpec.get_time_grains()
|
||||
time_grain_addon = time_grains[-1]
|
||||
self.assertEqual("PTXM", time_grain_addon.duration)
|
||||
self.assertEqual("x seconds", time_grain_addon.label)
|
||||
|
||||
def test_engine_time_grain_validity(self):
|
||||
time_grains = set(builtin_time_grains.keys())
|
||||
# loop over all subclasses of BaseEngineSpec
|
||||
for engine in engines.values():
|
||||
if engine is not BaseEngineSpec:
|
||||
# make sure time grain functions have been defined
|
||||
self.assertGreater(len(engine.get_time_grain_functions()), 0)
|
||||
# make sure all defined time grains are supported
|
||||
defined_grains = {grain.duration for grain in engine.get_time_grains()}
|
||||
intersection = time_grains.intersection(defined_grains)
|
||||
self.assertSetEqual(defined_grains, intersection, engine)
|
||||
|
||||
def test_get_table_names(self):
|
||||
inspector = mock.Mock()
|
||||
inspector.get_table_names = mock.Mock(return_value=["schema.table", "table_2"])
|
||||
inspector.get_foreign_table_names = mock.Mock(return_value=["table_3"])
|
||||
|
||||
""" Make sure base engine spec removes schema name from table name
|
||||
ie. when try_remove_schema_from_table_name == True. """
|
||||
base_result_expected = ["table", "table_2"]
|
||||
base_result = BaseEngineSpec.get_table_names(
|
||||
database=mock.ANY, schema="schema", inspector=inspector
|
||||
)
|
||||
self.assertListEqual(base_result_expected, base_result)
|
||||
|
||||
def test_column_datatype_to_string(self):
|
||||
example_db = get_example_database()
|
||||
sqla_table = example_db.get_table("energy_usage")
|
||||
dialect = example_db.get_dialect()
|
||||
col_names = [
|
||||
example_db.db_engine_spec.column_datatype_to_string(c.type, dialect)
|
||||
for c in sqla_table.columns
|
||||
]
|
||||
if example_db.backend == "postgresql":
|
||||
expected = ["VARCHAR(255)", "VARCHAR(255)", "DOUBLE PRECISION"]
|
||||
else:
|
||||
expected = ["VARCHAR(255)", "VARCHAR(255)", "FLOAT"]
|
||||
self.assertEqual(col_names, expected)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec
|
||||
from superset.models.core import Database
|
||||
from tests.base_tests import SupersetTestCase
|
||||
|
||||
|
||||
class DbEngineSpecTestCase(SupersetTestCase):
|
||||
def sql_limit_regex(
|
||||
self, sql, expected_sql, engine_spec_class=MySQLEngineSpec, limit=1000
|
||||
):
|
||||
main = Database(database_name="test_database", sqlalchemy_uri="sqlite://")
|
||||
limited = engine_spec_class.apply_limit_to_sql(sql, limit, main)
|
||||
self.assertEqual(expected_sql, limited)
|
||||
@@ -0,0 +1,39 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from sqlalchemy import column
|
||||
|
||||
from superset.db_engine_specs.bigquery import BigQueryEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class BigQueryTestCase(DbEngineSpecTestCase):
|
||||
def test_bigquery_sqla_column_label(self):
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("Col").name)
|
||||
label_expected = "Col"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("SUM(x)").name)
|
||||
label_expected = "SUM_x__5f110"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("SUM[x]").name)
|
||||
label_expected = "SUM_x__7ebe1"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("12345_col").name)
|
||||
label_expected = "_12345_col_8d390"
|
||||
self.assertEqual(label, label_expected)
|
||||
@@ -0,0 +1,152 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from unittest import mock
|
||||
|
||||
from superset.db_engine_specs.hive import HiveEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class HiveTests(DbEngineSpecTestCase):
|
||||
def test_0_progress(self):
|
||||
log = """
|
||||
17/02/07 18:26:27 INFO log.PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
|
||||
17/02/07 18:26:27 INFO log.PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_number_of_jobs_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1_map_40_progress(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(10, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1_map_80_reduce_40_progress(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 80%, reduce = 40%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(30, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_2_stages_progress(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 80%, reduce = 40%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-2 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 100%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(12, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_2_launched_stage_2_stages_progress(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 100%, reduce = 0%
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 2 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(60, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_hive_error_msg(self):
|
||||
msg = (
|
||||
'{...} errorMessage="Error while compiling statement: FAILED: '
|
||||
"SemanticException [Error 10001]: Line 4"
|
||||
":5 Table not found 'fact_ridesfdslakj'\", statusCode=3, "
|
||||
"sqlState='42S02', errorCode=10001)){...}"
|
||||
)
|
||||
self.assertEqual(
|
||||
(
|
||||
"hive error: Error while compiling statement: FAILED: "
|
||||
"SemanticException [Error 10001]: Line 4:5 "
|
||||
"Table not found 'fact_ridesfdslakj'"
|
||||
),
|
||||
HiveEngineSpec.extract_error_message(Exception(msg)),
|
||||
)
|
||||
|
||||
e = Exception("Some string that doesn't match the regex")
|
||||
self.assertEqual(f"hive error: {e}", HiveEngineSpec.extract_error_message(e))
|
||||
|
||||
msg = (
|
||||
"errorCode=10001, "
|
||||
'errorMessage="Error while compiling statement"), operationHandle'
|
||||
'=None)"'
|
||||
)
|
||||
self.assertEqual(
|
||||
("hive error: Error while compiling statement"),
|
||||
HiveEngineSpec.extract_error_message(Exception(msg)),
|
||||
)
|
||||
|
||||
def test_hive_get_view_names_return_empty_list(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
self.assertEqual(
|
||||
[], HiveEngineSpec.get_view_names(mock.ANY, mock.ANY, mock.ANY)
|
||||
)
|
||||
@@ -0,0 +1,71 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from sqlalchemy import column, table
|
||||
from sqlalchemy.dialects import mssql
|
||||
from sqlalchemy.sql import select
|
||||
from sqlalchemy.types import String, UnicodeText
|
||||
|
||||
from superset.db_engine_specs.mssql import MssqlEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class MssqlEngineSpecTest(DbEngineSpecTestCase):
|
||||
def test_mssql_column_types(self):
|
||||
def assert_type(type_string, type_expected):
|
||||
type_assigned = MssqlEngineSpec.get_sqla_column_type(type_string)
|
||||
if type_expected is None:
|
||||
self.assertIsNone(type_assigned)
|
||||
else:
|
||||
self.assertIsInstance(type_assigned, type_expected)
|
||||
|
||||
assert_type("INT", None)
|
||||
assert_type("STRING", String)
|
||||
assert_type("CHAR(10)", String)
|
||||
assert_type("VARCHAR(10)", String)
|
||||
assert_type("TEXT", String)
|
||||
assert_type("NCHAR(10)", UnicodeText)
|
||||
assert_type("NVARCHAR(10)", UnicodeText)
|
||||
assert_type("NTEXT", UnicodeText)
|
||||
|
||||
def test_where_clause_n_prefix(self):
|
||||
dialect = mssql.dialect()
|
||||
spec = MssqlEngineSpec
|
||||
str_col = column("col", type_=spec.get_sqla_column_type("VARCHAR(10)"))
|
||||
unicode_col = column("unicode_col", type_=spec.get_sqla_column_type("NTEXT"))
|
||||
tbl = table("tbl")
|
||||
sel = (
|
||||
select([str_col, unicode_col])
|
||||
.select_from(tbl)
|
||||
.where(str_col == "abc")
|
||||
.where(unicode_col == "abc")
|
||||
)
|
||||
|
||||
query = str(
|
||||
sel.compile(dialect=dialect, compile_kwargs={"literal_binds": True})
|
||||
)
|
||||
query_expected = (
|
||||
"SELECT col, unicode_col \n"
|
||||
"FROM tbl \n"
|
||||
"WHERE col = 'abc' AND unicode_col = N'abc'"
|
||||
)
|
||||
self.assertEqual(query, query_expected)
|
||||
|
||||
def test_time_exp_mixd_case_col_1y(self):
|
||||
col = column("MixedCase")
|
||||
expr = MssqlEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(None, dialect=mssql.dialect()))
|
||||
self.assertEqual(result, "DATEADD(year, DATEDIFF(year, 0, [MixedCase]), 0)")
|
||||
@@ -0,0 +1,30 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
import unittest
|
||||
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class MySQLEngineSpecsTestCase(DbEngineSpecTestCase):
|
||||
@unittest.skipUnless(
|
||||
DbEngineSpecTestCase.is_module_installed("MySQLdb"), "mysqlclient not installed"
|
||||
)
|
||||
def test_get_datatype_mysql(self):
|
||||
"""Tests related to datatype mapping for MySQL"""
|
||||
self.assertEqual("TINY", MySQLEngineSpec.get_datatype(1))
|
||||
self.assertEqual("VARCHAR", MySQLEngineSpec.get_datatype(15))
|
||||
@@ -0,0 +1,36 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from sqlalchemy import column
|
||||
from sqlalchemy.dialects import oracle
|
||||
|
||||
from superset.db_engine_specs.oracle import OracleEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class OracleTestCase(DbEngineSpecTestCase):
|
||||
def test_oracle_sqla_column_name_length_exceeded(self):
|
||||
col = column("This_Is_32_Character_Column_Name")
|
||||
label = OracleEngineSpec.make_label_compatible(col.name)
|
||||
self.assertEqual(label.quote, True)
|
||||
label_expected = "3b26974078683be078219674eeb8f5"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
def test_oracle_time_expression_reserved_keyword_1m_grain(self):
|
||||
col = column("decimal")
|
||||
expr = OracleEngineSpec.get_timestamp_expr(col, None, "P1M")
|
||||
result = str(expr.compile(dialect=oracle.dialect()))
|
||||
self.assertEqual(result, "TRUNC(CAST(\"decimal\" as DATE), 'MONTH')")
|
||||
@@ -0,0 +1,33 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from sqlalchemy import column
|
||||
|
||||
from superset.db_engine_specs.pinot import PinotEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class PinotTestCase(DbEngineSpecTestCase):
|
||||
""" Tests pertaining to our Pinot database support """
|
||||
|
||||
def test_pinot_time_expression_sec_one_1m_grain(self):
|
||||
col = column("tstamp")
|
||||
expr = PinotEngineSpec.get_timestamp_expr(col, "epoch_s", "P1M")
|
||||
result = str(expr.compile())
|
||||
self.assertEqual(
|
||||
result,
|
||||
'DATETIMECONVERT(tstamp, "1:SECONDS:EPOCH", "1:SECONDS:EPOCH", "1:MONTHS")',
|
||||
) # noqa
|
||||
@@ -0,0 +1,72 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from unittest import mock
|
||||
|
||||
from sqlalchemy import column, literal_column
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
from superset.db_engine_specs.postgres import PostgresEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class PostgresTests(DbEngineSpecTestCase):
|
||||
def test_get_table_names(self):
|
||||
""" Make sure postgres doesn't try to remove schema name from table name
|
||||
ie. when try_remove_schema_from_table_name == False. """
|
||||
inspector = mock.Mock()
|
||||
inspector.get_table_names = mock.Mock(return_value=["schema.table", "table_2"])
|
||||
inspector.get_foreign_table_names = mock.Mock(return_value=["table_3"])
|
||||
|
||||
pg_result_expected = ["schema.table", "table_2", "table_3"]
|
||||
pg_result = PostgresEngineSpec.get_table_names(
|
||||
database=mock.ANY, schema="schema", inspector=inspector
|
||||
)
|
||||
self.assertListEqual(pg_result_expected, pg_result)
|
||||
|
||||
def test_time_exp_literal_no_grain(self):
|
||||
col = literal_column("COALESCE(a, b)")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, None)
|
||||
result = str(expr.compile(None, dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "COALESCE(a, b)")
|
||||
|
||||
def test_time_exp_literal_1y_grain(self):
|
||||
col = literal_column("COALESCE(a, b)")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(None, dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "DATE_TRUNC('year', COALESCE(a, b))")
|
||||
|
||||
def test_time_ex_lowr_col_no_grain(self):
|
||||
col = column("lower_case")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, None)
|
||||
result = str(expr.compile(None, dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "lower_case")
|
||||
|
||||
def test_time_exp_lowr_col_sec_1y(self):
|
||||
col = column("lower_case")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, "epoch_s", "P1Y")
|
||||
result = str(expr.compile(None, dialect=postgresql.dialect()))
|
||||
self.assertEqual(
|
||||
result,
|
||||
"DATE_TRUNC('year', "
|
||||
"(timestamp 'epoch' + lower_case * interval '1 second'))",
|
||||
)
|
||||
|
||||
def test_time_exp_mixd_case_col_1y(self):
|
||||
col = column("MixedCase")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(None, dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "DATE_TRUNC('year', \"MixedCase\")")
|
||||
@@ -0,0 +1,343 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
from unittest import mock, skipUnless
|
||||
|
||||
import pandas as pd
|
||||
from sqlalchemy.engine.result import RowProxy
|
||||
from sqlalchemy.sql import select
|
||||
|
||||
from superset.db_engine_specs.presto import PrestoEngineSpec
|
||||
from tests.db_engine_specs.base_tests import DbEngineSpecTestCase
|
||||
|
||||
|
||||
class PrestoTests(DbEngineSpecTestCase):
|
||||
@skipUnless(
|
||||
DbEngineSpecTestCase.is_module_installed("pyhive"), "pyhive not installed"
|
||||
)
|
||||
def test_get_datatype_presto(self):
|
||||
self.assertEqual("STRING", PrestoEngineSpec.get_datatype("string"))
|
||||
|
||||
def test_presto_get_view_names_return_empty_list(
|
||||
self
|
||||
): # pylint: disable=invalid-name
|
||||
self.assertEqual(
|
||||
[], PrestoEngineSpec.get_view_names(mock.ANY, mock.ANY, mock.ANY)
|
||||
)
|
||||
|
||||
def verify_presto_column(self, column, expected_results):
|
||||
inspector = mock.Mock()
|
||||
inspector.engine.dialect.identifier_preparer.quote_identifier = mock.Mock()
|
||||
keymap = {
|
||||
"Column": (None, None, 0),
|
||||
"Type": (None, None, 1),
|
||||
"Null": (None, None, 2),
|
||||
}
|
||||
row = RowProxy(mock.Mock(), column, [None, None, None, None], keymap)
|
||||
inspector.bind.execute = mock.Mock(return_value=[row])
|
||||
results = PrestoEngineSpec.get_columns(inspector, "", "")
|
||||
self.assertEqual(len(expected_results), len(results))
|
||||
for expected_result, result in zip(expected_results, results):
|
||||
self.assertEqual(expected_result[0], result["name"])
|
||||
self.assertEqual(expected_result[1], str(result["type"]))
|
||||
|
||||
def test_presto_get_column(self):
|
||||
presto_column = ("column_name", "boolean", "")
|
||||
expected_results = [("column_name", "BOOLEAN")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column(self):
|
||||
presto_column = ("column_name", "row(nested_obj double)", "")
|
||||
expected_results = [("column_name", "ROW"), ("column_name.nested_obj", "FLOAT")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column_with_name_containing_whitespace(self):
|
||||
presto_column = ("column name", "row(nested_obj double)", "")
|
||||
expected_results = [("column name", "ROW"), ("column name.nested_obj", "FLOAT")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column_with_tricky_nested_field_name(self):
|
||||
presto_column = ("column_name", 'row("Field Name(Tricky, Name)" double)', "")
|
||||
expected_results = [
|
||||
("column_name", "ROW"),
|
||||
('column_name."Field Name(Tricky, Name)"', "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_array_column(self):
|
||||
presto_column = ("column_name", "array(double)", "")
|
||||
expected_results = [("column_name", "ARRAY")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_row_within_array_within_row_column(self):
|
||||
presto_column = (
|
||||
"column_name",
|
||||
"row(nested_array array(row(nested_row double)), nested_obj double)",
|
||||
"",
|
||||
)
|
||||
expected_results = [
|
||||
("column_name", "ROW"),
|
||||
("column_name.nested_array", "ARRAY"),
|
||||
("column_name.nested_array.nested_row", "FLOAT"),
|
||||
("column_name.nested_obj", "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_array_within_row_within_array_column(self):
|
||||
presto_column = (
|
||||
"column_name",
|
||||
"array(row(nested_array array(double), nested_obj double))",
|
||||
"",
|
||||
)
|
||||
expected_results = [
|
||||
("column_name", "ARRAY"),
|
||||
("column_name.nested_array", "ARRAY"),
|
||||
("column_name.nested_obj", "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
def test_presto_get_fields(self):
|
||||
cols = [
|
||||
{"name": "column"},
|
||||
{"name": "column.nested_obj"},
|
||||
{"name": 'column."quoted.nested obj"'},
|
||||
]
|
||||
actual_results = PrestoEngineSpec._get_fields(cols)
|
||||
expected_results = [
|
||||
{"name": '"column"', "label": "column"},
|
||||
{"name": '"column"."nested_obj"', "label": "column.nested_obj"},
|
||||
{
|
||||
"name": '"column"."quoted.nested obj"',
|
||||
"label": 'column."quoted.nested obj"',
|
||||
},
|
||||
]
|
||||
for actual_result, expected_result in zip(actual_results, expected_results):
|
||||
self.assertEqual(actual_result.element.name, expected_result["name"])
|
||||
self.assertEqual(actual_result.name, expected_result["label"])
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_simple_structural_columns(self):
|
||||
cols = [
|
||||
{"name": "row_column", "type": "ROW(NESTED_OBJ VARCHAR)"},
|
||||
{"name": "array_column", "type": "ARRAY(BIGINT)"},
|
||||
]
|
||||
data = [
|
||||
{"row_column": ["a"], "array_column": [1, 2, 3]},
|
||||
{"row_column": ["b"], "array_column": [4, 5, 6]},
|
||||
]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{"name": "row_column", "type": "ROW(NESTED_OBJ VARCHAR)"},
|
||||
{"name": "row_column.nested_obj", "type": "VARCHAR"},
|
||||
{"name": "array_column", "type": "ARRAY(BIGINT)"},
|
||||
]
|
||||
|
||||
expected_data = [
|
||||
{"array_column": 1, "row_column": ["a"], "row_column.nested_obj": "a"},
|
||||
{"array_column": 2, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 3, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 4, "row_column": ["b"], "row_column.nested_obj": "b"},
|
||||
{"array_column": 5, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 6, "row_column": "", "row_column.nested_obj": ""},
|
||||
]
|
||||
|
||||
expected_expanded_cols = [{"name": "row_column.nested_obj", "type": "VARCHAR"}]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_complex_row_columns(self):
|
||||
cols = [
|
||||
{
|
||||
"name": "row_column",
|
||||
"type": "ROW(NESTED_OBJ1 VARCHAR, NESTED_ROW ROW(NESTED_OBJ2 VARCHAR))",
|
||||
}
|
||||
]
|
||||
data = [{"row_column": ["a1", ["a2"]]}, {"row_column": ["b1", ["b2"]]}]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{
|
||||
"name": "row_column",
|
||||
"type": "ROW(NESTED_OBJ1 VARCHAR, NESTED_ROW ROW(NESTED_OBJ2 VARCHAR))",
|
||||
},
|
||||
{"name": "row_column.nested_row", "type": "ROW(NESTED_OBJ2 VARCHAR)"},
|
||||
{"name": "row_column.nested_row.nested_obj2", "type": "VARCHAR"},
|
||||
{"name": "row_column.nested_obj1", "type": "VARCHAR"},
|
||||
]
|
||||
expected_data = [
|
||||
{
|
||||
"row_column": ["a1", ["a2"]],
|
||||
"row_column.nested_obj1": "a1",
|
||||
"row_column.nested_row": ["a2"],
|
||||
"row_column.nested_row.nested_obj2": "a2",
|
||||
},
|
||||
{
|
||||
"row_column": ["b1", ["b2"]],
|
||||
"row_column.nested_obj1": "b1",
|
||||
"row_column.nested_row": ["b2"],
|
||||
"row_column.nested_row.nested_obj2": "b2",
|
||||
},
|
||||
]
|
||||
|
||||
expected_expanded_cols = [
|
||||
{"name": "row_column.nested_obj1", "type": "VARCHAR"},
|
||||
{"name": "row_column.nested_row", "type": "ROW(NESTED_OBJ2 VARCHAR)"},
|
||||
{"name": "row_column.nested_row.nested_obj2", "type": "VARCHAR"},
|
||||
]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_complex_array_columns(self):
|
||||
cols = [
|
||||
{"name": "int_column", "type": "BIGINT"},
|
||||
{
|
||||
"name": "array_column",
|
||||
"type": "ARRAY(ROW(NESTED_ARRAY ARRAY(ROW(NESTED_OBJ VARCHAR))))",
|
||||
},
|
||||
]
|
||||
data = [
|
||||
{"int_column": 1, "array_column": [[[["a"], ["b"]]], [[["c"], ["d"]]]]},
|
||||
{"int_column": 2, "array_column": [[[["e"], ["f"]]], [[["g"], ["h"]]]]},
|
||||
]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{"name": "int_column", "type": "BIGINT"},
|
||||
{
|
||||
"name": "array_column",
|
||||
"type": "ARRAY(ROW(NESTED_ARRAY ARRAY(ROW(NESTED_OBJ VARCHAR))))",
|
||||
},
|
||||
{
|
||||
"name": "array_column.nested_array",
|
||||
"type": "ARRAY(ROW(NESTED_OBJ VARCHAR))",
|
||||
},
|
||||
{"name": "array_column.nested_array.nested_obj", "type": "VARCHAR"},
|
||||
]
|
||||
expected_data = [
|
||||
{
|
||||
"array_column": [[["a"], ["b"]]],
|
||||
"array_column.nested_array": ["a"],
|
||||
"array_column.nested_array.nested_obj": "a",
|
||||
"int_column": 1,
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["b"],
|
||||
"array_column.nested_array.nested_obj": "b",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["c"], ["d"]]],
|
||||
"array_column.nested_array": ["c"],
|
||||
"array_column.nested_array.nested_obj": "c",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["d"],
|
||||
"array_column.nested_array.nested_obj": "d",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["e"], ["f"]]],
|
||||
"array_column.nested_array": ["e"],
|
||||
"array_column.nested_array.nested_obj": "e",
|
||||
"int_column": 2,
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["f"],
|
||||
"array_column.nested_array.nested_obj": "f",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["g"], ["h"]]],
|
||||
"array_column.nested_array": ["g"],
|
||||
"array_column.nested_array.nested_obj": "g",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["h"],
|
||||
"array_column.nested_array.nested_obj": "h",
|
||||
"int_column": "",
|
||||
},
|
||||
]
|
||||
expected_expanded_cols = [
|
||||
{
|
||||
"name": "array_column.nested_array",
|
||||
"type": "ARRAY(ROW(NESTED_OBJ VARCHAR))",
|
||||
},
|
||||
{"name": "array_column.nested_array.nested_obj", "type": "VARCHAR"},
|
||||
]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
def test_presto_extra_table_metadata(self):
|
||||
db = mock.Mock()
|
||||
db.get_indexes = mock.Mock(return_value=[{"column_names": ["ds", "hour"]}])
|
||||
db.get_extra = mock.Mock(return_value={})
|
||||
df = pd.DataFrame({"ds": ["01-01-19"], "hour": [1]})
|
||||
db.get_df = mock.Mock(return_value=df)
|
||||
PrestoEngineSpec.get_create_view = mock.Mock(return_value=None)
|
||||
result = PrestoEngineSpec.extra_table_metadata(db, "test_table", "test_schema")
|
||||
self.assertEqual({"ds": "01-01-19", "hour": 1}, result["partitions"]["latest"])
|
||||
|
||||
def test_presto_where_latest_partition(self):
|
||||
db = mock.Mock()
|
||||
db.get_indexes = mock.Mock(return_value=[{"column_names": ["ds", "hour"]}])
|
||||
db.get_extra = mock.Mock(return_value={})
|
||||
df = pd.DataFrame({"ds": ["01-01-19"], "hour": [1]})
|
||||
db.get_df = mock.Mock(return_value=df)
|
||||
columns = [{"name": "ds"}, {"name": "hour"}]
|
||||
result = PrestoEngineSpec.where_latest_partition(
|
||||
"test_table", "test_schema", db, select(), columns
|
||||
)
|
||||
query_result = str(result.compile(compile_kwargs={"literal_binds": True}))
|
||||
self.assertEqual("SELECT \nWHERE ds = '01-01-19' AND hour = 1", query_result)
|
||||
@@ -1,810 +0,0 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
import pandas as pd
|
||||
from sqlalchemy import column, literal_column, table
|
||||
from sqlalchemy.dialects import mssql, oracle, postgresql
|
||||
from sqlalchemy.engine.result import RowProxy
|
||||
from sqlalchemy.sql import select
|
||||
from sqlalchemy.types import String, UnicodeText
|
||||
|
||||
from superset import app
|
||||
from superset.db_engine_specs import engines
|
||||
from superset.db_engine_specs.base import BaseEngineSpec, builtin_time_grains
|
||||
from superset.db_engine_specs.bigquery import BigQueryEngineSpec
|
||||
from superset.db_engine_specs.hive import HiveEngineSpec
|
||||
from superset.db_engine_specs.mssql import MssqlEngineSpec
|
||||
from superset.db_engine_specs.mysql import MySQLEngineSpec
|
||||
from superset.db_engine_specs.oracle import OracleEngineSpec
|
||||
from superset.db_engine_specs.pinot import PinotEngineSpec
|
||||
from superset.db_engine_specs.postgres import PostgresEngineSpec
|
||||
from superset.db_engine_specs.presto import PrestoEngineSpec
|
||||
from superset.db_engine_specs.sqlite import SqliteEngineSpec
|
||||
from superset.models.core import Database
|
||||
from superset.utils.core import get_example_database
|
||||
|
||||
from .base_tests import SupersetTestCase
|
||||
|
||||
|
||||
class DbEngineSpecsTestCase(SupersetTestCase):
|
||||
def test_0_progress(self):
|
||||
log = """
|
||||
17/02/07 18:26:27 INFO log.PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
|
||||
17/02/07 18:26:27 INFO log.PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_number_of_jobs_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1_0_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(0, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1_map_40_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(10, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_1_map_80_reduce_40_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 80%, reduce = 40%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(30, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_1_launched_stage_2_stages_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 80%, reduce = 40%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-2 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 100%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(12, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_job_2_launched_stage_2_stages_progress(self):
|
||||
log = """
|
||||
17/02/07 19:15:55 INFO ql.Driver: Total jobs = 2
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 1 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 100%, reduce = 0%
|
||||
17/02/07 19:15:55 INFO ql.Driver: Launching Job 2 out of 2
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 0%, reduce = 0%
|
||||
17/02/07 19:16:09 INFO exec.Task: 2017-02-07 19:16:09,173 Stage-1 map = 40%, reduce = 0%
|
||||
""".split(
|
||||
"\n"
|
||||
)
|
||||
self.assertEqual(60, HiveEngineSpec.progress(log))
|
||||
|
||||
def test_hive_error_msg(self):
|
||||
msg = (
|
||||
'{...} errorMessage="Error while compiling statement: FAILED: '
|
||||
"SemanticException [Error 10001]: Line 4"
|
||||
":5 Table not found 'fact_ridesfdslakj'\", statusCode=3, "
|
||||
"sqlState='42S02', errorCode=10001)){...}"
|
||||
)
|
||||
self.assertEqual(
|
||||
(
|
||||
"hive error: Error while compiling statement: FAILED: "
|
||||
"SemanticException [Error 10001]: Line 4:5 "
|
||||
"Table not found 'fact_ridesfdslakj'"
|
||||
),
|
||||
HiveEngineSpec.extract_error_message(Exception(msg)),
|
||||
)
|
||||
|
||||
e = Exception("Some string that doesn't match the regex")
|
||||
self.assertEqual(f"hive error: {e}", HiveEngineSpec.extract_error_message(e))
|
||||
|
||||
msg = (
|
||||
"errorCode=10001, "
|
||||
'errorMessage="Error while compiling statement"), operationHandle'
|
||||
'=None)"'
|
||||
)
|
||||
self.assertEqual(
|
||||
("hive error: Error while compiling statement"),
|
||||
HiveEngineSpec.extract_error_message(Exception(msg)),
|
||||
)
|
||||
|
||||
def get_generic_database(self):
|
||||
return Database(database_name="test_database", sqlalchemy_uri="sqlite://")
|
||||
|
||||
def sql_limit_regex(
|
||||
self, sql, expected_sql, engine_spec_class=MySQLEngineSpec, limit=1000
|
||||
):
|
||||
main = self.get_generic_database()
|
||||
limited = engine_spec_class.apply_limit_to_sql(sql, limit, main)
|
||||
self.assertEqual(expected_sql, limited)
|
||||
|
||||
def test_extract_limit_from_query(self, engine_spec_class=MySQLEngineSpec):
|
||||
q0 = "select * from table"
|
||||
q1 = "select * from mytable limit 10"
|
||||
q2 = "select * from (select * from my_subquery limit 10) where col=1 limit 20"
|
||||
q3 = "select * from (select * from my_subquery limit 10);"
|
||||
q4 = "select * from (select * from my_subquery limit 10) where col=1 limit 20;"
|
||||
q5 = "select * from mytable limit 20, 10"
|
||||
q6 = "select * from mytable limit 10 offset 20"
|
||||
q7 = "select * from mytable limit"
|
||||
q8 = "select * from mytable limit 10.0"
|
||||
q9 = "select * from mytable limit x"
|
||||
q10 = "select * from mytable limit 20, x"
|
||||
q11 = "select * from mytable limit x offset 20"
|
||||
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q0), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q1), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q2), 20)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q3), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q4), 20)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q5), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q6), 10)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q7), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q8), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q9), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q10), None)
|
||||
self.assertEqual(engine_spec_class.get_limit_from_sql(q11), None)
|
||||
|
||||
def test_wrapped_query(self):
|
||||
self.sql_limit_regex(
|
||||
"SELECT * FROM a",
|
||||
"SELECT * \nFROM (SELECT * FROM a) AS inner_qry\n LIMIT 1000 OFFSET 0",
|
||||
MssqlEngineSpec,
|
||||
)
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("MySQLdb"), "mysqlclient not installed"
|
||||
)
|
||||
def test_wrapped_semi_tabs(self):
|
||||
self.sql_limit_regex(
|
||||
"SELECT * FROM a \t \n ; \t \n ", "SELECT * FROM a\nLIMIT 1000"
|
||||
)
|
||||
|
||||
def test_simple_limit_query(self):
|
||||
self.sql_limit_regex("SELECT * FROM a", "SELECT * FROM a\nLIMIT 1000")
|
||||
|
||||
def test_modify_limit_query(self):
|
||||
self.sql_limit_regex("SELECT * FROM a LIMIT 9999", "SELECT * FROM a LIMIT 1000")
|
||||
|
||||
def test_limit_query_with_limit_subquery(self):
|
||||
self.sql_limit_regex(
|
||||
"SELECT * FROM (SELECT * FROM a LIMIT 10) LIMIT 9999",
|
||||
"SELECT * FROM (SELECT * FROM a LIMIT 10) LIMIT 1000",
|
||||
)
|
||||
|
||||
def test_limit_with_expr(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000""",
|
||||
)
|
||||
|
||||
def test_limit_expr_and_semicolon(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990 ;""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000""",
|
||||
)
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("MySQLdb"), "mysqlclient not installed"
|
||||
)
|
||||
def test_get_datatype_mysql(self):
|
||||
self.assertEqual("TINY", MySQLEngineSpec.get_datatype(1))
|
||||
self.assertEqual("VARCHAR", MySQLEngineSpec.get_datatype(15))
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("pyhive"), "pyhive not installed"
|
||||
)
|
||||
def test_get_datatype_presto(self):
|
||||
self.assertEqual("STRING", PrestoEngineSpec.get_datatype("string"))
|
||||
|
||||
def test_get_datatype(self):
|
||||
self.assertEqual("VARCHAR", BaseEngineSpec.get_datatype("VARCHAR"))
|
||||
|
||||
def test_limit_with_implicit_offset(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990, 999999""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990, 1000""",
|
||||
)
|
||||
|
||||
def test_limit_with_explicit_offset(self):
|
||||
self.sql_limit_regex(
|
||||
"""
|
||||
SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 99990
|
||||
OFFSET 999999""",
|
||||
"""SELECT
|
||||
'LIMIT 777' AS a
|
||||
, b
|
||||
FROM
|
||||
table
|
||||
LIMIT 1000
|
||||
OFFSET 999999""",
|
||||
)
|
||||
|
||||
def test_limit_with_non_token_limit(self):
|
||||
self.sql_limit_regex(
|
||||
"""SELECT 'LIMIT 777'""", """SELECT 'LIMIT 777'\nLIMIT 1000"""
|
||||
)
|
||||
|
||||
def test_time_grain_blacklist(self):
|
||||
with app.app_context():
|
||||
app.config["TIME_GRAIN_BLACKLIST"] = ["PT1M"]
|
||||
time_grain_functions = SqliteEngineSpec.get_time_grain_functions()
|
||||
self.assertNotIn("PT1M", time_grain_functions)
|
||||
|
||||
def test_time_grain_addons(self):
|
||||
with app.app_context():
|
||||
app.config["TIME_GRAIN_ADDONS"] = {"PTXM": "x seconds"}
|
||||
app.config["TIME_GRAIN_ADDON_FUNCTIONS"] = {
|
||||
"sqlite": {"PTXM": "ABC({col})"}
|
||||
}
|
||||
time_grains = SqliteEngineSpec.get_time_grains()
|
||||
time_grain_addon = time_grains[-1]
|
||||
self.assertEqual("PTXM", time_grain_addon.duration)
|
||||
self.assertEqual("x seconds", time_grain_addon.label)
|
||||
|
||||
def test_engine_time_grain_validity(self):
|
||||
time_grains = set(builtin_time_grains.keys())
|
||||
# loop over all subclasses of BaseEngineSpec
|
||||
for engine in engines.values():
|
||||
if engine is not BaseEngineSpec:
|
||||
# make sure time grain functions have been defined
|
||||
self.assertGreater(len(engine.get_time_grain_functions()), 0)
|
||||
# make sure all defined time grains are supported
|
||||
defined_grains = {grain.duration for grain in engine.get_time_grains()}
|
||||
intersection = time_grains.intersection(defined_grains)
|
||||
self.assertSetEqual(defined_grains, intersection, engine)
|
||||
|
||||
def test_presto_get_view_names_return_empty_list(self):
|
||||
self.assertEqual(
|
||||
[], PrestoEngineSpec.get_view_names(mock.ANY, mock.ANY, mock.ANY)
|
||||
)
|
||||
|
||||
def verify_presto_column(self, column, expected_results):
|
||||
inspector = mock.Mock()
|
||||
inspector.engine.dialect.identifier_preparer.quote_identifier = mock.Mock()
|
||||
keymap = {
|
||||
"Column": (None, None, 0),
|
||||
"Type": (None, None, 1),
|
||||
"Null": (None, None, 2),
|
||||
}
|
||||
row = RowProxy(mock.Mock(), column, [None, None, None, None], keymap)
|
||||
inspector.bind.execute = mock.Mock(return_value=[row])
|
||||
results = PrestoEngineSpec.get_columns(inspector, "", "")
|
||||
self.assertEqual(len(expected_results), len(results))
|
||||
for expected_result, result in zip(expected_results, results):
|
||||
self.assertEqual(expected_result[0], result["name"])
|
||||
self.assertEqual(expected_result[1], str(result["type"]))
|
||||
|
||||
def test_presto_get_column(self):
|
||||
presto_column = ("column_name", "boolean", "")
|
||||
expected_results = [("column_name", "BOOLEAN")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column(self):
|
||||
presto_column = ("column_name", "row(nested_obj double)", "")
|
||||
expected_results = [("column_name", "ROW"), ("column_name.nested_obj", "FLOAT")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column_with_name_containing_whitespace(self):
|
||||
presto_column = ("column name", "row(nested_obj double)", "")
|
||||
expected_results = [("column name", "ROW"), ("column name.nested_obj", "FLOAT")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_row_column_with_tricky_nested_field_name(self):
|
||||
presto_column = ("column_name", 'row("Field Name(Tricky, Name)" double)', "")
|
||||
expected_results = [
|
||||
("column_name", "ROW"),
|
||||
('column_name."Field Name(Tricky, Name)"', "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_simple_array_column(self):
|
||||
presto_column = ("column_name", "array(double)", "")
|
||||
expected_results = [("column_name", "ARRAY")]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_row_within_array_within_row_column(self):
|
||||
presto_column = (
|
||||
"column_name",
|
||||
"row(nested_array array(row(nested_row double)), nested_obj double)",
|
||||
"",
|
||||
)
|
||||
expected_results = [
|
||||
("column_name", "ROW"),
|
||||
("column_name.nested_array", "ARRAY"),
|
||||
("column_name.nested_array.nested_row", "FLOAT"),
|
||||
("column_name.nested_obj", "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_get_array_within_row_within_array_column(self):
|
||||
presto_column = (
|
||||
"column_name",
|
||||
"array(row(nested_array array(double), nested_obj double))",
|
||||
"",
|
||||
)
|
||||
expected_results = [
|
||||
("column_name", "ARRAY"),
|
||||
("column_name.nested_array", "ARRAY"),
|
||||
("column_name.nested_obj", "FLOAT"),
|
||||
]
|
||||
self.verify_presto_column(presto_column, expected_results)
|
||||
|
||||
def test_presto_get_fields(self):
|
||||
cols = [
|
||||
{"name": "column"},
|
||||
{"name": "column.nested_obj"},
|
||||
{"name": 'column."quoted.nested obj"'},
|
||||
]
|
||||
actual_results = PrestoEngineSpec._get_fields(cols)
|
||||
expected_results = [
|
||||
{"name": '"column"', "label": "column"},
|
||||
{"name": '"column"."nested_obj"', "label": "column.nested_obj"},
|
||||
{
|
||||
"name": '"column"."quoted.nested obj"',
|
||||
"label": 'column."quoted.nested obj"',
|
||||
},
|
||||
]
|
||||
for actual_result, expected_result in zip(actual_results, expected_results):
|
||||
self.assertEqual(actual_result.element.name, expected_result["name"])
|
||||
self.assertEqual(actual_result.name, expected_result["label"])
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_simple_structural_columns(self):
|
||||
cols = [
|
||||
{"name": "row_column", "type": "ROW(NESTED_OBJ VARCHAR)"},
|
||||
{"name": "array_column", "type": "ARRAY(BIGINT)"},
|
||||
]
|
||||
data = [
|
||||
{"row_column": ["a"], "array_column": [1, 2, 3]},
|
||||
{"row_column": ["b"], "array_column": [4, 5, 6]},
|
||||
]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{"name": "row_column", "type": "ROW(NESTED_OBJ VARCHAR)"},
|
||||
{"name": "row_column.nested_obj", "type": "VARCHAR"},
|
||||
{"name": "array_column", "type": "ARRAY(BIGINT)"},
|
||||
]
|
||||
|
||||
expected_data = [
|
||||
{"array_column": 1, "row_column": ["a"], "row_column.nested_obj": "a"},
|
||||
{"array_column": 2, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 3, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 4, "row_column": ["b"], "row_column.nested_obj": "b"},
|
||||
{"array_column": 5, "row_column": "", "row_column.nested_obj": ""},
|
||||
{"array_column": 6, "row_column": "", "row_column.nested_obj": ""},
|
||||
]
|
||||
|
||||
expected_expanded_cols = [{"name": "row_column.nested_obj", "type": "VARCHAR"}]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_complex_row_columns(self):
|
||||
cols = [
|
||||
{
|
||||
"name": "row_column",
|
||||
"type": "ROW(NESTED_OBJ1 VARCHAR, NESTED_ROW ROW(NESTED_OBJ2 VARCHAR))",
|
||||
}
|
||||
]
|
||||
data = [{"row_column": ["a1", ["a2"]]}, {"row_column": ["b1", ["b2"]]}]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{
|
||||
"name": "row_column",
|
||||
"type": "ROW(NESTED_OBJ1 VARCHAR, NESTED_ROW ROW(NESTED_OBJ2 VARCHAR))",
|
||||
},
|
||||
{"name": "row_column.nested_row", "type": "ROW(NESTED_OBJ2 VARCHAR)"},
|
||||
{"name": "row_column.nested_row.nested_obj2", "type": "VARCHAR"},
|
||||
{"name": "row_column.nested_obj1", "type": "VARCHAR"},
|
||||
]
|
||||
expected_data = [
|
||||
{
|
||||
"row_column": ["a1", ["a2"]],
|
||||
"row_column.nested_obj1": "a1",
|
||||
"row_column.nested_row": ["a2"],
|
||||
"row_column.nested_row.nested_obj2": "a2",
|
||||
},
|
||||
{
|
||||
"row_column": ["b1", ["b2"]],
|
||||
"row_column.nested_obj1": "b1",
|
||||
"row_column.nested_row": ["b2"],
|
||||
"row_column.nested_row.nested_obj2": "b2",
|
||||
},
|
||||
]
|
||||
|
||||
expected_expanded_cols = [
|
||||
{"name": "row_column.nested_obj1", "type": "VARCHAR"},
|
||||
{"name": "row_column.nested_row", "type": "ROW(NESTED_OBJ2 VARCHAR)"},
|
||||
{"name": "row_column.nested_row.nested_obj2", "type": "VARCHAR"},
|
||||
]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
@mock.patch.dict(
|
||||
"superset._feature_flags", {"PRESTO_EXPAND_DATA": True}, clear=True
|
||||
)
|
||||
def test_presto_expand_data_with_complex_array_columns(self):
|
||||
cols = [
|
||||
{"name": "int_column", "type": "BIGINT"},
|
||||
{
|
||||
"name": "array_column",
|
||||
"type": "ARRAY(ROW(NESTED_ARRAY ARRAY(ROW(NESTED_OBJ VARCHAR))))",
|
||||
},
|
||||
]
|
||||
data = [
|
||||
{"int_column": 1, "array_column": [[[["a"], ["b"]]], [[["c"], ["d"]]]]},
|
||||
{"int_column": 2, "array_column": [[[["e"], ["f"]]], [[["g"], ["h"]]]]},
|
||||
]
|
||||
actual_cols, actual_data, actual_expanded_cols = PrestoEngineSpec.expand_data(
|
||||
cols, data
|
||||
)
|
||||
expected_cols = [
|
||||
{"name": "int_column", "type": "BIGINT"},
|
||||
{
|
||||
"name": "array_column",
|
||||
"type": "ARRAY(ROW(NESTED_ARRAY ARRAY(ROW(NESTED_OBJ VARCHAR))))",
|
||||
},
|
||||
{
|
||||
"name": "array_column.nested_array",
|
||||
"type": "ARRAY(ROW(NESTED_OBJ VARCHAR))",
|
||||
},
|
||||
{"name": "array_column.nested_array.nested_obj", "type": "VARCHAR"},
|
||||
]
|
||||
expected_data = [
|
||||
{
|
||||
"array_column": [[["a"], ["b"]]],
|
||||
"array_column.nested_array": ["a"],
|
||||
"array_column.nested_array.nested_obj": "a",
|
||||
"int_column": 1,
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["b"],
|
||||
"array_column.nested_array.nested_obj": "b",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["c"], ["d"]]],
|
||||
"array_column.nested_array": ["c"],
|
||||
"array_column.nested_array.nested_obj": "c",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["d"],
|
||||
"array_column.nested_array.nested_obj": "d",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["e"], ["f"]]],
|
||||
"array_column.nested_array": ["e"],
|
||||
"array_column.nested_array.nested_obj": "e",
|
||||
"int_column": 2,
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["f"],
|
||||
"array_column.nested_array.nested_obj": "f",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": [[["g"], ["h"]]],
|
||||
"array_column.nested_array": ["g"],
|
||||
"array_column.nested_array.nested_obj": "g",
|
||||
"int_column": "",
|
||||
},
|
||||
{
|
||||
"array_column": "",
|
||||
"array_column.nested_array": ["h"],
|
||||
"array_column.nested_array.nested_obj": "h",
|
||||
"int_column": "",
|
||||
},
|
||||
]
|
||||
expected_expanded_cols = [
|
||||
{
|
||||
"name": "array_column.nested_array",
|
||||
"type": "ARRAY(ROW(NESTED_OBJ VARCHAR))",
|
||||
},
|
||||
{"name": "array_column.nested_array.nested_obj", "type": "VARCHAR"},
|
||||
]
|
||||
self.assertEqual(actual_cols, expected_cols)
|
||||
self.assertEqual(actual_data, expected_data)
|
||||
self.assertEqual(actual_expanded_cols, expected_expanded_cols)
|
||||
|
||||
def test_presto_extra_table_metadata(self):
|
||||
db = mock.Mock()
|
||||
db.get_indexes = mock.Mock(return_value=[{"column_names": ["ds", "hour"]}])
|
||||
db.get_extra = mock.Mock(return_value={})
|
||||
df = pd.DataFrame({"ds": ["01-01-19"], "hour": [1]})
|
||||
db.get_df = mock.Mock(return_value=df)
|
||||
PrestoEngineSpec.get_create_view = mock.Mock(return_value=None)
|
||||
result = PrestoEngineSpec.extra_table_metadata(db, "test_table", "test_schema")
|
||||
self.assertEqual({"ds": "01-01-19", "hour": 1}, result["partitions"]["latest"])
|
||||
|
||||
def test_presto_where_latest_partition(self):
|
||||
db = mock.Mock()
|
||||
db.get_indexes = mock.Mock(return_value=[{"column_names": ["ds", "hour"]}])
|
||||
db.get_extra = mock.Mock(return_value={})
|
||||
df = pd.DataFrame({"ds": ["01-01-19"], "hour": [1]})
|
||||
db.get_df = mock.Mock(return_value=df)
|
||||
columns = [{"name": "ds"}, {"name": "hour"}]
|
||||
result = PrestoEngineSpec.where_latest_partition(
|
||||
"test_table", "test_schema", db, select(), columns
|
||||
)
|
||||
query_result = str(result.compile(compile_kwargs={"literal_binds": True}))
|
||||
self.assertEqual("SELECT \nWHERE ds = '01-01-19' AND hour = 1", query_result)
|
||||
|
||||
def test_hive_get_view_names_return_empty_list(self):
|
||||
self.assertEqual(
|
||||
[], HiveEngineSpec.get_view_names(mock.ANY, mock.ANY, mock.ANY)
|
||||
)
|
||||
|
||||
def test_bigquery_sqla_column_label(self):
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("Col").name)
|
||||
label_expected = "Col"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("SUM(x)").name)
|
||||
label_expected = "SUM_x__5f110"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("SUM[x]").name)
|
||||
label_expected = "SUM_x__7ebe1"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
label = BigQueryEngineSpec.make_label_compatible(column("12345_col").name)
|
||||
label_expected = "_12345_col_8d390"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
def test_oracle_sqla_column_name_length_exceeded(self):
|
||||
col = column("This_Is_32_Character_Column_Name")
|
||||
label = OracleEngineSpec.make_label_compatible(col.name)
|
||||
self.assertEqual(label.quote, True)
|
||||
label_expected = "3b26974078683be078219674eeb8f5"
|
||||
self.assertEqual(label, label_expected)
|
||||
|
||||
def test_mssql_column_types(self):
|
||||
def assert_type(type_string, type_expected):
|
||||
type_assigned = MssqlEngineSpec.get_sqla_column_type(type_string)
|
||||
if type_expected is None:
|
||||
self.assertIsNone(type_assigned)
|
||||
else:
|
||||
self.assertIsInstance(type_assigned, type_expected)
|
||||
|
||||
assert_type("INT", None)
|
||||
assert_type("STRING", String)
|
||||
assert_type("CHAR(10)", String)
|
||||
assert_type("VARCHAR(10)", String)
|
||||
assert_type("TEXT", String)
|
||||
assert_type("NCHAR(10)", UnicodeText)
|
||||
assert_type("NVARCHAR(10)", UnicodeText)
|
||||
assert_type("NTEXT", UnicodeText)
|
||||
|
||||
def test_mssql_where_clause_n_prefix(self):
|
||||
dialect = mssql.dialect()
|
||||
spec = MssqlEngineSpec
|
||||
str_col = column("col", type_=spec.get_sqla_column_type("VARCHAR(10)"))
|
||||
unicode_col = column("unicode_col", type_=spec.get_sqla_column_type("NTEXT"))
|
||||
tbl = table("tbl")
|
||||
sel = (
|
||||
select([str_col, unicode_col])
|
||||
.select_from(tbl)
|
||||
.where(str_col == "abc")
|
||||
.where(unicode_col == "abc")
|
||||
)
|
||||
|
||||
query = str(
|
||||
sel.compile(dialect=dialect, compile_kwargs={"literal_binds": True})
|
||||
)
|
||||
query_expected = (
|
||||
"SELECT col, unicode_col \n"
|
||||
"FROM tbl \n"
|
||||
"WHERE col = 'abc' AND unicode_col = N'abc'"
|
||||
)
|
||||
self.assertEqual(query, query_expected)
|
||||
|
||||
def test_get_table_names(self):
|
||||
inspector = mock.Mock()
|
||||
inspector.get_table_names = mock.Mock(return_value=["schema.table", "table_2"])
|
||||
inspector.get_foreign_table_names = mock.Mock(return_value=["table_3"])
|
||||
|
||||
""" Make sure base engine spec removes schema name from table name
|
||||
ie. when try_remove_schema_from_table_name == True. """
|
||||
base_result_expected = ["table", "table_2"]
|
||||
base_result = BaseEngineSpec.get_table_names(
|
||||
database=mock.ANY, schema="schema", inspector=inspector
|
||||
)
|
||||
self.assertListEqual(base_result_expected, base_result)
|
||||
|
||||
""" Make sure postgres doesn't try to remove schema name from table name
|
||||
ie. when try_remove_schema_from_table_name == False. """
|
||||
pg_result_expected = ["schema.table", "table_2", "table_3"]
|
||||
pg_result = PostgresEngineSpec.get_table_names(
|
||||
database=mock.ANY, schema="schema", inspector=inspector
|
||||
)
|
||||
self.assertListEqual(pg_result_expected, pg_result)
|
||||
|
||||
def test_pg_time_expression_literal_no_grain(self):
|
||||
col = literal_column("COALESCE(a, b)")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, None)
|
||||
result = str(expr.compile(dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "COALESCE(a, b)")
|
||||
|
||||
def test_pg_time_expression_literal_1y_grain(self):
|
||||
col = literal_column("COALESCE(a, b)")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "DATE_TRUNC('year', COALESCE(a, b))")
|
||||
|
||||
def test_pg_time_expression_lower_column_no_grain(self):
|
||||
col = column("lower_case")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, None)
|
||||
result = str(expr.compile(dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "lower_case")
|
||||
|
||||
def test_pg_time_expression_lower_case_column_sec_1y_grain(self):
|
||||
col = column("lower_case")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, "epoch_s", "P1Y")
|
||||
result = str(expr.compile(dialect=postgresql.dialect()))
|
||||
self.assertEqual(
|
||||
result,
|
||||
"DATE_TRUNC('year', (timestamp 'epoch' + lower_case * interval '1 second'))",
|
||||
)
|
||||
|
||||
def test_pg_time_expression_mixed_case_column_1y_grain(self):
|
||||
col = column("MixedCase")
|
||||
expr = PostgresEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(dialect=postgresql.dialect()))
|
||||
self.assertEqual(result, "DATE_TRUNC('year', \"MixedCase\")")
|
||||
|
||||
def test_mssql_time_expression_mixed_case_column_1y_grain(self):
|
||||
col = column("MixedCase")
|
||||
expr = MssqlEngineSpec.get_timestamp_expr(col, None, "P1Y")
|
||||
result = str(expr.compile(dialect=mssql.dialect()))
|
||||
self.assertEqual(result, "DATEADD(year, DATEDIFF(year, 0, [MixedCase]), 0)")
|
||||
|
||||
def test_oracle_time_expression_reserved_keyword_1m_grain(self):
|
||||
col = column("decimal")
|
||||
expr = OracleEngineSpec.get_timestamp_expr(col, None, "P1M")
|
||||
result = str(expr.compile(dialect=oracle.dialect()))
|
||||
self.assertEqual(result, "TRUNC(CAST(\"decimal\" as DATE), 'MONTH')")
|
||||
|
||||
def test_pinot_time_expression_sec_1m_grain(self):
|
||||
col = column("tstamp")
|
||||
expr = PinotEngineSpec.get_timestamp_expr(col, "epoch_s", "P1M")
|
||||
result = str(expr.compile())
|
||||
self.assertEqual(
|
||||
result,
|
||||
'DATETIMECONVERT(tstamp, "1:SECONDS:EPOCH", "1:SECONDS:EPOCH", "1:MONTHS")',
|
||||
)
|
||||
|
||||
def test_column_datatype_to_string(self):
|
||||
example_db = get_example_database()
|
||||
sqla_table = example_db.get_table("energy_usage")
|
||||
dialect = example_db.get_dialect()
|
||||
col_names = [
|
||||
example_db.db_engine_spec.column_datatype_to_string(c.type, dialect)
|
||||
for c in sqla_table.columns
|
||||
]
|
||||
if example_db.backend == "postgresql":
|
||||
expected = ["VARCHAR(255)", "VARCHAR(255)", "DOUBLE PRECISION"]
|
||||
else:
|
||||
expected = ["VARCHAR(255)", "VARCHAR(255)", "FLOAT"]
|
||||
self.assertEqual(col_names, expected)
|
||||
@@ -24,6 +24,7 @@ from superset import db
|
||||
from superset.connectors.druid.models import DruidColumn, DruidDatasource, DruidMetric
|
||||
from superset.connectors.sqla.models import SqlaTable, SqlMetric, TableColumn
|
||||
from superset.utils.core import get_example_database
|
||||
from superset.utils.dict_import_export import export_to_dict
|
||||
|
||||
from .base_tests import SupersetTestCase
|
||||
|
||||
@@ -266,6 +267,26 @@ class DictImportExportTests(SupersetTestCase):
|
||||
imported_copy_table.export_to_dict(), imported_table.export_to_dict()
|
||||
)
|
||||
|
||||
def test_export_datasource_ui_cli(self):
|
||||
cli_export = export_to_dict(
|
||||
session=db.session,
|
||||
recursive=True,
|
||||
back_references=False,
|
||||
include_defaults=False,
|
||||
)
|
||||
self.get_resp("/login/", data=dict(username="admin", password="general"))
|
||||
resp = self.get_resp(
|
||||
"/databaseview/action_post", {"action": "yaml_export", "rowid": 1}
|
||||
)
|
||||
ui_export = yaml.safe_load(resp)
|
||||
self.assertEqual(
|
||||
ui_export["databases"][0]["database_name"],
|
||||
cli_export["databases"][0]["database_name"],
|
||||
)
|
||||
self.assertEqual(
|
||||
ui_export["databases"][0]["tables"], cli_export["databases"][0]["tables"]
|
||||
)
|
||||
|
||||
def test_import_druid_no_metadata(self):
|
||||
datasource, dict_datasource = self.create_druid_datasource(
|
||||
"pure_druid", id=ID_PREFIX + 1
|
||||
|
||||
@@ -221,6 +221,32 @@ class DruidFuncTestCase(unittest.TestCase):
|
||||
res = DruidDatasource.get_filters([filtr], [], column_dict)
|
||||
self.assertTrue(res.filter["filter"]["upperStrict"])
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("pydruid"), "pydruid not installed"
|
||||
)
|
||||
def test_get_filters_is_null_filter(self):
|
||||
filtr = {"col": "A", "op": "IS NULL"}
|
||||
col = DruidColumn(column_name="A")
|
||||
column_dict = {"A": col}
|
||||
res = DruidDatasource.get_filters([filtr], [], column_dict)
|
||||
self.assertEqual("selector", res.filter["filter"]["type"])
|
||||
self.assertEqual("", res.filter["filter"]["value"])
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("pydruid"), "pydruid not installed"
|
||||
)
|
||||
def test_get_filters_is_not_null_filter(self):
|
||||
filtr = {"col": "A", "op": "IS NOT NULL"}
|
||||
col = DruidColumn(column_name="A")
|
||||
column_dict = {"A": col}
|
||||
res = DruidDatasource.get_filters([filtr], [], column_dict)
|
||||
self.assertEqual("not", res.filter["filter"]["type"])
|
||||
self.assertIn("field", res.filter["filter"])
|
||||
self.assertEqual(
|
||||
"selector", res.filter["filter"]["field"].filter["filter"]["type"]
|
||||
)
|
||||
self.assertEqual("", res.filter["filter"]["field"].filter["filter"]["value"])
|
||||
|
||||
@unittest.skipUnless(
|
||||
SupersetTestCase.is_module_installed("pydruid"), "pydruid not installed"
|
||||
)
|
||||
|
||||
@@ -130,6 +130,7 @@ class UtilsTestCase(unittest.TestCase):
|
||||
self.assertEqual(parse_human_timedelta("now"), timedelta(0))
|
||||
self.assertEqual(parse_human_timedelta("1 year"), timedelta(366))
|
||||
self.assertEqual(parse_human_timedelta("-1 year"), timedelta(-365))
|
||||
self.assertEqual(parse_human_timedelta(None), timedelta(0))
|
||||
|
||||
@patch("superset.utils.core.datetime")
|
||||
def test_parse_past_timedelta(self, mock_datetime):
|
||||
|
||||
Reference in New Issue
Block a user