Commit Graph

35 Commits

Author SHA1 Message Date
Grace Guo
93bcace20f Revert "Make select_star work with SQL Lab views (#8598)" (#8930)
This reverts commit 964e6db8
2020-01-06 12:04:36 -08:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
serenajiang
1719c4b5a3 [fix][sqllab] nullable booleans in dataframe (#8833)
* [fix] nullable booleans in dataframe

* added comment
2019-12-16 09:59:27 -08:00
michellethomas
98a82a0720 Adding error message when there's no partition for a presto table using latest_partition (#8745) 2019-12-04 11:03:50 -08:00
Beto Dealmeida
5f4e3adfd2 Pass full response to query_cost_formatter (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Beto Dealmeida
964e6db800 Make select_star work with SQL Lab views (#8598)
* Make select_star work with SQL Lab views

* Fix mypy
2019-11-20 13:07:16 -08:00
Beto Dealmeida
338a2b1a51 Allow user to customize query cost estimate (#8470)
* Allow user to customize query estimate

* Add docs; run black

* Update docs with types
2019-11-04 11:08:00 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley
0a3b121244 [sql] Fixing datetime SQL literal (#8464) 2019-10-29 23:24:48 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Beto Dealmeida
04c3d1f58d Remove unused methods from Presto (#8350)
* Remove unused methods from Presto

* Remove tests

* Fix bug in PRESTO_SPLIT_VIEWS_FROM_TABLES

* Fix lint
2019-10-04 11:22:02 -07:00
Will Barrett
ec86d9de17 Fix lint in superset/db_engine_spec (#8338)
* Enable lint checking for files in db_engine_spec that have few to no
lint issues

* Enable lint and fix issue in db_engine_spec/mysql.py

* Enable pylint and fix lint for db_engine_spec/pinot.py

* Enable lint and fix issues for db_engine_specs/hive.py

* Enable lint and fix for db_engine_spec/presto.py

* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature

* Make flake8 happy after a number of pylint fixes

* Update db_engine_spec_test test cases related to Presto to support
different method naming

* automated reformatting

* One more pylint disable for druid.py

* Find the magic invocation that makes all the lint tools happy
2019-10-04 09:19:21 -07:00
Erik Ritter
8c708037e8 [Presto] Handle uncaught exception in get_create_view (#8304) 2019-10-01 07:45:12 -10:00
Beto Dealmeida
6df74c6b8e Remove __row_id (#8257) 2019-09-18 20:32:33 -07:00
Beto Dealmeida
12fb8e70cc Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida
4088a84eb4 Small fix for Presto dtype map (#8251)
* Small fix for Presto dtype map

* Add unit test
2019-09-18 12:46:50 -07:00
Beto Dealmeida
88777943fa Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida
4132d8fb0f Simplify and improve expand_data in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00
Beto Dealmeida
8847e10529 Allow users to estimate query cost before executing it (#8172)
* WIP

* Basic functionality working

* Enable per DB

* Show error message

* Small improvements

* Fix lint

* Address comments

* Address comments

* Remove logging, fix exception

* Fix tests

* Bump FAB requirements
2019-09-17 09:09:40 -07:00
Beto Dealmeida
b9be01fcd8 Handle int64 columns with missing data in SQL Lab (#8226)
* Handle int64 columns with missing data in SQL Lab

* Fix docstring

* Add unit test

* Small fix

* Small fixes

* Fix cursor description update

* Better fix

* Fix unit test, black

* Fix nan comparison in unit test
2019-09-17 08:16:09 -07:00
serenajiang
4e2d1c1a62 more detailed async error messages (#8164) 2019-09-09 09:09:15 -07:00
Ville Brofeldt
3250c5ac94 [bugfix] fix timegrain addon regression (#8165)
* Fix regression in time grain addons

* Revert privatization of time_grain_functions

* Fix test

* Rename variable

* Fix test

* Fix typing error

* Refactor and add tests

* Add TODO
2019-09-08 08:34:40 +02:00
serenajiang
624449816f [logging] add query id to SQL Lab logs (#8104)
* [logging] add query id to logs

* add query_id to hive and presto logging
2019-08-26 10:35:18 -07:00
Erik Ritter
5d8da6aeae [Hive Engine Spec] Fix latest partition logic (#8098) 2019-08-26 10:29:25 -07:00
Ville Brofeldt
fb51632e18 Add docstrings and typing to db_engine_specs and sql_parse (#8058)
* Add typing to db_engine_specs

* Add more type annotations and docstrings

* Add docstrings and typing to sql_parse and db_engine_specs

* Refine select_star

* Fix execute and add more docstrings

* Revert kwargs change from execute

* Remove redundant or

* Align view and table getter schema types

* Fix return type of latest_partition

* Remove some typing from presto

* Improve docstring for __extract_from_token
2019-08-22 06:29:32 +03:00
Beto Dealmeida
aae9b56232 Move more Presto nested types behind feature flag (#8075)
* Move more stuff behind ff

* Fix unit tests

* Fix more tests

* Fix lint
2019-08-20 16:10:55 -07:00
Beto Dealmeida
478d0969a8 Add feature flag for Presto expand data (#8056)
* Add feature flag for Presto expand data

* Fix unit tests

* Fix black

* Revert temporary file change
2019-08-15 20:10:05 -07:00
Erik Ritter
d58dbad076 [DB Engine] Support old and new Presto syntax (#7977) 2019-08-05 11:56:56 -07:00
Beto Dealmeida
a0ad18c8ba Quick fix (#7843) 2019-07-11 10:50:32 -07:00
John Bodley
ffa7c617ad [travis] Adding Black tox environment (#7809) 2019-07-01 18:05:15 -07:00
Beto Dealmeida
34ca2aefbe Truncate data that is expanded (#7777)
* Truncate data that is expanded

* Fix object check

* Small optimizations
2019-07-01 10:36:40 -07:00
Kim Truong
5994e432d4 feat: query based on all partitions (#7782)
* feat: query based on all partitions

* fix: address Beto's comments
2019-06-27 11:19:55 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
John Bodley
49390b3232 Revert "Allow bigint to be used as time column in Presto" #7670 (#7678) 2019-06-10 16:10:20 -07:00
Ville Brofeldt
95291facff Break up db_engine_specs (#7676)
* Refactor db_engine_specs into package

* Rename bigquery class and add epoch funcs

* Fix flake8 errors

* Dynamically load all engine specs

* Fix linting errors and unit tests

* Implement Snowflake epoch time funcs

* Implement Teradata epoch time func

* Fix presto datasource query and remove unused import

* Fix broken datasource query

* Add mypy ignore for false positive

* Add missing license files

* Make create_time_grains_tuple public

* Fix flake8 quote

* Fix incorrect licence header
2019-06-08 11:27:13 -07:00