Commit Graph

584 Commits

Author SHA1 Message Date
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
kevang
783667168b Add support for Exasol (#8343)
* Add support for Exasol

* add time grain functions for Exasol

* remove duplicate of

* override ExasolEngineSpec's fetch_data method

* remove duplicate https

* simplify super call
2019-10-06 14:43:45 +03:00
Beto Dealmeida
04c3d1f58d Remove unused methods from Presto (#8350)
* Remove unused methods from Presto

* Remove tests

* Fix bug in PRESTO_SPLIT_VIEWS_FROM_TABLES

* Fix lint
2019-10-04 11:22:02 -07:00
Will Barrett
ec86d9de17 Fix lint in superset/db_engine_spec (#8338)
* Enable lint checking for files in db_engine_spec that have few to no
lint issues

* Enable lint and fix issue in db_engine_spec/mysql.py

* Enable pylint and fix lint for db_engine_spec/pinot.py

* Enable lint and fix issues for db_engine_specs/hive.py

* Enable lint and fix for db_engine_spec/presto.py

* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature

* Make flake8 happy after a number of pylint fixes

* Update db_engine_spec_test test cases related to Presto to support
different method naming

* automated reformatting

* One more pylint disable for druid.py

* Find the magic invocation that makes all the lint tools happy
2019-10-04 09:19:21 -07:00
Erik Ritter
8c708037e8 [Presto] Handle uncaught exception in get_create_view (#8304) 2019-10-01 07:45:12 -10:00
Beto Dealmeida
6df74c6b8e Remove __row_id (#8257) 2019-09-18 20:32:33 -07:00
Beto Dealmeida
12fb8e70cc Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida
4088a84eb4 Small fix for Presto dtype map (#8251)
* Small fix for Presto dtype map

* Add unit test
2019-09-18 12:46:50 -07:00
Beto Dealmeida
88777943fa Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida
4132d8fb0f Simplify and improve expand_data in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00
Beto Dealmeida
8847e10529 Allow users to estimate query cost before executing it (#8172)
* WIP

* Basic functionality working

* Enable per DB

* Show error message

* Small improvements

* Fix lint

* Address comments

* Address comments

* Remove logging, fix exception

* Fix tests

* Bump FAB requirements
2019-09-17 09:09:40 -07:00
Beto Dealmeida
b9be01fcd8 Handle int64 columns with missing data in SQL Lab (#8226)
* Handle int64 columns with missing data in SQL Lab

* Fix docstring

* Add unit test

* Small fix

* Small fixes

* Fix cursor description update

* Better fix

* Fix unit test, black

* Fix nan comparison in unit test
2019-09-17 08:16:09 -07:00
serenajiang
4e2d1c1a62 more detailed async error messages (#8164) 2019-09-09 09:09:15 -07:00
Ville Brofeldt
3250c5ac94 [bugfix] fix timegrain addon regression (#8165)
* Fix regression in time grain addons

* Revert privatization of time_grain_functions

* Fix test

* Rename variable

* Fix test

* Fix typing error

* Refactor and add tests

* Add TODO
2019-09-08 08:34:40 +02:00
serenajiang
624449816f [logging] add query id to SQL Lab logs (#8104)
* [logging] add query id to logs

* add query_id to hive and presto logging
2019-08-26 10:35:18 -07:00
Erik Ritter
5d8da6aeae [Hive Engine Spec] Fix latest partition logic (#8098) 2019-08-26 10:29:25 -07:00
Ville Brofeldt
fb51632e18 Add docstrings and typing to db_engine_specs and sql_parse (#8058)
* Add typing to db_engine_specs

* Add more type annotations and docstrings

* Add docstrings and typing to sql_parse and db_engine_specs

* Refine select_star

* Fix execute and add more docstrings

* Revert kwargs change from execute

* Remove redundant or

* Align view and table getter schema types

* Fix return type of latest_partition

* Remove some typing from presto

* Improve docstring for __extract_from_token
2019-08-22 06:29:32 +03:00
Beto Dealmeida
aae9b56232 Move more Presto nested types behind feature flag (#8075)
* Move more stuff behind ff

* Fix unit tests

* Fix more tests

* Fix lint
2019-08-20 16:10:55 -07:00
Beto Dealmeida
478d0969a8 Add feature flag for Presto expand data (#8056)
* Add feature flag for Presto expand data

* Fix unit tests

* Fix black

* Revert temporary file change
2019-08-15 20:10:05 -07:00
Erik Ritter
d58dbad076 [DB Engine] Support old and new Presto syntax (#7977) 2019-08-05 11:56:56 -07:00
Ville Brofeldt
b856666ae2 Remove collation info from MSSQL column type (#7963) 2019-08-05 17:08:58 +03:00
Ville Brofeldt
07a76f83b1 [Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Michał Misiewicz
9b8bfbbe7f DruidEngineSpec: set allows_subquery to True (#7863) 2019-07-16 13:59:39 -07:00
Beto Dealmeida
a0ad18c8ba Quick fix (#7843) 2019-07-11 10:50:32 -07:00
Ville Brofeldt
346638a339 Add second and minute grains (#7810) 2019-07-03 23:04:34 +03:00
John Bodley
ffa7c617ad [travis] Adding Black tox environment (#7809) 2019-07-01 18:05:15 -07:00
Beto Dealmeida
34ca2aefbe Truncate data that is expanded (#7777)
* Truncate data that is expanded

* Fix object check

* Small optimizations
2019-07-01 10:36:40 -07:00
Kim Truong
5994e432d4 feat: query based on all partitions (#7782)
* feat: query based on all partitions

* fix: address Beto's comments
2019-06-27 11:19:55 -07:00
Ville Brofeldt
ebb7fbc5f0 bugfix: Oracle timegrains fail to render when wrapped in outer query (#7767)
* Add check if TimegrainExpression col is None

* Clean up TimestampExpression proxying logic

* Remove redundant optionality from init
2019-06-26 08:56:19 +03:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Maxime Beauchemin
859d6e7c6a chore: taking db dependencies out of requirements-dev.txt (#7605)
* chore: taking db dependencies out of requirements-dev.txt

The deps on mysqlclient and psycopg2

* Fix unit tests

* fix tox.ini

* fix tests
2019-06-23 22:37:41 -07:00
Ville Brofeldt
1c4092c61c Add csv upload support for BigQuery (#7756)
* Add extra_require for bigquery to setup.py

* Refactor df_to_db and add df upload capability for BigQuery

* Fix unit tests and clarify kwarg logic

* Fix flake8 errors

* Add minimum versions for bigquery dependencies

* wrap to_gbq in try-catch block and raise error if pandas-gbq is missing

* Fix linting error and make error more generic
2019-06-23 14:20:09 -07:00
John Bodley
49390b3232 Revert "Allow bigint to be used as time column in Presto" #7670 (#7678) 2019-06-10 16:10:20 -07:00
Ville Brofeldt
95291facff Break up db_engine_specs (#7676)
* Refactor db_engine_specs into package

* Rename bigquery class and add epoch funcs

* Fix flake8 errors

* Dynamically load all engine specs

* Fix linting errors and unit tests

* Implement Snowflake epoch time funcs

* Implement Teradata epoch time func

* Fix presto datasource query and remove unused import

* Fix broken datasource query

* Add mypy ignore for false positive

* Add missing license files

* Make create_time_grains_tuple public

* Fix flake8 quote

* Fix incorrect licence header
2019-06-08 11:27:13 -07:00