Commit Graph

55 Commits

Author SHA1 Message Date
Naren
3bedee75d3 Add support for Dremio as a new source (#8939)
* Added spec for Dremio

* Installation instructions for Dremio

* added dependency for dremio

* Update dremio.py

* ASF header, dttm, set min version in setup.py

* Update installation.rst

* Update installation.rst

* fix tox 'Title underline too short.'

* change URI example

Co-authored-by: Naren <41924335+naren-dremio@users.noreply.github.com>
2020-01-10 17:37:40 +02:00
Grace Guo
93bcace20f Revert "Make select_star work with SQL Lab views (#8598)" (#8930)
This reverts commit 964e6db8
2020-01-06 12:04:36 -08:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Rob DiCiuccio
8fc814fc62 Enable running of tests in tests/db_engine_specs (#8902)
* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip
2019-12-31 14:45:33 +02:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
serenajiang
1719c4b5a3 [fix][sqllab] nullable booleans in dataframe (#8833)
* [fix] nullable booleans in dataframe

* added comment
2019-12-16 09:59:27 -08:00
Will Barrett
cc49aae245 re-enable pylint for database compatibility layer (#8772)
* Re-enable pylint for superset/db_engines/hive.py

* re-enable pylint for superset/db_engine_specs/elasticsearch.py

* Re-enable superset/db_engine_specs/__init__.py

* Re-enable pylint for superset/db_engine_specs/exasol.py
2019-12-07 16:15:12 -08:00
michellethomas
98a82a0720 Adding error message when there's no partition for a presto table using latest_partition (#8745) 2019-12-04 11:03:50 -08:00
Beto Dealmeida
5f4e3adfd2 Pass full response to query_cost_formatter (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Beto Dealmeida
964e6db800 Make select_star work with SQL Lab views (#8598)
* Make select_star work with SQL Lab views

* Fix mypy
2019-11-20 13:07:16 -08:00
Will Barrett
d70e0fc359 Fix for BigQuery connection checks and CSV uploads (#8511)
* Fix for BigQuery connection checks and CSV uploads

* Don't assume encrypted_extra will be populated

* Fix undefined method error

* Refactor to avoid circular import strangeness
2019-11-19 14:50:47 -08:00
Ville Brofeldt
90275fe991 Add convert_dttm method to SnowflakeEngineSpec (#8551) 2019-11-13 11:13:49 +02:00
axuew
763f59fc58 Add support for database engine SAP Hana (#8411)
* Add support for database engine SAP Hana

* Support hana services

Increase time, minute, and second

* Fix hana return string

* Fix formatting errors
2019-11-12 08:42:44 +02:00
Ville Brofeldt
49ea232c3a [fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
Beto Dealmeida
338a2b1a51 Allow user to customize query cost estimate (#8470)
* Allow user to customize query estimate

* Add docs; run black

* Update docs with types
2019-11-04 11:08:00 -08:00
serenajiang
9a29116d6b [typing] superset/models/core.py (#8284) 2019-11-04 21:04:53 +02:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley
0a3b121244 [sql] Fixing datetime SQL literal (#8464) 2019-10-29 23:24:48 -07:00
Daniel Vaz Gaspar
a757b43164 [db engine] Add support for Elasticsearch (#8441)
* [db engine] Add support for Elasticsearch
2019-10-28 16:04:14 +00:00
Maxime Beauchemin
8f246cec34 chore: disable another flaky cypress test (#8423)
Also fix isort issue on master
2019-10-22 11:50:23 +01:00
Beto Dealmeida
cca689b712 Add improved typed casting to BigQuery (#8331) 2019-10-21 09:19:24 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
kevang
783667168b Add support for Exasol (#8343)
* Add support for Exasol

* add time grain functions for Exasol

* remove duplicate of

* override ExasolEngineSpec's fetch_data method

* remove duplicate https

* simplify super call
2019-10-06 14:43:45 +03:00
Beto Dealmeida
04c3d1f58d Remove unused methods from Presto (#8350)
* Remove unused methods from Presto

* Remove tests

* Fix bug in PRESTO_SPLIT_VIEWS_FROM_TABLES

* Fix lint
2019-10-04 11:22:02 -07:00
Will Barrett
ec86d9de17 Fix lint in superset/db_engine_spec (#8338)
* Enable lint checking for files in db_engine_spec that have few to no
lint issues

* Enable lint and fix issue in db_engine_spec/mysql.py

* Enable pylint and fix lint for db_engine_spec/pinot.py

* Enable lint and fix issues for db_engine_specs/hive.py

* Enable lint and fix for db_engine_spec/presto.py

* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature

* Make flake8 happy after a number of pylint fixes

* Update db_engine_spec_test test cases related to Presto to support
different method naming

* automated reformatting

* One more pylint disable for druid.py

* Find the magic invocation that makes all the lint tools happy
2019-10-04 09:19:21 -07:00
Erik Ritter
8c708037e8 [Presto] Handle uncaught exception in get_create_view (#8304) 2019-10-01 07:45:12 -10:00
Beto Dealmeida
6df74c6b8e Remove __row_id (#8257) 2019-09-18 20:32:33 -07:00
Beto Dealmeida
12fb8e70cc Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida
4088a84eb4 Small fix for Presto dtype map (#8251)
* Small fix for Presto dtype map

* Add unit test
2019-09-18 12:46:50 -07:00
Beto Dealmeida
88777943fa Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida
4132d8fb0f Simplify and improve expand_data in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00
Beto Dealmeida
8847e10529 Allow users to estimate query cost before executing it (#8172)
* WIP

* Basic functionality working

* Enable per DB

* Show error message

* Small improvements

* Fix lint

* Address comments

* Address comments

* Remove logging, fix exception

* Fix tests

* Bump FAB requirements
2019-09-17 09:09:40 -07:00
Beto Dealmeida
b9be01fcd8 Handle int64 columns with missing data in SQL Lab (#8226)
* Handle int64 columns with missing data in SQL Lab

* Fix docstring

* Add unit test

* Small fix

* Small fixes

* Fix cursor description update

* Better fix

* Fix unit test, black

* Fix nan comparison in unit test
2019-09-17 08:16:09 -07:00
serenajiang
4e2d1c1a62 more detailed async error messages (#8164) 2019-09-09 09:09:15 -07:00
Ville Brofeldt
3250c5ac94 [bugfix] fix timegrain addon regression (#8165)
* Fix regression in time grain addons

* Revert privatization of time_grain_functions

* Fix test

* Rename variable

* Fix test

* Fix typing error

* Refactor and add tests

* Add TODO
2019-09-08 08:34:40 +02:00
serenajiang
624449816f [logging] add query id to SQL Lab logs (#8104)
* [logging] add query id to logs

* add query_id to hive and presto logging
2019-08-26 10:35:18 -07:00
Erik Ritter
5d8da6aeae [Hive Engine Spec] Fix latest partition logic (#8098) 2019-08-26 10:29:25 -07:00
Ville Brofeldt
fb51632e18 Add docstrings and typing to db_engine_specs and sql_parse (#8058)
* Add typing to db_engine_specs

* Add more type annotations and docstrings

* Add docstrings and typing to sql_parse and db_engine_specs

* Refine select_star

* Fix execute and add more docstrings

* Revert kwargs change from execute

* Remove redundant or

* Align view and table getter schema types

* Fix return type of latest_partition

* Remove some typing from presto

* Improve docstring for __extract_from_token
2019-08-22 06:29:32 +03:00
Beto Dealmeida
aae9b56232 Move more Presto nested types behind feature flag (#8075)
* Move more stuff behind ff

* Fix unit tests

* Fix more tests

* Fix lint
2019-08-20 16:10:55 -07:00
Beto Dealmeida
478d0969a8 Add feature flag for Presto expand data (#8056)
* Add feature flag for Presto expand data

* Fix unit tests

* Fix black

* Revert temporary file change
2019-08-15 20:10:05 -07:00
Erik Ritter
d58dbad076 [DB Engine] Support old and new Presto syntax (#7977) 2019-08-05 11:56:56 -07:00
Ville Brofeldt
b856666ae2 Remove collation info from MSSQL column type (#7963) 2019-08-05 17:08:58 +03:00
Ville Brofeldt
07a76f83b1 [Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Michał Misiewicz
9b8bfbbe7f DruidEngineSpec: set allows_subquery to True (#7863) 2019-07-16 13:59:39 -07:00
Beto Dealmeida
a0ad18c8ba Quick fix (#7843) 2019-07-11 10:50:32 -07:00
Ville Brofeldt
346638a339 Add second and minute grains (#7810) 2019-07-03 23:04:34 +03:00
John Bodley
ffa7c617ad [travis] Adding Black tox environment (#7809) 2019-07-01 18:05:15 -07:00
Beto Dealmeida
34ca2aefbe Truncate data that is expanded (#7777)
* Truncate data that is expanded

* Fix object check

* Small optimizations
2019-07-01 10:36:40 -07:00
Kim Truong
5994e432d4 feat: query based on all partitions (#7782)
* feat: query based on all partitions

* fix: address Beto's comments
2019-06-27 11:19:55 -07:00
Ville Brofeldt
ebb7fbc5f0 bugfix: Oracle timegrains fail to render when wrapped in outer query (#7767)
* Add check if TimegrainExpression col is None

* Clean up TimestampExpression proxying logic

* Remove redundant optionality from init
2019-06-26 08:56:19 +03:00