Commit Graph

89 Commits

Author SHA1 Message Date
Hendrik Schultze
6a8f441d54 fix: make auth for hive optional (#10172)
* fix: make auth for hive optional

This will make allow the admin to setup a root connection which can be impersonated.

* style: black lint conform

* style: clean up unnecessary condition expansion
2020-06-29 07:40:24 +03:00
Ville Brofeldt
b205ce32b0 feat: Add new timegrains and convert_dttm to Druid engine spec (#10160)
* feat: Add new timegrains and convert_dttm to Druid engine spec

* Add TemporalType enum and fix test case

* Remove DATETIME for athena (original spec)
2020-06-25 12:18:37 +03:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Maxime Beauchemin
280ade826c fix: dashboard export raises schema issues (#10060)
* fix: dashboard export raises schema issues

* Fix isort
2020-06-15 21:43:05 -07:00
serenajiang
8744dadca8 [csv upload][hive] support other delimiters (#9971)
Co-authored-by: serena-jiang <serena.jiang@airbnb.com>
2020-06-10 17:08:02 -07:00
Erik Ritter
e17da58a39 feat: add replace option to hive csv upload (#9764) 2020-06-10 08:57:33 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Xiang Fu
1d9dbcd9dc fix: pinot select query logic (#9954)
* Update how pinot handles selection logic

* Change DATETIMECONVERT argument to use single quote for literals
2020-06-02 23:18:35 -07:00
John Bodley
7f6dbf838e style: enforcing mypy typing for connectors (#9824)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-25 12:32:49 -07:00
Maxime Beauchemin
ee991963c7 fix: don't create examples db on 'superset init' (#9881)
* chore: don't create examples db on 'superset init'

* fix isort
2020-05-21 21:34:53 -07:00
Bogdan
3e8e441bfc feat: implement csv upload configuration func for the schema enforcement (#9734)
* Implement csv upload func for schema enforcement

Implement function controlled csv upload schema

Refactor + fix tests

Fixing hive as well

* Add explore_db to the extras

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-21 13:49:53 -07:00
Daniel Vaz Gaspar
3cc540019f fix(mssql): reverts #9644 and displays a better error msg (#9752) 2020-05-14 17:00:02 +01:00
Erik Ritter
83ec7365a7 feat: convert backend chart errors to the new error type (#9753) 2020-05-12 18:15:53 -07:00
serenajiang
5d167afb94 [hive][csv upload] make INTs BIGINTs (#9716) 2020-05-03 09:28:31 -07:00
Daniel Vaz Gaspar
2b59075d57 tests(engine_specs): full bigquery engine coverage (#9702) 2020-04-30 18:31:15 +01:00
Daniel Vaz Gaspar
527bee5051 tests(engine_specs): full postgres engine coverage (#9682) 2020-04-29 18:20:25 +01:00
Daniel Vaz Gaspar
516bdf6db1 fix(mssql): apply limit and set alias for functions (#9644) 2020-04-27 09:23:08 +01:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
Ville Brofeldt
982c234a50 fix: remove character set and collate column info by default (#9316)
* fix: remove character set and collate column info by default

* lint

* remove collation and charset info before compile
2020-03-17 21:34:39 +02:00
Ville Brofeldt
3682702e91 fix: add connection testing params for snowflake (#9272)
* fix: add connection testingt params for snowflake

* Linting
2020-03-11 06:51:57 +02:00
Ville Brofeldt
ef2ebbd570 Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
Ville Brofeldt
969bc87431 fix: Oracle fetch_query and datetime conversion (#9240) 2020-03-04 23:43:28 +02:00
Ville Brofeldt
7a91498cf1 fix: share column type matching between model and result set (#9161)
* Share column type matching between model and result set

* Address comments
2020-03-04 16:33:22 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
John Bodley
9f5f8e5d92 [mypy] Enforcing typing for db_engine_specs (#9138) 2020-02-17 23:08:11 -08:00
Xiang Fu
c7e2785d38 Support human readable datetime type for PinotDB (#9142)
* Support more time types for Pinot

* Address comments

* Address comments
2020-02-18 07:05:38 +02:00
John Bodley
a7e433a512 [mypy] Disallowing implicit optional (#9150) 2020-02-16 22:34:15 -08:00
Arian Treffer
0cf354cc88 Add support for Cockroach DB (#9043)
* Add support for Cockroach DB

* Add documentation, fix installation and style for CockroachDB

* Fix formatting for black
2020-02-08 12:07:02 +02:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
John Bodley
f24f671a90 [fix] Temporary filename for CSV upload to Hive (#9093) 2020-02-05 15:57:22 -08:00
Daniel Vaz Gaspar
24ec89a800 [csv upload] Use python's named temp file (#9047)
* [csv upload] Use python's named temp file

* [csv upload] stream write uploaded file

* Added missing ensure that configured path exists

* Fix, lint

* [csv upload] remove python's tmp file default delete behaviour
2020-02-04 10:28:32 -08:00
Erik Ritter
922eec2db2 [SQL Lab] Cache function names query (#9019) 2020-01-24 16:45:41 -08:00
Erik Ritter
a4b8e0b9e4 [SQL Lab] Add function names to autocomplete (#9012) 2020-01-24 11:08:26 -08:00
Naren
3bedee75d3 Add support for Dremio as a new source (#8939)
* Added spec for Dremio

* Installation instructions for Dremio

* added dependency for dremio

* Update dremio.py

* ASF header, dttm, set min version in setup.py

* Update installation.rst

* Update installation.rst

* fix tox 'Title underline too short.'

* change URI example

Co-authored-by: Naren <41924335+naren-dremio@users.noreply.github.com>
2020-01-10 17:37:40 +02:00
Grace Guo
93bcace20f Revert "Make select_star work with SQL Lab views (#8598)" (#8930)
This reverts commit 964e6db8
2020-01-06 12:04:36 -08:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Rob DiCiuccio
8fc814fc62 Enable running of tests in tests/db_engine_specs (#8902)
* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip
2019-12-31 14:45:33 +02:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
serenajiang
1719c4b5a3 [fix][sqllab] nullable booleans in dataframe (#8833)
* [fix] nullable booleans in dataframe

* added comment
2019-12-16 09:59:27 -08:00
Will Barrett
cc49aae245 re-enable pylint for database compatibility layer (#8772)
* Re-enable pylint for superset/db_engines/hive.py

* re-enable pylint for superset/db_engine_specs/elasticsearch.py

* Re-enable superset/db_engine_specs/__init__.py

* Re-enable pylint for superset/db_engine_specs/exasol.py
2019-12-07 16:15:12 -08:00
michellethomas
98a82a0720 Adding error message when there's no partition for a presto table using latest_partition (#8745) 2019-12-04 11:03:50 -08:00
Beto Dealmeida
5f4e3adfd2 Pass full response to query_cost_formatter (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Beto Dealmeida
964e6db800 Make select_star work with SQL Lab views (#8598)
* Make select_star work with SQL Lab views

* Fix mypy
2019-11-20 13:07:16 -08:00
Will Barrett
d70e0fc359 Fix for BigQuery connection checks and CSV uploads (#8511)
* Fix for BigQuery connection checks and CSV uploads

* Don't assume encrypted_extra will be populated

* Fix undefined method error

* Refactor to avoid circular import strangeness
2019-11-19 14:50:47 -08:00
Ville Brofeldt
90275fe991 Add convert_dttm method to SnowflakeEngineSpec (#8551) 2019-11-13 11:13:49 +02:00
axuew
763f59fc58 Add support for database engine SAP Hana (#8411)
* Add support for database engine SAP Hana

* Support hana services

Increase time, minute, and second

* Fix hana return string

* Fix formatting errors
2019-11-12 08:42:44 +02:00
Ville Brofeldt
49ea232c3a [fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
Beto Dealmeida
338a2b1a51 Allow user to customize query cost estimate (#8470)
* Allow user to customize query estimate

* Add docs; run black

* Update docs with types
2019-11-04 11:08:00 -08:00
serenajiang
9a29116d6b [typing] superset/models/core.py (#8284) 2019-11-04 21:04:53 +02:00