Commit Graph

67 Commits

Author SHA1 Message Date
Beto Dealmeida
31f406a526 feat: API endpoint to validate databases using separate parameters (#14420)
* feat: new endpoint for validating database parameters

* Rebase

* Remove broken tests
2021-05-12 18:32:10 -07:00
John Bodley
d8bb2d3e62 refactor(db_engine_specs): Removing top-level import of app (#14366)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-04-28 15:47:32 +12:00
Beto Dealmeida
c7112d1c48 feat: error messages for Presto connections (#14172)
* chore: rename connection errors

* feat: error messages for Presto connections

* Add unit tests

* Update docs/src/pages/docs/Miscellaneous/issue_codes.mdx

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
2021-04-16 15:49:47 -04:00
Beto Dealmeida
21f973f0bd feat: error messages when connecting to MSSQL (#14093)
* feat: error messages when connecting to MSSQL

* Address comments
2021-04-14 10:57:58 -07:00
Beto Dealmeida
c60a93db9c feat: add extract_errors to Postgres (#13997)
* feat: add extract_errors to Postgres

* Add unit tests

* Fix lint

* Fix unit tests
2021-04-08 13:24:54 -07:00
Jesse Yang
4789074309 fix(sqla-query): order by aggregations in Presto and Hive (#13739) 2021-04-01 18:10:17 -07:00
Nikola Gigić
609c3594ef feat(explore): Postgres datatype conversion (#13294)
* test

* unnecessary import

* fix lint

* changes

* fix lint

* changes

* changes

* changes

* changes

* answering comments & changes

* answering comments

* answering comments

* changes

* changes

* changes

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests
2021-03-12 10:36:43 +02:00
rijojoseph07
efeb5a93e6 feat(presto): add support for user impersonation (#13214)
* changes to support presto impersionation with ldap

* renamed method to match 30 char limit

* import spell check

* added presto impersonation test

* refactored impersionation code to generalize for extension

* moving config_args mutation to the update_connect_args_for_impersonation

* moving config_args mutation to the update_connect_args_for_impersonation

* nits

* refactored update_impersonation_config method name to match lint rule

* reduced comment line length

* black reformats

Co-authored-by: rijojoseph01 <rijo.joseph@myntra.com>
2021-02-22 15:01:33 +02:00
Ricardo Gândara Pinto
51195af4fa fix: Presto column_type_mappings time and timestamp (#12861)
* Fix presto column_type_mappings time and timestamp

* Added unit tests
2021-02-01 17:41:57 +02:00
Beto Dealmeida
dcef5e1381 fix: remove explicit cursor closing for BQ (#12836) 2021-01-29 14:32:47 -08:00
Beto Dealmeida
877b153331 feat: estimate query cost in Postgres (#12130)
* feat: estimate query cost in Postgres

* Add example in config

* Fix lint
2020-12-18 16:08:53 -08:00
Bogdan
0396c705d4 chore: Support SET & SHOW commands as read only SQL commands (#11868)
* Support SET & SHOW commands as read only SQL commands

* Move is_readonly definition into the engine spec

* Rename & use super()

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-12-03 10:44:11 -08:00
Jesse Yang
4cfcaebb61 feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509)
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
2020-11-13 22:35:10 -08:00
Erik Ritter
fa5dab85c4 feat: more specific presto error messages (#11099) 2020-09-29 21:50:58 -07:00
Erik Ritter
0f7e90f1f7 feat: improve presto query perf (#11069) 2020-09-28 08:52:00 -07:00
Kasia Kucharczyk
af1e8e8839 fix: enable several pylint rules partially in db_engines_specs module (#11000)
* Enabled mssql

* Enabling pylint rules in `mysql.py` from `db_engine_specs`:
- added variable to shorten line and enable `line-too-long`
- `import-error` seems not prompting anymore
- specified exception instead broad-except

* Refactored too long line in `oracle.py` from `db_engine_specs`. Enabled too-long-line pylint rule.

* Changed fstring into string `pinot.py` to reenable `f-string-without-interpolation` rule.

* Enabled `unused-import` rule and changed line to adjust to correct number of characters in `db_engine_specs/postgres.py`.

* Enabled unused-import pylint check in `db_engine_specs/presto.py`

* Enabled unused-import pylint check in `db_engine_specs/snowflake.py`

* Enabled unused-import pylint check in `db_engine_specs/sqlite.py`
2020-09-23 10:47:56 -07:00
Ville Brofeldt
bd140e018a fix(presto): default unknown types to string type (#10753)
* fix(presto): default unknown types to string type

* lint
2020-09-22 13:16:54 +03:00
Bogdan
19a9bcc9c5 chore: ci Initial hive support (#10593)
* Initial hive support

* Clone hive setup

* Make hive tests work locally

* Debugging presto failure

* sleep in dataset test

* Address comments

* Address comments

* Pin ipython, exclude new pylint rules

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-27 09:49:18 -07:00
Ville Brofeldt
9461f9c1e0 fix(db_engine_specs): improve Presto column type matching (#10658)
* fix: improve Presto column type matching

* add optional callback to type map and add tests

* lint

* change private to public
2020-08-24 22:42:07 +03:00
Erik Ritter
2055ecc1ba feat: refactor error components and add database issue code (#10473)
* feat: refactor error components and add database issue code

* Apply suggestions from code review

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>
2020-08-06 13:22:24 -07:00
Beto Dealmeida
4f678272d6 fix(presto): Handle ROW data stored as string (#10456)
* Handle ROW data stored as string

* Use destringify

* Fix mypy

* Fix mypy with cast

* Bypass pylint
2020-07-28 16:05:58 -07:00
Erik Ritter
b686004294 feat: add configuration for Presto cursor poll interval (#10191) 2020-06-30 08:56:22 -07:00
Ville Brofeldt
b205ce32b0 feat: Add new timegrains and convert_dttm to Druid engine spec (#10160)
* feat: Add new timegrains and convert_dttm to Druid engine spec

* Add TemporalType enum and fix test case

* Remove DATETIME for athena (original spec)
2020-06-25 12:18:37 +03:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
Ville Brofeldt
ef2ebbd570 Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
John Bodley
9f5f8e5d92 [mypy] Enforcing typing for db_engine_specs (#9138) 2020-02-17 23:08:11 -08:00
John Bodley
a7e433a512 [mypy] Disallowing implicit optional (#9150) 2020-02-16 22:34:15 -08:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Erik Ritter
922eec2db2 [SQL Lab] Cache function names query (#9019) 2020-01-24 16:45:41 -08:00
Erik Ritter
a4b8e0b9e4 [SQL Lab] Add function names to autocomplete (#9012) 2020-01-24 11:08:26 -08:00
Grace Guo
93bcace20f Revert "Make select_star work with SQL Lab views (#8598)" (#8930)
This reverts commit 964e6db8
2020-01-06 12:04:36 -08:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
serenajiang
1719c4b5a3 [fix][sqllab] nullable booleans in dataframe (#8833)
* [fix] nullable booleans in dataframe

* added comment
2019-12-16 09:59:27 -08:00
michellethomas
98a82a0720 Adding error message when there's no partition for a presto table using latest_partition (#8745) 2019-12-04 11:03:50 -08:00
Beto Dealmeida
5f4e3adfd2 Pass full response to query_cost_formatter (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Beto Dealmeida
964e6db800 Make select_star work with SQL Lab views (#8598)
* Make select_star work with SQL Lab views

* Fix mypy
2019-11-20 13:07:16 -08:00
Beto Dealmeida
338a2b1a51 Allow user to customize query cost estimate (#8470)
* Allow user to customize query estimate

* Add docs; run black

* Update docs with types
2019-11-04 11:08:00 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley
0a3b121244 [sql] Fixing datetime SQL literal (#8464) 2019-10-29 23:24:48 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Beto Dealmeida
04c3d1f58d Remove unused methods from Presto (#8350)
* Remove unused methods from Presto

* Remove tests

* Fix bug in PRESTO_SPLIT_VIEWS_FROM_TABLES

* Fix lint
2019-10-04 11:22:02 -07:00
Will Barrett
ec86d9de17 Fix lint in superset/db_engine_spec (#8338)
* Enable lint checking for files in db_engine_spec that have few to no
lint issues

* Enable lint and fix issue in db_engine_spec/mysql.py

* Enable pylint and fix lint for db_engine_spec/pinot.py

* Enable lint and fix issues for db_engine_specs/hive.py

* Enable lint and fix for db_engine_spec/presto.py

* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature

* Make flake8 happy after a number of pylint fixes

* Update db_engine_spec_test test cases related to Presto to support
different method naming

* automated reformatting

* One more pylint disable for druid.py

* Find the magic invocation that makes all the lint tools happy
2019-10-04 09:19:21 -07:00
Erik Ritter
8c708037e8 [Presto] Handle uncaught exception in get_create_view (#8304) 2019-10-01 07:45:12 -10:00
Beto Dealmeida
6df74c6b8e Remove __row_id (#8257) 2019-09-18 20:32:33 -07:00
Beto Dealmeida
12fb8e70cc Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida
4088a84eb4 Small fix for Presto dtype map (#8251)
* Small fix for Presto dtype map

* Add unit test
2019-09-18 12:46:50 -07:00
Beto Dealmeida
88777943fa Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida
4132d8fb0f Simplify and improve expand_data in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00