Commit Graph

60 Commits

Author SHA1 Message Date
Antonio Rivero Martinez
ad758c0802 feat(ssh_tunnel): Rename allow_ssh_tunneling and change the default value to False (#22723) 2023-01-17 19:11:41 -08:00
Zamar
001100ddf0 feat(db_engine_specs): big query cost estimation (#21325)
Co-authored-by: zamar roura <zamar.roura@cabify.es>
Co-authored-by: Zamar Roura <zamarfazal@gmail.com>
2023-01-09 08:56:19 +02:00
Antonio Rivero Martinez
60a617eaba fix(bigquery): Properly display errors for BigQuery DBs (#22349) 2022-12-07 13:34:30 -05:00
EugeneTorap
7bc5f04368 chore: Bump bigquery libs (#21466) 2022-12-02 12:48:36 +02:00
Hugh A. Miles II
e23efefc46 feat: refactor all get_sqla_engine to use contextmanager in codebase (#21943) 2022-11-15 13:45:14 -05:00
Antonio Rivero Martinez
95b4c7b7fe chore(bigquery): Add extra logging for BigQuery exceptions so we can have better insight on exceptions (#22024) 2022-11-09 22:56:08 -05:00
Ville Brofeldt
dc7399540b chore: bubble up more db error messages (#21982) 2022-11-01 07:54:27 +02:00
Antonio Rivero Martinez
203b289021 feat(bigquery): Custom message when Service Account doesnt have the correct Roles and Permissions (#21838) 2022-10-26 19:44:09 -04:00
Elizabeth Thompson
e98943e580 refactor: pass all properties to validate_parameters (#21487) 2022-10-03 17:48:54 -07:00
Elizabeth Thompson
ef78ec6b30 fix: catch error when masking encrypted extra is none (#21570) 2022-10-02 19:49:01 -07:00
Beto Dealmeida
34a79add04 feat: filter parameters from DB API (#21248) 2022-09-02 11:50:04 -07:00
Beto Dealmeida
7e501cd816 fix: BigQuery get_parameters_from_uri (#20966) 2022-08-03 17:27:40 -07:00
Diego Medina
5afeba34bd fix(viz): BigQuery time grain 'minute'/'second' throws an error (#20350) 2022-06-21 10:06:52 -03:00
Ramunas Balukonis
694f75d376 fix: "Week Staring Monday" time grain for BigQuery (#20091)
* Week Staring From Monday added for BQ

* lint fix
2022-05-26 12:31:32 +03:00
Daniel Vaz Gaspar
4fdf230a56 feat: deprecate /superset/extra_table_metadata migrate to api v1 (#19921)
* feat: deprecate /superset/extra_table_metadata migrate to api v1

* use can_read to table_extra_metadata

* troubleshoot sqlite

* fix test

* fix test

* fix test

* fix frontend test on sqllab
2022-05-04 14:55:52 +01:00
Craig Rueda
f64d654de3 chore(database): Creating helper make_url_safe to wrap potential errors (#19526)
* Creating helper make_url_safe to wrap potential errors

* Fixing imports

* Fixing imports again

* Adding comment

* Linting

* Fixing test

* Fixing test again...

* Fixing import
2022-04-05 11:17:30 -07:00
Ville Brofeldt
a619cb4ea9 chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
aniaan
5a1c68177e fix(elasticsearch): time_zone setting does not work for cast datetime expressions (#17048)
* fix(elasticsearch): cast does not take effect for time zone settings

* test(elasticsearch): add test

* fix(test): fix typo

* docs(elasticsearch): add annotation

* docs(elasticsearch): add time_zone desc

* docs(elasticsearch): fix typo

* refactor(db_engine): change convert_dttm signature

* fix(test): fix test

* fix(es): add try catch

* fix(test): fix caplog

* fix(test): fix typo
2021-11-25 11:58:44 +02:00
Craig Rueda
6a63a21e06 chore: Bumping pyarrow / pandas (#17290)
* Bumping pyarrow / pandas

* Appeasing the pylint gods

* Adjusting UPDATING.md

* Buping testing requirements
2021-10-31 10:47:02 -07:00
Yongjie Zhao
bedb8f4dff fix(bigquery): calculated column cannot orderby in BigQuery (#17196)
* fix(bigquery): calculated column cannot orderby in BigQuery

* typo

* add ut

* fix lint
2021-10-22 22:42:58 +08:00
John Bodley
b9ff85d9ea chore(engine): Translate fractional time grains—requires @superset-ui bump (#17078)
* chore(engine): Translate fractional time grains

* Bump @superset-ui

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-10-12 21:40:48 -07:00
Beto Dealmeida
c993c5845f fix(BigQuery): explicitly quote columns in select_star (#16822)
* fix (BigQuery): explicitly quote columns in select_star

* Fix test

* Fix SELECT * in BQ

* Add unit tests

* Remove type changes
2021-10-06 07:43:32 -07:00
AAfghahi
aa747219ad feat: Add Private Google Sheets to dynamic form (#16628)
* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* Regenerate package-lock.json

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-28 17:08:50 -07:00
Yongjie Zhao
21f98ddc21 fix: catch exception when create connection (#16692)
* fix: catch exception when create connection

* fix lint

* added UT
2021-09-16 15:55:57 +08:00
John Bodley
be7065faf8 chore(pylint): Reenable raise-missing-from check (#16266)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 10:26:10 +12:00
John Bodley
0df15bf207 chore(pylint): Reenable import-outside-toplevel check (#16263)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 05:20:13 +12:00
Hugh A. Miles II
d4480f5c9a feat: Database Connection UI (#14881) 2021-07-01 14:40:27 -07:00
AAfghahi
743d9cc928 feat: Better Errors in SQL Lab (#15432)
* snowflake errors

* added big query

* added to setup error messages, first test

* all big query testing added

* added snowflake test

* added syntax error

* added syntax errors to most used databases
2021-06-29 16:48:27 -07:00
Beto Dealmeida
90d9097841 fix: validate DB-specific parameters (#15155)
* fix: validate DB-specific parameters

* Fix lint

* Update test

* Fix lint/test

* Fix lint

* Update superset/databases/api.py
2021-06-14 18:44:18 -07:00
Hugh A. Miles II
ae8378a50e fix: Big Query Edit Form (#14850)
* fix big quert edit form

* fix test

* fix api test

* fix test
2021-05-26 16:13:26 -04:00
Beto Dealmeida
8b1a117d24 feat: return parameters only for DB with default driver (#14803)
* WIP

* Fix sorting of DBs
2021-05-25 14:46:54 -07:00
Hugh A. Miles II
c72894725e fix: Fix Big Query API for POST w/ no parameters (#14822)
* Update schemas.py

* Update bigquery.py

* Fix tests

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-05-25 17:13:17 -04:00
Hugh A. Miles II
6d33432b58 feat: Create BigQuery Parameters for DatabaseModal (#14721) 2021-05-23 12:45:48 -04:00
Beto Dealmeida
31f406a526 feat: API endpoint to validate databases using separate parameters (#14420)
* feat: new endpoint for validating database parameters

* Rebase

* Remove broken tests
2021-05-12 18:32:10 -07:00
Ben Reinhart
2a1235c0c2 fix: Cleanup serialization and hashing code (#14317) 2021-04-26 14:04:40 -07:00
John Bodley
b0f8f6b6ad fix(hive): Use parquet rather than textfile when uploading CSV files to Hive (#14240)
* fix(hive): Use parquet rather than textfile when uploading CSV files

* [csv/excel]: Use stream rather than temporary file

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-04-24 18:17:30 +12:00
Hugh A. Miles II
392d8a8107 fix: Handle bad permission errors for bigquery test connections (#14147)
* starter make file

* yea

* move messaging to config

* Delete Makefile

* remove

* checkout

* check for db generic errors

* checkout bad files

* add proper message

* add docs for new permissions codes

* Update superset/errors.py

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>

* Update superset/errors.py

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>

* rename var

* starter regex

* fix

* fix

* fix linting

* update test

* yerp

* fixed test

* added regex

* Apply suggestions from code review

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* address comments

* update docs

* prettier

* fux

* add space

* Update errors.py

* Update types.ts

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-04-20 19:19:36 -04:00
Ville Brofeldt
7e99d768af feat(bigquery): implement custom minute time grains (#12581)
* feat(bigquery): implement custom minute time grains

* address review comment
2021-01-19 14:32:33 +02:00
Beto Dealmeida
54bf70733f feat: run BQ queries as single statement (#11904)
* feat: run BQ queries as single statement

* Update deps

* Fix lint

* Update superset/sql_lab.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-03 13:20:23 -08:00
Maxime Beauchemin
796a2a6924 fix: error around latest partition in BigQuery (#11274)
* fix: error around latest partition in BigQuery

* lint

* Going with a backend-first approach

* fix test

* add an extra test
2020-10-20 21:34:46 -07:00
Kasia Kucharczyk
a6f258747b fix: enable pylint rules in db_engine_specs module (#10998)
* Athena.py: cretaed variable for too long line, removed disabled pylint rule.

* Enabled `line-too-long` in `drill.py`. Added variable for too long string.

* Reformatted hana.py:
- changed return statement into two lines to keep line lenght limit

* Enabling pylint rule in hive.py:
- `no-name-in-module` for pyhive in several places
- `line-too-long` and adding new lines
- `unused-import` is not invalid anymore

* Enabled `line-too-long` in `kylin.py`. Added variable for too long string.

* Enabled `unused_import` in `base.py` from db_engine_specs module.

* Enabled `unused_import` in `bigquery.py` from db_engine_specs module.

* Enabled `unused_import` in `druid.py` from db_engine_specs module.

* fix: athena datetime string converting
2020-09-22 13:44:45 -07:00
Ville Brofeldt
dd0bc472e3 refactor(database): use SupersetResultSet on SqlaTable.get_df() (#10707)
* refactor(database): use SupersetResultSet on SqlaTable.get_df()

* lint

* change cypress test
2020-08-28 21:12:03 +03:00
Erik Ritter
2055ecc1ba feat: refactor error components and add database issue code (#10473)
* feat: refactor error components and add database issue code

* Apply suggestions from code review

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>
2020-08-06 13:22:24 -07:00
Ville Brofeldt
b205ce32b0 feat: Add new timegrains and convert_dttm to Druid engine spec (#10160)
* feat: Add new timegrains and convert_dttm to Druid engine spec

* Add TemporalType enum and fix test case

* Remove DATETIME for athena (original spec)
2020-06-25 12:18:37 +03:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Daniel Vaz Gaspar
2b59075d57 tests(engine_specs): full bigquery engine coverage (#9702) 2020-04-30 18:31:15 +01:00
Ville Brofeldt
ef2ebbd570 Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
John Bodley
9f5f8e5d92 [mypy] Enforcing typing for db_engine_specs (#9138) 2020-02-17 23:08:11 -08:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
serenajiang
1719c4b5a3 [fix][sqllab] nullable booleans in dataframe (#8833)
* [fix] nullable booleans in dataframe

* added comment
2019-12-16 09:59:27 -08:00