Commit Graph

210 Commits

Author SHA1 Message Date
cccs-Dustin
02ef9ca4cd feat(sqllab): Add a configuration option to disable data preview (#19104) 2022-03-17 15:22:57 +02:00
Sujith Kumar S
7e51b200b4 fix(mssql): support top syntax for limiting queries (#18746)
* SQL-TOP Fix For Database Engines

MSSQL is not supporting LIMIT syntax in SQLs. For limiting the rows, MSSQL having a different keyword TOP. Added fixes for handling the TOP and LIMIT clauses based on the database engines.

* Teradata code for top clause handling removed from teradata.py

Teradata code for top clause handling removed from teradata.py file, since we added generic section in base engine for the same.

* Changes to handle CTE along with TOP in complex SQLs

Added changes to handle TOP command in CTEs, for DB Engines which are not supporting inline CTEs.

* Test cases for TOP unit testing in MSSQL

Added multiple unit test cases for MSSQL top command handling and also along with CTEs

* Corrected the select_keywords name key in basengine

Corrected the select_keywords name key in basengine

* Changes based on as per review.

made the required corrections based on code review to keep good code readability and code cleanliness.

* Review changes to correct lint and typo issues

Made the changes according to the review comments.

* fix linting errors

* fix teradata tests

* add coverage

* lint

* Code cleanliness

Moved the top/limit flag check from sql_lab to core.

* Changed for code cleanliness

Changes for keeping code cleanliness

* Corrected lint issue

Corrected lint issue.

* Code cleanliness

Code cleanliness

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-02-21 09:58:39 +02:00
Beto Dealmeida
1c0b141e2d feat: allow assets to be managed externally (#18093)
* feat: allow assets to be managed externally

* Use server_default
2022-01-24 11:03:01 -08:00
Đặng Minh Dũng
0b67fe1beb feat: Trino Authentications (#17593)
* feat: support Trino Authentications

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* docs: Trino Authentications

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>
2022-01-14 21:02:47 -08:00
Mikhail Kumachev
d2d4f8eb44 feat: Add support for Azure Data Explorer (Kusto) db engine spec (#17898)
* Add two Kusto engine specs: KQL and SQL. Some minor changes in core code to support Kusto engine specs.

* Remove redundant imports and logging.

* docs: Kusto sqlalchemy docs

* fix: Fix mypy and linting errors

* fix: Handle Black vs Pylint checks

* fix: isort problem

* refactor: Merge kustosql and kustokql in the single kusto module

* test: Add tests for Kusto db spec

* feat: Schema override does not require in KQL anymore

* Removed redundant imports.

* Added ".show" queries to readonly query determination.

* Fixed some bugs.
Added tests for convert_dttm.

* Fixed major sqlalchemy-kusto version.

* Fixed by isort.

Co-authored-by: Eugene Bikkinin <xnegxneg@gmail.com>
Co-authored-by: k.tomak <k.tomak@dodopizza.com>
Co-authored-by: Eugene Bikkinin <e.bikkinin@dodopizza.com>
2022-01-10 14:42:20 +02:00
Grace Guo
a01c4c95db fix: [alert] should run alert query from report account (#17499)
* fix: [alert] should run alert query from report account

* add solution2: override username for get_df

* add integration test
2021-12-16 12:12:43 +00:00
Shiva Raisinghani
ef3afbde82 refactor: Repeated boilerplate code between upload to database forms (#16756)
* abstract boilerplate code into class and rename csv to file

* add db migration

* fix some stuff

* more renaming of csv to file

* rename in translations

* update down revision

* update down revision

* bump chart version

* switch to alter column name approach in db migration

* fix db migration for MySQL

* db migration conflict
2021-10-25 13:53:06 +03:00
ofekisr
0d0c759cfe refactor: sql lab command: separate concerns into different modules (#16917)
* chore move sql_execution_context to sqllab package

* add new helper methods into base Dao

* refactor separate get existing query concern from command

* refactor separate query access validation concern

* refactor separate get query's database concern from command

* refactor separate get query rendering concern from command

* refactor sqllab_execution_context

* refactor separate creating payload for view

* chore decouple command from superset app

* fix pylint issues

* fix failed tests

* fix pylint issues

* fix failed test

* fix failed black

* fix failed black

* fix failed test
2021-10-03 11:15:46 +03:00
AAfghahi
aa747219ad feat: Add Private Google Sheets to dynamic form (#16628)
* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* Regenerate package-lock.json

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-28 17:08:50 -07:00
Yongjie Zhao
21f98ddc21 fix: catch exception when create connection (#16692)
* fix: catch exception when create connection

* fix lint

* added UT
2021-09-16 15:55:57 +08:00
John Bodley
fb4650a6eb chore(pylint): Remove top-level disable (#16589)
* chore(pylint): Remove top-level disable

* Update examples.py

* Update command.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-09-15 09:30:23 -07:00
aniaan
9e00e4e8cc fix(dataset): create ES-View dataset raise exception #16623 (#16624)
* fix(dataset): create es-view dataset raise exception #16623

* fix(database): fix has_view logic

* refactor(database): fix logic

* style(lint): remove unused typing

* fix(test): add test case

* fix(test): fix test case
2021-09-14 22:29:29 +08:00
John Bodley
7e4c940314 chore(pylint): Enable useless-suppression check (#16388)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-24 03:58:41 +12:00
John Bodley
24b43beff9 chore(pylint): Bump Pylint to 2.9.6 (#16146)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-14 10:32:28 +12:00
Hugh A. Miles II
4ae88ce3b4 chore: bump py version for integration test (#16213)
* bump py version for integration test

* bump py

* bump py

* remove files

* lock pylint

* add not-callable
2021-08-13 12:44:32 -04:00
Hugh A. Miles II
7d0f6ab7f5 fix: Fixing schemas_allowed_for_upload field in database connection UX (#15750) 2021-07-19 12:12:37 -04:00
Amit Miran
62a8f2e193 chore(python-testing): move memoized tests to unit tests (#15507)
* chore: move memoized test into a separated file

create integration test workflow

* chore: create unit test workflow to run purely pytest

* fix: bad reference

* fix: remove pip requirements bc there aren't any yet

* temp: install unit dependencies directly

* fix: --rootdir=

* fix: try to run only unit test

* chore: decouple memoized as separated module

* fix: bring back dependencies bc superset top-level module is coupled to flask and others so no reason no to do it

* fix: reference

* fix: pre-commit

* fix: pylint
2021-07-12 10:00:18 +03:00
Beto Dealmeida
a59bbbc544 fix: edit BQ w/o encrypted_extra (#15048)
* fix: edit BQ w/o encrypted_extra

* Fix lint
2021-06-08 15:04:04 -07:00
Hugh A. Miles II
ae8378a50e fix: Big Query Edit Form (#14850)
* fix big quert edit form

* fix test

* fix api test

* fix test
2021-05-26 16:13:26 -04:00
Hugh A. Miles II
6d33432b58 feat: Create BigQuery Parameters for DatabaseModal (#14721) 2021-05-23 12:45:48 -04:00
Hugh A. Miles II
ec0a7922fe feat: Add Parameters fields to GET Database (#14653) 2021-05-19 10:25:20 -04:00
AAfghahi
4f5c537250 feat: Configuration Method and expanded parameters for Database Model (#14451)
* db migration for dbs

* first draft at logic

* added unit tests

* revisions

* use strings for db values

* lint and revisions to tests

* changed test back

* added revisions for testing

* Update superset/databases/commands/update.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/databases/schemas.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/models/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/databases/commands/update.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* got rid of extra imports added new test

* Update superset/databases/schemas.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-05-14 18:19:17 -07:00
Ville Brofeldt
97c9e37c24 perf: memoize db_engine_spec in database (#14638)
* perf: memoize db_engine_spec in sqla table classes

* remove extended cypress timeouts
2021-05-14 12:49:35 +03:00
AAfghahi
e507508b48 feat: Logic added to limiting factor column in Query model (#13521)
* Sqllab limit

* Add migration script

* Set default values

* initial push

* revisions

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* moving migration to separate PR

* with migration

* revisions

* Fix apply_limit_to_sql

* all but tests

* added unit tests

* revisions

* Update superset/sql_lab.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/sql_parse.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* fixed black issue

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* updated logic

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-04-30 15:15:18 -07:00
Sam Faber-Manning
11260b3117 chore: add stack trace to all calls of logger.error (#14382) 2021-04-30 09:22:05 -07:00
Beto Dealmeida
13bf023100 feat: improve engine spec discoverability (#14204)
* feat: improve engine spec discoverability

* Address comments

* Fix tests
2021-04-19 17:24:22 -07:00
Craig Rueda
a49e0b2037 feat: Adding encrypted field factory (#14109)
* First cut at adding enc type factory

* Finalized enc type factory

* Adding unit test

* PyLinting

* Adding license

* Apply suggestions from code review

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>

* Moving things from enc -> encrypt

* CI commit

* One more fix

* Tweaking config name

* Fixing broken test

* Fixing broken test again

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
2021-04-16 09:01:18 -07:00
Jesse Yang
bd1d6acb0c fix(query): order by adhoc metrics should trigger group by (#13434)
* fix(query): properly select adhoc metrics in orderby

* Throw error when sql is empty

* Allow `metrics` to be None

* Always use alias in orderby for metrics

* Bump table chart version and migrate histogram to typescript

* Fix Histogram without groupby

* Fix Presto birth names test

* Raw records mode should not aggregate
2021-03-16 21:00:03 -07:00
rijojoseph07
efeb5a93e6 feat(presto): add support for user impersonation (#13214)
* changes to support presto impersionation with ldap

* renamed method to match 30 char limit

* import spell check

* added presto impersonation test

* refactored impersionation code to generalize for extension

* moving config_args mutation to the update_connect_args_for_impersonation

* moving config_args mutation to the update_connect_args_for_impersonation

* nits

* refactored update_impersonation_config method name to match lint rule

* reduced comment line length

* black reformats

Co-authored-by: rijojoseph01 <rijo.joseph@myntra.com>
2021-02-22 15:01:33 +02:00
Ville Brofeldt
5ab613d89e fix(database): ensure pk_constraint is JSON serializable (#13147) 2021-02-16 12:17:29 +02:00
Beto Dealmeida
dcef5e1381 fix: remove explicit cursor closing for BQ (#12836) 2021-01-29 14:32:47 -08:00
Jesse Yang
e3db935c62 refactor: migrate table chart to new API (#10270)
* refactor: migrate table chart to new API

* chore: bump superset-ui to 0.17.0

* Fix Cypress tests

* Apply soft-conversion to numeric metrics

Fix time column formatting test

* Add translation to chart does not exist error

* Bump to 0.17.1
2021-01-29 13:12:09 +02:00
Amit Miran
b221417e19 chore: remove harmless incubator refs (#12289)
* chore: migrate all harmless incubator refs like comments and documentation files

* chore: migrate all superset.incubator.apache.org to superset.apache.org to

* chore: migrate dist.apache.org refs

* chore: removing incubating and Disclaimer

* fix: linting issues
2021-01-06 13:45:19 +02:00
Beto Dealmeida
877b153331 feat: estimate query cost in Postgres (#12130)
* feat: estimate query cost in Postgres

* Add example in config

* Fix lint
2020-12-18 16:08:53 -08:00
Beto Dealmeida
2b9695c520 feat: add modal to import databases (#11884)
* feat: add modal to import databases

* Fix test

* Improve hook

* Remove log and needless store.

* Change JS functions
2020-12-07 11:22:45 -08:00
Erik Ritter
77d362d306 fix: Remove expensive logs table migration (#11920) 2020-12-04 07:59:28 -08:00
Maxime Beauchemin
0504cf1a00 chore: improve analytics (#11714)
* chore: improve analytics

* lint

* log more events, add note in UPDATING.md

* handling base class

* more events\!

* get ref through

* right before @expose

* fix context

* touchups
2020-11-25 08:45:02 -08:00
Jesse Yang
4cfcaebb61 feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509)
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
2020-11-13 22:35:10 -08:00
Jesse Yang
ec8ccd4cf1 feat: keep modal open when saving database failed (#11618) 2020-11-12 16:10:14 -08:00
ʈᵃᵢ
edb9619731 refactor: reduce number of api calls needed to fetch favorite status for charts and dashboards (#11502) 2020-11-02 21:26:14 -08:00
Beto Dealmeida
f2b8d6aef8 chore: rename ImportMixin to ImportExportMixin (#11460)
* Add UUID to saved_query

* Reuse function from previous migration

* Point to new head

* feat: add backend to export saved queries using new format

* Rename ImportMixin to ImportExportMixin
2020-10-30 09:11:56 -07:00
Rob DiCiuccio
c4effe1261 Log warning for failure to get table/view names from DB (#11374) 2020-10-21 20:09:24 -07:00
Maxime Beauchemin
796a2a6924 fix: error around latest partition in BigQuery (#11274)
* fix: error around latest partition in BigQuery

* lint

* Going with a backend-first approach

* fix test

* add an extra test
2020-10-20 21:34:46 -07:00
Jesse Yang
2c649ac20f perf: cache dashboard bootstrap data (#11234) 2020-10-13 18:43:06 -07:00
Kasia Kucharczyk
cd77797802 Removed disabled linting redefined-outer-name. Url parameters are renamed where url was already defined in class attributes (#10911) 2020-09-16 20:22:46 -07:00
Yongjie Zhao
38edb69d95 Feat: Adding table comment and columns comment for SQLLab (#10844)
* Adding table comment and columns comment for backend

* fix mypy

* Fix CI

* adding wider catch

* use logger

* fix lint
2020-09-14 21:11:48 -07:00
Ville Brofeldt
dd0bc472e3 refactor(database): use SupersetResultSet on SqlaTable.get_df() (#10707)
* refactor(database): use SupersetResultSet on SqlaTable.get_df()

* lint

* change cypress test
2020-08-28 21:12:03 +03:00
Erik Ritter
b686004294 feat: add configuration for Presto cursor poll interval (#10191) 2020-06-30 08:56:22 -07:00
Jason Davis
72fc581b15 added explore database for ctas/cvas (#10174)
Co-authored-by: Jason Davis <@dropbox.com>
2020-06-29 11:13:54 -07:00
Jason Davis
9de9e1c19d feat: Added configuration to SQL Lab results "Explore" button (#10164)
* added configuration to virtual table explore button

* added description to mixin

* fixed unit tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-26 09:54:55 -07:00