Commit Graph

86 Commits

Author SHA1 Message Date
EugeneTorap
fc89718d48 chore: Update pylint to 2.17.4 (#24700)
Co-authored-by: John Bodley <john.bodley@gmail.com>
2023-07-24 17:13:49 -07:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
EugeneTorap
a7f9c66e57 chore: Bump sqlparse to 0.4.4 (#24045)
Co-authored-by: sebastianliebscher <liebscher.sebastian@protonmail.ch>
2023-05-23 10:15:24 +03:00
Beto Dealmeida
2a1a061a3e fix: handle comments in has_table_query (#23882) 2023-05-01 11:06:54 -07:00
Francisco Muniz de Paula Neto
e9b4022787 fix: is_select check for lowercase select with "WITH" clauses (#22370) 2023-04-18 08:59:50 -07:00
Hugh A. Miles II
b820eb8235 chore: Update pre-commit packages (#23173) 2023-03-13 17:05:13 -06:00
Josh Soref
dec6ac49f6 chore(backend): Spelling (#19677)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
2023-02-17 10:48:24 -07:00
Diego Medina
8b354b4b69 fix: Published Dashboard without charts don't show up for non admin users (#20638) 2022-07-27 12:40:59 -06:00
John Bodley
ad308fbde2 chore(rls): Remove passing global username (#20344)
* chore(rls): Remove passing global username

* Update manager.py

* Update manager.py

* Update manager.py

* Update manager.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-07-05 10:51:24 -07:00
Beto Dealmeida
f2881e5bbd feat: RLS for SQL Lab (#19999)
* feat: RLS for SQL Lab

* Small fixes

* Pass username to security manager

* Update docstrings

* Add tests

* Remove type from docstring
2022-05-12 11:03:01 -07:00
Jesse Yang
231716cb50 perf: refactor SIP-68 db migrations with INSERT SELECT FROM (#19421) 2022-04-19 18:58:18 -07:00
Craig Rueda
f64d654de3 chore(database): Creating helper make_url_safe to wrap potential errors (#19526)
* Creating helper make_url_safe to wrap potential errors

* Fixing imports

* Fixing imports again

* Adding comment

* Linting

* Fixing test

* Fixing test again...

* Fixing import
2022-04-05 11:17:30 -07:00
Beto Dealmeida
6828624f61 feat: improve adhoc SQL validation (#19454)
* feat: improve adhoc SQL validation

* Small changes

* Add more unit tests
2022-03-31 11:55:19 -07:00
Ville Brofeldt
f341025d80 feat: add support for comments in adhoc clauses (#19248)
* feat: add support for comments in adhoc clauses

* sanitize remaining freeform clauses

* sanitize adhoc having in frontend

* address review comment
2022-03-18 15:08:06 -07:00
Beto Dealmeida
8234395466 feat: helper functions for RLS (#19055)
* feat: helper functions for RLS

* Add function to inject RLS

* Add UNION tests

* Add tests for schema

* Add more tests; cleanup

* has_table_query via tree traversal

* Wrap existing predicate in parenthesis

* Clean up logic

* Improve table matching
2022-03-11 14:47:11 -08:00
Sujith Kumar S
7e51b200b4 fix(mssql): support top syntax for limiting queries (#18746)
* SQL-TOP Fix For Database Engines

MSSQL is not supporting LIMIT syntax in SQLs. For limiting the rows, MSSQL having a different keyword TOP. Added fixes for handling the TOP and LIMIT clauses based on the database engines.

* Teradata code for top clause handling removed from teradata.py

Teradata code for top clause handling removed from teradata.py file, since we added generic section in base engine for the same.

* Changes to handle CTE along with TOP in complex SQLs

Added changes to handle TOP command in CTEs, for DB Engines which are not supporting inline CTEs.

* Test cases for TOP unit testing in MSSQL

Added multiple unit test cases for MSSQL top command handling and also along with CTEs

* Corrected the select_keywords name key in basengine

Corrected the select_keywords name key in basengine

* Changes based on as per review.

made the required corrections based on code review to keep good code readability and code cleanliness.

* Review changes to correct lint and typo issues

Made the changes according to the review comments.

* fix linting errors

* fix teradata tests

* add coverage

* lint

* Code cleanliness

Moved the top/limit flag check from sql_lab to core.

* Changed for code cleanliness

Changes for keeping code cleanliness

* Corrected lint issue

Corrected lint issue.

* Code cleanliness

Code cleanliness

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-02-21 09:58:39 +02:00
Erik Ritter
14b9298ef7 fix: revert #17654 to fix subselect table name parsing (#18017) 2022-01-12 21:28:23 -08:00
John Bodley
63ca09e345 fix: Workaround for sqlparse issue #652 (#17995)
* fix: Workaround for sqlparse issue #652

* Update superset/sql_parse.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update sql_parse.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2022-01-12 14:03:56 +13:00
Ville Brofeldt
3a42071e0f chore(sql): clean up invalid filter clause exception types (#17702)
* chore(sql): clean up invalid filter clause exception types

* fix lint

* rename exception
2021-12-09 17:49:32 +02:00
Beto Dealmeida
8c25f2f356 fix: extract tables doesn't work with reserved keywords (#17654) 2021-12-08 05:50:42 -08:00
Beto Dealmeida
93bafa0e6a feat: improve logic in is_select (#17329)
* feat: improve logic in is_select

* Add more edge cases
2021-11-02 17:30:12 -07:00
Beto Dealmeida
48a61bacc5 fix: handle CTEs with comments on is_select (#16769) 2021-09-21 16:23:54 -07:00
John Bodley
7e4c940314 chore(pylint): Enable useless-suppression check (#16388)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-24 03:58:41 +12:00
AAfghahi
e507508b48 feat: Logic added to limiting factor column in Query model (#13521)
* Sqllab limit

* Add migration script

* Set default values

* initial push

* revisions

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* moving migration to separate PR

* with migration

* revisions

* Fix apply_limit_to_sql

* all but tests

* added unit tests

* revisions

* Update superset/sql_lab.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/sql_parse.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* fixed black issue

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* updated logic

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-04-30 15:15:18 -07:00
Erik Ritter
06d6d7f8e2 fix: ParsedQuery subselect edge case (#13602) 2021-03-12 14:54:02 -08:00
Daniel Vaz Gaspar
9568985b7b fix: engines that don't support comments (#13153)
* fix: engines that don't support comments

* fix: engines that don't support comments

* add quick inexpensive test

* add test
2021-02-17 18:01:34 +00:00
Beto Dealmeida
164db3e5a1 fix: CTAS on multiple statements (#12188)
* WIP

* Add unit tests for sql_parse

* Add unit tests for sql_lab
2021-01-04 09:22:35 -08:00
David Aaron Suddjian
b5dd0f32cc feat: Dynamically imported viz plugins (#10288)
* first attempts at dynamic plugin loading

* dynamic import working for explore

* memoize appropriately

* add a backend for dynamic plugins

* hack at getting dynamic plugins working with dashboards

* more work on making it work, + feature flag

* lint

* actions to fix explore state when plugins load

* handle dynamic control panel, functionify ExploreViewContainer

* fix: rearrange migrations branch

* fix: name and key as strings with length 50

* bundle url length 2000

* bundle url to text
for some reason not supported on my sql

* fix: too long varchart

* fix: pre-commit typing

* fix: licenses

* fix: add slice container was not initing feature flags

* fix: undo linting issue

* fix: adjust down revision again

* fix: adjust down revision again

* isort

* pylint

* god damn linters

* remove unnecessary(?) loading message

* only log non-standard errors

* testing

* python is terrible

* see above commit message

* fix imports in DynamicPluginProvider

* fix

* shift migration forward

* lint

* fix form data calculations to handle missing control config

* temp commit - waiting for superset-ui changes and crud fixes

* remove unnecessary todo

* use new superset-ui shared module function

* fetch the plugins instead of hardcoding the test one

* migration sort

* remove duplicated import statement

* format

* try moving the import 🙄

* copy

* fix frontend tests

* safe access

* comment out dead code

* isort

* disable pylint on necessary lines

* use @superset-ui/logging instead of console

* remove temp code

* rearrange some code

* try triggering mouseover in cypress before click

* use loading spinner instead of text

* trying to fix cypress

* attempt cypress fix

* customize permissions

* update package lock

* only admins can write to plugins by default

* better copy

* disable flaky tests

* use makeApi

* flaky tests

* cleanup code

* flaaaakkkyyyyyy

* dry

Co-authored-by: amitNielsen <amit.miran@nielsen.com>
2020-12-19 07:06:11 -08:00
Bogdan
0396c705d4 chore: Support SET & SHOW commands as read only SQL commands (#11868)
* Support SET & SHOW commands as read only SQL commands

* Move is_readonly definition into the engine spec

* Rename & use super()

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-12-03 10:44:11 -08:00
Hugh A. Miles II
dfbcbcce67 fix: Allow "EXPLAIN" queries when "Allow DML" setting is False (#11348)
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2020-10-21 12:58:57 -07:00
serenajiang
54c2ad4308 fix(sqla): allow 'unknown' type queries in explore view (#11365) 2020-10-21 10:24:52 -07:00
Ville Brofeldt
96b9ba3364 ci: bump pre-commit py36 to py37 (#10541)
* ci: bump pre-commit py36 to py37

* add 3rd party for py37

* lint

* yet more lint
2020-08-07 06:54:22 +03:00
Maxime Beauchemin
318e5347bc feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Maxime Beauchemin
280ade826c fix: dashboard export raises schema issues (#10060)
* fix: dashboard export raises schema issues

* Fix isort
2020-06-15 21:43:05 -07:00
Lily Kuang
f6cd3a917a refactor(sql): optimize sql query parser (#9673)
* optimize sql query parser

* update extract from token

* update doc string

* pylint doc string
2020-06-10 09:15:48 +03:00
John Bodley
244677cf5e style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
John Bodley
7f6dbf838e style: enforcing mypy typing for connectors (#9824)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-25 12:32:49 -07:00
Maxime Beauchemin
ee991963c7 fix: don't create examples db on 'superset init' (#9881)
* chore: don't create examples db on 'superset init'

* fix isort
2020-05-21 21:34:53 -07:00
John Bodley
c691a07948 Revert "[sqllab] fix, strip comments before parsing statements" (#9862)
This reverts commit 949c4eae28.

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-20 09:54:17 -07:00
Daniel Vaz Gaspar
3cc540019f fix(mssql): reverts #9644 and displays a better error msg (#9752) 2020-05-14 17:00:02 +01:00
Ville Brofeldt
5b430ead42 fix: implement force refresh in chart data request (#9762)
* fix: implement force refresh in chart data request

* unrelated lint
2020-05-07 09:33:41 -07:00
Maxime Beauchemin
9aabafe9fe fix: adhoc filter 'equals' doesn't let you save (#9652)
When altering a filter, when using `equals` or `does not equals`,
the save button is disabled even when it should be enalbed.
2020-05-06 08:38:10 -07:00
ʈᵃᵢ
4c522ac65f [sqllab] fix, strip comments before parsing statements (#9692) 2020-05-04 19:47:29 +03:00
John Bodley
3b0f8e9c8a [sql] Adding lighweight Table class (#9649)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-30 08:38:02 -07:00
Daniel Vaz Gaspar
516bdf6db1 fix(mssql): apply limit and set alias for functions (#9644) 2020-04-27 09:23:08 +01:00
Daniel Vaz Gaspar
76764acfc1 [sql_lab] Improve performance, only use slow func when needed (#9612) 2020-04-23 09:35:57 +01:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
Daniel Vaz Gaspar
8e4dcd0509 [log] Set detailed query info to log debug level (#9167) 2020-02-24 10:42:59 +00:00
John Bodley
9f5f8e5d92 [mypy] Enforcing typing for db_engine_specs (#9138) 2020-02-17 23:08:11 -08:00