Commit Graph

100 Commits

Author SHA1 Message Date
Beto Dealmeida
cd7972d05b feat: support for KQL in SQLScript (#27522) 2024-03-22 12:48:20 -04:00
John Bodley
b25dd0c055 fix(sql_parse): Ensure table extraction handles Jinja templating (#27470) 2024-03-22 13:39:28 +13:00
Beto Dealmeida
72a41c1642 fix: sqlglot SQL Server (#27577) 2024-03-21 17:09:40 -04:00
Beto Dealmeida
36fd3c0bf8 feat: improve _extract_tables_from_sql (#26748) 2024-03-18 13:02:58 -04:00
Beto Dealmeida
024b88a40d chore: add annotations to sql_parse.py (#27520) 2024-03-14 18:16:06 -04:00
Beto Dealmeida
26d8077e97 chore: improve SQL parsing (#26767) 2024-03-13 18:27:01 -04:00
John Bodley
2c564817f1 fix(sqlglot): Address regressions introduced in #26476 (#27217) 2024-02-24 08:47:36 +13:00
mapledan
f2bf9f72e4 fix: handle CRLF endings causing sqlglot failure (#26911) 2024-01-31 18:07:43 -08:00
Beto Dealmeida
c0b57bd1c3 feat(sqlparse): improve table parsing (#26476) 2024-01-22 11:16:50 -05:00
Beto Dealmeida
2bd611916d feat: safer insert RLS (#20323) 2023-11-08 22:52:25 -05:00
Beto Dealmeida
bb002d6147 fix: is_select with UNION (#25290) 2023-09-14 09:05:19 -07:00
Beto Dealmeida
2f68010729 fix: is_select (#25189) 2023-09-06 11:54:25 -07:00
Daniel Vaz Gaspar
357986103b fix: CTE queries with non-SELECT statements (#25014) 2023-08-19 15:49:15 +01:00
EugeneTorap
5f103072b0 fix: Python3.11 (str, Enum) issue (#24803) 2023-07-31 09:04:09 -07:00
EugeneTorap
fc89718d48 chore: Update pylint to 2.17.4 (#24700)
Co-authored-by: John Bodley <john.bodley@gmail.com>
2023-07-24 17:13:49 -07:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
EugeneTorap
a7f9c66e57 chore: Bump sqlparse to 0.4.4 (#24045)
Co-authored-by: sebastianliebscher <liebscher.sebastian@protonmail.ch>
2023-05-23 10:15:24 +03:00
Beto Dealmeida
2a1a061a3e fix: handle comments in has_table_query (#23882) 2023-05-01 11:06:54 -07:00
Francisco Muniz de Paula Neto
e9b4022787 fix: is_select check for lowercase select with "WITH" clauses (#22370) 2023-04-18 08:59:50 -07:00
Hugh A. Miles II
b820eb8235 chore: Update pre-commit packages (#23173) 2023-03-13 17:05:13 -06:00
Josh Soref
dec6ac49f6 chore(backend): Spelling (#19677)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
2023-02-17 10:48:24 -07:00
Diego Medina
8b354b4b69 fix: Published Dashboard without charts don't show up for non admin users (#20638) 2022-07-27 12:40:59 -06:00
John Bodley
ad308fbde2 chore(rls): Remove passing global username (#20344)
* chore(rls): Remove passing global username

* Update manager.py

* Update manager.py

* Update manager.py

* Update manager.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-07-05 10:51:24 -07:00
Beto Dealmeida
f2881e5bbd feat: RLS for SQL Lab (#19999)
* feat: RLS for SQL Lab

* Small fixes

* Pass username to security manager

* Update docstrings

* Add tests

* Remove type from docstring
2022-05-12 11:03:01 -07:00
Jesse Yang
231716cb50 perf: refactor SIP-68 db migrations with INSERT SELECT FROM (#19421) 2022-04-19 18:58:18 -07:00
Craig Rueda
f64d654de3 chore(database): Creating helper make_url_safe to wrap potential errors (#19526)
* Creating helper make_url_safe to wrap potential errors

* Fixing imports

* Fixing imports again

* Adding comment

* Linting

* Fixing test

* Fixing test again...

* Fixing import
2022-04-05 11:17:30 -07:00
Beto Dealmeida
6828624f61 feat: improve adhoc SQL validation (#19454)
* feat: improve adhoc SQL validation

* Small changes

* Add more unit tests
2022-03-31 11:55:19 -07:00
Ville Brofeldt
f341025d80 feat: add support for comments in adhoc clauses (#19248)
* feat: add support for comments in adhoc clauses

* sanitize remaining freeform clauses

* sanitize adhoc having in frontend

* address review comment
2022-03-18 15:08:06 -07:00
Beto Dealmeida
8234395466 feat: helper functions for RLS (#19055)
* feat: helper functions for RLS

* Add function to inject RLS

* Add UNION tests

* Add tests for schema

* Add more tests; cleanup

* has_table_query via tree traversal

* Wrap existing predicate in parenthesis

* Clean up logic

* Improve table matching
2022-03-11 14:47:11 -08:00
Sujith Kumar S
7e51b200b4 fix(mssql): support top syntax for limiting queries (#18746)
* SQL-TOP Fix For Database Engines

MSSQL is not supporting LIMIT syntax in SQLs. For limiting the rows, MSSQL having a different keyword TOP. Added fixes for handling the TOP and LIMIT clauses based on the database engines.

* Teradata code for top clause handling removed from teradata.py

Teradata code for top clause handling removed from teradata.py file, since we added generic section in base engine for the same.

* Changes to handle CTE along with TOP in complex SQLs

Added changes to handle TOP command in CTEs, for DB Engines which are not supporting inline CTEs.

* Test cases for TOP unit testing in MSSQL

Added multiple unit test cases for MSSQL top command handling and also along with CTEs

* Corrected the select_keywords name key in basengine

Corrected the select_keywords name key in basengine

* Changes based on as per review.

made the required corrections based on code review to keep good code readability and code cleanliness.

* Review changes to correct lint and typo issues

Made the changes according to the review comments.

* fix linting errors

* fix teradata tests

* add coverage

* lint

* Code cleanliness

Moved the top/limit flag check from sql_lab to core.

* Changed for code cleanliness

Changes for keeping code cleanliness

* Corrected lint issue

Corrected lint issue.

* Code cleanliness

Code cleanliness

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-02-21 09:58:39 +02:00
Erik Ritter
14b9298ef7 fix: revert #17654 to fix subselect table name parsing (#18017) 2022-01-12 21:28:23 -08:00
John Bodley
63ca09e345 fix: Workaround for sqlparse issue #652 (#17995)
* fix: Workaround for sqlparse issue #652

* Update superset/sql_parse.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update sql_parse.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2022-01-12 14:03:56 +13:00
Ville Brofeldt
3a42071e0f chore(sql): clean up invalid filter clause exception types (#17702)
* chore(sql): clean up invalid filter clause exception types

* fix lint

* rename exception
2021-12-09 17:49:32 +02:00
Beto Dealmeida
8c25f2f356 fix: extract tables doesn't work with reserved keywords (#17654) 2021-12-08 05:50:42 -08:00
Beto Dealmeida
93bafa0e6a feat: improve logic in is_select (#17329)
* feat: improve logic in is_select

* Add more edge cases
2021-11-02 17:30:12 -07:00
Beto Dealmeida
48a61bacc5 fix: handle CTEs with comments on is_select (#16769) 2021-09-21 16:23:54 -07:00
John Bodley
7e4c940314 chore(pylint): Enable useless-suppression check (#16388)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-24 03:58:41 +12:00
AAfghahi
e507508b48 feat: Logic added to limiting factor column in Query model (#13521)
* Sqllab limit

* Add migration script

* Set default values

* initial push

* revisions

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* moving migration to separate PR

* with migration

* revisions

* Fix apply_limit_to_sql

* all but tests

* added unit tests

* revisions

* Update superset/sql_lab.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/sql_parse.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* fixed black issue

* Update superset/views/core.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* updated logic

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-04-30 15:15:18 -07:00
Erik Ritter
06d6d7f8e2 fix: ParsedQuery subselect edge case (#13602) 2021-03-12 14:54:02 -08:00
Daniel Vaz Gaspar
9568985b7b fix: engines that don't support comments (#13153)
* fix: engines that don't support comments

* fix: engines that don't support comments

* add quick inexpensive test

* add test
2021-02-17 18:01:34 +00:00
Beto Dealmeida
164db3e5a1 fix: CTAS on multiple statements (#12188)
* WIP

* Add unit tests for sql_parse

* Add unit tests for sql_lab
2021-01-04 09:22:35 -08:00
David Aaron Suddjian
b5dd0f32cc feat: Dynamically imported viz plugins (#10288)
* first attempts at dynamic plugin loading

* dynamic import working for explore

* memoize appropriately

* add a backend for dynamic plugins

* hack at getting dynamic plugins working with dashboards

* more work on making it work, + feature flag

* lint

* actions to fix explore state when plugins load

* handle dynamic control panel, functionify ExploreViewContainer

* fix: rearrange migrations branch

* fix: name and key as strings with length 50

* bundle url length 2000

* bundle url to text
for some reason not supported on my sql

* fix: too long varchart

* fix: pre-commit typing

* fix: licenses

* fix: add slice container was not initing feature flags

* fix: undo linting issue

* fix: adjust down revision again

* fix: adjust down revision again

* isort

* pylint

* god damn linters

* remove unnecessary(?) loading message

* only log non-standard errors

* testing

* python is terrible

* see above commit message

* fix imports in DynamicPluginProvider

* fix

* shift migration forward

* lint

* fix form data calculations to handle missing control config

* temp commit - waiting for superset-ui changes and crud fixes

* remove unnecessary todo

* use new superset-ui shared module function

* fetch the plugins instead of hardcoding the test one

* migration sort

* remove duplicated import statement

* format

* try moving the import 🙄

* copy

* fix frontend tests

* safe access

* comment out dead code

* isort

* disable pylint on necessary lines

* use @superset-ui/logging instead of console

* remove temp code

* rearrange some code

* try triggering mouseover in cypress before click

* use loading spinner instead of text

* trying to fix cypress

* attempt cypress fix

* customize permissions

* update package lock

* only admins can write to plugins by default

* better copy

* disable flaky tests

* use makeApi

* flaky tests

* cleanup code

* flaaaakkkyyyyyy

* dry

Co-authored-by: amitNielsen <amit.miran@nielsen.com>
2020-12-19 07:06:11 -08:00
Bogdan
0396c705d4 chore: Support SET & SHOW commands as read only SQL commands (#11868)
* Support SET & SHOW commands as read only SQL commands

* Move is_readonly definition into the engine spec

* Rename & use super()

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-12-03 10:44:11 -08:00
Hugh A. Miles II
dfbcbcce67 fix: Allow "EXPLAIN" queries when "Allow DML" setting is False (#11348)
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2020-10-21 12:58:57 -07:00
serenajiang
54c2ad4308 fix(sqla): allow 'unknown' type queries in explore view (#11365) 2020-10-21 10:24:52 -07:00
Ville Brofeldt
96b9ba3364 ci: bump pre-commit py36 to py37 (#10541)
* ci: bump pre-commit py36 to py37

* add 3rd party for py37

* lint

* yet more lint
2020-08-07 06:54:22 +03:00
Maxime Beauchemin
318e5347bc feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Maxime Beauchemin
280ade826c fix: dashboard export raises schema issues (#10060)
* fix: dashboard export raises schema issues

* Fix isort
2020-06-15 21:43:05 -07:00
Lily Kuang
f6cd3a917a refactor(sql): optimize sql query parser (#9673)
* optimize sql query parser

* update extract from token

* update doc string

* pylint doc string
2020-06-10 09:15:48 +03:00