Commit Graph

53 Commits

Author SHA1 Message Date
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Maxime Beauchemin
280ade826c fix: dashboard export raises schema issues (#10060)
* fix: dashboard export raises schema issues

* Fix isort
2020-06-15 21:43:05 -07:00
Lily Kuang
f6cd3a917a refactor(sql): optimize sql query parser (#9673)
* optimize sql query parser

* update extract from token

* update doc string

* pylint doc string
2020-06-10 09:15:48 +03:00
John Bodley
244677cf5e style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
John Bodley
7f6dbf838e style: enforcing mypy typing for connectors (#9824)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-25 12:32:49 -07:00
Maxime Beauchemin
ee991963c7 fix: don't create examples db on 'superset init' (#9881)
* chore: don't create examples db on 'superset init'

* fix isort
2020-05-21 21:34:53 -07:00
John Bodley
c691a07948 Revert "[sqllab] fix, strip comments before parsing statements" (#9862)
This reverts commit 949c4eae28.

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-20 09:54:17 -07:00
Daniel Vaz Gaspar
3cc540019f fix(mssql): reverts #9644 and displays a better error msg (#9752) 2020-05-14 17:00:02 +01:00
Ville Brofeldt
5b430ead42 fix: implement force refresh in chart data request (#9762)
* fix: implement force refresh in chart data request

* unrelated lint
2020-05-07 09:33:41 -07:00
Maxime Beauchemin
9aabafe9fe fix: adhoc filter 'equals' doesn't let you save (#9652)
When altering a filter, when using `equals` or `does not equals`,
the save button is disabled even when it should be enalbed.
2020-05-06 08:38:10 -07:00
ʈᵃᵢ
4c522ac65f [sqllab] fix, strip comments before parsing statements (#9692) 2020-05-04 19:47:29 +03:00
John Bodley
3b0f8e9c8a [sql] Adding lighweight Table class (#9649)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-30 08:38:02 -07:00
Daniel Vaz Gaspar
516bdf6db1 fix(mssql): apply limit and set alias for functions (#9644) 2020-04-27 09:23:08 +01:00
Daniel Vaz Gaspar
76764acfc1 [sql_lab] Improve performance, only use slow func when needed (#9612) 2020-04-23 09:35:57 +01:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
Daniel Vaz Gaspar
8e4dcd0509 [log] Set detailed query info to log debug level (#9167) 2020-02-24 10:42:59 +00:00
John Bodley
9f5f8e5d92 [mypy] Enforcing typing for db_engine_specs (#9138) 2020-02-17 23:08:11 -08:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
562aeab1aa Fix a bunch of files with pylint disabled (#8743)
* Re-enable pylint for superset/jinja_context.py

* Re-enable pylint for superset/sql_lab.py

* Re-enable pylint for superset/sql_parse.py

* Re-enable pylint for superset/exceptions.py

* Re-enable lint for superset/translations/utils.py

* Re-enable pylint for superset/views/schedules.py

* Re-enable pylint for superset/views/base.py

* Re-enable pylint for superset/views/log/views.py

* Re-enable pylint for superset/views/annotations.py

* black

* PR feedback, pylint, isort fixes

* Black, one more time...

* Move ungrouped-imports to a global disable
2019-12-11 10:14:24 -08:00
Ville Brofeldt
fb51632e18 Add docstrings and typing to db_engine_specs and sql_parse (#8058)
* Add typing to db_engine_specs

* Add more type annotations and docstrings

* Add docstrings and typing to sql_parse and db_engine_specs

* Refine select_star

* Fix execute and add more docstrings

* Revert kwargs change from execute

* Remove redundant or

* Align view and table getter schema types

* Fix return type of latest_partition

* Remove some typing from presto

* Improve docstring for __extract_from_token
2019-08-22 06:29:32 +03:00
Ville Brofeldt
a1261d7f9a Bump sqlparse to 0.3.0 (#7973)
* Black

* Bump sqlparse to 0.3.0

* Convert str.format() to f-string
2019-08-05 17:08:05 +03:00
Ville Brofeldt
72d1011023 [sqllab] Fix limit parsing bug when using limit-offset comma notation (#7912)
* Fix limit parsing bug when using limit-offset comma notation

* Use native sqlparse semantics to find limit

* black
2019-07-24 08:18:39 +03:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Maxime Beauchemin
859d6e7c6a chore: taking db dependencies out of requirements-dev.txt (#7605)
* chore: taking db dependencies out of requirements-dev.txt

The deps on mysqlclient and psycopg2

* Fix unit tests

* fix tox.ini

* fix tests
2019-06-23 22:37:41 -07:00
John Bodley
45b41aadcc [fix] SQL parsing of table names (#7490) 2019-06-03 11:07:57 -07:00
Maxime Beauchemin
d8be0a7dd5 Break line before LIMIT statement to prevent trailing comment issue (#7485)
* Break line before LIMIT statement to prevent trailing comment issue

This may not be a perfect solution but it addresses the issue in 7483

closes https://github.com/apache/incubator-superset/issues/7483

* fix tests
2019-05-13 00:34:34 -05:00
John Bodley
fb627ba376 [fix] Fixing SQL parsing issue (#7374) 2019-05-01 22:07:01 -07:00
Maxime Beauchemin
68d2880bce Remove super verbose logging.debug call (#7122)
I left this behind on a previous PR, and it gets kind of crazy on large
SQL statements, so decided it's probably a good thing to clean this up.
2019-03-26 08:42:13 -07:00
John Bodley
07c340cf82 [sqlparse] Fixing table name extraction for ill-defined query (#7029) 2019-03-15 09:19:30 -07:00
John Bodley
3e076cb60b [sql-parse] Fixing LIMIT exceptions (#6963) 2019-03-05 09:36:08 -08:00
Maxime Beauchemin
5a40f71710 [sql lab] improve table name detection in free form SQL (#6793)
* [sql lab] improve table name detection in free form SQL

* flake

* Addressing comments
2019-02-04 16:03:23 -08:00
Beto Dealmeida
99d6dc07d6 Trim query before parsing (#6703)
* Ignore empty lines

* Fix unit tests

* Use stripped

* Fix another unit test
2019-01-17 09:16:35 -08:00
Maxime Beauchemin
1dd4d7a587 Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Maxime Beauchemin
d427db0a8b [SQL Lab] Allow running multiple statements (#6112)
* Allow running multiple statements from SQL Lab

* fix tests

* More tests

* merge heads

* fix heads
2018-12-22 10:28:22 -08:00
Maxime Beauchemin
cc3a625a4b Use py3's f-strings instead of s.format(**locals()) (#6448)
* Use py3's f-strings instead of s.format(**locals())

In light of the bug reported here
https://github.com/apache/incubator-superset/issues/6347, which seems
like an odd `.format()` issue in py3, I greped and replaced all
instances of `.format(**locals())` using py3's f-strings

* lint

* fix tests
2018-12-02 13:50:49 -08:00
timifasubaa
46c86672c8 remove utf8 declaration (#6096) 2018-10-15 11:53:24 -07:00
timifasubaa
dd9eeda03e remove future (#6065) 2018-10-13 09:39:04 -07:00
timifasubaa
d5d592788c fix sqlparse bug and refactor (#5768) 2018-08-29 08:30:49 -07:00
timifasubaa
5c49514e2f [sqlparse] fix sqlparse bug (#5703)
* fix sqlparse bug

* add one more test case
2018-08-23 11:20:25 -07:00
Maxime Beauchemin
9331cf79b5 [sql lab] allow EXPlAIN queries (#5558)
* [sql lab] allow EXPlAIN queries

closes https://github.com/andialbrecht/sqlparse/issues/421

* typo
2018-08-03 15:33:33 -07:00
timifasubaa
f8a6e09220 [sqllab] Fix sqllab limit regex issue with sqlparse (#5295)
* include items after limit to the modified query

* use sqlparse
2018-07-16 15:27:30 -07:00
Maxime Beauchemin
b839608c32 [sql lab] a better approach at limiting queries (#4947)
* [sql lab] a better approach at limiting queries

Currently there are two mechanisms that we use to enforce the row
limiting constraints, depending on the database engine:
1. use dbapi's `cursor.fetchmany()`
2. wrap the SQL into a limiting subquery

Method 1 isn't great as it can result in the database server storing
larger than required result sets in memory expecting another fetch
command while we know we don't need that.

Method 2 has a positive side of working with all database engines,
whether they use LIMIT, ROWNUM, TOP or whatever else since sqlalchemy
does the work as specified for the dialect. On the downside though
the query optimizer might not be able to optimize this as much as an
approach that doesn't use a subquery.

Since most modern DBs use the LIMIT syntax, this adds a regex approach
to modify the query and force a LIMIT clause without using a subquery
for the database that support this syntax and uses method 2 for all
others.

* Fixing build

* Fix lint

* Added more tests

* Fix tests
2018-05-14 14:44:05 -05:00
John Bodley
d533ce0967 [pylint] prepping for enabling pylint for non-errors (#4884) 2018-04-28 20:08:09 -07:00
John Bodley
d57a37e341 [flake8] Adding flake8-coding (#4477) 2018-02-25 15:06:11 -08:00
John Bodley
ac57780607 [flake8] Resolving Q??? errors (#3847) 2017-11-13 21:06:51 -08:00
John Bodley
e2bca47421 [flake8] Resolve I??? errors (#3797) 2017-11-07 20:23:40 -08:00
Maxime Beauchemin
ed9f56448f Adding missing future imports (#3493) 2017-09-19 11:20:22 -07:00
Grace Guo
ad604aed09 fix encoding error in sql lab logging (#3424) 2017-09-14 11:18:34 -07:00
Maxime Beauchemin
10773f96a7 URL Params macro (#2537) 2017-04-05 10:45:32 -07:00