Commit Graph

96 Commits

Author SHA1 Message Date
Maxime Beauchemin
b839608c32 [sql lab] a better approach at limiting queries (#4947)
* [sql lab] a better approach at limiting queries

Currently there are two mechanisms that we use to enforce the row
limiting constraints, depending on the database engine:
1. use dbapi's `cursor.fetchmany()`
2. wrap the SQL into a limiting subquery

Method 1 isn't great as it can result in the database server storing
larger than required result sets in memory expecting another fetch
command while we know we don't need that.

Method 2 has a positive side of working with all database engines,
whether they use LIMIT, ROWNUM, TOP or whatever else since sqlalchemy
does the work as specified for the dialect. On the downside though
the query optimizer might not be able to optimize this as much as an
approach that doesn't use a subquery.

Since most modern DBs use the LIMIT syntax, this adds a regex approach
to modify the query and force a LIMIT clause without using a subquery
for the database that support this syntax and uses method 2 for all
others.

* Fixing build

* Fix lint

* Added more tests

* Fix tests
2018-05-14 14:44:05 -05:00
Yongjie Zhao
7a4a89b195 Update Apache Kylin dbengine with supported week/quarter grains (#4965) 2018-05-14 14:11:59 -05:00
Ville Brofeldt
b391676544 Force lowercase column names for Snowflake and Oracle (#4994)
* Force lowercase column names for Snowflake and Oracle

* Force lowercase column names for Snowflake and Oracle

* Remove lowercasing of DB2 columns

* Remove DB2 lowercasing

* Fix test cases
2018-05-14 13:43:13 -05:00
timifasubaa
d87504cb42 Merge pull request #4833 from timifasubaa/help_sqllab_forget_the_past
[sqllab] Help sqllab forget query history
2018-05-07 10:56:39 -07:00
Timi Fasubaa
ab958c67e6 make queries older than 6 hours timeout 2018-05-07 10:14:37 -07:00
Yongjie Zhao
5d6e59aa8a Support Apache Kylin in EngineSpec (#4925)
* Support Apache Kylin in EngineSpec

* Fix flake8
2018-05-03 08:42:43 -07:00
Maxime Beauchemin
5f6a1cea47 Fix typos from linting (#4918)
Caused by https://github.com/apache/incubator-superset/pull/3847

Fixes https://github.com/apache/incubator-superset/issues/4915
2018-05-01 13:34:10 -07:00
Beto Dealmeida
13da5a8742 Fix for week_start_sunday and week_ending_saturday (#4911)
* Handle locked weeks

* Fix spelling

* Fix druid

* Clean unit tests
2018-05-01 13:27:56 -07:00
John Bodley
d533ce0967 [pylint] prepping for enabling pylint for non-errors (#4884) 2018-04-28 20:08:09 -07:00
Ville Brofeldt
fa3da8c888 Implement Snowflake engine with supported time grains (#4882)
* Implement Snowflake engine with supported time grains

* Fix typo in second grain
2018-04-25 14:44:24 -07:00
Riccardo Magliocchetti
3b18fbf9e3 db_engine_specs: use correct sqlite week time grain (#4831)
We want the week of year %W and not the day of week %w
when using week as time grain.

Reference:
https://www.sqlite.org/lang_datefunc.html
2018-04-15 16:15:31 -07:00
timifasubaa
20f46eede5 call next() the right way (#4804) 2018-04-11 13:20:14 -07:00
Beto Dealmeida
426c34ee86 Pass granularity from backend to frontend as ISO duration (#4755)
* Add ISO duration to time grains

* Use ISO duration

* Remove debugging code

* Add module to yarn.lock

* Remove autolint

* Druid granularity as ISO

* Remove dangling comma
2018-04-06 16:19:17 -07:00
Maxime Beauchemin
93ec76f757 [sql lab] reduce the number of metadata calls when loading a table (#4593) 2018-03-15 17:53:34 -07:00
John Bodley
4250e239a2 Merge pull request #4590 from michellethomas/fixing_double_escape_presto
Removing escape_sql so we dont double escape
2018-03-13 12:19:44 -07:00
Hugh A. Miles II
2bc089ef8d Added new exception class and start of better exception/error handling (#4514)
* rebase and linting

* change back

* wip

* fixed broken test

* fix flake8

* fix test
2018-03-11 22:07:51 -07:00
Michelle Thomas
e1af421f0c Removing escape_sql so we dont double escape 2018-03-09 15:37:17 -08:00
Kyle Travis
31a995714d [bug] Fix CSV upload feature for DB with password (#4562)
* Use sqlalchemy_uri_decrypted in create_engine calls

* Update tox mysql uri

* Include mysql charset=utf8 for py2.7 in tox.ini
2018-03-07 17:42:52 -08:00
John Bodley
150768ee30 [presto] Removing patched presto (#4530) 2018-03-05 23:16:02 -08:00
timifasubaa
404e2d552a fixes to csv - hive upload (#4488) 2018-02-27 22:13:06 -08:00
John Bodley
d57a37e341 [flake8] Adding flake8-coding (#4477) 2018-02-25 15:06:11 -08:00
mxmzdlv
4a6adb2f27 Check class name string instead of checking the instance (#4404)
This fixes an issue with the newer Google's DBAPI library, which moved Row from google.cloud.bigquery._helper.Row to google.cloud.bigquery.Row, causing ImportError
2018-02-12 21:16:42 -08:00
Maciej Bryński
387bf766d3 Superset issue #4323 (#4353)
Adding configurable poll interval for Hive Engine
2018-02-07 21:29:31 -08:00
Beto Dealmeida
f14c1bb593 Add hour grain to Sqlite (#4333) 2018-02-07 14:07:15 -08:00
Teemu Haapoja
3b35ddf135 convert postgresql date_trunc() to UTC to prevent pandas error (#4319)
* cast postgresql date_trunc() to timestamp without time zone to prevent pandas error

* fix formatting for flake8

* change cast to timezone conversion instead
2018-02-07 08:18:11 -08:00
timifasubaa
2d8a0cc6c9 fix uri form data' (#4345)
'
2018-02-05 20:49:37 -08:00
timifasubaa
6d37d97ba5 Refactor import csv (#4298)
* move helpers to utils

* make form use queryselector

* refactor exception throwing and handling

* update db_connection access point

* nits
2018-02-03 20:22:06 -08:00
fabianmenges
a9e1e685ba [BugFix]: Creating a PostgresBaseEngineSpec so changes to the Postgre… (#4224)
* [BugFix]: Creating a PostgresBaseEngineSpec so changes to the PostgresEngineSpec don't affect every subclass

* Empty engine for abstract Engine
2018-02-03 20:03:02 -08:00
Alexander Tronchin-James
c9e47f0bb3 Check for non-None database before using. (#4162)
Some valid sqlalchemy uri's return a URL object with database=None, which causes the following error:
```
2018-01-05 17:59:47,560:ERROR:root:argument of type 'NoneType' is not iterable
Traceback (most recent call last):
  File "/opt/incubator-superset/superset/sql_lab.py", line 186, in execute_sql
    user_name=user_name,
  File "/opt/incubator-superset/superset/utils.py", line 124, in __call__
    return self.func(*args, **kwargs)
  File "/opt/incubator-superset/superset/models/core.py", line 644, in get_sqla_engine
    url = self.db_engine_spec.adjust_database_uri(url, schema)
  File "/opt/incubator-superset/superset/db_engine_specs.py", line 505, in adjust_database_uri
    if '/' in database:
TypeError: argument of type 'NoneType' is not iterable
```
This patch corrects that problem.
2018-01-05 13:54:17 -08:00
Beto Dealmeida
686023c8dd Druid support via SQLAlchemy (#4163)
* Use druiddb

* Remove auto formatting

* Show prequeries

* Fix subtle bug with lists

* Move arguments to query object

* Fix druid run_query
2018-01-05 13:52:58 -08:00
timifasubaa
e498f2fcb6 fix variable name (#4139) 2018-01-02 14:32:24 -08:00
kuriancheeramelil
281ae45495 Fix for SQL editor throwing can't deserialize google.cloud.bigquery._helpers.Row with BigQuery (#4071)
* fix for SQL editor throwing cant deserialize google.cloud.bigquery._helpers.Row with BigQuery

* linted code

* disable pylint import error of bigquery row

* fixed spacing issue before inline-comment
2017-12-18 21:22:34 -08:00
Nic
e79d05fd77 #4058 Fix Oracle timestamps (Oracle "ORA-00907: missing right parenthesis" error) (#4065) 2017-12-15 21:31:09 -08:00
Maxime Beauchemin
cf1d9ce1e6 Add db_engine_spec for Druid (#4063)
The `druiddb` pypi package provides a dbapi and sqlalchemy dialect for
Druid. This PR hooks adds some superset-specific conf.
2017-12-15 11:47:00 -08:00
Dmitry Goryunov
e02d35ed5c Add support of another DatabaseError format (#4019) 2017-12-06 21:49:03 -08:00
timifasubaa
268edcfedd Import CSV (#3643)
* add upload csv button to sources dropdown

* upload csv to non-hive datasources

* upload csv to hive datasource

* update FAQ page

* add tests

* fix linting errors and merge conflicts

* Update .travis.yml

* Update tox.ini
2017-11-27 21:07:12 -08:00
Maxime Beauchemin
2d37dec5ff [bugfix] remove quotes from Postgres time grains (#3913) 2017-11-21 11:24:17 -08:00
Yu Xiao
feb15a30a2 fix the schema-fetching problem for impala in sql_lab (#3906)
* fix the schema-fetching problem for impala in sql_lab

* delete redundant print

* remove blank lines...

* minior corrections
2017-11-21 09:28:31 -08:00
John Bodley
ac57780607 [flake8] Resolving Q??? errors (#3847) 2017-11-13 21:06:51 -08:00
Mike Schiller
630604bc6b adding support for getting list of foreign tables for PostgreSQL (#3856)
* adding support for getting list of foreign tables for PostgreSQL

* need extra newline to pass lint
2017-11-13 21:05:22 -08:00
John Bodley
ba96984048 [flake8] Resolving E3?? errors (#3814) 2017-11-10 17:52:34 -08:00
John Bodley
690de862e8 [flake8] Resolve E1?? errors (#3805) 2017-11-10 12:06:22 -08:00
John Bodley
1ea4521d0c [flake8] Resolving E7?? errors (#3816) 2017-11-09 20:23:59 -08:00
John Bodley
ec21d5af21 [flake8] Resolving E2?? errors (#3812) 2017-11-08 20:34:23 -08:00
John Bodley
17623f71d4 [flake8] Resolving C??? errors (#3787) 2017-11-07 21:32:45 -08:00
John Bodley
e2bca47421 [flake8] Resolve I??? errors (#3797) 2017-11-07 20:23:40 -08:00
Alejandro Fernandez
b059506afa DI-1113. ADDENDUM. Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) (#3697) 2017-11-06 10:20:38 -08:00
Ryan Harmuth
814b70ffd8 Escape columns names for time grains - postgres (#3736) 2017-10-30 21:02:46 -07:00
Alejandro Fernandez
adef519583 DI-1113. Authentication: Enable user impersonation for Superset to HiveServer2 using hive.server2.proxy.user (a.fernandez) (#3652) 2017-10-16 20:15:16 -07:00
Maxime Beauchemin
ccd5fd44cf Removing dependency on pythrifthiveapi (#3494)
Since the latest pyhive, we don't need pythrifthiveapi as they ship with
the latest version.

There's actually a conflict between the new pyhive and pythrifthiveapi
and this fixes it.
2017-09-19 11:19:49 -07:00