Commit Graph

57 Commits

Author SHA1 Message Date
Ville Brofeldt
acb00f509c fix(chart-data-api): assert referenced columns are present in datasource (#10451)
* fix(chart-data-api): assert requested columns are present in datasource

* add filter tests

* add column_names to AnnotationDatasource

* add assertion for simple metrics

* lint
2020-08-14 20:58:24 +03:00
Ville Brofeldt
08358d623b fix: handle query exceptions gracefully (#10548)
* fix: handle query exceptions gracefully

* add more recasts

* add test

* disable test for presto

* switch to SQLA error
2020-08-07 17:37:40 +03:00
Erik Ritter
fd2d1c58c5 Revert "chore: Cleanup database sessions (#10427)" (#10537)
This reverts commit 7645fc85c3.
2020-08-06 15:33:48 -07:00
John Bodley
7645fc85c3 chore: Cleanup database sessions (#10427)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-30 23:07:56 -07:00
Ville Brofeldt
c44ee06b5d fix(chart-data-api): improve chart data endpoint errors (#10300)
* fix: improve chart data error response

* Populate error_message in QueryResult

* add tests

* Lint + fix incorrect raise

* add more tests
2020-07-14 12:40:00 +03:00
Ville Brofeldt
14260f9843 feat: add contribution operation and fix cache_key bug (#10286)
* feat: add contribution operation and fix cache_key_bug

* Add contribution schema
2020-07-10 17:06:05 +03:00
John Bodley
cf60f664a4 fix: Remove double pickling for cached payloads (#10222)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-02 16:46:36 -07:00
John Bodley
aefef9ca55 chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Ville Brofeldt
a3393c1bc4 fix: native annotations (#10037)
* fix: native annotations

* Add test

* Add comment to test
2020-06-12 21:53:26 +03:00
Ville Brofeldt
315518d2d2 feat: add support for query offset (#10010)
* feat: add support for query offset

* Address comments and add new tests
2020-06-09 11:46:28 +03:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Ville Brofeldt
5c4d4f16b3 feat: finalize Word Cloud move to new chart data endpoint (#9975)
* remove word cloud from viz.py

* Fix Run in SQL Lab

* remove deprecated python tests

* break out legacy endpoint type into function

* Break out exploreChart from exportChart and implement results type

* Fix jest tests and refactor accordingly

* lint

* Rename v1 payload function

* Add dashboard id to v1 chart data request url params

* Add support for domain sharding to v1 chart data request
2020-06-05 14:08:46 +03:00
John Bodley
ee777acd57 fix(mypy): Resolves regression introducted in #9824 (#9973)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 00:20:34 -07:00
Ville Brofeldt
38a6bd79da feat: expand new chart data endpoint coverage (#9903)
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
2020-06-02 10:47:28 +03:00
Ville Brofeldt
a43a1d6303 feat: Add new result formats and types to chart data API (#9841)
* feat: Add new result formats and types to chart data API

* lint

* Linting

* Add language to query payload

* Fix tests

* simplify tests
2020-05-20 21:36:14 +03:00
Bogdan
283504479e Fix ENABLE_ROW_LEVEL_SECURITY flag usage (#9689)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-29 23:57:06 +03:00
Ville Brofeldt
a6cedaaa87 chore: Improve chart data API + schemas + tests (#9599)
* Make all fields optional in QueryObject and fix having_druid schema

* fix: datasource type sql to table

* lint

* Add missing fields

* Refactor tests

* Linting

* Refactor query context fixtures

* Add typing to test func
2020-04-23 14:30:48 +03:00
Ville Brofeldt
0472c11157 fix: Add deprecated fields to QueryObject schema (#9579)
* fix: Add deprecated fields to QueryObject schema

* linting
2020-04-18 20:28:57 +03:00
Ville Brofeldt
8e439b1115 chore: Add OpenAPI docs to /api/v1/chart/data EP (#9556)
* Add OpenAPI docs to /api/v1/chart/data EP

* Add missing fields to QueryObject, fix linting and unit test errors

* Fix unit test errors

* abc

* Fix errors uncovered by schema validation and add unit test for invalid payload

* Add schema for response

* Remove unnecessary pylint disable
2020-04-17 16:44:16 +03:00
Ville Brofeldt
9447381549 deprecate groupby controls in query_obj (#9366)
* Deprecate groupby from query_obj

* Fix query_object bug

* Fix histogram

* Remove groupby from legacy druid connector and fix first batch of unit tests

* Deprecate some unnecessary tests + fix a few others

* Address comments

* hide SIP-38 changes behind feature flag

* Break out further SIP-38 related tests

* Reslove test errors

* Add feature flag to QueryContext

* Resolve tests and bad rebase

* Backport recent changes from viz.py and fix broken DeckGL charts

* Fix bad rebase

* backport #9522 and address comments
2020-04-14 12:06:10 +03:00
Ville Brofeldt
a8ce3bccdf feat: Add post processing to QueryObject (#9427)
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
2020-04-10 20:50:11 +03:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
John Bodley
5e55e09e3e [mypy] Enforcing typing for some modules (#9416)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-04 12:45:14 -07:00
John Bodley
19da9042ff [cache] Cleaning up viz/cache logic (#9375)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-03-26 09:22:06 -07:00
Ville Brofeldt
7a91498cf1 fix: share column type matching between model and result set (#9161)
* Share column type matching between model and result set

* Address comments
2020-03-04 16:33:22 +02:00
John Bodley
7b2c2e8a33 [fix] Adding SIP-15 support for the query context (#9219) 2020-03-02 10:10:54 -08:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Erik Ritter
c214cf1a43 [Viz/Query] Improve logging around cache hits (#9058) 2020-01-31 08:46:12 -08:00
John Bodley
dc60db2a43 [fix] Reverting metic logic from #8901 (#9030) 2020-01-27 22:28:42 -08:00
Ville Brofeldt
c087a48d52 fix: add datasource.changed_on to cache_key (#8901)
* Add datasource.changed_on to cache_key and add+fix related unit tests

* Add note to UPDATING.md

* Remove redundant comment about metric names
2020-01-16 00:51:13 -05:00
John Bodley
0fe047171b [fix] Enforce the QueryResult.df to be a pandas.DataFrame (Phase II) (#8948) 2020-01-12 21:35:44 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
7f8db7ab8d Re-enable pylint for common, connector, and test files (#8774)
* re-enable pylint for superset/common/query_context.py

* Re-enable pylint for superset/connectors/base/views.py

* re-enable pylint for superset/connectors/connector_registry.py

* Re-enable pylint for tests/fixtures/pyodbcRow.py

* re-enable pylint for superset/cli.py

* Fix for mypy

* Improve variable naming

* A few more naming fixes
2019-12-09 22:32:54 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley
786d7706e7 [fix] Updating parse_human_timedelta typing (#8436) 2019-10-23 16:04:46 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
dfb3bf69a0 [typing] add typing for superset/connectors and superset/common (#8138) 2019-09-19 16:51:01 -07:00
Erik Ritter
ee75d03df4 Fix cache set kwargs (#8106) 2019-08-23 16:54:37 -07:00
Erik Ritter
01aa17cb70 Fix sqla query cache keys function (#8105) 2019-08-23 14:14:38 -07:00
Beto Dealmeida
10f00cdde3 [fix] Isolate and improve performance on tagging system (#7858)
* Fix tag perf

* Add ASF header

* Make script idempotent

* Add CLI to sync tags

* Add missing file

* Merge heads

* Fix lint

* Remove script
2019-07-31 09:19:39 -07:00
Ville Brofeldt
07a76f83b1 [Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Ville Brofeldt
4568b2a532 Add cache_key_wrapper to Jinja template processor (#7816) 2019-07-20 09:12:35 -07:00
Maxime Beauchemin
87f50ba0d8 Make SHOW_STACKTRACE config apply across the board (#7845) 2019-07-12 08:58:44 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Ville Brofeldt
c1712e5d10 Add relative start param for time filters (#7525)
* Add relative start param for time filters

* Fix typo and add types to parse_human_datetime

* Add relative start/end to query_object

* Fix linting error
2019-05-31 16:55:26 +03:00
Conglei
fda1002654 fix: cache issue for api/v1/query (#7507) 2019-05-14 14:21:05 -07:00
John Bodley
80d6f5a090 [mypy] Adding mypy linting (#7053) 2019-03-24 20:35:48 -07:00
John Bodley
61add606ca [missing values] Removing replacing missing values (#4905) 2019-03-20 17:14:15 -07:00