Commit Graph

39 Commits

Author SHA1 Message Date
Ville Brofeldt
acb00f509c fix(chart-data-api): assert referenced columns are present in datasource (#10451)
* fix(chart-data-api): assert requested columns are present in datasource

* add filter tests

* add column_names to AnnotationDatasource

* add assertion for simple metrics

* lint
2020-08-14 20:58:24 +03:00
Ville Brofeldt
08358d623b fix: handle query exceptions gracefully (#10548)
* fix: handle query exceptions gracefully

* add more recasts

* add test

* disable test for presto

* switch to SQLA error
2020-08-07 17:37:40 +03:00
Erik Ritter
fd2d1c58c5 Revert "chore: Cleanup database sessions (#10427)" (#10537)
This reverts commit 7645fc85c3.
2020-08-06 15:33:48 -07:00
John Bodley
7645fc85c3 chore: Cleanup database sessions (#10427)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-30 23:07:56 -07:00
Ville Brofeldt
c44ee06b5d fix(chart-data-api): improve chart data endpoint errors (#10300)
* fix: improve chart data error response

* Populate error_message in QueryResult

* add tests

* Lint + fix incorrect raise

* add more tests
2020-07-14 12:40:00 +03:00
John Bodley
cf60f664a4 fix: Remove double pickling for cached payloads (#10222)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-02 16:46:36 -07:00
John Bodley
aefef9ca55 chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Ville Brofeldt
a3393c1bc4 fix: native annotations (#10037)
* fix: native annotations

* Add test

* Add comment to test
2020-06-12 21:53:26 +03:00
Ville Brofeldt
315518d2d2 feat: add support for query offset (#10010)
* feat: add support for query offset

* Address comments and add new tests
2020-06-09 11:46:28 +03:00
Ville Brofeldt
5c4d4f16b3 feat: finalize Word Cloud move to new chart data endpoint (#9975)
* remove word cloud from viz.py

* Fix Run in SQL Lab

* remove deprecated python tests

* break out legacy endpoint type into function

* Break out exploreChart from exportChart and implement results type

* Fix jest tests and refactor accordingly

* lint

* Rename v1 payload function

* Add dashboard id to v1 chart data request url params

* Add support for domain sharding to v1 chart data request
2020-06-05 14:08:46 +03:00
Ville Brofeldt
38a6bd79da feat: expand new chart data endpoint coverage (#9903)
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
2020-06-02 10:47:28 +03:00
Ville Brofeldt
a43a1d6303 feat: Add new result formats and types to chart data API (#9841)
* feat: Add new result formats and types to chart data API

* lint

* Linting

* Add language to query payload

* Fix tests

* simplify tests
2020-05-20 21:36:14 +03:00
Bogdan
283504479e Fix ENABLE_ROW_LEVEL_SECURITY flag usage (#9689)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-29 23:57:06 +03:00
Ville Brofeldt
a8ce3bccdf feat: Add post processing to QueryObject (#9427)
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
2020-04-10 20:50:11 +03:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
John Bodley
5e55e09e3e [mypy] Enforcing typing for some modules (#9416)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-04 12:45:14 -07:00
John Bodley
19da9042ff [cache] Cleaning up viz/cache logic (#9375)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-03-26 09:22:06 -07:00
Ville Brofeldt
7a91498cf1 fix: share column type matching between model and result set (#9161)
* Share column type matching between model and result set

* Address comments
2020-03-04 16:33:22 +02:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Erik Ritter
c214cf1a43 [Viz/Query] Improve logging around cache hits (#9058) 2020-01-31 08:46:12 -08:00
Ville Brofeldt
c087a48d52 fix: add datasource.changed_on to cache_key (#8901)
* Add datasource.changed_on to cache_key and add+fix related unit tests

* Add note to UPDATING.md

* Remove redundant comment about metric names
2020-01-16 00:51:13 -05:00
John Bodley
0fe047171b [fix] Enforce the QueryResult.df to be a pandas.DataFrame (Phase II) (#8948) 2020-01-12 21:35:44 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
7f8db7ab8d Re-enable pylint for common, connector, and test files (#8774)
* re-enable pylint for superset/common/query_context.py

* Re-enable pylint for superset/connectors/base/views.py

* re-enable pylint for superset/connectors/connector_registry.py

* Re-enable pylint for tests/fixtures/pyodbcRow.py

* re-enable pylint for superset/cli.py

* Fix for mypy

* Improve variable naming

* A few more naming fixes
2019-12-09 22:32:54 -08:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
dfb3bf69a0 [typing] add typing for superset/connectors and superset/common (#8138) 2019-09-19 16:51:01 -07:00
Erik Ritter
ee75d03df4 Fix cache set kwargs (#8106) 2019-08-23 16:54:37 -07:00
Erik Ritter
01aa17cb70 Fix sqla query cache keys function (#8105) 2019-08-23 14:14:38 -07:00
Ville Brofeldt
4568b2a532 Add cache_key_wrapper to Jinja template processor (#7816) 2019-07-20 09:12:35 -07:00
Maxime Beauchemin
87f50ba0d8 Make SHOW_STACKTRACE config apply across the board (#7845) 2019-07-12 08:58:44 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Conglei
fda1002654 fix: cache issue for api/v1/query (#7507) 2019-05-14 14:21:05 -07:00
John Bodley
80d6f5a090 [mypy] Adding mypy linting (#7053) 2019-03-24 20:35:48 -07:00
John Bodley
61add606ca [missing values] Removing replacing missing values (#4905) 2019-03-20 17:14:15 -07:00
Conglei
d5b9795f87 Enhancement of query context and object. (#6962)
* added more functionalities for query context and object.

* fixed cache logic

* added default value for groupby

* updated comments and removed print
2019-03-04 18:06:59 -08:00
Maxime Beauchemin
1dd4d7a587 Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Christine Chambers
81349df625 Parse datasource_id as integer on the server side (#6359)
- Datasource ids are being serialized into strings on the client side and python's default json.load doesn't deserialize strings representing integers as integers. Adding an integer parsing step when initializing QueryContext.
2018-11-10 15:45:03 -08:00
Christine Chambers
8c0551ed46 [SIP-5] Open a new /api/v1/query endpoint that takes query_obj (#6220)
* [SIP-5] Open a new /api/v1/query endpoint that takes query_obj

- Introduce a new handle_superset_exception decorator to avoid repeating the logic for catching SupersetExceptions
- Create a query_obj_backfill method that takes form_data and constructs a query_obj that will be constructed in the client in the future. Use the backfill in explore_json.
- Create a new /api/v1/query endpoint that takes query_obj only and returns the payload data. Note the query_obj is constructed in the client. The endpoint currently only handles query_obj for table view viz (we'll be adding support to new viz types as we go).
- Unit test to verify the new endpoint for table view

* fix tests and lint errors

* - Move the new query endpoint into its own api.py view.
- Create QueryObject and QueryContext class to encapsulate query_object to be built from the client and additional info (e.g. datasource) needed to get the data payload for a given query
- Remove the query_obj_backfill as we'll start building the first query_object on the client so it no longer makes sense to have a short-lived backfill for the matter of days.

* Fixing lint and test errors

* Fixing additional lint error from the previous rebase.

* fixing additional lint error

* addressing additional pr comments

* Make /query accept a list of queries in the query_context object.

* fixing a lint error

* - Move time_shift based calculation and since, until check into util
- Add typing info for get_since_until
- Add new unit tests to verify time_shift calculation and the since until check
2018-11-07 17:23:52 -08:00