QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-04-12 04:37:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,276 Commits 421 Branches 541 Tags
9dfa33fedfc2f8524d07934fcf5dfd6dc4db794a
Commit Graph

65 Commits

Author SHA1 Message Date
John Bodley
be7065faf8 chore(pylint): Reenable raise-missing-from check (#16266) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2021-08-17 10:26:10 +12:00
John Bodley
b5c7ed9f18 chore(pylint): Reenable too-many-locals check (#16268) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2021-08-17 04:23:40 +12:00
Yongjie Zhao
32d2aa0c40 feat: run extra query on QueryObject and add compare operator for post_processing (#15279) 
* rebase master and resolve conflicts

* pylint to makefile

* fix crash when pivot operator

* fix comments

* add precision argument

* query test

* wip

* fix ut

* rename

* set time_offsets to cache key

wip

* refactor get_df_payload

wip

* extra query cache

* cache ut

* normalize df

* fix timeoffset

* fix ut

* make cache key logging sense

* resolve conflicts

* backend follow up iteration 1

wip

* rolling window type

* rebase master

* py lint and minor follow ups

* pylintrc
 2021-07-28 22:34:39 +08:00
ofekisr
15796ea4ac refactor: type hints should not be load in runtime (#15540) 2021-07-05 16:48:53 +03:00
Sam Faber-Manning
11260b3117 chore: add stack trace to all calls of logger.error (#14382) 2021-04-30 09:22:05 -07:00
Ben Reinhart
55ba47ec2e fix(#13734): Properly escape special characters in CSV output (#13735) 
* fix: Escape csv content during downloads

* Reuse CsvResponse object

* Use correct mimetype for csv responses

* Ensure that headers are also escaped

* Update escaping logic
 2021-03-26 15:22:00 -07:00
Jesse Yang
bd1d6acb0c fix(query): order by adhoc metrics should trigger group by (#13434) 
* fix(query): properly select adhoc metrics in orderby

* Throw error when sql is empty

* Allow `metrics` to be None

* Always use alias in orderby for metrics

* Bump table chart version and migrate histogram to typescript

* Fix Histogram without groupby

* Fix Presto birth names test

* Raw records mode should not aggregate
 2021-03-16 21:00:03 -07:00
Ville Brofeldt
70e12ed27d chore: bump pyarrow and pandas (#12882) 
* bump pyarrow and pandas

* remove df copy
 2021-03-02 19:44:53 +02:00
Ville Brofeldt
0a00153375 feat(chart-data): add rowcount, timegrain and column result types (#13271) 
* feat(chart-data): add rowcount, timegrain and column result types

* break out actions from query_context

* rename module
 2021-02-24 07:43:47 +02:00
Ville Brofeldt
d8c32b8097 fix(chart-data-api): support numeric temporal columns (#13138) 2021-02-16 09:51:22 +02:00
Ville Brofeldt
ac73991913 fix(viz): improve dtype inference logic (#12933) 2021-02-04 10:47:27 -08:00
Jesse Yang
e3db935c62 refactor: migrate table chart to new API (#10270) 
* refactor: migrate table chart to new API

* chore: bump superset-ui to 0.17.0

* Fix Cypress tests

* Apply soft-conversion to numeric metrics

Fix time column formatting test

* Add translation to chart does not exist error

* Bump to 0.17.1
 2021-01-29 13:12:09 +02:00
Rob DiCiuccio
d7cbd53fce fix(async queries): Remove "force" param on cached data retrieval (#12103) 
* Async queries: remove force cache param on data retrieval

* Assert equal query_object cache keys

* Decouple etag_cache from permission checks

* Fix query_context test

* Use marshmallow EnumField for validation
 2021-01-27 10:16:57 -08:00
Jesse Yang
deebd6f738 fix(viz): missing groupby and broken adhoc metrics for boxplot (#12556) 2021-01-15 17:03:12 -08:00
Rob DiCiuccio
4d329071a1 feat(SIP-39): Async query support for charts (#11499) 
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
 2020-12-10 20:21:56 -08:00
Ville Brofeldt
327a2817d3 feat: add event and interval annotation support to chart data ep (#11665) 
* feat: add event and interval annotation support to chart data ep

* add tests + refactor fixtures

* use chart dao
 2020-12-04 14:40:31 +02:00
Jesse Yang
4cfcaebb61 feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509) 
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
 2020-11-13 22:35:10 -08:00
Ville Brofeldt
77dff0e4e0 fix(chart-data): ignore orderby on sample result type (#11656) 2020-11-11 16:27:56 +02:00
Ville Brofeldt
600a6fa92a chore(rls): move to feature flag and disable related view (#11575) 
* chore(rls): move to feature flag and disable related view

* rename feature flag
 2020-11-05 12:19:48 +02:00
David Aaron Suddjian
18658f45be feat(dashboards): Filter status indicators (#10936) 
* Initial commit of new filters badge.

* refactor applied/rejected filters code

* finished filter indicators

* filter badge tested

* unnecessary imports

* formatting and types

* fixes

* license

* code quality tweaks

* state management for showing focused filter scope

* clean up filter key extraction code

* remove unnecessary styles

* temp css to demonstrate highlighting

* fix focused filter logic

* no more color badges

* new toys for highlighting dash components (#11144)

* tweak style for the filter chart when filter is focused

* style: Filters p0 css2 (#11151)

* nixing background tweak

* src paths

* another quick theme color

* src paths, adjusting pill icon color, changing icons, showing applied/busted counts

* linting stuff

* fixing and tweaking tests

* show filter indicator when filters are not active

* chart title bar cleanup

* open the right panel when popover opens

* unused import

* fix EditableTitle tests

* margin on dashboard header

* show the chart dropdown menu

* fix blur filter breaking dropdowns

* style tweak - no pointer events when irrelevant charts are blurred

* fix box shadow on filter highlight

* it's an array

* attempt fixing e2e

* style: filters p0 icon churn (#11215)

* new filters icon

* icon styling

* bigger icons in list views

* better sizing of table actions and favStars

* more icon sizing...

* fixing more button size jankiness

* linting

* Filters performance (#11255)

* fixing time filter "ok" button

* making unset filter menu collapsible

* sort alphabetically

* fix highlighting when removing items

* try a flex layout (for browser render perf)

* more specific transitioning

* temp: comment out some code as a test

* temp: comment out more code

* temp: remove possibly expensive computations from ChartHolder

* Revert "temp: comment out some code as a test"

This reverts commit 309b880e90.

* Revert "temp: comment out more code"

This reverts commit 64c88b2cba.

* Revert "temp: remove possibly expensive computations from ChartHolder"

This reverts commit 37ce0214f0.

* experiment: upgrade react-select to v3

* Revert "experiment: upgrade react-select to v3"

This reverts commit c3972ba486.

* fix the damn problem

* remove code used for testing purposes

* awful hack to avoid adding a class to a container

* approaching infinity... and not beyond!

* fix ref forwarding

* add theme to tests as necessary

* fix(extra-filters): add logic for identifying applied extra filters (#11325)

* fix: use dashboard id for stable cache key (#11293)

* fix: button translations missing (#11187)

* button translations missing

* blank space before text

* feat: update time_compare description and choices (#11294)

* feat: update time_compare description and choices

* Update sections.jsx

* fix(extra-filters): add logic for identifying applied extra filters

* lint

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>

* address design feedback

* slight tweak to panel logic, keep panels open that user has opened

* rearrange code to be more graceful

* fix: bump superset-ui/core (#11385)

* use is_dttm instead of is_temporal

* types, names

* only show unset filter panel if there are unset filters

* fix highlighting the filter control

* fix filterbox layout

* translations

* fix cypress

* actually add the test attribute

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* formatting

* add link comment to hack

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* stop importing lodash

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* skip broken test

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* adjust colors of titles

* linting

* no indicators when chart is loading

* support all time fields

* fix lock file

Co-authored-by: Natalie Ruhe <natalie@preset.io>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
 2020-10-28 15:46:24 -07:00
Ville Brofeldt
e230865216 chore(chart-data): add annotations and url params to chart data schema (#11393) 
* chore(chart-data): add annotations to chart data schema

* bump superset-ui packages

* xit flaky cypress tests
 2020-10-28 23:41:39 +02:00
Ville Brofeldt
917d02efbd fix(chart-api): cached-indicator always shows value is cached (#11326) 2020-10-19 16:46:42 +03:00
Ville Brofeldt
ada66e30dd fix(chart-data-api): ignore missing filters (#11112) 2020-09-30 09:34:23 +01:00
Kasia Kucharczyk
f01c488664 fix: setting specific exceptions common/query_context.py (#10942) 
* Specified an exceptions in reading cache in `get_df_payload()` method

* Reverted change after review:
- added broad exception in `get_df_payload` method
 2020-09-21 08:46:08 -07:00
Kasia Kucharczyk
e21a354b3b fix: update pylint disabled checks in common/query_context.py (#10941) 
* Updated common/query_context.py
- removed disabled pylint rule no-self-use from `df_metrics_to_num` since it has `@staticmethod`
- applied black on the file

* Removed disabled lint check `too-many-locals` in get_df_payload method. Applied black.

* Method `get_data()` has self param:
- removing # pylint: disable=no-self-use
- autoformatting
 2020-09-17 21:30:02 -07:00
Bogdan
4572ebb600 chore: log cache keys to the logs (#10678) 
* Log cache keys to the logs

* Add tests

* Use separate table for the cache keys

* Add migration for the cache lookup table

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
 2020-09-01 09:41:25 -07:00
Ville Brofeldt
acb00f509c fix(chart-data-api): assert referenced columns are present in datasource (#10451) 
* fix(chart-data-api): assert requested columns are present in datasource

* add filter tests

* add column_names to AnnotationDatasource

* add assertion for simple metrics

* lint
 2020-08-14 20:58:24 +03:00
Ville Brofeldt
08358d623b fix: handle query exceptions gracefully (#10548) 
* fix: handle query exceptions gracefully

* add more recasts

* add test

* disable test for presto

* switch to SQLA error
 2020-08-07 17:37:40 +03:00
Erik Ritter
fd2d1c58c5 Revert "chore: Cleanup database sessions (#10427)" (#10537) 
This reverts commit 7645fc85c3.
 2020-08-06 15:33:48 -07:00
John Bodley
7645fc85c3 chore: Cleanup database sessions (#10427) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2020-07-30 23:07:56 -07:00
Ville Brofeldt
c44ee06b5d fix(chart-data-api): improve chart data endpoint errors (#10300) 
* fix: improve chart data error response

* Populate error_message in QueryResult

* add tests

* Lint + fix incorrect raise

* add more tests
 2020-07-14 12:40:00 +03:00
John Bodley
cf60f664a4 fix: Remove double pickling for cached payloads (#10222) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2020-07-02 16:46:36 -07:00
John Bodley
aefef9ca55 chore(security): Updating assert logic (#10034) 
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2020-06-23 20:49:39 -07:00
Ville Brofeldt
a3393c1bc4 fix: native annotations (#10037) 
* fix: native annotations

* Add test

* Add comment to test
 2020-06-12 21:53:26 +03:00
Ville Brofeldt
315518d2d2 feat: add support for query offset (#10010) 
* feat: add support for query offset

* Address comments and add new tests
 2020-06-09 11:46:28 +03:00
Ville Brofeldt
5c4d4f16b3 feat: finalize Word Cloud move to new chart data endpoint (#9975) 
* remove word cloud from viz.py

* Fix Run in SQL Lab

* remove deprecated python tests

* break out legacy endpoint type into function

* Break out exploreChart from exportChart and implement results type

* Fix jest tests and refactor accordingly

* lint

* Rename v1 payload function

* Add dashboard id to v1 chart data request url params

* Add support for domain sharding to v1 chart data request
 2020-06-05 14:08:46 +03:00
Ville Brofeldt
38a6bd79da feat: expand new chart data endpoint coverage (#9903) 
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
 2020-06-02 10:47:28 +03:00
Ville Brofeldt
a43a1d6303 feat: Add new result formats and types to chart data API (#9841) 
* feat: Add new result formats and types to chart data API

* lint

* Linting

* Add language to query payload

* Fix tests

* simplify tests
 2020-05-20 21:36:14 +03:00
Bogdan
283504479e Fix ENABLE_ROW_LEVEL_SECURITY flag usage (#9689) 
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
 2020-04-29 23:57:06 +03:00
Ville Brofeldt
a8ce3bccdf feat: Add post processing to QueryObject (#9427) 
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
 2020-04-10 20:50:11 +03:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460) 
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
 2020-04-08 20:32:26 +03:00
John Bodley
5e55e09e3e [mypy] Enforcing typing for some modules (#9416) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2020-04-04 12:45:14 -07:00
John Bodley
19da9042ff [cache] Cleaning up viz/cache logic (#9375) 
Co-authored-by: John Bodley <john.bodley@airbnb.com>
 2020-03-26 09:22:06 -07:00
Ville Brofeldt
7a91498cf1 fix: share column type matching between model and result set (#9161) 
* Share column type matching between model and result set

* Address comments
 2020-03-04 16:33:22 +02:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699) 
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
 2020-02-22 11:21:31 +02:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099) 
* Moving away from using the root logger everywhere

* self.logger -> logger
 2020-02-07 23:38:48 -08:00
Erik Ritter
c214cf1a43 [Viz/Query] Improve logging around cache hits (#9058) 2020-01-31 08:46:12 -08:00
Ville Brofeldt
c087a48d52 fix: add datasource.changed_on to cache_key (#8901) 
* Add datasource.changed_on to cache_key and add+fix related unit tests

* Add note to UPDATING.md

* Remove redundant comment about metric names
 2020-01-16 00:51:13 -05:00
John Bodley
0fe047171b [fix] Enforce the QueryResult.df to be a pandas.DataFrame (Phase II) (#8948) 2020-01-12 21:35:44 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900) 
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
 2019-12-31 09:26:23 +02:00