Commit Graph

58 Commits

Author SHA1 Message Date
Yongjie Zhao
987740aa8d fix: contribution operator meets nan value (#18782) 2022-02-18 23:28:28 +08:00
Ville Brofeldt
3a42071e0f chore(sql): clean up invalid filter clause exception types (#17702)
* chore(sql): clean up invalid filter clause exception types

* fix lint

* rename exception
2021-12-09 17:49:32 +02:00
ofekisr
683d1dab13 refactor(QueryObject): decouple from superset (#17479)
* refactor: queryObject - decouple from superset

* refactor: queryObject - decouple from superset
2021-11-18 19:53:56 +02:00
ofekisr
377db1bd71 refactor: queryObject - add QueryObjectFactory (#17466) 2021-11-18 17:38:17 +02:00
ofekisr
b914e2d497 refactor: queryObject - decouple from queryContext and clean code (#17465) 2021-11-18 14:28:51 +02:00
Ville Brofeldt
e2a429b0c8 feat(explore): adhoc column expressions [ID-3] (#17379)
* add support for adhoc columns to api and sqla model

* fix some types

* fix duplicates in column names

* fix more lint

* fix schema and dedup

* clean up some logic

* first pass at fixing viz.py

* Add frontend support for adhoc columns

* Add title edit

* Fix showing custom title

* Use column name as default value in sql editor

* fix: Adds a loading message when needed in the Select component (#16531)

* fix(tests): make parquet select deterministic with order by (#16570)

* bump emotion to help with cache clobbering (#16559)

* fix: Support Jinja template functions in global async queries (#16412)

* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check

* fix: impersonate user label/tooltip (#16573)

* docs: update for small typos (#16568)

* feat: Add Aurora Data API engine spec (#16535)

* feat: Add Aurora Data API engine spec

* Fix lint

* refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)

* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues

* refactor sql_json view endpoint: separate concern into ad hod method (#16595)

* feat: Experimental cross-filter plugins (#16594)

* fix:fix get permission function

* feat: add cross filter chart in charts gallery under FF

* chore(deps): bump superset-ui to 0.18.2 (#16601)

* update type guard references

* fix imports

* update series_columns schema

* Add changes that got lost in rebase

* Use current columns name or expression as sql editor init value

* add integration test and do minor fixes

* Bump superset-ui

* fix linting issue

* bump superset-ui to 0.18.22

* resolve merge conflict

* lint

* fix select filter infinite loop

* bump superset-ui to 0.18.23

* Fix auto setting column popover title

* Enable adhoc columns only if UX_BETA enabled

* put back removed test

* Move popover height and width to constants

* Refactor big ternary expression

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>
2021-11-15 12:50:08 +02:00
ofekisr
45480f7ae5 refactor move ChartDataResult enums to common (#17399) 2021-11-11 11:41:37 +02:00
ofekisr
c05c6b3db4 fix missing series validation not raised an exception (#16946) 2021-10-03 14:29:56 +03:00
Ville Brofeldt
4e3d4f6daf feat: add global max row limit (#16683)
* feat: add global max limit

* fix lint and tests

* leave SAMPLES_ROW_LIMIT unchanged

* fix sample rowcount test

* replace max global limit with existing sql max row limit

* fix test

* make max_limit optional in util

* improve comments
2021-09-16 19:33:41 +03:00
Ville Brofeldt
836b5e2c86 feat: add support for generic series limit (#16660)
* feat: add support for generic series limit

* refine series_columns logic

* update docs

* bump superset-ui

* add note to UPDATING.md

* remove default value for timeseries_limit
2021-09-16 12:09:08 +03:00
John Bodley
7e4c940314 chore(pylint): Enable useless-suppression check (#16388)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-24 03:58:41 +12:00
Ville Brofeldt
b61c34f7c9 fix(viz): deduce metric name if empty (#16194)
* fix(viz): deduce metric name if empty

* fix unit test
2021-08-12 11:16:05 +03:00
Ville Brofeldt
63ace7b288 feat(cross-filters): add support for temporal filters (#16139)
* feat(cross-filters): add support for temporal filters

* fix test

* make filter optional

* remove mocks

* fix more tests

* remove unnecessary optionality

* fix even more tests

* bump superset-ui

* add isExtra to schema

* address comments

* fix presto test
2021-08-10 17:18:46 +01:00
Yongjie Zhao
32d2aa0c40 feat: run extra query on QueryObject and add compare operator for post_processing (#15279)
* rebase master and resolve conflicts

* pylint to makefile

* fix crash when pivot operator

* fix comments

* add precision argument

* query test

* wip

* fix ut

* rename

* set time_offsets to cache key

wip

* refactor get_df_payload

wip

* extra query cache

* cache ut

* normalize df

* fix timeoffset

* fix ut

* make cache key logging sense

* resolve conflicts

* backend follow up iteration 1

wip

* rolling window type

* rebase master

* py lint and minor follow ups

* pylintrc
2021-07-28 22:34:39 +08:00
John Bodley
ab4e3b9bf9 chore: Enforce Mypy for non-tests (#15757)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-22 06:46:43 +12:00
Ville Brofeldt
822eb2e27e feat(sql): add jinja support to metrics and expressions (#15247)
* feat(sql): add jinja support to metrics and expressions

* add test
2021-06-19 08:29:04 +03:00
Ben Reinhart
2a1235c0c2 fix: Cleanup serialization and hashing code (#14317) 2021-04-26 14:04:40 -07:00
Jesse Yang
bd1d6acb0c fix(query): order by adhoc metrics should trigger group by (#13434)
* fix(query): properly select adhoc metrics in orderby

* Throw error when sql is empty

* Allow `metrics` to be None

* Always use alias in orderby for metrics

* Bump table chart version and migrate histogram to typescript

* Fix Histogram without groupby

* Fix Presto birth names test

* Raw records mode should not aggregate
2021-03-16 21:00:03 -07:00
Ville Brofeldt
375797f649 feat(native-filters): add timegrain and column filter (#13484)
* feat(native-filters): add timegrain and column filter

* add fetch values predicate

* bump deps

* lint

* fix test

* add python test for legacy merge

* fix default value and isInitialized to not check strict equality

* Address comments

* add FilterValue type

* address review comments
2021-03-09 17:27:46 +02:00
John Bodley
694ae6f90e fix(query-object): extra time-range-endpoints (#13331)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-03-03 13:22:07 +13:00
Ville Brofeldt
0a00153375 feat(chart-data): add rowcount, timegrain and column result types (#13271)
* feat(chart-data): add rowcount, timegrain and column result types

* break out actions from query_context

* rename module
2021-02-24 07:43:47 +02:00
Ville Brofeldt
2f6d1ff4cd chore: remove SIP-38 feature flag (#12894) 2021-02-03 09:41:44 +02:00
Jesse Yang
e3db935c62 refactor: migrate table chart to new API (#10270)
* refactor: migrate table chart to new API

* chore: bump superset-ui to 0.17.0

* Fix Cypress tests

* Apply soft-conversion to numeric metrics

Fix time column formatting test

* Add translation to chart does not exist error

* Bump to 0.17.1
2021-01-29 13:12:09 +02:00
Jesse Yang
deebd6f738 fix(viz): missing groupby and broken adhoc metrics for boxplot (#12556) 2021-01-15 17:03:12 -08:00
Yongjie Zhao
9b0e6d0cc0 refactor: from superset.utils.core break down date_parser (#12408) 2021-01-11 14:16:42 -08:00
Jesse Yang
d2da25a621 refactor: remove queryFields in QueryObject and update chart control configs (#12091)
* Clean up queryFields

* Clean up unused vars

* Bump chart plugins

* Bringing changes in #12147
2020-12-22 17:10:19 -08:00
Rob DiCiuccio
4d329071a1 feat(SIP-39): Async query support for charts (#11499)
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-10 20:21:56 -08:00
Ville Brofeldt
327a2817d3 feat: add event and interval annotation support to chart data ep (#11665)
* feat: add event and interval annotation support to chart data ep

* add tests + refactor fixtures

* use chart dao
2020-12-04 14:40:31 +02:00
David Aaron Suddjian
18658f45be feat(dashboards): Filter status indicators (#10936)
* Initial commit of new filters badge.

* refactor applied/rejected filters code

* finished filter indicators

* filter badge tested

* unnecessary imports

* formatting and types

* fixes

* license

* code quality tweaks

* state management for showing focused filter scope

* clean up filter key extraction code

* remove unnecessary styles

* temp css to demonstrate highlighting

* fix focused filter logic

* no more color badges

* new toys for highlighting dash components (#11144)

* tweak style for the filter chart when filter is focused

* style: Filters p0 css2 (#11151)

* nixing background tweak

* src paths

* another quick theme color

* src paths, adjusting pill icon color, changing icons, showing applied/busted counts

* linting stuff

* fixing and tweaking tests

* show filter indicator when filters are not active

* chart title bar cleanup

* open the right panel when popover opens

* unused import

* fix EditableTitle tests

* margin on dashboard header

* show the chart dropdown menu

* fix blur filter breaking dropdowns

* style tweak - no pointer events when irrelevant charts are blurred

* fix box shadow on filter highlight

* it's an array

* attempt fixing e2e

* style: filters p0 icon churn (#11215)

* new filters icon

* icon styling

* bigger icons in list views

* better sizing of table actions and favStars

* more icon sizing...

* fixing more button size jankiness

* linting

* Filters performance (#11255)

* fixing time filter "ok" button

* making unset filter menu collapsible

* sort alphabetically

* fix highlighting when removing items

* try a flex layout (for browser render perf)

* more specific transitioning

* temp: comment out some code as a test

* temp: comment out more code

* temp: remove possibly expensive computations from ChartHolder

* Revert "temp: comment out some code as a test"

This reverts commit 309b880e90.

* Revert "temp: comment out more code"

This reverts commit 64c88b2cba.

* Revert "temp: remove possibly expensive computations from ChartHolder"

This reverts commit 37ce0214f0.

* experiment: upgrade react-select to v3

* Revert "experiment: upgrade react-select to v3"

This reverts commit c3972ba486.

* fix the damn problem

* remove code used for testing purposes

* awful hack to avoid adding a class to a container

* approaching infinity... and not beyond!

* fix ref forwarding

* add theme to tests as necessary

* fix(extra-filters): add logic for identifying applied extra filters (#11325)

* fix: use dashboard id for stable cache key (#11293)

* fix: button translations missing (#11187)

* button translations missing

* blank space before text

* feat: update time_compare description and choices (#11294)

* feat: update time_compare description and choices

* Update sections.jsx

* fix(extra-filters): add logic for identifying applied extra filters

* lint

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>

* address design feedback

* slight tweak to panel logic, keep panels open that user has opened

* rearrange code to be more graceful

* fix: bump superset-ui/core (#11385)

* use is_dttm instead of is_temporal

* types, names

* only show unset filter panel if there are unset filters

* fix highlighting the filter control

* fix filterbox layout

* translations

* fix cypress

* actually add the test attribute

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* formatting

* add link comment to hack

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* stop importing lodash

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* skip broken test

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* adjust colors of titles

* linting

* no indicators when chart is loading

* support all time fields

* fix lock file

Co-authored-by: Natalie Ruhe <natalie@preset.io>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2020-10-28 15:46:24 -07:00
Ville Brofeldt
e230865216 chore(chart-data): add annotations and url params to chart data schema (#11393)
* chore(chart-data): add annotations to chart data schema

* bump superset-ui packages

* xit flaky cypress tests
2020-10-28 23:41:39 +02:00
Ville Brofeldt
14260f9843 feat: add contribution operation and fix cache_key bug (#10286)
* feat: add contribution operation and fix cache_key_bug

* Add contribution schema
2020-07-10 17:06:05 +03:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Ville Brofeldt
315518d2d2 feat: add support for query offset (#10010)
* feat: add support for query offset

* Address comments and add new tests
2020-06-09 11:46:28 +03:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
John Bodley
ee777acd57 fix(mypy): Resolves regression introducted in #9824 (#9973)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 00:20:34 -07:00
Ville Brofeldt
a6cedaaa87 chore: Improve chart data API + schemas + tests (#9599)
* Make all fields optional in QueryObject and fix having_druid schema

* fix: datasource type sql to table

* lint

* Add missing fields

* Refactor tests

* Linting

* Refactor query context fixtures

* Add typing to test func
2020-04-23 14:30:48 +03:00
Ville Brofeldt
0472c11157 fix: Add deprecated fields to QueryObject schema (#9579)
* fix: Add deprecated fields to QueryObject schema

* linting
2020-04-18 20:28:57 +03:00
Ville Brofeldt
8e439b1115 chore: Add OpenAPI docs to /api/v1/chart/data EP (#9556)
* Add OpenAPI docs to /api/v1/chart/data EP

* Add missing fields to QueryObject, fix linting and unit test errors

* Fix unit test errors

* abc

* Fix errors uncovered by schema validation and add unit test for invalid payload

* Add schema for response

* Remove unnecessary pylint disable
2020-04-17 16:44:16 +03:00
Ville Brofeldt
9447381549 deprecate groupby controls in query_obj (#9366)
* Deprecate groupby from query_obj

* Fix query_object bug

* Fix histogram

* Remove groupby from legacy druid connector and fix first batch of unit tests

* Deprecate some unnecessary tests + fix a few others

* Address comments

* hide SIP-38 changes behind feature flag

* Break out further SIP-38 related tests

* Reslove test errors

* Add feature flag to QueryContext

* Resolve tests and bad rebase

* Backport recent changes from viz.py and fix broken DeckGL charts

* Fix bad rebase

* backport #9522 and address comments
2020-04-14 12:06:10 +03:00
Ville Brofeldt
a8ce3bccdf feat: Add post processing to QueryObject (#9427)
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
2020-04-10 20:50:11 +03:00
John Bodley
5e55e09e3e [mypy] Enforcing typing for some modules (#9416)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-04 12:45:14 -07:00
John Bodley
7b2c2e8a33 [fix] Adding SIP-15 support for the query context (#9219) 2020-03-02 10:10:54 -08:00
John Bodley
dc60db2a43 [fix] Reverting metic logic from #8901 (#9030) 2020-01-27 22:28:42 -08:00
Ville Brofeldt
c087a48d52 fix: add datasource.changed_on to cache_key (#8901)
* Add datasource.changed_on to cache_key and add+fix related unit tests

* Add note to UPDATING.md

* Remove redundant comment about metric names
2020-01-16 00:51:13 -05:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley
786d7706e7 [fix] Updating parse_human_timedelta typing (#8436) 2019-10-23 16:04:46 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
dfb3bf69a0 [typing] add typing for superset/connectors and superset/common (#8138) 2019-09-19 16:51:01 -07:00
Ville Brofeldt
07a76f83b1 [Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Ville Brofeldt
4568b2a532 Add cache_key_wrapper to Jinja template processor (#7816) 2019-07-20 09:12:35 -07:00