Commit Graph

88 Commits

Author SHA1 Message Date
Beto Dealmeida
16e6452b8c feat: Explorable protocol (#36245) 2025-12-04 13:18:34 -05:00
Beto Dealmeida
a2267d869b refactor: refactor get_query_result (#36057) 2025-11-21 18:14:52 -05:00
squalou
ce72a0ac27 fix: set columns numeric datatypes when exporting to excel (#27229)
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2024-08-23 01:39:47 -06:00
John Bodley
601432ad82 fix(pylint): Address errors/warnings introduced by #27867 (#27889) 2024-04-03 18:41:21 -07:00
John Bodley
07551dc3d4 chore(connector): Cleanup base models and views according to SIP-92 (#24773) 2023-11-21 10:11:50 -08:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
Ville Brofeldt
500d90058f feat: add ability to disable cache (#23439) 2023-03-31 11:41:57 +03:00
Mayur
5ea9249059 fix: respect chart cache timeout setting (#21637) 2022-09-29 08:26:19 -06:00
Ville Brofeldt
a619cb4ea9 chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
Kamil Gabryjelski
07e8837093 feat(explore): export csv data pivoted for Pivot Table [ID-9] (#17512)
* feat(explore): export csv data pivoted for Pivot Table

* Implement dropdown with download csv options

* Change label to "Original"

* Add tests

* Add form data to query context

* Add form data to query context generator

* Explicitly make form_data optional
2021-12-03 12:42:28 +01:00
ofekisr
66d756955b refactor(QueryContext): separate all processing concerns into QueryContextProcessor (#17497) 2021-11-22 10:01:01 +02:00
ofekisr
4b7d06c14b refactor(QueryContext): move df methods utils to utils package (#17496)
* chore(common.utils): modified utils from module based to package based

* refactor(common.utils): move QueryCacheManager to ad-hoc module

* refactor(QueryContext): move df method utils to utils module
2021-11-21 16:01:18 +02:00
ofekisr
8a6ecd3b97 refactor(QueryContext): add QueryContextFactory (#17495) 2021-11-21 14:35:46 +02:00
ofekisr
683d1dab13 refactor(QueryObject): decouple from superset (#17479)
* refactor: queryObject - decouple from superset

* refactor: queryObject - decouple from superset
2021-11-18 19:53:56 +02:00
ofekisr
377db1bd71 refactor: queryObject - add QueryObjectFactory (#17466) 2021-11-18 17:38:17 +02:00
ofekisr
b914e2d497 refactor: queryObject - decouple from queryContext and clean code (#17465) 2021-11-18 14:28:51 +02:00
Ville Brofeldt
e2a429b0c8 feat(explore): adhoc column expressions [ID-3] (#17379)
* add support for adhoc columns to api and sqla model

* fix some types

* fix duplicates in column names

* fix more lint

* fix schema and dedup

* clean up some logic

* first pass at fixing viz.py

* Add frontend support for adhoc columns

* Add title edit

* Fix showing custom title

* Use column name as default value in sql editor

* fix: Adds a loading message when needed in the Select component (#16531)

* fix(tests): make parquet select deterministic with order by (#16570)

* bump emotion to help with cache clobbering (#16559)

* fix: Support Jinja template functions in global async queries (#16412)

* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check

* fix: impersonate user label/tooltip (#16573)

* docs: update for small typos (#16568)

* feat: Add Aurora Data API engine spec (#16535)

* feat: Add Aurora Data API engine spec

* Fix lint

* refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)

* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues

* refactor sql_json view endpoint: separate concern into ad hod method (#16595)

* feat: Experimental cross-filter plugins (#16594)

* fix:fix get permission function

* feat: add cross filter chart in charts gallery under FF

* chore(deps): bump superset-ui to 0.18.2 (#16601)

* update type guard references

* fix imports

* update series_columns schema

* Add changes that got lost in rebase

* Use current columns name or expression as sql editor init value

* add integration test and do minor fixes

* Bump superset-ui

* fix linting issue

* bump superset-ui to 0.18.22

* resolve merge conflict

* lint

* fix select filter infinite loop

* bump superset-ui to 0.18.23

* Fix auto setting column popover title

* Enable adhoc columns only if UX_BETA enabled

* put back removed test

* Move popover height and width to constants

* Refactor big ternary expression

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>
2021-11-15 12:50:08 +02:00
ofekisr
45480f7ae5 refactor move ChartDataResult enums to common (#17399) 2021-11-11 11:41:37 +02:00
Ville Brofeldt
d7834f17e3 fix(filter-indicator): show filters handled by jinja as applied (#17140) 2021-10-18 19:28:05 +02:00
Yongjie Zhao
94282b7ecd fix: time comparison can't guarantee the accuracy (#16895)
* fix: time comparison can't guarantee the accuracy

* fix multiple series

* fix lint

* fix ut

* fix lint

* more ut

* fix typo
2021-09-30 19:59:57 +08:00
ofekisr
c520eb79b0 refactor: sqllab: move sqllab ralated enumns and utils to more logical place (#16843)
* refactor move QueryStatus to common

* refactor move apply_display_max_row_limit to sqllab package

* refactor move limiting_factor to sqllab package

* fix pylint issues
2021-09-26 21:15:57 +03:00
Ville Brofeldt
4e3d4f6daf feat: add global max row limit (#16683)
* feat: add global max limit

* fix lint and tests

* leave SAMPLES_ROW_LIMIT unchanged

* fix sample rowcount test

* replace max global limit with existing sql max row limit

* fix test

* make max_limit optional in util

* improve comments
2021-09-16 19:33:41 +03:00
Ville Brofeldt
836b5e2c86 feat: add support for generic series limit (#16660)
* feat: add support for generic series limit

* refine series_columns logic

* update docs

* bump superset-ui

* add note to UPDATING.md

* remove default value for timeseries_limit
2021-09-16 12:09:08 +03:00
John Bodley
be7065faf8 chore(pylint): Reenable raise-missing-from check (#16266)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 10:26:10 +12:00
John Bodley
b5c7ed9f18 chore(pylint): Reenable too-many-locals check (#16268)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 04:23:40 +12:00
Yongjie Zhao
32d2aa0c40 feat: run extra query on QueryObject and add compare operator for post_processing (#15279)
* rebase master and resolve conflicts

* pylint to makefile

* fix crash when pivot operator

* fix comments

* add precision argument

* query test

* wip

* fix ut

* rename

* set time_offsets to cache key

wip

* refactor get_df_payload

wip

* extra query cache

* cache ut

* normalize df

* fix timeoffset

* fix ut

* make cache key logging sense

* resolve conflicts

* backend follow up iteration 1

wip

* rolling window type

* rebase master

* py lint and minor follow ups

* pylintrc
2021-07-28 22:34:39 +08:00
ofekisr
15796ea4ac refactor: type hints should not be load in runtime (#15540) 2021-07-05 16:48:53 +03:00
Sam Faber-Manning
11260b3117 chore: add stack trace to all calls of logger.error (#14382) 2021-04-30 09:22:05 -07:00
Ben Reinhart
55ba47ec2e fix(#13734): Properly escape special characters in CSV output (#13735)
* fix: Escape csv content during downloads

* Reuse CsvResponse object

* Use correct mimetype for csv responses

* Ensure that headers are also escaped

* Update escaping logic
2021-03-26 15:22:00 -07:00
Jesse Yang
bd1d6acb0c fix(query): order by adhoc metrics should trigger group by (#13434)
* fix(query): properly select adhoc metrics in orderby

* Throw error when sql is empty

* Allow `metrics` to be None

* Always use alias in orderby for metrics

* Bump table chart version and migrate histogram to typescript

* Fix Histogram without groupby

* Fix Presto birth names test

* Raw records mode should not aggregate
2021-03-16 21:00:03 -07:00
Ville Brofeldt
70e12ed27d chore: bump pyarrow and pandas (#12882)
* bump pyarrow and pandas

* remove df copy
2021-03-02 19:44:53 +02:00
Ville Brofeldt
0a00153375 feat(chart-data): add rowcount, timegrain and column result types (#13271)
* feat(chart-data): add rowcount, timegrain and column result types

* break out actions from query_context

* rename module
2021-02-24 07:43:47 +02:00
Ville Brofeldt
d8c32b8097 fix(chart-data-api): support numeric temporal columns (#13138) 2021-02-16 09:51:22 +02:00
Ville Brofeldt
ac73991913 fix(viz): improve dtype inference logic (#12933) 2021-02-04 10:47:27 -08:00
Jesse Yang
e3db935c62 refactor: migrate table chart to new API (#10270)
* refactor: migrate table chart to new API

* chore: bump superset-ui to 0.17.0

* Fix Cypress tests

* Apply soft-conversion to numeric metrics

Fix time column formatting test

* Add translation to chart does not exist error

* Bump to 0.17.1
2021-01-29 13:12:09 +02:00
Rob DiCiuccio
d7cbd53fce fix(async queries): Remove "force" param on cached data retrieval (#12103)
* Async queries: remove force cache param on data retrieval

* Assert equal query_object cache keys

* Decouple etag_cache from permission checks

* Fix query_context test

* Use marshmallow EnumField for validation
2021-01-27 10:16:57 -08:00
Jesse Yang
deebd6f738 fix(viz): missing groupby and broken adhoc metrics for boxplot (#12556) 2021-01-15 17:03:12 -08:00
Rob DiCiuccio
4d329071a1 feat(SIP-39): Async query support for charts (#11499)
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-10 20:21:56 -08:00
Ville Brofeldt
327a2817d3 feat: add event and interval annotation support to chart data ep (#11665)
* feat: add event and interval annotation support to chart data ep

* add tests + refactor fixtures

* use chart dao
2020-12-04 14:40:31 +02:00
Jesse Yang
4cfcaebb61 feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509)
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
2020-11-13 22:35:10 -08:00
Ville Brofeldt
77dff0e4e0 fix(chart-data): ignore orderby on sample result type (#11656) 2020-11-11 16:27:56 +02:00
Ville Brofeldt
600a6fa92a chore(rls): move to feature flag and disable related view (#11575)
* chore(rls): move to feature flag and disable related view

* rename feature flag
2020-11-05 12:19:48 +02:00
David Aaron Suddjian
18658f45be feat(dashboards): Filter status indicators (#10936)
* Initial commit of new filters badge.

* refactor applied/rejected filters code

* finished filter indicators

* filter badge tested

* unnecessary imports

* formatting and types

* fixes

* license

* code quality tweaks

* state management for showing focused filter scope

* clean up filter key extraction code

* remove unnecessary styles

* temp css to demonstrate highlighting

* fix focused filter logic

* no more color badges

* new toys for highlighting dash components (#11144)

* tweak style for the filter chart when filter is focused

* style: Filters p0 css2 (#11151)

* nixing background tweak

* src paths

* another quick theme color

* src paths, adjusting pill icon color, changing icons, showing applied/busted counts

* linting stuff

* fixing and tweaking tests

* show filter indicator when filters are not active

* chart title bar cleanup

* open the right panel when popover opens

* unused import

* fix EditableTitle tests

* margin on dashboard header

* show the chart dropdown menu

* fix blur filter breaking dropdowns

* style tweak - no pointer events when irrelevant charts are blurred

* fix box shadow on filter highlight

* it's an array

* attempt fixing e2e

* style: filters p0 icon churn (#11215)

* new filters icon

* icon styling

* bigger icons in list views

* better sizing of table actions and favStars

* more icon sizing...

* fixing more button size jankiness

* linting

* Filters performance (#11255)

* fixing time filter "ok" button

* making unset filter menu collapsible

* sort alphabetically

* fix highlighting when removing items

* try a flex layout (for browser render perf)

* more specific transitioning

* temp: comment out some code as a test

* temp: comment out more code

* temp: remove possibly expensive computations from ChartHolder

* Revert "temp: comment out some code as a test"

This reverts commit 309b880e90.

* Revert "temp: comment out more code"

This reverts commit 64c88b2cba.

* Revert "temp: remove possibly expensive computations from ChartHolder"

This reverts commit 37ce0214f0.

* experiment: upgrade react-select to v3

* Revert "experiment: upgrade react-select to v3"

This reverts commit c3972ba486.

* fix the damn problem

* remove code used for testing purposes

* awful hack to avoid adding a class to a container

* approaching infinity... and not beyond!

* fix ref forwarding

* add theme to tests as necessary

* fix(extra-filters): add logic for identifying applied extra filters (#11325)

* fix: use dashboard id for stable cache key (#11293)

* fix: button translations missing (#11187)

* button translations missing

* blank space before text

* feat: update time_compare description and choices (#11294)

* feat: update time_compare description and choices

* Update sections.jsx

* fix(extra-filters): add logic for identifying applied extra filters

* lint

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>

* address design feedback

* slight tweak to panel logic, keep panels open that user has opened

* rearrange code to be more graceful

* fix: bump superset-ui/core (#11385)

* use is_dttm instead of is_temporal

* types, names

* only show unset filter panel if there are unset filters

* fix highlighting the filter control

* fix filterbox layout

* translations

* fix cypress

* actually add the test attribute

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* formatting

* add link comment to hack

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* stop importing lodash

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* skip broken test

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* adjust colors of titles

* linting

* no indicators when chart is loading

* support all time fields

* fix lock file

Co-authored-by: Natalie Ruhe <natalie@preset.io>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2020-10-28 15:46:24 -07:00
Ville Brofeldt
e230865216 chore(chart-data): add annotations and url params to chart data schema (#11393)
* chore(chart-data): add annotations to chart data schema

* bump superset-ui packages

* xit flaky cypress tests
2020-10-28 23:41:39 +02:00
Ville Brofeldt
917d02efbd fix(chart-api): cached-indicator always shows value is cached (#11326) 2020-10-19 16:46:42 +03:00
Ville Brofeldt
ada66e30dd fix(chart-data-api): ignore missing filters (#11112) 2020-09-30 09:34:23 +01:00
Kasia Kucharczyk
f01c488664 fix: setting specific exceptions common/query_context.py (#10942)
* Specified an exceptions in reading cache in `get_df_payload()` method

* Reverted change after review:
- added broad exception in `get_df_payload` method
2020-09-21 08:46:08 -07:00
Kasia Kucharczyk
e21a354b3b fix: update pylint disabled checks in common/query_context.py (#10941)
* Updated common/query_context.py
- removed disabled pylint rule no-self-use from `df_metrics_to_num` since it has `@staticmethod`
- applied black on the file

* Removed disabled lint check `too-many-locals` in get_df_payload method. Applied black.

* Method `get_data()` has self param:
- removing # pylint: disable=no-self-use
- autoformatting
2020-09-17 21:30:02 -07:00
Bogdan
4572ebb600 chore: log cache keys to the logs (#10678)
* Log cache keys to the logs

* Add tests

* Use separate table for the cache keys

* Add migration for the cache lookup table

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-01 09:41:25 -07:00
Ville Brofeldt
acb00f509c fix(chart-data-api): assert referenced columns are present in datasource (#10451)
* fix(chart-data-api): assert requested columns are present in datasource

* add filter tests

* add column_names to AnnotationDatasource

* add assertion for simple metrics

* lint
2020-08-14 20:58:24 +03:00