Commit Graph

658 Commits

Author SHA1 Message Date
Ville Brofeldt
0807ab44a5 fix(native-filters): merge_extra_form_data extras processing (#14244)
* fix: merge_extra_form_data extras processing

* move props that are later moved back in viz.py
2021-04-20 14:24:33 +03:00
Craig Rueda
a49e0b2037 feat: Adding encrypted field factory (#14109)
* First cut at adding enc type factory

* Finalized enc type factory

* Adding unit test

* PyLinting

* Adding license

* Apply suggestions from code review

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>

* Moving things from enc -> encrypt

* CI commit

* One more fix

* Tweaking config name

* Fixing broken test

* Fixing broken test again

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
2021-04-16 09:01:18 -07:00
Lily Kuang
df7e2b6a8e feat(alert/report): chart as csv format attachment for email and slack (#13828)
* add ui for setting report format

* refactor default notification format

* init csv data alert report

* add report format to report_schedule model

* add ALERTS_ATTACH_REPORTS feature flag

* fix lint

* update check image tag

* fix migrations

Co-authored-by: samtfm <sam@preset.io>
2021-04-15 14:07:49 -07:00
Beto Dealmeida
c1cb3619ab feat: script to benchmark DB migrations (#13561) 2021-04-15 10:55:35 -07:00
simcha90
8ef572a412 refactor(native-filters): update dataMask and ExtraFormData schema (#13983)
* refactor: updates usage of `ownFilters` to `ownState`

* refactor: update dataMask (final)

* lint: fix lint

* refactor: revert feat

* fix: fix missed chart configuration

* add filter set migration

* apply new changes

* fix migration revision

* update migration

* fix jest mock

* js lint

* fix test types

* update tests and types

* remove append_form_data from tests

* fix findExistingFilterSet tests

* add migration test

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2021-04-15 17:43:29 +03:00
Hugh A. Miles II
4b23d0ecca fix: logs table - user_id is NULL (#14057)
* add user back to session

* add test for logging None on exceptions

* fix this updated test

* reformat

* reformat

* Update log.py
2021-04-13 10:08:34 -04:00
Ben Reinhart
1c6173c7ee feat: Make async query JWT cookie domain configurable (#14007) 2021-04-11 20:33:33 -07:00
Rob DiCiuccio
806fb73d25 feat(SIP-39): Websocket sidecar app (#11498)
* WIP node.js websocket app

* Load testing

* Multi-stream publish with blocking reads

* Use JWT for auth and channel ID

* Update ws jwt cookie name

* Typescript

* Frontend WebSocket transport support

* ws server ping/pong and GC logic

* ws server unit tests

* GC interval config, debug logging

* Cleanup JWT cookie logic

* Refactor asyncEvents.ts to support non-Redux use cases

* Update tests for refactored asyncEvents

* Add eslint, write READMEs, reorg files

* CI workflow

* Moar Apache license headers

* pylint found something

* adjust GH actions workflow

* Improve documentation & comments

* Prettier

* Add configurable logging via Winston

* Add SSL support for Redis connections

* Fix incompatible logger statements

* Apply suggestions from code review

Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>

* rename streamPrefix config

Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>
2021-04-08 11:12:03 -07:00
Sam Faber-Manning
31b14dcc88 fix: log exceptions for thumbnail generation as warnings (#13737)
* log errors for thumbnail generation as warnings

* fix: ex_info -> exc_info
2021-04-05 11:43:51 -07:00
Jesse Yang
4789074309 fix(sqla-query): order by aggregations in Presto and Hive (#13739) 2021-04-01 18:10:17 -07:00
Ben Reinhart
55ba47ec2e fix(#13734): Properly escape special characters in CSV output (#13735)
* fix: Escape csv content during downloads

* Reuse CsvResponse object

* Use correct mimetype for csv responses

* Ensure that headers are also escaped

* Update escaping logic
2021-03-26 15:22:00 -07:00
Ville Brofeldt
13f7e0d755 feat(native-filters): add temporal support to select filter (#13622) 2021-03-18 16:15:43 +02:00
Beto Dealmeida
db57f90a34 feat: better error message when adding DBs (#13601)
* WIP

* Adding tests

* Add unit tests

* Show error message

* Fix lint

* Fix after rebase
2021-03-17 20:29:26 -07:00
Yongjie Zhao
7b74eb2730 chore(explore): added debounce in DateFilter (#13660) 2021-03-17 09:04:50 -06:00
Jesse Yang
bd1d6acb0c fix(query): order by adhoc metrics should trigger group by (#13434)
* fix(query): properly select adhoc metrics in orderby

* Throw error when sql is empty

* Allow `metrics` to be None

* Always use alias in orderby for metrics

* Bump table chart version and migrate histogram to typescript

* Fix Histogram without groupby

* Fix Presto birth names test

* Raw records mode should not aggregate
2021-03-16 21:00:03 -07:00
Nikola Gigić
609c3594ef feat(explore): Postgres datatype conversion (#13294)
* test

* unnecessary import

* fix lint

* changes

* fix lint

* changes

* changes

* changes

* changes

* answering comments & changes

* answering comments

* answering comments

* changes

* changes

* changes

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests
2021-03-12 10:36:43 +02:00
Beto Dealmeida
1470e70c10 feat: add option to load big/wide tables (#13448)
* WIP

* feat: add option to load big/wide tables

* Fix lint

* Address comments
2021-03-11 17:17:29 -08:00
Ville Brofeldt
375797f649 feat(native-filters): add timegrain and column filter (#13484)
* feat(native-filters): add timegrain and column filter

* add fetch values predicate

* bump deps

* lint

* fix test

* add python test for legacy merge

* fix default value and isInitialized to not check strict equality

* Address comments

* add FilterValue type

* address review comments
2021-03-09 17:27:46 +02:00
Hugh A. Miles II
c91c45574b feat: add event_logger to test_connection and create_database commands (#13468)
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-03-09 08:17:13 -05:00
Hugh A. Miles II
b17e7aa5c9 feat: refactor on DBEventLogger to allow for context management (#13441)
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-03-05 15:12:42 -05:00
Ville Brofeldt
70e12ed27d chore: bump pyarrow and pandas (#12882)
* bump pyarrow and pandas

* remove df copy
2021-03-02 19:44:53 +02:00
İbrahim Ercan
b04aebfa99 fix(alerts&reports): Alerts & Reports will use values from WEBDRIVER_WINDOW option (#13157)
* fix: thumbnails and reports will be use WEBDRIVER_WINDOW option

* changes reformatted

* config change reverted. thumbnails sizes changed to original

* typo fix

* bugfix

defining defaults in thumbnails.py caused thumbnail caches invalidated.
they moved to init.

Co-authored-by: Ibrahim Ercan <ibrahim.ercan@vlmedia.com.tr>
2021-03-02 10:49:22 +00:00
Yongjie Zhao
94d0bb9e7f fix: date picker support date unit with singular and plural (#13330)
* fix: date picker support date unit with singular and plural

* fix grammar
2021-02-25 17:38:23 +08:00
Daniel Vaz Gaspar
0b114fcbc5 feat(reports): send notification on error with grace (#13135)
* fix: add config to disable dataset ownership on the old api

* fix CI docker build

* fix logic

* add deprecation comment on the config

* feat: send alerts reports errors to recipients

* update

* feat(reports): send notification on error with grace

* merge and revert config

* fix lint and MySQL test

* fix mysql tests
2021-02-24 13:31:31 -08:00
Ville Brofeldt
0a00153375 feat(chart-data): add rowcount, timegrain and column result types (#13271)
* feat(chart-data): add rowcount, timegrain and column result types

* break out actions from query_context

* rename module
2021-02-24 07:43:47 +02:00
bolkedebruin
99a0c8a8a1 refactor: Move raise_for_dashboard_access to security manager (#13235)
raise_for_dashboard_access is part of dashboard and cannot
be overridden by a security manager.
2021-02-23 10:52:34 +00:00
Ville Brofeldt
d8c32b8097 fix(chart-data-api): support numeric temporal columns (#13138) 2021-02-16 09:51:22 +02:00
simcha90
d6fc720f4c feat(native-filters): Time native filter (#12992)
* Add Time Filter component

* Improve Time Filter component

* Fix import errors

* Display Time Filter

* Remove console logs

* Change Control Panel

* Remove unnecessary files

* Use time range override

* test: fix tests

* feat: re run pipeline

* fix: fix some case for Time filter

* fix: merge with master

* use original time range

* fix height

* add cross filter behavior

* apply filters on initialization

* add applied filter to overrides

* add unit tests for merge_extra_form_data

Co-authored-by: Agata Stawarz-Pastewska <agata.stawarz-pastewska@polidea.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2021-02-13 12:09:54 +02:00
simcha90
c5781cde60 feat(style): hide dashboard header by url parameter (#12918)
* feat(native-filters): hide dashboard header bu url parameter

* lint: fix lint

* test: add tests

* test: fix test

* refactor: upgrade standalone param

* fix: pre-commit and extract to method is_standalone_mode

* test: fix tests

* test: fix tests

* fix: fix standalone statement

* refactor: fix CR notes

* chore: pre-commit

* fix: fix sticky tabs + update CR notes

* lint: fix lint

* lint: fix lint

* fix: fix CR notes

* fix: fix CR notes

* lint: fix lint

* refactor: fix cr notes

Co-authored-by: amitmiran137 <amit.miran@nielsen.com>
2021-02-11 15:05:35 +02:00
Beto Dealmeida
7b8a0e184b fix: timeout context manager on Windows (#13041)
* fix: timeout decorator in Windows

* Fix lint
2021-02-10 10:40:28 -08:00
Jesse Yang
bbcb41149e fix: time filter db migration optimization (#13015) 2021-02-09 10:49:50 -08:00
Ville Brofeldt
ac73991913 fix(viz): improve dtype inference logic (#12933) 2021-02-04 10:47:27 -08:00
Amit Miran
b472d1841c feat(dashboard_rbac): dashboard_view access enforcement (#12875)
* test: dashboard_view_test failing

* test: tests works first time

* fix: pre-commit and  some refactoring

* fix:  after CR

* fix:  replace not_published with draft

* fix:  after CR

* fix: pre-commit fixes

* fix: pre-commit and lint fixes

* fix: remove unused

* fix: remove unused import

* fix: wrap the decorator to not block others

* chore: reuse dashboard from decorator into function
2021-02-04 13:23:53 -05:00
Jesse Yang
e3db935c62 refactor: migrate table chart to new API (#10270)
* refactor: migrate table chart to new API

* chore: bump superset-ui to 0.17.0

* Fix Cypress tests

* Apply soft-conversion to numeric metrics

Fix time column formatting test

* Add translation to chart does not exist error

* Bump to 0.17.1
2021-01-29 13:12:09 +02:00
Daniel Vaz Gaspar
a0e05a5eff fix(reports): handle exceptions properly in scope (#12811)
* fix(reports): handle exceptions properly in scope

* list and test commit removal

* revert removing commits from tests

* lint
2021-01-28 19:06:19 +00:00
Rob DiCiuccio
d7cbd53fce fix(async queries): Remove "force" param on cached data retrieval (#12103)
* Async queries: remove force cache param on data retrieval

* Assert equal query_object cache keys

* Decouple etag_cache from permission checks

* Fix query_context test

* Use marshmallow EnumField for validation
2021-01-27 10:16:57 -08:00
Jesse Yang
55c8f9ba60 feat(explore): allow opening charts with missing dataset (#12705) 2021-01-25 15:09:03 -08:00
Craig Rueda
0fed1e04ef feat: Adding option to set_database_uri CLI command (#12740)
* Adding option to set_database_uri CLI command

* Fixing flag logic
2021-01-25 12:49:22 -08:00
Yongjie Zhao
9e58eb809e fix(explore): deprecated x periods pattern in new time picker value (#12552) 2021-01-23 23:41:15 -08:00
Beto Dealmeida
4255c22d01 feat: add decorator to guard public APIs (#12635)
* feat: add decorator to guard public APIs

* Add unit tests

* Refactor to use unit tests

* Paramterize tests

* Remove decorator
2021-01-22 13:06:14 -08:00
bryanck
e7def7e0e2 fix: missing key when verifying adhoc filters in merge_extra_filters (#12620)
Co-authored-by: Bryan Keller <bkeller@netflix.com>
2021-01-20 14:22:05 -08:00
Jesse Yang
c14ed80f28 refactor: rename DbColumnType to GenericDataType (#12617) 2021-01-20 10:07:42 -08:00
Yongjie Zhao
b22e458108 fix(timepicker): make pyparsing thread safe (#12489)
* fix: make pyparsing thread safe

* remove parenthesis for decorator
2021-01-13 17:25:58 +02:00
Yongjie Zhao
9b0e6d0cc0 refactor: from superset.utils.core break down date_parser (#12408) 2021-01-11 14:16:42 -08:00
Yongjie Zhao
9eb911dca3 fix(explore): refine previous calendar range (#12308) 2021-01-06 10:09:13 -08:00
Yongjie Zhao
1f8347228e fix(explore): timepicker show human readable string in pill (#12300)
* fix(explore): timepicker show human readable string in pill

* fix lint
2021-01-06 22:37:35 +08:00
Amit Miran
b221417e19 chore: remove harmless incubator refs (#12289)
* chore: migrate all harmless incubator refs like comments and documentation files

* chore: migrate all superset.incubator.apache.org to superset.apache.org to

* chore: migrate dist.apache.org refs

* chore: removing incubating and Disclaimer

* fix: linting issues
2021-01-06 13:45:19 +02:00
David Aaron Suddjian
865beae3d8 feat(dashboard): Dashboard-Native Filters (#11814)
* wip: filter create modal

* add a feature flag

* automatic changes to package lock

* wip

* filter sidebar and basic state management

* move create button to the sidebar

* first step for edit filterconfig

* partially fix tests...

* edits to types and comments

* respect feature flag on the filter sidebar

* add filterconfig form

* get input state working

* feat: tree filter scopes UI

* fix: turn on flag

* sticky filter bar

* stop preferring default export

* feat: finish filter scoping

* fix: under toggle

* fix: title

* fix: add licence

* refactor: update TS

* fix: fix on reopen modal + validation

* new filter bar menu

* adding, but commenting out, bulk scoping action

* adding some placeholder buttons and styles therefor

* feat: add filter chart

* add relative path to package.json

* update modal

* a little input styling... just getting warmed up

* Revert "feat: add filter chart"

This reverts commit b1302d35b6.

* Revert "add relative path to package.json"

This reverts commit 26a7b40e18.

* https package lock idk

* feat: add filter chart

* add relative path to package.json

* flexboxes all the way down

* dynamically generate groupby and datasource in select control

* big wip

* fix target column name

* no importing nonexistent things

* styles and name editing

* Add hook for retrieval of all filter states

* start with a new filter when clicking add filter

* handle removed filters gracefully

* fix incorrect default filter configuration

* add fields to useAllFilterState

* add redux for filterconfigs

* add support for native_filters

* remove consoles

* improve filter removal

* unbreak infinite loop

* basic sidebar toggling working!

* collapsing and menu working more smoothly

* linting

* make dataset and column inputs work

* save filter values properly

* add dashboard event for filter updates

* guarded

* apply filters properly

* fix schema

* making New Filter button a link

* gridunits ftw

* centering modal

* tis not a button anymore! nixing type.

* plus and collapse buttons instead of "more" menu

* updating full size filter icons

* adding icons to filter collapsing/expanding

* turning off animation, but leaving class-based animation css

* fix linting error

* fix native filters for legacy charts

* updates test

* no individual apply buttons

* fix bugs with filter config modal

* remove redundant code

* switch to the filter with validation errors on submit

* separate form validation

* switch config button from add to edit

* update tests

* oops forgot to add the fancy new useChangeEffect hook

* comments and code reorganization

* rename native_filters to extr_form_data and move hook

* disable native filters in viz selector

* add cascading

* implement new extra form data api

* cleanup

* updates tests

* bump npm packages

* fix bad merge on package.json + lock

* lint

* replace in and not in with uppercase

* lint

* lint

* lint

* lint

* bulk test fix

* Sort select input alphabetically

* Change type for sorting elements

* fix rest of unit tests

* make filter operators all uppercase

* Hide Filter bar when there are no filters

* Show edit button for dashboard owners only

* Add visible argument to filters toggle function to avoid future regression

* Improve Toggle filters bar function

* lint

* fix js lint + set createNewOnOpen

* Handle setting extra form data in Filter Bar instead of Filter Control

* Add Handle apply filter function to Apply button

* Allow applying changes instantly

* Fix types

* remove console logs

* Add Error Boundary component to Filter bar and Filter Config Modal

* fix jest tests

* update native filters tests to pass

* reset cypress baseUrl

* remove unnecessary field

* cleanup: remove unused state fields

* move unrelated types to an appropriate location

* remove misplaced resource fetch error logic

* fix cascadeParentIds error

* fix cypress password

* initial attempt at fixing scope issue

* fix bad merge

* fix lint

* trying out makeApi for saving filters

* remove unused import

* fix test

* silence bad test

* add native-filter feat flag config

* oops fix here

* remove space

* Update superset-frontend/src/common/components/index.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/cypress-base/cypress/integration/dashboard/nativeFilters.test.ts

Co-authored-by: Evan Rusackas <evan@preset.io>

* use styledMount in tests

* comment

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterBar.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigForm.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigModal.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigurationLink.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* address PR feedback

* fix package lock

* null guards

* Fix charts resizing

* fix cypress tests

* add in nativefilters to form data

* fix lint and test

Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com>
Co-authored-by: Simcha Shats <simcha.shats@nielsen.com>
Co-authored-by: amitNielsen <amit.miran@nielsen.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Agata Stawarz-Pastewska <agata.stawarz-pastewska@polidea.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-18 17:06:37 -08:00
Yongjie Zhao
b592cc7e73 feat(explore): time picker enhancement (#11418) 2020-12-17 18:27:21 -08:00
Beto Dealmeida
8bda6b0bd9 feat: show missing parameters in query (#12049)
* feat: show missing parameters in query

* Fix lint

* Address comments

* Simplify error message

* Use f-string in helper function
2020-12-15 18:47:40 -08:00