Commit Graph

95 Commits

Author SHA1 Message Date
ofekisr
16e012fe5d refactor(chart.commands): separate commands into two different modules (#17509)
refactor: move imports under TYPE_CHECKING
2021-11-22 19:57:52 +02:00
ofekisr
3f2129b895 refactor: chartDataCommand - remove the responsibly of creating query context from command (#17461) 2021-11-17 17:13:40 +02:00
ofekisr
5d3e1b5c2c refactor: ChartDataCommand into two separate commands (#17425) 2021-11-15 01:00:08 +02:00
Elizabeth Thompson
f10bc6d8fe fix: add fallback and validation for report and cron timezones (#17338)
* add fallback and validation for report and cron timezones

* add logging to exception catch

* Run black

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-11-12 12:28:17 -08:00
John Bodley
fb4650a6eb chore(pylint): Remove top-level disable (#16589)
* chore(pylint): Remove top-level disable

* Update examples.py

* Update command.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-09-15 09:30:23 -07:00
Jack Fragassi
4dc859f89e fix: Ensure alerts & reports aren't schduled when flag is off (#16639)
* Don't schedule alerts & reports when flag is off

* Fix test function name

* Fix test

* Oops

* Another tweak

* Try to lint by hand

* Fix mock
2021-09-10 13:37:21 +01:00
Amit Miran
420dd5b94a chore: Pylint downgrade (#16617)
* Update testing.in

* chore: downgrade pylint to 2.9.6

* downgrade dependence for pylint

* trim space

* pylint rules

Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>
2021-09-08 08:19:15 +03:00
Rob DiCiuccio
4e380db3fd fix: Support Jinja template functions in global async queries (#16412)
* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check
2021-09-03 14:33:29 +03:00
Ville Brofeldt
ac1d779a30 chore(ci): bump pylint to 2.10.2 (#16463) 2021-08-26 20:30:41 +03:00
John Bodley
7e4c940314 chore(pylint): Enable useless-suppression check (#16388)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-24 03:58:41 +12:00
John Bodley
0df15bf207 chore(pylint): Reenable import-outside-toplevel check (#16263)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 05:20:13 +12:00
John Bodley
8e07dd28bc chore(pylint): Re-enable super-with-arguments check (#16138)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-17 04:00:24 +12:00
Ville Brofeldt
d46dc9aa45 chore: upgrade mypy and add type guards (#16227) 2021-08-14 06:31:45 +03:00
John Bodley
24b43beff9 chore(pylint): Bump Pylint to 2.9.6 (#16146)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-14 10:32:28 +12:00
Elizabeth Thompson
ea49aa3d2d feat: add timezones to report cron (#15849)
* add timezones to report cron

* fix test
2021-07-27 18:28:24 -07:00
John Bodley
ab4e3b9bf9 chore: Enforce Mypy for non-tests (#15757)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-22 06:46:43 +12:00
Beto Dealmeida
c9dad05f2b chore: remove retry dependency in favor of backoff (#15788)
* chore: remove retry dep in favor of backoff

* Fix lint
2021-07-20 10:33:37 -07:00
Daniel Vaz Gaspar
727847d2e5 fix: remove unnecessary app context on celery (#15422)
* fix: remove unnecessary app context on celery

* fix lint

* fix lint
2021-06-29 12:16:16 +01:00
Ben Reinhart
a06a2f30af fix: Set g.user to anon user in Celery (#14742)
* Set g.user to anon user in Celery

* Add test

* Fix bug in logic
2021-05-21 14:33:08 -07:00
Sam Faber-Manning
11260b3117 chore: add stack trace to all calls of logger.error (#14382) 2021-04-30 09:22:05 -07:00
Ben Reinhart
e7f5100833 fix: Fix unintended cache misses with async queries (#14291)
* bug: Fix unintended cache misses with async queries

* Ensure sort order

* Ensure columns are sorted

* Update failing tests
2021-04-28 12:14:55 -07:00
Lily Kuang
7980b767c0 feat: Implement Celery SoftTimeLimit handling (#13740)
* log soft time limit error

* lint

* update test
2021-04-12 13:18:17 -07:00
Daniel Vaz Gaspar
89817d4cee fix(alerts/reports): working timeout with celery kill and logic fix (#13911)
* fix: working timeout with celery kill and logic fix

* add config flags

* fix typo

* fix python lint

* log query time for alerts

* add tests

* fix lint
2021-04-08 11:23:31 +01:00
Ben Reinhart
ca506e9396 fix(#13378): Ensure g.user is set for impersonation (#13878) 2021-03-31 11:22:56 -07:00
Jack Fragassi
304e3b36eb feat(alerts & reports): Easier to read execution logs (#13752)
* Prep for migration

* Migration for execution id column

* Generate execution ids for alerts and reports

* Change execution id range

* Add execution id to API endpoint

* Add execution id to execution log view

* Change execution id range

* Change execution id to a uuid

* Fix execution id type

* Switch state and exec. id columns

* Change db column to UUIDType

* Python lint

* Fix failing frontend tests

* execution_id -> uuid

* Fix migration head

* lint

* Use celery task id as the execution id

* lint

* lint for real

* Fix tests
2021-03-30 08:46:16 -07:00
ʈᵃᵢ
3078c84d40 feat(reports): SLACK_API_TOKEN as callable or str (#13634) 2021-03-16 10:06:29 -06:00
Daniel Vaz Gaspar
94fc5d586e docs: deprecate old alerts and dash/charts reports (#13440)
* docs: deprecated old alerts and dash/charts reports

* revert celery default config

* fix lint
2021-03-05 09:40:08 +00:00
Rob DiCiuccio
b4ca39ceeb fix(celery): Reset DB connection pools for forked worker processes (#13350)
* Reset sqlalchemy connection pool on celery process fork

* Fix race condition with async chart loading state

* pylint: ignore

* prettier
2021-02-26 08:05:40 -08:00
Daniel Vaz Gaspar
a0e05a5eff fix(reports): handle exceptions properly in scope (#12811)
* fix(reports): handle exceptions properly in scope

* list and test commit removal

* revert removing commits from tests

* lint
2021-01-28 19:06:19 +00:00
İbrahim Ercan
49e6e420da fix: session error fixed related to thumbnails. (#12760)
* fix: session error fixed related to thumbnails.

* compute_and_cache moved to session scope

* lint fix done
2021-01-27 11:04:14 +00:00
Rob DiCiuccio
e4ae17def5 fix: Stabilize and deprecate legacy alerts module (#12627)
* Add deprecation notice to superset.tasks.schedules

* Reduce retries and schedule window for alerts
2021-01-21 09:51:53 -08:00
Yongjie Zhao
9b0e6d0cc0 refactor: from superset.utils.core break down date_parser (#12408) 2021-01-11 14:16:42 -08:00
Amit Miran
b221417e19 chore: remove harmless incubator refs (#12289)
* chore: migrate all harmless incubator refs like comments and documentation files

* chore: migrate all superset.incubator.apache.org to superset.apache.org to

* chore: migrate dist.apache.org refs

* chore: removing incubating and Disclaimer

* fix: linting issues
2021-01-06 13:45:19 +02:00
Daniel Vaz Gaspar
0171a6bee4 fix(reports): don't log user errors and state change has errors (#12277) 2021-01-06 08:57:10 +00:00
Daniel Vaz Gaspar
1a20552c2b fix(reports): increase crontab size and alert fixes (#12056)
* fix(reports): increase crontab size

* update to current alembic revision

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* lint

* update alembic revision

* fix related fields

* fix test
2020-12-17 18:03:05 +00:00
Rob DiCiuccio
4d329071a1 feat(SIP-39): Async query support for charts (#11499)
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-10 20:21:56 -08:00
Daniel Vaz Gaspar
1e3aaab590 fix(reports): validator_config, report state machine, working_timeout (#11890)
* fix(reports): expect more exceptions and fix validator config

* use a state pattern on command reports

* use a state pattern on command reports continue

* fix multiple heads

* fix unittests

* add more tests

* fix api tests after enum rename

* fix alembic multiple heads

* fix tests

* fix fixture cleanup

* fix mysql tests

* fix initial and not found state

* fix schema, and private public methods, addressing comments

* add new col to the API
2020-12-09 18:19:07 +00:00
Daniel Vaz Gaspar
7e6f04fce4 fix(alerts, thumbnails): celery tasks using flask-sqlalchemy sessions (#11950)
* fix(alerts, thumbnails): celery tasks using flask-sqlalchemy sessions

* selenium user is case sensitive
2020-12-08 13:03:32 +00:00
Daniel Vaz Gaspar
f27ebc4be5 feat: new reports scheduler (#11711)
* feat(reports): scheduler and delivery system

* working version

* improvements and fix grace_period

* add tests and fix bugs

* fix report API test

* test MySQL test fail

* delete-orphans

* fix MySQL tests

* address comments

* lint
2020-11-25 08:50:30 +00:00
Grace Guo
014fe9fd8c fix: [email report] add multiple schedules per hour (#11414) 2020-10-27 09:37:42 -07:00
Bogdan
cb3f649a7f chore: simplify alerting data model to leverage a single class (#11179)
* Collapse alerting models into a single one

Fixing upgrade migration & tests

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-10-09 19:20:12 -07:00
Kasia Kucharczyk
89bf76553e Updated lint in schedules.py in tasks: unused_import, ungrouped_import and changed argument name to remove unused-argument (#11045) 2020-09-29 08:10:46 -07:00
Bogdan
b2fdf12f62 Support jinja templates (#11008)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-24 10:29:44 -07:00
Bogdan
56d001835e fix: use nullpool even for user lookup in the celery (#10938)
* Use nullpool even for user lookup in the celery

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-21 10:34:03 -07:00
Bogdan
801fb4063c Allow empty observations in alerting (#10939)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-21 08:48:02 -07:00
Bogdan
c3f1720456 fix: superset alerting misc fixes (#10891)
* Superset alerting misc fixes

* Test 0 threshold

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-16 11:09:11 -07:00
Bogdan
ac2937a6c5 fix: use nullpool in the celery workers (#10819)
* Use nullpool in the celery workers

* Address comments

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-10 13:29:57 -07:00
Lily Kuang
8a3ac70c06 feat(databases): test connection api (#10723)
* test connection api on databases

* update test connection tests

* update database api test and open api description

* moved test connection to commands

* update error message

* fix isort

* fix mypy

* fix black

* fix mypy pre commit
2020-09-09 13:37:48 -07:00
Jason Davis
54ae3b044f fix: add validator information to email/slack alerts (#10762)
* added validator info to alerts

* adjusted format of messages

* added nits

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-03 08:15:05 -07:00
Jason Davis
b59f6b1833 feat: refractored SQL-based alerting framework (#10605)
* added new tables for alerting refractor

* reformatted inheritance structure

* added workflow for updated framework

* added suggested changes

* cleaned up changes

* added obervations to alert table to enable view

* added comments

* added requested changes

* fix tests

* added styling changes

* mypy

* added requested changes

* updated operator logic

* requested changes, 1 validator, styling changes

* refactored tests

* fix test alert workflow

* fixed create_alert in test

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-01 13:36:02 -07:00