Commit Graph

251 Commits

Author SHA1 Message Date
Jesse Yang
b89a5d6660 fix: dashboard cache invalid join query (#11369)
* bugfix: dashboard cache invalid join query

* Use engine instead of session
2020-10-22 12:47:59 -07:00
Rob DiCiuccio
c4effe1261 Log warning for failure to get table/view names from DB (#11374) 2020-10-21 20:09:24 -07:00
Maxime Beauchemin
796a2a6924 fix: error around latest partition in BigQuery (#11274)
* fix: error around latest partition in BigQuery

* lint

* Going with a backend-first approach

* fix test

* add an extra test
2020-10-20 21:34:46 -07:00
Beto Dealmeida
94e23bfc82 feat: export databases as a ZIP bundle (#11229)
* Export databases as Zip file

* Fix tests

* Address comments

* Implement mulexport for database

* Fix lint

* Fix lint
2020-10-16 11:10:39 -07:00
Jesse Yang
dde69d82b3 fix: use dashboard id for stable cache key (#11293) 2020-10-15 17:09:03 -07:00
ʈᵃᵢ
233e67a854 fix: add schema name to datasource field in chart list (#11278) 2020-10-15 14:12:21 -07:00
Jesse Yang
0262daa228 fix: delete the correct dashboard cache key (#11273)
* bugfix: delete dashboard cache key correctly

* update version number
2020-10-14 15:37:59 -07:00
Jesse Yang
2c649ac20f perf: cache dashboard bootstrap data (#11234) 2020-10-13 18:43:06 -07:00
Grace Guo
4f4367edf3 feat: prevent co-edit dashboard collision (#11220)
* feat: prevent co-edit dashboard collision

* fix comments
2020-10-12 17:58:32 -07:00
Bogdan
cb3f649a7f chore: simplify alerting data model to leverage a single class (#11179)
* Collapse alerting models into a single one

Fixing upgrade migration & tests

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-10-09 19:20:12 -07:00
Beto Dealmeida
9785667a0d feat: add UUID column to ImportMixin (#11098)
* Add UUID column to ImportMixin

* Fix default value

* Fix lint

* Fix order of downgrade

* Add logging when downgrade fails

* Migrate position_json to contain UUIDs, and add schedule tables

* Save UUID when adding charts to dashboard

* Fix heads

* Rename migration file

* Fix dashboard serialization

* Fix migration script with Postgres

* Fix unique contraint name

* Handle UUID when exporting dashboard

* Fix Dataset PUT

* Add UUID JSON serialization

* Fix tests

* Simplify logic

* Try binary=True
2020-10-07 09:00:55 -07:00
Daniel Vaz Gaspar
4c85d33109 feat: custom favorite filter for dashboards, charts and saved queries (#11083)
* feat: custom favorite filter for dashboards

* lint and sort

* add favored for charts

* fix tests and lint

* more tests and saved query filter

* fix tests

* fix tests

* lint

* lint and fix conflict

* remove unnecessary prop

* separate tests
2020-10-01 15:08:40 -07:00
Kasia Kucharczyk
92dd257ef4 Changed tags.py and helpers.py in models module: removed disabled pylint rule unused_import, changed unused arguments to private and removed disabled rule unused-argument. Removed redundant rules. (#11037) 2020-09-24 13:50:09 -07:00
Kasia Kucharczyk
63d0e801ad chore: updated lint rules in models module (#11036)
* Removed disabled pylint rule `unused_import` from `dashboards.py` in `models` module. Added missing break in too long line. Formatting.

* Changed `datasource_access_request.py` in models` module: removed disabled pylint rule `unused_import`, removed `no-member` disabled rules which aren't needed. Formatting

* Changed `slice.py` in models` module: removed disabled pylint rule `unused_import`, changed unused arguments to private and removed disabled rule `unused-argument`
2020-09-24 13:49:00 -07:00
Kasia Kucharczyk
2003442b32 fix: several disabled pylint rules in models/helpers.py (#10909)
* Removed conflicting lint and isort check in model helpers seems it's not appearing anymore

* Removed disabled linting for accessing private method. `parent_foreign_key_mappings` becomes public because it is accessed by other instance than `self`.

* Updated model's helper - removed unecessary exception and replaced with check while accessing global context to reset ownerships.

* Updated model's helper - renamed unused attribute to private in user link method.

* Updated model's helper - added specific exception for adding extra json column. Removed disabled pylint rule.

* Applied changes after review to `models/helpers.py`:
- removed unecesary function's param rename
- added extra JSON content in exception

* Removed self.extra_json content from exception message.
2020-09-18 16:32:02 +01:00
Kasia Kucharczyk
141ef4a188 Updated type: ignore in model's helper: (#10906)
- added sql alchemy mapper as a type
2020-09-17 13:49:14 -07:00
Kasia Kucharczyk
cd77797802 Removed disabled linting redefined-outer-name. Url parameters are renamed where url was already defined in class attributes (#10911) 2020-09-16 20:22:46 -07:00
Bogdan
c3f1720456 fix: superset alerting misc fixes (#10891)
* Superset alerting misc fixes

* Test 0 threshold

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-16 11:09:11 -07:00
Kasia Kucharczyk
2a6bcbb1eb Reviewed and repaired disabled pylint in dashboard file (#10877)
* Improved exceptions and unused method arguments which were disabled in linting

* Applied black to dashboard.py
2020-09-15 08:26:04 -07:00
Yongjie Zhao
38edb69d95 Feat: Adding table comment and columns comment for SQLLab (#10844)
* Adding table comment and columns comment for backend

* fix mypy

* Fix CI

* adding wider catch

* use logger

* fix lint
2020-09-14 21:11:48 -07:00
Daniel Vaz Gaspar
136f90f87c feat: SavedQuery REST API for bulk delete and new API fields (#10793)
* feat: SavedQuery REST API for bulk delete

* fix, singular msg and test

* remove 403 from OpenAPI spec

* filter by current user using created_by add sql_tables field

* fixes for new filter, add user field on pre_update, pre_add

* add lru cache to property

* Revert "add lru cache to property"

This reverts commit ad0d9428
2020-09-11 09:36:47 +01:00
Jason Davis
54ae3b044f fix: add validator information to email/slack alerts (#10762)
* added validator info to alerts

* adjusted format of messages

* added nits

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-03 08:15:05 -07:00
Jason Davis
b59f6b1833 feat: refractored SQL-based alerting framework (#10605)
* added new tables for alerting refractor

* reformatted inheritance structure

* added workflow for updated framework

* added suggested changes

* cleaned up changes

* added obervations to alert table to enable view

* added comments

* added requested changes

* fix tests

* added styling changes

* mypy

* added requested changes

* updated operator logic

* requested changes, 1 validator, styling changes

* refactored tests

* fix test alert workflow

* fixed create_alert in test

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-01 13:36:02 -07:00
Bogdan
4572ebb600 chore: log cache keys to the logs (#10678)
* Log cache keys to the logs

* Add tests

* Use separate table for the cache keys

* Add migration for the cache lookup table

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-01 09:41:25 -07:00
Ville Brofeldt
dd0bc472e3 refactor(database): use SupersetResultSet on SqlaTable.get_df() (#10707)
* refactor(database): use SupersetResultSet on SqlaTable.get_df()

* lint

* change cypress test
2020-08-28 21:12:03 +03:00
Ville Brofeldt
9461f9c1e0 fix(db_engine_specs): improve Presto column type matching (#10658)
* fix: improve Presto column type matching

* add optional callback to type map and add tests

* lint

* change private to public
2020-08-24 22:42:07 +03:00
Jason Davis
c0bb86d0d6 feat: slack integration for SQL-based alerts (#10566)
* add slack functionality

* deleted unused variable

* updated test

* black

* fix rebase

* added nits

* added slack no screenshot integration

* isort

* added namedtuple for screenshot

* added test

* fix precommit

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-11 11:15:31 -07:00
Ville Brofeldt
363abfa1a5 fix: remove unnecessary exception when exploring non-legacy viz plugins (#10538)
* fix: remove unnecessary exception when exploring non-legacy viz plugins

* lint
2020-08-07 08:41:39 +03:00
Erik Ritter
fd2d1c58c5 Revert "chore: Cleanup database sessions (#10427)" (#10537)
This reverts commit 7645fc85c3.
2020-08-06 15:33:48 -07:00
John Bodley
7645fc85c3 chore: Cleanup database sessions (#10427)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-30 23:07:56 -07:00
Daniel Vaz Gaspar
2fd37b18e3 fix(api): fixes perf on charts and introduces sorting by database on datasets (#10392) 2020-07-23 11:19:05 -07:00
Erik Ritter
5fa4680447 feat: update timeout error UX (#10274) 2020-07-20 15:32:17 -07:00
Daniel Vaz Gaspar
2b061fc64b fix: modified by column on charts and dashboards (#10340) 2020-07-16 23:15:47 +01:00
Daniel Vaz Gaspar
74cb82e1ad fix: humanised changed on UTC on dashboards and charts (#10321)
* fix: API marshmallow3 drop utc for naive datetime fields

* fix: API marshmallow3 drop utc for naive datetime fields

* fix, tests

* isort and test

* black

* add and fix test

* fix comment
2020-07-15 19:09:32 +01:00
Ville Brofeldt
ac85aebe4a chore: prefer allow/deny terminology (#10320)
* chore: prefer allow/deny terminology

* fix tests

* add PR reference
2020-07-15 19:22:46 +03:00
Daniel Vaz Gaspar
6224edd42d fix: dashboard endpoint sig changed (#10220)
* fix(thumbnails): dashboard endpoint sig changed

* fix, flask get url for Superset.dashboard

* add simple test
2020-07-09 11:42:38 +01:00
Maxime Beauchemin
318e5347bc feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Maxime Beauchemin
2314aad450 feat: add database dropdown to dashboard import (#10118)
* feat: add database dropdown to dashboard import

Currently, when importing a database from a JSON file, the process
looks at the database name from the source (the info is in the file)
and matches the datasources to that name. If no database by that name
exists, it simply fails.

With this PR, we add a database dropdown that allows the user to specify
which databases the datasources should target as the get upserted.

I want to stress that the code in this area is not in a great shape,
and that the challenge of serializing/deser the nested objects is
challenging, but that there should be a much better way to do this.
One of the improvement (out of scope for this PR) that would allow to
simplify those import/export would be to use UUIDs for
importable/exportable objects.

Another identified issue is the indirections between
`utils/import_expor_{model}.py` on top of `{Model}.import_object`. Not
addressing that here.

Next topic is the MVC stuff. Decided to stick with it for now as this is
more of a [obious missing feat:] than a rewrite.

* isort \!? 0%^$%Y$&?%$^?%0^?

* fix tests

* pre-committing to py3.6

* address dpgaspar's comments

* revert isort
2020-07-05 15:08:37 -07:00
Erik Ritter
b686004294 feat: add configuration for Presto cursor poll interval (#10191) 2020-06-30 08:56:22 -07:00
Jason Davis
72fc581b15 added explore database for ctas/cvas (#10174)
Co-authored-by: Jason Davis <@dropbox.com>
2020-06-29 11:13:54 -07:00
Jason Davis
9de9e1c19d feat: Added configuration to SQL Lab results "Explore" button (#10164)
* added configuration to virtual table explore button

* added description to mixin

* fixed unit tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-26 09:54:55 -07:00
Maxime Beauchemin
1781ebbaa4 fix: [search query view] edit link is broken (#10128)
* fix: [search query view] edit link is broken

* eslint + mypy

* rever app.py changes

* addressing comments

* use api/v1/query

* fix test
2020-06-25 12:20:10 -07:00
John Bodley
37777f382f fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
John Bodley
aefef9ca55 chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
John Bodley
15f267d586 style(mypy): Fix memoize watch type (#9970)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-02 16:02:07 -07:00
John Bodley
e789a35558 [mypy] Enforcing typing for superset.models (#9883)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-22 20:31:21 -07:00