Commit Graph

197 Commits

Author SHA1 Message Date
Yongjie Zhao
9b0e6d0cc0 refactor: from superset.utils.core break down date_parser (#12408) 2021-01-11 14:16:42 -08:00
Amit Miran
4cf3f99ee8 chore: remove additional incubator references (#12336)
* chore: remove additional incubator references
remove incubator.png from the main docs page

* fix: improve text for contributing-page.mdx

* fix: lint no-unused-vars
2021-01-11 06:09:10 -08:00
Daniel Vaz Gaspar
74f3faf1cd fix(reports): make name unique between alerts and reports (#12196)
* fix(reports): make name unique between alerts and reports

* add missing migration

* make it work for mySQL and PG only (yet)

* fixing sqlite crazy unique drop

* fixing sqlite missing one col
2020-12-24 09:18:46 +00:00
Beto Dealmeida
a85bb4a3f2 fix: remove uniqueness for MySQL (#12185) 2020-12-22 13:39:56 -08:00
David Aaron Suddjian
b5dd0f32cc feat: Dynamically imported viz plugins (#10288)
* first attempts at dynamic plugin loading

* dynamic import working for explore

* memoize appropriately

* add a backend for dynamic plugins

* hack at getting dynamic plugins working with dashboards

* more work on making it work, + feature flag

* lint

* actions to fix explore state when plugins load

* handle dynamic control panel, functionify ExploreViewContainer

* fix: rearrange migrations branch

* fix: name and key as strings with length 50

* bundle url length 2000

* bundle url to text
for some reason not supported on my sql

* fix: too long varchart

* fix: pre-commit typing

* fix: licenses

* fix: add slice container was not initing feature flags

* fix: undo linting issue

* fix: adjust down revision again

* fix: adjust down revision again

* isort

* pylint

* god damn linters

* remove unnecessary(?) loading message

* only log non-standard errors

* testing

* python is terrible

* see above commit message

* fix imports in DynamicPluginProvider

* fix

* shift migration forward

* lint

* fix form data calculations to handle missing control config

* temp commit - waiting for superset-ui changes and crud fixes

* remove unnecessary todo

* use new superset-ui shared module function

* fetch the plugins instead of hardcoding the test one

* migration sort

* remove duplicated import statement

* format

* try moving the import 🙄

* copy

* fix frontend tests

* safe access

* comment out dead code

* isort

* disable pylint on necessary lines

* use @superset-ui/logging instead of console

* remove temp code

* rearrange some code

* try triggering mouseover in cypress before click

* use loading spinner instead of text

* trying to fix cypress

* attempt cypress fix

* customize permissions

* update package lock

* only admins can write to plugins by default

* better copy

* disable flaky tests

* use makeApi

* flaky tests

* cleanup code

* flaaaakkkyyyyyy

* dry

Co-authored-by: amitNielsen <amit.miran@nielsen.com>
2020-12-19 07:06:11 -08:00
Daniel Vaz Gaspar
1a20552c2b fix(reports): increase crontab size and alert fixes (#12056)
* fix(reports): increase crontab size

* update to current alembic revision

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* Merge branch 'master' into feat/security-converge-datasets

# Conflicts:
#	tests/security_tests.py

* lint

* update alembic revision

* fix related fields

* fix test
2020-12-17 18:03:05 +00:00
Daniel Vaz Gaspar
5d9721e6ba feat(queries): security perm simplification (#12072)
* feat(queries): security perm simplification

* fix mig

* update alembic down revision
2020-12-17 14:27:14 +00:00
Daniel Vaz Gaspar
790ac5ef87 feat(databases): security perm simplification (#12036)
* feat(databases): security perm simplification

* fix tests

* fix JS tests
2020-12-17 12:37:05 +00:00
Daniel Vaz Gaspar
dd5cdb1b5f feat(dashboards): security permissions simplification (#12012)
* feat(dashboards): security perm simplification

* fix tests

* fix tests

* remove unnecessary mapping

* fix JS tests

* fix cypress test and explore endpoint

* update alembic down revision

* Merge branch 'master' into feat/security-converge-dashboards

# Conflicts:
#	superset/constants.py
#	tests/security_tests.py
2020-12-17 09:53:13 +00:00
Kasia Kucharczyk
700dee6db6 feat(logs): security permissions simplification (#12061)
* Added migration for logs security converge

* Changed class permission name and method permission in LogModelView and LogRestApi

* Updated recent revision and filename

* Changed name of Log perm in manager. Updated TestRolePermission to have correct menu and permission.

* Updated latest migration revision

* Updated latest migration revision
2020-12-17 00:50:31 -08:00
Daniel Vaz Gaspar
2302adb61a feat(datasets): security perm simplification (#12000)
* feat(datasets): security perm simplification

* feat(datasets): security perm simplification

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* include SqlMetricInlineView converge and fix JS tests

* update to current alembic revision
2020-12-16 11:49:03 +00:00
Kasia Kucharczyk
9c8b65d03f feat(annotations): security permissions simplification (#12014)
* Changed security permissions for annotations and annotation layers

* Updated permissions in annotation layers list

* Created test for retrieving premissions info. Updated uris from f-strings to strings

* Updated annotations in security_tests and added annotations to NEW_SECURITY_CONVERGE_VIEWS

* Added migration for annotations security converge

* Updated current revision after rebase master

* Updated migration name to annotations security converge

* Updated annotations permissions names in AnnotationLayersList and updated test since 'can_write' has wider permissions

* Updated annotations migration to current
2020-12-16 09:08:06 +00:00
Daniel Vaz Gaspar
f79e52f48e feat(charts): security perm simplification (#11981)
* feat(charts): security perm simplification

* fix superset explore

* fix JS test

* fix cypress test

* fix split heads

* fix favorite permission

* fix permission

* update with new async permission

* fix new permission coming from master

* fix core permission assert

* black

* update alembic down revision
2020-12-15 11:27:06 +00:00
Daniel Vaz Gaspar
0f979dea06 feat(reports): security perm simplification (#11853)
* feat: security converge report

* black

* fix: comment

* add frontend changes and rebase

* fix multiple heads
2020-12-15 08:48:00 +00:00
Daniel Vaz Gaspar
1e3aaab590 fix(reports): validator_config, report state machine, working_timeout (#11890)
* fix(reports): expect more exceptions and fix validator config

* use a state pattern on command reports

* use a state pattern on command reports continue

* fix multiple heads

* fix unittests

* add more tests

* fix api tests after enum rename

* fix alembic multiple heads

* fix tests

* fix fixture cleanup

* fix mysql tests

* fix initial and not found state

* fix schema, and private public methods, addressing comments

* add new col to the API
2020-12-09 18:19:07 +00:00
Erik Ritter
77d362d306 fix: Remove expensive logs table migration (#11920) 2020-12-04 07:59:28 -08:00
Daniel Vaz Gaspar
71c567bf3a feat(css templates): security perm simplification (#11856)
* feat: security converge css templates

* fix security tests

* fix JS test

* add migration

* black and fix migration
2020-12-03 08:45:32 +00:00
Daniel Vaz Gaspar
ac9761c730 fix: dashboard iframe to markdown db migration (#11845)
* fix: dashboard iframe to markdown db migration

* Update superset/migrations/versions/978245563a02_migrate_iframe_to_dash_markdown.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-01 20:50:35 +00:00
Daniel Vaz Gaspar
b1a963b93f fix: security converge downgrade procedure (#11852) 2020-11-30 17:50:12 +00:00
Daniel Vaz Gaspar
9dd33d5566 feat(saved queries): security perm simplification (#11764)
* feat(saved queries): security perm simplification

* migration script and frontend

* add downgrade procedure

* downgrade procedure, inferred from the upgrade data

* fix JS test

* improve code

* add tests for role migration

* more tests and a short description for each one

* Improve readability

* simplify dataclass creation

* fix dataclass
2020-11-30 13:07:39 +00:00
Maxime Beauchemin
0504cf1a00 chore: improve analytics (#11714)
* chore: improve analytics

* lint

* log more events, add note in UPDATING.md

* handling base class

* more events\!

* get ref through

* right before @expose

* fix context

* touchups
2020-11-25 08:45:02 -08:00
Daniel Vaz Gaspar
bd79bd2a54 feat: new report schedule models (#11550)
* feat: new report schedule models

* lint and unique constraint

* support sqlite

* fix sqlite

* add audit mixin and minor fixes

* fix FK's

* address comments

* lint
2020-11-06 09:33:32 +00:00
Beto Dealmeida
98daf0a435 feat: add UUID column to saved_query for export/import (#11397)
* Add UUID to saved_query

* Reuse function from previous migration

* Point to new head
2020-10-27 08:49:09 -07:00
Daniel Vaz Gaspar
144b279aa2 feat: saved queries with execution info (#11391)
* feat: add rows and last_run info to saved queries

* feat: add rows to saved query

* refactor and tests

* lint

* fix tests
2020-10-26 11:20:07 -07:00
Jesse Yang
d7eb1d476f perf: speed up uuid column generation (#11209) 2020-10-13 13:29:49 -07:00
Bogdan
cb3f649a7f chore: simplify alerting data model to leverage a single class (#11179)
* Collapse alerting models into a single one

Fixing upgrade migration & tests

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-10-09 19:20:12 -07:00
Beto Dealmeida
9785667a0d feat: add UUID column to ImportMixin (#11098)
* Add UUID column to ImportMixin

* Fix default value

* Fix lint

* Fix order of downgrade

* Add logging when downgrade fails

* Migrate position_json to contain UUIDs, and add schedule tables

* Save UUID when adding charts to dashboard

* Fix heads

* Rename migration file

* Fix dashboard serialization

* Fix migration script with Postgres

* Fix unique contraint name

* Handle UUID when exporting dashboard

* Fix Dataset PUT

* Add UUID JSON serialization

* Fix tests

* Simplify logic

* Try binary=True
2020-10-07 09:00:55 -07:00
John Bodley
e90c63fc7b fix: Alembic migration 18532d70ab98 (#11136) 2020-10-02 09:12:03 +01:00
Daniel Vaz Gaspar
50d80405a9 fix: alembic migration error msg trying to delete constraint on tables (#11115)
* fix: alembic migration fails by deleting non existent constraint on tables

* Revert "fix: alembic migration fails by deleting non existent constraint on tables"

This reverts commit 3a359b0577.

* mantain migration but just for MySQL and add downgrade procedure

* skip the downgrade
2020-10-01 12:35:12 +01:00
Bogdan
94d4d55908 Fix alembic migration (#11110)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-29 16:35:00 -07:00
Bogdan
03eebd3019 fix: fix table existence validation function (#11066)
* Fix table existance validation function

* Drop left over table name index in mysql db

* Do not modify model

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-29 15:01:01 -07:00
Daniel Vaz Gaspar
5d08a426d3 fix(api): unable to delete virtual dataset, wrong permission name (#11019)
* fix(api): unable to delete virtual dataset because of wrong permission name

* Still delete the dataset even when no permission was found

* migration script to fix possible existing faulty permissions on the db

* black

* fix db migration and one more test

* add more comments to the migration script

* freeze a partial schema of the model on the migration step

* fix mig script

* Update superset/migrations/versions/3fbbc6e8d654_fix_data_access_permissions_for_virtual_.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-09-29 12:33:07 +01:00
Ville Brofeldt
448a41a4e7 feat(row-level-security): add base filter type and filter grouping (#10946)
* feat(row-level-security): add filter type and group key

* simplify tests and add custom list widget

* address comments

* use enum value to ensure case sensitive value is used
2020-09-22 08:54:10 +03:00
Jason Davis
c1f2214d7c merge db migrations (#10758)
Co-authored-by: Jason Davis <@dropbox.com>
2020-09-01 17:59:02 -07:00
Jason Davis
b59f6b1833 feat: refractored SQL-based alerting framework (#10605)
* added new tables for alerting refractor

* reformatted inheritance structure

* added workflow for updated framework

* added suggested changes

* cleaned up changes

* added obervations to alert table to enable view

* added comments

* added requested changes

* fix tests

* added styling changes

* mypy

* added requested changes

* updated operator logic

* requested changes, 1 validator, styling changes

* refactored tests

* fix test alert workflow

* fixed create_alert in test

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-01 13:36:02 -07:00
Bogdan
4572ebb600 chore: log cache keys to the logs (#10678)
* Log cache keys to the logs

* Add tests

* Use separate table for the cache keys

* Add migration for the cache lookup table

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-01 09:41:25 -07:00
Grace Guo
ca9ca99510 refactor: [migration] convert iframe chart into dashboard markdown component (#10590)
* refactor: [migration] convert iframe chart into dashboard markdown component

* remove 3 viz_types

* fix comments
2020-08-16 14:43:30 -07:00
Erik Ritter
bd88e12b8b feat: add extra column to tables and sql_metrics (#10592) 2020-08-12 15:27:29 -07:00
Jason Davis
c0bb86d0d6 feat: slack integration for SQL-based alerts (#10566)
* add slack functionality

* deleted unused variable

* updated test

* black

* fix rebase

* added nits

* added slack no screenshot integration

* isort

* added namedtuple for screenshot

* added test

* fix precommit

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-11 11:15:31 -07:00
John Bodley
cda764fd4b chore: Adding pip-compile-multi et al. (#10499)
* chore: Adding pip-compile-multi et al

* Specify requirements.txt path for fossa

* [ci] Fixing CI

Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
2020-08-04 19:31:52 -07:00
John Bodley
821916a681 chore: pre-commit run --all-files (#10500)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-08-02 14:32:17 -07:00
Maxime Beauchemin
318e5347bc feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Will Barrett
df71fac1e0 refactor: Fix lint on superset/utils/core.py (#10120)
* Fix lint on superset/utils/core.py

* black

* mypy

* Fix some missing renames
2020-06-26 08:49:12 -07:00
Bogdan
42a56e29a1 Fix alembic migrations order (#10156)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 11:41:47 -07:00
John Bodley
37777f382f fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Ville Brofeldt
d615949820 chore: Bump sqlalchemy-utils and cachelib and refresh requirements.txt (#10117)
* chore: Bump sqlalchemy-utils and cachelib and refresh requirements.txt

* undeprecate EncryptedType

* black

* reference correct class

Co-authored-by: Maxime Beauchemin <maximebeauchemin@gmail.com>
2020-06-22 16:01:46 +03:00
Aliaksei Kushniarevich
550e78ff7c feat: Add multiple table filters for Row Level Security (#9751)
* Add multiple table filters for Row Level Security

* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)

* Merge DB migrations

* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite

* Support db records migration also

* Support downgrading from the new-fashioned formatted records

* Straighten up migrations

* Update migration's down_revision to comply master branch
2020-06-22 12:51:08 +03:00
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
Erik Ritter
e95af7f783 fix: migrate all slices off the old time grain format (#9684) 2020-04-30 08:38:13 -07:00