Commit Graph

155 Commits

Author SHA1 Message Date
Will Barrett
df71fac1e0 refactor: Fix lint on superset/utils/core.py (#10120)
* Fix lint on superset/utils/core.py

* black

* mypy

* Fix some missing renames
2020-06-26 08:49:12 -07:00
Bogdan
42a56e29a1 Fix alembic migrations order (#10156)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 11:41:47 -07:00
John Bodley
37777f382f fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Ville Brofeldt
d615949820 chore: Bump sqlalchemy-utils and cachelib and refresh requirements.txt (#10117)
* chore: Bump sqlalchemy-utils and cachelib and refresh requirements.txt

* undeprecate EncryptedType

* black

* reference correct class

Co-authored-by: Maxime Beauchemin <maximebeauchemin@gmail.com>
2020-06-22 16:01:46 +03:00
Aliaksei Kushniarevich
550e78ff7c feat: Add multiple table filters for Row Level Security (#9751)
* Add multiple table filters for Row Level Security

* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)

* Merge DB migrations

* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite

* Support db records migration also

* Support downgrading from the new-fashioned formatted records

* Straighten up migrations

* Update migration's down_revision to comply master branch
2020-06-22 12:51:08 +03:00
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
Erik Ritter
e95af7f783 fix: migrate all slices off the old time grain format (#9684) 2020-04-30 08:38:13 -07:00
Lily Kuang
b27a81e0da [logging] deprecation notices for SQLite (#9662)
* [logging] deprecation notices for SQLite

* [logging] use warning for SQLite notice

* [logging] update grammar
2020-04-28 14:28:09 -07:00
John Bodley
636ee5d19a [mypy] Enforcing typing for superset.migrations (#9583)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-18 23:24:32 -07:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
John Bodley
e63cb52630 [fix] Fixing cache key inconsistencies (#9382)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-03-28 12:33:16 -07:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Grace Guo
f4ad15e644 [migration] metadata for dashboard filters (#9109)
* [migration] metadata for dashboard filters

* fix comments

* fix extra code review comments
2020-02-14 13:26:04 -08:00
serenajiang
1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Will Barrett
68aca88c54 Re-enable pylint on some configuration files (#8767)
* re-enable pylint for superset/config.py

* re-enable pylint on superset/migrations/env.py

* re-enable pylint for superset/legacy.py

* re-enable pylint for superset/forms.py

* re-enable pylint for superset/stats_logger.py

* Tweaks to make mypy and pylint happy

* black
2019-12-06 14:30:11 -08:00
Beto Dealmeida
5b934bb377 Change column to longtext (#8739)
* Change column to longtext

* Manual merge heads
2019-12-04 11:24:23 -08:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
John Bodley
7bfa24d0da [fix] Re-cleanup legacy filters (#8523) 2019-11-12 12:29:00 -08:00
Will Barrett
0730261342 Fix column type on dbs.encrypted_extra, add instructions for testing migration downgrades (#8493)
* Fix column type on dbs.encrypted_extra

* Add instructions for testing migration downgrades

* Account for non-Postgres DBs in migration

* Use batch_alter_table to make SQLite happy

* Another CI-appeasing tweak
2019-11-04 23:13:21 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Will Barrett
148cec4690 Add UI-only database configuration method for extended authorization scenarios (#8448)
* Add encrypted_extra to dbs

* WIP - UI-based BigQuery connection configuration

* Fix 500 bubbling to the surface when adding a database connection

* Add check for valid json

* black formatting:

* isort

* Incorporate PR feedback. Thanks all!

* black

* Typo fix in CONTRIBUTING.md
2019-10-28 15:49:40 +00:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
876d329474 [fix] make names non-nullable (#8371) 2019-10-15 16:51:04 -07:00
John Bodley
03b35b3c11 [migration] Migrating since/until to time_range (#8372) 2019-10-11 13:41:23 -07:00
serenajiang
7e7ea3d9a0 Revert "[fix] make datasource names non-nullable (#8332)" (#8363)
This reverts commit 65a05ca47e.
2019-10-09 11:38:57 -07:00
serenajiang
65a05ca47e [fix] make datasource names non-nullable (#8332) 2019-10-04 09:13:25 -07:00
John Bodley
d46510779f [migration] Stripping leading and trailing whitespace (#8261) 2019-09-23 09:13:01 -07:00
Maxime Beauchemin
9d4b955cc7 chore: deprecate restricted metrics (#8197)
* chore: deprecate restricted metrics

An early community contribution added the concept of restricted metrics.

The idea was to allow for some metrics to be restricted, and if a metric
was tagged as such, a user would need to be given access to that metric
more explicitely, through a special perm we would maintain for that
metric.

Now since the new concept of "Adhoc Metrics", the popover that lets a
user pick a column and an aggregate function or to write their own SQL
expression inline, this restriction is completely bypassed. Adhoc
metrics was developed without the restricted metrics in mind.

Anyhow, in the near future, we'll be rethinking the ideas behind
data-access permissions, and things like column-level or row-level
security will be redesigned from scratch.

By deprecating this feature, we're removing a confusing and mostly broken
feature, and making it easy to move forward

* Use context manager to drop columns

* disable jest's maxWorkers
2019-09-09 17:20:32 -07:00
Beto Dealmeida
10f00cdde3 [fix] Isolate and improve performance on tagging system (#7858)
* Fix tag perf

* Add ASF header

* Make script idempotent

* Add CLI to sync tags

* Add missing file

* Merge heads

* Fix lint

* Remove script
2019-07-31 09:19:39 -07:00
Grace Guo
9dd6a385e6 [migration] add unique constraint on dashboard_slices table (#7880)
1. remove duplicated entries in many-to-many relation tbl dashboard_slices
2. add unique constraint on tbl
3. update the model to include the uniqueness constraint
2019-07-22 16:48:41 -07:00
1AB9502
9eccae3b4f Update down_revision to fix migrations (#7850) 2019-07-11 12:32:34 -07:00
Tresdon Jones
97ffb762d0 Add "Published" feature to dashboards (#4725)
* Allow users to publish dashboards

* Rework publish dashboards feature

- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status

* Fix linter errors

* Update javascript tests

* Add tests and change DashboardFilter

Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within

* Fix some linter errors

* Remove commas from core.py

* Fix some failing tests

* More linter errors I introduced

* Fix more linter errors I introduced

* update alembic migration

* Update design of publish dash feature

* Upgrade migration version

* Secure publish endpoint

* Remove bad quotes

* Give publish span its own style

* fix publish rendering

* Add new test for publish feature

* Update migration

* update slug in test

* Update migration

* Address reviwer comments

* Fix linter errors

* Add licenses

* Remove fetchPublished(), use bootstrap data

* Update migration

* Update croniter to existing version

* Fix linter errors

* Upgrade DB Revisions

* Fix flake8 linter error

* Set all dashboards to published on migration

* Migration proper line spacing

* Fix migration to work with postgres

* UPDATE statement works with postgresql and sqlite hopefully

* Update wording to kick off travis
2019-07-10 23:14:13 -07:00
John Bodley
9dac805e7e [schema] Deprecating the table_columns.database_expression column (#7653) 2019-07-03 14:05:59 -07:00
John Bodley
6d7701e684 [fix] Updating Pandas resample logic (#7799) 2019-07-02 11:10:50 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Erik Ritter
f7812a3f27 [SQL Lab] Clarify SQL Lab query and display limits (#7641) 2019-06-07 17:16:38 -07:00
Kim Truong
ca48f32a07 fix: alter sql columns to long text #7463 (#7476)
Merge lyft-release-sp8@7bfe7bc to master
2019-05-08 10:30:43 -07:00
John Bodley
f488a3281d [schema] Updating the base metric schema (#5453) 2019-04-24 21:50:14 -07:00
John Bodley
3a359fd892 [schema] Adding commits and removing unnecessary foreign-key definitions (#7371) 2019-04-24 12:33:33 -07:00
John Bodley
f58e7b204e Update 937d04c16b64_update_datasources.py (#7361) 2019-04-23 18:34:41 -07:00
John Bodley
1388452169 [schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
John Bodley
621cef7048 [schema] Updating the datasources schema (#5451) 2019-04-23 13:22:29 -07:00
John Bodley
cf87173c21 [wtforms] Strip leading/trailing whitespace (#7084) 2019-04-23 12:04:50 -07:00
Maxime Beauchemin
0c3e46f8ae Deprecate npm run backend-sync and related logic (#7211) 2019-04-10 08:46:05 -07:00