Commit Graph

149 Commits

Author SHA1 Message Date
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
Erik Ritter
e95af7f783 fix: migrate all slices off the old time grain format (#9684) 2020-04-30 08:38:13 -07:00
Lily Kuang
b27a81e0da [logging] deprecation notices for SQLite (#9662)
* [logging] deprecation notices for SQLite

* [logging] use warning for SQLite notice

* [logging] update grammar
2020-04-28 14:28:09 -07:00
John Bodley
636ee5d19a [mypy] Enforcing typing for superset.migrations (#9583)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-18 23:24:32 -07:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
John Bodley
e63cb52630 [fix] Fixing cache key inconsistencies (#9382)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-03-28 12:33:16 -07:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Grace Guo
f4ad15e644 [migration] metadata for dashboard filters (#9109)
* [migration] metadata for dashboard filters

* fix comments

* fix extra code review comments
2020-02-14 13:26:04 -08:00
serenajiang
1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Will Barrett
68aca88c54 Re-enable pylint on some configuration files (#8767)
* re-enable pylint for superset/config.py

* re-enable pylint on superset/migrations/env.py

* re-enable pylint for superset/legacy.py

* re-enable pylint for superset/forms.py

* re-enable pylint for superset/stats_logger.py

* Tweaks to make mypy and pylint happy

* black
2019-12-06 14:30:11 -08:00
Beto Dealmeida
5b934bb377 Change column to longtext (#8739)
* Change column to longtext

* Manual merge heads
2019-12-04 11:24:23 -08:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
John Bodley
7bfa24d0da [fix] Re-cleanup legacy filters (#8523) 2019-11-12 12:29:00 -08:00
Will Barrett
0730261342 Fix column type on dbs.encrypted_extra, add instructions for testing migration downgrades (#8493)
* Fix column type on dbs.encrypted_extra

* Add instructions for testing migration downgrades

* Account for non-Postgres DBs in migration

* Use batch_alter_table to make SQLite happy

* Another CI-appeasing tweak
2019-11-04 23:13:21 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Will Barrett
148cec4690 Add UI-only database configuration method for extended authorization scenarios (#8448)
* Add encrypted_extra to dbs

* WIP - UI-based BigQuery connection configuration

* Fix 500 bubbling to the surface when adding a database connection

* Add check for valid json

* black formatting:

* isort

* Incorporate PR feedback. Thanks all!

* black

* Typo fix in CONTRIBUTING.md
2019-10-28 15:49:40 +00:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
876d329474 [fix] make names non-nullable (#8371) 2019-10-15 16:51:04 -07:00
John Bodley
03b35b3c11 [migration] Migrating since/until to time_range (#8372) 2019-10-11 13:41:23 -07:00
serenajiang
7e7ea3d9a0 Revert "[fix] make datasource names non-nullable (#8332)" (#8363)
This reverts commit 65a05ca47e.
2019-10-09 11:38:57 -07:00
serenajiang
65a05ca47e [fix] make datasource names non-nullable (#8332) 2019-10-04 09:13:25 -07:00
John Bodley
d46510779f [migration] Stripping leading and trailing whitespace (#8261) 2019-09-23 09:13:01 -07:00
Maxime Beauchemin
9d4b955cc7 chore: deprecate restricted metrics (#8197)
* chore: deprecate restricted metrics

An early community contribution added the concept of restricted metrics.

The idea was to allow for some metrics to be restricted, and if a metric
was tagged as such, a user would need to be given access to that metric
more explicitely, through a special perm we would maintain for that
metric.

Now since the new concept of "Adhoc Metrics", the popover that lets a
user pick a column and an aggregate function or to write their own SQL
expression inline, this restriction is completely bypassed. Adhoc
metrics was developed without the restricted metrics in mind.

Anyhow, in the near future, we'll be rethinking the ideas behind
data-access permissions, and things like column-level or row-level
security will be redesigned from scratch.

By deprecating this feature, we're removing a confusing and mostly broken
feature, and making it easy to move forward

* Use context manager to drop columns

* disable jest's maxWorkers
2019-09-09 17:20:32 -07:00
Beto Dealmeida
10f00cdde3 [fix] Isolate and improve performance on tagging system (#7858)
* Fix tag perf

* Add ASF header

* Make script idempotent

* Add CLI to sync tags

* Add missing file

* Merge heads

* Fix lint

* Remove script
2019-07-31 09:19:39 -07:00
Grace Guo
9dd6a385e6 [migration] add unique constraint on dashboard_slices table (#7880)
1. remove duplicated entries in many-to-many relation tbl dashboard_slices
2. add unique constraint on tbl
3. update the model to include the uniqueness constraint
2019-07-22 16:48:41 -07:00
1AB9502
9eccae3b4f Update down_revision to fix migrations (#7850) 2019-07-11 12:32:34 -07:00
Tresdon Jones
97ffb762d0 Add "Published" feature to dashboards (#4725)
* Allow users to publish dashboards

* Rework publish dashboards feature

- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status

* Fix linter errors

* Update javascript tests

* Add tests and change DashboardFilter

Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within

* Fix some linter errors

* Remove commas from core.py

* Fix some failing tests

* More linter errors I introduced

* Fix more linter errors I introduced

* update alembic migration

* Update design of publish dash feature

* Upgrade migration version

* Secure publish endpoint

* Remove bad quotes

* Give publish span its own style

* fix publish rendering

* Add new test for publish feature

* Update migration

* update slug in test

* Update migration

* Address reviwer comments

* Fix linter errors

* Add licenses

* Remove fetchPublished(), use bootstrap data

* Update migration

* Update croniter to existing version

* Fix linter errors

* Upgrade DB Revisions

* Fix flake8 linter error

* Set all dashboards to published on migration

* Migration proper line spacing

* Fix migration to work with postgres

* UPDATE statement works with postgresql and sqlite hopefully

* Update wording to kick off travis
2019-07-10 23:14:13 -07:00
John Bodley
9dac805e7e [schema] Deprecating the table_columns.database_expression column (#7653) 2019-07-03 14:05:59 -07:00
John Bodley
6d7701e684 [fix] Updating Pandas resample logic (#7799) 2019-07-02 11:10:50 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Erik Ritter
f7812a3f27 [SQL Lab] Clarify SQL Lab query and display limits (#7641) 2019-06-07 17:16:38 -07:00
Kim Truong
ca48f32a07 fix: alter sql columns to long text #7463 (#7476)
Merge lyft-release-sp8@7bfe7bc to master
2019-05-08 10:30:43 -07:00
John Bodley
f488a3281d [schema] Updating the base metric schema (#5453) 2019-04-24 21:50:14 -07:00
John Bodley
3a359fd892 [schema] Adding commits and removing unnecessary foreign-key definitions (#7371) 2019-04-24 12:33:33 -07:00
John Bodley
f58e7b204e Update 937d04c16b64_update_datasources.py (#7361) 2019-04-23 18:34:41 -07:00
John Bodley
1388452169 [schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
John Bodley
621cef7048 [schema] Updating the datasources schema (#5451) 2019-04-23 13:22:29 -07:00
John Bodley
cf87173c21 [wtforms] Strip leading/trailing whitespace (#7084) 2019-04-23 12:04:50 -07:00
Maxime Beauchemin
0c3e46f8ae Deprecate npm run backend-sync and related logic (#7211) 2019-04-10 08:46:05 -07:00
Grace Guo
1104aca5bd update migration Revision for #6945 (#7258) 2019-04-09 17:38:12 -07:00
Grace Guo
139f299ab3 add parent ids into Dashboard layout metadata (#6945) 2019-04-09 15:41:57 -07:00
John Bodley
36a6fade90 [migration] Fixing issue with fb13d49b72f9 downgrade (#7145)
* [migration] Fixing issue with fb13d49b72f9 downgrade

This PR fixes an issue with the downgrade step of migration fb13d49b72f9 which wrongfully labeled the field `metrics` rather than `metric`.

to: @graceguo-supercat @michellethomas @mistercrunch

* Update fb13d49b72f9_better_filters.py
2019-03-28 10:07:55 -07:00
John Bodley
6e79e84b02 [migration] Fixing issue with c82ee8a39623 downgrade (#7144)
* [migration] Fixing downgrade

* Trigger notification
2019-03-27 22:23:21 -07:00
Don Bowman
4c5e6ca208 Add HTTP Basic Auth for Druid (Issue #4776) (#6795)
* Add HTTP Basic Auth for Druid (Issue #4776)

This should resolve [Issue #4776](https://github.com/apache/incubator-superset/issues/4776)

Add HTTP Basic Auth to the Druid Broker.

Signed-off-by: Don Bowman <don@agilicus.com>

* Encrypt broker password in database

Signed-off-by: Don Bowman <don@agilicus.com>

* Merge from master, new migration needed

Signed-off-by: Don Bowman <don@agilicus.com>

* Do not expose hashed druid basic auth password

* docs: add description_columns for druid basic auth
2019-03-25 16:20:12 -07:00
Ville Brofeldt
23e823f0c1 Bump SQLAlchemy to 1.3 (#7099)
* Bump sqla to >=1.3.1

* Refine mssql column types to only use N-prefixing when necessary

* make join explicit

* replace set with list

* Add additional test case for N-prefix

* Replace engine with dialect and fix linting error

* Remove unneeded import
2019-03-24 20:44:05 -07:00