Commit Graph

139 Commits

Author SHA1 Message Date
serenajiang
1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Will Barrett
ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Will Barrett
68aca88c54 Re-enable pylint on some configuration files (#8767)
* re-enable pylint for superset/config.py

* re-enable pylint on superset/migrations/env.py

* re-enable pylint for superset/legacy.py

* re-enable pylint for superset/forms.py

* re-enable pylint for superset/stats_logger.py

* Tweaks to make mypy and pylint happy

* black
2019-12-06 14:30:11 -08:00
Beto Dealmeida
5b934bb377 Change column to longtext (#8739)
* Change column to longtext

* Manual merge heads
2019-12-04 11:24:23 -08:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
John Bodley
7bfa24d0da [fix] Re-cleanup legacy filters (#8523) 2019-11-12 12:29:00 -08:00
Will Barrett
0730261342 Fix column type on dbs.encrypted_extra, add instructions for testing migration downgrades (#8493)
* Fix column type on dbs.encrypted_extra

* Add instructions for testing migration downgrades

* Account for non-Postgres DBs in migration

* Use batch_alter_table to make SQLite happy

* Another CI-appeasing tweak
2019-11-04 23:13:21 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Will Barrett
148cec4690 Add UI-only database configuration method for extended authorization scenarios (#8448)
* Add encrypted_extra to dbs

* WIP - UI-based BigQuery connection configuration

* Fix 500 bubbling to the surface when adding a database connection

* Add check for valid json

* black formatting:

* isort

* Incorporate PR feedback. Thanks all!

* black

* Typo fix in CONTRIBUTING.md
2019-10-28 15:49:40 +00:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang
876d329474 [fix] make names non-nullable (#8371) 2019-10-15 16:51:04 -07:00
John Bodley
03b35b3c11 [migration] Migrating since/until to time_range (#8372) 2019-10-11 13:41:23 -07:00
serenajiang
7e7ea3d9a0 Revert "[fix] make datasource names non-nullable (#8332)" (#8363)
This reverts commit 65a05ca47e.
2019-10-09 11:38:57 -07:00
serenajiang
65a05ca47e [fix] make datasource names non-nullable (#8332) 2019-10-04 09:13:25 -07:00
John Bodley
d46510779f [migration] Stripping leading and trailing whitespace (#8261) 2019-09-23 09:13:01 -07:00
Maxime Beauchemin
9d4b955cc7 chore: deprecate restricted metrics (#8197)
* chore: deprecate restricted metrics

An early community contribution added the concept of restricted metrics.

The idea was to allow for some metrics to be restricted, and if a metric
was tagged as such, a user would need to be given access to that metric
more explicitely, through a special perm we would maintain for that
metric.

Now since the new concept of "Adhoc Metrics", the popover that lets a
user pick a column and an aggregate function or to write their own SQL
expression inline, this restriction is completely bypassed. Adhoc
metrics was developed without the restricted metrics in mind.

Anyhow, in the near future, we'll be rethinking the ideas behind
data-access permissions, and things like column-level or row-level
security will be redesigned from scratch.

By deprecating this feature, we're removing a confusing and mostly broken
feature, and making it easy to move forward

* Use context manager to drop columns

* disable jest's maxWorkers
2019-09-09 17:20:32 -07:00
Beto Dealmeida
10f00cdde3 [fix] Isolate and improve performance on tagging system (#7858)
* Fix tag perf

* Add ASF header

* Make script idempotent

* Add CLI to sync tags

* Add missing file

* Merge heads

* Fix lint

* Remove script
2019-07-31 09:19:39 -07:00
Grace Guo
9dd6a385e6 [migration] add unique constraint on dashboard_slices table (#7880)
1. remove duplicated entries in many-to-many relation tbl dashboard_slices
2. add unique constraint on tbl
3. update the model to include the uniqueness constraint
2019-07-22 16:48:41 -07:00
1AB9502
9eccae3b4f Update down_revision to fix migrations (#7850) 2019-07-11 12:32:34 -07:00
Tresdon Jones
97ffb762d0 Add "Published" feature to dashboards (#4725)
* Allow users to publish dashboards

* Rework publish dashboards feature

- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status

* Fix linter errors

* Update javascript tests

* Add tests and change DashboardFilter

Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within

* Fix some linter errors

* Remove commas from core.py

* Fix some failing tests

* More linter errors I introduced

* Fix more linter errors I introduced

* update alembic migration

* Update design of publish dash feature

* Upgrade migration version

* Secure publish endpoint

* Remove bad quotes

* Give publish span its own style

* fix publish rendering

* Add new test for publish feature

* Update migration

* update slug in test

* Update migration

* Address reviwer comments

* Fix linter errors

* Add licenses

* Remove fetchPublished(), use bootstrap data

* Update migration

* Update croniter to existing version

* Fix linter errors

* Upgrade DB Revisions

* Fix flake8 linter error

* Set all dashboards to published on migration

* Migration proper line spacing

* Fix migration to work with postgres

* UPDATE statement works with postgresql and sqlite hopefully

* Update wording to kick off travis
2019-07-10 23:14:13 -07:00
John Bodley
9dac805e7e [schema] Deprecating the table_columns.database_expression column (#7653) 2019-07-03 14:05:59 -07:00
John Bodley
6d7701e684 [fix] Updating Pandas resample logic (#7799) 2019-07-02 11:10:50 -07:00
John Bodley
5c58fd1802 [format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Erik Ritter
f7812a3f27 [SQL Lab] Clarify SQL Lab query and display limits (#7641) 2019-06-07 17:16:38 -07:00
Kim Truong
ca48f32a07 fix: alter sql columns to long text #7463 (#7476)
Merge lyft-release-sp8@7bfe7bc to master
2019-05-08 10:30:43 -07:00
John Bodley
f488a3281d [schema] Updating the base metric schema (#5453) 2019-04-24 21:50:14 -07:00
John Bodley
3a359fd892 [schema] Adding commits and removing unnecessary foreign-key definitions (#7371) 2019-04-24 12:33:33 -07:00
John Bodley
f58e7b204e Update 937d04c16b64_update_datasources.py (#7361) 2019-04-23 18:34:41 -07:00
John Bodley
1388452169 [schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
John Bodley
621cef7048 [schema] Updating the datasources schema (#5451) 2019-04-23 13:22:29 -07:00
John Bodley
cf87173c21 [wtforms] Strip leading/trailing whitespace (#7084) 2019-04-23 12:04:50 -07:00
Maxime Beauchemin
0c3e46f8ae Deprecate npm run backend-sync and related logic (#7211) 2019-04-10 08:46:05 -07:00
Grace Guo
1104aca5bd update migration Revision for #6945 (#7258) 2019-04-09 17:38:12 -07:00
Grace Guo
139f299ab3 add parent ids into Dashboard layout metadata (#6945) 2019-04-09 15:41:57 -07:00
John Bodley
36a6fade90 [migration] Fixing issue with fb13d49b72f9 downgrade (#7145)
* [migration] Fixing issue with fb13d49b72f9 downgrade

This PR fixes an issue with the downgrade step of migration fb13d49b72f9 which wrongfully labeled the field `metrics` rather than `metric`.

to: @graceguo-supercat @michellethomas @mistercrunch

* Update fb13d49b72f9_better_filters.py
2019-03-28 10:07:55 -07:00
John Bodley
6e79e84b02 [migration] Fixing issue with c82ee8a39623 downgrade (#7144)
* [migration] Fixing downgrade

* Trigger notification
2019-03-27 22:23:21 -07:00
Don Bowman
4c5e6ca208 Add HTTP Basic Auth for Druid (Issue #4776) (#6795)
* Add HTTP Basic Auth for Druid (Issue #4776)

This should resolve [Issue #4776](https://github.com/apache/incubator-superset/issues/4776)

Add HTTP Basic Auth to the Druid Broker.

Signed-off-by: Don Bowman <don@agilicus.com>

* Encrypt broker password in database

Signed-off-by: Don Bowman <don@agilicus.com>

* Merge from master, new migration needed

Signed-off-by: Don Bowman <don@agilicus.com>

* Do not expose hashed druid basic auth password

* docs: add description_columns for druid basic auth
2019-03-25 16:20:12 -07:00
Ville Brofeldt
23e823f0c1 Bump SQLAlchemy to 1.3 (#7099)
* Bump sqla to >=1.3.1

* Refine mssql column types to only use N-prefixing when necessary

* make join explicit

* replace set with list

* Add additional test case for N-prefix

* Replace engine with dialect and fix linting error

* Remove unneeded import
2019-03-24 20:44:05 -07:00
Maxime Beauchemin
b210742ad2 Fix filter_box migration PR #6523 (#7066)
* Fix filter_box migration PR #6523

* Fix druid-related bug
2019-03-20 21:41:33 -07:00
Boris Hajduk
5137539a86 [fix] Use HTTPS, not HTTP wherever practical (#7040)
* Download RAT binary via HTTPS, not HTTP

* Merge branch 'patch-1' of github.com:hajdbo/incubator-superset into patch-1
2019-03-17 23:21:32 -07:00
Christine Chambers
e47a1b2868 Split tags migration (#7002)
This PR removes the iteration over charts, dashboards and saved queries to create tags in the original migration, leaving only the logic to create the tags and the tagged objects tables.
Tested locally by running `superset db downgrade` to revert to the previous migration and then running `superset db upgrade` to the current version.
2019-03-08 16:27:13 -08:00
Beto Dealmeida
8041b63af6 Backend only tagging system (#6823)
This PR introduces the backend changes for a tagging system for Superset, allowing dashboards, charts and queries to be tagged. It also allows searching for a given tag, and will be the basis for a new landing page (see #5327).

# Implicit tags
Dashboard, chart and (saved) queries have implicit tags related to their owners, types and favorites. For example, all objects owned by the admin have the tag `owner:1`. All charts have the tag `type:chart`. Objects favorited by the admin have the tag `favorited_by:1`.

These tags are automatically added by a migration script, and kept in sync through SQLAlchemy event listeners. They are currently not surfaced to the user, but can be searched for. For example, it's possible to search for `owner:1` in the welcome page to see all objects owned by the admin, or even search for `owner:{{ current_user_id() }}`.
2019-02-06 13:42:42 -08:00
John Bodley
e1b907783a [wtforms] Using wtforms-json which supports None (#5445) 2019-02-04 09:35:40 -08:00
Maxime Beauchemin
ba64ae6974 Add missing license (#6801) 2019-02-01 14:42:37 -08:00
John Bodley
2631558ac4 Revert "Removing uniqueness constraints on tables table" (#6777)
* Revert "creating new circular-json safe stringify and replacing one call (#6772)"

This reverts commit 11a7ad00b7.

* Revert "Improve Unicode support for MSSQL (#6690)"

This reverts commit c44ae612df.

* Revert "Fix uniqueness constraints on tables table (#6718)"

This reverts commit c4fb7a0a87.
2019-01-31 09:57:51 -08:00
agrawaldevesh
c4fb7a0a87 Fix uniqueness constraints on tables table (#6718)
Summary: Superset code enforces (in Table crud view pre_add) that the
table is unique within <database, schema, table_name). Indeed in commit
15b67b2c6c (in 2016), the model was
updated to reflect that. However, it was never ported over to a
migration.

I am fixing that in this diff. I am choosing to make this be a new
migration instead of fixing an existing one since I want to fix existing
installations also cleanly.

I also considered removing the uniqueness constraint, but that won't
work: First because anyway there are other places where the <database,
schema, table> uniqueness is enforced in code. But also, the .sql field
isn't a first citizen yet: The schema of the table is picked up from the
table-name and the sql part is only used when creating the explore
query. So indeed we want this uniqueness constraint. (Also it breaks the
unit tests in dict_import_export_tests.py)
[Perhaps it can be removed when we have true .sql support, but for now
the user would have to create a database view and he can use that as the
'table name'. That way he gets schema inference also]

Also added INFO logging to the alembic migration.
2019-01-28 22:49:31 -08:00
bolkedebruin
1fece0d2fa Add licenses to translations (#6732)
* Add licenses

* More licenses

* Ignore messages.json as they are generated

* More licenses

* Also typescript

* Fix alignment

* Add to svg

* Many more licenses

* more licenses

* Better excludes

* Add licenses to docs and md files

* Pre-finalize licenses

* Fix newlines

* Finalize all sourde licenses

* Fix lint
2019-01-22 08:21:13 -08:00
Beto Dealmeida
ef79757973 Merge heads (#6729)
* Merge heads

* Add ASF header
2019-01-18 16:04:06 -08:00