Commit Graph

459 Commits

Author SHA1 Message Date
ʈᵃᵢ
8764ae3852 [sqllab] fix exception caused by casting string to int with psycopg2 (#9287)
* [sqllab] fix exception caused by casting string to int with psycopg2

* rollback session on exception

* add SQLLAB_BACKEND_PERSISTENCE to default feature flags

* cast tab_state_id to str in python instead of sql

* remove import
2020-03-16 10:51:41 -07:00
Erik Ritter
f80fadff0e Reduce dashboard bootstrap payload (#9284) 2020-03-13 12:18:22 -07:00
Ville Brofeldt
3682702e91 fix: add connection testing params for snowflake (#9272)
* fix: add connection testingt params for snowflake

* Linting
2020-03-11 06:51:57 +02:00
Grace Guo
724b8a3c31 [fix] copy filter_scopes with duplicate charts (#9271) 2020-03-10 10:23:15 -07:00
Maxime Beauchemin
116200cf73 fix: change database save in DatasourceEditor (#9255)
* fix: change database save in DatasourceEditor

This addresses the issue where pointing a datasource to another database
in the datasource editor is not reflected.

Also addresses:
- a minorcosmetic issue in the datasource editor.
- user/owners list not getting populated

* tests
2020-03-10 09:20:37 -07:00
Ville Brofeldt
7a91498cf1 fix: share column type matching between model and result set (#9161)
* Share column type matching between model and result set

* Address comments
2020-03-04 16:33:22 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
David Aaron Suddjian
e01f24f833 Prevent database connections to sqlite (#9218)
* prevent database connections to sqlite

* tweaks and tests

* add entry to UPDATING.md
2020-03-02 13:13:11 -08:00
Grace Guo
ccd6e44edf refactor copy filter_scopes and add tests (#9224)
* refactor copy filter_scopes and add tests

* fix review comments
2020-03-02 12:55:30 -08:00
Grace Guo
ca2bc8b15f [fix] remove chart id from filter_scopes metadata if chart is not in dash anymore (#9213)
* [fix] remove chart id from filter_scopes metadata if chart is not in dashboard anymore

* fix review comments, and add check for overwrite dash function
2020-02-26 17:33:01 -08:00
Grace Guo
421aeb4605 [dashboard] fix filter_scopes when copy dashboard with duplicate_slices (#9188)
* [dashboard] fix filter_scopes when copy dashboard with duplicate_slices

* code cleanup
2020-02-24 13:45:29 -08:00
Daniel Vaz Gaspar
27b371a485 [core] Fix, sanitize errors returned from testconn (#9178) 2020-02-24 10:40:51 +00:00
John Bodley
141570636e [fix] SQL query source (#9173) 2020-02-20 11:15:12 -08:00
Daniel Vaz Gaspar
e55fe43ca6 [database] Fix, tables API endpoint (#9144) 2020-02-20 10:15:22 +00:00
Grace Guo
c1750af54a [dashboard] clean up usage for old filter immune metadata (#9146)
* [dashboard] clean up usage for filter_immune metadata

* add fix per comment
2020-02-19 09:58:34 -08:00
John Bodley
a7e433a512 [mypy] Disallowing implicit optional (#9150) 2020-02-16 22:34:15 -08:00
Daniel Vaz Gaspar
f993bdc7ef [database] new, select star API migration (#9054) 2020-02-14 14:30:49 +00:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Daniel Vaz Gaspar
ea399047d9 [table] [columns] remove generic checkbox API (#9044) 2020-02-06 20:01:24 +00:00
Daniel Vaz Gaspar
916d184076 [query] deprecate can_only_access_owned_queries (#9046) 2020-02-05 08:58:11 +00:00
Will Barrett
fc1c9428e3 Do not show stacktraces on some intentionally-thrown errors (#9056)
* Do not show stacktraces on some intentionally-thrown errors

* Fix pylint error

* Fix JavaScript looking for the wrong key in error response from connection check

* format -> f-string

* Run black because PyCharm missed it on the last save
2020-02-05 07:01:43 +02:00
Will Barrett
fd9a713c7e Remove endpoints allowing arbitrary cache access (#9049) 2020-01-29 15:41:57 -08:00
Daniel Vaz Gaspar
d5dbfc886d [database] new, API table metadata (#9002)
Migrates and completely removes: '/superset/table/<database_id>/<table_name>/<schema>/'
- Guarantees database filtering based on the user permissions
- Full OpenAPI spec
- Better error handling
2020-01-29 09:51:06 +00:00
Will Barrett
a4ed0758b0 Remove redirect endpoint /superset/explorev2 (#9038) 2020-01-29 00:33:08 -05:00
Maxime Beauchemin
315a11dfe2 fix: shut off unneeded endpoints (#8960)
* fix: shut off all uneeded endpoints

We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.

First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.

Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.

Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
  been using endpoints in their environment for other purposes may
  experience loss of functionality

* Tweaking

* Reduce the amount of endpoints using white lists

* Fix, included needed endpoints for dashboard and druid

* Drying things up

* fixes

* limiting more endpoints

* Read only on some FAB model views

* fixing some tests

* fixes

* Fixing more tests

* Addressing comments

* Drying up route_methods

* further drying

Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
2020-01-23 11:25:15 -05:00
Daniel Vaz Gaspar
74158694c5 [charts] New, REST API (#8917)
* [charts] New REST API

* [charts] Small improvements

* [charts] Fix, lint

* [charts] Tests and datasource validation

* [charts] Fix, lint

* [charts] DRY post schemas

* [charts] lint and improve type declarations

* [charts] DRY owned REST APIs

* [charts] Small fixes

* [charts] More tests

* [charts] Tests and DRY

* [charts] Tests for update

* [charts] More tests

* [charts] Fix, isort

* [charts] DRY and improve quality

* [charts] DRY and more tests

* [charts] Refactor base for api and schemas

* [charts] Fix bug on partial updates for dashboards

* [charts] Fix missing apache license

* black app.py after merge

* [charts] Fix, missing imports and black

* [api] Log on sqlalchemy error

* [api] isort
2020-01-21 13:04:52 -05:00
Daniel Vaz Gaspar
5b2499984d [dashboard] Deprecate superset published API (#8914) 2020-01-10 08:48:17 +00:00
Craig Rueda
2a94150097 Moving appbuilder.xxx out of view files and into app.py (#8912)
* Moving appbuilder.xxx out of view files ands into app.py

* Pulled url map converters out into their own file

* Adding license blurb

* Linting

* Linting again...
2020-01-08 18:04:05 +00:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
David Aaron Suddjian
3a468a53d9 fix a broken model reference (#8866) 2019-12-20 10:46:25 +00:00
Evan Rusackas
b6639504b6 Removing some dividers to make header dropdown menus more consistent (#8857)
* separators removed, "childs" changed to "children"

* changing `children` back to `childs`... it's baked in somewhere in Fab

* linting

* reverting changes to MenuObject

* adding separator above Druid menu items
2019-12-19 10:17:43 -08:00
David Aaron Suddjian
016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
David Aaron Suddjian
7a68cb7ca0 factor out datasource_access_request model (#8809) 2019-12-17 16:17:49 -08:00
Daniel Vaz Gaspar
408196f304 [superset] Core SQLAlchemy query refactor (#8280)
* [superset] Slight performance improvement, remove SQLA first

* [superset] Slight performance improvement, remove SQLA first

* [superset] black it

* [superset] Revert warm up cache change

* [superset] Fix warm up cache slice_id may be none

* [superset] Reverted changes of methods one by get

* [superset] Reverted changes of methods one by get
2019-12-16 23:50:55 -08:00
Kim Truong
930353286f fix: add default value to log param in sql_json (#8851)
* fix: add default params to log

* fix: add optional typing

* fix: linting
2019-12-16 23:27:06 -08:00
Kim Truong
2d42272e60 Add user agent logs (#8826)
* feat: add user agent logging

* fix: lint

* fix: address feedback

* fix: formatting
2019-12-16 14:28:00 -08:00
Daniel Vaz Gaspar
2f2ac00a09 [dashboard] feat: REST API (#8694) 2019-12-16 21:10:33 +00:00
Kim Truong
6c130b32ad Update sql validator status (#8799)
* fix: update status code for sql validator error

* fix: inspect error

* fix: lint issues

* fix: lint issues 2
2019-12-10 11:55:36 -08:00
Beto Dealmeida
5477f42740 Limit payload returned when using SQLLAB_BACKEND_PERSISTENCE (#8769)
* Limit queries

* WIP

* Adding unit test

* WIP

* Add unit test

* Fix test

* Try to fix remote unit tests

* Fix unit test

* Fix  for postgres
2019-12-09 16:12:40 -08:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Dave Smith
aafbfd3b4e slightly decouple sql_json, queries, and results http endpoints from … (#8626)
* slightly decouple sql_json, queries, and results http endpoints from biz logic

* fix syntax errors

* add some type annotations, fix a bug

* remove unnecessary var decl and assign

* add a lot more type annotations to fix tests

* fix mypy issues
2019-11-22 10:12:48 -08:00
Erik Ritter
9303672121 [SQL Lab] Wrap more logic with feature flag (#8629) 2019-11-21 17:08:11 -08:00
Ville Brofeldt
7104b04817 Pass url parameters from dashboard to charts (#8536)
* Pass url_params from dashboard to charts

* Update params to form_data instead of overwriting

* Add cypress tests

* Add python test

* Add docs

* Move reserved url params to utils

* Bump cypress
2019-11-21 07:07:08 +02:00
Craig Rueda
e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
ʈᵃᵢ
9d36fa348f Move from MVC FAB menu to Superset React menu (#8289)
* adds react based menu for welcome view

* bump fab

* better logic in common_bootstrap_payload

* split up files

* fix spacing and module scope styles

* lint

* render correct template for welcome
2019-11-19 22:56:27 -08:00
Will Barrett
d70e0fc359 Fix for BigQuery connection checks and CSV uploads (#8511)
* Fix for BigQuery connection checks and CSV uploads

* Don't assume encrypted_extra will be populated

* Fix undefined method error

* Refactor to avoid circular import strangeness
2019-11-19 14:50:47 -08:00
Beto Dealmeida
3b97ae3b9d Set owner when creating SQL Lab view (#8597)
* Set owner when creating SQL Lab view

* Add unit test

* Small fix
2019-11-19 10:48:42 -08:00
Grace Guo
ff6ab10893 [Feature] Dashboard scoped filter (#8590)
* [WIP][dashboard scoped filter] part 1: scope selector modal (#8557)

* filter scope selector modal

* add single-field-edit in multi-edit mode switch

* fix code review comments (round 1)

* refactory after design review

* fix a few props initial value

* [WIP][dashboard scoped filter] part 2: add algorithm to convert checked ids to scope object (#8564)

* convert ids to scope object

* use lodash helpers to make code readable

* [WIP][dashboard scoped filter] part 3: merge filter scope settings into dashboard redux state (#8522)

* merge filter scope settings into dashboard redux state

* fix/add unit tests

* minor bug fixes

* fix save filter Scopes behavior

* resolve review comments

* fix save filter scope settings

* minor comments

* [dashboard scoped filter] Improve scrollbar inside modal (#8553)

* improve scroll inside modal

* make left pane and right pane scroll separately

* fix review comments

* force show filter_box as unchecked (#8587)
2019-11-18 13:02:25 -08:00
Beto Dealmeida
78f41b38e1 Fix more places where feature flags are serialized (#8567) 2019-11-14 10:01:57 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00