Commit Graph

383 Commits

Author SHA1 Message Date
Daniel Vaz Gaspar
5b2499984d [dashboard] Deprecate superset published API (#8914) 2020-01-10 08:48:17 +00:00
Craig Rueda
2a94150097 Moving appbuilder.xxx out of view files and into app.py (#8912)
* Moving appbuilder.xxx out of view files ands into app.py

* Pulled url map converters out into their own file

* Adding license blurb

* Linting

* Linting again...
2020-01-08 18:04:05 +00:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
David Aaron Suddjian
3a468a53d9 fix a broken model reference (#8866) 2019-12-20 10:46:25 +00:00
Evan Rusackas
b6639504b6 Removing some dividers to make header dropdown menus more consistent (#8857)
* separators removed, "childs" changed to "children"

* changing `children` back to `childs`... it's baked in somewhere in Fab

* linting

* reverting changes to MenuObject

* adding separator above Druid menu items
2019-12-19 10:17:43 -08:00
David Aaron Suddjian
016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
David Aaron Suddjian
7a68cb7ca0 factor out datasource_access_request model (#8809) 2019-12-17 16:17:49 -08:00
Daniel Vaz Gaspar
408196f304 [superset] Core SQLAlchemy query refactor (#8280)
* [superset] Slight performance improvement, remove SQLA first

* [superset] Slight performance improvement, remove SQLA first

* [superset] black it

* [superset] Revert warm up cache change

* [superset] Fix warm up cache slice_id may be none

* [superset] Reverted changes of methods one by get

* [superset] Reverted changes of methods one by get
2019-12-16 23:50:55 -08:00
Kim Truong
930353286f fix: add default value to log param in sql_json (#8851)
* fix: add default params to log

* fix: add optional typing

* fix: linting
2019-12-16 23:27:06 -08:00
Kim Truong
2d42272e60 Add user agent logs (#8826)
* feat: add user agent logging

* fix: lint

* fix: address feedback

* fix: formatting
2019-12-16 14:28:00 -08:00
Daniel Vaz Gaspar
2f2ac00a09 [dashboard] feat: REST API (#8694) 2019-12-16 21:10:33 +00:00
Kim Truong
6c130b32ad Update sql validator status (#8799)
* fix: update status code for sql validator error

* fix: inspect error

* fix: lint issues

* fix: lint issues 2
2019-12-10 11:55:36 -08:00
Beto Dealmeida
5477f42740 Limit payload returned when using SQLLAB_BACKEND_PERSISTENCE (#8769)
* Limit queries

* WIP

* Adding unit test

* WIP

* Add unit test

* Fix test

* Try to fix remote unit tests

* Fix unit test

* Fix  for postgres
2019-12-09 16:12:40 -08:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Dave Smith
aafbfd3b4e slightly decouple sql_json, queries, and results http endpoints from … (#8626)
* slightly decouple sql_json, queries, and results http endpoints from biz logic

* fix syntax errors

* add some type annotations, fix a bug

* remove unnecessary var decl and assign

* add a lot more type annotations to fix tests

* fix mypy issues
2019-11-22 10:12:48 -08:00
Erik Ritter
9303672121 [SQL Lab] Wrap more logic with feature flag (#8629) 2019-11-21 17:08:11 -08:00
Ville Brofeldt
7104b04817 Pass url parameters from dashboard to charts (#8536)
* Pass url_params from dashboard to charts

* Update params to form_data instead of overwriting

* Add cypress tests

* Add python test

* Add docs

* Move reserved url params to utils

* Bump cypress
2019-11-21 07:07:08 +02:00
Craig Rueda
e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
ʈᵃᵢ
9d36fa348f Move from MVC FAB menu to Superset React menu (#8289)
* adds react based menu for welcome view

* bump fab

* better logic in common_bootstrap_payload

* split up files

* fix spacing and module scope styles

* lint

* render correct template for welcome
2019-11-19 22:56:27 -08:00
Will Barrett
d70e0fc359 Fix for BigQuery connection checks and CSV uploads (#8511)
* Fix for BigQuery connection checks and CSV uploads

* Don't assume encrypted_extra will be populated

* Fix undefined method error

* Refactor to avoid circular import strangeness
2019-11-19 14:50:47 -08:00
Beto Dealmeida
3b97ae3b9d Set owner when creating SQL Lab view (#8597)
* Set owner when creating SQL Lab view

* Add unit test

* Small fix
2019-11-19 10:48:42 -08:00
Grace Guo
ff6ab10893 [Feature] Dashboard scoped filter (#8590)
* [WIP][dashboard scoped filter] part 1: scope selector modal (#8557)

* filter scope selector modal

* add single-field-edit in multi-edit mode switch

* fix code review comments (round 1)

* refactory after design review

* fix a few props initial value

* [WIP][dashboard scoped filter] part 2: add algorithm to convert checked ids to scope object (#8564)

* convert ids to scope object

* use lodash helpers to make code readable

* [WIP][dashboard scoped filter] part 3: merge filter scope settings into dashboard redux state (#8522)

* merge filter scope settings into dashboard redux state

* fix/add unit tests

* minor bug fixes

* fix save filter Scopes behavior

* resolve review comments

* fix save filter scope settings

* minor comments

* [dashboard scoped filter] Improve scrollbar inside modal (#8553)

* improve scroll inside modal

* make left pane and right pane scroll separately

* fix review comments

* force show filter_box as unchecked (#8587)
2019-11-18 13:02:25 -08:00
Beto Dealmeida
78f41b38e1 Fix more places where feature flags are serialized (#8567) 2019-11-14 10:01:57 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
Erik Ritter
1a5c2573b7 [Dashboard Import] Log error on dashboard import failure (#8550) 2019-11-13 09:41:48 -08:00
Beto Dealmeida
a58b3920db Encode feature flags to JSON pessimistically (#8529)
* Encode feature flags to JSON pessimistically

* Add unit test

* Remove old imports
2019-11-12 11:10:36 -08:00
Ville Brofeldt
49ea232c3a [fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
Juliette Tisseyre
ec476fc67e Default page: remove title attribute to fall back on appbuilder.app_name (#8427) 2019-11-06 13:10:39 -08:00
Beto Dealmeida
338a2b1a51 Allow user to customize query cost estimate (#8470)
* Allow user to customize query estimate

* Add docs; run black

* Update docs with types
2019-11-04 11:08:00 -08:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Kim Truong
3cba1bd990 feat: add expand_data parameter (#8472)
* feat: add expand_data parameter

* fix: reformat files
2019-10-30 11:45:35 -07:00
John Bodley
8b74745f9e [SIP-15] Adding initial framework (#8398)
* [sip-15] Adding initial framework

* [toast] Addressing etr2460's comments

* [fix] Addressing etr2460's comments
2019-10-28 14:23:12 -07:00
Beto Dealmeida
e704e29174 Allow fetching all rows from results endpoint (#8389)
* Allow bypassing DISPLAY_MAX_ROW

* Add unit tests and docs

* Fix tests

* Fix mock

* Fix unit test

* Revert config change after test

* Change behavior

* Address comments
2019-10-25 10:22:16 -07:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Erik Ritter
d3406e0a04 [Config] Cache static resources (#8370) 2019-10-17 09:09:25 -07:00
John Bodley
fcb39f9091 [form-data] Deleting obsolete Superset.get_form_data method (#8390)
* Update core.py

* Update core.py
2019-10-15 09:11:55 -07:00
John Bodley
05b67673c3 [metric] Adding security for restricted metrics (#8175) 2019-10-01 16:52:15 -07:00
David Aaron Suddjian
fbbc5f0577 Allow overwriting a SQLLab query that has previously been saved (#8298)
* ignore direnv

* allow overwriting saved queries

* simplify state management a little bit

* fix tests and linting
2019-09-30 23:09:45 -07:00
Daniel Vaz Gaspar
d59e0c4641 [superset] Fix, missing return on error and improved query (#8239) 2019-09-30 00:09:30 +01:00
John Bodley
5d1bf4245d [sql_json] Ensuring the request body is JSON encoded (#8256) 2019-09-23 09:09:12 -07:00
Daniel Vaz Gaspar
8bc5cd7dc0 [SQLLab] Refactor sql json endpoint (#8163)
* [superset] sql_json endpoint refactor

* [superset] sql_json endpoint refactor sync

* [superset] Fix, wrong parameter name

* [superset] flake8 and black

* [SQLLab] Don't need to pass request to helper SQLJsonParams

* [SQLLab] try except on query add

* [SQLLab] Remove unwanted changes

* [SQLLab] refactor (cont.)

* [superset] black

* [SQLLab] Remove SQLJsonParams class

* [SQLLab] Fix, add missing type annotations

* [superset] Better exception handling when saving query

* [superset] Remove unnecessary commit

* [superset] one or none instead of first

* [superset] Fix, missing return on error sql_json

* [superset] Fix, black

* [database] Use f strings
2019-09-19 12:15:40 -07:00
serenajiang
0c94f0fd82 [sqllab] create query slightly earlier (#8224) 2019-09-18 14:29:07 -07:00
Beto Dealmeida
12fb8e70cc Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida
88777943fa Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida
8847e10529 Allow users to estimate query cost before executing it (#8172)
* WIP

* Basic functionality working

* Enable per DB

* Show error message

* Small improvements

* Fix lint

* Address comments

* Address comments

* Remove logging, fix exception

* Fix tests

* Bump FAB requirements
2019-09-17 09:09:40 -07:00
John Bodley
762edf430b [talisman] Enforcing HTTP for status checks (#8214) 2019-09-12 15:54:57 -07:00
Dave Smith
1a02d61582 change log from csv_export to data_export; fix db name in log (#8204)
* change log from csv_export to data_export; fix db name in log

* add exported_format field to data_export event log
2019-09-09 11:08:47 -07:00
Maxime Beauchemin
68c4c3a0b9 Prevent 'main' database connection creation (#8038)
* prevent 'main' database connection creation

* fix tests

* removing get_main_database

* Kill get_main_database

* Point to examples tables
2019-09-08 10:18:09 -07:00
John Bodley
9dfa0a3f8e Update core.py (#8191) 2019-09-07 23:32:17 -07:00
serenajiang
be33934b83 [fix] retry stop_query on all exceptions (#8184) 2019-09-06 10:16:08 -07:00