Commit Graph

186 Commits

Author SHA1 Message Date
ʈᵃᵢ
3414f35792 refactor: better naming for config flags (#10180) 2020-06-26 17:36:50 -07:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Bogdan
be6b9b8fec feat: implement dttm column configuration through db extra config (#9444)
* Implement table mutator and examples how to set date defaults

Fix tests

* Fix flaky test

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 13:46:45 -07:00
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
John Bodley
244677cf5e style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
Grace Guo
dcac860f3e feat: [dashboard] notification and warning for auto force refresh (#9886)
* feat: [dashboard] notification and warning for auto force refresh

* fix review comments
2020-06-03 10:20:56 -07:00
Ville Brofeldt
38a6bd79da feat: expand new chart data endpoint coverage (#9903)
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
2020-06-02 10:47:28 +03:00
John Bodley
b296a0f250 [mypy] Enforcing typing for superset.utils (#9905)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 22:57:30 -07:00
Bogdan
3e8e441bfc feat: implement csv upload configuration func for the schema enforcement (#9734)
* Implement csv upload func for schema enforcement

Implement function controlled csv upload schema

Refactor + fix tests

Fixing hive as well

* Add explore_db to the extras

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-21 13:49:53 -07:00
Bogdan
1d3b264a36 Add extra configration to the email reports (#9342)
Superset email. Fix slice url

Make WEBDRIVER_BASEURL a function

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-13 20:14:12 +03:00
Ville Brofeldt
b532c4609f chore: bump python dependencies (#9786)
* chore: bump pyhton dependencies

* bump flask-testing

* Add note to UPDATING and bump flask-testing in requirements-dev.txt

* fix old sqlparse test case that is fixed with sqlparse==0.3.1

* Fix migration and sqlparse set test

* downgrade alembic

* Downgrade sqlalchemy-utils

* pin sqlalchemy-utils due to regression in 0.36.5

* Refine comment in setup.py

* make cachelib an explicit requirement
2020-05-13 18:49:34 +03:00
Erik Ritter
083dd6ae1a [config] Enable dashboard bootstrap payload reduction by default (#9585) 2020-04-19 15:41:36 -07:00
Daniel Vaz Gaspar
d81f720502 [thumbnails] API and celery task for dashboards and charts (#8947) 2020-04-15 09:40:14 +01:00
Ville Brofeldt
9447381549 deprecate groupby controls in query_obj (#9366)
* Deprecate groupby from query_obj

* Fix query_object bug

* Fix histogram

* Remove groupby from legacy druid connector and fix first batch of unit tests

* Deprecate some unnecessary tests + fix a few others

* Address comments

* hide SIP-38 changes behind feature flag

* Break out further SIP-38 related tests

* Reslove test errors

* Add feature flag to QueryContext

* Resolve tests and bad rebase

* Backport recent changes from viz.py and fix broken DeckGL charts

* Fix bad rebase

* backport #9522 and address comments
2020-04-14 12:06:10 +03:00
ʈᵃᵢ
f90824fa17 [Dashboard] new listview filters & emotion infra (#9462)
* [Dashboard] listview filters to match new design

* use title_or_slug filter

* use ?. operator

* move components to components folder
2020-04-13 13:39:55 -07:00
dandanhub
72f051f3ce [sqllab] Add CUSTOM_TEMPLATE_PROCESSOR config (#9376)
Co-authored-by: Dandan Shi <dshi@dropbox.com>
2020-04-07 21:00:42 +01:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
John Bodley
742887bf00 [sip-15] fix messaging (#9397) 2020-03-27 08:20:47 -07:00
John Bodley
5d9857544a [config] Fixing GET_FEATURE_FLAGS_FUNC example (#9339)
* [config] Fixing GET_FEATURE_FLAGS_FUNC example

* Adding typing for GET_FEATURE_FLAGS_FUNC

* Update config.py
2020-03-23 13:53:37 -07:00
ʈᵃᵢ
8764ae3852 [sqllab] fix exception caused by casting string to int with psycopg2 (#9287)
* [sqllab] fix exception caused by casting string to int with psycopg2

* rollback session on exception

* add SQLLAB_BACKEND_PERSISTENCE to default feature flags

* cast tab_state_id to str in python instead of sql

* remove import
2020-03-16 10:51:41 -07:00
Erik Ritter
f80fadff0e Reduce dashboard bootstrap payload (#9284) 2020-03-13 12:18:22 -07:00
Ville Brofeldt
ef2ebbd570 Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
David Aaron Suddjian
e01f24f833 Prevent database connections to sqlite (#9218)
* prevent database connections to sqlite

* tweaks and tests

* add entry to UPDATING.md
2020-03-02 13:13:11 -08:00
Daniel Vaz Gaspar
23fc9a7a72 [config] Disable FAB's permission and view menus views (#9133) 2020-02-25 00:49:56 +00:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Will Barrett
38f3fd0c9f Add feature flags to control query sharing, KV exposure (#9120)
* Add feature flags to control query sharing, KV exposure

* Add tests, fix bug

* Skip test for kv endpoints when they are disabled

* ESLint fixes

* Remove unnecessary binds

* Fix eslint errors

* Add note to UPDATING.md RE: new feature flag options

* Use expanded version of RBAC

* Enable KV_STORE and SHARE_QUERIES_VIA_KV_STORE feature flags in the test environment

* Fix black
2020-02-19 09:51:50 -08:00
David Aaron Suddjian
966d321703 [docker] fix, Dockerfile for frontend builds (#9114)
* fix dockerfile for frontend builds

* fix paths

* empty commit for CI

* adjust version_info.json

* making version_info work

* rat

* fixing docker-compose node service

* address feedback
2020-02-12 15:48:24 +00:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Will Barrett
47b5780c5c Wrap tagging endpoints in a feature flag (disabled by default) (#9078)
* Wrap tagging endpoints in a feature flag (disabled by default)

* Add tests for new feature flag

* Reuse existing feature flag

* Add TAGGING_SYSTEM to feature flags defaulted to False
2020-02-05 11:10:13 -08:00
Daniel Vaz Gaspar
24ec89a800 [csv upload] Use python's named temp file (#9047)
* [csv upload] Use python's named temp file

* [csv upload] stream write uploaded file

* Added missing ensure that configured path exists

* Fix, lint

* [csv upload] remove python's tmp file default delete behaviour
2020-02-04 10:28:32 -08:00
John Bodley
848c565c8c [sip-15] Enabling SIP-15 by default (#9017) 2020-02-03 10:37:29 -08:00
ʈᵃᵢ
d0aab05e79 [dashboard] fix, add config to optionally enable react replacement fo… (#9025)
* [dashboard] fix, add config to optionally enable react replacement for CRUD views

* move app.config to function scope
2020-01-27 11:28:24 -08:00
Erik Ritter
68e85ab1b6 [Caching] Ensure cache is always created (#9020)
* [Caching] Ensure cache is always created

* Update cache_manager.py

* Refactoring cache typing
2020-01-25 22:49:05 +00:00
serenajiang
f079b6dad0 [i18n] enable spanish (#9010) 2020-01-24 11:27:05 -08:00
John Bodley
d56abc3d34 [druid] Making scaning/refreshing Druid datasource view items optional (#8995) 2020-01-22 15:53:16 -08:00
Kelvin Jayanoris
28563ad088 Fix docstrings in superset/config.py (#8931)
- Replace ldap with LDAP
- Replace `(username/password()` with `(username/password)`
2020-01-07 11:00:32 +00:00
Hossein Torabi
478e445a5a Add config for celery email task soft time (#8887) 2020-01-04 09:20:57 +02:00
Rob DiCiuccio
6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Ville Brofeldt
5b690f9411 chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Evan Rusackas
36c6f4ca3a moving favicon(s) to config, allowing multiple favicons (#8856)
* moving favicon to config, allowing multiple favicons

* black...
2019-12-20 12:58:36 -08:00
Evan Rusackas
5780d9d53b Version number and SHA added to user menu (#8850)
* working!

* sigh... linting.

* more linting
2019-12-16 23:48:27 -08:00
Will Barrett
68aca88c54 Re-enable pylint on some configuration files (#8767)
* re-enable pylint for superset/config.py

* re-enable pylint on superset/migrations/env.py

* re-enable pylint for superset/legacy.py

* re-enable pylint for superset/forms.py

* re-enable pylint for superset/stats_logger.py

* Tweaks to make mypy and pylint happy

* black
2019-12-06 14:30:11 -08:00
Evan Rusackas
b107cc03bf SHA display touchups (#8614)
* Only showing last 8 chars of SHA

* visual touchups

* Grabbing first 8 rather than last 8

* 10 char SHA rather than 8

* atting config var for SHA length
2019-12-04 13:13:51 -08:00
Daniel Vaz Gaspar
385484eb9d [config] Enables swagger UI by default (#8732)
* [config] Enables swagger UI by default
2019-12-04 09:50:56 +00:00
Beto Dealmeida
f8e15e1c55 Add protocol to cache warmup (#8721)
* Add protocol to cache warmup

* Update unit tests

* Fix wrong commit

* Update UPDATING.md

* Fix pylint
2019-12-03 10:51:06 -08:00
Will Barrett
16c0599470 Disable deprecated druid connector by default (#8512)
* Disable deprecated druid connector by default

* Add a line in UPDATING.md for the configuration change

* Remove security tests related default-disabled feature

* More test updates

* black
2019-11-27 11:28:23 -08:00
Evan Rusackas
ac665d360c showing version and SHA (subtly) in user menu. (#8562)
* showing version and SHA (subtly) in user menu.

* not using VERSION_INFO_FILE after all.

* cleanup... excess linebraks...

* Just trying to make you happy, Black!

* adding some spaces before opening brackets for CSS selectors

* setting version_sha from appbuilder.app.config['VERSION_SHA']
2019-11-19 21:35:18 -08:00
Beto Dealmeida
d66bc5ad90 SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00