Commit Graph

203 Commits

Author SHA1 Message Date
ʈᵃᵢ
7301a363c1 feat: setup react page with submenu for datasources listview (#10642) 2020-08-20 11:42:17 -07:00
Grace Guo
3bc79191c4 refactor: remove slice level label_colors from dashboard init load (#10603) 2020-08-18 18:10:06 -07:00
Daniel Vaz Gaspar
4dd1d1df5e fix(thumbnails): missing field, logging and new config var (#10562)
* fix(thumbnails): missing field, logging and new config var

* deprecate EMAIL_REPORTS_WEBDRIVER

* fix after merge and conflicts

* fix tests

* black
2020-08-14 14:12:44 +01:00
Craig Rueda
2aaa4d92d9 chore: Migrating reports to AuthWebdriverProxy (#10567)
* Migrating reports to AuthWebdriverProxy

* Extracting out webdriver proxy / Adding thumbnail tests to CI

* Adding license

* Adding license again

* Empty commit

* Adding thumbnail tests to CI

* Switching thumbnail test to Postgres

* Linting

* Adding mypy:ignore / removing thumbnail tests from CI

* Putting ignore statement back

* Updating docs

* First cut at authprovider

* First cut at authprovider mostly working - still needs more tests

* Auth provider tests added

* Linting

* Linting again...

* Linting again...

* Busting CI cache

* Reverting workflow change

* Fixing dataclasses

* Reverting back to master

* linting?

* Reverting installation.rst

* Reverting package-lock.json

* Addressing feedback

* Blacking

* Lazy logging strings

* UPDATING.md note
2020-08-12 13:28:41 -07:00
John Bodley
cda764fd4b chore: Adding pip-compile-multi et al. (#10499)
* chore: Adding pip-compile-multi et al

* Specify requirements.txt path for fossa

* [ci] Fixing CI

Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
2020-08-04 19:31:52 -07:00
Jason Davis
72ced53d2a feat: make screenshot timeout configurable (#10517)
* made screenshot timeout configurable

* added default value to config and refractored use

* black

* updated config comment

* moves config variables to thumbnail section

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-04 17:16:31 -07:00
ʈᵃᵢ
6f56cd5e9d feat(listviews): SIP-34 filters for charts, dashboards, datasets (#10335) 2020-07-27 10:14:11 -07:00
Daniel Vaz Gaspar
e0264060c3 fix(charts): disable CSRF for chart data endpoint (#10397) 2020-07-23 10:44:25 +01:00
Ville Brofeldt
9b8da40295 feat: add TXT as default CSV extension (#10371) 2020-07-21 17:04:29 +03:00
John Bodley
a7a96430cf chore: Silencing unused thumbnail cache config (#10291) 2020-07-17 10:09:12 -07:00
Ville Brofeldt
ac85aebe4a chore: prefer allow/deny terminology (#10320)
* chore: prefer allow/deny terminology

* fix tests

* add PR reference
2020-07-15 19:22:46 +03:00
Maxime Beauchemin
a460fdf0f8 docs(fix): /swaggerview/v1 has moved to /swagger/v1 (#10276) 2020-07-09 15:16:52 -07:00
Evan Rusackas
569e4a7c50 chore: Revised logo (new color), w/CSS for vert alignment (#10237)
* out with the old, in with the new

* vertically centering logo once and for all

* lint: black instead of IDE formating

* touch config.py

* Styles -> StyledHeader

* nixing vestigial SVG

* removing trademark

Co-authored-by: Maxime Beauchemin <maximebeauchemin@gmail.com>
2020-07-06 17:30:18 -07:00
Bogdan
84f8a51458 feat: support nulls in the csv uploads (#10208)
* Support more table properties for the hive upload

Refactor

Add tests, and refactor them to be pytest friendly

Use lowercase table names

Ignore isort

* Use sql params

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-06 13:26:43 -07:00
Maxime Beauchemin
318e5347bc feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
John Bodley
774c23a6f1 chore: Deprecating ENABLE_FLASK_COMPRESS (#10233)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-04 12:46:34 -07:00
Hossein Torabi
fdd28c1a5e Upload excel (#9825) 2020-07-03 08:28:30 +03:00
ʈᵃᵢ
3414f35792 refactor: better naming for config flags (#10180) 2020-06-26 17:36:50 -07:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
Bogdan
be6b9b8fec feat: implement dttm column configuration through db extra config (#9444)
* Implement table mutator and examples how to set date defaults

Fix tests

* Fix flaky test

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 13:46:45 -07:00
Bogdan
29e9f2c70b feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
John Bodley
244677cf5e style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
Grace Guo
dcac860f3e feat: [dashboard] notification and warning for auto force refresh (#9886)
* feat: [dashboard] notification and warning for auto force refresh

* fix review comments
2020-06-03 10:20:56 -07:00
Ville Brofeldt
38a6bd79da feat: expand new chart data endpoint coverage (#9903)
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
2020-06-02 10:47:28 +03:00
John Bodley
b296a0f250 [mypy] Enforcing typing for superset.utils (#9905)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 22:57:30 -07:00
Bogdan
3e8e441bfc feat: implement csv upload configuration func for the schema enforcement (#9734)
* Implement csv upload func for schema enforcement

Implement function controlled csv upload schema

Refactor + fix tests

Fixing hive as well

* Add explore_db to the extras

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-21 13:49:53 -07:00
Bogdan
1d3b264a36 Add extra configration to the email reports (#9342)
Superset email. Fix slice url

Make WEBDRIVER_BASEURL a function

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-13 20:14:12 +03:00
Ville Brofeldt
b532c4609f chore: bump python dependencies (#9786)
* chore: bump pyhton dependencies

* bump flask-testing

* Add note to UPDATING and bump flask-testing in requirements-dev.txt

* fix old sqlparse test case that is fixed with sqlparse==0.3.1

* Fix migration and sqlparse set test

* downgrade alembic

* Downgrade sqlalchemy-utils

* pin sqlalchemy-utils due to regression in 0.36.5

* Refine comment in setup.py

* make cachelib an explicit requirement
2020-05-13 18:49:34 +03:00
Erik Ritter
083dd6ae1a [config] Enable dashboard bootstrap payload reduction by default (#9585) 2020-04-19 15:41:36 -07:00
Daniel Vaz Gaspar
d81f720502 [thumbnails] API and celery task for dashboards and charts (#8947) 2020-04-15 09:40:14 +01:00
Ville Brofeldt
9447381549 deprecate groupby controls in query_obj (#9366)
* Deprecate groupby from query_obj

* Fix query_object bug

* Fix histogram

* Remove groupby from legacy druid connector and fix first batch of unit tests

* Deprecate some unnecessary tests + fix a few others

* Address comments

* hide SIP-38 changes behind feature flag

* Break out further SIP-38 related tests

* Reslove test errors

* Add feature flag to QueryContext

* Resolve tests and bad rebase

* Backport recent changes from viz.py and fix broken DeckGL charts

* Fix bad rebase

* backport #9522 and address comments
2020-04-14 12:06:10 +03:00
ʈᵃᵢ
f90824fa17 [Dashboard] new listview filters & emotion infra (#9462)
* [Dashboard] listview filters to match new design

* use title_or_slug filter

* use ?. operator

* move components to components folder
2020-04-13 13:39:55 -07:00
dandanhub
72f051f3ce [sqllab] Add CUSTOM_TEMPLATE_PROCESSOR config (#9376)
Co-authored-by: Dandan Shi <dshi@dropbox.com>
2020-04-07 21:00:42 +01:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
John Bodley
742887bf00 [sip-15] fix messaging (#9397) 2020-03-27 08:20:47 -07:00
John Bodley
5d9857544a [config] Fixing GET_FEATURE_FLAGS_FUNC example (#9339)
* [config] Fixing GET_FEATURE_FLAGS_FUNC example

* Adding typing for GET_FEATURE_FLAGS_FUNC

* Update config.py
2020-03-23 13:53:37 -07:00
ʈᵃᵢ
8764ae3852 [sqllab] fix exception caused by casting string to int with psycopg2 (#9287)
* [sqllab] fix exception caused by casting string to int with psycopg2

* rollback session on exception

* add SQLLAB_BACKEND_PERSISTENCE to default feature flags

* cast tab_state_id to str in python instead of sql

* remove import
2020-03-16 10:51:41 -07:00
Erik Ritter
f80fadff0e Reduce dashboard bootstrap payload (#9284) 2020-03-13 12:18:22 -07:00
Ville Brofeldt
ef2ebbd570 Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
Bogdan
4e1fa95035 Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
David Aaron Suddjian
e01f24f833 Prevent database connections to sqlite (#9218)
* prevent database connections to sqlite

* tweaks and tests

* add entry to UPDATING.md
2020-03-02 13:13:11 -08:00
Daniel Vaz Gaspar
23fc9a7a72 [config] Disable FAB's permission and view menus views (#9133) 2020-02-25 00:49:56 +00:00
altef
dee16de03e [SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Will Barrett
38f3fd0c9f Add feature flags to control query sharing, KV exposure (#9120)
* Add feature flags to control query sharing, KV exposure

* Add tests, fix bug

* Skip test for kv endpoints when they are disabled

* ESLint fixes

* Remove unnecessary binds

* Fix eslint errors

* Add note to UPDATING.md RE: new feature flag options

* Use expanded version of RBAC

* Enable KV_STORE and SHARE_QUERIES_VIA_KV_STORE feature flags in the test environment

* Fix black
2020-02-19 09:51:50 -08:00
David Aaron Suddjian
966d321703 [docker] fix, Dockerfile for frontend builds (#9114)
* fix dockerfile for frontend builds

* fix paths

* empty commit for CI

* adjust version_info.json

* making version_info work

* rat

* fixing docker-compose node service

* address feedback
2020-02-12 15:48:24 +00:00
Craig Rueda
607cfd1f29 Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Will Barrett
47b5780c5c Wrap tagging endpoints in a feature flag (disabled by default) (#9078)
* Wrap tagging endpoints in a feature flag (disabled by default)

* Add tests for new feature flag

* Reuse existing feature flag

* Add TAGGING_SYSTEM to feature flags defaulted to False
2020-02-05 11:10:13 -08:00
Daniel Vaz Gaspar
24ec89a800 [csv upload] Use python's named temp file (#9047)
* [csv upload] Use python's named temp file

* [csv upload] stream write uploaded file

* Added missing ensure that configured path exists

* Fix, lint

* [csv upload] remove python's tmp file default delete behaviour
2020-02-04 10:28:32 -08:00
John Bodley
848c565c8c [sip-15] Enabling SIP-15 by default (#9017) 2020-02-03 10:37:29 -08:00