Commit Graph

49 Commits

Author SHA1 Message Date
ʈᵃᵢ
7301a363c1 feat: setup react page with submenu for datasources listview (#10642) 2020-08-20 11:42:17 -07:00
Maxime Beauchemin
796f8e7d26 style: make database list less bloated (#10462) 2020-07-29 16:14:39 -07:00
Hossein Torabi
259a344fd1 test: upload excel (#10467) 2020-07-29 12:50:39 -07:00
pphszx
473fe1003a fix: excel sheet upload is not working (#10450)
* remove conflicts with csv upload

* revert StringField

* change description

* remove redundant space

* apply string approach
2020-07-29 10:20:44 +03:00
Daniel Vaz Gaspar
4529812a10 refactor: database API OpenAPI spec and location (#10303)
* refactor: database API OpenAPI spec and location

* fix isort and license

* fix description
2020-07-16 09:01:56 +01:00
Bogdan
84f8a51458 feat: support nulls in the csv uploads (#10208)
* Support more table properties for the hive upload

Refactor

Add tests, and refactor them to be pytest friendly

Use lowercase table names

Ignore isort

* Use sql params

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-06 13:26:43 -07:00
Hossein Torabi
fdd28c1a5e Upload excel (#9825) 2020-07-03 08:28:30 +03:00
Jason Davis
72fc581b15 added explore database for ctas/cvas (#10174)
Co-authored-by: Jason Davis <@dropbox.com>
2020-06-29 11:13:54 -07:00
Jason Davis
9de9e1c19d feat: Added configuration to SQL Lab results "Explore" button (#10164)
* added configuration to virtual table explore button

* added description to mixin

* fixed unit tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-26 09:54:55 -07:00
Bogdan
3db76c6fdc Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
Will Barrett
8e23d4f369 chore: Upgrade pylint to 2.5.3 and fix most new rules (#10101)
* Bump pylint version to 2.5.3

* Add a global disable for the most common new pylint error

* Fix a bunch of files containing very few errors

* More pylint tweakage, low-hanging fruit

* More easy stuff...

* Fix more erroring files

* Fix the last couple of errors, clean pylint!

* Black

* Fix mypy issue in connectors/druid/models.py
2020-06-18 14:03:42 -07:00
John Bodley
9532bff48f chore(security): Renaming access methods (#10031)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 13:12:23 -07:00
John Bodley
54c6ddbdb7 chore(security): Renaming schemas_accessible_by_user (#10030)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 11:29:43 -07:00
John Bodley
91517a56a3 style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Daniel Vaz Gaspar
543a3ca6f2 fix(api): Wrong parameter name on database openapi spec (#9965) 2020-06-04 15:21:23 +01:00
John Bodley
b010c35887 style(mypy): Enforcing typing for views.database (#9920)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 21:04:48 -07:00
tooptoop4
bd959919e3 Fix missing line break in Database 'extra' instructions (#9895) 2020-05-23 17:36:51 -07:00
Bogdan
3e8e441bfc feat: implement csv upload configuration func for the schema enforcement (#9734)
* Implement csv upload func for schema enforcement

Implement function controlled csv upload schema

Refactor + fix tests

Fixing hive as well

* Add explore_db to the extras

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-21 13:49:53 -07:00
John Bodley
3b0f8e9c8a [sql] Adding lighweight Table class (#9649)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-30 08:38:02 -07:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
Ville Brofeldt
e33f6c244d Add check for SSL certificate and add form validators (#9436)
* Add check for server_cert falsy and add form validators

* Address comments
2020-04-02 08:17:17 +03:00
Ville Brofeldt
499f9c8fca feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
David Aaron Suddjian
e01f24f833 Prevent database connections to sqlite (#9218)
* prevent database connections to sqlite

* tweaks and tests

* add entry to UPDATING.md
2020-03-02 13:13:11 -08:00
Daniel Vaz Gaspar
27b371a485 [core] Fix, sanitize errors returned from testconn (#9178) 2020-02-24 10:40:51 +00:00
Daniel Vaz Gaspar
e55fe43ca6 [database] Fix, tables API endpoint (#9144) 2020-02-20 10:15:22 +00:00
John Bodley
a7e433a512 [mypy] Disallowing implicit optional (#9150) 2020-02-16 22:34:15 -08:00
Daniel Vaz Gaspar
f993bdc7ef [database] new, select star API migration (#9054) 2020-02-14 14:30:49 +00:00
Daniel Vaz Gaspar
24ec89a800 [csv upload] Use python's named temp file (#9047)
* [csv upload] Use python's named temp file

* [csv upload] stream write uploaded file

* Added missing ensure that configured path exists

* Fix, lint

* [csv upload] remove python's tmp file default delete behaviour
2020-02-04 10:28:32 -08:00
Daniel Vaz Gaspar
d5dbfc886d [database] new, API table metadata (#9002)
Migrates and completely removes: '/superset/table/<database_id>/<table_name>/<schema>/'
- Guarantees database filtering based on the user permissions
- Full OpenAPI spec
- Better error handling
2020-01-29 09:51:06 +00:00
Erik Ritter
a4b8e0b9e4 [SQL Lab] Add function names to autocomplete (#9012) 2020-01-24 11:08:26 -08:00
Maxime Beauchemin
315a11dfe2 fix: shut off unneeded endpoints (#8960)
* fix: shut off all uneeded endpoints

We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.

First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.

Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.

Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
  been using endpoints in their environment for other purposes may
  experience loss of functionality

* Tweaking

* Reduce the amount of endpoints using white lists

* Fix, included needed endpoints for dashboard and druid

* Drying things up

* fixes

* limiting more endpoints

* Read only on some FAB model views

* fixing some tests

* fixes

* Fixing more tests

* Addressing comments

* Drying up route_methods

* further drying

Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
2020-01-23 11:25:15 -05:00
Craig Rueda
2a94150097 Moving appbuilder.xxx out of view files and into app.py (#8912)
* Moving appbuilder.xxx out of view files ands into app.py

* Pulled url map converters out into their own file

* Adding license blurb

* Linting

* Linting again...
2020-01-08 18:04:05 +00:00
Daniel Vaz Gaspar
4f8bf2b04d [database] [log] Fix, Limit the amount of info on response (#8918) 2020-01-03 16:35:12 +00:00
Bogdan
003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Will Barrett
bca2b91417 Split functionality out of views/database/__init__.py, re-enable pylint (#8713)
* Split functionality out of __init__.py, re-enable pylint

* Black, isort

* Test fix

* Remove commented-out code bit
2019-12-02 14:33:35 -08:00
Daniel Vaz Gaspar
517a643bd4 [databases] [mvc] remove encrypted extra from list of possible filter (#8679) 2019-11-28 09:02:45 +00:00
Will Barrett
d70e0fc359 Fix for BigQuery connection checks and CSV uploads (#8511)
* Fix for BigQuery connection checks and CSV uploads

* Don't assume encrypted_extra will be populated

* Fix undefined method error

* Refactor to avoid circular import strangeness
2019-11-19 14:50:47 -08:00
Daniel Vaz Gaspar
650c848243 [cli] Fix, import datasources exported by UI (#8477)
* [cli] Fix, import datasources exported by UI
2019-11-10 16:29:51 +00:00
Ville Brofeldt
49ea232c3a [fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
Will Barrett
e4ca44e95f Use config[] not config.get() (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Will Barrett
148cec4690 Add UI-only database configuration method for extended authorization scenarios (#8448)
* Add encrypted_extra to dbs

* WIP - UI-based BigQuery connection configuration

* Fix 500 bubbling to the surface when adding a database connection

* Add check for valid json

* black formatting:

* isort

* Incorporate PR feedback. Thanks all!

* black

* Typo fix in CONTRIBUTING.md
2019-10-28 15:49:40 +00:00
John Bodley
9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Daniel Vaz Gaspar
f3065a763f [database] Improve form and API validation for invalid URI (#8240)
* [database] Improve form and API validation for invalid URI

* [database] Added missing EOL

* [database] lint
2019-09-23 08:28:08 -07:00
Beto Dealmeida
8847e10529 Allow users to estimate query cost before executing it (#8172)
* WIP

* Basic functionality working

* Enable per DB

* Show error message

* Small improvements

* Fix lint

* Address comments

* Address comments

* Remove logging, fix exception

* Fix tests

* Bump FAB requirements
2019-09-17 09:09:40 -07:00
Daniel Vaz Gaspar
73cdad2375 [SQLLab] Fix, database api unlimited page size v2 (#8002)
* [database] Fix, Removes the limit for the page size, Bump FAB to 2.1.8

Old FAB API had no limits by default, this will keep this behaviour
but only for this endpoint

* [sqllab] Add test for database API

* [sqllab] Add test for database API

* [sqllab] Include page zero on request

* [sqllab] Fix, Black and requirements

* [sqllab] Make database API return unlimited results

* [sqllab] just a test

* [sqllab] Bump FAB to 2.1.9

* [sqllab] Remove unused import
2019-08-08 11:37:00 -07:00
Erik Ritter
b62c122b4d Revert "[database] Fix, Removes the limit for the page size (#7987)" (#7995)
This reverts commit 8db419fe99.
2019-08-06 16:14:27 -07:00
Daniel Vaz Gaspar
8db419fe99 [database] Fix, Removes the limit for the page size (#7987)
Old FAB API had no limits by default, this will keep this behaviour
but only for this endpoint
2019-08-05 22:52:43 -07:00
Erik Ritter
d58dbad076 [DB Engine] Support old and new Presto syntax (#7977) 2019-08-05 11:56:56 -07:00
Daniel Vaz Gaspar
b65ab51b63 [api] [database] New, migrate to new FAB API side by side (#7665)
* [api] [database] New, migrate to new FAB API side by side

* [database] view and api module reorganization

* [style] Fix, flake8

* [test] [database] Fix, tests for new database view structure

* [style] [database] black

* [database] Fix, missing apache license

* Migrate databaasync api to the new api

* flake8

* More efficient api call

* Revert querySearch to old API, test

* dummy commit

* revert config.py commit

* Remove unused import

* Remove old API view

* Remove new API view

* Add database filter

* Add database filter

* Add database filter

* Remove comments on js

* Fix JS test

* Fix pylint

* QuerySearch new databaseasync API endpoint

* Fix, query search new api endpoint

* Reintroduce old API for smooth side by side migration

* Better naming and use common hooks for view and API

* black it

* Fix, database mixin must come first

* Fix, lint
2019-07-22 21:25:16 -07:00