Commit Graph

63 Commits

Author SHA1 Message Date
Gabriel Torres Ruiz
0a75bac2a1 fix(error-handling): jinja2 error handling improvements (#34803) 2025-09-02 17:29:07 -04:00
Beto Dealmeida
1204507d68 fix: make get_image() always return BytesIO (#34801) 2025-08-25 09:01:36 -04:00
Beto Dealmeida
edc60914f6 chore: 100% test coverage for SQL parsing (#33568) 2025-06-04 22:18:09 -04:00
Beto Dealmeida
a26e1d822a chore: remove sqlparse (#33564) 2025-06-04 19:31:41 -04:00
Beto Dealmeida
cf315388f2 feat(sqllab): use sqlglot instead of sqlparse (#33542) 2025-05-30 17:08:19 -04:00
Maxime Beauchemin
e51b95ffa8 chore: enforce more ruff rules (#31447)
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2024-12-18 17:41:34 -08:00
Beto Dealmeida
a098809294 feat(SQL Lab): better SQL parsing error messages (#30501) 2024-10-04 19:17:56 -04:00
Beto Dealmeida
bdf29cb7c2 chore: organize SQL parsing files (#30258) 2024-09-13 16:24:19 -04:00
Beto Dealmeida
0415ed34ce feat: allow create/update OAuth2 DB (#30071) 2024-09-03 19:22:38 -04:00
Ville Brofeldt
7d6e933348 chore(key-value): convert command to dao (#29344) 2024-07-01 20:22:11 +03:00
Daniel Vaz Gaspar
5dfbab5424 fix: adds the ability to disallow SQL functions per engine (#28639) 2024-05-29 10:51:28 +01:00
Beto Dealmeida
68a982dfe6 feat(sip-95): new endpoint for extra table metadata (#28063) 2024-04-18 10:42:53 -04:00
Beto Dealmeida
9022f5c519 feat(SIP-85): OAuth2 for databases (#27631) 2024-04-02 22:05:33 -04:00
Beto Dealmeida
26d8077e97 chore: improve SQL parsing (#26767) 2024-03-13 18:27:01 -04:00
Beto Dealmeida
3e63c82ecc feat: generic marshmallow error component (#25303) 2023-10-03 11:35:28 -07:00
Hugh A. Miles II
3b46511439 feat(Tags): Allow users to favorite Tags on CRUD Listview page (#24701) 2023-07-27 13:17:26 -04:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
Hugh A. Miles II
b820eb8235 chore: Update pre-commit packages (#23173) 2023-03-13 17:05:13 -06:00
Kamil Gabryjelski
42980a69a7 fix(dashboard): Charts crashing when cross filter on adhoc column is applied (#23238)
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2023-03-04 07:57:35 +01:00
Diego Medina
3ed288d4ee chore: Migrate /superset/stop_query/ to API v1 (#22624) 2023-01-16 14:07:52 +00:00
Diego Medina
383313b105 fix(sql lab): Syntax errors should return with 422 status (#20491)
* fix(sql lab): Syntax errors should return with 422 status

* refactor
2022-07-27 12:41:17 -06:00
Daniel Vaz Gaspar
b9a98aae79 fix: None dataset and schema permissions (#20108)
* fix: None dataset and schema permissions

* fix pylint

* add migration and test

* fix migration
2022-05-20 09:40:10 +01:00
cccs-RyanS
ddc01ea781 feat(business-types): initial implementation of SIP-78 (#18794)
* add BUSINESS_TYPE_ADDONS to config with example callback

* Removing uneeded whitespace

* [Work in progress] Modifying cidr function to allow for  single ip and adding port outline

* Added test REST endpoint, added some more ports

I've thrown in a test.py script as well that will try to connect to the
business_type endpoint.

* Moving code from config.py into the business api

Very simple api is exposed that will allow someone to call a checkport
endpoint and get back a response.

* Removing commented out bits.

* Adding fucntion dict back to the config

* Moving business_type endpoint to charts

* Adding schema for get endpoint

* Removing imports, updating docstring, fixing typo

Just some small changes as described in the title.  I've updated the
test.py as well so it functions with the endpoint changes.

* Adding translation dict

* Fixing ops

* Adding check for list

* Modifying changes to add quotes where needed

Also changed BusinessTypeResponse to resp.

* Adding in some code to call the filter config

If a column starts with "cidr_" it will call the code in config.py to
try to translate the filter.  Nothing is changed in the JSON being
executed, just some information is dumped to console.

* Porting Ryan's changes

* Adding migration script (as per Ryan's PR)

* Fixing typo

* Prettier fixes

* [CLDN-1043] Adding rough version of filter changes for business types

* fix down migration

* Fixing bugs after merge

* adding functionality to appy filters in back end

* Fixing linting issues

* fix down revision

* Changing conversion callback to handle multiple values at once

* Adding string representation of values

* Code cleanup plus fixing debouce to only be called once for each entry

* Removing non needed logginh

* Changing operator list to use sting values

* Using text value operators

* Removing clear operator call

* Moving business type endpoints

* fix down revision

* Adding port functions

* update migration

* fix bad rebase and add ff

* implement validator

* dont add invalid values to response

* [CLDN-1205] Added a new exception type for a business type translation error. Added the error message in the display_value field within the business type response. Modified the IP and Port business types to populate the error message field in the response if an error occurs

* [CLDN-1205] Added meaningful error message for port translation errors

* Removing status field from businesstype Response and adding in error message

* [CLDN-1205] Added check to make sure the port business type is within the valid range of ports, if it is not, it will populate the error message

* [CLDN-1205] Fixed the if statement that checks to see if the string_value is in the valid range of port numbers. It did not corrently verify this before now.

* [CLDN-1205] Fixed an error where it was trying to use string_value in <= statements. I just casted string_value to an integer if it is numeric, which allows <= operators to be used on it

* [CLDN-1207] Added unit tests for the cidr_func and port_translation_func functions which are located in /superset/config.py

* [CLDN-1207] removed the assertRaises line as it does not work with the cidr_func and port_translation_func functions

* [CLDN-1207] Added the skeleton of the test_cidr_translate_filter_func unit test, still need to update what the expected response from the function will be.

* [CLDN-1207] Added the remainder of the back-end unit tests for the business types

* [CLDN-1207] Fixed the syntax error which caused the test_cidr_translate_filter_func_NOT_IN_double unit test to fail

* [CLDN-1207] Removed the logging that was added for debugging purposes

* [CLDN-1207] Formatted the commands_tests.py file to make it nicer to look at/read through

* [CLDN-1207] Fixed the code so that it conformed to the pylint requirements (i.e., pylint no longer complains about the code in commands_tests.py)

* [CLDN-1207] Modified some of the docstrings so they made better use of the 100 character per line, line limit

* [CLDN-1207] Added the beginnings of the unit tests for the
business types API

* [CLDN-1207] Added a comment to the top of the commands_tests.py file explaining how to run the unit tests. This prevents the next person who tries to run them from having to waste time trying the different forms of testing that Superset supports (e.g., pytest, tox, etc.)

* [CLDN-1207] Added a grammar fix to the comments describing how to run the unit tests

* [CLDN-1207] Modified the description of the business_type API endpoints as they did not represent what the API was actually doing

* [CLDN-1207] Added further instructions on how to run the unit tests that are within the business_type/api_tests.py file

* add request validation

* disable request if business type missing

* [CLDN-1207] Unit tests for the business type API are now working, however, they need to be modified to make use of @mock as we don't want to have to run the server to be able to run the unit tests

* Removing businesss types deffinitons from config

* Adding select to only show valid business types

* Fixed Enzyme tests

* Added scalfolding for selecting filter dropdown

* Adding intigration tests

* fix revision

* fix typos and unnecessary requests

* break out useBusinessTypes

* Added front-end RTL unit tests for the business type API endpoint

* Fixed error from unit tests

* Added a unit test to ensure the operator list is updated after a business type API response is received

* Removing elect compoenet for business types

* Adding feature flag and allowing saving when no business type present

* fixing useEffect hooks

* Adding feature flag to model

* Changing behavior such that an empty string returns a default response

* add form validation

* Modified comments in unit test as command to run test has changed

* Modified comments in unit test as filename to run test has changed

* Modified the api_tests.py file to conform to the linting requirements

* Changed the name of one of the tests to reflect what the test is actually testing

* Added cypress back to the package.json

* Added informative comments

* Updated comments in files as well as removed imports which were not being used

* Changes made by npm run prettier

* Fixed spelling mistakes

* Updated models.py to remove placeholder comments used in development

* Added feature flag mocking in unit test

* Fixing open api failure

* Fixing business types to pass unit tests

* Reverting unsafe connections back to false

* Removing print statement

* Adding business tpye to export test

* setting default feature flag to false for business type

* Reverting pre commit

* Reverting pre commit and running pre commit

* Reverting pre commit and running pre commit

* Fixing formatting

* Adding license

* Fixing Linting

* Protecting api enpoints

* updating model

* Fixing code path when business type exists

* Linting

* Linting

* Fixing linting

* Fixing spelling

* Fixing schemas

* Fixing app import

* fixing item render

* Added RTL test to make sure business type operator list is updated after API response

* Fixing linting

* fix migration

* Changing unit tests

* Fixing import and DB migration after rebase

* Renaming to advanced types

* Fixing Linting

* More renaming

* Removing uneeded change

* Fixing linting and test errors

* Removing unused imports

* linting

* Adding more detailed name for migration

* Moving files to plugins

* more renaming

* Fixing schema name

* Disabling feature flag that should not be enabled by default

* Adding extra cehck

* NameChange

* formatting

* Fixing equals check

* Moveing all advanced type classes and types to one file, and converting tests to functional

* Adding advanced type to test and fix linitng

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Dan Parent <daniel.parent@cse-cst.gc.ca>
Co-authored-by: GITHUB_USERNAME <EMAIL>
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
2022-05-16 17:58:21 +03:00
Ville Brofeldt
a619cb4ea9 chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
Yongjie Zhao
375c03e084 feat(advanced analysis): support MultiIndex column in post processing stage (#19116) 2022-03-23 13:46:28 +08:00
Evan Rusackas
3aa11614b5 fix: change 401 response to a 403 for Security Exceptions (#17768)
* fix: change 401 to 403 for Security Exceptions

* updating tests to reflect new (proper) status code

* another test update
2021-12-15 17:11:04 -07:00
Ville Brofeldt
3a42071e0f chore(sql): clean up invalid filter clause exception types (#17702)
* chore(sql): clean up invalid filter clause exception types

* fix lint

* rename exception
2021-12-09 17:49:32 +02:00
AAfghahi
8a27c6760d change to 422 for stop query (#17292) 2021-10-29 16:34:41 -07:00
ofekisr
3f784cc1c7 refactor: sql lab: handling command exceptions (#16852)
* chore: support error_type in SupersetException and method to convert the exception to dictionary

* chore: support error_type in SupersetException and method to convert the exception to dictionary

* refactor handling command exceptions   fix update query status when query was not created
2021-09-29 16:20:42 +03:00
John Bodley
4ba17092fa fix: Ensure SupersetError.extra is always a dict (#15848)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-24 10:19:30 +12:00
Peter Kosztolanyi
02032ee8a4 feat: cancel db query on stop (#15403)
* feat: cancel db query on stop

* fix pylint

* Add unit tests

* Do not bind multiple times

* Stop only running queries

* Postgres to cancel only the required query

* Remove extra log

* Add docstring

* Better types, docstring and naming

* Use python3 format strings

* Update superset/sql_lab.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Add cancel_query_on_windows_unload option to database

* Return cancel_query as bool

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-13 12:09:22 -04:00
Beto Dealmeida
e606477ec1 feat: more SIP-40 errors (#15482) 2021-06-30 16:32:59 -07:00
Beto Dealmeida
149c5fd594 feat: add more SIP-40 errors to SQL Lab (#15436) 2021-06-29 18:17:16 -07:00
Beto Dealmeida
21d1fb55c6 feat: more error messages (#15409) 2021-06-28 14:51:31 -07:00
Beto Dealmeida
ba5d66cb0a fix: DB parameter validation (#14636) 2021-05-14 14:07:34 -07:00
Beto Dealmeida
31f406a526 feat: API endpoint to validate databases using separate parameters (#14420)
* feat: new endpoint for validating database parameters

* Rebase

* Remove broken tests
2021-05-12 18:32:10 -07:00
Erik Ritter
3d357c661c feat: handle chart/data API errors (#14040) 2021-04-09 09:39:02 -07:00
Beto Dealmeida
a82d72fef6 feat: initial work to make v1 API compatible with SIP-40 and SIP-41 (#13960)
* WIP

* Use errorhandler

* Add response schema

* Fix status on HTTPException

* s/found/encountered/g

* Fix test

* Fix lint

* Fix lint and test
2021-04-06 22:06:32 -07:00
Hugh A. Miles II
d54cc6d82a fix: change status code for generic errors to 400 (#13868)
* change status code for generic errors to 422

* Update exceptions.py
2021-03-31 16:19:15 +08:00
Beto Dealmeida
6e7f276775 feat: initial custom error messages for SQL Lab (#12080) 2020-12-17 22:58:21 -08:00
Rob DiCiuccio
4d329071a1 feat(SIP-39): Async query support for charts (#11499)
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-10 20:21:56 -08:00
Daniel Vaz Gaspar
f27ebc4be5 feat: new reports scheduler (#11711)
* feat(reports): scheduler and delivery system

* working version

* improvements and fix grace_period

* add tests and fix bugs

* fix report API test

* test MySQL test fail

* delete-orphans

* fix MySQL tests

* address comments

* lint
2020-11-25 08:50:30 +00:00
Ville Brofeldt
e647286393 chore(sql-lab): catch PyArrow deserialization error (#11201) 2020-10-12 10:16:00 +03:00
Erik Ritter
5fa4680447 feat: update timeout error UX (#10274) 2020-07-20 15:32:17 -07:00
Maxime Beauchemin
2314aad450 feat: add database dropdown to dashboard import (#10118)
* feat: add database dropdown to dashboard import

Currently, when importing a database from a JSON file, the process
looks at the database name from the source (the info is in the file)
and matches the datasources to that name. If no database by that name
exists, it simply fails.

With this PR, we add a database dropdown that allows the user to specify
which databases the datasources should target as the get upserted.

I want to stress that the code in this area is not in a great shape,
and that the challenge of serializing/deser the nested objects is
challenging, but that there should be a much better way to do this.
One of the improvement (out of scope for this PR) that would allow to
simplify those import/export would be to use UUIDs for
importable/exportable objects.

Another identified issue is the indirections between
`utils/import_expor_{model}.py` on top of `{Model}.import_object`. Not
addressing that here.

Next topic is the MVC stuff. Decided to stick with it for now as this is
more of a [obious missing feat:] than a rewrite.

* isort \!? 0%^$%Y$&?%$^?%0^?

* fix tests

* pre-committing to py3.6

* address dpgaspar's comments

* revert isort
2020-07-05 15:08:37 -07:00
John Bodley
244677cf5e style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
Erik Ritter
d02f2d1fa7 feat: return security errors in the SIP-40 format (#9796) 2020-05-13 17:10:37 -07:00
John Bodley
1c656feb95 [mypy] Enforcing typing for a number of modules (#9586)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-24 10:07:35 -07:00
Ville Brofeldt
a8ce3bccdf feat: Add post processing to QueryObject (#9427)
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
2020-04-10 20:50:11 +03:00
Ville Brofeldt
e33f6c244d Add check for SSL certificate and add form validators (#9436)
* Add check for server_cert falsy and add form validators

* Address comments
2020-04-02 08:17:17 +03:00