Commit Graph

290 Commits

Author SHA1 Message Date
Craig Rueda
d1c24f81f2 chore(migrations): Renaming migration files so that they're easier to keep track of (#20284)
* Renaming migration files so that they're easier to keep track of

* Fixing broken import

* Fixing broken import again

* Fixing broken tests
2022-06-07 10:30:09 -07:00
Daniel Vaz Gaspar
b9a98aae79 fix: None dataset and schema permissions (#20108)
* fix: None dataset and schema permissions

* fix pylint

* add migration and test

* fix migration
2022-05-20 09:40:10 +01:00
cccs-RyanS
ddc01ea781 feat(business-types): initial implementation of SIP-78 (#18794)
* add BUSINESS_TYPE_ADDONS to config with example callback

* Removing uneeded whitespace

* [Work in progress] Modifying cidr function to allow for  single ip and adding port outline

* Added test REST endpoint, added some more ports

I've thrown in a test.py script as well that will try to connect to the
business_type endpoint.

* Moving code from config.py into the business api

Very simple api is exposed that will allow someone to call a checkport
endpoint and get back a response.

* Removing commented out bits.

* Adding fucntion dict back to the config

* Moving business_type endpoint to charts

* Adding schema for get endpoint

* Removing imports, updating docstring, fixing typo

Just some small changes as described in the title.  I've updated the
test.py as well so it functions with the endpoint changes.

* Adding translation dict

* Fixing ops

* Adding check for list

* Modifying changes to add quotes where needed

Also changed BusinessTypeResponse to resp.

* Adding in some code to call the filter config

If a column starts with "cidr_" it will call the code in config.py to
try to translate the filter.  Nothing is changed in the JSON being
executed, just some information is dumped to console.

* Porting Ryan's changes

* Adding migration script (as per Ryan's PR)

* Fixing typo

* Prettier fixes

* [CLDN-1043] Adding rough version of filter changes for business types

* fix down migration

* Fixing bugs after merge

* adding functionality to appy filters in back end

* Fixing linting issues

* fix down revision

* Changing conversion callback to handle multiple values at once

* Adding string representation of values

* Code cleanup plus fixing debouce to only be called once for each entry

* Removing non needed logginh

* Changing operator list to use sting values

* Using text value operators

* Removing clear operator call

* Moving business type endpoints

* fix down revision

* Adding port functions

* update migration

* fix bad rebase and add ff

* implement validator

* dont add invalid values to response

* [CLDN-1205] Added a new exception type for a business type translation error. Added the error message in the display_value field within the business type response. Modified the IP and Port business types to populate the error message field in the response if an error occurs

* [CLDN-1205] Added meaningful error message for port translation errors

* Removing status field from businesstype Response and adding in error message

* [CLDN-1205] Added check to make sure the port business type is within the valid range of ports, if it is not, it will populate the error message

* [CLDN-1205] Fixed the if statement that checks to see if the string_value is in the valid range of port numbers. It did not corrently verify this before now.

* [CLDN-1205] Fixed an error where it was trying to use string_value in <= statements. I just casted string_value to an integer if it is numeric, which allows <= operators to be used on it

* [CLDN-1207] Added unit tests for the cidr_func and port_translation_func functions which are located in /superset/config.py

* [CLDN-1207] removed the assertRaises line as it does not work with the cidr_func and port_translation_func functions

* [CLDN-1207] Added the skeleton of the test_cidr_translate_filter_func unit test, still need to update what the expected response from the function will be.

* [CLDN-1207] Added the remainder of the back-end unit tests for the business types

* [CLDN-1207] Fixed the syntax error which caused the test_cidr_translate_filter_func_NOT_IN_double unit test to fail

* [CLDN-1207] Removed the logging that was added for debugging purposes

* [CLDN-1207] Formatted the commands_tests.py file to make it nicer to look at/read through

* [CLDN-1207] Fixed the code so that it conformed to the pylint requirements (i.e., pylint no longer complains about the code in commands_tests.py)

* [CLDN-1207] Modified some of the docstrings so they made better use of the 100 character per line, line limit

* [CLDN-1207] Added the beginnings of the unit tests for the
business types API

* [CLDN-1207] Added a comment to the top of the commands_tests.py file explaining how to run the unit tests. This prevents the next person who tries to run them from having to waste time trying the different forms of testing that Superset supports (e.g., pytest, tox, etc.)

* [CLDN-1207] Added a grammar fix to the comments describing how to run the unit tests

* [CLDN-1207] Modified the description of the business_type API endpoints as they did not represent what the API was actually doing

* [CLDN-1207] Added further instructions on how to run the unit tests that are within the business_type/api_tests.py file

* add request validation

* disable request if business type missing

* [CLDN-1207] Unit tests for the business type API are now working, however, they need to be modified to make use of @mock as we don't want to have to run the server to be able to run the unit tests

* Removing businesss types deffinitons from config

* Adding select to only show valid business types

* Fixed Enzyme tests

* Added scalfolding for selecting filter dropdown

* Adding intigration tests

* fix revision

* fix typos and unnecessary requests

* break out useBusinessTypes

* Added front-end RTL unit tests for the business type API endpoint

* Fixed error from unit tests

* Added a unit test to ensure the operator list is updated after a business type API response is received

* Removing elect compoenet for business types

* Adding feature flag and allowing saving when no business type present

* fixing useEffect hooks

* Adding feature flag to model

* Changing behavior such that an empty string returns a default response

* add form validation

* Modified comments in unit test as command to run test has changed

* Modified comments in unit test as filename to run test has changed

* Modified the api_tests.py file to conform to the linting requirements

* Changed the name of one of the tests to reflect what the test is actually testing

* Added cypress back to the package.json

* Added informative comments

* Updated comments in files as well as removed imports which were not being used

* Changes made by npm run prettier

* Fixed spelling mistakes

* Updated models.py to remove placeholder comments used in development

* Added feature flag mocking in unit test

* Fixing open api failure

* Fixing business types to pass unit tests

* Reverting unsafe connections back to false

* Removing print statement

* Adding business tpye to export test

* setting default feature flag to false for business type

* Reverting pre commit

* Reverting pre commit and running pre commit

* Reverting pre commit and running pre commit

* Fixing formatting

* Adding license

* Fixing Linting

* Protecting api enpoints

* updating model

* Fixing code path when business type exists

* Linting

* Linting

* Fixing linting

* Fixing spelling

* Fixing schemas

* Fixing app import

* fixing item render

* Added RTL test to make sure business type operator list is updated after API response

* Fixing linting

* fix migration

* Changing unit tests

* Fixing import and DB migration after rebase

* Renaming to advanced types

* Fixing Linting

* More renaming

* Removing uneeded change

* Fixing linting and test errors

* Removing unused imports

* linting

* Adding more detailed name for migration

* Moving files to plugins

* more renaming

* Fixing schema name

* Disabling feature flag that should not be enabled by default

* Adding extra cehck

* NameChange

* formatting

* Fixing equals check

* Moveing all advanced type classes and types to one file, and converting tests to functional

* Adding advanced type to test and fix linitng

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Dan Parent <daniel.parent@cse-cst.gc.ca>
Co-authored-by: GITHUB_USERNAME <EMAIL>
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
2022-05-16 17:58:21 +03:00
Ville Brofeldt
4435e53901 fix(dataset): handle missing sqla uri in migration (#20073) 2022-05-16 10:17:28 +03:00
John Bodley
89a844a40f fix(reports): Clear last value when state is WORKING (#19941)
* fix(reports): Clear last value when state is WORKING

* Update cbe71abde154_fix_report_schedule_and_log.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-05-12 21:54:23 -07:00
Josh Soref
c32c505742 chore(docs): Spelling (#19675)
* spelling: adding

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: aggregate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: avoid

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: blacklist

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cached

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: discontinue

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exhaustive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: from

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: github

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: hybrid

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: implicit

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: interim

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: introduced

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: javascript

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: logstash

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: metadata

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: password

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: recommended

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: redshift

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: refactored

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: referencing

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: sqlite

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: the

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: undoes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: very

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-04-26 13:17:15 -03:00
Ville Brofeldt
a1bd5b283c fix(key_value): use longblob on mysql (#19805)
* fix(key_value): use longblob on mysql

* set length
2022-04-22 13:12:29 +03:00
Jesse Yang
fa680369ea fix(SIP-68): handle empty table name during migration (#19793) 2022-04-21 10:49:25 -07:00
Cemre Mengu
9c20299039 fix(migrations): sl_columns is_temporal mapping (#19786) 2022-04-20 07:31:40 -07:00
Jesse Yang
231716cb50 perf: refactor SIP-68 db migrations with INSERT SELECT FROM (#19421) 2022-04-19 18:58:18 -07:00
Hugh A. Miles II
7e92340c70 fix: Fix migration for removing time_range_endpoints 3 (#19767)
* fix migration

* so dumb

* update test

* add code change

* retest
2022-04-19 09:49:50 -07:00
Hugh A. Miles II
86642025ba fix: Removetime_range_endpoints from query context object pt 2 (#19728) 2022-04-15 13:17:05 -04:00
Jesse Yang
a975af3e9e chore: clean up unused imports in db migration scripts (#19630) 2022-04-09 07:48:59 -07:00
Ville Brofeldt
d9343a4639 fix(dataset): handle missing python_type gracefully (#19553)
* fix(dataset): handle missing python_type gracefully

* refactor TEMPORAL_TYPES
2022-04-07 14:30:11 +03:00
Beto Dealmeida
6136942759 fix: sqloxide optional (#19570) 2022-04-06 16:11:38 -07:00
Elizabeth Thompson
444b4f5832 merge multiple db heads (#19577) 2022-04-06 15:14:39 -07:00
Diego Medina
350f21d9d3 fix: cannot delete a database if team member has SQL editor tab that uses that db (#19243)
* fix: update sql lab models constraints to support db deletion

* update with master

* solve db migration conflict

* fix tests
2022-04-06 08:46:21 -07:00
Ville Brofeldt
e1ef2baad7 fix(migrations): reorder skipped 1.4 migrations (#19543) 2022-04-06 10:47:12 +03:00
Beto Dealmeida
1b4d8ddf71 chore: blacklist unsafe functions (#19537) 2022-04-05 14:55:30 -07:00
Ville Brofeldt
1eef923b31 chore: postpone timerange endpoint removal (#19513) 2022-04-05 10:10:29 +03:00
Jesse Yang
90dbe8d340 perf: speed up db migration for deprecating time_range_endpoints (#19495) 2022-04-01 16:44:41 -07:00
serenajiang
7f85da6e5b fix: handle null params in #18936 migration (#19474) 2022-03-31 12:47:02 -07:00
Hugh A. Miles II
129063d544 fix: Removetime_range_endpoints from query context object (#19423)
* template for remove time_range_endpoints from query context

* Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py

Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>

* Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py

Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>

* fix reference

* fix column reference

* fix

* Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2022-03-30 17:26:10 -07:00
Beto Dealmeida
63b5e2e4fa perf: improve perf in SIP-68 migration (#19416)
* chore: improve perf in SIP-68 migration

* Small fixes

* Create tables referenced in SQL

* Update logic in SqlaTable as well

* Fix unit tests
2022-03-29 22:33:15 -07:00
Ville Brofeldt
a619cb4ea9 chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
Diego Medina
aa5c80bda6 fix(sql lab): deleting the last saved query or the last executed from history (#19225)
* fix: fix issue when deleting the last saved query or the last executed query

* merge migration
2022-03-17 15:01:18 -06:00
Ville Brofeldt
b7a0559aaf feat: add permalink to dashboard and explore (#19078)
* rename key_value to temporary_cache

* add migration

* create new key_value package

* add commands

* lots of new stuff

* fix schema reference

* remove redundant filter state from bootstrap data

* add missing license headers

* fix pylint

* fix dashboard permalink access

* use valid json mocks for filter state tests

* fix temporary cache tests

* add anchors to dashboard state

* lint

* fix util test

* fix url shortlink button tests

* remove legacy shortner

* remove unused imports

* fix js tests

* fix test

* add native filter state to anchor link

* add UPDATING.md section

* address comments

* address comments

* lint

* fix test

* add utils tests + other test stubs

* add key_value integration tests

* add filter box state to permalink state

* fully support persisting url parameters

* lint, add redirects and a few integration tests

* fix test + clean up trailing comma

* fix anchor bug

* change value to LargeBinary to support persisting binary values

* fix urlParams type and simplify urlencode

* lint

* add optional entry expiration

* fix incorrect chart id + add test
2022-03-17 01:15:52 +02:00
Kamil Gabryjelski
9f834e8317 fix(plugin-chart-echarts): Apply temporary filters to Query B in explore (#18998)
* fix(explore): Extra filters not applied to query b in mixed timeseries

* Add return type

* Apply review comment

* Fix non-dnd filters
2022-03-04 17:48:41 +01:00
Beto Dealmeida
50bb86d666 fix: remove unneeded complexity in migration (#19022) 2022-03-03 16:56:38 -08:00
John Bodley
26486d01c1 chore: Remove legacy SIP-15 interim logic/flags (#18936)
* chore: Remove legacy SIP-15 logic

* Update ab9a9d86e695_deprecate_time_range_endpoints.py

* Update UPDATING.md

* Update UPDATING.md

* Update UPDATING.md

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-03-04 09:15:36 +13:00
John Bodley
19eb73b316 fix(sqllab): Replace stringified 'null' schema column values with NULL (#18992)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-03-03 07:12:52 +13:00
Ville Brofeldt
2baceddbf1 fix(dataset): handle missing database in migration (#18948) 2022-02-25 20:50:55 +02:00
Beto Dealmeida
00c99c91ec feat: new dataset/table/column models (#17543)
* feat: new models for SIP-68

* feat: new dataset models DB migration

* Add shadow write (WIP)

* Physical dataset shadow write (WIP)

* Virtual dataset shadow write (WIP)

* Update migration to populate models

* Cascade delete columns

* after_delete

* Update hook

* Add metric test

* Do not rename tables

* Small fixes

* Fix migration

* Fix tests

* Fix more tests

* Even more tests

* Addressing comments (WIP)

* Fix migration

* Rebase and update head

* Fix unit tests

* Add external management cols

* Small fixes
2022-02-24 11:02:01 -08:00
Lily Kuang
1fbdabd2cf feat: embedded dashboard (#18561)
* add embedded dashboard

* json

* lint

* change json to text

* uuid
2022-02-08 00:18:52 -08:00
Beto Dealmeida
1c0b141e2d feat: allow assets to be managed externally (#18093)
* feat: allow assets to be managed externally

* Use server_default
2022-01-24 11:03:01 -08:00
AAfghahi
5e84fd19a9 refactor: Migration for Tabbed Sql Editor (#17925)
* a lot of console logs

* migration for tabbed Sql Editors
2022-01-12 12:25:24 -05:00
Beto Dealmeida
4479fc9222 fix: force_screenshot migration (#17928) 2022-01-04 18:17:18 -08:00
Beto Dealmeida
2cd8054358 feat: add force option to report screenshots (#17853)
* Update existing tests

* Add backend test

* feat: add force option to report screenshots

* Add tests

* Rebase fixes

* Do not force screenshot on dashboard alerts
2021-12-22 12:16:04 -08:00
Kamil Gabryjelski
2c3f39f3f2 feat(plugin-chart-pivot-table): support series limit (#17803)
* feat(plugin-chart-pivot-table): support series limit

* Add a migration

* Use non-legacy series limit controls

* Add a todo comment

* Bug fix
2021-12-21 16:58:54 +01:00
Craig Rueda
b3db6140c8 fix(alembic): Making Alembic logger config optional (#17774)
* Making Alembic logger config optional

* Switching to app config
2021-12-15 16:16:48 -08:00
Kamil Gabryjelski
124af4c566 chore(explore): Migrate BigNumber to v1 api [ID-28][ID-55] (#17587)
* chore(explore): Migrate BigNumber to v1 api

* Move to echarts

* Use Echarts trendline

* Fix imports

* Fix parsing dates as strings

* Add from_dttm and to_dttm to v1 chart response

* Fix post processing

* Fix timeRangeFixed

* Fix tests

* Remove from and to dttm from cache

* Cleanup date formatting

* Fix storybook

* Fix missing types

* Fix timestamp with timezone

* Add types to demo's tsconfig

* bug fix

* fix import

* Fix cypress tests

* add sort

* add resample to handle missing values properly

* Sync ChartDataResponseResult schema with ts interface

* Lint fix

* Add migration

* Fix migration

* Remove pass

* Re-raise the exception in migration

* Typo fix

* Update revision

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2021-12-15 10:15:14 +01:00
cccs-joel
e6db62c469 fix: Change datatype of column type in BaseColumn to allow larger datatype names for complexed columns (#17360)
* Change datatype of column type in BaseColumn to allow larger datatype names for complexed columns

* Fixed formatting

* Added an entry in the UPDATING.md file as a potential downtime

* Update UPDATING.md

Accept proposed changes.

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Updated down revision number to reflect new head

Co-authored-by: cccs-joel <cccs-joel@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-12-13 14:28:21 -08:00
Beto Dealmeida
0d2299cb60 fix: migration out-of-scope bind (#17728)
* fix: migration out-of-scope bind

* Bypass flaky test
2021-12-13 13:04:35 -08:00
Ajay M
1377465a7d feat(alerts): Column for select tabs to send (#17604)
* init

* init

* Modify migration

* Modify migration

* revert config, delete test file

* Rename column, modify migration

* rename a column

* Make extra non-nullable

* Fix typo
2021-12-03 10:28:25 -08:00
Geido
83e49fc9ee feat: Certify Charts and Dashboards (#17335)
* Certify charts

* Format

* Certify dashboards

* Format

* Refactor card certification

* Clear details when certified by empty

* Show certification in detail page

* Add RTL tests

* Test charts api

* Enhance integration tests

* Lint

* Fix dashboards count

* Format

* Handle empty value

* Handle empty slice

* Downgrade migration

* Indent

* Use alter

* Fix revision

* Fix revision
2021-11-24 13:42:52 +02:00
Shiva Raisinghani
0ca4312212 fix: rename to schemas_allowed_for_file_upload in dbs.extra (#17323)
* rename to schemas_allowed_for_file_upload in dbs.extra

* black

* I should really setup pre-commit hooks

* Apply suggestions

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>

* move changes to a seperate migration

* fix spaces

* black

Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>
2021-11-14 09:57:09 -08:00
Beto Dealmeida
8ee50deac6 Drop constraint (#17275) 2021-10-29 09:33:25 -07:00
Shiva Raisinghani
ef3afbde82 refactor: Repeated boilerplate code between upload to database forms (#16756)
* abstract boilerplate code into class and rename csv to file

* add db migration

* fix some stuff

* more renaming of csv to file

* rename in translations

* update down revision

* update down revision

* bump chart version

* switch to alter column name approach in db migration

* fix db migration for MySQL

* db migration conflict
2021-10-25 13:53:06 +03:00
John Bodley
b9ff85d9ea chore(engine): Translate fractional time grains—requires @superset-ui bump (#17078)
* chore(engine): Translate fractional time grains

* Bump @superset-ui

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-10-12 21:40:48 -07:00
Yongjie Zhao
bbe7aa3fc3 chore: db migrate timeseries_limit_metric to legacy_order_by in pivottable2 (#16849) 2021-09-27 16:25:51 +08:00