Commit Graph

46 Commits

Author SHA1 Message Date
mapledan
c2ab9bba29 fix: change the validation logic for python_date_format (#25510)
Co-authored-by: John Bodley <john.bodley@gmail.com>
2024-01-24 10:44:36 +13:00
Beto Dealmeida
3e63c82ecc feat: generic marshmallow error component (#25303) 2023-10-03 11:35:28 -07:00
Stepan
14c324950a feat(datasource): Checkbox for always filtering main dttm in datasource (#25204)
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2023-09-14 15:57:08 -07:00
Ville Brofeldt
f94dc49fd0 fix(snowflake): opt-in denormalization of column names (#24982) 2023-08-15 16:32:54 -07:00
Sebastian Liebscher
14a27b1ba3 chore(api): clean up API spec (#24838) 2023-07-31 20:03:58 +03:00
EugeneTorap
fc89718d48 chore: Update pylint to 2.17.4 (#24700)
Co-authored-by: John Bodley <john.bodley@gmail.com>
2023-07-24 17:13:49 -07:00
Vitor Avila
65fb8e10ba fix(dataset-import): support empty strings for extra fields (#24663) 2023-07-13 16:19:00 -07:00
Kamil Gabryjelski
83ff4cd86a feat: Implement currencies formatter for saved metrics (#24517) 2023-06-28 20:51:40 +02:00
Jack Fragassi
5af298e1f6 chore: Migrate warm up cache endpoint to api v1 (#23853) 2023-06-20 12:08:29 +01:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
Ville Brofeldt
8375baac99 fix(dataset-api): allow undefined schema in post (#24242) 2023-05-30 17:48:54 +03:00
Jack Fragassi
2b236cbbd6 fix: Errors when saving dataset (#24113) 2023-05-22 13:21:30 -04:00
Stepan
742c2cd8fb fix(datasource): saving new calculated columns (#23783) 2023-04-24 12:56:30 -07:00
Sebastian Liebscher
b31efbae7c chore: remove RemovedInMarshmallow4 warnings (#23704) 2023-04-21 09:11:46 +03:00
Jack Fragassi
44557f5a23 chore(api v1): Deprecate datasource/save and datasource/get endpoints (#23678) 2023-04-18 17:51:24 -07:00
Jack Fragassi
3fae7dbc34 chore: Migrate get_or_create_table endpoint to api v1 (#22931) 2023-02-15 10:38:51 +00:00
Shubham Sinha
decb2fb591 feat(datasets): Change in API to create virtual datasets (#21840) 2022-10-19 15:05:21 +01:00
Reese
16032ed3e2 feat: Virtual dataset duplication (#20309)
* Inital duplicate functionality

* Fix formatting

* Create dedicated duplicate API

* Make use of new API

* Make use of new api permissions

* Add integration tests for duplicating datasets

* Add licenses

* Fix linting errors

* Change confirm button to 'Duplicate'

* Fix HTTP status code and response

* Add missing import

* Use user id instead of user object

* Remove stray debug print

* Fix sqlite tests

* Specify type of extra

* Add frontend tests

* Add match statement to test
2022-08-26 16:07:56 -06:00
Diego Medina
6ce027d2f8 fix(import): error with disable_data_preview field (#20932) 2022-08-02 14:16:45 -06:00
cccs-RyanS
ddc01ea781 feat(business-types): initial implementation of SIP-78 (#18794)
* add BUSINESS_TYPE_ADDONS to config with example callback

* Removing uneeded whitespace

* [Work in progress] Modifying cidr function to allow for  single ip and adding port outline

* Added test REST endpoint, added some more ports

I've thrown in a test.py script as well that will try to connect to the
business_type endpoint.

* Moving code from config.py into the business api

Very simple api is exposed that will allow someone to call a checkport
endpoint and get back a response.

* Removing commented out bits.

* Adding fucntion dict back to the config

* Moving business_type endpoint to charts

* Adding schema for get endpoint

* Removing imports, updating docstring, fixing typo

Just some small changes as described in the title.  I've updated the
test.py as well so it functions with the endpoint changes.

* Adding translation dict

* Fixing ops

* Adding check for list

* Modifying changes to add quotes where needed

Also changed BusinessTypeResponse to resp.

* Adding in some code to call the filter config

If a column starts with "cidr_" it will call the code in config.py to
try to translate the filter.  Nothing is changed in the JSON being
executed, just some information is dumped to console.

* Porting Ryan's changes

* Adding migration script (as per Ryan's PR)

* Fixing typo

* Prettier fixes

* [CLDN-1043] Adding rough version of filter changes for business types

* fix down migration

* Fixing bugs after merge

* adding functionality to appy filters in back end

* Fixing linting issues

* fix down revision

* Changing conversion callback to handle multiple values at once

* Adding string representation of values

* Code cleanup plus fixing debouce to only be called once for each entry

* Removing non needed logginh

* Changing operator list to use sting values

* Using text value operators

* Removing clear operator call

* Moving business type endpoints

* fix down revision

* Adding port functions

* update migration

* fix bad rebase and add ff

* implement validator

* dont add invalid values to response

* [CLDN-1205] Added a new exception type for a business type translation error. Added the error message in the display_value field within the business type response. Modified the IP and Port business types to populate the error message field in the response if an error occurs

* [CLDN-1205] Added meaningful error message for port translation errors

* Removing status field from businesstype Response and adding in error message

* [CLDN-1205] Added check to make sure the port business type is within the valid range of ports, if it is not, it will populate the error message

* [CLDN-1205] Fixed the if statement that checks to see if the string_value is in the valid range of port numbers. It did not corrently verify this before now.

* [CLDN-1205] Fixed an error where it was trying to use string_value in <= statements. I just casted string_value to an integer if it is numeric, which allows <= operators to be used on it

* [CLDN-1207] Added unit tests for the cidr_func and port_translation_func functions which are located in /superset/config.py

* [CLDN-1207] removed the assertRaises line as it does not work with the cidr_func and port_translation_func functions

* [CLDN-1207] Added the skeleton of the test_cidr_translate_filter_func unit test, still need to update what the expected response from the function will be.

* [CLDN-1207] Added the remainder of the back-end unit tests for the business types

* [CLDN-1207] Fixed the syntax error which caused the test_cidr_translate_filter_func_NOT_IN_double unit test to fail

* [CLDN-1207] Removed the logging that was added for debugging purposes

* [CLDN-1207] Formatted the commands_tests.py file to make it nicer to look at/read through

* [CLDN-1207] Fixed the code so that it conformed to the pylint requirements (i.e., pylint no longer complains about the code in commands_tests.py)

* [CLDN-1207] Modified some of the docstrings so they made better use of the 100 character per line, line limit

* [CLDN-1207] Added the beginnings of the unit tests for the
business types API

* [CLDN-1207] Added a comment to the top of the commands_tests.py file explaining how to run the unit tests. This prevents the next person who tries to run them from having to waste time trying the different forms of testing that Superset supports (e.g., pytest, tox, etc.)

* [CLDN-1207] Added a grammar fix to the comments describing how to run the unit tests

* [CLDN-1207] Modified the description of the business_type API endpoints as they did not represent what the API was actually doing

* [CLDN-1207] Added further instructions on how to run the unit tests that are within the business_type/api_tests.py file

* add request validation

* disable request if business type missing

* [CLDN-1207] Unit tests for the business type API are now working, however, they need to be modified to make use of @mock as we don't want to have to run the server to be able to run the unit tests

* Removing businesss types deffinitons from config

* Adding select to only show valid business types

* Fixed Enzyme tests

* Added scalfolding for selecting filter dropdown

* Adding intigration tests

* fix revision

* fix typos and unnecessary requests

* break out useBusinessTypes

* Added front-end RTL unit tests for the business type API endpoint

* Fixed error from unit tests

* Added a unit test to ensure the operator list is updated after a business type API response is received

* Removing elect compoenet for business types

* Adding feature flag and allowing saving when no business type present

* fixing useEffect hooks

* Adding feature flag to model

* Changing behavior such that an empty string returns a default response

* add form validation

* Modified comments in unit test as command to run test has changed

* Modified comments in unit test as filename to run test has changed

* Modified the api_tests.py file to conform to the linting requirements

* Changed the name of one of the tests to reflect what the test is actually testing

* Added cypress back to the package.json

* Added informative comments

* Updated comments in files as well as removed imports which were not being used

* Changes made by npm run prettier

* Fixed spelling mistakes

* Updated models.py to remove placeholder comments used in development

* Added feature flag mocking in unit test

* Fixing open api failure

* Fixing business types to pass unit tests

* Reverting unsafe connections back to false

* Removing print statement

* Adding business tpye to export test

* setting default feature flag to false for business type

* Reverting pre commit

* Reverting pre commit and running pre commit

* Reverting pre commit and running pre commit

* Fixing formatting

* Adding license

* Fixing Linting

* Protecting api enpoints

* updating model

* Fixing code path when business type exists

* Linting

* Linting

* Fixing linting

* Fixing spelling

* Fixing schemas

* Fixing app import

* fixing item render

* Added RTL test to make sure business type operator list is updated after API response

* Fixing linting

* fix migration

* Changing unit tests

* Fixing import and DB migration after rebase

* Renaming to advanced types

* Fixing Linting

* More renaming

* Removing uneeded change

* Fixing linting and test errors

* Removing unused imports

* linting

* Adding more detailed name for migration

* Moving files to plugins

* more renaming

* Fixing schema name

* Disabling feature flag that should not be enabled by default

* Adding extra cehck

* NameChange

* formatting

* Fixing equals check

* Moveing all advanced type classes and types to one file, and converting tests to functional

* Adding advanced type to test and fix linitng

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Dan Parent <daniel.parent@cse-cst.gc.ca>
Co-authored-by: GITHUB_USERNAME <EMAIL>
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
2022-05-16 17:58:21 +03:00
Beto Dealmeida
c7f9060a2f feat: import external management columns (#19315)
* feat: import flags

* Add tests
2022-03-24 10:41:22 -07:00
Beto Dealmeida
9766726b26 feat: external management flags in CRUD (#19318) 2022-03-23 15:38:33 -07:00
Beto Dealmeida
00c99c91ec feat: new dataset/table/column models (#17543)
* feat: new models for SIP-68

* feat: new dataset models DB migration

* Add shadow write (WIP)

* Physical dataset shadow write (WIP)

* Virtual dataset shadow write (WIP)

* Update migration to populate models

* Cascade delete columns

* after_delete

* Update hook

* Add metric test

* Do not rename tables

* Small fixes

* Fix migration

* Fix tests

* Fix more tests

* Even more tests

* Addressing comments (WIP)

* Fix migration

* Rebase and update head

* Fix unit tests

* Add external management cols

* Small fixes
2022-02-24 11:02:01 -08:00
Beto Dealmeida
30c0459808 fix: extra column in metrics (#17784) 2021-12-22 10:39:29 -08:00
Beto Dealmeida
c49545aec1 fix: dataset extra import/export (#17740)
* fix: dataset extra import/export

* Update superset/datasets/commands/importers/v1/utils.py
2021-12-22 09:41:31 -08:00
Beto Dealmeida
37cc2c4d15 fix: column extra in import/export (#17738) 2021-12-14 18:33:52 -08:00
cccs-joel
e6db62c469 fix: Change datatype of column type in BaseColumn to allow larger datatype names for complexed columns (#17360)
* Change datatype of column type in BaseColumn to allow larger datatype names for complexed columns

* Fixed formatting

* Added an entry in the UPDATING.md file as a potential downtime

* Update UPDATING.md

Accept proposed changes.

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Updated down revision number to reflect new head

Co-authored-by: cccs-joel <cccs-joel@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-12-13 14:28:21 -08:00
Ville Brofeldt
0544bee74e fix(api): incorrect api schema definitions (#17620) 2021-12-01 21:47:22 +02:00
Phillip Kelley-Dotson
a198dbb19b feat: add certifiedby & certification details fields to the edit dataset columns fields (#16454)
* add migration

* add backend and frontend for certified

* update migration with batch

* fix integration test and update Updating.md

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* change method name

* add tooltip info

* add mixin

* merge heads

* address comments

* fix select label styles

* add extra field

* fix test?

* add extra field to put schema

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>
2021-09-22 15:09:30 -07:00
Beto Dealmeida
5e95d4638c fix: import dataset with extra; Vertica URI (#14698)
* fix: import dataset with extra; Vertica URI

* Fix lint
2021-05-18 20:51:33 -07:00
Beto Dealmeida
3acfd826de feat: add new dashboard with FCC data (#12163) 2020-12-23 14:54:50 -08:00
Beto Dealmeida
5e811a14ef feat: load examples from config instead of code (#12026)
* feat: load examples from config instead of code

* Remove database

* Update data URL
2020-12-15 13:44:23 -08:00
Beto Dealmeida
e0079bb5ae fix: small fixes to the new import/export (#12064) 2020-12-15 12:24:24 -08:00
Hugh A. Miles II
cc44a2c01b feat: Sqllab to Explore UX improvements (#11755)
* create boiler modal component

* hello world modal

* setup modal flow

* setup savemodal for components

* flake8

* fix onclick reference

* working create datasource boiler

* saving spot for callback on text input

* working dataset with input box

* working redirect on completion

* get data for owners dropdown

* fix build with pull from master

* fix the filteroption

* 💯

* move state to upper component

* add overwrite state

* hacked overwrite process

* linting

* fix filter

* cleaning up the coe

* Delete preset.code-workspace

* remove unused code

* remove visualize

* update default value

* remove unneeded vars

* checkout package-lock.json

* linting

* get user id

* remove page filter

* setup proper call for updating columns in dataset

* add move to explore flow

* linting

* add param for overriding columns

* linting

* change title

* fix some tings

* cleanup

* linting

* add types in some places

* save toast

* use moment

* add error toast

* create enum for radio states

* initial state for saving query

* add tpying

* addressing concerns

* update propTypes

* add functionality for CTAS explor btn

* handle CTAS state

* fix onclick to reference upper level component

* formatting

* reset state after closing

* add error message when user doesn't pick an already selected dataset

* remove unneeded todo

* remove styling

* move async calls to api directory

* remove console.log

* add user id param

* typing

* littty

* move put to seperate file

* save

* dsf

* fix typing errors

* adding more types

* fix typing erros

* linting

* add basic spec test

* create dataset modal

* add components reference

* Rename SaveDatasetModal_spec.jsx to SaveDatasetModal_spec.tsx

* remove sinon for now

* fix typing errors on modal files

* fix linting

* address comments

* attempt to fix linting

* add props

* fix test

* fix the linting

* yerp

* fix this references

* spaces

* handleOverwriteCancel reference cleanup

* rename bool value for shouldOverwriteDataset

* fix typing for onChange

* you still the best in the world

* fix spec

* align branches

* push

* fix key names

* fix dataset reference

* lowercase

* fix save bug with tiem

* fixed styling

* fix date state after push to explore

* add disabling states

* plz refactor this

* this is working fully now

* do some renaming

* renaming

* remove console.logs

* still refactoring

* remove unneeded code

* remove unneeded variables

* still cleaning

* added interface

* fix typing issues

* cleanup unused code

* fix npm lnit

* fix initial problems

* add props to test

* remove unneeded files

* skip linting

* saving

* this works

* remove old test

* remove old test

* fix linting

* fix broken test

* remove jsx file

* refactoring

* cleanup

* remove comments

* reset user object

* fix functions

* fix this

* reverting CTAS btn flow

* remove onclick

* save frontend work

* allow for database id to be passed as param in body

* use enum

* fix linting

* style alignment

* get rid of .then

* add function to compute default value with tiem

* lit

* remove ts-error

* fix typing
2020-12-08 17:29:41 -08:00
Beto Dealmeida
33325f9fa6 feat: add modal to import charts (#11956)
* WIP

* Splat props
2020-12-07 18:33:59 -08:00
Beto Dealmeida
fbb458fa8b feat: add modal to import datasets (#11910) 2020-12-07 16:20:25 -08:00
Beto Dealmeida
7bc353f8a8 feat: new import commands for dataset and databases (#11670)
* feat: commands for importing databases and datasets

* Refactor code
2020-11-16 17:11:20 -08:00
Erik Ritter
ca40877640 feat: add certifications to tables (#11450) 2020-10-30 11:28:01 -07:00
Daniel Vaz Gaspar
9e9dac68f7 feat(datasets): REST API bulk delete (#11237)
* feat(datasets): REST API bulk delete

* doc HTTP 400
2020-10-12 13:40:05 +01:00
Beto Dealmeida
9785667a0d feat: add UUID column to ImportMixin (#11098)
* Add UUID column to ImportMixin

* Fix default value

* Fix lint

* Fix order of downgrade

* Add logging when downgrade fails

* Migrate position_json to contain UUIDs, and add schedule tables

* Save UUID when adding charts to dashboard

* Fix heads

* Rename migration file

* Fix dashboard serialization

* Fix migration script with Postgres

* Fix unique contraint name

* Handle UUID when exporting dashboard

* Fix Dataset PUT

* Add UUID JSON serialization

* Fix tests

* Simplify logic

* Try binary=True
2020-10-07 09:00:55 -07:00
Daniel Vaz Gaspar
7ecff0e513 fix: datasets related objects, apply filter and openapi spec (#10252)
* fix, datasets related objects, apply filter and openapi spec

* fix, add 404 test

* fix, add 404 test

* fix test
2020-07-08 10:04:05 +01:00
Ville Brofeldt
980dd2fd41 pylint: accept specific 2 character names by default (#9460)
* lint: accept 2 letter names by default

* Address review comments

* Remove e and d from good-names
2020-04-08 20:32:26 +03:00
Daniel Vaz Gaspar
4be827544e [datasets] Add strict type annotation (#9437)
* [datasets] Add strict type annotation

* Fix refresh endpoint

* Improve logic on update
2020-04-07 16:22:57 +01:00
Daniel Vaz Gaspar
40bc1b33d2 [dataset] New, export API endpoint (#9367)
* [dataset] New, export API endpoint

* Fix, lint
2020-03-27 09:30:23 +00:00
Daniel Vaz Gaspar
ccb22dc976 [dataset] columns and metrics API (nested) (#9268)
* [dataset] columns and metrics API (nested)

* [dataset] tests and validation

* [datasets] Fix, revert list field name to database_name
2020-03-24 17:24:08 +00:00
Daniel Vaz Gaspar
52c59d6890 [datasets] new, API using command pattern (#9129)
* [datasets] new, API using command pattern

* [datasets] tests and improvements

* [datasets] lint

* [database] address comments

* [datasets] lint

* [datasets] Address PR comments

* [dataset] Fix, dataset expects a Dict now

* [dataset] lint and optional commits

* [dataset] mypy

* [dataset] Fix, license and parent class

* [dataset] Make CRUD DAO raise exceptions
2020-03-08 09:13:08 +00:00