* WIP - Add easily interpretable negative timedelta formatting
* Add unit tests for utils.core.timedelta_f
* Reformat tests.utils_tests.test_timedelta_f test code. Fixes#8274.
* Add type annotations to utils.core.timedelta_f (#8274)
* Replace recursion with absolute value in utils.core.timedelta_f (#8274)
* Rename utils.core.timedelta_f to format_timedelta (#8274)
* Fix small typo (#8274)
* Add support for msgpack results_backend serialization
* Serialize DataFrame with PyArrow rather than JSON
* Adjust dependencies, de-lint
* Add tests for (de)serialization methods
* Add MessagePack config info to Installation docs
* Enable msgpack/arrow serialization by default
* [Fix] Prevent msgpack serialization on synchronous queries
* Add type annotations
* 7620: Start removing dependencies on requests
* Patch urllib.request.urlopen instead of requests.get
* Try to fix flake8
* More work on flake8 import errors
* First attempt at using urllib with cookies
* Fix pylint/flake8
* Fix test_deliver_slice_csv_attachment
* Fix test_deliver_slice_csv_inline
* Import requests and pydruid conditionally, remove dependency on prison
* Fix flake errors
* Fix load_examples
* Please flake
* Skip tests depending on optional deps
* Try to please flake
* Address review comments
* Remove Druid-related UI
* Revert "Remove Druid-related UI"
This reverts commit d7e0f166cc3f3dd2496b4a666e177f0c191aeb0f.
* Skip a few tests more
* Put imports in right order
* Apply black patch
* Please flake
* Please black, silence flake
* Use flake8 silencing the right way
* Add deps for CI
* Allow to pass user for dashboard import cli
Dashboard import assign current user from flask context during import.
But in case of cli import there is no flask user and imported charts
don't have an owner which prevents ability to edit them.
* Reset ownership on dashboard import
For overriding existing charts it requires `owners` property to be set.
* Add tests for reset ownership
* Use ORM to decode dashboards json
Creating instances using ORM allows to normally work with relations
* Fix test_import_dashboard_1_slice test
Previously tests used side-effect of slices import which kept id from
json on insert into db.
* [WiP] improve load_examples
related to #7472, longer term we will generate the examples by exporting
them into tarball as in #7472. In the meantime, we need this subset of
the features:
* allowing specifying an alternate database connection for examples
* allowing a --only-metadata flag to `load_examples` to load only
dashboard and chart definitions, no actual data is loaded
* Improve logging
* Rename data->examples
* Load only if not exist
* By default do not load, add a force flag
* fix build
* set published to true
* [log] New, make action log configurable and generic
* [log] Fix, missing apache license
* [log] Fix, user_id is a required parameter on event logs
* [log] Fix, Rename Action to Event
* [log] Fix, flake8
* [logger] Change all log_this decorators to new abstract one
* [logger] [docs] Simple docs to show how to override the event log
* [style] Fix, single quote to double quote
* [style] Fix, single quote to double quote
* Move schema name handling in table names from frontend to backend
* Rename all_schema_names to get_all_schema_names
* Fix js errors
* Fix additional js linting errors
* Refactor datasource getters and fix linting errors
* Update js unit tests
* Add python unit test for get_table_names method
* Add python unit test for get_table_names method
* Fix js linting error
* Bugfix to SQL Lab to support tables and schemas with characters that require quoting
* Remove debugging prints
* Add uri encoding to secondary tables call
* Quote schema names for presto
* Quote selected_schema on Snowflake, MySQL and Hive
* Remove redundant parens
* Add python unit tests
* Add js unit test
* Fix flake8 linting error
* Bump FAB to 2.0.0
* [tests] whitelist SecurityApi login and refresh endpoints
* [style] Fix, C812 missing trailing commas
* [security] Remove SUPERSET_UPDATE_PERMS flag
Registering sources needs to be performed after the views are
initialized on UPDATE_PERMS=False configuration
* [docs] New, FAB_UPDATE_PERMS and flask fab cli
* [docs] Fix, db upgrade needs to come first, create-admin needs a db
* [cli] New, superset init bootstraps all permissions for FAB and Superset
* [style] Fix, flakes
* Sparkline dates aren't formatting in Time Series Table (#6976)
* Exclude venv for python linter to ignore
* Fix NaN error
* Fix the white background shown in SQL editor on drag (#7021)
This PR sets the background-color css property on `.ace_scroller` instead of `.ace_content` to prevent the white background shown during resizing of the SQL editor before drag ends.
* Show tooltip with time frame (#6979)
* Fix time filter control (#6978)
* Enhancement of query context and object. (#6962)
* added more functionalities for query context and object.
* fixed cache logic
* added default value for groupby
* updated comments and removed print
(cherry picked from commit d5b9795f87)
* [fix] /superset/slice/id url is too long (#6989)
(cherry picked from commit 6a4d507ab6)
* [WIP] fix user specified JSON metadata not updating dashboard on refresh (#7027)
(cherry picked from commit cc58f0e661)
* feat: add ability to change font size in big number (#7003)
* Add ability to change font sizes in Big Number
* rename big number to header
* Add comment to clarify font size values
* Allow LIMIT to be specified in parameters (#7052)
* [fix] Cursor jumping when editing chart and dashboard titles (#7038)
(cherry picked from commit fc1770f7b7)
* Changing time table viz to pass formatTime a date (#7020)
(cherry picked from commit 7f3c145b1f)
* [db-engine-spec] Aligning Hive/Presto partition logic (#7007)
(cherry picked from commit 05be866117)
* [fix] explore chart from dashboard missed slice title (#7046)
(cherry picked from commit a6d48d4052)
* fix inaccurate data calculation with adata rolling and contribution (#7035)
(cherry picked from commit 0782e831cd)
* Adding warning message for sqllab save query (#7028)
(cherry picked from commit ead3d48133)
* [datasource] Ensuring consistent behavior of datasource editing/saving. (#7037)
* Update datasource.py
* Update datasource.py
(cherry picked from commit c771625f10)
* [csv-upload] Fixing message encoding (#6971)
(cherry picked from commit 48431ab5b9)
* [sql-parse] Fixing LIMIT exceptions (#6963)
(cherry picked from commit 3e076cb60b)
* Adding custom control overrides (#6956)
* Adding extraOverrides to line chart
* Updating extraOverrides to fit with more cases
* Moving extraOverrides to index.js
* Removing webpack-merge in package.json
* Fixing metrics control clearing metric
(cherry picked from commit e6194051f4)
* [sqlparse] Fixing table name extraction for ill-defined query (#7029)
(cherry picked from commit 07c340cf82)
* [missing values] Removing replacing missing values (#4905)
(cherry picked from commit 61add606ca)
* [SQL Lab] Improved query and results tabs rendering reliability (#7082)
closes#7080
(cherry picked from commit 9b58e9f492)
* Fix filter_box migration PR #6523 (#7066)
* Fix filter_box migration PR #6523
* Fix druid-related bug
(cherry picked from commit b210742ad2)
* SQL editor layout makeover (#7102)
This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the center of the page across the board
(cherry picked from commit 71f1bbd2ec)
* [forms] Fix handling of NULLs
(cherry picked from commit e83a07d3df)
* handle null column_name in sqla and druid models
(cherry picked from commit 2ff721ae07)
* Use metric name instead of metric in filter box (#7106)
(cherry picked from commit 003364e74e)
* Bump python lib croniter to an existing version (#7132)
Package maintainers should really never delete packages, but it appears
this happened with croniter and resulted in breaking our builds.
This PR bumps to a more recent existing version of the library
(cherry picked from commit 215ed392a1)
* Revert PR #6933 (#7162)
* Add decorator for etag cache
* Fetch charts with GET
* Small fixes
* Fix typo
* Compute correct cache key; fix logging
* Check perms on cached response
* Revert change
* If perms fail, return naked response
* Fix lint
* Compute cache key from all form data
* Pass extra_filters in GET request
* Fix pylint
* Fix flake8
* Use ETags even if no cache is set
* Handle adhoc filters
* Raise in debug mode
* Rename actions
* Fix integration tests
* Do POST request on new charts
* Set extra/adhoc filters only in GET requests
* Raise if check_perms fails
* Refactor auth
* Fix flake8
* Fix js unit tests
* Fix js unit tests that fail in lyftga
* Fix js
* Sparkline dates aren't formatting in Time Series Table (#6976)
* Exclude venv for python linter to ignore
* Fix NaN error
* Changing time table viz to pass formatTime a date (#7020)
(cherry picked from commit 7f3c145b1f)
* SQL editor layout makeover (#7102)
This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the center of the page across the board
(cherry picked from commit 71f1bbd2ec)
* Add decorator for etag cache
* Fetch charts with GET
* Small fixes
* Fix typo
* Compute correct cache key; fix logging
* Check perms on cached response
* Revert change
* If perms fail, return naked response
* Fix lint
* Compute cache key from all form data
* Pass extra_filters in GET request
* Fix pylint
* Fix flake8
* Use ETags even if no cache is set
* Handle adhoc filters
* Raise in debug mode
* Rename actions
* Fix integration tests
* Do POST request on new charts
* Set extra/adhoc filters only in GET requests
* Raise if check_perms fails
* Refactor auth
* Fix flake8
* Fix js unit tests
* Fix js unit tests that fail in lyftga
* Fix js
* Fix bad merge
* Use far future when max_age=0
Introduce a config DEFAULT_RELATIVE_END_TIME which is used when
computing the "Last X days". The default behavior (as currently) is to
let that be 'today', which actually means the 0th second of today. We
can also let it be 'now' which means the data as of now (ie at query
time).
Secondly, also introduce another config IS_EPOCH_S_TRULY_UTC, which
means that the logged time in epoch_s format is actually in UTC.
Currently (as the default) is that it it is in the local (superset
webserver) timezone.
** There is no backward incompatibility with thes config features since
the default behavior hasn't changed. **
* Add licenses
* More licenses
* Ignore messages.json as they are generated
* More licenses
* Also typescript
* Fix alignment
* Add to svg
* Many more licenses
* more licenses
* Better excludes
* Add licenses to docs and md files
* Pre-finalize licenses
* Fix newlines
* Finalize all sourde licenses
* Fix lint
* Making thrift, pyhive and tableschema as extra_requires
Looking at the dependency tree for license related questions, I noticed
that tableschema had a huge tree, and only people running Hive really
need it. Making this as well as pyhive and thrift optional.
Also bumping some python dependencies
* Run pip-compile
* Removing refs to past.builtins (from future lib)
* Add thrift
* [scheduled reports] Add support for scheduled reports
* Scheduled email reports for slice and dashboard visualization
(attachment or inline)
* Scheduled email reports for slice data (CSV attachment on inline table)
* Each schedule has a list of recipients (all of them can receive a single mail,
or separate mails)
* All outgoing mails can have a mandatory bcc - for audit purposes.
* Each dashboard/slice can have multiple schedules.
In addition, this PR also makes a few minor improvements to the celery
infrastructure.
* Create a common celery app
* Added more celery annotations for the tasks
* Introduced celery beat
* Update docs about concurrency / pools
* [scheduled reports] - Debug mode for scheduled emails
* [scheduled reports] - Ability to send test mails
* [scheduled reports] - Test email functionality - minor improvements
* [scheduled reports] - Rebase with master. Minor fixes
* [scheduled reports] - Add warning messages
* [scheduled reports] - flake8
* [scheduled reports] - fix rebase
* [scheduled reports] - fix rebase
* [scheduled reports] - fix flake8
* [scheduled reports] Rebase in prep for merge
* Fixed alembic tree after rebase
* Updated requirements to latest version of packages (and tested)
* Removed py2 stuff
* [scheduled reports] - fix flake8
* [scheduled reports] - address review comments
* [scheduled reports] - rebase with master
* Deprecate database attribute allow_run_sync
There's really 2 modes of operations in SQL Lab, sync or async
though currently there are 2 boolean flags: allow_run_sync and
allow_run_async, leading to 4 potential combinations, only 2 of which
actually makes sense.
The original vision is that we'd expose the choice to users and they
would decide which `Run` or `Run Async` button to hit.
Later on we decided to have a
single button and for each database to be either sync or async.
This PR cleans up allow_run_sync by removing references to it.
* Fix build
* Add db migration
* using batch_op
* [SIP-5] Open a new /api/v1/query endpoint that takes query_obj
- Introduce a new handle_superset_exception decorator to avoid repeating the logic for catching SupersetExceptions
- Create a query_obj_backfill method that takes form_data and constructs a query_obj that will be constructed in the client in the future. Use the backfill in explore_json.
- Create a new /api/v1/query endpoint that takes query_obj only and returns the payload data. Note the query_obj is constructed in the client. The endpoint currently only handles query_obj for table view viz (we'll be adding support to new viz types as we go).
- Unit test to verify the new endpoint for table view
* fix tests and lint errors
* - Move the new query endpoint into its own api.py view.
- Create QueryObject and QueryContext class to encapsulate query_object to be built from the client and additional info (e.g. datasource) needed to get the data payload for a given query
- Remove the query_obj_backfill as we'll start building the first query_object on the client so it no longer makes sense to have a short-lived backfill for the matter of days.
* Fixing lint and test errors
* Fixing additional lint error from the previous rebase.
* fixing additional lint error
* addressing additional pr comments
* Make /query accept a list of queries in the query_context object.
* fixing a lint error
* - Move time_shift based calculation and since, until check into util
- Add typing info for get_since_until
- Add new unit tests to verify time_shift calculation and the since until check
* [utils] gathering/refactoring into a "utils/" folder
Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.
Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.
* Improve tests
* Make loading examples in scope for tests
* Remove test class attrs examples_loaded and requires_examples
