* fix: SqlaColumn.type overflow on mysql
Hitting a new error when loading examples around a datatype that exceeds
the 32 chars limit on SqlaColumn.type
Type includes the 'COLLATE utf8mb4_general_ci' suffix which is too
verbose and not needed in that context.
* fix tests
* feat: rough check in for Presto rows and arrays
* fix: presto arrays
* fix: return selected and expanded columns
* fix: add helper methods and unit tests
* fix: only allow exploration of selected columns
* fix: address Beto's comments and add more unit tests
* Move schema name handling in table names from frontend to backend
* Rename all_schema_names to get_all_schema_names
* Fix js errors
* Fix additional js linting errors
* Refactor datasource getters and fix linting errors
* Update js unit tests
* Add python unit test for get_table_names method
* Add python unit test for get_table_names method
* Fix js linting error
* Bugfix to SQL Lab to support tables and schemas with characters that require quoting
* Remove debugging prints
* Add uri encoding to secondary tables call
* Quote schema names for presto
* Quote selected_schema on Snowflake, MySQL and Hive
* Remove redundant parens
* Add python unit tests
* Add js unit test
* Fix flake8 linting error
* Merge lastest from master into lyft-release-sp8 (#7405)
* filter out all nan series (#7313)
* improve not rich tooltip (#7345)
* Create issue_label_bot.yaml (#7341)
* fix: do not save colors without a color scheme (#7347)
* [wtforms] Strip leading/trailing whitespace (#7084)
* [schema] Updating the datasources schema (#5451)
* limit tables/views returned if schema is not provided (#7358)
* limit tables/views returned if schema is not provided
* fix typo
* improve code performance
* handle the case when table name or view name does not present a schema
* Add type anno (#7342)
* Updated local dev instructions to include missing step
* First pass at type annotations
* [schema] Updating the base column schema (#5452)
* Update 937d04c16b64_update_datasources.py (#7361)
* Feature flag for client cache (#7348)
* Feature flag for client cache
* Fix integration test
* Revert "Fix integration test"
This reverts commit 58434ab98a.
* Feature flag for client cache
* Fix integration tests
* Add feature flag to config.py
* Add another feature check
* Fix more integration tests
* Fix raw HTML in SliceAdder (#7338)
* remove backendSync.json (#7331)
* [bubbles] issue when using duplicated metrics (#7087)
* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)
* SUPERSET-8: Update text in docs copyright footer (#7360)
* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04
* SUPERSET-8: Extra text in docs copyright footer
* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)
* Store last selected dashboard in sessionStorage (#7181)
* Store last selected dashboard in sessionStorage
* Fix tests
* [schema] Updating the base metric schema (#5453)
* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)
* feat: see Presto row and array data types (#7391)
* feat: see Presto row and array data types
* fix: address PR comments
* fix: lint and build issues
* fix: add types
* Incorporate feedback from initial PR (prematurely merged to lyft-release-sp8) (#7415)
* add stronger type hints where possible
* fix: lint issues and add select_star func in Hive
* add missing pkg init
* fix: build issues
* fix: pylint issues
* fix: use logging instead of print
* feat: view presto row objects in data grid
* fix: address feedback
* fix: spacing
* Merge lastest from master into lyft-release-sp8 (#7405)
* filter out all nan series (#7313)
* improve not rich tooltip (#7345)
* Create issue_label_bot.yaml (#7341)
* fix: do not save colors without a color scheme (#7347)
* [wtforms] Strip leading/trailing whitespace (#7084)
* [schema] Updating the datasources schema (#5451)
* limit tables/views returned if schema is not provided (#7358)
* limit tables/views returned if schema is not provided
* fix typo
* improve code performance
* handle the case when table name or view name does not present a schema
* Add type anno (#7342)
* Updated local dev instructions to include missing step
* First pass at type annotations
* [schema] Updating the base column schema (#5452)
* Update 937d04c16b64_update_datasources.py (#7361)
* Feature flag for client cache (#7348)
* Feature flag for client cache
* Fix integration test
* Revert "Fix integration test"
This reverts commit 58434ab98a.
* Feature flag for client cache
* Fix integration tests
* Add feature flag to config.py
* Add another feature check
* Fix more integration tests
* Fix raw HTML in SliceAdder (#7338)
* remove backendSync.json (#7331)
* [bubbles] issue when using duplicated metrics (#7087)
* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)
* SUPERSET-8: Update text in docs copyright footer (#7360)
* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04
* SUPERSET-8: Extra text in docs copyright footer
* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)
* Store last selected dashboard in sessionStorage (#7181)
* Store last selected dashboard in sessionStorage
* Fix tests
* [schema] Updating the base metric schema (#5453)
* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)
* feat: see Presto row and array data types (#7391)
* feat: see Presto row and array data types
* fix: address PR comments
* fix: lint and build issues
* fix: add types
* add stronger type hints where possible
* fix: lint issues and add select_star func in Hive
* add missing pkg init
* fix: build issues
* fix: pylint issues
* fix: use logging instead of print
* Bump sqla to >=1.3.1
* Refine mssql column types to only use N-prefixing when necessary
* make join explicit
* replace set with list
* Add additional test case for N-prefix
* Replace engine with dialect and fix linting error
* Remove unneeded import
* Force quotes on non-expression time grains on postgres
* Change to or semantics
* Return lower case column name as unmutated
* Add testcases for postgres time grains
* Make expression MixedCase
* Merge dataframe and column name mutation logic, add flag for disabling column aliases and add column name length checking
* Remove custome mutate_label from oracle spec
* Move hashing from mutate_label() to make_label_compatible()
* Remove empty line
* Make label mutating and truncating more robust
* Rename variables and make proposed changes from review
* Always execute labels_expected codepath
* Fix linting error
* Add comments and fix subquery errors
* Refine column compatibility
* Simplify label assignment
* Add unit tests for BQ and Oracle
* Linting
Summary: Added limited support for visualizations with Pinot via
Sqlalchemy.
Pinot QL (PQL) is a bit weird and limited, and this patch hacks superset to
deal with that weirdness:
1. Pinot's grouping by time is best done as a long epoch. Grouping by a
time string is really slow and times out.
2. Pinot's response does not respect column aliases. So columns are not
named what they are expected to. So we remember the given column aliases
and then stamp them back onto the dataframe
3. Pinot's Json rest call does not return the output types. Instead
everything is cast to string. So when grouping by time, the group key
is integral and has to be treated specially when casting back to the
dataframe __timestamp column.
4. Finally, pinot does support grouping by on expressions. But those
expressions cannot then appear on the select clause. They are returned
regardless in the response. ie, 'select foo, count(*) from bar group by
foo' is okay, but 'select expr(foo), count(*) from bar group by
expr(foo)' ain't. One must use 'select count(*) from bar group by
expr(foo)'.
I also fixed a couple of things that looked like bugs to me: for
example, the row-ordering-limit should come at the end always.
Test Plan: Tested with the modified pinotdb sqlalchemy driver and an
internal pinot cluster. The pinotdb driver changes are in
https://github.com/agrawaldevesh/pinot-dbapi.
Pinot does not support orderby-limit for aggregated queries. To annotate
a query as an aggregate query, this patch adds a hint to the prepared
select statement that the pinotdb sqlalchemy driver then heeds.
Introduce a config DEFAULT_RELATIVE_END_TIME which is used when
computing the "Last X days". The default behavior (as currently) is to
let that be 'today', which actually means the 0th second of today. We
can also let it be 'now' which means the data as of now (ie at query
time).
Secondly, also introduce another config IS_EPOCH_S_TRULY_UTC, which
means that the logged time in epoch_s format is actually in UTC.
Currently (as the default) is that it it is in the local (superset
webserver) timezone.
** There is no backward incompatibility with thes config features since
the default behavior hasn't changed. **
* Making thrift, pyhive and tableschema as extra_requires
Looking at the dependency tree for license related questions, I noticed
that tableschema had a huge tree, and only people running Hive really
need it. Making this as well as pyhive and thrift optional.
Also bumping some python dependencies
* Run pip-compile
* Removing refs to past.builtins (from future lib)
* Add thrift
* Conditionally mutate and quote sqla labels decouple sqla logic from viz.py
* Prefix hashed label with underscore if bigquery label exceeds 128 chars
* Add comments for label cache
* Rename to mutated_labels and simply
* Rename mutated_label to get_label and simplify make_label_compatible in db_engine_specs
* Add note about deterministic and unique mutated labels
* add hash to label that has been prefixed with underscore
* Fix PEP8 escape warning
* Fix DeckPathViz get_metric_label call
* Use py3's f-strings instead of s.format(**locals())
In light of the bug reported here
https://github.com/apache/incubator-superset/issues/6347, which seems
like an odd `.format()` issue in py3, I greped and replaced all
instances of `.format(**locals())` using py3's f-strings
* lint
* fix tests
* [utils] gathering/refactoring into a "utils/" folder
Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.
Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.
* Improve tests
* Make loading examples in scope for tests
* Remove test class attrs examples_loaded and requires_examples
* Allow user to force refresh metadata
* fix javascript test error
* nit
* fix styling
* allow custom cache timeout configuration on any database
* minor improvement
* nit
* fix test
* nit
* preserve the old endpoint
* Enable Teradata
New DB engine spec for Teradata:
- LimitMethod should be WRAP_SQL since Teradata does not supporting "LIMIT" clause (TOP)
- Timegrains for Teradata is added
* Update formatting to pass flake8 tests
* Replace dataframe label override logic with table column override
* Add mutation to any_date_col
* Linting
* Add mutation to oracle and redshift
* Fine tune how and which labels are mutated
* Implement alias quoting logic for oracle-like databases
* Fix and align column and metric sqla_col methods
* Clean up typos and redundant logic
* Move new attribute to old location
* Linting
* Replace old sqla_col property references with function calls
* Remove redundant calls to mutate_column_label
* Move duplicated logic to common function
* Add db_engine_specs to all sqla_col calls
* Add missing mydb
* Add note about snowflake-sqlalchemy regression
* Make db_engine_spec mandatory in sqla_col
* Small refactoring and cleanup
* Remove db_engine_spec from get_from_clause call
* Make db_engine_spec mandatory in adhoc_metric_to_sa
* Remove redundant mutate_expression_label call
* Add missing db_engine_specs to adhoc_metric_to_sa
* Rename arg label_name to label in get_column_label()
* Rename label function and add docstring
* Remove redundant db_engine_spec args
* Rename col_label to label
* Remove get_column_name wrapper and make direct calls to db_engine_spec
* Remove unneeded db_engine_specs
* Rename sa_ vars to sqla_
* Bug: fixing async syntax for python 3.7
Rename async to async_ so superset installs for python 3.7.
* Addressing PR comments. Use kwargs instead of explicitly specifying async_ so downstream engines (e.g. PyHive) that supports async can choose to use the async_ in pythonwq3.7 and async in <=python3.6
* addressing additional pr comments
* Field names in big query can contain only alphanumeric and underscore
* bad quote
* better place for mutating labels
* lint
* bug fix thanks to mistercrunch
* lint
* lint again
