Druid sometimes returns error message that are contained in "<>", as in
`<urlopen error [Errno 61] Connection refused>`. Since Superset's
approach is often to bubble up messages coming from external library,
it's impossible to predict whether it will contain special characters.
There are some cases where our error handling does return some html
(presto?),
but we should manage that upstream. Plus the current setup has security concerns,
so let's move away from that.
fillna would miss out on identifying STRING columns for Druid and
replace None in string columns with a numeric `0`. This
mixed type column would confuse
pandas down the line on some operations like `df.pivot_table`.
Though flask has a builtin web debugger, ipdb some times still work
better. So I think add ipdb to dev dependencies is a good option for
people who prefer CLI debugging.
* cast postgresql date_trunc() to timestamp without time zone to prevent pandas error
* fix formatting for flake8
* change cast to timezone conversion instead
* Update installation.rst for Ubuntu 16.04 LTS
Ubuntu 16.04 by default install python2.7 alongside with python 3.5 and set python2.7 as default. If you have created a virtualenv with python3.5 compilation fails due to wrong python-dev library installed.
If you install ``python3.5-dev`` the build for the wheel package of ``cryptography`` run fine.
* Add options in D3_TIME_FORMAT_OPTIONS for non-english Time Format.
Added '%d/%m/%Y' and '%d/%m/%Y %H:%M:%S' as valid drop-down option for Axis Format
* [cli] permission cleanup on 'superset init'
FAB sometimes creates NULL/None permissions in the database,
presumably a race condition when multiple gunicorn workers start at the
same time, and those create issues raising "AttributeError: 'NoneType'
object has no attribute 'name'"
* Linting
Ubuntu 16.04 by default install python2.7 alongside with python 3.5 and set python2.7 as default. If you have created a virtualenv with python3.5 compilation fails due to wrong python-dev library installed.
If you install ``python3.5-dev`` the build for the wheel package of ``cryptography`` run fine.
* Use the query_obj as the basis for the cache key
When we recently moved from hashing form_data to define the cache_key
towards using the rendered query instead,
it made is such that non deterministic form
control values like relative times specified in "from" and "until" time
bound resulted in making those miss cache 100% of the time.
Here we move away from using the rendered query and using the query_obj
instead.
* Deprecating using form_data in templates
Filters applied to deck_multi will be passed down to layers as.
If the column isn't set as "filterable", the filter is ignored.
Also note that Dashboard configuration in regards to
"filter_immune_slices" and such will be disregarded in this context as
it isn't the dashboard controller passing down the filter and that
context is not easily accessible here.
Moving from having the user define an interceptor function that operates
on one object at a time.
By passing the entire array, it's possible to do multiple pass where
needed. A common pattern might be to figure out the max value in order
to define a scaler function. That's only possible if dealing with the
whole array.
* conditional check on datatype of results before converting to df
fix type checking
fix conditional checks
remove trailing whitespace and fix df_data fallback def
actually remove trailing whitespace
generalized type check to check all columns for dict
refactor dict col check
* move df conversion to helper and add unit test
add missing newlines
another missing newline
fix quotes
more quote fixes
Currently, even though `get_sqla_engine` calls get memoized, engines are
still short lived since they are attached to an models.Database ORM
object. All engines created through this method have the scope of a web
request.
Knowing that the SQLAlchemy objects are short lived means that
a related connection pool would also be short lived and mostly useless.
I think it's pretty rare that connections get reused within the context
of a view or Celery worker task.
We've noticed on Redshift that Superset was leaving many connections
opened (hundreds). This is probably due to a combination of the current
process not garbage collecting connections properly, and perhaps the
absence of connection timeout on the redshift side of things. This
could also be related to the fact that we experience web requests timeouts
(enforced by gunicorn) and that process-killing may not allow SQLAlchemy
to clean up connections as they occur (which this PR may not help
fixing...)
For all these reasons, it seems like the right thing to do to use
NullPool for external connection (but not for our connection to the metadata
db!).
Opening the PR for conversation. Putting this query into our staging
today to run some tests.
* Working polygon layer for deckGL
* add js controls
* add thumbnail
* better description
* refactor to leverage line_column controls
* templates: open code and documentation on a new tab (#4217)
As they are external resources.
* Fix tutorial doesn't match the current interface #4138 (#4215)
* [bugfix] markup and iframe viz raise 'Empty query' (#4225)
closes https://github.com/apache/incubator-superset/issues/4222
Related to: https://github.com/apache/incubator-superset/pull/4016
* [bugfix] time_pivot entry got missing in merge conflict (#4221)
PR here https://github.com/apache/incubator-superset/pull/3518 missed a
line of code while merging conflicts with time_pivot viz
* Improve deck.gl GeoJSON visualization (#4220)
* Improve geoJSON
* Addressing comments
* lint
* refactor to leverage line_column controls
* refactor to use DeckPathViz
* oops
Funky datatypes in some databases like BLOBs will have the DBAPI return
python types that can't be serialized to JSON out of the box.
Currently, when this happens SQL Lab fails in a bad way with a gigantic
HTML error message.
This allows specifying a pessimistic JSON serializer handler that will
simply show "Unserializable [type]"
* Using JS to customize spatial viz and tooltips
* Add missing deck_multi.png
* Improve GeoJSON layer with JS support and extra controls
* Addressing comments
When generating an auto SUM() metric on a column, Superset currently
will go `SUM(table_name.column_name)`. This is an issue when moving to
point to another table. It's common to work on some temporary table or
work table and eventually need to point Superset to an alternate table.
* Make Welcome page into a simple React app
This removes a dependency on datatables, we should be able to get rid
of it as we re-write the Table and PivotTable viz
* tests/lint
* Bump node version to latest
Some valid sqlalchemy uri's return a URL object with database=None, which causes the following error:
```
2018-01-05 17:59:47,560:ERROR:root:argument of type 'NoneType' is not iterable
Traceback (most recent call last):
File "/opt/incubator-superset/superset/sql_lab.py", line 186, in execute_sql
user_name=user_name,
File "/opt/incubator-superset/superset/utils.py", line 124, in __call__
return self.func(*args, **kwargs)
File "/opt/incubator-superset/superset/models/core.py", line 644, in get_sqla_engine
url = self.db_engine_spec.adjust_database_uri(url, schema)
File "/opt/incubator-superset/superset/db_engine_specs.py", line 505, in adjust_database_uri
if '/' in database:
TypeError: argument of type 'NoneType' is not iterable
```
This patch corrects that problem.
Delay creating DATA_DIR until config is fully imported.
This allows superset_config.py to override DATA_DIR before superset
attempts to create it in a potentially unwriteable location.
* Fix USA's state geojson for 'Country Map' visualization
Turns out the ISO codes were missing from the geojson file, this adds it
and uses human-readable indents.
* using proper ISO codes
* Linting
New linting rules started applying, I'm guessing a new version of
pylint?
* added deckgl geojson layer
* linting
* fixed comments
* addressed comments
* added override with controls.color_picker > 0
* set var properly
* set colors if property doesnt exist at all
* refacator on property mapping
* Introduce Javascript controls
This allows power-users to perform intricate transformations on data and
objects using javascript code.
The operations allowed are "sanboxed" or limited using node's vm
`runInNewContext`
https://nodejs.org/api/vm.html#vm_vm_runinnewcontext_code_sandbox_options
For now I'm only enabling in the line chart visualization, but the plan
would be to go towards offering more power to people who can write some
JS moving forward.
* Not applied
* Adding full Annotation Framework
* Viz types
* Re organizing native annotations
* liniting
* Bug fix
* Handle no data
* Cleanup
* Refactor slice form_data to data
For context, the viewport gets updated dynamically from the user
actions on the map. This is done on a timer every second or so to keep
the form data updated with the viewport settings.
With renderTrigger=true on that control that generates re-renders which
introduces glitches while zooming/panning. So turning it off as we don't
really expect users to input viewport info directly in the control
anyways.
Before this PR the only way to query lat/long is in the shape of 2
columns that contains lat and long.
Now we're adding 2 more options:
* a single column that has lat and long with a delimiter in between
* support for geohashes - geohashes are cool
* change reference for slices to chart
* change profile page reference
* change reference for Associated Slices
* change back to single quotes
* fix other single quotes
* linting
* last one
* fix test
The nvd3 docs say that it if the fastdom library is present it makes use
of it.
"Including Fastdom in your project can greatly increase the performance
of the line chart (particularly in Firefox and Internet Explorer) by
batching DOM read and write operations to avoid layout thrashing. NVD3
will take advantage of Fastdom if present."
* [sql lab] fix position of 'save query' Popover
The "Save Query" popover renders on the upper left corner as opposed to
where it should (relative to the Save Query button). After a bit of
research, it seems like Popover won't render in the right place when
parents are absolute.
I'm guessing this stopped working properly when I added the resizable
panes.
Anyhow, the solution here is to use a modal instead.
* Fixing tests
* fix 'superset db history'
Related Error msg when running `superset db history`:
"NameError: Can't invoke function 'get_bind', as the proxy object has not
yet been established for the Alembic 'Operations' class. Try placing
this code inside a callable."
* Lint
A bug in to_dict(orient="records") in pandas/core/frame.py prevents
datetimes with time zones to be worked with. This works around the
issue in superset by re-implementing the logic of pandas in the
correct way. Until pandas fixes the issue this code should stay.
https://github.com/pandas-dev/pandas/issues/18372
This closes#1929
Users can define `label_colors` in a dashboard's JSON metadata that
enforces a label to color mapping.
This also makes the function that maps labels to colors case insensitive.
* DECKGL integration
Adding a new set of geospatial visualizations building on top of the
awesome deck.gl library. https://github.com/uber/deck.gl
While the end goal it to expose all types of layers and let users bind
their data to control most props exposed by the deck.gl API, this
PR focusses on a first set of visualizations and props:
* ScatterLayer
* HexagonLayer
* GridLayer
* ScreenGridLayer
* Addressing comments
* lint
* Linting
* Addressing chri's comments
I was wondering what was left to do in order to remove Dashboard.jsx
and superset.js, and it looks like they can just be pulled out.
I am so happy to get rid of what used to be the messiest JS files in the
whole repo.
Thanks @graceguo!
* README: add the mailing list
And mark the google group as deprecated
* README: move the screenshots at the end
So hopefully it'll be easier to find the resources
I was getting some "Could not connect to server" when there was
a proper json payload with an `error` key, the change here makes sure to
prioritize those messages over the generic one.
Making it clear that master has a `dev`-suffixed version. Proper release
number will be created in release branches.
There are constraints as to what npm and setuptools will accept as a
proper version number and N.N.Ndev seems to work so I'm rolling with it
Open to suggestion as to how to tag `master`
I was having issues getting an RPM to build. My error was
Processing files: superset-0.20.1-1.noarch
error: File must begin with "/": Lockup
error: File must begin with "/": With
error: File must begin with "/": Text.svg
error: File must begin with "/": Lockup
error: File must begin with "/": With
error: File must begin with "/": Text@2x.png
error: File must begin with "/": Lockup
error: File must begin with "/": Without
error: File must begin with "/": Text@1x.svg
error: File must begin with "/": Lockup
error: File must begin with "/": Without
error: File must begin with "/": Text@2x.png
error: File must begin with "/": Mark.png
error: File must begin with "/": Mark@1x.svg
for ref https://github.com/pypa/setuptools/issues/767
File renaming fixes this issue
In `assets/static` we have a symlink pointing to `docs/_build/html`
which is .gitignored and missing in the repo since we don't want the
html-generated docs in the repo, meaning the symlink is broken
Having the symlink broken creates issues with pip-compile somehow.
Adding a dummy file here addresses that
fixes https://github.com/apache/incubator-superset/issues/3507
This prevents malformed JSON from getting saved in a slice's params. It
also prevents the issue described in #3507 from happening though as a
result malformed slices will render using default control values.
* Fix the ISO code description of region/province/department
* Missing the data of one province and two regions of China
* Revert " Missing the data of one province and two regions of China"
This reverts commit abff4555cd.
* Add srcdoc-polyfill tosupport Internet Explorer iframes in markup slices. Add allow-top-navigation and allow-popups to support links within iframes
* Remove jquery from markup.js
* ColorSchemeControl: fixing bad use of PropTypes
Accessing PropTypes via the main React package is deprecated, and will be
removed in React v16.0. Use the latest available v15.* prop-types package from
npm instead. For info on usage, compatibility, migration and more, see
https://fb.me/prop-types-docs
* Control: adding PropTypes.func in types allowed inside `value` prop
This removes a warning during yarn tests
Fix#3589
* tests(QueryStateLabel): removing missing prop warning
```
Warning: Failed prop type: The prop `query` is marked as required in
`QueryStateLabel`, but its value is `undefined`.
in QueryStateLabel
```
* SaveQuery: removing invalid prop `target` supplied to `Overlay`.
This removes a warning during yarn tests:
```
Warning: Failed prop type: Invalid prop `target` supplied to `Overlay`.
```
* RunQueryActionButton: removing `isRequired` from queryState props
This removes a warning during yarn tests:
```
Warning: Failed prop type: The prop `queryState` is marked as required in
`RunQueryActionButton`, but its value is `null`.
```
When the database impersonation flag is on, a query using a template
fails. It has to do with templating using a database connection without
a username being specified by the caller, along with the fact that the
work is taking place on a worker, outside a web request, where
referencing g.user raises this exception.
Somehow the nature of `g` in Flask has changed where `g.user` used to
be provided outside the web request scope and its not anymore.
The fix here should address that.
* Feature: added datepicker and time granularity options to dashboard filter
* Added option for Druid datasource time filters
* added more checkbox control over dashboard time filters
it was downgraded in commit 7045018d86.
The old version used React.PropTypes and it was deprecated.
This version is compatible with React 15+.
Fix#3588
* removing deprecated babel-preset-es2015 in favor of babel-preset-env
This removes this yarn warning during install:
```
warning babel-preset-es2015@6.24.1:
Thanks for using Babel: we recommend using babel-preset-env now:
please read babeljs.io/env to update!
```
* updating sinon to 4.0.0
This removes wrench from its dependencies and this yarn warning during install:
```
warning wrench@1.3.9: wrench.js is deprecated!
You should check out fs-extra (https://github.com/jprichardson/node-fs-extra)
for any operations you were using wrench for.
Thanks for all the usage over the years.
```
* Adding annotations to backend
* Auto fetching Annotations on the backend
* Closing the loop
* Adding missing files
* annotation layers UI
for https://github.com/apache/incubator-superset/issues/3502
* a few fixes per code review.
- add annotation input sanity check before add and before update.
- make SelectAsyncControl component statelesis, and generic
- add annotation description in d3 tool tip
- use less variable to replace hard-coded color
* exposed API endpoint to get querystring for a slice
* Added unit tests for endpoint
* fixed test case for python3
* moved get querystring logic into its own func
* renamed query string endpoint
* 1. after user make sql query and visualize, allow user click title to update slice title, and create a new slice at the same time.
2. don't save new title if it is empty. Will still show old title.
* change saveSlice call response and update explore view
- if current user is allowed to edit dashboard, we will allow this user to edit slice name.
- show different tooltip given allowed/not-allowed to update slice name.
- user will click slice name and update.
- after user submit edit, if he doesn't have right to alter slice, server-side will return error message to client-side. Slice name will not be changed or saved.
- will show notification after save slice name.
* Chinese page
* Using react-intl-universal to improve multi language in react page
* Using react-intl-universal to improve multi language in react page
* react_intl_universal
* change
* change
* change
* change
* change
* change
* change
* merge
* multiple page in js
* merge
* merge
* merge
* merge
* Js Translations
* JS Translation
* JS Translations
* Js translation
* JS translations
* JS translations
* Js translaion
* JS en Translation
* JS Translation
* upgrade document
Fixing the damn build (#3179)
* Fixing the build
* Going deeper
[bugfix] only filterable columns should show up in FilterBox list (#3105)
* [bugfix] only filterable columns should show up in FilterBox list
* Touchups
Datasource cannot be empty (#3035)
add title description to model view (#3045)
* add title description to model view
* add missing import
Add 'show/hide totals' option to pivot table vis (#3101)
[bugfix] numeric value for date fields in table viz (#3036)
Bug was present only when using the NOT GROUPED BY option
fixes https://github.com/ApacheInfra/superset/issues/3027
fix hive.fetch_logs (#2968)
add Zalando to the list of organizations (#3171)
docs: fixup installation examples code indentation (#3169)
[bugfix] fix bar order (#3180)
[bugfix] visualize flow error: 'Metric x is not valid' (#3181)
The metric name in the frontend doesn't match the one generated on the
backend. It turns out the explore view will default to the first
metric so specifying one isn't needed.
Fix the segment interval for pulling metadata (#3174)
The end of the interval would be on the truncated today date, which
means that you will exclude today. If your realtime ingestion job
runs shorter than a day, the metadata cannot be pulled from the
druid cluster.
Bump cryptography to 1.9 (#3065)
As 1.7.2 doesn't compile here with openssl 1.1.0f
Escaping the user's SQL in the explore view (#3186)
* Escaping the user's SQL in the explore view
When executing SQL from SQL Lab, we use a lower level API to the
database which doesn't require escaping the SQL. When going through
the explore view, the stack chain leading to the same method may need
escaping depending on how the DBAPI driver is written, and that is the
case for Presto (and perhaps other drivers).
* Using regex to avoid doubling doubles
[sqllab] improve Hive support (#3187)
* [sqllab] improve Hive support
* Fix "Transport not open" bug
* Getting progress bar to show
* Bump pyhive to 0.4.0
* Getting [Track Job] button to show
* Fix testzz
Add BigQuery engine specifications (#3193)
As contributed by @mxmzdlv on issue #945
[bugfix] fix merge conflict that broke Hive support (#3196)
Adding 'apache' to docs (#3194)
[druid] Allow custom druid postaggregators (#3146)
* [druid] Allow custom druid postaggregators
Also, fix the postaggregation for approxHistogram quantiles so it adds
the dependent field and that can show up in the graphs/tables.
In general, postAggregators add significant power, we should probably
support including custom postAggregators. Plywood has standard
postAggregators here, and a customAggregator escape hatch that allows
you to define custom postAggregators.
This commit adds a similar capability for Superset and a additional
field/fields/fieldName breakdown of the typical naming for dependent
aggregations, which should make it significantly easier to develop
approxHistogram and custom postAggregation-required dashboards.
* [druid] Minor style cleanup in tests file.
* [druid] Apply code review suggestions
* break out CustomPostAggregator into separate class. This just cleans
up the creation of the postaggregator a little bit.
* minor style issues.
* move the function around so the git diff is more readable
add combine config for metrics in pivot table (#3086)
* add combine config for metrics in pivot table
* change method to stack/unstack
* update backendSync
Autofocus search input in VizTypeControl modal onEnter (#2929)
Speed up JS build time (#3203)
Also bumping a few related libs
JS Translation
JS translations
js translation
fix issue 3204 (#3205)
[bugfix] capture Hive job_id pre-url transformation (#3213)
js translation
fix issue 3204 (#3205)
[bugfix] capture Hive job_id pre-url transformation (#3213)
[docs] update url in CONTRIBUTING.md (#3212)
[sqllab/cosmetics] add margin-top for labels in query history (#3222)
[explore] nvd3 sort values in rich tooltip (#3197)
[sqllab] fix UI shows 'The query returned no results' momentarily (#3214)
this is visible when running async queries between the fetching and
success state as the rows are getting cached in the component
[explore] DatasourceControl to pick datasource in modal (#3210)
* [explore] DatasourceControl to pick datasource in modal
Makes it easier to change datasource, also makes it such that the list
of all datasources doesn't need to be loaded upfront.
* Adding more metadata
* Js translation
* js tran
* js trans
* js trans
* js tran
* js trans
* js trans
* js tran
* js translation
* js trans
* js translation
* try load language pack async
* Backend translations things
* create language pack inside common data
* performance improvement for js i18n.
- js bundle should not contain localized content
- we populate translation content from server-side, in boostrap.common.language_pack
- in client-side, use promise to wrap around translation content. text will be translated after translation content arrived/parsed.
- fix linting
* fix Timer unit test
* 1. add global hook for all tests, to make translation pack avaialble before each test starts.
2. fix unit test for Timer component
3. remove noused method get_locale, and modules
4. fix page reload after user change page language
* parse and build i18n dictionary as a module
* fix sync-backend task, which should run without DOM
Since the latest pyhive, we don't need pythrifthiveapi as they ship with
the latest version.
There's actually a conflict between the new pyhive and pythrifthiveapi
and this fixes it.
* Add "Impersonate user" setting to Datasource
* Add tests
* Use g.user.username for all the sync cases
* use uri.username instead of uri.user
* Small refactoring
* Fixed bugs when removing filter, switching operators, and switching columns
* Fixed lint errors for code style
* Added more unit tests for FilterControl
* Code format changes to meet standards
* add lanscape?
* prevent empty queries from being sent to db
* Revert "add lanscape?"
This reverts commit b3d6e80af4.
* improve code style
* nit
* unnit
* renit
* [heatmap] numerous improvements
* flexibility as to how to sort X and Y axis (alpha/value, desc/asc)
* option to show a legend
* fixed margins, maximize real estate
* allowed users to define bounds
* Tunning
Currently the heatmap axis are sorted randomly, this PR makes it such
that it's properly sorted.
Also allowing for specifying the left and bottom margins
* Bumping npm
* Fixing the build
This addresses long standing issues and hacks around react-map-gl
that have been creating all sorts of build issues over time.
It appears that recent changes broke things further.
This PR upgrades to the latest `react-map-gl` and removes related hacks.
Previously all 'since' date expression evaluated in the future like
`30 days` would be reassigned to the past (now - `30 days`). It would
extend to fixed dates which is a bad thing and was removed.
Now we have reports and dashboards in the wild that use things like `30
days` and we'd like to not break those as we roll out the next version.
This fix should allow for that.
* [explore] fixed padding bug on filter section
* fix slice_id logging issue
* [superset-sqllab] fix slice_id population in appropriate column
* [explore-logging] test the slice_id logging fix
* fix travis errors
* fix nits pointed out in PR comments
* cleanup tests
* made python more beautiful
* made python even more beautiful
* made python even more more beautiful
* made python even more more more beautiful
* fix lint error
* make exception handling more specific
* fixed silly error
* fixed argument indentation
The left panel of the explore view has become crowded and overwhelming
overtime. This PR adds functionality to collapse the control sections,
and sets most sections to be collapse by default as the explore view
opens up.
* breakdown `Query` section for most viz
* bring filters to the top, under Query section
* collapse most sections by default
* removed confusing outdated description for Filter section
* Update to the version of react-ace with the fixed sizing issues
* Make ace editor resizable
* Use small util method for offset calculation instead of $
* Test ResizableAceEditor
* Make the right pane of the Sql Lab scrollable
* Add default and min height to the ResizableAceEditor
* Implement SplitPane
* Make Splitter fullscreen
* React on resize of the window
* Implement min and max
* Get rid of a magic number + add margin
* Handle resize event with delay + cleanup the code
* Make ResultSet adjustable
* Make QueryHistory adjustable
* Remove ResizableAceEditor
* Make linter happy
* Test SplitPane
* Init sizes properly
number of issues are addressed here:
* x axis formatting doesn't apply in bubble chart
* adding option to show/hide bounds of x and y axis
* with a flaky 'auto' margin mode, allow user to specify hard values or
auto
* x label font-size was different than y axis
* show more options and reorg Control panels for Axes in `line` viz
for chart type: bubble, area, treemap and country map, color scheme change cannot take effect immediately. So have to disable immediately re-render function and hide lightening icon for these chart type.
* Allow for multiple color schemes
1. create ColorSchemeControl component
2. using the same new control component for linear colors
* add color spectum for linear color scheme
* remove dup css
* fix controls setting for linear color scheme
* minor fix by code review comment
Style wasn't working right for separator widget since the iframe sandboxing of
the markup widget. This addresses this small issue and also now allows
for html in the separator widget
* [mapbox] fix viewport alterations
Since explorev2 it appears that altering the viewport hasn't been
changing the controls as it used to. This PR addresses it.
* lint
Somehow Ace's "changeSelection" event is triggered when hitting
backspace (and shouldn't!).
changeSelection on our side triggers enough work to make the
holding backspace sluggish and laggy.
This fix ignores selection with a length of 1, avoiding mutating the
state altogether when hitting/holding backspace.
* improve superset list view page layout
- less header spaces and stacks
- move pagination down to bottom
- apply material design style to 'add' action button
- will apply to all superset list view, like slices list, security tab lists etc.
* improve superset list view page layout
- less header spaces and stacks
- move pagination down to bottom
- apply material design style to 'add' action button
- will apply to all superset list view, like slices list, security tab lists etc.
* improve superset list view page layout
- less header spaces and stacks
- move pagination down to bottom
- apply material design style to 'add' action button
- will apply to all superset list view, like slices list, security tab lists etc.
* improve superset list view page layout
- less header spaces and stacks
- move pagination down to bottom
- apply material design style to 'add' action button
- will apply to all superset list view, like slices list, security tab lists etc.
* improve superset list view page layout
- less header spaces and stacks
- move pagination down to bottom
- apply material design style to 'add' action button
- will apply to all superset list view, like slices list, security tab lists etc.
* remove tabs from indentation
* fix merge conflicts
* adjust css after code merge
parse_human_datetime parses date-only strings, e.g. "today", returning the correct date but time set at 9am. This is an internal implementation in parsedatetime. This patch resets to midnight. If time is specified and parsed, it is correctly returned.
* add translatable columns in label_columns of the view
* display the verbose_name of columns in list view, just like in the metrics list
* Revert "display the verbose_name of columns in list view, just like in the metrics list"
This reverts commit f815d3b3ed.
* [add] Save filters to dashboard
* format code
* fix CI error
* add semicolon semi
* fix none object
* add test data
optimize the js code
fix the compatibility issue
* fix urllib to urllib.parse
* add space
* update test case
* remove 'return'
* fix error
* update test case
* split reducer logic for ExploreViewContainer
* fix saveModal component and unit tests
* revert changes in SaveModal_spec.
will make another commit just to improve test coverage for SaveModal component.
* improve test coverage for explore view components:
- SaveModal component
- URLShortLinkButton
* remove comment-out code
* [bugfix] wrong 'Cant have overlap between Series and Breakdowns' (#3254)
* [explore] make edit datasource a basic link (#3244)
* Relying on FAB for font-awesome.min.css (#3261)
* Modernize SQLA pessimistic handling (#3256)
Looks like SQLAlchemy has redefined the best practice around
pessimistic connection handling.
* [webpack] break CSS and JS files while webpackin' (#3262)
* [webpack] break CSS and JS files while webpackin'
* cleaning up some templates
* Fix pylint issue
* import logging (#3264)
* [bugfix] preserve order in groupby (#3268)
Recently in
4c3313b01c
I introduced an issue where the order of groupby fields might change.
This addresses this issue and will preserve ordering.
* Explicitly add Flask as dependancy (#3252)
* Use sane Celery defaults to prevent tasks from being delayed (#3267)
* Improve the chart type of Visualize in sqllab (#3241)
* Improve the chart type of Visualize in sqllab & Add some css & Fix the link address in the navbar
* add vizTypes filter
* Set default ports Druid (#3266)
For Druid set the default port for the broker and coordinator.
* [explore] Split large reducer logic in ExploreViewContainer (#3088)
* split reducer logic for ExploreViewContainer
* fix saveModal component and unit tests
* revert changes in SaveModal_spec.
will make another commit just to improve test coverage for SaveModal component.
* remove comment-out code
* fix merge confilicts
* split reducer logic for ExploreViewContainer
* fix saveModal component and unit tests
* revert changes in SaveModal_spec.
will make another commit just to improve test coverage for SaveModal component.
* remove comment-out code
* fix merge confilicts
* [explore] DatasourceControl to pick datasource in modal
Makes it easier to change datasource, also makes it such that the list
of all datasources doesn't need to be loaded upfront.
* Adding more metadata
* [druid] Allow custom druid postaggregators
Also, fix the postaggregation for approxHistogram quantiles so it adds
the dependent field and that can show up in the graphs/tables.
In general, postAggregators add significant power, we should probably
support including custom postAggregators. Plywood has standard
postAggregators here, and a customAggregator escape hatch that allows
you to define custom postAggregators.
This commit adds a similar capability for Superset and a additional
field/fields/fieldName breakdown of the typical naming for dependent
aggregations, which should make it significantly easier to develop
approxHistogram and custom postAggregation-required dashboards.
* [druid] Minor style cleanup in tests file.
* [druid] Apply code review suggestions
* break out CustomPostAggregator into separate class. This just cleans
up the creation of the postaggregator a little bit.
* minor style issues.
* move the function around so the git diff is more readable
* [sqllab] improve Hive support
* Fix "Transport not open" bug
* Getting progress bar to show
* Bump pyhive to 0.4.0
* Getting [Track Job] button to show
* Fix testzz
* Escaping the user's SQL in the explore view
When executing SQL from SQL Lab, we use a lower level API to the
database which doesn't require escaping the SQL. When going through
the explore view, the stack chain leading to the same method may need
escaping depending on how the DBAPI driver is written, and that is the
case for Presto (and perhaps other drivers).
* Using regex to avoid doubling doubles
The end of the interval would be on the truncated today date, which
means that you will exclude today. If your realtime ingestion job
runs shorter than a day, the metadata cannot be pulled from the
druid cluster.
The metric name in the frontend doesn't match the one generated on the
backend. It turns out the explore view will default to the first
metric so specifying one isn't needed.
Related Msg:
sqlalchemy.orm.exc.ObjectDeletedError: Instance '<PermissionView at
0x7f10306b0e90>' has been deleted, or its row is otherwise not present.
* upgrade celery to 4.0.2
* using Redis for unit tests (sqla broker not supported in Celery 4)
* Setting Celery's soft_time_limit based on `SQLLAB_ASYNC_TIME_LIMIT_SEC` config
* Better error handling in async tasks
* Better statsd logging in async tasks
* show [pending/running] query status in Results tab
* systematically using sqla NullPool on worker (async) to limit number
of database connections
* move to explore folder and delete explorev2 folder
* add tests for fetchDatasourceMetadata
* tests for fetchDatasources, fetchDatasourceMetadata
* use $.ajax for fetch dashboards and write test
* sort explicitly on label
* add simple test for /slicemodelview/add endpoint
* make comments and method names more clear
* fix test name
* be more explicit, test status_code
* get compiled js file names
* make manifest available as template var
* use script src directly to avoid flash of unstyled content in the case of csstheme.js
* linting
* attempt to fix tests
* exception
* print the path when no manifest file found
* handle case when manifest.json is not present for some reason, or in the case of tests
* Fix handling of Chunked requests
Add fix for handling chunk encoding requests.
If ENABLE_CHUNK_ENCODING is set to true, for requests with transfer
encoding set to true. It will set wsgi.input_terminated to true which
tells werkzeug to ignore content-length and read the stream till the
end.
break comment in multiple lines
* remove debug print logging
* dont set first datasource as default
* not used
* add a disabled class if datasource is not selected
* sort datasources alphabettically
* make btn disabled is no datasource is selected
* fix linting
* [big number] various improvements
* dynamic number of X axis ticks based on width to prevent label overlap
* corrected overflow on the x axis
* improved tooltips (precise arrow and visible data point circle on hover)
* Fixing tooltips in heatmap viz
* initial structure for add new slice page
* simplify add slice form
* add a test
* fix long line
* use underscore for template name
* fix controls path
* fix vis types select
Some python developers use Python version manager to create and switch Python
enviroments, pyenv (http://github.com/yyuu/pyenv) is one of the version managers.
When you use `pyenv local x.x.x`, `pyenv` will put a `.python-version` file in
project directory, which, in general, should not be commited to git.
Same as explore view, if query takes > 45 seconds not returned, we will show Query timeout warning message. Otherwise user will see 504 Gateway timeout error.
* CONTRIBUTING: document starter_task label
* CONTRIBUTING: PR should work travis python versions
* CONTRIBUTING: move PR guidelines at the top
* CONTRIBUTING: no need to open a new pr after review
in SqlLab view, if query takes over 45 seconds, we will show advise to store a summarized data set before user clicks on Visualize button.
This advise will not block Visualize button.
fixes https://github.com/airbnb/superset/issues/2733
* rename spec folder
* remove special handling for viz_type control since it now uses VizTypeControl
* add test for getOptions
* linting
* add test for cwp
* linting
The reason is that it should be able to create arbitrary filters
over attributes with very high cardinality, where loading all
possible combinations into the filter box is infeasible,
e.g. IPv6 addresses.
* feat(visualization): Create new visualization's tools that display country with geojson file. Currently only france and Spain are loaded
* refacto(visualization): fix issues created by codeclimate
* refacto(visualization): fix issues created by travis for mysql databases
* refacto(visualization): fix issues created by codeclimate
* refacto(visualization): fix issues created by codeclimate
* refacto(visualization): fix issues created by codeclimate
* refacto(visualization): fix issues created by codeclimate
* fix(visualization): fix issues
* doc(visualization): Init visualization documentation
* doc(visualization): init visualization documentation for country
* fix(visualization): implement iso 3166-2 for id of country
* fix(visualization): implement iso 3166-2 for id of country
* doc(visualization): init visualization documentation for country
* fix(visualization): implement iso 3166-2 for id of country
* feat(country_map): add ukraine map in component
* fix(visualization): Test dont working and add color based on metrics
* refacto(visualization): fix issues for es6 and color component
* Show clear and actionable query timeout error message
1. Instead of waiting for a long time or server-side response 504 Gateway timeout, explore view now add a query timeout threshold. After timeout it will show specific querytimeout message.
2. fix alert box close button position.
* Show clear and actionable query timeout error message
1. Instead of waiting for a long time or server-side response 504 Gateway timeout, explore view now add a query timeout threshold. After timeout it will show specific querytimeout message.
2. fix alert box close button position.
3. fix a linting error.
Load times on list view pages like Slices, Dashboards, Tables and Database
have grown to be terrible over time.
After a bit of digging, I found that the not specifying `search_columns`
in ModelViews actually means "all columns" and that for each filter,
FAB goes and fetches a list of all values to prepopulate the
filter dropdowns. That means that the list of tables would fetch all
slices and all users upfront which is horrible. Worse, database list
view would fetch all queries with is insane.
This picks a subset of columns for search/filters and changes the
default to show only 100 elements per page instead of 500
auto select should only apply to cases where select control doesn't have pre-selected value prop. If select control has pre-selected value (passed in from value prop), auto select first avaliable options should not apply.
* [sql lab] improvements to the left panel
* better error handling with error notifications
* table is added instantly with a loading image
* Fixing tests
* Fixed tests
* Decimal is a valid numeric type
This will allow automatic creation of sum__ and avg__ metrics
* formatting for line length at 80 chars
* Reformating again, for the mandatory 79 chars maximum.
Somehow the CSS trick we use to display histogram-type visual elements
in table views started showing some odd glitches since
what I believe was a not-so-recent version of webkit.
The bug is around the `linear-gradient` call under `background-image`
prop being used to show non-gradient or "hard" colors, while using
transparency.
I was able to find a workaround that addresses things in chrome by using
a fake [minimal] gradient instead of a flat color.
* db_engine_specs: added engine spec for clickhouse
* models: group by time without join on subquery
if this is enabled, it will remove the subquery when grouping by time,
and group by time will be rewritten as group by timeslot, ...
this particularly fixes clickhouse backend
* docs: added clickhouse
* models: allow using secondary dttm columns as constraints
* superset/forms: configurable default slice since time
* superset/models: respect dialect quoting
* dttm handling, codeclimate tweaks
* react: using prop-types package to fix deprecated React.PropTypes property warning
https://facebook.github.io/react/warnings/dont-call-proptypes.html
* removing babel devDependency because has been deprecated in favor of babel-cli
this fixes a warning during `npm install`:
```
npm WARN deprecated babel@6.23.0:
In 6.x, the babel package has been deprecated in favor of babel-cli.
Check https://opencollective.com/babel to support the Babel maintainers
```
* js: setting ExploreActionButtons.queryEndpoint PropType as required
because it's required in the child component DisplayQueryButton
* js(tests): using object in expandedSlices prop type of SliceCell tests
* js(tests): adding required props to SqlEditor mockedProps
* js(tests): adding required prop editorHeight to TabbedSqlEditors mockedProps
* js: removing unused moments dependency
* Fixing PropTypes warning message
React recently started warning on the upcoming deprecation of
React.PropTypes, the new approach is to use the `prop-types`
npm package instead.
* Fixing the tests
The standalone view doesn't need to fetch the datasources asynchronously
to fill in the dataources dropdown, it also doesn't need to render any
of the controls.
* fix scrolling for query search, getHeight was returning NAN
* add styling for query search pagination
* remove console.log
* make pagination buttons match bootstrap theme small buttons
At Airbnb we use these WARNING_MSG configuration element to make it
clear that we're in the staging environment.
Before this PR it would render under the navbar and mess up some of the
heights configurations, where you wouldn't be able to scroll all the way
to the bottom of the page. This fixes it.
* make react-virtualized table work
use dynamic sizing for cell width
enable filtering
require height prop for result set component
* fix tests and linting
* move some state to props
* move getTextWidth to visUtils
* make striped rows optional
* fix striped proptype
* update name to FilterableTable
* add basic test and fix linting
* accept array of columns keys rather than an array of objects that needs to be mapped
* move container div inside the component
* rename styles
* fit table component to width if it's smaller than parent container
* move stylesheet to javascript folder otherwise it throws an error on npm run prod
* move css to index.jsx
* fix result set spec
* fix linting and test
* fix result set props
* keep list immutable
Introducing a nice component as a label that show when data was
loaded from cache, when the cache was taken (in humanize duration as in
`a few minutes ago`) in a tooltip, and it can act as a button that
can trigger a force-refresh.
While working on it, it became clear that it was going to be hard to
use this component in the Dashboard view since it's not pure React.
I'm planning on refactoring the dashboard view with proper React/Redux
and introducing the CachedLabel component at that point.
While digging around in the Dashboard view I realized that there was a
bunch on unused code around managing timers that was used in explorev1
and decided to rip it out.
* Add UNIX socket option to runserver
Added an optional parameter to runserver to server from a UNIX socket instead of an address:port. I believe it is fairly common to server from sockets when using a web server like Nginx on the same host.
* Collapsed if/else logic for address or socket
Also wrapped help description for socket parameter
* [explore] fix and clean
Currently it's not possible to view queries while they are running, the
spinner appears endlessly. I decided to rearrange things a bit while
debugging so I could see clearly through it.
* Adding NotImplemented methods to base classes
* Fixing
* Piling up
Column functions are currently escaped with the
same dialect that is being used by Superset for its
metadata storage. Instead the dialect of the target
table should be used when escaping column names.
* [bugfix] clarifying how to create a slice
First time users [rightfully] think that the proper way to create a
slice is to click the `+` button on the `Slices` CRUD page.
Changing / simplifying the redirect logic and message flashing that was
failing here somehow.
* Changing message from danger to info
* Revert "[big num] make sure scatterplot dots align properly (#2559)"
This reverts commit f24ddfd467.
* Revert "Revert "measure x axis labels too and use the longest to determine margins" (#2550)"
This reverts commit fe68bc31c3.
* Revert "[bug num vis] fix sizing for single digits (#2548)"
This reverts commit 3d2c791ff1.
* revert all big num changes.
* fix linting
* make document global, fix linting
* sql_lab.py: compress via utils
* utils.py: added zlib_compress and zlib_compress_to_string
* core.py: converted to use zlib_decompress_to_string; renamed uncompress to decompress in utils.py
* utils_tests.py: added test for compress/decompress
* fixed broken utils test; removed redundant code and empty lines from utils.py
* utils.py: corrected docstrings, removed unnecessary 'else'
* removed yet another superfluous else
* fix js error, make should render viz more readable, add tooltips, style axis and add min/max to xaxis
* put getTextWidth in utils
* add document to eslint globals config
* Initial commit to support the athena DB
This work was done at tobii.com (in collaboration with knowit.se), and depends on:
- A patched version of PyAthenaJDBC (https://github.com/dwa/PyAthenaJDBC/tree/dwa-tobii-dict_formatter)
- A patched version of PyHive (https://github.com/dwa/PyHive/tree/dwa-tobii-sqlalchemy-athena)
And can be used like so:
athena://<user>:<password>@athena.us-east-1.amazonaws.com/?region_name=<region>&s3_staging_dir=s3%3A//<staging_bucket_of_choice>
* Rebased, and fixed two lint issues
* rename athena engine: athena -> awsathena
* remove network status feature
* only fetch queries if there are started or running queries
* don't use local storage
* remove last network status from actions
* don't remove support for local storage
* address pr comments and linting
* use .some rather than .forEach
* [sqllab] assign types for visualize flow
Somehow when using the visualize flow, the types were not
assigned at all, creating some bugs downstream. This PR attempts to get
the information required based on what pandas is knows and the types in
the data itself.
* Fixing tests
* Fixing tests
* Fixing more tests
* Fixing the last py3 tests
* use fast filter for tables select
* react-virtualized needs this package, throws error otherwise...
* continue to use async select and this.getTableNamesBySubStr when a schema is not selected
* import styles in same place as react-select css import
When adding a Redshift database, the docs said to use a connection
string that started with `redshift+psycopg2://...`, but this results
in a Python error from how SQLAlchemy executes the `get_schema_names`
method (first reported [here](https://github.com/airbnb/superset/issues/2364#issuecomment-284705179)).
The fix is to use `postgresql+psycog2://...` for the Redshift connection
string.
* Support more filter operators
* more filter operators [>, <, >=, <=, ==, !=, LIKE]
* Fix need to escape/double `%` in LIKE clauses
* spinner while loading values when changing column
* datasource config elements to allow to applying predicates when
fetching filter values
* refactor
* Removing doubling parens
* rebasing
* Merging migrations
* [table viz] allow showing time granularity in table (#2284)
The time granularity currently does not show up in table viz. Now
defining a granularity will add an extra ISO-formatted time column in
the table. The column will be added for both grouped by and not grouped
by.
* Changes based on comments
The time granularity currently does not show up in table viz. Now
defining a granularity will add an extra ISO-formatted time column in
the table. The column will be added for both grouped by and not grouped
by.
The time granularity currently does not show up in table viz. Now
defining a granularity will add an extra ISO-formatted time column in
the table. The column will be added for both grouped by and not grouped
by.
Caching wasn't working after deprecate_v1, this addresses it. Also surfacing
whether the data is served from cache in explore view and a way to force
run the query bypassing the cache.
on startup, FAB spits out a bunch of logging messages that aren't useful
in most cases. This shuts them down by default. They can be turned back
on with `config.SILENCE_FAB = True`
Also shushing a flask-cache warning around setting up a null (default) cache
* Simplifying the viz interface (#2005)
* Working on dashes
* Making this a collaborative branch
* Fixing some bugs
* Fixing bugs
* More improvements
* Add datasource back in bootstrap data
* Decent state
* Linting
* Moving forward
* Some more linting
* Fix the timer
* Triggering events through state
* Lingint
* Put filters in an array instead of flt strings (#2090)
* Put filters in an array instead of flt strings
* Remove query_filter(), put opChoices into Filter
* Update version_info.json
* Fix migrations
* More renderTrigger=true
* Fixing bugs
* Working on standalone
* getting standalone to work
* Fixed forcedHeight for standalone =view
* Linting
* Get save slice working in v2 (#2106)
* Filter bugfix
* Fixing empty series limit bug
* Fixed dashboard view
* Fixing short urls
* Only allow owners to overwrite slice (#2142)
* Raise exception when date range is wrong
* Only allow owner to overwrite a slice
* Fix tests for deprecate v1 (#2140)
* Fixed tests for control panels container and filters
* Fixed python tests for explorev2
* Fix linting errors
* Add in stop button during slice querying/rendering (#2121)
* Add in stop button during slice querying/rendering
* Abort ajax request on stop
* Adding missing legacy module
* Removing select2.sortable.js because of license
* Allow query to display while slice is loading (#2100)
* Allow query to display while slice is loading
* Put latestQueryFormData in store
* Reorganized query function, got rid of tu[le return values
* Merging migrations
* Wrapping up shortner migration
* Fixing tests
* Add folder creation to syncBackend
* Fixing edit URL in explore view
* Fix look of Stop button
* Adding syntax highlighting to query modal
* Fix cast_form_data and flase checkbox on dash
* Bugfix
* Going deeper
* Fix filtering
* Deleing invalid filters when changing datasource
* Minor adjustments
* Fixing calendar heatmap examples
* Moving edit datasource button to header's right side
* Fixing mapbox example
* Show stack trace when clicking alert
* Adding npm sync-backend command to build instruction
* Bumping up JS dependencies
* rm dep on select2
* Fix py3 urlparse
* rm superset-select2.js
* Improving migration scripts
* Bugfixes on staging
* Fixing Markup viz
* Fix werkzeug instance was created twice in Debug Mode (#2135)
* add reloader option for flask (#2136)
* using --no-reload option for flask
* divide a line of code into two lines for PEP8
* Rename rv => o in the decorator.
* Address comments.
* Permissions cleanup: remove none and duplicates. (#1967)
* Updates
* Rename var and dropdown text
* Cleanup
* Resolve comments.
* Add user to the perm check.
* fix axis label size bug and accommodate dual axis chart
* don't adjust margins for bar time series
* handling this below now
* apply different margin padding if explore or dashboard
* fix linting
* re-render chart after adjusting for long axis labels
* measure all of the labels and take the max height
* add missing isTimeSeries var
* fix linting
* use jQuery to get text ticks
* rotate 45 rather than 90
* Add KeyValue model for storing id-value pairs
use it for storing shared queries
* Change string to text and added test
* Put getQueryLink in one place
* Changed migration down version
* Changes based on comments
* Update bcf3126872fc_add_keyvalue.py
In Vertica, inner queries require different aliases than the main query.
This is an example of query generated before this patch:
SELECT chain AS chain,
weekstartday AS __timestamp,
SUM(inventory) AS "Inventory"
FROM mytable
JOIN
(SELECT chain AS chain__,
SUM(inventory) AS "Inventory"
FROM mytable
WHERE weekstartday >= '2016-01-24 00:00:00'
AND weekstartday <= '2017-01-17 00:00:00'
GROUP BY chain
ORDER BY "Inventory" DESC LIMIT 50) AS anon_1 ON chain = chain__
WHERE weekstartday >= '2016-01-24 00:00:00'
AND weekstartday <= '2017-01-17 00:00:00'
GROUP BY chain,
weekstartday
ORDER BY "Inventory" DESC LIMIT 50000
Which in Vertica produces the error:
Error: ('42702', '[42702] ERROR 2671: Column reference "inventory" is ambiguous\n (2671) (SQLExecDirectW)')
And this is the same example after the patch:
SELECT chain AS chain,
weekstartday AS __timestamp,
SUM(inventory) AS "Inventory"
FROM mytable
JOIN
(SELECT chain AS chain__,
SUM(inventory) AS mme_inner__
FROM mytable
WHERE weekstartday >= '2016-01-24 00:00:00'
AND weekstartday <= '2017-01-17 00:00:00'
GROUP BY chain
ORDER BY mme_inner__ DESC LIMIT 50) AS anon_1 ON chain = chain__
WHERE weekstartday >= '2016-01-24 00:00:00'
AND weekstartday <= '2017-01-17 00:00:00'
GROUP BY chain,
weekstartday
ORDER BY "Inventory" DESC LIMIT 50000
Related PR:
19f5371787
* Cleanup fulfilled requests after approve
* Modified tests
* Moved to separate test, add user to access functions
* Moved to separate test and added test cases
* Fixed issue with dryrun
* More changes based on comments
I had issues compiling the src, I think it's because the syntax is different than the one used here?
With the dist it works fine.
I also saw there were other related issues in the past, so I'm suggesting to switch to the dist file.
* [explorev2] using label in 'Visualization Type' Select
* moved url related logic upstream in the caller
* moved option creation logic from render method to the constructor as
it only needs to be executed once
* Refactoring out getOptions
Fix cache set after c14c7ed update json serializer behaviour
but forget to update one call site.
While at it move payload serialization outside of catch all try
block since we want to see explosions if any.
Fix#1910
* Druid dashboard import
* Fix tests.
* Parse params with trailing commas
* Resolved issue with datasource is not in DB yet when slice perms are set
* Finish rebase
* Fix heads alembic problem.
* Druid dashboard import
* Fix tests.
* Parse params with trailing commas
* Resolved issue with datasource is not in DB yet when slice perms are set
* Finish rebase
* Add #views and #distinct users to created dashboard on profile page
* Added index on logs to speed up query
* Added #views and #users for slice table
* Add a views column to dashboards and slices, prepopulate
them with Log data
* Remove index on Log model
* Remove unused index
* Update 1b2c3f7c96f9_.py
fix multiple heads
* Exclude postgres in prepopulating views column
* Add dual-axis line chart to viz
- Add a new viz using two y axis for multi-chart in nvd3
- Option to set metric and axis format for two y axis
* Resolve conflicts
* Fixed x axis and resized thumbnail
* Added example to __init__.py
* Change get_df to match with format
* only show single run query button, allow async if possible
* only pass the needed props, rather than entire objects to the component
* add simple test
* fix linting
* Initial
* rewrite some line to make it short and setting merge variable temporarily
* rewrite commit author
* add emitted attribute
* Fix typo
* fix test error
* fix typo
* test added
* Improving logging with duration and referrer
* Handling case where referrer is None
* Providing log_this with its own session
* Attempting to fix tests
* Fixing tests
* Enable freeform-select with fetched column values for filter values
- db migration to add filter_select_enabled
- add freeform-multi option for Selectfield
- modify formatFilter() function on query to accomodate filter-select
* Fix js tests
* Fix codeclimate issue
* Changes based on comments
* Add test for filter endpoint
* Extract out renderFilterFormField function from render
* Fix landscape issues
* Stop ChartContainer from rendering twice on chartStatus change
* Make spinner overlay and dim chart while laoding
* Added timeout to make render more stable on resize
* Put viz in state and call resize at browser size change
* add render after height change since resize depends on render called on the same object
* Change name of renderVis to renderVisOnChange
* Only call resize at height change, persist viz in state
* Call resize wihout render at window size change
* Sort searched queries by recency
* Fixed sqllab tests
* Add one more tr to QueryTable spec for pagination
* Change desc to asc as we reverse queries in component
This should fix issue #1339.
IE 11 and lower has a long standing issue: out-of-document element's
pathname has no leading '/'. See
https://connect.microsoft.com/IE/feedbackdetail/view/1002846/pathname-incorrect-for-out-of-document-elements
And Superset's Slice.jsonEndpoint() method relies on pathname() to build
JSON API URL for slices:
```javascript
jsonEndpoint() {
const parser = document.createElement('a');
parser.href = data.json_endpoint;
let endpoint = parser.pathname + this.querystring();
endpoint += '&json=true';
endpoint += '&force=' + this.force;
return endpoint;
},
```
`parser` above is exactly an out-of-document element. Therefore when
running in IE <= 11, Superset would build wrong JSON endpoint URLs,
hence the 404 errors for loading data for slices.
This commit adds a simple workaround when leading '/' is missing in the
value returned by pathname().
* add config option for explore v2 beta users, and send through v2 path
* fix long lines
* use role rather than user ids in config
* add test
* simplify role check
* remove extra line
* use different test user for v2 tests
* Not working errors
* Remove update_explore endpoint, only update explore endpoints in reducer on query
* Change scroll to auto and make container reponse to height:
* Remove update_explore endpoint
* Remove can_update_explore perm
* Add NVD3's bullet chart.
* Add empty lines before nested function definitions.
* Add thumbnail for bullet chart.
* Add bullet chart to gallery.rst.
* Add "requiresTime: false", fix indentation.
* Avoid scaling bullet chart vertically.
* Use a default if no range is specified.
* Fix coloring of bullet chart.
* Add http to copied url and move function to componentWillReceiveProps
* Added getText() to CopyToClipbaord to enable ajax calls for getting copy text
* Set ajax call to synchronous (document.execCommand only works in synchronous mode
* Make cell-click filter in table viz optional
- Added table_filter checkbox in table viz
- If set to false, clicking on a cell in table will not apply filter to
dashboard
* Fix codeclimate issue
* Change default to false
* Fixed js error when results are not available
* Flush data and query in results when running new query, keeping columns
* add exception for columns
* Move setState from componentWillMount to componentWillReceiveProps
* Nit
* Fix superset cli for python3
dict.iteritems() has been removed since dict.items() returns an
iterable in python3. Shouldn't be a big deal for python2 to load
all the data into a list.
Fix#1756
* bin/superset: avoid some work when reading config
We don't need to unpack and then pack again a dictionary.
* Support running superset via pex
* [superset] Update default port in superset/bin/superset
* Fix codeclimate line length issues
* Fix another line length issue, in config.py
* Add trivial utils test to increase test coverage
* Clean up runserver handling
* Make chart container more responsive
**Leave chart empty when theres error
**Make all boolean fields auto-query chart when changed
* Replace forEach with some
* Added fields to autoQueryFields
* Bug fixes in Save Modal
Issues solved:
- Save button doesn't pass in gotodash
- slice_name was passed in from store as original slice_name instead of
new one in 'saveas' action
- datasource_type wasn't passed in to defaultViz and defaultForm
function
* Change css filename to exploreV2
* Moved out utils
* Implement table name extraction tests.
* Address comments.
* Fix tests and reimplement the token processing.
* Exclude aliases.
* Clean up print statements and code.
* Reverse select test.
* Fix failing test.
* Test JOINs
* refactore as a class
* Check for permissions in SQL Lab.
* Implement permissions check for the datasources in sql_lab
* Address comments.
* Add per schema permissions.
* Address comments.
* Add schema_access perms to the alpha and admin
* Create permissions on addition databases and datasources.
* Remove hybrid_property. Linter complains.
* Fixed table_name does not exist in druid
* Make Chart container scrollable for large chart
* Fixed bug of action buttons not clickable in heatmap
* Solve codeclimate issue
* Limit overflow to x
* render table name if in table view
* only render fave star and edit button if slice, not table
* fix error when table view
* use [table-name] - untitled format
* remove extra fave star
* Added filter in ControlPanelsContainer
* Move function for getting url params object to utils
* Fixed python test
* Move Filter to separate component
* Added specs and made changes based on comments
* Moved specs to right folder
* For some fields we would like to re-render chart once field is
* changed, saving user the time to click query button
* Such fields are stored in an array in store, could add more fields in
* the future
* Get sections to render when switching datasource
- Move sectionsToRender in store and use is for defaultFormData
- Change some SelectField to FreeFormSelect according to forms.py
* Solved the css not found problem in staging
* Fixed js tests
* Added specs for SaveModal
* Move datasource_id and datasource_name to form_data
* Add comments
* Deleted redundant fetchDashboard
* Replcae has_key for python3
* More react and less jquery
* Added alert for save slice
* Small changes based on comments
* Use react bootstrap
* Added Alert for ControlPanel and ChartContainer
Done:
- Add alert for Control Panel when fetch_datasource_metadata failes
- Add alert for Chart Container when update_explore query fails
* Changed color to warning-yellow
* Solve linter issue
* Fixed indent and delete error_redirect
- Issue: when switching from a viz_type outside nvd3 to a viz_type in
nvd3, the Chart Container doesn't draw new graph
- Fix: The reason was somehow the function inside nv.addGraph() wasn't
called, extract the function outside and explicitly calling it solve
the problem
* Fixed bugs with viz in exploreV2
Done:
- fix typo in pie viz
- add metrics/groupby to dist_bar viz
- fix typo in heatmap viz
- add big_number_total to viz
- fixed problem with fetching columns for datasource
- add sqlaTimeSeries to viz
- change row_limit and limit from number to strings so that we
don't need to parse integers in bootstrap data
* Fix python tests
* Added order_bars checkbox for dist_bar viz
* Move datasource from global store object to form_data
* Moved datasource_id and datasource_name to form_data
* Refetch defaultFormData when switching datasource
* Fixed js tests
* Get query buttonw working in explorev2
- Create new endpoint for updating explore viz
- Send over new form_data when query button is pressed
* Added endpoint test
* Changes based on comments
* Added docstring for endpoint, and query spec
* Remove white space around docstring
* Added different Select Fields
- Switched FormGroup to react-select
- Added multi and freeform to select, now it can take customized user
input and insert it as options
* Fixed tests
* Small nit based on comments
* Change in files
* Renamin files and folders
* cleaning up a single piece of lint
* Removing boat picture from docs
* add superset word mark
* Update rename note in docs
* Fixing images
* Pinning datatables
* Fixing issues with mapbox-gl
* Forgot to rename one file
* Linting
* v0.13.0
* adding pyyaml to dev-reqs
* Link fields to store by firing setFormData action
* Moved onChange to ControlPanelsContainer, retrieve defaultFormData from fields in store
* Pass data from store to checkbox/select/text Field
* Fixed tests
* Changed reducer back to old Object.assign() style
* pass field options in viz json
* move field options to fetch_datasource_metadata
* on control panels container mount, fetch datasource meta data and set dynamic field choices
* render options for select fields
* use component class rather than sic
* fix linting
* fix whitespace
* delete unused var
* only render fields once datasource meta has returned
* fix typo
* add datasources and fix column formatting
* fix tests
* never used function
* fix tests
* add test for fetch_datasource_metadata
* remove unneeded props
* Put default form_data and viz in store
* Link fields to store by firing setFormData action
* Fixed tests for Container and actions
* Moved onChange to ControlPanelsContainer, retrieve defaultFormData from fields in store
* Deleted switch statement in reducer
* Removed resetFormData and refactored setFormData in reducers
* Added text for fields
* Changed test statements
* Added scroll bar and option to collapse for Sql Editor tool bar
Done:
- Added scroll-bar to Sql Editor tool bar
- Added hide/expand tool bar option to dropdown menu of tab
* Add more margin to give space to scroll-bar
* Add scroll to right panel independently
* Put data preview in south pane
Before: data preview of a selected table appears as a modal, but for
some cases users may want to view data and edit sql at the same time
After:
- data preview of a selected table pops up a new tab in South Pane
- data are saved to local state and flushed in global store in
ResultSet component
* Moved dataPreviewId to table object
* Put back preview icon for fetching preview data
* Revert "Put back preview icon for fetching preview data"
This reverts commit b6f5dcfe64.
* Added option to retrieve preview results after refresh
* make fieldset conditions more clear
* make label required
* use render* pattern
* use slugify util for turning labels into ids
* use field rather than html
* don't need panel-title class here
* Support week_ending_saturday for Druid.
* Use period granularity
* Use ISO 8601 for period definitions.
* Fix tests
* More flexibility for the freeform choices.
* export functions directly rather than object at the bottom
* move viztypes to controlPanelMappings, add fieldset rows and section data
* for each viz type, render a controlPanelsContainer, controlPanelSections, FieldSetRows, and FieldsSets
* add comments, move mappings to store
* organize store and add default sections
* render all the needed sections
* add tooltip to sections
* remove console log
* use only panel panel-default, not panel-body, no need the padding
* render fields for all fields in field set
* add the rest of the control panel sections and field overrides
* fix naming
* add fieldTypes array
* don't use default section
* pass only needed state via mapStateToProps
* fix code climate errors
* linting
* move field components to their own files
* render field sets as lists
* fix field components
* use SFC
* update modal trigger test to be more accurate
* add FieldSetRow test
* add test for controlpanelsContainer
* fix test
* make code climate happy
* add freeform select field
* [druid] optimize Druid queries where possible
Trying to use timeseries, topn where possible, falling back on 2-phases
groupby only where needed
* Fixing py3 bug
* Moved queriesArray from render() to local state, so that QueriesArray
is only reloaded only during switching tabs or queries object is updated.
* Changed object comparison function to take length into consideration
* Added time filter to query search page
* Added start date
* Updated python endpoint test
* changed spec
* Added specs and tests
* Modified python/js tests and some function/file names
based on code review comments
* Resolved conflicts in DashboardSelect_spec and QuerySearch_spec
* Break python tests for separate functions, Move sql queries to setUp()
* Get around eslint error for spec
* Small changes based on comments
* Highlight affected slices for filter change in dashboard view
Done:
- When user adds/deletes a filter in dashboard, affected slices will
have their header highlighted for 2 seconds
* Modified highlight to a more subtle box shadow
* Added slice-cell class for highlight transition
* Changed class name to slice-cell-highlight
* Added dashboard standalone page
* Deleted additional template, parameterized dashboard.html for standalone
* Only wrap add-slice-container for standalone instead of including whole modal
* Use standalone_mode argument passed from both explore view and dashboard view
* Adjusted top margin of heatmap plot to get it working in V2
Problem:
The heatmap in V2 was shifted towards the top margin of slice
container, this was because in v2 slice name header was part of the
container body, while in v1 the header was separately defined in
explore.html template.
Solution:
To get heatmap properly shown in V2, we need to
take into account the height of the slice_name header. Adding to
margin_top will shift the plot in V1 too, but it won't make a big
difference to the look.
Ideally when we renovate slice container in future PR we would defined a
height for slice_name header and take it into account for all
visualization files.
* Added panel header height to margin_top for explore v2
* Use getBoundingClientRect to get header height
* Use slice-header for id of panel-title
* Added url shortner for sharing query link
* Move shortener outside CopyToClipboard and move ajax call to common.js
* transfer dbId to int to avoid failed prop
* Added d3format() function to mock slice in explorev2
Problem:
table viz was not working in explorev2 due to d3format() not defined in
mock slice.
* Change column_formats to camel case
* [sqllab] add support for results backends
Long running SQL queries (beyond the scope of a web request) can now use
a k/v store to hold their result sets.
* Addressing comments, fixed js tests
* Fixing mysql has gone away
* Adressing more comments
* Touchups
A primary goal of Apache Superset is to be inclusive to the largest number of contributors, with the most varied and diverse backgrounds possible. As such, we are committed to providing a friendly, safe and welcoming environment for all, regardless of gender, sexual orientation, ability, ethnicity, socioeconomic status, and religion (or lack thereof).
This code of conduct outlines our expectations for all those who participate in our community, as well as the consequences for unacceptable behavior.
We invite all those who participate in Apache Superset to help us create safe and positive experiences for everyone.
## 2. Open Source Citizenship
A supplemental goal of this Code of Conduct is to increase open source citizenship by encouraging participants to recognize and strengthen the relationships between our actions and their effects on our community.
Communities mirror the societies in which they exist and positive action is essential to counteract the many forms of inequality and abuses of power that exist in society.
If you see someone who is making an extra effort to ensure our community is welcoming, friendly, and encourages all participants to contribute to the fullest extent, we want to know.
## 3. Expected Behavior
The following behaviors are expected and requested of all community members:
* Participate in an authentic and active way. In doing so, you contribute to the health and longevity of this community.
* Exercise consideration and respect in your speech and actions.
* Attempt collaboration before conflict.
* Refrain from demeaning, discriminatory, or harassing behavior and speech.
* Be mindful of your surroundings and of your fellow participants. Alert community leaders if you notice a dangerous situation, someone in distress, or violations of this Code of Conduct, even if they seem inconsequential.
* Remember that community event venues may be shared with members of the public; please be respectful to all patrons of these locations.
## 4. Unacceptable Behavior
The following behaviors are considered harassment and are unacceptable within our community:
* Violence, threats of violence or violent language directed against another person.
* Sexist, racist, homophobic, transphobic, ableist or otherwise discriminatory jokes and language.
* Posting or displaying sexually explicit or violent material.
* Posting or threatening to post other people’s personally identifying information ("doxing").
* Personal insults, particularly those related to gender, sexual orientation, race, religion, or disability.
* Inappropriate photography or recording.
* Inappropriate physical contact. You should have someone’s consent before touching them.
* Unwelcome sexual attention. This includes, sexualized comments or jokes; inappropriate touching, groping, and unwelcomed sexual advances.
* Deliberate intimidation, stalking or following (online or in person).
* Advocating for, or encouraging, any of the above behavior.
* Sustained disruption of community events, including talks and presentations.
## 5. Consequences of Unacceptable Behavior
Unacceptable behavior from any community member, including sponsors and those with decision-making authority, will not be tolerated.
Anyone asked to stop unacceptable behavior is expected to comply immediately.
If a community member engages in unacceptable behavior, the community organizers may take any action they deem appropriate, up to and including a temporary ban or permanent expulsion from the community without warning (and without refund in the case of a paid event).
## 6. Reporting Guidelines
If you are subject to or witness unacceptable behavior, or have any other concerns, please notify a community organizer as soon as possible. dev@superset.incubator.apache.org .
Additionally, community organizers are available to help community members engage with local law enforcement or to otherwise help those experiencing unacceptable behavior feel safe. In the context of in-person events, organizers will also provide escorts as desired by the person experiencing distress.
## 7. Addressing Grievances
If you feel you have been falsely or unfairly accused of violating this Code of Conduct, you should notify Apache with a concise description of your grievance. Your grievance will be handled in accordance with our existing governing policies.
## 8. Scope
We expect all community participants (contributors, paid or otherwise; sponsors; and other guests) to abide by this Code of Conduct in all community venues–online and in-person–as well as in all one-on-one communications pertaining to community business.
This code of conduct and its related procedures also applies to unacceptable behavior occurring outside the scope of community activities when such behavior has the potential to adversely affect the safety and well-being of community members.
## 9. Contact info
dev@superset.incubator.apache.org
## 10. License and attribution
This Code of Conduct is distributed under a [Creative Commons Attribution-ShareAlike license](http://creativecommons.org/licenses/by-sa/3.0/).
Portions of text derived from the [Django Code of Conduct](https://www.djangoproject.com/conduct/) and the [Geek Feminism Anti-Harassment Policy](http://geekfeminism.wikia.com/wiki/Conference_anti-harassment/Policy).
Retrieved on November 22, 2016 from [http://citizencodeofconduct.org/](http://citizencodeofconduct.org/)
Look through the GitHub issues for bugs. Anything tagged with "bug" is
@@ -26,12 +29,12 @@ open to whoever wants to implement it.
### Implement Features
Look through the GitHub issues for features. Anything tagged with
"feature" is open to whoever wants to implement it.
"feature" or "starter_task" is open to whoever wants to implement it.
### Documentation
Caravel could always use better documentation,
whether as part of the official Caravel docs,
Superset could always use better documentation,
whether as part of the official Superset docs,
in docstrings, `docs/*.rst` or even on the web as blog posts or
articles.
@@ -46,52 +49,161 @@ If you are proposing a feature:
implement.
- Remember that this is a volunteer-driven project, and that
contributions are welcome :)
### Questions
## Latest Documentation
There is a dedicated [tag](https://stackoverflow.com/questions/tagged/apache-superset) on [stackoverflow](https://stackoverflow.com/). Please use it when asking questions.
Latest documentation and tutorial are available [here](http://airbnb.io/caravel)
## Pull Request Guidelines
Before you submit a pull request from your forked repo, check that it
meets these guidelines:
1. The pull request should include tests, either as doctests,
unit tests, or both.
2. If the pull request adds functionality, the docs should be updated
as part of the same PR. Doc string are often sufficient, make
sure to follow the sphinx compatible standards.
3. The pull request should work for Python 2.7, and ideally python 3.4+.
``from __future__ import`` will be required in every `.py` file soon.
4. Code will be reviewed by re running the unittests, flake8 and syntax
should be as rigorous as the core Python project.
5. Please rebase and resolve all conflicts before submitting.
6. If you are asked to update your pull request with some changes there's
no need to create a new one. Push your changes to the same branch.
## Documentation
The latest documentation and tutorial are available [here](https://superset.incubator.apache.org/).
Contributing to the official documentation is relatively easy, once you've setup
your environment and done an edit end-to-end. The docs can be found in the
`docs/` subdirectory of the repository, and are written in the
Note that before running specific tests, you have to both setup the local testing environment and run all tests.
We use [Mocha](https://mochajs.org/), [Chai](http://chaijs.com/) and [Enzyme](http://airbnb.io/enzyme/) to test Javascript. Tests can be run with:
cd /caravel/caravel/assets/javascripts
cd /superset/superset/assets/javascripts
npm i
npm run test
@@ -155,15 +286,14 @@ We use [Mocha](https://mochajs.org/), [Chai](http://chaijs.com/) and [Enzyme](ht
Lint the project with:
# for python changes
flake8 changes tests
flake8 changes caravel
# for python
flake8
# for javascript
npm run lint
## Linting with codeclimate
Codeclimate is a service we use to measure code quality and test coverage. To get codeclimate's report on your branch, ideally before sending your PR, you can setup codeclimate against your Caravel fork. After you push to your fork, you should be able to get the report at http://codeclimate.com . Alternatively, if you prefer to work locally, you can install the codeclimate cli tool.
Codeclimate is a service we use to measure code quality and test coverage. To get codeclimate's report on your branch, ideally before sending your PR, you can setup codeclimate against your Superset fork. After you push to your fork, you should be able to get the report at http://codeclimate.com . Alternatively, if you prefer to work locally, you can install the codeclimate cli tool.
*Install the codeclimate cli tool*
```
@@ -193,42 +323,27 @@ Generate the documentation with:
cd docs && ./build.sh
## CSS Themes
As part of the npm build process, CSS for Caravel is compiled from `Less`, a dynamic stylesheet language.
As part of the npm build process, CSS for Superset is compiled from `Less`, a dynamic stylesheet language.
It's possible to customize or add your own theme to Caravel, either by overriding CSS rules or preferably
It's possible to customize or add your own theme to Superset, either by overriding CSS rules or preferably
by modifying the Less variables or files in `assets/stylesheets/less/`.
The `variables.less` and `bootswatch.less` files that ship with Caravel are derived from
The `variables.less` and `bootswatch.less` files that ship with Superset are derived from
[Bootswatch](https://bootswatch.com) and thus extend Bootstrap. Modify variables in these files directly, or
swap them out entirely with the equivalent files from other Bootswatch (themes)[https://github.com/thomaspark/bootswatch.git]
## Pull Request Guidelines
Before you submit a pull request from your forked repo, check that it
meets these guidelines:
1. The pull request should include tests, either as doctests,
unit tests, or both.
2. If the pull request adds functionality, the docs should be updated
as part of the same PR. Doc string are often sufficient, make
sure to follow the sphinx compatible standards.
3. The pull request should work for Python 2.6, 2.7, and ideally python 3.3.
``from __future__ import`` will be required in every `.py` file soon.
4. Code will be reviewed by re running the unittests, flake8 and syntax
should be as rigorous as the core Python project.
5. Please rebase and resolve all conflicts before submitting.
## Translations
We use [Babel](http://babel.pocoo.org/en/latest/) to translate Caravel. The
We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset. The
key is to instrument the strings that need translation using
`from flask_babel import lazy_gettext as _`. Once this is imported in
a module, all you have to do is to `_("Wrap your strings")` using the
underscore `_` "function".
We use `import {t, tn, TCT} from locales;` in js, JSX file, locales is in `./superset/assets/javascripts/` directory.
To enable changing language in your environment, you can simply add the
`LANGUAGES` parameter to your `caravel_config.py`. Having more than one
`LANGUAGES` parameter to your `superset_config.py`. Having more than one
options here will add a language selection dropdown on the right side of the
navigation bar.
@@ -239,25 +354,39 @@ navigation bar.
}
As per the [Flask AppBuilder documentation] about translation, to create a
new language dictionary, run the following command:
new language dictionary, run the following command (where `es` is replaced with
the language code for your target language):
pybabel init -i ./babel/messages.pot -d caravel/translations -l es
pybabel init -i superset/translations/messages.pot -d superset/translations -l es
Then it's a matter of running the statement below to gather all stings that
Then it's a matter of running the statement below to gather all strings that
[](https://gitter.im/airbnb/caravel?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[](https://gitter.im/airbnb/superset?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.