Files
superset2/docs-v2/docs/installation/cache.mdx
Hugh A. Miles II ac2545e357 feat: Upgrade documentation V2 (#17411)
* setup docusaurus

* rename

* add introduction content

* chore(docsV2): move content from docs to docsV2 (#17714)

* add FAQs and contribution pages

* chore: add api, security, and roadmap pages, include swaggerui in dependency for api page

* chore: move api page header below imports

* chore: change API page info alert to use built in Infima class instead of custom class

Co-authored-by: Corbin Robb <corbin@Corbins-MacBook-Pro.local>

* chore(docs-v2): moving more markdown content to new documentation site (#17736)

* chore: move markdown content and images for docs installation directory to docs-v2

* chore: move docs miscellaneous directory content to docs-v2

* chore(docs-v2): move over connecting to databases content and rename some files to .mdx

Co-authored-by: Corbin Robb <corbin@Corbins-MacBook-Pro.local>

* Update styling and logo (#17990)

* update styling

* update colors

* chore(docs-v2): remove blog and tutorial and update some styling (#17929)

* add superset logo and favicon, change styles to better match current docs, add prettierrc

* change file types to mdx

* Add simple superset dark mode freindly logo

* clean up default pages - blog and tutorial docs

Co-authored-by: Corbin Robb <corbin@Corbins-MacBook-Pro.local>

* Chore: moving charts and dashboard to docusaurus (#18036)

* add contributing add creating charts and dashboards

* delete extra images

* update rat-excludes

* Port homepage (#18115)

* Port community page (#18128)

* chore: add seo redirects for Docs v@ (#18092)

* fix: handle null values in time-series table (#18039)

* cleanup column_type_mappings (#17569)

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* important change to MakeFile (#18037)

* Update superset-e2e.yml (#18041)

* Revert "Update superset-e2e.yml (#18041)" (#18051)

This reverts commit b5652739c9.

* feat: Trino Authentications (#17593)

* feat: support Trino Authentications

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* docs: Trino Authentications

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* chore(supeset.utils.core): move all database utils to database utils module (#18058)

* chore(plugin-chart-echarts): add types to controls (#18059)

* fix(generator): more cleanup to plugin framework (#18027)

* fix(generator): more cleanup to plugin framework

* fix typo and package name

* add docs

* fix typo

* Update superset-frontend/webpack.config.js

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>

* fix generator reference

* add steps to tutorial and fix package version

* refine docs/readme

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>

* feat(advanced analytics): support groupby in resample (#18045)

* fix(dashboard): scope of nativefilter not update (#18048)

* fix(generator): add lockfile and fix styling issues (#18073)

* fix(generator): add lockfile and fix styling issues

* fix margins and remove redundant scroll

* update tutorial

* refactor(sql_lab): SQL Lab Persistent Saved State (#17771)

* a lot of console logs

* testing

* test

* added saved_query to remoteId

* created useEffect so that title properly changes in modal

* Update superset-frontend/src/SqlLab/actions/sqlLab.js

Co-authored-by: Lyndsi Kay Williams <55605634+lyndsiWilliams@users.noreply.github.com>

Co-authored-by: Lyndsi Kay Williams <55605634+lyndsiWilliams@users.noreply.github.com>

* refactor(example_data): replace the way the birth_names data is loaded to DB (#18060)

* refactor: replace the way the birth_names data is loaded to DB

* fix failed unit test

* fix failed unit test

* fix failed tests

* fix pass wrong flag of support datetime type

* remove unused fixture

* feat: add chart description in info tooltip (#17207)

* feat: add chart list description

* fix: text overflow

* fix: text-overflow with line-height

* Correction of proper names format in README (#18087)

* chore: added SEO routes

* fix can't use examples helpers on non app context based environment (#18086)

* chore: split CLI into multiple files (#18082)

* chore: split CLI into multiple files

* Update tests

* Who fixes the fixtures?

* Add subcommands dynamically

* Rebase

* fix misspelling (#18097)

* refactor: sqleditorleftbar to functional (#17807)

* Working on converting sqleditorleftbar to functional component

* Creating draft PR to address bug

* Still working on solving re rendering bug

* infinite rerender fix

* Creating draft PR to address bug

* Cleaning up in preparation for push

* Made changes suggested by Elizabeth

* Fixed issues as per Lindsey's comment

Co-authored-by: Arash <arash.afghahi@gmail.com>

* fix rat excludes and headers

* fix(docs): fix path of image for "Create New Chart" (#18089)

* Migrate Checkbox story to tsx - see #18100 (#18101)

Looks good!

* refactor: migrate RowCountLabel to TypeScript & added story (#18105)

* enable superbook for explore component

* migrate RowCountLabel to TypeScript

* add storybook for RowCountLabel

* fix: logging warning on dataframe (don't use python's warnings) (#18111)

* fix: logging warning on dataframe (don't use python's warnings)

* lint

* update changelog and updating for 1.4.0 (#18083)

* feat: Adds a key-value endpoint to store charts form data (#17882)

* feat: Adds a key-value endpoint to store charts form data

* Fixes linting problems

* Removes the query_params from the endpoints

* Refactors the commands

* Removes unused imports

* Changes the parameters to use dataclass

* Adds more access tests

* Gets the first dataset while testing

* Adds unit tests for the check_access function

* Changes the can_access check

* Always check for dataset access

* fix(explore): fix chart embed code modal glitch (#17843)

* feat(plugin-chart-echarts): support non-timeseries x-axis (#17917)

* feat(plugin-chart-echarts): support non-timeseries x-axis

* fix tests

* change formula return type from Date to number

* add x_axis test coverage

* rename func and improve coverage

* add x-axis control to bar chart

* remove redundant console.log

* fix description

* make x-axis control mandatory

* 🙃

* fix x-axis formatter

* fix showValues

* fix implicit rDTTM_ALIAS references in postProcessing

* replace TIME_COLUMN with DTTM_ALIAS

* fix remaining implicit indexes

* fix: Disable filtering on wide result sets (#18021)

* fix: handle null values in time-series table (#18039)

* cleanup column_type_mappings (#17569)

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* important change to MakeFile (#18037)

* add missing is_timeseries to pivot op

Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Grace Guo <grace.guo@airbnb.com>
Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>
Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>

* feat(country-map): added new countries in country-chart-map (#18081)

* chore: migrating storybook jsx to typescript #18100 (#18133)

* Migrating storybook jsx to typescript #18100

* Migrating storybook jsx to typescript

Co-authored-by: Jayakrishnan Karolil <jayakrishnan.karolil@nielsen.com>

* feat(annotation): add toast feedback to annotation templates (#18116)

* feat(dashboard): add toast feedback to dashboard actions (#18114)

* feat(explore): more toast feedback on user actions in Explore (#18108)

* feat(explore): add toasts feedback when user copies chart url

* Show toast message when updating chart properties

* Change toast type to success when saving chart

* Use success toast from props

* Fix tests

* Use withToasts instead of dispatch

* Use PropertiesModalProps instead of any

* Docs: fix typo (#18125)

* fix: undefined error when adding extra sequential color scheme (#18152)

* feat: allow assets to be managed externally (#18093)

* feat: allow assets to be managed externally

* Use server_default

* chore: use pkg_resources for cleaner config (#18130)

* refactor: Moves the Explore form_data endpoint (#18151)

* refactor: Moves the Explore form_data endpoint

* Removes unused imports

* Fixes openapi schema error

* Fixes typo

* Renames and UPDATING.md

Co-authored-by: Grace Guo <grace.guo@airbnb.com>
Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>
Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: Hugh A. Miles II <hughmil3s@gmail.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>
Co-authored-by: Stephen Liu <750188453@qq.com>
Co-authored-by: Lyndsi Kay Williams <55605634+lyndsiWilliams@users.noreply.github.com>
Co-authored-by: Adam Dobrawy <ad-m@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: Emily Wu <86927881+em0227@users.noreply.github.com>
Co-authored-by: Josue Lugaro <82119536+JosueLugaro@users.noreply.github.com>
Co-authored-by: Arash <arash.afghahi@gmail.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Hammad-Raza <hammadraza42@hotmail.com>
Co-authored-by: jayakrishnankk <kk.jayakrishnan@gmail.com>
Co-authored-by: Jayakrishnan Karolil <jayakrishnan.karolil@nielsen.com>
Co-authored-by: Farid Rener <proteusvacuum@users.noreply.github.com>

* remove unneeded requirement

Co-authored-by: Corbin Robb <31329271+corbinrobb@users.noreply.github.com>
Co-authored-by: Corbin Robb <corbin@Corbins-MacBook-Pro.local>
Co-authored-by: Daniel W <61300812+The-hyphen-user@users.noreply.github.com>
Co-authored-by: Geido <60598000+geido@users.noreply.github.com>
Co-authored-by: Srini Kadamati <skadamat@gmail.com>
Co-authored-by: Grace Guo <grace.guo@airbnb.com>
Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>
Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>
Co-authored-by: Stephen Liu <750188453@qq.com>
Co-authored-by: Lyndsi Kay Williams <55605634+lyndsiWilliams@users.noreply.github.com>
Co-authored-by: Adam Dobrawy <ad-m@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: Emily Wu <86927881+em0227@users.noreply.github.com>
Co-authored-by: Josue Lugaro <82119536+JosueLugaro@users.noreply.github.com>
Co-authored-by: Arash <arash.afghahi@gmail.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Hammad-Raza <hammadraza42@hotmail.com>
Co-authored-by: jayakrishnankk <kk.jayakrishnan@gmail.com>
Co-authored-by: Jayakrishnan Karolil <jayakrishnan.karolil@nielsen.com>
Co-authored-by: Farid Rener <proteusvacuum@users.noreply.github.com>
2022-01-27 14:54:53 -08:00

124 lines
4.0 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Caching
hide_title: true
sidebar_position: 5
version: 1
---
## Caching
Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purpose. For security reasons,
there are two separate cache configs for Superset's own metadata (`CACHE_CONFIG`) and charting data queried from
connected datasources (`DATA_CACHE_CONFIG`). However, Query results from SQL Lab are stored in another backend
called `RESULTS_BACKEND`, See [Async Queries via Celery](/docs/installation/async-queries-celery) for details.
Configuring caching is as easy as providing `CACHE_CONFIG` and `DATA_CACHE_CONFIG` in your
`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the
local filesystem.
- Memcached: we recommend using [pylibmc](https://pypi.org/project/pylibmc/) client library as
`python-memcached` does not handle storing binary data correctly.
- Redis: we recommend the [redis](https://pypi.python.org/pypi/redis) Python package
Both of these libraries can be installed using pip.
For chart data, Superset goes up a “timeout search path”, from a slice's configuration
to the datasources, the databases, then ultimately falls back to the global default
defined in `DATA_CACHE_CONFIG`.
```
DATA_CACHE_CONFIG = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24, # 1 day default (in secs)
'CACHE_KEY_PREFIX': 'superset_results',
'CACHE_REDIS_URL': 'redis://localhost:6379/0',
}
```
Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics.
Superset has a Celery task that will periodically warm up the cache based on different strategies.
To use it, add the following to the `CELERYBEAT_SCHEDULE` section in `config.py`:
```python
CELERYBEAT_SCHEDULE = {
'cache-warmup-hourly': {
'task': 'cache-warmup',
'schedule': crontab(minute=0, hour='*'), # hourly
'kwargs': {
'strategy_name': 'top_n_dashboards',
'top_n': 5,
'since': '7 days ago',
},
},
}
```
This will cache all the charts in the top 5 most popular dashboards every hour. For other
strategies, check the `superset/tasks/cache.py` file.
### Caching Thumbnails
This is an optional feature that can be turned on by activating its feature flag on config:
```
FEATURE_FLAGS = {
"THUMBNAILS": True,
"THUMBNAILS_SQLA_LISTENERS": True,
}
```
For this feature you will need a cache system and celery workers. All thumbnails are stored on cache
and are processed asynchronously by the workers.
An example config where images are stored on S3 could be:
```python
from flask import Flask
from s3cache.s3cache import S3Cache
...
class CeleryConfig(object):
BROKER_URL = "redis://localhost:6379/0"
CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks", "superset.tasks.thumbnails")
CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
CELERYD_PREFETCH_MULTIPLIER = 10
CELERY_ACKS_LATE = True
CELERY_CONFIG = CeleryConfig
def init_thumbnail_cache(app: Flask) -> S3Cache:
return S3Cache("bucket_name", 'thumbs_cache/')
THUMBNAIL_CACHE_CONFIG = init_thumbnail_cache
# Async selenium thumbnail task will use the following user
THUMBNAIL_SELENIUM_USER = "Admin"
```
Using the above example cache keys for dashboards will be `superset_thumb__dashboard__{ID}`. You can
override the base URL for selenium using:
```
WEBDRIVER_BASEURL = "https://superset.company.com"
```
Additional selenium web drive configuration can be set using `WEBDRIVER_CONFIGURATION`. You can
implement a custom function to authenticate selenium. The default function uses the `flask-login`
session cookie. Here's an example of a custom function signature:
```python
def auth_driver(driver: WebDriver, user: "User") -> WebDriver:
pass
```
Then on configuration:
```
WEBDRIVER_AUTH_FUNC = auth_driver
```