QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-05-07 08:54:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
5fdf8e4441bcf8294de5d38369829e5c9744cc0e
superset2/installation.html

693 lines
45 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Installation &amp; Configuration &mdash; Superset&#39;s documentation documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="index" title="Index"
href="genindex.html"/>
<link rel="search" title="Search" href="search.html"/>
<link rel="top" title="Superset&#39;s documentation documentation" href="index.html"/>
<link rel="next" title="Tutorial for Superset Administrators" href="tutorial.html"/>
<link rel="prev" title="Superset’s documentation" href="index.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<a href="index.html" class="icon icon-home"> Superset's documentation
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Installation &amp; Configuration</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#getting-started">Getting Started</a></li>
<li class="toctree-l2"><a class="reference internal" href="#os-dependencies">OS dependencies</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-virtualenv">Python virtualenv</a></li>
<li class="toctree-l2"><a class="reference internal" href="#python-s-setup-tools-and-pip">Python&#8217;s setup tools and pip</a></li>
<li class="toctree-l2"><a class="reference internal" href="#superset-installation-and-initialization">Superset installation and initialization</a></li>
<li class="toctree-l2"><a class="reference internal" href="#configuration-behind-a-load-balancer">Configuration behind a load balancer</a></li>
<li class="toctree-l2"><a class="reference internal" href="#configuration">Configuration</a></li>
<li class="toctree-l2"><a class="reference internal" href="#database-dependencies">Database dependencies</a></li>
<li class="toctree-l2"><a class="reference internal" href="#aws-athena">(AWS) Athena</a></li>
<li class="toctree-l2"><a class="reference internal" href="#caching">Caching</a></li>
<li class="toctree-l2"><a class="reference internal" href="#deeper-sqlalchemy-integration">Deeper SQLAlchemy integration</a></li>
<li class="toctree-l2"><a class="reference internal" href="#schemas-postgres-redshift">Schemas (Postgres &amp; Redshift)</a></li>
<li class="toctree-l2"><a class="reference internal" href="#ssl-access-to-databases">SSL Access to databases</a></li>
<li class="toctree-l2"><a class="reference internal" href="#druid">Druid</a></li>
<li class="toctree-l2"><a class="reference internal" href="#cors">CORS</a></li>
<li class="toctree-l2"><a class="reference internal" href="#middleware">MIDDLEWARE</a></li>
<li class="toctree-l2"><a class="reference internal" href="#upgrading">Upgrading</a></li>
<li class="toctree-l2"><a class="reference internal" href="#sql-lab">SQL Lab</a></li>
<li class="toctree-l2"><a class="reference internal" href="#making-your-own-build">Making your own build</a></li>
<li class="toctree-l2"><a class="reference internal" href="#blueprints">Blueprints</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial for Superset Administrators</a></li>
<li class="toctree-l1"><a class="reference internal" href="security.html">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="sqllab.html">SQL Lab</a></li>
<li class="toctree-l1"><a class="reference internal" href="visualization.html">Visualization Tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="videos.html">Videos</a></li>
<li class="toctree-l1"><a class="reference internal" href="gallery.html">Gallery</a></li>
<li class="toctree-l1"><a class="reference internal" href="druid.html">Druid</a></li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">Superset's documentation</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html">Docs</a> &raquo;</li>
<li>Installation &amp; Configuration</li>
<li class="wy-breadcrumbs-aside">
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="installation-configuration">
<h1>Installation &amp; Configuration<a class="headerlink" href="#installation-configuration" title="Permalink to this headline"></a></h1>
<div class="section" id="getting-started">
<h2>Getting Started<a class="headerlink" href="#getting-started" title="Permalink to this headline"></a></h2>
<p>Superset is tested against Python <code class="docutils literal"><span class="pre">2.7</span></code> and Python <code class="docutils literal"><span class="pre">3.4</span></code>.
Airbnb currently uses 2.7.* in production. We do not plan on supporting
Python <code class="docutils literal"><span class="pre">2.6</span></code>.</p>
</div>
<div class="section" id="os-dependencies">
<h2>OS dependencies<a class="headerlink" href="#os-dependencies" title="Permalink to this headline"></a></h2>
<p>Superset stores database connection information in its metadata database.
For that purpose, we use the <code class="docutils literal"><span class="pre">cryptography</span></code> Python library to encrypt
connection passwords. Unfortunately this library has OS level dependencies.</p>
<p>You may want to attempt the next step
(&#8220;Superset installation and initialization&#8221;) and come back to this step if
you encounter an error.</p>
<p>Here&#8217;s how to install them:</p>
<p>For <strong>Debian</strong> and <strong>Ubuntu</strong>, the following command will ensure that
the required dependencies are installed:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">build</span><span class="o">-</span><span class="n">essential</span> <span class="n">libssl</span><span class="o">-</span><span class="n">dev</span> <span class="n">libffi</span><span class="o">-</span><span class="n">dev</span> <span class="n">python</span><span class="o">-</span><span class="n">dev</span> <span class="n">python</span><span class="o">-</span><span class="n">pip</span> <span class="n">libsasl2</span><span class="o">-</span><span class="n">dev</span> <span class="n">libldap2</span><span class="o">-</span><span class="n">dev</span>
</pre></div>
</div>
<p>For <strong>Fedora</strong> and <strong>RHEL-derivatives</strong>, the following command will ensure
that the required dependencies are installed:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">yum</span> <span class="n">upgrade</span> <span class="n">python</span><span class="o">-</span><span class="n">setuptools</span>
<span class="n">sudo</span> <span class="n">yum</span> <span class="n">install</span> <span class="n">gcc</span> <span class="n">gcc</span><span class="o">-</span><span class="n">c</span><span class="o">++</span> <span class="n">libffi</span><span class="o">-</span><span class="n">devel</span> <span class="n">python</span><span class="o">-</span><span class="n">devel</span> <span class="n">python</span><span class="o">-</span><span class="n">pip</span> <span class="n">python</span><span class="o">-</span><span class="n">wheel</span> <span class="n">openssl</span><span class="o">-</span><span class="n">devel</span> <span class="n">libsasl2</span><span class="o">-</span><span class="n">devel</span> <span class="n">openldap</span><span class="o">-</span><span class="n">devel</span>
</pre></div>
</div>
<p><strong>OSX</strong>, system python is not recommended. brew&#8217;s python also ships with pip</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">brew</span> <span class="n">install</span> <span class="n">pkg</span><span class="o">-</span><span class="n">config</span> <span class="n">libffi</span> <span class="n">openssl</span> <span class="n">python</span>
<span class="n">env</span> <span class="n">LDFLAGS</span><span class="o">=</span><span class="s2">&quot;-L$(brew --prefix openssl)/lib&quot;</span> <span class="n">CFLAGS</span><span class="o">=</span><span class="s2">&quot;-I$(brew --prefix openssl)/include&quot;</span> <span class="n">pip</span> <span class="n">install</span> <span class="n">cryptography</span><span class="o">==</span><span class="mf">1.7</span><span class="o">.</span><span class="mi">2</span>
</pre></div>
</div>
<p><strong>Windows</strong> isn&#8217;t officially supported at this point, but if you want to
attempt it, download <a class="reference external" href="https://bootstrap.pypa.io/get-pip.py">get-pip.py</a>, and run <code class="docutils literal"><span class="pre">python</span> <span class="pre">get-pip.py</span></code> which may need admin access. Then run the following:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">C</span><span class="p">:</span>\<span class="o">&gt;</span> <span class="n">pip</span> <span class="n">install</span> <span class="n">cryptography</span>
<span class="c1"># You may also have to create C:\Temp</span>
<span class="n">C</span><span class="p">:</span>\<span class="o">&gt;</span> <span class="n">md</span> <span class="n">C</span><span class="p">:</span>\<span class="n">Temp</span>
</pre></div>
</div>
</div>
<div class="section" id="python-virtualenv">
<h2>Python virtualenv<a class="headerlink" href="#python-virtualenv" title="Permalink to this headline"></a></h2>
<p>It is recommended to install Superset inside a virtualenv. Python 3 already ships virtualenv, for
Python 2 you need to install it. If it&#8217;s packaged for your operating systems install it from there
otherwise you can install from pip:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">virtualenv</span>
</pre></div>
</div>
<p>You can create and activate a virtualenv by:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="c1"># virtualenv is shipped in Python 3 as pyvenv</span>
<span class="n">virtualenv</span> <span class="n">venv</span>
<span class="o">.</span> <span class="o">./</span><span class="n">venv</span><span class="o">/</span><span class="nb">bin</span><span class="o">/</span><span class="n">activate</span>
</pre></div>
</div>
<p>On windows the syntax for activating it is a bit different:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">venv</span>\<span class="n">Scripts</span>\<span class="n">activate</span>
</pre></div>
</div>
<p>Once you activated your virtualenv everything you are doing is confined inside the virtualenv.
To exit a virtualenv just type <code class="docutils literal"><span class="pre">deactivate</span></code>.</p>
</div>
<div class="section" id="python-s-setup-tools-and-pip">
<h2>Python&#8217;s setup tools and pip<a class="headerlink" href="#python-s-setup-tools-and-pip" title="Permalink to this headline"></a></h2>
<p>Put all the chances on your side by getting the very latest <code class="docutils literal"><span class="pre">pip</span></code>
and <code class="docutils literal"><span class="pre">setuptools</span></code> libraries.:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="o">--</span><span class="n">upgrade</span> <span class="n">setuptools</span> <span class="n">pip</span>
</pre></div>
</div>
</div>
<div class="section" id="superset-installation-and-initialization">
<h2>Superset installation and initialization<a class="headerlink" href="#superset-installation-and-initialization" title="Permalink to this headline"></a></h2>
<p>Follow these few simple steps to install Superset.:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="c1"># Install superset</span>
<span class="n">pip</span> <span class="n">install</span> <span class="n">superset</span>
<span class="c1"># Create an admin user (you will be prompted to set username, first and last name before setting a password)</span>
<span class="n">fabmanager</span> <span class="n">create</span><span class="o">-</span><span class="n">admin</span> <span class="o">--</span><span class="n">app</span> <span class="n">superset</span>
<span class="c1"># Initialize the database</span>
<span class="n">superset</span> <span class="n">db</span> <span class="n">upgrade</span>
<span class="c1"># Load some data to play with</span>
<span class="n">superset</span> <span class="n">load_examples</span>
<span class="c1"># Create default roles and permissions</span>
<span class="n">superset</span> <span class="n">init</span>
<span class="c1"># Start the web server on port 8088, use -p to bind to another port</span>
<span class="n">superset</span> <span class="n">runserver</span>
<span class="c1"># To start a development web server, use the -d switch</span>
<span class="c1"># superset runserver -d</span>
</pre></div>
</div>
<p>After installation, you should be able to point your browser to the right
hostname:port <a class="reference external" href="http://localhost:8088">http://localhost:8088</a>, login using
the credential you entered while creating the admin account, and navigate to
<cite>Menu -&gt; Admin -&gt; Refresh Metadata</cite>. This action should bring in all of
your datasources for Superset to be aware of, and they should show up in
<cite>Menu -&gt; Datasources</cite>, from where you can start playing with your data!</p>
<p>Please note that <em>gunicorn</em>, Superset default application server, does not
work on Windows so you need to use the development web server.
The development web server though is not intended to be used on production systems
so better use a supported platform that can run <em>gunicorn</em>.</p>
</div>
<div class="section" id="configuration-behind-a-load-balancer">
<h2>Configuration behind a load balancer<a class="headerlink" href="#configuration-behind-a-load-balancer" title="Permalink to this headline"></a></h2>
<p>If you are running superset behind a load balancer or reverse proxy (e.g. NGINX
or ELB on AWS), you may need to utilise a healthcheck endpoint so that your
load balancer knows if your superset instance is running. This is provided
at <code class="docutils literal"><span class="pre">/health</span></code> which will return a 200 response containing &#8220;OK&#8221; if the
webserver is running.</p>
<p>If the load balancer is inserting X-Forwarded-For/X-Forwarded-Proto headers, you
should set <cite>ENABLE_PROXY_FIX = True</cite> in the superset config file to extract and use
the headers.</p>
</div>
<div class="section" id="configuration">
<h2>Configuration<a class="headerlink" href="#configuration" title="Permalink to this headline"></a></h2>
<p>To configure your application, you need to create a file (module)
<code class="docutils literal"><span class="pre">superset_config.py</span></code> and make sure it is in your PYTHONPATH. Here are some
of the parameters you can copy / paste in that configuration module:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="c1">#---------------------------------------------------------</span>
<span class="c1"># Superset specific config</span>
<span class="c1">#---------------------------------------------------------</span>
<span class="n">ROW_LIMIT</span> <span class="o">=</span> <span class="mi">5000</span>
<span class="n">SUPERSET_WORKERS</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">SUPERSET_WEBSERVER_PORT</span> <span class="o">=</span> <span class="mi">8088</span>
<span class="c1">#---------------------------------------------------------</span>
<span class="c1">#---------------------------------------------------------</span>
<span class="c1"># Flask App Builder configuration</span>
<span class="c1">#---------------------------------------------------------</span>
<span class="c1"># Your App secret key</span>
<span class="n">SECRET_KEY</span> <span class="o">=</span> <span class="s1">&#39;</span><span class="se">\2\1</span><span class="s1">thisismyscretkey</span><span class="se">\1\2</span><span class="s1">\e\y\y\h&#39;</span>
<span class="c1"># The SQLAlchemy connection string to your database backend</span>
<span class="c1"># This connection defines the path to the database that stores your</span>
<span class="c1"># superset metadata (slices, connections, tables, dashboards, ...).</span>
<span class="c1"># Note that the connection information to connect to the datasources</span>
<span class="c1"># you want to explore are managed directly in the web UI</span>
<span class="n">SQLALCHEMY_DATABASE_URI</span> <span class="o">=</span> <span class="s1">&#39;sqlite:////path/to/superset.db&#39;</span>
<span class="c1"># Flask-WTF flag for CSRF</span>
<span class="n">CSRF_ENABLED</span> <span class="o">=</span> <span class="kc">True</span>
<span class="c1"># Set this API key to enable Mapbox visualizations</span>
<span class="n">MAPBOX_API_KEY</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span>
</pre></div>
</div>
<p>This file also allows you to define configuration parameters used by
Flask App Builder, the web framework used by Superset. Please consult
the <a class="reference external" href="http://flask-appbuilder.readthedocs.org/en/latest/config.html">Flask App Builder Documentation</a>
for more information on how to configure Superset.</p>
<p>Please make sure to change:</p>
<ul class="simple">
<li><em>SQLALCHEMY_DATABASE_URI</em>, by default it is stored at <em>~/.superset/superset.db</em></li>
<li><em>SECRET_KEY</em>, to a long random string</li>
</ul>
</div>
<div class="section" id="database-dependencies">
<h2>Database dependencies<a class="headerlink" href="#database-dependencies" title="Permalink to this headline"></a></h2>
<p>Superset does not ship bundled with connectivity to databases, except
for Sqlite, which is part of the Python standard library.
You&#8217;ll need to install the required packages for the database you
want to use as your metadata database as well as the packages needed to
connect to the databases you want to access through Superset.</p>
<p>Here&#8217;s a list of some of the recommended packages.</p>
<table border="1" class="docutils">
<colgroup>
<col width="15%" />
<col width="37%" />
<col width="49%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">database</th>
<th class="head">pypi package</th>
<th class="head">SQLAlchemy URI prefix</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>MySQL</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">mysqlclient</span></code></td>
<td><code class="docutils literal"><span class="pre">mysql://</span></code></td>
</tr>
<tr class="row-odd"><td>Postgres</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">psycopg2</span></code></td>
<td><code class="docutils literal"><span class="pre">postgresql+psycopg2://</span></code></td>
</tr>
<tr class="row-even"><td>Presto</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">pyhive</span></code></td>
<td><code class="docutils literal"><span class="pre">presto://</span></code></td>
</tr>
<tr class="row-odd"><td>Oracle</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">cx_Oracle</span></code></td>
<td><code class="docutils literal"><span class="pre">oracle://</span></code></td>
</tr>
<tr class="row-even"><td>sqlite</td>
<td>&nbsp;</td>
<td><code class="docutils literal"><span class="pre">sqlite://</span></code></td>
</tr>
<tr class="row-odd"><td>Redshift</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">sqlalchemy-redshift</span></code></td>
<td><code class="docutils literal"><span class="pre">postgresql+psycopg2://</span></code></td>
</tr>
<tr class="row-even"><td>MSSQL</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">pymssql</span></code></td>
<td><code class="docutils literal"><span class="pre">mssql://</span></code></td>
</tr>
<tr class="row-odd"><td>Impala</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">impyla</span></code></td>
<td><code class="docutils literal"><span class="pre">impala://</span></code></td>
</tr>
<tr class="row-even"><td>SparkSQL</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">pyhive</span></code></td>
<td><code class="docutils literal"><span class="pre">jdbc+hive://</span></code></td>
</tr>
<tr class="row-odd"><td>Greenplum</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">psycopg2</span></code></td>
<td><code class="docutils literal"><span class="pre">postgresql+psycopg2://</span></code></td>
</tr>
<tr class="row-even"><td>Athena</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">&quot;PyAthenaJDBC&gt;1.0.9&quot;</span></code></td>
<td><code class="docutils literal"><span class="pre">awsathena+jdbc://</span></code></td>
</tr>
<tr class="row-odd"><td>Vertica</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span>
<span class="pre">sqlalchemy-vertica-python</span></code></td>
<td><code class="docutils literal"><span class="pre">vertica+vertica_python://</span></code></td>
</tr>
<tr class="row-even"><td>ClickHouse</td>
<td><code class="docutils literal"><span class="pre">pip</span> <span class="pre">install</span>
<span class="pre">sqlalchemy-clickhouse</span></code></td>
<td><code class="docutils literal"><span class="pre">clickhouse://</span></code></td>
</tr>
</tbody>
</table>
<p>Note that many other database are supported, the main criteria being the
existence of a functional SqlAlchemy dialect and Python driver. Googling
the keyword <code class="docutils literal"><span class="pre">sqlalchemy</span></code> in addition of a keyword that describes the
database you want to connect to should get you to the right place.</p>
</div>
<div class="section" id="aws-athena">
<h2>(AWS) Athena<a class="headerlink" href="#aws-athena" title="Permalink to this headline"></a></h2>
<p>This currently relies on an unreleased future version of <a class="reference external" href="https://github.com/laughingman7743/PyAthenaJDBC">PyAthenaJDBC</a>. If you&#8217;re adventurous or simply impatient, you can install directly from git:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">git</span><span class="o">+</span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">laughingman7743</span><span class="o">/</span><span class="n">PyAthenaJDBC</span><span class="nd">@support_sqlalchemy</span>
</pre></div>
</div>
<p>The connection string for Athena looks like this</p>
<div class="highlight-default"><div class="highlight"><pre><span></span>awsathena+jdbc://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}&amp;...
</pre></div>
</div>
<p>Where you need to escape/encode at least the s3_staging_dir, i.e.,</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">s3</span><span class="p">:</span><span class="o">//...</span> <span class="o">-&gt;</span> <span class="n">s3</span><span class="o">%</span><span class="mi">3</span><span class="n">A</span><span class="o">//...</span>
</pre></div>
</div>
</div>
<div class="section" id="caching">
<h2>Caching<a class="headerlink" href="#caching" title="Permalink to this headline"></a></h2>
<p>Superset uses <a class="reference external" href="https://pythonhosted.org/Flask-Cache/">Flask-Cache</a> for
caching purpose. Configuring your caching backend is as easy as providing
a <code class="docutils literal"><span class="pre">CACHE_CONFIG</span></code>, constant in your <code class="docutils literal"><span class="pre">superset_config.py</span></code> that
complies with the Flask-Cache specifications.</p>
<p>Flask-Cache supports multiple caching backends (Redis, Memcached,
SimpleCache (in-memory), or the local filesystem). If you are going to use
Memcached please use the <cite>pylibmc</cite> client library as <cite>python-memcached</cite> does
not handle storing binary data correctly. If you use Redis, please install
the <a class="reference external" href="https://pypi.python.org/pypi/redis">redis</a> Python package:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">redis</span>
</pre></div>
</div>
<p>For setting your timeouts, this is done in the Superset metadata and goes
up the &#8220;timeout searchpath&#8221;, from your slice configuration, to your
data source&#8217;s configuration, to your database&#8217;s and ultimately falls back
into your global default defined in <code class="docutils literal"><span class="pre">CACHE_CONFIG</span></code>.</p>
</div>
<div class="section" id="deeper-sqlalchemy-integration">
<h2>Deeper SQLAlchemy integration<a class="headerlink" href="#deeper-sqlalchemy-integration" title="Permalink to this headline"></a></h2>
<p>It is possible to tweak the database connection information using the
parameters exposed by SQLAlchemy. In the <code class="docutils literal"><span class="pre">Database</span></code> edit view, you will
find an <code class="docutils literal"><span class="pre">extra</span></code> field as a <code class="docutils literal"><span class="pre">JSON</span></code> blob.</p>
<a class="reference internal image-reference" href="_images/add_db.png"><img alt="_images/add_db.png" src="_images/add_db.png" style="width: 534.0px; height: 370.8px;" /></a>
<p>This JSON string contains extra configuration elements. The <code class="docutils literal"><span class="pre">engine_params</span></code>
object gets unpacked into the
<a class="reference external" href="http://docs.sqlalchemy.org/en/latest/core/engines.html#sqlalchemy.create_engine">sqlalchemy.create_engine</a> call,
while the <code class="docutils literal"><span class="pre">metadata_params</span></code> get unpacked into the
<a class="reference external" href="http://docs.sqlalchemy.org/en/rel_1_0/core/metadata.html#sqlalchemy.schema.MetaData">sqlalchemy.MetaData</a> call. Refer to the SQLAlchemy docs for more information.</p>
</div>
<div class="section" id="schemas-postgres-redshift">
<h2>Schemas (Postgres &amp; Redshift)<a class="headerlink" href="#schemas-postgres-redshift" title="Permalink to this headline"></a></h2>
<p>Postgres and Redshift, as well as other database,
use the concept of <strong>schema</strong> as a logical entity
on top of the <strong>database</strong>. For Superset to connect to a specific schema,
there&#8217;s a <strong>schema</strong> parameter you can set in the table form.</p>
</div>
<div class="section" id="ssl-access-to-databases">
<h2>SSL Access to databases<a class="headerlink" href="#ssl-access-to-databases" title="Permalink to this headline"></a></h2>
<p>This example worked with a MySQL database that requires SSL. The configuration
may differ with other backends. This is what was put in the <code class="docutils literal"><span class="pre">extra</span></code>
parameter</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;metadata_params&quot;</span><span class="p">:</span> <span class="p">{},</span>
<span class="s2">&quot;engine_params&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;connect_args&quot;</span><span class="p">:{</span>
<span class="s2">&quot;sslmode&quot;</span><span class="p">:</span><span class="s2">&quot;require&quot;</span><span class="p">,</span>
<span class="s2">&quot;sslrootcert&quot;</span><span class="p">:</span> <span class="s2">&quot;/path/to/my/pem&quot;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="druid">
<h2>Druid<a class="headerlink" href="#druid" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li>From the UI, enter the information about your clusters in the
<code class="docutils literal"><span class="pre">Admin-&gt;Clusters</span></code> menu by hitting the + sign.</li>
<li>Once the Druid cluster connection information is entered, hit the
<code class="docutils literal"><span class="pre">Admin-&gt;Refresh</span> <span class="pre">Metadata</span></code> menu item to populate</li>
<li>Navigate to your datasources</li>
</ul>
<p>Note that you can run the <code class="docutils literal"><span class="pre">superset</span> <span class="pre">refresh_druid</span></code> command to refresh the
metadata from your Druid cluster(s)</p>
</div>
<div class="section" id="cors">
<h2>CORS<a class="headerlink" href="#cors" title="Permalink to this headline"></a></h2>
<p>The extra CORS Dependency must be installed:</p>
<blockquote>
<div>superset[cors]</div></blockquote>
<p>The following keys in <cite>superset_config.py</cite> can be specified to configure CORS:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">ENABLE_CORS</span></code>: Must be set to True in order to enable CORS</li>
<li><code class="docutils literal"><span class="pre">CORS_OPTIONS</span></code>: options passed to Flask-CORS (<cite>documentation &lt;http://flask-cors.corydolphin.com/en/latest/api.html#extension&gt;</cite>)</li>
</ul>
</div>
<div class="section" id="middleware">
<h2>MIDDLEWARE<a class="headerlink" href="#middleware" title="Permalink to this headline"></a></h2>
<p>Superset allows you to add your own middleware. To add your own middleware, update the <code class="docutils literal"><span class="pre">ADDITIONAL_MIDDLEWARE</span></code> key in
your <cite>superset_config.py</cite>. <code class="docutils literal"><span class="pre">ADDITIONAL_MIDDLEWARE</span></code> should be a list of your additional middleware classes.</p>
<p>For example, to use AUTH_REMOTE_USER from behind a proxy server like nginx, you have to add a simple middleware class to
add the value of <code class="docutils literal"><span class="pre">HTTP_X_PROXY_REMOTE_USER</span></code> (or any other custom header from the proxy) to Gunicorn&#8217;s <code class="docutils literal"><span class="pre">REMOTE_USER</span></code>
environment variable:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">RemoteUserMiddleware</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">app</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">app</span> <span class="o">=</span> <span class="n">app</span>
<span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">environ</span><span class="p">,</span> <span class="n">start_response</span><span class="p">):</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">environ</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s1">&#39;HTTP_X_PROXY_REMOTE_USER&#39;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="n">environ</span><span class="p">[</span><span class="s1">&#39;REMOTE_USER&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">user</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">app</span><span class="p">(</span><span class="n">environ</span><span class="p">,</span> <span class="n">start_response</span><span class="p">)</span>
<span class="n">ADDITIONAL_MIDDLEWARE</span> <span class="o">=</span> <span class="p">[</span><span class="n">RemoteUserMiddleware</span><span class="p">,</span> <span class="p">]</span>
</pre></div>
</div>
<p><em>Adapted from http://flask.pocoo.org/snippets/69/</em></p>
</div>
<div class="section" id="upgrading">
<h2>Upgrading<a class="headerlink" href="#upgrading" title="Permalink to this headline"></a></h2>
<p>Upgrading should be as straightforward as running:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">superset</span> <span class="o">--</span><span class="n">upgrade</span>
<span class="n">superset</span> <span class="n">db</span> <span class="n">upgrade</span>
<span class="n">superset</span> <span class="n">init</span>
</pre></div>
</div>
</div>
<div class="section" id="sql-lab">
<h2>SQL Lab<a class="headerlink" href="#sql-lab" title="Permalink to this headline"></a></h2>
<p>SQL Lab is a powerful SQL IDE that works with all SQLAlchemy compatible
databases. By default, queries are executed in the scope of a web
request so they
may eventually timeout as queries exceed the maximum duration of a web
request in your environment, whether it&#8217;d be a reverse proxy or the Superset
server itself.</p>
<p>On large analytic databases, it&#8217;s common to run queries that
execute for minutes or hours.
To enable support for long running queries that
execute beyond the typical web request&#8217;s timeout (30-60 seconds), it is
necessary to configure an asynchronous backend for Superset which consist of:</p>
<ul class="simple">
<li>one or many Superset worker (which is implemented as a Celery worker), and
can be started with the <code class="docutils literal"><span class="pre">superset</span> <span class="pre">worker</span></code> command, run
<code class="docutils literal"><span class="pre">superset</span> <span class="pre">worker</span> <span class="pre">--help</span></code> to view the related options</li>
<li>a celery broker (message queue) for which we recommend using Redis
or RabbitMQ</li>
<li>a results backend that defines where the worker will persist the query
results</li>
</ul>
<p>Configuring Celery requires defining a <code class="docutils literal"><span class="pre">CELERY_CONFIG</span></code> in your
<code class="docutils literal"><span class="pre">superset_config.py</span></code>. Both the worker and web server processes should
have the same configuration.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">CeleryConfig</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="n">BROKER_URL</span> <span class="o">=</span> <span class="s1">&#39;redis://localhost:6379/0&#39;</span>
<span class="n">CELERY_IMPORTS</span> <span class="o">=</span> <span class="p">(</span><span class="s1">&#39;superset.sql_lab&#39;</span><span class="p">,</span> <span class="p">)</span>
<span class="n">CELERY_RESULT_BACKEND</span> <span class="o">=</span> <span class="s1">&#39;redis://localhost:6379/0&#39;</span>
<span class="n">CELERY_ANNOTATIONS</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;tasks.add&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;rate_limit&#39;</span><span class="p">:</span> <span class="s1">&#39;10/s&#39;</span><span class="p">}}</span>
<span class="n">CELERY_CONFIG</span> <span class="o">=</span> <span class="n">CeleryConfig</span>
</pre></div>
</div>
<p>To setup a result backend, you need to pass an instance of a derivative
of <code class="docutils literal"><span class="pre">werkzeug.contrib.cache.BaseCache</span></code> to the <code class="docutils literal"><span class="pre">RESULTS_BACKEND</span></code>
configuration key in your <code class="docutils literal"><span class="pre">superset_config.py</span></code>. It&#8217;s possible to use
Memcached, Redis, S3 (<a class="reference external" href="https://pypi.python.org/pypi/s3werkzeugcache">https://pypi.python.org/pypi/s3werkzeugcache</a>),
memory or the file system (in a single server-type setup or for testing),
or to write your own caching interface. Your <code class="docutils literal"><span class="pre">superset_config.py</span></code> may
look something like:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span> <span class="c1"># On S3</span>
<span class="kn">from</span> <span class="nn">s3cache.s3cache</span> <span class="kn">import</span> <span class="n">S3Cache</span>
<span class="n">S3_CACHE_BUCKET</span> <span class="o">=</span> <span class="s1">&#39;foobar-superset&#39;</span>
<span class="n">S3_CACHE_KEY_PREFIX</span> <span class="o">=</span> <span class="s1">&#39;sql_lab_result&#39;</span>
<span class="n">RESULTS_BACKEND</span> <span class="o">=</span> <span class="n">S3Cache</span><span class="p">(</span><span class="n">S3_CACHE_BUCKET</span><span class="p">,</span> <span class="n">S3_CACHE_KEY_PREFIX</span><span class="p">)</span>
<span class="c1"># On Redis</span>
<span class="kn">from</span> <span class="nn">werkzeug.contrib.cache</span> <span class="kn">import</span> <span class="n">RedisCache</span>
<span class="n">RESULTS_BACKEND</span> <span class="o">=</span> <span class="n">RedisCache</span><span class="p">(</span>
<span class="n">host</span><span class="o">=</span><span class="s1">&#39;localhost&#39;</span><span class="p">,</span> <span class="n">port</span><span class="o">=</span><span class="mi">6379</span><span class="p">,</span> <span class="n">key_prefix</span><span class="o">=</span><span class="s1">&#39;superset_results&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>Also note that SQL Lab supports Jinja templating in queries, and that it&#8217;s
possible to overload
the default Jinja context in your environment by defining the
<code class="docutils literal"><span class="pre">JINJA_CONTEXT_ADDONS</span></code> in your superset configuration. Objects referenced
in this dictionary are made available for users to use in their SQL.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">JINJA_CONTEXT_ADDONS</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">&#39;my_crazy_macro&#39;</span><span class="p">:</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">*</span><span class="mi">2</span><span class="p">,</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="making-your-own-build">
<h2>Making your own build<a class="headerlink" href="#making-your-own-build" title="Permalink to this headline"></a></h2>
<p>For more advanced users, you may want to build Superset from sources. That
would be the case if you fork the project to add features specific to
your environment.:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span># assuming $SUPERSET_HOME as the root of the repo
cd $SUPERSET_HOME/superset/assets
npm install
npm run build
cd $SUPERSET_HOME
python setup.py install
</pre></div>
</div>
</div>
<div class="section" id="blueprints">
<h2>Blueprints<a class="headerlink" href="#blueprints" title="Permalink to this headline"></a></h2>
<p><a class="reference external" href="http://flask.pocoo.org/docs/0.12/blueprints/">Blueprints are Flask&#8217;s reusable apps</a>.
Superset allows you to specify an array of Blueprints
in your <code class="docutils literal"><span class="pre">superset_config</span></code> module. Here&#8217;s
an example on how this can work with a simple Blueprint. By doing
so, you can expect Superset to serve a page that says &#8220;OK&#8221;
at the <code class="docutils literal"><span class="pre">/simple_page</span></code> url. This can allow you to run other things such
as custom data visualization applications alongside Superset, on the
same server.</p>
<p>..code</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">flask</span> <span class="k">import</span> <span class="n">Blueprint</span>
<span class="n">simple_page</span> <span class="o">=</span> <span class="n">Blueprint</span><span class="p">(</span><span class="s1">&#39;simple_page&#39;</span><span class="p">,</span> <span class="n">__name__</span><span class="p">,</span>
<span class="n">template_folder</span><span class="o">=</span><span class="s1">&#39;templates&#39;</span><span class="p">)</span>
<span class="nd">@simple_page</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">,</span> <span class="n">defaults</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;page&#39;</span><span class="p">:</span> <span class="s1">&#39;index&#39;</span><span class="p">})</span>
<span class="nd">@simple_page</span><span class="o">.</span><span class="n">route</span><span class="p">(</span><span class="s1">&#39;/&lt;page&gt;&#39;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">show</span><span class="p">(</span><span class="n">page</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">&quot;Ok&quot;</span>
<span class="n">BLUEPRINTS</span> <span class="o">=</span> <span class="p">[</span><span class="n">simple_page</span><span class="p">]</span>
</pre></div>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="tutorial.html" class="btn btn-neutral float-right" title="Tutorial for Superset Administrators" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="index.html" class="btn btn-neutral" title="Superset’s documentation" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink