QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-04-19 08:04:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improvements related to ASF release process (#7539)

Browse Source 
* [WiP] Improvements related to ASF release process

* Removing translation binaries (.mo)
* Working on 2 docker files to help package and validate releases
  * Dockerfile.from_tarball: takes a VERSION as input, downloads
    official source release fro svn and bakes it into a functional
    docker image that can be validated
  * Dockerfile.make_tarball: helps a maintainer to package and sign a
    release candidate or release

* Address COPYRIGHT + LICENSE issues

(cherry picked from commit 5ae2836b25)

* Add Roboto font to LICENSE, remove glyphicons files

(cherry picked from commit 9615f39f7a556afe217b4e3288c9116f8ffaab91)

* remove unused LICENSE entries

* Change babytux to open image in birth dashboard

(cherry picked from commit 2776d11893596a029b8af37b96fd6ce1a9f12059)

* Improve instructions

* Docker tweaks

(cherry picked from commit df6e6462458ea8cff1482c142921b20f5607ad8e)

* Include image
This commit is contained in:
Maxime Beauchemin
2019-07-03 22:33:14 -07:00
committed by GitHub
parent 08b4a1735a
commit 5c7a50c104
44 changed files with 254 additions and 244 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
RELEASING/Dockerfile.from_tarball Normal file
View File

@@ -0,0 +1,71 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
FROM python:3.6-jessie
RUN useradd --user-group --create-home --no-log-init --shell /bin/bash superset
# Configure environment
ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8
RUN apt-get update -y
# Install dependencies to fix `curl https support error` and `elaying package configuration warning`
RUN apt-get install -y apt-transport-https apt-utils
# Install superset dependencies
# https://superset.incubator.apache.org/installation.html#os-dependencies
RUN apt-get install -y build-essential libssl-dev \
libffi-dev python3-dev libsasl2-dev libldap2-dev libxi-dev
# Install nodejs for custom build
# https://superset.incubator.apache.org/installation.html#making-your-own-build
# https://nodejs.org/en/download/package-manager/
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
&& apt-get install -y nodejs
RUN mkdir -p /home/superset
RUN chown superset /home/superset
WORKDIR /home/superset
ARG VERSION
RUN svn co https://dist.apache.org/repos/dist/dev/incubator/superset/$VERSION ./
RUN tar -xvf *.tar.gz
WORKDIR apache-superset-incubating-$VERSION
RUN cd superset/assets \
&& npm ci \
&& npm run build \
&& rm -rf node_modules
WORKDIR /home/superset/apache-superset-incubating-$VERSION
RUN pip install --upgrade setuptools pip \
&& pip install -r requirements.txt -r requirements-dev.txt \
&& pip install -e . \
&& rm -rf /root/.cache/pip
RUN fabmanager babel-compile --target superset/translations
RUN pip install -e . \
&& rm -rf /root/.cache/pip
ENV PATH=/home/superset/superset/bin:$PATH \
PYTHONPATH=/home/superset/superset/:$PYTHONPATH
COPY from_tarball_entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

19
RELEASING/Dockerfile.make_tarball Normal file
View File

@@ -0,0 +1,19 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
FROM python:3.6-jessie
COPY make_tarball.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

163
RELEASING/README.md Normal file
View File

@@ -0,0 +1,163 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Apache Releases
To make a fresh tarball of a git reference on apache/incubator-superset
(push your tag first!)
```bash
docker build -t make_tarball -f Dockerfile.make_tarball . --build-arg VERSION=0.33.0rc1
docker run make_tarball -f Dockerfile.make_tarball --env VERSION=0.33.0rc1
```
To make a working build given a tarball
```bash
# Building a docker from a tarball
VERSION=0.33.0rc2 && \
docker build -t apache-superset:$VERSION -f Dockerfile.from_tarball . --build-arg VERSION=$VERSION
# testing the resulting docker
docker run -p 5001:8088 apache-superset:0.33.0rc2
# you should be able to access localhost:5001 on your browser
# login using admin/admin
```
## Refresh documentation website
Every once in a while we want to compile the documentation and publish it.
Here's how to do it.
```bash
# install doc dependencies
pip install -r docs/requirements.txt
# build the docs
python setup.py build_sphinx
# copy html files to temp folder
cp -r docs/_build/html/ /tmp/tmp_superset_docs/
# clone the docs repo
cd ~/
git clone https://git-wip-us.apache.org/repos/asf/incubator-superset-site.git
# copy
cp -r /tmp/tmp_superset_docs/ ~/incubator-superset-site.git/
# commit and push to `asf-site` branch
cd ~/incubator-superset-site.git/
git checkout asf-site
git add .
git commit -a -m "New doc version"
git push origin master
```
# Apache Releases
You'll probably want to run these commands manually and understand what
they do prior to doing so.
## Release setup
First you need to setup a few things. This is a one-off and doesn't
need to be done at every release.
```bash
# Create PGP Key, and use your @apache.org email address
gpg --gen-key
# Checkout ASF dist repo
svn checkout https://dist.apache.org/repos/dist/dev/incubator/superset/ ~/svn/superset_dev
svn checkout https://dist.apache.org/repos/dist/release/incubator/superset/ ~/svn/superset
cd ~/svn/superset
# Add your GPG pub key to KEYS file. Replace "Maxime Beauchemin" with your name
export FULLNAME="Maxime Beauchemin"
(gpg --list-sigs $FULLNAME && gpg --armor --export $FULLNAME ) >> KEYS
# Commit the changes
svn commit -m "Add PGP keys of new Superset committer"
```
## Crafting tarball and signatures
Now let's craft a source release
```bash
# Assuming these commands are executed from the root of the repo
# Setting a VERSION var will be useful
export VERSION=0.31.0rc18
export RELEASE=apache-superset-incubating-${VERSION}
export RELEASE_TARBAL=${RELEASE}-source.tar.gz
# Let's create a git tag
git tag -f ${VERSION}
# [WARNING!] This command wipes everything in your repo that is
# gitignored in preparation for the source release.
# You may want to check that there's nothing your care about here first.
# Alternatively you could clone the repo into another location as in
# git clone git@github.com:apache/incubator-superset.git superset-releases
git clean -fxd
# Create the target folder
mkdir -p ~/svn/superset_dev/${VERSION}/
git archive \
--format=tar.gz ${VERSION} \
--prefix=${RELEASE}/ \
-o ~/svn/superset_dev/${VERSION}/${RELEASE_TARBALL}
cd ~/svn/superset_dev/
scripts/sign.sh ${RELEASE}-source.tar.gz
```
## Shipping to SVN
Now let's ship this RC into svn's dev folder
```bash
# cp or mv the files over to the svn repo
mkdir ~/svn/superset_dev/${VERSION}/
cp ${RELEASE_TARBALL} ~/svn/superset_dev/${VERSION}/
cd ~/svn/superset_dev/
svn add ${VERSION}
svn commit
```
Now you're ready to start the VOTE thread.
## Validating a release
https://www.apache.org/info/verification.html
## Publishing a successful release
Upon a successful vote, you'll have to copy the folder into the non-"dev/"
folder.
```bash
cp -r ~/svn/superset_dev/${VERSION}/ ~/svn/superset/${VERSION}/
cd ~/svn/superset/
svn add ${VERSION}
svn commit
```
Now you can announce the release on the mailing list, make sure to use the
proper template

42
RELEASING/from_tarball_entrypoint.sh Executable file
View File

@@ -0,0 +1,42 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -ex
echo "[WARNING] this entrypoint creates an admin/admin user"
echo "[WARNING] it should only be used for lightweight testing/validation"
# Create an admin user (you will be prompted to set username, first and last name before setting a password)
fabmanager create-admin \
--app superset \
--username admin \
--firstname admin \
--lastname admin \
--email admin@admin.com \
--password admin
# Initialize the database
superset db upgrade
# Loading examples
superset load_examples
# Create default roles and permissions
superset init
FLASK_ENV=development FLASK_APP=superset:app \
flask run -p 8088 --with-threads --reload --debugger --host=0.0.0.0

38
RELEASING/make_tarball.sh Executable file
View File

@@ -0,0 +1,38 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -ex
if [ -z "$VERSION" ]; then
echo "VERSION is required to run this container"
exit 1
fi
echo "version: $VERSION"
cd /tmp
git clone --depth 1 --branch $VERSION https://github.com/apache/incubator-superset.git
mkdir ~/$VERSION
cd incubator-superset && \
git archive \
--format=tar.gz \
--prefix=apache-superset-incubating-$VERSION/ \
HEAD \
-o ~/$VERSION/apache-superset-incubating.tar.gz
gpg --armor --output apache-superset-incubating.tar.gz.asc --detach-sig apache-superset-incubating.tar.gz
gpg --print-md SHA512 apache-superset-incubating.tar.gz > apache-superset-incubating.tar.gz.sha512