[load_examples] download data at runtime (#7314)

* [load_examples] download data at runtime

When running `superset load_examples` to load example data sets,
Superset used to load from the local package. This created a few issues
notably around licensing (what are these datasets licensed as?) and
around package size.

For now, I moved the data sets here:
https://github.com/apache-superset/examples-data

Altered the logic to download the data from where it is stored.

* flakes
This commit is contained in:
Maxime Beauchemin
2019-04-17 13:19:14 -07:00
committed by GitHub
parent 9341995803
commit 3d08266714
26 changed files with 64 additions and 203 deletions

View File

@@ -16,7 +16,6 @@
# under the License.
import datetime
import json
import os
import random
import pandas as pd
@@ -27,7 +26,7 @@ from superset.utils import core as utils
from .helpers import (
config,
Dash,
DATA_FOLDER,
get_example_data,
get_slice_json,
merge_slice,
Slice,
@@ -38,8 +37,9 @@ from .helpers import (
def load_unicode_test_data():
"""Loading unicode test dataset from a csv file in the repo"""
df = pd.read_csv(os.path.join(DATA_FOLDER, 'unicode_utf8_unixnl_test.csv'),
encoding='utf-8')
data = get_example_data(
'unicode_utf8_unixnl_test.csv', is_gzip=False, make_bytes=True)
df = pd.read_csv(data, encoding='utf-8')
# generate date/numeric data
df['dttm'] = datetime.datetime.now().date()
df['value'] = [random.randint(1, 100) for _ in range(len(df))]