Mike Bridge 58a1a1a8d1 build(scripts): add stress-test data generator for migration timing
Add ``scripts/seed_junction_load.py``, a backend-agnostic script that
bulk-inserts synthetic parent rows (dashboards, slices, users, roles,
tables, dbs) and many-to-many junction rows for the four largest
association tables targeted by the composite-PK migration:
``dashboard_slices``, ``slice_user``, ``dashboard_user``,
``dashboard_roles``.

Designed for measuring migration runtime at varying scales — run with
a series of size flags (100K / 1M / 5M / 10M for the target table)
and time the migration at each scale to verify the predicted
``O(N log N)`` extrapolation against real numbers.

Properties:
- **Reproducible**: deterministic cross-product walk through parent IDs
  produces a stable pair sequence; re-running is replayable.
- **Idempotent**: re-running with the same target is a no-op; with a
  higher target, only new rows are added.
- **Backend-agnostic**: connects via Superset's standard ``DATABASE_*``
  env vars (or ``SUPERSET__SQLALCHEMY_DATABASE_URI``). Branches on
  dialect for ``BINARY(16)`` vs ``UUID`` vs TEXT/BLOB UUID columns.
- **Batched**: bulk INSERT 10K rows per statement.
- **Per-phase timing**: logs elapsed wall time for the parents phase,
  the junctions phase as a whole, and per junction-table.
- **Avoidance set**: loads existing junction pairs into a Python set
  so re-runs on top of pre-existing data don't collide on the
  uniqueness constraint.

Usage (inside the Superset container):

    docker exec superset-superset-1 \\
        /app/.venv/bin/python /app/scripts/seed_junction_load.py \\
        --dashboard-slices 1000000

Defaults target a "large multi-team install" shape: 1M
``dashboard_slices``, 100K each ``slice_user`` / ``dashboard_user``,
10K ``dashboard_roles``. Override per-table via flags.

Tested locally on MySQL (the user's current eval stack):
- 200/100/100/50 row mini-run produced expected counts.
- Re-running at the same target is a no-op (idempotent).
- ``--dry-run`` plans without writing.

Junction tables not yet covered (``sqlatable_user``, ``rls_filter_*``,
``report_schedule_user``) are typically small in production and
require additional parent seeding (RLS filters, report schedules)
that wasn't worth the scope here. Adding them is straightforward by
extending ``JUNCTIONS`` and writing the corresponding parent seeder.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 18:42:05 -06:00
2025-12-04 13:18:34 -05:00
2024-04-15 11:21:42 -06:00

Superset

License Latest Release on Github Build Status PyPI version PyPI GitHub Stars Contributors Last Commit Open Issues Open PRs Get on Slack Documentation

Superset logo (light)

A modern, enterprise-ready business intelligence web application.

Documentation

  • User Guide — For analysts and business users. Explore data, build charts, create dashboards, and connect databases.
  • Administrator Guide — Install, configure, and operate Superset. Covers security, scaling, and database drivers.
  • Developer Guide — Contribute to Superset or build on its REST API and extension framework.

Why Superset? | Supported Databases | Release Notes | Get Involved | Resources | Organizations Using Superset

Why Superset?

Superset is a modern data exploration and data visualization platform. Superset can replace or augment proprietary business intelligence tools for many teams. Superset integrates well with a variety of data sources.

Superset provides:

  • A no-code interface for building charts quickly
  • A powerful, web-based SQL Editor for advanced querying
  • A lightweight semantic layer for quickly defining custom dimensions and metrics
  • Out of the box support for nearly any SQL database or data engine
  • A wide array of beautiful visualizations to showcase your data, ranging from simple bar charts to geospatial visualizations
  • Lightweight, configurable caching layer to help ease database load
  • Highly extensible security roles and authentication options
  • An API for programmatic customization
  • A cloud-native architecture designed from the ground up for scale

Screenshots & Gifs

Video Overview

superset-video-1080p.webm


Large Gallery of Visualizations


Craft Beautiful, Dynamic Dashboards


No-Code Chart Builder


Powerful SQL Editor


Supported Databases

Superset can query data from any SQL-speaking datastore or data engine (Presto, Trino, Athena, and more) that has a Python DB-API driver and a SQLAlchemy dialect.

Here are some of the major database solutions that are supported:

Amazon Athena   Amazon DynamoDB   Amazon Redshift   Apache Doris   Apache Drill   Apache Druid   Apache Hive   Apache Impala   Apache Kylin   Apache Pinot   Apache Solr   Apache Spark SQL   Ascend   Aurora MySQL (Data API)   Aurora PostgreSQL (Data API)   Azure Data Explorer   Azure Synapse   ClickHouse   Cloudflare D1   CockroachDB   Couchbase   CrateDB   Databend   Databricks   Denodo   Dremio   DuckDB   Elasticsearch   Exasol   Firebird   Firebolt   Google BigQuery   Google Sheets   Greenplum   Hologres   IBM Db2   IBM Netezza Performance Server   MariaDB   Microsoft SQL Server   MonetDB   MongoDB   MotherDuck   OceanBase   Oracle   Presto   RisingWave   SAP HANA   SAP Sybase   Shillelagh   SingleStore   Snowflake   SQLite   StarRocks   Superset meta database   TDengine   Teradata   TimescaleDB   Trino   Vertica   YDB   YugabyteDB

A more comprehensive list of supported databases along with the configuration instructions can be found here.

Want to add support for your datastore or data engine? Read more here about the technical requirements.

Installation and Configuration

Try out Superset's quickstart guide or learn about the options for production deployments.

Get Involved

Contributor Guide

Interested in contributing? Check out our Developer Guide to find resources around contributing along with a detailed guide on how to set up a development environment.

Resources

Understanding the Superset Points of View

Repo Activity

Performance Stats of apache/superset - Last 28 days
Languages
Python 38%
TypeScript 37.8%
Jupyter Notebook 21.3%
HTML 2.3%
JavaScript 0.3%
Other 0.2%