diff --git a/docs/.claude/instructions.md b/docs/.claude/instructions.md index 54db2f6f121..4e3b5ccd538 100644 --- a/docs/.claude/instructions.md +++ b/docs/.claude/instructions.md @@ -111,5 +111,5 @@ InteractiveMenu.parameters = { - **Generator**: `docs/scripts/generate-superset-components.mjs` - **Wrapper**: `docs/src/components/StorybookWrapper.jsx` -- **Output**: `docs/developer_portal/components/` +- **Output**: `docs/developer_docs/components/` - **Stories**: `superset-frontend/packages/superset-ui-core/src/components/*/` diff --git a/docs/DOCS_CLAUDE.md b/docs/DOCS_CLAUDE.md index 039b701ba1d..1ac49462afe 100644 --- a/docs/DOCS_CLAUDE.md +++ b/docs/DOCS_CLAUDE.md @@ -416,7 +416,7 @@ If versions don't appear in dropdown: - [Docusaurus Documentation](https://docusaurus.io/docs) - [MDX Documentation](https://mdxjs.com/) -- [Superset Developer Portal](https://superset.apache.org/developer_portal/) +- [Superset Developer Docs](https://superset.apache.org/developer-docs/) - [Main Superset Documentation](https://superset.apache.org/docs/intro) ## 📖 Real Examples and Patterns diff --git a/docs/README.md b/docs/README.md index dbf7a51238b..6f3a6b4902e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -19,15 +19,16 @@ under the License. This is the public documentation site for Superset, built using [Docusaurus 3](https://docusaurus.io/). See the -[Developer Portal](https://superset.apache.org/developer_portal/contributing/development-setup#documentation) +[Developer Docs](https://superset.apache.org/developer-docs/contributing/development-setup#documentation) for documentation on contributing to documentation. ## Version Management -The Superset documentation site uses Docusaurus versioning with three independent versioned sections: +The Superset documentation site uses Docusaurus versioning with four independent sections: -- **Main Documentation** (`/docs/`) - Core Superset documentation -- **Developer Portal** (`/developer_portal/`) - Developer guides and tutorials +- **User Documentation** (`/user-docs/`) - End-user guides and tutorials +- **Admin Documentation** (`/admin-docs/`) - Installation, configuration, and security +- **Developer Docs** (`/developer-docs/`) - Developer guides, contributing, and extensions - **Component Playground** (`/components/`) - Interactive component examples (currently disabled) Each section maintains its own version history and can be versioned independently. diff --git a/docs/developer_docs/contributing/development-setup.md b/docs/developer_docs/contributing/development-setup.md index 44ff39ada47..dc1e9c4ed17 100644 --- a/docs/developer_docs/contributing/development-setup.md +++ b/docs/developer_docs/contributing/development-setup.md @@ -1012,7 +1012,7 @@ When contributing new React components to Superset, please try to add a Story al The topic of authoring new plugins, whether you'd like to contribute it back or not has been well documented in the -[the documentation](https://superset.apache.org/docs/contributing/creating-viz-plugins), and in [this blog post](https://preset.io/blog/building-custom-viz-plugins-in-superset-v2). +[the documentation](/developer-docs/contributing/howtos#creating-visualization-plugins), and in [this blog post](https://preset.io/blog/building-custom-viz-plugins-in-superset-v2). To contribute a plugin to Superset, your plugin must meet the following criteria: diff --git a/docs/developer_docs/contributing/howtos.md b/docs/developer_docs/contributing/howtos.md index e4469dfb07a..e01d54de334 100644 --- a/docs/developer_docs/contributing/howtos.md +++ b/docs/developer_docs/contributing/howtos.md @@ -256,6 +256,30 @@ For debugging the Flask backend: 2. Set breakpoints and press F5 to debug +### Debugging Server App in Kubernetes Environment + +To debug Flask running in a POD inside a kubernetes cluster, you'll need to make sure the pod runs as root and is granted the `SYS_PTRACE` capability. These settings should not be used in production environments. + +```yaml + securityContext: + capabilities: + add: ["SYS_PTRACE"] +``` + +See [set capabilities for a container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container) for more details. + +Once the pod is running as root and has the `SYS_PTRACE` capability it will be able to debug the Flask app. + +You can follow the same instructions as in `docker compose`. Enter the pod and install the required library and packages: gdb, netstat and debugpy. + +Often in a Kubernetes environment nodes are not addressable from outside the cluster. VSCode will thus be unable to remotely connect to port 5678 on a Kubernetes node. In order to do this you need to create a tunnel that port forwards 5678 to your local machine. + +```bash +kubectl port-forward pod/superset- 5678:5678 +``` + +You can now launch your VSCode debugger with the same config as above. VSCode will connect to 127.0.0.1:5678 which is forwarded by kubectl to your remote kubernetes POD. + ### Storybook See the dedicated [Storybook documentation](../testing/storybook) for information on running Storybook locally and adding new stories. diff --git a/docs/developer_docs/contributing/pkg-resources-migration.md b/docs/developer_docs/contributing/pkg-resources-migration.md new file mode 100644 index 00000000000..7300b14bc42 --- /dev/null +++ b/docs/developer_docs/contributing/pkg-resources-migration.md @@ -0,0 +1,94 @@ +--- +title: pkg_resources Migration Guide +sidebar_position: 9 +--- + + + +# pkg_resources Deprecation and Migration Guide + +## Background + +As of setuptools 81.0.0, the `pkg_resources` API is deprecated and will be removed. This affects several packages in the Python ecosystem. + +## Current Status + +### Superset Codebase + +The Superset codebase has already migrated away from `pkg_resources` to the modern `importlib.metadata` API: + +- `superset/db_engine_specs/__init__.py` - Uses `from importlib.metadata import entry_points` +- All entry point loading uses the modern API + +### Production Dependencies + +Some third-party dependencies may still use `pkg_resources`. Monitor your dependency tree for packages that haven't migrated yet. + +## Migration Path + +### Short-term Solution + +Pin setuptools to version 80.x to prevent breaking changes: + +```python +# requirements/base.in +setuptools<81 +``` + +This prevents the removal of `pkg_resources` while dependent packages are updated. + +### Long-term Solution + +Update all dependencies to use `importlib.metadata` instead of `pkg_resources`: + +#### Migration Example + +**Old (deprecated):** +```python +import pkg_resources + +version = pkg_resources.get_distribution("package_name").version +entry_points = pkg_resources.iter_entry_points("group_name") +``` + +**New (recommended):** +```python +from importlib.metadata import version, entry_points + +pkg_version = version("package_name") +eps = entry_points(group="group_name") +``` + +## Action Items + +### For Superset Maintainers +1. The Superset codebase already uses `importlib.metadata` +2. Monitor third-party dependencies for updates +3. Update setuptools pin once the ecosystem is ready + +### For Extension Developers +1. **Update your packages** to use `importlib.metadata` instead of `pkg_resources` +2. **Test with setuptools >= 81.0.0** once all packages are migrated + +## References + +- [setuptools pkg_resources deprecation notice](https://setuptools.pypa.io/en/latest/pkg_resources.html) +- [importlib.metadata documentation](https://docs.python.org/3/library/importlib.metadata.html) +- [Migration guide](https://setuptools.pypa.io/en/latest/deprecated/pkg_resources.html) diff --git a/docs/developer_docs/contributing/resources.md b/docs/developer_docs/contributing/resources.md index 78af327ae07..bd159c984f0 100644 --- a/docs/developer_docs/contributing/resources.md +++ b/docs/developer_docs/contributing/resources.md @@ -110,7 +110,7 @@ flowchart TD ## Entity-Relationship Diagram -For the full interactive Entity-Relationship Diagram, please visit the [main documentation](https://superset.apache.org/docs/contributing/resources). +For the full interactive Entity-Relationship Diagram, please visit the [developer documentation](/developer-docs/contributing/resources). You can also [download the .svg](https://github.com/apache/superset/tree/master/docs/static/img/erd.svg) directly from GitHub. @@ -119,7 +119,7 @@ You can also [download the .svg](https://github.com/apache/superset/tree/master/ ### Official Documentation - [Apache Superset Documentation](https://superset.apache.org/docs/intro) - [API Documentation](https://superset.apache.org/docs/api) -- [Configuration Guide](https://superset.apache.org/docs/installation/configuring-superset) +- [Configuration Guide](https://superset.apache.org/admin-docs/configuration/configuring-superset) ### Community Resources - [Apache Superset Blog](https://preset.io/blog/) diff --git a/docs/developer_docs/index.md b/docs/developer_docs/index.md index fe567fcc5a9..b883a08d6b6 100644 --- a/docs/developer_docs/index.md +++ b/docs/developer_docs/index.md @@ -29,14 +29,14 @@ Welcome to the Apache Superset Developer Portal - your comprehensive resource fo ## 🚀 Quick Start ### New Contributors -- [Contributing Overview](/developer_portal/contributing/overview) -- [Development Setup](/developer_portal/contributing/development-setup) -- [Your First PR](/developer_portal/contributing/submitting-pr) +- [Contributing Overview](/developer-docs/contributing/overview) +- [Development Setup](/developer-docs/contributing/development-setup) +- [Your First PR](/developer-docs/contributing/submitting-pr) ### Extension Development -- [Extension Project Structure](/developer_portal/extensions/extension-project-structure) -- [Extension Architecture](/developer_portal/extensions/architecture) -- [Quick Start](/developer_portal/extensions/quick-start) +- [Extension Development](/developer-docs/extensions/development) +- [Extension Architecture](/developer-docs/extensions/architecture) +- [Quick Start](/developer-docs/extensions/quick-start) ## 📚 Documentation Sections @@ -83,7 +83,7 @@ Everything you need to contribute to the Apache Superset project. This section i ### External Documentation - **[User Documentation](https://superset.apache.org/docs/intro)** - Using Superset - **[API Documentation](https://superset.apache.org/docs/api)** - REST API reference -- **[Configuration Guide](https://superset.apache.org/docs/configuration/configuring-superset)** - Setup and configuration +- **[Configuration Guide](https://superset.apache.org/admin-docs/configuration/configuring-superset)** - Setup and configuration ### Important Files - **[CLAUDE.md](https://github.com/apache/superset/blob/master/CLAUDE.md)** - LLM development guide @@ -96,17 +96,17 @@ Everything you need to contribute to the Apache Superset project. This section i **I want to contribute code** -1. [Set up development environment](/developer_portal/contributing/development-setup) +1. [Set up development environment](/developer-docs/contributing/development-setup) 2. [Find a good first issue](https://github.com/apache/superset/labels/good%20first%20issue) -3. [Submit your first PR](/developer_portal/contributing/submitting-pr) +3. [Submit your first PR](/developer-docs/contributing/submitting-pr) **I want to build an extension** -1. [Start with Quick Start](/developer_portal/extensions/quick-start) -2. [Learn extension structure](/developer_portal/extensions/extension-project-structure) -3. [Explore architecture](/developer_portal/extensions/architecture) +1. [Start with Quick Start](/developer-docs/extensions/quick-start) +2. [Learn extension development](/developer-docs/extensions/development) +3. [Explore architecture](/developer-docs/extensions/architecture) @@ -115,8 +115,8 @@ Everything you need to contribute to the Apache Superset project. This section i **I found a bug** 1. [Search existing issues](https://github.com/apache/superset/issues) -2. [Report the bug](/developer_portal/contributing/issue-reporting) -3. [Submit a fix](/developer_portal/contributing/submitting-pr) +2. [Report the bug](/developer-docs/contributing/issue-reporting) +3. [Submit a fix](/developer-docs/contributing/submitting-pr) diff --git a/docs/docs/intro.md b/docs/docs/intro.md new file mode 100644 index 00000000000..4921692e769 --- /dev/null +++ b/docs/docs/intro.md @@ -0,0 +1,257 @@ +--- +hide_title: true +sidebar_position: 1 +--- + + +# Superset + +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/license/apache-2-0) +[![Latest Release on Github](https://img.shields.io/github/v/release/apache/superset?sort=semver)](https://github.com/apache/superset/releases/latest) +[![Build Status](https://github.com/apache/superset/actions/workflows/superset-python-unittest.yml/badge.svg)](https://github.com/apache/superset/actions) +[![PyPI version](https://badge.fury.io/py/apache_superset.svg)](https://badge.fury.io/py/apache_superset) +[![PyPI](https://img.shields.io/pypi/pyversions/apache_superset.svg?maxAge=2592000)](https://pypi.python.org/pypi/apache_superset) +[![GitHub Stars](https://img.shields.io/github/stars/apache/superset?style=social)](https://github.com/apache/superset/stargazers) +[![Contributors](https://img.shields.io/github/contributors/apache/superset)](https://github.com/apache/superset/graphs/contributors) +[![Last Commit](https://img.shields.io/github/last-commit/apache/superset)](https://github.com/apache/superset/commits/master) +[![Open Issues](https://img.shields.io/github/issues/apache/superset)](https://github.com/apache/superset/issues) +[![Open PRs](https://img.shields.io/github/issues-pr/apache/superset)](https://github.com/apache/superset/pulls) +[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](http://bit.ly/join-superset-slack) +[![Documentation](https://img.shields.io/badge/docs-apache.org-blue.svg)](https://superset.apache.org) + + + + Superset logo (light) + + +A modern, enterprise-ready business intelligence web application. + +[**Why Superset?**](#why-superset) | +[**Supported Databases**](#supported-databases) | +[**Installation and Configuration**](#installation-and-configuration) | +[**Release Notes**](https://github.com/apache/superset/blob/master/RELEASING/README.md#release-notes-for-recent-releases) | +[**Get Involved**](#get-involved) | +[**Contributor Guide**](#contributor-guide) | +[**Resources**](#resources) | +[**Organizations Using Superset**](https://superset.apache.org/inTheWild) + +## Why Superset? + +Superset is a modern data exploration and data visualization platform. Superset can replace or augment proprietary business intelligence tools for many teams. Superset integrates well with a variety of data sources. + +Superset provides: + +- A **no-code interface** for building charts quickly +- A powerful, web-based **SQL Editor** for advanced querying +- A **lightweight semantic layer** for quickly defining custom dimensions and metrics +- Out of the box support for **nearly any SQL** database or data engine +- A wide array of **beautiful visualizations** to showcase your data, ranging from simple bar charts to geospatial visualizations +- Lightweight, configurable **caching layer** to help ease database load +- Highly extensible **security roles and authentication** options +- An **API** for programmatic customization +- A **cloud-native architecture** designed from the ground up for scale + +## Screenshots & Gifs + +**Video Overview** + + + +[superset-video-1080p.webm](https://github.com/user-attachments/assets/b37388f7-a971-409c-96a7-90c4e31322e6) + +
+ +**Large Gallery of Visualizations** + +
+ +**Craft Beautiful, Dynamic Dashboards** + +
+ +**No-Code Chart Builder** + +
+ +**Powerful SQL Editor** + +
+ +## Supported Databases + +Superset can query data from any SQL-speaking datastore or data engine (Presto, Trino, Athena, [and more](https://superset.apache.org/docs/databases)) that has a Python DB-API driver and a SQLAlchemy dialect. + +Here are some of the major database solutions that are supported: + + +

+ Amazon Athena   + Amazon DynamoDB   + Amazon Redshift   + Apache Doris   + Apache Drill   + Apache Druid   + Apache Hive   + Apache Impala   + Apache Kylin   + Apache Pinot   + Apache Solr   + Apache Spark SQL   + Ascend   + Aurora MySQL (Data API)   + Aurora PostgreSQL (Data API)   + Azure Data Explorer   + Azure Synapse   + ClickHouse   + Cloudflare D1   + CockroachDB   + Couchbase   + CrateDB   + Databend   + Databricks   + Denodo   + Dremio   + DuckDB   + Elasticsearch   + Exasol   + Firebird   + Firebolt   + Google BigQuery   + Google Sheets   + Greenplum   + Hologres   + IBM Db2   + IBM Netezza Performance Server   + MariaDB   + Microsoft SQL Server   + MonetDB   + MongoDB   + MotherDuck   + OceanBase   + Oracle   + Presto   + RisingWave   + SAP HANA   + SAP Sybase   + Shillelagh   + SingleStore   + Snowflake   + SQLite   + StarRocks   + Superset meta database   + TDengine   + Teradata   + TimescaleDB   + Trino   + Vertica   + YDB   + YugabyteDB +

+ + +**A more comprehensive list of supported databases** along with the configuration instructions can be found [here](https://superset.apache.org/docs/databases). + +Want to add support for your datastore or data engine? Read more [here](https://superset.apache.org/docs/frequently-asked-questions#does-superset-work-with-insert-database-engine-here) about the technical requirements. + +## Installation and Configuration + +Try out Superset's [quickstart](https://superset.apache.org/docs/quickstart/) guide or learn about [the options for production deployments](https://superset.apache.org/docs/installation/architecture/). + +## Get Involved + +- Ask and answer questions on [StackOverflow](https://stackoverflow.com/questions/tagged/apache-superset) using the **apache-superset** tag +- [Join our community's Slack](http://bit.ly/join-superset-slack) + and please read our [Slack Community Guidelines](https://github.com/apache/superset/blob/master/CODE_OF_CONDUCT.md#slack-community-guidelines) +- [Join our dev@superset.apache.org Mailing list](https://lists.apache.org/list.html?dev@superset.apache.org). To join, simply send an email to [dev-subscribe@superset.apache.org](mailto:dev-subscribe@superset.apache.org) +- If you want to help troubleshoot GitHub Issues involving the numerous database drivers that Superset supports, please consider adding your name and the databases you have access to on the [Superset Database Familiarity Rolodex](https://docs.google.com/spreadsheets/d/1U1qxiLvOX0kBTUGME1AHHi6Ywel6ECF8xk_Qy-V9R8c/edit#gid=0) +- Join Superset's Town Hall and [Operational Model](https://preset.io/blog/the-superset-operational-model-wants-you/) recurring meetings. Meeting info is available on the [Superset Community Calendar](https://superset.apache.org/community) + +## Contributor Guide + +Interested in contributing? Check out our +[Developer Portal](https://superset.apache.org/developer-docs/) +to find resources around contributing along with a detailed guide on +how to set up a development environment. + +## Resources + +- [Superset "In the Wild"](https://superset.apache.org/inTheWild) - see who's using Superset, and [add your organization](https://github.com/apache/superset/edit/master/RESOURCES/INTHEWILD.yaml) to the list! +- [Feature Flags](https://superset.apache.org/docs/configuration/feature-flags) - the status of Superset's Feature Flags. +- [Standard Roles](https://github.com/apache/superset/blob/master/RESOURCES/STANDARD_ROLES.md) - How RBAC permissions map to roles. +- [Superset Wiki](https://github.com/apache/superset/wiki) - Tons of additional community resources: best practices, community content and other information. +- [Superset SIPs](https://github.com/orgs/apache/projects/170) - The status of Superset's SIPs (Superset Improvement Proposals) for both consensus and implementation status. + +Understanding the Superset Points of View + +- [The Case for Dataset-Centric Visualization](https://preset.io/blog/dataset-centric-visualization/) +- [Understanding the Superset Semantic Layer](https://preset.io/blog/understanding-superset-semantic-layer/) + +- Getting Started with Superset + - [Superset in 2 Minutes using Docker Compose](https://superset.apache.org/docs/installation/docker-compose#installing-superset-locally-using-docker-compose) + - [Installing Database Drivers](https://superset.apache.org/docs/configuration/databases#installing-database-drivers) + - [Building New Database Connectors](https://preset.io/blog/building-database-connector/) + - [Create Your First Dashboard](https://superset.apache.org/docs/using-superset/creating-your-first-dashboard/) + - [Comprehensive Tutorial for Contributing Code to Apache Superset + ](https://preset.io/blog/tutorial-contributing-code-to-apache-superset/) +- [Resources to master Superset by Preset](https://preset.io/resources/) + +- Deploying Superset + + - [Official Docker image](https://hub.docker.com/r/apache/superset) + - [Helm Chart](https://github.com/apache/superset/tree/master/helm/superset) + +- Recordings of Past [Superset Community Events](https://preset.io/events) + + - [Mixed Time Series Charts](https://preset.io/events/mixed-time-series-visualization-in-superset-workshop/) + - [How the Bing Team Customized Superset for the Internal Self-Serve Data & Analytics Platform](https://preset.io/events/how-the-bing-team-heavily-customized-superset-for-their-internal-data/) + - [Live Demo: Visualizing MongoDB and Pinot Data using Trino](https://preset.io/events/2021-04-13-visualizing-mongodb-and-pinot-data-using-trino/) + - [Introduction to the Superset API](https://preset.io/events/introduction-to-the-superset-api/) + - [Building a Database Connector for Superset](https://preset.io/events/2021-02-16-building-a-database-connector-for-superset/) + +- Visualizations + + - [Creating Viz Plugins](https://superset.apache.org/developer-docs/contributing/howtos#creating-visualization-plugins) + - [Managing and Deploying Custom Viz Plugins](https://medium.com/nmc-techblog/apache-superset-manage-custom-viz-plugins-in-production-9fde1a708e55) + - [Why Apache Superset is Betting on Apache ECharts](https://preset.io/blog/2021-4-1-why-echarts/) + +- [Superset API](https://superset.apache.org/docs/rest-api) + +## Repo Activity + + + + + Performance Stats of apache/superset - Last 28 days + + + + + + + diff --git a/docs/sidebarAdminDocs.js b/docs/sidebarAdminDocs.js index 1acfaface7f..cf59bc1c1a2 100644 --- a/docs/sidebarAdminDocs.js +++ b/docs/sidebarAdminDocs.js @@ -53,7 +53,7 @@ const sidebars = { { type: 'link', label: 'Database Drivers', - href: '/docs/databases', + href: '/user-docs/databases', description: 'See User Docs for database connection guides', }, { diff --git a/docs/sidebarTutorials.js b/docs/sidebarTutorials.js index b786478c0b1..fbcf39d8dad 100644 --- a/docs/sidebarTutorials.js +++ b/docs/sidebarTutorials.js @@ -42,6 +42,7 @@ const sidebars = { 'contributing/howtos', 'contributing/release-process', 'contributing/resources', + 'contributing/pkg-resources-migration', 'guidelines/design-guidelines', { type: 'category', @@ -91,6 +92,7 @@ const sidebars = { collapsed: true, items: [ 'extensions/extension-points/sqllab', + 'extensions/extension-points/editors', ], }, 'extensions/development', @@ -129,7 +131,7 @@ const sidebars = { { type: 'link', label: 'API Reference', - href: '/docs/api', + href: '/user-docs/api', }, ], }; diff --git a/docs/versioned_docs/version-6.0.0/intro.md b/docs/versioned_docs/version-6.0.0/intro.md index b928eb7d8bb..841ee0f7e55 100644 --- a/docs/versioned_docs/version-6.0.0/intro.md +++ b/docs/versioned_docs/version-6.0.0/intro.md @@ -165,7 +165,7 @@ Try out Superset's [quickstart](https://superset.apache.org/docs/quickstart/) gu ## Contributor Guide Interested in contributing? Check out our -[Developer Portal](https://superset.apache.org/developer_portal/) +[Developer Docs](https://superset.apache.org/developer-docs/) to find resources around contributing along with a detailed guide on how to set up a development environment. diff --git a/docs/versioned_docs/version-6.0.0/quickstart.mdx b/docs/versioned_docs/version-6.0.0/quickstart.mdx index b9795106b74..640ebe5794b 100644 --- a/docs/versioned_docs/version-6.0.0/quickstart.mdx +++ b/docs/versioned_docs/version-6.0.0/quickstart.mdx @@ -73,7 +73,7 @@ processes by running Docker Compose `stop` command. By doing so, you can avoid d From this point on, you can head on to: -- [Create your first Dashboard](/docs/using-superset/creating-your-first-dashboard) +- [Create your first Dashboard](/docs/6.0.0/using-superset/creating-your-first-dashboard) - [Connect to a Database](/docs/6.0.0/configuration/databases) - [Using Docker Compose](/docs/6.0.0/installation/docker-compose) - [Configure Superset](/docs/6.0.0/configuration/configuring-superset/)