From fe214850656eb99d164078838e749e17deeec729 Mon Sep 17 00:00:00 2001 From: "Michael S. Molina" <70410625+michael-s-molina@users.noreply.github.com> Date: Wed, 26 Nov 2025 16:48:36 -0500 Subject: [PATCH] docs: Reorganizes the extensions documentation (#36298) Co-authored-by: Evan Rusackas --- .github/CODEOWNERS | 1 + .../extensions/architecture.md | 4 +- ...ibution-types.md => contribution-types.md} | 46 +++- .../{deploying-extension.md => deployment.md} | 24 +- .../extensions/development-mode.md | 48 ---- .../extensions/development.md | 239 ++++++++++++++++++ .../extensions/extension-metadata.md | 55 ---- .../extensions/extension-points/sqllab.md | 209 +++++++++++++++ .../extensions/extension-project-structure.md | 78 ------ .../extensions/interacting-with-host.md | 129 ---------- docs/developer_portal/extensions/mcp.md | 7 +- docs/developer_portal/extensions/overview.md | 56 ++-- .../extensions/quick-start.md | 9 +- .../{security-implications.md => security.md} | 10 +- docs/developer_portal/sidebars.js | 20 +- docs/sidebarTutorials.js | 20 +- 16 files changed, 559 insertions(+), 396 deletions(-) rename docs/developer_portal/extensions/{frontend-contribution-types.md => contribution-types.md} (66%) rename docs/developer_portal/extensions/{deploying-extension.md => deployment.md} (59%) delete mode 100644 docs/developer_portal/extensions/development-mode.md create mode 100644 docs/developer_portal/extensions/development.md delete mode 100644 docs/developer_portal/extensions/extension-metadata.md create mode 100644 docs/developer_portal/extensions/extension-points/sqllab.md delete mode 100644 docs/developer_portal/extensions/extension-project-structure.md delete mode 100644 docs/developer_portal/extensions/interacting-with-host.md rename docs/developer_portal/extensions/{security-implications.md => security.md} (82%) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 237907f74d3..357078af86b 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -33,6 +33,7 @@ # Notify PMC members of changes to extension-related files +/docs/developer_portal/extensions/ @michael-s-molina @villebro @rusackas /superset-core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje /superset-extensions-cli/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje /superset/core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje diff --git a/docs/developer_portal/extensions/architecture.md b/docs/developer_portal/extensions/architecture.md index 313596320de..a5a01bdb29f 100644 --- a/docs/developer_portal/extensions/architecture.md +++ b/docs/developer_portal/extensions/architecture.md @@ -248,6 +248,6 @@ This architecture provides several key benefits: Now that you understand the architecture, explore: -- **[Extension Project Structure](./extension-project-structure)** - How to organize your extension code -- **[Frontend Contribution Types](./frontend-contribution-types)** - What kinds of extensions you can build - **[Quick Start](./quick-start)** - Build your first extension +- **[Contribution Types](./contribution-types)** - What kinds of extensions you can build +- **[Development](./development)** - Project structure, APIs, and development workflow diff --git a/docs/developer_portal/extensions/frontend-contribution-types.md b/docs/developer_portal/extensions/contribution-types.md similarity index 66% rename from docs/developer_portal/extensions/frontend-contribution-types.md rename to docs/developer_portal/extensions/contribution-types.md index 51857ad0c93..6ed4d77b957 100644 --- a/docs/developer_portal/extensions/frontend-contribution-types.md +++ b/docs/developer_portal/extensions/contribution-types.md @@ -1,6 +1,6 @@ --- -title: Frontend Contribution Types -sidebar_position: 5 +title: Contribution Types +sidebar_position: 4 --- -# Frontend Contribution Types +# Contribution Types -To facilitate the development of extensions, we will define a set of well-defined contribution types that extensions can implement. These contribution types will serve as the building blocks for extensions, allowing them to interact with the host application and provide new functionality. The initial set of contribution types will include: +To facilitate the development of extensions, we define a set of well-defined contribution types that extensions can implement. These contribution types serve as the building blocks for extensions, allowing them to interact with the host application and provide new functionality. -## Views +## Frontend + +Frontend contribution types allow extensions to extend Superset's user interface with new views, commands, and menu items. + +### Views Extensions can add new views or panels to the host application, such as custom SQL Lab panels, dashboards, or other UI components. Each view is registered with a unique ID and can be activated or deactivated as needed. Contribution areas are uniquely identified (e.g., `sqllab.panels` for SQL Lab panels), enabling seamless integration into specific parts of the application. @@ -41,7 +45,7 @@ Extensions can add new views or panels to the host application, such as custom S }, ``` -## Commands +### Commands Extensions can define custom commands that can be executed within the host application, such as context-aware actions or menu options. Each command can specify properties like a unique command identifier, an icon, a title, and a description. These commands can be invoked by users through menus, keyboard shortcuts, or other UI elements, enabling extensions to add rich, interactive functionality to Superset. @@ -56,7 +60,7 @@ Extensions can define custom commands that can be executed within the host appli ] ``` -## Menus +### Menus Extensions can contribute new menu items or context menus to the host application, providing users with additional actions and options. Each menu item can specify properties such as the target view, the command to execute, its placement (primary, secondary, or context), and conditions for when it should be displayed. Menu contribution areas are uniquely identified (e.g., `sqllab.editor` for the SQL Lab editor), allowing extensions to seamlessly integrate their functionality into specific menus and workflows within Superset. @@ -88,3 +92,31 @@ Extensions can contribute new menu items or context menus to the host applicatio }, } ``` + +## Backend + +Backend contribution types allow extensions to extend Superset's server-side capabilities with new API endpoints, MCP tools, and MCP prompts. + +### REST API Endpoints + +Extensions can register custom REST API endpoints under the `/api/v1/extensions/` namespace. This dedicated namespace prevents conflicts with built-in endpoints and provides a clear separation between core and extension functionality. + +``` json +"backend": { + "entryPoints": ["my_extension.entrypoint"], + "files": ["backend/src/my_extension/**/*.py"] +} +``` + +The entry point module registers the API with Superset: + +``` python +from superset_core.api.rest_api import add_extension_api +from .api import MyExtensionAPI + +add_extension_api(MyExtensionAPI) +``` + +### MCP Tools and Prompts + +Extensions can contribute Model Context Protocol (MCP) tools and prompts that AI agents can discover and use. See [MCP Integration](./mcp) for detailed documentation. diff --git a/docs/developer_portal/extensions/deploying-extension.md b/docs/developer_portal/extensions/deployment.md similarity index 59% rename from docs/developer_portal/extensions/deploying-extension.md rename to docs/developer_portal/extensions/deployment.md index 48582bceae8..2817dcfbbdc 100644 --- a/docs/developer_portal/extensions/deploying-extension.md +++ b/docs/developer_portal/extensions/deployment.md @@ -1,6 +1,6 @@ --- -title: Deploying an Extension -sidebar_position: 8 +title: Deployment +sidebar_position: 6 --- -# Deploying an Extension +# Deployment Once an extension has been developed, the deployment process involves packaging and uploading it to the host application. @@ -33,13 +33,17 @@ Packaging is handled by the `superset-extensions bundle` command, which: 3. Generates a `manifest.json` with build-time metadata, including the contents of `extension.json` and references to built assets. 4. Packages everything into a `.supx` file (a zip archive with a specific structure required by Superset). -Uploading is accomplished through Superset's REST API at `/api/v1/extensions/import/`. The endpoint accepts the `.supx` file as form data and processes it by: +To deploy an extension, place the `.supx` file in the extensions directory configured via `EXTENSIONS_PATH` in your `superset_config.py`: -1. Extracting and validating the extension metadata and manifest. -2. Storing extension assets in the metadata database for dynamic loading. -3. Registering the extension in the metadata database, including its name, version, author, and capabilities. -4. Automatically activating the extension, making it immediately available for use and management via the Superset UI or API. +``` python +EXTENSIONS_PATH = "/path/to/extensions" +``` -This API-driven approach enables automated deployment workflows and simplifies extension management for administrators. Extensions can be uploaded through the Swagger UI, programmatically via scripts, or through the management interface: +During application startup, Superset automatically discovers and loads all `.supx` files from this directory: -https://github.com/user-attachments/assets/98b16cdd-8ec5-4812-9d5e-9915badd8f0d +1. Scans the configured directory for `.supx` files. +2. Validates each file is a properly formatted zip archive. +3. Extracts and validates the extension manifest and metadata. +4. Loads the extension, making it available for use. + +This file-based approach simplifies deployment in containerized environments and enables version control of extensions alongside infrastructure configuration. diff --git a/docs/developer_portal/extensions/development-mode.md b/docs/developer_portal/extensions/development-mode.md deleted file mode 100644 index d2fdf36e055..00000000000 --- a/docs/developer_portal/extensions/development-mode.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: Development Mode -sidebar_position: 10 ---- - - - -# Development Mode - -Development mode accelerates extension development by letting developers see changes in Superset quickly, without the need for repeated packaging and uploading. To enable development mode, set the `LOCAL_EXTENSIONS` configuration in your `superset_config.py`: - -``` python -LOCAL_EXTENSIONS = [ - "/path/to/your/extension1", - "/path/to/your/extension2", -] -``` - -This instructs Superset to load and serve extensions directly from disk, so you can iterate quickly. Running `superset-extensions dev` watches for file changes and rebuilds assets automatically, while the Webpack development server (started separately with `npm run dev-server`) serves updated files as soon as they're modified. This enables immediate feedback for React components, styles, and other frontend code. Changes to backend files are also detected automatically and immediately synced, ensuring that both frontend and backend updates are reflected in your development environment. - -Example output when running in development mode: - -``` -superset-extensions dev - -⚙️ Building frontend assets… -✅ Frontend rebuilt -✅ Backend files synced -✅ Manifest updated -👀 Watching for changes in: /dataset_references/frontend, /dataset_references/backend -``` diff --git a/docs/developer_portal/extensions/development.md b/docs/developer_portal/extensions/development.md new file mode 100644 index 00000000000..0550a789838 --- /dev/null +++ b/docs/developer_portal/extensions/development.md @@ -0,0 +1,239 @@ +--- +title: Development +sidebar_position: 5 +--- + + + +# Development + +This guide covers everything you need to know about developing extensions for Superset, from project structure to development workflow. + +## Project Structure + +The [apache-superset-extensions-cli](https://github.com/apache/superset/tree/master/superset-extensions-cli) package provides a command-line interface (CLI) that streamlines the extension development workflow. It offers the following commands: + +``` +superset-extensions init: Generates the initial folder structure and scaffolds a new extension project. + +superset-extensions build: Builds extension assets. + +superset-extensions bundle: Packages the extension into a .supx file. + +superset-extensions dev: Automatically rebuilds the extension as files change. +``` + +When creating a new extension with `superset-extensions init `, the CLI generates a standardized folder structure: + +``` +dataset_references/ +├── extension.json +├── frontend/ +│ ├── src/ +│ ├── webpack.config.js +│ ├── tsconfig.json +│ └── package.json +├── backend/ +│ ├── src/ +│ └── dataset_references/ +│ ├── tests/ +│ ├── pyproject.toml +│ └── requirements.txt +├── dist/ +│ ├── manifest.json +│ ├── frontend +│ └── dist/ +│ ├── remoteEntry.d7a9225d042e4ccb6354.js +│ └── 900.038b20cdff6d49cfa8d9.js +│ └── backend +│ └── dataset_references/ +│ ├── __init__.py +│ ├── api.py +│ └── entrypoint.py +├── dataset_references-1.0.0.supx +└── README.md +``` + +The `extension.json` file serves as the declared metadata for the extension, containing the extension's name, version, author, description, and a list of capabilities. This file is essential for the host application to understand how to load and manage the extension. + +The `frontend` directory contains the source code for the frontend components of the extension, including React components, styles, and assets. The `webpack.config.js` file is used to configure Webpack for building the frontend code, while the `tsconfig.json` file defines the TypeScript configuration for the project. The `package.json` file specifies the dependencies and scripts for building and testing the frontend code. + +The `backend` directory contains the source code for the backend components of the extension, including Python modules, tests, and configuration files. The `pyproject.toml` file is used to define the Python package and its dependencies, while the `requirements.txt` file lists the required Python packages for the extension. The `src` folder contains the functional backend source files, `tests` directory contains unit tests for the backend code, ensuring that the extension behaves as expected and meets the defined requirements. + +The `dist` directory is built when running the `build` or `dev` command, and contains the files that will be included in the bundle. The `manifest.json` file contains critical metadata about the extension, including the majority of the contents of the `extension.json` file, but also other build-time information, like the name of the built Webpack Module Federation remote entry file. The files in the `dist` directory will be zipped into the final `.supx` file. Although this file is technically a zip archive, the `.supx` extension makes it clear that it is a Superset extension package and follows a specific file layout. This packaged file can be distributed and installed in Superset instances. + +The `README.md` file provides documentation and instructions for using the extension, including how to install, configure, and use its functionality. + +## Extension Metadata + +The `extension.json` file contains all metadata necessary for the host application to understand and manage the extension: + +```json +{ + "name": "dataset_references", + "version": "1.0.0", + "frontend": { + "contributions": { + "views": { + "sqllab.panels": [ + { + "id": "dataset_references.main", + "name": "Dataset references" + } + ] + } + }, + "moduleFederation": { + "exposes": ["./index"] + } + }, + "backend": { + "entryPoints": ["dataset_references.entrypoint"], + "files": ["backend/src/dataset_references/**/*.py"] + } +} +``` + +The `contributions` section declares how the extension extends Superset's functionality through views, commands, menus, and other contribution types. The `backend` section specifies entry points and files to include in the bundle. + +## Interacting with the Host + +Extensions interact with Superset through well-defined, versioned APIs provided by the `@apache-superset/core` (frontend) and `apache-superset-core` (backend) packages. These APIs are designed to be stable, discoverable, and consistent for both built-in and external extensions. + +**Note**: The `superset_core.api` module provides abstract classes that are replaced with concrete implementations via dependency injection when Superset initializes. This allows extensions to use the same interfaces as the host application. + +### Frontend APIs + +The frontend extension APIs (via `@apache-superset/core`) are organized into logical namespaces such as `authentication`, `commands`, `extensions`, `sqlLab`, and others. Each namespace groups related functionality, making it easy for extension authors to discover and use the APIs relevant to their needs. For example, the `sqlLab` namespace provides events and methods specific to SQL Lab, allowing extensions to react to user actions and interact with the SQL Lab environment: + +```typescript +export const getCurrentTab: () => Tab | undefined; + +export const getDatabases: () => Database[]; + +export const getTabs: () => Tab[]; + +export const onDidChangeEditorContent: Event; + +export const onDidClosePanel: Event; + +export const onDidChangeActivePanel: Event; + +export const onDidChangeTabTitle: Event; + +export const onDidQueryRun: Event; + +export const onDidQueryStop: Event; +``` + +The following code demonstrates more examples of the existing frontend APIs: + +```typescript +import { core, commands, sqlLab, authentication, Button } from '@apache-superset/core'; +import MyPanel from './MyPanel'; + +export function activate(context) { + // Register a new panel (view) in SQL Lab and use shared UI components in your extension's React code + const panelDisposable = core.registerView('my_extension.panel',