QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-04-19 08:04:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(docs): Populate Developer Portal with comprehensive documentation framework (#35217)

Browse Source 
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Evan Rusackas
2025-10-08 03:18:40 -04:00
committed by GitHub
parent d383b2bf3c
commit 441e043bff
74 changed files with 12216 additions and 1793 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/developer_portal/extensions/architectural-principles.md Normal file
View File

@@ -0,0 +1,36 @@
---
title: Architectural Principles
sidebar_position: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Architectural Principles
Realizing this vision requires a strong architectural foundation. To ensure the resulting system is robust, maintainable, and adaptable, we have defined a set of architectural principles that will guide the design and implementation of the changes. These principles serve as the basis for all technical decisions and help create an environment where extensions can be developed safely and predictably, while minimizing technical debt and fragmentation.
The architectural principles guiding this proposal include:
1. **Lean core**: Superset's core should remain as minimal as possible, with many features and capabilities delegated to extensions. Wherever possible, built-in features should be implemented using the same APIs and extension mechanisms available to external extension authors. This approach reduces maintenance burden and complexity in the core application, encourages modularity, and allows the community to innovate and iterate on features independently of the main codebase.
2. **Explicit contribution points**: All extension points must be clearly defined and documented, so extension authors know exactly where and how they can interact with the host system. Each extension must also declare its capabilities in a metadata file, enabling the host to manage the extension lifecycle and provide a consistent user experience.
3. **Versioned and stable APIs**: Public interfaces for extensions should be versioned and follow semantic versioning, allowing for safe evolution and backward compatibility.
4. **Lazy loading and activation**: Extensions should be loaded and activated only when needed, minimizing performance overhead and resource consumption.
5. **Composability and reuse**: The architecture should encourage the reuse of extension points and patterns across different modules, promoting consistency and reducing duplication.
6. **Community-driven evolution**: The system should be designed to evolve based on real-world feedback and contributions, allowing new extension points and capabilities to be added as needs emerge.

36
docs/developer_portal/extensions/built-in-features.md Normal file
View File

@@ -0,0 +1,36 @@
---
title: What This Means for Superset's Built-in Features
sidebar_position: 13
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# What This Means for Superset's Built-in Features
Transitioning to a well-defined, versioned API model has significant implications for Superset's built-in features. By exposing stable, public APIs for core functionality, we enable both extensions and internal modules to interact with the application in a consistent and predictable way. This approach brings several key benefits:
- **Unified API Surface**: All important features of Superset will be accessible through documented, versioned APIs. This not only empowers extension authors but also ensures that built-in features use the same mechanisms, validating the APIs through real-world usage and making it easier to replace or enhance individual features over time.
- **Dogfooding and Replaceability**: By building Superset's own features using the same APIs available to extensions, we ensure that these APIs are robust, flexible, and well-tested. This also means that any built-in feature can potentially be replaced or extended by a third-party extension, increasing modularity and adaptability.
- **Versioned and Stable Contracts**: Public APIs will be versioned and follow semantic versioning, providing stability for both internal and external consumers. This stability is critical for long-term maintainability, but it also means that extra care must be taken to avoid breaking changes and to provide clear migration paths when changes are necessary.
- **Improved Inter-Module Communication**: With clearly defined APIs and a command-based architecture, modules and extensions can communicate through explicit interfaces rather than relying on direct Redux store access or tightly coupled state management. This decouples modules, reduces the risk of unintended side effects, and makes the codebase easier to reason about and maintain.
- **Facilitated Refactoring and Evolution**: As the application evolves, having a stable API layer allows for internal refactoring and optimization without breaking consumers. This makes it easier to modernize or optimize internal implementations while preserving compatibility.
- **Clearer Documentation and Onboarding**: A public, versioned API surface makes it easier to document and onboard new contributors, both for core development and for extension authors.
Overall, this shift represents a move toward a more modular, maintainable, and extensible architecture, where both built-in features and extensions are first-class citizens, and where the boundaries between core and community-driven innovation are minimized.

45
docs/developer_portal/extensions/deploying-extension.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: Deploying an Extension
sidebar_position: 8
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Deploying an Extension
Once an extension has been developed, the deployment process involves packaging and uploading it to the host application.
Packaging is handled by the `superset-extensions bundle` command, which:
1. Builds frontend assets using Webpack (with Module Federation configuration).
2. Collects backend Python source files and all necessary resources.
3. Generates a `manifest.json` with build-time metadata, including the contents of `extension.json` and references to built assets.
4. Packages everything into a `.supx` file (a zip archive with a specific structure required by Superset).
Uploading is accomplished through Superset's REST API at `/api/v1/extensions/import/`. The endpoint accepts the `.supx` file as form data and processes it by:
1. Extracting and validating the extension metadata and manifest.
2. Storing extension assets in the metadata database for dynamic loading.
3. Registering the extension in the metadata database, including its name, version, author, and capabilities.
4. Automatically activating the extension, making it immediately available for use and management via the Superset UI or API.
This API-driven approach enables automated deployment workflows and simplifies extension management for administrators. Extensions can be uploaded through the Swagger UI, programmatically via scripts, or through the management interface:
https://github.com/user-attachments/assets/98b16cdd-8ec5-4812-9d5e-9915badd8f0d

48
docs/developer_portal/extensions/development-mode.md Normal file
View File

@@ -0,0 +1,48 @@
---
title: Development Mode
sidebar_position: 10
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Development Mode
Development mode accelerates extension development by letting developers see changes in Superset quickly, without the need for repeated packaging and uploading. To enable development mode, set the `LOCAL_EXTENSIONS` configuration in your `superset_config.py`:
``` python
LOCAL_EXTENSIONS = [
"/path/to/your/extension1",
"/path/to/your/extension2",
]
```
This instructs Superset to load and serve extensions directly from disk, so you can iterate quickly. Running `superset-extensions dev` watches for file changes and rebuilds assets automatically, while the Webpack development server (started separately with `npm run dev-server`) serves updated files as soon as they're modified. This enables immediate feedback for React components, styles, and other frontend code. Changes to backend files are also detected automatically and immediately synced, ensuring that both frontend and backend updates are reflected in your development environment.
Example output when running in development mode:
```
superset-extensions dev
⚙️ Building frontend assets…
✅ Frontend rebuilt
✅ Backend files synced
✅ Manifest updated
👀 Watching for changes in: /dataset_references/frontend, /dataset_references/backend
```

84
docs/developer_portal/extensions/dynamic-module-loading.md Normal file
View File

@@ -0,0 +1,84 @@
---
title: Dynamic Module Loading
sidebar_position: 7
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Dynamic Module Loading
The extension architecture leverages Webpack's Module Federation to enable dynamic loading of frontend assets at runtime. This sophisticated mechanism involves several key concepts:
**Module Federation** allows extensions to be built and deployed independently while sharing dependencies with the host application. Extensions expose their entry points through the federation configuration:
``` typescript
new ModuleFederationPlugin({
name: 'my_extension',
filename: 'remoteEntry.[contenthash].js',
exposes: {
'./index': './src/index.tsx',
},
externalsType: 'window',
externals: {
'@apache-superset/core': 'superset',
},
shared: {
react: { singleton: true },
'react-dom': { singleton: true },
'antd-v5': { singleton: true }
}
})
```
`externals` and `externalsType` ensure that extensions use the host's implementation of shared packages rather than bundling their own copies.
`shared` dependencies prevent duplication of common libraries like React and Ant Design avoiding version conflicts and reducing bundle size.
This configuration tells Webpack that when the extension imports from `@apache-superset/core`, it should resolve to `window.superset` at runtime, where the host application provides the actual implementation. The following diagram illustrates how this works in practice:
<img width="913" height="558" alt="Image" src="https://github.com/user-attachments/assets/e5e4d2ae-e8b5-4d17-a2a1-3667c65f25ca" />
During extension registration, the host application fetches the remote entry file and dynamically loads the extension's modules without requiring a rebuild or restart of Superset.
On the host application side, the `@apache-superset/core` package will be mapped to the corresponding implementations during bootstrap in the `setupExtensionsAPI` function.
``` typescript
import * as supersetCore from '@apache-superset/core';
import {
authentication,
core,
commands,
environment,
extensions,
sqlLab,
} from 'src/extensions';
export default function setupExtensionsAPI() {
window.superset = {
...supersetCore,
authentication,
core,
commands,
environment,
extensions,
sqlLab,
};
}
```

55
docs/developer_portal/extensions/extension-metadata.md Normal file
View File

@@ -0,0 +1,55 @@
---
title: Extension Metadata
sidebar_position: 4
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Extension Metadata
The `extension.json` file contains all metadata necessary for the host application to understand and manage the extension:
``` json
{
"name": "dataset_references",
"version": "1.0.0",
"frontend": {
"contributions": {
"views": {
"sqllab.panels": [
{
"id": "dataset_references.main",
"name": "Dataset references"
}
]
}
},
"moduleFederation": {
"exposes": ["./index"]
}
},
"backend": {
"entryPoints": ["dataset_references.entrypoint"],
"files": ["backend/src/dataset_references/**/*.py"]
},
}
```
The `contributions` section declares how the extension extends Superset's functionality through views, commands, menus, and other contribution types. The `backend` section specifies entry points and files to include in the bundle.

78
docs/developer_portal/extensions/extension-project-structure.md Normal file
View File

@@ -0,0 +1,78 @@
---
title: Extension Project Structure
sidebar_position: 3
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Extension Project Structure
The `apache-superset-extensions-cli` package provides a command-line interface (CLI) that streamlines the extension development workflow. It offers the following commands:
```
superset-extensions init: Generates the initial folder structure and scaffolds a new extension project.
superset-extensions build: Builds extension assets.
superset-extensions bundle: Packages the extension into a .supx file.
superset-extensions dev: Automatically rebuilds the extension as files change.
```
When creating a new extension with `superset-extensions init <extension-name>`, the CLI generates a standardized folder structure:
```
dataset_references/
├── extension.json
├── frontend/
│ ├── src/
│ ├── webpack.config.js
│ ├── tsconfig.json
│ └── package.json
├── backend/
│ ├── src/
│ └── dataset_references/
│ ├── tests/
│ ├── pyproject.toml
│ └── requirements.txt
├── dist/
│ ├── manifest.json
│ ├── frontend
│ └── dist/
│ ├── remoteEntry.d7a9225d042e4ccb6354.js
│ └── 900.038b20cdff6d49cfa8d9.js
│ └── backend
│ └── dataset_references/
│ ├── __init__.py
│ ├── api.py
│ └── entrypoint.py
├── dataset_references-1.0.0.supx
└── README.md
```
The `extension.json` file serves as the declared metadata for the extension, containing the extension's name, version, author, description, and a list of capabilities. This file is essential for the host application to understand how to load and manage the extension.
The `frontend` directory contains the source code for the frontend components of the extension, including React components, styles, and assets. The `webpack.config.js` file is used to configure Webpack for building the frontend code, while the `tsconfig.json` file defines the TypeScript configuration for the project. The `package.json` file specifies the dependencies and scripts for building and testing the frontend code.
The `backend` directory contains the source code for the backend components of the extension, including Python modules, tests, and configuration files. The `pyproject.toml` file is used to define the Python package and its dependencies, while the r`equirements.txt` file lists the required Python packages for the extension. The `src` folder contains the functional backend source files, `tests` directory contains unit tests for the backend code, ensuring that the extension behaves as expected and meets the defined requirements.
The `dist` directory is built when running the `build` or `dev` command, and contains the files that will be included in the bundle. The `manifest.json` file contains critical metadata about the extension, including the majority of the contents of the `extension.json` file, but also other build-time information, like the name of the built Webpack Module Federation remote entry file. The files in the `dist` directory will be zipped into the final `.supx` file. Although this file is technically a zip archive, the `.supx` extension makes it clear that it is a Superset extension package and follows a specific file layout. This packaged file can be distributed and installed in Superset instances.
The `README.md` file provides documentation and instructions for using the extension, including how to install, configure, and use its functionality.

90
docs/developer_portal/extensions/frontend-contribution-types.md Normal file
View File

@@ -0,0 +1,90 @@
---
title: Frontend Contribution Types
sidebar_position: 5
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Frontend Contribution Types
To facilitate the development of extensions, we will define a set of well-defined contribution types that extensions can implement. These contribution types will serve as the building blocks for extensions, allowing them to interact with the host application and provide new functionality. The initial set of contribution types will include:
## Views
Extensions can add new views or panels to the host application, such as custom SQL Lab panels, dashboards, or other UI components. Each view is registered with a unique ID and can be activated or deactivated as needed. Contribution areas are uniquely identified (e.g., `sqllab.panels` for SQL Lab panels), enabling seamless integration into specific parts of the application.
``` json
"views": {
"sqllab.panels": [
{
"id": "dataset_references.main",
"name": "Table references"
}
]
},
```
## Commands
Extensions can define custom commands that can be executed within the host application, such as context-aware actions or menu options. Each command can specify properties like a unique command identifier, an icon, a title, and a description. These commands can be invoked by users through menus, keyboard shortcuts, or other UI elements, enabling extensions to add rich, interactive functionality to Superset.
``` json
"commands": [
{
"command": "extension1.copy_query",
"icon": "CopyOutlined",
"title": "Copy Query",
"description": "Copy the current query to clipboard"
},
]
```
## Menus
Extensions can contribute new menu items or context menus to the host application, providing users with additional actions and options. Each menu item can specify properties such as the target view, the command to execute, its placement (primary, secondary, or context), and conditions for when it should be displayed. Menu contribution areas are uniquely identified (e.g., `sqllab.editor` for the SQL Lab editor), allowing extensions to seamlessly integrate their functionality into specific menus and workflows within Superset.
``` json
"menus": {
"sqllab.editor": {
"primary": [
{
"view": "builtin.editor",
"command": "extension1.copy_query"
}
],
"secondary": [
{
"view": "builtin.editor",
"command": "extension1.prettify"
}
],
"context": [
{
"view": "builtin.editor",
"command": "extension1.clear"
},
{
"view": "builtin.editor",
"command": "extension1.refresh"
}
]
},
}
```

41
docs/developer_portal/extensions/high-level-architecture.md Normal file
View File

@@ -0,0 +1,41 @@
---
title: High-level Architecture
sidebar_position: 2
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# High-level Architecture
It is important to note that this SIP is not intended to address every aspect of the architecture in a single document. Rather, it should be seen as a chapter in an ongoing book: it establishes foundational concepts and direction, while recognizing that many important topics remain to be explored. Many topics not covered here are expected to be addressed in subsequent SIPs as the architecture and community needs evolve.
The following diagram illustrates the overall architecture, emphasizing the relationships between the host application, extensions, and the supporting packages that will be developed to enable this ecosystem.
<img width="955" height="586" alt="Image" src="https://github.com/user-attachments/assets/cc2a41df-55a4-48c8-b056-35f7a1e567c6" />
On the frontend side, as a result of discussions in [[SIP-169] Proposal for Extracting and Publishing Superset Core UI Components](https://github.com/apache/superset/issues/33441), the `@apache-superset/core` package will be created to provide all the essential building blocks for both the host application and extensions, including shared UI components, utility functions, APIs, and type definitions. By centralizing these resources, extensions and built-in features can use the same APIs and components, ensuring consistency, type safety, and a seamless user experience across the Superset ecosystem. This package will be versioned to support safe evolution of the platform while maintaining compatibility for both internal and external features.
On the backend side, the `apache-superset-core` package will expose key classes and APIs needed by extensions that provide backend functionality such as extending Superset's API, providing database connectors, or customizing the security manager. It will contain dependencies on critical libraries like FAB, SQLAlchemy, etc and like `@apache-superset/core`, it will be versioned to ensure compatibility and stability.
The `apache-superset-extensions-cli` package will provide a comprehensive set of CLI commands for extension development, including tools for code generation, building, and packaging extensions. These commands streamline the development workflow, making it easier for developers to scaffold new projects, build and bundle their code, and distribute extensions efficiently. By standardizing these processes, the CLI helps ensure that extensions are built in a consistent manner, remain compatible with evolving versions of Superset, and adhere to best practices across the ecosystem.
Extension projects might have references to all packages, depending on their specific needs. For example, an extension that provides a custom SQL editor might depend on the `apache-superset-core` package for backend functionality, while also using the `@apache-superset/core` package for UI components and type definitions. This modular approach allows extension authors to choose the dependencies that best suit their needs, while also promoting consistency and reusability across the ecosystem.
The host application (Superset) will depend on core packages and is responsible for providing the implementation of the APIs defined in them. It will expose a new endpoint (`/api/v1/extensions`) for extension registration and management, and a dedicated UI for managing extensions. Registered extensions will be stored in the metadata database in a table called `extensions`, which will contain information such as the extension's name, version, author, contributed features, exposed modules, relevant metadata and the built frontend and/or backend code.

120
docs/developer_portal/extensions/interacting-with-host.md Normal file
View File

@@ -0,0 +1,120 @@
---
title: Interacting with the Host
sidebar_position: 6
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Interacting with the Host
Extensions interact with Superset through well-defined, versioned APIs provided by the `@apache-superset/core` (frontend) and `apache-superset-core` (backend) packages. These APIs are designed to be stable, discoverable, and consistent for both built-in and external extensions.
**Frontend APIs** (via `@apache-superset/core)`:
The frontend extension APIs in Superset are organized into logical namespaces such as `authentication`, `commands`, `extensions`, `sqlLab`, and others. Each namespace groups related functionality, making it easy for extension authors to discover and use the APIs relevant to their needs. For example, the `sqlLab` namespace provides events and methods specific to SQL Lab, allowing extensions to react to user actions and interact with the SQL Lab environment:
``` typescript
export const getCurrentTab: () => Tab | undefined;
export const getDatabases: () => Database[];
export const getTabs: () => Tab[];
export const onDidChangeEditorContent: Event<string>;
export const onDidClosePanel: Event<Panel>;
export const onDidChangeActivePanel: Event<Panel>;
export const onDidChangeTabTitle: Event<string>;
export const onDidQueryRun: Event<Editor>;
export const onDidQueryStop: Event<Editor>;
```
The following code demonstrates more examples of the existing frontend APIs:
``` typescript
import { core, commands, sqlLab, authentication, Button } from '@apache-superset/core';
import MyPanel from './MyPanel';
export function activate(context) {
// Register a new panel (view) in SQL Lab and use shared UI components in your extension's React code
const panelDisposable = core.registerView('my_extension.panel', <MyPanel><Button/></MyPanel>);
// Register a custom command
const commandDisposable = commands.registerCommand('my_extension.copy_query', {
title: 'Copy Query',
execute: () => {
// Command logic here
},
});
// Listen for query run events in SQL Lab
const eventDisposable = sqlLab.onDidQueryRun(editor => {
// Handle query execution event
});
// Access a CSRF token for secure API requests
authentication.getCSRFToken().then(token => {
// Use token as needed
});
// Add all disposables for automatic cleanup on deactivation
context.subscriptions.push(panelDisposable, commandDisposable, eventDisposable);
}
```
**Backend APIs** (via `apache-superset-core`):
Backend APIs follow a similar pattern, providing access to Superset's models, sessions, and query capabilities. Extensions can register REST API endpoints, access the metadata database, and interact with Superset's core functionality.
Extension endpoints are registered under a dedicated `/extensions` namespace to avoid conflicting with built-in endpoints and also because they don't share the same version constraints. By grouping all extension endpoints under `/extensions`, Superset establishes a clear boundary between core and extension functionality, making it easier to manage, document, and secure both types of APIs.
``` python
from superset_core.api import rest_api, models, query
from .api import DatasetReferencesAPI
# Register a new extension REST API
rest_api.add_extension_api(DatasetReferencesAPI)
# Access Superset models with simple queries that filter out entities that
# the user doesn't have access to
databases = models.get_databases(id=database_id)
if not databases:
return self.response_404()
database = databases[0]
# Perform complex queries using SQLAlchemy BaseQuery, also filtering
# out inaccessible entities
session = models.get_session()
db_model = models.get_database_model())
database_query = session.query(db_model.database_name.ilike("%abc%")
databases_containing_abc = models.get_databases(query)
# Bypass security model for highly custom use cases
session = models.get_session()
db_model = models.get_database_model())
all_databases_containg_abc = session.query(db_model.database_name.ilike("%abc%").all()
```
In the future, we plan to expand the backend APIs to support configuring security models, database engines, SQL Alchemy dialects, etc.

41
docs/developer_portal/extensions/lifecycle-management.md Normal file
View File

@@ -0,0 +1,41 @@
---
title: Lifecycle and Management
sidebar_position: 9
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Lifecycle and Management
Superset will manage the full lifecycle of extensions, including activation, deactivation, and cleanup. The lifecycle is designed to ensure that extensions can safely register resources and reliably clean them up when no longer needed.
## Frontend lifecycle
- When an extension is activated, its `activate(context)` function is called. The extension should register all event listeners, commands, views, and other contributions using the provided context, and add any disposables to `context.disposables`.
- When the extension is deactivated (e.g., disabled or uninstalled), Superset automatically calls `dispose()` on all items in `context.disposables`, ensuring that event listeners, commands, and UI contributions are removed and memory leaks are avoided.
## Backend lifecycle
- Backend entry points, which can add REST API endpoints or execute arbitrary backend code, are eagerly evaluated during startup. Other backend code is lazily loaded when needed, ensuring minimal startup latency.
- In the future, we plan to leverage mechanisms like Redis pub/sub (already an optional dependency of Superset) to dynamically manage the lifecycle of extensions' backend functionality. This will ensure that all running instances of the Superset backend have the same available extensions without requiring a restart. This will be addressed in a follow-up SIP.
The proof-of-concept (POC) code for this SIP already implements a management module where administrators can upload, delete, enable/disable, and inspect the manifest for installed extensions via the Superset UI, making extension operations straightforward. These operations are currently supported dynamically for frontend extensions. For backend extensions, dynamic upload, deletion, and enable/disable are planned for a future iteration; at present, changes to backend extensions (such as uploading, deleting, or enabling/disabling) still require a server restart.
https://github.com/user-attachments/assets/4eb7064b-3290-4e4c-b88b-52d8d1c11245

78
docs/developer_portal/extensions/overview.md Normal file
View File

@@ -0,0 +1,78 @@
---
title: Extensions Overview
sidebar_position: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Extensions Overview
Apache Superset's extension system allows developers to enhance and customize Superset's functionality through a modular, plugin-based architecture. Extensions can add new visualization types, custom UI components, data processing capabilities, and integration points.
## What are Superset Extensions?
Superset extensions are self-contained packages that extend the core platform's capabilities. They follow a standardized architecture that ensures compatibility, security, and maintainability while providing powerful customization options.
## Extension Architecture
- **[Architectural Principles](./architectural-principles)** - Core design principles guiding extension development
- **[High-level Architecture](./high-level-architecture)** - System overview and component relationships
- **[Extension Project Structure](./extension-project-structure)** - Standard project layout and organization
- **[Extension Metadata](./extension-metadata)** - Configuration and manifest structure
## Development Guide
- **[Frontend Contribution Types](./frontend-contribution-types)** - Types of UI contributions available
- **[Interacting with Host](./interacting-with-host)** - Communication patterns with Superset core
- **[Dynamic Module Loading](./dynamic-module-loading)** - Runtime loading and dependency management
- **[Development Mode](./development-mode)** - Tools and workflows for extension development
## Deployment & Management
- **[Deploying Extension](./deploying-extension)** - Packaging and distribution strategies
- **[Lifecycle Management](./lifecycle-management)** - Installation, updates, and removal
- **[Versioning](./versioning)** - Version management and compatibility
- **[Security Implications](./security-implications)** - Security considerations and best practices
## Hands-on Examples
- **[Proof of Concept](./proof-of-concept)** - Complete Hello World extension walkthrough
## Extension Capabilities
Extensions can provide:
- **Custom Visualizations**: New chart types and data visualization components
- **UI Enhancements**: Custom dashboards, panels, and interactive elements
- **Data Connectors**: Integration with external data sources and APIs
- **Workflow Automation**: Custom actions and batch processing capabilities
- **Authentication Providers**: SSO and custom authentication mechanisms
- **Theme Customization**: Custom styling and branding options
## Getting Started
1. **Learn the Architecture**: Start with [Architectural Principles](./architectural-principles) to understand the design philosophy
2. **Set up Development**: Follow the [Development Mode](./development-mode) guide to configure your environment
3. **Build Your First Extension**: Complete the [Proof of Concept](./proof-of-concept) tutorial
4. **Deploy and Share**: Use the [Deploying Extension](./deploying-extension) guide to package your extension
## Extension Ecosystem
The extension system is designed to foster a vibrant ecosystem of community-contributed functionality. By following the established patterns and guidelines, developers can create extensions that seamlessly integrate with Superset while maintaining the platform's reliability and performance standards.

288
docs/developer_portal/extensions/proof-of-concept.md Normal file
View File

@@ -0,0 +1,288 @@
---
title: "Example: Hello World"
sidebar_position: 14
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Example: Hello World
:::warning Work in Progress
This documentation is under active development. Some features described may not be fully implemented yet.
:::
This guide walks you through creating your first Superset extension - a simple "Hello World" panel for SQL Lab. You'll learn how to create, build, package, install, and test a basic extension.
## Prerequisites
Before starting, ensure you have:
- Node.js 18+ and npm installed
- Python 3.9+ installed
- A running Superset development environment with `ENABLE_EXTENSIONS = True` in your config
- Basic knowledge of React and TypeScript
## Step 1: Initialize Your Extension
First, install the Superset extension CLI and create a new extension project:
```bash
# Install the CLI globally
pip install apache-superset-extensions-cli
# Create a new extension
superset-extensions init hello-world
cd hello-world
```
This creates the following structure:
```
hello-world/
├── extension.json # Extension metadata
├── frontend/ # Frontend code
│ ├── src/
│ │ └── index.tsx # Main entry point
│ ├── package.json
│ └── webpack.config.js
├── backend/ # Backend code (optional)
│ └── src/
└── README.md
```
## Step 2: Configure Extension Metadata
Edit `extension.json` to define your extension's capabilities:
```json
{
"name": "hello_world",
"displayName": "Hello World Extension",
"version": "1.0.0",
"description": "A simple Hello World panel for SQL Lab",
"author": "Your Name",
"license": "Apache-2.0",
"superset": {
"minVersion": "4.0.0"
},
"frontend": {
"contributions": {
"views": {
"sqllab.panels": [
{
"id": "hello_world.main",
"name": "Hello World",
"icon": "SmileOutlined"
}
]
},
"commands": [
{
"command": "hello_world.greet",
"title": "Say Hello",
"description": "Display a greeting message"
}
]
}
}
}
```
## Step 3: Create the Frontend Component
Create `frontend/src/HelloWorldPanel.tsx`:
```tsx
import React, { useState, useEffect } from 'react';
import { Card, Button, Alert } from '@apache-superset/core';
export const HelloWorldPanel: React.FC = () => {
const [greeting, setGreeting] = useState('Hello, Superset!');
const [queryCount, setQueryCount] = useState(0);
// Listen for query executions
useEffect(() => {
const handleQueryRun = () => {
setQueryCount(prev => prev + 1);
setGreeting(`Hello! You've run ${queryCount + 1} queries.`);
};
// Subscribe to SQL Lab events
const disposable = window.superset?.sqlLab?.onDidQueryRun?.(handleQueryRun);
return () => disposable?.dispose?.();
}, [queryCount]);
return (
<Card title="Hello World Extension">
<Alert
message={greeting}
type="success"
showIcon
/>
<p style={{ marginTop: 16 }}>
This is your first Superset extension! 🎉
</p>
<p>
Queries executed this session: <strong>{queryCount}</strong>
</p>
<Button
onClick={() => setGreeting('Hello from the button!')}
style={{ marginTop: 16 }}
>
Click Me
</Button>
</Card>
);
};
```
## Step 4: Set Up the Entry Point
Update `frontend/src/index.tsx`:
```tsx
import { ExtensionContext } from '@apache-superset/core';
import { HelloWorldPanel } from './HelloWorldPanel';
export function activate(context: ExtensionContext) {
console.log('Hello World extension is activating!');
// Register the panel
const panel = context.registerView(
'hello_world.main',
<HelloWorldPanel />
);
// Register the command
const command = context.registerCommand('hello_world.greet', {
execute: () => {
console.log('Hello from the command!');
// You could trigger panel updates or show notifications here
}
});
// Add disposables for cleanup
context.subscriptions.push(panel, command);
}
export function deactivate() {
console.log('Hello World extension is deactivating!');
}
```
## Step 5: Build the Extension
Build your extension assets:
```bash
# Install dependencies
cd frontend
npm install
# Build the extension
cd ..
superset-extensions build
# This creates a dist/ folder with your built assets
```
> **Note:** The `superset-extensions` CLI handles webpack configuration automatically. Superset already has Module Federation configured, so you don't need to set up webpack yourself unless you have specific advanced requirements.
## Step 6: Package the Extension
Create the distributable `.supx` file:
```bash
superset-extensions bundle
# This creates hello_world-1.0.0.supx
```
## Step 7: Install in Superset
Upload your extension to a running Superset instance:
### Option A: Via API
```bash
curl -X POST http://localhost:8088/api/v1/extensions/import/ \
-H "Authorization: Bearer YOUR_TOKEN" \
-F "bundle=@hello_world-1.0.0.supx"
```
### Option B: Via UI
1. Navigate to Settings → Extensions
2. Click "Upload Extension"
3. Select your `hello_world-1.0.0.supx` file
4. Click "Install"
## Step 9: Test Your Extension
1. Open SQL Lab in Superset
2. Look for the "Hello World" panel in the panels dropdown or sidebar
3. The panel should display your greeting message
4. Run a SQL query and watch the query counter increment
5. Click the button to see the greeting change
## Step 10: Development Mode (Optional)
For faster development iteration, use local development mode:
1. Add to your `superset_config.py`:
```python
LOCAL_EXTENSIONS = [
"/path/to/hello-world"
]
ENABLE_EXTENSIONS = True
```
2. Run the extension in watch mode:
```bash
superset-extensions dev
```
3. Changes will be reflected immediately without rebuilding
## Troubleshooting
### Extension Not Loading
- Check that `ENABLE_EXTENSIONS = True` in your Superset config
- Verify the extension is listed in Settings → Extensions
- Check browser console for errors
- Ensure all dependencies are installed
### Build Errors
- Make sure you have the correct Node.js version (18+)
- Clear node_modules and reinstall: `rm -rf node_modules && npm install`
- Check that webpack.config.js is properly configured
### Panel Not Visible
- Verify the contribution point in extension.json matches `sqllab.panels`
- Check that the panel ID is unique
- Restart Superset after installing the extension
## Next Steps
Now that you have a working Hello World extension, you can:
- Add more complex UI components
- Integrate with Superset's API to fetch data
- Add backend functionality for data processing
- Create custom commands and menu items
- Listen to more SQL Lab events
For more advanced examples, explore the other pages in this documentation section.

33
docs/developer_portal/extensions/security-implications.md Normal file
View File

@@ -0,0 +1,33 @@
---
title: Security Implications and Responsibilities
sidebar_position: 12
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Security Implications and Responsibilities
By default, extensions are disabled and must be explicitly enabled by setting the `ENABLE_EXTENSIONS` feature flag. Built-in extensions are included as part of the Superset codebase and are held to the same security standards and review processes as the rest of the application.
For external extensions, administrators are responsible for evaluating and verifying the security of any extensions they choose to install, just as they would when installing third-party NPM or PyPI packages. At this stage, all extensions run in the same context as the host application, without additional sandboxing. This means that external extensions can impact the security and performance of a Superset environment in the same way as any other installed dependency.
We plan to introduce an optional sandboxed execution model for extensions in the future (as part of an additional SIP). Until then, administrators should exercise caution and follow best practices when selecting and deploying third-party extensions. A directory of known Superset extensions may be maintained in a means similar to [this page](https://github.com/apache/superset/wiki/Superset-Third%E2%80%90Party-Plugins-Directory) on the wiki. We also discussed the possibility of introducing a shared registry for vetted extensions but decided to leave it out of the initial scope of the project. We might introduce a registry at a later stage depending on the evolution of extensions created by the community.
Any performance or security vulnerabilities introduced by external extensions should be reported directly to the extension author, not as Superset vulnerabilities. Any security concerns regarding built-in extensions (included in Superset's monorepo) should be reported to the Superset Security mailing list for triage and resolution by maintainers.

31
docs/developer_portal/extensions/versioning.md Normal file
View File

@@ -0,0 +1,31 @@
---
title: Versioning
sidebar_position: 11
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Versioning
All core packages published to NPM and PyPI—including `@apache-superset/core`, `apache-superset-core`, and `apache-superset-extensions-cli`—will follow semantic versioning ([semver](https://semver.org/)). This means that any breaking changes to public APIs, interfaces, or extension points will result in a major version bump, while new features and non-breaking changes will be released as minor or patch updates.
During the initial development phase, packages will remain under version 0.x, allowing for rapid iteration and the introduction of breaking changes as the architecture evolves based on real-world feedback. Once the APIs and extension points are considered stable, we will release version 1.0.0 and begin enforcing strict server practices.
To minimize disruption, breaking changes will be carefully planned and communicated in advance. We will provide clear migration guides and changelogs to help extension authors and core contributors adapt to new versions. Where possible, we will deprecate APIs before removing them, giving developers time to transition to newer interfaces.