From 441e043bff46a1c77f2fb613a4771075a56309bf Mon Sep 17 00:00:00 2001 From: Evan Rusackas Date: Wed, 8 Oct 2025 03:18:40 -0400 Subject: [PATCH] feat(docs): Populate Developer Portal with comprehensive documentation framework (#35217) Co-authored-by: Claude --- .pre-commit-config.yaml | 2 +- CONTRIBUTING.md | 22 +- docs/DOCS_CLAUDE.md | 642 ++++++++++ docs/developer_portal/api/frontend.md | 999 +++++++++------ .../developer_portal/architecture/overview.md | 440 +++---- .../capabilities/common-capabilities.md | 55 + .../capabilities/extending-workbench.md | 62 + .../developer_portal/capabilities/overview.md | 53 + docs/developer_portal/capabilities/theming.md | 61 + docs/developer_portal/cli/overview.md | 758 ++++++----- .../coding-guidelines/overview.md | 44 + .../contributing/code-review.md | 339 +++++ .../contributing/development-setup.md | 1108 +++++++++++++++++ .../contributing/guidelines.md | 415 ++++++ docs/developer_portal/contributing/howtos.md | 528 ++++++++ .../contributing/issue-reporting.md | 418 +++++++ .../developer_portal/contributing/overview.md | 166 +++ .../contributing/release-process.md | 469 +++++++ .../contributing/resources.md | 133 ++ .../contributing/submitting-pr.md | 321 +++++ docs/developer_portal/examples/index.md | 464 ------- .../extensions/architectural-principles.md | 36 + .../extensions/built-in-features.md | 36 + .../extensions/deploying-extension.md | 45 + .../extensions/development-mode.md | 48 + .../extensions/dynamic-module-loading.md | 84 ++ .../extensions/extension-metadata.md | 55 + .../extensions/extension-project-structure.md | 78 ++ .../extensions/frontend-contribution-types.md | 90 ++ .../extensions/high-level-architecture.md | 41 + .../extensions/interacting-with-host.md | 120 ++ .../extensions/lifecycle-management.md | 41 + docs/developer_portal/extensions/overview.md | 78 ++ .../extensions/proof-of-concept.md | 288 +++++ .../extensions/security-implications.md | 33 + .../developer_portal/extensions/versioning.md | 31 + .../developer_portal/getting-started/index.md | 248 ---- .../guidelines/backend-style-guidelines.md | 318 +++++ .../backend/dao-style-guidelines.md | 365 ++++++ .../guidelines/design-guidelines.md | 148 +++ .../guidelines/frontend-style-guidelines.md | 44 + .../frontend/component-style-guidelines.md | 258 ++++ .../frontend/emotion-styling-guidelines.md | 346 +++++ .../guidelines/frontend/testing-guidelines.md | 297 +++++ .../guides/command-palette.md | 61 + .../developer_portal/guides/custom-editors.md | 64 + docs/developer_portal/guides/overview.md | 58 + .../guides/virtual-documents.md | 63 + docs/developer_portal/guides/webviews.md | 61 + docs/developer_portal/index.md | 184 +-- .../references/activation-events.md | 549 ++++++++ docs/developer_portal/references/api.md | 101 ++ .../references/contribution-points.md | 475 +++++++ docs/developer_portal/references/manifest.md | 526 ++++++++ docs/developer_portal/references/overview.md | 74 ++ docs/developer_portal/sidebars.js | 27 + .../testing/backend-testing.md | 68 + docs/developer_portal/testing/ci-cd.md | 70 ++ docs/developer_portal/testing/e2e-testing.md | 80 ++ .../testing/frontend-testing.md | 61 + docs/developer_portal/testing/overview.md | 164 +++ docs/developer_portal/ux/accessibility.md | 70 ++ docs/developer_portal/ux/best-practices.md | 73 ++ docs/developer_portal/ux/design-principles.md | 68 + docs/developer_portal/ux/overview.md | 62 + docs/developer_portal/viz-plugins/controls.md | 77 ++ .../viz-plugins/creating-viz-plugin.md | 80 ++ docs/developer_portal/viz-plugins/overview.md | 67 + .../viz-plugins/transforming-data.md | 76 ++ docs/docs/configuration/alerts-reports.mdx | 2 +- docs/docusaurus.config.ts | 28 +- docs/package.json | 2 +- docs/sidebarTutorials.js | 88 +- docs/versions-config.json | 3 +- 74 files changed, 12216 insertions(+), 1793 deletions(-) create mode 100644 docs/DOCS_CLAUDE.md create mode 100644 docs/developer_portal/capabilities/common-capabilities.md create mode 100644 docs/developer_portal/capabilities/extending-workbench.md create mode 100644 docs/developer_portal/capabilities/overview.md create mode 100644 docs/developer_portal/capabilities/theming.md create mode 100644 docs/developer_portal/coding-guidelines/overview.md create mode 100644 docs/developer_portal/contributing/code-review.md create mode 100644 docs/developer_portal/contributing/development-setup.md create mode 100644 docs/developer_portal/contributing/guidelines.md create mode 100644 docs/developer_portal/contributing/howtos.md create mode 100644 docs/developer_portal/contributing/issue-reporting.md create mode 100644 docs/developer_portal/contributing/overview.md create mode 100644 docs/developer_portal/contributing/release-process.md create mode 100644 docs/developer_portal/contributing/resources.md create mode 100644 docs/developer_portal/contributing/submitting-pr.md delete mode 100644 docs/developer_portal/examples/index.md create mode 100644 docs/developer_portal/extensions/architectural-principles.md create mode 100644 docs/developer_portal/extensions/built-in-features.md create mode 100644 docs/developer_portal/extensions/deploying-extension.md create mode 100644 docs/developer_portal/extensions/development-mode.md create mode 100644 docs/developer_portal/extensions/dynamic-module-loading.md create mode 100644 docs/developer_portal/extensions/extension-metadata.md create mode 100644 docs/developer_portal/extensions/extension-project-structure.md create mode 100644 docs/developer_portal/extensions/frontend-contribution-types.md create mode 100644 docs/developer_portal/extensions/high-level-architecture.md create mode 100644 docs/developer_portal/extensions/interacting-with-host.md create mode 100644 docs/developer_portal/extensions/lifecycle-management.md create mode 100644 docs/developer_portal/extensions/overview.md create mode 100644 docs/developer_portal/extensions/proof-of-concept.md create mode 100644 docs/developer_portal/extensions/security-implications.md create mode 100644 docs/developer_portal/extensions/versioning.md delete mode 100644 docs/developer_portal/getting-started/index.md create mode 100644 docs/developer_portal/guidelines/backend-style-guidelines.md create mode 100644 docs/developer_portal/guidelines/backend/dao-style-guidelines.md create mode 100644 docs/developer_portal/guidelines/design-guidelines.md create mode 100644 docs/developer_portal/guidelines/frontend-style-guidelines.md create mode 100644 docs/developer_portal/guidelines/frontend/component-style-guidelines.md create mode 100644 docs/developer_portal/guidelines/frontend/emotion-styling-guidelines.md create mode 100644 docs/developer_portal/guidelines/frontend/testing-guidelines.md create mode 100644 docs/developer_portal/guides/command-palette.md create mode 100644 docs/developer_portal/guides/custom-editors.md create mode 100644 docs/developer_portal/guides/overview.md create mode 100644 docs/developer_portal/guides/virtual-documents.md create mode 100644 docs/developer_portal/guides/webviews.md create mode 100644 docs/developer_portal/references/activation-events.md create mode 100644 docs/developer_portal/references/api.md create mode 100644 docs/developer_portal/references/contribution-points.md create mode 100644 docs/developer_portal/references/manifest.md create mode 100644 docs/developer_portal/references/overview.md create mode 100644 docs/developer_portal/testing/backend-testing.md create mode 100644 docs/developer_portal/testing/ci-cd.md create mode 100644 docs/developer_portal/testing/e2e-testing.md create mode 100644 docs/developer_portal/testing/frontend-testing.md create mode 100644 docs/developer_portal/testing/overview.md create mode 100644 docs/developer_portal/ux/accessibility.md create mode 100644 docs/developer_portal/ux/best-practices.md create mode 100644 docs/developer_portal/ux/design-principles.md create mode 100644 docs/developer_portal/ux/overview.md create mode 100644 docs/developer_portal/viz-plugins/controls.md create mode 100644 docs/developer_portal/viz-plugins/creating-viz-plugin.md create mode 100644 docs/developer_portal/viz-plugins/overview.md create mode 100644 docs/developer_portal/viz-plugins/transforming-data.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1a2a799bb9e..2a1586c5937 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -68,7 +68,7 @@ repos: files: ^superset-frontend/.*\.(js|jsx|ts|tsx)$ - id: eslint-docs name: eslint (docs) - entry: bash -c 'cd docs && FILES=$(echo "$@" | sed "s|docs/||g") && yarn eslint --fix --ext .js,.jsx,.ts,.tsx --quiet $FILES' + entry: bash -c 'cd docs && FILES=$(echo "$@" | sed "s|docs/||g") && yarn eslint --fix --quiet $FILES' language: system pass_filenames: true files: ^docs/.*\.(js|jsx|ts|tsx)$ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 06ed3b146c5..620eb66de58 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,7 +5,7 @@ regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at + with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 @@ -16,9 +16,23 @@ specific language governing permissions and limitations under the License. --> +# Contributing to Apache Superset + Contributions are welcome and are greatly appreciated! Every little bit helps, and credit will always be given. -All matters related to contributions have moved to [this section of -the official Superset documentation](https://superset.apache.org/docs/contributing/). Source for the documentation is -[located here](https://github.com/apache/superset/tree/master/docs/docs). +## Developer Portal + +All developer and contribution documentation has moved to the Apache Superset Developer Portal: + +**[📚 View the Developer Portal →](https://superset.apache.org/docs/developer-portal/)** + +The Developer Portal includes comprehensive guides for: +- [Contributing Overview](https://superset.apache.org/docs/developer-portal/contributing/overview) +- [Development Setup](https://superset.apache.org/docs/developer-portal/contributing/development-setup) +- [Submitting Pull Requests](https://superset.apache.org/docs/developer-portal/contributing/submitting-pr) +- [Contribution Guidelines](https://superset.apache.org/docs/developer-portal/contributing/guidelines) +- [Code Review Process](https://superset.apache.org/docs/developer-portal/contributing/code-review) +- [Development How-tos](https://superset.apache.org/docs/developer-portal/contributing/howtos) + +Source for the Developer Portal documentation is [located here](https://github.com/apache/superset/tree/master/docs/developer_portal). diff --git a/docs/DOCS_CLAUDE.md b/docs/DOCS_CLAUDE.md new file mode 100644 index 00000000000..42fd0ad71a9 --- /dev/null +++ b/docs/DOCS_CLAUDE.md @@ -0,0 +1,642 @@ + + +# LLM Context Guide for Apache Superset Documentation + +This guide helps LLMs work with the Apache Superset documentation site built with Docusaurus 3. + +## 📍 Current Directory Context + +You are currently in the `/docs` subdirectory of the Apache Superset repository. When referencing files from the main codebase, use `../` to access the parent directory. + +``` +/Users/evan_1/GitHub/superset/ # Main repository root +├── superset/ # Python backend code +├── superset-frontend/ # React/TypeScript frontend +└── docs/ # Documentation site (YOU ARE HERE) + ├── docs/ # Main documentation content + ├── developer_portal/ # Developer guides (currently disabled) + ├── components/ # Component playground (currently disabled) + └── docusaurus.config.ts # Site configuration +``` + +## 🚀 Quick Commands + +```bash +# Development +yarn start # Start dev server on http://localhost:3000 +yarn stop # Stop running dev server +yarn build # Build production site +yarn serve # Serve built site locally + +# Version Management (USE THESE, NOT docusaurus commands) +yarn version:add:docs # Add new docs version +yarn version:add:developer_portal # Add developer portal version +yarn version:add:components # Add components version +yarn version:remove:docs # Remove docs version +yarn version:remove:developer_portal # Remove developer portal version +yarn version:remove:components # Remove components version + +# Quality Checks +yarn typecheck # TypeScript validation +yarn eslint # Lint TypeScript/JavaScript files +``` + +## 📁 Documentation Structure + +### Main Documentation (`/docs`) +The primary documentation lives in `/docs` with this structure: + +``` +docs/ +├── intro.md # Auto-generated from ../README.md +├── quickstart.mdx # Getting started guide +├── api.mdx # API reference with Swagger UI +├── faq.mdx # Frequently asked questions +├── installation/ # Installation guides +│ ├── installation-methods.mdx +│ ├── docker-compose.mdx +│ ├── docker-builds.mdx +│ ├── kubernetes.mdx +│ ├── pypi.mdx +│ └── architecture.mdx +├── configuration/ # Configuration guides +│ ├── configuring-superset.mdx +│ ├── alerts-reports.mdx +│ ├── caching.mdx +│ ├── databases.mdx +│ └── [more config docs] +├── using-superset/ # User guides +│ ├── creating-your-first-dashboard.md +│ ├── exploring-data.mdx +│ └── [more user docs] +├── contributing/ # Contributor guides +│ ├── development.mdx +│ ├── testing-locally.mdx +│ └── [more contributor docs] +└── security/ # Security documentation + ├── security.mdx + └── [security guides] +``` + +### Developer Portal (`/developer_portal`) - Currently Disabled +When enabled, contains developer-focused content: +- API documentation +- Architecture guides +- CLI tools +- Code examples + +### Component Playground (`/components`) - Currently Disabled +When enabled, provides interactive component examples for UI development. + +## 📝 Documentation Standards + +### File Types +- **`.md` files**: Basic Markdown documents +- **`.mdx` files**: Markdown with JSX - can include React components +- **`.tsx` files in `/src`**: Custom React components and pages + +### Frontmatter Structure +Every documentation page should have frontmatter: + +```yaml +--- +title: Page Title +description: Brief description for SEO +sidebar_position: 1 # Optional: controls order in sidebar +--- +``` + +### MDX Component Usage +MDX files can import and use React components: + +```mdx +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + + + + ```bash + npm install superset + ``` + + + ```bash + yarn add superset + ``` + + +``` + +### Code Blocks +Use triple backticks with language identifiers: + +````markdown +```python +def hello_world(): + print("Hello, Superset!") +``` + +```sql title="Example Query" +SELECT * FROM users WHERE active = true; +``` + +```bash +# Installation command +pip install apache-superset +``` +```` + +### Admonitions +Docusaurus supports various admonition types: + +```markdown +:::note +This is a note +::: + +:::tip +This is a tip +::: + +:::warning +This is a warning +::: + +:::danger +This is a danger warning +::: + +:::info +This is an info box +::: +``` + +## 🔄 Version Management + +### Version Configuration +Versions are managed through `versions-config.json`: + +```json +{ + "docs": { + "disabled": false, + "lastVersion": "6.0.0", // Default version shown + "includeCurrentVersion": true, // Show "Next" version + "onlyIncludeVersions": ["current", "6.0.0"], + "versions": { + "current": { + "label": "Next", + "path": "", + "banner": "unreleased" // Shows warning banner + }, + "6.0.0": { + "label": "6.0.0", + "path": "6.0.0", + "banner": "none" + } + } + } +} +``` + +### Creating New Versions +**IMPORTANT**: Always use the custom scripts, NOT native Docusaurus commands: + +```bash +# ✅ CORRECT - Updates both Docusaurus and versions-config.json +yarn version:add:docs 6.1.0 + +# ❌ WRONG - Only updates Docusaurus, breaks version dropdown +yarn docusaurus docs:version 6.1.0 +``` + +### Version Files Created +When versioning, these files are created: +- `versioned_docs/version-X.X.X/` - Snapshot of current docs +- `versioned_sidebars/version-X.X.X-sidebars.json` - Sidebar config +- `versions.json` - List of all versions + +## 🎨 Styling and Theming + +### Custom CSS +Add custom styles in `/src/css/custom.css`: + +```css +:root { + --ifm-color-primary: #20a7c9; + --ifm-code-font-size: 95%; +} +``` + +### Custom Components +Create React components in `/src/components/`: + +```tsx +// src/components/FeatureCard.tsx +import React from 'react'; + +export default function FeatureCard({title, description}) { + return ( +
+

{title}

+

{description}

+
+ ); +} +``` + +Use in MDX: + +```mdx +import FeatureCard from '@site/src/components/FeatureCard'; + + +``` + +## 📦 Key Dependencies + +- **Docusaurus 3.8.1**: Static site generator +- **React 18.3**: UI framework +- **Ant Design 5.26**: Component library +- **@superset-ui/core**: Superset UI components +- **Swagger UI React**: API documentation +- **Prism**: Syntax highlighting + +## 🔗 Linking Strategies + +### Internal Links +Use relative paths for internal documentation: + +```markdown +[Installation Guide](./installation/docker-compose) +[Configuration](../configuration/configuring-superset) +``` + +### External Links +Always use full URLs: + +```markdown +[Apache Superset GitHub](https://github.com/apache/superset) +``` + +### Linking to Code +Reference code in the main repository: + +```markdown +See the [main configuration file](https://github.com/apache/superset/blob/master/superset/config.py) +``` + +## 🛠️ Common Documentation Tasks + +### Adding a New Guide +1. Create the `.mdx` file in the appropriate directory +2. Add frontmatter with title and description +3. Update sidebar if needed (for manual sidebar configs) + +### Adding API Documentation +The API docs use Swagger UI embedded in `/docs/api.mdx`: + +```mdx +import SwaggerUI from "swagger-ui-react"; +import "swagger-ui-react/swagger-ui.css"; + + +``` + +### Adding Interactive Examples +Use MDX to create interactive documentation: + +```mdx +import CodeBlock from '@theme/CodeBlock'; +import MyComponentExample from '!!raw-loader!../examples/MyComponent.tsx'; + +{MyComponentExample} +``` + +## 📋 Documentation Checklist + +When creating or updating documentation: + +- [ ] Clear, descriptive title in frontmatter +- [ ] Description for SEO in frontmatter +- [ ] Proper heading hierarchy (h1 -> h2 -> h3) +- [ ] Code examples with language identifiers +- [ ] Links verified (internal and external) +- [ ] Images have alt text +- [ ] Admonitions used for important notes +- [ ] Tested locally with `yarn start` +- [ ] No broken links (check with `yarn build`) + +## 🔍 Searching and Navigation + +### Sidebar Configuration +Sidebars are configured in `/sidebars.js`: + +```javascript +module.exports = { + CustomSidebar: [ + { + type: 'doc', + label: 'Introduction', + id: 'intro', + }, + { + type: 'category', + label: 'Installation', + items: [ + { + type: 'autogenerated', + dirName: 'installation', + }, + ], + }, + ], +}; +``` + +### Search +Docusaurus includes Algolia DocSearch integration configured in `docusaurus.config.ts`. + +## 🚫 Common Pitfalls to Avoid + +1. **Never use `yarn docusaurus docs:version`** - Use `yarn version:add:docs` instead +2. **Don't edit versioned docs directly** - Edit current docs and create new version +3. **Avoid absolute paths in links** - Use relative paths for maintainability +4. **Don't forget frontmatter** - Every doc needs title and description +5. **Test builds locally** - Run `yarn build` before committing + +## 🔧 Troubleshooting + +### Dev Server Issues +```bash +yarn stop # Kill any running servers +yarn clear # Clear cache +yarn start # Restart +``` + +### Build Failures +```bash +# Check for broken links +yarn build + +# TypeScript issues +yarn typecheck + +# Linting issues +yarn eslint +``` + +### Version Issues +If versions don't appear in dropdown: +1. Check `versions-config.json` includes the version +2. Verify version files exist in `versioned_docs/` +3. Restart dev server + +## 📚 Resources + +- [Docusaurus Documentation](https://docusaurus.io/docs) +- [MDX Documentation](https://mdxjs.com/) +- [Superset Contributing Guide](../CONTRIBUTING.md) +- [Main Superset Documentation](https://superset.apache.org/docs/intro) + +## 📖 Real Examples and Patterns + +### Example: Configuration Documentation Pattern +From `docs/configuration/configuring-superset.mdx`: + +```mdx +--- +title: Configuring Superset +hide_title: true +sidebar_position: 1 +version: 1 +--- + +# Configuring Superset + +## superset_config.py + +Superset exposes hundreds of configurable parameters through its +[config.py module](https://github.com/apache/superset/blob/master/superset/config.py). + +```bash +export SUPERSET_CONFIG_PATH=/app/superset_config.py +``` +``` + +**Key patterns:** +- Links to source code for reference +- Code blocks with bash/python examples +- Environment variable documentation +- Step-by-step configuration instructions + +### Example: Tutorial Documentation Pattern +From `docs/using-superset/creating-your-first-dashboard.mdx`: + +```mdx +import useBaseUrl from "@docusaurus/useBaseUrl"; + +## Creating Your First Dashboard + +:::tip +In addition to this site, [Preset.io](http://preset.io/) maintains an updated set of end-user +documentation at [docs.preset.io](https://docs.preset.io/). +::: + +### Connecting to a new database + + +``` + +**Key patterns:** +- Import Docusaurus hooks for dynamic URLs +- Use of admonitions (:::tip) for helpful information +- Screenshots with useBaseUrl for proper path resolution +- Clear section hierarchy with ### subheadings +- Step-by-step visual guides + +### Example: API Documentation Pattern +From `docs/api.mdx`: + +```mdx +import SwaggerUI from "swagger-ui-react"; +import "swagger-ui-react/swagger-ui.css"; + +## API Documentation + + +``` + +**Key patterns:** +- Embedding interactive Swagger UI +- Importing necessary CSS +- Direct API spec integration + +### Common Image Patterns + +```mdx +// For images in static folder +import useBaseUrl from "@docusaurus/useBaseUrl"; + + + +// With caption +
+ Dashboard view +
Superset Dashboard Interface
+
+``` + +### Multi-Tab Code Examples + +```mdx +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + + + + ```bash + docker-compose up + ``` + + + ```bash + kubectl apply -f superset.yaml + ``` + + + ```bash + pip install apache-superset + ``` + + +``` + +### Configuration File Examples + +```mdx +```python title="superset_config.py" +# Database connection example +SQLALCHEMY_DATABASE_URI = 'postgresql://user:password@localhost/superset' + +# Security configuration +SECRET_KEY = 'YOUR_SECRET_KEY_HERE' +WTF_CSRF_ENABLED = True + +# Feature flags +FEATURE_FLAGS = { + 'ENABLE_TEMPLATE_PROCESSING': True, + 'DASHBOARD_NATIVE_FILTERS': True, +} +``` +``` + +### Cross-Referencing Pattern + +```mdx +For detailed configuration options, see: +- [Configuring Superset](./configuration/configuring-superset) +- [Database Connections](./configuration/databases) +- [Security Settings](./security/security) + +External resources: +- [SQLAlchemy Documentation](https://docs.sqlalchemy.org/) +- [Flask Configuration](https://flask.palletsprojects.com/config/) +``` + +### Writing Installation Guides + +```mdx +## Prerequisites + +:::warning +Ensure you have Python 3.9+ and Node.js 16+ installed before proceeding. +::: + +## Installation Steps + +1. **Clone the repository** + ```bash + git clone https://github.com/apache/superset.git + cd superset + ``` + +2. **Install Python dependencies** + ```bash + pip install -e . + ``` + +3. **Initialize the database** + ```bash + superset db upgrade + superset init + ``` + +:::tip Success Check +Navigate to http://localhost:8088 and login with admin/admin +::: +``` + +### Documenting API Endpoints + +```mdx +## Chart API + +### GET /api/v1/chart/ + +Returns a list of charts. + +**Parameters:** +- `page` (optional): Page number +- `page_size` (optional): Number of items per page + +**Example Request:** +```bash +curl -X GET "http://localhost:8088/api/v1/chart/" \ + -H "Authorization: Bearer YOUR_ACCESS_TOKEN" +``` + +**Example Response:** +```json +{ + "count": 42, + "result": [ + { + "id": 1, + "slice_name": "Sales Dashboard", + "viz_type": "line" + } + ] +} +``` +``` + +--- + +**Note**: This documentation site serves as the primary resource for Superset users, administrators, and contributors. Always prioritize clarity, accuracy, and completeness when creating or updating documentation. diff --git a/docs/developer_portal/api/frontend.md b/docs/developer_portal/api/frontend.md index e2d66ae9f0a..1f75a9f5d13 100644 --- a/docs/developer_portal/api/frontend.md +++ b/docs/developer_portal/api/frontend.md @@ -1,477 +1,772 @@ --- title: Frontend API Reference sidebar_position: 1 -hide_title: true --- -# Frontend API Reference +# Frontend Extension API Reference -The `@apache-superset/core` package provides comprehensive APIs for frontend extension development. All APIs are organized into logical namespaces for easy discovery and use. +The `@apache-superset/core` package provides comprehensive APIs for frontend extensions to interact with Apache Superset. All APIs are versioned and follow semantic versioning principles. -## Core API +## Core APIs -The core namespace provides fundamental extension functionality. +### Extension Context -### registerView - -Registers a new view or panel in the specified contribution point. +Every extension receives a context object during activation that provides access to the extension system. ```typescript -core.registerView( - id: string, - component: React.ComponentType +interface ExtensionContext { + // Unique extension identifier + extensionId: string; + + // Extension metadata + extensionPath: string; + extensionUri: Uri; + + // Storage paths + globalStorageUri: Uri; + workspaceStorageUri: Uri; + + // Subscription management + subscriptions: Disposable[]; + + // State management + globalState: Memento; + workspaceState: Memento; + + // Extension-specific APIs + registerView(viewId: string, component: React.Component): Disposable; + registerCommand(commandId: string, handler: CommandHandler): Disposable; +} +``` + +### Lifecycle Methods + +```typescript +// Required: Called when extension is activated +export function activate(context: ExtensionContext): void | Promise { + console.log('Extension activated'); +} + +// Optional: Called when extension is deactivated +export function deactivate(): void | Promise { + console.log('Extension deactivated'); +} +``` + +## SQL Lab APIs + +The `sqlLab` namespace provides APIs specific to SQL Lab functionality. + +### Query Management + +```typescript +// Get current query editor content +sqlLab.getCurrentQuery(): string | undefined + +// Get active tab information +sqlLab.getCurrentTab(): Tab | undefined + +// Get all open tabs +sqlLab.getTabs(): Tab[] + +// Get available databases +sqlLab.getDatabases(): Database[] + +// Get schemas for a database +sqlLab.getSchemas(databaseId: number): Promise + +// Get tables for a schema +sqlLab.getTables(databaseId: number, schema: string): Promise + +// Insert text at cursor position +sqlLab.insertText(text: string): void + +// Replace entire query +sqlLab.replaceQuery(query: string): void + +// Execute current query +sqlLab.executeQuery(): Promise + +// Stop query execution +sqlLab.stopQuery(queryId: string): Promise +``` + +### Event Subscriptions + +```typescript +// Query execution events +sqlLab.onDidQueryRun( + listener: (event: QueryRunEvent) => void +): Disposable + +sqlLab.onDidQueryComplete( + listener: (event: QueryCompleteEvent) => void +): Disposable + +sqlLab.onDidQueryFail( + listener: (event: QueryFailEvent) => void +): Disposable + +// Editor events +sqlLab.onDidChangeEditorContent( + listener: (content: string) => void +): Disposable + +sqlLab.onDidChangeActiveTab( + listener: (tab: Tab) => void +): Disposable + +// Panel events +sqlLab.onDidOpenPanel( + listener: (panel: Panel) => void +): Disposable + +sqlLab.onDidClosePanel( + listener: (panel: Panel) => void ): Disposable ``` -**Example:** -```typescript -const panel = context.core.registerView('my-extension.panel', () => ( - -)); -``` - -### getActiveView - -Gets the currently active view in a contribution area. +### Types ```typescript -core.getActiveView(area: string): View | undefined +interface Tab { + id: string; + title: string; + query: string; + database: Database; + schema?: string; + isActive: boolean; + queryId?: string; + status?: 'pending' | 'running' | 'success' | 'error'; +} + +interface Database { + id: number; + name: string; + backend: string; + allows_subquery: boolean; + allows_ctas: boolean; + allows_cvas: boolean; +} + +interface QueryResult { + queryId: string; + status: 'success' | 'error'; + data?: any[]; + columns?: Column[]; + error?: string; + startTime: number; + endTime: number; + rows: number; +} ``` ## Commands API -Manages command registration and execution. +Register and execute commands within Superset. -### registerCommand - -Registers a new command that can be triggered by menus, shortcuts, or programmatically. +### Registration ```typescript +interface CommandHandler { + execute(...args: any[]): any | Promise; + isEnabled?(): boolean; + isVisible?(): boolean; +} + +// Register a command commands.registerCommand( - id: string, + commandId: string, handler: CommandHandler ): Disposable -interface CommandHandler { +// Register with metadata +commands.registerCommand( + commandId: string, + metadata: CommandMetadata, + handler: (...args: any[]) => any +): Disposable + +interface CommandMetadata { title: string; + category?: string; icon?: string; - execute: (...args: any[]) => any; - isEnabled?: (...args: any[]) => boolean; + enablement?: string; + when?: string; } ``` -**Example:** +### Execution + ```typescript -const cmd = context.commands.registerCommand('my-extension.analyze', { - title: 'Analyze Query', - icon: 'BarChartOutlined', - execute: () => { - const query = context.sqlLab.getCurrentQuery(); - // Perform analysis - }, - isEnabled: () => { - return context.sqlLab.hasActiveEditor(); - } -}); +// Execute a command +commands.executeCommand( + commandId: string, + ...args: any[] +): Promise + +// Get all registered commands +commands.getCommands(): Promise + +// Check if command exists +commands.hasCommand(commandId: string): boolean ``` -### executeCommand - -Executes a registered command by ID. +### Built-in Commands ```typescript -commands.executeCommand(id: string, ...args: any[]): Promise +// SQL Lab commands +'sqllab.executeQuery' +'sqllab.formatQuery' +'sqllab.saveQuery' +'sqllab.shareQuery' +'sqllab.downloadResults' + +// Editor commands +'editor.action.formatDocument' +'editor.action.commentLine' +'editor.action.findReferences' + +// Extension commands +'extensions.installExtension' +'extensions.uninstallExtension' +'extensions.enableExtension' +'extensions.disableExtension' ``` -## SQL Lab API +## UI Components -Provides access to SQL Lab functionality and events. +Pre-built components from `@apache-superset/core` for consistent UI. -### Query Access +### Basic Components ```typescript -// Get current tab -sqlLab.getCurrentTab(): Tab | undefined - -// Get all tabs -sqlLab.getTabs(): Tab[] - -// Get current query -sqlLab.getCurrentQuery(): string - -// Get selected text -sqlLab.getSelectedText(): string | undefined +import { + Button, + Input, + Select, + Checkbox, + Radio, + Switch, + Slider, + DatePicker, + TimePicker, + Tooltip, + Popover, + Modal, + Drawer, + Alert, + Message, + Notification, + Spin, + Progress +} from '@apache-superset/core'; ``` -### Database Access +### Data Display ```typescript -// Get available databases -sqlLab.getDatabases(): Database[] - -// Get database by ID -sqlLab.getDatabase(id: number): Database | undefined - -// Get schemas for database -sqlLab.getSchemas(databaseId: number): Promise - -// Get tables for schema -sqlLab.getTables( - databaseId: number, - schema: string -): Promise +import { + Table, + List, + Card, + Collapse, + Tabs, + Tag, + Badge, + Statistic, + Timeline, + Tree, + Empty, + Result +} from '@apache-superset/core'; ``` -### Events +### Form Components ```typescript -// Query execution events -sqlLab.onDidQueryRun: Event -sqlLab.onDidQueryStop: Event -sqlLab.onDidQueryFail: Event - -// Editor events -sqlLab.onDidChangeEditorContent: Event -sqlLab.onDidChangeSelection: Event - -// Tab events -sqlLab.onDidChangeActiveTab: Event -sqlLab.onDidCloseTab: Event -sqlLab.onDidChangeTabTitle: Event<{tab: Tab, title: string}> - -// Panel events -sqlLab.onDidOpenPanel: Event -sqlLab.onDidClosePanel: Event -sqlLab.onDidChangeActivePanel: Event -``` - -**Event Usage Example:** -```typescript -const disposable = context.sqlLab.onDidQueryRun((result) => { - console.log('Query executed:', result.query); - console.log('Rows returned:', result.rowCount); - console.log('Execution time:', result.executionTime); -}); - -// Remember to dispose when done -context.subscriptions.push(disposable); +import { + Form, + FormItem, + FormList, + InputNumber, + TextArea, + Upload, + Rate, + Cascader, + AutoComplete, + Mentions +} from '@apache-superset/core'; ``` ## Authentication API -Handles authentication and security tokens. - -### getCSRFToken - -Gets the current CSRF token for API requests. +Access authentication and user information. ```typescript -authentication.getCSRFToken(): Promise -``` - -### getCurrentUser - -Gets information about the current user. - -```typescript -authentication.getCurrentUser(): User +// Get current user +authentication.getCurrentUser(): User | undefined interface User { id: number; username: string; email: string; + firstName: string; + lastName: string; roles: Role[]; - permissions: Permission[]; -} -``` - -### hasPermission - -Checks if the current user has a specific permission. - -```typescript -authentication.hasPermission(permission: string): boolean -``` - -## Extensions API - -Manages extension lifecycle and inter-extension communication. - -### getExtension - -Gets information about an installed extension. - -```typescript -extensions.getExtension(id: string): Extension | undefined - -interface Extension { - id: string; - name: string; - version: string; isActive: boolean; - metadata: ExtensionMetadata; + isAnonymous: boolean; } -``` -### getActiveExtensions +// Get CSRF token for API requests +authentication.getCSRFToken(): Promise -Gets all currently active extensions. +// Check permissions +authentication.hasPermission( + permission: string, + resource?: string +): boolean -```typescript -extensions.getActiveExtensions(): Extension[] -``` +// Get user preferences +authentication.getPreferences(): UserPreferences -### Events - -```typescript -// Extension lifecycle events -extensions.onDidActivateExtension: Event -extensions.onDidDeactivateExtension: Event -``` - -## UI Components - -Import pre-built UI components from `@apache-superset/core`: - -```typescript -import { - Button, - Select, - Input, - Table, - Modal, - Alert, - Tabs, - Card, - Dropdown, - Menu, - Tooltip, - Icon, - // ... many more -} from '@apache-superset/core'; -``` - -### Example Component Usage - -```typescript -import { Button, Alert } from '@apache-superset/core'; - -function MyExtensionPanel() { - return ( -
- - -
- ); -} +// Update preferences +authentication.setPreference( + key: string, + value: any +): Promise ``` ## Storage API -Provides persistent storage for extension data. +Persist data across sessions. -### Local Storage +### Global Storage ```typescript -// Store data -storage.local.set(key: string, value: any): Promise +// Shared across all workspaces +const globalState = context.globalState; -// Retrieve data -storage.local.get(key: string): Promise +// Get value +const value = globalState.get(key: string): T | undefined -// Remove data -storage.local.remove(key: string): Promise +// Set value +await globalState.update(key: string, value: any): Promise -// Clear all extension data -storage.local.clear(): Promise +// Get all keys +globalState.keys(): readonly string[] ``` ### Workspace Storage -Workspace storage is shared across all users for collaborative features. - ```typescript -storage.workspace.set(key: string, value: any): Promise -storage.workspace.get(key: string): Promise -storage.workspace.remove(key: string): Promise +// Specific to current workspace +const workspaceState = context.workspaceState; + +// Same API as globalState +workspaceState.get(key: string): T | undefined +workspaceState.update(key: string, value: any): Promise +workspaceState.keys(): readonly string[] ``` -## Network API - -Utilities for making API calls to Superset. - -### fetch - -Enhanced fetch with CSRF token handling. +### Secrets Storage ```typescript -network.fetch(url: string, options?: RequestInit): Promise +// Secure storage for sensitive data +secrets.store(key: string, value: string): Promise +secrets.get(key: string): Promise +secrets.delete(key: string): Promise ``` -### API Client +## Events API -Type-safe API client for Superset endpoints. +Subscribe to and emit custom events. ```typescript -// Get chart data -network.api.charts.get(id: number): Promise +// Create an event emitter +const onDidChange = new EventEmitter(); -// Query database -network.api.sqlLab.execute( - databaseId: number, - query: string -): Promise +// Expose as event +export const onChange = onDidChange.event; -// Get datasets -network.api.datasets.list(): Promise +// Fire event +onDidChange.fire({ + type: 'update', + data: newData +}); + +// Subscribe to event +const disposable = onChange((event) => { + console.log('Changed:', event); +}); + +// Cleanup +disposable.dispose(); ``` -## Utility Functions +## Window API -### Formatting +Interact with the UI window. + +### Notifications ```typescript -// Format numbers -utils.formatNumber(value: number, format?: string): string +// Show info message +window.showInformationMessage( + message: string, + ...items: string[] +): Promise -// Format dates -utils.formatDate(date: Date, format?: string): string +// Show warning +window.showWarningMessage( + message: string, + ...items: string[] +): Promise -// Format SQL -utils.formatSQL(sql: string): string -``` +// Show error +window.showErrorMessage( + message: string, + ...items: string[] +): Promise -### Validation +// Show with options +window.showInformationMessage( + message: string, + options: MessageOptions, + ...items: MessageItem[] +): Promise -```typescript -// Validate SQL syntax -utils.validateSQL(sql: string): ValidationResult - -// Check if valid database ID -utils.isValidDatabaseId(id: any): boolean -``` - -## TypeScript Types - -Import common types for type safety: - -```typescript -import type { - Database, - Dataset, - Chart, - Dashboard, - Query, - QueryResult, - Tab, - Panel, - User, - Role, - Permission, - ExtensionContext, - Disposable, - Event, - // ... more types -} from '@apache-superset/core'; -``` - -## Extension Context - -The context object passed to your extension's `activate` function: - -```typescript -interface ExtensionContext { - // Subscription management - subscriptions: Disposable[]; - - // Extension metadata - extensionId: string; - extensionPath: string; - - // API namespaces - core: CoreAPI; - commands: CommandsAPI; - sqlLab: SqlLabAPI; - authentication: AuthenticationAPI; - extensions: ExtensionsAPI; - storage: StorageAPI; - network: NetworkAPI; - utils: UtilsAPI; - - // Logging - logger: Logger; +interface MessageOptions { + modal?: boolean; + detail?: string; } ``` -## Event Handling - -Events follow the VS Code pattern with subscribe/dispose: +### Input Dialogs ```typescript -// Subscribe to event -const disposable = sqlLab.onDidQueryRun((result) => { - // Handle event +// Show input box +window.showInputBox( + options?: InputBoxOptions +): Promise + +interface InputBoxOptions { + title?: string; + prompt?: string; + placeHolder?: string; + value?: string; + password?: boolean; + validateInput?(value: string): string | null; +} + +// Show quick pick +window.showQuickPick( + items: string[] | QuickPickItem[], + options?: QuickPickOptions +): Promise + +interface QuickPickOptions { + title?: string; + placeHolder?: string; + canPickMany?: boolean; + matchOnDescription?: boolean; + matchOnDetail?: boolean; +} +``` + +### Progress + +```typescript +// Show progress +window.withProgress( + options: ProgressOptions, + task: (progress: Progress<{message?: string}>) => Promise +): Promise + +interface ProgressOptions { + location: ProgressLocation; + title?: string; + cancellable?: boolean; +} + +// Example usage +await window.withProgress( + { + location: ProgressLocation.Notification, + title: "Processing", + cancellable: true + }, + async (progress) => { + progress.report({ message: 'Step 1...' }); + await step1(); + progress.report({ message: 'Step 2...' }); + await step2(); + } +); +``` + +## Workspace API + +Access workspace information and configuration. + +```typescript +// Get workspace folders +workspace.workspaceFolders: readonly WorkspaceFolder[] + +// Get configuration +workspace.getConfiguration( + section?: string +): WorkspaceConfiguration + +// Update configuration +workspace.getConfiguration('myExtension') + .update('setting', value, ConfigurationTarget.Workspace) + +// Watch for configuration changes +workspace.onDidChangeConfiguration( + listener: (e: ConfigurationChangeEvent) => void +): Disposable + +// File system operations +workspace.fs.readFile(uri: Uri): Promise +workspace.fs.writeFile(uri: Uri, content: Uint8Array): Promise +workspace.fs.delete(uri: Uri): Promise +workspace.fs.rename(oldUri: Uri, newUri: Uri): Promise +workspace.fs.copy(source: Uri, destination: Uri): Promise +workspace.fs.createDirectory(uri: Uri): Promise +workspace.fs.readDirectory(uri: Uri): Promise<[string, FileType][]> +workspace.fs.stat(uri: Uri): Promise +``` + +## HTTP Client API + +Make HTTP requests from extensions. + +```typescript +import { api } from '@apache-superset/core'; + +// GET request +const response = await api.get('/api/v1/chart/'); + +// POST request +const response = await api.post('/api/v1/chart/', { + data: chartData }); -// Dispose when done -disposable.dispose(); +// PUT request +const response = await api.put('/api/v1/chart/123', { + data: updatedData +}); -// Or add to context for automatic cleanup -context.subscriptions.push(disposable); +// DELETE request +const response = await api.delete('/api/v1/chart/123'); + +// Custom headers +const response = await api.get('/api/v1/chart/', { + headers: { + 'X-Custom-Header': 'value' + } +}); + +// Query parameters +const response = await api.get('/api/v1/chart/', { + params: { + page: 1, + page_size: 20 + } +}); +``` + +## Theming API + +Access and customize theme settings. + +```typescript +// Get current theme +theme.getActiveTheme(): Theme + +interface Theme { + name: string; + isDark: boolean; + colors: ThemeColors; + typography: Typography; + spacing: Spacing; +} + +// Listen for theme changes +theme.onDidChangeTheme( + listener: (theme: Theme) => void +): Disposable + +// Get theme colors +const colors = theme.colors; +colors.primary +colors.success +colors.warning +colors.error +colors.info +colors.text +colors.background +colors.border +``` + +## Disposable Pattern + +Manage resource cleanup consistently. + +```typescript +interface Disposable { + dispose(): void; +} + +// Create a disposable +class MyDisposable implements Disposable { + dispose() { + // Cleanup logic + } +} + +// Combine disposables +const composite = Disposable.from( + disposable1, + disposable2, + disposable3 +); + +// Dispose all at once +composite.dispose(); + +// Use in extension +export function activate(context: ExtensionContext) { + // All disposables added here are cleaned up on deactivation + context.subscriptions.push( + registerCommand(...), + registerView(...), + onDidChange(...) + ); +} +``` + +## Type Definitions + +Complete TypeScript definitions are available: + +```typescript +import type { + ExtensionContext, + Disposable, + Event, + EventEmitter, + Uri, + Command, + QuickPickItem, + InputBoxOptions, + Progress, + CancellationToken +} from '@apache-superset/core'; +``` + +## Version Compatibility + +The API follows semantic versioning: + +```typescript +// Check API version +const version = superset.version; + +// Version components +version.major // Breaking changes +version.minor // New features +version.patch // Bug fixes + +// Check minimum version +if (version.major < 1) { + throw new Error('Requires Superset API v1.0.0 or higher'); +} +``` + +## Migration Guide + +### From v0.x to v1.0 + +```typescript +// Before (v0.x) +sqlLab.runQuery(query); + +// After (v1.0) +sqlLab.executeQuery(); + +// Before (v0.x) +core.registerPanel(id, component); + +// After (v1.0) +context.registerView(id, component); ``` ## Best Practices -1. **Always dispose subscriptions** to prevent memory leaks -2. **Use TypeScript** for better IDE support and type safety -3. **Handle errors gracefully** with try-catch blocks -4. **Check permissions** before sensitive operations -5. **Use provided UI components** for consistency -6. **Cache API responses** when appropriate -7. **Validate user input** before processing +### Error Handling -## Version Compatibility - -The frontend API follows semantic versioning: - -- **Major version**: Breaking changes -- **Minor version**: New features, backward compatible -- **Patch version**: Bug fixes - -Check compatibility in your `extension.json`: - -```json -{ - "engines": { - "@apache-superset/core": "^1.0.0" +```typescript +export async function activate(context: ExtensionContext) { + try { + await initializeExtension(); + } catch (error) { + console.error('Failed to initialize:', error); + window.showErrorMessage( + `Extension failed to activate: ${error.message}` + ); } } ``` + +### Resource Management + +```typescript +// Always use disposables +const disposables: Disposable[] = []; + +disposables.push( + commands.registerCommand(...), + sqlLab.onDidQueryRun(...), + workspace.onDidChangeConfiguration(...) +); + +// Cleanup in deactivate +export function deactivate() { + disposables.forEach(d => d.dispose()); +} +``` + +### Type Safety + +```typescript +// Use type guards +function isDatabase(obj: any): obj is Database { + return obj && typeof obj.id === 'number' && typeof obj.name === 'string'; +} + +// Use generics +function getValue(key: string, defaultValue: T): T { + return context.globalState.get(key) ?? defaultValue; +} +``` diff --git a/docs/developer_portal/architecture/overview.md b/docs/developer_portal/architecture/overview.md index 452e8fe1688..3c8ccc1c131 100644 --- a/docs/developer_portal/architecture/overview.md +++ b/docs/developer_portal/architecture/overview.md @@ -1,348 +1,194 @@ --- -title: Architecture Overview +title: Extension Architecture sidebar_position: 1 -hide_title: true --- -# Extension Architecture Overview +# Superset Extension Architecture -The Superset extension architecture is designed to be modular, secure, and performant. This document provides a comprehensive overview of how extensions work and interact with the Superset host application. +Apache Superset's extension architecture enables developers to enhance and customize the platform without modifying the core codebase. Inspired by the successful VS Code Extensions model, this architecture provides well-defined, versioned APIs and clear contribution points that allow the community to build upon and extend Superset's functionality. -## Core Principles +## Core Concepts -### 1. Lean Core -Superset's core remains minimal, with features delegated to extensions wherever possible. Built-in features use the same APIs as external extensions, ensuring API quality through dogfooding. +### Extensions vs Plugins -### 2. Explicit Contribution Points -All extension points are clearly defined and documented. Extensions declare their capabilities in metadata files, enabling predictable lifecycle management. +We use the term "extensions" rather than "plugins" to better convey the idea of enhancing and expanding Superset's core capabilities in a modular and integrated way. Extensions can add new features, modify existing behavior, and integrate deeply with the host application through well-defined APIs. -### 3. Versioned APIs -Public interfaces follow semantic versioning, ensuring backward compatibility and safe evolution of the platform. +### Lean Core Philosophy -### 4. Lazy Loading -Extensions load only when needed, minimizing performance impact and resource consumption. +Superset's core remains minimal, with many features delegated to extensions. Built-in features are implemented using the same APIs available to external extension authors, ensuring consistency and validating the extension architecture through real-world usage. -### 5. Composability -Architecture patterns and APIs are reusable across different Superset modules, promoting consistency. +## Architecture Overview -### 6. Community-Driven -The system evolves based on real-world feedback, with new extension points added as needs emerge. - -## System Architecture - -```mermaid -graph TB - subgraph "Superset Host Application" - Core[Core Application] - API[Extension APIs] - Loader[Extension Loader] - Manager[Extension Manager] - end - - subgraph "Core Packages" - FrontendCore["@apache-superset/core
(Frontend)"] - BackendCore["apache-superset-core
(Backend)"] - CLI["apache-superset-extensions-cli"] - end - - subgraph "Extension" - Metadata[extension.json] - Frontend[Frontend Code] - Backend[Backend Code] - Bundle[.supx Bundle] - end - - Core --> API - API --> FrontendCore - API --> BackendCore - Loader --> Manager - Manager --> Bundle - Frontend --> FrontendCore - Backend --> BackendCore - CLI --> Bundle -``` - -## Key Components - -### Host Application - -The Superset host application provides: - -- **Extension APIs**: Well-defined interfaces for extensions to interact with Superset -- **Extension Manager**: Handles lifecycle, activation, and deactivation -- **Module Loader**: Dynamically loads extension code using Webpack Module Federation -- **Security Context**: Manages permissions and sandboxing for extensions +The extension architecture consists of several key components: ### Core Packages #### @apache-superset/core (Frontend) -- Shared UI components and utilities -- TypeScript type definitions -- Frontend API implementations -- Event system and command registry +Provides essential building blocks for extensions: +- Shared UI components +- Utility functions +- Type definitions +- Frontend APIs for interacting with the host #### apache-superset-core (Backend) -- Python base classes and utilities +Exposes backend functionality: - Database access APIs -- Security and permission helpers -- REST API registration +- Security models +- REST API extensions +- SQLAlchemy models and utilities -#### apache-superset-extensions-cli -- Project scaffolding -- Build and bundling tools -- Development server -- Package management +### Extension CLI -### Extension Structure +The `apache-superset-extensions-cli` package provides commands for: +- Scaffolding new extension projects +- Building and bundling extensions +- Development workflows with hot-reload +- Packaging extensions for distribution -Each extension consists of: +### Host Application -- **Metadata** (`extension.json`): Declares capabilities and requirements -- **Frontend**: React components and TypeScript code -- **Backend**: Python modules and API endpoints -- **Assets**: Styles, images, and other resources -- **Bundle** (`.supx`): Packaged distribution format +Superset acts as the host, providing: +- Extension registration and management +- Dynamic loading of extension assets +- API implementation for extensions +- Lifecycle management (activation/deactivation) -## Module Federation +## Extension Points -Extensions use Webpack Module Federation for dynamic loading: +Extensions can contribute to various parts of Superset: -```javascript -// Extension webpack.config.js -new ModuleFederationPlugin({ - name: 'my_extension', - filename: 'remoteEntry.[contenthash].js', - exposes: { - './index': './src/index.tsx', - }, - externals: { - '@apache-superset/core': 'superset', - }, - shared: { - react: { singleton: true }, - 'react-dom': { singleton: true }, - } -}) -``` +### SQL Lab Extensions +- Custom panels (left, right, bottom) +- Editor enhancements +- Query processors +- Autocomplete providers +- Execution plan visualizers -This allows: -- **Independent builds**: Extensions compile separately from Superset -- **Shared dependencies**: Common libraries like React aren't duplicated -- **Dynamic loading**: Extensions load at runtime without rebuilding Superset -- **Version compatibility**: Extensions declare compatible core versions +### Dashboard Extensions (Future) +- Custom widget types +- Filter components +- Interaction handlers -## Extension Lifecycle +### Chart Extensions (Future) +- New visualization types +- Data transformers +- Export formats + +## Technical Foundation + +### Module Federation + +Frontend extensions leverage Webpack Module Federation for dynamic loading: +- Extensions are built independently +- Dependencies are shared with the host +- No rebuild of Superset required +- Runtime loading of extension assets + +### API Versioning + +All public APIs follow semantic versioning: +- Breaking changes require major version bumps +- Extensions declare compatibility requirements +- Backward compatibility maintained within major versions + +### Security Model + +- Extensions disabled by default (require `ENABLE_EXTENSIONS` flag) +- Built-in extensions follow same security standards as core +- External extensions run in same context as host (sandboxing planned) +- Administrators responsible for vetting third-party extensions + +## Development Workflow + +1. **Initialize**: Use CLI to scaffold new extension +2. **Develop**: Work with hot-reload in development mode +3. **Build**: Bundle frontend and backend assets +4. **Package**: Create `.supx` distribution file +5. **Deploy**: Upload through API or management UI + +## Example: Dataset References Extension + +A practical example demonstrating the architecture: -### 1. Registration ```typescript -// Extension registered with host -extensionManager.register({ - name: 'my-extension', - version: '1.0.0', - manifest: manifestData -}); -``` - -### 2. Activation -```typescript -// activate() called when extension loads -export function activate(context: ExtensionContext) { - // Register contributions - const disposables = []; - - // Add panel - disposables.push( - context.core.registerView('my-panel', MyPanel) +// Frontend activation +export function activate(context) { + // Register a new SQL Lab panel + const panel = core.registerView('dataset_references.main', + ); - // Register command - disposables.push( - context.commands.registerCommand('my-command', { - execute: () => { /* ... */ } - }) - ); + // Listen to query changes + const listener = sqlLab.onDidQueryRun(editor => { + // Analyze query and update panel + }); - // Store for cleanup - context.subscriptions.push(...disposables); + // Cleanup on deactivation + context.subscriptions.push(panel, listener); } ``` -### 3. Runtime -- Extension responds to events -- Provides UI components when requested -- Executes commands when triggered -- Accesses APIs as needed - -### 4. Deactivation -```typescript -// Automatic cleanup of registered items -export function deactivate() { - // context.subscriptions automatically disposed - // Additional cleanup if needed -} -``` - -## Contribution Types - -### Views -Extensions can add panels and UI components: - -```json -{ - "views": { - "sqllab.panels": [{ - "id": "my-panel", - "name": "My Panel", - "icon": "ToolOutlined" - }] - } -} -``` - -### Commands -Define executable actions: - -```json -{ - "commands": [{ - "command": "my-extension.run", - "title": "Run Analysis", - "icon": "PlayCircleOutlined" - }] -} -``` - -### Menus -Add items to existing menus: - -```json -{ - "menus": { - "sqllab.editor": { - "primary": [{ - "command": "my-extension.run", - "when": "editorHasSelection" - }] - } - } -} -``` - -### API Endpoints -Register backend REST endpoints: - ```python +# Backend API extension from superset_core.api import rest_api +from .api import DatasetReferencesAPI -@rest_api.route('/my-endpoint') -def my_endpoint(): - return {'data': 'value'} +# Register custom REST endpoints +rest_api.add_extension_api(DatasetReferencesAPI) ``` -## Security Model - -### Permissions -- Extensions run with user's permissions -- No elevation of privileges -- Access controlled by Superset's RBAC - -### Sandboxing -- Frontend code runs in browser context -- Backend code runs in Python process -- Future: Optional sandboxed execution - -### Validation -- Manifest validation on upload -- Signature verification (future) -- Dependency scanning - -## Performance Considerations - -### Lazy Loading -- Extensions load only when features are accessed -- Code splitting for large extensions -- Cached after first load - -### Bundle Optimization -- Tree shaking removes unused code -- Minification reduces size -- Compression for network transfer - -### Resource Management -- Automatic cleanup on deactivation -- Memory leak prevention -- Event listener management - -## Development vs Production - -### Development Mode -```python -# superset_config.py -ENABLE_EXTENSIONS = True -LOCAL_EXTENSIONS = ['/path/to/extension'] -``` -- Hot reloading -- Source maps -- Debug logging - -### Production Mode -- Optimized bundles -- Cached assets -- Performance monitoring - -## Future Enhancements - -### Planned Features -- Enhanced sandboxing -- Extension marketplace -- Inter-extension communication -- Theme contributions -- Chart type extensions - -### API Expansion -- Dashboard extensions -- Database connector API -- Security provider interface -- Workflow automation - ## Best Practices -### Do's -- ✅ Use TypeScript for type safety -- ✅ Follow semantic versioning -- ✅ Handle errors gracefully -- ✅ Clean up resources properly -- ✅ Document your extension +### Extension Design +- Keep extensions focused on specific functionality +- Use versioned APIs for stability +- Handle cleanup properly on deactivation +- Follow Superset's coding standards -### Don'ts -- ❌ Access private APIs -- ❌ Modify global state directly -- ❌ Block the main thread -- ❌ Store sensitive data insecurely -- ❌ Assume API stability in 0.x versions +### Performance +- Lazy load assets when possible +- Minimize bundle sizes +- Share dependencies with host +- Cache expensive operations -## Learn More +### Compatibility +- Declare API version requirements +- Test across Superset versions +- Provide migration guides for breaking changes +- Document compatibility clearly -- [API Reference](../api/frontend) -- [Development Guide](../getting-started) -- [Security Guidelines](./security) -- [Performance Optimization](./performance) +## Future Roadmap + +Planned enhancements include: +- JavaScript sandboxing for untrusted extensions +- Extension marketplace and registry +- Inter-extension communication +- Advanced theming capabilities +- Backend hot-reload without restart + +## Getting Started + +Ready to build your first extension? Check out: +- [Extension Project Structure](/developer_portal/extensions/extension-project-structure) +- [API Reference](/developer_portal/api/frontend) +- [CLI Documentation](/developer_portal/cli/overview) +- [Frontend Contribution Types](/developer_portal/extensions/frontend-contribution-types) diff --git a/docs/developer_portal/capabilities/common-capabilities.md b/docs/developer_portal/capabilities/common-capabilities.md new file mode 100644 index 00000000000..dc1adf66c6f --- /dev/null +++ b/docs/developer_portal/capabilities/common-capabilities.md @@ -0,0 +1,55 @@ +--- +title: Common Plugin Capabilities +sidebar_position: 2 +--- + + + +# Common Plugin Capabilities + +🚧 **Coming Soon** 🚧 + +Explore the shared functionality and common patterns available to all Superset plugins. + +## Topics to be covered: + +- Plugin lifecycle hooks (initialization, activation, deactivation) +- Accessing Superset's core services and APIs +- State management and data persistence +- Event handling and plugin communication +- Internationalization (i18n) support +- Error handling and logging +- Plugin configuration management +- Accessing user context and permissions +- Working with datasets and queries +- Plugin metadata and manifests + +## Core Services Available + +- **API Client** - HTTP client for backend communication +- **State Store** - Redux store access +- **Theme Provider** - Access to current theme settings +- **User Context** - Current user information and permissions +- **Dataset Service** - Working with data sources +- **Chart Service** - Chart rendering utilities + +--- + +*This documentation is under active development. Check back soon for updates!* diff --git a/docs/developer_portal/capabilities/extending-workbench.md b/docs/developer_portal/capabilities/extending-workbench.md new file mode 100644 index 00000000000..07cead0abf5 --- /dev/null +++ b/docs/developer_portal/capabilities/extending-workbench.md @@ -0,0 +1,62 @@ +--- +title: Extending the Workbench +sidebar_position: 4 +--- + + + +# Extending the Workbench + +🚧 **Coming Soon** 🚧 + +Discover how to extend Superset's main interface and workbench with custom components and functionality. + +## Topics to be covered: + +- Adding custom menu items and navigation +- Creating custom dashboard components +- Extending the SQL Lab interface +- Adding custom sidebar panels +- Creating floating panels and modals +- Integrating with the command palette +- Custom toolbar buttons and actions +- Workspace state management +- Plugin-specific keyboard shortcuts +- Context menu extensions + +## Extension Points + +- **Main navigation** - Top-level menu items +- **Dashboard builder** - Custom components and layouts +- **SQL Lab** - Query editor extensions +- **Chart explorer** - Visualization building tools +- **Settings panels** - Configuration interfaces +- **Data source explorer** - Database navigation + +## UI Integration Patterns + +- React component composition +- Portal-based rendering +- Event-driven UI updates +- Responsive layout adaptation + +--- + +*This documentation is under active development. Check back soon for updates!* diff --git a/docs/developer_portal/capabilities/overview.md b/docs/developer_portal/capabilities/overview.md new file mode 100644 index 00000000000..22fe81e1e00 --- /dev/null +++ b/docs/developer_portal/capabilities/overview.md @@ -0,0 +1,53 @@ +--- +title: Plugin Capabilities Overview +sidebar_position: 1 +--- + + + +# Plugin Capabilities Overview + +🚧 **Coming Soon** 🚧 + +This section provides a comprehensive overview of what Superset plugins can do and how they integrate with the core platform. + +## Topics to be covered: + +- Plugin architecture and lifecycle +- Available extension points +- Core APIs and services +- Plugin communication patterns +- Configuration and settings management +- Plugin permissions and security +- Performance considerations +- Best practices for plugin development + +## Plugin Types + +Superset supports several types of plugins: +- **Visualization plugins** - Custom chart types and data visualizations +- **Database connectors** - New data source integrations +- **UI extensions** - Custom dashboard components and interfaces +- **Theme plugins** - Custom styling and branding +- **Filter plugins** - Custom filter components + +--- + +*This documentation is under active development. Check back soon for updates!* diff --git a/docs/developer_portal/capabilities/theming.md b/docs/developer_portal/capabilities/theming.md new file mode 100644 index 00000000000..1ad673f5671 --- /dev/null +++ b/docs/developer_portal/capabilities/theming.md @@ -0,0 +1,61 @@ +--- +title: Theming and Styling +sidebar_position: 3 +--- + + + +# Theming and Styling + +🚧 **Coming Soon** 🚧 + +Learn how to create custom themes and style your plugins to match Superset's design system. + +## Topics to be covered: + +- Understanding Superset's theme architecture +- Using the theme provider in plugins +- Creating custom color palettes +- Responsive design considerations +- Dark mode and light mode support +- Customizing chart colors and styling +- Brand customization and white-labeling +- CSS-in-JS best practices +- Working with Ant Design components +- Accessibility in custom themes + +## Theme Structure + +- **Color tokens** - Primary, secondary, and semantic colors +- **Typography** - Font families, sizes, and weights +- **Spacing** - Grid system and layout tokens +- **Component styles** - Default component appearances +- **Chart themes** - Color schemes for visualizations + +## Supported Theming APIs + +- Theme provider context +- CSS custom properties +- Emotion/styled-components integration +- Chart color palette API + +--- + +*This documentation is under active development. Check back soon for updates!* diff --git a/docs/developer_portal/cli/overview.md b/docs/developer_portal/cli/overview.md index 6aa43dfcf4a..5e92a6e4c8b 100644 --- a/docs/developer_portal/cli/overview.md +++ b/docs/developer_portal/cli/overview.md @@ -1,466 +1,578 @@ - --- -title: CLI Documentation +title: Extension CLI sidebar_position: 1 -hide_title: true --- -# Superset Extensions CLI + + +# Superset Extension CLI + +The `apache-superset-extensions-cli` package provides command-line tools for creating, developing, and packaging Apache Superset extensions. It streamlines the entire extension development workflow from initialization to deployment. ## Installation +Install the CLI globally using pip: + ```bash pip install apache-superset-extensions-cli ``` -## Commands +Or install locally in your project: + +```bash +pip install --user apache-superset-extensions-cli +``` + +Verify installation: + +```bash +superset-extensions --version +# Output: apache-superset-extensions-cli version 1.0.0 +``` + +## Commands Overview + +| Command | Description | +|---------|-------------| +| `init` | Create a new extension project | +| `dev` | Start development mode with hot reload | +| `build` | Build extension assets for production | +| `bundle` | Package extension into a .supx file | +| `validate` | Validate extension metadata and structure | +| `publish` | Publish extension to registry (future) | + +## Command Reference ### init -Creates a new extension project with the standard folder structure. +Creates a new extension project with the standard structure and boilerplate code. ```bash -superset-extensions init [options] +superset-extensions init [options] ``` -**Options:** -- `--template