QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-07-26 16:42:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(extensions): add a Storage API for extensions (#39171)

Browse Source 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Amin Ghadersohi <amin.ghadersohi@gmail.com>
Co-authored-by: Benyovszky Máté <benyovszky.mate.work@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Michael S. Molina
2026-07-21 16:23:56 -03:00
committed by GitHub
parent 0dd06f3b7b
commit 34ebe3d22a
72 changed files with 9590 additions and 138 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/developer_docs/extensions/architecture.md
View File

@@ -163,10 +163,6 @@ The architecture leverages Webpack's Module Federation to enable dynamic loading
Extensions configure Webpack to expose their entry points:
```javascript
externalsType: 'window',
externals: {
'@apache-superset/core': 'superset',
},
plugins: [
new ModuleFederationPlugin({
name: 'my_extension',
@@ -178,6 +174,7 @@ plugins: [
react: { singleton: true, import: false },
'react-dom': { singleton: true, import: false },
antd: { singleton: true, import: false },
'@apache-superset/core': { singleton: true, import: false },
},
}),
]
@@ -187,9 +184,7 @@ This configuration does several important things:
**`exposes`** - Declares which modules are available to the host application. Superset always loads extensions by requesting the `./index` module from the remote container — this is a fixed convention, not a configurable value. Extensions must expose exactly `'./index': './src/index.tsx'` and place all API registrations (views, commands, menus, editors, event listeners) in that file. The module is executed as a side effect when the extension loads, so any call to `views.registerView`, `commands.registerCommand`, etc. made at the top level of `index.tsx` will run automatically.
**`externals` and `externalsType`** - Tell Webpack that when the extension imports `@apache-superset/core`, it should use `window.superset` at runtime instead of bundling its own copy. This ensures extensions use the host's implementation of shared packages.
**`shared`** - Prevents duplication of common libraries like React and Ant Design. The `singleton: true` setting ensures only one instance of each library exists, avoiding version conflicts and reducing bundle size.
**`shared`** - Prevents duplication of common libraries like React and Ant Design, and, for `@apache-superset/core`, is the mechanism that gives each extension an isolated context (see below). The `singleton: true` setting ensures only one *logical* instance of each library exists — for `react`/`react-dom`/`antd` that means the host's actual instance is reused; for `@apache-superset/core` it means the extension's container defers to whatever module the host's loader supplies for it at init time, which is not the same object for every extension (see [Runtime Resolution](#runtime-resolution)).
### Runtime Resolution
@@ -200,25 +195,30 @@ The following diagram illustrates the module loading process:
Here's what happens at runtime:
1. **Extension Registration**: When an extension is registered, Superset stores its remote entry URL
2. **Dynamic Loading**: When the extension is activated, the host fetches the remote entry file
3. **Module Resolution**: The extension imports `@apache-superset/core`, which resolves to `window.superset`
4. **Execution**: The extension code runs with access to the host's APIs and shared dependencies
2. **Dynamic Loading**: When the extension is activated, the host fetches the remote entry file, then initializes Module Federation sharing (`__webpack_init_sharing__`) and looks up the extension's container on `window`
3. **Per-Extension Scope Injection**: Before calling `container.init()`, the loader builds a **per-container copy** of the webpack share scope in which the `@apache-superset/core` entry is replaced with a synthetic module — a copy of the host's own `window.superset` implementations with `extensions.getContext` bound to that one extension's isolated context object. Because Module Federation caches the resolved module per container, every import of `@apache-superset/core` inside that extension resolves to this pre-bound copy for the lifetime of the container, and because each container gets its own independent scope object, extensions loading in parallel cannot see each other's context.
4. **Execution**: The extension code runs with access to the host's APIs (via its per-extension `@apache-superset/core` instance) and shared dependencies (`react`, `react-dom`, `antd`)
### Host API Setup
On the Superset side, the APIs are mapped to `window.superset` during application bootstrap:
On the Superset side, the real implementations backing `@apache-superset/core` (`commands`, `views`, `menus`, `extensions`, etc.) are assigned onto `window.superset` during application bootstrap, once the current user is known:
```typescript
// eslint-disable-next-line no-restricted-syntax
import * as supersetCore from '@apache-superset/core';
import { commands, views, menus, extensions /* ... */ } from 'src/core';
export default function setupExtensionsAPI() {
window.superset = {
...supersetCore,
};
}
window.superset = {
...supersetCore,
commands,
views,
menus,
extensions,
// ...
};
```
This function runs before any extensions are loaded, ensuring the APIs are available when extensions import from `@apache-superset/core`.
This runs before any extensions are loaded. `window.superset` is not itself what an extension's `@apache-superset/core` import resolves to — it is the source object the loader reads from when building each extension's per-container scoped instance (see [Runtime Resolution](#runtime-resolution)), so every extension ends up sharing the same underlying `commands`/`views`/`menus` singletons but with its own `extensions.getContext()`.
### Benefits

21
docs/developer_docs/extensions/overview.md
View File

@@ -47,13 +47,14 @@ Extension developers have access to pre-built UI components via `@apache-superse
## Next Steps
- **[Quick Start](./quick-start.md)** - Build your first extension with a complete walkthrough
- **[Architecture](./architecture.md)** - Design principles and system overview
- **[Dependencies](./dependencies.md)** - Managing dependencies and understanding API stability
- **[Contribution Types](./contribution-types.md)** - Available extension points
- **[Development](./development.md)** - Project structure, APIs, and development workflow
- **[Deployment](./deployment.md)** - Packaging and deploying extensions
- **[MCP Integration](./mcp.md)** - Adding AI agent capabilities using extensions
- **[Security](./security.md)** - Security considerations and best practices
- **[Tasks](./tasks.md)** - Framework for creating and managing long running tasks
- **[Community Extensions](./registry.md)** - Browse extensions shared by the community
- **[Quick Start](./quick-start)** - Build your first extension with a complete walkthrough
- **[Architecture](./architecture)** - Design principles and system overview
- **[Dependencies](./dependencies)** - Managing dependencies and understanding API stability
- **[Contribution Types](./contribution-types)** - Available extension points
- **[Development](./development)** - Project structure, APIs, and development workflow
- **[Deployment](./deployment)** - Packaging and deploying extensions
- **[MCP Integration](./mcp)** - Adding AI agent capabilities using extensions
- **[Security](./security)** - Security considerations and best practices
- **[Storage](./storage)** - Managed storage API for persisting extension data
- **[Tasks](./tasks)** - Framework for creating and managing long running tasks
- **[Community Extensions](./registry)** - Browse extensions shared by the community

13
docs/developer_docs/extensions/quick-start.md
View File

@@ -223,7 +223,7 @@ The `@apache-superset/core` package must be listed in both `peerDependencies` (t
**`frontend/webpack.config.js`**
The webpack configuration requires specific settings for Module Federation. Key settings include `externalsType: "window"` and `externals` to map `@apache-superset/core` to `window.superset` at runtime, `import: false` for shared modules to use the host's React instead of bundling a separate copy, and `remoteEntry.[contenthash].js` for cache busting.
The webpack configuration requires specific settings for Module Federation. `@apache-superset/core` is declared as a `shared` singleton alongside `react`/`react-dom`/`antd` — Superset injects a per-extension instance of the package into the container's share scope at load time, so the extension's own webpack config only needs to mark it `shared` with `singleton: true`, not resolve it itself. `import: false` on each shared entry means the extension doesn't bundle its own copy and instead uses the instance the host (or, for `@apache-superset/core`, the loader) provides. `remoteEntry.[contenthash].js` gives the built remote entry file a content hash for cache busting.
**Convention**: Superset always loads extensions by requesting the `./index` module from the Module Federation container. The `exposes` entry must be exactly `'./index': './src/index.tsx'` — do not rename or add additional entries. All API registrations must be reachable from that file. See [Architecture](./architecture.md#module-federation) for a full explanation.
@@ -255,11 +255,6 @@ module.exports = (env, argv) => {
resolve: {
extensions: ['.ts', '.tsx', '.js', '.jsx'],
},
// Map @apache-superset/core imports to window.superset at runtime
externalsType: 'window',
externals: {
'@apache-superset/core': 'superset',
},
module: {
rules: [
{
@@ -292,6 +287,10 @@ module.exports = (env, argv) => {
requiredVersion: packageConfig.peerDependencies['antd'],
import: false,
},
'@apache-superset/core': {
singleton: true,
import: false, // Resolved to a per-extension instance by the loader
},
},
}),
],
@@ -498,7 +497,7 @@ Here's what happens when your extension loads:
1. **Superset starts**: Reads `manifest.json` from the `.supx` bundle and loads the backend entrypoint
2. **Backend registration**: `entrypoint.py` imports your API class, triggering the [`@api`](https://github.com/apache/superset/blob/master/superset-core/src/superset_core/rest_api/decorators.py) decorator to register it automatically
3. **Frontend loads**: When SQL Lab opens, Superset fetches the remote entry file
4. **Module Federation**: Webpack loads your extension module and resolves `@apache-superset/core` to `window.superset`
4. **Module Federation**: Webpack loads your extension module; the extension's `@apache-superset/core` import resolves to a per-extension instance injected into its container's share scope by the loader
5. **Registration**: The module executes at load time, calling `views.registerView` to register your panel
6. **Rendering**: When the user opens your panel, React renders `<HelloWorldPanel />`
7. **API call**: The component fetches data from `/extensions/my-org/hello-world/message`

465
docs/developer_docs/extensions/storage.md Normal file
View File

@@ -0,0 +1,465 @@
---
title: Storage
sidebar_position: 8
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Storage
Superset Extensions have access to a managed storage API for persisting data. The storage system provides multiple tiers with different persistence characteristics, allowing extensions to choose the right storage for their needs.
Each extension receives its own isolated storage namespace. When Superset loads your extension, it binds storage to your extension's unique identifier, ensuring data privacy—two extensions using the same key will never collide, and extensions cannot access each other's data.
Storage is always accessed through the extension context via `getContext()`. This binding to the context is what ties every operation to the current extension's namespace.
## Storage Tiers
| Tier | Storage Type | Context Property | Use Case |
| ---- | ----------------- | ------------------------------------------ | -------------------------------------- |
| 1 | Browser storage | `ctx.storage.local`, `ctx.storage.session` | UI state, wizard progress, draft forms |
| 2 | Server-side cache | `ctx.storage.ephemeral` | Job progress, temporary results |
| 3 | Database | `ctx.storage.persistent` | Saved artifacts, durable config |
## Tier 1: Local State
Browser-based storage that persists on the user's device. Use this for UI state and settings that don't need to sync across devices.
### Why Use the API Instead of localStorage Directly?
You might wonder why extensions should use `ctx.storage.local` instead of directly accessing `window.localStorage`. The managed API provides several benefits:
- **Automatic namespacing**: Each extension's data is isolated. Two extensions using the same key name won't collide.
- **User isolation**: By default, data is scoped to the current user, preventing data leakage between users on shared devices.
- **Clean uninstall**: When an extension is uninstalled, all its data can be cleanly removed using prefix-based deletion.
- **Future sandboxing**: The async API is designed for a future sandboxed execution model where extensions run in isolated contexts without direct DOM access.
- **Consistent patterns**: The same API shape works across all storage tiers, making it easy to switch between them.
### localState
Data persists across browser sessions until explicitly deleted or the user clears browser storage.
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
// Save sidebar state
await ctx.storage.local.set('sidebar_collapsed', true);
// Retrieve it later
const isCollapsed = await ctx.storage.local.get('sidebar_collapsed');
// Remove it
await ctx.storage.local.remove('sidebar_collapsed');
```
### sessionState
Data is cleared when the browser tab is closed. Use for transient state within a single session.
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
// Save wizard progress (lost when tab closes)
await ctx.storage.session.set('wizard_step', 3);
await ctx.storage.session.set('unsaved_form', { name: 'Draft' });
// Retrieve on page reload within same tab
const step = await ctx.storage.session.get('wizard_step');
```
Tier 1 has no `shared` accessor. Browser storage is per-device, so a "shared" value would only be visible to other users of that same browser, not to other users of the extension generally — see `.shared` on [Tier 2](#tier-2-ephemeral-state) or [Tier 3](#tier-3-persistent-state) for storage that's actually shared across users.
### When to Use Tier 1
- UI state (sidebar collapsed, panel sizes)
- Recently used items
- Draft form values
- Any data acceptable to lose if user clears browser
### Limitations
- Per-browser, per-device (not shared across devices)
- Subject to browser storage quotas (~5-10 MB)
- Not accessible from backend code
## Tier 2: Ephemeral State
Server-side cache storage with automatic TTL expiration. Use for temporary data that needs to be shared between frontend and backend, or persist across page reloads.
TTL is required for every `set` call. The maximum allowed TTL is controlled by `MAX_TTL` in the server configuration.
### Frontend Usage
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
// Store with a 5-minute TTL
await ctx.storage.ephemeral.set('job_progress', { pct: 42, status: 'running' }, { ttl: 300 });
// Retrieve
const progress = await ctx.storage.ephemeral.get('job_progress');
// Remove
await ctx.storage.ephemeral.remove('job_progress');
```
### Backend Usage
```python
from superset_core.extensions.context import get_context
from superset_core.extensions.storage.ephemeral import EphemeralSetOptions
ctx = get_context()
# Store job progress with a 5-minute TTL
ctx.storage.ephemeral.set(
'job_progress', {'pct': 42, 'status': 'running'}, EphemeralSetOptions(ttl=300)
)
# Retrieve
progress = ctx.storage.ephemeral.get('job_progress')
# Remove
ctx.storage.ephemeral.remove('job_progress')
```
### Shared State
For data that needs to be visible to all users:
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
await ctx.storage.ephemeral.shared.set('shared_result', { data: [1, 2, 3] }, { ttl: 3600 });
const result = await ctx.storage.ephemeral.shared.get('shared_result');
```
```python
from superset_core.extensions.context import get_context
from superset_core.extensions.storage.ephemeral import EphemeralSetOptions
ctx = get_context()
ctx.storage.ephemeral.shared.set(
'shared_result', {'data': [1, 2, 3]}, EphemeralSetOptions(ttl=3600)
)
result = ctx.storage.ephemeral.shared.get('shared_result')
```
### When to Use Tier 2
- Background job progress indicators
- Cross-request intermediate state
- Query result previews
- Temporary computation results
- Any data that can be recomputed if lost
### Limitations
- Not guaranteed to survive server restarts
- Subject to cache eviction under memory pressure
- TTL-based expiration (data disappears after timeout)
- TTL is required on every `set` call
## Tier 3: Persistent State
Database-backed storage that survives server restarts, cache evictions, and browser clears. Use for any data that must not be lost.
### Frontend Usage
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
// Store a saved SQL snippet
await ctx.storage.persistent.set('snippet:top_customers', { sql: 'SELECT ...' });
// Retrieve
const snippet = await ctx.storage.persistent.get('snippet:top_customers');
// Remove
await ctx.storage.persistent.remove('snippet:top_customers');
```
### Backend Usage
```python
from superset_core.extensions.context import get_context
ctx = get_context()
# Store a saved SQL snippet
ctx.storage.persistent.set('snippet:top_customers', {'sql': 'SELECT ...'})
# Retrieve
snippet = ctx.storage.persistent.get('snippet:top_customers')
# Remove
ctx.storage.persistent.remove('snippet:top_customers')
```
### Shared State
For data that should be visible to all users of the extension:
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
await ctx.storage.persistent.shared.set('global_config', { version: 2 });
const config = await ctx.storage.persistent.shared.get('global_config');
```
```python
from superset_core.extensions.context import get_context
ctx = get_context()
ctx.storage.persistent.shared.set('global_config', {'version': 2})
config = ctx.storage.persistent.shared.get('global_config')
```
### Listing Entries
Both the frontend and backend accessors support listing entries in the caller's scope, without needing to know every key in advance. `page` and `pageSize`/`page_size` are required on every call — `list()` always returns one page of results, never the whole result set, so there's no default that could make that fact easy to miss. Check the returned `count` (total entries matching the scope/filters, across all pages) against `pageSize`/`page_size` to know whether more pages exist.
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
// Filter by resource
const result = await ctx.storage.persistent.list({
resourceType: 'dashboard',
resourceUuid: '1234-uuid',
page: 0,
pageSize: 25,
});
result.entries.forEach(entry => {
console.log(entry.key, entry.value, entry.codec);
});
console.log(result.count); // total matching entries across all pages
// Shared (global) scope
const shared = await ctx.storage.persistent.shared.list({ page: 0, pageSize: 10 });
```
```python
from superset_core.extensions.context import get_context
from superset_core.extensions.storage.persistent import PersistentListOptions
ctx = get_context()
# Filter by resource
result = ctx.storage.persistent.list(
PersistentListOptions(
resource_type="dashboard",
resource_uuid="1234-uuid",
page=0,
page_size=25,
)
)
for entry in result.entries:
print(entry.key, entry.value, entry.codec)
print(result.count) # total matching entries across all pages
# Shared (global) scope
shared = ctx.storage.persistent.shared.list(PersistentListOptions(page=0, page_size=10))
```
`resource_type`/`resource_uuid` filter to entries linked to one resource — since a single resource can have several keyed entries (e.g. `"layout"`, `"notes"`), this is how an extension enumerates all of them without tracking key names itself.
There is no fixed limit on `page_size`, but the combined size of a requested page's values is checked against `MAX_LIST_PAYLOAD_SIZE` (see [Quotas](#quotas)) before any row's value is read from the database — a page that would exceed it is rejected rather than silently truncated, so reduce `page_size` and retry if that happens.
Backend `list()` decodes every entry's value unconditionally. The REST API backing the frontend's `list()` call is more restrictive: an entry written with a codec that isn't safe to decode over the wire (e.g. `pickle`) comes back with `value: null` (its `codec` is still reported) rather than being decoded.
### Enumerating and Managing Storage from the Backend
`ctx.storage.persistent.list()` covers the common case of listing an extension's own entries. Backend extension code that needs lower-level bulk operations — for example, a cleanup routine that removes storage linked to resources that no longer exist, using filters `list()` doesn't expose — can use `superset_core.extensions.storage.dao.ExtensionStorageDAO` instead:
```python
from superset_core.extensions.storage.dao import ExtensionStorageDAO
# All entries this extension has linked to a given resource type
entries = ExtensionStorageDAO.filter_by(resource_type="my-resource-type")
# Bulk-delete rows for resources that no longer exist
orphaned = [e for e in entries if e.resource_uuid not in my_extension_live_resource_ids()]
ExtensionStorageDAO.delete(orphaned)
```
`ExtensionStorageDAO` is automatically scoped to the calling extension's own rows — `extension_id` is never a parameter you pass, so there is no way to reach another extension's storage through this API. It supports the standard DAO operations `find_all`, `find_one_or_none`, `filter_by`, `query`, `update`, and `delete`. `create()` is not supported — it's a raw insert with no upsert dedup, quota check, or locking against the key `.set()` writes to, so use `ctx.storage.persistent.set()` to write a value instead.
### When to Use Tier 3
- Extension configuration that must survive restarts
- User-specific saved artifacts that need to roam across devices and browsers (e.g. saved SQL snippets, annotations)
- Any data where loss is unacceptable
### Limitations
- Higher latency than Tiers 1–2 (database round-trip per operation)
- Subject to the per-value size limit configured via `EXTENSIONS_PERSISTENT_STORAGE.MAX_VALUE_SIZE` (see [Quotas](#quotas))
- Subject to the per-extension quota configured via `EXTENSIONS_PERSISTENT_STORAGE.QUOTA_PER_EXTENSION` (see [Quotas](#quotas))
- Requires a database migration when first deployed
## Working with Binary Data
Tiers 2 and 3 accept a `codec` option describing how `value` is encoded — `"json"` (the default) for JSON-native values, or `"binary"` for raw bytes. `list()` and `get()` report which codec an entry was written with via `entry.codec`.
### Backend
Backend code talks to the storage DAO directly and works with real Python `bytes` — no extra flag is needed:
```python
from superset_core.extensions.context import get_context
from superset_core.extensions.storage.persistent import PersistentSetOptions
ctx = get_context()
with open('logo.png', 'rb') as f:
png_bytes = f.read()
ctx.storage.persistent.set('logo', png_bytes, PersistentSetOptions(codec='binary'))
stored = ctx.storage.persistent.get('logo') # raw bytes back, unchanged
```
### Frontend
The frontend SDK talks to the backend over a JSON REST API, which has no byte type — a binary value must be base64-encoded to travel in a JSON request body. There is no way for the SDK to infer this from a JS value's type, so it is never detected automatically: set `isBinary: true` explicitly whenever `value` is binary.
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
const pngBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47 /* ... */]);
await ctx.storage.persistent.set('logo', pngBytes, { isBinary: true });
```
`get()` returns a binary entry's value exactly as stored on the wire — a base64 string, not a `Uint8Array` — so decode it yourself:
```typescript
const base64 = await ctx.storage.persistent.get<string>('logo');
const bytes = Uint8Array.from(atob(base64), c => c.charCodeAt(0));
```
`list()` entries report `isBinary` alongside `codec`, so you know which entries need decoding before you read `value`:
```typescript
const result = await ctx.storage.persistent.list({ page: 0, pageSize: 10 });
result.entries.forEach(entry => {
const value = entry.isBinary
? Uint8Array.from(atob(entry.value as string), c => c.charCodeAt(0))
: entry.value;
});
```
## Key Patterns
All storage keys are automatically namespaced:
| Scope | Key Pattern |
| ---------------------- | -------------------------------------------------- |
| User-scoped | `superset-ext:{extension_id}:user:{user_id}:{key}` |
| Shared (Tiers 2 and 3) | `superset-ext:{extension_id}:shared:{key}` |
This ensures:
- Extensions cannot accidentally access each other's data
- Users cannot see other users' data (by default)
- Clean prefix-based deletion on uninstall
## Configuration
### Tier 2: Ephemeral Storage
Administrators can configure the server-side cache backend in `superset_config.py`:
```python
EXTENSIONS_EPHEMERAL_STORAGE = {
# Use Redis for better performance in production
"CACHE_TYPE": "RedisCache",
"CACHE_REDIS_URL": "redis://localhost:6379/2",
"MAX_TTL": 604800, # 7 days maximum TTL
"MAX_VALUE_SIZE": 100 * 1024, # 100 KB maximum per stored value
}
```
For development, the default `SupersetMetastoreCache` stores data in the metadata database.
### Tier 3: Persistent Storage
Tier 3 values are stored in the `extension_storage` database table. Values are stored unencrypted by default; encryption is opt-in per write, available via a `PersistentSetOptions`-shaped `encrypt` option:
```typescript
import { extensions } from '@apache-superset/core';
const ctx = extensions.getContext();
await ctx.storage.persistent.set('api_token', 'sk-...', { encrypt: true });
const token = await ctx.storage.persistent.get('api_token');
```
```python
from superset_core.extensions.context import get_context
from superset_core.extensions.storage.persistent import PersistentSetOptions
ctx = get_context()
ctx.storage.persistent.set('api_token', 'sk-...', PersistentSetOptions(encrypt=True))
token = ctx.storage.persistent.get('api_token')
```
Encryption reuses Superset's existing `EncryptedType` (from `sqlalchemy-utils`) rather than a separate mechanism — the same infrastructure used for database connection credentials. The key is not configured directly: user-scoped values are encrypted with a key derived per-user via HMAC-SHA256 from the deployment's `SECRET_KEY`, so ciphertext for one user cannot be decrypted as another's; shared/global values (written via `.shared`) use `SECRET_KEY` directly. The encryption engine (AES-CBC by default) can be changed for the whole deployment via the existing `SQLALCHEMY_ENCRYPTED_FIELD_ENGINE` config, the same setting that controls encryption for database credentials elsewhere in Superset — there is no separate key list or rotation mechanism specific to extension storage. When `SECRET_KEY` is rotated, extension storage rows are re-encrypted by the existing `SecretsMigrator` tooling alongside database credentials, with no extension-specific steps required.
### Quotas
Persistent storage is subject to per-value, per-list-page, and per-extension size limits, configured in `superset_config.py`:
```python
EXTENSIONS_PERSISTENT_STORAGE = {
# Maximum total bytes an extension may store across all of its rows
# (global, every user's, and shared). Defaults to 100 MB.
"QUOTA_PER_EXTENSION": 100 * 1024 * 1024,
# Maximum size (in bytes) of a single stored value. Defaults to 1 MB.
"MAX_VALUE_SIZE": 1024 * 1024,
# Maximum combined value size (in bytes) that a single list() page may
# return. Defaults to 10 MB.
"MAX_LIST_PAYLOAD_SIZE": 10 * 1024 * 1024,
}
```
A write that would push the extension's total stored size over `QUOTA_PER_EXTENSION` is rejected — the frontend SDK's `set()` call rejects with an HTTP 413 response, and backend code calling `ctx.storage.persistent.set()` sees the corresponding exception raised. Overwriting an existing key nets out that key's own current size against usage first, so replacing a value with one of the same or smaller size never spuriously fails even when the extension is already near its quota.
A write whose encoded value exceeds `MAX_VALUE_SIZE` is rejected outright, independently of the total quota.
A `list()` call whose requested page would return more than `MAX_LIST_PAYLOAD_SIZE` combined bytes of value data is rejected — reduce `page`/`pageSize` and retry. This check runs against the page's stored sizes before any value is read from the database, so an oversized request fails fast. Because `list()`'s response is consumed as JSON in the browser, raising `MAX_LIST_PAYLOAD_SIZE` substantially above the default risks client-side memory and parse-time issues — treat the default as already generous rather than a floor to build up from.