Files
superset2/docs/developer_docs/extensions/extension-points/persistent-storage.md
Amin Ghadersohi 02dbaeb9cf polish(tier3-storage): docs, test coverage, API cleanup
- Rename migration file to match the table it creates (extension_storage)
- Replace unsafe assert patterns in API with proper None guards
- Add direct import of ExtensionStorage into api.py (removes runtime import)
- Add missing requestBody OpenAPI spec to set_user and set_resource
- Add _key_to_fernet docstring
- Add test_list_resource_filtered_by_category (parity with list_global/list_user)
- Add docs/developer_docs/extensions/extension-points/persistent-storage.md
  covering all three scopes, REST API, frontend/backend usage examples,
  encryption, key rotation, and data model table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 11:23:24 -07:00

7.1 KiB

title, sidebar_position
title sidebar_position
Persistent Storage 3

Persistent Storage (Tier 3)

Extensions can store arbitrary key-value data in Superset's database without contributing model code to the core codebase. The storage layer supports three scopes, optional category filtering for cheap listing, optional at-rest encryption, and key rotation with zero downtime.

Storage Scopes

Scope When to use user_fk resource_type
Global Configuration shared across all users of the extension null null
User Per-user preferences or saved artefacts set null
Resource State tied to a specific Superset resource (dashboard, chart, …) any set

REST API

All endpoints live under:

/api/v1/extensions/{publisher}/{name}/storage/persistent/

Global scope

Method Path Description
GET .../global/ List entries (?page=0&page_size=25&category=…)
GET .../global/{key} Read a value (response body is the raw bytes)
PUT .../global/{key} Create or update
DELETE .../global/{key} Delete

User scope

Method Path Description
GET .../user/ List the authenticated user's entries
GET .../user/{key} Read a value
PUT .../user/{key} Create or update
DELETE .../user/{key} Delete

Resource scope

Method Path Description
GET .../resources/{type}/{uuid}/ List entries for the resource
GET .../resources/{type}/{uuid}/{key} Read a value
PUT .../resources/{type}/{uuid}/{key} Create or update
DELETE .../resources/{type}/{uuid}/{key} Delete

PUT request body

All write endpoints accept a JSON body:

{
  "value": "<string or base64 bytes>",
  "value_type": "application/json",
  "category": "my-category",
  "description": "Human-readable label for cheap listing",
  "is_encrypted": false
}
Field Required Default Notes
value yes "" The payload to store
value_type no application/json MIME type returned on GET
category no null Used to filter list results
description no null Returned in list results without fetching value
is_encrypted no false Encrypts value at rest with Fernet

List response

{
  "count": 42,
  "result": [
    {
      "key": "my-key",
      "uuid": "...",
      "value_type": "application/json",
      "category": "my-category",
      "description": "...",
      "is_encrypted": false
    }
  ]
}

The value blob is not included in list responses. Fetch individual keys to retrieve values.

Frontend usage

From a TypeScript extension frontend, use standard fetch with the Superset CSRF token:

const BASE = '/api/v1/extensions/acme/my-ext/storage/persistent/user';

// Save
await fetch(`${BASE}/my-key`, {
  method: 'PUT',
  headers: {
    'Content-Type': 'application/json',
    'X-CSRFToken': await authentication.getCSRFToken(),
  },
  body: JSON.stringify({
    value: JSON.stringify({ foo: 'bar' }),
    value_type: 'application/json',
    category: 'preferences',
    description: 'My display name',  // shown in list without fetching value
  }),
});

// List
const { result } = await fetch(`${BASE}/?category=preferences`).then(r => r.json());

// Read
const raw = await fetch(`${BASE}/my-key`).then(r => r.text());
const data = JSON.parse(raw);

// Delete
await fetch(`${BASE}/my-key`, { method: 'DELETE', headers: { 'X-CSRFToken': ... } });

Backend usage (Python)

Within extension backend code, use ExtensionStorageDAO directly:

from superset_core.common.daos import ExtensionStorageDAO

EXT_ID = "acme.my-ext"

# Write
ExtensionStorageDAO.set(EXT_ID, "config", b'{"theme":"dark"}', category="global-config")

# Read
value: bytes | None = ExtensionStorageDAO.get_value(EXT_ID, "config")

# User-scoped
ExtensionStorageDAO.set(EXT_ID, "prefs", b'{"lang":"fr"}', user_fk=user_id)
value = ExtensionStorageDAO.get_value(EXT_ID, "prefs", user_fk=user_id)

# Resource-scoped
ExtensionStorageDAO.set(
    EXT_ID, "state", payload,
    resource_type="dashboard", resource_uuid=str(dashboard.uuid),
)

:::note ExtensionStorageDAO methods call db.session.flush() but do not commit. The caller (or the @transaction() decorator on the API endpoint) owns the transaction. :::

Encryption at rest

Set "is_encrypted": true in the PUT body (frontend) or is_encrypted=True in ExtensionStorageDAO.set() (backend). Values are encrypted with Fernet using a key derived from SECRET_KEY (or the first entry of EXTENSION_STORAGE_ENCRYPTION_KEYS if configured). Decryption is transparent on read.

Key rotation

To rotate encryption keys without downtime:

  1. Prepend the new key to EXTENSION_STORAGE_ENCRYPTION_KEYS in your Superset config and restart. Existing encrypted rows are still readable (old key is tried as a fallback); new writes use the new key.

    # superset_config.py
    EXTENSION_STORAGE_ENCRYPTION_KEYS = [
        "new-strong-secret",   # used for new encryptions
        "old-secret",          # still tried for decryption
    ]
    
  2. Re-encrypt all existing rows with the new key:

    superset rotate-extension-storage-keys
    
  3. Remove the old key from the list and restart again.

Data model

The extension_storage table schema:

Column Type Notes
id integer PK
uuid UUID Unique, auto-generated
extension_id varchar(255) {publisher}.{name}
user_fk integer FK null = global or resource scope
resource_type varchar(64) e.g. dashboard, chart
resource_uuid varchar(36) UUID of the linked resource
key varchar(255) Identifier within a scope
category varchar(64) Optional grouping tag
description text Summary for list endpoints
value binary (up to 16 MB) The stored payload
value_type varchar(255) MIME type, default application/json
is_encrypted boolean Whether value is Fernet-encrypted
created_on, changed_on datetime Audit fields
created_by_fk, changed_by_fk integer FK Audit fields