mirror of
https://github.com/apache/superset.git
synced 2026-05-04 23:44:23 +00:00
Compare commits
11 Commits
fix/postgr
...
6.1.0rc1
| Author | SHA1 | Date | |
|---|---|---|---|
|
249c21655f | ||
|
|
224a922341 | ||
|
|
a9b24da0a2 | ||
|
|
ab64ad7ac7 | ||
|
|
d266146bbb | ||
|
|
10415fe8be | ||
|
|
d42caf744f | ||
|
|
d8e346d52d | ||
|
|
8d7a36df5a | ||
|
|
77c7f9b5e8 | ||
|
|
6ebaf5919f |
@@ -50,3 +50,4 @@ under the License.
|
||||
- [4.1.4](./CHANGELOG/4.1.4.md)
|
||||
- [5.0.0](./CHANGELOG/5.0.0.md)
|
||||
- [6.0.0](./CHANGELOG/6.0.0.md)
|
||||
- [6.1.0](./CHANGELOG/6.1.0.md)
|
||||
|
||||
1338
CHANGELOG/6.1.0.md
Normal file
1338
CHANGELOG/6.1.0.md
Normal file
File diff suppressed because it is too large
Load Diff
132
UPDATING.md
132
UPDATING.md
@@ -24,46 +24,12 @@ assists people when migrating to a new version.
|
||||
|
||||
## Next
|
||||
|
||||
## 6.1.0
|
||||
|
||||
### ClickHouse minimum driver version bump
|
||||
|
||||
The minimum required version of `clickhouse-connect` has been raised to `>=0.13.0`. If you are using the ClickHouse connector, please upgrade your `clickhouse-connect` package. The `_mutate_label` workaround that appended hash suffixes to column aliases has also been removed, as it is no longer needed with modern versions of the driver.
|
||||
|
||||
### MCP Tool Observability
|
||||
|
||||
MCP (Model Context Protocol) tools now include enhanced observability instrumentation for monitoring and debugging:
|
||||
|
||||
**Two-layer instrumentation:**
|
||||
1. **Middleware layer** (`LoggingMiddleware`): Automatically logs all MCP tool calls with `duration_ms` and `success` status in the audit log (Action Log UI, logs table)
|
||||
2. **Sub-operation tracking**: All 19 MCP tools include granular `event_logger.log_context()` blocks for tracking individual operations like validation, database writes, and query execution
|
||||
|
||||
**Action naming convention:**
|
||||
- Tool-level logs: `mcp_tool_call` (via middleware)
|
||||
- Sub-operation logs: `mcp.{tool_name}.{operation}` (e.g., `mcp.generate_chart.validation`, `mcp.execute_sql.query_execution`)
|
||||
|
||||
**Querying MCP logs:**
|
||||
```sql
|
||||
-- Top slowest MCP operations
|
||||
SELECT action, COUNT(*) as calls, AVG(duration_ms) as avg_ms
|
||||
FROM logs
|
||||
WHERE action LIKE 'mcp.%'
|
||||
GROUP BY action
|
||||
ORDER BY avg_ms DESC
|
||||
LIMIT 20;
|
||||
|
||||
-- MCP tool success rate
|
||||
SELECT
|
||||
json_extract(curated_payload, '$.tool') as tool,
|
||||
COUNT(*) as total_calls,
|
||||
SUM(CASE WHEN json_extract(curated_payload, '$.success') = 'true' THEN 1 ELSE 0 END) as successful,
|
||||
ROUND(100.0 * SUM(CASE WHEN json_extract(curated_payload, '$.success') = 'true' THEN 1 ELSE 0 END) / COUNT(*), 2) as success_rate
|
||||
FROM logs
|
||||
WHERE action = 'mcp_tool_call'
|
||||
GROUP BY tool
|
||||
ORDER BY total_calls DESC;
|
||||
```
|
||||
|
||||
**Security note:** Sensitive parameters (passwords, API keys, tokens) are automatically redacted in logs as `[REDACTED]`.
|
||||
|
||||
### Distributed Coordination Backend
|
||||
|
||||
A new `DISTRIBUTED_COORDINATION_CONFIG` configuration provides a unified Redis-based backend for real-time coordination features in Superset. This backend enables:
|
||||
@@ -75,6 +41,7 @@ A new `DISTRIBUTED_COORDINATION_CONFIG` configuration provides a unified Redis-b
|
||||
The distributed coordination is used by the Global Task Framework (GTF) for abort notifications and task completion signaling, and will eventually replace `GLOBAL_ASYNC_QUERIES_CACHE_BACKEND` as the standard signaling backend. Configuring this is recommended for Redis enabled production deployments.
|
||||
|
||||
Example configuration in `superset_config.py`:
|
||||
|
||||
```python
|
||||
DISTRIBUTED_COORDINATION_CONFIG = {
|
||||
"CACHE_TYPE": "RedisCache",
|
||||
@@ -89,9 +56,11 @@ See `superset/config.py` for complete configuration options.
|
||||
### WebSocket config for GAQ with Docker
|
||||
|
||||
[35896](https://github.com/apache/superset/pull/35896) and [37624](https://github.com/apache/superset/pull/37624) updated documentation on how to run and configure Superset with Docker. Specifically for the WebSocket configuration, a new `docker/superset-websocket/config.example.json` was added to the repo, so that users could copy it to create a `docker/superset-websocket/config.json` file. The existing `docker/superset-websocket/config.json` was removed and git-ignored, so if you're using GAQ / WebSocket make sure to:
|
||||
|
||||
- Stash/backup your existing `config.json` file, to re-apply it after (will get git-ignored going forward)
|
||||
- Update the `volumes` configuration for the `superset-websocket` service in your `docker-compose.override.yml` file, to include the `docker/superset-websocket/config.json` file. For example:
|
||||
``` yaml
|
||||
|
||||
```yaml
|
||||
services:
|
||||
superset-websocket:
|
||||
volumes:
|
||||
@@ -104,7 +73,9 @@ services:
|
||||
### Example Data Loading Improvements
|
||||
|
||||
#### New Directory Structure
|
||||
|
||||
Examples are now organized by name with data and configs co-located:
|
||||
|
||||
```
|
||||
superset/examples/
|
||||
├── _shared/ # Shared database & metadata configs
|
||||
@@ -116,31 +87,12 @@ superset/examples/
|
||||
└── ...
|
||||
```
|
||||
|
||||
#### Simplified Parquet-based Loading
|
||||
- Auto-discovery: create `superset/examples/my_dataset/data.parquet` to add a new example
|
||||
- Parquet is an Apache project format: compressed (~27% smaller), self-describing schema
|
||||
- YAML configs define datasets, charts, and dashboards declaratively
|
||||
- Removed Python-based data generation from individual example files
|
||||
|
||||
#### Test Data Reorganization
|
||||
- Moved `big_data.py` to `superset/cli/test_loaders.py` - better reflects its purpose as a test utility
|
||||
- Fixed inverted logic for `--load-test-data` flag (now correctly includes .test.yaml files when flag is set)
|
||||
- Clarified CLI flags:
|
||||
- `--force` / `-f`: Force reload even if tables exist
|
||||
- `--only-metadata` / `-m`: Create table metadata without loading data
|
||||
- `--load-test-data` / `-t`: Include test dashboards and .test.yaml configs
|
||||
- `--load-big-data` / `-b`: Generate synthetic stress-test data
|
||||
|
||||
#### Bug Fixes
|
||||
- Fixed numpy array serialization for PostgreSQL (converts complex types to JSON strings)
|
||||
- Fixed KeyError for `allow_csv_upload` field in database configs (now optional with default)
|
||||
- Fixed test data loading logic that was incorrectly filtering files
|
||||
|
||||
### MCP Service
|
||||
|
||||
The MCP (Model Context Protocol) service enables AI assistants and automation tools to interact programmatically with Superset.
|
||||
|
||||
#### New Features
|
||||
|
||||
- MCP service infrastructure with FastMCP framework
|
||||
- Tools for dashboards, charts, datasets, SQL Lab, and instance metadata
|
||||
- Optional dependency: install with `pip install apache-superset[fastmcp]`
|
||||
@@ -150,6 +102,7 @@ The MCP (Model Context Protocol) service enables AI assistants and automation to
|
||||
#### New Configuration Options
|
||||
|
||||
**Development** (single-user, local testing):
|
||||
|
||||
```python
|
||||
# superset_config.py
|
||||
MCP_DEV_USERNAME = "admin" # User for MCP authentication
|
||||
@@ -158,6 +111,7 @@ MCP_SERVICE_PORT = 5008
|
||||
```
|
||||
|
||||
**Production** (JWT-based, multi-user):
|
||||
|
||||
```python
|
||||
# superset_config.py
|
||||
MCP_AUTH_ENABLED = True
|
||||
@@ -203,12 +157,14 @@ superset mcp run --port 5008 --use-factory-config
|
||||
The MCP service runs as a **separate process** from the Superset web server.
|
||||
|
||||
**Important**:
|
||||
|
||||
- Requires same Python environment and configuration as Superset
|
||||
- Shares database connections with main Superset app
|
||||
- Can be scaled independently from web server
|
||||
- Requires `fastmcp` package (optional dependency)
|
||||
|
||||
**Installation**:
|
||||
|
||||
```bash
|
||||
# Install with MCP support
|
||||
pip install apache-superset[fastmcp]
|
||||
@@ -222,6 +178,7 @@ Use systemd, supervisord, or Kubernetes to manage the MCP service process.
|
||||
See `superset/mcp_service/PRODUCTION.md` for deployment guides.
|
||||
|
||||
**Security**:
|
||||
|
||||
- Development: Uses `MCP_DEV_USERNAME` for single-user access
|
||||
- Production: **MUST** configure JWT authentication
|
||||
- See `superset/mcp_service/SECURITY.md` for details
|
||||
@@ -234,14 +191,50 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
|
||||
- Developer Guide: `superset/mcp_service/CLAUDE.md`
|
||||
- Quick Start: `superset/mcp_service/README.md`
|
||||
|
||||
---
|
||||
### MCP Tool Observability
|
||||
|
||||
- [35621](https://github.com/apache/superset/pull/35621): The default hash algorithm has changed from MD5 to SHA-256 for improved security and FedRAMP compliance. This affects cache keys for thumbnails, dashboard digests, chart digests, and filter option names. Existing cached data will be invalidated upon upgrade. To opt out of this change and maintain backward compatibility, set `HASH_ALGORITHM = "md5"` in your `superset_config.py`.
|
||||
- [35062](https://github.com/apache/superset/pull/35062): Changed the function signature of `setupExtensions` to `setupCodeOverrides` with options as arguments.
|
||||
MCP (Model Context Protocol) tools now include enhanced observability instrumentation for monitoring and debugging:
|
||||
|
||||
**Two-layer instrumentation:**
|
||||
|
||||
1. **Middleware layer** (`LoggingMiddleware`): Automatically logs all MCP tool calls with `duration_ms` and `success` status in the audit log (Action Log UI, logs table)
|
||||
2. **Sub-operation tracking**: All 19 MCP tools include granular `event_logger.log_context()` blocks for tracking individual operations like validation, database writes, and query execution
|
||||
|
||||
**Action naming convention:**
|
||||
|
||||
- Tool-level logs: `mcp_tool_call` (via middleware)
|
||||
- Sub-operation logs: `mcp.{tool_name}.{operation}` (e.g., `mcp.generate_chart.validation`, `mcp.execute_sql.query_execution`)
|
||||
|
||||
**Querying MCP logs:**
|
||||
|
||||
```sql
|
||||
-- Top slowest MCP operations
|
||||
SELECT action, COUNT(*) as calls, AVG(duration_ms) as avg_ms
|
||||
FROM logs
|
||||
WHERE action LIKE 'mcp.%'
|
||||
GROUP BY action
|
||||
ORDER BY avg_ms DESC
|
||||
LIMIT 20;
|
||||
|
||||
-- MCP tool success rate
|
||||
SELECT
|
||||
json_extract(curated_payload, '$.tool') as tool,
|
||||
COUNT(*) as total_calls,
|
||||
SUM(CASE WHEN json_extract(curated_payload, '$.success') = 'true' THEN 1 ELSE 0 END) as successful,
|
||||
ROUND(100.0 * SUM(CASE WHEN json_extract(curated_payload, '$.success') = 'true' THEN 1 ELSE 0 END) / COUNT(*), 2) as success_rate
|
||||
FROM logs
|
||||
WHERE action = 'mcp_tool_call'
|
||||
GROUP BY tool
|
||||
ORDER BY total_calls DESC;
|
||||
```
|
||||
|
||||
**Security note:** Sensitive parameters (passwords, API keys, tokens) are automatically redacted in logs as `[REDACTED]`.
|
||||
|
||||
### APP_NAME configuration
|
||||
|
||||
### Breaking Changes
|
||||
- [37370](https://github.com/apache/superset/pull/37370): The `APP_NAME` configuration variable no longer controls the browser window/tab title or other frontend branding. Application names should now be configured using the theme system with the `brandAppName` token. The `APP_NAME` config is still used for backend contexts (MCP service, logs, etc.) and serves as a fallback if `brandAppName` is not set.
|
||||
- **Migration:**
|
||||
|
||||
```python
|
||||
# Before (Superset 5.x)
|
||||
APP_NAME = "My Custom App"
|
||||
@@ -260,16 +253,22 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
|
||||
APP_NAME = "My Custom App"
|
||||
# But you should migrate to THEME_DEFAULT.token.brandAppName
|
||||
```
|
||||
|
||||
- **Note:** For dark mode, set the same tokens in `THEME_DARK` configuration.
|
||||
|
||||
### CUSTOM_FONT_URLS configuration
|
||||
|
||||
- [36317](https://github.com/apache/superset/pull/36317): The `CUSTOM_FONT_URLS` configuration option has been removed. Use the new per-theme `fontUrls` token in `THEME_DEFAULT` or database-managed themes instead.
|
||||
- **Before:**
|
||||
|
||||
```python
|
||||
CUSTOM_FONT_URLS = [
|
||||
"https://fonts.example.com/myfont.css",
|
||||
]
|
||||
```
|
||||
|
||||
- **After:**
|
||||
|
||||
```python
|
||||
THEME_DEFAULT = {
|
||||
"token": {
|
||||
@@ -281,7 +280,13 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
|
||||
}
|
||||
```
|
||||
|
||||
### Other
|
||||
|
||||
- [35621](https://github.com/apache/superset/pull/35621): The default hash algorithm has changed from MD5 to SHA-256 for improved security and FedRAMP compliance. This affects cache keys for thumbnails, dashboard digests, chart digests, and filter option names. Existing cached data will be invalidated upon upgrade. To opt out of this change and maintain backward compatibility, set `HASH_ALGORITHM = "md5"` in your `superset_config.py`.
|
||||
- [35062](https://github.com/apache/superset/pull/35062): Changed the function signature of `setupExtensions` to `setupCodeOverrides` with options as arguments.
|
||||
|
||||
## 6.0.0
|
||||
|
||||
- [33055](https://github.com/apache/superset/pull/33055): Upgrades Flask-AppBuilder to 5.0.0. The AUTH_OID authentication type has been deprecated and is no longer available as an option in Flask-AppBuilder. OpenID (OID) is considered a deprecated authentication protocol - if you are using AUTH_OID, you will need to migrate to an alternative authentication method such as OAuth, LDAP, or database authentication before upgrading.
|
||||
- [34871](https://github.com/apache/superset/pull/34871): Fixed Jest test hanging issue from Ant Design v5 upgrade. MessageChannel is now mocked in test environment to prevent rc-overflow from causing Jest to hang. Test environment only - no production impact.
|
||||
- [34782](https://github.com/apache/superset/pull/34782): Dataset exports now include the dataset ID in their file name (similar to charts and dashboards). If managing assets as code, make sure to rename existing dataset YAMLs to include the ID (and avoid duplicated files).
|
||||
@@ -290,8 +295,8 @@ See `superset/mcp_service/PRODUCTION.md` for deployment guides.
|
||||
- Change any hex color values to one of: `"success"`, `"processing"`, `"error"`, `"warning"`, `"default"`
|
||||
- Custom colors are no longer supported to maintain consistency with Ant Design components
|
||||
- [34561](https://github.com/apache/superset/pull/34561) Added tiled screenshot functionality for Playwright-based reports to handle large dashboards more efficiently. When enabled (default: `SCREENSHOT_TILED_ENABLED = True`), dashboards with 20+ charts or height exceeding 5000px will be captured using multiple viewport-sized tiles and combined into a single image. This improves report generation performance and reliability for large dashboards.
|
||||
Note: Pillow is now a required dependency (previously optional) to support image processing for tiled screenshots.
|
||||
`thumbnails` optional dependency is now deprecated and will be removed in the next major release (7.0).
|
||||
Note: Pillow is now a required dependency (previously optional) to support image processing for tiled screenshots.
|
||||
`thumbnails` optional dependency is now deprecated and will be removed in the next major release (7.0).
|
||||
- [33084](https://github.com/apache/superset/pull/33084) The DISALLOWED_SQL_FUNCTIONS configuration now includes additional potentially sensitive database functions across PostgreSQL, MySQL, SQLite, MS SQL Server, and ClickHouse. Existing queries using these functions may now be blocked. Review your SQL Lab queries and dashboards if you encounter "disallowed function" errors after upgrading
|
||||
- [34235](https://github.com/apache/superset/pull/34235) CSV exports now use `utf-8-sig` encoding by default to include a UTF-8 BOM, improving compatibility with Excel.
|
||||
- [34258](https://github.com/apache/superset/pull/34258) changing the default in Dockerfile to INCLUDE_CHROMIUM="false" (from "true") in the past. This ensures the `lean` layer is lean by default, and people can opt-in to the `chromium` layer by setting the build arg `INCLUDE_CHROMIUM=true`. This is a breaking change for anyone using the `lean` layer, as it will no longer include Chromium by default.
|
||||
@@ -681,7 +686,6 @@ Note: Pillow is now a required dependency (previously optional) to support image
|
||||
|
||||
- [11509](https://github.com/apache/superset/pull/12491): Dataset metadata updates check user ownership, only owners or an Admin are allowed.
|
||||
- Security simplification (SIP-19), the following permission domains were simplified:
|
||||
|
||||
- [12072](https://github.com/apache/superset/pull/12072): `Query` with `can_read`, `can_write`
|
||||
- [12036](https://github.com/apache/superset/pull/12036): `Database` with `can_read`, `can_write`.
|
||||
- [12012](https://github.com/apache/superset/pull/12036): `Dashboard` with `can_read`, `can_write`.
|
||||
|
||||
120
docs/src/theme/ApiExplorer/MethodEndpoint/index.tsx
Normal file
120
docs/src/theme/ApiExplorer/MethodEndpoint/index.tsx
Normal file
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*
|
||||
* Swizzled from docusaurus-theme-openapi-docs to fix SSG crash.
|
||||
*
|
||||
* The original component calls useTypedSelector (Redux) at the top level,
|
||||
* which fails during static site generation because no Redux store is
|
||||
* available. This version moves the hook into a browser-only child component
|
||||
* so SSG can render the page without a store context.
|
||||
*/
|
||||
|
||||
import React from "react";
|
||||
|
||||
import BrowserOnly from "@docusaurus/BrowserOnly";
|
||||
import { useSelector } from "react-redux";
|
||||
|
||||
interface ServerVariable {
|
||||
default?: string;
|
||||
}
|
||||
|
||||
interface ServerValue {
|
||||
url: string;
|
||||
variables?: Record<string, ServerVariable>;
|
||||
}
|
||||
|
||||
interface StoreState {
|
||||
server: { value: ServerValue | null };
|
||||
}
|
||||
|
||||
function colorForMethod(method: string) {
|
||||
switch (method.toLowerCase()) {
|
||||
case "get":
|
||||
return "primary";
|
||||
case "post":
|
||||
return "success";
|
||||
case "delete":
|
||||
return "danger";
|
||||
case "put":
|
||||
return "info";
|
||||
case "patch":
|
||||
return "warning";
|
||||
case "head":
|
||||
return "secondary";
|
||||
case "event":
|
||||
return "secondary";
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export interface Props {
|
||||
method: string;
|
||||
path: string;
|
||||
context?: "endpoint" | "callback";
|
||||
}
|
||||
|
||||
// Inner component rendered only in the browser, where the Redux store exists.
|
||||
function ServerUrl() {
|
||||
const serverValue = useSelector((state: StoreState) => state.server.value);
|
||||
|
||||
if (serverValue && serverValue.variables) {
|
||||
let serverUrlWithVariables = serverValue.url.replace(/\/$/, "");
|
||||
Object.keys(serverValue.variables).forEach((variable) => {
|
||||
serverUrlWithVariables = serverUrlWithVariables.replace(
|
||||
`{${variable}}`,
|
||||
serverValue.variables?.[variable].default ?? ""
|
||||
);
|
||||
});
|
||||
return <>{serverUrlWithVariables}</>;
|
||||
}
|
||||
|
||||
if (serverValue && serverValue.url) {
|
||||
return <>{serverValue.url}</>;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function MethodEndpoint({ method, path, context }: Props) {
|
||||
const renderServerUrl = () => {
|
||||
if (context === "callback") {
|
||||
return "";
|
||||
}
|
||||
return <BrowserOnly>{() => <ServerUrl />}</BrowserOnly>;
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<pre className="openapi__method-endpoint">
|
||||
<span className={"badge badge--" + colorForMethod(method)}>
|
||||
{method === "event" ? "Webhook" : method.toUpperCase()}
|
||||
</span>{" "}
|
||||
{method !== "event" && (
|
||||
<h2 className="openapi__method-endpoint-path">
|
||||
{renderServerUrl()}
|
||||
{`${path.replace(/{([a-z0-9-_]+)}/gi, ":$1")}`}
|
||||
</h2>
|
||||
)}
|
||||
</pre>
|
||||
<div className="openapi__divider" />
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
export default MethodEndpoint;
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superset",
|
||||
"version": "0.0.0-dev",
|
||||
"version": "6.1.0",
|
||||
"description": "Superset is a data exploration platform designed to be visual, intuitive, and interactive.",
|
||||
"keywords": [
|
||||
"big",
|
||||
|
||||
@@ -118,7 +118,7 @@ export interface ResultSetProps {
|
||||
const ResultContainer = styled.div`
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
row-gap: ${({ theme }) => theme.sizeUnit * 2}px;
|
||||
row-gap: ${({ theme }) => theme.sizeUnit * 3}px;
|
||||
height: 100%;
|
||||
`;
|
||||
|
||||
|
||||
@@ -63,6 +63,7 @@ const TABS_KEYS = {
|
||||
const StyledPane = styled.div`
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
|
||||
.ant-tabs .ant-tabs-content-holder {
|
||||
overflow: visible;
|
||||
}
|
||||
@@ -79,6 +80,7 @@ const StyledPane = styled.div`
|
||||
${({ theme }) => theme.sizeUnit * 2}px;
|
||||
}
|
||||
.ant-tabs-tabpane {
|
||||
padding-top: ${({ theme }) => theme.sizeUnit * 3}px;
|
||||
.scrollable {
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
@@ -251,23 +251,28 @@ def initialize_core_mcp_dependencies() -> None:
|
||||
|
||||
Also imports MCP service app to register all host tools BEFORE extension loading.
|
||||
"""
|
||||
import superset_core.mcp.decorators
|
||||
|
||||
try:
|
||||
# Replace the abstract decorators with concrete implementations
|
||||
from fastmcp.tools import Tool # noqa: F401
|
||||
except ImportError:
|
||||
logger.info(
|
||||
"fastmcp is not installed, skipping MCP initialization. "
|
||||
"Install it with: pip install 'apache-superset[fastmcp]'"
|
||||
)
|
||||
return
|
||||
|
||||
import superset_core.mcp.decorators
|
||||
# Replace the abstract decorators with concrete implementations
|
||||
superset_core.mcp.decorators.tool = create_tool_decorator
|
||||
superset_core.mcp.decorators.prompt = create_prompt_decorator
|
||||
|
||||
superset_core.mcp.decorators.tool = create_tool_decorator
|
||||
superset_core.mcp.decorators.prompt = create_prompt_decorator
|
||||
|
||||
logger.info("MCP dependency injection initialized successfully")
|
||||
logger.info("MCP dependency injection initialized successfully")
|
||||
|
||||
try:
|
||||
# Import MCP service app to register host tools BEFORE extension loading
|
||||
# This prevents host tools from being registered during extension context
|
||||
|
||||
from superset.mcp_service import app # noqa: F401
|
||||
|
||||
logger.info("MCP service app imported - host tools registered")
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Failed to initialize MCP dependencies: %s", e)
|
||||
raise
|
||||
logger.error("Failed to register MCP host tools: %s", e)
|
||||
|
||||
@@ -142,6 +142,33 @@ Query Examples:
|
||||
- My dashboards:
|
||||
filters=[{{"col": "created_by_fk", "opr": "eq", "value": <user_id>}}]
|
||||
|
||||
To modify an existing chart (add filters, change metrics, change dimensions, etc.):
|
||||
1. get_chart_info(chart_id) -> examine current configuration
|
||||
2. update_chart(chart_id, config) -> apply changes (filters, metrics, dimensions)
|
||||
Do NOT use execute_sql for chart modifications. Use update_chart instead.
|
||||
|
||||
CRITICAL RULES - NEVER VIOLATE:
|
||||
- NEVER fabricate or invent URLs. ALL URLs must come from tool call results.
|
||||
If you need a link, call the appropriate tool (generate_explore_link, generate_chart,
|
||||
open_sql_lab_with_context, etc.) and use the URL it returns.
|
||||
- To modify an existing chart's filters, metrics, or dimensions, use update_chart.
|
||||
Do NOT use execute_sql for chart modifications.
|
||||
- Parameter name reminders: open_sql_lab_with_context uses "sql" (not "query"),
|
||||
execute_sql uses "sql" (not "query").
|
||||
|
||||
IMPORTANT - Tool-Only Interaction:
|
||||
- Do NOT generate code artifacts, HTML pages, JavaScript snippets, or any code intended
|
||||
for the user to run. All visualization, data retrieval, and authentication are handled
|
||||
by the provided MCP tools.
|
||||
- Always call the appropriate tool directly instead of writing code. For example, use
|
||||
generate_chart to create visualizations rather than generating plotting code.
|
||||
- When a tool returns a URL (chart URL, dashboard URL, explore link, SQL Lab link),
|
||||
return that URL to the user. Do NOT attempt to recreate the visualization in code.
|
||||
- Do NOT generate HTML dashboards, embed scripts, or custom frontend code. Use
|
||||
generate_dashboard and add_chart_to_existing_dashboard for dashboard operations.
|
||||
- If a user asks for something the tools cannot do, explain the limitation and suggest
|
||||
the closest available tool rather than generating code as a workaround.
|
||||
|
||||
General usage tips:
|
||||
- All listing tools use 1-based pagination (first page is 1)
|
||||
- Use get_schema to discover filterable columns, sortable columns, and default columns
|
||||
|
||||
16
superset/mcp_service/chart/__init__.py
Normal file
16
superset/mcp_service/chart/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
@@ -789,34 +789,165 @@ def map_filter_operator(op: str) -> str:
|
||||
return operator_map.get(op, op)
|
||||
|
||||
|
||||
def _humanize_column(col: ColumnRef) -> str:
|
||||
"""Return a human-readable label for a column reference."""
|
||||
if col.label:
|
||||
return col.label
|
||||
name = col.name.replace("_", " ").title()
|
||||
if col.aggregate:
|
||||
return f"{col.aggregate.capitalize()}({name})"
|
||||
return name
|
||||
|
||||
|
||||
def _summarize_filters(
|
||||
filters: list[Any] | None,
|
||||
) -> str | None:
|
||||
"""Extract a short context string from filter configs."""
|
||||
if not filters:
|
||||
return None
|
||||
parts: list[str] = []
|
||||
for f in filters[:2]:
|
||||
col = getattr(f, "column", "")
|
||||
val = getattr(f, "value", "")
|
||||
if isinstance(val, list):
|
||||
val = ", ".join(str(v) for v in val[:3])
|
||||
parts.append(f"{str(col).replace('_', ' ').title()} {val}")
|
||||
return ", ".join(parts) if parts else None
|
||||
|
||||
|
||||
def _truncate(name: str, max_length: int = 60) -> str:
|
||||
"""Truncate to *max_length*, preserving the en-dash context portion."""
|
||||
if len(name) <= max_length:
|
||||
return name
|
||||
if " \u2013 " in name:
|
||||
what, _context = name.split(" \u2013 ", 1)
|
||||
if len(what) <= max_length:
|
||||
return what
|
||||
return name[: max_length - 1] + "\u2026"
|
||||
|
||||
|
||||
def _table_chart_what(config: TableChartConfig, dataset_name: str | None) -> str:
|
||||
"""Build the descriptive fragment for a table chart."""
|
||||
has_agg = any(col.aggregate for col in config.columns)
|
||||
if has_agg:
|
||||
metrics = [col for col in config.columns if col.aggregate]
|
||||
what = ", ".join(_humanize_column(m) for m in metrics[:2])
|
||||
return f"{what} Summary"
|
||||
if dataset_name:
|
||||
return f"{dataset_name} Records"
|
||||
cols = ", ".join(_humanize_column(c) for c in config.columns[:3])
|
||||
return f"{cols} Table"
|
||||
|
||||
|
||||
def _xy_chart_what(config: XYChartConfig) -> str:
|
||||
"""Build the descriptive fragment for an XY chart."""
|
||||
primary_metric = _humanize_column(config.y[0]) if config.y else "Value"
|
||||
dimension = _humanize_column(config.x)
|
||||
|
||||
if config.kind in ("line", "area") and config.group_by is None:
|
||||
return f"{primary_metric} Over Time"
|
||||
if config.group_by is not None:
|
||||
group_label = _humanize_column(config.group_by)
|
||||
return f"{primary_metric} by {group_label}"
|
||||
if config.kind == "scatter":
|
||||
return f"{primary_metric} vs {dimension}"
|
||||
return f"{primary_metric} by {dimension}"
|
||||
|
||||
|
||||
_GRAIN_MAP: dict[str, str] = {
|
||||
"PT1H": "Hourly",
|
||||
"P1D": "Daily",
|
||||
"P1W": "Weekly",
|
||||
"P1M": "Monthly",
|
||||
"P3M": "Quarterly",
|
||||
"P1Y": "Yearly",
|
||||
}
|
||||
|
||||
|
||||
def _xy_chart_context(config: XYChartConfig) -> str | None:
|
||||
"""Build context (time grain / filters) for an XY chart name."""
|
||||
parts: list[str] = []
|
||||
if config.time_grain:
|
||||
grain_val = (
|
||||
config.time_grain.value
|
||||
if hasattr(config.time_grain, "value")
|
||||
else str(config.time_grain)
|
||||
)
|
||||
grain_str = _GRAIN_MAP.get(grain_val, grain_val)
|
||||
parts.append(grain_str)
|
||||
if filter_ctx := _summarize_filters(config.filters):
|
||||
parts.append(filter_ctx)
|
||||
return ", ".join(parts) if parts else None
|
||||
|
||||
|
||||
def _pie_chart_what(config: PieChartConfig) -> str:
|
||||
"""Build the 'what' portion for a pie chart name."""
|
||||
dim = config.dimension.name
|
||||
metric_label = config.metric.label or config.metric.name
|
||||
return f"{dim} by {metric_label}"
|
||||
|
||||
|
||||
def _pivot_table_what(config: PivotTableChartConfig) -> str:
|
||||
"""Build the 'what' portion for a pivot table chart name."""
|
||||
row_names = ", ".join(r.name for r in config.rows)
|
||||
return f"Pivot Table \u2013 {row_names}"
|
||||
|
||||
|
||||
def _mixed_timeseries_what(config: MixedTimeseriesChartConfig) -> str:
|
||||
"""Build the 'what' portion for a mixed timeseries chart name."""
|
||||
primary = config.y[0].label or config.y[0].name if config.y else "primary"
|
||||
secondary = (
|
||||
config.y_secondary[0].label or config.y_secondary[0].name
|
||||
if config.y_secondary
|
||||
else "secondary"
|
||||
)
|
||||
return f"{primary} + {secondary}"
|
||||
|
||||
|
||||
def generate_chart_name(
|
||||
config: TableChartConfig
|
||||
| XYChartConfig
|
||||
| PieChartConfig
|
||||
| PivotTableChartConfig
|
||||
| MixedTimeseriesChartConfig,
|
||||
dataset_name: str | None = None,
|
||||
) -> str:
|
||||
"""Generate a chart name based on the configuration."""
|
||||
"""Generate a descriptive chart name following a standard format.
|
||||
|
||||
Format conventions (by chart type):
|
||||
Aggregated (bar/scatter with group_by): [Metric] by [Dimension]
|
||||
Time-series (line/area, no group_by): [Metric] Over Time
|
||||
Table (no aggregates): [Dataset] Records
|
||||
Table (with aggregates): [Metric] Summary
|
||||
Pie: [Dimension] by [Metric]
|
||||
Pivot Table: Pivot Table – [Row1, Row2]
|
||||
Mixed Timeseries: [Primary] + [Secondary]
|
||||
An en-dash followed by context (filters / time grain) is appended
|
||||
when such information is available.
|
||||
"""
|
||||
if isinstance(config, TableChartConfig):
|
||||
return f"Table Chart - {', '.join(col.name for col in config.columns)}"
|
||||
what = _table_chart_what(config, dataset_name)
|
||||
context = _summarize_filters(config.filters)
|
||||
elif isinstance(config, XYChartConfig):
|
||||
chart_type = config.kind.capitalize()
|
||||
x_col = config.x.name
|
||||
y_cols = ", ".join(col.name for col in config.y)
|
||||
return f"{chart_type} Chart - {x_col} vs {y_cols}"
|
||||
what = _xy_chart_what(config)
|
||||
context = _xy_chart_context(config)
|
||||
elif isinstance(config, PieChartConfig):
|
||||
metric_label = config.metric.label or config.metric.name
|
||||
return f"Pie Chart - {config.dimension.name} by {metric_label}"
|
||||
what = _pie_chart_what(config)
|
||||
context = _summarize_filters(config.filters)
|
||||
elif isinstance(config, PivotTableChartConfig):
|
||||
rows = ", ".join(col.name for col in config.rows)
|
||||
return f"Pivot Table - {rows}"
|
||||
what = _pivot_table_what(config)
|
||||
context = _summarize_filters(config.filters)
|
||||
elif isinstance(config, MixedTimeseriesChartConfig):
|
||||
primary = ", ".join(col.name for col in config.y)
|
||||
secondary = ", ".join(col.name for col in config.y_secondary)
|
||||
return f"Mixed Chart - {primary} + {secondary}"
|
||||
what = _mixed_timeseries_what(config)
|
||||
context = _summarize_filters(config.filters)
|
||||
else:
|
||||
return "Chart"
|
||||
|
||||
name = what
|
||||
if context:
|
||||
name = f"{what} \u2013 {context}"
|
||||
return _truncate(name)
|
||||
|
||||
|
||||
def analyze_chart_capabilities(chart: Any | None, config: Any) -> ChartCapabilities:
|
||||
"""Analyze chart capabilities based on type and configuration."""
|
||||
|
||||
@@ -103,6 +103,7 @@ def generate_preview_from_form_data(
|
||||
|
||||
# Execute query
|
||||
command = ChartDataCommand(query_context_obj)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
if not result or not result.get("queries"):
|
||||
|
||||
@@ -101,6 +101,7 @@ def _compile_chart(
|
||||
)
|
||||
|
||||
command = ChartDataCommand(query_context)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
warnings: List[str] = []
|
||||
@@ -263,10 +264,6 @@ async def generate_chart( # noqa: C901
|
||||
await ctx.report_progress(2, 5, "Creating chart in database")
|
||||
from superset.commands.chart.create import CreateChartCommand
|
||||
|
||||
# Use custom chart name if provided, otherwise auto-generate
|
||||
chart_name = request.chart_name or generate_chart_name(request.config)
|
||||
await ctx.debug("Chart name: chart_name=%s" % (chart_name,))
|
||||
|
||||
# Find the dataset to get its numeric ID
|
||||
from superset.daos.dataset import DatasetDAO
|
||||
|
||||
@@ -343,6 +340,15 @@ async def generate_chart( # noqa: C901
|
||||
}
|
||||
)
|
||||
|
||||
# Generate chart name after dataset lookup so we can include dataset name
|
||||
dataset_name = getattr(dataset, "datasource_name", None) or getattr(
|
||||
dataset, "table_name", None
|
||||
)
|
||||
chart_name = request.chart_name or generate_chart_name(
|
||||
request.config, dataset_name=dataset_name
|
||||
)
|
||||
await ctx.debug("Chart name: chart_name=%s" % (chart_name,))
|
||||
|
||||
try:
|
||||
with event_logger.log_context(action="mcp.generate_chart.db_write"):
|
||||
command = CreateChartCommand(
|
||||
|
||||
@@ -462,6 +462,7 @@ async def get_chart_data( # noqa: C901
|
||||
# Execute the query
|
||||
with event_logger.log_context(action="mcp.get_chart_data.query_execution"):
|
||||
command = ChartDataCommand(query_context)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
# Handle empty query results for certain chart types
|
||||
|
||||
@@ -160,6 +160,7 @@ class ASCIIPreviewStrategy(PreviewFormatStrategy):
|
||||
)
|
||||
|
||||
command = ChartDataCommand(query_context)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
data = []
|
||||
@@ -234,6 +235,7 @@ class TablePreviewStrategy(PreviewFormatStrategy):
|
||||
)
|
||||
|
||||
command = ChartDataCommand(query_context)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
data = []
|
||||
@@ -340,6 +342,7 @@ class VegaLitePreviewStrategy(PreviewFormatStrategy):
|
||||
|
||||
# Execute the query
|
||||
command = ChartDataCommand(query_context)
|
||||
command.validate()
|
||||
result = command.run()
|
||||
|
||||
# Extract data from result
|
||||
|
||||
16
superset/mcp_service/dashboard/__init__.py
Normal file
16
superset/mcp_service/dashboard/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
16
superset/mcp_service/dataset/__init__.py
Normal file
16
superset/mcp_service/dataset/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
# Licensed to the Apache Software Foundation (ASF) under one
|
||||
# or more contributor license agreements. See the NOTICE file
|
||||
# distributed with this work for additional information
|
||||
# regarding copyright ownership. The ASF licenses this file
|
||||
# to you under the Apache License, Version 2.0 (the
|
||||
# "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing,
|
||||
# software distributed under the License is distributed on an
|
||||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
# KIND, either express or implied. See the License for the
|
||||
# specific language governing permissions and limitations
|
||||
# under the License.
|
||||
@@ -30,7 +30,11 @@ import uvicorn
|
||||
|
||||
from superset.mcp_service.app import create_mcp_app, init_fastmcp_server
|
||||
from superset.mcp_service.mcp_config import get_mcp_factory_config, MCP_STORE_CONFIG
|
||||
from superset.mcp_service.middleware import create_response_size_guard_middleware
|
||||
from superset.mcp_service.middleware import (
|
||||
create_response_size_guard_middleware,
|
||||
GlobalErrorHandlerMiddleware,
|
||||
LoggingMiddleware,
|
||||
)
|
||||
from superset.mcp_service.storage import _create_redis_store
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -224,16 +228,24 @@ def run_server(
|
||||
auth_provider = _create_auth_provider(flask_app)
|
||||
|
||||
# Build middleware list
|
||||
# FastMCP wraps handlers so that the LAST-added middleware is
|
||||
# outermost. Order here is innermost → outermost.
|
||||
middleware_list = []
|
||||
|
||||
# Add caching middleware (innermost – runs closest to the tool)
|
||||
caching_middleware = create_response_caching_middleware()
|
||||
if caching_middleware:
|
||||
middleware_list.append(caching_middleware)
|
||||
|
||||
# Add response size guard (protects LLM clients from huge responses)
|
||||
if size_guard_middleware := create_response_size_guard_middleware():
|
||||
middleware_list.append(size_guard_middleware)
|
||||
|
||||
# Add caching middleware
|
||||
caching_middleware = create_response_caching_middleware()
|
||||
if caching_middleware:
|
||||
middleware_list.append(caching_middleware)
|
||||
# Add logging middleware (logs all tool calls with duration tracking)
|
||||
middleware_list.append(LoggingMiddleware())
|
||||
|
||||
# Add global error handler (outermost – catches all exceptions)
|
||||
middleware_list.append(GlobalErrorHandlerMiddleware())
|
||||
|
||||
mcp_instance = init_fastmcp_server(
|
||||
auth=auth_provider,
|
||||
|
||||
@@ -33,6 +33,7 @@ from superset.mcp_service.sql_lab.schemas import (
|
||||
SqlLabResponse,
|
||||
)
|
||||
from superset.mcp_service.utils.schema_utils import parse_request
|
||||
from superset.mcp_service.utils.url_utils import get_superset_base_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -95,7 +96,7 @@ def open_sql_lab_with_context(
|
||||
|
||||
# Construct SQL Lab URL
|
||||
query_string = urlencode(params)
|
||||
url = f"/sqllab?{query_string}"
|
||||
url = f"{get_superset_base_url()}/sqllab?{query_string}"
|
||||
|
||||
logger.info(
|
||||
"Generated SQL Lab URL for database %s", request.database_connection_id
|
||||
|
||||
@@ -424,6 +424,7 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods
|
||||
("can_read", "Theme"),
|
||||
# Embedded dashboard support
|
||||
("can_read", "EmbeddedDashboard"),
|
||||
("can_read", "CurrentUserRestApi"),
|
||||
# Datasource metadata for chart rendering
|
||||
("can_get", "Datasource"),
|
||||
("can_external_metadata", "Datasource"),
|
||||
|
||||
@@ -490,8 +490,8 @@ class TestMapConfigToFormData:
|
||||
class TestGenerateChartName:
|
||||
"""Test generate_chart_name function"""
|
||||
|
||||
def test_generate_table_chart_name(self) -> None:
|
||||
"""Test generating name for table chart"""
|
||||
def test_table_no_aggregates(self) -> None:
|
||||
"""Table without aggregates uses column names."""
|
||||
config = TableChartConfig(
|
||||
chart_type="table",
|
||||
columns=[
|
||||
@@ -501,25 +501,138 @@ class TestGenerateChartName:
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Table Chart - product, revenue"
|
||||
assert result == "Product, Revenue Table"
|
||||
|
||||
def test_generate_xy_chart_name(self) -> None:
|
||||
"""Test generating name for XY chart"""
|
||||
def test_table_no_aggregates_with_dataset_name(self) -> None:
|
||||
"""Table without aggregates includes dataset name when available."""
|
||||
config = TableChartConfig(
|
||||
chart_type="table",
|
||||
columns=[ColumnRef(name="product")],
|
||||
)
|
||||
|
||||
result = generate_chart_name(config, dataset_name="Orders")
|
||||
assert result == "Orders Records"
|
||||
|
||||
def test_table_with_aggregates(self) -> None:
|
||||
"""Table with aggregates produces a summary name."""
|
||||
config = TableChartConfig(
|
||||
chart_type="table",
|
||||
columns=[
|
||||
ColumnRef(name="product"),
|
||||
ColumnRef(name="revenue", aggregate="SUM"),
|
||||
],
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Sum(Revenue) Summary"
|
||||
|
||||
def test_line_chart_over_time(self) -> None:
|
||||
"""Line chart without group_by uses 'Over Time' format."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="date"),
|
||||
y=[ColumnRef(name="revenue"), ColumnRef(name="orders")],
|
||||
x=ColumnRef(name="order_date"),
|
||||
y=[ColumnRef(name="revenue", aggregate="SUM")],
|
||||
kind="line",
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Line Chart - date vs revenue, orders"
|
||||
assert result == "Sum(Revenue) Over Time"
|
||||
|
||||
def test_generate_chart_name_unsupported(self) -> None:
|
||||
"""Test generating name for unsupported config type"""
|
||||
def test_bar_chart_by_dimension(self) -> None:
|
||||
"""Bar chart uses 'by [X]' format."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="product_category"),
|
||||
y=[ColumnRef(name="order_count", aggregate="COUNT")],
|
||||
kind="bar",
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Count(Order Count) by Product Category"
|
||||
|
||||
def test_line_chart_with_group_by(self) -> None:
|
||||
"""Line chart with group_by uses 'by [group]' format."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="date"),
|
||||
y=[ColumnRef(name="revenue", aggregate="SUM")],
|
||||
kind="line",
|
||||
group_by=ColumnRef(name="sales_rep"),
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Sum(Revenue) by Sales Rep"
|
||||
|
||||
def test_scatter_plot(self) -> None:
|
||||
"""Scatter plot uses 'Y vs X' format."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="age"),
|
||||
y=[ColumnRef(name="income")],
|
||||
kind="scatter",
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Income vs Age"
|
||||
|
||||
def test_time_grain_in_context(self) -> None:
|
||||
"""Time grain is appended as context."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="date"),
|
||||
y=[ColumnRef(name="revenue", aggregate="SUM")],
|
||||
kind="line",
|
||||
time_grain="P1M",
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Sum(Revenue) Over Time \u2013 Monthly"
|
||||
|
||||
def test_filter_context(self) -> None:
|
||||
"""Filters are appended as context."""
|
||||
config = TableChartConfig(
|
||||
chart_type="table",
|
||||
columns=[ColumnRef(name="product")],
|
||||
filters=[FilterConfig(column="region", op="=", value="West")],
|
||||
)
|
||||
|
||||
result = generate_chart_name(config, dataset_name="Orders")
|
||||
assert result == "Orders Records \u2013 Region West"
|
||||
|
||||
def test_name_truncation(self) -> None:
|
||||
"""Names exceeding 60 chars are truncated."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="date"),
|
||||
y=[
|
||||
ColumnRef(
|
||||
name="very_long_metric_name_that_goes_on_and_on", aggregate="SUM"
|
||||
)
|
||||
],
|
||||
kind="line",
|
||||
group_by=ColumnRef(name="another_very_long_dimension_name_here"),
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert len(result) <= 60
|
||||
|
||||
def test_unsupported_config_type(self) -> None:
|
||||
"""Unsupported config type returns generic name."""
|
||||
result = generate_chart_name("invalid_config") # type: ignore
|
||||
assert result == "Chart"
|
||||
|
||||
def test_custom_labels_used(self) -> None:
|
||||
"""Column labels are preferred over names."""
|
||||
config = XYChartConfig(
|
||||
chart_type="xy",
|
||||
x=ColumnRef(name="ds", label="Date"),
|
||||
y=[ColumnRef(name="cnt", aggregate="COUNT", label="Order Count")],
|
||||
kind="bar",
|
||||
)
|
||||
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Order Count by Date"
|
||||
|
||||
|
||||
class TestGenerateExploreLink:
|
||||
"""Test generate_explore_link function"""
|
||||
|
||||
@@ -723,7 +723,7 @@ class TestGenerateChartNameNewTypes:
|
||||
metric=ColumnRef(name="revenue", aggregate="SUM"),
|
||||
)
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Pie Chart - product by revenue"
|
||||
assert result == "product by revenue"
|
||||
|
||||
def test_pie_chart_name_with_custom_label(self) -> None:
|
||||
config = PieChartConfig(
|
||||
@@ -732,7 +732,7 @@ class TestGenerateChartNameNewTypes:
|
||||
metric=ColumnRef(name="revenue", aggregate="SUM", label="Total Revenue"),
|
||||
)
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Pie Chart - product by Total Revenue"
|
||||
assert result == "product by Total Revenue"
|
||||
|
||||
def test_pivot_table_chart_name(self) -> None:
|
||||
config = PivotTableChartConfig(
|
||||
@@ -741,7 +741,7 @@ class TestGenerateChartNameNewTypes:
|
||||
metrics=[ColumnRef(name="revenue", aggregate="SUM")],
|
||||
)
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Pivot Table - product, region"
|
||||
assert result == "Pivot Table \u2013 product, region"
|
||||
|
||||
def test_mixed_timeseries_chart_name(self) -> None:
|
||||
config = MixedTimeseriesChartConfig(
|
||||
@@ -751,7 +751,7 @@ class TestGenerateChartNameNewTypes:
|
||||
y_secondary=[ColumnRef(name="orders", aggregate="COUNT")],
|
||||
)
|
||||
result = generate_chart_name(config)
|
||||
assert result == "Mixed Chart - revenue + orders"
|
||||
assert result == "revenue + orders"
|
||||
|
||||
|
||||
# ============================================================
|
||||
|
||||
@@ -671,3 +671,190 @@ class TestGetChartDataRequestSchema:
|
||||
assert data["identifier"] == 123
|
||||
assert data["limit"] == 50
|
||||
assert data["format"] == "json"
|
||||
|
||||
|
||||
class TestChartDataCommandValidation:
|
||||
"""Tests that ChartDataCommand.validate() is called before run().
|
||||
|
||||
These tests verify the security fix (CWE-862) that adds command.validate()
|
||||
before command.run() in MCP chart data tools. The validate() call enforces
|
||||
row-level security, guest user guards, and schema-level permissions.
|
||||
"""
|
||||
|
||||
def test_preview_utils_calls_validate_before_run(self):
|
||||
"""Test that generate_preview_from_form_data calls validate() before run()."""
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
call_order: list[str] = []
|
||||
mock_query_result = {"queries": [{"data": [{"col1": "a", "col2": 1}]}]}
|
||||
|
||||
mock_command = MagicMock()
|
||||
mock_command.validate.side_effect = lambda: call_order.append("validate")
|
||||
mock_command.run.side_effect = lambda: (
|
||||
call_order.append("run"),
|
||||
mock_query_result,
|
||||
)[1]
|
||||
|
||||
mock_dataset = MagicMock()
|
||||
mock_dataset.id = 10
|
||||
|
||||
# ChartDataCommand is module-level import in preview_utils;
|
||||
# db and QueryContextFactory are local imports inside the function.
|
||||
with (
|
||||
patch("superset.extensions.db") as mock_db,
|
||||
patch(
|
||||
"superset.mcp_service.chart.preview_utils.ChartDataCommand",
|
||||
return_value=mock_command,
|
||||
),
|
||||
patch(
|
||||
"superset.common.query_context_factory.QueryContextFactory"
|
||||
) as mock_factory,
|
||||
):
|
||||
mock_db.session.query.return_value.get.return_value = mock_dataset
|
||||
mock_factory.return_value.create.return_value = MagicMock()
|
||||
|
||||
from superset.mcp_service.chart.preview_utils import (
|
||||
generate_preview_from_form_data,
|
||||
)
|
||||
|
||||
generate_preview_from_form_data(
|
||||
form_data={"metrics": [{"label": "count"}], "viz_type": "table"},
|
||||
dataset_id=10,
|
||||
preview_format="table",
|
||||
)
|
||||
|
||||
mock_command.validate.assert_called_once()
|
||||
mock_command.run.assert_called_once()
|
||||
assert call_order == ["validate", "run"]
|
||||
|
||||
def test_preview_utils_security_exception_from_validate(self):
|
||||
"""Test that SupersetSecurityException from validate() is propagated."""
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
from superset.mcp_service.chart.schemas import ChartError
|
||||
|
||||
security_error = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="Access denied",
|
||||
error_type=SupersetErrorType.DATASOURCE_SECURITY_ACCESS_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
|
||||
mock_command = MagicMock()
|
||||
mock_command.validate.side_effect = security_error
|
||||
|
||||
mock_dataset = MagicMock()
|
||||
mock_dataset.id = 10
|
||||
|
||||
with (
|
||||
patch("superset.extensions.db") as mock_db,
|
||||
patch(
|
||||
"superset.mcp_service.chart.preview_utils.ChartDataCommand",
|
||||
return_value=mock_command,
|
||||
),
|
||||
patch(
|
||||
"superset.common.query_context_factory.QueryContextFactory"
|
||||
) as mock_factory,
|
||||
):
|
||||
mock_db.session.query.return_value.get.return_value = mock_dataset
|
||||
mock_factory.return_value.create.return_value = MagicMock()
|
||||
|
||||
from superset.mcp_service.chart.preview_utils import (
|
||||
generate_preview_from_form_data,
|
||||
)
|
||||
|
||||
result = generate_preview_from_form_data(
|
||||
form_data={"metrics": [{"label": "count"}], "viz_type": "table"},
|
||||
dataset_id=10,
|
||||
preview_format="table",
|
||||
)
|
||||
|
||||
# SupersetSecurityException is caught by the broad except and
|
||||
# returned as a ChartError
|
||||
assert isinstance(result, ChartError)
|
||||
assert "Access denied" in result.error
|
||||
mock_command.run.assert_not_called()
|
||||
|
||||
def test_compile_chart_calls_validate_before_run(self):
|
||||
"""Test that _compile_chart calls validate() before run()."""
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
call_order: list[str] = []
|
||||
mock_query_result = {"queries": [{"data": [{"col1": 1}]}]}
|
||||
|
||||
mock_command = MagicMock()
|
||||
mock_command.validate.side_effect = lambda: call_order.append("validate")
|
||||
mock_command.run.side_effect = lambda: (
|
||||
call_order.append("run"),
|
||||
mock_query_result,
|
||||
)[1]
|
||||
|
||||
# Both ChartDataCommand and QueryContextFactory are local imports
|
||||
# inside _compile_chart, so patch at source.
|
||||
with (
|
||||
patch(
|
||||
"superset.commands.chart.data.get_data_command.ChartDataCommand",
|
||||
return_value=mock_command,
|
||||
),
|
||||
patch(
|
||||
"superset.common.query_context_factory.QueryContextFactory"
|
||||
) as mock_factory,
|
||||
):
|
||||
mock_factory.return_value.create.return_value = MagicMock()
|
||||
|
||||
from superset.mcp_service.chart.tool.generate_chart import _compile_chart
|
||||
|
||||
result = _compile_chart(
|
||||
form_data={"metrics": [{"label": "count"}], "viz_type": "table"},
|
||||
dataset_id=10,
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
mock_command.validate.assert_called_once()
|
||||
mock_command.run.assert_called_once()
|
||||
assert call_order == ["validate", "run"]
|
||||
|
||||
def test_compile_chart_security_exception_from_validate(self):
|
||||
"""Test that _compile_chart propagates security exception from validate()."""
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from superset.errors import ErrorLevel, SupersetError, SupersetErrorType
|
||||
from superset.exceptions import SupersetSecurityException
|
||||
|
||||
security_error = SupersetSecurityException(
|
||||
SupersetError(
|
||||
message="Row-level security violation",
|
||||
error_type=SupersetErrorType.DATASOURCE_SECURITY_ACCESS_ERROR,
|
||||
level=ErrorLevel.ERROR,
|
||||
)
|
||||
)
|
||||
|
||||
mock_command = MagicMock()
|
||||
mock_command.validate.side_effect = security_error
|
||||
|
||||
with (
|
||||
patch(
|
||||
"superset.commands.chart.data.get_data_command.ChartDataCommand",
|
||||
return_value=mock_command,
|
||||
),
|
||||
patch(
|
||||
"superset.common.query_context_factory.QueryContextFactory"
|
||||
) as mock_factory,
|
||||
):
|
||||
mock_factory.return_value.create.return_value = MagicMock()
|
||||
|
||||
from superset.mcp_service.chart.tool.generate_chart import _compile_chart
|
||||
|
||||
# SupersetSecurityException is not caught by _compile_chart's
|
||||
# specific except blocks, so it propagates to the caller
|
||||
# (generate_chart's broad except handler).
|
||||
with pytest.raises(SupersetSecurityException):
|
||||
_compile_chart(
|
||||
form_data={"metrics": [{"label": "count"}], "viz_type": "table"},
|
||||
dataset_id=10,
|
||||
)
|
||||
|
||||
mock_command.run.assert_not_called()
|
||||
|
||||
@@ -37,3 +37,56 @@ def test_mcp_prompts_registered():
|
||||
|
||||
prompts = mcp._prompt_manager._prompts
|
||||
assert len(prompts) > 0
|
||||
|
||||
|
||||
def test_mcp_resources_registered():
|
||||
"""Test that MCP resources are registered.
|
||||
|
||||
Resources are registered via @mcp.resource() decorators in resource files.
|
||||
They require __init__.py in parent packages for find_packages() to include
|
||||
them in distributions. This test ensures all expected resources are found.
|
||||
"""
|
||||
from superset.mcp_service.app import mcp
|
||||
|
||||
resource_manager = mcp._resource_manager
|
||||
resources = resource_manager._resources
|
||||
assert len(resources) > 0, "No MCP resources registered"
|
||||
|
||||
# Verify the two documented resources are registered
|
||||
resource_uris = set(resources.keys())
|
||||
assert "chart://configs" in resource_uris, (
|
||||
"chart://configs resource not registered - "
|
||||
"check superset/mcp_service/chart/__init__.py exists"
|
||||
)
|
||||
assert "instance://metadata" in resource_uris, (
|
||||
"instance://metadata resource not registered - "
|
||||
"check superset/mcp_service/system/resources/ imports"
|
||||
)
|
||||
|
||||
|
||||
def test_mcp_packages_discoverable_by_setuptools():
|
||||
"""Test that all MCP sub-packages have __init__.py for setuptools.
|
||||
|
||||
setuptools.find_packages() only discovers directories with __init__.py.
|
||||
Without __init__.py, sub-packages (tool, resources, prompts) are excluded
|
||||
from built distributions, causing missing module errors in deployments.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
mcp_root = Path(__file__).parents[3] / "superset" / "mcp_service"
|
||||
assert mcp_root.is_dir(), f"MCP service root not found: {mcp_root}"
|
||||
|
||||
# All immediate sub-directories that contain Python files should be packages
|
||||
missing = []
|
||||
for subdir in sorted(mcp_root.iterdir()):
|
||||
if not subdir.is_dir() or subdir.name.startswith(("_", ".")):
|
||||
continue
|
||||
# Check if it has any .py files in it or its subdirectories
|
||||
has_py = any(subdir.rglob("*.py"))
|
||||
if has_py and not (subdir / "__init__.py").exists():
|
||||
missing.append(subdir.name)
|
||||
|
||||
assert not missing, (
|
||||
f"MCP sub-packages missing __init__.py (will be excluded from "
|
||||
f"setuptools distributions): {missing}"
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user