Compare commits

...

6 Commits

Author SHA1 Message Date
Amin Ghadersohi
7c2135a3ad docs(mcp): update Semantic Layer MCP research with Beto's answers and final architecture
Incorporate answers from betodealmeida (2026-06-04) to the six open
questions, which materially change the proposed architecture:

- list_metrics is the primary LLM entry point (not admin discovery tools)
- Metrics must include compatible_dimensions inline in the response
- Unified get_table replaces separate query_dataset + query_semantic_view
- Global metric search with no layer selection required (single source of truth)
- get_compatible_metrics/dimensions and get_dimension_values/get_row_count
  are now explicit tools, mapping directly to view.py abstract methods
- Snowflake is first external implementation target; plugin mechanism is stable

Adds a Final Architecture Proposal section with revised tool designs,
phase breakdown, and updated effort estimates (~4 weeks total).
2026-06-30 22:05:33 +00:00
Amin Ghadersohi
0f220e976e docs: add Semantic Layer MCP research spike (SC-98803)
Research covering Superset's two semantic layer models (built-in SqlaTable
and external SemanticLayer/SemanticView), the existing MCP tool surface,
proposed new tools (Tier 1–3), landscape survey, and effort estimates.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 22:55:10 +00:00
Evan Rusackas
a75f9b67b2 chore(superset-ui-switchboard): forward-compat fixes for TypeScript 6.0 (Phase E) (#40028)
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 15:37:52 -07:00
Evan Rusackas
3f0858e35d chore(sql-lab): migrate useDispatch to useAppDispatch (#40037)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-05-20 15:36:27 -07:00
Beto Dealmeida
68c145adc3 feat(semantic layers): add metadata on additive metrics (#40279) 2026-05-20 18:29:28 -04:00
Evan Rusackas
4a9aecda4a fix(dashboard-import): remap chartsInScope on import (#26338) (#40140)
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Claude <claude@anthropic.com>
2026-05-20 13:41:14 -07:00
27 changed files with 855 additions and 49 deletions

540
RESEARCH.md Normal file
View File

@@ -0,0 +1,540 @@
# Semantic Layer MCP — Research
**Story**: SC-98803 — Semantic Layer & MCP
**Branch**: `research-semantic-layer-mcp`
**Date**: 2026-05-20
---
## Story Summary
Story #98803 is titled "Semantic Layer & MCP" with no written description or acceptance criteria. Two
comments clarify intent:
1. "Probably worth meeting with Beto to start discussing what Semantic Layer work means for MCP"
2. "not putting any more stories in mcp v1 and rather triaging future work in the Preset MCP V2 epic"
This is a **research spike** to map out what "Semantic Layer MCP" would mean, what already exists, what
gaps remain, and what would go into MCP v2. No implementation is expected from this story.
---
## What Superset's Semantic Layer Is
Superset has **two distinct semantic layer concepts** that must be treated separately:
### 1. Built-in Dataset Semantic Layer (SqlaTable)
Every Superset dataset (`SqlaTable`) carries a lightweight semantic layer composed of:
| Concept | Model | Storage |
|---|---|---|
| **Saved metrics** | `SqlMetric` | `sql_metrics` table; `expression` column holds SQL (e.g. `COUNT(*)`, `SUM(revenue)`) |
| **Calculated columns** | `TableColumn.expression` | `table_columns` table; non-null `expression` = virtual column |
| **Dimensions** | `TableColumn` with `groupby=True` | Physical or calculated columns marked as groupable |
| **Filters** | `TableColumn` with `filterable=True` | Drives filter dropdowns |
| **Time grains** | `SqlaTable.main_dttm_col` | Primary datetime column for temporal slicing |
| **RLS** | `RowLevelSecurityFilter` | Per-user WHERE clause predicates |
At query time, `TableColumn.get_sqla_col()` either wraps the physical column (`column(name)`) or inlines
the SQL expression as `literal_column(expression)`. Jinja2 templates are processed before injection.
Saved metrics follow the same pattern via `SqlMetric.get_sqla_col()`.
Physical vs. virtual datasets:
- **Physical** — `sql=NULL`; metadata discovered from DB schema
- **Virtual** — `sql=<SELECT ...>`; metadata discovered by running the SQL; `is_sqllab_view=True`
### 2. External Semantic Layer (SemanticLayer / SemanticView)
A newer plugin-based system for connecting to external semantic catalogs:
- `SemanticLayer` model (`superset/semantic_layers/models.py`) — a connection record (type, encrypted config)
- `SemanticView` model — a queryable view within a layer (maps to a dbt model, a Cube cube, etc.)
- `superset/semantic_layers/registry.py` — plugin registry mapping `type → implementation class`
- `superset/semantic_layers/mapper.py` — translates Superset's `QueryObject``SemanticQuery` for
dispatch to the implementation; handles metrics, dimensions, filters, time grains, and time comparison
- REST API: `GET/POST /api/v1/semantic_layer/`, `GET/POST /api/v1/semantic_layer/{id}/semantic_view/`
- Query path uses `datasource_type="semantic_view"` (distinct from `"table"` for SqlaTable)
The registry is currently empty in OSS (`registry: dict[str, type[SemanticLayer]] = {}`); implementations
are contributed via the extension/plugin mechanism referenced by the TODO in `models.py`.
---
## Existing MCP Tool Surface
The MCP service (`superset/mcp_service/`) already covers Superset's built-in semantic layer well:
| Tool | Tag | What it does |
|---|---|---|
| `list_datasets` | discovery | List/search datasets by name, schema, database |
| `get_dataset_info` | discovery | Full dataset metadata: **columns** (dimensions) + **metrics** (saved metrics) |
| `query_dataset` | data | Query using metric names + column names + filters + time range — **no SQL needed** |
| `create_virtual_dataset` | mutate | Save a SQL SELECT as a named virtual dataset |
| `execute_sql` | data | Raw SQL against any DB (bypasses semantic layer entirely) |
`query_dataset` is the core semantic layer MCP tool. It:
1. Resolves dataset by ID or UUID
2. Validates requested metric/column names against the dataset's saved metrics and columns (with typo
suggestions via `difflib.get_close_matches`)
3. Builds a `QueryContext` with `datasource_type="table"` and dispatches via `ChartDataCommand`
4. Returns tabular rows, column metadata, cache status, and applied filters
**What is NOT covered:**
- No MCP tools for `SemanticLayer` or `SemanticView` models (external semantic layers)
- No cross-dataset metric/dimension discovery (no `list_metrics`, `list_dimensions` tools)
- No tool for querying external semantic layers via the `datasource_type="semantic_view"` path
---
## Proposed MCP Tools
The following analysis groups proposed tools by priority tier.
### Tier 1 — External Semantic Layer Discovery (highest value, unblocked)
These tools mirror what `list_datasets` / `get_dataset_info` do for SqlaTable but for the external
semantic layer model.
---
**`list_semantic_layers`**
- **Description**: List available external semantic layer connections (dbt, Cube, etc.)
- **Inputs**: `page`, `page_size`, `search` (optional name filter)
- **Outputs**: `[{id, uuid, name, type, description, view_count}]`, pagination
- **Pattern**: `ModelListCore(SemanticLayerDAO, ...)`
- **Notes**: Requires `SemanticLayer` FAB permission; respects `semantic_layer_access` PVM.
---
**`get_semantic_layer_info`**
- **Description**: Get details of a semantic layer including its registered views
- **Inputs**: `identifier` (UUID)
- **Outputs**: `{uuid, name, type, description, views: [{id, uuid, name, description}]}`
- **Pattern**: `ModelGetInfoCore(SemanticLayerDAO, ...)`
- **Notes**: Views list drives the LLM to choose the right view before querying.
---
**`list_semantic_views`**
- **Description**: List views within a specific semantic layer
- **Inputs**: `semantic_layer_uuid`, `page`, `page_size`, `search`
- **Outputs**: `[{id, uuid, name, description, metric_count, dimension_count}]`
- **Pattern**: DAO query on `SemanticView` filtered by `semantic_layer_uuid`
- **Notes**: Separate from `get_semantic_layer_info` to support pagination on large layers.
---
### Tier 2 — External Semantic Layer Query
---
**`get_semantic_view_schema`**
- **Description**: Get available metrics and dimensions for a semantic view
- **Inputs**: `semantic_view_id` (int or UUID)
- **Outputs**: `{metrics: [{name, type, description, ...}], dimensions: [{name, type, is_dttm, ...}]}`
- **Notes**: Calls the layer's `implementation` to fetch live schema from the external system. This is
distinct from `get_dataset_info` because the metadata comes from the external catalog, not Superset's DB.
The `MetricMetadata` and `ColumnMetadata` dataclasses in `semantic_layers/models.py` define the shape.
---
**`query_semantic_view`**
- **Description**: Query an external semantic layer view by metric and dimension names
- **Inputs**: `semantic_view_id`, `metrics: list[str]`, `dimensions: list[str]`, `filters`, `time_range`,
`time_grain`, `row_limit`
- **Outputs**: Same shape as `query_dataset` (tabular rows + column metadata + performance)
- **Notes**: Uses the `datasource_type="semantic_view"` path through `QueryContextFactory`. The
`mapper.py` `get_results()` function already handles the translation. This is a peer to `query_dataset`,
not a replacement.
- **Key design question**: How to handle authentication to the external system (dbt Cloud token, Cube API
key, etc.) — stored in encrypted `SemanticLayer.configuration`, fetched at query time.
---
### Tier 3 — Cross-Dataset Discovery (nice-to-have, lower priority)
These are useful for LLM agents that need to find metrics without knowing which dataset to start from.
---
**`list_metrics`**
- **Description**: Search saved metrics across all accessible datasets
- **Inputs**: `search` (metric name / description), `dataset_id` (optional), `page`, `page_size`
- **Outputs**: `[{metric_name, expression, dataset_id, dataset_name, description, d3format}]`
- **Notes**: Direct DAO query on `SqlMetric` with RBAC filtering. High value for LLMs that start with
"find me a revenue metric" rather than "query dataset 42."
---
**`list_dimensions`**
- **Description**: Search groupable columns across accessible datasets
- **Inputs**: `search`, `dataset_id` (optional), `is_dttm` (optional), `page`, `page_size`
- **Outputs**: `[{column_name, type, is_dttm, dataset_id, dataset_name, description}]`
- **Notes**: Same pattern as `list_metrics` but on `TableColumn` with `groupby=True`.
Complementary to `list_metrics` for fully metric-driven workflows.
---
### Summary Table
| Tool | Tier | Effort | Depends on |
|---|---|---|---|
| `list_semantic_layers` | 1 | S | `SemanticLayerDAO` (exists) |
| `get_semantic_layer_info` | 1 | S | `SemanticLayerDAO` (exists) |
| `list_semantic_views` | 1 | S | `SemanticLayerDAO` (exists) |
| `get_semantic_view_schema` | 2 | M | External layer implementation |
| `query_semantic_view` | 2 | M | `mapper.py` (exists), `QueryContextFactory` |
| `list_metrics` | 3 | S | `SqlMetric` DAO query |
| `list_dimensions` | 3 | S | `TableColumn` DAO query |
Effort scale: S = 12 days, M = 35 days, L = 12 weeks, XL = 2+ weeks.
---
## Landscape Survey
### How Other Tools Do It
**Looker** (Google)
- Looker has a full Semantic Layer API: `/api/4.0/lookml_models/{name}/explores/{name}/fields` returns
dimensions, measures, and filters from LookML
- LookML is the semantic definition language; AI/LLM access is via Looker's Explore REST API
- Looker Studio integration reads the semantic layer directly
**Cube.dev**
- Cube ships an official MCP server (`@cubejs-backend/mcp`) with:
- `list_cubes` — returns all cubes with measure/dimension/segment definitions
- `query_cube` — runs a semantic query by cube+measures+dimensions+filters
- `get_cube_meta` — live schema for a specific cube
- This is the closest prior art to what Superset's Tier 1+2 tools would look like
**dbt Semantic Layer (MetricFlow)**
- dbt Cloud exposes metrics via JDBC/GraphQL Semantic Layer API
- MetricFlow's conceptual model: `Metric`, `Dimension`, `Entity`, `TimeDimension`
- LLM integrations (e.g., dbt Copilot) use `list_metrics`, `list_dimensions`, and a `query_metrics`
endpoint that returns results as Arrow IPC
**MetricFlow**
- Query API: `list_metrics()`, `list_dimensions()`, `query_metrics(metrics=[], groupBy=[])`
- Self-describing schema: each metric knows its required dimensions and time grains
**Key takeaway**: The industry pattern is consistent — a 3-tool surface: `list_metrics`, `list_dimensions`,
`query_metrics`. Superset's `get_dataset_info` + `query_dataset` already implement this pattern for the
built-in semantic layer. The gap is the external semantic layer equivalents and cross-dataset discovery.
### Prior Art in the Superset Community
- The `semantic_layers/` module (models, mapper, registry, API) was built ahead of any specific
implementation — it is the extension point for dbt, Cube, Snowflake Cortex, etc.
- No OSS implementations are registered; implementations come via the Preset plugin mechanism
- No Preset-specific semantic layer MCP tools exist in `superset-shell/preset/mcp/` today
---
## Implementation Approach
### Phase 1: External Semantic Layer Discovery (Tier 1)
The `SemanticLayerDAO` already exists and the REST API is live. Adding `list_semantic_layers`,
`get_semantic_layer_info`, and `list_semantic_views` follows the exact same pattern as the existing
dataset tools. Use `ModelListCore` and `ModelGetInfoCore` from `mcp_core.py`. This is straightforward
work (3 × S = ~34 days total) and unblocked.
Create `superset/mcp_service/semantic_layer/` module with:
- `schemas.py` — Pydantic schemas for SemanticLayer and SemanticView
- `tool/list_semantic_layers.py`
- `tool/get_semantic_layer_info.py`
- `tool/list_semantic_views.py`
- `tool/__init__.py`
Register all three in `app.py`.
### Phase 2: External Semantic Layer Query (Tier 2)
`get_semantic_view_schema` requires calling the external implementation's schema method, which may do
a network call. This needs timeout handling and graceful error messaging.
`query_semantic_view` is the most complex: it must go through the `mapper.py` + `QueryContextFactory`
path with `datasource_type="semantic_view"`. The query execution path exists but is untested from MCP.
Recommend writing this alongside integration tests against a mock external layer.
### Phase 3: Cross-Dataset Discovery (Tier 3)
`list_metrics` and `list_dimensions` are pure DAO queries on `SqlMetric` and `TableColumn`. They need
RBAC filtering (only return metrics/columns from datasets the user can access) which adds complexity.
The simplest safe approach is to call `DatasetDAO.find_all()` to get accessible datasets and then filter
metrics/columns in Python — this avoids writing a custom RBAC-aware JOIN.
### Key Design Decisions
1. **Unified `query` tool vs. split tools**: Should `query_dataset` be extended to detect
`semantic_view_id` inputs, or should `query_semantic_view` be a separate tool? Recommendation:
**separate tools** — the query paths are genuinely different (different datasource types, different
schema shapes) and separate tools are clearer for LLMs.
2. **Schema exposure**: Should `get_semantic_view_schema` return the full external schema or only the
subset Superset knows about? Recommendation: **live from external system** — this is the value prop.
3. **`list_metrics` RBAC**: Should it join dataset permissions in SQL or filter in Python? Recommendation:
**Python filtering via DatasetDAO** until proven to be a performance problem.
4. **Prompt updates**: `DEFAULT_INSTRUCTIONS` in `app.py` should be updated to mention the semantic layer
tools and when to prefer them over `execute_sql`.
---
## Questions for Beto — with Answers (2026-06-04)
1. **What does MCP v2 mean architecturally?** The story comment says these tools belong in "MCP v2
epic." Are there breaking changes (new tool naming, new auth model, new transport) that affect how
these tools should be designed?
> **💬 Beto:** No. It's just the second release of the MCP service — not a breaking change. Build
> these tools with the same conventions as v1.
2. **Which external semantic layer implementations exist at Preset?** The `registry` is empty in OSS.
What types are registered in `superset-shell`? Which implementations should `query_semantic_view`
be tested against first (dbt, Snowflake Cortex, Cube)?
> **💬 Beto:** Our first target is Snowflake, followed by MetricFlow, Cube, DJ. But it shouldn't
> matter — the interface is identical for all of them.
3. **Is `SemanticLayer.implementation` wiring complete?** There's a `TODO` comment in
`semantic_layers/models.py` referencing an `extension_manager` that isn't fully wired yet (currently
falls back directly to `registry[self.type]`). Is the plugin mechanism stable enough to build MCP
tools on top of?
> **💬 Beto:** It is. The TODO in the discovery mechanism is because we've been discussing changing
> how installed extensions are discovered in the future, but it won't affect any APIs — it's purely
> internal. Safe to build on top of today.
4. **RLS for external semantic layers**: SqlaTable RLS is handled by `get_sqla_row_level_filters()`.
External layers have their own permission models (dbt Cloud access, Cube tenant isolation). How should
MCP enforce per-user data access when querying external layers?
> **💬 Beto:** We don't currently support RLS in semantic layers/views, but will eventually. Don't
> worry about this now — when we add support it will be transparent to the MCP service.
5. **Should `list_metrics` / `list_dimensions` span built-in + external layers?** A unified
`list_metrics` spanning both SqlaTable saved metrics and external semantic layer metrics could be
very powerful but complex. Or should they be separate surfaces?
> **💬 Beto:** Yes — unified. Most users will have only a single semantic layer (the whole point is
> a single source of truth). Because of this, we should never force the user to choose a layer before
> listing metrics — most of the time there will be only one, and we should never prompt to choose from
> a single-element list. Even when there are multiple, start with global search.
6. **What is the intended LLM workflow for external semantic layers?** For built-in datasets the
workflow is: `list_datasets``get_dataset_info``query_dataset`. What's the analogous flow
for external layers?
> **💬 Beto:** The flow is `list_metrics` → metrics come with related dimensions already — then
> `get_table(metrics, dimensions, filters)`. Also worth exposing the `get_compatible_*()` methods
> from `superset-core/src/superset_core/semantic_layers/view.py`. When listing metrics, compatible
> dimensions should be part of the metric payload itself.
---
## Effort Estimate (original)
| Scope | Effort | Notes |
|---|---|---|
| `list_semantic_layers` | S (12 days) | Follows exact ModelListCore pattern |
| `get_semantic_layer_info` | S (12 days) | Follows exact ModelGetInfoCore pattern |
| `list_semantic_views` | S (1 day) | Subset of get_semantic_layer_info |
| `get_semantic_view_schema` | M (3 days) | Network call to external system + error handling |
| `query_semantic_view` | M (45 days) | New query path + integration tests |
| `list_metrics` | S (2 days) | DAO query + RBAC filter |
| `list_dimensions` | S (12 days) | Same pattern as list_metrics |
| Prompt/instructions updates | S (1 day) | Update DEFAULT_INSTRUCTIONS + tool docs |
| **Total Tier 1** | **~1 week** | |
| **Total Tier 1+2** | **~2.5 weeks** | Assuming external implementation is available |
| **Total Tier 1+2+3** | **~3.5 weeks** | |
---
## Final Architecture Proposal
*Synthesized from the research above and Beto's 2026-06-04 answers.*
### Core Insight: Flat Metric-First Workflow
Beto's answer to Q6 collapses the original three-tier hierarchy into a much flatter design. The intended
LLM workflow is:
```
list_metrics → get_table(metrics, dimensions, filters)
```
Not the hierarchical: `list_semantic_layers → list_semantic_views → get_semantic_view_schema → query_semantic_view`.
This means `list_metrics` is the **primary entry point** — not an administrative helper — and it must
work globally across both built-in datasets and external semantic layers without the user choosing a
layer first. Compatible dimensions travel with each metric in the response payload.
The abstract interface in `superset-core/src/superset_core/semantic_layers/view.py` is the contract
we're wrapping. Every proposed MCP tool maps to a method there:
| MCP Tool | `view.py` method | Notes |
|---|---|---|
| `list_metrics` | `get_metrics()` | + `SqlMetric` for built-in |
| `get_compatible_dimensions` | `get_compatible_dimensions(metrics, dims)` | Refinement during selection |
| `get_compatible_metrics` | `get_compatible_metrics(metrics, dims)` | Refinement during selection |
| `get_table` | `get_table(query)` | Unified query; replaces `query_dataset` + `query_semantic_view` |
| `get_dimension_values` | `get_values(dimension, filters)` | Distinct values for filter UI |
| `get_row_count` | `get_row_count(query)` | Preview row count before full fetch |
### Revised Tool Design
#### `list_metrics` ★ Primary tool
Search all accessible metrics globally — spanning both built-in (`SqlMetric`) and external semantic
layer metrics (via `SemanticView.get_metrics()`). No layer selection required.
- **Inputs:** `search` (optional text filter), `page`, `page_size`
- **Outputs:**
```json
[{
"name": "total_revenue",
"description": "...",
"source": {"type": "dataset" | "semantic_view", "id": "<uuid>", "name": "Sales"},
"compatible_dimensions": [
{"name": "region", "type": "VARCHAR", "is_dttm": false},
{"name": "order_date", "type": "TIMESTAMP", "is_dttm": true}
]
}]
```
- **RBAC:** Built-in metrics: filter by accessible datasets via `DatasetDAO`. External: filter by
accessible `SemanticLayer` connections.
- **Effort:** M (34 days) — unified across both sources is more complex than a single DAO query.
---
#### `get_table` ★ Unified query tool
Execute a semantic query against any metric source. Routes to the built-in path or the external
`SemanticView.get_table()` path based on where the metrics come from.
- **Inputs:** `metrics: list[str]`, `dimensions: list[str]`, `filters`, `time_range`, `time_grain`,
`row_limit`
- **Outputs:** Tabular rows + column metadata + performance info (same shape as existing `query_dataset`)
- **Routing logic:**
- All metrics from `SqlMetric` → existing `ChartDataCommand` path (`datasource_type="table"`)
- Metrics from `SemanticView` → `mapper.py` + `datasource_type="semantic_view"` path
- Mixed → error (metrics must come from a single source)
- **Notes:** This is the peer/replacement for `query_dataset` in semantic-layer-first workflows.
Existing `query_dataset` is unchanged — it remains the direct-dataset tool.
- **Effort:** M (45 days) — routing logic + integration tests against both paths.
---
#### `get_compatible_dimensions`
Refine dimension selection given already-chosen metrics and dimensions. Surfaces
`view.get_compatible_dimensions()`.
- **Inputs:** `metric_names: list[str]`, `selected_dimensions: list[str]` (already picked)
- **Outputs:** `[{name, type, is_dttm, description}]` — dimensions compatible with the current selection
- **Use case:** LLM is helping user build a query step-by-step; user has picked 2 metrics and now asks
"what dimensions can I group by?"
- **Effort:** S (12 days)
---
#### `get_compatible_metrics`
Mirror of `get_compatible_dimensions` — refine metric selection given selected dimensions.
- **Inputs:** `selected_metrics: list[str]`, `dimension_names: list[str]`
- **Outputs:** `[{name, description, source}]` — metrics compatible with the selected dimensions
- **Effort:** S (1 day)
---
#### `get_dimension_values`
Fetch distinct values for a dimension (for filter UI or LLM-driven filtering). Surfaces
`view.get_values()`.
- **Inputs:** `dimension_name: str`, `source_id` (dataset UUID or semantic_view UUID), `filters`
(optional pre-filters)
- **Outputs:** `[{value, label}]`
- **Effort:** S (12 days)
---
#### `get_row_count`
Count rows a query would return before executing it — useful for previews and safety checks. Surfaces
`view.get_row_count()`.
- **Inputs:** `metrics: list[str]`, `dimensions: list[str]`, `filters`, `time_range`
- **Outputs:** `{row_count: int}`
- **Effort:** S (1 day)
---
#### Admin Discovery Tools (lower priority)
These are useful for admin/setup workflows but are NOT part of the primary LLM query flow:
| Tool | Effort | Purpose |
|---|---|---|
| `list_semantic_layers` | S (12 days) | List configured external connections |
| `get_semantic_layer_info` | S (12 days) | Details of a connection + its views |
| `list_semantic_views` | S (1 day) | Paginated views within a layer |
---
### What Changed from the Original Proposal
| Original | Revised | Reason |
|---|---|---|
| Tier 3 (`list_metrics`) = lowest priority | `list_metrics` = **highest priority** | Beto: it's the primary entry point |
| Separate `query_dataset` + `query_semantic_view` | Unified `get_table` | Beto: single query tool for the semantic workflow |
| `get_semantic_view_schema` for schema discovery | `list_metrics` includes compatible dims inline | Beto: schema travels with metrics |
| Layer selection required before listing metrics | Global search, no layer selection | Beto: single source of truth, never prompt to choose |
| `get_compatible_*()` not in scope | Now explicit MCP tools | Beto: expose these directly |
### Implementation Phases
**Phase 1 — Core semantic workflow (~2 weeks, mostly unblocked)**
All built-in layer support; external layer support added in Phase 2.
1. `list_metrics` (built-in SqlMetric + stub for external)
2. `get_table` (built-in path first; `datasource_type="table"`)
3. `get_compatible_dimensions` / `get_compatible_metrics`
4. Update `DEFAULT_INSTRUCTIONS` in `app.py`
**Phase 2 — External layer support (~1.5 weeks, needs Snowflake plugin)**
1. Extend `list_metrics` to query external `SemanticView.get_metrics()`
2. Extend `get_table` to route to `datasource_type="semantic_view"` path
3. `get_dimension_values` (via `view.get_values()`)
4. `get_row_count` (via `view.get_row_count()`)
**Phase 3 — Admin tools (~0.5 weeks, independent)**
1. `list_semantic_layers` / `get_semantic_layer_info` / `list_semantic_views`
### Revised Effort Summary
| Tool | Phase | Effort | Blocked? |
|---|---|---|---|
| `list_metrics` (built-in) | 1 | M (3 days) | No |
| `get_table` (built-in path) | 1 | M (34 days) | No |
| `get_compatible_dimensions` | 1 | S (12 days) | No |
| `get_compatible_metrics` | 1 | S (1 day) | No |
| `DEFAULT_INSTRUCTIONS` update | 1 | S (1 day) | No |
| `list_metrics` (external) | 2 | M (2 days) | Yes — needs Snowflake plugin |
| `get_table` (external path) | 2 | M (23 days) | Yes — needs Snowflake plugin |
| `get_dimension_values` | 2 | S (12 days) | Yes — needs Snowflake plugin |
| `get_row_count` | 2 | S (1 day) | Yes — needs Snowflake plugin |
| `list_semantic_layers` | 3 | S (12 days) | No |
| `get_semantic_layer_info` | 3 | S (12 days) | No |
| `list_semantic_views` | 3 | S (1 day) | No |
| **Total Phase 1** | | **~2 weeks** | |
| **Total Phase 1+2** | | **~3.5 weeks** | |
| **Total Phase 1+2+3** | | **~4 weeks** | |

View File

@@ -92,6 +92,26 @@ class Dimension:
grain: Grain | None = None
class AggregationType(str, enum.Enum):
"""
Aggregation function applied by a metric.
Additivity (across an arbitrary set of grouping dimensions):
* ``SUM``, ``COUNT``: fully additive — sub-group sums roll up via ``sum``.
* ``MIN``, ``MAX``: roll up via ``min`` / ``max`` of sub-group values.
* ``AVG``, ``COUNT_DISTINCT``, ``OTHER``: not safely roll-uppable from
sub-aggregates without auxiliary data.
"""
SUM = "SUM"
COUNT = "COUNT"
MIN = "MIN"
MAX = "MAX"
AVG = "AVG"
COUNT_DISTINCT = "COUNT_DISTINCT"
OTHER = "OTHER"
@dataclass(frozen=True)
class Metric:
id: str
@@ -100,6 +120,7 @@ class Metric:
definition: str
description: str | None = None
aggregation: AggregationType | None = None
@dataclass(frozen=True)

View File

@@ -95,8 +95,11 @@ class FakeMessageChannel {
const port2 = new FakeMessagePort();
port1.otherPort = port2;
port2.otherPort = port1;
this.port1 = port1;
this.port2 = port2;
// FakeMessagePort only implements the subset of MessagePort that
// Switchboard exercises; cast at the boundary so the fake satisfies
// the consumer signature without weakening the production type.
this.port1 = port1 as unknown as MessagePort;
this.port2 = port2 as unknown as MessagePort;
}
}

View File

@@ -88,7 +88,7 @@ function isError(message: Message): message is ErrorMessage {
* Calling methods on the switchboard causes messages to be sent through the channel.
*/
export class Switchboard {
port: MessagePort;
port!: MessagePort;
name = '';
@@ -97,9 +97,9 @@ export class Switchboard {
// used to make unique ids
incrementor = 1;
debugMode: boolean;
debugMode = false;
private isInitialised: boolean;
private isInitialised = false;
constructor(params?: Params) {
if (!params) {

View File

@@ -1712,7 +1712,7 @@ export function createDatasource(
export function createCtasDatasource(
vizOptions: Record<string, unknown>,
): SqlLabThunkAction<Promise<{ id: number }>> {
): SqlLabThunkAction<Promise<{ table_id: number }>> {
return (dispatch: AppDispatch) => {
dispatch(createDatasourceStarted());
return SupersetClient.post({
@@ -1720,9 +1720,14 @@ export function createCtasDatasource(
jsonPayload: vizOptions,
})
.then(({ json }) => {
dispatch(createDatasourceSuccess(json.result));
const result = json.result as { table_id: number };
// The endpoint's `result.table_id` IS the dataset id; normalize so
// createDatasourceSuccess's `${data.id}__table` resolves correctly.
// Without this, the CTAS Explore button silently produced
// `"undefined__table"` because `result.id` doesn't exist.
dispatch(createDatasourceSuccess({ id: result.table_id }));
return json.result;
return result;
})
.catch(() => {
const errorMsg = t('An error occurred while creating the data source');

View File

@@ -19,7 +19,8 @@
import { useRef, useEffect, FC, useMemo } from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { logging } from '@apache-superset/core/utils';
import {
SqlLabRootState,
@@ -86,7 +87,7 @@ const EditorAutoSync: FC = () => {
const editorTabLastUpdatedAt = useSelector<SqlLabRootState, number>(
state => state.sqlLab.editorTabLastUpdatedAt,
);
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const lastSavedTimestampRef = useRef<number>(editorTabLastUpdatedAt);
const currentQueryEditorId = useSelector<SqlLabRootState, string>(

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useState, useEffect, useRef, useCallback, useMemo } from 'react';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { usePrevious } from '@superset-ui/core';
import { css, useTheme } from '@apache-superset/core/theme';
import { Global } from '@emotion/react';
@@ -136,7 +137,7 @@ const EditorWrapper = ({
height,
hotkeys,
}: EditorWrapperProps) => {
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const queryEditor = useQueryEditor(queryEditorId, [
'id',
'dbId',

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useEffect, useMemo, useRef } from 'react';
import { useDispatch, useStore } from 'react-redux';
import { useStore } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { t } from '@apache-superset/core/translation';
import { getExtensionsRegistry } from '@superset-ui/core';
@@ -68,7 +69,7 @@ export function useKeywords(
catalog,
schema,
});
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const hasFetchedKeywords = useRef(false);
// skipFetch is used to prevent re-evaluating memoized keywords
// due to updated api results by skip flag

View File

@@ -16,9 +16,10 @@
* specific language governing permissions and limitations
* under the License.
*/
import { useSelector, useDispatch } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { t } from '@apache-superset/core/translation';
import { JsonObject, VizType } from '@superset-ui/core';
import { VizType } from '@superset-ui/core';
import {
createCtasDatasource,
addInfoToast,
@@ -45,7 +46,7 @@ const ExploreCtasResultsButton = ({
const errorMessage = useSelector(
(state: SqlLabRootState) => state.sqlLab.errorMessage,
);
const dispatch = useDispatch<(dispatch: any) => Promise<JsonObject>>();
const dispatch = useAppDispatch();
const buildVizOptions = {
table_name: table,
@@ -56,7 +57,7 @@ const ExploreCtasResultsButton = ({
const visualize = () => {
dispatch(createCtasDatasource(buildVizOptions))
.then((data: { table_id: number }) => {
.then(data => {
const formData = {
datasource: `${data.table_id}__table`,
metrics: ['count'],

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useEffect, useState } from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import URI from 'urijs';
import { pick } from 'lodash';
import { useComponentDidUpdate } from '@superset-ui/core';
@@ -49,7 +50,7 @@ const PopEditorTab: React.FC<{ children?: React.ReactNode }> = ({
({ sqlLab: { tabHistory } }) => tabHistory.slice(-1)[0],
);
const [updatedUrl, setUpdatedUrl] = useState<string>(SQL_LAB_URL);
const dispatch = useDispatch();
const dispatch = useAppDispatch();
useComponentDidUpdate(() => {
setQueryEditorId(assigned => assigned ?? activeQueryEditorId);
if (activeQueryEditorId) {

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useRef } from 'react';
import { useSelector, useDispatch } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { isObject } from 'lodash';
import rison from 'rison';
import {
@@ -82,7 +83,7 @@ function QueryAutoRefresh({
.map(({ id }) => id),
),
);
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const checkForRefresh = () => {
const shouldRequestChecking = shouldCheckForQueries(queries);

View File

@@ -16,7 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
import { useDispatch } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { t } from '@apache-superset/core/translation';
import { Dropdown, Button } from '@superset-ui/core/components';
import { Menu } from '@superset-ui/core/components/Menu';
@@ -75,7 +75,7 @@ const QueryLimitSelect = ({
maxRow,
defaultQueryLimit,
}: QueryLimitSelectProps) => {
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const queryEditor = useQueryEditor(queryEditorId, ['id', 'queryLimit']);
const queryLimit = queryEditor.queryLimit || defaultQueryLimit;

View File

@@ -30,7 +30,8 @@ import ProgressBar from '@superset-ui/core/components/ProgressBar';
import { t } from '@apache-superset/core/translation';
import { QueryResponse, QueryState } from '@superset-ui/core';
import { useTheme } from '@apache-superset/core/theme';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import {
queryEditorSetSql,
@@ -92,7 +93,7 @@ const QueryTable = ({
latestQueryId,
}: QueryTableProps) => {
const theme = useTheme();
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const [selectedQuery, setSelectedQuery] = useState<QueryResponse | null>(
null,
);

View File

@@ -27,7 +27,8 @@ import {
} from 'react';
import AutoSizer from 'react-virtualized-auto-sizer';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { useHistory } from 'react-router-dom';
import { pick } from 'lodash';
import {
@@ -231,7 +232,7 @@ const ResultSet = ({
canCopyClipboardSqlLab: canCopyClipboard,
} = usePermissions();
const history = useHistory();
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const logAction = useLogAction({ queryId, sqlEditorId: query.sqlEditorId });
const { showConfirm, ConfirmModal } = useConfirmModal();

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { createRef, useCallback, useMemo } from 'react';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { nanoid } from 'nanoid';
import Tabs from '@superset-ui/core/components/Tabs';
import { t } from '@apache-superset/core/translation';
@@ -105,7 +106,7 @@ const SouthPane = ({
const { id, tabViewId } = useQueryEditor(queryEditorId, ['tabViewId']);
const editorId = tabViewId ?? id;
const theme = useTheme();
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const viewItems = views.getViews(ViewLocations.sqllab.panels) || [];
const { offline, tables } = useSelector(
({ sqlLab: { offline, tables } }: SqlLabRootState) => ({

View File

@@ -30,7 +30,8 @@ import {
import type { editors } from '@apache-superset/core';
import useEffectEvent from 'src/hooks/useEffectEvent';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import AutoSizer from 'react-virtualized-auto-sizer';
import { t } from '@apache-superset/core/translation';
import {
@@ -237,7 +238,7 @@ const SqlEditor: FC<Props> = ({
scheduleQueryWarning,
}) => {
const theme = useTheme();
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const { database, latestQuery, currentQueryEditorId, hasSqlStatement } =
useSelector<

View File

@@ -17,7 +17,7 @@
* under the License.
*/
import { useCallback, useState } from 'react';
import { useDispatch } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { resetState } from 'src/SqlLab/actions/sqlLab';
import {
@@ -69,7 +69,7 @@ const SqlEditorLeftBar = ({ queryEditorId }: SqlEditorLeftBarProps) => {
const { db, catalog, schema, onDbChange, onCatalogChange, onSchemaChange } =
dbSelectorProps;
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const shouldShowReset = window.location.search === '?reset=1';
// Modal state for Database/Catalog/Schema selector

View File

@@ -19,7 +19,8 @@
import { useMemo, FC } from 'react';
import { bindActionCreators } from 'redux';
import { useSelector, useDispatch, shallowEqual } from 'react-redux';
import { useSelector, shallowEqual } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { MenuDotsDropdown } from '@superset-ui/core/components';
import { Menu, MenuItemType } from '@superset-ui/core/components/Menu';
import { t } from '@apache-superset/core/translation';
@@ -90,7 +91,7 @@ const SqlEditorTabHeader: FC<Props> = ({ queryEditor }) => {
);
const StatusIcon = queryState ? STATE_ICONS[queryState] : STATE_ICONS.running;
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const actions = useMemo(
() =>
bindActionCreators(

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useEffect, useCallback, useMemo, useState } from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { SqlLabRootState } from 'src/SqlLab/types';
import {
@@ -41,7 +42,7 @@ export default function useDatabaseSelector(queryEditorId: string) {
SqlLabRootState,
SqlLabRootState['sqlLab']['databases']
>(({ sqlLab }) => sqlLab.databases);
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const queryEditor = useQueryEditor(queryEditorId, [
'dbId',
'catalog',

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { useState, useRef, useEffect } from 'react';
import { useDispatch, useSelector } from 'react-redux';
import { useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import type { QueryEditor, SqlLabRootState, Table } from 'src/SqlLab/types';
import {
ButtonGroup,
@@ -75,7 +76,7 @@ const Fade = styled.div`
const TableElement = ({ table, ...props }: TableElementProps) => {
const { dbId, catalog, schema, name, expanded, id } = table;
const theme = useTheme();
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const {
currentData: tableMetadata,
isSuccess: isMetadataSuccess,

View File

@@ -25,7 +25,8 @@ import {
type ChangeEvent,
useMemo,
} from 'react';
import { useSelector, useDispatch, shallowEqual } from 'react-redux';
import { useSelector, shallowEqual } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { styled, css, useTheme } from '@apache-superset/core/theme';
import { t } from '@apache-superset/core/translation';
import AutoSizer from 'react-virtualized-auto-sizer';
@@ -163,7 +164,7 @@ const savePinnedSchemasToStorage = (
};
const TableExploreTree: React.FC<Props> = ({ queryEditorId }) => {
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const theme = useTheme();
const treeRef = useRef<TreeApi<TreeNodeData>>(null);
const tables = useSelector(

View File

@@ -17,7 +17,7 @@
* under the License.
*/
import { useMemo, useReducer, useCallback } from 'react';
import { useDispatch } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { t } from '@apache-superset/core/translation';
import {
Table,
@@ -130,7 +130,7 @@ const useTreeData = ({
catalog,
pinnedTables,
}: UseTreeDataParams): UseTreeDataResult => {
const reduxDispatch = useDispatch();
const reduxDispatch = useAppDispatch();
// Schema data from API
const {
currentData: schemaData,

View File

@@ -17,7 +17,8 @@
* under the License.
*/
import { type FC, useCallback, useMemo, useRef, useState } from 'react';
import { shallowEqual, useDispatch, useSelector } from 'react-redux';
import { shallowEqual, useSelector } from 'react-redux';
import { useAppDispatch } from 'src/views/store';
import { nanoid } from 'nanoid';
import { t } from '@apache-superset/core/translation';
import { ClientErrorObject, getExtensionsRegistry } from '@superset-ui/core';
@@ -110,7 +111,7 @@ const renderWell = (partitions: TableMetaData['partitions']) => {
};
const TablePreview: FC<Props> = ({ dbId, catalog, schema, tableName }) => {
const dispatch = useDispatch();
const dispatch = useAppDispatch();
const theme = useTheme();
const [databaseName, backend, disableDataPreview] = useSelector<
SqlLabRootState,

View File

@@ -22,12 +22,13 @@ import {
createListenerMiddleware,
StoreEnhancer,
} from '@reduxjs/toolkit';
import type { AnyAction } from 'redux';
import {
useDispatch,
useSelector,
type TypedUseSelectorHook,
} from 'react-redux';
import thunk from 'redux-thunk';
import thunk, { type ThunkDispatch } from 'redux-thunk';
import { api } from 'src/hooks/apiResources/queryApi';
import messageToastReducer from 'src/components/MessageToasts/reducers';
import charts from 'src/components/Chart/chartReducer';
@@ -188,6 +189,14 @@ export type RootState = ReturnType<typeof store.getState>;
// thunks resolve correctly), and `useAppSelector` infers `RootState` without
// callers having to annotate every selector. Required ahead of the
// react-redux v8+ bump, which tightens dispatch typing — see #39927.
export type AppDispatch = typeof store.dispatch;
//
// AppDispatch is declared as ThunkDispatch & store.dispatch rather than
// `typeof store.dispatch` because Superset annotates getMiddleware as
// ConfigureStoreOptions['middleware'], which erases the middleware tuple type
// and leaves store.dispatch typed as Dispatch<AnyAction>. The intersection
// restores thunk support without requiring a wider refactor of the middleware
// setup.
export type AppDispatch = ThunkDispatch<RootState, undefined, AnyAction> &
typeof store.dispatch;
export const useAppDispatch: () => AppDispatch = useDispatch;
export const useAppSelector: TypedUseSelectorHook<RootState> = useSelector;

View File

@@ -62,6 +62,21 @@ def build_uuid_to_id_map(position: dict[str, Any]) -> dict[str, int]:
}
def _remap_charts_in_scope(container: dict[str, Any], id_map: dict[int, int]) -> None:
"""Remap source-env chart IDs in ``container["chartsInScope"]`` in place.
``chartsInScope`` is a denormalized cache of the charts a filter (native
or cross-filter) currently applies to. Both surfaces share this contract,
so they share this remap. Unresolvable IDs are dropped rather than
passed through, matching the convention used for ``scope.excluded``.
"""
charts_in_scope = container.get("chartsInScope")
if isinstance(charts_in_scope, list):
container["chartsInScope"] = [
id_map[old_id] for old_id in charts_in_scope if old_id in id_map
]
def update_id_refs( # pylint: disable=too-many-locals # noqa: C901
config: dict[str, Any],
chart_ids: dict[str, int],
@@ -145,6 +160,8 @@ def update_id_refs( # pylint: disable=too-many-locals # noqa: C901
id_map[old_id] for old_id in scope_excluded if old_id in id_map
]
_remap_charts_in_scope(native_filter, id_map)
# fix display control dataset references
for customization in (
fixed.get("metadata", {}).get("chart_customization_config") or []
@@ -170,7 +187,7 @@ def update_id_refs( # pylint: disable=too-many-locals # noqa: C901
return fixed
def update_cross_filter_scoping(
def update_cross_filter_scoping( # noqa: C901
config: dict[str, Any], id_map: dict[int, int]
) -> dict[str, Any]:
# fix cross filter references
@@ -185,6 +202,9 @@ def update_cross_filter_scoping(
id_map[old_id] for old_id in scope_excluded if old_id in id_map
]
# Global cross-filter chartsInScope mirrors the native-filter case.
_remap_charts_in_scope(cross_filter_global_config, id_map)
if "chart_configuration" in (metadata := fixed.get("metadata", {})):
# Build remapped configuration in a single pass for clarity/readability.
new_chart_configuration: dict[str, Any] = {}
@@ -212,6 +232,11 @@ def update_cross_filter_scoping(
if old_id in id_map
]
# Cross-filter chartsInScope mirrors the native-filter case.
cross_filters = chart_config.get("crossFilters")
if isinstance(cross_filters, dict):
_remap_charts_in_scope(cross_filters, id_map)
new_chart_configuration[str(new_id)] = chart_config
metadata["chart_configuration"] = new_chart_configuration

View File

@@ -238,6 +238,136 @@ def test_update_native_filter_config_default_rootpath_preserved():
assert scope["excluded"] == []
def test_update_id_refs_remaps_charts_in_scope():
"""
Regression for #26338: ``chartsInScope`` on a native filter holds chart
IDs and must be remapped from source-env IDs to destination-env IDs
during import.
The export side already converts ``chartsInScope`` IDs to UUIDs (see
``export_example.py:325``). The import side must symmetrically convert
them back to the destination environment's chart IDs. Without that
remap, the field carries stale source IDs into the imported dashboard
and breaks ``filtersInScope`` / ``filtersOutScope`` computation —
filters end up applied to the wrong charts (or none at all).
"""
from superset.commands.dashboard.importers.v1.utils import update_id_refs
config: dict[str, Any] = {
"position": {
"CHART1": {
"id": "CHART1",
"meta": {"chartId": 101, "uuid": "uuid1"},
"type": "CHART",
},
"CHART2": {
"id": "CHART2",
"meta": {"chartId": 102, "uuid": "uuid2"},
"type": "CHART",
},
},
"metadata": {
"native_filter_configuration": [
{
"id": "NATIVE_FILTER-region",
"scope": {"rootPath": ["ROOT_ID"], "excluded": []},
# chartsInScope contains source-env chart IDs.
"chartsInScope": [101, 102, 103],
}
],
},
}
chart_ids = {"uuid1": 1, "uuid2": 2}
dataset_info: dict[str, dict[str, Any]] = {}
fixed = update_id_refs(config, chart_ids, dataset_info)
filter_config = fixed["metadata"]["native_filter_configuration"][0]
# Resolved IDs are remapped; unknown IDs (103) are dropped rather than
# left to silently bind to whatever chart owns that integer in the
# destination environment.
assert filter_config["chartsInScope"] == [1, 2]
def test_update_id_refs_remaps_cross_filter_charts_in_scope():
"""
Companion to test_update_id_refs_remaps_charts_in_scope. Cross-filter
config also stores ``chartsInScope`` (under ``crossFilters`` per chart)
and must be remapped on import for the same reason.
"""
from superset.commands.dashboard.importers.v1.utils import update_id_refs
config: dict[str, Any] = {
"position": {
"CHART1": {
"id": "CHART1",
"meta": {"chartId": 101, "uuid": "uuid1"},
"type": "CHART",
},
"CHART2": {
"id": "CHART2",
"meta": {"chartId": 102, "uuid": "uuid2"},
"type": "CHART",
},
},
"metadata": {
"chart_configuration": {
"101": {
"id": 101,
"crossFilters": {
"scope": {"rootPath": ["ROOT_ID"], "excluded": []},
"chartsInScope": [101, 102, 103],
},
},
},
},
}
chart_ids = {"uuid1": 1, "uuid2": 2}
dataset_info: dict[str, dict[str, Any]] = {}
fixed = update_id_refs(config, chart_ids, dataset_info)
cross_filters = fixed["metadata"]["chart_configuration"]["1"]["crossFilters"]
assert cross_filters["chartsInScope"] == [1, 2]
def test_update_id_refs_remaps_global_chart_configuration_charts_in_scope():
"""
Per-chart and native-filter ``chartsInScope`` are remapped by their own
branches; ``global_chart_configuration.chartsInScope`` lives next to
``global_chart_configuration.scope.excluded`` and needs the same treatment
so the global cross-filter scope cache doesn't keep stale source-env IDs.
"""
from superset.commands.dashboard.importers.v1.utils import update_id_refs
config: dict[str, Any] = {
"position": {
"CHART1": {
"id": "CHART1",
"meta": {"chartId": 101, "uuid": "uuid1"},
"type": "CHART",
},
"CHART2": {
"id": "CHART2",
"meta": {"chartId": 102, "uuid": "uuid2"},
"type": "CHART",
},
},
"metadata": {
"global_chart_configuration": {
"scope": {"rootPath": ["ROOT_ID"], "excluded": []},
"chartsInScope": [101, 102, 103],
},
},
}
chart_ids = {"uuid1": 1, "uuid2": 2}
dataset_info: dict[str, dict[str, Any]] = {}
fixed = update_id_refs(config, chart_ids, dataset_info)
assert fixed["metadata"]["global_chart_configuration"]["chartsInScope"] == [1, 2]
def test_update_id_refs_cross_filter_chart_configuration_key_and_excluded_mapping():
from superset.commands.dashboard.importers.v1.utils import update_id_refs

View File

@@ -0,0 +1,58 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from __future__ import annotations
import pyarrow as pa
from superset_core.semantic_layers.types import AggregationType, Metric
def test_metric_aggregation_defaults_to_none() -> None:
metric = Metric(
id="x",
name="x",
type=pa.float64(),
definition="sum(x)",
)
assert metric.aggregation is None
def test_metric_accepts_aggregation_type() -> None:
metric = Metric(
id="x",
name="x",
type=pa.float64(),
definition="sum(x)",
aggregation=AggregationType.SUM,
)
assert metric.aggregation is AggregationType.SUM
def test_aggregation_type_is_string_enum() -> None:
# Behaves as a string for equality and serialization, so it can be sent
# over JSON without an explicit converter.
assert AggregationType.SUM == "SUM"
assert AggregationType.COUNT_DISTINCT.value == "COUNT_DISTINCT"
assert {a.value for a in AggregationType} == {
"SUM",
"COUNT",
"MIN",
"MAX",
"AVG",
"COUNT_DISTINCT",
"OTHER",
}