Compare commits

...
Author SHA1 Message Date
Maxime Beauchemin c778c15259 docs(ci): auto-regenerate openapi files used for docs 2025-05-06 16:02:41 -07:00
Beto Dealmeida dc0d542054 chore: regenerate openapi.json (#33378) 2025-05-06 15:56:00 -07:00
0cd3a12daa chore(🦾): bump python markdown 3.7 -> 3.8 (#33279)
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Maxime Beauchemin <maximebeauchemin@gmail.com>
2025-05-06 09:06:59 -07:00
amaannawab923andAmaan Nawab 35b30480f0 fix: Exclude Filter Values (#33271)
Co-authored-by: Amaan Nawab <nelsondrew07@gmail.com>
2025-05-06 13:40:27 +02:00
github-actions[bot]andGitHub Action 6d1f17bd46 chore(🦾): bump python sshtunnel subpackage(s) (#33370)
Co-authored-by: GitHub Action <action@github.com>
2025-05-05 18:30:08 -07:00
github-actions[bot]andGitHub Action ab899e71e7 chore(🦾): bump python cryptography 44.0.2 -> 44.0.3 (#33371)
Co-authored-by: GitHub Action <action@github.com>
2025-05-05 18:29:40 -07:00
github-actions[bot]andGitHub Action 6b9d8708d3 chore(🦾): bump python humanize 4.12.2 -> 4.12.3 (#33369)
Co-authored-by: GitHub Action <action@github.com>
2025-05-05 18:29:10 -07:00
github-actions[bot]andGitHub Action bc1e8e07cf chore(🦾): bump python sqlglot 26.16.2 -> 26.16.4 (#33368)
Co-authored-by: GitHub Action <action@github.com>
2025-05-05 18:28:56 -07:00
github-actions[bot]andGitHub Action 82526865d2 chore(🦾): bump python h11 0.14.0 -> 0.16.0 (#33339)
Co-authored-by: GitHub Action <action@github.com>
2025-05-06 07:41:10 +08:00
Daniel Vaz Gaspar 02c8c9c752 fix: bump FAB to 4.6.3 (#33363) 2025-05-06 00:02:25 +01:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 6475188e6a chore(deps): bump swagger-ui-react from 5.20.2 to 5.21.0 in /docs (#33318)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 16:34:20 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 6e485c9f70 chore(deps-dev): update ts-loader requirement from ^9.5.1 to ^9.5.2 in /superset-frontend/packages/superset-ui-demo (#33323)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 16:33:58 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> b49e5857c9 chore(deps): bump uuid from 11.0.2 to 11.1.0 in /superset-websocket (#33311)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 16:33:18 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 13ced58261 chore(deps-dev): bump @eslint/js from 9.17.0 to 9.25.1 in /superset-websocket (#33312)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 16:32:59 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> ed36674a99 chore(deps): bump less from 4.2.2 to 4.3.0 in /docs (#33317)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 16:32:28 -06:00
Jonas DOREL 99aa3a6507 docs(docker-builds.mdx): clarify dockerize images (#33350) 2025-05-05 16:31:33 -06:00
Maxime Beauchemin f045a73e2d fix: loading examples from raw.githubusercontent.com fails with 429 errors (#33354) 2025-05-05 13:07:23 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 7791674f24 chore(deps-dev): bump eslint-config-prettier from 10.1.1 to 10.1.2 in /docs (#33315)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-03 15:37:33 -06:00
Vitor Avila 9f0ae77341 fix: Edge case with metric not getting quoted in sort by when normalize_columns is enabled (#33337) 2025-05-02 18:20:57 -07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 5a9e366c0a chore(deps-dev): bump typescript from 5.8.2 to 5.8.3 in /docs (#33320)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-02 11:00:13 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> c22c532a5c chore(deps-dev): bump eslint-plugin-react from 7.37.4 to 7.37.5 in /docs (#33314)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-02 10:59:53 -06:00
Michael S. Molina 6db3a4d9d2 fix: Temporal filter conversion in viz migrations (#33224) 2025-05-02 08:27:49 -03:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 17d7b72f3b chore(deps-dev): bump webpack from 5.98.0 to 5.99.7 in /docs (#33316)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-02 14:31:29 +07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> fee33dd0cf chore(deps): bump @rjsf/validator-ajv8 from 5.24.1 to 5.24.9 in /superset-frontend (#33321)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-02 14:29:44 +07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 65605b4a54 chore(deps-dev): bump @babel/plugin-transform-runtime from 7.25.9 to 7.27.1 in /superset-frontend (#33332)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-02 14:28:41 +07:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> e304f2d5ad chore(deps): bump react-intersection-observer from 9.15.1 to 9.16.0 in /superset-frontend (#33333)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-01 16:22:16 -07:00
Beto Dealmeida 4e0c261c9d fix: show only filterable columns on filter dropdown (#33338) 2025-05-01 18:36:32 -04:00
Beto Dealmeida 22de26cd77 fix: metric.currency should be JSON, not string (#33303) 2025-05-01 18:16:51 -04:00
Beto Dealmeida 339ba96600 fix: improve function detection (#33306) 2025-05-01 13:45:03 -04:00
Phillip LeBlancandVille Brofeldt 3c6091144b chore(deps): Upgrade pyarrow to 18.1.0 (#31476)
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2025-04-30 22:31:47 -06:00
JUST.in DO IT ef14b529b8 fix(echarts): rename time series shifted colnames (#33269) 2025-04-30 14:18:18 -03:00
github-actions[bot]andGitHub Action 2a97a6ec1f chore(🦾): bump python importlib-metadata 8.6.1 -> 8.7.0 (#33277)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 10:03:18 -07:00
github-actions[bot]andGitHub Action fa6548939e chore(🦾): bump python mako 1.3.9 -> 1.3.10 (#33280)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 10:02:45 -07:00
github-actions[bot]andGitHub Action 418c673699 chore(🦾): bump python pyparsing 3.2.2 -> 3.2.3 (#33281)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 10:01:42 -07:00
github-actions[bot]andGitHub Action 13f77a7416 chore(🦾): bump python celery 5.4.0 -> 5.5.2 (#33257)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:28:39 -07:00
github-actions[bot]andGitHub Action 303a80a316 chore(🦾): bump python packaging 24.2 -> 25.0 (#33259)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:27:48 -07:00
github-actions[bot]andGitHub Action 2392ac6827 chore(🦾): bump python deprecation subpackage(s) (#33260)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:27:25 -07:00
github-actions[bot]andGitHub Action 01ce4b987e chore(🦾): bump python python-dotenv 1.0.1 -> 1.1.0 (#33262)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:26:59 -07:00
github-actions[bot]andGitHub Action 2f308a85d8 chore(🦾): bump python pandas subpackage(s) (#33263)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:26:40 -07:00
github-actions[bot]andGitHub Action e8d60509a0 chore(🦾): bump python sqlglot 26.11.1 -> 26.16.2 (#33266)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:26:01 -07:00
github-actions[bot]andGitHub Action d6f80eaae7 chore(🦾): bump python gunicorn subpackage(s) (#33265)
Co-authored-by: GitHub Action <action@github.com>
2025-04-29 08:25:23 -07:00
Emad Rad a5f986fec5 feat: Persian translations (#29580) 2025-04-29 09:01:34 -06:00
Beto Dealmeida 141d0252f2 fix: mask password on DB import (#33267) 2025-04-29 10:27:03 -04:00
Daniel Vaz Gaspar c029b532d4 fix: LocalProxy is not mapped warning (#33025) 2025-04-28 23:01:26 -06:00
github-actions[bot]andGitHub Action 13816443ba chore(🦾): bump python croniter subpackage(s) (#33258)
Co-authored-by: GitHub Action <action@github.com>
2025-04-28 16:52:09 -07:00
Elizabeth Thompson 2c4e22e598 chore: add some utils tests (#33236) 2025-04-28 15:00:32 -07:00
Hamir Mahal aea776a131 fix: Unexpected input(s) 'depth' CI warnings (#33254) 2025-04-28 11:07:13 -06:00
Evan Rusackas d2360b533b fix(histogram): remove extra single quotes (#33248) 2025-04-25 16:45:05 -06:00
Vitor Avila de84a534ac fix(DB update): Gracefully handle querry error during DB update (#33250) 2025-04-25 15:38:59 -03:00
Sam Firke ac636c73ae fix(heatmap): correctly render int and boolean falsy values on axes (#33238) 2025-04-25 11:25:50 -04:00
Levis Mbote 6a586fe4fd fix(chart): Restore subheader used in bignumber with trendline (#33196) 2025-04-25 09:39:07 -03:00
Vitor Avila fbd8ae2888 fix(sqllab permalink): Commit SQL Lab permalinks (#33237) 2025-04-24 22:41:15 -03:00
Vitor Avila 7e4fde7a14 fix(standalone): Ensure correct URL param value for standalone mode (#33234) 2025-04-24 16:41:42 -03:00
Evan Rusackasanddykoffi 150b9a0168 feat(maps): Adding Republic of Serbia to country maps (#33208)
Co-authored-by: dykoffi <dykoffi@users.noreply.github.com>
2025-04-23 11:29:35 -06:00
Vitor Avila f7b7aace38 fix(export): Full CSV/Excel exports respecting SQL_MAX_ROW config (#33214) 2025-04-23 13:13:07 -03:00
Sam Firke f78c94c988 docs(installation): compare installation methods (#33137) 2025-04-23 11:57:33 -04:00
sha174n 74ff8dc724 docs: Add note on SQL execution security considerations (#33210) 2025-04-23 13:58:33 +01:00
Shao Yu-Lung (Allen)andShao Yu-Lung 8aa127eac2 feat(i18n): Frontend add zh_TW Option (#33192)
Co-authored-by: Shao Yu-Lung (Allen) <mis@cendai.com.tw>
2025-04-22 15:36:09 -06:00
KalaiandEvan Rusackas 3729016a0d docs: improve documentation(docs): clarify URL encoding requirement for connection strings (#30047)
Co-authored-by: Evan Rusackas <evan@preset.io>
2025-04-22 15:30:19 -06:00
Elizabeth Thompson b6628cdfd2 chore: migrate to more db migration utils (#33155) 2025-04-22 11:26:54 -07:00
Evan Rusackasanddykoffi ae48dba3e1 feat(maps): Adding Ivory Coast / Côte d'Ivoire (#33198)
Co-authored-by: dykoffi <dykoffi@users.noreply.github.com>
2025-04-22 10:04:19 -06:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 09364d182c chore(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /superset-frontend (#33197)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-22 09:31:10 -06:00
Geido 99ed968289 fix(Native Filters): Keep default filter values when configuring creatable behavior (#33205) 2025-04-22 16:32:30 +02:00
Geido 8fa3b8d7e3 fix(Native Filters): Keep default filter values when configuring creatable behavior (#33205) 2025-04-22 16:30:36 +02:00
Maxime Alay-EddineandMaxime ALAY-EDDINE 7530487760 feat(country-map): fix France Regions IDF region code - Fixes #32627 (#32695)
Co-authored-by: Maxime ALAY-EDDINE <maxime@galeax.com>
2025-04-21 20:15:27 -06:00
Maxime Beauchemin 79afc2b545 docs: add a high-level architecture diagram to the docs (#33173) 2025-04-21 11:15:29 -07:00
JUST.in DO IT 8c94f9c435 fix(sqllab): Invalid SQL Error breaks SQL Lab (#33164) 2025-04-18 13:31:54 -07:00
Evan Rusackas b589d44dfb fix(deckgl): Update Arc to properly adjust line width (#33154) 2025-04-18 10:07:40 -06:00
Elizabeth Thompson 4140261797 fix: subheader should show as subtitle (#33172) 2025-04-18 13:03:20 +08:00
120 changed files with 50627 additions and 26138 deletions
+5
View File
@@ -127,6 +127,11 @@
- any-glob-to-any-file:
- 'superset/translations/es/**'
"i18n:persian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/fa/**'
############################################
# Sub-projects and monorepo packages
############################################
+1 -2
View File
@@ -17,13 +17,12 @@ jobs:
check-python-deps:
runs-on: ubuntu-22.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
depth: 1
fetch-depth: 1
- name: Setup Python
if: steps.check.outputs.python
@@ -41,6 +41,8 @@ jobs:
node-version-file: './docs/.nvmrc'
- name: Setup Python
uses: ./.github/actions/setup-backend/
- name: Update openapi docs
run: superset update_api_docs
- uses: actions/setup-java@v4
with:
distribution: 'zulu'
+2
View File
@@ -86,6 +86,7 @@
"Israel",
"Italy",
"Italy (regions)",
"Ivory Coast",
"Japan",
"Jordan",
"Kazakhstan",
@@ -143,6 +144,7 @@
"Poland",
"Portugal",
"Qatar",
"Republic Of Serbia",
"Romania",
"Russia",
"Rwanda",
+7
View File
@@ -1293,6 +1293,13 @@ The connection string for SQL Server looks like this:
mssql+pyodbc:///?odbc_connect=Driver%3D%7BODBC+Driver+17+for+SQL+Server%7D%3BServer%3Dtcp%3A%3Cmy_server%3E%2C1433%3BDatabase%3Dmy_database%3BUid%3Dmy_user_name%3BPwd%3Dmy_password%3BEncrypt%3Dyes%3BConnection+Timeout%3D30
```
:::note
You might have noticed that some special charecters are used in the above connection string. For example see the `odbc_connect` parameter. The value is `Driver%3D%7BODBC+Driver+17+for+SQL+Server%7D%3B` which is a URL-encoded form of `Driver={ODBC+Driver+17+for+SQL+Server};`. It's important to give the connection string is URL encoded.
For more information about this check the [sqlalchemy documentation](https://docs.sqlalchemy.org/en/20/core/engines.html#escaping-special-characters-such-as-signs-in-passwords). Which says `When constructing a fully formed URL string to pass to create_engine(), special characters such as those that may be used in the user and password need to be URL encoded to be parsed correctly. This includes the @ sign.`
:::
#### StarRocks
The [sqlalchemy-starrocks](https://pypi.org/project/starrocks/) library is the recommended
+86
View File
@@ -4,9 +4,95 @@ version: 1
---
import InteractiveSVG from '../../src/components/InteractiveERDSVG';
import Mermaid from '@theme/Mermaid';
# Resources
## High Level Architecture
<div style={{ maxWidth: "600px", margin: "0 auto", marginLeft: 0, marginRight: "auto" }}>
```mermaid
flowchart TD
%% Top Level
LB["<b>Load Balancer(s)</b><br/>(optional)"]
LB -.-> WebServers
%% Web Servers
subgraph WebServers ["<b>Web Server(s)</b>"]
WS1["<b>Frontend</b><br/>(React, AntD, ECharts, AGGrid)"]
WS2["<b>Backend</b><br/>(Python, Flask, SQLAlchemy, Pandas, ...)"]
end
%% Infra
subgraph InfraServices ["<b>Infra</b>"]
DB[("<b>Metadata Database</b><br/>(Postgres / MySQL)")]
subgraph Caching ["<b>Caching Subservices<br/></b>(Redis, memcache, S3, ...)"]
direction LR
DummySpace[" "]:::invisible
QueryCache["<b>Query Results Cache</b><br/>(Accelerated Dashboards)"]
CsvCache["<b>CSV Exports Cache</b>"]
ThumbnailCache["<b>Thumbnails Cache</b>"]
AlertImageCache["<b>Alert/Report Images Cache</b>"]
QueryCache -- " " --> CsvCache
linkStyle 1 stroke:transparent;
ThumbnailCache -- " " --> AlertImageCache
linkStyle 2 stroke:transparent;
end
Broker(("<b>Message Queue</b><br/>(Redis / RabbitMQ / SQS)"))
end
AsyncBackend["<b>Async Workers (Celery)</b><br>required for Alerts & Reports, thumbnails, CSV exports, long-running workloads, ..."]
%% External DBs
subgraph ExternalDatabases ["<b>Analytics Databases</b>"]
direction LR
BigQuery[(BigQuery)]
Snowflake[(Snowflake)]
Redshift[(Redshift)]
Postgres[(Postgres)]
Postgres[(... any ...)]
end
%% Connections
LB -.-> WebServers
WebServers --> DB
WebServers -.-> Caching
WebServers -.-> Broker
WebServers -.-> ExternalDatabases
Broker -.-> AsyncBackend
AsyncBackend -.-> ExternalDatabases
AsyncBackend -.-> Caching
%% Legend styling
classDef requiredNode stroke-width:2px,stroke:black;
class Required requiredNode;
class Optional optionalNode;
%% Hide real arrow
linkStyle 0 stroke:transparent;
%% Styling
classDef optionalNode stroke-dasharray: 5 5, opacity:0.9;
class LB optionalNode;
class Caching optionalNode;
class AsyncBackend optionalNode;
class Broker optionalNode;
class QueryCache optionalNode;
class CsvCache optionalNode;
class ThumbnailCache optionalNode;
class AlertImageCache optionalNode;
class Celery optionalNode;
classDef invisible fill:transparent,stroke:transparent;
```
</div>
## Entity-Relationship Diagram
Here is our interactive ERD:
+2 -2
View File
@@ -1,7 +1,7 @@
---
title: Docker Builds
hide_title: true
sidebar_position: 6
sidebar_position: 7
version: 1
---
@@ -44,7 +44,7 @@ Here are the build presets that are exposed through the `supersetbot docker` uti
- `py311`, e.g., Py311: Similar to lean but with a different Python version (in this example, 3.11).
- `ci`: For certain CI workloads.
- `websocket`: For Superset clusters supporting advanced features.
- `dockerize`: Used by Helm.
- `dockerize`: Used by Helm in initContainers to wait for database dependencies to be available.
## Key tags examples
+1 -1
View File
@@ -1,7 +1,7 @@
---
title: Docker Compose
hide_title: true
sidebar_position: 4
sidebar_position: 5
version: 1
---
@@ -0,0 +1,58 @@
---
title: Installation Methods
hide_title: true
sidebar_position: 2
version: 1
---
import useBaseUrl from "@docusaurus/useBaseUrl";
# Installation Methods
How should you install Superset? Here's a comparison of the different options. It will help if you've first read the [Architecture](/docs/installation/architecture.mdx) page to understand Superset's different components.
The fundamental trade-off is between you needing to do more of the detail work yourself vs. using a more complex deployment route that handles those details.
## [Docker Compose](/docs/installation/docker-compose.mdx)
**Summary:** This takes advantage of containerization while remaining simpler than Kubernetes. This is the best way to try out Superset; it's also useful for developing & contributing back to Superset.
If you're not just demoing the software, you'll need a moderate understanding of Docker to customize your deployment and avoid a few risks. Even when fully-optimized this is not as robust a method as Kubernetes when it comes to large-scale production deployments.
You manage a superset-config.py file and a docker-compose.yml file. Docker Compose brings up all the needed services - the Superset application, a Postgres metadata DB, Redis cache, Celery worker and beat. They are automatically connected to each other.
**Responsibilities**
You will need to back up your metadata DB. That could mean backing up the service running as a Docker container and its volume; ideally you are running Postgres as a service outside of that container and backing up that service.
You will also need to extend the Superset docker image. The default `lean` images do not contain drivers needed to access your metadata database (Postgres or MySQL), nor to access your data warehouse, nor the headless browser needed for Alerts & Reports. You could run a `-dev` image while demoing Superset, which has some of this, but you'll still need to install the driver for your data warehouse. The `-dev` images run as root, which is not recommended for production.
Ideally you will build your own image of Superset that extends `lean`, adding what your deployment needs.
See [Docker Build Presets](/docs/installation/docker-builds/#build-presets) for more information about the different image versions you can extend.
## [Kubernetes (K8s)](/docs/installation/kubernetes.mdx)
**Summary:** This is the best-practice way to deploy a production instance of Superset, but has the steepest skill requirement - someone who knows Kubernetes.
You will deploy Superset into a K8s cluster. The most common method is using the community-maintained Helm chart, though work is now underway to implement [SIP-149 - a Kubernetes Operator for Superset](https://github.com/apache/superset/issues/31408).
A K8s deployment can scale up and down based on usage and deploy rolling updates with zero downtime - features that big deployments appreciate.
**Responsibilities**
You will need to build your own Docker image, and back up your metadata DB, both as described in Docker Compose above. You'll also need to customize your Helm chart values and deploy and maintain your Kubernetes cluster.
## [PyPI (Python)](/docs/installation/pypi.mdx)
**Summary:** This is the only method that requires no knowledge of containers. It requires the most hands-on work to deploy, connect, and maintain each component.
You install Superset as a Python package and run it that way, providing your own metadata database. Superset has documentation on how to install this way, but it is updated infrequently.
If you want caching, you'll set up Redis or RabbitMQ. If you want Alerts & Reports, you'll set up Celery.
**Responsibilities**
You will need to get the component services running and communicating with each other. You'll need to arrange backups of your metadata database.
When upgrading, you'll need to manage the system environment and packages and ensure all components have functional dependencies.
+1 -1
View File
@@ -1,7 +1,7 @@
---
title: Kubernetes
hide_title: true
sidebar_position: 2
sidebar_position: 3
version: 1
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
title: PyPI
hide_title: true
sidebar_position: 3
sidebar_position: 4
version: 1
---
@@ -1,7 +1,7 @@
---
title: Upgrading Superset
hide_title: true
sidebar_position: 5
sidebar_position: 6
version: 1
---
+20
View File
@@ -64,6 +64,26 @@ tables in the **Permissions** dropdown. To select the data sources you want to a
You can then confirm with users assigned to the **Gamma** role that they see the
objects (dashboards and slices) associated with the tables you just extended them.
### SQL Execution Security Considerations
Apache Superset includes features designed to provide safeguards when interacting with connected databases, such as the `DISALLOWED_SQL_FUNCTIONS` configuration setting. This aims to prevent the execution of potentially harmful database functions or system variables directly from Superset interfaces like SQL Lab.
However, it is crucial to understand the following:
**Superset is Not a Database Firewall**: Superset's built-in checks, like `DISALLOWED_SQL_FUNCTIONS`, provide a layer of protection but cannot guarantee complete security against all database-level threats or advanced bypass techniques (like specific comment injection methods). They should be viewed as a supplement to, not a replacement for, robust database security.
**Configuration is Key**: The effectiveness of Superset's safeguards heavily depends on proper configuration by the Superset administrator. This includes maintaining the `DISALLOWED_SQL_FUNCTIONS` list, carefully managing feature flags (like `ENABLE_TEMPLATE_PROCESSING`), and configuring other security settings appropriately.
**Database Security is Paramount**: The ultimate responsibility for securing database access, controlling permissions, and preventing unauthorized function execution lies with the database administrators (DBAs) and security teams managing the underlying database instance.
**Recommended Database Practices**: We strongly recommend implementing security best practices at the database level, including:
* **Least Privilege**: Connecting Superset using dedicated database user accounts with the minimum permissions required for Superset's operation (typically read-only access to necessary schemas/tables).
* **Database Roles & Permissions**: Utilizing database-native roles and permissions to restrict access to sensitive functions, system variables (like `@@hostname`), schemas, or tables.
* **Network Security**: Employing network-level controls like database firewalls or proxies to restrict connections.
* **Auditing**: Enabling database-level auditing to monitor executed queries and access patterns.
By combining Superset's configurable safeguards with strong database-level security practices, you can achieve a more robust and layered security posture.
### REST API for user & role management
Flask-AppBuilder supports a REST API for user CRUD,
+4 -1
View File
@@ -31,10 +31,13 @@ const config: Config = {
baseUrl: '/',
onBrokenLinks: 'throw',
onBrokenMarkdownLinks: 'throw',
markdown: {
mermaid: true,
},
favicon: '/img/favicon.ico',
organizationName: 'apache',
projectName: 'superset',
themes: ['@saucelabs/theme-github-codeblock'],
themes: ['@saucelabs/theme-github-codeblock', '@docusaurus/theme-mermaid'],
plugins: [
[
'docusaurus-plugin-less',
+10 -9
View File
@@ -19,22 +19,23 @@
},
"dependencies": {
"@ant-design/icons": "^5.5.2",
"@docusaurus/core": "^3.5.2",
"@docusaurus/plugin-client-redirects": "^3.5.2",
"@docusaurus/preset-classic": "^3.5.2",
"@docusaurus/core": "3.7.0",
"@docusaurus/plugin-client-redirects": "3.7.0",
"@docusaurus/preset-classic": "3.7.0",
"@docusaurus/theme-mermaid": "3.7.0",
"@emotion/styled": "^10.0.27",
"@saucelabs/theme-github-codeblock": "^0.3.0",
"@superset-ui/style": "^0.14.23",
"antd": "^5.24.5",
"docusaurus-plugin-less": "^2.0.2",
"less": "^4.2.2",
"less": "^4.3.0",
"less-loader": "^11.0.0",
"prism-react-renderer": "^2.4.1",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-github-btn": "^1.4.0",
"react-svg-pan-zoom": "^3.13.1",
"swagger-ui-react": "^5.20.2"
"swagger-ui-react": "^5.21.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.7.0",
@@ -43,12 +44,12 @@
"@typescript-eslint/eslint-plugin": "^5.0.0",
"@typescript-eslint/parser": "^5.0.0",
"eslint": "^8.0.0",
"eslint-config-prettier": "^10.1.1",
"eslint-config-prettier": "^10.1.2",
"eslint-plugin-prettier": "^4.0.0",
"eslint-plugin-react": "^7.0.0",
"eslint-plugin-react": "^7.37.5",
"prettier": "^2.0.0",
"typescript": "~5.8.2",
"webpack": "^5.98.0"
"typescript": "~5.8.3",
"webpack": "^5.99.7"
},
"browserslist": {
"production": [
+1 -1
View File
@@ -111,7 +111,7 @@ const StyledTitleContainer = styled('div')`
}
`;
const StyledButton = styled(Link)`
const StyledButton = styled(Link as React.ComponentType<any>)`
border-radius: 10px;
font-size: 20px;
font-weight: bold;
+4222 -2067
View File
File diff suppressed because it is too large Load Diff
+4141 -2825
View File
File diff suppressed because it is too large Load Diff
+4 -2
View File
@@ -44,7 +44,7 @@ dependencies = [
"cryptography>=42.0.4, <45.0.0",
"deprecation>=2.1.0, <2.2.0",
"flask>=2.2.5, <3.0.0",
"flask-appbuilder>=4.6.1, <5.0.0",
"flask-appbuilder>=4.6.3, <5.0.0",
"flask-caching>=2.1.0, <3",
"flask-compress>=1.13, <2.0",
"flask-talisman>=1.0.0, <2.0",
@@ -81,7 +81,7 @@ dependencies = [
"python-dateutil",
"python-dotenv", # optional dependencies for Flask but required for Superset, see https://flask.palletsprojects.com/en/stable/installation/#optional-dependencies
"python-geohash",
"pyarrow>=14.0.1, <15",
"pyarrow>=18.1.0, <19",
"pyyaml>=6.0.0, <7.0.0",
"PyJWT>=2.4.0, <3.0",
"redis>=4.6.0, <5.0",
@@ -371,12 +371,14 @@ authorized_licenses = [
"apache software",
"apache software, bsd",
"bsd",
"bsd-3-clause",
"isc license (iscl)",
"isc license",
"mit",
"mozilla public license 2.0 (mpl 2.0)",
"osi approved",
"osi approved",
"psf-2.0",
"python software foundation",
"the unlicense (unlicense)",
"the unlicense",
+19 -20
View File
@@ -44,7 +44,7 @@ cachetools==5.5.2
# via google-auth
cattrs==24.1.2
# via requests-cache
celery==5.4.0
celery==5.5.2
# via apache-superset (pyproject.toml)
certifi==2025.1.31
# via
@@ -82,7 +82,7 @@ cron-descriptor==1.4.5
# via apache-superset (pyproject.toml)
croniter==6.0.0
# via apache-superset (pyproject.toml)
cryptography==44.0.2
cryptography==44.0.3
# via
# apache-superset (pyproject.toml)
# paramiko
@@ -118,7 +118,7 @@ flask==2.3.3
# flask-session
# flask-sqlalchemy
# flask-wtf
flask-appbuilder==4.6.1
flask-appbuilder==4.6.3
# via apache-superset (pyproject.toml)
flask-babel==2.0.0
# via flask-appbuilder
@@ -158,22 +158,23 @@ greenlet==3.1.1
# via
# apache-superset (pyproject.toml)
# shillelagh
# sqlalchemy
gunicorn==23.0.0
# via apache-superset (pyproject.toml)
h11==0.14.0
h11==0.16.0
# via wsproto
hashids==1.3.1
# via apache-superset (pyproject.toml)
holidays==0.25
# via apache-superset (pyproject.toml)
humanize==4.12.2
humanize==4.12.3
# via apache-superset (pyproject.toml)
idna==3.10
# via
# email-validator
# requests
# trio
importlib-metadata==8.6.1
importlib-metadata==8.7.0
# via apache-superset (pyproject.toml)
isodate==0.7.2
# via apache-superset (pyproject.toml)
@@ -191,17 +192,17 @@ jsonschema==4.23.0
# via flask-appbuilder
jsonschema-specifications==2024.10.1
# via jsonschema
kombu==5.5.0
kombu==5.5.3
# via celery
korean-lunar-calendar==0.3.1
# via holidays
limits==4.4.1
limits==5.1.0
# via flask-limiter
mako==1.3.9
mako==1.3.10
# via
# apache-superset (pyproject.toml)
# alembic
markdown==3.7
markdown==3.8
# via apache-superset (pyproject.toml)
markdown-it-py==3.0.0
# via rich
@@ -235,7 +236,6 @@ numpy==1.26.4
# bottleneck
# numexpr
# pandas
# pyarrow
odfpy==1.4.1
# via pandas
openpyxl==3.1.5
@@ -244,7 +244,7 @@ ordered-set==4.1.0
# via flask-limiter
outcome==1.3.0.post0
# via trio
packaging==24.2
packaging==25.0
# via
# apache-superset (pyproject.toml)
# apispec
@@ -271,9 +271,9 @@ polyline==2.0.2
# via apache-superset (pyproject.toml)
prison==0.2.1
# via flask-appbuilder
prompt-toolkit==3.0.50
prompt-toolkit==3.0.51
# via click-repl
pyarrow==14.0.2
pyarrow==18.1.0
# via apache-superset (pyproject.toml)
pyasn1==0.6.1
# via
@@ -294,7 +294,7 @@ pynacl==1.5.0
# via paramiko
pyopenssl==25.0.0
# via shillelagh
pyparsing==3.2.2
pyparsing==3.2.3
# via apache-superset (pyproject.toml)
pysocks==1.7.1
# via urllib3
@@ -307,11 +307,11 @@ python-dateutil==2.9.0.post0
# holidays
# pandas
# shillelagh
python-dotenv==1.0.1
python-dotenv==1.1.0
# via apache-superset (pyproject.toml)
python-geohash==0.8.5
# via apache-superset (pyproject.toml)
pytz==2025.1
pytz==2025.2
# via
# croniter
# flask-babel
@@ -373,7 +373,7 @@ sqlalchemy-utils==0.38.3
# via
# apache-superset (pyproject.toml)
# flask-appbuilder
sqlglot==26.11.1
sqlglot==26.16.4
# via apache-superset (pyproject.toml)
sqlparse==0.5.3
# via apache-superset (pyproject.toml)
@@ -398,9 +398,8 @@ typing-extensions==4.12.2
# rich
# selenium
# shillelagh
tzdata==2025.1
tzdata==2025.2
# via
# celery
# kombu
# pandas
url-normalize==1.4.3
+19 -20
View File
@@ -72,7 +72,7 @@ cattrs==24.1.2
# via
# -c requirements/base.txt
# requests-cache
celery==5.4.0
celery==5.5.2
# via
# -c requirements/base.txt
# apache-superset
@@ -138,7 +138,7 @@ croniter==6.0.0
# via
# -c requirements/base.txt
# apache-superset
cryptography==44.0.2
cryptography==44.0.3
# via
# -c requirements/base.txt
# apache-superset
@@ -202,7 +202,7 @@ flask==2.3.3
# flask-sqlalchemy
# flask-testing
# flask-wtf
flask-appbuilder==4.6.1
flask-appbuilder==4.6.3
# via
# -c requirements/base.txt
# apache-superset
@@ -318,6 +318,7 @@ greenlet==3.1.1
# apache-superset
# gevent
# shillelagh
# sqlalchemy
grpcio==1.71.0
# via
# apache-superset
@@ -329,7 +330,7 @@ gunicorn==23.0.0
# via
# -c requirements/base.txt
# apache-superset
h11==0.14.0
h11==0.16.0
# via
# -c requirements/base.txt
# wsproto
@@ -342,7 +343,7 @@ holidays==0.25
# -c requirements/base.txt
# apache-superset
# prophet
humanize==4.12.2
humanize==4.12.3
# via
# -c requirements/base.txt
# apache-superset
@@ -354,7 +355,7 @@ idna==3.10
# email-validator
# requests
# trio
importlib-metadata==8.6.1
importlib-metadata==8.7.0
# via
# -c requirements/base.txt
# apache-superset
@@ -395,7 +396,7 @@ jsonschema-specifications==2024.10.1
# openapi-schema-validator
kiwisolver==1.4.7
# via matplotlib
kombu==5.5.0
kombu==5.5.3
# via
# -c requirements/base.txt
# celery
@@ -405,16 +406,16 @@ korean-lunar-calendar==0.3.1
# holidays
lazy-object-proxy==1.10.0
# via openapi-spec-validator
limits==4.4.1
limits==5.1.0
# via
# -c requirements/base.txt
# flask-limiter
mako==1.3.9
mako==1.3.10
# via
# -c requirements/base.txt
# alembic
# apache-superset
markdown==3.7
markdown==3.8
# via
# -c requirements/base.txt
# apache-superset
@@ -472,7 +473,6 @@ numpy==1.26.4
# pandas
# pandas-gbq
# prophet
# pyarrow
oauthlib==3.2.2
# via requests-oauthlib
odfpy==1.4.1
@@ -495,7 +495,7 @@ outcome==1.3.0.post0
# via
# -c requirements/base.txt
# trio
packaging==24.2
packaging==25.0
# via
# -c requirements/base.txt
# apache-superset
@@ -565,7 +565,7 @@ prison==0.2.1
# flask-appbuilder
progress==1.6
# via apache-superset
prompt-toolkit==3.0.50
prompt-toolkit==3.0.51
# via
# -c requirements/base.txt
# click-repl
@@ -586,7 +586,7 @@ psutil==6.1.0
# via apache-superset
psycopg2-binary==2.9.6
# via apache-superset
pyarrow==14.0.2
pyarrow==18.1.0
# via
# -c requirements/base.txt
# apache-superset
@@ -635,7 +635,7 @@ pyopenssl==25.0.0
# via
# -c requirements/base.txt
# shillelagh
pyparsing==3.2.2
pyparsing==3.2.3
# via
# -c requirements/base.txt
# apache-superset
@@ -668,7 +668,7 @@ python-dateutil==2.9.0.post0
# pyhive
# shillelagh
# trino
python-dotenv==1.0.1
python-dotenv==1.1.0
# via
# -c requirements/base.txt
# apache-superset
@@ -678,7 +678,7 @@ python-geohash==0.8.5
# apache-superset
python-ldap==3.4.4
# via apache-superset
pytz==2025.1
pytz==2025.2
# via
# -c requirements/base.txt
# croniter
@@ -799,7 +799,7 @@ sqlalchemy-utils==0.38.3
# -c requirements/base.txt
# apache-superset
# flask-appbuilder
sqlglot==26.11.1
sqlglot==26.16.4
# via
# -c requirements/base.txt
# apache-superset
@@ -850,10 +850,9 @@ typing-extensions==4.12.2
# rich
# selenium
# shillelagh
tzdata==2025.1
tzdata==2025.2
# via
# -c requirements/base.txt
# celery
# kombu
# pandas
tzlocal==5.2
@@ -56,7 +56,6 @@ describe('Visualization > Big Number with Trendline', () => {
it('should work', () => {
verify(BIG_NUMBER_FORM_DATA);
cy.get('.chart-container .header-line');
cy.get('.chart-container .subheader-line');
cy.get('.chart-container canvas');
});
@@ -66,7 +65,7 @@ describe('Visualization > Big Number with Trendline', () => {
compare_lag: null,
});
cy.get('.chart-container .header-line');
cy.get('.chart-container .subheader-line').should('not.exist');
cy.get('.chart-container .subtitle-line').should('not.exist');
cy.get('.chart-container canvas');
});
@@ -76,7 +75,6 @@ describe('Visualization > Big Number with Trendline', () => {
show_trend_line: false,
});
cy.get('[data-test="chart-container"] .header-line');
cy.get('[data-test="chart-container"] .subheader-line');
cy.get('[data-test="chart-container"] canvas').should('not.exist');
});
});
+79 -65
View File
@@ -23,7 +23,7 @@
"@reduxjs/toolkit": "^1.9.3",
"@rjsf/core": "^5.21.1",
"@rjsf/utils": "^5.24.3",
"@rjsf/validator-ajv8": "^5.22.3",
"@rjsf/validator-ajv8": "^5.24.9",
"@scarf/scarf": "^1.4.0",
"@superset-ui/chart-controls": "file:./packages/superset-ui-chart-controls",
"@superset-ui/core": "file:./packages/superset-ui-core",
@@ -115,7 +115,7 @@
"react-dom": "^17.0.2",
"react-draggable": "^4.4.6",
"react-hot-loader": "^4.13.1",
"react-intersection-observer": "^9.10.2",
"react-intersection-observer": "^9.16.0",
"react-js-cron": "^2.1.2",
"react-json-tree": "^0.17.0",
"react-lines-ellipsis": "^0.15.4",
@@ -162,7 +162,7 @@
"@babel/plugin-proposal-private-methods": "^7.18.6",
"@babel/plugin-syntax-dynamic-import": "^7.8.3",
"@babel/plugin-transform-modules-commonjs": "^7.26.3",
"@babel/plugin-transform-runtime": "^7.25.9",
"@babel/plugin-transform-runtime": "^7.27.1",
"@babel/preset-env": "^7.26.7",
"@babel/preset-react": "^7.26.3",
"@babel/preset-typescript": "^7.26.0",
@@ -1146,14 +1146,14 @@
}
},
"node_modules/@babel/code-frame": {
"version": "7.26.2",
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.26.2.tgz",
"integrity": "sha512-RJlIHRueQgwWitWgF8OdFYGZX328Ax5BCemNGlqHfplnRT9ESi8JkFlvaVYbS+UubVY6dpv87Fs2u5M29iNFVQ==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.27.1.tgz",
"integrity": "sha512-cjQ7ZlQ0Mv3b47hABuTevyTuYN4i+loJKGeV9flcCgIK37cCXRh+L1bd3iBHlynerhQ7BhCkn2BPbQUL+rGqFg==",
"license": "MIT",
"dependencies": {
"@babel/helper-validator-identifier": "^7.25.9",
"@babel/helper-validator-identifier": "^7.27.1",
"js-tokens": "^4.0.0",
"picocolors": "^1.0.0"
"picocolors": "^1.1.1"
},
"engines": {
"node": ">=6.9.0"
@@ -1240,13 +1240,13 @@
}
},
"node_modules/@babel/generator": {
"version": "7.26.5",
"resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.26.5.tgz",
"integrity": "sha512-2caSP6fN9I7HOe6nqhtft7V4g7/V/gfDsC3Ag4W7kEzzvRGKqiv0pu0HogPiZ3KaVSoNDhUws6IJjDjpfmYIXw==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/generator/-/generator-7.27.1.tgz",
"integrity": "sha512-UnJfnIpc/+JO0/+KRVQNGU+y5taA5vCbwN8+azkX6beii/ZF+enZJSOKo11ZSzGJjlNfJHfQtmQT8H+9TXPG2w==",
"license": "MIT",
"dependencies": {
"@babel/parser": "^7.26.5",
"@babel/types": "^7.26.5",
"@babel/parser": "^7.27.1",
"@babel/types": "^7.27.1",
"@jridgewell/gen-mapping": "^0.3.5",
"@jridgewell/trace-mapping": "^0.3.25",
"jsesc": "^3.0.2"
@@ -1387,13 +1387,13 @@
}
},
"node_modules/@babel/helper-module-imports": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/helper-module-imports/-/helper-module-imports-7.25.9.tgz",
"integrity": "sha512-tnUA4RsrmflIM6W6RFTLFSXITtl0wKjgpnLgXyowocVPrbYrLUXSBXDgTs8BlbmIzIdlBySRQjINYs2BAkiLtw==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/helper-module-imports/-/helper-module-imports-7.27.1.tgz",
"integrity": "sha512-0gSFWUPNXNopqtIPQvlD5WgXYI5GY2kP2cCvoT8kczjbfcfuIljTbcWrulD1CIPIX2gt1wghbDy08yE1p+/r3w==",
"license": "MIT",
"dependencies": {
"@babel/traverse": "^7.25.9",
"@babel/types": "^7.25.9"
"@babel/traverse": "^7.27.1",
"@babel/types": "^7.27.1"
},
"engines": {
"node": ">=6.9.0"
@@ -1431,9 +1431,9 @@
}
},
"node_modules/@babel/helper-plugin-utils": {
"version": "7.26.5",
"resolved": "https://registry.npmjs.org/@babel/helper-plugin-utils/-/helper-plugin-utils-7.26.5.tgz",
"integrity": "sha512-RS+jZcRdZdRFzMyr+wcsaqOmld1/EqTghfaBGQQd/WnRdzdlvSZ//kF7U8VQTxf1ynZ4cjUcYgjVGx13ewNPMg==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/helper-plugin-utils/-/helper-plugin-utils-7.27.1.tgz",
"integrity": "sha512-1gn1Up5YXka3YYAHGKpbideQ5Yjf1tDa9qYcgysz+cNCXukyLl6DjPXhD3VRwSb8c0J9tA4b2+rHEZtc6R0tlw==",
"dev": true,
"license": "MIT",
"engines": {
@@ -1491,18 +1491,18 @@
}
},
"node_modules/@babel/helper-string-parser": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.25.9.tgz",
"integrity": "sha512-4A/SCr/2KLd5jrtOMFzaKjVtAei3+2r/NChoBNoZ3EyP/+GlhoaEGoWOZUmFmoITP7zOJyHIMm+DYRd8o3PvHA==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.27.1.tgz",
"integrity": "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA==",
"license": "MIT",
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/helper-validator-identifier": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.25.9.tgz",
"integrity": "sha512-Ed61U6XJc3CVRfkERJWDz4dJwKe7iLmmJsbOGu9wSloNSFttHV0I8g6UAgb7qnK5ly5bGLPd4oXZlxCdANBOWQ==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.27.1.tgz",
"integrity": "sha512-D2hP9eA+Sqx1kBZgzxZh0y1trbuU+JoDkiEwqhQ36nodYqJwyEIhPSdMNd7lOm/4io72luTPWH20Yda0xOuUow==",
"license": "MIT",
"engines": {
"node": ">=6.9.0"
@@ -1572,12 +1572,12 @@
}
},
"node_modules/@babel/parser": {
"version": "7.26.7",
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.26.7.tgz",
"integrity": "sha512-kEvgGGgEjRUutvdVvZhbn/BxVt+5VSpwXz1j3WYXQbXDo8KzFOPNG2GQbdAiNq8g6wn1yKk7C/qrke03a84V+w==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.27.1.tgz",
"integrity": "sha512-I0dZ3ZpCrJ1c04OqlNsQcKiZlsrXf/kkE4FXzID9rIOYICsAbA8mMDzhW/luRNAHdCNt7os/u8wenklZDlUVUQ==",
"license": "MIT",
"dependencies": {
"@babel/types": "^7.26.7"
"@babel/types": "^7.27.1"
},
"bin": {
"parser": "bin/babel-parser.js"
@@ -2849,16 +2849,16 @@
}
},
"node_modules/@babel/plugin-transform-runtime": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/plugin-transform-runtime/-/plugin-transform-runtime-7.25.9.tgz",
"integrity": "sha512-nZp7GlEl+yULJrClz0SwHPqir3lc0zsPrDHQUcxGspSL7AKrexNSEfTbfqnDNJUO13bgKyfuOLMF8Xqtu8j3YQ==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/plugin-transform-runtime/-/plugin-transform-runtime-7.27.1.tgz",
"integrity": "sha512-TqGF3desVsTcp3WrJGj4HfKokfCXCLcHpt4PJF0D8/iT6LPd9RS82Upw3KPeyr6B22Lfd3DO8MVrmp0oRkUDdw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/helper-module-imports": "^7.25.9",
"@babel/helper-plugin-utils": "^7.25.9",
"@babel/helper-module-imports": "^7.27.1",
"@babel/helper-plugin-utils": "^7.27.1",
"babel-plugin-polyfill-corejs2": "^0.4.10",
"babel-plugin-polyfill-corejs3": "^0.10.6",
"babel-plugin-polyfill-corejs3": "^0.11.0",
"babel-plugin-polyfill-regenerator": "^0.6.1",
"semver": "^6.3.1"
},
@@ -2869,6 +2869,20 @@
"@babel/core": "^7.0.0-0"
}
},
"node_modules/@babel/plugin-transform-runtime/node_modules/babel-plugin-polyfill-corejs3": {
"version": "0.11.1",
"resolved": "https://registry.npmjs.org/babel-plugin-polyfill-corejs3/-/babel-plugin-polyfill-corejs3-0.11.1.tgz",
"integrity": "sha512-yGCqvBT4rwMczo28xkH/noxJ6MZ4nJfkVYdoDaC/utLtWrXxv27HVrzAeSbqR8SxDsp46n0YF47EbHoixy6rXQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/helper-define-polyfill-provider": "^0.6.3",
"core-js-compat": "^3.40.0"
},
"peerDependencies": {
"@babel/core": "^7.4.0 || ^8.0.0-0 <8.0.0"
}
},
"node_modules/@babel/plugin-transform-runtime/node_modules/semver": {
"version": "6.3.1",
"resolved": "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz",
@@ -3279,30 +3293,30 @@
}
},
"node_modules/@babel/template": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/template/-/template-7.25.9.tgz",
"integrity": "sha512-9DGttpmPvIxBb/2uwpVo3dqJ+O6RooAFOS+lB+xDqoE2PVCE8nfoHMdZLpfCQRLwvohzXISPZcgxt80xLfsuwg==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/template/-/template-7.27.1.tgz",
"integrity": "sha512-Fyo3ghWMqkHHpHQCoBs2VnYjR4iWFFjguTDEqA5WgZDOrFesVjMhMM2FSqTKSoUSDO1VQtavj8NFpdRBEvJTtg==",
"license": "MIT",
"dependencies": {
"@babel/code-frame": "^7.25.9",
"@babel/parser": "^7.25.9",
"@babel/types": "^7.25.9"
"@babel/code-frame": "^7.27.1",
"@babel/parser": "^7.27.1",
"@babel/types": "^7.27.1"
},
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/traverse": {
"version": "7.26.7",
"resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.26.7.tgz",
"integrity": "sha512-1x1sgeyRLC3r5fQOM0/xtQKsYjyxmFjaOrLJNtZ81inNjyJHGIolTULPiSc/2qe1/qfpFLisLQYFnnZl7QoedA==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/traverse/-/traverse-7.27.1.tgz",
"integrity": "sha512-ZCYtZciz1IWJB4U61UPu4KEaqyfj+r5T1Q5mqPo+IBpcG9kHv30Z0aD8LXPgC1trYa6rK0orRyAhqUgk4MjmEg==",
"license": "MIT",
"dependencies": {
"@babel/code-frame": "^7.26.2",
"@babel/generator": "^7.26.5",
"@babel/parser": "^7.26.7",
"@babel/template": "^7.25.9",
"@babel/types": "^7.26.7",
"@babel/code-frame": "^7.27.1",
"@babel/generator": "^7.27.1",
"@babel/parser": "^7.27.1",
"@babel/template": "^7.27.1",
"@babel/types": "^7.27.1",
"debug": "^4.3.1",
"globals": "^11.1.0"
},
@@ -3311,13 +3325,13 @@
}
},
"node_modules/@babel/types": {
"version": "7.26.9",
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.26.9.tgz",
"integrity": "sha512-Y3IR1cRnOxOCDvMmNiym7XpXQ93iGDDPHx+Zj+NM+rg0fBaShfQLkg+hKPaZCEvg5N/LeCo4+Rj/i3FuJsIQaw==",
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/types/-/types-7.27.1.tgz",
"integrity": "sha512-+EzkxvLNfiUeKMgy/3luqfsCWFRXLb7U6wNQTk60tovuckwB15B191tJWvpp4HjiQWdJkCxO3Wbvc6jlk3Xb2Q==",
"license": "MIT",
"dependencies": {
"@babel/helper-string-parser": "^7.25.9",
"@babel/helper-validator-identifier": "^7.25.9"
"@babel/helper-string-parser": "^7.27.1",
"@babel/helper-validator-identifier": "^7.27.1"
},
"engines": {
"node": ">=6.9.0"
@@ -8498,9 +8512,9 @@
}
},
"node_modules/@rjsf/validator-ajv8": {
"version": "5.24.1",
"resolved": "https://registry.npmjs.org/@rjsf/validator-ajv8/-/validator-ajv8-5.24.1.tgz",
"integrity": "sha512-p6URehglU9yFUAoQXE1ryqZjLYSjc6qdbiUfCVvEFAzUuMECsIFomz2hH3CPlt10K72sAFdzwVvrKn1iWTnxDw==",
"version": "5.24.9",
"resolved": "https://registry.npmjs.org/@rjsf/validator-ajv8/-/validator-ajv8-5.24.9.tgz",
"integrity": "sha512-leHb39Qa612QhAfvw36qi/ubWa7LQ6hrPN4Ge93QBlWywRfV/M0Wmx9bPccCGgIL4Qnn1Wmt53EWV8kQT28xTA==",
"license": "Apache-2.0",
"dependencies": {
"ajv": "^8.12.0",
@@ -26682,9 +26696,9 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.7.tgz",
"integrity": "sha512-fgVY8AV7qU7z/MmXJ/rxwbrtQH4jBQ9m7kp3llF0liB7glmFeVZFBepQb32T3y8n8k2+AEYuMPCpinYW+/CuRA==",
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -39476,9 +39490,9 @@
}
},
"node_modules/react-intersection-observer": {
"version": "9.15.1",
"resolved": "https://registry.npmjs.org/react-intersection-observer/-/react-intersection-observer-9.15.1.tgz",
"integrity": "sha512-vGrqYEVWXfH+AGu241uzfUpNK4HAdhCkSAyFdkMb9VWWXs6mxzBLpWCxEy9YcnDNY2g9eO6z7qUtTBdA9hc8pA==",
"version": "9.16.0",
"resolved": "https://registry.npmjs.org/react-intersection-observer/-/react-intersection-observer-9.16.0.tgz",
"integrity": "sha512-w9nJSEp+DrW9KmQmeWHQyfaP6b03v+TdXynaoA964Wxt7mdR3An11z4NNCQgL4gKSK7y1ver2Fq+JKH6CWEzUA==",
"license": "MIT",
"peerDependencies": {
"react": "^17.0.0 || ^18.0.0 || ^19.0.0",
+3 -3
View File
@@ -90,7 +90,7 @@
"@reduxjs/toolkit": "^1.9.3",
"@rjsf/core": "^5.21.1",
"@rjsf/utils": "^5.24.3",
"@rjsf/validator-ajv8": "^5.22.3",
"@rjsf/validator-ajv8": "^5.24.9",
"@scarf/scarf": "^1.4.0",
"@superset-ui/chart-controls": "file:./packages/superset-ui-chart-controls",
"@superset-ui/core": "file:./packages/superset-ui-core",
@@ -182,7 +182,7 @@
"react-dom": "^17.0.2",
"react-draggable": "^4.4.6",
"react-hot-loader": "^4.13.1",
"react-intersection-observer": "^9.10.2",
"react-intersection-observer": "^9.16.0",
"react-js-cron": "^2.1.2",
"react-json-tree": "^0.17.0",
"react-lines-ellipsis": "^0.15.4",
@@ -229,7 +229,7 @@
"@babel/plugin-proposal-private-methods": "^7.18.6",
"@babel/plugin-syntax-dynamic-import": "^7.8.3",
"@babel/plugin-transform-modules-commonjs": "^7.26.3",
"@babel/plugin-transform-runtime": "^7.25.9",
"@babel/plugin-transform-runtime": "^7.27.1",
"@babel/preset-env": "^7.26.7",
"@babel/preset-react": "^7.26.3",
"@babel/preset-typescript": "^7.26.0",
@@ -21,7 +21,6 @@ import { Dataset } from './types';
export const TestDataset: Dataset = {
column_formats: {},
currency_formats: {},
columns: [
{
advanced_data_type: undefined,
@@ -26,6 +26,7 @@ import {
} from '@superset-ui/core';
import { PostProcessingFactory } from './types';
import { getMetricOffsetsMap, isTimeComparison } from './utils';
import { TIME_COMPARISON_SEPARATOR } from './utils/constants';
export const renameOperator: PostProcessingFactory<PostProcessingRename> = (
formData,
@@ -37,50 +38,60 @@ export const renameOperator: PostProcessingFactory<PostProcessingRename> = (
);
const { truncate_metric } = formData;
const xAxisLabel = getXAxisLabel(formData);
const isTimeComparisonValue = isTimeComparison(formData, queryObject);
// remove or rename top level of column name(metric name) in the MultiIndex when
// 1) only 1 metric
// 1) at least 1 metric
// 2) dimension exist
// 3) xAxis exist
// 4) time comparison exist, and comparison type is "actual values"
// 5) truncate_metric in form_data and truncate_metric is true
// 4) truncate_metric in form_data and truncate_metric is true
if (
metrics.length === 1 &&
metrics.length > 0 &&
columns.length > 0 &&
xAxisLabel &&
!(
// todo: we should provide an approach to handle derived metrics
(
isTimeComparison(formData, queryObject) &&
[
ComparisonType.Difference,
ComparisonType.Ratio,
ComparisonType.Percentage,
].includes(formData.comparison_type)
)
) &&
truncate_metric !== undefined &&
!!truncate_metric
) {
const renamePairs: [string, string | null][] = [];
if (
// "actual values" will add derived metric.
// we will rename the "metric" from the metricWithOffset label
// for example: "count__1 year ago" => "1 year ago"
isTimeComparison(formData, queryObject) &&
formData.comparison_type === ComparisonType.Values
isTimeComparisonValue
) {
const metricOffsetMap = getMetricOffsetsMap(formData, queryObject);
const timeOffsets = ensureIsArray(formData.time_compare);
[...metricOffsetMap.keys()].forEach(metricWithOffset => {
const offsetLabel = timeOffsets.find(offset =>
metricWithOffset.includes(offset),
);
renamePairs.push([metricWithOffset, offsetLabel]);
});
[...metricOffsetMap.entries()].forEach(
([metricWithOffset, metricOnly]) => {
const offsetLabel = timeOffsets.find(offset =>
metricWithOffset.includes(offset),
);
renamePairs.push([
formData.comparison_type === ComparisonType.Values
? metricWithOffset
: [formData.comparison_type, metricOnly, metricWithOffset].join(
TIME_COMPARISON_SEPARATOR,
),
metrics.length > 1 ? `${metricOnly}, ${offsetLabel}` : offsetLabel,
]);
},
);
}
renamePairs.push([getMetricLabel(metrics[0]), null]);
if (
![
ComparisonType.Difference,
ComparisonType.Percentage,
ComparisonType.Ratio,
].includes(formData.comparison_type) &&
metrics.length === 1
) {
renamePairs.push([getMetricLabel(metrics[0]), null]);
}
if (renamePairs.length === 0) {
return undefined;
}
return {
operation: 'rename',
@@ -69,7 +69,6 @@ export interface Dataset {
columns: ColumnMeta[];
metrics: Metric[];
column_formats: Record<string, string>;
currency_formats: Record<string, Currency>;
verbose_map: Record<string, string>;
main_dttm_col: string;
// eg. ['["ds", true]', 'ds [asc]']
@@ -43,12 +43,12 @@ const queryObject: QueryObject = {
post_processing: [],
};
test('should skip renameOperator if exists multiple metrics', () => {
test('should skip renameOperator for empty metrics', () => {
expect(
renameOperator(formData, {
...queryObject,
...{
metrics: ['count(*)', 'sum(sales)'],
metrics: [],
},
}),
).toEqual(undefined);
@@ -77,7 +77,23 @@ test('should skip renameOperator if does not exist x_axis and is_timeseries', ()
).toEqual(undefined);
});
test('should skip renameOperator if exists derived metrics', () => {
test('should skip renameOperator if not is_timeseries and multi metrics', () => {
expect(
renameOperator(formData, {
...queryObject,
...{ is_timeseries: false, metrics: ['count(*)', 'sum(val)'] },
}),
).toEqual(undefined);
});
test('should add renameOperator', () => {
expect(renameOperator(formData, queryObject)).toEqual({
operation: 'rename',
options: { columns: { 'count(*)': null }, inplace: true, level: 0 },
});
});
test('should add renameOperator if exists derived metrics', () => {
[
ComparisonType.Difference,
ComparisonType.Ratio,
@@ -99,14 +115,14 @@ test('should skip renameOperator if exists derived metrics', () => {
},
},
),
).toEqual(undefined);
});
});
test('should add renameOperator', () => {
expect(renameOperator(formData, queryObject)).toEqual({
operation: 'rename',
options: { columns: { 'count(*)': null }, inplace: true, level: 0 },
).toEqual({
operation: 'rename',
options: {
columns: { [`${type}__count(*)__count(*)__1 year ago`]: '1 year ago' },
inplace: true,
level: 0,
},
});
});
});
@@ -170,6 +186,61 @@ test('should add renameOperator if exist "actual value" time comparison', () =>
});
});
test('should add renameOperator if derived time comparison exists', () => {
expect(
renameOperator(
{
...formData,
...{
comparison_type: ComparisonType.Ratio,
time_compare: ['1 year ago', '1 year later'],
},
},
queryObject,
),
).toEqual({
operation: 'rename',
options: {
columns: {
'ratio__count(*)__count(*)__1 year ago': '1 year ago',
'ratio__count(*)__count(*)__1 year later': '1 year later',
},
inplace: true,
level: 0,
},
});
});
test('should add renameOperator if multiple metrics exist', () => {
expect(
renameOperator(
{
...formData,
...{
comparison_type: ComparisonType.Values,
time_compare: ['1 year ago'],
},
},
{
...queryObject,
...{
metrics: ['count(*)', 'sum(sales)'],
},
},
),
).toEqual({
operation: 'rename',
options: {
columns: {
'count(*)__1 year ago': 'count(*), 1 year ago',
'sum(sales)__1 year ago': 'sum(sales), 1 year ago',
},
inplace: true,
level: 0,
},
});
});
test('should remove renameOperator', () => {
expect(
renameOperator(
@@ -53,7 +53,6 @@ describe('columnChoices()', () => {
],
verbose_map: {},
column_formats: { fiz: 'NUMERIC', about: 'STRING', foo: 'DATE' },
currency_formats: {},
datasource_name: 'my_datasource',
description: 'this is my datasource',
}),
@@ -105,7 +104,6 @@ describe('columnChoices()', () => {
],
verbose_map: {},
column_formats: { fiz: 'NUMERIC', about: 'STRING', foo: 'DATE' },
currency_formats: {},
datasource_name: 'my_datasource',
description: 'this is my datasource',
}),
@@ -41,7 +41,6 @@ describe('defineSavedMetrics', () => {
columns: [],
verbose_map: {},
column_formats: {},
currency_formats: {},
datasource_name: 'my_datasource',
description: 'this is my datasource',
};
@@ -35,7 +35,8 @@ export type Locale =
| 'pt'
| 'pt_BR'
| 'ru'
| 'zh'; // supported locales in Superset
| 'zh'
| 'zh_TW'; // supported locales in Superset
/**
* Language pack provided to `jed`.
@@ -60,7 +60,7 @@
"@storybook/react-webpack5": "8.2.9",
"babel-loader": "^9.1.3",
"fork-ts-checker-webpack-plugin": "^9.0.2",
"ts-loader": "^9.5.1",
"ts-loader": "^9.5.2",
"typescript": "^5.7.2"
},
"peerDependencies": {
File diff suppressed because one or more lines are too long
@@ -103,6 +103,7 @@ import iran from './countries/iran.geojson';
import israel from './countries/israel.geojson';
import italy from './countries/italy.geojson';
import italy_regions from './countries/italy_regions.geojson';
import ivory_coast from './countries/ivory_coast.geojson';
import japan from './countries/japan.geojson';
import jordan from './countries/jordan.geojson';
import kazakhstan from './countries/kazakhstan.geojson';
@@ -160,6 +161,7 @@ import philippines_regions from './countries/philippines_regions.geojson';
import poland from './countries/poland.geojson';
import portugal from './countries/portugal.geojson';
import qatar from './countries/qatar.geojson';
import republic_of_serbia from './countries/republic_of_serbia.geojson';
import romania from './countries/romania.geojson';
import russia from './countries/russia.geojson';
import rwanda from './countries/rwanda.geojson';
@@ -304,6 +306,7 @@ export const countries = {
israel,
italy,
italy_regions,
ivory_coast,
japan,
jordan,
kazakhstan,
@@ -361,6 +364,7 @@ export const countries = {
poland,
portugal,
qatar,
republic_of_serbia,
romania,
russia,
rwanda,
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -77,7 +77,7 @@ export function getLayer(
getTargetColor: (d: any) =>
d.targetColor || d.color || [tc.r, tc.g, tc.b, 255 * tc.a],
id: `path-layer-${fd.slice_id}` as const,
strokeWidth: fd.stroke_width ? fd.stroke_width : 3,
getWidth: fd.stroke_width ? fd.stroke_width : 3,
...commonLayerProps(fd, setTooltip, setTooltipContent(fd)),
});
}
@@ -0,0 +1,97 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { SqlaFormData } from '@superset-ui/core';
import * as ChartControls from '@superset-ui/chart-controls';
import controlPanel from './controlPanel';
const { __mockShiftMetric } = ChartControls as any;
jest.mock('@superset-ui/core', () => ({
GenericDataType: { Numeric: 'numeric' },
SMART_DATE_ID: 'SMART_DATE_ID',
t: (str: string) => str,
}));
jest.mock('@superset-ui/chart-controls', () => {
// Define the mock function inside the factory
const mockShiftMetric = jest.fn(() => 'shiftedMetric');
return {
ControlPanelConfig: {},
D3_FORMAT_DOCS: 'Format docs',
D3_TIME_FORMAT_OPTIONS: [['', 'default']],
getStandardizedControls: () => ({
shiftMetric: mockShiftMetric,
}),
// Optional export to let tests access the mock
__mockShiftMetric: mockShiftMetric,
};
});
describe('BigNumber Total Control Panel Config', () => {
it('should have the required control panel sections', () => {
expect(controlPanel).toHaveProperty('controlPanelSections');
const sections = controlPanel.controlPanelSections;
expect(Array.isArray(sections)).toBe(true);
expect(sections.length).toBe(2);
// First section should have label 'Query' and contain rows with metric and adhoc_filters
expect(sections[0]!.label).toBe('Query');
expect(Array.isArray(sections[0]!.controlSetRows)).toBe(true);
expect(sections[0]!.controlSetRows[0]).toEqual(['metric']);
expect(sections[0]!.controlSetRows[1]).toEqual(['adhoc_filters']);
// Second section should contain a control named subtitle
const secondSectionRow = sections[1]!.controlSetRows[1];
expect(secondSectionRow[0]).toHaveProperty('name', 'subtitle');
// Second section should include controls for time_format and conditional_formatting
const thirdSection = sections[1]!.controlSetRows;
// Check time_format control exists in one of the rows
const timeFormatRow = thirdSection.find(row =>
row.some((control: any) => control.name === 'time_format'),
);
expect(timeFormatRow).toBeTruthy();
// Check conditional_formatting control exists in one of the rows
const conditionalFormattingRow = thirdSection.find(row =>
row.some((control: any) => control.name === 'conditional_formatting'),
);
expect(conditionalFormattingRow).toBeTruthy();
});
it('should have y_axis_format override with correct label', () => {
expect(controlPanel).toHaveProperty('controlOverrides');
expect(controlPanel.controlOverrides).toHaveProperty('y_axis_format');
expect(controlPanel.controlOverrides!.y_axis_format!.label).toBe(
'Number format',
);
});
it('should override formData metric using getStandardizedControls', () => {
const dummyFormData = { someProp: 'test' } as unknown as SqlaFormData;
const newFormData = controlPanel.formDataOverrides!(dummyFormData);
// The original properties are spread correctly.
expect(newFormData.someProp).toBe('test');
// The metric property should be replaced by the output of shiftMetric.
expect(newFormData.metric).toBe('shiftedMetric');
// Ensure that the mockShiftMetric function was called.
expect(__mockShiftMetric).toHaveBeenCalled();
});
});
@@ -0,0 +1,252 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { GenericDataType } from '@superset-ui/core';
import { getColorFormatters } from '@superset-ui/chart-controls';
import { BigNumberTotalChartProps } from '../types';
import transformProps from './transformProps';
jest.mock('@superset-ui/chart-controls', () => ({
getColorFormatters: jest.fn(),
}));
jest.mock('@superset-ui/core', () => ({
GenericDataType: { Temporal: 2, String: 1 },
getMetricLabel: jest.fn(metric => metric),
extractTimegrain: jest.fn(() => 'P1D'),
getValueFormatter: jest.fn(() => (v: any) => `$${v}`),
}));
jest.mock('../utils', () => ({
getDateFormatter: jest.fn(() => (v: any) => `${v}pm`),
parseMetricValue: jest.fn(val => Number(val)),
}));
describe('BigNumberTotal transformProps', () => {
const onContextMenu = jest.fn();
const baseFormData = {
headerFontSize: 20,
metric: 'value',
subheader: 'sub header text',
subheaderFontSize: 14,
forceTimestampFormatting: false,
timeFormat: 'YYYY-MM-DD',
yAxisFormat: 'SMART_NUMBER',
conditionalFormatting: [{ color: 'red', op: '>', value: 0 }],
currencyFormat: { symbol: '$', symbolPosition: 'prefix' },
};
const baseDatasource = {
currencyFormats: { value: '$0,0.00' },
columnFormats: { value: '$0,0.00' },
metrics: [{ metric_name: 'value', d3format: '.2f' }],
};
const baseHooks = { onContextMenu };
const baseRawFormData = { dummy: 'raw' };
it('should return null bigNumber when no data is provided', () => {
const chartProps = {
width: 400,
height: 300,
queriesData: [{ data: [], coltypes: [] }],
formData: baseFormData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const result = transformProps(
chartProps as unknown as BigNumberTotalChartProps,
);
expect(result.bigNumber).toBeNull();
expect(result.width).toBe(400);
expect(result.height).toBe(300);
expect(result.subtitle).toBe(baseFormData.subheader);
expect(result.onContextMenu).toBe(onContextMenu);
expect(result.refs).toEqual({});
// headerFormatter should be set even if there's no data
expect(typeof result.headerFormatter).toBe('function');
// colorThresholdFormatters fallback to empty array when getColorFormatters returns falsy
expect(result.colorThresholdFormatters).toEqual([]);
});
it('should convert subheader to subtitle', () => {
const chartProps = {
width: 400,
height: 300,
queriesData: [{ data: [], coltypes: [] }],
formData: { ...baseFormData, subheader: 'test' },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const result = transformProps(
chartProps as unknown as BigNumberTotalChartProps,
);
expect(result.subtitle).toBe('test');
});
const baseChartProps = {
width: 400,
height: 300,
queriesData: [{ data: [], coltypes: [] }],
rawFormData: { dummy: 'raw' },
hooks: { onContextMenu: jest.fn() },
datasource: {
currencyFormats: { value: '$0,0.00' },
columnFormats: { value: '$0,0.00' },
metrics: [{ metric_name: 'value', d3format: '.2f' }],
},
};
it('uses subtitle font size when subtitle is provided', () => {
const result = transformProps({
...baseChartProps,
formData: {
subtitle: 'Subtitle wins',
subheader: 'Fallback subheader',
subtitleFontSize: 0.4,
subheaderFontSize: 0.99,
metric: 'value',
headerFontSize: 0.3,
yAxisFormat: 'SMART_NUMBER',
timeFormat: 'smart_date',
},
} as unknown as BigNumberTotalChartProps);
expect(result.subtitle).toBe('Subtitle wins');
expect(result.subtitleFontSize).toBe(0.4);
});
it('should compute bigNumber using parseMetricValue when data exists', () => {
const chartProps = {
width: 500,
height: 400,
queriesData: [
{ data: [{ value: '456' }], coltypes: [GenericDataType.String] },
],
formData: { ...baseFormData, forceTimestampFormatting: false },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
sortBy: 'value',
};
const result = transformProps(
chartProps as unknown as BigNumberTotalChartProps,
);
// parseMetricValue converts '456' to number 456 by our mock
expect(result.bigNumber).toEqual(456);
});
it('should use formatTime as headerFormatter for Temporal or String types or forced formatting', () => {
// Case 1: Temporal type
const chartPropsTemporal = {
width: 600,
height: 450,
queriesData: [
{ data: [{ value: '789' }], coltypes: [GenericDataType.Temporal] },
],
formData: { ...baseFormData, forceTimestampFormatting: false },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const resultTemporal = transformProps(
chartPropsTemporal as unknown as BigNumberTotalChartProps,
);
expect(resultTemporal.headerFormatter(5)).toBe('5pm');
// Case 2: String type regardless of forcing formatting
const chartPropsString = {
width: 600,
height: 450,
queriesData: [
{ data: [{ value: '789' }], coltypes: [GenericDataType.String] },
],
formData: { ...baseFormData, forceTimestampFormatting: false },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const resultString = transformProps(
chartPropsString as unknown as BigNumberTotalChartProps,
);
expect(resultString.headerFormatter(5)).toBe('5pm');
// Case 3: Forced timestamp formatting
const chartPropsForced = {
width: 600,
height: 450,
queriesData: [{ data: [{ value: '789' }], coltypes: [0] }], // non-temporal/non-string
formData: { ...baseFormData, forceTimestampFormatting: true },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const resultForced = transformProps(
chartPropsForced as unknown as BigNumberTotalChartProps,
);
expect(resultForced.headerFormatter(5)).toBe('5pm');
});
it('should use numberFormatter as headerFormatter when not Temporal/String and no forced formatting', () => {
const chartProps = {
width: 700,
height: 500,
queriesData: [{ data: [{ value: '321' }], coltypes: [0] }], // non-temporal/non-string
formData: { ...baseFormData, forceTimestampFormatting: false },
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const result = transformProps(
chartProps as unknown as BigNumberTotalChartProps,
);
expect(result.headerFormatter(500)).toBe('$500');
});
it('should propagate colorThresholdFormatters from getColorFormatters', () => {
// Override the getColorFormatters mock to return specific value
const mockFormatters = [{ formatter: 'red' }];
(getColorFormatters as jest.Mock).mockReturnValueOnce(mockFormatters);
const chartProps = {
width: 800,
height: 600,
queriesData: [
{ data: [{ value: '100' }], coltypes: [GenericDataType.Temporal] },
],
formData: baseFormData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
};
const result = transformProps(
chartProps as unknown as BigNumberTotalChartProps,
);
expect(result.colorThresholdFormatters).toEqual(mockFormatters);
});
});
@@ -47,19 +47,24 @@ export default function transformProps(
const {
headerFontSize,
metric = 'value',
subtitle = '',
subtitle,
subtitleFontSize,
forceTimestampFormatting,
timeFormat,
yAxisFormat,
conditionalFormatting,
currencyFormat,
subheader,
subheaderFontSize,
} = formData;
const refs: Refs = {};
const { data = [], coltypes = [] } = queriesData[0];
const granularity = extractTimegrain(rawFormData as QueryFormData);
const metricName = getMetricLabel(metric);
const formattedSubtitle = subtitle;
const formattedSubtitle = subtitle?.trim() ? subtitle : subheader || '';
const formattedSubtitleFontSize = subtitle?.trim()
? (subtitleFontSize ?? 1)
: (subheaderFontSize ?? 1);
const bigNumber =
data.length === 0 ? null : parseMetricValue(data[0][metricName]);
@@ -105,10 +110,9 @@ export default function transformProps(
bigNumber,
headerFormatter,
headerFontSize,
subtitleFontSize,
subheaderFontSize,
subtitleFontSize: formattedSubtitleFontSize,
subtitle: formattedSubtitle,
subheader: '',
subheaderFontSize: subtitleFontSize,
onContextMenu,
refs,
colorThresholdFormatters,
@@ -188,31 +188,26 @@ class BigNumberVis extends PureComponent<BigNumberVizProps> {
);
}
renderSubheader(maxHeight: number) {
const { bigNumber, subheader, width, bigNumberFallback } = this.props;
rendermetricComparisonSummary(maxHeight: number) {
const { subheader, width } = this.props;
let fontSize = 0;
const NO_DATA_OR_HASNT_LANDED = t(
'No data after filtering or data is NULL for the latest time record',
);
const NO_DATA = t(
'Try applying different filters or ensuring your datasource has data',
);
let text = subheader;
if (bigNumber === null) {
text = bigNumberFallback ? NO_DATA : NO_DATA_OR_HASNT_LANDED;
}
const text = subheader;
if (text) {
const container = this.createTemporaryContainer();
document.body.append(container);
fontSize = computeMaxFontSize({
text,
maxWidth: width * 0.9, // max width reduced
maxHeight,
className: 'subheader-line',
container,
});
container.remove();
try {
fontSize = computeMaxFontSize({
text,
maxWidth: width * 0.9,
maxHeight,
className: 'subheader-line',
container,
});
} finally {
container.remove();
}
return (
<div
@@ -230,34 +225,46 @@ class BigNumberVis extends PureComponent<BigNumberVizProps> {
}
renderSubtitle(maxHeight: number) {
const { subtitle, width } = this.props;
const { subtitle, width, bigNumber, bigNumberFallback } = this.props;
let fontSize = 0;
if (subtitle) {
const NO_DATA_OR_HASNT_LANDED = t(
'No data after filtering or data is NULL for the latest time record',
);
const NO_DATA = t(
'Try applying different filters or ensuring your datasource has data',
);
let text = subtitle;
if (bigNumber === null) {
text =
subtitle || (bigNumberFallback ? NO_DATA : NO_DATA_OR_HASNT_LANDED);
}
if (text) {
const container = this.createTemporaryContainer();
document.body.append(container);
try {
fontSize = computeMaxFontSize({
text: subtitle,
maxWidth: width * 0.9,
maxHeight,
className: 'subtitle-line',
container,
});
} finally {
container.remove();
}
fontSize = computeMaxFontSize({
text,
maxWidth: width * 0.9,
maxHeight,
className: 'subtitle-line',
container,
});
container.remove();
return (
<div
className="subtitle-line"
style={{
fontSize,
height: maxHeight,
}}
>
{subtitle}
</div>
<>
<div
className="subtitle-line subheader-line"
style={{
fontSize: `${fontSize}px`,
height: maxHeight,
}}
>
{text}
</div>
</>
);
}
return null;
@@ -315,8 +322,8 @@ class BigNumberVis extends PureComponent<BigNumberVizProps> {
height,
kickerFontSize,
headerFontSize,
subheaderFontSize,
subtitleFontSize,
subheaderFontSize,
} = this.props;
const className = this.getClassName();
@@ -336,7 +343,7 @@ class BigNumberVis extends PureComponent<BigNumberVizProps> {
{this.renderHeader(
Math.ceil(headerFontSize * (1 - PROPORTION.TRENDLINE) * height),
)}
{this.renderSubheader(
{this.rendermetricComparisonSummary(
Math.ceil(
subheaderFontSize * (1 - PROPORTION.TRENDLINE) * height,
),
@@ -355,7 +362,9 @@ class BigNumberVis extends PureComponent<BigNumberVizProps> {
{this.renderFallbackWarning()}
{this.renderKicker((kickerFontSize || 0) * height)}
{this.renderHeader(Math.ceil(headerFontSize * height))}
{this.renderSubheader(Math.ceil(subheaderFontSize * height))}
{this.rendermetricComparisonSummary(
Math.ceil(subheaderFontSize * height),
)}
{this.renderSubtitle(Math.ceil(subtitleFontSize * height))}
</div>
);
@@ -407,12 +416,12 @@ export default styled(BigNumberVis)`
.subheader-line {
line-height: 1em;
padding-bottom: 0.3em;
padding-bottom: 0;
}
.subtitle-line {
line-height: 1em;
padding-top: 0.3em;
padding-bottom: 0;
}
&.is-fallback-value {
@@ -0,0 +1,196 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { GenericDataType } from '@superset-ui/core';
import transformProps from './transformProps';
import { BigNumberWithTrendlineChartProps, BigNumberDatum } from '../types';
jest.mock('@superset-ui/core', () => ({
GenericDataType: { Temporal: 2, String: 1 },
extractTimegrain: jest.fn(() => 'P1D'),
getMetricLabel: jest.fn(metric => metric),
getXAxisLabel: jest.fn(() => '__timestamp'),
getValueFormatter: jest.fn(() => ({
format: (v: number) => `$${v}`,
})),
getNumberFormatter: jest.fn(() => (v: number) => `${(v * 100).toFixed(1)}%`),
t: jest.fn(v => v),
tooltipHtml: jest.fn(() => '<div>tooltip</div>'),
NumberFormats: {
PERCENT_SIGNED_1_POINT: '.1%',
},
}));
jest.mock('../utils', () => ({
getDateFormatter: jest.fn(() => (v: any) => `${v}pm`),
parseMetricValue: jest.fn(val => Number(val)),
}));
jest.mock('../../utils/tooltip', () => ({
getDefaultTooltip: jest.fn(() => ({})),
}));
describe('BigNumberWithTrendline transformProps', () => {
const onContextMenu = jest.fn();
const baseFormData = {
headerFontSize: 20,
metric: 'value',
subtitle: 'subtitle message',
subtitleFontSize: 14,
forceTimestampFormatting: false,
timeFormat: 'YYYY-MM-DD',
yAxisFormat: 'SMART_NUMBER',
compareLag: 1,
compareSuffix: 'WoW',
colorPicker: { r: 0, g: 0, b: 0 },
currencyFormat: { symbol: '$', symbolPosition: 'prefix' },
};
const baseDatasource = {
currencyFormats: { value: '$0,0.00' },
columnFormats: { value: '$0,0.00' },
metrics: [{ metric_name: 'value', d3format: '.2f' }],
};
const baseHooks = { onContextMenu };
const baseRawFormData = { dummy: 'raw' };
it('should return null bigNumber when no data is provided', () => {
const chartProps = {
width: 400,
height: 300,
queriesData: [{ data: [] as unknown as BigNumberDatum[], coltypes: [] }],
formData: baseFormData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
theme: { colors: { grayscale: { light5: '#eee' } } },
};
const result = transformProps(
chartProps as unknown as BigNumberWithTrendlineChartProps,
);
expect(result.bigNumber).toBeNull();
expect(result.subtitle).toBe('subtitle message');
});
it('should calculate subheader as percent change with suffix', () => {
const chartProps = {
width: 500,
height: 400,
queriesData: [
{
data: [
{ __timestamp: 2, value: 110 },
{ __timestamp: 1, value: 100 },
] as unknown as BigNumberDatum[],
colnames: ['__timestamp', 'value'],
coltypes: ['TEMPORAL', 'NUMERIC'],
},
],
formData: baseFormData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
theme: { colors: { grayscale: { light5: '#eee' } } },
};
const result = transformProps(
chartProps as unknown as BigNumberWithTrendlineChartProps,
);
expect(result.subheader).toBe('10.0% WoW');
});
it('should compute bigNumber from parseMetricValue', () => {
const chartProps = {
width: 600,
height: 450,
queriesData: [
{
data: [
{ __timestamp: 2, value: '456' },
] as unknown as BigNumberDatum[],
colnames: ['__timestamp', 'value'],
coltypes: [GenericDataType.Temporal, GenericDataType.String],
},
],
formData: baseFormData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
theme: { colors: { grayscale: { light5: '#eee' } } },
};
const result = transformProps(
chartProps as unknown as BigNumberWithTrendlineChartProps,
);
expect(result.bigNumber).toEqual(456);
});
it('should use formatTime as headerFormatter for Temporal/String or forced', () => {
const formData = { ...baseFormData, forceTimestampFormatting: true };
const chartProps = {
width: 600,
height: 450,
queriesData: [
{
data: [
{ __timestamp: 2, value: '123' },
] as unknown as BigNumberDatum[],
colnames: ['__timestamp', 'value'],
coltypes: [0, GenericDataType.String],
},
],
formData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
theme: { colors: { grayscale: { light5: '#eee' } } },
};
const result = transformProps(
chartProps as unknown as BigNumberWithTrendlineChartProps,
);
expect(result.headerFormatter(5)).toBe('5pm');
});
it('should use numberFormatter when not Temporal/String and not forced', () => {
const formData = { ...baseFormData, forceTimestampFormatting: false };
const chartProps = {
width: 600,
height: 450,
queriesData: [
{
data: [{ __timestamp: 2, value: 500 }] as unknown as BigNumberDatum[],
colnames: ['__timestamp', 'value'],
coltypes: [0, 0],
},
],
formData,
rawFormData: baseRawFormData,
hooks: baseHooks,
datasource: baseDatasource,
theme: { colors: { grayscale: { light5: '#eee' } } },
};
const result = transformProps(
chartProps as unknown as BigNumberWithTrendlineChartProps,
);
expect(result.headerFormatter.format(500)).toBe('$500');
});
});
@@ -77,7 +77,7 @@ export type BigNumberVizProps = {
formatTime?: TimeFormatter;
headerFontSize: number;
kickerFontSize?: number;
subheader: string;
subheader?: string;
subtitle: string;
subheaderFontSize: number;
subtitleFontSize: number;
@@ -150,7 +150,7 @@ export default function transformProps(
data: data.map(row =>
colnames.map(col => {
const value = row[col];
if (!value) {
if (value === null || value === undefined) {
return NULL_STRING;
}
if (typeof value === 'boolean' || typeof value === 'bigint') {
@@ -84,7 +84,7 @@ export default function transformProps(
.filter(key => !groupbySet.has(key))
.map(key => {
const array = key.split(' - ').map(value => parseFloat(value));
return `${xAxisFormatter(array[0])} '-' ${xAxisFormatter(array[1])}`;
return `${xAxisFormatter(array[0])} - ${xAxisFormatter(array[1])}`;
});
const barSeries: BarSeriesOption[] = data.map(datum => {
const seriesName =
@@ -36,6 +36,21 @@ const defaultProps = {
subtitle: 'Test subtitle',
};
const missingExtraProps = {
...defaultProps,
error: {
error_type: ErrorTypeEnum.INVALID_SQL_ERROR,
message: 'SQLStatement should have exactly one statement',
level: 'error' as ErrorLevel,
extra: {
sql: null,
line: null,
column: null,
engine: null,
},
},
};
const renderComponent = (overrides = {}) =>
render(<InvalidSQLErrorMessage {...defaultProps} {...overrides} />);
@@ -60,6 +75,12 @@ describe('InvalidSQLErrorMessage', () => {
unmount();
});
it('renders the error message with the empty extra properties', () => {
const { getByText } = renderComponent(missingExtraProps);
expect(getByText('Unable to parse SQL')).toBeInTheDocument();
expect(getByText(missingExtraProps.error.message)).toBeInTheDocument();
});
it('displays the SQL error line and column indicator', async () => {
const { getByText, container, unmount } = renderComponent();
@@ -36,20 +36,22 @@ function InvalidSQLErrorMessage({
source,
subtitle,
}: ErrorMessageComponentProps<SupersetParseErrorExtra>) {
const { extra, level } = error;
const { extra, level, message } = error;
const { sql, line, column } = extra;
const lines = sql.split('\n');
const lines = sql?.split('\n');
let errorLine;
if (line !== null) errorLine = lines[line - 1];
else if (lines.length > 0) {
if (line !== null && Number.isInteger(line)) errorLine = lines[line - 1];
else if (lines?.length > 0) {
errorLine = lines[0];
}
const body = errorLine && (
const body = errorLine ? (
<>
<pre>{errorLine}</pre>
{column !== null && <pre>{' '.repeat(column - 1)}^</pre>}
</>
) : (
message
);
return (
<ErrorAlert
@@ -440,7 +440,7 @@ const Select = forwardRef(
const bulkSelectComponent = useMemo(
() => (
<StyledBulkActionsContainer size={0}>
<StyledBulkActionsContainer className="select-bulk-actions" size={0}>
<Button
type="link"
buttonSize="xsmall"
@@ -150,6 +150,9 @@ const Chart = props => {
const emitCrossFilters = useSelector(
state => !!state.dashboardInfo.crossFiltersEnabled,
);
const maxRows = useSelector(
state => state.dashboardInfo.common.conf.SQL_MAX_ROW,
);
const datasource = useSelector(
state =>
(chart &&
@@ -360,9 +363,7 @@ const Chart = props => {
is_cached: props.isCached,
});
exportChart({
formData: isFullCSV
? { ...formData, row_limit: props.maxRows }
: formData,
formData: isFullCSV ? { ...formData, row_limit: maxRows } : formData,
resultType: isPivot ? 'post_processed' : 'full',
resultFormat: format,
force: true,
@@ -34,7 +34,7 @@ const props = {
height: 100,
updateSliceName() {},
// from redux
maxRows: 666,
maxRows: 500, // will be overwritten with SQL_MAX_ROW from conf
formData: chartQueries[queryId].form_data,
datasource: mockDatasource[sliceEntities.slices[queryId].datasource],
sliceName: sliceEntities.slices[queryId].slice_name,
@@ -78,7 +78,7 @@ const defaultState = {
superset_can_explore: false,
superset_can_share: false,
superset_can_csv: false,
common: { conf: { SUPERSET_WEBSERVER_TIMEOUT: 0 } },
common: { conf: { SUPERSET_WEBSERVER_TIMEOUT: 0, SQL_MAX_ROW: 666 } },
},
};
@@ -128,6 +128,14 @@ const VerticalFormItem = styled(StyledFormItem)<{
width: 140px;
`}
}
.select-bulk-actions {
${({ inverseSelection }) =>
inverseSelection &&
`
flex-direction: column;
`}
}
`;
const HorizontalFormItem = styled(StyledFormItem)<{
@@ -164,6 +172,10 @@ const HorizontalFormItem = styled(StyledFormItem)<{
width: 164px;
`}
}
.select-bulk-actions {
flex-direction: column;
}
`;
const HorizontalOverflowFormItem = VerticalFormItem;
@@ -46,6 +46,7 @@ export const FilterTitle = styled.div`
}
}
&.errored div, &.errored .warning {
align-items: center;
color: ${theme.colors.error.base};
}
`}
@@ -120,7 +121,7 @@ const FilterTitleContainer = forwardRef<HTMLDivElement, Props>(
{isRemoved ? t('(Removed)') : getFilterTitle(id)}
</div>
{!removedFilters[id] && isErrored && (
<StyledWarning className="warning" />
<StyledWarning className="warning" iconSize="s" />
)}
{isRemoved && (
<span
@@ -99,6 +99,7 @@ export function ColumnSelect({
'columns.column_name',
'columns.is_dttm',
'columns.type_generic',
'columns.filterable',
],
})}`,
})
@@ -20,11 +20,38 @@ import { Filter, NativeFilterType } from '@superset-ui/core';
import { render, screen, userEvent } from 'spec/helpers/testing-library';
import { FormInstance } from 'src/components';
import getControlItemsMap, { ControlItemsProps } from './getControlItemsMap';
import { getControlItems, setNativeFilterFieldValues } from './utils';
import {
getControlItems,
setNativeFilterFieldValues,
doesColumnMatchFilterType,
} from './utils';
jest.mock('./utils', () => ({
getControlItems: jest.fn(),
setNativeFilterFieldValues: jest.fn(),
doesColumnMatchFilterType: jest.fn(),
}));
// Mock ColumnSelect to test filterValues logic
jest.mock('./ColumnSelect', () => ({
ColumnSelect: ({
filterValues,
}: {
filterValues: (column: any) => boolean;
}) => {
const columns = [
{ name: 'col1', filterable: true },
{ name: 'col2', filterable: false },
{ name: 'col3', filterable: true },
];
return (
<>
{columns.filter(filterValues).map(column => (
<div key={column.name}>{column.name}</div>
))}
</>
);
},
}));
const formMock: FormInstance = {
@@ -62,7 +89,7 @@ const filterMock: Filter = {
description: '',
};
const createProps: () => ControlItemsProps = () => ({
const createProps = (): ControlItemsProps => ({
expanded: false,
datasetId: 1,
disabled: false,
@@ -179,3 +206,44 @@ test('Clicking on checkbox when resetConfig:false', () => {
expect(props.forceUpdate).toHaveBeenCalled();
expect(setNativeFilterFieldValues).not.toHaveBeenCalled();
});
describe('ColumnSelect filterValues behavior', () => {
beforeEach(() => {
(getControlItems as jest.Mock).mockReturnValue([
{
name: 'groupby',
config: { label: 'Column', multiple: false, required: false },
},
]);
});
test('only renders filterable columns when doesColumnMatchFilterType returns true', () => {
(doesColumnMatchFilterType as jest.Mock).mockReturnValue(true);
const props = {
...createProps(),
formFilter: { filterType: 'filterType' },
};
// @ts-ignore: bypass incomplete formFilter type for test
const element = getControlItemsMap(props).mainControlItems.groupby
.element as React.ReactElement;
render(element);
expect(screen.getByText('col1')).toBeInTheDocument();
expect(screen.getByText('col3')).toBeInTheDocument();
expect(screen.queryByText('col2')).not.toBeInTheDocument();
});
test('renders no columns when doesColumnMatchFilterType returns false', () => {
(doesColumnMatchFilterType as jest.Mock).mockReturnValue(false);
const props = {
...createProps(),
formFilter: { filterType: 'filterType' },
};
// @ts-ignore: bypass incomplete formFilter type for test
const element = getControlItemsMap(props).mainControlItems.groupby
.element as React.ReactElement;
render(element);
expect(screen.queryByText('col1')).not.toBeInTheDocument();
expect(screen.queryByText('col3')).not.toBeInTheDocument();
expect(screen.queryByText('col2')).not.toBeInTheDocument();
});
});
@@ -131,7 +131,10 @@ export default function getControlItemsMap({
filterId={filterId}
datasetId={datasetId}
filterValues={column =>
doesColumnMatchFilterType(formFilter?.filterType || '', column)
doesColumnMatchFilterType(
formFilter?.filterType || '',
column,
) && !!column?.filterable
}
onChange={() => {
// We need reset default value when column changed
@@ -29,7 +29,6 @@ export const PLACEHOLDER_DATASOURCE: Datasource = {
column_types: [],
metrics: [],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '',
description: '',
+25
View File
@@ -27,10 +27,13 @@ import {
Filter,
FilterConfiguration,
Filters,
FilterState,
ExtraFormData,
} from '@superset-ui/core';
import { NATIVE_FILTER_PREFIX } from 'src/dashboard/components/nativeFilters/FiltersConfigModal/utils';
import { HYDRATE_DASHBOARD } from 'src/dashboard/actions/hydrate';
import { SaveFilterChangesType } from 'src/dashboard/components/nativeFilters/FiltersConfigModal/types';
import { isEqual } from 'lodash';
import {
AnyDataMaskAction,
CLEAR_DATA_MASK_STATE,
@@ -39,6 +42,11 @@ import {
} from './actions';
import { areObjectsEqual } from '../reduxUtils';
type FilterWithExtaFromData = Filter & {
extraFormData?: ExtraFormData;
filterState?: FilterState;
};
export function getInitialDataMask(
id?: string | number,
moreProps: DataMask = {},
@@ -106,10 +114,27 @@ function updateDataMaskForFilterChanges(
});
filterChanges.modified.forEach((filter: Filter) => {
const existingFilter = draftDataMask[filter.id] as FilterWithExtaFromData;
// Check if targets are equal
const areTargetsEqual = isEqual(existingFilter?.targets, filter?.targets);
// Preserve state only if filter exists, has enableEmptyFilter=true and targets match
const shouldPreserveState =
existingFilter &&
areTargetsEqual &&
(filter.controlValues?.enableEmptyFilter ||
filter.controlValues?.defaultToFirstItem);
mergedDataMask[filter.id] = {
...getInitialDataMask(filter.id),
...filter.defaultDataMask,
...filter,
// Preserve extraFormData and filterState if conditions match
...(shouldPreserveState && {
extraFormData: existingFilter.extraFormData,
filterState: existingFilter.filterState,
}),
};
});
@@ -41,7 +41,6 @@ const CURRENT_DATASOURCE = {
columns: [],
metrics: [],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '__timestamp',
// eg. ['["ds", true]', 'ds [asc]']
@@ -55,7 +54,6 @@ const NEW_DATASOURCE = {
columns: [],
metrics: [],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '__timestamp',
// eg. ['["ds", true]', 'ds [asc]']
@@ -734,6 +734,17 @@ const retainQueryModeRequirements = hiddenFormData =>
key => !QUERY_MODE_REQUISITES.has(key),
);
function patchBigNumberTotalFormData(form_data, slice) {
if (
form_data.viz_type === 'big_number_total' &&
!form_data.subtitle &&
slice?.form_data?.subheader
) {
return { ...form_data, subtitle: slice.form_data.subheader };
}
return form_data;
}
function mapStateToProps(state) {
const {
explore,
@@ -768,6 +779,25 @@ function mapStateToProps(state) {
dashboardId = undefined;
}
if (
form_data.viz_type === 'big_number_total' &&
slice?.form_data?.subheader &&
(!controls.subtitle?.value || controls.subtitle.value === '')
) {
controls.subtitle = {
...controls.subtitle,
value: slice.form_data.subheader,
};
if (slice?.form_data?.subheader_font_size) {
controls.subtitle_font_size = {
...controls.subtitle_font_size,
value: slice.form_data.subheader_font_size,
};
}
}
const patchedFormData = patchBigNumberTotalFormData(form_data, slice);
return {
isDatasourceMetaLoading: explore.isDatasourceMetaLoading,
datasource,
@@ -789,7 +819,7 @@ function mapStateToProps(state) {
slice,
sliceName: explore.sliceName ?? slice?.slice_name ?? null,
triggerRender: explore.triggerRender,
form_data,
form_data: patchedFormData,
table_name: datasource.table_name,
vizType: form_data.viz_type,
standalone: !!explore.standalone,
@@ -56,7 +56,6 @@ describe('controlUtils', () => {
{ metric_name: 'second', uuid: '2' },
],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '',
datasource_name: '1__table',
@@ -35,7 +35,6 @@ const sampleDatasource: Dataset = {
],
metrics: [{ metric_name: 'saved_metric_2', uuid: '1' }],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '',
datasource_name: 'Sample Dataset',
@@ -138,7 +138,6 @@ export const exploreInitialData: ExplorePageInitialData = {
{ metric_name: 'second', uuid: '2' },
],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '',
datasource_name: '8__table',
@@ -157,7 +156,6 @@ export const fallbackExploreInitialData: ExplorePageInitialData = {
columns: [],
metrics: [],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '',
owners: [],
@@ -78,7 +78,6 @@ export type DatasetObject = {
always_filter_main_dttm: boolean;
type: DatasourceType;
column_formats: Record<string, string>;
currency_formats: Record<string, Currency>;
datasource_name: string | null;
verbose_map: Record<string, string>;
};
@@ -17,7 +17,7 @@
* under the License.
*/
/* eslint-disable no-param-reassign */
import { useCallback, useEffect, useMemo, useState } from 'react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import {
AppSection,
DataMask,
@@ -84,7 +84,10 @@ function reducer(draft: DataMask, action: DataMaskAction) {
}
}
const StyledSpace = styled(Space)<{ $inverseSelection: boolean }>`
const StyledSpace = styled(Space)<{
inverseSelection: boolean;
appSection: AppSection;
}>`
display: flex;
align-items: center;
width: 100%;
@@ -96,12 +99,17 @@ const StyledSpace = styled(Space)<{ $inverseSelection: boolean }>`
&.ant-space {
.ant-space-item {
width: ${({ $inverseSelection }) =>
!$inverseSelection ? '100%' : 'auto'};
width: ${({ inverseSelection, appSection }) =>
!inverseSelection || appSection === AppSection.FilterConfigModal
? '100%'
: 'auto'};
}
}
`;
// Keep track of orientation changes outside component with filter ID
const orientationMap = new Map<string, FilterBarOrientation>();
export default function PluginFilterSelect(props: PluginFilterSelectProps) {
const {
coltypeMap,
@@ -158,6 +166,30 @@ export default function PluginFilterSelect(props: PluginFilterSelectProps) {
: filterState?.excludeFilterValues,
);
const prevExcludeFilterValues = useRef(excludeFilterValues);
const hasOnlyOrientationChanged = useRef(false);
useEffect(() => {
// Get previous orientation for this specific filter
const previousOrientation = orientationMap.get(formData.nativeFilterId);
// Check if only orientation changed for this filter
if (
previousOrientation !== undefined &&
previousOrientation !== filterBarOrientation
) {
hasOnlyOrientationChanged.current = true;
} else {
hasOnlyOrientationChanged.current = false;
}
// Update orientation for this filter
if (filterBarOrientation) {
orientationMap.set(formData.nativeFilterId, filterBarOrientation);
}
}, [filterBarOrientation]);
const updateDataMask = useCallback(
(values: SelectValue) => {
const emptyFilter =
@@ -287,35 +319,44 @@ export default function PluginFilterSelect(props: PluginFilterSelectProps) {
[formData.sortAscending],
);
// Use effect for initialisation for filter plugin
// this should run only once when filter is configured & saved
// & shouldnt run when the component is remounted on change of
// orientation of filter bar
useEffect(() => {
if (defaultToFirstItem && filterState.value === undefined) {
// initialize to first value if set to default to first item
// Skip if only orientation changed
if (hasOnlyOrientationChanged.current) {
return;
}
// Case 1: Handle disabled state first
if (isDisabled) {
updateDataMask(null);
return;
}
// Case 2: Handle the default to first Value case
if (defaultToFirstItem) {
// Set to first item if defaultToFirstItem is true
const firstItem: SelectValue = data[0]
? (groupby.map(col => data[0][col]) as string[])
: null;
// firstItem[0] !== undefined for a case when groupby changed but new data still not fetched
// TODO: still need repopulate default value in config modal when column changed
if (firstItem?.[0] !== undefined) {
updateDataMask(firstItem);
}
} else if (isDisabled) {
// empty selection if filter is disabled
updateDataMask(null);
} else {
// reset data mask based on filter state
updateDataMask(filterState.value);
} else if (formData?.defaultValue) {
// Case 3 : Handle defalut value case
updateDataMask(formData.defaultValue);
}
}, [
col,
isDisabled,
defaultToFirstItem,
enableEmptyFilter,
inverseSelection,
excludeFilterValues,
updateDataMask,
defaultToFirstItem,
formData?.defaultValue,
data,
groupby,
JSON.stringify(filterState.value),
col,
inverseSelection,
]);
useEffect(() => {
@@ -323,23 +364,26 @@ export default function PluginFilterSelect(props: PluginFilterSelectProps) {
}, [JSON.stringify(dataMask)]);
useEffect(() => {
dispatchDataMask({
type: 'filterState',
extraFormData: getSelectExtraFormData(
col,
filterState.value,
!filterState.value?.length,
excludeFilterValues && inverseSelection,
),
filterState: {
...(filterState as {
value: SelectValue;
label?: string;
excludeFilterValues?: boolean;
}),
excludeFilterValues,
},
});
if (prevExcludeFilterValues.current !== excludeFilterValues) {
dispatchDataMask({
type: 'filterState',
extraFormData: getSelectExtraFormData(
col,
filterState.value,
!filterState.value?.length,
excludeFilterValues && inverseSelection,
),
filterState: {
...(filterState as {
value: SelectValue;
label?: string;
excludeFilterValues?: boolean;
}),
excludeFilterValues,
},
});
prevExcludeFilterValues.current = excludeFilterValues;
}
}, [excludeFilterValues]);
const handleExclusionToggle = (value: string) => {
@@ -352,8 +396,11 @@ export default function PluginFilterSelect(props: PluginFilterSelectProps) {
validateStatus={filterState.validateStatus}
extra={formItemExtra}
>
<StyledSpace $inverseSelection={inverseSelection}>
{inverseSelection && (
<StyledSpace
appSection={appSection}
inverseSelection={inverseSelection}
>
{appSection !== AppSection.FilterConfigModal && inverseSelection && (
<Select
className="exclude-select"
value={`${excludeFilterValues}`}
@@ -74,7 +74,6 @@ const config: ControlPanelConfig = {
type: 'CheckboxControl',
label: t('Allow creation of new values'),
default: creatable,
resetConfig: true,
affectsDataMask: true,
renderTrigger: true,
},
@@ -26,7 +26,6 @@ const TEST_DATASOURCE = {
columns: [],
metrics: [],
column_formats: {},
currency_formats: {},
verbose_map: {},
main_dttm_col: '__timestamp',
// eg. ['["ds", true]', 'ds [asc]']
+32 -14
View File
@@ -14,12 +14,12 @@
"ioredis": "^4.28.0",
"jsonwebtoken": "^9.0.2",
"lodash": "^4.17.21",
"uuid": "^11.0.2",
"uuid": "^11.1.0",
"winston": "^3.17.0",
"ws": "^8.18.0"
},
"devDependencies": {
"@eslint/js": "^9.16.0",
"@eslint/js": "^9.25.1",
"@types/cookie": "^0.6.0",
"@types/eslint__js": "^8.42.3",
"@types/ioredis": "^4.27.8",
@@ -826,10 +826,11 @@
}
},
"node_modules/@eslint/js": {
"version": "9.17.0",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.17.0.tgz",
"integrity": "sha512-Sxc4hqcs1kTu0iID3kcZDW3JHq2a77HO9P8CP6YEA/FpH3Ll8UXE2r/86Rz9YJLKme39S9vU5OWNjC6Xl0Cr3w==",
"version": "9.25.1",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.25.1.tgz",
"integrity": "sha512-dEIwmjntEx8u3Uvv+kr3PDeeArL8Hw07H9kyYxCjnM9pBjfEhk6uLXSchxxzgiwtRhhzVzqmUSDFBOi1TuZ7qg==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
@@ -3279,6 +3280,16 @@
"url": "https://opencollective.com/eslint"
}
},
"node_modules/eslint/node_modules/@eslint/js": {
"version": "9.17.0",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.17.0.tgz",
"integrity": "sha512-Sxc4hqcs1kTu0iID3kcZDW3JHq2a77HO9P8CP6YEA/FpH3Ll8UXE2r/86Rz9YJLKme39S9vU5OWNjC6Xl0Cr3w==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
}
},
"node_modules/eslint/node_modules/escape-string-regexp": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
@@ -6811,13 +6822,14 @@
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw=="
},
"node_modules/uuid": {
"version": "11.0.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-11.0.2.tgz",
"integrity": "sha512-14FfcOJmqdjbBPdDjFQyk/SdT4NySW4eM0zcG+HqbHP5jzuH56xO3J1DGhgs/cEMCfwYi3HQI1gnTO62iaG+tQ==",
"version": "11.1.0",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-11.1.0.tgz",
"integrity": "sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==",
"funding": [
"https://github.com/sponsors/broofa",
"https://github.com/sponsors/ctavan"
],
"license": "MIT",
"bin": {
"uuid": "dist/esm/bin/uuid"
}
@@ -7615,9 +7627,9 @@
}
},
"@eslint/js": {
"version": "9.17.0",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.17.0.tgz",
"integrity": "sha512-Sxc4hqcs1kTu0iID3kcZDW3JHq2a77HO9P8CP6YEA/FpH3Ll8UXE2r/86Rz9YJLKme39S9vU5OWNjC6Xl0Cr3w==",
"version": "9.25.1",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.25.1.tgz",
"integrity": "sha512-dEIwmjntEx8u3Uvv+kr3PDeeArL8Hw07H9kyYxCjnM9pBjfEhk6uLXSchxxzgiwtRhhzVzqmUSDFBOi1TuZ7qg==",
"dev": true
},
"@eslint/object-schema": {
@@ -9388,6 +9400,12 @@
"optionator": "^0.9.3"
},
"dependencies": {
"@eslint/js": {
"version": "9.17.0",
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.17.0.tgz",
"integrity": "sha512-Sxc4hqcs1kTu0iID3kcZDW3JHq2a77HO9P8CP6YEA/FpH3Ll8UXE2r/86Rz9YJLKme39S9vU5OWNjC6Xl0Cr3w==",
"dev": true
},
"escape-string-regexp": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
@@ -12007,9 +12025,9 @@
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw=="
},
"uuid": {
"version": "11.0.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-11.0.2.tgz",
"integrity": "sha512-14FfcOJmqdjbBPdDjFQyk/SdT4NySW4eM0zcG+HqbHP5jzuH56xO3J1DGhgs/cEMCfwYi3HQI1gnTO62iaG+tQ=="
"version": "11.1.0",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-11.1.0.tgz",
"integrity": "sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A=="
},
"v8-compile-cache-lib": {
"version": "3.0.1",
+2 -2
View File
@@ -22,12 +22,12 @@
"ioredis": "^4.28.0",
"jsonwebtoken": "^9.0.2",
"lodash": "^4.17.21",
"uuid": "^11.0.2",
"uuid": "^11.1.0",
"winston": "^3.17.0",
"ws": "^8.18.0"
},
"devDependencies": {
"@eslint/js": "^9.16.0",
"@eslint/js": "^9.25.1",
"@types/cookie": "^0.6.0",
"@types/eslint__js": "^8.42.3",
"@types/ioredis": "^4.27.8",
@@ -69,7 +69,11 @@ def import_database(
# Before it gets removed in import_from_dict
ssh_tunnel_config = config.pop("ssh_tunnel", None)
# set SQLAlchemy URI via `set_sqlalchemy_uri` so that the password gets masked
sqlalchemy_uri = config.pop("sqlalchemy_uri")
database: Database = Database.import_from_dict(config, recursive=False)
database.set_sqlalchemy_uri(sqlalchemy_uri)
if database.id is None:
db.session.flush()
+14 -3
View File
@@ -80,9 +80,19 @@ class UpdateDatabaseCommand(BaseCommand):
# existing personal tokens.
self._handle_oauth2()
# build new DB
# Some DBs require running a query to get the default catalog.
# In these cases, if the current connection is broken then
# `get_default_catalog` would raise an exception. We need to
# gracefully handle that so that the connection can be fixed.
original_database_name = self._model.database_name
original_catalog = self._model.get_default_catalog()
force_update: bool = False
try:
original_catalog = self._model.get_default_catalog()
except Exception:
original_catalog = None
force_update = True
# build new DB
database = DatabaseDAO.update(self._model, self._properties)
database.set_sqlalchemy_uri(database.sqlalchemy_uri)
ssh_tunnel = self._handle_ssh_tunnel(database)
@@ -92,7 +102,8 @@ class UpdateDatabaseCommand(BaseCommand):
# configured with multi-catalog support; if it was enabled or is enabled in the
# update we don't update the assets
if (
new_catalog != original_catalog
force_update
or new_catalog != original_catalog
and not self._model.allow_multi_catalog
and not database.allow_multi_catalog
):
+25 -12
View File
@@ -15,14 +15,21 @@
# specific language governing permissions and limitations
# under the License.
import logging
from functools import partial
from typing import Any
from sqlalchemy.exc import SQLAlchemyError
from superset import db
from superset.commands.sql_lab.permalink.base import BaseSqlLabPermalinkCommand
from superset.daos.key_value import KeyValueDAO
from superset.key_value.exceptions import KeyValueCodecEncodeException
from superset.key_value.exceptions import (
KeyValueCodecEncodeException,
KeyValueCreateFailedError,
)
from superset.key_value.utils import encode_permalink_key
from superset.sqllab.permalink.exceptions import SqlLabPermalinkCreateFailedError
from superset.utils.decorators import on_error, transaction
logger = logging.getLogger(__name__)
@@ -31,19 +38,25 @@ class CreateSqlLabPermalinkCommand(BaseSqlLabPermalinkCommand):
def __init__(self, state: dict[str, Any]):
self._properties = state.copy()
@transaction(
on_error=partial(
on_error,
catches=(
KeyValueCodecEncodeException,
KeyValueCreateFailedError,
SQLAlchemyError,
),
reraise=SqlLabPermalinkCreateFailedError,
),
)
def run(self) -> str:
self.validate()
try:
entry = KeyValueDAO.create_entry(
self.resource, self._properties, self.codec
)
db.session.flush()
key = entry.id
if key is None:
raise SqlLabPermalinkCreateFailedError("Unexpected missing key id")
return encode_permalink_key(key=key, salt=self.salt)
except KeyValueCodecEncodeException as ex:
raise SqlLabPermalinkCreateFailedError(str(ex)) from ex
entry = KeyValueDAO.create_entry(self.resource, self._properties, self.codec)
db.session.flush()
key = entry.id
if key is None:
raise SqlLabPermalinkCreateFailedError("Unexpected missing key id")
return encode_permalink_key(key=key, salt=self.salt)
def validate(self) -> None:
pass
+1 -16
View File
@@ -313,10 +313,6 @@ class BaseDatasource(AuditMixinNullable, ImportExportMixin): # pylint: disable=
def column_formats(self) -> dict[str, str | None]:
return {m.metric_name: m.d3format for m in self.metrics if m.d3format}
@property
def currency_formats(self) -> dict[str, dict[str, str | None] | None]:
return {m.metric_name: m.currency_json for m in self.metrics if m.currency_json}
def add_missing_metrics(self, metrics: list[SqlMetric]) -> None:
existing_metrics = {m.metric_name for m in self.metrics}
for metric in metrics:
@@ -379,7 +375,6 @@ class BaseDatasource(AuditMixinNullable, ImportExportMixin): # pylint: disable=
"id": self.id,
"uid": self.uid,
"column_formats": self.column_formats,
"currency_formats": self.currency_formats,
"description": self.description,
"database": self.database.data, # pylint: disable=no-member
"default_endpoint": self.default_endpoint,
@@ -1046,7 +1041,7 @@ class SqlMetric(AuditMixinNullable, ImportExportMixin, CertificationMixin, Model
metric_type = Column(String(32))
description = Column(utils.MediumText())
d3format = Column(String(128))
currency = Column(String(128))
currency = Column(JSON, nullable=True)
warning_text = Column(Text)
table_id = Column(Integer, ForeignKey("tables.id", ondelete="CASCADE"))
expression = Column(utils.MediumText(), nullable=False)
@@ -1101,16 +1096,6 @@ class SqlMetric(AuditMixinNullable, ImportExportMixin, CertificationMixin, Model
def get_perm(self) -> str | None:
return self.perm
@property
def currency_json(self) -> dict[str, str | None] | None:
try:
return json.loads(self.currency or "{}") or None
except (TypeError, json.JSONDecodeError) as exc:
logger.error(
"Unable to load currency json: %r. Leaving empty.", exc, exc_info=True
)
return None
@property
def data(self) -> dict[str, Any]:
attrs = (
-1
View File
@@ -259,7 +259,6 @@ class DashboardDatasetSchema(Schema):
id = fields.Int()
uid = fields.Str()
column_formats = fields.Dict()
currency_formats = fields.Dict()
database = fields.Nested(DatabaseSchema)
default_endpoint = fields.String()
filter_select = fields.Bool()
-1
View File
@@ -221,7 +221,6 @@ class DatasetRestApi(BaseSupersetModelRestApi):
"datasource_name",
"name",
"column_formats",
"currency_formats",
"granularity_sqla",
"time_grain_sqla",
"order_by_choices",
+6
View File
@@ -440,6 +440,8 @@ class DatabricksNativeEngineSpec(DatabricksDynamicBaseEngineSpec):
"""
Return the default catalog.
It's optionally specified in `connect_args.catalog`. If not:
The default behavior for Databricks is confusing. When Unity Catalog is not
enabled we have (the DB engine spec hasn't been tested with it enabled):
@@ -451,6 +453,10 @@ class DatabricksNativeEngineSpec(DatabricksDynamicBaseEngineSpec):
To handle permissions correctly we use the result of `SHOW CATALOGS` when a
single catalog is returned.
"""
connect_args = cls.get_extra_params(database)["engine_params"]["connect_args"]
if default_catalog := connect_args.get("catalog"):
return default_catalog
with database.get_sqla_engine() as engine:
catalogs = {catalog for (catalog,) in engine.execute("SHOW CATALOGS")}
if len(catalogs) == 1:
+57 -8
View File
@@ -14,6 +14,34 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Helpers for loading Superset example datasets.
All Superset example data files (CSV, JSON, etc.) are fetched via the
jsDelivr CDN instead of raw.githubusercontent.com to avoid GitHub API
rate limits (60 anonymous requests/hour/IP).
jsDelivr is a multiCDN front for public GitHub repos and supports
arbitrary paths including nested folders. It doesnt use the GitHub REST API
and advertises unlimited bandwidth for open-source use.
Example URL::
https://cdn.jsdelivr.net/gh/apache-superset/examples-data@master/datasets/examples/slack/messages.csv
Environment knobs
-----------------
``SUPERSET_EXAMPLES_DATA_REF`` (default: ``master``)
Tag / branch / SHA to pin so builds remain reproducible.
``SUPERSET_EXAMPLES_BASE_URL``
Override the base completely if you want to host the files elsewhere
(internal mirror, S3 bucket, ASF downloads, ). **Include any query
string required by your hosting (e.g. ``?raw=true`` if you point back
to a GitHub *blob* URL).**
"""
from __future__ import annotations
import os
from typing import Any
@@ -22,27 +50,41 @@ from superset.connectors.sqla.models import SqlaTable
from superset.models.slice import Slice
from superset.utils import json
BASE_URL = "https://github.com/apache-superset/examples-data/blob/master/"
# ---------------------------------------------------------------------------
# Public sampledata mirror configuration
# ---------------------------------------------------------------------------
BASE_COMMIT: str = os.getenv("SUPERSET_EXAMPLES_DATA_REF", "master")
BASE_URL: str = os.getenv(
"SUPERSET_EXAMPLES_BASE_URL",
f"https://cdn.jsdelivr.net/gh/apache-superset/examples-data@{BASE_COMMIT}/",
)
misc_dash_slices: set[str] = set() # slices assembled in a 'Misc Chart' dashboard
# Slices assembled into a 'Misc Chart' dashboard
misc_dash_slices: set[str] = set()
# ---------------------------------------------------------------------------
# Utility functions
# ---------------------------------------------------------------------------
def get_table_connector_registry() -> Any:
"""Return the SqlaTable registry so we can mock it in unit tests."""
return SqlaTable
def get_examples_folder() -> str:
"""Return local path to the examples folder (when vendored)."""
return os.path.join(app.config["BASE_DIR"], "examples")
def update_slice_ids(pos: dict[Any, Any]) -> list[Slice]:
"""Update slice ids in position_json and return the slices found."""
"""Update slice ids in ``position_json`` and return the slices found."""
slice_components = [
component
for component in pos.values()
if isinstance(component, dict) and component.get("type") == "CHART"
]
slices = {}
slices: dict[str, Slice] = {}
for name in {component["meta"]["sliceName"] for component in slice_components}:
slc = db.session.query(Slice).filter_by(slice_name=name).first()
if slc:
@@ -56,17 +98,24 @@ def update_slice_ids(pos: dict[Any, Any]) -> list[Slice]:
def merge_slice(slc: Slice) -> None:
o = db.session.query(Slice).filter_by(slice_name=slc.slice_name).first()
if o:
db.session.delete(o)
"""Upsert a Slice by name."""
existing = db.session.query(Slice).filter_by(slice_name=slc.slice_name).first()
if existing:
db.session.delete(existing)
db.session.add(slc)
def get_slice_json(defaults: dict[Any, Any], **kwargs: Any) -> str:
"""Return JSON string for a chart definition, merging extra kwargs."""
defaults_copy = defaults.copy()
defaults_copy.update(kwargs)
return json.dumps(defaults_copy, indent=4, sort_keys=True)
def get_example_url(filepath: str) -> str:
return f"{BASE_URL}{filepath}?raw=true"
"""Return an absolute URL to *filepath* under the examplesdata repo.
All calls are routed through jsDelivr unless overridden. Supports nested
paths like ``datasets/examples/slack/messages.csv``.
"""
return f"{BASE_URL}{filepath}"
-1
View File
@@ -25,7 +25,6 @@ class DatasetSchema(Schema):
}
)
column_formats = fields.Dict(metadata={"description": "Column formats."})
currency_formats = fields.Dict(metadata={"description": "Currency formats."})
columns = fields.List(fields.Dict(), metadata={"description": "Columns metadata."})
database = fields.Dict(
metadata={"description": "Database associated with the dataset."}
+27 -7
View File
@@ -26,6 +26,7 @@ from superset import conf
from superset.constants import TimeGrain
from superset.migrations.shared.utils import paginated_update, try_load_json
from superset.utils import json
from superset.utils.date_parser import get_since_until
logger = logging.getLogger("alembic")
@@ -113,14 +114,33 @@ class MigrateViz:
}
if isinstance(granularity_sqla, dict):
temporal_filter["comparator"] = None
temporal_filter["expressionType"] = "SQL"
temporal_filter["subject"] = granularity_sqla["label"]
temporal_filter["sqlExpression"] = granularity_sqla["sqlExpression"]
since, until = get_since_until(time_range=time_range)
if not since and not until:
temporal_filter = {}
else:
temporal_filter["comparator"] = None
temporal_filter["expressionType"] = "SQL"
temporal_filter["subject"] = granularity_sqla["label"]
rv_data["adhoc_filters"] = (rv_data.get("adhoc_filters") or []) + [
temporal_filter
]
start_date = since.isoformat() if since else None
end_date = until.isoformat() if until else None
if start_date and end_date:
temporal_filter["sqlExpression"] = (
f"{granularity_sqla['sqlExpression']} >= '{start_date}' AND "
f"{granularity_sqla['sqlExpression']} < '{end_date}'"
)
elif start_date:
temporal_filter["sqlExpression"] = (
f"{granularity_sqla['sqlExpression']} >= '{start_date}'"
)
elif end_date:
temporal_filter["sqlExpression"] = (
f"{granularity_sqla['sqlExpression']} < '{end_date}'"
)
rv_data["adhoc_filters"] = rv_data.get("adhoc_filters") or []
if temporal_filter:
rv_data["adhoc_filters"].append(temporal_filter)
@classmethod
def upgrade_slice(cls, slc: Slice) -> None:
+126 -3
View File
@@ -22,7 +22,17 @@ from typing import Any, Callable, Optional, Union
from uuid import uuid4
from alembic import op
from sqlalchemy import Column, inspect
from sqlalchemy import (
Column,
inspect,
JSON,
MetaData,
select,
String,
Table,
text,
update,
)
from sqlalchemy.dialects.mysql.base import MySQLDialect
from sqlalchemy.dialects.postgresql.base import PGDialect
from sqlalchemy.dialects.sqlite.base import SQLiteDialect # noqa: E402
@@ -303,7 +313,7 @@ def add_columns(table_name: str, *columns: Column) -> None:
"""
Adds new columns to an existing database table.
If a column already exist, it logs an informational message and skips the adding process.
If a column already exist, or the table doesn't exist, it logs an informational message and skips the adding process.
Otherwise, it proceeds to add the new column to the table.
The operation is performed using Alembic's batch_alter_table.
@@ -333,7 +343,7 @@ def drop_columns(table_name: str, *columns: str) -> None:
"""
Drops specified columns from an existing database table.
If a column does not exist, it logs an informational message and skips the dropping process.
If a column or table does not exist, it logs an informational message and skips the dropping process.
Otherwise, it proceeds to remove the column from the table.
The operation is performed using Alembic's batch_alter_table.
@@ -468,3 +478,116 @@ def create_fks_for_table(
remote_cols,
ondelete=ondelete,
)
def cast_text_column_to_json(
table: str,
column: str,
pk: str = "id",
nullable: bool = True,
suffix: str = "_tmp",
) -> None:
"""
Cast a text column to JSON.
SQLAlchemy now has a nice abstraction for JSON columns, even if the underlying
database doesn't support the type natively. We should always use it when storing
JSON payloads.
:param table: The name of the table.
:param column: The name of the column to be cast.
:param pk: The name of the primary key column.
:param nullable: Whether the new column should be nullable.
:param suffix: The suffix to be added to the temporary column name.
"""
conn = op.get_bind()
if isinstance(conn.dialect, PGDialect):
conn.execute(
text(
f"""
ALTER TABLE {table}
ALTER COLUMN {column} TYPE jsonb
USING {column}::jsonb
"""
)
)
return
tmp_column = column + suffix
op.add_column(
table,
Column(tmp_column, JSON(), nullable=nullable),
)
meta = MetaData()
t = Table(table, meta, autoload_with=conn)
stmt_select = select(t.c[pk], t.c[column]).where(t.c[column].is_not(None))
for row_pk, value in conn.execute(stmt_select):
stmt_update = update(t).where(t.c[pk] == row_pk).values({tmp_column: value})
conn.execute(stmt_update)
op.drop_column(table, column)
op.alter_column(table, tmp_column, existing_type=JSON(), new_column_name=column)
return
def cast_json_column_to_text(
table: str,
column: str,
pk: str = "id",
nullable: bool = True,
suffix: str = "_tmp",
length: int = 128,
) -> None:
"""
Cast a JSON column back to text.
:param table: The name of the table.
:param column: The name of the column to be cast.
:param pk: The name of the primary key column.
:param nullable: Whether the new column should be nullable.
:param suffix: The suffix to be added to the temporary column name.
:param length: The length of the text column.
"""
conn = op.get_bind()
if isinstance(conn.dialect, PGDialect):
conn.execute(
text(
f"""
ALTER TABLE {table}
ALTER COLUMN {column} TYPE text
USING {column}::text
"""
)
)
return
tmp_column = column + suffix
op.add_column(
table,
Column(tmp_column, String(length=length), nullable=nullable),
)
meta = MetaData()
t = Table(table, meta, autoload_with=conn)
stmt_select = select(t.c[pk], t.c[column]).where(t.c[column].is_not(None))
for row_pk, value in conn.execute(stmt_select):
stmt_update = (
update(t).where(t.c[pk] == row_pk).values({tmp_column: json.dumps(value)})
)
conn.execute(stmt_update)
op.drop_column(table, column)
op.alter_column(
table,
tmp_column,
existing_type=String(length=length),
new_column_name=column,
)
return
@@ -22,19 +22,20 @@ Create Date: 2022-04-05 13:27:06.028908
"""
from alembic import op
from superset.migrations.shared.utils import create_index, drop_index
# revision identifiers, used by Alembic.
revision = "cdcf3d64daf4"
down_revision = "7fb8bca906d2"
from alembic import op # noqa: E402
def upgrade():
op.create_index(
op.f("ix_logs_user_id_dttm"), "logs", ["user_id", "dttm"], unique=False
create_index(
"logs", op.f("ix_logs_user_id_dttm"), ["user_id", "dttm"], unique=False
)
def downgrade():
op.drop_index(op.f("ix_logs_user_id_dttm"), table_name="logs")
drop_index(index_name=op.f("ix_logs_user_id_dttm"), table_name="logs")
@@ -22,21 +22,20 @@ Create Date: 2023-06-21 14:02:08.200955
"""
import sqlalchemy as sa
from superset.migrations.shared.utils import add_columns, drop_columns
# revision identifiers, used by Alembic.
revision = "90139bf715e4"
down_revision = "83e1abbe777f"
import sqlalchemy as sa # noqa: E402
from alembic import op # noqa: E402
def upgrade():
op.add_column("metrics", sa.Column("currency", sa.String(128), nullable=True))
op.add_column("sql_metrics", sa.Column("currency", sa.String(128), nullable=True))
add_columns("metrics", sa.Column("currency", sa.String(128), nullable=True))
add_columns("sql_metrics", sa.Column("currency", sa.String(128), nullable=True))
def downgrade():
with op.batch_alter_table("sql_metrics") as batch_op_sql_metrics:
batch_op_sql_metrics.drop_column("currency")
with op.batch_alter_table("metrics") as batch_op_metrics:
batch_op_metrics.drop_column("currency")
drop_columns("sql_metrics", "currency")
drop_columns("metrics", "currency")
@@ -26,37 +26,41 @@ Create Date: 2024-01-05 16:20:31.598995
revision = "65a167d4c62e"
down_revision = "06dd9ff00fe8"
from alembic import op # noqa: E402
from superset.migrations.shared.utils import create_index, drop_index # noqa: E402
def upgrade():
op.create_index(
"ix_report_execution_log_report_schedule_id",
create_index(
"report_execution_log",
"ix_report_execution_log_report_schedule_id",
["report_schedule_id"],
unique=False,
)
op.create_index(
"ix_report_execution_log_start_dttm",
create_index(
"report_execution_log",
"ix_report_execution_log_start_dttm",
["start_dttm"],
unique=False,
)
op.create_index(
"ix_report_recipient_report_schedule_id",
create_index(
"report_recipient",
"ix_report_recipient_report_schedule_id",
["report_schedule_id"],
unique=False,
)
def downgrade():
op.drop_index(
"ix_report_recipient_report_schedule_id", table_name="report_recipient"
drop_index(
index_name="ix_report_recipient_report_schedule_id",
table_name="report_recipient",
)
op.drop_index(
"ix_report_execution_log_start_dttm", table_name="report_execution_log"
drop_index(
index_name="ix_report_execution_log_start_dttm",
table_name="report_execution_log",
)
op.drop_index(
"ix_report_execution_log_report_schedule_id", table_name="report_execution_log"
drop_index(
index_name="ix_report_execution_log_report_schedule_id",
table_name="report_execution_log",
)
@@ -27,6 +27,7 @@ from alembic import op
from sqlalchemy_utils import EncryptedType
from superset.migrations.shared.utils import (
create_index,
create_table,
drop_fks_for_table,
)
@@ -77,9 +78,9 @@ def upgrade():
),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"idx_user_id_database_id",
create_index(
"database_user_oauth2_tokens",
"idx_user_id_database_id",
["user_id", "database_id"],
)
@@ -22,28 +22,26 @@ Create Date: 2024-05-02 13:40:23.126659
"""
from alembic import op
from superset.migrations.shared.utils import create_index, drop_index
# revision identifiers, used by Alembic.
revision = "3dfd0e78650e"
down_revision = "5f57af97bc3f"
from alembic import op # noqa: E402
from superset.migrations.shared.utils import table_has_index # noqa: E402
table = "query"
index = "ix_sql_editor_id"
def upgrade():
if not table_has_index(table, index):
op.create_index(
op.f(index),
table,
["sql_editor_id"],
unique=False,
)
create_index(
table,
op.f(index),
["sql_editor_id"],
unique=False,
)
def downgrade():
if table_has_index(table, index):
op.drop_index(op.f(index), table_name=table)
drop_index(index_name=op.f(index), table_name=table)
@@ -0,0 +1,46 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""metric currency should be JSON
Revision ID: f1edd4a4d4f2
Revises: 378cecfdba9f
Create Date: 2025-04-30 11:04:39.105229
"""
from superset.migrations.shared.utils import (
cast_json_column_to_text,
cast_text_column_to_json,
)
# revision identifiers, used by Alembic.
revision = "f1edd4a4d4f2"
down_revision = "378cecfdba9f"
def upgrade():
"""
Convert the currency column to JSON.
"""
cast_text_column_to_json("sql_metrics", "currency")
def downgrade():
"""
Convert the currency column back to text.
"""
cast_json_column_to_text("sql_metrics", "currency")
+4 -4
View File
@@ -1585,10 +1585,6 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
# use the existing instance.
col = metrics_exprs_by_expr.get(str(col), col)
need_groupby = True
elif col in columns_by_name:
col = self.convert_tbl_column_to_sqla_col(
columns_by_name[col], template_processor=template_processor
)
elif col in metrics_exprs_by_label:
col = metrics_exprs_by_label[col]
need_groupby = True
@@ -1597,6 +1593,10 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
template_processor=template_processor
)
need_groupby = True
elif col in columns_by_name:
col = self.convert_tbl_column_to_sqla_col(
columns_by_name[col], template_processor=template_processor
)
if isinstance(col, ColumnElement):
orderby_exprs.append(col)
+40
View File
@@ -32,6 +32,7 @@ from deprecation import deprecated
from sqlglot import exp
from sqlglot.dialects.dialect import Dialect, Dialects
from sqlglot.errors import ParseError
from sqlglot.expressions import Func
from sqlglot.optimizer.pushdown_predicates import pushdown_predicates
from sqlglot.optimizer.scope import Scope, ScopeType, traverse_scope
@@ -453,6 +454,23 @@ class SQLStatement(BaseSQLStatement[exp.Expression]):
return SQLStatement(sql, self.engine, optimized)
def check_functions_present(self, functions: set[str]) -> bool:
"""
Check if any of the given functions are present in the script.
:param functions: List of functions to check for
:return: True if any of the functions are present
"""
present = {
(
function.sql_name()
if function.sql_name() != "ANONYMOUS"
else function.name.upper()
)
for function in self._parsed.find_all(Func)
}
return any(function.upper() in present for function in functions)
class KQLSplitState(enum.Enum):
"""
@@ -619,6 +637,16 @@ class KustoKQLStatement(BaseSQLStatement[str]):
"""
return KustoKQLStatement(self._sql, self.engine, self._parsed)
def check_functions_present(self, functions: set[str]) -> bool:
"""
Check if any of the given functions are present in the script.
:param functions: List of functions to check for
:return: True if any of the functions are present
"""
logger.warning("Kusto KQL doesn't support checking for functions present.")
return True
class SQLScript:
"""
@@ -684,6 +712,18 @@ class SQLScript:
return script
def check_functions_present(self, functions: set[str]) -> bool:
"""
Check if any of the given functions are present in the script.
:param functions: List of functions to check for
:return: True if any of the functions are present
"""
return any(
statement.check_functions_present(functions)
for statement in self.statements
)
def extract_tables_from_statement(
statement: exp.Expression,
+2 -30
View File
@@ -31,7 +31,6 @@ from sqlalchemy import and_
from sqlparse import keywords
from sqlparse.lexer import Lexer
from sqlparse.sql import (
Function,
Identifier,
IdentifierList,
Parenthesis,
@@ -181,7 +180,7 @@ def check_sql_functions_exist(
:param function_list: The list of functions to search for
:param engine: The engine to use for parsing the SQL statement
"""
return ParsedQuery(sql, engine=engine).check_functions_exist(function_list)
return SQLScript(sql, engine=engine).check_functions_present(function_list)
def strip_comments_from_sql(statement: str, engine: str = "base") -> str:
@@ -229,34 +228,6 @@ class ParsedQuery:
self._tables = self._extract_tables_from_sql()
return self._tables
def _check_functions_exist_in_token(
self, token: Token, functions: set[str]
) -> bool:
if (
isinstance(token, Function)
and token.get_name() is not None
and token.get_name().lower() in functions
):
return True
if hasattr(token, "tokens"):
for inner_token in token.tokens:
if self._check_functions_exist_in_token(inner_token, functions):
return True
return False
def check_functions_exist(self, functions: set[str]) -> bool:
"""
Check if the SQL statement contains any of the specified functions.
:param functions: A set of functions to search for
:return: True if the statement contains any of the specified functions
"""
for statement in self._parsed:
for token in statement.tokens:
if self._check_functions_exist_in_token(token, functions):
return True
return False
def _extract_tables_from_sql(self) -> set[Table]:
"""
Extract all table references in a query.
@@ -277,6 +248,7 @@ class ParsedQuery:
"You may have an error in your SQL statement. {message}"
).format(message=ex.error.message),
level=ErrorLevel.ERROR,
extra=ex.error.extra,
)
) from ex
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More