Compare commits

..

9 Commits

Author SHA1 Message Date
Maxime Beauchemin
5d32c8834d touch file to trigger 2025-07-23 01:18:21 -07:00
Maxime Beauchemin
40164300e5 feat: optimize setup-backend to go faster on py 3.12 2025-07-23 01:13:36 -07:00
Maxime Beauchemin
c444eed63e chore(docker): use editable mode in docker images (#34146) 2025-07-22 16:14:31 -07:00
Mehmet Salih Yavuz
1df5e59fdf fix(theming): Theming visual fixes (#34253) 2025-07-23 01:55:32 +03:00
Maxime Beauchemin
77f66e7434 fix: build issues on master with 'npm run dev' (#34272) 2025-07-22 15:55:18 -07:00
Maxime Beauchemin
2c81eb6c39 feat(docker): do not include chromium (headless browser) by default in Dockerfile (#34258) 2025-07-22 13:12:55 -07:00
Maxime Beauchemin
09c4afc894 feat: introduce comprehensive LLM context guides for AI-powered development (#34194)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-22 12:22:13 -07:00
JUST.in DO IT
229d92590a fix: Matching errorType on superset api error with SupersetError (#34261) 2025-07-22 11:51:42 -07:00
Kamil Gabryjelski
f4f516c64c fix: Missing ownState and isCached props in Chart.jsx (#34259) 2025-07-22 18:58:28 +02:00
32 changed files with 446 additions and 310 deletions

View File

@@ -0,0 +1,125 @@
---
description: Apache Superset development standards and guidelines for Cursor IDE
globs: ["**/*.py", "**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.sql", "**/*.md"]
alwaysApply: true
---
# Apache Superset Development Standards for Cursor IDE
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
**These migrations are actively happening - avoid deprecated patterns:**
### Frontend Modernization
- **NO `any` types** - Use proper TypeScript types
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
- **NO Enzyme** - Use React Testing Library/Jest (Enzyme fully removed)
- **Use @superset-ui/core** - Don't import Ant Design directly
### Testing Strategy Migration
- **Prefer unit tests** over integration tests
- **Prefer integration tests** over Cypress end-to-end tests
- **Cypress is last resort** - Actively moving away from Cypress
- **Use Jest + React Testing Library** for component testing
### Backend Type Safety
- **Add type hints** - All new Python code needs proper typing
- **MyPy compliance** - Run `pre-commit run mypy` to validate
- **SQLAlchemy typing** - Use proper model annotations
## Code Standards
### TypeScript Frontend
- **NO `any` types** - Use proper TypeScript
- **Functional components** with hooks
- **@superset-ui/core** for UI components (not direct antd)
- **Jest** for testing (NO Enzyme)
- **Redux** for global state, hooks for local
### Python Backend
- **Type hints required** for all new code
- **MyPy compliant** - run `pre-commit run mypy`
- **SQLAlchemy models** with proper typing
- **pytest** for testing
### Apache License Headers
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
- **LLM instruction files are excluded** - Files like LLMS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
## Key Directory Structure
```
superset/
├── superset/ # Python backend (Flask, SQLAlchemy)
│ ├── views/api/ # REST API endpoints
│ ├── models/ # Database models
│ └── connectors/ # Database connections
├── superset-frontend/src/ # React TypeScript frontend
│ ├── components/ # Reusable components
│ ├── explore/ # Chart builder
│ ├── dashboard/ # Dashboard interface
│ └── SqlLab/ # SQL editor
├── superset-frontend/packages/
│ └── superset-ui-core/ # UI component library (USE THIS)
├── tests/ # Python/integration tests
├── docs/ # Documentation (UPDATE FOR CHANGES)
└── UPDATING.md # Breaking changes log
```
## Architecture Patterns
### Dataset-Centric Approach
Charts built from enriched datasets containing:
- Dimension columns with labels/descriptions
- Predefined metrics as SQL expressions
- Self-service analytics within defined contexts
### Security & Features
- **RBAC**: Role-based access via Flask-AppBuilder
- **Feature flags**: Control feature rollouts
- **Row-level security**: SQL-based data access control
## Test Utilities
### Python Test Helpers
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
- **`@with_config`** - Config mocking decorator
- **`@with_feature_flags`** - Feature flag testing
- **`login_as()`, `login_as_admin()`** - Authentication helpers
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
### TypeScript Test Helpers
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
- **`createWrapper()`** - Redux/Router/Theme wrapper
- **`selectOption()`** - Select component helper
- **React Testing Library** - NO Enzyme (removed)
## Pre-commit Validation
**Use pre-commit hooks for quality validation:**
```bash
# Install hooks
pre-commit install
# Quick validation (faster than --all-files)
pre-commit run # Staged files only
pre-commit run mypy # Python type checking
pre-commit run prettier # Code formatting
pre-commit run eslint # Frontend linting
```
## Development Guidelines
- **Documentation**: Update docs/ for any user-facing changes
- **Breaking Changes**: Add to UPDATING.md
- **Docstrings**: Required for new functions/classes
- **Follow existing patterns**: Mimic code style, use existing libraries and utilities
- **Type Safety**: This codebase is actively modernizing toward full TypeScript and type safety
- **Always run `pre-commit run`** to validate changes before committing
---
**Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

View File

@@ -28,7 +28,6 @@ runs:
if [ "${{ inputs.python-version }}" = "current" ]; then
echo "PYTHON_VERSION=3.11" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "next" ]; then
# currently disabled in GHA matrixes because of library compatibility issues
echo "PYTHON_VERSION=3.12" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "previous" ]; then
echo "PYTHON_VERSION=3.10" >> $GITHUB_ENV
@@ -40,7 +39,17 @@ runs:
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: ${{ inputs.cache }}
- name: Cache uv packages
uses: actions/cache@v4
with:
path: ~/.cache/uv
key: uv-${{ runner.os }}-python${{ env.PYTHON_VERSION }}-${{ hashFiles('requirements/development.txt', 'requirements/base.txt') }}
restore-keys: |
uv-${{ runner.os }}-python${{ env.PYTHON_VERSION }}-
- name: Install dependencies
env:
UV_CACHE_DIR: ~/.cache/uv
UV_PREFER_BINARY: "1"
run: |
if [ "${{ inputs.install-superset }}" = "true" ]; then
sudo apt-get update && sudo apt-get -y install libldap2-dev libsasl2-dev
@@ -48,11 +57,11 @@ runs:
pip install --upgrade pip setuptools wheel uv
if [ "${{ inputs.requirements-type }}" = "dev" ]; then
uv pip install --system -r requirements/development.txt
uv pip install --system --prefer-binary -r requirements/development.txt
elif [ "${{ inputs.requirements-type }}" = "base" ]; then
uv pip install --system -r requirements/base.txt
uv pip install --system --prefer-binary -r requirements/base.txt
fi
uv pip install --system -e .
uv pip install --system --prefer-binary -e .
fi
shell: bash

1
.github/copilot-instructions.md vendored Symbolic link
View File

@@ -0,0 +1 @@
../LLMS.md

4
.gitignore vendored
View File

@@ -127,5 +127,7 @@ docker/*local*
# Jest test report
test-report.html
superset/static/stats/statistics.html
.aider*
# LLM-related
CLAUDE.local.md
.aider*

View File

@@ -76,3 +76,11 @@ ydb.svg
erd.puml
erd.svg
intro_header.txt
# for LLMs
llm-context.md
LLMS.md
CLAUDE.md
CURSOR.md
GEMINI.md
GPT.md

1
CLAUDE.md Symbolic link
View File

@@ -0,0 +1 @@
LLMS.md

View File

@@ -167,7 +167,7 @@ RUN mkdir -p \
&& touch superset/static/version_info.json
# Install Playwright and optionally setup headless browsers
ARG INCLUDE_CHROMIUM="true"
ARG INCLUDE_CHROMIUM="false"
ARG INCLUDE_FIREFOX="false"
RUN --mount=type=cache,target=${SUPERSET_HOME}/.cache/uv \
if [ "$INCLUDE_CHROMIUM" = "true" ] || [ "$INCLUDE_FIREFOX" = "true" ]; then \
@@ -223,7 +223,7 @@ RUN --mount=type=cache,target=${SUPERSET_HOME}/.cache/uv \
/app/docker/pip-install.sh --requires-build-essential -r requirements/base.txt
# Install the superset package
RUN --mount=type=cache,target=${SUPERSET_HOME}/.cache/uv \
uv pip install .
uv pip install -e .
RUN python -m compileall /app/superset
USER superset
@@ -246,7 +246,7 @@ RUN --mount=type=cache,target=${SUPERSET_HOME}/.cache/uv \
/app/docker/pip-install.sh --requires-build-essential -r requirements/development.txt
# Install the superset package
RUN --mount=type=cache,target=${SUPERSET_HOME}/.cache/uv \
uv pip install .
uv pip install -e .
RUN uv pip install .[postgres]
RUN python -m compileall /app/superset

1
GEMINI.md Symbolic link
View File

@@ -0,0 +1 @@
LLMS.md

1
GPT.md Symbolic link
View File

@@ -0,0 +1 @@
LLMS.md

148
LLMS.md Normal file
View File

@@ -0,0 +1,148 @@
# LLM Context Guide for Apache Superset
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
**These migrations are actively happening - avoid deprecated patterns:**
### Frontend Modernization
- **NO `any` types** - Use proper TypeScript types
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
- **Use @superset-ui/core** - Don't import Ant Design directly
### Testing Strategy Migration
- **Prefer unit tests** over integration tests
- **Prefer integration tests** over Cypress end-to-end tests
- **Cypress is last resort** - Actively moving away from Cypress
- **Use Jest + React Testing Library** for component testing
### Backend Type Safety
- **Add type hints** - All new Python code needs proper typing
- **MyPy compliance** - Run `pre-commit run mypy` to validate
- **SQLAlchemy typing** - Use proper model annotations
## Key Directories
```
superset/
├── superset/ # Python backend (Flask, SQLAlchemy)
│ ├── views/api/ # REST API endpoints
│ ├── models/ # Database models
│ └── connectors/ # Database connections
├── superset-frontend/src/ # React TypeScript frontend
│ ├── components/ # Reusable components
│ ├── explore/ # Chart builder
│ ├── dashboard/ # Dashboard interface
│ └── SqlLab/ # SQL editor
├── superset-frontend/packages/
│ └── superset-ui-core/ # UI component library (USE THIS)
├── tests/ # Python/integration tests
├── docs/ # Documentation (UPDATE FOR CHANGES)
└── UPDATING.md # Breaking changes log
```
## Code Standards
### TypeScript Frontend
- **Avoid `any` types** - Use proper TypeScript, reuse existing types
- **Functional components** with hooks
- **@superset-ui/core** for UI components (not direct antd)
- **Jest** for testing (NO Enzyme)
- **Redux** for global state where it exists, hooks for local
### Python Backend
- **Type hints required** for all new code
- **MyPy compliant** - run `pre-commit run mypy`
- **SQLAlchemy models** with proper typing
- **pytest** for testing
### Apache License Headers
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
- **LLM instruction files are excluded** - Files like LLMS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
## Documentation Requirements
- **docs/**: Update for any user-facing changes
- **UPDATING.md**: Add breaking changes here
- **Docstrings**: Required for new functions/classes
## Architecture Patterns
### Security & Features
- **RBAC**: Role-based access via Flask-AppBuilder
- **Feature flags**: Control feature rollouts
- **Row-level security**: SQL-based data access control
## Test Utilities
### Python Test Helpers
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
- **`@with_config`** - Config mocking decorator
- **`@with_feature_flags`** - Feature flag testing
- **`login_as()`, `login_as_admin()`** - Authentication helpers
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
### TypeScript Test Helpers
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
- **`createWrapper()`** - Redux/Router/Theme wrapper
- **`selectOption()`** - Select component helper
- **React Testing Library** - NO Enzyme (removed)
### Running Tests
```bash
# Frontend
npm run test # All tests
npm run test -- filename.test.tsx # Single file
# Backend
pytest # All tests
pytest tests/unit_tests/specific_test.py # Single file
pytest tests/unit_tests/ # Directory
# If pytest fails with database/setup issues, ask the user to run test environment setup
```
## Environment Validation
**Quick Setup Check (run this first):**
```bash
# Verify Superset is running
curl -f http://localhost:8088/health || echo "❌ Setup required - see https://superset.apache.org/docs/contributing/development#working-with-llms"
```
**If health checks fail:**
"It appears you aren't set up properly. Please refer to the [Working with LLMs](https://superset.apache.org/docs/contributing/development#working-with-llms) section in the development docs for setup instructions."
**Key Project Files:**
- `superset-frontend/package.json` - Frontend build scripts (`npm run dev` on port 9000, `npm run test`, `npm run lint`)
- `pyproject.toml` - Python tooling (ruff, mypy configs)
- `requirements/` folder - Python dependencies (base.txt, development.txt)
## Pre-commit Validation
**Use pre-commit hooks for quality validation:**
```bash
# Install hooks
pre-commit install
# Quick validation (faster than --all-files)
pre-commit run # Staged files only
pre-commit run mypy # Python type checking
pre-commit run prettier # Code formatting
pre-commit run eslint # Frontend linting
```
## Platform-Specific Instructions
- **[CLAUDE.md](CLAUDE.md)** - For Claude/Anthropic tools
- **[.github/copilot-instructions.md](.github/copilot-instructions.md)** - For GitHub Copilot
- **[GEMINI.md](GEMINI.md)** - For Google Gemini tools
- **[GPT.md](GPT.md)** - For OpenAI/ChatGPT tools
- **[.cursor/rules/dev-standard.mdc](.cursor/rules/dev-standard.mdc)** - For Cursor editor
---
**LLM Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

View File

@@ -23,6 +23,7 @@ This file documents any backwards-incompatible changes in Superset and
assists people when migrating to a new version.
## Next
- [34258](https://github.com/apache/superset/pull/34258) changing the default in Dockerfile to INCLUDE_CHROMIUM="false" (from "true") in the past. This ensures the `lean` layer is lean by default, and people can opt-in to the `chromium` layer by setting the build arg `INCLUDE_CHROMIUM=true`. This is a breaking change for anyone using the `lean` layer, as it will no longer include Chromium by default.
- [34204](https://github.com/apache/superset/pull/33603) OpenStreetView has been promoted as the new default for Deck.gl visualization since it can be enabled by default without requiring an API key. If you have Mapbox set up and want to disable OpenStreeView in your environment, please follow the steps documented here [https://superset.apache.org/docs/configuration/map-tiles].
- [33116](https://github.com/apache/superset/pull/33116) In Echarts Series charts (e.g. Line, Area, Bar, etc.) charts, the `x_axis_sort_series` and `x_axis_sort_series_ascending` form data items have been renamed with `x_axis_sort` and `x_axis_sort_asc`.
There's a migration added that can potentially affect a significant number of existing charts.

View File

@@ -194,6 +194,48 @@ You can also run the pre-commit checks manually in various ways:
Replace `<hook_id>` with the ID of the specific hook you want to run. You can find the list
of available hooks in the `.pre-commit-config.yaml` file.
## Working with LLMs
### Environment Setup
Ensure Docker Compose is running before starting LLM sessions:
```bash
docker compose up
```
Validate your environment:
```bash
curl -f http://localhost:8088/health && echo "✅ Superset ready"
```
### LLM Session Best Practices
- Always validate environment setup first using the health checks above
- Use focused validation commands: `pre-commit run` (not `--all-files`)
- **Read [LLMS.md](https://github.com/apache/superset/blob/master/LLMS.md) first** - Contains comprehensive development guidelines, coding standards, and critical refactor information
- **Check platform-specific files** when available:
- `CLAUDE.md` - For Claude/Anthropic tools
- `CURSOR.md` - For Cursor editor
- `GEMINI.md` - For Google Gemini tools
- `GPT.md` - For OpenAI/ChatGPT tools
- Follow the TypeScript migration guidelines and avoid deprecated patterns listed in LLMS.md
### Key Development Commands
```bash
# Frontend development
cd superset-frontend
npm run dev # Development server on http://localhost:9000
npm run test # Run all tests
npm run test -- filename.test.tsx # Run single test file
npm run lint # Linting and type checking
# Backend validation
pre-commit run mypy # Type checking
pytest # Run all tests
pytest tests/unit_tests/specific_test.py # Run single test file
pytest tests/unit_tests/ # Run all tests in directory
```
For detailed development context, environment setup, and coding guidelines, see [LLMS.md](https://github.com/apache/superset/blob/master/LLMS.md).
## Alternatives to `docker compose`
:::caution

View File

@@ -148,7 +148,6 @@ hive = [
impala = ["impyla>0.16.2, <0.17"]
kusto = ["sqlalchemy-kusto>=3.0.0, <4"]
kylin = ["kylinpy>=2.8.1, <2.9"]
metricflow = ["shillelagh[dbtmetricflowapi]>=1.4.2, <2"]
motherduck = ["duckdb==0.10.2", "duckdb-engine>=0.12.1, <0.13"]
mssql = ["pymssql>=2.2.8, <3"]
mysql = ["mysqlclient>=2.1.0, <3"]

View File

@@ -202,7 +202,7 @@ export function AsyncAceEditor(
/* Basic editor styles with dark mode support */
.ace_editor.ace-github,
.ace_editor.ace-textmate {
.ace_editor.ace-tm {
background-color: ${token.colorBgContainer} !important;
color: ${token.colorText} !important;
}

View File

@@ -70,7 +70,6 @@ export function Button(props: ButtonProps) {
if (!buttonStyle || buttonStyle === 'primary') {
variant = 'solid';
antdType = 'primary';
color = 'primary';
} else if (buttonStyle === 'secondary') {
variant = 'filled';
color = 'primary';
@@ -78,7 +77,6 @@ export function Button(props: ButtonProps) {
variant = 'outlined';
color = 'default';
} else if (buttonStyle === 'dashed') {
color = 'primary';
variant = 'dashed';
antdType = 'dashed';
} else if (buttonStyle === 'danger') {
@@ -134,6 +132,11 @@ export function Button(props: ButtonProps) {
'& > span > :first-of-type': {
marginRight: firstChildMargin,
},
':not(:hover)': effectiveButtonStyle === 'secondary' && {
// NOTE: This is the best we can do contrast wise for the secondary button using antd tokens
// and abusing the semantics. Should be revisited when possible. https://github.com/apache/superset/pull/34253#issuecomment-3104834692
color: `${theme.colorPrimaryTextHover} !important`,
},
}}
icon={icon}
{...restProps}

View File

@@ -31,11 +31,6 @@ const StyledDiv = styled.div`
}
`;
const DescriptionContainer = styled.div`
line-height: ${({ theme }) => theme.sizeUnit * 4}px;
padding-top: 16px;
`;
export function DeleteModal({
description,
onConfirm,
@@ -81,12 +76,12 @@ export function DeleteModal({
onHide={hide}
onHandledPrimaryAction={confirm}
primaryButtonName={t('Delete')}
primaryButtonType="danger"
primaryButtonStyle="danger"
show={open}
title={title}
centered
>
<DescriptionContainer>{description}</DescriptionContainer>
{description}
<StyledDiv>
<FormLabel htmlFor="delete">
{t('Type "%s" to confirm', t('DELETE'))}

View File

@@ -33,7 +33,7 @@ export const InteractiveModal = (props: ModalProps) => (
InteractiveModal.args = {
disablePrimaryButton: false,
primaryButtonName: 'Danger',
primaryButtonType: 'danger',
primaryButtonStyle: 'danger',
show: true,
title: "I'm a modal!",
resizable: false,

View File

@@ -77,7 +77,7 @@ export const StyledModal = styled(BaseModal)<StyledModalProps>`
.ant-modal-header {
flex: 0 0 auto;
border-radius: ${theme.borderRadius}px ${theme.borderRadius}px 0 0;
padding: ${theme.sizeUnit * 4}px ${theme.sizeUnit * 6}px;
padding: ${theme.sizeUnit * 4}px ${theme.sizeUnit * 4}px;
.ant-modal-title {
font-weight: ${theme.fontWeightStrong};
@@ -122,6 +122,7 @@ export const StyledModal = styled(BaseModal)<StyledModalProps>`
.ant-modal-body {
flex: 0 1 auto;
padding: ${theme.sizeUnit * 4}px;
padding-bottom: ${theme.sizeUnit * 2}px;
overflow: auto;
${!resizable && height && `height: ${height};`}
}
@@ -208,7 +209,7 @@ const CustomModal = ({
onHide,
onHandledPrimaryAction,
primaryButtonName = t('OK'),
primaryButtonType = 'primary',
primaryButtonStyle = 'primary',
show,
name,
title,
@@ -261,7 +262,7 @@ const CustomModal = ({
</Button>,
<Button
key="submit"
buttonStyle={primaryButtonType}
buttonStyle={primaryButtonStyle}
disabled={disablePrimaryButton}
tooltip={primaryTooltipMessage}
loading={primaryButtonLoading}

View File

@@ -20,6 +20,7 @@ import type { CSSProperties, ReactNode } from 'react';
import type { ModalFuncProps } from 'antd';
import type { ResizableProps } from 're-resizable';
import type { DraggableProps } from 'react-draggable';
import { ButtonStyle } from '../Button/types';
export interface ModalProps {
className?: string;
@@ -30,7 +31,7 @@ export interface ModalProps {
onHide: () => void;
onHandledPrimaryAction?: () => void;
primaryButtonName?: string;
primaryButtonType?: 'primary' | 'danger';
primaryButtonStyle?: ButtonStyle;
show: boolean;
name?: string;
title: ReactNode;

View File

@@ -24,7 +24,7 @@ import {
getSequentialSchemeRegistry,
CategoricalColorNamespace,
} from '@superset-ui/core';
import Datamap from 'datamaps/dist/datamaps.world.min';
import Datamap from 'datamaps/dist/datamaps.all.min';
import { ColorBy } from './utils';
const propTypes = {

View File

@@ -54,7 +54,8 @@ export function ErrorMessageWithStackTrace({
// Check if a custom error message component was registered for this message
if (error) {
const ErrorMessageComponent = getErrorMessageComponentRegistry().get(
error.error_type,
// @ts-ignore: plan to modify this part so that all errors in Superset 6.0 are standardized as Superset API error types
error.errorType ?? error.error_type,
);
if (ErrorMessageComponent) {
return (

View File

@@ -362,7 +362,7 @@ export const ImportModal: FunctionComponent<ImportModelsModalProps> = ({
onHandledPrimaryAction={onUpload}
onHide={hide}
primaryButtonName={needsOverwriteConfirm ? t('Overwrite') : t('Import')}
primaryButtonType={needsOverwriteConfirm ? 'danger' : 'primary'}
primaryButtonStyle={needsOverwriteConfirm ? 'danger' : 'primary'}
width="750px"
show={show}
title={<h4>{t('Import %s', resourceLabel)}</h4>}

View File

@@ -129,6 +129,9 @@ const Chart = props => {
);
const chart = useSelector(state => state.charts[props.id] || EMPTY_OBJECT);
const { queriesResponse, chartUpdateEndTime, chartStatus, annotationQuery } =
chart;
const slice = useSelector(
state => state.sliceEntities.slices[props.id] || EMPTY_OBJECT,
);
@@ -163,6 +166,12 @@ const Chart = props => {
);
const dashboardInfo = useSelector(state => state.dashboardInfo);
const isCached = useMemo(
// eslint-disable-next-line camelcase
() => queriesResponse?.map(({ is_cached }) => is_cached) || [],
[queriesResponse],
);
const [descriptionHeight, setDescriptionHeight] = useState(0);
const [height, setHeight] = useState(props.height);
const [width, setWidth] = useState(props.width);
@@ -249,9 +258,9 @@ const Chart = props => {
const logExploreChart = useCallback(() => {
boundActionCreators.logEvent(LOG_ACTIONS_EXPLORE_DASHBOARD_CHART, {
slice_id: slice.slice_id,
is_cached: props.isCached,
is_cached: isCached,
});
}, [boundActionCreators.logEvent, slice.slice_id, props.isCached]);
}, [boundActionCreators.logEvent, slice.slice_id, isCached]);
const chartConfiguration = useSelector(
state => state.dashboardInfo.metadata?.chart_configuration,
@@ -365,22 +374,22 @@ const Chart = props => {
: LOG_ACTIONS_EXPORT_XLSX_DASHBOARD_CHART;
boundActionCreators.logEvent(logAction, {
slice_id: slice.slice_id,
is_cached: props.isCached,
is_cached: isCached,
});
exportChart({
formData: isFullCSV ? { ...formData, row_limit: maxRows } : formData,
resultType: isPivot ? 'post_processed' : 'full',
resultFormat: format,
force: true,
ownState: props.ownState,
ownState: dataMask[props.id]?.ownState,
});
},
[
slice.slice_id,
props.isCached,
isCached,
formData,
props.maxRows,
props.ownState,
dataMask[props.id]?.ownState,
boundActionCreators.logEvent,
],
);
@@ -408,7 +417,7 @@ const Chart = props => {
const forceRefresh = useCallback(() => {
boundActionCreators.logEvent(LOG_ACTIONS_FORCE_REFRESH_CHART, {
slice_id: slice.slice_id,
is_cached: props.isCached,
is_cached: isCached,
});
return boundActionCreators.refreshChart(chart.id, true, props.dashboardId);
}, [
@@ -416,7 +425,7 @@ const Chart = props => {
chart.id,
props.dashboardId,
slice.slice_id,
props.isCached,
isCached,
boundActionCreators.logEvent,
]);
@@ -424,11 +433,7 @@ const Chart = props => {
return <MissingChart height={getChartHeight()} />;
}
const { queriesResponse, chartUpdateEndTime, chartStatus, annotationQuery } =
chart;
const isLoading = chartStatus === 'loading';
// eslint-disable-next-line camelcase
const isCached = queriesResponse?.map(({ is_cached }) => is_cached) || [];
const cachedDttm =
// eslint-disable-next-line camelcase
queriesResponse?.map(({ cached_dttm }) => cached_dttm) || [];

View File

@@ -46,7 +46,7 @@ const containerStyle = (theme: SupersetTheme) => css`
display: flex;
&& > .filter-clear-all-button {
color: ${theme.colors.grayscale.base};
color: ${theme.colorTextSecondary};
margin-left: 0;
&:hover {
color: ${theme.colorPrimaryText};
@@ -54,7 +54,7 @@ const containerStyle = (theme: SupersetTheme) => css`
&[disabled],
&[disabled]:hover {
color: ${theme.colors.grayscale.light1};
color: ${theme.colorTextDisabled};
}
}
`;
@@ -62,7 +62,6 @@ const containerStyle = (theme: SupersetTheme) => css`
const verticalStyle = (theme: SupersetTheme, width: number) => css`
flex-direction: column;
align-items: center;
pointer-events: none;
position: fixed;
z-index: 100;
@@ -74,14 +73,10 @@ const verticalStyle = (theme: SupersetTheme, width: number) => css`
padding-top: ${theme.sizeUnit * 6}px;
background: linear-gradient(
${rgba(theme.colors.grayscale.light5, 0)},
${theme.colors.grayscale.light5} 60%
${rgba(theme.colorBgLayout, 0)},
${theme.colorBgElevated} 20%
);
& > button {
pointer-events: auto;
}
& > .filter-apply-button {
margin-bottom: ${theme.sizeUnit * 3}px;
}
@@ -94,13 +89,6 @@ const horizontalStyle = (theme: SupersetTheme) => css`
text-transform: capitalize;
font-weight: ${theme.fontWeightNormal};
}
& > .filter-apply-button {
&[disabled],
&[disabled]:hover {
color: ${theme.colors.grayscale.light1};
background: ${theme.colors.grayscale.light3};
}
}
`;
const ButtonsContainer = styled.div<{ isVertical: boolean; width: number }>`

View File

@@ -28,6 +28,7 @@ import {
import { Icons } from '@superset-ui/core/components/Icons';
import { Tooltip } from '@superset-ui/core/components/Tooltip';
import { Typography } from '@superset-ui/core/components';
import DatasourcePanelDragOption from './DatasourcePanelDragOption';
import { DndItemType } from '../DndItemType';
import { DndItemValue, FlattenedItem, Folder } from './types';
@@ -94,7 +95,7 @@ const SectionHeaderTextContainer = styled.div`
width: 100%;
`;
const SectionHeader = styled.span`
const SectionHeader = styled(Typography.Text)`
${({ theme }) => css`
font-size: ${theme.fontSize}px;
font-weight: ${theme.fontWeightStrong};

View File

@@ -95,6 +95,7 @@ const Styles = styled.div`
}
.error-alert {
margin: ${({ theme }) => 2 * theme.sizeUnit}px;
min-height: 150px;
}
.ant-dropdown-trigger {
margin-left: ${({ theme }) => 2 * theme.sizeUnit}px;
@@ -454,16 +455,14 @@ class DatasourceControl extends PureComponent {
{isMissingDatasource && !isMissingParams && (
<div className="error-alert">
{extra?.error ? (
<div className="error-alert">
<ErrorMessageWithStackTrace
title={extra.error.statusText || extra.error.message}
subtitle={
extra.error.statusText ? extra.error.message : undefined
}
error={extra.error}
source="explore"
/>
</div>
<ErrorMessageWithStackTrace
title={extra.error.statusText || extra.error.message}
subtitle={
extra.error.statusText ? extra.error.message : undefined
}
error={extra.error}
source="explore"
/>
) : (
<ErrorAlert
type="warning"

View File

@@ -102,9 +102,9 @@ const getCommonElements = () => ({
cancelButton: screen.getByRole('button', { name: 'Cancel' }),
uploadButton: screen.getByRole('button', { name: 'Upload' }),
selectButton: screen.getByRole('button', { name: 'Select' }),
panel1: screen.getByRole('heading', { name: /General information/i }),
panel2: screen.getByRole('heading', { name: /file settings/i }),
panel3: screen.getByRole('heading', { name: /columns/i }),
panel1: screen.getByText(/General information/i, { selector: 'strong' }),
panel2: screen.getByText(/file settings/i, { selector: 'strong' }),
panel3: screen.getByText(/columns/i, { selector: 'strong' }),
selectDatabase: screen.getByRole('combobox', { name: /select a database/i }),
inputTableName: screen.getByRole('textbox', { name: /table name/i }),
inputSchema: screen.getByRole('combobox', { name: /schema/i }),
@@ -130,7 +130,7 @@ describe('UploadDataModal - General Information Elements', () => {
const common = getCommonElements();
const title = screen.getByRole('heading', { name: /csv upload/i });
const panel4 = screen.getByRole('heading', { name: /rows/i });
const panel4 = screen.getByText(/rows/i);
const selectDelimiter = screen.getByRole('combobox', {
name: /choose a delimiter/i,
});
@@ -156,7 +156,7 @@ describe('UploadDataModal - General Information Elements', () => {
const common = getCommonElements();
const title = screen.getByRole('heading', { name: /excel upload/i });
const panel4 = screen.getByRole('heading', { name: /rows/i });
const panel4 = screen.getByText(/rows/i);
const selectSheetName = screen.getByRole('combobox', {
name: /choose sheet name/i,
});
@@ -177,9 +177,7 @@ describe('UploadDataModal - General Information Elements', () => {
]);
// Check elements that should NOT be visible
expect(
screen.queryByRole('heading', { name: /csv upload/i }),
).not.toBeInTheDocument();
expect(screen.queryByText(/csv upload/i)).not.toBeInTheDocument();
expect(
screen.queryByRole('combobox', { name: /choose a delimiter/i }),
).not.toBeInTheDocument();
@@ -206,8 +204,8 @@ describe('UploadDataModal - General Information Elements', () => {
// Check elements that should NOT be visible
expectElementsNotVisible([
screen.queryByRole('heading', { name: /csv upload/i }),
screen.queryByRole('heading', { name: /rows/i }),
screen.queryByText(/csv upload/i),
screen.queryByText(/rows/i),
screen.queryByRole('combobox', { name: /choose a delimiter/i }),
screen.queryByRole('combobox', { name: /choose sheet name/i }),
]);
@@ -216,7 +214,7 @@ describe('UploadDataModal - General Information Elements', () => {
describe('UploadDataModal - File Settings Elements', () => {
const openFileSettings = async () => {
const panelHeader = screen.getByRole('heading', { name: /file settings/i });
const panelHeader = screen.getByText(/file settings/i);
await userEvent.click(panelHeader);
};
@@ -294,7 +292,7 @@ describe('UploadDataModal - File Settings Elements', () => {
describe('UploadDataModal - Columns Elements', () => {
const openColumns = async () => {
const panelHeader = screen.getByRole('heading', { name: /columns/i });
const panelHeader = screen.getByText(/columns/i, { selector: 'strong' });
await userEvent.click(panelHeader);
};
@@ -365,7 +363,7 @@ describe('UploadDataModal - Rows Elements', () => {
test('CSV/Excel rows render correctly', async () => {
render(<UploadDataModal {...csvProps} />, { useRedux: true });
const panelHeader = screen.getByRole('heading', { name: /rows/i });
const panelHeader = screen.getByText(/rows/i);
await userEvent.click(panelHeader);
const elements = [
@@ -380,7 +378,7 @@ describe('UploadDataModal - Rows Elements', () => {
test('Columnar does not render rows', () => {
render(<UploadDataModal {...columnarProps} />, { useRedux: true });
const panelHeader = screen.queryByRole('heading', { name: /rows/i });
const panelHeader = screen.queryByText(/rows/i);
expect(panelHeader).not.toBeInTheDocument();
});
});
@@ -608,11 +606,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information Upload a file to a database./i,
name: /expanded General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings Adjust how spaces, blank lines, null values are handled and other file wide settings./i,
name: /collapsed File settings/i,
});
userEvent.click(fileSettingsTab);
expect(fileSettingsTab).toHaveAttribute('aria-expanded', 'true');
@@ -626,11 +624,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information Upload a file to a database./i,
name: /expanded General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings Adjust how spaces, blank lines, null values are handled and other file wide settings./i,
name: /collapsed File settings/i,
});
userEvent.click(fileSettingsTab);
expect(fileSettingsTab).toHaveAttribute('aria-expanded', 'true');
@@ -644,11 +642,11 @@ describe('UploadDataModal Collapse Tabs', () => {
useRedux: true,
});
const generalInfoTab = screen.getByRole('tab', {
name: /expanded General information Upload a file to a database./i,
name: /expanded General information/i,
});
expect(generalInfoTab).toHaveAttribute('aria-expanded', 'true');
const fileSettingsTab = screen.getByRole('tab', {
name: /collapsed File settings Adjust how spaces, blank lines, null values are handled and other file wide settings./i,
name: /collapsed File settings/i,
});
userEvent.click(fileSettingsTab);
expect(fileSettingsTab).toHaveAttribute('aria-expanded', 'true');

View File

@@ -26,6 +26,7 @@ import {
} from 'react';
import {
css,
getClientErrorObject,
SupersetClient,
SupersetTheme,
@@ -45,6 +46,7 @@ import {
Upload,
type UploadChangeParam,
type UploadFile,
Typography,
} from '@superset-ui/core/components';
import { Switch, SwitchProps } from '@superset-ui/core/components/Switch';
import { Icons } from '@superset-ui/core/components/Icons';
@@ -576,7 +578,17 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
const UploadTitle: FC = () => {
const title = uploadTitles[type] || t('Upload');
return <h4>{title}</h4>;
return (
<Typography.Title
level={5}
css={css`
margin-top: 0;
margin-bottom: 0;
`}
>
{title}
</Typography.Title>
);
};
return (
@@ -592,7 +604,7 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
onHandledPrimaryAction={form.submit}
onHide={onClose}
width="500px"
primaryButtonName="Upload"
primaryButtonName={t('Upload')}
centered
show={show}
title={<UploadTitle />}
@@ -615,10 +627,9 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
{
key: 'general',
label: (
<div>
<h4>{t('General information')}</h4>
<p className="helper">{t('Upload a file to a database.')}</p>
</div>
<Typography.Text strong>
{t('General information')}
</Typography.Text>
),
children: (
<>
@@ -772,14 +783,7 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
{
key: 'file-settings',
label: (
<div>
<h4>{t('File settings')}</h4>
<p className="helper">
{t(
'Adjust how spaces, blank lines, null values are handled and other file wide settings.',
)}
</p>
</div>
<Typography.Text strong>{t('File settings')}</Typography.Text>
),
children: (
<>
@@ -901,16 +905,7 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
},
{
key: 'columns',
label: (
<div>
<h4>{t('Columns')}</h4>
<p className="helper">
{t(
'Adjust column settings such as specifying the columns to read, how duplicates are handled, column data types, and more.',
)}
</p>
</div>
),
label: <Typography.Text strong>{t('Columns')}</Typography.Text>,
children: (
<>
<Row>
@@ -1010,14 +1005,7 @@ const UploadDataModal: FunctionComponent<UploadDataModalProps> = ({
{
key: 'rows',
label: (
<div>
<h4>{t('Rows')}</h4>
<p className="helper">
{t(
'Set header rows and the number of rows to read or skip.',
)}
</p>
</div>
<Typography.Text strong>{t('Rows')}</Typography.Text>
),
children: (
<Row>

View File

@@ -46,7 +46,7 @@ export const antDModalNoPaddingStyles = css`
export const formStyles = (theme: SupersetTheme) => css`
.switch-label {
color: ${theme.colors.grayscale.base};
color: ${theme.colorTextSecondary};
margin-left: ${theme.sizeUnit * 4}px;
}
`;

View File

@@ -319,6 +319,13 @@ const config = {
'./node_modules/@storybook/react-dom-shim/dist/react-16',
),
),
// Workaround for react-color trying to import non-existent icon
'@icons/material/UnfoldMoreHorizontalIcon': path.resolve(
path.join(
APP_DIR,
'./node_modules/@icons/material/CubeUnfoldedIcon.js',
),
),
},
extensions: ['.ts', '.tsx', '.js', '.jsx', '.yml'],
fallback: {

View File

@@ -598,8 +598,8 @@ DEFAULT_FEATURE_FLAGS: dict[str, bool] = {
# This is intended to use for theme creation, visual review and theming-debugging
# purposes.
"THEME_ALLOW_THEME_EDITOR_BETA": False,
# Allow users to optionally specify date formats in email subjects, which will
# be parsed if enabled
# Allow users to optionally specify date formats in email subjects, which
# will be parsed if enabled
"DATE_FORMAT_IN_EMAIL_SUBJECT": False,
# Allow metrics and columns to be grouped into (potentially nested) folders in the
# chart builder

View File

@@ -1,189 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
An interface to dbt's semantic layer, Metric Flow.
"""
from __future__ import annotations
from datetime import timedelta
from typing import Any, TYPE_CHECKING
from shillelagh.backends.apsw.dialects.base import get_adapter_for_table_name
from shillelagh.backends.apsw.dialects.metricflow import TABLE_NAME
from sqlalchemy import event
from sqlalchemy.engine import Engine
from sqlalchemy.pool import _ConnectionRecord
from sqlalchemy.pool.base import _ConnectionFairy
from superset.connectors.sqla.models import SqlaTable
from superset.constants import TimeGrain
from superset.db_engine_specs.base import MetricType, ValidColumnsType
from superset.db_engine_specs.shillelagh import ShillelaghEngineSpec
from superset.extensions import cache_manager
from superset.utils.cache import memoized_func
if TYPE_CHECKING:
from shillelagh.backends.apsw.dialects.metricflow import TableMetricFlowAPI
from shillelagh.fields import Field
from sqlalchemy.engine.reflection import Inspector
from superset.models.core import Database
from superset.sql.parse import Table
from superset.superset_typing import ResultSetColumnType
@event.listens_for(Engine, "connect")
def receive_connect(
dbapi_connection: _ConnectionFairy,
connection_record: _ConnectionRecord,
) -> None:
"""
Called when a new DB connection is created.
This hook adds a cache to the `_build_column_from_dimension` method of the Metric
Flow adapter, since it's called frequently and can be expensive.
"""
engine = connection_record.info.get("engine")
if (
not engine
or not engine.name == "metricflow"
or getattr(engine.dialect, "_patched", False)
):
return
original_method = engine.dialect._build_column_from_dimension
@memoized_func(
key="metricflow:dimension:{name}",
cache=cache_manager.data_cache,
)
def cached_build_column_from_dimension(
self: TableMetricFlowAPI,
name: str,
*args: Any,
**kwargs: Any,
) -> Field:
return original_method(
self,
name,
cache_timeout=int(timedelta(days=1).total_seconds()),
)
engine.dialect._build_column_from_dimension = cached_build_column_from_dimension
engine.dialect._patched = True
SELECT_STAR_MESSAGE = (
'The dbt semantic layer does not support data preview, since the "metrics" table '
"is a virtual table that is not materialized. An administrator should configure "
'the database in Apache Superset so that the "Disable SQL Lab data preview '
'queries" option under "Advanced""SQL Lab" is enabled.'
)
class DbtMetricFlowEngineSpec(ShillelaghEngineSpec):
"""
Engine for the the dbt semantic layer.
"""
engine = "metricflow"
engine_name = "dbt Metric Flow"
sqlalchemy_uri_placeholder = (
"metricflow://[ab123.us1.dbt.com]/<environment_id>"
"?service_token=<service_token>"
)
supports_dynamic_columns = True
_time_grain_expressions = {
TimeGrain.DAY: "{col}__day",
TimeGrain.WEEK: "{col}__week",
TimeGrain.MONTH: "{col}__month",
TimeGrain.QUARTER: "{col}__quarter",
TimeGrain.YEAR: "{col}__year",
}
@classmethod
def select_star(cls, *args: Any, **kwargs: Any) -> str:
"""
Return a ``SELECT *`` query.
"""
message = SELECT_STAR_MESSAGE.replace("'", "''")
return f"SELECT '{message}' AS warning"
@classmethod
def get_columns(
cls,
inspector: Inspector,
table: Table,
options: dict[str, Any] | None = None,
) -> list[ResultSetColumnType]:
columns: list[ResultSetColumnType] = []
for column in inspector.get_columns(table.table, table.schema):
# ignore metrics
if "computed" in column:
continue
column["column_name"] = column["name"]
columns.append(column)
return columns
@classmethod
def get_metrics(
cls,
database: Database,
inspector: Inspector,
table: Table,
) -> list[MetricType]:
"""
Get all metrics.
"""
return [
{
"metric_name": column["name"],
"expression": column["computed"]["sqltext"],
"description": column["comment"],
}
for column in inspector.get_columns(table.table, table.schema)
if "computed" in column
]
@classmethod
def get_valid_metrics_and_dimensions(
cls,
database: Database,
table: SqlaTable,
dimensions: set[str],
metrics: set[str],
) -> ValidColumnsType:
"""
Get valid metrics and dimensions.
Given a datasource, and sets of selected metrics and dimensions, return the
sets of valid metrics and dimensions that can further be selected.
"""
with database.get_sqla_engine() as engine:
connection = engine.connect()
adapter = get_adapter_for_table_name(connection, TABLE_NAME)
return {
"metrics": adapter._get_metrics_for_dimensions(dimensions),
"dimensions": adapter._get_dimensions_for_metrics(metrics),
}