Compare commits

..
Author SHA1 Message Date
Maxime BeaucheminandClaude b7b0f1c795 feat: add comprehensive configuration metadata with AgGrid table
- Add enriched metadata for 348 Superset configurations
- Replace basic HTML table with AgGrid for better UX
- Add search, sort, filter, and CSV export capabilities
- Include configuration details, groups, and categories
- Add .scratchpad to gitignore for temporary files
- Update config schema with proper descriptions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-03 17:21:25 -07:00
Maxime Beauchemin ed73ac4737 rollback 2025-07-31 21:18:15 -07:00
Maxime BeaucheminandClaude 59fa496221 feat: comprehensive configuration improvements with Flask extension references
- Enhanced 151 config descriptions with Flask extension references and documentation links
- Added config_objects.py module to handle complex Python objects that can't be JSON serialized
- Fixed ConfigurationTable.tsx error by adding missing impact/requires_restart fields
- Improved AST parsing in extract_config_schema.py with better type detection
- Added fully qualified type names with module paths for complex objects
- Enhanced markdown formatting with proper emphasis, code blocks, and links
- Cleaned up formatting issues (dashes, extra whitespace) in descriptions
- Added comprehensive type detection for callables, instances, and complex objects

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:15 -07:00
Maxime BeaucheminandClaude 590d39abeb feat: enhance config descriptions with markdown formatting
- Added markdown formatting to key configuration descriptions
- Enhanced SECRET_KEY description with bold emphasis and code formatting
- Updated SQLALCHEMY_ENGINE_OPTIONS with link to flask-sqlalchemy documentation
- Regenerated config schema and metadata with markdown-formatted content
- Configuration table now properly renders:
  - **Bold text** for important warnings and recommendations
  - `Code blocks` for configuration examples and commands
  - [Links](url) that open in new tabs for external documentation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:15 -07:00
Maxime BeaucheminandClaude c67143592b feat: add markdown support and dedicated configuration reference page
- Added react-markdown dependency for rendering markdown in config descriptions
- Created comprehensive configuration reference page at /configuration/configuration-reference
- Configured ReactMarkdown with custom components for table context:
  - Links open in new tabs with proper styling
  - Inline code blocks with background highlighting
  - Bold and italic text rendering
  - Paragraph tags converted to spans for table compatibility
- Updated sidebar positioning to make config reference first in configuration section
- Added detailed usage instructions and security notes
- Enhanced documentation with configuration precedence explanation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:15 -07:00
Maxime Beauchemin 6e469eb922 style: remove quotes from search message to avoid ESLint warnings 2025-07-31 21:18:15 -07:00
Maxime BeaucheminandClaude ffe1a0c9ee feat: add search functionality to configuration table
- Added search input box that filters by configuration name and description
- Real-time filtering as user types with case-insensitive matching
- Clear search button (✕) appears when search term is entered
- Improved layout with flexbox for search and category controls
- Enhanced results messaging: shows "Found X matching settings" when searching
- Better empty state messaging with option to clear search
- Responsive design with flex-wrap for mobile compatibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:15 -07:00
Maxime BeaucheminandClaude c2a05ea919 feat: make configuration table denser with smaller fonts
- Reduced cell padding from 12px to 8px for more compact layout
- Decreased font sizes: headers 13px, content 12px, badges 10px
- Removed title-cased labels in leftmost column, showing only config key
- Made badges smaller with reduced padding (1px 6px instead of 2px 8px)
- Improved information density while maintaining readability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:15 -07:00
Maxime Beauchemin 54f17134b6 hackin' 2025-07-31 21:18:14 -07:00
Maxime BeaucheminandClaude ad8d0bb2fb refactor: simplify safe_eval function in config schema extraction
- Removed complex AST node handling for function calls and attributes
- Simplified to handle only basic types: constants, lists, dicts, names
- Reduced complexity from 19 to 6 branches
- Maintains same functionality for actual config values while being more maintainable

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:14 -07:00
Maxime BeaucheminandClaude a21a1824e3 fix: add missing impact and requires_restart fields to config metadata
- Added infer_impact() function to determine configuration impact levels
- Added infer_requires_restart() function to determine restart requirements
- Updated export_config_metadata.py to include these fields in JSON output
- Fixes ConfigurationTable.tsx error: "Cannot read properties of undefined (reading 'toUpperCase')"
- All 218 configuration settings now include impact and requires_restart fields
- Fixed type annotations and linting issues in extract_config_schema.py

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-31 21:18:14 -07:00
Maxime Beauchemin 30e731a15b improve docs generation 2025-07-31 21:18:14 -07:00
Maxime Beauchemin faef33d6ba improve docs generation 2025-07-31 21:18:12 -07:00
Maxime Beauchemin 92bf3b9d4e touchups 2025-07-31 21:17:26 -07:00
Maxime Beauchemin 29b4c480f3 docs 2025-07-31 21:17:26 -07:00
Maxime Beauchemin 1a9da0ff78 feat: enhance flask config system with derived class 2025-07-31 21:17:26 -07:00
106 changed files with 20116 additions and 3328 deletions
+3 -1
View File
@@ -47,10 +47,12 @@ jobs:
java-version: '21'
- name: Install Graphviz
run: sudo apt-get install -y graphviz
- name: Compute Entity Relationship diagram (ERD)
- name: Generate documentation artifacts
env:
SUPERSET_SECRET_KEY: not-a-secret
CI: true
run: |
# Generate ERD
python scripts/erd/erd.py
curl -L http://sourceforge.net/projects/plantuml/files/1.2023.7/plantuml.1.2023.7.jar/download > ~/plantuml.jar
java -jar ~/plantuml.jar -v -tsvg -r -o "${{ github.workspace }}/docs/static/img/" "${{ github.workspace }}/scripts/erd/erd.puml"
@@ -64,6 +64,11 @@ jobs:
uses: actions/setup-node@v4
with:
node-version-file: './docs/.nvmrc'
- name: Setup Python Backend
uses: ./.github/actions/setup-backend
with:
python-version: 'current'
requirements-type: 'base'
- name: yarn install
run: |
yarn install --check-cache
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
steps:
- name: Welcome Message
uses: actions/first-interaction@v2
uses: actions/first-interaction@v1
continue-on-error: true
with:
repo-token: ${{ github.token }}
+3
View File
@@ -131,3 +131,6 @@ superset/static/stats/statistics.html
# LLM-related
CLAUDE.local.md
.aider*
# Temporary scratchpad for development
.scratchpad/
@@ -0,0 +1,86 @@
---
title: Configuration Reference
hide_title: true
sidebar_position: 1
version: 1
---
import ConfigurationTable from '@site/src/components/ConfigurationTable';
# Configuration Reference
This page provides a comprehensive reference for all Superset configuration options. These settings can be configured in your `superset_config.py` file or through environment variables.
## How to Use This Reference
- **Search**: Use the search box to find specific configuration settings
- **Filter by Category**: Use the dropdown to filter by configuration category
- **Environment Variables**: All configurations can be set via environment variables with the `SUPERSET__` prefix
- **Impact Level**: Each setting shows its impact level (low, medium, high)
- **Restart Required**: Settings marked with "RESTART" require a server restart to take effect
## Configuration Settings
<ConfigurationTable showEnvironmentVariables={true} />
## Setting Configuration Values
### In superset_config.py
```python
# Example configuration in superset_config.py
SECRET_KEY = 'your-secret-key-here'
SQLALCHEMY_DATABASE_URI = 'postgresql://user:pass@localhost/superset'
CACHE_DEFAULT_TIMEOUT = 300
```
### Via Environment Variables
```bash
# All configuration keys can be set via environment variables
export SUPERSET__SECRET_KEY="your-secret-key-here"
export SUPERSET__SQLALCHEMY_DATABASE_URI="postgresql://user:pass@localhost/superset"
export SUPERSET__CACHE_DEFAULT_TIMEOUT=300
```
### Configuration Precedence
Configuration values are loaded in the following order (later values override earlier ones):
1. **Default values** from `superset/config_defaults.py`
2. **Base configuration** from `superset/config.py`
3. **Custom configuration file** (if specified via `SUPERSET_CONFIG_PATH`)
4. **superset_config module** (if available in PYTHONPATH)
5. **Environment variables** with `SUPERSET__` prefix
## Configuration Categories
The configuration settings are organized into the following categories:
- **Security**: Authentication, authorization, and security-related settings
- **Database**: Database connection and SQL-related configurations
- **Performance**: Caching, timeouts, and performance optimization settings
- **Features**: Feature flags and optional functionality toggles
- **UI**: User interface and theming configurations
- **Logging**: Logging and monitoring configurations
- **Email**: Email and notification settings
- **Async**: Asynchronous processing and Celery settings
- **General**: Miscellaneous configuration options
## Important Security Notes
- Always set a strong `SECRET_KEY` in production
- Use environment variables for sensitive configuration values
- Never commit sensitive configuration values to version control
- Regularly rotate secrets and passwords
- Review security-related configurations before deploying
## Need Help?
For detailed information about specific configuration topics, see:
- [Configuring Superset](./configuring-superset.mdx) - General configuration guide
- [Security](../security/security.mdx) - Security configuration
- [Database Configuration](./databases.mdx) - Database-specific settings
- [Cache Configuration](./cache.mdx) - Caching setup
- [Async Queries](./async-queries-celery.mdx) - Celery configuration
@@ -1,19 +1,36 @@
---
title: Configuring Superset
hide_title: true
sidebar_position: 1
sidebar_position: 2
version: 1
---
# Configuring Superset
## superset_config.py
## Configuration Overview
Superset provides a flexible, multi-layered configuration system that supports:
1. **File-based configuration** - Traditional Python configuration files
2. **Environment variables** - For containerized deployments and CI/CD
3. **Database-backed settings** - Runtime configuration changes (coming soon)
4. **Structured metadata** - Rich documentation and validation schemas
### Configuration Priority
Configuration values are loaded in the following order (later values override earlier ones):
1. **Default configuration** - Built-in defaults from `superset/config_defaults.py`
2. **Environment variables** - Values prefixed with `SUPERSET__`
3. **User configuration file** - Your custom `superset_config.py` file
### superset_config.py
Superset exposes hundreds of configurable parameters through its
[config.py module](https://github.com/apache/superset/blob/master/superset/config.py). The
[config_defaults.py module](https://github.com/apache/superset/blob/master/superset/config_defaults.py). The
variables and objects exposed act as a public interface of the bulk of what you may want
to configure, alter and interface with. In this python module, you'll find all these
parameters, sensible defaults, as well as rich documentation in the form of comments
parameters, sensible defaults, as well as structured metadata documentation.
To configure your application, you need to create your own configuration module, which
will allow you to override few or many of these parameters. Instead of altering the core module,
@@ -77,12 +94,12 @@ MAPBOX_API_KEY = ''
:::tip
Note that it is typical to copy and paste [only] the portions of the
core [superset/config.py](https://github.com/apache/superset/blob/master/superset/config.py) that
core [superset/config_defaults.py](https://github.com/apache/superset/blob/master/superset/config_defaults.py) that
you want to alter, along with the related comments into your own `superset_config.py` file.
:::
All the parameters and default values defined
in [superset/config.py](https://github.com/apache/superset/blob/master/superset/config.py)
in [superset/config_defaults.py](https://github.com/apache/superset/blob/master/superset/config_defaults.py)
can be altered in your local `superset_config.py`. Administrators will want to read through the file
to understand what can be configured locally as well as the default values in place.
@@ -97,6 +114,102 @@ for more information on how to configure it.
At the very least, you'll want to change `SECRET_KEY` and `SQLALCHEMY_DATABASE_URI`. Continue reading for more about each of these.
## Environment Variables Configuration
For containerized deployments and CI/CD pipelines, Superset supports configuration through environment variables. This is particularly useful for:
- **Docker deployments** - Configure containers without rebuilding images
- **Kubernetes environments** - Use ConfigMaps and Secrets
- **CI/CD pipelines** - Set configuration dynamically based on environment
- **Development workflows** - Override settings locally without changing files
### Environment Variable Format
All Superset environment variables must use the `SUPERSET__` prefix (note the double underscore):
```bash
# Basic settings
export SUPERSET__ROW_LIMIT=100000
export SUPERSET__SQLLAB_TIMEOUT=60
# Database configuration
export SUPERSET__SQLALCHEMY_DATABASE_URI="postgresql://user:pass@localhost/superset"
# Secret key
export SUPERSET__SECRET_KEY="your-secret-key-here"
```
### JSON and Complex Values
Environment variables automatically parse JSON values for complex configuration:
```bash
# Feature flags as JSON
export SUPERSET__FEATURE_FLAGS='{"ENABLE_TEMPLATE_PROCESSING": true, "ENABLE_EXPLORE_DRAG_AND_DROP": true}'
# Database configuration
export SUPERSET__DATABASE_CONFIG='{"timeout": 60, "pool_size": 10}'
```
### Nested Configuration
For nested configuration objects, use triple underscores (`___`) to separate levels:
```bash
# This sets FEATURE_FLAGS["ENABLE_TEMPLATE_PROCESSING"] = true
export SUPERSET__FEATURE_FLAGS__ENABLE_TEMPLATE_PROCESSING=true
# This sets THEME_DEFAULT["token"]["colorPrimary"] = "#ff0000"
export SUPERSET__THEME_DEFAULT__token__colorPrimary="#ff0000"
```
### Environment Variable Examples
You can view examples of environment variable configuration:
```bash
# Show all available environment variable examples
superset config env-examples
# Show current configuration and sources
superset config show --verbose
# Get a specific configuration value
superset config get ROW_LIMIT
```
### Docker Environment Variables
When using Docker, you can set environment variables in your `docker-compose.yml`:
```yaml
services:
superset:
image: apache/superset:latest
environment:
- SUPERSET__ROW_LIMIT=100000
- SUPERSET__SQLLAB_TIMEOUT=60
- SUPERSET__FEATURE_FLAGS__ENABLE_TEMPLATE_PROCESSING=true
# ... other configuration
```
Or use an environment file:
```bash
# .env file
SUPERSET__ROW_LIMIT=100000
SUPERSET__SQLLAB_TIMEOUT=60
SUPERSET__SECRET_KEY=your-secret-key-here
```
```yaml
services:
superset:
image: apache/superset:latest
env_file:
- .env
```
## Specifying a SECRET_KEY
### Adding an initial SECRET_KEY
@@ -546,3 +659,93 @@ FEATURE_FLAGS = {
```
A current list of feature flags can be found in [RESOURCES/FEATURE_FLAGS.md](https://github.com/apache/superset/blob/master/RESOURCES/FEATURE_FLAGS.md).
## Configuration Introspection
Superset provides CLI commands to inspect and understand your current configuration:
### View Current Configuration
```bash
# Show all configuration as YAML
superset config show
# Filter configuration by pattern
superset config show --filter "ROW_LIMIT"
# Show configuration with sources (where each value comes from)
superset config show --verbose
```
### Get Specific Configuration Values
```bash
# Get a specific configuration value with source information
superset config get ROW_LIMIT
# Output shows both the value and where it came from:
# ROW_LIMIT:
# source: environment (SUPERSET__ROW_LIMIT)
# value: 100000
```
### Environment Variable Examples
```bash
# Show examples of environment variables for all documented settings
superset config env-examples
# This shows:
# - Basic environment variable syntax
# - JSON formatting examples
# - Nested configuration examples
# - All documented settings with their metadata
```
### Configuration Sources
The CLI will show you where each configuration value comes from:
- **`environment (SUPERSET__KEY)`** - Value set via environment variable
- **`superset_config.py`** - Value set in your custom configuration file
- **`config_defaults.py`** - Default value from Superset's built-in configuration
This helps you understand the configuration precedence and troubleshoot configuration issues.
## Configuration Reference
The following table shows all documented configuration settings with their metadata:
import ConfigurationTable from '@site/src/components/ConfigurationTable';
<ConfigurationTable showEnvironmentVariables={true} />
## Environment Variables Examples
Here are ready-to-use environment variable examples:
import EnvironmentVariablesExample from '@site/src/components/EnvironmentVariablesExample';
<EnvironmentVariablesExample />
## Configuration Metadata and Documentation
Superset's configuration system includes rich metadata for many settings, providing:
- **Type information** - Whether a setting expects an integer, string, boolean, or object
- **Validation rules** - Minimum/maximum values, allowed options
- **Documentation** - Detailed descriptions of what each setting does
- **Impact levels** - How significant changes to this setting are
- **Restart requirements** - Whether changing this setting requires a restart
This metadata is used for:
- **CLI documentation** - The `superset config env-examples` command shows this information
- **Future admin UI** - Settings management interface (coming soon)
- **Validation** - Ensuring configuration values are valid
- **API documentation** - Automatic generation of configuration schemas
You can also access this information via CLI:
```bash
superset config env-examples
```
+3 -6
View File
@@ -28,9 +28,6 @@ const globals = require('globals');
const { defineConfig, globalIgnores } = require('eslint/config');
module.exports = defineConfig([
{
files: ['**/*.{js,jsx,ts,tsx}'],
},
globalIgnores(['build/**/*', '.docusaurus/**/*', 'node_modules/**/*']),
js.configs.recommended,
...ts.configs.recommended,
@@ -39,7 +36,7 @@ module.exports = defineConfig([
files: ['eslint.config.js'],
rules: {
'@typescript-eslint/no-require-imports': 'off',
},
}
},
{
languageOptions: {
@@ -71,5 +68,5 @@ module.exports = defineConfig([
version: 'detect',
},
},
},
]);
}
])
+13 -9
View File
@@ -6,8 +6,9 @@
"scripts": {
"docusaurus": "docusaurus",
"_init": "cat src/intro_header.txt ../README.md > docs/intro.md",
"start": "yarn run _init && docusaurus start",
"build": "yarn run _init && DEBUG=docusaurus:* docusaurus build",
"_update-config": "bash scripts/generate_docs.sh",
"start": "yarn run _init && yarn run _update-config && docusaurus start",
"build": "yarn run _init && yarn run _update-config && DEBUG=docusaurus:* docusaurus build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
@@ -15,7 +16,7 @@
"write-translations": "docusaurus write-translations",
"write-heading-ids": "docusaurus write-heading-ids",
"typecheck": "tsc",
"eslint": "eslint ."
"eslint": "eslint . --ext .js,.jsx,.ts,.tsx"
},
"dependencies": {
"@ant-design/icons": "^6.0.0",
@@ -26,33 +27,36 @@
"@emotion/styled": "^10.0.27",
"@saucelabs/theme-github-codeblock": "^0.3.0",
"@superset-ui/style": "^0.14.23",
"antd": "^5.26.7",
"ag-grid-community": "^34.1.0",
"ag-grid-react": "^34.1.0",
"antd": "^5.26.3",
"docusaurus-plugin-less": "^2.0.2",
"less": "^4.4.0",
"less": "^4.3.0",
"less-loader": "^12.3.0",
"prism-react-renderer": "^2.4.1",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-github-btn": "^1.4.0",
"react-markdown": "^10.1.0",
"react-svg-pan-zoom": "^3.13.1",
"swagger-ui-react": "^5.27.1"
"swagger-ui-react": "^5.26.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.8.1",
"@docusaurus/tsconfig": "^3.8.1",
"@eslint/js": "^9.32.0",
"@eslint/js": "^9.31.0",
"@types/react": "^19.1.8",
"@typescript-eslint/eslint-plugin": "^8.37.0",
"@typescript-eslint/parser": "^8.37.0",
"eslint": "^9.31.0",
"eslint-config-prettier": "^10.1.8",
"eslint-config-prettier": "^10.1.5",
"eslint-plugin-prettier": "^5.5.1",
"eslint-plugin-react": "^7.37.5",
"globals": "^16.3.0",
"prettier": "^3.6.2",
"typescript": "~5.8.3",
"typescript-eslint": "^8.37.0",
"webpack": "^5.101.0"
"webpack": "^5.99.9"
},
"browserslist": {
"production": [
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""
Export configuration metadata to JSON for documentation generation.
This script loads configuration metadata from the Python metadata module
and exports it in JSON format for the documentation React components.
This script is called by docs/scripts/generate_docs.sh as part of the
unified documentation generation process.
"""
import json as json_module
import sys
from pathlib import Path
from typing import Any, Dict, List
# Add the superset directory to Python path
superset_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(superset_root))
def infer_impact(key: str) -> str:
"""Infer the impact level based on the configuration key name."""
name_lower = key.lower()
# High impact - security, database, core functionality
if any(
term in name_lower
for term in [
"secret",
"key",
"password",
"database",
"uri",
"url",
"security",
"auth",
]
):
return "high"
# Medium impact - performance, features, UI
elif any(
term in name_lower
for term in ["limit", "timeout", "cache", "feature", "flag", "theme"]
):
return "medium"
# Low impact - logging, debugging, minor settings
else:
return "low"
def infer_requires_restart(key: str) -> bool:
"""Infer if the configuration requires a restart based on the key name."""
name_lower = key.lower()
# These typically require restart
if any(
term in name_lower
for term in [
"secret",
"key",
"database",
"uri",
"url",
"security",
"auth",
"ssl",
"tls",
]
):
return True
# These typically don't require restart
elif any(
term in name_lower for term in ["limit", "timeout", "cache", "log", "debug"]
):
return False
# Default to requiring restart for safety
return True
def export_config_metadata() -> List[Dict[str, Any]]:
"""Export configuration metadata as JSON."""
try:
# Import from Python metadata module
from superset.config_metadata import export_for_documentation
# Get metadata from Python source
metadata_export = export_for_documentation()
# Export as JSON for documentation
output_dir = Path(__file__).parent.parent / "src" / "resources"
output_dir.mkdir(exist_ok=True)
# Write the full export (includes categories, etc.)
with open(output_dir / "config_metadata.json", "w") as f:
json_module.dump(metadata_export, f, indent=2)
output_file = output_dir / "config_metadata.json"
print(
f"Exported {len(metadata_export['all_settings'])} configuration settings to {output_file}"
)
return metadata_export["all_settings"]
except ImportError as e:
print(f"Error importing config_metadata: {e}")
print("Please ensure superset/config_metadata.py exists")
return []
if __name__ == "__main__":
export_config_metadata()
+101
View File
@@ -0,0 +1,101 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Unified documentation generation script
# This script generates all dynamic documentation artifacts needed for the docs build
set -e
echo "🚀 Generating documentation artifacts..."
# Navigate to the docs directory
cd "$(dirname "$0")/.."
# Track any failures
FAILED_TASKS=()
# 1. Extract configuration schema and export metadata
echo "📊 Extracting configuration schema and exporting metadata..."
if python ../scripts/extract_config_schema.py && python scripts/export_config_metadata.py; then
echo "✅ Configuration metadata exported successfully"
else
echo "⚠️ Warning: Failed to export configuration metadata"
echo " The documentation build will continue with existing metadata"
FAILED_TASKS+=("config_metadata")
fi
# 2. Generate OpenAPI documentation
echo "🔌 Generating OpenAPI documentation..."
if python -c "
import sys
sys.path.insert(0, '..')
from superset.app import create_app
from superset.cli.update import update_api_docs
import os
# Set required environment variables
os.environ['SUPERSET_SECRET_KEY'] = 'not-a-secret'
app = create_app()
with app.app_context():
update_api_docs()
"; then
echo "✅ OpenAPI documentation generated successfully"
else
echo "⚠️ Warning: Failed to generate OpenAPI documentation"
echo " The documentation build will continue with existing OpenAPI spec"
FAILED_TASKS+=("openapi")
fi
# 3. Generate ERD (Entity Relationship Diagram) if in CI environment
if [ -n "$CI" ] && [ -f "../scripts/erd/erd.py" ]; then
echo "🗂️ Generating Entity Relationship Diagram..."
if python ../scripts/erd/erd.py; then
echo "✅ ERD generated successfully"
else
echo "⚠️ Warning: Failed to generate ERD"
echo " The documentation build will continue without updated ERD"
FAILED_TASKS+=("erd")
fi
fi
# Summary
echo ""
echo "📝 Documentation generation summary:"
echo " - Configuration metadata: ${FAILED_TASKS[*]}" | grep -q "config_metadata" && echo " - Configuration metadata: ❌ Failed" || echo " - Configuration metadata: ✅ Success"
echo " - OpenAPI documentation: ${FAILED_TASKS[*]}" | grep -q "openapi" && echo " - OpenAPI documentation: ❌ Failed" || echo " - OpenAPI documentation: ✅ Success"
if [ -n "$CI" ]; then
echo " - ERD generation: ${FAILED_TASKS[*]}" | grep -q "erd" && echo " - ERD generation: ❌ Failed" || echo " - ERD generation: ✅ Success"
fi
if [ ${#FAILED_TASKS[@]} -eq 0 ]; then
echo ""
echo "🎉 All documentation artifacts generated successfully!"
else
echo ""
echo "⚠️ Some tasks failed but documentation build can continue"
echo " Failed tasks: ${FAILED_TASKS[*]}"
echo " To fix missing dependencies, run: pip install -e ."
fi
echo ""
echo "📁 Generated files:"
[ -f "src/resources/config_metadata.json" ] && echo " - src/resources/config_metadata.json"
[ -f "static/resources/openapi.json" ] && echo " - static/resources/openapi.json"
[ -f "static/img/erd.svg" ] && echo " - static/img/erd.svg"
+329
View File
@@ -0,0 +1,329 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import React, { useState, useMemo, useCallback } from 'react';
import { AgGridReact } from 'ag-grid-react';
import { ColDef, GridReadyEvent, GridApi, ModuleRegistry, AllCommunityModule } from 'ag-grid-community';
import 'ag-grid-community/styles/ag-grid.css';
import 'ag-grid-community/styles/ag-theme-material.css';
import configMetadata from '../resources/config_metadata.json';
// Register AG Grid modules
ModuleRegistry.registerModules([AllCommunityModule]);
// ConfigSetting interface is defined for type safety but not directly used
// as AG Grid uses dynamic property access
// interface ConfigSetting {
// key: string;
// title: string;
// description: string;
// details: string;
// type: string;
// category: string;
// group: string;
// default: any;
// env_var: string;
// external: boolean;
// source: string;
// supports_callable: boolean;
// }
interface ConfigurationTableProps {
category?: string;
showEnvironmentVariables?: boolean;
}
// Custom cell renderers
const KeyCellRenderer = (props: { value: string }) => {
return <span style={{ fontWeight: 'bold' }}>{props.value}</span>;
};
const TypeCellRenderer = (props: { value: string }) => {
return <code>{props.value}</code>;
};
const DefaultCellRenderer = (props: { value: unknown }) => {
const formatDefault = (value: unknown): string => {
if (value === null || value === undefined || value === 'None') return 'None';
if (typeof value === 'object') {
try {
return JSON.stringify(value, null, 2);
} catch {
return String(value);
}
}
return String(value);
};
const formatted = formatDefault(props.value);
const isLong = formatted.length > 50;
return (
<code
style={{
whiteSpace: isLong ? 'pre-wrap' : 'nowrap',
wordBreak: isLong ? 'break-all' : 'normal',
}}
title={isLong ? formatted : undefined}
>
{isLong ? formatted.substring(0, 50) + '...' : formatted}
</code>
);
};
const BooleanCellRenderer = (props: { value: boolean }) => {
return props.value ? '✅ Yes' : '❌ No';
};
const GroupCellRenderer = (props: { value: string | null }) => {
if (!props.value) return null;
return (
<span
style={{
backgroundColor: '#f0f0f0',
padding: '2px 8px',
borderRadius: '4px',
fontSize: '0.9em',
}}
>
{props.value}
</span>
);
};
const DescriptionCellRenderer = (props: { value: string; data: { details?: string } }) => {
const hasDetails = props.data.details && props.data.details.trim() !== '';
return (
<div style={{ display: 'flex', alignItems: 'center', gap: '6px' }}>
<span>{props.value || 'No description available'}</span>
{hasDetails && (
<span
style={{
display: 'inline-flex',
alignItems: 'center',
justifyContent: 'center',
width: '16px',
height: '16px',
backgroundColor: '#e8e8e8',
color: '#666',
borderRadius: '50%',
fontSize: '0.8em',
fontWeight: 'bold',
cursor: 'help',
flexShrink: 0,
border: '1px solid #d0d0d0',
}}
title={props.data.details}
>
i
</span>
)}
</div>
);
};
const ConfigurationTable: React.FC<ConfigurationTableProps> = ({
category, // eslint-disable-line @typescript-eslint/no-unused-vars
showEnvironmentVariables = false,
}) => {
const [gridApi, setGridApi] = useState<GridApi | null>(null);
const [searchText, setSearchText] = useState('');
// Process data to include only enriched configs
const rowData = useMemo(() => {
return configMetadata.all_settings;
}, []);
// Column definitions
const columnDefs = useMemo<ColDef[]>(() => {
const columns: ColDef[] = [
{
field: 'key',
headerName: 'Configuration Key',
cellRenderer: KeyCellRenderer,
width: 280,
pinned: 'left',
filter: 'agTextColumnFilter',
floatingFilter: true,
},
{
field: 'description',
headerName: 'Description',
cellRenderer: DescriptionCellRenderer,
flex: 2,
minWidth: 300,
wrapText: true,
autoHeight: true,
filter: 'agTextColumnFilter',
floatingFilter: true,
},
{
field: 'type',
headerName: 'Type',
cellRenderer: TypeCellRenderer,
width: 120,
filter: 'agTextColumnFilter',
},
{
field: 'default',
headerName: 'Default',
cellRenderer: DefaultCellRenderer,
width: 200,
filter: 'agTextColumnFilter',
},
{
field: 'category',
headerName: 'Category',
width: 120,
filter: 'agTextColumnFilter',
floatingFilter: true,
},
{
field: 'group',
headerName: 'Group',
cellRenderer: GroupCellRenderer,
width: 180,
filter: 'agTextColumnFilter',
floatingFilter: true,
},
];
if (showEnvironmentVariables) {
columns.push({
field: 'env_var',
headerName: 'Environment Variable',
width: 250,
filter: 'agTextColumnFilter',
cellRenderer: (props: { value: string }) => (
<code>{props.value}</code>
),
});
}
columns.push(
{
field: 'external',
headerName: 'External',
cellRenderer: BooleanCellRenderer,
width: 100,
filter: true,
},
);
return columns;
}, [showEnvironmentVariables]);
const defaultColDef = useMemo<ColDef>(() => ({
sortable: true,
resizable: true,
}), []);
const onGridReady = useCallback((params: GridReadyEvent) => {
setGridApi(params.api);
}, []);
const onFilterTextBoxChanged = useCallback(() => {
if (gridApi) {
gridApi.setGridOption('quickFilterText', searchText);
}
}, [gridApi, searchText]);
const exportToCsv = useCallback(() => {
if (gridApi) {
gridApi.exportDataAsCsv({
fileName: 'superset_configuration.csv',
});
}
}, [gridApi]);
return (
<div style={{ width: '100%', height: '800px' }}>
{/* Controls */}
<div style={{ marginBottom: '20px', display: 'flex', gap: '15px', alignItems: 'center' }}>
<div style={{ flex: 1 }}>
<input
type="text"
placeholder="Quick filter across all columns..."
value={searchText}
onChange={(e) => {
setSearchText(e.target.value);
onFilterTextBoxChanged();
}}
style={{
padding: '8px 12px',
border: '1px solid #ddd',
borderRadius: '4px',
width: '100%',
maxWidth: '400px',
}}
/>
</div>
<button
onClick={exportToCsv}
style={{
padding: '8px 16px',
backgroundColor: '#1890ff',
color: 'white',
border: 'none',
borderRadius: '4px',
cursor: 'pointer',
fontSize: '14px',
}}
>
Export to CSV
</button>
<div style={{ color: '#666' }}>
{rowData.length} configurations
</div>
</div>
{/* AG Grid */}
<div className="ag-theme-material" style={{ height: '100%', width: '100%' }}>
<AgGridReact
rowData={rowData}
columnDefs={columnDefs}
defaultColDef={defaultColDef}
onGridReady={onGridReady}
animateRows={true}
enableCellTextSelection={true}
ensureDomOrder={true}
tooltipShowDelay={500}
pagination={true}
paginationPageSize={50}
paginationPageSizeSelector={[20, 50, 100, 200]}
/>
</div>
{/* Help text */}
<div style={{ marginTop: '15px', color: '#666' }}>
<p>
<strong>Tips:</strong> Click column headers to sort. Use the filter row below headers for column-specific filtering.
Hold Shift to sort by multiple columns. Right-click headers for more options.
</p>
</div>
</div>
);
};
export default ConfigurationTable;
@@ -0,0 +1,181 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import React, { useState } from 'react';
import configMetadata from '../resources/config_metadata.json';
interface EnvironmentVariablesExampleProps {
category?: string;
}
const EnvironmentVariablesExample: React.FC<
EnvironmentVariablesExampleProps
> = ({ category }) => {
const [showAll, setShowAll] = useState(false);
// Get settings based on category
const getSettings = () => {
if (category && configMetadata.by_category[category]) {
return configMetadata.by_category[category];
}
return configMetadata.all_settings;
};
const settings = getSettings();
const displaySettings = showAll ? settings : settings.slice(0, 5);
const formatDefaultForEnv = (value: unknown): string => {
if (value === null || value === undefined) return '""';
if (typeof value === 'object') {
return `'${JSON.stringify(value)}'`;
}
if (typeof value === 'string') {
return `"${value}"`;
}
return String(value);
};
const copyToClipboard = (text: string) => {
navigator.clipboard.writeText(text);
};
const generateEnvExample = (setting: { default: unknown; env_var: string }): string => {
const example = formatDefaultForEnv(setting.default);
return `export ${setting.env_var}=${example}`;
};
const generateAllEnvVars = (): string => {
return [
'# Superset Configuration Environment Variables',
'# Generated from configuration metadata',
'',
...displaySettings.map(setting =>
[
`# ${setting.title}`,
`# ${setting.description}`,
`# Type: ${setting.type}`,
`# Impact: ${setting.impact}${
setting.requires_restart ? ' (requires restart)' : ''
}`,
generateEnvExample(setting),
'',
].join('\n'),
),
].join('\n');
};
return (
<div style={{ margin: '20px 0' }}>
<div
style={{
backgroundColor: '#f6f8fa',
border: '1px solid #e1e4e8',
borderRadius: '6px',
padding: '16px',
position: 'relative',
}}
>
<div
style={{
display: 'flex',
justifyContent: 'space-between',
alignItems: 'center',
marginBottom: '10px',
}}
>
<h4 style={{ margin: 0, color: '#24292e' }}>
Environment Variables {category && `(${category})`}
</h4>
<button
onClick={() => copyToClipboard(generateAllEnvVars())}
style={{
backgroundColor: '#0366d6',
color: 'white',
border: 'none',
padding: '6px 12px',
borderRadius: '4px',
cursor: 'pointer',
fontSize: '12px',
}}
title="Copy all environment variables"
>
📋 Copy All
</button>
</div>
<pre
style={{
backgroundColor: '#f6f8fa',
border: 'none',
padding: '0',
margin: '0',
fontFamily:
'SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace',
fontSize: '12px',
lineHeight: '1.45',
overflow: 'auto',
maxHeight: '400px',
}}
>
<code>{generateAllEnvVars()}</code>
</pre>
{!showAll && settings.length > 5 && (
<div
style={{
textAlign: 'center',
marginTop: '10px',
borderTop: '1px solid #e1e4e8',
paddingTop: '10px',
}}
>
<button
onClick={() => setShowAll(true)}
style={{
backgroundColor: 'transparent',
border: '1px solid #0366d6',
color: '#0366d6',
padding: '6px 12px',
borderRadius: '4px',
cursor: 'pointer',
fontSize: '12px',
}}
>
Show all {settings.length} settings
</button>
</div>
)}
</div>
<div
style={{
marginTop: '10px',
fontSize: '14px',
color: '#586069',
}}
>
<strong>Usage:</strong> Save to a <code>.env</code> file or export
directly in your shell.
{category && ` Showing ${settings.length} ${category} settings.`}
</div>
</div>
);
};
export default EnvironmentVariablesExample;
File diff suppressed because it is too large Load Diff
+2835 -58
View File
File diff suppressed because it is too large Load Diff
+113 -62
View File
@@ -2205,16 +2205,11 @@
minimatch "^3.1.2"
strip-json-comments "^3.1.1"
"@eslint/js@9.31.0":
"@eslint/js@9.31.0", "@eslint/js@^9.31.0":
version "9.31.0"
resolved "https://registry.yarnpkg.com/@eslint/js/-/js-9.31.0.tgz#adb1f39953d8c475c4384b67b67541b0d7206ed8"
integrity sha512-LOm5OVt7D4qiKCqoiPbA7LWmI+tbw1VbTUowBcUMgQSuM6poJufkFkYDcQpo5KfgD39TnNySV26QjOh7VFpSyw==
"@eslint/js@^9.32.0":
version "9.32.0"
resolved "https://registry.yarnpkg.com/@eslint/js/-/js-9.32.0.tgz#a02916f58bd587ea276876cb051b579a3d75d091"
integrity sha512-BBpRFZK3eX6uMLKz8WxFOBIFFcGFJ/g8XuwjTHCqHROSIsopI+ddn/d5Cfh36+7+e5edVS8dbSHnBNhrLEX0zg==
"@eslint/object-schema@^2.1.6":
version "2.1.6"
resolved "https://registry.yarnpkg.com/@eslint/object-schema/-/object-schema-2.1.6.tgz#58369ab5b5b3ca117880c0f6c0b0f32f6950f24f"
@@ -2517,10 +2512,10 @@
classnames "^2.3.2"
rc-util "^5.24.4"
"@rc-component/trigger@^2.0.0", "@rc-component/trigger@^2.1.1", "@rc-component/trigger@^2.3.0":
version "2.3.0"
resolved "https://registry.yarnpkg.com/@rc-component/trigger/-/trigger-2.3.0.tgz#9499ada078daca9dd99d01f0f0743ee1ab9e398b"
integrity sha512-iwaxZyzOuK0D7lS+0AQEtW52zUWxoGqTGkke3dRyb8pYiShmRpCjB/8TzPI4R6YySCH7Vm9BZj/31VPiiQTLBg==
"@rc-component/trigger@^2.0.0", "@rc-component/trigger@^2.1.1", "@rc-component/trigger@^2.2.7":
version "2.2.7"
resolved "https://registry.yarnpkg.com/@rc-component/trigger/-/trigger-2.2.7.tgz#a2b97ecbb93280a3c424e51fa415b371b355d76a"
integrity sha512-Qggj4Z0AA2i5dJhzlfFSmg1Qrziu8dsdHOihROL5Kl18seO2Eh/ZaTYt2c8a/CyGaTChnFry7BEYew1+/fhSbA==
dependencies:
"@babel/runtime" "^7.23.2"
"@rc-component/portal" "^1.1.0"
@@ -3427,10 +3422,10 @@
dependencies:
"@types/estree" "*"
"@types/estree@*", "@types/estree@^1.0.0", "@types/estree@^1.0.6", "@types/estree@^1.0.8":
version "1.0.8"
resolved "https://registry.yarnpkg.com/@types/estree/-/estree-1.0.8.tgz#958b91c991b1867ced318bedea0e215ee050726e"
integrity sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==
"@types/estree@*", "@types/estree@^1.0.0", "@types/estree@^1.0.6":
version "1.0.7"
resolved "https://registry.yarnpkg.com/@types/estree/-/estree-1.0.7.tgz#4158d3105276773d5b7695cd4834b1722e4f37a8"
integrity sha512-w28IoSUCJpidD/TGviZwwMJckNESJZXFu7NBZ5YJ4mEUnNraUn9Pm8HSZm/jDF1pDWYKspWE7oVphigUPRakIQ==
"@types/express-serve-static-core@*", "@types/express-serve-static-core@^5.0.0":
version "5.0.6"
@@ -3971,11 +3966,6 @@ accepts@~1.3.4, accepts@~1.3.8:
mime-types "~2.1.34"
negotiator "0.6.3"
acorn-import-phases@^1.0.3:
version "1.0.4"
resolved "https://registry.yarnpkg.com/acorn-import-phases/-/acorn-import-phases-1.0.4.tgz#16eb850ba99a056cb7cbfe872ffb8972e18c8bd7"
integrity sha512-wKmbr/DDiIXzEOiWrTTUcDm24kQ2vGfZQvM2fwg2vXqR5uW6aapr7ObPtj1th32b9u90/Pf4AItvdTh42fBmVQ==
acorn-jsx@^5.0.0, acorn-jsx@^5.3.2:
version "5.3.2"
resolved "https://registry.yarnpkg.com/acorn-jsx/-/acorn-jsx-5.3.2.tgz#7ed5bb55908b3b2f1bc55c6af1653bada7f07937"
@@ -3988,7 +3978,12 @@ acorn-walk@^8.0.0:
dependencies:
acorn "^8.11.0"
acorn@^8.0.0, acorn@^8.0.4, acorn@^8.11.0, acorn@^8.14.0, acorn@^8.15.0, acorn@^8.8.2:
acorn@^8.0.0, acorn@^8.0.4, acorn@^8.11.0, acorn@^8.14.0, acorn@^8.8.2:
version "8.14.1"
resolved "https://registry.yarnpkg.com/acorn/-/acorn-8.14.1.tgz#721d5dc10f7d5b5609a891773d47731796935dfb"
integrity sha512-OvQ/2pUDKmgfCg++xsTX1wGxfTaszcHVcTctW4UJB4hibJx2HXxxO5UmVgyjMa+ZDsiaf5wWLXYpRWMmBI0QHg==
acorn@^8.15.0:
version "8.15.0"
resolved "https://registry.yarnpkg.com/acorn/-/acorn-8.15.0.tgz#a360898bc415edaac46c8241f6383975b930b816"
integrity sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==
@@ -3998,6 +3993,26 @@ address@^1.0.1:
resolved "https://registry.yarnpkg.com/address/-/address-1.2.2.tgz#2b5248dac5485a6390532c6a517fda2e3faac89e"
integrity sha512-4B/qKCfeE/ODUaAUpSwfzazo5x29WD4r3vXiWsB7I2mSDAihwEqKO+g8GELZUQSSAo5e1XTYh3ZVfLyxBc12nA==
ag-charts-types@12.1.0:
version "12.1.0"
resolved "https://registry.yarnpkg.com/ag-charts-types/-/ag-charts-types-12.1.0.tgz#75104b90e5f6ae01b7248ec3f6a8dabc65c3cbb6"
integrity sha512-qeODwJ1EqKjpwEbp0mQ2wQ0arRNYaZo2BafdAGfcuOwjOBlagSwJvUg5MCvAYZ/W/mg2uEmt7jKMNfDy4ul4+Q==
ag-grid-community@34.1.0, ag-grid-community@^34.1.0:
version "34.1.0"
resolved "https://registry.yarnpkg.com/ag-grid-community/-/ag-grid-community-34.1.0.tgz#6356562b3a544a50bbab6a3d0929029567bbd7bc"
integrity sha512-3rZiOyyCGqSNqqTsrWafDVj1WfK43jfb53Ka5sqzdOG/yu6ySUFmdc0h/OuGLnkzwW5PC29coQwbS2rkb4c9dA==
dependencies:
ag-charts-types "12.1.0"
ag-grid-react@^34.1.0:
version "34.1.0"
resolved "https://registry.yarnpkg.com/ag-grid-react/-/ag-grid-react-34.1.0.tgz#9d89a75f5994a5187cbdf1f44132eb06c2741623"
integrity sha512-CY1p4/JnvcwOt2HipmsqME9CWz7M21nb3OB1DhJGOvNUaxo1wF6Hb/pKpa20F3F/E93wQCNmCz+gfrSLPuJrQw==
dependencies:
ag-grid-community "34.1.0"
prop-types "^15.8.1"
aggregate-error@^3.0.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/aggregate-error/-/aggregate-error-3.1.0.tgz#92670ff50f5359bdb7a3e0d40d0ec30c5737687a"
@@ -4112,10 +4127,10 @@ ansi-styles@^6.1.0:
resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-6.2.1.tgz#0e62320cf99c21afff3b3012192546aacbfb05c5"
integrity sha512-bN798gFfQX+viw3R7yrGWRqnrN2oRkEkUjjl4JNn4E8GxxbjtG3FbrEIIY3l8/hrwUwIeCZvi4QuOTP4MErVug==
antd@^5.26.7:
version "5.26.7"
resolved "https://registry.yarnpkg.com/antd/-/antd-5.26.7.tgz#e2f7e37330b27eec0de7a7789767975373f61602"
integrity sha512-iCyXN6+i2CUVEOSzzJKfbKeg115qoJhGvSkCh5uzAf9hANwHUOJQhsMn+KtN+Lx/2NQ6wfM7nGZ+7NPNO5Pn1w==
antd@^5.26.3:
version "5.26.3"
resolved "https://registry.yarnpkg.com/antd/-/antd-5.26.3.tgz#cbbb7e1b48a972dc7b6ee8b6948f51cc91c263f8"
integrity sha512-M/s9Q39h/+G7AWnS6fbNxmAI9waTH4ti022GVEXBLq2j810V1wJ3UOQps13nEilzDNcyxnFN/EIbqIgS7wSYaA==
dependencies:
"@ant-design/colors" "^7.2.1"
"@ant-design/cssinjs" "^1.23.0"
@@ -4128,7 +4143,7 @@ antd@^5.26.7:
"@rc-component/mutate-observer" "^1.1.0"
"@rc-component/qrcode" "~1.0.0"
"@rc-component/tour" "~1.15.1"
"@rc-component/trigger" "^2.3.0"
"@rc-component/trigger" "^2.2.7"
classnames "^2.5.1"
copy-to-clipboard "^3.3.3"
dayjs "^1.11.11"
@@ -4158,7 +4173,7 @@ antd@^5.26.7:
rc-switch "~4.1.0"
rc-table "~7.51.1"
rc-tabs "~15.6.1"
rc-textarea "~1.10.1"
rc-textarea "~1.10.0"
rc-tooltip "~6.4.0"
rc-tree "~5.13.1"
rc-tree-select "~5.27.0"
@@ -4513,7 +4528,17 @@ braces@^3.0.3, braces@~3.0.2:
dependencies:
fill-range "^7.1.1"
browserslist@^4.0.0, browserslist@^4.23.0, browserslist@^4.24.0, browserslist@^4.24.4, browserslist@^4.25.0:
browserslist@^4.0.0, browserslist@^4.23.0, browserslist@^4.24.0, browserslist@^4.24.4:
version "4.24.4"
resolved "https://registry.yarnpkg.com/browserslist/-/browserslist-4.24.4.tgz#c6b2865a3f08bcb860a0e827389003b9fe686e4b"
integrity sha512-KDi1Ny1gSePi1vm0q4oxSF8b4DR44GF4BbmS2YdhPLOEqd8pDviZOGH/GsmRwoWJ2+5Lr085X7naowMwKHDG1A==
dependencies:
caniuse-lite "^1.0.30001688"
electron-to-chromium "^1.5.73"
node-releases "^2.0.19"
update-browserslist-db "^1.1.1"
browserslist@^4.25.0:
version "4.25.0"
resolved "https://registry.yarnpkg.com/browserslist/-/browserslist-4.25.0.tgz#986aa9c6d87916885da2b50d8eb577ac8d133b2c"
integrity sha512-PJ8gYKeS5e/whHBh8xrwYK+dAvEj7JXtz6uTucnMRB8OiGTsKccFekoRrjajPBHV8oOY+2tI4uxeceSimKwMFA==
@@ -4615,7 +4640,7 @@ caniuse-api@^3.0.0:
lodash.memoize "^4.1.2"
lodash.uniq "^4.5.0"
caniuse-lite@^1.0.0, caniuse-lite@^1.0.30001702:
caniuse-lite@^1.0.0, caniuse-lite@^1.0.30001688, caniuse-lite@^1.0.30001702:
version "1.0.30001714"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001714.tgz#cfd27ff07e6fa20a0f45c7a10d28a0ffeaba2122"
integrity sha512-mtgapdwDLSSBnCI3JokHM7oEQBLxiJKVRtg10AxM1AyeiKcM96f0Mkbqeq+1AbiCtvMcHRulAAEMu693JrSWqg==
@@ -5898,6 +5923,11 @@ electron-to-chromium@^1.5.160:
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.5.170.tgz#9f6697de4339e24da8b234e4492a9ecb91f5989c"
integrity sha512-GP+M7aeluQo9uAyiTCxgIj/j+PrWhMlY7LFVj8prlsPljd0Fdg9AprlfUi+OCSFWy9Y5/2D/Jrj9HS8Z4rpKWA==
electron-to-chromium@^1.5.73:
version "1.5.138"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.5.138.tgz#319e775179bd0889ed96a04d4390d355fb315a44"
integrity sha512-FWlQc52z1dXqm+9cCJ2uyFgJkESd+16j6dBEjsgDNuHjBpuIzL8/lRc0uvh1k8RNI6waGo6tcy2DvwkTBJOLDg==
emoji-regex@^8.0.0:
version "8.0.0"
resolved "https://registry.yarnpkg.com/emoji-regex/-/emoji-regex-8.0.0.tgz#e818fd69ce5ccfcb404594f842963bf53164cc37"
@@ -5942,10 +5972,10 @@ encodeurl@~2.0.0:
resolved "https://registry.yarnpkg.com/encodeurl/-/encodeurl-2.0.0.tgz#7b8ea898077d7e409d3ac45474ea38eaf0857a58"
integrity sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==
enhanced-resolve@^5.17.2:
version "5.18.2"
resolved "https://registry.yarnpkg.com/enhanced-resolve/-/enhanced-resolve-5.18.2.tgz#7903c5b32ffd4b2143eeb4b92472bd68effd5464"
integrity sha512-6Jw4sE1maoRJo3q8MsSIn2onJFbLTOjY9hlx4DZXmOKvLRd1Ok2kXmAGXaafL2+ijsJZ1ClYbl/pmqr9+k4iUQ==
enhanced-resolve@^5.17.1:
version "5.18.1"
resolved "https://registry.yarnpkg.com/enhanced-resolve/-/enhanced-resolve-5.18.1.tgz#728ab082f8b7b6836de51f1637aab5d3b9568faf"
integrity sha512-ZSW3ma5GkcQBIpwZTSRAI8N71Uuwgs93IezB7mf7R60tC8ZbJideoDNKjHn2O9KIlx6rkGTTEk1xUCK2E1Y2Yg==
dependencies:
graceful-fs "^4.2.4"
tapable "^2.2.0"
@@ -6151,10 +6181,10 @@ escape-string-regexp@^5.0.0:
resolved "https://registry.yarnpkg.com/escape-string-regexp/-/escape-string-regexp-5.0.0.tgz#4683126b500b61762f2dbebace1806e8be31b1c8"
integrity sha512-/veY75JbMK4j1yjvuUxuVsiS/hr/4iHs9FTT6cgTexxdE0Ly/glccBAkloH/DofkjRbZU3bnoj38mOmhkZ0lHw==
eslint-config-prettier@^10.1.8:
version "10.1.8"
resolved "https://registry.yarnpkg.com/eslint-config-prettier/-/eslint-config-prettier-10.1.8.tgz#15734ce4af8c2778cc32f0b01b37b0b5cd1ecb97"
integrity sha512-82GZUjRS0p/jganf6q1rEO25VSoHH0hKPCTrgillPjdI/3bgBhAE1QzHrHTizjpRvy6pGAvKjDJtk2pF9NDq8w==
eslint-config-prettier@^10.1.5:
version "10.1.5"
resolved "https://registry.yarnpkg.com/eslint-config-prettier/-/eslint-config-prettier-10.1.5.tgz#00c18d7225043b6fbce6a665697377998d453782"
integrity sha512-zc1UmCpNltmVY34vuLRV61r1K27sWuX39E+uyUnY8xS2Bex88VV9cugG+UZbRSRGtGyFboj+D8JODyme1plMpw==
eslint-plugin-prettier@^5.5.1:
version "5.5.1"
@@ -7210,6 +7240,11 @@ html-tags@^3.3.1:
resolved "https://registry.yarnpkg.com/html-tags/-/html-tags-3.3.1.tgz#a04026a18c882e4bba8a01a3d39cfe465d40b5ce"
integrity sha512-ztqyC3kLto0e9WbNp0aeP+M3kTt+nbaIveGmUxAtZa+8iFgKLUOD4YKM5j+f3QD89bra7UeumolZHKuOXnTmeQ==
html-url-attributes@^3.0.0:
version "3.0.1"
resolved "https://registry.yarnpkg.com/html-url-attributes/-/html-url-attributes-3.0.1.tgz#83b052cd5e437071b756cd74ae70f708870c2d87"
integrity sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ==
html-void-elements@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/html-void-elements/-/html-void-elements-3.0.0.tgz#fc9dbd84af9e747249034d4d62602def6517f1d7"
@@ -8053,10 +8088,10 @@ less-loader@^12.3.0:
resolved "https://registry.yarnpkg.com/less-loader/-/less-loader-12.3.0.tgz#d4a00361568be86a97da3df4f16954b0d4c15340"
integrity sha512-0M6+uYulvYIWs52y0LqN4+QM9TqWAohYSNTo4htE8Z7Cn3G/qQMEmktfHmyJT23k+20kU9zHH2wrfFXkxNLtVw==
less@^4.4.0:
version "4.4.0"
resolved "https://registry.yarnpkg.com/less/-/less-4.4.0.tgz#deaf881f4880ee80691beae925b8fac699d3a76d"
integrity sha512-kdTwsyRuncDfjEs0DlRILWNvxhDG/Zij4YLO4TMJgDLW+8OzpfkdPnRgrsRuY1o+oaxJGWsps5f/RVBgGmmN0w==
less@^4.3.0:
version "4.3.0"
resolved "https://registry.yarnpkg.com/less/-/less-4.3.0.tgz#ef0cfc260a9ca8079ed8d0e3512bda8a12c82f2a"
integrity sha512-X9RyH9fvemArzfdP8Pi3irr7lor2Ok4rOttDXBhlwDg+wKQsXOXgHWduAJE1EsF7JJx0w0bcO6BC6tCKKYnXKA==
dependencies:
copy-anything "^2.0.1"
parse-node-version "^1.0.1"
@@ -10704,10 +10739,10 @@ rc-tabs@~15.6.1:
rc-resize-observer "^1.0.0"
rc-util "^5.34.1"
rc-textarea@~1.10.0, rc-textarea@~1.10.1:
version "1.10.2"
resolved "https://registry.yarnpkg.com/rc-textarea/-/rc-textarea-1.10.2.tgz#459e3574a95c32939c6793045a1e4db04cb514cc"
integrity sha512-HfaeXiaSlpiSp0I/pvWpecFEHpVysZ9tpDLNkxQbMvMz6gsr7aVZ7FpWP9kt4t7DB+jJXesYS0us1uPZnlRnwQ==
rc-textarea@~1.10.0:
version "1.10.0"
resolved "https://registry.yarnpkg.com/rc-textarea/-/rc-textarea-1.10.0.tgz#f8f962ef83be0b8e35db97cf03dbfb86ddd9c46c"
integrity sha512-ai9IkanNuyBS4x6sOL8qu/Ld40e6cEs6pgk93R+XLYg0mDSjNBGey6/ZpDs5+gNLD7urQ14po3V6Ck2dJLt9SA==
dependencies:
"@babel/runtime" "^7.10.1"
classnames "^2.2.1"
@@ -10877,6 +10912,23 @@ react-loadable-ssr-addon-v5-slorber@^1.0.1:
dependencies:
"@types/react" "*"
react-markdown@^10.1.0:
version "10.1.0"
resolved "https://registry.yarnpkg.com/react-markdown/-/react-markdown-10.1.0.tgz#e22bc20faddbc07605c15284255653c0f3bad5ca"
integrity sha512-qKxVopLT/TyA6BX3Ue5NwabOsAzm0Q7kAPwq6L+wWDwisYs7R8vZ0nRXqq6rkueboxpkjvLGU9fWifiX/ZZFxQ==
dependencies:
"@types/hast" "^3.0.0"
"@types/mdast" "^4.0.0"
devlop "^1.0.0"
hast-util-to-jsx-runtime "^2.0.0"
html-url-attributes "^3.0.0"
mdast-util-to-hast "^13.0.0"
remark-parse "^11.0.0"
remark-rehype "^11.0.0"
unified "^11.0.0"
unist-util-visit "^5.0.0"
vfile "^6.0.0"
react-redux@^9.2.0:
version "9.2.0"
resolved "https://registry.yarnpkg.com/react-redux/-/react-redux-9.2.0.tgz#96c3ab23fb9a3af2cb4654be4b51c989e32366f5"
@@ -12090,10 +12142,10 @@ swagger-client@^3.35.5:
ramda "^0.30.1"
ramda-adjunct "^5.1.0"
swagger-ui-react@^5.27.1:
version "5.27.1"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.27.1.tgz#315b59970c33933a5f62ca0f702789741dcedc7c"
integrity sha512-wwDoavIeJI/Pwiavn32FMJ5dfptz0BAOKjSrj7EdU22QdP3gdk9+MZHdzzjxWURmVj0kc0XoQfsFgjln0toJaw==
swagger-ui-react@^5.26.0:
version "5.26.0"
resolved "https://registry.yarnpkg.com/swagger-ui-react/-/swagger-ui-react-5.26.0.tgz#b15a903d556cc0ec2a56a969beb9d5bc9ea52910"
integrity sha512-4e6bP9bdJyh+SqQW0lxulPn/SDno4+oWrKXsuon5Z9kjtV0zeoWEJ1c70Qxp8kN/c3caFwec8OyxDNhvo14pkw==
dependencies:
"@babel/runtime-corejs3" "^7.27.1"
"@scarf/scarf" "=1.4.0"
@@ -12515,7 +12567,7 @@ unraw@^3.0.0:
resolved "https://registry.npmjs.org/unraw/-/unraw-3.0.0.tgz"
integrity sha512-08/DA66UF65OlpUDIQtbJyrqTR0jTAlJ+jsnkQ4jxR7+K5g5YG1APZKQSMCE1vqqmD+2pv6+IdEjmopFatacvg==
update-browserslist-db@^1.1.3:
update-browserslist-db@^1.1.1, update-browserslist-db@^1.1.3:
version "1.1.3"
resolved "https://registry.yarnpkg.com/update-browserslist-db/-/update-browserslist-db-1.1.3.tgz#348377dd245216f9e7060ff50b15a1b740b75420"
integrity sha512-UxhIZQ+QInVdunkDAaiazvvT/+fXL5Osr0JZlJulepYu6Jd7qJtDZjlur0emRlT71EN3ScPoE7gvsuIKKNavKw==
@@ -12784,27 +12836,26 @@ webpack-merge@^6.0.1:
flat "^5.0.2"
wildcard "^2.0.1"
webpack-sources@^3.3.3:
version "3.3.3"
resolved "https://registry.yarnpkg.com/webpack-sources/-/webpack-sources-3.3.3.tgz#d4bf7f9909675d7a070ff14d0ef2a4f3c982c723"
integrity sha512-yd1RBzSGanHkitROoPFd6qsrxt+oFhg/129YzheDGqeustzX0vTZJZsSsQjVQC4yzBQ56K55XU8gaNCtIzOnTg==
webpack-sources@^3.2.3:
version "3.2.3"
resolved "https://registry.yarnpkg.com/webpack-sources/-/webpack-sources-3.2.3.tgz#2d4daab8451fd4b240cc27055ff6a0c2ccea0cde"
integrity sha512-/DyMEOrDgLKKIG0fmvtz+4dUX/3Ghozwgm6iPp8KRhvn+eQf9+Q7GWxVNMk3+uCPWfdXYC4ExGBckIXdFEfH1w==
webpack@^5.101.0, webpack@^5.88.1, webpack@^5.95.0:
version "5.101.0"
resolved "https://registry.yarnpkg.com/webpack/-/webpack-5.101.0.tgz#4b81407ffad9857f81ff03f872e3369b9198cc9d"
integrity sha512-B4t+nJqytPeuZlHuIKTbalhljIFXeNRqrUGAQgTGlfOl2lXXKXw+yZu6bicycP+PUlM44CxBjCFD6aciKFT3LQ==
webpack@^5.88.1, webpack@^5.95.0, webpack@^5.99.9:
version "5.99.9"
resolved "https://registry.yarnpkg.com/webpack/-/webpack-5.99.9.tgz#d7de799ec17d0cce3c83b70744b4aedb537d8247"
integrity sha512-brOPwM3JnmOa+7kd3NsmOUOwbDAj8FT9xDsG3IW0MgbN9yZV7Oi/s/+MNQ/EcSMqw7qfoRyXPoeEWT8zLVdVGg==
dependencies:
"@types/eslint-scope" "^3.7.7"
"@types/estree" "^1.0.8"
"@types/estree" "^1.0.6"
"@types/json-schema" "^7.0.15"
"@webassemblyjs/ast" "^1.14.1"
"@webassemblyjs/wasm-edit" "^1.14.1"
"@webassemblyjs/wasm-parser" "^1.14.1"
acorn "^8.15.0"
acorn-import-phases "^1.0.3"
acorn "^8.14.0"
browserslist "^4.24.0"
chrome-trace-event "^1.0.2"
enhanced-resolve "^5.17.2"
enhanced-resolve "^5.17.1"
es-module-lexer "^1.2.1"
eslint-scope "5.1.1"
events "^3.2.0"
@@ -12818,7 +12869,7 @@ webpack@^5.101.0, webpack@^5.88.1, webpack@^5.95.0:
tapable "^2.1.1"
terser-webpack-plugin "^5.3.11"
watchpack "^2.4.1"
webpack-sources "^3.3.3"
webpack-sources "^3.2.3"
webpackbar@^6.0.1:
version "6.0.1"
+291
View File
@@ -0,0 +1,291 @@
#!/usr/bin/env python3
"""
Extract configuration schema from config_defaults.py.
This script parses the existing config_defaults.py file and extracts:
- All configuration keys and their default values
- Comments above each key as descriptions
- Types inferred from the default values
The output is a comprehensive JSON schema that can be used for:
- Documentation generation
- Configuration validation
- IDE autocomplete
"""
import ast
import json
import sys
from pathlib import Path
from typing import Any, Dict, List
# Import the complex object handlers
sys.path.append(str(Path(__file__).parent.parent))
try:
from superset.config_objects import (
get_default_for_complex_object,
get_fully_qualified_type,
get_object_import_info,
is_complex_object,
)
except ImportError:
# Fallback if import fails
def get_default_for_complex_object(key: str) -> tuple[Any, str]:
return f"<Complex object: {key}>", "unknown"
def is_complex_object(key: str) -> bool:
return False
def get_fully_qualified_type(obj: Any) -> str:
return type(obj).__name__
def get_object_import_info(obj: Any) -> dict[str, Any]:
return {
"module": None,
"name": str(type(obj).__name__),
"import_statement": None,
}
def infer_type(value: Any) -> str:
"""Infer the configuration type from the default value."""
if value is None:
return "null"
elif isinstance(value, bool):
return "boolean"
elif isinstance(value, int):
return "integer"
elif isinstance(value, float):
return "number"
elif isinstance(value, str):
return "string"
elif isinstance(value, (list, tuple)):
return "array"
elif isinstance(value, dict):
return "object"
else:
return "unknown"
def extract_comments_before_line(lines: List[str], line_num: int) -> List[str]:
"""Extract comments immediately before a configuration line."""
comments: List[str] = []
current_line = line_num - 2 # line_num is 1-based, so -2 to get previous line
# Look backwards for comments, but only go back a few lines to avoid
# picking up unrelated comments
max_lookback = min(5, current_line + 1)
for i in range(max_lookback):
if current_line - i < 0:
break
line = lines[current_line - i].strip()
if line.startswith("#"):
# Remove the '#' and clean up the comment
comment = line[1:].strip()
if comment: # Only add non-empty comments
comments.insert(0, comment)
elif line == "":
# Empty line - continue looking
continue
else:
# Non-comment, non-empty line - stop looking
break
return comments
def safe_eval(node: ast.AST) -> Any: # noqa: C901
"""Safely evaluate an AST node to get its value."""
try:
# Handle basic constant values
if isinstance(node, ast.Constant):
return node.value
elif isinstance(node, ast.Num): # Python < 3.8
return node.n
elif isinstance(node, ast.Str): # Python < 3.8
return node.s
elif isinstance(node, ast.List):
return [safe_eval(item) for item in node.elts]
elif isinstance(node, ast.Dict):
return {
safe_eval(k): safe_eval(v)
for k, v in zip(node.keys, node.values, strict=False)
if k is not None
}
elif isinstance(node, ast.Name):
# Handle common constants
if node.id in ("True", "False", "None"):
return {"True": True, "False": False, "None": None}[node.id]
else:
return f"<{node.id}>" # Placeholder for variables
elif isinstance(node, ast.Call):
# Handle function calls - try to identify the function being called
if isinstance(node.func, ast.Name):
func_name = node.func.id
if func_name in ("int", "float", "str", "bool"):
# Handle type constructors
if node.args:
arg_val = safe_eval(node.args[0])
if isinstance(arg_val, (int, float, str, bool)):
try:
return eval(func_name)(arg_val) # noqa: S307
except Exception:
return f"<{func_name}()>"
return f"<{func_name}()>"
elif func_name == "timedelta":
# Handle timedelta calls
return "<timedelta()>"
else:
return f"<{func_name}()>"
elif isinstance(node.func, ast.Attribute):
# Handle method calls like obj.method()
method_name = (
ast.unparse(node.func) if hasattr(ast, "unparse") else "method_call"
)
return f"<{method_name}()>"
else:
return "<function_call>"
elif isinstance(node, ast.Attribute):
# Handle attribute access like obj.attr
try:
attr_str = ast.unparse(node) if hasattr(ast, "unparse") else "attribute"
return f"<{attr_str}>"
except Exception:
return "<attribute>"
else:
# For everything else, just return a descriptive placeholder
return f"<{type(node).__name__}>"
except Exception:
return "<unknown>"
def extract_config_schema(config_file: Path) -> Dict[str, Any]:
"""Extract configuration schema from config_defaults.py."""
with open(config_file, "r") as f:
content = f.read()
lines = content.splitlines()
# Parse the Python file
tree = ast.parse(content)
schema = {}
for node in ast.walk(tree):
if isinstance(node, ast.Assign):
# Check if this is a simple assignment to a variable
if len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
var_name = node.targets[0].id
# Only include uppercase variables (configuration convention)
if var_name.isupper():
# Get the default value
default_value = safe_eval(node.value)
# Check if this is a complex object
if is_complex_object(var_name):
# Get the proper default value and type for complex objects
default_value, type_name = get_default_for_complex_object(
var_name
)
config_type = type_name
else:
# Infer type from default value
config_type = infer_type(default_value)
# Get comments before this line
comments = extract_comments_before_line(lines, node.lineno)
description = " ".join(comments) if comments else ""
# Determine category based on variable name patterns
category = categorize_config(var_name)
schema[var_name] = {
"type": config_type,
"default": default_value,
"description": description,
"category": category,
}
# Add additional metadata for complex objects
if is_complex_object(var_name):
schema[var_name]["is_complex_object"] = True
return schema
def categorize_config(var_name: str) -> str:
"""Categorize configuration variables based on their names."""
name_lower = var_name.lower()
if any(term in name_lower for term in ["limit", "timeout", "cache", "pool"]):
return "performance"
elif any(term in name_lower for term in ["feature", "flag", "enable", "disable"]):
return "features"
elif any(term in name_lower for term in ["theme", "color", "style", "ui"]):
return "ui"
elif any(term in name_lower for term in ["db", "database", "sql", "query"]):
return "database"
elif any(term in name_lower for term in ["auth", "security", "login", "oauth"]):
return "security"
elif any(term in name_lower for term in ["log", "debug", "stats"]):
return "logging"
elif any(term in name_lower for term in ["mail", "smtp", "email"]):
return "email"
elif any(term in name_lower for term in ["celery", "async", "worker"]):
return "async"
else:
return "general"
def main() -> None:
"""Extract configuration schema and save to JSON."""
superset_root = Path(__file__).parent.parent
config_file = superset_root / "superset" / "config_defaults.py"
if not config_file.exists():
print(f"Error: {config_file} not found")
sys.exit(1)
print("Extracting configuration schema...")
schema = extract_config_schema(config_file)
# Create output structure
output = {
"metadata": {
"generated_from": str(config_file),
"total_configs": len(schema),
"description": (
"Superset configuration schema extracted from config_defaults.py"
),
},
"configs": schema,
"by_category": {},
}
# Group by category
for key, config in schema.items():
category = config["category"]
if category not in output["by_category"]:
output["by_category"][category] = {}
output["by_category"][category][key] = config
# Save to JSON
output_file = superset_root / "superset" / "config_schema.json"
with open(output_file, "w") as f:
json.dump(output, f, indent=2, default=str)
print("✅ Schema extracted successfully!")
print(f"📊 Total configurations: {len(schema)}")
print(f"📂 Categories: {list(output['by_category'].keys())}")
print(f"💾 Saved to: {output_file}")
# Show some stats
print("\n📈 Category breakdown:")
for category, configs in output["by_category"].items():
print(f" {category}: {len(configs)} configs")
if __name__ == "__main__":
main()
+324
View File
@@ -0,0 +1,324 @@
#!/usr/bin/env python3
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Extract configuration types from runtime inspection of config.py.
This script imports the actual config module and extracts type information
through runtime introspection, providing more accurate type data than
static analysis.
"""
import ast
import inspect
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional
# Add superset to path
sys.path.insert(0, str(Path(__file__).parent.parent))
def get_source_comment(module_path: str, var_name: str) -> Optional[str]:
"""Extract comment from source code for a variable."""
try:
with open(module_path, "r") as f:
content = f.read()
tree = ast.parse(content)
lines = content.splitlines()
for node in ast.walk(tree):
if isinstance(node, ast.Assign):
if len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
if node.targets[0].id == var_name:
# Look for comments above this line
line_num = node.lineno - 1 # Convert to 0-based
comments = []
# Look backwards for comments
for i in range(min(5, line_num)):
check_line = line_num - i - 1
if check_line < 0:
break
line = lines[check_line].strip()
if line.startswith("#"):
comment = line[1:].strip()
if comment:
comments.insert(0, comment)
elif line and not line.startswith("#"):
break
return " ".join(comments) if comments else None
return None
except Exception:
return None
def analyze_value(value: Any) -> Dict[str, Any]:
"""Analyze a configuration value to extract type information."""
analysis = {
"python_type": type(value),
"type_name": type(value).__name__,
"module": getattr(type(value), "__module__", None),
"is_callable": callable(value),
"is_none": value is None,
}
# Basic type categorization
if value is None:
analysis["category"] = "null"
elif isinstance(value, bool):
analysis["category"] = "boolean"
elif isinstance(value, int):
analysis["category"] = "integer"
elif isinstance(value, float):
analysis["category"] = "number"
elif isinstance(value, str):
analysis["category"] = "string"
elif isinstance(value, (list, tuple)):
analysis["category"] = "array"
# Sample item types
if value:
item_types = list(set(type(item).__name__ for item in value[:5]))
analysis["item_types"] = item_types
elif isinstance(value, dict):
analysis["category"] = "object"
# Sample key/value types
if value:
keys = list(value.keys())[:5]
key_types = list(set(type(k).__name__ for k in keys))
val_types = list(set(type(value[k]).__name__ for k in keys))
analysis["key_types"] = key_types
analysis["value_types"] = val_types
elif callable(value):
analysis["category"] = "function"
try:
analysis["signature"] = str(inspect.signature(value))
except Exception:
pass
else:
analysis["category"] = "object"
analysis["class_name"] = f"{type(value).__module__}.{type(value).__name__}"
# Serialization check
try:
import json
json.dumps(value)
analysis["serializable"] = True
except Exception:
analysis["serializable"] = False
return analysis
def categorize_config_key(key: str) -> str:
"""Categorize a configuration key based on its name."""
key_lower = key.lower()
if any(
term in key_lower
for term in ["secret", "key", "password", "auth", "oauth", "login"]
):
return "security"
elif any(
term in key_lower for term in ["db", "database", "sql", "query", "engine"]
):
return "database"
elif any(
term in key_lower for term in ["limit", "timeout", "cache", "pool", "async"]
):
return "performance"
elif any(term in key_lower for term in ["feature", "flag", "enable", "disable"]):
return "features"
elif any(
term in key_lower for term in ["theme", "color", "style", "ui", "frontend"]
):
return "ui"
elif any(term in key_lower for term in ["log", "debug", "stats", "event"]):
return "logging"
elif any(term in key_lower for term in ["mail", "smtp", "email"]):
return "email"
elif any(term in key_lower for term in ["celery", "worker", "beat", "task"]):
return "async"
else:
return "general"
def extract_config_types() -> Dict[str, Any]:
"""Extract type information from the config module."""
try:
# Import the config module
from superset import config
# Get module path for comment extraction
config_path = inspect.getfile(config)
results = {}
# Get all uppercase attributes (configuration convention)
for name in dir(config):
if name.isupper() and not name.startswith("_"):
value = getattr(config, name)
# Analyze the value
analysis = analyze_value(value)
# Get source comment
comment = get_source_comment(config_path, name)
# Categorize
category = categorize_config_key(name)
results[name] = {
"key": name,
"value_analysis": analysis,
"description": comment,
"category": category,
"current_value": value
if analysis.get("serializable")
else f"<{analysis['type_name']} instance>",
}
return results
except ImportError as e:
print(f"Error importing config: {e}")
return {}
def compare_with_metadata() -> Dict[str, Any]:
"""Compare runtime config with defined metadata."""
from superset.config_metadata import CONFIG_METADATA
runtime_configs = extract_config_types()
comparison = {
"in_metadata_only": [],
"in_runtime_only": [],
"type_mismatches": [],
"matching": [],
}
metadata_keys = set(CONFIG_METADATA.keys())
runtime_keys = set(runtime_configs.keys())
# Keys only in metadata
comparison["in_metadata_only"] = sorted(metadata_keys - runtime_keys)
# Keys only in runtime
comparison["in_runtime_only"] = sorted(runtime_keys - metadata_keys)
# Check for type mismatches
for key in metadata_keys & runtime_keys:
metadata_type = CONFIG_METADATA[key].type
runtime_type = runtime_configs[key]["value_analysis"]["python_type"]
if metadata_type != runtime_type:
comparison["type_mismatches"].append(
{
"key": key,
"metadata_type": str(metadata_type),
"runtime_type": str(runtime_type),
}
)
else:
comparison["matching"].append(key)
return comparison
def suggest_metadata_entries() -> List[str]:
"""Suggest metadata entries for configs not yet documented."""
runtime_configs = extract_config_types()
from superset.config_metadata import CONFIG_METADATA
suggestions = []
for key, info in runtime_configs.items():
if key not in CONFIG_METADATA:
analysis = info["value_analysis"]
# Build suggested metadata entry
suggestion = f""" "{key}": ConfigMetadata(
key="{key}",
type={analysis["type_name"]},
default={repr(info["current_value"]) if analysis["serializable"] else f"{analysis['type_name']}()"},
description="{info.get("description", "TODO: Add description")}",
category="{info["category"]}",
impact="medium",
requires_restart={"True" if info["category"] in ["security", "database"] else "False"},"""
if analysis["category"] == "integer":
suggestion += "\n min_value=1,"
if not analysis["serializable"]:
suggestion += f'\n serializable=False,\n doc_default="<{analysis["type_name"]} instance>",'
suggestion += "\n ),"
suggestions.append(suggestion)
return suggestions
def main():
"""Main function to run type extraction."""
print("Extracting configuration types from runtime...")
# Extract types
runtime_configs = extract_config_types()
print(f"Found {len(runtime_configs)} configuration variables")
# Compare with metadata
print("\nComparing with defined metadata...")
comparison = compare_with_metadata()
print(f" - Matching: {len(comparison['matching'])}")
print(f" - Only in metadata: {len(comparison['in_metadata_only'])}")
print(f" - Only in runtime: {len(comparison['in_runtime_only'])}")
print(f" - Type mismatches: {len(comparison['type_mismatches'])}")
if comparison["in_runtime_only"]:
print(f"\nConfigs missing metadata: {len(comparison['in_runtime_only'])}")
print("Generating suggestions...")
suggestions = suggest_metadata_entries()
# Save suggestions to file
output_file = Path(__file__).parent / "suggested_metadata.py"
with open(output_file, "w") as f:
f.write("# Suggested metadata entries for undocumented configs\n\n")
f.write("\n\n".join(suggestions))
print(f"Suggestions saved to: {output_file}")
# Show type distribution
type_dist = {}
for config in runtime_configs.values():
cat = config["value_analysis"]["category"]
type_dist[cat] = type_dist.get(cat, 0) + 1
print("\nType distribution:")
for cat, count in sorted(type_dist.items()):
print(f" {cat}: {count}")
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
@@ -589,7 +589,7 @@ const config: ControlPanelConfig = {
name: 'show_cell_bars',
config: {
type: 'CheckboxControl',
label: t('Show cell bars'),
label: t('Show Cell bars'),
renderTrigger: true,
default: true,
description: t(
@@ -617,7 +617,7 @@ const config: ControlPanelConfig = {
name: 'color_pn',
config: {
type: 'CheckboxControl',
label: t('Add colors to cell bars for +/-'),
label: t('add colors to cell bars for +/-'),
renderTrigger: true,
default: true,
description: t(
@@ -631,7 +631,7 @@ const config: ControlPanelConfig = {
name: 'comparison_color_enabled',
config: {
type: 'CheckboxControl',
label: t('Basic conditional formatting'),
label: t('basic conditional formatting'),
renderTrigger: true,
visibility: ({ controls }) =>
!isEmpty(controls?.time_compare?.value),
@@ -672,7 +672,7 @@ const config: ControlPanelConfig = {
config: {
type: 'ConditionalFormattingControl',
renderTrigger: true,
label: t('Custom conditional formatting'),
label: t('Custom Conditional Formatting'),
extraColorChoices: [
{
value: ColorSchemeEnum.Green,
@@ -95,27 +95,27 @@ function getTotalValuePadding({
top: donut ? 'middle' : '0',
left: 'center',
};
const LEGEND_HEIGHT = 15;
const LEGEND_WIDTH = 215;
if (chartPadding.top) {
padding.top = donut
? `${50 + (chartPadding.top / height / 2) * 100}%`
: `${(chartPadding.top / height) * 100}%`;
? `${50 + ((chartPadding.top - LEGEND_HEIGHT) / height / 2) * 100}%`
: `${((chartPadding.top + LEGEND_HEIGHT) / height) * 100}%`;
}
if (chartPadding.bottom) {
padding.top = donut
? `${50 - (chartPadding.bottom / height / 2) * 100}%`
? `${50 - ((chartPadding.bottom + LEGEND_HEIGHT) / height / 2) * 100}%`
: '0';
}
if (chartPadding.left) {
// When legend is on the left, shift text right to center it in the available space
const leftPaddingPercent = (chartPadding.left / width) * 100;
const adjustedLeftPercent = 50 + leftPaddingPercent * 0.25;
padding.left = `${adjustedLeftPercent}%`;
padding.left = `${
50 + ((chartPadding.left - LEGEND_WIDTH) / width / 2) * 100
}%`;
}
if (chartPadding.right) {
// When legend is on the right, shift text left to center it in the available space
const rightPaddingPercent = (chartPadding.right / width) * 100;
const adjustedLeftPercent = 50 - rightPaddingPercent * 0.75;
padding.left = `${adjustedLeftPercent}%`;
padding.left = `${
50 - ((chartPadding.right + LEGEND_WIDTH) / width / 2) * 100
}%`;
}
return padding;
}
@@ -220,7 +220,7 @@ export default function transformProps(
name: otherName,
value: otherSum,
itemStyle: {
color: theme.colorText,
color: theme.colors.grayscale.dark1,
opacity:
filterState.selectedValues &&
!filterState.selectedValues.includes(otherName)
@@ -368,7 +368,7 @@ export default function transformProps(
const defaultLabel = {
formatter,
show: showLabels,
color: theme.colorText,
color: theme.colors.grayscale.dark2,
};
const chartPadding = getChartPadding(
@@ -403,7 +403,7 @@ export default function transformProps(
label: {
show: true,
fontWeight: 'bold',
backgroundColor: theme.colorBgContainer,
backgroundColor: theme.colors.grayscale.light5,
},
},
data: transformedData,
@@ -445,7 +445,6 @@ export default function transformProps(
text: t('Total: %s', numberFormatter(totalValue)),
fontSize: 16,
fontWeight: 'bold',
fill: theme.colorText,
},
z: 10,
}
@@ -71,7 +71,6 @@ export const DEFAULT_FORM_DATA: EchartsTimeseriesFormData = {
seriesType: EchartsTimeseriesSeriesType.Line,
stack: false,
tooltipTimeFormat: 'smart_date',
xAxisTimeFormat: 'smart_date',
truncateXAxis: true,
truncateYAxis: false,
yAxisBounds: [null, null],
@@ -174,8 +174,6 @@ function Echart(
if (!chartRef.current) {
chartRef.current = init(divRef.current, null, { locale });
}
// did mount
handleSizeChange({ width, height });
setDidMount(true);
});
}, [locale]);
@@ -237,6 +235,9 @@ function Echart(
echartOptions,
);
chartRef.current?.setOption(themedEchartOptions, true);
// did mount
handleSizeChange({ width, height });
}
}, [didMount, echartOptions, eventHandlers, zrEventHandlers, theme]);
@@ -221,157 +221,6 @@ describe('Pie label string template', () => {
});
});
describe('Total value positioning with legends', () => {
const getChartPropsWithLegend = (
showTotal = true,
showLegend = true,
legendOrientation = 'right',
donut = true,
): EchartsPieChartProps => {
const formData: SqlaFormData = {
colorScheme: 'bnbColors',
datasource: '3__table',
granularity_sqla: 'ds',
metric: 'sum__num',
groupby: ['category'],
viz_type: 'pie',
show_total: showTotal,
show_legend: showLegend,
legend_orientation: legendOrientation,
donut,
};
return new ChartProps({
formData,
width: 800,
height: 600,
queriesData: [
{
data: [
{ category: 'A', sum__num: 10, sum__num__contribution: 0.4 },
{ category: 'B', sum__num: 15, sum__num__contribution: 0.6 },
],
},
],
theme: supersetTheme,
}) as EchartsPieChartProps;
};
it('should center total text when legend is on the right', () => {
const props = getChartPropsWithLegend(true, true, 'right', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
left: expect.stringMatching(/^\d+(\.\d+)?%$/),
top: 'middle',
style: expect.objectContaining({
text: expect.stringContaining('Total:'),
}),
}),
);
// The left position should be less than 50% (shifted left)
const leftValue = parseFloat(
(transformed.echartOptions.graphic as any).left.replace('%', ''),
);
expect(leftValue).toBeLessThan(50);
expect(leftValue).toBeGreaterThan(30); // Should be reasonable positioning
});
it('should center total text when legend is on the left', () => {
const props = getChartPropsWithLegend(true, true, 'left', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
left: expect.stringMatching(/^\d+(\.\d+)?%$/),
top: 'middle',
}),
);
// The left position should be greater than 50% (shifted right)
const leftValue = parseFloat(
(transformed.echartOptions.graphic as any).left.replace('%', ''),
);
expect(leftValue).toBeGreaterThan(50);
expect(leftValue).toBeLessThan(70); // Should be reasonable positioning
});
it('should center total text when legend is on top', () => {
const props = getChartPropsWithLegend(true, true, 'top', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
left: 'center',
top: expect.stringMatching(/^\d+(\.\d+)?%$/),
}),
);
// The top position should be adjusted for top legend
const topValue = parseFloat(
(transformed.echartOptions.graphic as any).top.replace('%', ''),
);
expect(topValue).toBeGreaterThan(50); // Shifted down for top legend
});
it('should center total text when legend is on bottom', () => {
const props = getChartPropsWithLegend(true, true, 'bottom', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
left: 'center',
top: expect.stringMatching(/^\d+(\.\d+)?%$/),
}),
);
// The top position should be adjusted for bottom legend
const topValue = parseFloat(
(transformed.echartOptions.graphic as any).top.replace('%', ''),
);
expect(topValue).toBeLessThan(50); // Shifted up for bottom legend
});
it('should use default positioning when no legend is shown', () => {
const props = getChartPropsWithLegend(true, false, 'right', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
left: 'center',
top: 'middle',
}),
);
});
it('should handle regular pie chart (non-donut) positioning', () => {
const props = getChartPropsWithLegend(true, true, 'right', false);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toEqual(
expect.objectContaining({
type: 'text',
top: '0', // Non-donut charts use '0' as default top position
left: expect.stringMatching(/^\d+(\.\d+)?%$/), // Should still adjust left for right legend
}),
);
});
it('should not show total graphic when showTotal is false', () => {
const props = getChartPropsWithLegend(false, true, 'right', true);
const transformed = transformProps(props);
expect(transformed.echartOptions.graphic).toBeNull();
});
});
describe('Other category', () => {
const defaultFormData: SqlaFormData = {
colorScheme: 'bnbColors',
@@ -1,204 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import controlPanel from '../../../src/Timeseries/Regular/Bar/controlPanel';
describe('Bar Chart Control Panel', () => {
describe('x_axis_time_format control', () => {
it('should include x_axis_time_format control in the panel', () => {
const config = controlPanel;
// Look for x_axis_time_format control in all sections and rows
let foundTimeFormatControl = false;
for (const section of config.controlPanelSections) {
if (section && section.controlSetRows) {
for (const row of section.controlSetRows) {
for (const control of row) {
if (
typeof control === 'object' &&
control !== null &&
'name' in control &&
control.name === 'x_axis_time_format'
) {
foundTimeFormatControl = true;
break;
}
}
if (foundTimeFormatControl) break;
}
if (foundTimeFormatControl) break;
}
}
expect(foundTimeFormatControl).toBe(true);
});
it('should have correct default value for x_axis_time_format', () => {
const config = controlPanel;
// Find the x_axis_time_format control
let timeFormatControl: any = null;
for (const section of config.controlPanelSections) {
if (section && section.controlSetRows) {
for (const row of section.controlSetRows) {
for (const control of row) {
if (
typeof control === 'object' &&
control !== null &&
'name' in control &&
control.name === 'x_axis_time_format'
) {
timeFormatControl = control;
break;
}
}
if (timeFormatControl) break;
}
if (timeFormatControl) break;
}
}
expect(timeFormatControl).toBeDefined();
expect(timeFormatControl.config).toBeDefined();
expect(timeFormatControl.config.default).toBe('smart_date');
});
it('should have visibility function for x_axis_time_format', () => {
const config = controlPanel;
// Find the x_axis_time_format control
let timeFormatControl: any = null;
for (const section of config.controlPanelSections) {
if (section && section.controlSetRows) {
for (const row of section.controlSetRows) {
for (const control of row) {
if (
typeof control === 'object' &&
control !== null &&
'name' in control &&
control.name === 'x_axis_time_format'
) {
timeFormatControl = control;
break;
}
}
if (timeFormatControl) break;
}
if (timeFormatControl) break;
}
}
expect(timeFormatControl).toBeDefined();
expect(timeFormatControl.config.visibility).toBeDefined();
expect(typeof timeFormatControl.config.visibility).toBe('function');
// The visibility function exists - the exact logic is tested implicitly through UI behavior
// The important part is that the control has proper visibility configuration
});
it('should have proper control configuration', () => {
const config = controlPanel;
// Find the x_axis_time_format control
let timeFormatControl: any = null;
for (const section of config.controlPanelSections) {
if (section && section.controlSetRows) {
for (const row of section.controlSetRows) {
for (const control of row) {
if (
typeof control === 'object' &&
control !== null &&
'name' in control &&
control.name === 'x_axis_time_format'
) {
timeFormatControl = control;
break;
}
}
if (timeFormatControl) break;
}
if (timeFormatControl) break;
}
}
expect(timeFormatControl).toBeDefined();
expect(timeFormatControl.config).toMatchObject({
default: 'smart_date',
disableStash: true,
resetOnHide: false,
});
// Should have a description that includes D3 time format docs
expect(timeFormatControl.config.description).toContain('D3');
});
});
describe('Control panel structure for bar charts', () => {
it('should have Chart Orientation section', () => {
const config = controlPanel;
const orientationSection = config.controlPanelSections.find(
section => section && section.label === 'Chart Orientation',
);
expect(orientationSection).toBeDefined();
expect(orientationSection!.expanded).toBe(true);
});
it('should have Chart Options section with X Axis controls', () => {
const config = controlPanel;
const chartOptionsSection = config.controlPanelSections.find(
section => section && section.label === 'Chart Options',
);
expect(chartOptionsSection).toBeDefined();
expect(chartOptionsSection!.expanded).toBe(true);
// Should contain X Axis subsection header - this is sufficient proof
expect(chartOptionsSection!.controlSetRows).toBeDefined();
expect(chartOptionsSection!.controlSetRows!.length).toBeGreaterThan(0);
});
it('should have proper form data overrides', () => {
const config = controlPanel;
expect(config.formDataOverrides).toBeDefined();
expect(typeof config.formDataOverrides).toBe('function');
// Test the form data override function
const mockFormData = {
datasource: '1__table',
viz_type: 'echarts_timeseries_bar',
metrics: ['test_metric'],
groupby: ['test_column'],
other_field: 'test',
};
const result = config.formDataOverrides!(mockFormData);
expect(result).toHaveProperty('metrics');
expect(result).toHaveProperty('groupby');
expect(result).toHaveProperty('other_field', 'test');
});
});
});
@@ -1,353 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { ChartProps, SqlaFormData, supersetTheme } from '@superset-ui/core';
import { EchartsTimeseriesChartProps } from '../../../src/types';
import transformProps from '../../../src/Timeseries/transformProps';
import { DEFAULT_FORM_DATA } from '../../../src/Timeseries/constants';
import { EchartsTimeseriesSeriesType } from '../../../src/Timeseries/types';
describe('Bar Chart X-axis Time Formatting', () => {
const baseFormData: SqlaFormData = {
...DEFAULT_FORM_DATA,
colorScheme: 'bnbColors',
datasource: '3__table',
granularity_sqla: '__timestamp',
metric: ['Sales', 'Marketing', 'Operations'],
groupby: [],
viz_type: 'echarts_timeseries_bar',
seriesType: EchartsTimeseriesSeriesType.Bar,
orientation: 'vertical',
};
const timeseriesData = [
{
data: [
{ Sales: 100, __timestamp: 1609459200000 }, // 2021-01-01
{ Marketing: 150, __timestamp: 1612137600000 }, // 2021-02-01
{ Operations: 200, __timestamp: 1614556800000 }, // 2021-03-01
],
colnames: ['Sales', 'Marketing', 'Operations', '__timestamp'],
coltypes: ['BIGINT', 'BIGINT', 'BIGINT', 'TIMESTAMP'],
},
];
const baseChartPropsConfig = {
width: 800,
height: 600,
queriesData: timeseriesData,
theme: supersetTheme,
};
describe('Default xAxisTimeFormat', () => {
it('should use smart_date as default xAxisTimeFormat', () => {
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: baseFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// Check that the x-axis has a formatter applied
expect(transformedProps.echartOptions.xAxis).toHaveProperty('axisLabel');
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(typeof xAxis.axisLabel.formatter).toBe('function');
});
it('should apply xAxisTimeFormat from DEFAULT_FORM_DATA when not explicitly set', () => {
const formDataWithoutTimeFormat = {
...baseFormData,
};
delete formDataWithoutTimeFormat.xAxisTimeFormat;
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: formDataWithoutTimeFormat,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// Should still have a formatter since DEFAULT_FORM_DATA includes xAxisTimeFormat
expect(transformedProps.echartOptions.xAxis).toHaveProperty('axisLabel');
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
});
});
describe('Custom xAxisTimeFormat', () => {
it('should respect custom xAxisTimeFormat when explicitly set', () => {
const customFormData = {
...baseFormData,
xAxisTimeFormat: '%Y-%m-%d',
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: customFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// Verify the formatter function exists and is applied
expect(transformedProps.echartOptions.xAxis).toHaveProperty('axisLabel');
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(typeof xAxis.axisLabel.formatter).toBe('function');
// The key test is that a formatter exists - the actual formatting is handled by d3-time-format
const { formatter } = xAxis.axisLabel;
expect(formatter).toBeDefined();
expect(typeof formatter).toBe('function');
});
it('should handle different time format options', () => {
const timeFormats = [
'%Y-%m-%d',
'%Y/%m/%d',
'%m/%d/%Y',
'%b %d, %Y',
'smart_date',
];
timeFormats.forEach(timeFormat => {
const customFormData = {
...baseFormData,
xAxisTimeFormat: timeFormat,
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: customFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(typeof xAxis.axisLabel.formatter).toBe('function');
});
});
});
describe('Orientation-specific behavior', () => {
it('should apply time formatting to x-axis in vertical bar charts', () => {
const verticalFormData = {
...baseFormData,
orientation: 'vertical',
xAxisTimeFormat: '%Y-%m',
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: verticalFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// In vertical orientation, time should be on x-axis
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(typeof xAxis.axisLabel.formatter).toBe('function');
});
it('should apply time formatting to y-axis in horizontal bar charts', () => {
const horizontalFormData = {
...baseFormData,
orientation: 'horizontal',
xAxisTimeFormat: '%Y-%m',
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: horizontalFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// In horizontal orientation, axes are swapped, so time should be on y-axis
const yAxis = transformedProps.echartOptions.yAxis as any;
expect(yAxis.axisLabel).toHaveProperty('formatter');
expect(typeof yAxis.axisLabel.formatter).toBe('function');
});
});
describe('Integration with existing features', () => {
it('should work with axis bounds', () => {
const formDataWithBounds = {
...baseFormData,
xAxisTimeFormat: '%Y-%m-%d',
truncateXAxis: true,
xAxisBounds: [null, null] as [number | null, number | null],
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: formDataWithBounds,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
// The xAxis should be configured with the time formatting
expect(transformedProps.echartOptions.xAxis).toBeDefined();
});
it('should work with label rotation', () => {
const formDataWithRotation = {
...baseFormData,
xAxisTimeFormat: '%Y-%m-%d',
xAxisLabelRotation: 45,
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: formDataWithRotation,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(xAxis.axisLabel).toHaveProperty('rotate', 45);
});
it('should maintain time formatting consistency with tooltip', () => {
const formDataWithTooltip = {
...baseFormData,
xAxisTimeFormat: '%Y-%m-%d',
tooltipTimeFormat: '%Y-%m-%d',
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: formDataWithTooltip,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// Both axis and tooltip should have formatters
const xAxis = transformedProps.echartOptions.xAxis as any;
expect(xAxis.axisLabel).toHaveProperty('formatter');
expect(transformedProps.xValueFormatter).toBeDefined();
expect(typeof transformedProps.xValueFormatter).toBe('function');
});
});
describe('Regression test for Issue #30373', () => {
it('should not be stuck on adaptive formatting', () => {
// Test the exact scenario described in the issue
const issueFormData = {
...baseFormData,
xAxisTimeFormat: '%Y-%m-%d %H:%M:%S', // Non-adaptive format
};
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: issueFormData,
});
const transformedProps = transformProps(
chartProps as EchartsTimeseriesChartProps,
);
// Verify formatter exists - this is the key fix, ensuring xAxisTimeFormat is used
const xAxis = transformedProps.echartOptions.xAxis as any;
const { formatter } = xAxis.axisLabel;
expect(formatter).toBeDefined();
expect(typeof formatter).toBe('function');
// The important part is that the xAxisTimeFormat is being used from formData
// The actual formatting is handled by the underlying time formatter
});
it('should allow changing from smart_date to other formats', () => {
// First create with smart_date (default)
const smartDateFormData = {
...baseFormData,
xAxisTimeFormat: 'smart_date',
};
const smartDateChartProps = new ChartProps({
...baseChartPropsConfig,
formData: smartDateFormData,
});
const smartDateProps = transformProps(
smartDateChartProps as EchartsTimeseriesChartProps,
);
// Then change to a different format
const customFormatFormData = {
...baseFormData,
xAxisTimeFormat: '%b %Y',
};
const customFormatChartProps = new ChartProps({
...baseChartPropsConfig,
formData: customFormatFormData,
});
const customFormatProps = transformProps(
customFormatChartProps as EchartsTimeseriesChartProps,
);
// Both should have formatters - the key is that they're not undefined
const smartDateXAxis = smartDateProps.echartOptions.xAxis as any;
const customFormatXAxis = customFormatProps.echartOptions.xAxis as any;
expect(smartDateXAxis.axisLabel.formatter).toBeDefined();
expect(customFormatXAxis.axisLabel.formatter).toBeDefined();
// Both should be functions that can format time
expect(typeof smartDateXAxis.axisLabel.formatter).toBe('function');
expect(typeof customFormatXAxis.axisLabel.formatter).toBe('function');
});
it('should have xAxisTimeFormat in formData by default', () => {
// This test specifically verifies our fix - that DEFAULT_FORM_DATA includes xAxisTimeFormat
const chartProps = new ChartProps({
...baseChartPropsConfig,
formData: baseFormData,
});
expect(chartProps.formData.xAxisTimeFormat).toBeDefined();
expect(chartProps.formData.xAxisTimeFormat).toBe('smart_date');
});
});
});
@@ -1,43 +0,0 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { DEFAULT_FORM_DATA } from '../../src/Timeseries/constants';
describe('Timeseries constants', () => {
describe('DEFAULT_FORM_DATA', () => {
it('should include xAxisTimeFormat in default form data', () => {
expect(DEFAULT_FORM_DATA).toHaveProperty('xAxisTimeFormat');
expect(DEFAULT_FORM_DATA.xAxisTimeFormat).toBe('smart_date');
});
it('should include tooltipTimeFormat in default form data', () => {
expect(DEFAULT_FORM_DATA).toHaveProperty('tooltipTimeFormat');
expect(DEFAULT_FORM_DATA.tooltipTimeFormat).toBe('smart_date');
});
it('should have consistent time format defaults', () => {
expect(DEFAULT_FORM_DATA.xAxisTimeFormat).toBe(
DEFAULT_FORM_DATA.tooltipTimeFormat,
);
});
it('should have vertical orientation as default', () => {
expect(DEFAULT_FORM_DATA.orientation).toBe('vertical');
});
});
});
@@ -646,7 +646,7 @@ const config: ControlPanelConfig = {
name: 'show_cell_bars',
config: {
type: 'CheckboxControl',
label: t('Show cell bars'),
label: t('Show Cell bars'),
renderTrigger: true,
default: true,
description: t(
@@ -674,7 +674,7 @@ const config: ControlPanelConfig = {
name: 'color_pn',
config: {
type: 'CheckboxControl',
label: t('Add colors to cell bars for +/-'),
label: t('add colors to cell bars for +/-'),
renderTrigger: true,
default: true,
description: t(
@@ -688,7 +688,7 @@ const config: ControlPanelConfig = {
name: 'comparison_color_enabled',
config: {
type: 'CheckboxControl',
label: t('Basic conditional formatting'),
label: t('basic conditional formatting'),
renderTrigger: true,
visibility: ({ controls }) =>
!isEmpty(controls?.time_compare?.value),
@@ -729,7 +729,7 @@ const config: ControlPanelConfig = {
config: {
type: 'ConditionalFormattingControl',
renderTrigger: true,
label: t('Custom conditional formatting'),
label: t('Custom Conditional Formatting'),
extraColorChoices: [
{
value: ColorSchemeEnum.Green,
+171
View File
@@ -0,0 +1,171 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Configuration introspection CLI commands."""
import re
from typing import Any
import click
import yaml
from flask.cli import with_appcontext
from superset import app
def serialize_config_value(value: Any) -> Any:
"""Serialize config values for YAML output, handling callables and objects."""
if callable(value):
name = value.__name__ if hasattr(value, "__name__") else repr(value)
return f"<callable: {name}>"
elif hasattr(value, "__module__") and hasattr(value, "__name__"):
return f"<object: {value.__module__}.{value.__name__}>"
elif isinstance(value, type):
return f"<class: {value.__module__}.{value.__name__}>"
else:
try:
# Try to serialize with yaml to check if it's serializable
yaml.safe_dump(value)
return value
except yaml.YAMLError:
return repr(value)
def get_config_source(key: str) -> str:
"""Determine where a config value comes from."""
import os
# Check if it's from environment variables (with double underscore prefix)
env_key = f"SUPERSET__{key}"
if env_key in os.environ:
return f"environment ({env_key})"
# Check if it's from superset_config.py user override
try:
import superset_config
if hasattr(superset_config, key):
return "superset_config.py"
except ImportError:
pass
# Otherwise it's from defaults
return "config_defaults.py"
@click.group()
def config() -> None:
"""Configuration introspection commands."""
pass
@config.command()
@with_appcontext
@click.option("--filter", "-f", help="Filter config keys (regex pattern)")
@click.option(
"--verbose", "-v", is_flag=True, help="Show source information for each key"
)
def show(filter: str, verbose: bool) -> None:
"""Show effective configuration as YAML."""
config_dict = {}
# Get actual config keys (not Flask Config methods)
# Use app.config.keys() to get only the configuration values
for key in app.config.keys():
# Apply filter if provided
if filter and not re.search(filter, key, re.IGNORECASE):
continue
value = app.config[key]
serialized_value = serialize_config_value(value)
if verbose:
source = get_config_source(key)
config_dict[key] = {"value": serialized_value, "source": source}
else:
config_dict[key] = serialized_value
# Output as YAML
print(yaml.dump(config_dict, default_flow_style=False, sort_keys=True))
@config.command()
@with_appcontext
@click.argument("key")
def get(key: str) -> None:
"""Get a specific configuration value."""
if key not in app.config:
click.echo(f"Configuration key '{key}' not found", err=True)
return
value = app.config[key]
serialized_value = serialize_config_value(value)
source = get_config_source(key)
result = {key: {"value": serialized_value, "source": source}}
print(yaml.dump(result, default_flow_style=False))
@config.command()
@with_appcontext
def env_examples() -> None:
"""Show example environment variables for configuration."""
from superset.config_extensions import SupersetConfig
examples = [
"# Superset configuration via environment variables",
"# All environment variables must start with SUPERSET__ prefix "
"(note double underscore)",
"",
"# Basic settings",
"export SUPERSET__ROW_LIMIT=100000",
"export SUPERSET__SAMPLES_ROW_LIMIT=10000",
"export SUPERSET__SQLLAB_TIMEOUT=60",
"",
"# Feature flags (JSON format)",
'export SUPERSET__FEATURE_FLAGS=\'{"ENABLE_TEMPLATE_PROCESSING": true, '
'"ENABLE_EXPLORE_DRAG_AND_DROP": true}\'',
"",
"# Or use triple underscore for nested values",
"export SUPERSET__FEATURE_FLAGS__ENABLE_TEMPLATE_PROCESSING=true",
"export SUPERSET__FEATURE_FLAGS__ENABLE_EXPLORE_DRAG_AND_DROP=true",
"",
"# Theme configuration",
'export SUPERSET__THEME_DEFAULT=\'{"colors": '
'{"primary": {"base": "#1985a1"}}}\'',
"",
"# Lists and complex types",
'export SUPERSET__FAB_ROLES=\'["Admin", "Alpha", "Gamma"]\'',
"",
]
for line in examples:
click.echo(line)
# Show documented settings if using SupersetConfig
if isinstance(app.config, SupersetConfig):
click.echo("\n# Documented settings with metadata:")
for key, schema in app.config.DATABASE_SETTINGS_SCHEMA.items():
click.echo(f"\n# {schema.get('title', key)}")
click.echo(f"# {schema.get('description', '')}")
click.echo(f"# Type: {schema.get('type', 'unknown')}")
if "minimum" in schema or "maximum" in schema:
click.echo(
f"# Range: {schema.get('minimum', 'N/A')} - "
"{schema.get('maximum', 'N/A')}"
)
click.echo(f"export SUPERSET__{key}={schema.get('default', '...')}")
+16
View File
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+47
View File
@@ -0,0 +1,47 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from superset.commands.exceptions import CommandException
class SettingsCommandException(CommandException):
"""Base exception for settings commands."""
pass
class SettingNotFoundError(SettingsCommandException):
"""Exception raised when a setting is not found."""
pass
class SettingNotAllowedInDatabaseError(SettingsCommandException):
"""Exception raised when a setting cannot be stored in the database."""
pass
class SettingValidationError(SettingsCommandException):
"""Exception raised when a setting value is invalid."""
pass
class SettingAlreadyExistsError(SettingsCommandException):
"""Exception raised when trying to create a setting that already exists."""
pass
+2 -2
View File
@@ -145,7 +145,7 @@ def _get_samples(
query_obj = copy.copy(query_obj)
query_obj.is_timeseries = False
query_obj.orderby = []
query_obj.metrics = []
query_obj.metrics = None
query_obj.post_processing = []
qry_obj_cols = []
for o in datasource.columns:
@@ -168,7 +168,7 @@ def _get_drill_detail(
query_obj = copy.copy(query_obj)
query_obj.is_timeseries = False
query_obj.orderby = []
query_obj.metrics = []
query_obj.metrics = None
query_obj.post_processing = []
qry_obj_cols = []
for o in datasource.columns:
+11 -467
View File
@@ -86,8 +86,6 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
apply_fetch_values_predicate: bool
columns: list[Column]
datasource: BaseDatasource | None
columns_by_name: dict[str, Any]
metrics_by_name: dict[str, Any]
extras: dict[str, Any]
filter: list[QueryObjectFilterClause]
from_dttm: datetime | None
@@ -96,7 +94,7 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
inner_to_dttm: datetime | None
is_rowcount: bool
is_timeseries: bool
metrics: list[Metric]
metrics: list[Metric] | None
order_desc: bool
orderby: list[OrderBy]
post_processing: list[dict[str, Any]]
@@ -143,30 +141,6 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
self.apply_fetch_values_predicate = apply_fetch_values_predicate or False
self.columns = columns or []
self.datasource = datasource
# Build datasource mappings for easy lookup
self.columns_by_name: dict[str, Any] = {}
self.metrics_by_name: dict[str, Any] = {}
if datasource:
try:
if hasattr(datasource, "columns") and datasource.columns is not None:
self.columns_by_name = {
col.column_name: col for col in datasource.columns
}
except (TypeError, AttributeError):
# Handle mocked datasources or other non-iterable cases
pass
try:
if hasattr(datasource, "metrics") and datasource.metrics is not None:
self.metrics_by_name = {
metric.metric_name: metric for metric in datasource.metrics
}
except (TypeError, AttributeError):
# Handle mocked datasources or other non-iterable cases
pass
self.extras = extras or {}
self.filter = filters or []
self.granularity = granularity
@@ -218,12 +192,9 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
def is_str_or_adhoc(metric: Metric) -> bool:
return isinstance(metric, str) or is_adhoc_metric(metric)
# Track whether metrics was originally None (for need_groupby logic)
self._metrics_is_not_none = metrics is not None
self.metrics = [
self.metrics = metrics and [
x if is_str_or_adhoc(x) else x["label"] # type: ignore
for x in (metrics or [])
for x in metrics
]
def _set_post_processing(
@@ -255,20 +226,15 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
field.new_name,
)
value = kwargs[field.old_name]
if value is not None:
# Only override if the new field is not already populated with data
current_value = getattr(self, field.new_name, None)
if (
current_value
): # If field already has truthy data, don't override
if value:
if hasattr(self, field.new_name):
logger.warning(
"The field `%s` is already populated, "
"not replacing with contents from deprecated `%s`.",
"replacing value with contents from `%s`.",
field.new_name,
field.old_name,
)
else:
setattr(self, field.new_name, value)
setattr(self, field.new_name, value)
def _move_deprecated_extra_fields(self, kwargs: dict[str, Any]) -> None:
# move deprecated extras fields to extras
@@ -281,8 +247,8 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
field.new_name,
)
value = kwargs[field.old_name]
if value is not None and value != "": # Don't add empty string values
if field.new_name in self.extras:
if value:
if hasattr(self.extras, field.new_name):
logger.warning(
"The field `%s` is already populated in "
"`extras`, replacing value with contents "
@@ -296,7 +262,7 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
def metric_names(self) -> list[str]:
"""Return metrics names (labels), coerce adhoc metrics to strings."""
return get_metric_names(
self.metrics,
self.metrics or [],
(
self.datasource.verbose_map
if self.datasource and hasattr(self.datasource, "verbose_map")
@@ -310,428 +276,6 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
and metrics are non-empty, otherwise returns column labels."""
return get_column_names(self.columns)
@property
def time_grain(self) -> str | None:
"""Get time grain from extras."""
return (self.extras or {}).get("time_grain_sqla")
@property
def need_groupby(self) -> bool:
"""Determine if GROUP BY is needed based on metrics and columns."""
# GROUP BY is needed when there are metrics or when metrics is explicitly
# provided (even as empty list). When metrics=None, columns are just for
# selection without aggregation, so no GROUP BY needed.
return self._metrics_is_not_none
@property
def groupby(self) -> list[Column]:
"""Alias for columns (for backward compatibility/clarity)."""
return self.columns or []
def get_series_limit_prequery_obj(
self,
granularity: str | None,
inner_from_dttm: datetime | None,
inner_to_dttm: datetime | None,
orderby: list[OrderBy] | None = None,
) -> dict[str, Any]:
"""Build prequery object for series limit queries.
This is used to determine top groups when series_limit is set.
Args:
granularity: The time column name
inner_from_dttm: Inner from datetime (if different from main query)
inner_to_dttm: Inner to datetime (if different from main query)
orderby: Optional orderby to override (for series_limit_metric)
Returns:
Dictionary suitable for passing to query()
"""
from superset.utils.core import get_non_base_axis_columns
return {
"is_timeseries": False,
"row_limit": self.series_limit,
"metrics": self.metrics,
"granularity": granularity,
"groupby": self.groupby,
"from_dttm": inner_from_dttm or self.from_dttm,
"to_dttm": inner_to_dttm or self.to_dttm,
"filter": self.filter,
"orderby": orderby or [],
"extras": self.extras or {},
"columns": get_non_base_axis_columns(self.columns),
"order_desc": True,
}
def build_select_expressions( # noqa: C901
self,
granularity: str | None,
series_column_labels: set[str],
datasource: Any, # BaseDatasource
template_processor: Any,
) -> tuple[list[Any], dict[str, Any], dict[str, Any]]:
"""Build SELECT expressions for the query.
Args:
granularity: The time column name
series_column_labels: Labels of series columns
datasource: The datasource being queried
template_processor: Template processor for SQL templating
Returns:
Tuple of (select_exprs, groupby_all_columns, groupby_series_columns)
"""
from sqlalchemy import literal_column
from superset.utils.core import (
DTTM_ALIAS,
is_adhoc_column,
)
select_exprs = []
groupby_all_columns = {}
groupby_series_columns = {}
# Filter out the pseudo column __timestamp from columns
columns = [col for col in self.columns if col != DTTM_ALIAS]
if self.need_groupby:
# dedup columns while preserving order
columns = self.groupby or self.columns
for selected in columns:
if isinstance(selected, str):
# if groupby field/expr equals granularity field/expr
if selected == granularity:
table_col = self.columns_by_name[selected]
outer = table_col.get_timestamp_expression(
time_grain=self.time_grain,
label=selected,
template_processor=template_processor,
)
# if groupby field equals a selected column
elif selected in self.columns_by_name:
outer = datasource.convert_tbl_column_to_sqla_col(
self.columns_by_name[selected],
template_processor=template_processor,
)
else:
# Import here to avoid circular imports
from superset.models.helpers import validate_adhoc_subquery
selected = validate_adhoc_subquery(
selected,
datasource.database,
datasource.catalog,
datasource.schema,
datasource.database.db_engine_spec.engine,
)
outer = literal_column(f"({selected})")
outer = datasource.make_sqla_column_compatible(outer, selected)
else:
outer = datasource.adhoc_column_to_sqla(
col=selected,
template_processor=template_processor,
)
groupby_all_columns[outer.name] = outer
if (
self.is_timeseries and not series_column_labels
) or outer.name in series_column_labels:
groupby_series_columns[outer.name] = outer
select_exprs.append(outer)
elif self.columns:
with datasource.database.get_sqla_engine() as engine:
quote = engine.dialect.identifier_preparer.quote
for selected in self.columns:
if is_adhoc_column(selected):
_sql = selected["sqlExpression"]
_column_label = selected["label"]
elif isinstance(selected, str):
_sql = quote(selected)
_column_label = selected
# Import here to avoid circular imports
from superset.models.helpers import validate_adhoc_subquery
selected = validate_adhoc_subquery(
_sql,
datasource.database,
datasource.catalog,
datasource.schema,
datasource.database.db_engine_spec.engine,
)
select_exprs.append(
datasource.convert_tbl_column_to_sqla_col(
self.columns_by_name[selected],
template_processor=template_processor,
label=_column_label,
)
if selected in self.columns_by_name
else datasource.make_sqla_column_compatible(
literal_column(selected), _column_label
)
)
return select_exprs, groupby_all_columns, groupby_series_columns
def build_filter_clauses( # noqa: C901
self,
datasource: Any, # BaseDatasource
template_processor: Any,
time_filters: list[Any],
removed_filters: set[str],
applied_adhoc_filters_columns: list[Any],
rejected_adhoc_filters_columns: list[Any],
is_timeseries: bool,
dttm_col: Any,
) -> tuple[list[Any], list[Any]]:
"""Build WHERE and HAVING filter clauses for the query.
Args:
datasource: The datasource being queried
template_processor: Template processor for SQL templating
time_filters: Time-based filters to apply
removed_filters: Set of filter column names handled by Jinja templates
applied_adhoc_filters_columns: List to track applied adhoc filters
rejected_adhoc_filters_columns: List to track rejected adhoc filters
is_timeseries: Whether this is a timeseries query
dttm_col: The datetime column object
Returns:
Tuple of (where_clause_and, having_clause_and)
"""
from flask import current_app
from sqlalchemy import or_
from superset import feature_flag_manager
from superset.common.utils.time_range_utils import (
get_since_until_from_time_range,
)
from superset.exceptions import QueryObjectValidationError
from superset.utils.core import (
DTTM_ALIAS,
FilterOperator,
GenericDataType,
get_column_name,
is_adhoc_column,
)
where_clause_and = []
having_clause_and = []
# Process regular filters
for flt in self.filter:
if not all(flt.get(s) for s in ["col", "op"]):
continue
flt_col = flt["col"]
val = flt.get("val")
flt_grain = flt.get("grain")
op = FilterOperator(flt["op"].upper())
col_obj = None
sqla_col = None
if flt_col == DTTM_ALIAS and is_timeseries and dttm_col:
col_obj = dttm_col
elif is_adhoc_column(flt_col):
try:
sqla_col = datasource.adhoc_column_to_sqla(
flt_col, force_type_check=True
)
applied_adhoc_filters_columns.append(flt_col)
except Exception: # ColumnNotFoundException
rejected_adhoc_filters_columns.append(flt_col)
continue
else:
col_obj = self.columns_by_name.get(str(flt_col))
filter_grain = flt.get("grain")
if get_column_name(flt_col) in removed_filters:
# Skip generating SQLA filter when the jinja template handles it.
continue
if col_obj or sqla_col is not None:
db_engine_spec = datasource.database.db_engine_spec
if sqla_col is not None:
pass
elif col_obj and filter_grain:
sqla_col = col_obj.get_timestamp_expression(
time_grain=filter_grain, template_processor=template_processor
)
elif col_obj:
sqla_col = datasource.convert_tbl_column_to_sqla_col(
tbl_column=col_obj, template_processor=template_processor
)
col_type = col_obj.type if col_obj else None
col_spec = db_engine_spec.get_column_spec(native_type=col_type)
is_list_target = op in (
FilterOperator.IN,
FilterOperator.NOT_IN,
)
col_advanced_data_type = col_obj.advanced_data_type if col_obj else ""
if col_spec and not col_advanced_data_type:
target_generic_type = col_spec.generic_type
else:
target_generic_type = GenericDataType.STRING
eq = datasource.filter_values_handler(
values=val,
operator=op,
target_generic_type=target_generic_type,
target_native_type=col_type,
is_list_target=is_list_target,
db_engine_spec=db_engine_spec,
)
# Get ADVANCED_DATA_TYPES from config when needed
ADVANCED_DATA_TYPES = current_app.config.get("ADVANCED_DATA_TYPES", {}) # noqa: N806
if (
col_advanced_data_type != ""
and feature_flag_manager.is_feature_enabled(
"ENABLE_ADVANCED_DATA_TYPES"
)
and col_advanced_data_type in ADVANCED_DATA_TYPES
and eq is not None
):
where_clause_and.append(
datasource._apply_advanced_data_type_filter(
sqla_col, col_advanced_data_type, op, eq
)
)
elif is_list_target:
assert isinstance(eq, (tuple, list))
if len(eq) == 0:
raise QueryObjectValidationError(
"Filter value list cannot be empty"
)
if len(eq) > len(
eq_without_none := [x for x in eq if x is not None]
):
is_null_cond = sqla_col.is_(None)
if eq:
cond = or_(is_null_cond, sqla_col.in_(eq_without_none))
else:
cond = is_null_cond
else:
cond = sqla_col.in_(eq)
if op == FilterOperator.NOT_IN:
cond = ~cond
where_clause_and.append(cond)
elif op in {
FilterOperator.IS_NULL,
FilterOperator.IS_NOT_NULL,
}:
where_clause_and.append(
db_engine_spec.handle_null_filter(sqla_col, op)
)
elif op == FilterOperator.IS_TRUE:
where_clause_and.append(
db_engine_spec.handle_boolean_filter(sqla_col, op, True)
)
elif op == FilterOperator.IS_FALSE:
where_clause_and.append(
db_engine_spec.handle_boolean_filter(sqla_col, op, False)
)
else:
if (
op
not in {
FilterOperator.EQUALS,
FilterOperator.NOT_EQUALS,
}
and eq is None
):
raise QueryObjectValidationError(
"Must specify a value for filters with comparison operators"
)
if op in {
FilterOperator.EQUALS,
FilterOperator.NOT_EQUALS,
FilterOperator.GREATER_THAN,
FilterOperator.LESS_THAN,
FilterOperator.GREATER_THAN_OR_EQUALS,
FilterOperator.LESS_THAN_OR_EQUALS,
}:
where_clause_and.append(
db_engine_spec.handle_comparison_filter(sqla_col, op, eq)
)
elif op in {
FilterOperator.ILIKE,
FilterOperator.LIKE,
}:
if target_generic_type != GenericDataType.STRING:
import sqlalchemy as sa
sqla_col = sa.cast(sqla_col, sa.String)
if op == FilterOperator.LIKE:
where_clause_and.append(sqla_col.like(eq))
else:
where_clause_and.append(sqla_col.ilike(eq))
elif op in {FilterOperator.NOT_LIKE}:
if target_generic_type != GenericDataType.STRING:
import sqlalchemy as sa
sqla_col = sa.cast(sqla_col, sa.String)
where_clause_and.append(sqla_col.not_like(eq))
elif (
op == FilterOperator.TEMPORAL_RANGE
and isinstance(eq, str)
and col_obj is not None
):
_since, _until = get_since_until_from_time_range(
time_range=eq,
time_shift=self.time_shift,
extras=self.extras or {},
)
where_clause_and.append(
datasource.get_time_filter(
time_col=col_obj,
start_dttm=_since,
end_dttm=_until,
time_grain=flt_grain,
label=sqla_col.key,
template_processor=template_processor,
)
)
else:
raise QueryObjectValidationError(
f"Invalid filter operation type: {op}"
)
# Process WHERE and HAVING extras
if self.extras:
where = self.extras.get("where")
if where:
where = datasource._process_sql_expression(
expression=where,
database_id=datasource.database_id,
engine=datasource.database.backend,
schema=datasource.schema,
template_processor=template_processor,
)
where_clause_and += [datasource.text(where)]
having = self.extras.get("having")
if having:
having = datasource._process_sql_expression(
expression=having,
database_id=datasource.database_id,
engine=datasource.database.backend,
schema=datasource.schema,
template_processor=template_processor,
)
having_clause_and += [datasource.text(having)]
return where_clause_and, having_clause_and
def validate(
self, raise_exceptions: bool | None = True
) -> QueryObjectValidationError | None:
@@ -806,7 +350,7 @@ class QueryObject: # pylint: disable=too-many-instance-attributes
"inner_to_dttm": self.inner_to_dttm,
"is_rowcount": self.is_rowcount,
"is_timeseries": self.is_timeseries,
"metrics": self.metrics if self.metrics else None,
"metrics": self.metrics,
"order_desc": self.order_desc,
"orderby": self.orderby,
"row_limit": self.row_limit,
File diff suppressed because it is too large Load Diff
+184
View File
@@ -0,0 +1,184 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Enhanced configuration system for Superset
This module provides the SupersetConfig class and supporting infrastructure
for the new configuration system.
"""
import logging
from typing import Any, Dict, Optional
from flask import Config
logger = logging.getLogger(__name__)
class SupersetConfig(Config):
"""Enhanced configuration class for Superset.
This class extends Flask's Config class to provide additional features:
- Rich metadata system for configuration values
- Database-backed settings support
- Environment variable integration
- JSON schema generation for UI forms
"""
# Metadata is now stored in config_metadata.py
# This provides a reference to the metadata module
def __init__(
self, root_path: Optional[str] = None, defaults: Optional[Dict[str, Any]] = None
):
"""Initialize SupersetConfig with enhanced features."""
super().__init__(root_path, defaults)
def get_setting_metadata(self, key: str) -> Optional[Dict[str, Any]]:
"""Get metadata for a configuration setting."""
try:
from superset.config_metadata import get_metadata
metadata = get_metadata(key)
if metadata:
return metadata.to_doc_dict()
return None
except ImportError:
return None
def get_settings_by_category(self, category: str) -> Dict[str, Any]:
"""Get all settings for a specific category."""
try:
from superset.config_metadata import CONFIG_METADATA
return {
key: metadata.to_doc_dict()
for key, metadata in CONFIG_METADATA.items()
if metadata.category == category
}
except ImportError:
return {}
def validate_setting(self, key: str, value: Any) -> bool:
"""Validate a setting value against its schema."""
try:
from superset.config_metadata import validate_config_value
return validate_config_value(key, value)
except ImportError:
return True # No validation if metadata not available
def to_json_schema(self) -> Dict[str, Any]:
"""Generate JSON schema for all database settings.
This can be used to generate forms in the frontend.
"""
try:
from superset.config_metadata import CONFIG_METADATA
properties = {}
required = []
for key, metadata in CONFIG_METADATA.items():
if metadata.deprecated:
continue
property_schema = {
"type": metadata._type_to_string().lower(),
"title": key.replace("_", " ").title(),
"description": metadata.description,
"default": metadata.doc_default
if metadata.doc_default is not None
else metadata._serialize_default(),
}
if isinstance(metadata.type, type) and issubclass(
metadata.type, (int, float)
):
if metadata.min_value is not None:
property_schema["minimum"] = metadata.min_value
if metadata.max_value is not None:
property_schema["maximum"] = metadata.max_value
if metadata.choices is not None:
property_schema["enum"] = metadata.choices
properties[key] = property_schema
return {
"type": "object",
"properties": properties,
"required": required,
}
except ImportError:
return {"type": "object", "properties": {}, "required": []}
def get_database_setting(self, key: str, default: Any = None) -> Any:
"""Get a setting value from the database (future implementation)."""
# This would integrate with the SettingsDAO
# For now, return the regular config value
return self.get(key, default)
def set_database_setting(self, key: str, value: Any) -> bool:
"""Set a setting value in the database (future implementation)."""
# This would integrate with the SettingsDAO
# For now, just validate the value
if not self.validate_setting(key, value):
return False
# Would save to database here
# settings_dao.set_value(key, value)
return True
def from_database(self) -> None:
"""Load settings from database (future implementation)."""
# This would load database-backed settings
# For now, this is a placeholder
pass
def load_from_environment(self, prefix: str = "SUPERSET_") -> bool:
"""Load configuration from environment variables.
Uses Flask's built-in from_prefixed_env method to load environment
variables with the SUPERSET__ prefix (note the double underscore).
This provides automatic JSON parsing and nested dictionary support.
The double underscore clearly separates the system prefix from the
configuration key name.
Examples:
SUPERSET__ROW_LIMIT=100000
SUPERSET__SQLLAB_TIMEOUT=60
SUPERSET__FEATURE_FLAGS='{"ENABLE_TEMPLATE_PROCESSING": true}'
SUPERSET__FEATURE_FLAGS__ENABLE_TEMPLATE_PROCESSING=true
Args:
prefix: The environment variable prefix (default: "SUPERSET_")
Returns:
bool: True if any values were loaded
"""
# Use Flask's built-in method which handles JSON parsing automatically
# Note: Flask will add one more underscore, so SUPERSET_ becomes SUPERSET__
return self.from_prefixed_env(prefix)
def export_settings(self) -> Dict[str, Any]:
"""Export current settings with metadata."""
result = {}
for key, schema in self.DATABASE_SETTINGS_SCHEMA.items():
if key in self:
result[key] = {"value": self[key], "metadata": schema}
return result
+371
View File
@@ -0,0 +1,371 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Configuration metadata for Apache Superset.
This module contains the authoritative metadata for all configuration
settings in Superset. It serves as the single source of truth for:
1. Configuration documentation
2. Type information
3. Validation rules
4. Default values
5. Environment variable mappings
The metadata can reference actual Python objects (non-serializable)
and provides an export mechanism for documentation generation.
"""
from datetime import timedelta
from typing import Any, Callable, Dict, List, Optional, Type, Union
from superset.stats_logger import DummyStatsLogger
from superset.utils.log import DBEventLogger
from superset.utils.logging_configurator import DefaultLoggingConfigurator
class ConfigMetadata:
"""Metadata for a configuration setting."""
def __init__(
self,
key: str,
type: Type,
default: Any,
description: str,
category: str = "general",
impact: str = "medium",
requires_restart: bool = False,
min_value: Optional[Union[int, float]] = None,
max_value: Optional[Union[int, float]] = None,
choices: Optional[List[Any]] = None,
example: Optional[str] = None,
documentation_url: Optional[str] = None,
deprecated: bool = False,
deprecated_message: Optional[str] = None,
validator: Optional[Callable[[Any], bool]] = None,
converter: Optional[Callable[[Any], Any]] = None,
serializable: bool = True,
doc_default: Optional[Any] = None,
):
"""Initialize configuration metadata.
Args:
key: Configuration key name
type: Python type of the configuration value
default: Default value
description: Human-readable description
category: Category for grouping (performance, security, etc.)
impact: Impact level (low, medium, high)
requires_restart: Whether changing requires restart
min_value: Minimum value for numeric types
max_value: Maximum value for numeric types
choices: List of valid choices
example: Example usage
documentation_url: Link to detailed docs
deprecated: Whether this setting is deprecated
deprecated_message: Message for deprecated settings
validator: Custom validation function
converter: Function to convert from env var or other format
serializable: Whether the value can be JSON serialized
doc_default: Alternative default for documentation (if not serializable)
"""
self.key = key
self.type = type
self.default = default
self.description = description
self.category = category
self.impact = impact
self.requires_restart = requires_restart
self.min_value = min_value
self.max_value = max_value
self.choices = choices
self.example = example
self.documentation_url = documentation_url
self.deprecated = deprecated
self.deprecated_message = deprecated_message
self.validator = validator
self.converter = converter
self.serializable = serializable
self.doc_default = doc_default
def to_doc_dict(self) -> Dict[str, Any]:
"""Convert to dictionary for documentation export."""
return {
"key": self.key,
"type": self._type_to_string(),
"default": self.doc_default
if self.doc_default is not None
else self._serialize_default(),
"description": self.description,
"category": self.category,
"impact": self.impact,
"requires_restart": self.requires_restart,
"min_value": self.min_value,
"max_value": self.max_value,
"choices": self.choices,
"example": self.example,
"documentation_url": self.documentation_url,
"deprecated": self.deprecated,
"deprecated_message": self.deprecated_message,
"env_var": f"SUPERSET__{self.key}",
}
def _type_to_string(self) -> str:
"""Convert Python type to string representation."""
if hasattr(self.type, "__name__"):
return self.type.__name__
return str(self.type)
def _serialize_default(self) -> Any:
"""Serialize default value for documentation."""
if not self.serializable:
return f"<{self._type_to_string()} instance>"
if callable(self.default):
return "<function>"
if self.default is None:
return None
if isinstance(self.default, (str, int, float, bool, list, dict)):
return self.default
return str(self.default)
# Configuration metadata registry
CONFIG_METADATA: Dict[str, ConfigMetadata] = {
# Performance settings
"ROW_LIMIT": ConfigMetadata(
key="ROW_LIMIT",
type=int,
default=50000,
description="Maximum number of rows returned for any query or data request. "
"This is a hard limit to prevent memory issues and ensure reasonable response times.",
category="performance",
impact="medium",
requires_restart=False,
min_value=1,
max_value=1000000,
example="export SUPERSET__ROW_LIMIT=100000",
),
"SAMPLES_ROW_LIMIT": ConfigMetadata(
key="SAMPLES_ROW_LIMIT",
type=int,
default=1000,
description="Default row limit when requesting samples from datasource. "
"Used in dataset exploration and SQL Lab table preview.",
category="performance",
impact="low",
requires_restart=False,
min_value=1,
max_value=10000,
),
"SQLLAB_TIMEOUT": ConfigMetadata(
key="SQLLAB_TIMEOUT",
type=int,
default=30,
description="Timeout duration for SQL Lab synchronous queries in seconds. "
"Queries taking longer will be killed. For async queries, see SQLLAB_ASYNC_TIME_LIMIT_SEC.",
category="performance",
impact="high",
requires_restart=False,
min_value=1,
max_value=3600,
converter=lambda x: int(x) if isinstance(x, str) else x,
),
"SQLLAB_ASYNC_TIME_LIMIT_SEC": ConfigMetadata(
key="SQLLAB_ASYNC_TIME_LIMIT_SEC",
type=int,
default=int(timedelta(hours=6).total_seconds()),
description="Maximum duration for SQL Lab async queries in seconds. "
"This is enforced by Celery workers.",
category="performance",
impact="high",
requires_restart=True,
min_value=60,
max_value=86400, # 24 hours
doc_default=21600, # 6 hours
),
# Security settings
"SECRET_KEY": ConfigMetadata(
key="SECRET_KEY",
type=str,
default="CHANGE_ME_SECRET_KEY",
description="**CRITICAL**: Secret key for signing cookies and CSRF tokens. "
"**Must be changed** from default in production. Generate with: "
"`openssl rand -base64 42`",
category="security",
impact="high",
requires_restart=True,
example='export SUPERSET__SECRET_KEY="$(openssl rand -base64 42)"',
documentation_url="https://superset.apache.org/docs/configuration/security",
),
"WTF_CSRF_ENABLED": ConfigMetadata(
key="WTF_CSRF_ENABLED",
type=bool,
default=True,
description="Enable CSRF protection. Should always be True in production. "
"Only disable for testing or if you have your own CSRF protection.",
category="security",
impact="high",
requires_restart=True,
),
# Feature flags
"FEATURE_FLAGS": ConfigMetadata(
key="FEATURE_FLAGS",
type=dict,
default={},
description="Feature flags to enable/disable functionality. "
"Can be set as JSON in environment or as nested values.",
category="features",
impact="high",
requires_restart=True,
example="export SUPERSET__FEATURE_FLAGS='{\"ENABLE_TEMPLATE_PROCESSING\": true}'",
),
# UI/Theme settings
"THEME_DEFAULT": ConfigMetadata(
key="THEME_DEFAULT",
type=dict,
default={},
description="Default theme configuration in Ant Design format. "
"Customize colors, fonts, and other design tokens.",
category="ui",
impact="medium",
requires_restart=False,
example='export SUPERSET__THEME_DEFAULT__token__colorPrimary="#1890ff"',
documentation_url="https://ant.design/docs/react/customize-theme",
),
# Logging
"STATS_LOGGER": ConfigMetadata(
key="STATS_LOGGER",
type=DummyStatsLogger,
default=DummyStatsLogger(),
description="Statistics logger instance for metrics collection. "
"Use StatsdStatsLogger for production metrics.",
category="logging",
impact="low",
requires_restart=True,
serializable=False,
doc_default="<DummyStatsLogger instance>",
),
"EVENT_LOGGER": ConfigMetadata(
key="EVENT_LOGGER",
type=DBEventLogger,
default=DBEventLogger(),
description="Event logger for audit trails and user activity tracking. "
"DBEventLogger stores in database, StdOutEventLogger for debugging.",
category="logging",
impact="medium",
requires_restart=True,
serializable=False,
doc_default="<DBEventLogger instance>",
),
"LOGGING_CONFIGURATOR": ConfigMetadata(
key="LOGGING_CONFIGURATOR",
type=DefaultLoggingConfigurator,
default=DefaultLoggingConfigurator(),
description="Logging configuration handler. Customize to integrate with "
"your logging infrastructure.",
category="logging",
impact="medium",
requires_restart=True,
serializable=False,
doc_default="<DefaultLoggingConfigurator instance>",
),
# Add more configuration metadata as needed...
}
def export_for_documentation() -> Dict[str, Any]:
"""Export metadata in JSON-serializable format for documentation.
Returns:
Dictionary with all settings metadata suitable for JSON export
"""
all_settings = []
by_category = {}
for key, metadata in CONFIG_METADATA.items():
doc_dict = metadata.to_doc_dict()
all_settings.append(doc_dict)
category = metadata.category
if category not in by_category:
by_category[category] = []
by_category[category].append(doc_dict)
# Sort settings within each category
for category in by_category:
by_category[category].sort(key=lambda x: x["key"])
all_settings.sort(key=lambda x: x["key"])
return {
"all_settings": all_settings,
"by_category": by_category,
"categories": sorted(by_category.keys()),
"metadata": {
"total_configs": len(all_settings),
"description": "Superset configuration metadata",
"generated_from": "config_metadata.py",
},
}
def get_metadata(key: str) -> Optional[ConfigMetadata]:
"""Get metadata for a specific configuration key.
Args:
key: Configuration key name
Returns:
ConfigMetadata instance or None if not found
"""
return CONFIG_METADATA.get(key)
def validate_config_value(key: str, value: Any) -> bool:
"""Validate a configuration value against its metadata.
Args:
key: Configuration key name
value: Value to validate
Returns:
True if valid, False otherwise
"""
metadata = get_metadata(key)
if not metadata:
return True # No metadata means any value is valid
# Type check
if not isinstance(value, metadata.type):
return False
# Range check for numeric types
if isinstance(value, (int, float)):
if metadata.min_value is not None and value < metadata.min_value:
return False
if metadata.max_value is not None and value > metadata.max_value:
return False
# Choice validation
if metadata.choices is not None and value not in metadata.choices:
return False
# Custom validator
if metadata.validator is not None:
return metadata.validator(value)
return True
+336
View File
@@ -0,0 +1,336 @@
"""
Configuration objects and complex types that can't be JSON serialized.
This module contains all the complex Python objects used in configuration
that need to be imported and used as actual Python objects rather than
simple JSON-serializable values.
"""
from collections import OrderedDict
from datetime import timedelta
from typing import Any
from superset.advanced_data_type.plugins.internet_address import internet_address
from superset.advanced_data_type.plugins.internet_port import internet_port
from superset.constants import CHANGE_ME_SECRET_KEY, NO_TIME_RANGE
from superset.db_engine_specs.utils import SQLAlchemyUtilsAdapter
from superset.extensions import feature_flag_manager
from superset.key_value.types import JsonKeyValueCodec
from superset.stats_logger import DummyStatsLogger
from superset.utils.core import parse_boolean_string
from superset.utils.log import DBEventLogger
from superset.utils.logging_configurator import DefaultLoggingConfigurator
# Define complex objects that can't be JSON serialized
COMPLEX_CONFIG_OBJECTS = {
"STATS_LOGGER": DummyStatsLogger(),
"EVENT_LOGGER": DBEventLogger(),
"NO_TIME_RANGE": NO_TIME_RANGE,
"CHANGE_ME_SECRET_KEY": CHANGE_ME_SECRET_KEY,
"LOGGING_CONFIGURATOR": DefaultLoggingConfigurator(),
"JSON_KEY_VALUE_CODEC": JsonKeyValueCodec(),
"SQLALCHEMY_UTILS_ADAPTER": SQLAlchemyUtilsAdapter,
"ADVANCED_DATA_TYPES": [internet_address, internet_port],
"PARSE_BOOLEAN_STRING": parse_boolean_string,
"FEATURE_FLAG_MANAGER": feature_flag_manager,
}
# Default collections and dictionaries
DEFAULT_COLLECTIONS = {
"EXCEL_EXTENSIONS": {"xlsx", "xls"},
"CSV_EXTENSIONS": {"csv", "tsv", "txt"},
"COLUMNAR_EXTENSIONS": {"parquet", "zip"},
"CURRENCIES": ["USD", "EUR", "GBP", "INR", "MXN", "JPY", "CNY"],
"BLUEPRINTS": [],
"ADDITIONAL_MIDDLEWARE": [],
"CUSTOM_TEMPLATE_PROCESSORS": {},
"ALLOWED_EXTRA_AUTHENTICATIONS": {},
"DBS_AVAILABLE_DENYLIST": {},
"QUERY_COST_FORMATTERS_BY_ENGINE": {},
"SQLGLOT_DIALECTS_EXTENSIONS": {},
"DB_POLL_INTERVAL_SECONDS": {},
"DEFAULT_HTTP_HEADERS": {},
"OVERRIDE_HTTP_HEADERS": {},
"HTTP_HEADERS": {},
"JINJA_CONTEXT_ADDONS": {},
"WTF_CSRF_EXEMPT_LIST": [
"superset.views.core.log",
"superset.views.core.explore_json",
"superset.charts.data.api.data",
"superset.dashboards.api.cache_dashboard_screenshot",
],
"ROBOT_PERMISSION_ROLES": ["Public", "Gamma", "Alpha", "Admin", "sql_lab"],
"VIZ_TYPE_DENYLIST": [],
"PREFERRED_DATABASES": [
"PostgreSQL",
"Presto",
"MySQL",
"SQLite",
],
"DASHBOARD_AUTO_REFRESH_INTERVALS": [
[0, "Don't refresh"],
[10, "10 seconds"],
[30, "30 seconds"],
[60, "1 minute"],
[300, "5 minutes"],
[1800, "30 minutes"],
[3600, "1 hour"],
[21600, "6 hours"],
[43200, "12 hours"],
[86400, "24 hours"],
],
"LANGUAGES": {
"en": {"flag": "us", "name": "English"},
"es": {"flag": "es", "name": "Spanish"},
"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
"zh": {"flag": "cn", "name": "Chinese"},
"zh_TW": {"flag": "tw", "name": "Traditional Chinese"},
"ja": {"flag": "jp", "name": "Japanese"},
"pt": {"flag": "pt", "name": "Portuguese"},
"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
"ru": {"flag": "ru", "name": "Russian"},
"ko": {"flag": "kr", "name": "Korean"},
"sk": {"flag": "sk", "name": "Slovak"},
"sl": {"flag": "si", "name": "Slovenian"},
"nl": {"flag": "nl", "name": "Dutch"},
"uk": {"flag": "uk", "name": "Ukrainian"},
},
"FAVICONS": [{"href": "/static/assets/images/favicon.png"}],
"PROXY_FIX_CONFIG": {
"x_for": 1,
"x_proto": 1,
"x_host": 1,
"x_port": 1,
"x_prefix": 1,
},
"WEBDRIVER_WINDOW": {
"dashboard": (1600, 2000),
"slice": (3000, 1200),
"pixel_density": 1,
},
"WEBDRIVER_CONFIGURATION": {
"options": {"capabilities": {}, "preferences": {}, "binary_location": ""},
"service": {
"log_output": "/dev/null",
"service_args": [],
"port": 0,
"env": {},
},
},
"WEBDRIVER_OPTION_ARGS": ["--headless"],
"CSV_EXPORT": {"encoding": "utf-8"},
"EXCEL_EXPORT": {},
"GLOBAL_ASYNC_QUERIES_CACHE_BACKEND": {
"CACHE_TYPE": "RedisCache",
"CACHE_REDIS_HOST": "localhost",
"CACHE_REDIS_PORT": 6379,
"CACHE_REDIS_USER": "",
"CACHE_REDIS_PASSWORD": "",
"CACHE_REDIS_DB": 0,
"CACHE_DEFAULT_TIMEOUT": 300,
},
"TALISMAN_CONFIG": {
"content_security_policy": {
"base-uri": ["'self'"],
"default-src": ["'self'"],
"img-src": ["'self'", "blob:", "data:"],
"worker-src": ["'self'", "blob:"],
"connect-src": ["'self'"],
"object-src": "'none'",
"style-src": ["'self'", "'unsafe-inline'"],
"script-src": ["'self'", "'strict-dynamic'"],
},
"content_security_policy_nonce_in": ["script-src"],
"force_https": False,
"session_cookie_secure": False,
},
"TALISMAN_DEV_CONFIG": {
"content_security_policy": {
"base-uri": ["'self'"],
"default-src": ["'self'"],
"img-src": ["'self'", "blob:", "data:"],
"worker-src": ["'self'", "blob:"],
"connect-src": ["'self'"],
"object-src": "'none'",
"style-src": ["'self'", "'unsafe-inline'"],
"script-src": ["'self'", "'unsafe-inline'", "'unsafe-eval'"],
},
"content_security_policy_nonce_in": ["script-src"],
"force_https": False,
"session_cookie_secure": False,
},
}
# Cache configurations
CACHE_CONFIGS = {
"CACHE_CONFIG": {"CACHE_TYPE": "NullCache"},
"DATA_CACHE_CONFIG": {"CACHE_TYPE": "NullCache"},
"THUMBNAIL_CACHE_CONFIG": {
"CACHE_TYPE": "NullCache",
"CACHE_DEFAULT_TIMEOUT": int(timedelta(days=7).total_seconds()),
"CACHE_NO_NULL_WARNING": True,
},
"EXPLORE_FORM_DATA_CACHE_CONFIG": {
"CACHE_TYPE": "NullCache",
"CACHE_DEFAULT_TIMEOUT": int(timedelta(days=7).total_seconds()),
"REFRESH_TIMEOUT_ON_RETRIEVAL": True,
"CODEC": JsonKeyValueCodec(),
},
}
# Computed values that depend on other variables
COMPUTED_VALUES = {
"ALLOWED_EXTENSIONS": lambda: {
*DEFAULT_COLLECTIONS["EXCEL_EXTENSIONS"],
*DEFAULT_COLLECTIONS["CSV_EXTENSIONS"],
*DEFAULT_COLLECTIONS["COLUMNAR_EXTENSIONS"],
},
"DEFAULT_MODULE_DS_MAP": lambda: OrderedDict(
[
("superset.connectors.sqla.models", ["SqlaTable"]),
]
),
}
# Lambda functions and callables
LAMBDA_FUNCTIONS = {
"TRACKING_URL_TRANSFORMER": lambda url: url,
"SQLA_TABLE_MUTATOR": lambda table: table,
"SQL_QUERY_MUTATOR": lambda sql, **kwargs: sql,
"EMAIL_HEADER_MUTATOR": lambda msg, **kwargs: msg,
}
# Timeout values using timedelta
TIMEOUT_VALUES = {
"SUPERSET_WEBSERVER_TIMEOUT": int(timedelta(minutes=1).total_seconds()),
"CELERY_BEAT_SCHEDULER_EXPIRES": timedelta(weeks=1),
"SQLLAB_TIMEOUT": int(timedelta(seconds=30).total_seconds()),
"SQLLAB_VALIDATION_TIMEOUT": int(timedelta(seconds=10).total_seconds()),
"SQLLAB_ASYNC_TIME_LIMIT_SEC": int(timedelta(hours=6).total_seconds()),
"SQLLAB_QUERY_COST_ESTIMATE_TIMEOUT": int(timedelta(seconds=10).total_seconds()),
"CACHE_DEFAULT_TIMEOUT": int(timedelta(days=1).total_seconds()),
"THUMBNAIL_ERROR_CACHE_TTL": int(timedelta(days=1).total_seconds()),
"SCREENSHOT_LOCATE_WAIT": int(timedelta(seconds=10).total_seconds()),
"SCREENSHOT_LOAD_WAIT": int(timedelta(minutes=1).total_seconds()),
"SCREENSHOT_PLAYWRIGHT_DEFAULT_TIMEOUT": int(
timedelta(seconds=60).total_seconds() * 1000
),
"WTF_CSRF_TIME_LIMIT": int(timedelta(weeks=1).total_seconds()),
"EMAIL_PAGE_RENDER_WAIT": int(timedelta(seconds=30).total_seconds()),
"TEST_DATABASE_CONNECTION_TIMEOUT": timedelta(seconds=30),
"DATABASE_OAUTH2_TIMEOUT": timedelta(seconds=30),
"SEND_FILE_MAX_AGE_DEFAULT": int(timedelta(days=365).total_seconds()),
"GLOBAL_ASYNC_QUERIES_POLLING_DELAY": int(
timedelta(milliseconds=500).total_seconds() * 1000
),
"ALERT_REPORTS_WORKING_TIME_OUT_LAG": int(timedelta(seconds=10).total_seconds()),
"ALERT_REPORTS_WORKING_SOFT_TIME_OUT_LAG": int(
timedelta(seconds=1).total_seconds()
),
"ALERT_MINIMUM_INTERVAL": int(timedelta(minutes=0).total_seconds()),
"REPORT_MINIMUM_INTERVAL": int(timedelta(minutes=0).total_seconds()),
"SLACK_CACHE_TIMEOUT": int(timedelta(days=1).total_seconds()),
}
def get_object_type_description(obj: Any) -> str:
"""Get a human-readable description of a Python object type."""
if callable(obj):
if hasattr(obj, "__name__"):
# Try to get the full module path
module = getattr(obj, "__module__", None)
if module:
return f"Function: {module}.{obj.__name__}"
else:
return f"Function: {obj.__name__}"
else:
return "Callable function"
elif hasattr(obj, "__class__"):
# Get fully qualified class name
cls = obj.__class__
module = getattr(cls, "__module__", None)
if module and module != "builtins":
return f"Instance of {module}.{cls.__name__}"
else:
return f"Instance of {cls.__name__}"
else:
# Get type information
obj_type = type(obj)
module = getattr(obj_type, "__module__", None)
if module and module != "builtins":
return f"Object of type {module}.{obj_type.__name__}"
else:
return f"Object of type {obj_type.__name__}"
def get_fully_qualified_type(obj: Any) -> str:
"""Get the fully qualified type name of an object."""
if obj is None:
return "NoneType"
obj_type = type(obj)
module = getattr(obj_type, "__module__", None)
if module and module != "builtins":
return f"{module}.{obj_type.__name__}"
else:
return obj_type.__name__
def get_default_for_complex_object(key: str) -> tuple[Any, str]:
"""Get a string representation and type of complex objects for documentation."""
if key in COMPLEX_CONFIG_OBJECTS:
obj = COMPLEX_CONFIG_OBJECTS[key]
return f"<{get_object_type_description(obj)}>", get_fully_qualified_type(obj)
elif key in DEFAULT_COLLECTIONS:
obj = DEFAULT_COLLECTIONS[key]
return obj, get_fully_qualified_type(obj)
elif key in CACHE_CONFIGS:
obj = CACHE_CONFIGS[key]
return obj, get_fully_qualified_type(obj)
elif key in COMPUTED_VALUES:
# For computed values, try to call them to get the actual value
try:
func = COMPUTED_VALUES[key]
obj = func() # type: ignore
return obj, get_fully_qualified_type(obj)
except Exception:
return f"<Computed: {key}>", "callable"
elif key in LAMBDA_FUNCTIONS:
obj = LAMBDA_FUNCTIONS[key]
return "<Lambda function>", get_fully_qualified_type(obj)
elif key in TIMEOUT_VALUES:
obj = TIMEOUT_VALUES[key]
return obj, get_fully_qualified_type(obj)
else:
return f"<Complex object: {key}>", "unknown"
def is_complex_object(key: str) -> bool:
"""Check if a configuration key represents a complex object."""
return (
key in COMPLEX_CONFIG_OBJECTS
or key in COMPUTED_VALUES
or key in LAMBDA_FUNCTIONS
)
def get_object_import_info(obj: Any) -> dict[str, Any]:
"""Get import information for an object."""
if hasattr(obj, "__module__") and hasattr(obj, "__name__"):
return {
"module": obj.__module__,
"name": obj.__name__,
"import_statement": f"from {obj.__module__} import {obj.__name__}",
}
elif hasattr(obj, "__class__"):
cls = obj.__class__
if hasattr(cls, "__module__"):
return {
"module": cls.__module__,
"name": cls.__name__,
"import_statement": f"from {cls.__module__} import {cls.__name__}",
}
return {"module": None, "name": str(type(obj).__name__), "import_statement": None}
File diff suppressed because it is too large Load Diff
+7 -12
View File
@@ -71,7 +71,6 @@ from sqlalchemy.types import JSON
from superset import db, is_feature_enabled, security_manager
from superset.commands.dataset.exceptions import DatasetNotFoundError
from superset.common.db_query_status import QueryStatus
from superset.common.query_object import QueryObject
from superset.connectors.sqla.utils import (
get_columns_description,
get_physical_table_metadata,
@@ -721,7 +720,7 @@ class AnnotationDatasource(BaseDatasource):
def query(self, query_obj: QueryObjectDict) -> QueryResult:
error_message = None
qry = db.session.query(Annotation)
qry = qry.filter(Annotation.layer_id == query_obj["filters"][0]["val"])
qry = qry.filter(Annotation.layer_id == query_obj["filter"][0]["val"])
if query_obj["from_dttm"]:
qry = qry.filter(Annotation.start_dttm >= query_obj["from_dttm"])
if query_obj["to_dttm"]:
@@ -1515,19 +1514,18 @@ class SqlaTable(
def _get_series_orderby(
self,
series_limit_metric: Metric,
query_obj: QueryObject,
metrics_by_name: dict[str, SqlMetric],
columns_by_name: dict[str, TableColumn],
template_processor: BaseTemplateProcessor | None = None,
) -> Column:
if utils.is_adhoc_metric(series_limit_metric):
assert isinstance(series_limit_metric, dict)
ob = self.adhoc_metric_to_sqla(
series_limit_metric, query_obj.columns_by_name
)
ob = self.adhoc_metric_to_sqla(series_limit_metric, columns_by_name)
elif (
isinstance(series_limit_metric, str)
and series_limit_metric in query_obj.metrics_by_name
and series_limit_metric in metrics_by_name
):
ob = query_obj.metrics_by_name[series_limit_metric].get_sqla_col(
ob = metrics_by_name[series_limit_metric].get_sqla_col(
template_processor=template_processor
)
else:
@@ -1859,10 +1857,7 @@ class SqlaTable(
"""
extra_cache_keys = super().get_extra_cache_keys(query_obj)
if self.has_extra_cache_key_calls(query_obj):
from superset.common.query_object import QueryObject
query_object = QueryObject(datasource=self, **query_obj)
sqla_query = self.get_sqla_query(query_object)
sqla_query = self.get_sqla_query(**query_obj)
extra_cache_keys += sqla_query.extra_cache_keys
return list(set(extra_cache_keys))
+159
View File
@@ -0,0 +1,159 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from __future__ import annotations
from typing import Any, Dict, Optional
from superset.daos.base import BaseDAO
from superset.models.core import Settings
from superset.utils import json
class SettingsDAO(BaseDAO[Settings]):
"""
Data Access Object for Settings model.
Provides methods to manage configuration settings stored in the database.
"""
id_column_name = "key" # Settings uses 'key' as primary key, not 'id'
@classmethod
def find_by_key(cls, key: str) -> Optional[Settings]:
"""Find a setting by its key."""
return cls.find_by_id(key)
@classmethod
def get_value(cls, key: str, default: Any = None) -> Any:
"""
Get the parsed value for a setting key.
Args:
key: The setting key
default: Default value if setting not found
Returns:
The parsed JSON value or default
"""
if setting := cls.find_by_key(key):
try:
return json.loads(setting.json_data)
except json.JSONDecodeError:
return setting.json_data
return default
@classmethod
def set_value(
cls,
key: str,
value: Any,
namespace: Optional[str] = None,
is_sensitive: bool = False,
) -> Settings:
"""
Set a setting value.
Args:
key: The setting key
value: The value to set (will be JSON serialized)
namespace: Optional namespace for organizing settings
is_sensitive: Whether this setting contains sensitive data
Returns:
The Settings object
"""
json_value = json.dumps(value)
if setting := cls.find_by_key(key):
# Update existing setting
setting.json_data = json_value
if namespace is not None:
setting.namespace = namespace
setting.is_sensitive = is_sensitive
return cls.update(setting)
else:
# Create new setting
return cls.create(
attributes={
"key": key,
"json_data": json_value,
"namespace": namespace,
"is_sensitive": is_sensitive,
}
)
@classmethod
def get_all_as_dict(cls) -> Dict[str, Any]:
"""
Get all settings as a dictionary with parsed values.
Returns:
Dictionary mapping setting keys to their parsed values
"""
settings = cls.find_all()
result = {}
for setting in settings:
try:
result[setting.key] = json.loads(setting.json_data)
except json.JSONDecodeError:
result[setting.key] = setting.json_data
return result
@classmethod
def get_by_namespace(cls, namespace: str) -> Dict[str, Any]:
"""
Get all settings in a specific namespace.
Args:
namespace: The namespace to filter by
Returns:
Dictionary mapping setting keys to their parsed values
"""
from superset.extensions import db
settings = (
db.session.query(Settings).filter(Settings.namespace == namespace).all()
)
result = {}
for setting in settings:
try:
result[setting.key] = json.loads(setting.json_data)
except json.JSONDecodeError:
result[setting.key] = setting.json_data
return result
@classmethod
def delete_by_key(cls, key: str) -> bool:
"""
Delete a setting by key.
Args:
key: The setting key to delete
Returns:
True if setting was deleted, False if not found
"""
if setting := cls.find_by_key(key):
cls.delete([setting])
return True
return False
+3 -3
View File
@@ -21,7 +21,7 @@ from datetime import datetime
from typing import Any, Optional
from urllib import parse
from flask import current_app as app
from flask import current_app
from sqlalchemy import types
from sqlalchemy.engine import URL
@@ -498,8 +498,8 @@ class SingleStoreSpec(BasicParametersMixin, BaseEngineSpec):
"conn_attrs",
{
"_connector_name": "SingleStore Superset Database Engine",
"_connector_version": app.config.get("VERSION_STRING", "dev"),
"_product_version": app.config.get("VERSION_STRING", "dev"),
"_connector_version": current_app.config.get("VERSION_STRING", "dev"),
"_product_version": current_app.config.get("VERSION_STRING", "dev"),
},
)
return uri, connect_args
+2
View File
@@ -160,6 +160,7 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
SecurityRestApi,
UserRegistrationsRestAPI,
)
from superset.settings.api import SettingsRestApi
from superset.sqllab.api import SqlLabRestApi
from superset.sqllab.permalink.api import SqlLabPermalinkRestApi
from superset.tags.api import TagRestApi
@@ -242,6 +243,7 @@ class SupersetAppInitializer: # pylint: disable=too-many-public-methods
appbuilder.add_api(ReportExecutionLogRestApi)
appbuilder.add_api(RLSRestApi)
appbuilder.add_api(SavedQueryRestApi)
appbuilder.add_api(SettingsRestApi)
appbuilder.add_api(TagRestApi)
appbuilder.add_api(SqlLabRestApi)
appbuilder.add_api(SqlLabPermalinkRestApi)
+1 -1
View File
@@ -940,7 +940,7 @@ def dataset_macro(
metrics = [metric.metric_name for metric in dataset.metrics]
query_obj = {
"is_timeseries": False,
"filters": [],
"filter": [],
"metrics": metrics if include_metrics else None,
"columns": columns,
"from_dttm": from_dttm,
+1 -1
View File
@@ -37,7 +37,7 @@ from superset.migrations.shared.security_converge import (
)
from superset.models.core import Database
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
Base: Type[Any] = declarative_base()
@@ -28,7 +28,7 @@ from superset.migrations.shared.utils import paginated_update, try_load_json
from superset.utils import json
from superset.utils.date_parser import get_since_until
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
Base = declarative_base()
@@ -29,7 +29,7 @@ from sqlalchemy import (
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import Load, relationship, Session
logger = logging.getLogger("alembic.env")
logger = logging.getLogger(__name__)
Base = declarative_base()
+1 -1
View File
@@ -49,7 +49,7 @@ YELLOW = "\033[33m"
RED = "\033[31m"
LRED = "\033[91m"
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
DEFAULT_BATCH_SIZE = int(os.environ.get("BATCH_SIZE", 1000))
@@ -33,8 +33,6 @@ from superset.utils.core import generic_find_constraint_name
revision = "1226819ee0e3"
down_revision = "956a063c52b3"
logger = logging.getLogger("alembic.env")
naming_convention = {
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s"
@@ -63,7 +61,7 @@ def upgrade():
["datasource_name"],
)
except: # noqa: E722
logger.warning("Could not find or drop constraint on `columns`")
logging.warning("Could not find or drop constraint on `columns`")
def downgrade():
@@ -38,8 +38,6 @@ from superset.utils.core import generic_find_constraint_name
revision = "3b626e2a6783"
down_revision = "eca4694defa7"
logger = logging.getLogger("alembic.env")
def upgrade():
# cleanup after: https://github.com/airbnb/superset/pull/1078
@@ -62,7 +60,7 @@ def upgrade():
batch_op.drop_column("druid_datasource_id")
batch_op.drop_column("table_id")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
# fixed issue: https://github.com/airbnb/superset/issues/466
try:
@@ -71,18 +69,18 @@ def upgrade():
None, "datasources", ["datasource_name"], ["datasource_name"]
)
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
try:
with op.batch_alter_table("query") as batch_op:
batch_op.create_unique_constraint("client_id", ["client_id"])
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
try:
with op.batch_alter_table("query") as batch_op:
batch_op.drop_column("name")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
def downgrade():
@@ -90,7 +88,7 @@ def downgrade():
with op.batch_alter_table("tables") as batch_op:
batch_op.create_index("table_name", ["table_name"], unique=True)
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
try:
with op.batch_alter_table("slices") as batch_op:
@@ -115,7 +113,7 @@ def downgrade():
)
batch_op.create_foreign_key("slices_ibfk_2", "tables", ["table_id"], ["id"])
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
try:
fk_columns = generic_find_constraint_name(
@@ -127,11 +125,11 @@ def downgrade():
with op.batch_alter_table("columns") as batch_op:
batch_op.drop_constraint(fk_columns, type_="foreignkey")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
op.add_column("query", sa.Column("name", sa.String(length=256), nullable=True))
try:
with op.batch_alter_table("query") as batch_op:
batch_op.drop_constraint("client_id", type_="unique")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
@@ -31,8 +31,6 @@ import logging # noqa: E402
import sqlalchemy as sa # noqa: E402
from alembic import op # noqa: E402
logger = logging.getLogger("alembic.env")
def upgrade():
op.add_column("tables", sa.Column("params", sa.Text(), nullable=True))
@@ -42,4 +40,4 @@ def downgrade():
try:
op.drop_column("tables", "params")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex))
@@ -31,8 +31,6 @@ import logging # noqa: E402
import sqlalchemy as sa # noqa: E402
from alembic import op # noqa: E402
logger = logging.getLogger("alembic.env")
def upgrade():
op.add_column(
@@ -46,7 +44,7 @@ def upgrade():
op.create_unique_constraint(None, "dbs", ["verbose_name"])
op.create_unique_constraint(None, "clusters", ["verbose_name"])
except Exception:
logger.info("Constraint not created, expected when using sqlite")
logging.info("Constraint not created, expected when using sqlite")
def downgrade():
@@ -54,4 +52,4 @@ def downgrade():
op.drop_column("dbs", "verbose_name")
op.drop_column("clusters", "verbose_name")
except Exception as ex:
logger.exception(ex)
logging.exception(ex)
@@ -37,8 +37,6 @@ from superset.utils.core import (
revision = "4736ec66ce19"
down_revision = "f959a6652acd"
logger = logging.getLogger("alembic.env")
conv = {
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"uq": "uq_%(table_name)s_%(column_0_name)s",
@@ -121,13 +119,13 @@ def upgrade():
type_="unique",
)
except Exception as ex:
logger.warning(
logging.warning(
"Constraint drop failed, you may want to do this "
"manually on your database. For context, this is a known "
"issue around nondeterministic constraint names on Postgres "
"and perhaps more databases through SQLAlchemy."
)
logger.exception(ex)
logging.exception(ex)
def downgrade():
@@ -22,16 +22,12 @@ Create Date: 2018-05-09 23:45:14.296283
"""
import logging
import sqlalchemy as sa # noqa: E402
from alembic import op # noqa: E402
# revision identifiers, used by Alembic.
revision = "e502db2af7be"
down_revision = "5ccf602336a0"
logger = logging.getLogger("alembic.env")
import sqlalchemy as sa # noqa: E402
from alembic import op # noqa: E402
def upgrade():
@@ -42,4 +38,4 @@ def downgrade():
try:
op.drop_column("tables", "template_params")
except Exception as ex:
logger.warning(str(ex))
logging.warning(str(ex)) # noqa: F821
@@ -35,8 +35,6 @@ from superset.utils import json
revision = "fb13d49b72f9"
down_revision = "de021a1ca60d"
logger = logging.getLogger("alembic.env")
Base = declarative_base()
@@ -51,7 +49,7 @@ class Slice(Base):
def upgrade_slice(slc):
params = json.loads(slc.params)
logger.info(f"Upgrading {slc.slice_name}")
logging.info(f"Upgrading {slc.slice_name}")
cols = params.get("groupby")
metric = params.get("metric")
if cols:
@@ -81,8 +79,8 @@ def upgrade():
for slc in filter_box_slices.all():
try:
upgrade_slice(slc)
except Exception as e:
logger.exception(e)
except Exception:
logging.exception(e) # noqa: F821
session.commit()
session.close()
@@ -96,7 +94,7 @@ def downgrade():
for slc in filter_box_slices.all():
try:
params = json.loads(slc.params)
logger.info(f"Downgrading {slc.slice_name}")
logging.info(f"Downgrading {slc.slice_name}")
flts = params.get("filter_configs")
if not flts:
continue
@@ -104,7 +102,7 @@ def downgrade():
params["groupby"] = [o.get("column") for o in flts]
slc.params = json.dumps(params, sort_keys=True)
except Exception as ex:
logger.exception(ex)
logging.exception(ex)
session.commit()
session.close()
@@ -41,8 +41,6 @@ from superset.utils.core import ( # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
class Slice(Base):
__tablename__ = "slices"
@@ -65,7 +63,7 @@ def upgrade():
if source != target:
slc.params = json.dumps(target, sort_keys=True)
except Exception as ex:
logger.warning(ex)
logging.warn(ex)
session.commit()
session.close()
@@ -37,8 +37,6 @@ from superset.utils.dashboard_filter_scopes_converter import convert_filter_scop
revision = "3325d4caccc8"
down_revision = "e96dbf2cfef0"
logger = logging.getLogger("alembic.env")
Base = declarative_base()
@@ -88,7 +86,7 @@ def upgrade():
if filters:
filter_scopes = convert_filter_scopes(json_metadata, filters)
json_metadata["filter_scopes"] = filter_scopes
logger.info(
logging.info(
f"Adding filter_scopes for dashboard {dashboard.id}: {json.dumps(filter_scopes)}" # noqa: E501
)
@@ -102,7 +100,7 @@ def upgrade():
else:
dashboard.json_metadata = None
except Exception as ex:
logger.exception(f"dashboard {dashboard.id} has error: {ex}")
logging.exception(f"dashboard {dashboard.id} has error: {ex}")
session.commit()
session.close()
@@ -38,8 +38,6 @@ from sqlalchemy_utils import UUIDType # noqa: E402
from superset import db # noqa: E402
logger = logging.getLogger("alembic.env")
add_uuid_column_to_import_mixin = import_module(
"superset.migrations.versions."
"2020-09-28_17-57_b56500de1855_add_uuid_column_to_import_mixin",
@@ -54,9 +52,9 @@ def has_uuid_column(table_name, bind):
columns = {column["name"] for column in inspector.get_columns(table_name)}
has_uuid_column = "uuid" in columns
if has_uuid_column:
logger.info("Table %s already has uuid column, skipping...", table_name)
logging.info("Table %s already has uuid column, skipping...", table_name)
else:
logger.info("Table %s doesn't have uuid column, adding...", table_name)
logging.info("Table %s doesn't have uuid column, adding...", table_name)
return has_uuid_column
@@ -38,8 +38,6 @@ down_revision = "31b2a1039d4a"
Base = declarative_base()
logger = logging.getLogger("alembic.env")
class Database(Base):
__tablename__ = "dbs"
@@ -58,7 +56,7 @@ def upgrade():
try:
extra = json.loads(database.extra)
except json.JSONDecodeError as ex:
logger.warning(str(ex))
logging.warning(str(ex))
continue
schemas_allowed_for_csv_upload = extra.get("schemas_allowed_for_csv_upload")
@@ -35,8 +35,6 @@ from sqlalchemy import func # noqa: E402
from superset import db # noqa: E402
from superset.connectors.sqla.models import SqlaTable # noqa: E402
logger = logging.getLogger("alembic.env")
def upgrade():
with op.batch_alter_table("tables") as batch_op:
@@ -64,12 +62,12 @@ def remove_value_if_too_long():
for row in rows:
row.fetch_values_predicate = None
logger.info("%d values deleted", len(rows))
logging.info("%d values deleted", len(rows))
session.commit()
session.close()
except Exception as ex:
logger.warning(ex)
logging.warning(ex)
def downgrade():
@@ -37,7 +37,7 @@ from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
class Dashboard(Base):
@@ -37,8 +37,6 @@ from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
class Database(Base):
__tablename__ = "dbs"
@@ -54,7 +52,7 @@ def upgrade():
try:
extra = json.loads(database.extra)
except json.JSONDecodeError as ex:
logger.warning(str(ex))
logging.warning(str(ex))
continue
if "schemas_allowed_for_csv_upload" in extra:
@@ -76,7 +74,7 @@ def downgrade():
try:
extra = json.loads(database.extra)
except json.JSONDecodeError as ex:
logger.warning(str(ex))
logging.warning(str(ex))
continue
if "schemas_allowed_for_file_upload" in extra:
@@ -37,7 +37,7 @@ from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
class Slice(Base):
@@ -38,7 +38,7 @@ from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
class Slice(Base):
@@ -37,7 +37,7 @@ from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
logger = logging.getLogger(__name__)
class Slice(Base): # type: ignore
@@ -38,7 +38,7 @@ from superset.migrations.shared.utils import paginated_update # noqa: E402
from superset.utils import json # noqa: E402
Base = declarative_base()
logger = logging.getLogger("alembic.env")
logger = logging.getLogger(__name__)
class Dashboard(Base):
@@ -41,7 +41,7 @@ from superset.utils.date_parser import get_since_until
revision = "f84fde59123a"
down_revision = "9621c6d56ffb"
logger = logging.getLogger("alembic.env")
logger = logging.getLogger(__name__)
Base = declarative_base()
@@ -34,7 +34,7 @@ from superset.utils.core import generic_find_uq_constraint_name
revision = "df3d7e2eb9a4"
down_revision = "48cbb571fa3a"
logger = logging.getLogger("alembic.env")
logger = logging.getLogger(__name__)
def upgrade():
@@ -32,7 +32,8 @@ from sqlalchemy.ext.declarative import declarative_base
from superset import db
from superset.migrations.shared.utils import paginated_update
logger = logging.getLogger("alembic.env")
logger = logging.getLogger("alembic")
logger.setLevel(logging.INFO)
# revision identifiers, used by Alembic.
revision = "363a9b1e8992"
@@ -0,0 +1,66 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""settings table
Revision ID: 02b237514781
Revises: 363a9b1e8992
Create Date: 2025-07-17 00:30:13.162356
"""
import sqlalchemy as sa
import sqlalchemy_utils
from alembic import op
from sqlalchemy.dialects import mysql
# revision identifiers, used by Alembic.
revision = "02b237514781"
down_revision = "363a9b1e8992"
def upgrade():
op.create_table(
"settings",
sa.Column("uuid", sqlalchemy_utils.types.uuid.UUIDType(), nullable=True),
sa.Column("created_on", sa.DateTime(), nullable=True),
sa.Column("changed_on", sa.DateTime(), nullable=True),
sa.Column("key", sa.String(length=255), nullable=False),
sa.Column(
"json_data",
sa.Text().with_variant(mysql.MEDIUMTEXT(), "mysql"),
nullable=False,
),
sa.Column("namespace", sa.String(length=100), nullable=True),
sa.Column("schema_version", sa.Integer(), nullable=False),
sa.Column("is_sensitive", sa.Boolean(), nullable=False),
sa.Column("created_by_fk", sa.Integer(), nullable=True),
sa.Column("changed_by_fk", sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(
["changed_by_fk"],
["ab_user.id"],
),
sa.ForeignKeyConstraint(
["created_by_fk"],
["ab_user.id"],
),
sa.PrimaryKeyConstraint("key"),
sa.UniqueConstraint("uuid"),
)
def downgrade():
op.drop_table("settings")
+29
View File
@@ -1249,3 +1249,32 @@ class FavStar(UUIDMixin, Model):
class_name = Column(String(50))
obj_id = Column(Integer)
dttm = Column(DateTime, default=datetime.utcnow)
class Settings(Model, AuditMixinNullable, UUIDMixin):
"""
Database-backed configuration settings for Superset.
This model extends the Flask configuration system by storing runtime-configurable
settings in the database. These settings can be modified without restarting the
application and take precedence over static configuration files.
The settings system supports:
- JSON-serializable values stored in json_data field
- Import/export across Superset instances via UUID
- Audit trail via AuditMixinNullable
- Namespace organization for logical grouping
"""
__tablename__ = "settings"
key = Column(String(255), primary_key=True)
json_data = Column(utils.MediumText(), nullable=False) # JSON-serialized value
namespace = Column(
String(100), nullable=True
) # e.g., 'features', 'ui', 'performance'
schema_version = Column(Integer, nullable=False, default=1) # For migrations
is_sensitive = Column(Boolean, nullable=False, default=False) # Flag for encryption
def __repr__(self) -> str:
return f"<Settings(key='{self.key}', namespace='{self.namespace}')>"
+541 -558
View File
File diff suppressed because it is too large Load Diff
+16
View File
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+379
View File
@@ -0,0 +1,379 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import logging
from typing import Any, Dict, Optional
from flask import current_app, request
from flask_appbuilder.api import expose, protect, safe
from flask_appbuilder.models.sqla.interface import SQLAInterface
from marshmallow import ValidationError
from superset.config_extensions import SupersetConfig
from superset.constants import MODEL_API_RW_METHOD_PERMISSION_MAP
from superset.daos.settings import SettingsDAO
from superset.models.core import Settings
from superset.settings.schemas import (
SettingCreateSchema,
SettingListResponseSchema,
SettingResponseSchema,
SettingUpdateSchema,
)
from superset.views.base_api import BaseSupersetModelRestApi
logger = logging.getLogger(__name__)
class SettingsRestApi(BaseSupersetModelRestApi):
"""REST API for configuration settings management."""
datamodel = SQLAInterface(Settings)
resource_name = "settings"
allow_browser_login = True
# Only allow specific methods
include_route_methods = {
"get_list",
"get",
"post",
"put",
"delete",
"info",
}
# Schemas for serialization/deserialization
list_schema = SettingListResponseSchema()
show_schema = SettingResponseSchema()
add_schema = SettingCreateSchema()
edit_schema = SettingUpdateSchema()
# Permissions - only admins can read/write settings
method_permission_name = MODEL_API_RW_METHOD_PERMISSION_MAP.copy()
method_permission_name.update(
{
"get_list": "can_read",
"get": "can_read",
"post": "can_write",
"put": "can_write",
"delete": "can_write",
"validate": "can_read",
"metadata": "can_read",
"effective_config": "can_read",
}
)
# Only Admin role can access settings
class_permission_name = "Settings"
def _get_setting_metadata(self, key: str) -> Optional[Dict[str, Any]]:
"""Get metadata for a configuration setting."""
if isinstance(current_app.config, SupersetConfig):
return current_app.config.get_setting_metadata(key)
return None
def _get_setting_source(self, key: str) -> str:
"""Determine where a setting value comes from."""
import os
# Check if it's from environment variables
env_key = f"SUPERSET__{key}"
if env_key in os.environ:
return f"environment ({env_key})"
# Check if it's from superset_config.py
try:
import superset_config
if hasattr(superset_config, key):
return "superset_config.py"
except ImportError:
pass
# Check if it's in database
if SettingsDAO.find_by_key(key):
return "database"
# Otherwise it's from defaults
return "config_defaults.py"
def _is_setting_allowed_in_database(self, key: str) -> bool:
"""Check if a setting is allowed to be stored in the database."""
metadata = self._get_setting_metadata(key)
if not metadata:
return False
# Only allow settings that don't require restart and aren't readonly
return not metadata.get("requires_restart", True) and not metadata.get(
"readonly", False
)
def _validate_setting_value(self, key: str, value: Any) -> tuple[bool, list[str]]:
"""Validate a setting value against its metadata."""
metadata = self._get_setting_metadata(key)
if not metadata:
return True, []
if isinstance(current_app.config, SupersetConfig):
if current_app.config.validate_setting(key, value):
return True, []
else:
return False, [
f"Value does not match expected type or constraints for {key}"
]
return True, []
@expose("/", methods=["GET"])
@protect()
@safe
def get_list(self) -> str:
"""Get list of all database settings."""
# Parse query parameters
args = request.args
namespace = args.get("namespace")
category = args.get("category")
include_metadata = args.get("include_metadata", "false").lower() == "true"
include_source = args.get("include_source", "false").lower() == "true"
# Get settings from database
if namespace:
settings = SettingsDAO.get_by_namespace(namespace)
else:
settings = SettingsDAO.get_all_as_dict()
# Build response
result = []
for key, value in settings.items():
setting_data = {
"key": key,
"value": value,
"namespace": namespace, # This would need to be fetched from the model
}
if include_metadata:
setting_data["metadata"] = self._get_setting_metadata(key)
if include_source:
setting_data["source"] = self._get_setting_source(key)
# Filter by category if specified
if category:
metadata = self._get_setting_metadata(key)
if not metadata or metadata.get("category") != category:
continue
result.append(setting_data)
return self.response(200, result=result, count=len(result))
@expose("/<pk>", methods=["GET"])
@protect()
@safe
def get(self, pk: str) -> str:
"""Get a specific setting by key."""
# Parse query parameters
args = request.args
include_metadata = args.get("include_metadata", "false").lower() == "true"
include_source = args.get("include_source", "false").lower() == "true"
# Get setting value
value = SettingsDAO.get_value(pk)
if value is None:
return self.response_404()
# Build response
setting_data = {
"key": pk,
"value": value,
}
if include_metadata:
setting_data["metadata"] = self._get_setting_metadata(pk)
if include_source:
setting_data["source"] = self._get_setting_source(pk)
return self.response(200, **setting_data)
@expose("/", methods=["POST"])
@protect()
@safe
def post(self) -> str:
"""Create a new setting."""
try:
item = self.add_schema.load(request.json)
except ValidationError as error:
return self.response_422(message=error.messages)
key = item["key"]
value = item["value"]
namespace = item.get("namespace")
# Check if setting is allowed in database
if not self._is_setting_allowed_in_database(key):
return self.response_422(
message=f"Setting '{key}' cannot be stored in database. "
f"It may require a restart or be read-only."
)
# Validate value
is_valid, errors = self._validate_setting_value(key, value)
if not is_valid:
return self.response_422(message={"validation_errors": errors})
# Check if setting already exists
if SettingsDAO.find_by_key(key):
return self.response_422(
message=f"Setting '{key}' already exists. Use PUT to update."
)
# Create setting
try:
SettingsDAO.set_value(key, value, namespace)
return self.response(201, key=key, value=value)
except Exception as ex:
logger.exception("Error creating setting")
return self.response_422(message=str(ex))
@expose("/<pk>", methods=["PUT"])
@protect()
@safe
def put(self, pk: str) -> str:
"""Update an existing setting."""
try:
item = self.edit_schema.load(request.json)
except ValidationError as error:
return self.response_422(message=error.messages)
value = item["value"]
namespace = item.get("namespace")
# Check if setting is allowed in database
if not self._is_setting_allowed_in_database(pk):
return self.response_422(
message=f"Setting '{pk}' cannot be stored in database. "
f"It may require a restart or be read-only."
)
# Validate value
is_valid, errors = self._validate_setting_value(pk, value)
if not is_valid:
return self.response_422(message={"validation_errors": errors})
# Update setting
try:
SettingsDAO.set_value(pk, value, namespace)
return self.response(200, key=pk, value=value)
except Exception as ex:
logger.exception("Error updating setting")
return self.response_422(message=str(ex))
@expose("/<pk>", methods=["DELETE"])
@protect()
@safe
def delete(self, pk: str) -> str:
"""Delete a setting."""
# Check if setting exists
if not SettingsDAO.find_by_key(pk):
return self.response_404()
# Delete setting
try:
SettingsDAO.delete_by_key(pk)
return self.response(200, message=f"Setting '{pk}' deleted")
except Exception as ex:
logger.exception("Error deleting setting")
return self.response_422(message=str(ex))
@expose("/validate", methods=["POST"])
@protect()
@safe
def validate(self) -> str:
"""Validate a setting value without saving it."""
if not request.json:
return self.response_400()
key = request.json.get("key")
value = request.json.get("value")
if not key:
return self.response_422(message="Key is required")
# Get metadata
metadata = self._get_setting_metadata(key)
# Validate value
is_valid, errors = self._validate_setting_value(key, value)
# Check if allowed in database
allowed_in_db = self._is_setting_allowed_in_database(key)
response_data = {
"key": key,
"value": value,
"valid": is_valid,
"errors": errors,
"metadata": metadata,
"allowed_in_database": allowed_in_db,
}
return self.response(200, **response_data)
@expose("/metadata", methods=["GET"])
@protect()
@safe
def metadata(self) -> str:
"""Get metadata for all documented settings."""
if not isinstance(current_app.config, SupersetConfig):
return self.response(200, metadata={})
metadata = current_app.config.DATABASE_SETTINGS_SCHEMA
# Filter by category if specified
if category := request.args.get("category"):
metadata = current_app.config.get_settings_by_category(category)
return self.response(200, metadata=metadata)
@expose("/effective_config", methods=["GET"])
@protect()
@safe
def effective_config(self) -> str:
"""Get effective configuration (database + env + defaults)."""
# Get current config values
config_dict = {}
# Get documented settings
if isinstance(current_app.config, SupersetConfig):
for key in current_app.config.DATABASE_SETTINGS_SCHEMA:
if key in current_app.config:
config_dict[key] = {
"value": current_app.config[key],
"source": self._get_setting_source(key),
"metadata": self._get_setting_metadata(key),
}
# Add database settings
db_settings = SettingsDAO.get_all_as_dict()
for key, value in db_settings.items():
if key not in config_dict:
config_dict[key] = {
"value": value,
"source": "database",
"metadata": self._get_setting_metadata(key),
}
return self.response(200, config=config_dict)
+110
View File
@@ -0,0 +1,110 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from marshmallow import fields, Schema, validate
class SettingCreateSchema(Schema):
"""Schema for creating a new setting."""
key = fields.String(required=True, validate=validate.Length(min=1, max=255))
value = fields.Raw(required=True)
namespace = fields.String(allow_none=True, validate=validate.Length(max=100))
class SettingUpdateSchema(Schema):
"""Schema for updating an existing setting."""
value = fields.Raw(required=True)
namespace = fields.String(allow_none=True, validate=validate.Length(max=100))
class SettingResponseSchema(Schema):
"""Schema for setting response."""
key = fields.String()
value = fields.Raw()
namespace = fields.String(allow_none=True)
created_on = fields.DateTime()
changed_on = fields.DateTime()
created_by = fields.Nested("UserSchema", only=["id", "first_name", "last_name"])
changed_by = fields.Nested("UserSchema", only=["id", "first_name", "last_name"])
metadata = fields.Dict(allow_none=True) # Configuration metadata if available
source = fields.String(
allow_none=True
) # Source of the setting (database, env, defaults)
class SettingListResponseSchema(Schema):
"""Schema for listing settings."""
result = fields.List(fields.Nested(SettingResponseSchema))
count = fields.Integer()
class SettingMetadataSchema(Schema):
"""Schema for configuration metadata."""
title = fields.String()
description = fields.String()
type = fields.String()
category = fields.String()
impact = fields.String()
requires_restart = fields.Boolean()
default = fields.Raw()
minimum = fields.Integer(allow_none=True)
maximum = fields.Integer(allow_none=True)
readonly = fields.Boolean()
documentation_url = fields.String(allow_none=True)
class SettingValidationSchema(Schema):
"""Schema for setting validation."""
key = fields.String(required=True)
value = fields.Raw(required=True)
valid = fields.Boolean()
errors = fields.List(fields.String())
metadata = fields.Nested(SettingMetadataSchema, allow_none=True)
# Query schemas for API endpoints
setting_get_schema = {
"type": "object",
"properties": {
"include_metadata": {"type": "boolean"},
"include_source": {"type": "boolean"},
},
}
setting_list_schema = {
"type": "object",
"properties": {
"namespace": {"type": "string"},
"category": {"type": "string"},
"include_metadata": {"type": "boolean"},
"include_source": {"type": "boolean"},
},
}
setting_validate_schema = {
"type": "object",
"properties": {
"key": {"type": "string"},
"value": {}, # Any type
},
"required": ["key", "value"],
}
+1 -1
View File
@@ -70,7 +70,7 @@ SQLGLOT_DIALECTS = {
# "denodo": ???
"dremio": Dremio,
"drill": Dialects.DRILL,
# "druid": Dialects.DRUID, # DRUID dialect not available in current sqlglot version
"druid": Dialects.DRUID,
"duckdb": Dialects.DUCKDB,
# "dynamodb": ???
# "elasticsearch": ???
@@ -3382,7 +3382,7 @@ msgid "Custom"
msgstr "تقليد"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "التنسيق الشرطي"
msgid "Custom Plugin"
@@ -9862,7 +9862,7 @@ msgid "Show CREATE VIEW statement"
msgstr "عرض بيان إنشاء عرض"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "عرض أشرطة الخلايا"
msgid "Show Dashboard"
@@ -13915,7 +13915,7 @@ msgstr "يجب أن يكون `row_offset` أكبر من أو يساوي 0"
msgid "`width` must be greater or equal to 0"
msgstr "يجب أن يكون «العرض» أكبر أو يساوي 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13962,7 +13962,7 @@ msgid "background"
msgstr "خلفية"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "التنسيق الشرطي"
msgid "basis"
@@ -3353,7 +3353,7 @@ msgstr "Actualment renderitzat: %s"
msgid "Custom"
msgstr "Personalitzat"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Format Condicional Personalitzat"
msgid "Custom Plugin"
@@ -9647,7 +9647,7 @@ msgstr "Mostrar Bombolles"
msgid "Show CREATE VIEW statement"
msgstr "Mostrar sentència CREATE VIEW"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Mostrar barres de cel·la"
msgid "Show Dashboard"
@@ -13708,7 +13708,7 @@ msgstr "`row_offset` ha de ser major o igual a 0"
msgid "`width` must be greater or equal to 0"
msgstr "`width` ha de ser major o igual a 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "afegir colors a les barres de cel·la per +/-"
msgid "aggregate"
@@ -13753,7 +13753,7 @@ msgstr "auto"
msgid "background"
msgstr "fons"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "formatat condicional bàsic"
msgid "basis"
@@ -3494,7 +3494,7 @@ msgstr "Derzeit dargestellt: %s"
msgid "Custom"
msgstr "Angepasst"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Benutzerdefinierte bedingte Formatierung"
msgid "Custom Plugin"
@@ -10043,7 +10043,7 @@ msgstr "Blasen anzeigen"
msgid "Show CREATE VIEW statement"
msgstr "CREATE VIEW-Anweisung anzeigen"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Zellenbalken anzeigen"
msgid "Show Dashboard"
@@ -14358,7 +14358,7 @@ msgstr "\"row_offset\" muss größer oder gleich 0 sein"
msgid "`width` must be greater or equal to 0"
msgstr "\"Breite\" muss größer oder gleich 0 sein"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "Farben zu den Zellenbalken für +/- hinzufügen"
msgid "aggregate"
@@ -14403,7 +14403,7 @@ msgstr "automatisch"
msgid "background"
msgstr "Hintergrund"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "grundlegende bedingte Formatierung"
msgid "basis"
@@ -3138,7 +3138,7 @@ msgstr ""
msgid "Custom"
msgstr ""
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr ""
msgid "Custom Plugin"
@@ -9141,7 +9141,7 @@ msgstr ""
msgid "Show CREATE VIEW statement"
msgstr ""
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr ""
msgid "Show Dashboard"
@@ -12795,7 +12795,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -12840,7 +12840,7 @@ msgstr ""
msgid "background"
msgstr ""
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr ""
msgid "basis"
@@ -3138,7 +3138,7 @@ msgstr "Actualmente renderizado: %s"
msgid "Custom"
msgstr "Personalizado"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Formato condicional personalizado"
msgid "Custom Plugin"
@@ -9141,7 +9141,7 @@ msgstr "Mostrar burbujas"
msgid "Show CREATE VIEW statement"
msgstr "Mostrar instrucción CREAR VISTA"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Mostrar barras de celda"
msgid "Show Dashboard"
@@ -12795,7 +12795,7 @@ msgstr "«row_offset» debe ser mayor o igual a 0"
msgid "`width` must be greater or equal to 0"
msgstr "«width» debe ser mayor o igual a 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "añade colores a las barras de celdas para +/-"
msgid "aggregate"
@@ -12840,7 +12840,7 @@ msgstr "automático"
msgid "background"
msgstr "fondo"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "formato condicional básico"
msgid "basis"
@@ -3376,7 +3376,7 @@ msgstr "در حال حاضر رندر شده: %s"
msgid "Custom"
msgstr "سفارشی"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "قالب‌بندی شرطی سفارشی"
msgid "Custom Plugin"
@@ -9784,7 +9784,7 @@ msgstr "نمایش حباب‌ها"
msgid "Show CREATE VIEW statement"
msgstr "بیان CREATE VIEW را نشان بدهید"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "نمودار میله‌ای سلول‌ها را نمایش بدهید"
msgid "Show Dashboard"
@@ -13905,7 +13905,7 @@ msgstr "`row_offset` باید بزرگتر از یا برابر با ۰ باشد
msgid "`width` must be greater or equal to 0"
msgstr "`عرض` باید بزرگتر یا برابر با ۰ باشد."
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "رنگ‌ها را به نوارهای سلول برای + و - اضافه کنید"
msgid "aggregate"
@@ -13950,7 +13950,7 @@ msgstr "خودکار"
msgid "background"
msgstr "پس‌زمینه"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "فرمت‌بندی شرطی پایه"
msgid "basis"
@@ -3681,7 +3681,7 @@ msgid "Custom"
msgstr "Personnalisé"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Formatage conditionnel"
msgid "Custom Plugin"
@@ -10819,7 +10819,7 @@ msgid "Show CREATE VIEW statement"
msgstr "Afficher l’énoncé CREATE VIEW"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Afficher les barres de cellules"
msgid "Show Dashboard"
@@ -15453,7 +15453,7 @@ msgstr "« row_offset » doit être plus grand ou égal à 0"
msgid "`width` must be greater or equal to 0"
msgstr "« width » doit être plus grand ou égal à 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "ajouter des couleurs aux barres de cellules pour +/-"
msgid "aggregate"
@@ -15496,7 +15496,7 @@ msgstr "auto"
msgid "background"
msgstr "contexte"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "formatage conditionnel basique"
#, fuzzy
@@ -3374,7 +3374,7 @@ msgid "Custom"
msgstr ""
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Formato Datetime"
msgid "Custom Plugin"
@@ -9873,7 +9873,7 @@ msgid "Show CREATE VIEW statement"
msgstr ""
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Grafico a Proiettile"
msgid "Show Dashboard"
@@ -13738,7 +13738,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13787,7 +13787,7 @@ msgid "background"
msgstr ""
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Formato Datetime"
msgid "basis"
@@ -3209,7 +3209,7 @@ msgstr "現在レンダリング中: %s"
msgid "Custom"
msgstr "カスタム"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "カスタム条件付き書式"
msgid "Custom Plugin"
@@ -9277,7 +9277,7 @@ msgstr "バブルを表示"
msgid "Show CREATE VIEW statement"
msgstr "CREATE VIEW文を表示"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "セルバーを表示"
msgid "Show Dashboard"
@@ -13058,7 +13058,7 @@ msgstr "「row_offset」は 0 以上でなければなりません。"
msgid "`width` must be greater or equal to 0"
msgstr "「幅」は 0 以上でなければなりません。"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13103,7 +13103,7 @@ msgstr "自動"
msgid "background"
msgstr "背景"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "基本的な条件付き書式"
msgid "basis"
@@ -3352,7 +3352,7 @@ msgid "Custom"
msgstr ""
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "주석"
msgid "Custom Plugin"
@@ -9773,7 +9773,7 @@ msgid "Show CREATE VIEW statement"
msgstr ""
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "차트 추가"
msgid "Show Dashboard"
@@ -13592,7 +13592,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13641,7 +13641,7 @@ msgid "background"
msgstr ""
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "주석"
msgid "basis"
+4 -4
View File
@@ -3141,7 +3141,7 @@ msgstr ""
msgid "Custom"
msgstr ""
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr ""
msgid "Custom Plugin"
@@ -9128,7 +9128,7 @@ msgstr ""
msgid "Show CREATE VIEW statement"
msgstr ""
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr ""
msgid "Show Dashboard"
@@ -12780,7 +12780,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -12825,7 +12825,7 @@ msgstr ""
msgid "background"
msgstr ""
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr ""
msgid "basis"
@@ -3442,7 +3442,7 @@ msgid "Custom"
msgstr "Aangepast"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Voorwaardelijke opmaak"
msgid "Custom Plugin"
@@ -9958,7 +9958,7 @@ msgid "Show CREATE VIEW statement"
msgstr "Toon CREATE VIEW statement"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Toon cel balken"
msgid "Show Dashboard"
@@ -14170,7 +14170,7 @@ msgstr "`rij_offset` moet groter zijn dan of gelijk aan 0"
msgid "`width` must be greater or equal to 0"
msgstr "`breedte` moet groter of gelijk zijn aan 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -14217,7 +14217,7 @@ msgid "background"
msgstr "achtergrond"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Voorwaardelijke opmaak"
msgid "basis"
@@ -3557,7 +3557,7 @@ msgid "Custom"
msgstr "Niestandardowy"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Niestandardowe formatowanie warunkowe"
msgid "Custom Plugin"
@@ -10392,7 +10392,7 @@ msgid "Show CREATE VIEW statement"
msgstr "Pokaż instrukcję CREATE VIEW"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Pokaż paski komórek"
msgid "Show Dashboard"
@@ -14786,7 +14786,7 @@ msgstr "`row_offset` musi być większe lub równe 0"
msgid "`width` must be greater or equal to 0"
msgstr "`width` musi być większe lub równe 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "dodaj kolory do pasków komórek dla +/-"
msgid "aggregate"
@@ -14835,7 +14835,7 @@ msgid "background"
msgstr "tło"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Podstawowe formatowanie warunkowe"
#, fuzzy
@@ -3416,7 +3416,7 @@ msgid "Custom"
msgstr ""
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Metadados adicionais"
msgid "Custom Plugin"
@@ -10009,7 +10009,7 @@ msgid "Show CREATE VIEW statement"
msgstr ""
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Gráfico de bala"
msgid "Show Dashboard"
@@ -13939,7 +13939,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13987,7 +13987,7 @@ msgid "background"
msgstr ""
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Metadados adicionais"
msgid "basis"
@@ -3502,7 +3502,7 @@ msgid "Custom"
msgstr "Personalizado"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Formatação condicional"
msgid "Custom Plugin"
@@ -10106,7 +10106,7 @@ msgid "Show CREATE VIEW statement"
msgstr "Mostrar instrução CREATE VIEW"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Mostrar barras de células"
msgid "Show Dashboard"
@@ -14348,7 +14348,7 @@ msgstr "`row_offset` deve ser maior ou igual a 0"
msgid "`width` must be greater or equal to 0"
msgstr "`largura` deve ser maior ou igual a 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "adicionar cores para as barras para +/-"
msgid "aggregate"
@@ -14396,7 +14396,7 @@ msgid "background"
msgstr "fundo"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "formatação condicional básica"
msgid "basis"
@@ -3410,7 +3410,7 @@ msgstr "Сейчас отрисовано: %s"
msgid "Custom"
msgstr "Пользовательский"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Условное форматирование"
msgid "Custom Plugin"
@@ -9909,7 +9909,7 @@ msgstr "Показать пузыри"
msgid "Show CREATE VIEW statement"
msgstr "Показать выражение CREATE VIEW"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Наложить гистограммы на ячейки"
msgid "Show Dashboard"
@@ -14016,7 +14016,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr "Ширина должна быть не менее нуля"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "Окрасить столбцы для +/-"
msgid "aggregate"
@@ -14062,7 +14062,7 @@ msgstr "автоматически"
msgid "background"
msgstr ""
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Простое условное форматирование"
#, fuzzy
@@ -3168,7 +3168,7 @@ msgstr ""
msgid "Custom"
msgstr ""
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr ""
msgid "Custom Plugin"
@@ -9249,7 +9249,7 @@ msgid "Show CREATE VIEW statement"
msgstr ""
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Importovať dashboard"
msgid "Show Dashboard"
@@ -12929,7 +12929,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -12974,7 +12974,7 @@ msgstr ""
msgid "background"
msgstr ""
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr ""
msgid "basis"
@@ -3406,7 +3406,7 @@ msgstr "Trenutno izrisano: %s"
msgid "Custom"
msgstr "Prilagojen"
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Prilagojeno pogojno oblikovanje"
msgid "Custom Plugin"
@@ -9819,7 +9819,7 @@ msgstr "Prikaži mehurčke"
msgid "Show CREATE VIEW statement"
msgstr "Prikaži CREATE VIEW stavek"
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Prikaži grafe v celicah"
msgid "Show Dashboard"
@@ -13928,7 +13928,7 @@ msgstr "`row_offset` mora biti večja ali enaka 0"
msgid "`width` must be greater or equal to 0"
msgstr "`width` mora biti večja ali enaka 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr "dodajte barvo za graf v celici za +/-"
msgid "aggregate"
@@ -13973,7 +13973,7 @@ msgstr "samodejno"
msgid "background"
msgstr "ozadje"
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "osnovno pogojno oblikovanje"
msgid "basis"
@@ -3184,7 +3184,7 @@ msgid "Custom"
msgstr ""
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Ek bilgi"
msgid "Custom Plugin"
@@ -9284,7 +9284,7 @@ msgstr ""
msgid "Show CREATE VIEW statement"
msgstr ""
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr ""
msgid "Show Dashboard"
@@ -12972,7 +12972,7 @@ msgstr ""
msgid "`width` must be greater or equal to 0"
msgstr ""
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13019,7 +13019,7 @@ msgid "background"
msgstr ""
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Ek bilgi"
msgid "basis"
@@ -3436,7 +3436,7 @@ msgid "Custom"
msgstr "Звичайний"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "Умовне форматування"
msgid "Custom Plugin"
@@ -9968,7 +9968,7 @@ msgid "Show CREATE VIEW statement"
msgstr "Показати заяву про створення перегляду"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "Показати стільникові смуги"
msgid "Show Dashboard"
@@ -14148,7 +14148,7 @@ msgstr "`row_offset` повинен бути більшим або рівним
msgid "`width` must be greater or equal to 0"
msgstr "`width` повинна бути більшою або рівною 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -14195,7 +14195,7 @@ msgid "background"
msgstr "фон"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "Умовне форматування"
msgid "basis"
@@ -3393,7 +3393,7 @@ msgid "Custom"
msgstr "自定义"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "条件格式设置"
msgid "Custom Plugin"
@@ -9908,7 +9908,7 @@ msgid "Show CREATE VIEW statement"
msgstr "显示 CREATE VIEW 语句"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "显示单元格的柱"
msgid "Show Dashboard"
@@ -13820,7 +13820,7 @@ msgstr "`行偏移量` 必须大于或等于0"
msgid "`width` must be greater or equal to 0"
msgstr "`宽度` 必须大于或等于0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13870,7 +13870,7 @@ msgid "background"
msgstr "背景"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "条件格式设置"
#, fuzzy
@@ -3396,7 +3396,7 @@ msgid "Custom"
msgstr "自定義"
#, fuzzy
msgid "Custom conditional formatting"
msgid "Custom Conditional Formatting"
msgstr "條件格式設定"
msgid "Custom Plugin"
@@ -9922,7 +9922,7 @@ msgid "Show CREATE VIEW statement"
msgstr "顯示 CREATE VIEW 語句"
#, fuzzy
msgid "Show cell bars"
msgid "Show Cell bars"
msgstr "顯示單元格的柱"
msgid "Show Dashboard"
@@ -13834,7 +13834,7 @@ msgstr "`行偏移量` 必須大於或等於 0"
msgid "`width` must be greater or equal to 0"
msgstr "`寬度` 必須大於或等於 0"
msgid "Add colors to cell bars for +/-"
msgid "add colors to cell bars for +/-"
msgstr ""
msgid "aggregate"
@@ -13884,7 +13884,7 @@ msgid "background"
msgstr "背景"
#, fuzzy
msgid "Basic conditional formatting"
msgid "basic conditional formatting"
msgstr "條件格式設定"
#, fuzzy
+1 -1
View File
@@ -411,7 +411,7 @@ class BaseViz: # pylint: disable=too-many-public-methods
"groupby": groupby,
"metrics": metrics,
"row_limit": row_limit,
"filters": self.form_data.get("filters", []),
"filter": self.form_data.get("filters", []),
"timeseries_limit": limit,
"extras": extras,
"timeseries_limit_metric": timeseries_limit_metric,
+2 -2
View File
@@ -29,8 +29,8 @@ query_birth_names = {
"row_limit": 100,
"granularity": "ds",
"time_range": "100 years ago : now",
"series_limit": 0,
"series_limit_metric": None,
"timeseries_limit": 0,
"timeseries_limit_metric": None,
"order_desc": True,
"filters": [
{"col": "gender", "op": "==", "val": "boy"},
@@ -902,9 +902,6 @@ class TestPostChartDataApi(BaseTestChartDataApi):
request_payload["queries"][0]["columns"] = ["foo", "bar", "state"]
request_payload["queries"][0]["where"] = "':abc' != ':xyz:qwerty'"
request_payload["queries"][0]["orderby"] = None
request_payload["queries"][0]["granularity"] = (
None # Virtual table has no time column
)
request_payload["queries"][0]["metrics"] = [
{
"expressionType": AdhocMetricExpressionType.SQL,
@@ -1015,7 +1012,7 @@ class TestGetChartDataApi(BaseTestChartDataApi):
"orderby": [["sum__num", False]],
"annotation_layers": [],
"row_limit": 50000,
"series_limit": 0,
"timeseries_limit": 0,
"order_desc": True,
"url_params": {},
"custom_params": {},
@@ -1068,7 +1065,7 @@ class TestGetChartDataApi(BaseTestChartDataApi):
"orderby": [["sum__num", False]],
"annotation_layers": [],
"row_limit": 50000,
"series_limit": 0,
"timeseries_limit": 0,
"order_desc": True,
"url_params": {},
"custom_params": {},
@@ -1122,7 +1119,7 @@ class TestGetChartDataApi(BaseTestChartDataApi):
"orderby": [["sum__num", False]],
"annotation_layers": [],
"row_limit": 50000,
"series_limit": 0,
"timeseries_limit": 0,
"order_desc": True,
"url_params": {},
"custom_params": {},
+14 -22
View File
@@ -124,6 +124,17 @@ class TestDatasource(SupersetTestCase):
else:
return
query_obj = {
"columns": ["metric"],
"filter": [],
"from_dttm": datetime.now() - timedelta(days=1),
"granularity": "additional_dttm",
"orderby": [],
"to_dttm": datetime.now() + timedelta(days=1),
"series_columns": [],
"row_limit": 1000,
"row_offset": 0,
}
table = SqlaTable(
table_name="dummy_sql_table",
database=database,
@@ -138,28 +149,13 @@ class TestDatasource(SupersetTestCase):
sql=sql,
)
from superset.common.query_object import QueryObject
query_obj = QueryObject(
columns=["metric"],
filters=[],
from_dttm=datetime.now() - timedelta(days=1),
granularity="additional_dttm",
orderby=[],
to_dttm=datetime.now() + timedelta(days=1),
series_columns=[],
row_limit=1000,
row_offset=0,
datasource=table,
)
with create_and_cleanup_table(table):
table.always_filter_main_dttm = False
result = str(table.get_sqla_query(query_obj).sqla_query.whereclause)
result = str(table.get_sqla_query(**query_obj).sqla_query.whereclause)
assert "default_dttm" not in result and "additional_dttm" in result # noqa: PT018
table.always_filter_main_dttm = True
result = str(table.get_sqla_query(query_obj).sqla_query.whereclause)
result = str(table.get_sqla_query(**query_obj).sqla_query.whereclause)
assert "default_dttm" in result and "additional_dttm" in result # noqa: PT018
def test_external_metadata_for_virtual_table(self):
@@ -588,11 +584,7 @@ def test_get_samples_with_incorrect_cc(test_client, login_as_admin, virtual_data
)
rv = test_client.post(uri, json={})
assert rv.status_code == 422
# The error handling returns a simple error message for CommandInvalidError
assert "error" in rv.json
assert (
"DUMMY CC" in rv.json["error"]
) # Check the error mentions the problematic column
assert rv.json["errors"][0]["error_type"] == "INVALID_SQL_ERROR"
@with_feature_flags(ALLOW_ADHOC_SUBQUERY=True)

Some files were not shown because too many files have changed in this diff Show More