mirror of
https://github.com/apache/superset.git
synced 2026-05-22 16:25:49 +00:00
docs: cut 6.1.0 versions for docs, admin_docs, developer_docs, components
Snapshots all four versioned Docusaurus sections at v6.1.0. Built on top of the version-cutting tooling work in chore/docs-cut-6.1.0-versions so the snapshot benefits from: - Auto-gen refresh before snapshotting (database pages from engine spec metadata, API reference from openapi.json, component pages from Storybook stories) — captured at the SHA we cut from rather than whatever happened to be on disk. - Data-import freeze: country list, feature flag table, database diagnostics, and component metadata are copied into snapshot-local `_versioned_data/` dirs so the historical version doesn't silently mutate when the source files change. - Depth-aware import-path rewriter that handles deeply-nested component MDX files referencing `../../../src/` from the snapshot. Versioning behavior: `lastVersion` stays at `current` for every section, so the canonical URLs (`/docs/...`, `/admin-docs/...`, `/developer-docs/...`, `/components/...`) continue to render content from master. The `current` version is consistently labeled "Next" with an `unreleased` banner, and `6.1.0` is a historical pin accessible only via its explicit version segment. Component playground: previously `disabled: true` in versions-config.json, now enabled and versioned. The plugin block in docusaurus.config.ts was already gated only by the `disabled` flag, so no other code changes were needed to bring it back online. The frozen `databases.json` in the snapshot is the canonical 80-database artifact from the latest committed state in master (preserved by the generator's input-hash cache), not a fallback regenerated from a local Flask environment.
This commit is contained in:
Claude Code
@@ -0,0 +1,171 @@
|
||||
---
|
||||
title: Backend Testing
|
||||
sidebar_position: 3
|
||||
---
|
||||
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
-->
|
||||
|
||||
# Backend Testing
|
||||
|
||||
🚧 **Coming Soon** 🚧
|
||||
|
||||
Complete guide for testing Superset's Python backend, APIs, and database interactions.
|
||||
|
||||
## Topics to be covered:
|
||||
|
||||
- Pytest configuration and fixtures
|
||||
- Unit testing best practices
|
||||
- Integration testing with databases
|
||||
- API endpoint testing
|
||||
- Mocking strategies and patterns
|
||||
- Testing async operations with Celery
|
||||
- Security testing guidelines
|
||||
- Performance and load testing
|
||||
- Test database setup and teardown
|
||||
- Coverage requirements
|
||||
|
||||
## Quick Commands
|
||||
|
||||
```bash
|
||||
# Run all backend tests
|
||||
pytest
|
||||
|
||||
# Run specific test file
|
||||
pytest tests/unit_tests/specific_test.py
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=superset
|
||||
|
||||
# Run tests in parallel
|
||||
pytest -n auto
|
||||
|
||||
# Run only unit tests
|
||||
pytest tests/unit_tests/
|
||||
|
||||
# Run only integration tests
|
||||
pytest tests/integration_tests/
|
||||
```
|
||||
|
||||
## Testing Alerts & Reports with Celery and MailHog
|
||||
|
||||
The Alerts & Reports feature relies on Celery for task scheduling and execution. To test it locally, you need Redis (message broker), Celery Beat (scheduler), a Celery Worker (executor), and an SMTP server to receive email notifications.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Redis running on `localhost:6379`
|
||||
- [MailHog](https://github.com/mailhog/MailHog) installed (a local SMTP server with a web UI for viewing caught emails)
|
||||
|
||||
### superset_config.py
|
||||
|
||||
Your `CeleryConfig` **must** include `beat_schedule`. When you define a custom `CeleryConfig` class in `superset_config.py`, it replaces the default entirely. If you omit `beat_schedule`, Celery Beat will start but never schedule any report tasks.
|
||||
|
||||
```python
|
||||
from celery.schedules import crontab
|
||||
from superset.tasks.types import ExecutorType
|
||||
|
||||
REDIS_HOST = "localhost"
|
||||
REDIS_PORT = "6379"
|
||||
|
||||
class CeleryConfig:
|
||||
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
|
||||
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
|
||||
broker_connection_retry_on_startup = True
|
||||
imports = (
|
||||
"superset.sql_lab",
|
||||
"superset.tasks.scheduler",
|
||||
"superset.tasks.thumbnails",
|
||||
"superset.tasks.cache",
|
||||
)
|
||||
worker_prefetch_multiplier = 10
|
||||
task_acks_late = True
|
||||
beat_schedule = {
|
||||
"reports.scheduler": {
|
||||
"task": "reports.scheduler",
|
||||
"schedule": crontab(minute="*", hour="*"),
|
||||
},
|
||||
"reports.prune_log": {
|
||||
"task": "reports.prune_log",
|
||||
"schedule": crontab(minute=0, hour=0),
|
||||
},
|
||||
}
|
||||
|
||||
CELERY_CONFIG = CeleryConfig
|
||||
|
||||
# SMTP settings pointing to MailHog
|
||||
SMTP_HOST = "localhost"
|
||||
SMTP_PORT = 1025
|
||||
SMTP_STARTTLS = False
|
||||
SMTP_SSL = False
|
||||
SMTP_USER = ""
|
||||
SMTP_PASSWORD = ""
|
||||
SMTP_MAIL_FROM = "superset@localhost"
|
||||
|
||||
# Must match where your frontend is running
|
||||
WEBDRIVER_BASEURL = "http://localhost:9000/"
|
||||
|
||||
ALERT_REPORTS_EXECUTE_AS = [ExecutorType.OWNER]
|
||||
|
||||
FEATURE_FLAGS = {
|
||||
"ALERT_REPORTS": True,
|
||||
# Recommended for better screenshot support (WebGL/DeckGL charts)
|
||||
"PLAYWRIGHT_REPORTS_AND_THUMBNAILS": True,
|
||||
}
|
||||
```
|
||||
|
||||
:::note
|
||||
Do not include `"superset.tasks.async_queries"` in `CeleryConfig.imports` unless you need Global Async Queries. That module accesses `current_app.config` at import time and will crash the worker with a "Working outside of application context" error.
|
||||
:::
|
||||
|
||||
### Starting the Services
|
||||
|
||||
Start MailHog, then Celery Beat and Worker in separate terminals:
|
||||
|
||||
```bash
|
||||
# Terminal 1 - MailHog (SMTP on :1025, Web UI on :8025)
|
||||
MailHog
|
||||
|
||||
# Terminal 2 - Celery Beat (scheduler)
|
||||
celery --app=superset.tasks.celery_app:app beat --loglevel=info
|
||||
|
||||
# Terminal 3 - Celery Worker (executor)
|
||||
celery --app=superset.tasks.celery_app:app worker --concurrency=1 --loglevel=info
|
||||
```
|
||||
|
||||
Use `--concurrency=1` to limit resource usage on your dev machine.
|
||||
|
||||
### Verifying the Setup
|
||||
|
||||
1. **Beat** should log `Scheduler: Sending due task reports.scheduler (reports.scheduler)` once per minute
|
||||
2. **Worker** should log `Scheduling alert <name> eta: <timestamp>` for each active report
|
||||
3. Create a test report in **Settings > Alerts & Reports** with a `* * * * *` cron schedule
|
||||
4. Check **http://localhost:8025** (MailHog web UI) for the email within 1-2 minutes
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
| Problem | Solution |
|
||||
|---|---|
|
||||
| Beat shows no output | Ensure `beat_schedule` is defined in your `CeleryConfig` and `--loglevel=info` is set |
|
||||
| "Report Schedule is still working, refusing to re-compute" | Previous executions are stuck. Reset with: `UPDATE report_schedule SET last_state = 'Not triggered' WHERE id = <id>;` |
|
||||
| Task backlog overwhelming the worker | Flush Redis: `redis-cli FLUSHDB`, then restart Beat and Worker |
|
||||
| Screenshot timeout | Ensure your frontend dev server is running and `WEBDRIVER_BASEURL` matches its URL |
|
||||
|
||||
---
|
||||
|
||||
*This documentation is under active development. Check back soon for updates!*
|
||||
Reference in New Issue
Block a user