superset2/docs/developer_docs/guidelines/backend-style-guidelines.md

---
title: Overview
sidebar_position: 1
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

# Backend Style Guidelines

This is a list of statements that describe how we do backend development in Superset. While they might not be 100% true for all files in the repo, they represent the gold standard we strive towards for backend quality and style.

- We use a monolithic Python/Flask/Flask-AppBuilder backend, with small single-responsibility satellite services where necessary.
- Files are generally organized by feature or object type. Within each domain, we can have api controllers, models, schemas, commands, and data access objects (DAO).
  - See: [Proposal for Improving Superset's Python Code Organization](https://github.com/apache/superset/issues/9077)
- API controllers use Marshmallow Schemas to serialize/deserialize data.
- Authentication and authorization are controlled by the [security manager](https://github.com/apache/superset/blob/master/superset/security/manager).
- We use Pytest for unit and integration tests. These live in the `tests` directory.
- We add tests for every new piece of functionality added to the backend.
- We use pytest fixtures to share setup between tests.
- We use SQLAlchemy to access both Superset's application database, and users' analytics databases.
- We make changes backwards compatible whenever possible.
  - If a change cannot be made backwards compatible, it goes into a major release.
  - See: [Proposal For Semantic Versioning](https://github.com/apache/superset/issues/12566)
- We use Swagger for API documentation, with docs written inline on the API endpoint code.
- We prefer thin ORM models, putting shared functionality in other utilities.
- Several linters/checkers are used to maintain consistent code style and type safety: pylint, mypy, black, isort.
- `__init__.py` files are kept empty to avoid implicit dependencies.

## Code Organization

### Domain-Driven Structure

Organize code by business domain rather than technical layers:

```
superset/
├── dashboards/
│   ├── api.py          # API endpoints
│   ├── commands/       # Business logic
│   ├── dao.py          # Data access layer
│   ├── models.py       # Database models
│   └── schemas.py      # Serialization schemas
├── charts/
│   ├── api.py
│   ├── commands/
│   ├── dao.py
│   ├── models.py
│   └── schemas.py
```

### API Controllers

Use Flask-AppBuilder with Marshmallow schemas:

```python
from flask_appbuilder.api import BaseApi
from marshmallow import Schema, fields

class DashboardSchema(Schema):
    id = fields.Integer()
    dashboard_title = fields.String(required=True)
    created_on = fields.DateTime(dump_only=True)

class DashboardApi(BaseApi):
    resource_name = "dashboard"
    openapi_spec_tag = "Dashboards"

    @expose("/", methods=["GET"])
    @protect()
    @safe
    def get_list(self) -> Response:
        """Get a list of dashboards"""
        # Implementation
```

### Commands Pattern

Use commands for business logic:

```python
from typing import Optional
from superset.commands.base import BaseCommand
from superset.dashboards.dao import DashboardDAO

class CreateDashboardCommand(BaseCommand):
    def __init__(self, properties: Dict[str, Any]):
        self._properties = properties

    def run(self) -> Dashboard:
        self.validate()
        return DashboardDAO.create(self._properties)

    def validate(self) -> None:
        if not self._properties.get("dashboard_title"):
            raise ValidationError("Title is required")
```

### Data Access Objects (DAOs)

See: [DAO Style Guidelines and Best Practices](./backend/dao-style-guidelines.md)

## Testing

### Unit Tests

```python
import pytest
from unittest.mock import patch
from superset.dashboards.commands.create import CreateDashboardCommand

def test_create_dashboard_success():
    properties = {
        "dashboard_title": "Test Dashboard",
        "owners": [1]
    }

    command = CreateDashboardCommand(properties)
    dashboard = command.run()

    assert dashboard.dashboard_title == "Test Dashboard"

def test_create_dashboard_validation_error():
    properties = {}  # Missing required title

    command = CreateDashboardCommand(properties)

    with pytest.raises(ValidationError):
        command.run()
```

### Integration Tests

```python
import pytest
from superset.app import create_app
from superset.extensions import db

@pytest.fixture
def app():
    app = create_app()
    app.config["TESTING"] = True
    with app.app_context():
        db.create_all()
        yield app
        db.drop_all()

def test_dashboard_api_create(app, auth_headers):
    with app.test_client() as client:
        response = client.post(
            "/api/v1/dashboard/",
            json={"dashboard_title": "Test Dashboard"},
            headers=auth_headers
        )
        assert response.status_code == 201
```

## Security

### Authorization Decorators

```python
from flask_appbuilder.security.decorators import has_access

class DashboardApi(BaseApi):
    @expose("/", methods=["POST"])
    @protect()
    @has_access("can_write", "Dashboard")
    def post(self) -> Response:
        """Create a new dashboard"""
        # Implementation
```

### Input Validation

```python
from marshmallow import Schema, fields, validate

class DashboardSchema(Schema):
    dashboard_title = fields.String(
        required=True,
        validate=validate.Length(min=1, max=500)
    )
    slug = fields.String(
        validate=validate.Regexp(r'^[a-z0-9-]+$')
    )
```

## Database Operations

### Use SQLAlchemy ORM

```python
from sqlalchemy import Column, Integer, String, Text
from superset.models.helpers import AuditMixinNullable

class Dashboard(Model, AuditMixinNullable):
    __tablename__ = "dashboards"

    id = Column(Integer, primary_key=True)
    dashboard_title = Column(String(500))
    position_json = Column(Text)

    def __repr__(self) -> str:
        return f"<Dashboard {self.dashboard_title}>"
```

### Database Migrations

```python
# migration file
def upgrade():
    op.add_column(
        "dashboards",
        sa.Column("new_column", sa.String(255), nullable=True)
    )

def downgrade():
    op.drop_column("dashboards", "new_column")
```

## Error Handling

### Custom Exceptions

```python
class SupersetException(Exception):
    """Base exception for Superset"""
    pass

class ValidationError(SupersetException):
    """Raised when validation fails"""
    pass

class SecurityException(SupersetException):
    """Raised when security check fails"""
    pass
```

### Error Responses

```python
from flask import jsonify
from werkzeug.exceptions import BadRequest

@app.errorhandler(ValidationError)
def handle_validation_error(error):
    return jsonify({
        "message": str(error),
        "error_type": "VALIDATION_ERROR"
    }), 400
```

## Type Hints and Documentation

### Use Type Hints

```python
from typing import List, Optional, Dict, Any
from superset.models.dashboard import Dashboard

def get_dashboards_by_owner(owner_id: int) -> List[Dashboard]:
    """Get all dashboards owned by a specific user"""
    return db.session.query(Dashboard).filter_by(owner_id=owner_id).all()

def create_dashboard(properties: Dict[str, Any]) -> Optional[Dashboard]:
    """Create a new dashboard with the given properties"""
    # Implementation
```

### API Documentation

```python
from flask_appbuilder.api import BaseApi
from flask_appbuilder.api.schemas import get_list_schema

class DashboardApi(BaseApi):
    @expose("/", methods=["GET"])
    @protect()
    @safe
    def get_list(self) -> Response:
        """Get a list of dashboards
        ---
        get:
          description: >-
            Get a list of dashboards
          parameters:
          - in: query
            schema:
              type: integer
            name: page_size
            description: Number of results per page
          responses:
            200:
              description: Success
              content:
                application/json:
                  schema:
                    $ref: '#/components/schemas/DashboardListResponse'
        """
        # Implementation
```