Compare commits

..

1 Commits

Author SHA1 Message Date
Michael S. Molina
beb9ec77a5 chore: Adds 4.0.0 RC1 data to CHANGELOG.md and UPDATING.md 2024-02-20 11:42:05 -05:00
7698 changed files with 787790 additions and 1282198 deletions

View File

@@ -17,16 +17,7 @@
# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
---
notifications:
commits: commits@superset.apache.org
issues: notifications@superset.apache.org
pullrequests: notifications@superset.apache.org
discussions: notifications@superset.apache.org
github:
pull_requests:
del_branch_on_merge: true
allow_update_branch: true
description: "Apache Superset is a Data Visualization and Data Exploration Platform"
homepage: https://superset.apache.org/
labels:
@@ -56,17 +47,12 @@ github:
projects: true
# Enable wiki for documentation
wiki: true
# Enable discussions
discussions: true
enabled_merge_buttons:
squash: true
merge: false
rebase: false
ghp_branch: gh-pages
ghp_path: /
protected_branches:
master:
required_status_checks:
@@ -77,23 +63,24 @@ github:
# combination here.
contexts:
- lint-check
- cypress-matrix (0, chrome)
- cypress-matrix (1, chrome)
- cypress-matrix (2, chrome)
- cypress-matrix (3, chrome)
- cypress-matrix (4, chrome)
- cypress-matrix (5, chrome)
- dependency-review
- frontend-build
- playwright-tests (chromium)
- pre-commit (current)
- pre-commit (previous)
- test-mysql
- test-postgres (current)
- test-postgres-hive
- test-postgres-presto
- test-sqlite
- unit-tests (current)
- pre-commit (3.9)
- python-lint (3.9)
- test-mysql (3.9)
- test-postgres (3.9)
- test-postgres (3.10)
- test-sqlite (3.9)
- docker-build (dev, linux/amd64)
- docker-build (lean, linux/amd64)
- docker-build (py310, linux/arm64)
- docker-build (py310, linux/amd64)
- docker-build (websocket, linux/arm64)
- docker-build (websocket, linux/amd64)
- docker-build (dockerize, linux/arm64)
- docker-build (dockerize, linux/amd64)
required_pull_request_reviews:
dismiss_stale_reviews: false
@@ -101,10 +88,3 @@ github:
required_approving_review_count: 1
required_signatures: false
gh-pages:
required_pull_request_reviews:
dismiss_stale_reviews: false
require_code_owner_reviews: true
required_approving_review_count: 1
required_signatures: false

View File

@@ -1,10 +0,0 @@
# JavaScript to TypeScript Migration Command
## Usage
```
/js-to-ts <core-filename>
```
- `<core-filename>` - Path to CORE file relative to `superset-frontend/` (e.g., `src/utils/common.js`, `src/middleware/loggerMiddleware.js`)
## Agent Instructions
**See:** [../projects/js-to-ts/AGENT.md](../projects/js-to-ts/AGENT.md) for complete migration guide.

View File

@@ -1,684 +0,0 @@
# JavaScript to TypeScript Migration Agent Guide
**Complete technical reference for converting JavaScript/JSX files to TypeScript/TSX in Apache Superset frontend.**
**Agent Role:** Atomic migration unit - migrate the core file + ALL related tests/mocks as one cohesive unit. Use `git mv` to preserve history, NO `git commit`. NO global import changes. Report results upon completion.
---
## 🎯 Migration Principles
1. **Atomic migration units** - Core file + all related tests/mocks migrate together
2. **Zero `any` types** - Use proper TypeScript throughout
3. **Leverage existing types** - Reuse established definitions
4. **Type inheritance** - Derivatives extend base component types
5. **Strategic placement** - File types for maximum discoverability
6. **Surgical improvements** - Enhance existing types during migration
---
## Step 0: Dependency Check (MANDATORY)
**Command:**
```bash
grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" superset-frontend/{filename}
```
**Decision:**
- ✅ No matches → Proceed with atomic migration (core + tests + mocks)
- ❌ Matches found → EXIT with dependency report (see format below)
---
## Step 1: Identify Related Files (REQUIRED)
**Atomic Migration Scope:**
For core file `src/utils/example.js`, also migrate:
- `src/utils/example.test.js` / `src/utils/example.test.jsx`
- `src/utils/example.spec.js` / `src/utils/example.spec.jsx`
- `src/utils/__mocks__/example.js`
- Any other related test/mock files found by pattern matching
**Find all related test and mock files:**
```bash
# Pattern-based search for related files
basename=$(basename {filename} .js)
dirname=$(dirname superset-frontend/{filename})
# Find test files
find "$dirname" -name "${basename}.test.js" -o -name "${basename}.test.jsx"
find "$dirname" -name "${basename}.spec.js" -o -name "${basename}.spec.jsx"
# Find mock files
find "$dirname" -name "__mocks__/${basename}.js"
find "$dirname" -name "${basename}.mock.js"
```
**Migration Requirement:** All discovered related files MUST be migrated together as one atomic unit.
**Test File Creation:** If NO test files exist for the core file, CREATE a minimal test file using the following pattern:
- Location: Same directory as core file
- Name: `{basename}.test.ts` (e.g., `DebouncedMessageQueue.test.ts`)
- Content: Basic test structure importing and testing the main functionality
- Use proper TypeScript types in test file
---
## 🗺️ Type Reference Map
### From `@superset-ui/core`
```typescript
// Data & Query
QueryFormData, QueryData, JsonObject, AnnotationData, AdhocMetric
LatestQueryFormData, GenericDataType, DatasourceType, ExtraFormData
DataMaskStateWithId, NativeFilterScope, NativeFiltersState, NativeFilterTarget
// UI & Theme
FeatureFlagMap, LanguagePack, ColorSchemeConfig, SequentialSchemeConfig
```
### From `@superset-ui/chart-controls`
```typescript
Dataset, ColumnMeta, ControlStateMapping
```
### From Local Types (`src/types/`)
```typescript
// Authentication
User, UserWithPermissionsAndRoles, BootstrapUser, PermissionsAndRoles
// Dashboard
Dashboard, DashboardState, DashboardInfo, DashboardLayout, LayoutItem
ComponentType, ChartConfiguration, ActiveFilters
// Charts
Chart, ChartState, ChartStatus, ChartLinkedDashboard, Slice, SaveActionType
// Data
Datasource, Database, Owner, Role
// UI Components
TagType, FavoriteStatus, Filter, ImportResourceName
```
### From Domain Types
```typescript
// src/dashboard/types.ts
RootState, ChartsState, DatasourcesState, FilterBarOrientation
ChartCrossFiltersConfig, ActiveTabs, MenuKeys
// src/explore/types.ts
ExplorePageInitialData, ExplorePageState, ExploreResponsePayload, OptionSortType
// src/SqlLab/types.ts
[SQL Lab specific types]
```
---
## 🏗️ Type Organization Strategy
### Type Placement Hierarchy
1. **Component-Colocated** (90% of cases)
```typescript
// Same file as component
interface MyComponentProps {
title: string;
onClick: () => void;
}
```
2. **Feature-Shared**
```typescript
// src/[domain]/components/[Feature]/types.ts
export interface FilterConfiguration {
filterId: string;
targets: NativeFilterTarget[];
}
```
3. **Domain-Wide**
```typescript
// src/[domain]/types.ts
export interface ExploreFormData extends QueryFormData {
viz_type: string;
}
```
4. **Global**
```typescript
// src/types/[TypeName].ts
export interface ApiResponse<T> {
result: T;
count?: number;
}
```
### Type Discovery Commands
```bash
# Search existing types before creating
find superset-frontend/src -name "types.ts" -exec grep -l "[TypeConcept]" {} \;
grep -r "interface.*Props\|type.*Props" superset-frontend/src/
```
### Derivative Component Patterns
**Rule:** Components that extend others should extend their type interfaces.
```typescript
// ✅ Base component type
interface SelectProps {
value: string | number;
options: SelectOption[];
onChange: (value: string | number) => void;
disabled?: boolean;
}
// ✅ Derivative extends base
interface ChartSelectProps extends SelectProps {
charts: Chart[];
onChartSelect: (chart: Chart) => void;
}
// ✅ Derivative with modified props
interface DatabaseSelectProps extends Omit<SelectProps, 'value' | 'onChange'> {
value: number; // Narrowed type
onChange: (databaseId: number) => void; // Specific signature
}
```
**Common Patterns:**
- **Extension:** `extends BaseProps` - adds new props
- **Omission:** `Omit<BaseProps, 'prop'>` - removes props
- **Modification:** `Omit<BaseProps, 'prop'> & { prop: NewType }` - changes prop type
- **Restriction:** Override with narrower types (union → specific)
---
## 📋 Migration Recipe
### Step 2: File Conversion
```bash
# Use git mv to preserve history
git mv component.js component.ts
git mv Component.jsx Component.tsx
```
### Step 3: Import & Type Setup
```typescript
// Import order (enforced by linting)
import { FC, ReactNode } from 'react';
import { JsonObject, QueryFormData } from '@superset-ui/core';
import { Dataset } from '@superset-ui/chart-controls';
import type { Dashboard } from 'src/types/Dashboard';
```
### Step 4: Function & Component Typing
```typescript
// Functions with proper parameter/return types
export function processData(
data: Dataset[],
config: JsonObject
): ProcessedData[] {
// implementation
}
// Component props with inheritance
interface ComponentProps extends BaseProps {
data: Chart[];
onSelect: (id: number) => void;
}
const Component: FC<ComponentProps> = ({ data, onSelect }) => {
// implementation
};
```
### Step 5: State & Redux Typing
```typescript
// Hooks with specific types
const [data, setData] = useState<Chart[]>([]);
const [selected, setSelected] = useState<number | null>(null);
// Redux with existing RootState
const mapStateToProps = (state: RootState) => ({
charts: state.charts,
user: state.user,
});
```
---
## 🧠 Type Debugging Strategies (Real-World Learnings)
### The Evolution of Type Approaches
When you hit type errors, follow this debugging evolution:
#### 1. ❌ Idealized Union Types (First Attempt)
```typescript
// Looks clean but doesn't match reality
type DatasourceInput = Datasource | QueryEditor;
```
**Problem**: Real calling sites pass variations, not exact types.
#### 2. ❌ Overly Precise Types (Second Attempt)
```typescript
// Tried to match exact calling signatures
type DatasourceInput =
| IDatasource // From DatasourcePanel
| (QueryEditor & { columns: ColumnMeta[] }); // From SaveQuery
```
**Problem**: Too rigid, doesn't handle legacy variations.
#### 3. ✅ Flexible Interface (Final Solution)
```typescript
// Captures what the function actually needs
interface DatasourceInput {
name?: string | null; // Allow null for compatibility
datasource_name?: string | null; // Legacy variations
columns?: any[]; // Multiple column types accepted
database?: { id?: number };
// ... other optional properties
}
```
**Success**: Works with all calling sites, focuses on function needs.
### Type Debugging Process
1. **Start with compilation errors** - they show exact mismatches
2. **Examine actual usage** - look at calling sites, not idealized types
3. **Build flexible interfaces** - capture what functions need, not rigid contracts
4. **Iterate based on downstream validation** - let calling sites guide your types
---
## 🚨 Anti-Patterns to Avoid
```typescript
// ❌ Never use any
const obj: any = {};
// ✅ Use proper types
const obj: Record<string, JsonObject> = {};
// ❌ Don't recreate base component props
interface ChartSelectProps {
value: string; // Duplicated from SelectProps
onChange: () => void; // Duplicated from SelectProps
charts: Chart[]; // New prop
}
// ✅ Inherit and extend
interface ChartSelectProps extends SelectProps {
charts: Chart[]; // Only new props
}
// ❌ Don't create ad-hoc type variations
interface UserInfo {
name: string;
email: string;
}
// ✅ Extend existing types (DRY principle)
import { User } from 'src/types/bootstrapTypes';
type UserDisplayInfo = Pick<User, 'firstName' | 'lastName' | 'email'>;
// ❌ Don't create overly rigid unions
type StrictInput = ExactTypeA | ExactTypeB;
// ✅ Create flexible interfaces for function parameters
interface FlexibleInput {
// Focus on what the function actually needs
commonProperty: string;
optionalVariations?: any; // Allow for legacy variations
}
```
## 📍 DRY Type Guidelines (WHERE TYPES BELONG)
### Type Placement Rules
**CRITICAL**: Type variations must live close to where they belong, not scattered across files.
#### ✅ Proper Type Organization
```typescript
// ❌ Don't create one-off interfaces in utility files
// src/utils/datasourceUtils.ts
interface DatasourceInput { /* custom interface */ } // Wrong!
// ✅ Use existing types or extend them in their proper domain
// src/utils/datasourceUtils.ts
import { IDatasource } from 'src/explore/components/DatasourcePanel';
import { QueryEditor } from 'src/SqlLab/types';
// Create flexible interface that references existing types
interface FlexibleDatasourceInput {
// Properties that actually exist across variations
}
```
#### Type Location Hierarchy
1. **Domain Types**: `src/{domain}/types.ts` (dashboard, explore, SqlLab)
2. **Component Types**: Co-located with components
3. **Global Types**: `src/types/` directory
4. **Utility Types**: Only when they truly don't belong elsewhere
#### ✅ DRY Type Patterns
```typescript
// ✅ Extend existing domain types
interface SaveQueryData extends Pick<QueryEditor, 'sql' | 'dbId' | 'catalog'> {
columns: ColumnMeta[]; // Add what's needed
}
// ✅ Create flexible interfaces for cross-domain utilities
interface CrossDomainInput {
// Common properties that exist across different source types
name?: string | null; // Accommodate legacy null values
// Only include properties the function actually uses
}
```
---
## 🎯 PropTypes Auto-Generation (Elegant Approach)
**IMPORTANT**: Superset has `babel-plugin-typescript-to-proptypes` configured to automatically generate PropTypes from TypeScript interfaces. Use this instead of manual PropTypes duplication!
### ❌ Manual PropTypes Duplication (Avoid This)
```typescript
export interface MyComponentProps {
title: string;
count?: number;
}
// 8+ lines of manual PropTypes duplication 😱
const propTypes = PropTypes.shape({
title: PropTypes.string.isRequired,
count: PropTypes.number,
});
export default propTypes;
```
### ✅ Auto-Generated PropTypes (Use This)
```typescript
import { InferProps } from 'prop-types';
export interface MyComponentProps {
title: string;
count?: number;
}
// Single validator function - babel plugin auto-generates PropTypes! ✨
export default function MyComponentValidator(props: MyComponentProps) {
return null; // PropTypes auto-assigned by babel-plugin-typescript-to-proptypes
}
// Optional: For consumers needing PropTypes type inference
export type MyComponentPropsInferred = InferProps<typeof MyComponentValidator>;
```
### Migration Pattern for Type-Only Files
**When migrating type-only files with manual PropTypes:**
1. **Keep the TypeScript interfaces** (single source of truth)
2. **Replace manual PropTypes** with validator function
3. **Remove PropTypes imports** and manual shape definitions
4. **Add InferProps import** if type inference needed
**Example Migration:**
```typescript
// Before: 25+ lines with manual PropTypes duplication
export interface AdhocFilterType { /* ... */ }
const adhocFilterTypePropTypes = PropTypes.oneOfType([...]);
// After: 3 lines with auto-generation
export interface AdhocFilterType { /* ... */ }
export default function AdhocFilterValidator(props: { filter: AdhocFilterType }) {
return null; // Auto-generated PropTypes by babel plugin
}
```
### Component PropTypes Pattern
**For React components, the babel plugin works automatically:**
```typescript
interface ComponentProps {
title: string;
onClick: () => void;
}
const MyComponent: FC<ComponentProps> = ({ title, onClick }) => {
// Component implementation
};
// PropTypes automatically generated by babel plugin - no manual work needed!
export default MyComponent;
```
### Auto-Generation Benefits
- ✅ **Single source of truth**: TypeScript interfaces drive PropTypes
- ✅ **No duplication**: Eliminate 15-20 lines of manual PropTypes code
- ✅ **Automatic updates**: Changes to TypeScript automatically update PropTypes
- ✅ **Type safety**: Compile-time checking ensures PropTypes match interfaces
- ✅ **Backward compatibility**: Existing JavaScript components continue working
### Babel Plugin Configuration
The plugin is already configured in `babel.config.js`:
```javascript
['babel-plugin-typescript-to-proptypes', { loose: true }]
```
**No additional setup required** - just use TypeScript interfaces and the plugin handles the rest!
---
## 🧪 Test File Migration Patterns
### Test File Priority
- **Always migrate test files** alongside production files
- **Test files are often leaf nodes** - good starting candidates
- **Create tests if missing** - Leverage new TypeScript types for better test coverage
### Test-Specific Type Patterns
```typescript
// Mock interfaces for testing
interface MockStore {
getState: () => Partial<RootState>; // Partial allows minimal mocking
}
// Type-safe mocking for complex objects
const mockDashboardInfo: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
};
// Sinon stub typing
let postStub: sinon.SinonStub;
beforeEach(() => {
postStub = sinon.stub(SupersetClient, 'post');
});
// Use stub reference instead of original method
expect(postStub.callCount).toBe(1);
expect(postStub.getCall(0).args[0].endpoint).toMatch('/api/');
```
### Test Migration Recipe
1. **Migrate production file first** (if both need migration)
2. **Update test imports** to point to `.ts/.tsx` files
3. **Add proper mock typing** using `Partial<T> as T` pattern
4. **Fix stub typing** - Use stub references, not original methods
5. **Verify all tests pass** with TypeScript compilation
---
## 🔧 Type Conflict Resolution
### Multiple Type Definitions Issue
**Problem**: Same type name defined in multiple files causes compilation errors.
**Example**: `DashboardInfo` defined in both:
- `src/dashboard/reducers/types.ts` (minimal)
- `src/dashboard/components/Header/types.ts` (different shape)
- `src/dashboard/types.ts` (complete - used by RootState)
### Resolution Strategy
1. **Identify the authoritative type**:
```bash
# Find which type is used by RootState/main interfaces
grep -r "DashboardInfo" src/dashboard/types.ts
```
2. **Use import from authoritative source**:
```typescript
// ✅ Import from main domain types
import { RootState, DashboardInfo } from 'src/dashboard/types';
// ❌ Don't import from component-specific files
import { DashboardInfo } from 'src/dashboard/components/Header/types';
```
3. **Mock complex types in tests**:
```typescript
// For testing - provide minimal required fields
const mockInfo: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
// Only provide fields actually used in test
};
```
### Type Hierarchy Discovery Commands
```bash
# Find all definitions of a type
grep -r "interface.*TypeName\|type.*TypeName" src/
# Find import usage patterns
grep -r "import.*TypeName" src/
# Check what RootState uses
grep -A 10 -B 10 "TypeName" src/*/types.ts
```
---
## Agent Constraints (CRITICAL)
1. **Use git mv** - Run `git mv file.js file.ts` to preserve git history, but NO `git commit`
2. **NO global import changes** - Don't update imports across codebase
3. **Type files OK** - Can modify existing type files to improve/align types
4. **Single-File TypeScript Validation** (CRITICAL) - tsc has known issues with multi-file compilation:
- **Core Issue**: TypeScript's `tsc` has documented problems validating multiple files simultaneously in complex projects
- **Solution**: ALWAYS validate files one at a time using individual `tsc` calls
- **Command Pattern**: `cd superset-frontend && npx tscw --noEmit --allowJs --composite false --project tsconfig.json {single-file-path}`
- **Why**: Multi-file validation can produce false positives, miss real errors, and conflict during parallel agent execution
5. **Downstream Impact Validation** (CRITICAL) - Your migration affects calling sites:
- **Find downstream files**: `find superset-frontend/src -name "*.tsx" -o -name "*.ts" | xargs grep -l "your-core-filename" 2>/dev/null || echo "No files found"`
- **Validate each downstream file individually**: `cd superset-frontend && npx tscw --noEmit --allowJs --composite false --project tsconfig.json {each-downstream-file}`
- **Fix type mismatches** you introduced in calling sites
- **NEVER ignore downstream errors** - they indicate your types don't match reality
6. **Avoid Project-Wide Validation During Migration**:
- **NEVER use `npm run type`** during parallel agent execution - produces unreliable results
- **Single-file validation is authoritative** - trust individual file checks over project-wide scans
6. **ESLint validation** - Run `npm run eslint -- --fix {file}` for each migrated file to auto-fix formatting/linting issues
6. Zero `any` types - use proper TypeScript types
7. Search existing types before creating new ones
8. Follow patterns from this guide
---
## Success Report Format
```
SUCCESS: Atomic Migration of {core-filename}
## Files Migrated (Atomic Unit)
- Core: {core-filename} → {core-filename.ts/tsx}
- Tests: {list-of-test-files} → {list-of-test-files.ts/tsx} OR "CREATED: {basename}.test.ts"
- Mocks: {list-of-mock-files} → {list-of-mock-files.ts}
- Type files modified: {list-of-type-files}
## Types Created/Improved
- {TypeName}: {location} ({scope}) - {rationale}
- {ExistingType}: enhanced in {location} - {improvement-description}
## Documentation Recommendations
- ADD_TO_DIRECTORY: {TypeName} - {reason}
- NO_DOCUMENTATION: {TypeName} - {reason}
## Quality Validation
- **Single-File TypeScript Validation**: ✅ PASS - Core files individually validated
- Core file: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {core-file}`
- Test files: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {test-file}` (if exists)
- **Downstream Impact Check**: ✅ PASS - Found {N} files importing this module, all validate individually
- Downstream files: {list-of-files-that-import-your-module}
- Individual validation: `npx tscw --noEmit --allowJs --composite false --project tsconfig.json {each-downstream-file}`
- **ESLint validation**: ✅ PASS (using `npm run eslint -- --fix {files}` to auto-fix formatting)
- **Zero any types**: ✅ PASS
- **Local imports resolved**: ✅ PASS
- **Functionality preserved**: ✅ PASS
- **Tests pass** (if test file): ✅ PASS
- **Follow-up action required**: {YES/NO}
## Validation Strategy Notes
- **Single-file approach used**: Avoided multi-file tsc validation due to known TypeScript compilation issues
- **Project-wide validation skipped**: `npm run type` not used during parallel migration to prevent false positives
## Migration Learnings
- Type conflicts encountered: {describe any multiple type definitions}
- Mock patterns used: {describe test mocking approaches}
- Import hierarchy decisions: {note authoritative type sources used}
- PropTypes strategy: {AUTO_GENERATED via babel plugin | MANUAL_DUPLICATION_REMOVED | N/A}
## Improvement Suggestions for Documentation
- AGENT.md enhancement: {suggest additions to migration guide}
- Common pattern identified: {note reusable patterns for future migrations}
```
---
## Dependency Block Report Format
```
DEPENDENCY_BLOCK: Cannot migrate {filename}
## Blocking Dependencies
- {path}: {type} - {usage} - {priority}
## Impact Analysis
- Estimated types: {number}
- Expected locations: {list}
- Cross-domain: {YES/NO}
## Recommended Order
{ordered-list}
```
---
## 📚 Quick Reference
**Type Utilities:**
- `Record<K, V>` - Object with specific key/value types
- `Partial<T>` - All properties optional
- `Pick<T, K>` - Subset of properties
- `Omit<T, K>` - Exclude specific properties
- `NonNullable<T>` - Exclude null/undefined
**Event Types:**
- `MouseEvent<HTMLButtonElement>`
- `ChangeEvent<HTMLInputElement>`
- `FormEvent<HTMLFormElement>`
**React Types:**
- `FC<Props>` - Functional component
- `ReactNode` - Any renderable content
- `CSSProperties` - Style objects
---
**Remember:** Every type should add value and clarity. The goal is meaningful type safety that catches bugs and improves developer experience.

View File

@@ -1,199 +0,0 @@
# JS-to-TS Coordinator Workflow
**Role:** Strategic migration coordination - select leaf-node files, trigger agents, review results, handle integration, manage dependencies.
---
## 1. Core File Selection Strategy
**Target ONLY Core Files**: Coordinators identify core files (production code), agents handle related tests/mocks atomically.
**File Analysis Commands**:
```bash
# Find CORE files with no JS/JSX dependencies (exclude tests/mocks) - SIZE PRIORITIZED
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | xargs wc -l | sort -n | head -20
# Alternative: Get file sizes in lines with paths
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | while read file; do
lines=$(wc -l < "$file")
echo "$lines $file"
done | sort -n | head -20
# Check dependencies for core files only (start with smallest)
for file in <core-files-sorted-by-size>; do
echo "=== $file ($(wc -l < "$file") lines) ==="
grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" "$file" || echo "✅ LEAF CANDIDATE"
done
# Identify heavily imported files (migrate last)
grep -r "from.*utils/common" superset-frontend/src/ | wc -l
# Quick leaf analysis with size priority
find superset-frontend/src -name "*.js" -o -name "*.jsx" | grep -v "test\|spec\|mock" | head -30 | while read file; do
deps=$(grep -E "from '\.\./.*\.jsx?'|from '\./.*\.jsx?'|from 'src/.*\.jsx?'" "$file" | wc -l)
lines=$(wc -l < "$file")
if [ "$deps" -eq 0 ]; then
echo "✅ LEAF: $lines lines - $file"
fi
done | sort -n
```
**Priority Order** (Smallest files first for easier wins):
1. **Small leaf files** (<50 lines) - No JS/JSX imports, quick TypeScript conversion
2. **Medium leaf files** (50-200 lines) - Self-contained utilities and helpers
3. **Small dependency files** (<100 lines) - Import only already-migrated files
4. **Larger components** (200+ lines) - Complex but well-contained functionality
5. **Core foundational files** (utils/common.js, controls.jsx) - migrate last regardless of size
**Size-First Benefits**:
- Faster completion builds momentum
- Earlier validation of migration patterns
- Easier rollback if issues arise
- Better success rate for agent learning
**Migration Unit**: Each agent call migrates:
- 1 core file (primary target)
- All related `*.test.js/jsx` files
- All related `*.mock.js` files
- All related `__mocks__/` files
---
## 2. Task Creation & Agent Control
### Task Triggering
When triggering the `/js-to-ts` command:
- **Task Title**: Use the core filename as the task title (e.g., "DebouncedMessageQueue.js migration", "hostNamesConfig.js migration")
- **Task Description**: Include the full relative path to help agent locate the file
- **Reference**: Point agent to [AGENT.md](./AGENT.md) for technical instructions
### Post-Processing Workflow
After each agent completes:
1. **Review Agent Report**: Always read and analyze the complete agent report
2. **Share Summary**: Provide user with key highlights from agent's work:
- Files migrated (core + tests/mocks)
- Types created or improved
- Any validation issues or coordinator actions needed
3. **Quality Assessment**: Evaluate agent's TypeScript implementation against criteria:
-**Type Usage**: Proper types used, no `any` types
-**Type Filing**: Types placed in correct hierarchy (component → feature → domain → global)
-**Side Effects**: No unintended changes to other files
-**Import Alignment**: Proper .ts/.tsx import extensions
4. **Integration Decision**:
- **COMMIT**: If agent work is complete and high quality
- **FIX & COMMIT**: If minor issues need coordinator fixes
- **ROLLBACK**: If major issues require complete rework
5. **Next Action**: Ask user preference - commit this work or trigger next migration
---
## 3. Integration Decision Framework
**Automatic Integration** ✅:
- `npm run type` passes without errors
- Agent created clean TypeScript with proper types
- Types appropriately filed in hierarchy
**Coordinator Integration** (Fix Side-Effects) 🔧:
- `npm run type` fails BUT agent's work is high quality
- Good type usage, proper patterns, well-organized
- Side-effects are manageable TypeScript compilation errors
- **Coordinator Action**: Integrate the change, then fix global compilation issues
**Rollback Only** ❌:
- Agent introduced `any` types or poor type choices
- Types poorly organized or conflicting with existing patterns
- Fundamental approach issues requiring complete rework
**Integration Process**:
1. **Review**: Agent already used `git mv` to preserve history
2. **Fix Side-Effects**: Update dependent files with proper import extensions
3. **Resolve Types**: Fix any cascading type issues across codebase
4. **Validate**: Ensure `npm run type` passes after fixes
---
## 4. Common Integration Patterns
**Common Side-Effects (Expect These)**:
- **Type import conflicts**: Multiple definitions of same type name
- **Mock object typing**: Tests need complete type satisfaction
- **Stub method references**: Use stub vars instead of original methods
**Coordinator Fixes (Standard Process)**:
1. **Import Resolution**:
```bash
# Find authoritative type source
grep -r "TypeName" src/*/types.ts
# Import from domain types (src/dashboard/types.ts) not component types
```
2. **Test Mock Completion**:
```typescript
// Use Partial<T> as T pattern for minimal mocking
const mockDashboard: Partial<DashboardInfo> as DashboardInfo = {
id: 123,
json_metadata: '{}',
};
```
3. **Stub Reference Fixes**:
```typescript
// ✅ Use stub variable
expect(postStub.callCount).toBe(1);
// ❌ Don't use original method
expect(SupersetClient.post.callCount).toBe(1);
```
4. **Validation Commands**:
```bash
npm run type # TypeScript compilation
npm test -- filename # Test functionality
git status # Should show rename, not add/delete
```
---
## 5. File Categories for Planning
### Leaf Files (Start Here)
**Self-contained files with minimal JS/JSX dependencies**:
- Test files (80 files) - Usually only import the file being tested
- Utility files without internal dependencies
- Components importing only external libraries
### Heavily Imported Files (Migrate Last)
**Core files that many others depend on**:
- `utils/common.js` - Core utility functions
- `utils/reducerUtils.js` - Redux helpers
- `@superset-ui/core` equivalent files
- Major state management files (`explore/store.js`, `dashboard/actions/`)
### Complex Components (Middle Priority)
**Large files requiring careful type analysis**:
- `components/Datasource/DatasourceEditor.jsx` (1,809 lines)
- `explore/components/controls/AnnotationLayerControl/AnnotationLayer.jsx` (1,031 lines)
- `explore/components/ExploreViewContainer/index.jsx` (911 lines)
---
## 6. Success Metrics & Continuous Improvement
**Per-File Gates**:
- ✅ `npm run type` passes after each migration
- ✅ Zero `any` types introduced
- ✅ All imports properly typed
- ✅ Types filed in correct hierarchy
**Linear Scheduling**:
When agents report `DEPENDENCY_BLOCK`:
- Queue dependencies in linear order
- Process one file at a time to avoid conflicts
- Handle cascading type changes between files
**After Each Migration**:
1. **Update guides** with new patterns discovered
2. **Document coordinator fixes** that become common
3. **Enhance agent instructions** based on recurring issues
4. **Track success metrics** - automatic vs coordinator integration rates

View File

@@ -1,76 +0,0 @@
# JavaScript to TypeScript Migration Project
Progressive migration of 219 JS/JSX files to TypeScript in Apache Superset frontend.
## 📁 Project Documentation
- **[AGENT.md](./AGENT.md)** - Complete technical migration guide for agents (includes type reference, patterns, validation)
- **[COORDINATOR.md](./COORDINATOR.md)** - Strategic workflow for coordinators (file selection, task management, integration)
## 🎯 Quick Start
**For Agents:** Read [AGENT.md](./AGENT.md) for complete migration instructions
**For Coordinators:** Read [COORDINATOR.md](./COORDINATOR.md) for workflow and [AGENT.md](./AGENT.md) for supervision
**Command:** `/js-to-ts <filename>` - See [../../commands/js-to-ts.md](../../commands/js-to-ts.md)
## 📊 Migration Progress
**Scope**: 219 files total (112 JS + 107 JSX)
- Production files: 139 (63%)
- Test files: 80 (37%)
**Strategy**: Leaf-first migration with dependency-aware coordination
### Completed Migrations ✅
1. **roundDecimal** - `plugins/legacy-plugin-chart-map-box/src/utils/roundDecimal.js`
- Migrated core + test files
- Added proper TypeScript function signature with optional precision parameter
- All tests pass
2. **timeGrainSqlaAnimationOverrides** - `src/explore/controlPanels/timeGrainSqlaAnimationOverrides.js`
- Migrated to TypeScript with ControlPanelState and Dataset types
- Added TimeGrainOverrideState interface for return type
- Used type guards for safe property access
3. **DebouncedMessageQueue** - `src/utils/DebouncedMessageQueue.js`
- Migrated to TypeScript with proper generics
- Created DebouncedMessageQueueOptions interface
- **CREATED test file** with 4 comprehensive test cases
- Excellent class property typing with private/readonly modifiers
**Files Migrated**: 3/219 (1.4%)
**Tests Created**: 2 (roundDecimal had existing, DebouncedMessageQueue created)
### Next Candidates (Leaf Nodes) 🎯
**Identified leaf files with no JS/JSX dependencies:**
- `src/utils/hostNamesConfig.js` - Domain configuration utility
- `src/explore/controlPanels/Separator.js` - Control panel configuration
- `src/middleware/loggerMiddleware.js` - Logging middleware
**Migration Quality**: All completed migrations have:
- ✅ Zero `any` types
- ✅ Proper TypeScript compilation
- ✅ ESLint validation passed
- ✅ Test coverage (created where missing)
---
## 📈 Success Metrics
**Per-File Gates**:
-`npm run type` passes after each migration
- ✅ Zero `any` types introduced
- ✅ All imports properly typed
- ✅ Types filed in correct hierarchy
**Overall Progress**:
- **Automatic Integration Rate**: 100% (3/3 migrations required no coordinator fixes)
- **Test Coverage**: Improved (1 new test file created)
- **Type Safety**: Enhanced with proper interfaces and generics
---
*This is a claudette-managed progressive refactor. All documentation and coordination resources are organized under `.claude/projects/js-to-ts/`*

View File

@@ -1,15 +0,0 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "jq -r '.tool_input.command // \"\"' | grep -qE '^git commit' && cd \"$CLAUDE_PROJECT_DIR\" && echo '🔍 Running pre-commit before commit...' && pre-commit run || true"
}
]
}
]
}
}

View File

@@ -1,36 +0,0 @@
# .coveragerc to control coverage.py
[run]
branch = True
source = superset
# omit = bad_file.py
[paths]
source =
superset/
*/site-packages/
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain about missing debug-only code:
def __repr__
if self\.debug
# Don't complain if tests don't hit defensive assertion code:
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
# Ignore importlib backport
from importlib
if TYPE_CHECKING:
#fail_under = 100
show_missing = True

View File

@@ -1,125 +0,0 @@
---
description: Apache Superset development standards and guidelines for Cursor IDE
globs: ["**/*.py", "**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.sql", "**/*.md"]
alwaysApply: true
---
# Apache Superset Development Standards for Cursor IDE
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
**These migrations are actively happening - avoid deprecated patterns:**
### Frontend Modernization
- **NO `any` types** - Use proper TypeScript types
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
- **NO Enzyme** - Use React Testing Library/Jest (Enzyme fully removed)
- **Use @superset-ui/core** - Don't import Ant Design directly
### Testing Strategy Migration
- **Prefer unit tests** over integration tests
- **Prefer integration tests** over Cypress end-to-end tests
- **Cypress is last resort** - Actively moving away from Cypress
- **Use Jest + React Testing Library** for component testing
### Backend Type Safety
- **Add type hints** - All new Python code needs proper typing
- **MyPy compliance** - Run `pre-commit run mypy` to validate
- **SQLAlchemy typing** - Use proper model annotations
## Code Standards
### TypeScript Frontend
- **NO `any` types** - Use proper TypeScript
- **Functional components** with hooks
- **@superset-ui/core** for UI components (not direct antd)
- **Jest** for testing (NO Enzyme)
- **Redux** for global state, hooks for local
### Python Backend
- **Type hints required** for all new code
- **MyPy compliant** - run `pre-commit run mypy`
- **SQLAlchemy models** with proper typing
- **pytest** for testing
### Apache License Headers
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
- **LLM instruction files are excluded** - Files like LLMS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
## Key Directory Structure
```
superset/
├── superset/ # Python backend (Flask, SQLAlchemy)
│ ├── views/api/ # REST API endpoints
│ ├── models/ # Database models
│ └── connectors/ # Database connections
├── superset-frontend/src/ # React TypeScript frontend
│ ├── components/ # Reusable components
│ ├── explore/ # Chart builder
│ ├── dashboard/ # Dashboard interface
│ └── SqlLab/ # SQL editor
├── superset-frontend/packages/
│ └── superset-ui-core/ # UI component library (USE THIS)
├── tests/ # Python/integration tests
├── docs/ # Documentation (UPDATE FOR CHANGES)
└── UPDATING.md # Breaking changes log
```
## Architecture Patterns
### Dataset-Centric Approach
Charts built from enriched datasets containing:
- Dimension columns with labels/descriptions
- Predefined metrics as SQL expressions
- Self-service analytics within defined contexts
### Security & Features
- **RBAC**: Role-based access via Flask-AppBuilder
- **Feature flags**: Control feature rollouts
- **Row-level security**: SQL-based data access control
## Test Utilities
### Python Test Helpers
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
- **`@with_config`** - Config mocking decorator
- **`@with_feature_flags`** - Feature flag testing
- **`login_as()`, `login_as_admin()`** - Authentication helpers
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
### TypeScript Test Helpers
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
- **`createWrapper()`** - Redux/Router/Theme wrapper
- **`selectOption()`** - Select component helper
- **React Testing Library** - NO Enzyme (removed)
## Pre-commit Validation
**Use pre-commit hooks for quality validation:**
```bash
# Install hooks
pre-commit install
# Quick validation (faster than --all-files)
pre-commit run # Staged files only
pre-commit run mypy # Python type checking
pre-commit run prettier # Code formatting
pre-commit run eslint # Frontend linting
```
## Development Guidelines
- **Documentation**: Update docs/ for any user-facing changes
- **Breaking Changes**: Add to UPDATING.md
- **Docstrings**: Required for new functions/classes
- **Follow existing patterns**: Mimic code style, use existing libraries and utilities
- **Type Safety**: This codebase is actively modernizing toward full TypeScript and type safety
- **Always run `pre-commit run`** to validate changes before committing
---
**Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

View File

@@ -1,20 +0,0 @@
# Keep this in sync with the base image in the main Dockerfile (ARG PY_VER)
FROM python:3.11.13-trixie AS base
# Install system dependencies that Superset needs
# This layer will be cached across Codespace sessions
RUN apt-get update && apt-get install -y \
libsasl2-dev \
libldap2-dev \
libpq-dev \
tmux \
gh \
&& rm -rf /var/lib/apt/lists/*
# Install uv for fast Python package management
# This will also be cached in the image
RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \
echo 'export PATH="/root/.cargo/bin:$PATH"' >> /etc/bash.bashrc
# Set the cargo/bin directory in PATH for all users
ENV PATH="/root/.cargo/bin:${PATH}"

View File

@@ -1,5 +0,0 @@
# Superset Development with GitHub Codespaces
For complete documentation on using GitHub Codespaces with Apache Superset, please see:
**[Setting up a Development Environment - GitHub Codespaces](https://superset.apache.org/docs/contributing/development#github-codespaces-cloud-development)**

View File

@@ -1,62 +0,0 @@
# Superset Codespaces environment setup
# This file is appended to ~/.bashrc during Codespace setup
# Find the workspace directory (handles both 'superset' and 'superset-2' names)
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
if [ -n "$WORKSPACE_DIR" ]; then
# Check if virtual environment exists
if [ -d "$WORKSPACE_DIR/.venv" ]; then
# Activate the virtual environment
source "$WORKSPACE_DIR/.venv/bin/activate"
echo "✅ Python virtual environment activated"
# Verify pre-commit is installed and set up
if command -v pre-commit &> /dev/null; then
echo "✅ pre-commit is available ($(pre-commit --version))"
# Install git hooks if not already installed
if [ -d "$WORKSPACE_DIR/.git" ] && [ ! -f "$WORKSPACE_DIR/.git/hooks/pre-commit" ]; then
echo "🪝 Installing pre-commit hooks..."
cd "$WORKSPACE_DIR" && pre-commit install
fi
else
echo "⚠️ pre-commit not found. Run: pip install pre-commit"
fi
else
echo "⚠️ Python virtual environment not found at $WORKSPACE_DIR/.venv"
echo " Run: cd $WORKSPACE_DIR && .devcontainer/setup-dev.sh"
fi
# Always cd to the workspace directory for convenience
cd "$WORKSPACE_DIR"
fi
# Add helpful aliases for Superset development
alias start-superset="$WORKSPACE_DIR/.devcontainer/start-superset.sh"
alias setup-dev="$WORKSPACE_DIR/.devcontainer/setup-dev.sh"
# Show helpful message on login
echo ""
echo "🚀 Superset Codespaces Environment"
echo "=================================="
# Check if Superset is running
if docker ps 2>/dev/null | grep -q "superset"; then
echo "✅ Superset is running!"
echo " - Check the 'Ports' tab for your live Superset URL"
echo " - Initial startup takes 10-20 minutes"
echo " - Login: admin/admin"
else
echo "⚠️ Superset is not running. Use: start-superset"
# Check if there's a startup log
if [ -f "/tmp/superset-startup.log" ]; then
echo " 📋 Startup log found: cat /tmp/superset-startup.log"
fi
fi
echo ""
echo "Quick commands:"
echo " start-superset - Start Superset with Docker Compose"
echo " setup-dev - Set up Python environment (if not already done)"
echo " pre-commit run - Run pre-commit checks on staged files"
echo ""

View File

@@ -1,20 +0,0 @@
#!/bin/bash
# Script to build and push the devcontainer image to GitHub Container Registry
# This allows caching the image between Codespace sessions
# You'll need to run this with appropriate GitHub permissions
# gh auth login --scopes write:packages
REGISTRY="ghcr.io"
OWNER="apache"
REPO="superset"
TAG="devcontainer-base"
echo "Building devcontainer image..."
docker build -t $REGISTRY/$OWNER/$REPO:$TAG .devcontainer/
echo "Pushing to GitHub Container Registry..."
docker push $REGISTRY/$OWNER/$REPO:$TAG
echo "Done! Update .devcontainer/devcontainer.json to use:"
echo " \"image\": \"$REGISTRY/$OWNER/$REPO:$TAG\""

View File

@@ -1,19 +0,0 @@
{
// Extend the base configuration
"extends": "../devcontainer-base.json",
"name": "Apache Superset Development (Default)",
// Forward ports for development
"forwardPorts": [9001],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
}
},
// Auto-start Superset on Codespace resume
"postStartCommand": ".devcontainer/start-superset.sh"
}

View File

@@ -1,39 +0,0 @@
{
"name": "Apache Superset Development",
// Keep this in sync with the base image in Dockerfile (ARG PY_VER)
// Using the same base as Dockerfile, but non-slim for dev tools
"image": "python:3.11.13-bookworm",
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {
"moby": true,
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
"ghcr.io/devcontainers/features/sshd:1": {
"version": "latest"
}
},
// Run commands after container is created
"postCreateCommand": "chmod +x .devcontainer/setup-dev.sh && .devcontainer/setup-dev.sh",
// VS Code customizations
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}
}
}

View File

@@ -1,66 +0,0 @@
{
"name": "Apache Superset Development",
// Option 1: Use pre-built image directly
// "image": "ghcr.io/apache/superset:devcontainer-base",
// Option 2: Build from Dockerfile with cache (current approach)
"build": {
"dockerfile": "Dockerfile",
"context": ".",
// Cache from the Apache registry image
"cacheFrom": ["ghcr.io/apache/superset:devcontainer-base"]
},
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {
"moby": false,
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/common-utils:2": {
"configureZshAsDefaultShell": true
},
"ghcr.io/devcontainers/features/sshd:1": {
"version": "latest"
}
},
// Forward ports for development
"forwardPorts": [9001],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
}
},
// Run commands after container is created
"postCreateCommand": "bash .devcontainer/setup-dev.sh || echo '⚠️ Setup had issues - run .devcontainer/setup-dev.sh manually'",
// Auto-start Superset after ensuring Docker is ready
// Run in foreground to see any errors, but don't block on failures
"postStartCommand": "bash -c 'echo \"Waiting 30s for services to initialize...\"; sleep 30; .devcontainer/start-superset.sh || echo \"⚠️ Auto-start failed - run start-superset manually\"'",
// Set environment variables
"remoteEnv": {
// Removed automatic venv activation to prevent startup issues
// The setup script will handle this
},
// VS Code customizations
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode"
]
}
}
}

View File

@@ -1,32 +0,0 @@
#!/bin/bash
# Setup script for Superset Codespaces development environment
echo "🔧 Setting up Superset development environment..."
# The universal image has most tools, just need Superset-specific libs
echo "📦 Installing Superset-specific dependencies..."
sudo apt-get update
sudo apt-get install -y \
libsasl2-dev \
libldap2-dev \
libpq-dev \
tmux \
gh
# Install uv for fast Python package management
echo "📦 Installing uv..."
curl -LsSf https://astral.sh/uv/install.sh | sh
# Add cargo/bin to PATH for uv
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.zshrc
# Install Claude Code CLI via npm
echo "🤖 Installing Claude Code..."
npm install -g @anthropic-ai/claude-code
# Make the start script executable
chmod +x .devcontainer/start-superset.sh
echo "✅ Development environment setup complete!"
echo "🚀 Run '.devcontainer/start-superset.sh' to start Superset"

View File

@@ -1,69 +0,0 @@
#!/bin/bash
# Startup script for Superset in Codespaces
echo "🚀 Starting Superset in Codespaces..."
echo "🌐 Frontend will be available at port 9001"
# Check if MCP is enabled
if [ "$ENABLE_MCP" = "true" ]; then
echo "🤖 MCP Service will be available at port 5008"
fi
# Find the workspace directory (Codespaces clones as 'superset', not 'superset-2')
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
if [ -n "$WORKSPACE_DIR" ]; then
cd "$WORKSPACE_DIR"
echo "📁 Working in: $WORKSPACE_DIR"
else
echo "📁 Using current directory: $(pwd)"
fi
# Check if docker is running
if ! docker info > /dev/null 2>&1; then
echo "⏳ Waiting for Docker to start..."
sleep 5
fi
# Clean up any existing containers
echo "🧹 Cleaning up existing containers..."
docker-compose -f docker-compose-light.yml --profile mcp down
# Start services
echo "🏗️ Building and starting services..."
echo ""
echo "📝 Once started, login with:"
echo " Username: admin"
echo " Password: admin"
echo ""
echo "📋 Running in foreground with live logs (Ctrl+C to stop)..."
# Run docker-compose and capture exit code
if [ "$ENABLE_MCP" = "true" ]; then
echo "🤖 Starting with MCP Service enabled..."
docker-compose -f docker-compose-light.yml --profile mcp up
else
docker-compose -f docker-compose-light.yml up
fi
EXIT_CODE=$?
# If it failed, provide helpful instructions
if [ $EXIT_CODE -ne 0 ] && [ $EXIT_CODE -ne 130 ]; then # 130 is Ctrl+C
echo ""
echo "❌ Superset startup failed (exit code: $EXIT_CODE)"
echo ""
echo "🔄 To restart Superset, run:"
echo " .devcontainer/start-superset.sh"
echo ""
echo "🔧 For troubleshooting:"
echo " # View logs:"
echo " docker-compose -f docker-compose-light.yml logs"
echo ""
echo " # Clean restart (removes volumes):"
echo " docker-compose -f docker-compose-light.yml down -v"
echo " .devcontainer/start-superset.sh"
echo ""
echo " # Common issues:"
echo " - Network timeouts: Just retry, often transient"
echo " - Port conflicts: Check 'docker ps'"
echo " - Database issues: Try clean restart with -v"
fi

View File

@@ -1,29 +0,0 @@
{
// Extend the base configuration
"extends": "../devcontainer-base.json",
"name": "Apache Superset Development with MCP",
// Forward ports for development
"forwardPorts": [9001, 5008],
"portsAttributes": {
"9001": {
"label": "Superset (via Webpack Dev Server)",
"onAutoForward": "notify",
"visibility": "public"
},
"5008": {
"label": "MCP Service (Model Context Protocol)",
"onAutoForward": "notify",
"visibility": "private"
}
},
// Auto-start Superset with MCP on Codespace resume
"postStartCommand": "ENABLE_MCP=true .devcontainer/start-superset.sh",
// Environment variables
"containerEnv": {
"ENABLE_MCP": "true"
}
}

View File

@@ -34,6 +34,7 @@
**/*.sqllite
**/*.swp
**/.terser-plugin-cache/
**/.storybook/
**/node_modules/
tests/
@@ -41,8 +42,6 @@ docs/
install/
superset-frontend/cypress-base/
superset-frontend/coverage/
superset-frontend/.temp_cache/
superset/static/assets/
superset-websocket/dist/
venv
.venv

View File

@@ -1,41 +0,0 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Auto-configure Docker Compose for multi-instance support
# Requires direnv: https://direnv.net/
#
# Install: brew install direnv (or apt install direnv)
# Setup: Add 'eval "$(direnv hook bash)"' to ~/.bashrc (or ~/.zshrc)
# Allow: Run 'direnv allow' in this directory once
# Generate unique project name from directory
export COMPOSE_PROJECT_NAME=$(basename "$PWD" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g')
# Find available ports sequentially to avoid collisions
_is_free() { ! lsof -i ":$1" &>/dev/null 2>&1; }
_p=80; while ! _is_free $_p; do ((_p++)); done; export NGINX_PORT=$_p
_p=8088; while ! _is_free $_p; do ((_p++)); done; export SUPERSET_PORT=$_p
_p=9000; while ! _is_free $_p; do ((_p++)); done; export NODE_PORT=$_p
_p=8080; while ! _is_free $_p || [ $_p -eq $NGINX_PORT ]; do ((_p++)); done; export WEBSOCKET_PORT=$_p
_p=8081; while ! _is_free $_p || [ $_p -eq $WEBSOCKET_PORT ]; do ((_p++)); done; export CYPRESS_PORT=$_p
_p=5432; while ! _is_free $_p; do ((_p++)); done; export DATABASE_PORT=$_p
_p=6379; while ! _is_free $_p; do ((_p++)); done; export REDIS_PORT=$_p
unset _p _is_free
echo "🐳 Superset configured: http://localhost:$SUPERSET_PORT (dev: localhost:$NODE_PORT)"

3
.gitattributes vendored
View File

@@ -1,4 +1 @@
docker/**/*.sh text eol=lf
*.svg binary
*.ipynb binary
*.geojson binary

36
.github/CODEOWNERS vendored
View File

@@ -2,7 +2,7 @@
# https://github.com/apache/superset/issues/13351
/superset/migrations/ @mistercrunch @michael-s-molina @betodealmeida @eschutho @sadpandajoe
/superset/migrations/ @apache/superset-committers
# Notify some committers of changes in the components
@@ -12,41 +12,23 @@
# Notify Helm Chart maintainers about changes in it
/helm/superset/ @craig-rueda @dpgaspar @villebro @nytai @michael-s-molina @mistercrunch @rusackas @Antonio-RiveroMartnez
/helm/superset/ @craig-rueda @dpgaspar @villebro
# Notify E2E test maintainers of changes
/superset-frontend/cypress-base/ @sadpandajoe @geido @eschutho @rusackas @betodealmeida @mistercrunch
/superset-frontend/cypress-base/ @jinghua-qa @geido @eschutho @rusackas @betodealmeida
# Notify PMC members of changes to GitHub Actions
/.github/ @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @kgabryje @dpgaspar @sadpandajoe @hainenber
/.github/ @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @john-bodley @kgabryje @dpgaspar
# Notify PMC members of changes to CI-executed scripts (supply-chain risk:
# scripts/ files run directly in CI workflows and can execute arbitrary code)
# Notify PMC members of changes to required Github Actions
/scripts/ @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @kgabryje @dpgaspar @sadpandajoe @hainenber
# Notify PMC members of changes to required GitHub Actions
/.asf.yaml @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @kgabryje @dpgaspar @Antonio-RiveroMartnez
# Maps are a finicky contribution process we care about
/.asf.yaml @villebro @geido @eschutho @rusackas @betodealmeida @nytai @mistercrunch @craig-rueda @john-bodley @kgabryje @dpgaspar
# Maps are a finnicky contribution process we care about
**/*.geojson @villebro @rusackas
/superset-frontend/plugins/legacy-plugin-chart-country-map/ @villebro @rusackas
# Notify translation maintainers of changes to translations
/superset/translations/ @sfirke
# Notify PMC members of changes to extension-related files
/docs/developer_portal/extensions/ @michael-s-molina @villebro @rusackas
/superset-core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-extensions-cli/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset/core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset/extensions/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/packages/superset-core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/core/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
/superset-frontend/src/extensions/ @michael-s-molina @villebro @geido @eschutho @rusackas @kgabryje
# Translations are a finnicky contribution that we care about
/superset/translations/ @villebro @rusackas @michael-s-molina

View File

@@ -7,7 +7,6 @@ body:
value: |
Hello Superset Community member! Please keep things tidy by putting your post in the proper place:
🚨 Reporting a security issue: send an email to security@superset.apache.org. DO NOT USE GITHUB ISSUES TO REPORT SECURITY PROBLEMS.
🐛 Reporting a bug: use this form.
🙏 Asking a question or getting help: post in the [Superset Slack chat](http://bit.ly/join-superset-slack) or [GitHub Discussions](https://github.com/apache/superset/discussions) under "Q&A / Help".
💡 Requesting a new feature: Search [GitHub Discussions](https://github.com/apache/superset/discussions) to see if it exists already. If not, add a new post there under "Ideas".
@@ -15,9 +14,14 @@ body:
id: bug-description
attributes:
label: Bug description
description: A clear description of what the bug is, including reproduction steps and expected behavior.
description: A clear and concise description of what the bug is.
validations:
required: true
- type: textarea
id: repro-steps
attributes:
label: How to reproduce the bug
placeholder: |
The bug is that...
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
@@ -41,8 +45,8 @@ body:
label: Superset version
options:
- master / latest-dev
- "6.0.0"
- "5.0.0"
- "3.1.0"
- "3.0.4"
validations:
required: true
- type: dropdown

37
.github/SECURITY.md vendored
View File

@@ -18,32 +18,10 @@ e-mail address [security@superset.apache.org](mailto:security@superset.apache.or
More details can be found on the ASF website at
[ASF vulnerability reporting process](https://apache.org/security/#reporting-a-vulnerability)
**Submission Standards & AI Policy**
To ensure engineering focus remains on verified risks and to manage high reporting volumes, all reports must meet the following criteria:
- Plain Text Format: In accordance with Apache guidelines, please provide all details in plain text within the email body. Avoid sending PDFs, Word documents, or password-protected archives.
- Mandatory AI Disclosure: If you utilized Large Language Models (LLMs) or AI tools to identify a flaw or assist in writing a report, you must disclose this in your submission so our triage team can contextualize the findings.
- Human-Verified PoC: All submissions must include a manual, step-by-step Proof of Concept (PoC) performed on a supported release. Raw AI outputs, hypothetical chat transcripts, or unverified scanner logs will be closed as Invalid.
We kindly ask you to include the following information in your report to assist our developers in triaging and remediating issues efficiently:
- Version/Commit: The specific version of Apache Superset or the Git commit hash you are using.
- Configuration: A sanitized copy of your `superset_config.py` file or any config overrides.
- Environment: Your deployment method (e.g., Docker Compose, Helm, or source) and relevant OS/Browser details.
- Impacted Component: Identification of the affected area (e.g., Python backend, React frontend, or a specific database connector).
- Expected vs. Actual Behavior: A clear description of the intended system behavior versus the observed vulnerability.
- Detailed Reproduction Steps: Clear, manual steps to reproduce the vulnerability.
**Out of Scope Vulnerabilities**
To prioritize engineering efforts on genuine architectural risks, the following scenarios are explicitly out of scope and will not be issued a CVE:
- Attacks requiring Admin privileges: (e.g., CSS injection, template manipulation, dashboard ownership overrides, or modifying global system settings). Per the CVE vulnerability definition in CNA Operational Rules 4.1, a qualifying vulnerability must allow violation of a security policy. The Admin role is a fully trusted operational boundary defined by Apache Superset's security policy; actions within this boundary do not violate that policy and are therefore considered intended capabilities 'by design,' not vulnerabilities.
- Brute Force and Rate Limiting: Reports targeting a lack of resource exhaustion protections, generic rate-limiting, or volumetric Denial of Service (DoS) attempts.
- Theoretical attack vectors: Issues without a demonstrable, reproducible exploit path.
- Non-Exploitable Findings: Missing security headers, generic banner disclosures, or descriptive error messages that do not lead to a direct, documented exploit.
**Outcome of Reports**
Reports that are deemed out-of-scope for a CVE but represent valid security best practices or hardening opportunities may be converted into public GitHub issues. This allows the community to contribute to the general hardening of the platform even when a specific vulnerability threshold is not met.
We kindly ask you to include the following information in your report:
- Apache Superset version that you are using
- A sanitized copy of your `superset_config.py` file or any config overrides
- Detailed steps to reproduce the vulnerability
Note that Apache Superset is not responsible for any third-party dependencies that may
have security issues. Any vulnerabilities found in third-party dependencies should be
@@ -51,13 +29,6 @@ reported to the maintainers of those projects. Results from security scans of Ap
Superset dependencies found on its official Docker image can be remediated at release time
by extending the image itself.
**Vulnerability Aggregation & CVE Attribution**
In accordance with MITRE CNA Operational Rules (4.1.10, 4.1.11, and 4.2.13), Apache Superset issues CVEs based on the underlying architectural root cause rather than the number of affected endpoints or exploit payloads.
- Aggregation: If multiple exploit vectors stem from the same programmatic failure or shared vulnerable code, they must be aggregated into a single, comprehensive report.
- Independent Fixes: Separate CVEs will only be assigned if the vulnerabilities reside in decoupled architectural modules and can be fixed independently of one another.
Reports that fail to aggregate related findings will be merged during triage to ensure an accurate and defensible CVE record.
**Your responsible disclosure and collaboration are invaluable.**
## Extra Information

View File

@@ -1,34 +0,0 @@
name: Change Detector
description: Detects file changes for pull request and push events
inputs:
token:
description: GitHub token for authentication
required: true
outputs:
python:
description: Whether Python-related files were changed
value: ${{ steps.change-detector.outputs.python }}
frontend:
description: Whether frontend-related files were changed
value: ${{ steps.change-detector.outputs.frontend }}
docker:
description: Whether docker-related files were changed
value: ${{ steps.change-detector.outputs.docker }}
docs:
description: Whether docs-related files were changed
value: ${{ steps.change-detector.outputs.docs }}
superset-extensions-cli:
description: Whether superset-extensions-cli package-related files were changed
value: ${{ steps.change-detector.outputs.superset-extensions-cli }}
runs:
using: composite
steps:
- name: Detect file changes
id: change-detector
run: |
python --version
python scripts/change_detector.py
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.token }}
GITHUB_OUTPUT: ${{ github.output }}

View File

@@ -1,23 +0,0 @@
name: Label Draft PRs
on:
pull_request:
types:
- opened
- converted_to_draft
jobs:
label-draft:
runs-on: ubuntu-latest
steps:
- name: Check if the PR is a draft
id: check-draft
uses: actions/github-script@v8
with:
script: |
const isDraft = context.payload.pull_request.draft;
core.setOutput('isDraft', isDraft);
- name: Add `review:draft` Label
if: steps.check-draft.outputs.isDraft == 'true'
uses: actions-ecosystem/action-add-labels@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
labels: "review:draft"

View File

@@ -1,58 +0,0 @@
name: 'Setup Python Environment'
description: 'Set up Python and install dependencies with optional configurations.'
inputs:
python-version:
description: 'Python version to set up. Accepts a version number, "current", or "next".'
required: true
default: 'current'
cache:
description: 'Cache dependencies. Options: pip'
required: false
default: 'pip'
requirements-type:
description: 'Type of requirements to install. Options: base, development, default'
required: false
default: 'dev'
install-superset:
description: 'Whether to install Superset itself. If false, only python is installed'
required: false
default: 'true'
runs:
using: 'composite'
steps:
- name: Interpret Python Version
id: set-python-version
shell: bash
run: |
if [ "${{ inputs.python-version }}" = "current" ]; then
echo "PYTHON_VERSION=3.11" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "next" ]; then
# currently disabled in GHA matrixes because of library compatibility issues
echo "PYTHON_VERSION=3.12" >> $GITHUB_ENV
elif [ "${{ inputs.python-version }}" = "previous" ]; then
echo "PYTHON_VERSION=3.10" >> $GITHUB_ENV
else
echo "PYTHON_VERSION=${{ inputs.python-version }}" >> $GITHUB_ENV
fi
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: ${{ inputs.cache }}
- name: Install dependencies
run: |
if [ "${{ inputs.install-superset }}" = "true" ]; then
sudo apt-get update && sudo apt-get -y install libldap2-dev libsasl2-dev
pip install --upgrade pip setuptools wheel uv
if [ "${{ inputs.requirements-type }}" = "dev" ]; then
uv pip install --system -r requirements/development.txt
elif [ "${{ inputs.requirements-type }}" = "base" ]; then
uv pip install --system -r requirements/base.txt
fi
uv pip install --system -e .
fi
shell: bash

View File

@@ -1,69 +0,0 @@
name: "Setup Docker Environment"
description: "Reusable steps for setting up QEMU, Docker Buildx, DockerHub login, Supersetbot, and optionally Docker Compose"
inputs:
build:
description: "Used for building?"
required: false
default: "false"
dockerhub-user:
description: "DockerHub username"
required: false
dockerhub-token:
description: "DockerHub token"
required: false
install-docker-compose:
description: "Flag to install Docker Compose"
required: false
default: "true"
login-to-dockerhub:
description: "Whether you want to log into dockerhub"
required: false
default: "true"
outputs: {}
runs:
using: "composite"
steps:
- name: Set up QEMU
if: ${{ inputs.build == 'true' }}
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
- name: Set up Docker Buildx
if: ${{ inputs.build == 'true' }}
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Try to login to DockerHub
if: ${{ inputs.login-to-dockerhub == 'true' }}
continue-on-error: true
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
username: ${{ inputs.dockerhub-user }}
password: ${{ inputs.dockerhub-token }}
- name: Install Docker Compose
if: ${{ inputs.install-docker-compose == 'true' }}
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
# Download and save the Docker GPG key in the correct format
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
# Ensure the key file is readable
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the Docker repository using the correct key
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Update package lists and install Docker Compose plugin
sudo apt update
sudo apt install -y docker-compose-plugin
- name: Docker Version Info
shell: bash
run: docker info

View File

@@ -1,40 +0,0 @@
name: 'Setup supersetbot'
description: 'Sets up supersetbot npm lib from the repo or npm'
inputs:
from-npm:
description: 'Install from npm instead of local setup'
required: false
default: 'true' # Defaults to using the local setup
runs:
using: 'composite'
steps:
- name: Setup Node Env
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install supersetbot from npm
if: ${{ inputs.from-npm == 'true' }}
shell: bash
run: npm install -g supersetbot
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
if: ${{ inputs.from-npm == 'false' }}
uses: actions/checkout@v4
with:
repository: apache-superset/supersetbot
path: supersetbot
- name: Setup supersetbot from repo
if: ${{ inputs.from-npm == 'false' }}
shell: bash
working-directory: supersetbot
run: |
# simple trick to install globally with dependencies
npm pack
npm install -g ./supersetbot*.tgz
- name: echo supersetbot version
shell: bash
run: supersetbot version

View File

@@ -1 +0,0 @@
../AGENTS.md

203
.github/dependabot.yml vendored
View File

@@ -1,107 +1,51 @@
version: 2
enable-beta-ecosystems: true
updates:
- package-ecosystem: "github-actions"
directory: "/"
ignore:
# Ignore temporarily as release schedule is too mentally taxing for dep-handling maintainers
# Additionally, very few PRs are reviewed by this action.
- dependency-name: anthropics/claude-code-action
schedule:
interval: "daily"
interval: "monthly"
- package-ecosystem: "npm"
ignore:
# TODO: remove below entries until React >= 18.0.0
- dependency-name: "storybook"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- dependency-name: "@storybook*"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- dependency-name: "eslint-plugin-storybook"
- dependency-name: "react-error-boundary"
- dependency-name: "@rjsf/*"
# remark-gfm v4+ requires react-markdown v9+, which needs React 18
- dependency-name: "remark-gfm"
- dependency-name: "react-markdown"
# TODO: remove below entries until React >= 19.0.0
- dependency-name: "react-icons"
# JSDOM v30 doesn't play well with Jest v30
# Source: https://jestjs.io/blog#known-issues
# GH thread: https://github.com/jsdom/jsdom/issues/3492
- dependency-name: "jest-environment-jsdom"
# `@swc/plugin-transform-imports` doesn't work with current Webpack-SWC hybrid setup
# See https://github.com/apache/superset/pull/37384#issuecomment-3793991389
# TODO: remove the plugin once Lodash usage has been migrated to a more readily tree-shakeable alternative
- dependency-name: "@swc/plugin-transform-imports"
# `just-handlerbars-helpers` library in plugin-chart-handlebars requires `currencyformatter`` to be < 2
- dependency-name: "currencyformatter.js"
update-types: ["version-update:semver-major"]
# TODO: remove below clause once https://github.com/pmmmwh/react-refresh-webpack-plugin/pull/940 lands onto a future release
# and confirm the issue https://github.com/apache/superset/issues/39600 is fixed
- dependency-name: "react-checkbox-tree"
update-types: ["version-update:semver-major"]
groups:
storybook:
applies-to: version-updates
patterns:
- "@storybook*"
- "storybook"
update-types:
- "patch"
# not until node >= 18.12.0
- dependency-name: "css-minimizer-webpack-plugin"
directory: "/superset-frontend/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 30
versioning-strategy: increase
- package-ecosystem: "pip"
directory: "/"
open-pull-requests-limit: 10
directory: "/requirements/"
schedule:
interval: "weekly"
interval: "monthly"
labels:
- pip
- dependabot
open-pull-requests-limit: 30
- package-ecosystem: "npm"
directory: ".github/actions"
schedule:
interval: "daily"
interval: "monthly"
open-pull-requests-limit: 10
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/docs/"
ignore:
# TODO: remove below entries until React >= 18.0.0 in superset-frontend
- dependency-name: "storybook"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- dependency-name: "@storybook*"
update-types: ["version-update:semver-major", "version-update:semver-minor"]
- dependency-name: "eslint-plugin-storybook"
- dependency-name: "react-error-boundary"
groups:
storybook:
applies-to: version-updates
patterns:
- "@storybook*"
- "storybook"
update-types:
- "patch"
schedule:
interval: "daily"
interval: "monthly"
open-pull-requests-limit: 10
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-websocket/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -110,7 +54,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-websocket/utils/client-ws-app/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -122,7 +66,17 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-calendar/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-histogram/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -132,7 +86,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-partition/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -142,7 +96,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-world-map/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -151,11 +105,8 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-pivot-table/"
ignore:
# TODO: remove below entries until React >= 19.0.0
- dependency-name: "react-icons"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -165,7 +116,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-chord/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -175,7 +126,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-horizon/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -185,7 +136,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-rose/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -195,7 +146,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-deckgl/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -204,11 +155,8 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-table/"
ignore:
# TODO: remove below entries until React >= 19.0.0
- dependency-name: "react-icons"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -218,7 +166,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-country-map/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -228,7 +176,17 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-map-box/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sankey/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -238,7 +196,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-preset-chart-nvd3/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -248,7 +206,17 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-word-cloud/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-event-flow/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -258,7 +226,17 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-paired-t-test/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sankey-loop/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -268,7 +246,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-echarts/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -276,9 +254,9 @@ updates:
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-ag-grid-table/"
directory: "/superset-frontend/plugins/preset-chart-xy/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -286,9 +264,9 @@ updates:
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-cartodiagram/"
directory: "/superset-frontend/plugins/legacy-plugin-chart-heatmap/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -298,7 +276,17 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-parallel-coordinates/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/legacy-plugin-chart-sunburst/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -307,12 +295,8 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/plugins/plugin-chart-handlebars/"
ignore:
# `just-handlerbars-helpers` library in plugin-chart-handlebars requires `currencyformatter`` to be < 2
- dependency-name: "currencyformatter.js"
update-types: ["version-update:semver-major"]
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -322,7 +306,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/generator-superset/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -332,7 +316,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-chart-controls/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
@@ -341,13 +325,18 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-core/"
ignore:
# not until React >= 18.0.0
- dependency-name: "react-markdown"
- dependency-name: "remark-gfm"
- dependency-name: "react-error-boundary"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot
open-pull-requests-limit: 5
versioning-strategy: increase
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-demo/"
schedule:
interval: "monthly"
labels:
- npm
- dependabot
@@ -357,7 +346,7 @@ updates:
- package-ecosystem: "npm"
directory: "/superset-frontend/packages/superset-ui-switchboard/"
schedule:
interval: "daily"
interval: "monthly"
labels:
- npm
- dependabot

25
.github/labeler.yml vendored
View File

@@ -17,11 +17,6 @@
- any-glob-to-any-file:
- 'superset/migrations/**'
"risk:ci-script":
- changed-files:
- any-glob-to-any-file:
- 'scripts/**'
############################################
# Dependencies
############################################
@@ -77,16 +72,6 @@
- any-glob-to-any-file:
- 'superset/translations/zh/**'
"i18n:czech":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/cs/**'
"i18n:traditional-chinese":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/zh_TW/**'
"i18n:dutch":
- changed-files:
- any-glob-to-any-file:
@@ -127,11 +112,6 @@
- any-glob-to-any-file:
- 'superset/translations/sk/**'
"i18n:latvian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/lv/**'
"i18n:ukrainian":
- changed-files:
- any-glob-to-any-file:
@@ -142,11 +122,6 @@
- any-glob-to-any-file:
- 'superset/translations/es/**'
"i18n:persian":
- changed-files:
- any-glob-to-any-file:
- 'superset/translations/fa/**'
############################################
# Sub-projects and monorepo packages
############################################

View File

@@ -31,6 +31,20 @@ say() {
fi
}
# default command to run when the `run` input is empty
default-setup-command() {
apt-get-install
pip-upgrade
}
apt-get-install() {
say "::group::apt-get install dependencies"
sudo apt-get update && sudo apt-get install --yes \
libsasl2-dev \
libldap2-dev
say "::endgroup::"
}
pip-upgrade() {
say "::group::Upgrade pip"
pip install --upgrade pip
@@ -89,8 +103,6 @@ EOF
setup-mysql() {
say "::group::Initialize database"
mysql -h 127.0.0.1 -P 13306 -u root --password=root <<-EOF
SET GLOBAL transaction_isolation='READ-COMMITTED';
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
DROP DATABASE IF EXISTS superset;
CREATE DATABASE superset DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;
DROP DATABASE IF EXISTS sqllab_test_db;
@@ -117,44 +129,9 @@ testdata() {
say "::endgroup::"
}
playwright_testdata() {
cd "$GITHUB_WORKSPACE"
say "::group::Load all examples for Playwright tests"
# must specify PYTHONPATH to make `tests.superset_test_config` importable
export PYTHONPATH="$GITHUB_WORKSPACE"
pip install -e .
superset db upgrade
superset load_test_users
superset load_examples
superset init
# Enable DML on the examples database so Playwright tests can create/drop
# temporary tables via SQL Lab without depending on external data sources.
superset shell <<'PYEOF'
import sys
from superset.extensions import db
from superset.models.core import Database
examples_db = db.session.query(Database).filter_by(database_name='examples').first()
if not examples_db:
sys.exit('ERROR: examples database not found. load_examples may have failed.')
examples_db.allow_dml = True
db.session.commit()
print('Enabled allow_dml on examples database')
PYEOF
say "::endgroup::"
}
celery-worker() {
cd "$GITHUB_WORKSPACE"
say "::group::Start Celery worker"
# must specify PYTHONPATH to make `tests.superset_test_config` importable
export PYTHONPATH="$GITHUB_WORKSPACE"
celery \
--app=superset.tasks.celery_app:app \
worker \
--concurrency=2 \
--detach \
--optimization=fair
codecov() {
say "::group::Upload code coverage"
bash ".github/workflows/codecov.sh" "$@"
say "::endgroup::"
}
@@ -170,132 +147,73 @@ cypress-install() {
cache-save cypress
}
cypress-run-all() {
local USE_DASHBOARD=$1
local APP_ROOT=$2
# Run Cypress and upload coverage reports
cypress-run() {
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"
local page=$1
local group=${2:-Default}
local cypress="./node_modules/.bin/cypress run"
local browser=${CYPRESS_BROWSER:-chrome}
export TERM="xterm"
export ELECTRON_DISABLE_GPU=true # Attempt to disable GPU for Electron-based Cypress
say "::group::Run Cypress for [$page]"
if [[ -z $CYPRESS_KEY ]]; then
xvfb-run --auto-servernum --server-args='-screen 0, 1024x768x24' $cypress --spec "cypress/e2e/$page" --browser "$browser"
else
export CYPRESS_RECORD_KEY=$(echo $CYPRESS_KEY | base64 --decode)
# additional flags for Cypress dashboard recording
xvfb-run --auto-servernum --server-args='-screen 0, 1024x768x24' $cypress --spec "cypress/e2e/$page" --browser "$browser" \
--record --group "$group" --tag "${GITHUB_REPOSITORY},${GITHUB_EVENT_NAME}" \
--parallel --ci-build-id "${GITHUB_SHA:0:8}-${NONCE}"
fi
# don't add quotes to $record because we do want word splitting
say "::endgroup::"
}
cypress-run-all() {
# Start Flask and run it in background
# --no-debugger means disable the interactive debugger on the 500 page
# so errors can print to stderr.
local flasklog="${HOME}/flask.log"
local port=8081
CYPRESS_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
export SUPERSET_APP_ROOT=$APP_ROOT
CYPRESS_BASE_URL=${CYPRESS_BASE_URL}${APP_ROOT}
fi
export CYPRESS_BASE_URL
export CYPRESS_BASE_URL="http://localhost:${port}"
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
USE_DASHBOARD_FLAG=''
if [ "$USE_DASHBOARD" = "true" ]; then
USE_DASHBOARD_FLAG='--use-dashboard'
fi
# UNCOMMENT the next few commands to monitor memory usage
# monitor_memory & # Start memory monitoring in the background
# memoryMonitorPid=$!
python ../../scripts/cypress_run.py --parallelism $PARALLELISM --parallelism-id $PARALLEL_ID --group $PARALLEL_ID --retries 5 $USE_DASHBOARD_FLAG
# kill $memoryMonitorPid
cypress-run "*/**/*"
# After job is done, print out Flask log for debugging
echo "::group::Flask log for default run"
say "::group::Flask log for default run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
}
playwright-install() {
cd "$GITHUB_WORKSPACE/superset-frontend"
say "::group::Install Playwright browsers"
npx playwright install --with-deps chromium
# Create output directories for test results and debugging
mkdir -p playwright-results
mkdir -p test-results
say "::endgroup::"
}
playwright-run() {
local APP_ROOT=$1
local TEST_PATH=$2
# Start Flask from the project root (same as Cypress)
cd "$GITHUB_WORKSPACE"
local flasklog="${HOME}/flask-playwright.log"
local port=8081
PLAYWRIGHT_BASE_URL="http://localhost:${port}"
if [ -n "$APP_ROOT" ]; then
export SUPERSET_APP_ROOT=$APP_ROOT
PLAYWRIGHT_BASE_URL=${PLAYWRIGHT_BASE_URL}${APP_ROOT}/
fi
export PLAYWRIGHT_BASE_URL
# Rerun SQL Lab tests with backend persist disabled
export SUPERSET_CONFIG=tests.integration_tests.superset_test_config_sqllab_backend_persist_off
# Restart Flask with new configs
kill $flaskProcessId
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
# Ensure cleanup on exit
trap "kill $flaskProcessId 2>/dev/null || true" EXIT
cypress-run "sqllab/*" "Backend persist"
# Wait for server to be ready with health check
local timeout=60
say "Waiting for Flask server to start on port $port..."
while [ $timeout -gt 0 ]; do
if curl -f ${PLAYWRIGHT_BASE_URL}/health >/dev/null 2>&1; then
say "Flask server is ready"
break
fi
sleep 1
timeout=$((timeout - 1))
done
# Upload code coverage separately so each page can have separate flags
# -c will clean existing coverage reports, -F means add flags
# || true to prevent CI failure on codecov upload
codecov -c -F "cypress" || true
if [ $timeout -eq 0 ]; then
echo "::error::Flask server failed to start within 60 seconds"
echo "::group::Flask startup log"
cat "$flasklog"
echo "::endgroup::"
return 1
fi
# Change to frontend directory for Playwright execution
cd "$GITHUB_WORKSPACE/superset-frontend"
say "::group::Run Playwright tests"
echo "Running Playwright with baseURL: ${PLAYWRIGHT_BASE_URL}"
if [ -n "$TEST_PATH" ]; then
# Check if there are any test files in the specified path
if ! find "playwright/tests/${TEST_PATH}" -name "*.spec.ts" -type f 2>/dev/null | grep -q .; then
echo "No test files found in ${TEST_PATH} - skipping test run"
say "::endgroup::"
kill $flaskProcessId
return 0
fi
echo "Running tests: ${TEST_PATH}"
# Set INCLUDE_EXPERIMENTAL=true to allow experimental tests to run
export INCLUDE_EXPERIMENTAL=true
npx playwright test "${TEST_PATH}" --output=playwright-results
local status=$?
# Unset to prevent leaking into subsequent commands
unset INCLUDE_EXPERIMENTAL
else
echo "Running all required tests (experimental/ excluded via playwright.config.ts)"
npx playwright test --output=playwright-results
local status=$?
fi
say "::group::Flask log for backend persist"
cat "$flasklog"
say "::endgroup::"
# After job is done, print out Flask log for debugging
echo "::group::Flask log for Playwright run"
cat "$flasklog"
echo "::endgroup::"
# make sure the program exits
kill $flaskProcessId
return $status
}
eyes-storybook-dependencies() {
@@ -304,17 +222,27 @@ eyes-storybook-dependencies() {
say "::endgroup::"
}
monitor_memory() {
# This is a small utility to monitor memory usage. Useful for debugging memory in GHA.
# To use wrap your command as follows
#
# monitor_memory & # Start memory monitoring in the background
# memoryMonitorPid=$!
# YOUR_COMMAND_HERE
# kill $memoryMonitorPid
while true; do
echo "$(date) - Top 5 memory-consuming processes:"
ps -eo pid,comm,%mem --sort=-%mem | head -n 6 # First line is the header, next 5 are top processes
sleep 2
done
cypress-run-applitools() {
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"
local flasklog="${HOME}/flask.log"
local port=8081
local cypress="./node_modules/.bin/cypress run"
local browser=${CYPRESS_BROWSER:-chrome}
export CYPRESS_BASE_URL="http://localhost:${port}"
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!
$cypress --spec "cypress/e2e/*/**/*.applitools.test.ts" --browser "$browser" --headless --config ignoreTestFiles="[]"
codecov -c -F "cypress" || true
say "::group::Flask log for default run"
cat "$flasklog"
say "::endgroup::"
# make sure the program exits
kill $flaskProcessId
}

View File

@@ -1,81 +0,0 @@
name: Bump Python Package
on:
# Can be triggered manually
workflow_dispatch:
inputs:
package:
required: false
description: The python package to bump (all if empty)
group:
required: false
description: The optional dependency group to bump (as defined in pyproject.toml)
limit:
required: true
description: Max number of PRs to open (0 for no limit)
default: 5
extra-flags:
required: false
default: --only-base
description: Additional flags to pass to the bump-python command
#schedule:
# - cron: '0 0 * * *' # Runs daily at midnight UTC
jobs:
bump-python-package:
runs-on: ubuntu-24.04
permissions:
actions: write
contents: write
pull-requests: write
checks: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: true
ref: master
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.10"
- name: Install uv
run: pip install uv
- name: supersetbot bump-python -p "${{ github.event.inputs.package }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INPUT_PACKAGE: ${{ github.event.inputs.package }}
INPUT_GROUP: ${{ github.event.inputs.group }}
INPUT_EXTRA_FLAGS: ${{ github.event.inputs.extra-flags }}
INPUT_LIMIT: ${{ github.event.inputs.limit }}
run: |
git config --global user.email "action@github.com"
git config --global user.name "GitHub Action"
PACKAGE_OPT=""
if [ -n "${INPUT_PACKAGE}" ]; then
PACKAGE_OPT="-p ${INPUT_PACKAGE}"
fi
GROUP_OPT=""
if [ -n "${INPUT_GROUP}" ]; then
GROUP_OPT="-g ${INPUT_GROUP}"
fi
EXTRA_FLAGS="${INPUT_EXTRA_FLAGS}"
supersetbot bump-python \
--verbose \
--use-current-repo \
--include-subpackages \
--limit ${INPUT_LIMIT} \
$PACKAGE_OPT \
$GROUP_OPT \
$EXTRA_FLAGS

View File

@@ -9,7 +9,7 @@ on:
jobs:
cancel-duplicate-runs:
name: Cancel duplicate workflow runs
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
permissions:
actions: write
contents: read
@@ -31,7 +31,7 @@ jobs:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
if: steps.check_queued.outputs.count >= 20
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
- name: Cancel duplicate workflow runs
if: steps.check_queued.outputs.count >= 20

View File

@@ -1,59 +0,0 @@
name: Check python dependencies
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
check-python-deps:
runs-on: ubuntu-22.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
fetch-depth: 1
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
- name: Run uv
if: steps.check.outputs.python
run: ./scripts/uv-pip-compile.sh
- name: Check for uncommitted changes
if: steps.check.outputs.python
run: |
echo "Full diff (for logging/debugging):"
git diff
echo "Filtered diff (excluding comments and whitespace):"
filtered_diff=$(git diff -U0 | grep '^[-+]' | grep -vE '^[-+]{3}' | grep -vE '^[-+][[:space:]]*#' | grep -vE '^[-+][[:space:]]*$' || true)
echo "$filtered_diff"
if [[ -n "$filtered_diff" ]]; then
echo
echo "ERROR: The pinned dependencies are not up-to-date."
echo "Please run './scripts/uv-pip-compile.sh' and commit the changes."
echo "More info: https://github.com/apache/superset/tree/master/requirements"
exit 1
else
echo "Pinned dependencies are up-to-date."
fi

View File

@@ -4,8 +4,7 @@ on:
paths:
- "superset/migrations/**"
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
paths:
- "superset/migrations/**"
@@ -19,15 +18,15 @@ concurrency:
jobs:
check_db_migration_conflict:
name: Check DB migration conflict
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
- name: Check and notify
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
uses: actions/github-script@v7
with:
github-token: ${{ github.token }}
script: |
@@ -69,7 +68,7 @@ jobs:
`❗ @${pull.user.login} Your base branch \`${currentBranch}\` has ` +
'also updated `superset/migrations`.\n' +
'\n' +
'**Please consider rebasing your branch and [resolving potential db migration conflicts](https://superset.apache.org/docs/contributing/development#merging-db-migrations).**',
'**Please consider rebasing your branch and [resolving potential db migration conflicts](https://github.com/apache/superset/blob/master/CONTRIBUTING.md#merging-db-migrations).**',
});
}
}

72
.github/workflows/chromatic-master.yml vendored Normal file
View File

@@ -0,0 +1,72 @@
# .github/workflows/chromatic.yml
# see https://www.chromatic.com/docs/github-actions
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Workflow name
name: 'Chromatic Storybook Master'
# Event for the workflow
# Only run if changes were made in superset-frontend folder of repo on merge to Master
on:
# This will trigger when a branch merges to master when the PR has changes in the frontend folder updating the chromatic baseline
push:
branches:
- master
paths:
- "superset-frontend/**"
# List of jobs
jobs:
config:
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.CHROMATIC_PROJECT_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
chromatic-deployment:
needs: config
if: needs.config.outputs.has-secrets
# Operating System
runs-on: ubuntu-latest
# Job steps
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # 👈 Required to retrieve git history
- name: Install dependencies
run: npm ci
working-directory: superset-frontend
# 👇 Build and publish Storybook to Chromatic
- name: Build and publish Storybook to Chromatic
id: chromatic-master
uses: chromaui/action@v10
# Required options for the Chromatic GitHub Action
with:
# 👇 Location of package.json from root of mono-repo
workingDir: superset-frontend
# 👇 Chromatic projectToken, refer to the manage page to obtain it.
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
exitZeroOnChanges: true # 👈 Option to prevent the workflow from failing
autoAcceptChanges: true # 👈 Option to accept all changes when merging to master

View File

@@ -1,82 +0,0 @@
name: Claude PR Assistant
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
jobs:
check-permissions:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
runs-on: ubuntu-latest
outputs:
allowed: ${{ steps.check.outputs.allowed }}
steps:
- name: Check if user is allowed
id: check
run: |
# List of allowed users
ALLOWED_USERS="mistercrunch,rusackas"
# Get the commenter's username
COMMENTER="${{ github.event.comment.user.login }}"
echo "Checking permissions for user: $COMMENTER"
# Check if user is in allowed list
if [[ ",$ALLOWED_USERS," == *",$COMMENTER,"* ]]; then
echo "allowed=true" >> $GITHUB_OUTPUT
echo "✅ User $COMMENTER is allowed to use Claude"
else
echo "allowed=false" >> $GITHUB_OUTPUT
echo "❌ User $COMMENTER is not allowed to use Claude"
fi
deny-access:
needs: check-permissions
if: needs.check-permissions.outputs.allowed == 'false'
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Comment access denied
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const message = `👋 Hi @${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}!
Thanks for trying to use Claude Code, but currently only certain team members have access to this feature.
If you believe you should have access, please contact a project maintainer.`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: message
});
claude-code-action:
needs: check-permissions
if: needs.check-permissions.outputs.allowed == 'true'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 1
- name: Run Claude PR Action
uses: anthropics/claude-code-action@5fb899572b81d2bb648d4d187173a2f423a9677c # beta
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
timeout_minutes: "60"

1903
.github/workflows/codecov.sh vendored Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -2,12 +2,16 @@ name: "CodeQL"
on:
push:
branches: ["master", "[0-9].[0-9]*"]
branches: [ "master" ]
paths:
- 'superset/**'
pull_request:
# The branches below must be a subset of the branches above
branches: ["master"]
branches: [ "master" ]
paths:
- 'superset/**'
schedule:
- cron: "0 4 * * *"
- cron: '0 4 * * *'
# cancel previous workflow jobs for PRs
concurrency:
@@ -17,7 +21,7 @@ concurrency:
jobs:
analyze:
name: Analyze
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
permissions:
actions: read
contents: read
@@ -26,22 +30,16 @@ jobs:
strategy:
fail-fast: false
matrix:
language: ["python", "javascript"]
language: [ 'python', 'javascript' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
uses: actions/checkout@v4
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -52,7 +50,6 @@ jobs:
# queries: security-extended,security-and-quality
- name: Perform CodeQL Analysis
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"

View File

@@ -5,63 +5,23 @@
# Source repository: https://github.com/actions/dependency-review-action
# Public documentation: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement
name: "Dependency Review"
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
on: [pull_request]
permissions:
contents: read
jobs:
dependency-review:
if: github.event_name == 'pull_request'
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- name: "Checkout Repository"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
- name: "Dependency Review"
uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4.9.0
continue-on-error: true
uses: actions/dependency-review-action@v4
with:
fail-on-severity: critical
# compatible/incompatible licenses addressed here: https://www.apache.org/legal/resolved.html
# find SPDX identifiers here: https://spdx.org/licenses/
deny-licenses: MS-LPL, BUSL-1.1, QPL-1.0, Sleepycat, SSPL-1.0, CPOL-1.02, AGPL-3.0, GPL-1.0+, BSD-4-Clause-UC, NPL-1.0, NPL-1.1, JSON
# pkg:npm/store2@2.14.2
# adding an exception for an ambigious license on store2, which has been resolved in
# the latest version. It's MIT: https://github.com/nbubna/store/blob/master/LICENSE-MIT
# pkg:npm/node-forge@1.3.1
# selecting BSD-3-Clause licensing terms for node-forge to ensure compatibility with Apache
allow-dependencies-licenses: pkg:npm/store2@2.14.2, pkg:npm/node-forge@1.3.1, pkg:npm/rgbcolor, pkg:npm/jszip@3.10.1
python-dependency-liccheck:
# NOTE: Configuration for liccheck lives in our pyproject.yml.
# You cannot use a liccheck.ini file in this workflow.
runs-on: ubuntu-22.04
steps:
- name: "Checkout Repository"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Setup Python
uses: ./.github/actions/setup-backend/
with:
requirements-type: base
- name: "Set up liccheck"
run: |
uv pip install --system liccheck
- name: "Run liccheck"
run: |
# run the checks
liccheck -R output.txt
# Print the report
cat output.txt
# adding an exception for an ambigious license on store2, which has been resolved in the latest version. It's MIT: https://github.com/nbubna/store/blob/master/LICENSE-MIT
allow-dependencies-licenses: 'pkg:npm/store2@2.14.2'

89
.github/workflows/docker-release.yml vendored Normal file
View File

@@ -0,0 +1,89 @@
name: Docker Publish Release
on:
release:
types: [published, edited]
# Can be triggered manually
workflow_dispatch:
inputs:
release:
required: true
description: The version to generate
git-ref:
required: true
description: The git reference to checkout prior to running the docker build
force-latest:
required: true
type: choice
default: 'false'
description: Whether to force a latest tag on the release
options:
- true
- false
jobs:
config:
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.DOCKERHUB_USER != '' && secrets.DOCKERHUB_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
docker-release:
needs: config
if: needs.config.outputs.has-secrets
name: docker-release
runs-on: ubuntu-latest
strategy:
matrix:
build_preset: ["dev", "lean", "py310", "websocket", "dockerize"]
fail-fast: false
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.ref }}
fetch-depth: 0
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Docker Image
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
RELEASE="${{ github.event.release.tag_name }}"
FORCE_LATEST=""
EVENT="${{github.event_name}}"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# in the case of a manually-triggered run, read release from input
RELEASE="${{ github.event.inputs.release }}"
if [ "${{ github.event.inputs.force-latest }}" = "true" ]; then
FORCE_LATEST="--force-latest"
fi
# build_docker.py may not exist on that SHA, let's switcharoo to /tmp
cp ./scripts/build_docker.py /tmp
git checkout "${{ github.event.inputs.git-ref }}"
cp /tmp/build_docker.py scripts/
EVENT="release"
fi
pip install click
# Make a multi-platform image
./scripts/build_docker.py \
${{ matrix.build_preset }} \
"$EVENT" \
--build_context_ref "$RELEASE" $FORCE_LATEST \
--platform "linux/arm64" \
--platform "linux/amd64"

View File

@@ -1,140 +1,49 @@
name: Build & publish docker images
name: Docker
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
branches:
- "master"
types: [synchronize, opened, reopened, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
setup_matrix:
runs-on: ubuntu-24.04
outputs:
matrix_config: ${{ steps.set_matrix.outputs.matrix_config }}
steps:
- id: set_matrix
run: |
MATRIX_CONFIG=$(if [ "${{ github.event_name }}" == "pull_request" ]; then echo '["dev", "lean"]'; else echo '["dev", "lean", "py310", "websocket", "dockerize", "py311", "py312"]'; fi)
echo "matrix_config=${MATRIX_CONFIG}" >> $GITHUB_OUTPUT
echo $GITHUB_OUTPUT
docker-build:
name: docker-build
needs: setup_matrix
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
strategy:
matrix:
build_preset: ${{fromJson(needs.setup_matrix.outputs.matrix_config)}}
build_preset: ["dev", "lean", "py310", "websocket", "dockerize"]
platform: ["linux/amd64", "linux/arm64"]
exclude:
# disabling because slow! no python wheels for arm/py39 and
# QEMU is slow!
- build_preset: "dev"
platform: "linux/arm64"
- build_preset: "lean"
platform: "linux/arm64"
fail-fast: false
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
IMAGE_TAG: apache/superset:GHA-${{ matrix.build_preset }}-${{ github.run_id }}
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Setup Docker Environment
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
- name: Setup supersetbot
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
uses: ./.github/actions/setup-supersetbot/
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Docker Image
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
# Single platform builds in pull_request context to speed things up
if [ "${{ github.event_name }}" = "push" ]; then
PLATFORM_ARG="--platform linux/arm64 --platform linux/amd64"
# can only --load images in single-platform builds
PUSH_OR_LOAD="--push"
elif [ "${{ github.event_name }}" = "pull_request" ]; then
PLATFORM_ARG="--platform linux/amd64"
PUSH_OR_LOAD="--load"
fi
supersetbot docker \
$PUSH_OR_LOAD \
--preset ${{ matrix.build_preset }} \
--context "$EVENT" \
--context-ref "$RELEASE" $FORCE_LATEST \
--extra-flags "--build-arg INCLUDE_CHROMIUM=false --tag $IMAGE_TAG" \
$PLATFORM_ARG
# in the context of push (using multi-platform build), we need to pull the image locally
- name: Docker pull
if: github.event_name == 'push' && (steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker)
run: docker pull $IMAGE_TAG
- name: Print docker stats
if: steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker
run: |
echo "SHA: ${{ github.sha }}"
echo "IMAGE: $IMAGE_TAG"
docker images $IMAGE_TAG
docker history $IMAGE_TAG
- name: docker-compose sanity check
if: (steps.check.outputs.python || steps.check.outputs.frontend || steps.check.outputs.docker) && matrix.build_preset == 'dev'
shell: bash
run: |
export SUPERSET_BUILD_TARGET=${{ matrix.build_preset }}
# This should reuse the CACHED image built in the previous steps
docker compose build superset-init --build-arg DEV_MODE=false --build-arg INCLUDE_CHROMIUM=false
docker compose up superset-init --exit-code-from superset-init
docker-compose-image-tag:
# Run this job only on pushes to master (not for PRs)
# goal is to check that building the latest image works, not required for all PR pushes
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-24.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Docker Environment
if: steps.check.outputs.docker
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "false"
install-docker-compose: "true"
- name: docker-compose sanity check
if: steps.check.outputs.docker
shell: bash
run: |
docker compose -f docker-compose-image-tag.yml up superset-init --exit-code-from superset-init
pip install click
./scripts/build_docker.py ${{ matrix.build_preset }} ${{ github.event_name }} --platform ${{ matrix.platform }}

View File

@@ -3,12 +3,11 @@ name: Embedded SDK Release
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -16,24 +15,22 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${NPM_TOKEN}" ]; then
if [ -n "${{ (secrets.NPM_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
NPM_TOKEN: ${{ (secrets.NPM_TOKEN != '') || '' }}
build:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
defaults:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: './superset-embedded-sdk/.nvmrc'
node-version: "16"
registry-url: 'https://registry.npmjs.org'
- run: npm ci
- run: npm run ci:release

View File

@@ -13,15 +13,15 @@ concurrency:
jobs:
embedded-sdk-test:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
defaults:
run:
working-directory: superset-embedded-sdk
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: './superset-embedded-sdk/.nvmrc'
node-version: "16"
registry-url: 'https://registry.npmjs.org'
- run: npm ci
- run: npm test

View File

@@ -1,10 +1,4 @@
name: Cleanup ephemeral envs (PR close) [DEPRECATED]
# ⚠️ DEPRECATION NOTICE ⚠️
# This workflow is deprecated and will be removed in a future version.
# The new Superset Showtime workflow handles cleanup automatically.
# See .github/workflows/showtime.yml and showtime-cleanup.yml for replacements.
# Migration guide: https://github.com/mistercrunch/superset-showtime
name: Cleanup ephemeral envs (PR close)
on:
pull_request_target:
@@ -12,7 +6,7 @@ on:
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -20,22 +14,20 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${AWS_ACCESS_KEY_ID}" ]; then
if [ -n "${{ (secrets.AWS_ACCESS_KEY_ID != '' && secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
AWS_ACCESS_KEY_ID: ${{ (secrets.AWS_ACCESS_KEY_ID != '' && secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}
ephemeral-env-cleanup:
needs: config
if: needs.config.outputs.has-secrets
name: Cleanup ephemeral envs
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
@@ -58,7 +50,7 @@ jobs:
- name: Login to Amazon ECR
if: steps.describe-services.outputs.active == 'true'
id: login-ecr
uses: aws-actions/amazon-ecr-login@19d944daaa35f0fa1d3f7f8af1d3f2e5de25c5b7 # v2
uses: aws-actions/amazon-ecr-login@v2
- name: Delete ECR image tag
if: steps.describe-services.outputs.active == 'true'
@@ -71,7 +63,7 @@ jobs:
- name: Comment (success)
if: steps.describe-services.outputs.active == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
uses: actions/github-script@v7
with:
github-token: ${{github.token}}
script: |
@@ -79,5 +71,5 @@ jobs:
issue_number: ${{ github.event.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '⚠️ **DEPRECATED WORKFLOW** - Ephemeral environment shutdown and build artifacts deleted. Please migrate to the new Superset Showtime system for future PRs.'
body: 'Ephemeral environment shutdown and build artifacts deleted.'
})

View File

@@ -1,197 +1,135 @@
name: Ephemeral env workflow [DEPRECATED]
# ⚠️ DEPRECATION NOTICE ⚠️
# This workflow is deprecated and will be removed in a future version.
# Please use the new Superset Showtime workflow instead:
# - Use label "🎪 trigger-start" instead of "testenv-up"
# - Showtime provides better reliability and easier management
# - See .github/workflows/showtime.yml for the replacement
# - Migration guide: https://github.com/mistercrunch/superset-showtime
# Example manual trigger:
# gh workflow run ephemeral-env.yml --ref fix_ephemerals --field label_name="testenv-up" --field issue_number=666
name: Ephemeral env workflow
on:
pull_request_target:
types:
- labeled
workflow_dispatch:
inputs:
label_name:
description: 'Label name to simulate label-based /testenv trigger'
required: true
default: 'testenv-up'
issue_number:
description: 'Issue or PR number'
required: true
issue_comment:
types: [created]
jobs:
ephemeral-env-label:
config:
runs-on: "ubuntu-latest"
if: github.event.issue.pull_request
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.AWS_ACCESS_KEY_ID != '' && secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
ephemeral-env-comment:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-label
group: ${{ github.workflow }}-${{ github.event.issue.number || github.run_id }}-comment
cancel-in-progress: true
name: Evaluate ephemeral env label trigger
runs-on: ubuntu-24.04
needs: config
if: needs.config.outputs.has-secrets
name: Evaluate ephemeral env comment trigger (/testenv)
runs-on: ubuntu-latest
permissions:
pull-requests: write
outputs:
slash-command: ${{ steps.eval-label.outputs.result }}
slash-command: ${{ steps.eval-body.outputs.result }}
feature-flags: ${{ steps.eval-feature-flags.outputs.result }}
sha: ${{ steps.get-sha.outputs.sha }}
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
steps:
- name: Check for the "testenv-up" label
id: eval-label
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
LABEL_NAME="${INPUT_LABEL_NAME}"
else
LABEL_NAME="${{ github.event.label.name }}"
fi
- name: Debug
run: |
echo "Comment on PR #${{ github.event.issue.number }} by ${{ github.event.issue.user.login }}, ${{ github.event.comment.author_association }}"
echo "Evaluating label: $LABEL_NAME"
- name: Eval comment body for /testenv slash command
uses: actions/github-script@v7
id: eval-body
with:
result-encoding: string
script: |
const pattern = /^\/testenv (up|down)/
const result = pattern.exec(context.payload.comment.body)
return result === null ? 'noop' : result[1]
if [[ "$LABEL_NAME" == "testenv-up" ]]; then
echo "result=up" >> $GITHUB_OUTPUT
else
echo "result=noop" >> $GITHUB_OUTPUT
fi
- name: Eval comment body for feature flags
uses: actions/github-script@v7
id: eval-feature-flags
with:
script: |
const pattern = /FEATURE_(\w+)=(\w+)/g;
let results = [];
[...context.payload.comment.body.matchAll(pattern)].forEach(match => {
const config = {
name: `SUPERSET_FEATURE_${match[1]}`,
value: match[2],
};
results.push(config);
});
return results;
env:
INPUT_LABEL_NAME: ${{ github.event.inputs.label_name }}
- name: Get event SHA
id: get-sha
if: steps.eval-label.outputs.result == 'up'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
let prSha;
// If event is workflow_dispatch, use the issue_number from inputs
if (context.eventName === "workflow_dispatch") {
const prNumber = "${{ github.event.inputs.issue_number }}";
if (!prNumber) {
console.log("No PR number found.");
return;
}
// Fetch PR details using the provided issue_number
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber
});
prSha = pr.head.sha;
} else {
// If it's not workflow_dispatch, use the PR head sha from the event
prSha = context.payload.pull_request.head.sha;
}
console.log(`PR SHA: ${prSha}`);
core.setOutput("sha", prSha);
- name: Looking for feature flags in PR description
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
id: eval-feature-flags
if: steps.eval-label.outputs.result == 'up'
with:
script: |
const description = context.payload.pull_request
? context.payload.pull_request.body || ''
: context.payload.inputs.pr_description || '';
const pattern = /FEATURE_(\w+)=(\w+)/g;
let results = [];
[...description.matchAll(pattern)].forEach(match => {
const config = {
name: `SUPERSET_FEATURE_${match[1]}`,
value: match[2],
};
results.push(config);
});
return results;
- name: Reply with confirmation comment
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
if: steps.eval-label.outputs.result == 'up'
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const action = '${{ steps.eval-label.outputs.result }}';
const user = context.actor;
const runId = context.runId;
const workflowUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${runId}`;
const issueNumber = context.payload.pull_request
? context.payload.pull_request.number
: context.payload.inputs.issue_number;
if (!issueNumber) {
throw new Error("Issue number is not available.");
}
const body = `⚠️ **DEPRECATED WORKFLOW** ⚠️\n\n@${user} This workflow is deprecated! Please use the new **Superset Showtime** system instead:\n\n` +
`- Replace "testenv-up" label with "🎪 trigger-start"\n` +
`- Better reliability and easier management\n` +
`- See https://github.com/mistercrunch/superset-showtime for details\n\n` +
`Processing your ephemeral environment request [here](${workflowUrl}). Action: **${action}**.` +
` More information on [how to use or configure ephemeral environments]` +
`(https://superset.apache.org/docs/contributing/howtos/#github-ephemeral-environments)`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
body,
});
- name: Limit to committers
if: >
steps.eval-body.outputs.result != 'noop' &&
github.event.comment.author_association != 'MEMBER' &&
github.event.comment.author_association != 'OWNER'
uses: actions/github-script@v7
with:
github-token: ${{github.token}}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Ephemeral environment creation is currently limited to committers.'
github.rest.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
})
core.setFailed(errMsg)
ephemeral-docker-build:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-build
group: ${{ github.workflow }}-${{ github.event.issue.number || github.run_id }}-build
cancel-in-progress: true
needs: ephemeral-env-label
if: needs.ephemeral-env-label.outputs.slash-command == 'up'
needs: ephemeral-env-comment
name: ephemeral-docker-build
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ needs.ephemeral-env-label.outputs.sha }} : ${{steps.get-sha.outputs.sha}} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Get Info from comment
uses: actions/github-script@v7
id: get-pr-info
with:
ref: ${{ needs.ephemeral-env-label.outputs.sha }}
script: |
const request = {
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: ${{ github.event.issue.number }},
}
core.info(`Getting PR #${request.pull_number} from ${request.owner}/${request.repo}`)
const pr = await github.rest.pulls.get(request);
return pr.data;
- name: Debug
id: get-sha
run: |
echo "sha=${{ fromJSON(steps.get-pr-info.outputs.result).head.sha }}" >> $GITHUB_OUTPUT
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} : ${{steps.get-sha.outputs.sha}} )"
uses: actions/checkout@v4
with:
ref: ${{ steps.get-sha.outputs.sha }}
persist-credentials: false
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
install-docker-compose: "false"
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build ephemeral env image
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
supersetbot docker \
--push \
--load \
--preset ci \
--platform linux/amd64 \
--context-ref "$RELEASE" \
--extra-flags "--build-arg INCLUDE_CHROMIUM=false"
./scripts/build_docker.py \
"ci" \
"pull_request" \
--build_context_ref ${{ github.event.issue.number }}
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
@@ -199,152 +137,147 @@ jobs:
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@19d944daaa35f0fa1d3f7f8af1d3f2e5de25c5b7 # v2
uses: aws-actions/amazon-ecr-login@v2
- name: Load, tag and push image to ECR
id: push-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: superset-ci
IMAGE_TAG: apache/superset:${{ needs.ephemeral-env-label.outputs.sha }}-ci
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
IMAGE_TAG: apache/superset:${{ steps.get-sha.outputs.sha }}-ci
run: |
docker tag $IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:pr-$PR_NUMBER-ci
docker tag $IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:pr-${{ github.event.issue.number }}-ci
docker push -a $ECR_REGISTRY/$ECR_REPOSITORY
ephemeral-env-up:
needs: [ephemeral-env-label, ephemeral-docker-build]
if: needs.ephemeral-env-label.outputs.slash-command == 'up'
needs: [ephemeral-env-comment, ephemeral-docker-build]
if: needs.ephemeral-env-comment.outputs.slash-command == 'up'
name: Spin up an ephemeral environment
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@19d944daaa35f0fa1d3f7f8af1d3f2e5de25c5b7 # v2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Check target image exists in ECR
id: check-image
continue-on-error: true
env:
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
run: |
aws ecr describe-images \
--registry-id $(echo "${{ steps.login-ecr.outputs.registry }}" | grep -Eo "^[0-9]+") \
--repository-name superset-ci \
--image-ids imageTag=pr-$PR_NUMBER-ci
- name: Check target image exists in ECR
id: check-image
continue-on-error: true
run: |
aws ecr describe-images \
--registry-id $(echo "${{ steps.login-ecr.outputs.registry }}" | grep -Eo "^[0-9]+") \
--repository-name superset-ci \
--image-ids imageTag=pr-${{ github.event.issue.number }}-ci
- name: Fail on missing container image
if: steps.check-image.outcome == 'failure'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ github.token }}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Container image not yet published for this PR. Please try again when build is complete.';
github.rest.issues.createComment({
issue_number: ${{ github.event.inputs.issue_number || github.event.pull_request.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
});
core.setFailed(errMsg);
- name: Fail on missing container image
if: steps.check-image.outcome == 'failure'
uses: actions/github-script@v7
with:
github-token: ${{github.token}}
script: |
const errMsg = '@${{ github.event.comment.user.login }} Container image not yet published for this PR. Please try again when build is complete.'
github.rest.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: errMsg
})
core.setFailed(errMsg)
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@6853cfae8c3a7d978fbf68b5a55453395541dfbb # v1
with:
task-definition: .github/workflows/ecs-task-definition.json
container-name: superset-ci
image: ${{ steps.login-ecr.outputs.registry }}/superset-ci:pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-ci
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: .github/workflows/ecs-task-definition.json
container-name: superset-ci
image: ${{ steps.login-ecr.outputs.registry }}/superset-ci:pr-${{ github.event.issue.number }}-ci
- name: Update env vars in the Amazon ECS task definition
run: |
cat <<< "$(jq '.containerDefinitions[0].environment += ${{ needs.ephemeral-env-label.outputs.feature-flags }}' < ${{ steps.task-def.outputs.task-definition }})" > ${{ steps.task-def.outputs.task-definition }}
- name: Update env vars in the Amazon ECS task definition
run: |
cat <<< "$(jq '.containerDefinitions[0].environment += ${{ needs.ephemeral-env-comment.outputs.feature-flags }}' < ${{ steps.task-def.outputs.task-definition }})" > ${{ steps.task-def.outputs.task-definition }}
- name: Describe ECS service
id: describe-services
run: |
echo "active=$(aws ecs describe-services --cluster superset-ci --services pr-${INPUT_ISSUE_NUMBER}-service | jq '.services[] | select(.status == "ACTIVE") | any')" >> $GITHUB_OUTPUT
env:
INPUT_ISSUE_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
- name: Create ECS service
id: create-service
if: steps.describe-services.outputs.active != 'true'
env:
ECR_SUBNETS: subnet-0e15a5034b4121710,subnet-0e8efef4a72224974
ECR_SECURITY_GROUP: sg-092ff3a6ae0574d91
PR_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
run: |
aws ecs create-service \
--cluster superset-ci \
--service-name pr-$PR_NUMBER-service \
--task-definition superset-ci \
--launch-type FARGATE \
--desired-count 1 \
--platform-version LATEST \
--network-configuration "awsvpcConfiguration={subnets=[$ECR_SUBNETS],securityGroups=[$ECR_SECURITY_GROUP],assignPublicIp=ENABLED}" \
--tags key=pr,value=$PR_NUMBER key=github_user,value=${{ github.actor }}
- name: Deploy Amazon ECS task definition
id: deploy-task
uses: aws-actions/amazon-ecs-deploy-task-definition@a310a830f5c14e583e35d84e4e1ec7dd177c3c9c # v2
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: pr-${{ github.event.inputs.issue_number || github.event.pull_request.number }}-service
cluster: superset-ci
wait-for-service-stability: true
wait-for-minutes: 10
- name: Describe ECS service
id: describe-services
run: |
echo "active=$(aws ecs describe-services --cluster superset-ci --services pr-${{ github.event.issue.number }}-service | jq '.services[] | select(.status == "ACTIVE") | any')" >> $GITHUB_OUTPUT
- name: Create ECS service
if: steps.describe-services.outputs.active != 'true'
id: create-service
env:
ECR_SUBNETS: subnet-0e15a5034b4121710,subnet-0e8efef4a72224974
ECR_SECURITY_GROUP: sg-092ff3a6ae0574d91
run: |
aws ecs create-service \
--cluster superset-ci \
--service-name pr-${{ github.event.issue.number }}-service \
--task-definition superset-ci \
--launch-type FARGATE \
--desired-count 1 \
--platform-version LATEST \
--network-configuration "awsvpcConfiguration={subnets=[$ECR_SUBNETS],securityGroups=[$ECR_SECURITY_GROUP],assignPublicIp=ENABLED}" \
--tags key=pr,value=${{ github.event.issue.number }} key=github_user,value=${{ github.actor }}
- name: List tasks
id: list-tasks
run: |
echo "task=$(aws ecs list-tasks --cluster superset-ci --service-name pr-${INPUT_ISSUE_NUMBER}-service | jq '.taskArns | first')" >> $GITHUB_OUTPUT
env:
INPUT_ISSUE_NUMBER: ${{ github.event.inputs.issue_number || github.event.pull_request.number }}
- name: Get network interface
id: get-eni
run: |
echo "eni=$(aws ecs describe-tasks --cluster superset-ci --tasks ${{ steps.list-tasks.outputs.task }} | jq '.tasks[0].attachments[0].details | map(select(.name=="networkInterfaceId"))[0].value')" >> $GITHUB_OUTPUT
- name: Get public IP
id: get-ip
run: |
echo "ip=$(aws ec2 describe-network-interfaces --network-interface-ids ${{ steps.get-eni.outputs.eni }} | jq -r '.NetworkInterfaces | first | .Association.PublicIp')" >> $GITHUB_OUTPUT
- name: Comment (success)
if: ${{ success() }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{github.token}}
script: |
const issue_number = context.payload.inputs?.issue_number || context.issue.number;
github.rest.issues.createComment({
issue_number: issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `@${{ github.actor }} Ephemeral environment spinning up at http://${{ steps.get-ip.outputs.ip }}:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.`
});
- name: Comment (failure)
if: ${{ failure() }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{github.token}}
script: |
const issue_number = context.payload.inputs?.issue_number || context.issue.number;
github.rest.issues.createComment({
issue_number: issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.inputs.user_login || github.event.comment.user.login }} Ephemeral environment creation failed. Please check the Actions logs for details.'
})
- name: Deploy Amazon ECS task definition
id: deploy-task
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: pr-${{ github.event.issue.number }}-service
cluster: superset-ci
wait-for-service-stability: true
wait-for-minutes: 10
- name: List tasks
id: list-tasks
run: |
echo "task=$(aws ecs list-tasks --cluster superset-ci --service-name pr-${{ github.event.issue.number }}-service | jq '.taskArns | first')" >> $GITHUB_OUTPUT
- name: Get network interface
id: get-eni
run: |
echo "eni=$(aws ecs describe-tasks --cluster superset-ci --tasks ${{ steps.list-tasks.outputs.task }} | jq '.tasks | .[0] | .attachments | .[0] | .details | map(select(.name=="networkInterfaceId")) | .[0] | .value')" >> $GITHUB_OUTPUT
- name: Get public IP
id: get-ip
run: |
echo "ip=$(aws ec2 describe-network-interfaces --network-interface-ids ${{ steps.get-eni.outputs.eni }} | jq -r '.NetworkInterfaces | first | .Association.PublicIp')" >> $GITHUB_OUTPUT
- name: Comment (success)
if: ${{ success() }}
uses: actions/github-script@v7
with:
github-token: ${{github.token}}
script: |
github.rest.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.comment.user.login }} Ephemeral environment spinning up at http://${{ steps.get-ip.outputs.ip }}:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.'
})
- name: Comment (failure)
if: ${{ failure() }}
uses: actions/github-script@v7
with:
github-token: ${{github.token}}
script: |
github.rest.issues.createComment({
issue_number: ${{ github.event.issue.number }},
owner: context.repo.owner,
repo: context.repo.repo,
body: '@${{ github.event.comment.user.login }} Ephemeral environment creation failed. Please check the Actions logs for details.'
})

View File

@@ -4,11 +4,10 @@ on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -16,28 +15,26 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${FOSSA_API_KEY}" ]; then
if [ -n "${{ (secrets.FOSSA_API_KEY != '' ) || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
FOSSA_API_KEY: ${{ (secrets.FOSSA_API_KEY != '' ) || '' }}
license_check:
needs: config
if: needs.config.outputs.has-secrets
name: Generate Report
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Setup Java
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
uses: actions/setup-java@v4
with:
distribution: "temurin"
java-version: "11"
distribution: 'temurin'
java-version: '11'
- name: Generate fossa report
env:
FOSSA_API_KEY: ${{ secrets.FOSSA_API_KEY }}

View File

@@ -1,28 +0,0 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Inspired from https://github.com/mpalmer/action-validator?tab=readme-ov-file#pre-commit-hook-example
echo "Running pre-commit hook for GitHub Actions: https://github.com/mpalmer/action-validator"
for action in $(git ls-files .github/ | grep -E '^\.github/(workflows|actions)/.*\.ya?ml$'); do
if action-validator "$action"; then
echo "$action"
else
echo "$action"
exit 1
fi
done

View File

@@ -1,28 +0,0 @@
name: Validate All GitHub Actions
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
jobs:
validate-all-ghas:
runs-on: ubuntu-24.04
steps:
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: '20'
- name: Install Dependencies
run: npm install -g @action-validator/core @action-validator/cli --save-dev
- name: Run Script
run: bash .github/workflows/github-action-validator.sh

View File

@@ -1,34 +0,0 @@
name: supersetbot orglabel based on author
on:
issues:
types: [created, edited]
pull_request:
types: [created, edited]
jobs:
superbot-orglabel:
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute supersetbot orglabel command
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Label the issue with the appropriate org using supersetbot
# - this requires for the author to be publicly associated with their org
# - and for the org to be listed in `supersetbot/src/metadata.js`
supersetbot orglabel --issue ${{ github.event.number }} --repo ${{ github.repository }} || true

View File

@@ -7,9 +7,9 @@ jobs:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v6
- uses: actions/labeler@v5
with:
sync-labels: true

View File

@@ -6,13 +6,13 @@ on:
jobs:
latest-release:
name: Add/update tag to new release
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive

View File

@@ -12,15 +12,15 @@ concurrency:
jobs:
license_check:
name: License Check
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Setup Java
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '11'

View File

@@ -4,9 +4,6 @@ on:
pull_request:
types: [labeled, unlabeled, opened, reopened, synchronize]
permissions:
pull-requests: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -14,10 +11,10 @@ concurrency:
jobs:
check-hold-label:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- name: Check for 'hold' label
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
uses: actions/github-script@v7
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |

54
.github/workflows/no-op.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
# no-op.yml
#
# Purpose:
# This workflow provides a workaround for the "required status checks" feature in GitHub Actions
# when using path-specific conditions in other workflows. Required checks might remain in a "Pending"
# state if the conditions are not met, thus blocking pull requests from being merged.
# This no-op (no operation) workflow provides dummy success statuses for these required jobs when
# the real jobs do not run due to path-specific conditions.
#
# How it works:
# - It defines jobs with the same names as the required jobs in the main workflows.
# - These jobs simply execute a command (`exit 0`) to succeed immediately.
# - When a pull request is created or updated, both this no-op workflow and the main workflows are triggered.
# - If the main workflows' jobs don't run (due to path conditions), these no-op jobs provide successful statuses.
# - If the main workflows' jobs do run and fail, their failure statuses take precedence,
# ensuring that pull requests are not merged with failing checks.
#
# Usage:
# - Ensure that the job names in this workflow match exactly the names of the corresponding jobs in the main workflows.
# - This workflow should be kept as-is, without path-specific conditions.
name: No Operation Checks
on: pull_request
jobs:
frontend-build:
runs-on: ubuntu-latest
steps:
- name: No-op for frontend-build
run: |
echo "This is a no-op step for frontend-build to ensure a successful status."
exit 0
pre-commit:
strategy:
matrix:
python-version: ["3.9"]
runs-on: ubuntu-latest
steps:
- name: No-op for pre-commit
run: |
echo "This is a no-op step for pre-commit to ensure a successful status."
exit 0
python-lint:
strategy:
matrix:
python-version: ["3.9"]
runs-on: ubuntu-latest
steps:
- name: No-op for python-lint
run: |
echo "This is a no-op step for python-lint to ensure a successful status."
exit 0

View File

@@ -10,13 +10,13 @@ on:
jobs:
lint-check:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive

View File

@@ -3,14 +3,10 @@ name: pre-commit checks
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
permissions:
contents: read
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -18,22 +14,35 @@ concurrency:
jobs:
pre-commit:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["current", "previous", "next"]
python-version: ["3.9"]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: ./.github/actions/setup-backend/
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: |
requirements/base.txt
requirements/integration.txt
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/base.txt
pip install -r requirements/integration.txt
# Add brew to the path - see https://github.com/actions/runner-images/issues/6283
- name: Enable brew and helm-docs
# Add brew to the path - see https://github.com/actions/runner-images/issues/6283
run: |
echo "/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin" >> $GITHUB_PATH
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
@@ -41,56 +50,10 @@ jobs:
echo "HOMEBREW_CELLAR=$HOMEBREW_CELLAR" >>"${GITHUB_ENV}"
echo "HOMEBREW_REPOSITORY=$HOMEBREW_REPOSITORY" >>"${GITHUB_ENV}"
brew install norwoodj/tap/helm-docs
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: '20'
- name: Install Frontend Dependencies
run: |
cd superset-frontend
npm ci
- name: Install Docs Dependencies
run: |
cd docs
yarn install --immutable
- name: Cache pre-commit environments
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: ~/.cache/pre-commit
key: pre-commit-v2-${{ runner.os }}-py${{ matrix.python-version }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: |
pre-commit-v2-${{ runner.os }}-py${{ matrix.python-version }}-
- name: Get changed files
id: changed_files
uses: ./.github/actions/file-changes-action
with:
output: ' '
- name: pre-commit
run: |
set +e # Don't exit immediately on failure
export SKIP=type-checking-frontend
pre-commit run --files ${{ steps.changed_files.outputs.files }}
PRE_COMMIT_EXIT_CODE=$?
git diff --quiet --exit-code
GIT_DIFF_EXIT_CODE=$?
if [ "${PRE_COMMIT_EXIT_CODE}" -ne 0 ] || [ "${GIT_DIFF_EXIT_CODE}" -ne 0 ]; then
if [ "${PRE_COMMIT_EXIT_CODE}" -ne 0 ]; then
echo "❌ Pre-commit check failed (exit code: ${PRE_COMMIT_EXIT_CODE})."
echo "🔍 Modified files:"
git diff --name-only
else
echo "❌ Git working directory is dirty."
echo "📌 This likely means that pre-commit made changes that were not committed."
echo "🔍 Modified files:"
git diff --name-only
fi
echo "🚒 To prevent/address this CI issue, please install/use pre-commit locally."
echo "📖 More details here: https://superset.apache.org/docs/contributing/development#git-hooks"
if ! pre-commit run --all-files; then
git status
git diff
exit 1
fi

69
.github/workflows/prefer-typescript.yml vendored Normal file
View File

@@ -0,0 +1,69 @@
name: Prefer TypeScript
on:
push:
branches:
- 'master'
paths:
- "superset-frontend/src/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset-frontend/src/**"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
prefer_typescript:
if: github.ref == 'ref/heads/master' && github.event_name == 'pull_request'
name: Prefer TypeScript
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Get changed files
id: changed
uses: ./.github/actions/file-changes-action
with:
githubToken: ${{ github.token }}
- name: Determine if a .js or .jsx file was added
id: check
run: |
js_files_added() {
jq -r '
map(
select(
endswith(".js") or endswith(".jsx")
)
) | join("\n")
' ${HOME}/files_added.json
}
echo "js_files_added=$(js_files_added)" >> $GITHUB_OUTPUT
- if: steps.check.outputs.js_files_added
name: Add Comment to PR
uses: ./.github/actions/comment-on-pr
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
with:
msg: |
### WARNING: Prefer TypeScript
Looks like your PR contains new `.js` or `.jsx` files:
```
${{steps.check.outputs.js_files_added}}
```
As decided in [SIP-36](https://github.com/apache/superset/issues/9101), all new frontend code should be written in TypeScript. Please convert above files to TypeScript then re-request review.

View File

@@ -3,12 +3,11 @@ name: release-workflow
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -16,19 +15,23 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${NPM_TOKEN}" ]; then
if [ -n "${{ (secrets.NPM_TOKEN != '' && secrets.GH_PERSONAL_ACCESS_TOKEN != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
NPM_TOKEN: ${{ (secrets.NPM_TOKEN != '' && secrets.GH_PERSONAL_ACCESS_TOKEN != '') || '' }}
build:
needs: config
if: needs.config.outputs.has-secrets
name: Bump version and publish package(s)
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
strategy:
matrix:
node-version: [16]
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/checkout@v4
with:
# pulls all commits (needed for lerna / semantic release to correctly version)
fetch-depth: 0
@@ -42,15 +45,15 @@ jobs:
git fetch --prune --unshallow
git tag -d `git tag | grep -E '^trigger-'`
- name: Install Node.js
- name: Use Node.js ${{ matrix.node-version }}
if: env.HAS_TAGS
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: ${{ matrix.node-version }}
- name: Cache npm
if: env.HAS_TAGS
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
with:
path: ~/.npm # npm cache files are stored in `~/.npm` on Linux/macOS
key: ${{ runner.OS }}-node-${{ hashFiles('**/package-lock.json') }}
@@ -64,7 +67,7 @@ jobs:
run: echo "dir=$(npm config get cache)" >> $GITHUB_OUTPUT
- name: Cache npm
if: env.HAS_TAGS
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
id: npm-cache # use this to check for `cache-hit` (`steps.npm-cache.outputs.cache-hit != 'true'`)
with:
path: ${{ steps.npm-cache-dir-path.outputs.dir }}

View File

@@ -1,36 +0,0 @@
name: 🎪 Showtime Cleanup
# Scheduled cleanup of expired environments
on:
schedule:
- cron: '0 */6 * * *' # Every 6 hours
# Manual trigger for testing
workflow_dispatch:
# Common environment variables
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ vars.AWS_REGION || 'us-west-2' }}
GITHUB_ORG: ${{ github.repository_owner }}
GITHUB_REPO: ${{ github.event.repository.name }}
jobs:
cleanup-expired:
name: Clean up expired showtime environments
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Install Superset Showtime
run: pip install superset-showtime
- name: Cleanup expired environments
run: |
echo "Cleaning up environments respecting TTL labels"
python -m showtime cleanup --respect-ttl

View File

@@ -1,183 +0,0 @@
name: 🎪 Superset Showtime
# Ultra-simple: just sync on any PR state change
on:
pull_request_target:
types: [labeled, unlabeled, synchronize, closed]
# Manual testing
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to sync'
required: true
type: number
sha:
description: 'Specific SHA to deploy (optional, defaults to latest)'
required: false
type: string
# Common environment variables for all jobs (non-sensitive only)
env:
AWS_REGION: us-west-2
GITHUB_ORG: ${{ github.repository_owner }}
GITHUB_REPO: ${{ github.event.repository.name }}
GITHUB_ACTOR: ${{ github.actor }}
jobs:
sync:
name: 🎪 Sync PR to desired state
runs-on: ubuntu-latest
timeout-minutes: 90
permissions:
contents: read
pull-requests: write
steps:
- name: Security Check - Authorize Maintainers Only
id: auth
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
script: |
const actor = context.actor;
console.log(`🔍 Checking authorization for ${actor}`);
// Early exit for workflow_dispatch - assume authorized since it's manually triggered
if (context.eventName === 'workflow_dispatch') {
console.log(`✅ Workflow dispatch event - assuming authorized for ${actor}`);
core.setOutput('authorized', 'true');
return;
}
const { data: permission } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: actor
});
console.log(`📊 Permission level for ${actor}: ${permission.permission}`);
const authorized = ['write', 'admin'].includes(permission.permission);
// If this is a synchronize event from unauthorized user, check if Showtime is active and set blocked label
if (!authorized && context.eventName === 'pull_request_target' && context.payload.action === 'synchronize') {
console.log(`🔒 Synchronize event detected - checking if Showtime is active`);
// Check if PR has any circus tent labels (Showtime is in use)
const { data: issue } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number
});
const hasCircusLabels = issue.labels.some(label => label.name.startsWith('🎪 '));
if (hasCircusLabels) {
console.log(`🎪 Circus labels found - setting blocked label to prevent auto-deployment`);
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
labels: ['🎪 🔒 showtime-blocked']
});
console.log(`✅ Blocked label set - Showtime will detect and skip operations`);
} else {
console.log(` No circus labels found - Showtime not in use, skipping block`);
}
}
if (!authorized) {
console.log(`🚨 Unauthorized user ${actor} - skipping all operations`);
core.setOutput('authorized', 'false');
return;
}
console.log(`✅ Authorized maintainer: ${actor}`);
core.setOutput('authorized', 'true');
- name: Install Superset Showtime
if: steps.auth.outputs.authorized == 'true'
run: |
echo "::notice::Maintainer ${{ github.actor }} triggered deploy for PR ${PULL_REQUEST_NUMBER}"
pip install --upgrade superset-showtime
showtime version
env:
PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number || github.event.inputs.pr_number }}
- name: Check what actions are needed
if: steps.auth.outputs.authorized == 'true'
id: check
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INPUT_PR_NUMBER: ${{ github.event.inputs.pr_number }}
INPUT_SHA: ${{ github.event.inputs.sha }}
run: |
# Bulletproof PR number extraction
if [[ -n "${{ github.event.pull_request.number }}" ]]; then
PR_NUM="${{ github.event.pull_request.number }}"
elif [[ -n "${INPUT_PR_NUMBER}" ]]; then
PR_NUM="${INPUT_PR_NUMBER}"
else
echo "❌ No PR number found in event or inputs"
exit 1
fi
echo "Using PR number: $PR_NUM"
# Run sync check-only with optional SHA override
if [[ -n "${INPUT_SHA}" ]]; then
OUTPUT=$(python -m showtime sync $PR_NUM --check-only --sha "${INPUT_SHA}")
else
OUTPUT=$(python -m showtime sync $PR_NUM --check-only)
fi
echo "$OUTPUT"
# Extract the outputs we need for conditional steps
BUILD=$(echo "$OUTPUT" | grep "build_needed=" | cut -d'=' -f2)
SYNC=$(echo "$OUTPUT" | grep "sync_needed=" | cut -d'=' -f2)
PR_NUM_OUT=$(echo "$OUTPUT" | grep "pr_number=" | cut -d'=' -f2)
TARGET_SHA=$(echo "$OUTPUT" | grep "target_sha=" | cut -d'=' -f2)
echo "build_needed=$BUILD" >> $GITHUB_OUTPUT
echo "sync_needed=$SYNC" >> $GITHUB_OUTPUT
echo "pr_number=$PR_NUM_OUT" >> $GITHUB_OUTPUT
echo "target_sha=$TARGET_SHA" >> $GITHUB_OUTPUT
- name: Checkout PR code (only if build needed)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.build_needed == 'true'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
ref: ${{ steps.check.outputs.target_sha }}
persist-credentials: false
- name: Setup Docker Environment (only if build needed)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.build_needed == 'true'
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
build: "true"
install-docker-compose: "false"
- name: Execute sync (handles everything)
if: steps.auth.outputs.authorized == 'true' && steps.check.outputs.sync_needed == 'true'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
PR_NUM="${{ steps.check.outputs.pr_number }}"
TARGET_SHA="${{ steps.check.outputs.target_sha }}"
if [[ -n "$TARGET_SHA" ]]; then
python -m showtime sync $PR_NUM --sha "$TARGET_SHA"
else
python -m showtime sync $PR_NUM
fi

View File

@@ -1,67 +0,0 @@
name: Superset App CLI tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-load-examples:
runs-on: ubuntu-24.04
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
REDIS_PORT: 16379
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
services:
postgres:
image: postgres:17-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
# Use custom ports for services to avoid accidentally connecting to
# GitHub action runner's default installations
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
- name: Setup Postgres
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: superset init
if: steps.check.outputs.python
run: |
pip install -e .
superset db upgrade
superset load_test_users
- name: superset load_examples
if: steps.check.outputs.python
run: |
# load examples without test data
superset load_examples --load-big-data

View File

@@ -0,0 +1,103 @@
name: Applitools Cypress
on:
schedule:
- cron: "0 1 * * *"
jobs:
config:
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.APPLITOOLS_API_KEY != '' && secrets.APPLITOOLS_API_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
cypress-applitools:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-20.04
strategy:
fail-fast: false
matrix:
browser: ["chrome"]
node: [16]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
APPLITOOLS_APP_NAME: Superset
APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}
APPLITOOLS_BATCH_ID: ${{ github.sha }}
APPLITOOLS_BATCH_NAME: Superset Cypress
services:
postgres:
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
ref: master
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.9"
- name: OS dependencies
uses: ./.github/actions/cached-dependencies
with:
run: apt-get-install
- name: Install python dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
pip-upgrade
pip install -r requirements/testing.txt
- name: Setup postgres
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- name: Install npm dependencies
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install cypress
uses: ./.github/actions/cached-dependencies
with:
run: cypress-install
- name: Run Cypress
uses: ./.github/actions/cached-dependencies
env:
CYPRESS_BROWSER: ${{ matrix.browser }}
with:
run: cypress-run-applitools

View File

@@ -0,0 +1,55 @@
name: Applitools Storybook
on:
schedule:
- cron: "0 0 * * *"
env:
APPLITOOLS_APP_NAME: Superset
APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}
APPLITOOLS_BATCH_ID: ${{ github.sha }}
APPLITOOLS_BATCH_NAME: Superset Storybook
jobs:
config:
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${{ (secrets.APPLITOOLS_API_KEY != '' && secrets.APPLITOOLS_API_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
cron:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-20.04
strategy:
matrix:
node: [16]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
ref: master
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- name: Install eyes-storybook dependencies
uses: ./.github/actions/cached-dependencies
with:
run: eyes-storybook-dependencies
- name: Install NPM dependencies
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Run Applitools Eyes-Storybook
working-directory: ./superset-frontend
run: npx eyes-storybook -u https://superset-storybook.netlify.app/

80
.github/workflows/superset-cli.yml vendored Normal file
View File

@@ -0,0 +1,80 @@
name: Superset CLI tests
on:
push:
branches:
- 'master'
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-load-examples:
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
REDIS_PORT: 16379
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
services:
postgres:
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
# Use custom ports for services to avoid accidentally connecting to
# GitHub action runner's default installations
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check if python changes are present
id: check
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: superset init
if: steps.check.outcome == 'failure'
run: |
pip install -e .
superset db upgrade
superset load_test_users
- name: superset load_examples
if: steps.check.outcome == 'failure'
run: |
# load examples without test data
superset load_examples --load-big-data

View File

@@ -1,25 +1,15 @@
name: Docs Deployment
on:
# Deploy after integration tests complete on master
workflow_run:
workflows: ["Python-Integration"]
types: [completed]
branches: [master]
# Also allow manual trigger and direct pushes to docs
push:
paths:
- "docs/**"
- "README.md"
branches:
- "master"
workflow_dispatch: {}
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -27,81 +17,36 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${SUPERSET_SITE_BUILD}" ]; then
if [ -n "${{ (secrets.SUPERSET_SITE_BUILD != '' && secrets.SUPERSET_SITE_BUILD != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
SUPERSET_SITE_BUILD: ${{ (secrets.SUPERSET_SITE_BUILD != '' && secrets.SUPERSET_SITE_BUILD != '') || '' }}
build-deploy:
needs: config
if: needs.config.outputs.has-secrets
name: Build & Deploy
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
defaults:
run:
working-directory: docs
steps:
- name: "Checkout ${{ github.event.workflow_run.head_sha || github.sha }}"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
ref: ${{ github.event.workflow_run.head_sha || github.sha }}
persist-credentials: false
submodules: recursive
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Set up Node.js 16
uses: actions/setup-node@v4
with:
node-version-file: './docs/.nvmrc'
- name: Setup Python
uses: ./.github/actions/setup-backend/
- uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
with:
distribution: 'zulu'
java-version: '21'
- name: Install Graphviz
run: sudo apt-get install -y graphviz
- name: Compute Entity Relationship diagram (ERD)
env:
SUPERSET_SECRET_KEY: not-a-secret
run: |
python scripts/erd/erd.py
curl -L http://sourceforge.net/projects/plantuml/files/1.2023.7/plantuml.1.2023.7.jar/download > ~/plantuml.jar
java -jar ~/plantuml.jar -v -tsvg -r -o "${{ github.workspace }}/docs/static/img/" "${{ github.workspace }}/scripts/erd/erd.puml"
node-version: '16'
- name: yarn install
working-directory: docs
run: |
yarn install --check-cache
- name: Download database diagnostics (if triggered by integration tests)
if: github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success'
uses: dawidd6/action-download-artifact@b6e2e70617bc3265edd6dab6c906732b2f1ae151 # v21
continue-on-error: true
with:
workflow: superset-python-integrationtest.yml
run_id: ${{ github.event.workflow_run.id }}
name: database-diagnostics
path: docs/src/data/
- name: Try to download latest diagnostics (for push/dispatch triggers)
if: github.event_name != 'workflow_run'
uses: dawidd6/action-download-artifact@b6e2e70617bc3265edd6dab6c906732b2f1ae151 # v21
continue-on-error: true
with:
workflow: superset-python-integrationtest.yml
name: database-diagnostics
path: docs/src/data/
branch: master
search_artifacts: true
if_no_artifact_found: warn
- name: Use diagnostics artifact if available
working-directory: docs
run: |
if [ -f "src/data/databases-diagnostics.json" ]; then
echo "Using fresh diagnostics from integration tests"
mv src/data/databases-diagnostics.json src/data/databases.json
else
echo "Using committed databases.json (no artifact found)"
fi
- name: yarn build
working-directory: docs
run: |
yarn build
- name: deploy docs
if: github.ref == 'refs/heads/master'
uses: ./.github/actions/github-action-push-to-another-repository
env:
API_TOKEN_GITHUB: ${{ secrets.SUPERSET_SITE_BUILD }}
@@ -110,5 +55,5 @@ jobs:
destination-github-username: "apache"
destination-repository-name: "superset-site"
target-branch: "asf-site"
commit-message: "deploying docs: ${{ github.event.head_commit.message || 'triggered by integration tests' }} (apache/superset@${{ github.event.workflow_run.head_sha || github.sha }})"
commit-message: "deploying docs: ${{ github.event.head_commit.message }} (apache/superset@${{ github.sha }})"
user-email: dev@superset.apache.org

View File

@@ -4,77 +4,30 @@ on:
pull_request:
paths:
- "docs/**"
- "superset/db_engine_specs/**"
- ".github/workflows/superset-docs-verify.yml"
types: [synchronize, opened, reopened, ready_for_review]
workflow_run:
workflows: ["Python-Integration"]
types: [completed]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.workflow_run.head_sha || github.run_id }}
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
linkinator:
# See docs here: https://github.com/marketplace/actions/linkinator
# Only run on pull_request, not workflow_run
if: github.event_name == 'pull_request'
name: Link Checking
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
# Do not bump this linkinator-action version without opening
# an ASF Infra ticket to allow the new version first!
- uses: JustinBeckwith/linkinator-action@af984b9f30f63e796ae2ea5be5e07cb587f1bbd9 # v2.3
continue-on-error: true # This will make the job advisory (non-blocking, no red X)
with:
paths: "**/*.md, **/*.mdx"
linksToSkip: >-
^https://github.com/apache/(superset|incubator-superset)/(pull|issues)/\d+,
^https://github.com/apache/(superset|incubator-superset)/commit/[a-f0-9]+,
superset-frontend/.*CHANGELOG\.md,
http://localhost:8088/,
http://127.0.0.1:3000/,
http://localhost:9001/,
https://charts.bitnami.com/bitnami,
https://www.li.me/,
https://www.fanatics.com/,
https://tails.com/gb/,
https://www.techaudit.info/,
https://avetilearning.com/,
https://www.udemy.com/,
https://trustmedis.com/,
http://theiconic.com.au/,
https://dev.mysql.com/doc/refman/5.7/en/innodb-limits.html,
^https://img\.shields\.io/.*,
https://vkusvill.ru/,
https://www.linkedin.com/in/mark-thomas-b16751158/,
https://theiconic.com.au/,
https://wattbewerb.de/,
https://timbr.ai/,
https://opensource.org/license/apache-2-0,
https://www.plaidcloud.com/
build-on-pr:
# Build docs when PR changes docs/** (uses committed databases.json)
if: github.event_name == 'pull_request'
name: Build (PR trigger)
runs-on: ubuntu-24.04
build-deploy:
name: Build & Deploy
runs-on: ubuntu-20.04
defaults:
run:
working-directory: docs
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Set up Node.js 16
uses: actions/setup-node@v4
with:
node-version-file: './docs/.nvmrc'
node-version: '16'
- name: yarn install
run: |
yarn install --check-cache
@@ -84,51 +37,3 @@ jobs:
- name: yarn build
run: |
yarn build
build-after-tests:
# Build docs after integration tests complete (uses fresh diagnostics)
# Only runs if integration tests succeeded
if: >
github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'success'
name: Build (after integration tests)
runs-on: ubuntu-24.04
defaults:
run:
working-directory: docs
steps:
- name: "Checkout PR head: ${{ github.event.workflow_run.head_sha }}"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
ref: ${{ github.event.workflow_run.head_sha }}
persist-credentials: false
submodules: recursive
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './docs/.nvmrc'
- name: yarn install
run: |
yarn install --check-cache
- name: Download database diagnostics from integration tests
uses: dawidd6/action-download-artifact@b6e2e70617bc3265edd6dab6c906732b2f1ae151 # v21
with:
workflow: superset-python-integrationtest.yml
run_id: ${{ github.event.workflow_run.id }}
name: database-diagnostics
path: docs/src/data/
if_no_artifact_found: 'warning'
- name: Use fresh diagnostics
run: |
if [ -f "src/data/databases-diagnostics.json" ]; then
echo "Using fresh diagnostics from integration tests"
mv src/data/databases-diagnostics.json src/data/databases.json
else
echo "Warning: No diagnostics artifact found, using committed data"
fi
- name: yarn typecheck
run: |
yarn typecheck
- name: yarn build
run: |
yarn build

View File

@@ -3,24 +3,9 @@ name: E2E
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
workflow_dispatch:
inputs:
use_dashboard:
description: 'Use Cypress Dashboard (true/false) [paid service - trigger manually when needed]. You MUST provide a branch and/or PR number below for this to work.'
required: false
default: 'false'
ref:
description: 'The branch or tag to checkout'
required: false
default: ''
pr_id:
description: 'The pull request ID to checkout'
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
@@ -28,21 +13,19 @@ concurrency:
jobs:
cypress-matrix:
# Somehow one test flakes on 24.04 for unknown reasons, this is the only GHA left on 22.04
runs-on: ubuntu-22.04
runs-on: ubuntu-20.04
permissions:
contents: read
pull-requests: read
strategy:
# when one test fails, DO NOT cancel the other
# parallel_id, because this will kill Cypress processes
# containers, because this will kill Cypress processes
# leaving the Dashboard hanging ...
# https://github.com/cypress-io/github-action/issues/48
fail-fast: false
matrix:
parallel_id: [0, 1, 2, 3, 4, 5]
containers: [1, 2, 3]
browser: ["chrome"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -50,11 +33,9 @@ jobs:
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
# Only use dashboard when explicitly requested via workflow_dispatch
USE_DASHBOARD: ${{ github.event.inputs.use_dashboard == 'true' || 'false' }}
services:
postgres:
image: postgres:17-alpine
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -65,204 +46,83 @@ jobs:
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Checkout (pull) ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
if: github.event_name == 'push'
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Checkout (pull_request) ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
if: github.event_name == 'pull_request' || github.event_name == 'pull_request_target'
with:
ref: "refs/pull/${{ github.event.number }}/merge"
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
- name: Check if python or frontend changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python frontend
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: "3.9"
- name: OS dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: apt-get-install
- name: Install python dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
pip-upgrade
pip install -r requirements/testing.txt
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: "16"
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: cypress-install
- name: Run Cypress
if: steps.check.outputs.python || steps.check.outputs.frontend
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
env:
CYPRESS_BROWSER: ${{ matrix.browser }}
PARALLEL_ID: ${{ matrix.parallel_id }}
PARALLELISM: 6
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
NODE_OPTIONS: "--max-old-space-size=4096"
CYPRESS_KEY: YjljODE2MzAtODcwOC00NTA3LWE4NmMtMTU3YmFmMjIzOTRhCg==
with:
run: cypress-run-all ${{ env.USE_DASHBOARD }} ${{ matrix.app_root }}
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
run: cypress-run-all
- name: Upload Artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
uses: actions/upload-artifact@v4
if: failure()
with:
name: screenshots
path: ${{ github.workspace }}/superset-frontend/cypress-base/cypress/screenshots
name: cypress-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}-${{ matrix.parallel_id }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}
playwright-tests:
runs-on: ubuntu-22.04
permissions:
contents: read
pull-requests: read
strategy:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
services:
postgres:
image: postgres:17-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright_testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Required Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
with:
run: playwright-run "${{ matrix.app_root }}"
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
- name: Upload Playwright Artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
if: failure()
with:
path: |
${{ github.workspace }}/superset-frontend/playwright-results/
${{ github.workspace }}/superset-frontend/test-results/
name: playwright-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}

View File

@@ -1,64 +0,0 @@
name: Superset Extensions CLI Package Tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
test-superset-extensions-cli-package:
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: ["previous", "current", "next"]
defaults:
run:
working-directory: superset-extensions-cli
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.superset-extensions-cli
uses: ./.github/actions/setup-backend/
with:
python-version: ${{ matrix.python-version }}
requirements-type: dev
- name: Run pytest with coverage
if: steps.check.outputs.superset-extensions-cli
run: |
pytest --cov=superset_extensions_cli --cov-report=xml --cov-report=term-missing --cov-report=html -v --tb=short
- name: Upload coverage reports to Codecov
if: steps.check.outputs.superset-extensions-cli
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
file: ./coverage.xml
flags: superset-extensions-cli
name: superset-extensions-cli-coverage
fail_ci_if_error: false
- name: Upload HTML coverage report
if: steps.check.outputs.superset-extensions-cli
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: superset-extensions-cli-coverage-html
path: htmlcov/

View File

@@ -1,200 +1,88 @@
name: "Frontend Build CI (unit tests, linting & sanity checks)"
name: Frontend
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
paths:
- "superset-frontend/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset-frontend/**"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
env:
TAG: apache/superset:GHA-${{ github.run_id }}
jobs:
frontend-build:
runs-on: ubuntu-24.04
outputs:
should-run: ${{ steps.check.outputs.frontend }}
runs-on: ubuntu-20.04
steps:
- name: Checkout Code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Check for File Changes
submodules: recursive
- name: Check npm lock file version
run: ./scripts/ci_check_npm_lock_version.sh ./superset-frontend/package-lock.json
- name: Check if frontend changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Build Docker Image
if: steps.check.outputs.frontend
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "git rev-parse --short HEAD"
git rev-parse --short HEAD
echo "git show -s --format=raw HEAD"
git show -s --format=raw HEAD
docker buildx build \
-t $TAG \
--cache-from=type=registry,ref=apache/superset-cache:3.10-slim-trixie \
--target superset-node-ci \
.
- name: Save Docker Image as Artifact
if: steps.check.outputs.frontend
run: |
docker save $TAG | zstd -3 --threads=0 > docker-image.tar.zst
- name: Upload Docker Image Artifact
if: steps.check.outputs.frontend
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh frontend
- name: Setup Node.js
if: steps.check.outcome == 'failure'
uses: actions/setup-node@v4
with:
name: docker-image
path: docker-image.tar.zst
sharded-jest-tests:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
strategy:
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8]
fail-fast: false
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
node-version: "16"
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
name: docker-image
- name: Load Docker Image
run: npm-install
- name: eslint
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
zstd -d < docker-image.tar.zst | docker load
- name: npm run test with coverage
run: |
mkdir -p ${{ github.workspace }}/superset-frontend/coverage
docker run \
-v ${{ github.workspace }}/superset-frontend/coverage:/app/superset-frontend/coverage \
--rm $TAG \
bash -c \
"npm run test -- --coverage --shard=${{ matrix.shard }}/8 --coverageReporters=json"
- name: Upload Coverage Artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: coverage-artifacts-${{ matrix.shard }}
path: superset-frontend/coverage
report-coverage:
needs: [sharded-jest-tests]
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
permissions:
id-token: write
steps:
- name: Checkout Code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
fetch-depth: 0
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Download Coverage Artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
pattern: coverage-artifacts-*
path: coverage/
- name: Reorganize test result reports
run: |
find coverage/
for i in {1..8}; do
mv coverage/coverage-artifacts-${i}/coverage-final.json coverage/coverage-shard-${i}.json
done
shell: bash
- name: Merge Code Coverage
run: npx nyc merge coverage/ merged-output/coverage-summary.json
- name: Upload Code Coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: javascript
use_oidc: true
verbose: true
disable_search: true
files: merged-output/coverage-summary.json
slug: apache/superset
lint-frontend:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: docker-image
- name: Load Docker Image
run: |
zstd -d < docker-image.tar.zst | docker load
- name: lint
run: |
docker run --rm $TAG bash -c \
"npm i && npm run lint"
npm run eslint -- . --quiet
- name: tsc
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
docker run --rm $TAG bash -c \
"npm i && npm run plugins:build && npm run type"
validate-frontend:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: docker-image
- name: Load Docker Image
npm run type
- name: prettier
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
zstd -d < docker-image.tar.zst | docker load
- name: Build Plugins Packages
npm run prettier-check
- name: Build plugins packages
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: npm run plugins:build
- name: Build plugins Storybook
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: npm run plugins:build-storybook
- name: superset-ui/core coverage
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
docker run --rm $TAG bash -c \
"npm run plugins:build"
test-storybook:
needs: frontend-build
if: needs.frontend-build.outputs.should-run == 'true'
runs-on: ubuntu-24.04
steps:
- name: Download Docker Image Artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: docker-image
- name: Load Docker Image
npm run core:cover
- name: unit tests
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: |
zstd -d < docker-image.tar.zst | docker load
- name: Build Storybook and Run Tests
run: |
docker run --rm $TAG bash -c \
"npm run build-storybook && npx playwright install-deps && npx playwright install chromium && npm run test-storybook:ci"
npm run test -- --coverage --silent
# todo: remove this step when fix generator as a project in root jest.config.js
- name: generator-superset unit tests
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend/packages/generator-superset
run: npx jest
- name: Upload code coverage
if: steps.check.outcome == 'failure'
working-directory: ./superset-frontend
run: ../.github/workflows/codecov.sh -c -F javascript

View File

@@ -1,4 +1,4 @@
name: "Helm: lint and test charts"
name: Lint and Test Charts
on:
pull_request:
@@ -13,24 +13,23 @@ concurrency:
jobs:
lint-test:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
fetch-depth: 0
- name: Set up Helm
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
uses: azure/setup-helm@v3
with:
version: v3.16.4
version: v3.5.4
- name: Setup Python
uses: ./.github/actions/setup-backend/
- uses: actions/setup-python@v5
with:
install-superset: 'false'
python-version: "3.9"
- name: Set up chart-testing
uses: ./.github/actions/chart-testing-action

View File

@@ -1,38 +1,23 @@
# This workflow automates the release process for Helm charts.
# The workflow creates a new branch for the release and opens a pull request against the 'gh-pages' branch,
# allowing the changes to be reviewed and merged manually.
name: "Helm: release charts"
name: Release Charts
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
paths:
- "helm/**"
workflow_dispatch:
inputs:
ref:
description: "The branch, tag, or commit SHA to check out"
required: false
default: "master"
jobs:
release:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
ref: ${{ inputs.ref || github.ref_name }}
persist-credentials: true
persist-credentials: false
submodules: recursive
fetch-depth: 0
@@ -42,84 +27,18 @@ jobs:
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Install Helm
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
uses: azure/setup-helm@v3
with:
version: v3.5.4
- name: Add bitnami repo dependency
run: helm repo add bitnami https://charts.bitnami.com/bitnami
- name: Fetch/list all tags
run: |
# Debugging tags
git fetch --tags --force
git tag -d superset-helm-chart-0.13.4 || true
echo "DEBUG TAGS"
git show-ref --tags
- name: Create unique pages branch name
id: vars
run: echo "branch_name=helm-publish-${GITHUB_SHA:0:7}" >> $GITHUB_ENV
- name: Force recreate branch from gh-pages
run: |
# Ensure a clean working directory
git reset --hard
git clean -fdx
git checkout -b local_gha_temp
git submodule update
# Fetch the latest gh-pages branch
git fetch origin gh-pages
# Check out and reset the target branch based on gh-pages
git checkout -B ${{ env.branch_name }} origin/gh-pages
# Remove submodules from the branch
git submodule deinit -f --all
# Force push to the remote branch
git push origin ${{ env.branch_name }} --force
# Return to the original branch
git checkout local_gha_temp
- name: Fetch/list all tags
run: |
git submodule update
cat .github/actions/chart-releaser-action/action.yml
- name: Run chart-releaser
uses: ./.github/actions/chart-releaser-action
with:
version: v1.6.0
charts_dir: helm
mark_as_latest: false
pages_branch: ${{ env.branch_name }}
env:
CR_TOKEN: "${{ github.token }}"
CR_RELEASE_NAME_TEMPLATE: "superset-helm-chart-{{ .Version }}"
- name: Open Pull Request
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const branchName = '${{ env.branch_name }}';
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/');
if (!branchName) {
throw new Error("Branch name is not defined.");
}
const pr = await github.rest.pulls.create({
owner,
repo,
title: `Helm chart release for ${branchName}`,
head: branchName,
base: "gh-pages", // Adjust if the target branch is different
body: `This PR releases Helm charts to the gh-pages branch.`,
});
core.info(`Pull request created: ${pr.data.html_url}`);
env:
BRANCH_NAME: ${{ env.branch_name }}

View File

@@ -1,142 +0,0 @@
name: Playwright Experimental Tests
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
workflow_dispatch:
inputs:
ref:
description: 'The branch or tag to checkout'
required: false
default: ''
pr_id:
description: 'The pull request ID to checkout'
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
# NOTE: Required Playwright tests are in superset-e2e.yml (E2E / playwright-tests)
# This workflow contains only experimental tests that run in shadow mode
playwright-tests-experimental:
runs-on: ubuntu-22.04
continue-on-error: true
permissions:
contents: read
pull-requests: read
strategy:
fail-fast: false
matrix:
browser: ["chromium"]
app_root: ["", "/app/prefix"]
env:
SUPERSET_ENV: development
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
PYTHONPATH: ${{ github.workspace }}
REDIS_PORT: 16379
GITHUB_TOKEN: ${{ github.token }}
services:
postgres:
image: postgres:17-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
ports:
- 15432:5432
redis:
image: redis:7-alpine
ports:
- 16379:6379
steps:
# -------------------------------------------------------
# Conditional checkout based on context (same as Cypress workflow)
- name: Checkout for push or pull_request event
if: github.event_name == 'push' || github.event_name == 'pull_request'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
submodules: recursive
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Checkout using ref (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.ref != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: ${{ github.event.inputs.ref }}
submodules: recursive
- name: Checkout using PR ID (workflow_dispatch)
if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_id != ''
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
ref: refs/pull/${{ github.event.inputs.pr_id }}/merge
submodules: recursive
# -------------------------------------------------------
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python || steps.check.outputs.frontend
- name: Setup postgres
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Import test data
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright_testdata
- name: Setup Node.js
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version-file: './superset-frontend/.nvmrc'
- name: Install npm dependencies
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: Build javascript packages
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: build-instrumented-assets
- name: Install Playwright
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: playwright-install
- name: Run Playwright (Experimental Tests)
if: steps.check.outputs.python || steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
env:
NODE_OPTIONS: "--max-old-space-size=4096"
with:
run: playwright-run "${{ matrix.app_root }}" experimental/
- name: Set safe app root
if: failure()
id: set-safe-app-root
run: |
APP_ROOT="${{ matrix.app_root }}"
SAFE_APP_ROOT=${APP_ROOT//\//_}
echo "safe_app_root=$SAFE_APP_ROOT" >> $GITHUB_OUTPUT
- name: Upload Playwright Artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
if: failure()
with:
path: |
${{ github.workspace }}/superset-frontend/playwright-results/
${{ github.workspace }}/superset-frontend/test-results/
name: playwright-experimental-artifact-${{ github.run_id }}-${{ github.job }}-${{ matrix.browser }}--${{ steps.set-safe-app-root.outputs.safe_app_root }}

View File

@@ -4,8 +4,7 @@ name: Python-Integration
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
@@ -15,9 +14,10 @@ concurrency:
jobs:
test-mysql:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -26,16 +26,11 @@ jobs:
mysql+mysqldb://superset:superset@127.0.0.1:13306/superset?charset=utf8mb4&binary_prefix=true
services:
mysql:
image: mysql:8.0
image: mysql:5.7
env:
MYSQL_ROOT_PASSWORD: root
ports:
- 13306:3306
options: >-
--health-cmd="mysqladmin ping --silent"
--health-interval=10s
--health-timeout=5s
--health-retries=5
redis:
image: redis:7-alpine
options: --entrypoint redis-server
@@ -43,76 +38,51 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup MySQL
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-mysql
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-mysql
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (MySQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: python,mysql
verbose: true
use_oidc: true
slug: apache/superset
- name: Generate database diagnostics for docs
if: steps.check.outputs.python
env:
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
SUPERSET__SQLALCHEMY_DATABASE_URI: |
mysql+mysqldb://superset:superset@127.0.0.1:13306/superset?charset=utf8mb4&binary_prefix=true
if: steps.check.outcome == 'failure'
run: |
python -c "
import json
from superset.app import create_app
from superset.db_engine_specs.lib import generate_yaml_docs
app = create_app()
with app.app_context():
docs = generate_yaml_docs()
# Wrap in the expected format
output = {
'generated': '$(date -Iseconds)',
'databases': docs
}
with open('databases-diagnostics.json', 'w') as f:
json.dump(output, f, indent=2, default=str)
print(f'Generated diagnostics for {len(docs)} databases')
"
- name: Upload database diagnostics artifact
if: steps.check.outputs.python
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: database-diagnostics
path: databases-diagnostics.json
retention-days: 7
bash .github/workflows/codecov.sh -c -F python -F mysql
test-postgres:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["current", "previous", "next"]
python-version: ["3.9", "3.10"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -120,7 +90,7 @@ jobs:
SUPERSET__SQLALCHEMY_DATABASE_URI: postgresql+psycopg2://superset:superset@127.0.0.1:15432/superset
services:
postgres:
image: postgres:17-alpine
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -134,47 +104,51 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Setup Postgres
if: steps.check.outputs.python
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: python,postgres
verbose: true
use_oidc: true
slug: apache/superset
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F postgres
test-sqlite:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -190,38 +164,42 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
# sqlite needs this working directory
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
mkdir ${{ github.workspace }}/.temp
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python integration tests (SQLite)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: python,sqlite
verbose: true
use_oidc: true
slug: apache/superset
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F sqlite

View File

@@ -0,0 +1,86 @@
# Python Misc unit tests
name: Python Misc
on:
push:
branches:
- 'master'
paths:
- "superset/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset/**"
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
python-lint:
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check if python changes are present
id: check
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: "requirements/testing.txt"
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
- name: pylint
if: steps.check.outcome == 'failure'
# `-j 0` run Pylint in parallel
run: pylint -j 0 superset
babel-extract:
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: "requirements/base.txt"
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/base.txt
- name: Test babel extraction
run: flask fab babel-extract --target superset/translations --output superset/translations/messages.pot --config superset/translations/babel.cfg -k _,__,t,tn,tct

View File

@@ -4,10 +4,13 @@ name: Python Presto/Hive
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
paths:
- "superset/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset/**"
# cancel previous workflow jobs for PRs
concurrency:
@@ -16,9 +19,10 @@ concurrency:
jobs:
test-postgres-presto:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -27,7 +31,7 @@ jobs:
SUPERSET__SQLALCHEMY_EXAMPLES_URI: presto://localhost:15433/memory/default
services:
postgres:
image: postgres:17-alpine
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -50,46 +54,51 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python == 'true'
- name: Setup Postgres
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: |
echo "${{ steps.check.outputs.python }}"
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: python,presto
verbose: true
use_oidc: true
slug: apache/superset
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F presto
test-postgres-hive:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
env:
PYTHONPATH: ${{ github.workspace }}
SUPERSET_CONFIG: tests.integration_tests.superset_test_config
@@ -99,7 +108,7 @@ jobs:
UPLOAD_FOLDER: /tmp/.superset/uploads/
services:
postgres:
image: postgres:17-alpine
image: postgres:15-alpine
env:
POSTGRES_USER: superset
POSTGRES_PASSWORD: superset
@@ -113,46 +122,51 @@ jobs:
- 16379:6379
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Create csv upload directory
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: sudo mkdir -p /tmp/.superset/uploads
- name: Give write access to the csv upload directory
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: sudo chown -R $USER:$USER /tmp/.superset
- name: Start hadoop and hive
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: docker compose -f scripts/databases/hive/docker-compose.yml up -d
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
- name: Setup Postgres
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
run: setup-postgres
- name: Start Celery worker
if: steps.check.outputs.python
uses: ./.github/actions/cached-dependencies
with:
run: celery-worker
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
setup-postgres
- name: Run celery
if: steps.check.outcome == 'failure'
run: celery --app=superset.tasks.celery_app:app worker -Ofair -c 2 &
- name: Python unit tests (PostgreSQL)
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
run: |
pip install -e .[hive]
./scripts/python_tests.sh -m 'chart_data_flow or sql_json_flow'
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
with:
flags: python,hive
verbose: true
use_oidc: true
slug: apache/superset
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F hive

View File

@@ -4,10 +4,13 @@ name: Python-Unit
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
paths:
- "superset/**"
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
paths:
- "superset/**"
# cancel previous workflow jobs for PRs
concurrency:
@@ -16,48 +19,50 @@ concurrency:
jobs:
unit-tests:
runs-on: ubuntu-24.04
permissions:
id-token: write
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["previous", "current", "next"]
python-version: ["3.9", "3.10"]
env:
PYTHONPATH: ${{ github.workspace }}
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
- name: Check if python changes are present
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
env:
GITHUB_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
continue-on-error: true
run: ./scripts/ci_check_no_file_changes.sh python
- name: Setup Python
uses: ./.github/actions/setup-backend/
if: steps.check.outputs.python
if: steps.check.outcome == 'failure'
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Python unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear --maxfail=50
- name: Python 100% coverage unit tests
if: steps.check.outputs.python
env:
SUPERSET_TESTENV: true
SUPERSET_SECRET_KEY: not-a-secret
run: |
pytest --durations-min=0.5 --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100
- name: Upload code coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5
cache: 'pip'
cache-dependency-path: 'requirements/testing.txt'
# TODO: separated requirements.txt file just for unit tests
- name: Install dependencies
if: steps.check.outcome == 'failure'
uses: ./.github/actions/cached-dependencies
with:
flags: python,unit
verbose: true
use_oidc: true
slug: apache/superset
run: |
apt-get-install
pip-upgrade
pip install wheel
pip install -r requirements/testing.txt
mkdir ${{ github.workspace }}/.temp
- name: Python unit tests
if: steps.check.outcome == 'failure'
env:
SUPERSET_TESTENV: true
run: |
pytest --durations-min=0.5 --cov-report= --cov=superset ./tests/common ./tests/unit_tests --cache-clear
- name: Upload code coverage
if: steps.check.outcome == 'failure'
run: |
bash .github/workflows/codecov.sh -c -F python -F unit

View File

@@ -3,8 +3,7 @@ name: Translations
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
pull_request:
types: [synchronize, opened, reopened, ready_for_review]
@@ -14,58 +13,48 @@ concurrency:
cancel-in-progress: true
jobs:
frontend-check-translations:
runs-on: ubuntu-24.04
frontend-check:
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Node.js
if: steps.check.outputs.frontend
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: '16'
- name: Install dependencies
if: steps.check.outputs.frontend
uses: ./.github/actions/cached-dependencies
with:
run: npm-install
- name: lint
if: steps.check.outputs.frontend
working-directory: ./superset-frontend
run: |
npm run build-translation
npm run check-translation
babel-extract:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.9"]
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Check for file changes
id: check
uses: ./.github/actions/change-detector/
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Python
if: steps.check.outputs.python
uses: ./.github/actions/setup-backend/
- name: Install msgcat
run: sudo apt update && sudo apt install gettext
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
uses: ./.github/actions/cached-dependencies
with:
run: |
apt-get-install
pip-upgrade
pip install -r requirements/base.txt
- name: Test babel extraction
if: steps.check.outputs.python
run: ./scripts/translations/babel_update.sh
run: ./scripts/babel_update.sh

View File

@@ -2,8 +2,7 @@ name: WebSocket server
on:
push:
branches:
- "master"
- "[0-9].[0-9]*"
- 'master'
paths:
- "superset-websocket/**"
pull_request:
@@ -18,10 +17,10 @@ concurrency:
jobs:
app-checks:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install dependencies

View File

@@ -1,56 +0,0 @@
name: SupersetBot Workflow
on:
issue_comment:
types: [created, edited]
# Making the workflow testable since `issue_comment` only triggers on
# the default branch
workflow_dispatch:
inputs:
comment_body:
description: 'Comment Body'
required: true
type: string
jobs:
supersetbot:
runs-on: ubuntu-24.04
if: >
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@supersetbot'))
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Quickly add thumbs up!
if: github.event_name == 'issue_comment' && contains(github.event.comment.body, '@supersetbot')
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/')
await github.rest.reactions.createForIssueComment({
owner,
repo,
comment_id: context.payload.comment.id,
content: '+1'
});
- name: "Checkout ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute custom Node.js script
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_ISSUE_NUMBER: ${{ github.event.issue.number }}
COMMENT_BODY: ${{ github.event.comment.body }}${{ github.event.inputs.comment_body }}
run: |
supersetbot run "$COMMENT_BODY"

View File

@@ -1,140 +0,0 @@
name: Publish a Release
on:
release:
types: [published, edited]
# Can be triggered manually
workflow_dispatch:
inputs:
release:
required: true
description: The version to generate
git-ref:
required: true
description: The git reference to checkout prior to running the docker build
force-latest:
required: true
type: choice
default: 'false'
description: Whether to force a latest tag on the release
options:
- 'true'
- 'false'
jobs:
config:
runs-on: ubuntu-24.04
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
- name: "Check for secrets"
id: check
shell: bash
run: |
if [ -n "${DOCKERHUB_USER}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
DOCKERHUB_USER: ${{ (secrets.DOCKERHUB_USER != '' && secrets.DOCKERHUB_TOKEN != '') || '' }}
docker-release:
needs: config
if: needs.config.outputs.has-secrets
name: docker-release
runs-on: ubuntu-24.04
strategy:
matrix:
build_preset: ["dev", "lean", "py310", "websocket", "dockerize", "py311", "py312"]
fail-fast: false
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 0
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker
with:
dockerhub-user: ${{ secrets.DOCKERHUB_USER }}
dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }}
install-docker-compose: "false"
build: "true"
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 20
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Execute custom Node.js script
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INPUT_RELEASE: ${{ github.event.inputs.release }}
INPUT_FORCE_LATEST: ${{ github.event.inputs.force-latest }}
INPUT_GIT_REF: ${{ github.event.inputs.git-ref }}
run: |
RELEASE="${{ github.event.release.tag_name }}"
FORCE_LATEST=""
EVENT="${{github.event_name}}"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# in the case of a manually-triggered run, read release from input
RELEASE="${INPUT_RELEASE}"
if [ "${INPUT_FORCE_LATEST}" = "true" ]; then
FORCE_LATEST="--force-latest"
fi
git checkout "${INPUT_GIT_REF}"
EVENT="release"
fi
supersetbot docker \
--push \
--preset ${{ matrix.build_preset }} \
--context "$EVENT" \
--context-ref "$RELEASE" $FORCE_LATEST \
--platform "linux/arm64" \
--platform "linux/amd64"
# Returning to master to support closing setup-supersetbot
git checkout master
update-prs-with-release-info:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
permissions:
contents: read
pull-requests: write
steps:
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 0
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 20
- name: Setup supersetbot
uses: ./.github/actions/setup-supersetbot/
- name: Label the PRs with the right release-related labels
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INPUT_RELEASE: ${{ github.event.inputs.release }}
run: |
export GITHUB_ACTOR=""
git fetch --all --tags
git checkout master
RELEASE="${{ github.event.release.tag_name }}"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# in the case of a manually-triggered run, read release from input
RELEASE="${INPUT_RELEASE}"
fi
supersetbot release-label $RELEASE

View File

@@ -4,14 +4,10 @@ on:
push:
branches:
- master
- "[0-9].[0-9]*"
permissions:
contents: read
jobs:
config:
runs-on: ubuntu-24.04
runs-on: "ubuntu-latest"
outputs:
has-secrets: ${{ steps.check.outputs.has-secrets }}
steps:
@@ -19,33 +15,31 @@ jobs:
id: check
shell: bash
run: |
if [ -n "${GSHEET_KEY}" ]; then
if [ -n "${{ (secrets.GSHEET_KEY != '' ) || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
env:
GSHEET_KEY: ${{ (secrets.GSHEET_KEY != '' ) || '' }}
process-and-upload:
needs: config
if: needs.config.outputs.has-secrets
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
name: Generate Reports
steps:
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version-file: './superset-frontend/.nvmrc'
node-version: '16'
- name: Install Dependencies
run: npm ci
run: npm install
working-directory: ./superset-frontend
- name: Run Script
env:
SPREADSHEET_ID: "1oABNnzxJYzwUrHjr_c9wfYEq9dFL1ScVof9LlaAdxvo"
SPREADSHEET_ID: '1oABNnzxJYzwUrHjr_c9wfYEq9dFL1ScVof9LlaAdxvo'
SERVICE_ACCOUNT_KEY: ${{ secrets.GSHEET_KEY }}
run: npm run lint-stats
continue-on-error: true

View File

@@ -0,0 +1,47 @@
name: Update Lockfiles for Dependabot Monorepo PRs
on:
pull_request:
paths:
- 'superset-frontend/packages/**/package.json'
- 'superset-frontend/plugins/**/package.json'
types: [opened, synchronize, reopened]
issue_comment:
types: [created]
# cancel previous workflow jobs for PRs
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
cancel-in-progress: true
jobs:
update-lock-file:
runs-on: ubuntu-latest
if: >
(github.event_name == 'pull_request' && github.event.pull_request.user.login == 'dependabot[bot]') ||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@supersetbot trigger-dependabot-lockfile') && github.event.issue.pull_request)
defaults:
run:
working-directory: superset-frontend
steps:
- name: Checkout Code
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref || github.head_ref }} # Checkout the branch that made the PR or the comment's PR branch
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '16'
- name: Install Dependencies and Update Lock File
run: |
npm install
- name: Commit and Push Changes
run: |
git config user.name "GitHub-Actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add package-lock.json
# Push the changes back to the branch if they exist, and pass if there are no changes
git diff --staged --quiet || (git commit -m "Update lock file for Dependabot PR" -a && git push https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} ${{ github.event.pull_request.head.ref || github.head_ref }})

View File

@@ -6,13 +6,13 @@ on:
jobs:
welcome:
runs-on: ubuntu-24.04
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Welcome Message
uses: actions/first-interaction@v3
uses: actions/first-interaction@v1
continue-on-error: true
with:
repo-token: ${{ github.token }}

47
.gitignore vendored
View File

@@ -21,7 +21,6 @@
*.swp
__pycache__
.aider*
.local
.cache
.bento*
@@ -33,7 +32,6 @@ cover
.env
.envrc
.idea
.roo
.mypy_cache
.python-version
.tox
@@ -44,7 +42,7 @@ _modules
_static
build
app.db
*.egg-info/
apache_superset.egg-info/
changelog.sh
dist
dump.rdb
@@ -52,6 +50,7 @@ env
venv*
env_py3
envpy3
env36
local_config.py
/superset_config.py
/superset_text.yml
@@ -61,19 +60,18 @@ tmp
rat-results.txt
superset/app/
superset-websocket/config.json
.direnv
*.log
# Node.js, webpack artifacts, storybook
*.entry.js
*.js.map
node_modules
npm-debug.log*
superset/static/*
superset/static/assets/*
!superset/static/assets/.gitkeep
superset/static/uploads/*
!superset/static/uploads/.gitkeep
superset/static/assets
superset/static/version_info.json
superset-frontend/**/esm/*
superset-frontend/**/lib/*
superset-frontend/**/storybook-static/*
superset-frontend/migration-storybook.log
yarn-error.log
*.map
*.min.js
@@ -91,7 +89,6 @@ scripts/*.zip
# IntelliJ
*.iml
venv
.venv
@eaDir/
# PyCharm
@@ -107,35 +104,11 @@ ghostdriver.log
testCSV.csv
.terser-plugin-cache/
apache-superset-*.tar.gz*
apache_superset-*.tar.gz*
release.json
# Translation-related files
# these json files are generated by ./scripts/po2json.sh
superset/translations/**/messages.json
# these mo binary files are generated by `pybabel compile`
superset/translations/**/messages.mo
# Translation binaries
messages.mo
docker/requirements-local.txt
cache/
docker/*local*
docker/superset-websocket/config.json
docker-compose.override.yml
.temp_cache
# Jest test report
test-report.html
superset/static/stats/statistics.html
# LLM-related
CLAUDE.local.md
PROJECT.md
.aider*
.claude_rc*
.claude/settings.local.json
.env.local
oxc-custom-build/
*.code-workspace
*.duckdb

View File

@@ -16,18 +16,33 @@
#
repos:
- repo: https://github.com/MarcoGorelli/auto-walrus
rev: 0.3.4
rev: v0.2.2
hooks:
- id: auto-walrus
- repo: https://github.com/asottile/pyupgrade
rev: v3.4.0
hooks:
- id: pyupgrade
args:
- --py39-plus
- repo: https://github.com/hadialqattan/pycln
rev: v2.1.2
hooks:
- id: pycln
args:
- --disable-all-dunder-policy
- --exclude=superset/config.py
- --extend-exclude=tests/integration_tests/superset_test_config.*.py
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
rev: v1.3.0
hooks:
- id: mypy
name: mypy (main)
args: [--check-untyped-defs]
exclude: ^superset-extensions-cli/
additional_dependencies: [
types-cachetools,
types-simplejson,
types-python-dateutil,
types-requests,
@@ -41,59 +56,33 @@ repos:
types-paramiko,
types-Markdown,
]
- id: mypy
name: mypy (superset-extensions-cli)
args: [--check-untyped-defs]
files: ^superset-extensions-cli/
- repo: https://github.com/peterdemin/pip-compile-multi
rev: v2.6.2
hooks:
- id: pip-compile-multi-verify
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
rev: v4.4.0
hooks:
- id: check-docstring-first
- id: check-added-large-files
exclude: ^.*\.(geojson)$|^docs/static/img/screenshots/.*|^superset-frontend/CHANGELOG\.md$|^superset/examples/.*/data\.parquet$
exclude: \.(geojson)$
- id: check-yaml
exclude: ^helm/superset/templates/
- id: debug-statements
- id: end-of-file-fixer
exclude: .*/lerna\.json$|^docs/static/img/logos/
- id: trailing-whitespace
exclude: ^.*\.(snap)
args: ["--markdown-linebreak-ext=md"]
- repo: local
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: prettier-frontend
name: prettier (frontend)
entry: bash -c 'cd superset-frontend && for file in "$@"; do npx prettier --write "${file#superset-frontend/}"; done'
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx|css|scss|sass|json)$
- repo: local
- id: black
language_version: python3
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0 # Use the sha or tag you want to point at
hooks:
- id: oxlint-frontend
name: oxlint (frontend)
entry: ./scripts/oxlint.sh
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx)$
- id: custom-rules-frontend
name: custom rules (frontend)
entry: ./scripts/check-custom-rules.sh
language: system
pass_filenames: true
files: ^superset-frontend/.*\.(js|jsx|ts|tsx)$
- id: eslint-docs
name: eslint (docs)
entry: bash -c 'cd docs && FILES=$(printf "%s\n" "$@" | sed "s|^docs/||" | tr "\n" " ") && yarn eslint --fix --quiet $FILES'
language: system
pass_filenames: true
files: ^docs/.*\.(js|jsx|ts|tsx)$
- id: type-checking-frontend
name: Type-Checking (Frontend)
entry: ./scripts/check-type.js package=superset-frontend excludeDeclarationDir=cypress-base
language: system
files: ^superset-frontend\/.*\.(js|jsx|ts|tsx)$
exclude: ^superset-frontend/cypress-base\/
require_serial: true
- id: prettier
args: ["--ignore-path=./superset-frontend/.prettierignore"]
files: "superset-frontend"
# blacklist unsafe functions like make_url (see #19526)
- repo: https://github.com/skorokithakis/blacklist-pre-commit-hook
rev: e2f070289d8eddcaec0b580d3bde29437e7c8221
@@ -101,60 +90,7 @@ repos:
- id: blacklist
args: ["--blacklisted-names=make_url", "--ignore=tests/"]
- repo: https://github.com/norwoodj/helm-docs
rev: v1.14.2
rev: v1.11.0
hooks:
- id: helm-docs
files: helm
verbose: false
args: ["--log-level", "error"]
# Using local hooks ensures ruff version matches requirements/development.txt
- repo: local
hooks:
- id: ruff-format
name: ruff-format
entry: ruff format
language: system
types: [python]
- id: ruff
name: ruff
entry: ruff check --fix --show-fixes
language: system
types: [python]
- repo: local
hooks:
- id: pylint
name: pylint with custom Superset plugins
entry: bash
language: system
types: [python]
exclude: ^(tests/|superset/migrations/|scripts/|RELEASING/|docker/)
args:
- -c
- |
TARGET_BRANCH=${GITHUB_BASE_REF:-master}
# Only fetch if we're not in CI (CI already has all refs)
if [ -z "$CI" ]; then
git fetch --no-recurse-submodules origin "$TARGET_BRANCH" 2>/dev/null || true
fi
BASE=$(git merge-base origin/"$TARGET_BRANCH" HEAD 2>/dev/null) || BASE="HEAD"
files=$(git diff --name-only --diff-filter=ACM "$BASE"..HEAD 2>/dev/null | grep '^superset/.*\.py$' || true)
if [ -n "$files" ]; then
pylint --rcfile=.pylintrc --load-plugins=superset.extensions.pylint --reports=no $files
else
echo "No Python files to lint."
fi
- id: db-engine-spec-metadata
name: database engine spec metadata validation
entry: python superset/db_engine_specs/lint_metadata.py --strict
language: system
files: ^superset/db_engine_specs/.*\.py$
exclude: ^superset/db_engine_specs/(base|lib|lint_metadata|__init__)\.py$
pass_filenames: false
- repo: local
hooks:
- id: feature-flags-sync
name: feature flags documentation sync
entry: bash -c 'python scripts/extract_feature_flags.py > docs/static/feature-flags.json.tmp && if ! diff -q docs/static/feature-flags.json docs/static/feature-flags.json.tmp > /dev/null 2>&1; then mv docs/static/feature-flags.json.tmp docs/static/feature-flags.json && echo "Updated docs/static/feature-flags.json" && exit 1; else rm docs/static/feature-flags.json.tmp; fi'
language: system
files: ^superset/config\.py$
pass_filenames: false

View File

@@ -36,7 +36,7 @@ persistent=yes
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=superset.extensions.pylint
load-plugins=
# Use multiple processes to speed up Pylint.
jobs=2
@@ -52,9 +52,33 @@ extension-pkg-whitelist=pyarrow
[MESSAGES CONTROL]
disable=all
enable=json-import,disallowed-sql-import,consider-using-transaction
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=
useless-suppression,
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=
cyclic-import, # re-enable once this no longer raises false positives
missing-docstring,
duplicate-code,
unspecified-encoding,
too-many-instance-attributes # re-enable once this no longer raises false positives
[REPORTS]
@@ -147,7 +171,7 @@ max-nested-blocks=5
[FORMAT]
# Maximum number of characters on a single line.
max-line-length=100
max-line-length=90
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$

View File

@@ -11,7 +11,6 @@
.nvmrc
.prettierrc
.rat-excludes
.swcrc
.*log
.*pyc
.*lock
@@ -33,8 +32,6 @@ apache_superset.egg-info
# json and csv in general cannot have comments
.*json
.*csv
# jinja templates often need to be as-is
.*j2
# Generated doc files
env/*
docs/.htaccess*
@@ -64,25 +61,8 @@ tsconfig.tsbuildinfo
generator-superset/*
temporary_superset_ui/*
# skip license checks for auto-generated test snapshots
.*snap
# docs third-party logos (database logos, org logos, etc.)
databases/*
logos/*
# docs-related
erd.puml
erd.svg
intro_header.txt
TODO.md
# for LLMs
llm-context.md
llms.txt
AGENTS.md
LLMS.md
CLAUDE.md
CURSOR.md
GEMINI.md
GPT.md
# docs overrides for third party logos we don't have the rights to
google-big-query.svg
google-sheets.svg
postgresql.svg
snowflake.svg

269
AGENTS.md
View File

@@ -1,269 +0,0 @@
# LLM Context Guide for Apache Superset
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
## ⚠️ CRITICAL: Always Run Pre-commit Before Pushing
**ALWAYS run `pre-commit run --all-files` before pushing commits.** CI will fail if pre-commit checks don't pass. This is non-negotiable.
```bash
# Stage your changes first
git add .
# Run pre-commit on all files
pre-commit run --all-files
# If there are auto-fixes, stage them and commit
git add .
git commit --amend # or new commit
```
Common pre-commit failures:
- **Formatting** - black, prettier, eslint will auto-fix
- **Type errors** - mypy failures need manual fixes
- **Linting** - ruff, pylint issues need manual fixes
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
**These migrations are actively happening - avoid deprecated patterns:**
### Frontend Modernization
- **NO `any` types** - Use proper TypeScript types
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
- **Use @superset-ui/core** - Don't import Ant Design directly, prefer Ant Design component wrappers from @superset-ui/core/components
- **Use antd theming tokens** - Prefer antd tokens over legacy theming tokens
- **Avoid custom css and styles** - Follow antd best practices and avoid styling and custom CSS whenever possible
### Testing Strategy Migration
- **Prefer unit tests** over integration tests
- **Prefer integration tests** over end-to-end tests
- **Use Playwright for E2E tests** - Migrating from Cypress
- **Cypress is deprecated** - Will be removed once migration is completed
- **Use Jest + React Testing Library** for component testing
- **Use `test()` instead of `describe()`** - Follow [avoid nesting when testing](https://kentcdodds.com/blog/avoid-nesting-when-youre-testing) principles
### Backend Type Safety
- **Add type hints** - All new Python code needs proper typing
- **MyPy compliance** - Run `pre-commit run mypy` to validate
- **SQLAlchemy typing** - Use proper model annotations
### UUID Migration
- **Prefer UUIDs over auto-incrementing IDs** - New models should use UUID primary keys
- **External API exposure** - Use UUIDs in public APIs instead of internal integer IDs
- **Existing models** - Add UUID fields alongside integer IDs for gradual migration
## Key Directories
```
superset/
├── superset/ # Python backend (Flask, SQLAlchemy)
│ ├── views/api/ # REST API endpoints
│ ├── models/ # Database models
│ └── connectors/ # Database connections
├── superset-frontend/src/ # React TypeScript frontend
│ ├── components/ # Reusable components
│ ├── explore/ # Chart builder
│ ├── dashboard/ # Dashboard interface
│ └── SqlLab/ # SQL editor
├── superset-frontend/packages/
│ └── superset-ui-core/ # UI component library (USE THIS)
├── tests/ # Python/integration tests
├── docs/ # Documentation (UPDATE FOR CHANGES)
└── UPDATING.md # Breaking changes log
```
## Code Standards
### TypeScript Frontend
- **Avoid `any` types** - Use proper TypeScript, reuse existing types
- **Functional components** with hooks
- **@superset-ui/core** for UI components (not direct antd)
- **Jest** for testing (NO Enzyme)
- **Redux** for global state where it exists, hooks for local
### Python Backend
- **Type hints required** for all new code
- **MyPy compliant** - run `pre-commit run mypy`
- **SQLAlchemy models** with proper typing
- **pytest** for testing
### Apache License Headers
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
- **LLM instruction files are excluded** - Files like AGENTS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
### Code Comments
- **Avoid time-specific language** - Don't use words like "now", "currently", "today" in code comments as they become outdated
- **Write timeless comments** - Comments should remain accurate regardless of when they're read
## Documentation Requirements
- **docs/**: Update for any user-facing changes
- **UPDATING.md**: Add breaking changes here
- **Docstrings**: Required for new functions/classes
## Developer Portal: Storybook-to-MDX Documentation
The Developer Portal auto-generates MDX documentation from Storybook stories. **Stories are the single source of truth.**
### Core Philosophy
- **Fix issues in the STORY, not the generator** - When something doesn't render correctly, update the story file first
- **Generator should be lightweight** - It extracts and passes through data; avoid special cases
- **Stories define everything** - Props, controls, galleries, examples all come from story metadata
### Story Requirements for Docs Generation
- Use `export default { title: '...' }` (inline), not `const meta = ...; export default meta;`
- Name interactive stories `Interactive${ComponentName}` (e.g., `InteractiveButton`)
- Define `args` for default prop values
- Define `argTypes` at the story level (not meta level) with control types and descriptions
- Use `parameters.docs.gallery` for size×style variant grids
- Use `parameters.docs.sampleChildren` for components that need children
- Use `parameters.docs.liveExample` for custom live code blocks
- Use `parameters.docs.staticProps` for complex object props that can't be parsed inline
### Generator Location
- Script: `docs/scripts/generate-superset-components.mjs`
- Wrapper: `docs/src/components/StorybookWrapper.jsx`
- Output: `docs/developer_portal/components/`
## Architecture Patterns
### Security & Features
- **RBAC**: Role-based access via Flask-AppBuilder
- **Feature flags**: Control feature rollouts
- **Row-level security**: SQL-based data access control
## Test Utilities
### Python Test Helpers
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
- **`@with_config`** - Config mocking decorator
- **`@with_feature_flags`** - Feature flag testing
- **`login_as()`, `login_as_admin()`** - Authentication helpers
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
### TypeScript Test Helpers
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
- **`createWrapper()`** - Redux/Router/Theme wrapper
- **`selectOption()`** - Select component helper
- **React Testing Library** - NO Enzyme (removed)
### Test Database Patterns
- **Mock patterns**: Use `MagicMock()` for config objects, avoid `AsyncMock` for synchronous code
- **API tests**: Update expected columns when adding new model fields
### Running Tests
```bash
# Frontend
npm run test # All tests
npm run test -- filename.test.tsx # Single file
# E2E Tests (Playwright - NEW)
npm run playwright:test # All Playwright tests
npm run playwright:ui # Interactive UI mode
npm run playwright:headed # See browser during tests
npx playwright test tests/auth/login.spec.ts # Single file
npm run playwright:debug tests/auth/login.spec.ts # Debug specific file
# E2E Tests (Cypress - DEPRECATED)
cd superset-frontend/cypress-base
npm run cypress-run-chrome # All Cypress tests (headless)
npm run cypress-debug # Interactive Cypress UI
# Backend
pytest # All tests
pytest tests/unit_tests/specific_test.py # Single file
pytest tests/unit_tests/ # Directory
# If pytest fails with database/setup issues, ask the user to run test environment setup
```
## Environment Validation
**Quick Setup Check (run this first):**
```bash
# Verify Superset is running
curl -f http://localhost:8088/health || echo "❌ Setup required - see https://superset.apache.org/docs/contributing/development#working-with-llms"
```
**If health checks fail:**
"It appears you aren't set up properly. Please refer to the [Working with LLMs](https://superset.apache.org/docs/contributing/development#working-with-llms) section in the development docs for setup instructions."
**Key Project Files:**
- `superset-frontend/package.json` - Frontend build scripts (`npm run dev` on port 9000, `npm run test`, `npm run lint`)
- `pyproject.toml` - Python tooling (ruff, mypy configs)
- `requirements/` folder - Python dependencies (base.txt, development.txt)
## SQLAlchemy Query Best Practices
- **Use negation operator**: `~Model.field` instead of `== False` to avoid ruff E712 errors
- **Example**: `~Model.is_active` instead of `Model.is_active == False`
## Pull Request Guidelines
**When creating pull requests:**
1. **Read the current PR template**: Always check `.github/PULL_REQUEST_TEMPLATE.md` for the latest format
2. **Use the template sections**: Include all sections from the template (SUMMARY, BEFORE/AFTER, TESTING INSTRUCTIONS, ADDITIONAL INFORMATION)
3. **Follow PR title conventions**: Use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
- Format: `type(scope): description`
- Example: `fix(dashboard): load charts correctly`
- Types: `fix`, `feat`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`
**Important**: Always reference the actual template file at `.github/PULL_REQUEST_TEMPLATE.md` instead of using cached content, as the template may be updated over time.
## Pre-commit Validation
**Use pre-commit hooks for quality validation:**
```bash
# Install hooks
pre-commit install
# IMPORTANT: Stage your changes first!
git add . # Pre-commit only checks staged files
# Quick validation (faster than --all-files)
pre-commit run # Staged files only
pre-commit run mypy # Python type checking
pre-commit run prettier # Code formatting
pre-commit run eslint # Frontend linting
```
**Important pre-commit usage notes:**
- **Stage files first**: Run `git add .` before `pre-commit run` to check only changed files (much faster)
- **Virtual environment**: Activate your Python virtual environment before running pre-commit
```bash
# Common virtual environment locations (yours may differ):
source .venv/bin/activate # if using .venv
source venv/bin/activate # if using venv
source ~/venvs/superset/bin/activate # if using a central location
```
If you get a "command not found" error, ask the user which virtual environment to activate
- **Auto-fixes**: Some hooks auto-fix issues (e.g., trailing whitespace). Re-run after fixes are applied
## Common File Patterns
### API Structure
- **`/api.py`** - REST endpoints with decorators and OpenAPI docstrings
- **`/schemas.py`** - Marshmallow validation schemas for OpenAPI spec
- **`/commands/`** - Business logic classes with @transaction() decorators
- **`/models/`** - SQLAlchemy database models
- **OpenAPI docs**: Auto-generated at `/swagger/v1` from docstrings and schemas
### Migration Files
- **Location**: `superset/migrations/versions/`
- **Naming**: `YYYY-MM-DD_HH-MM_hash_description.py`
- **Utilities**: Use helpers from `superset.migrations.shared.utils` for database compatibility
- **Pattern**: Import utilities instead of raw SQLAlchemy operations
## Platform-Specific Instructions
- **[CLAUDE.md](CLAUDE.md)** - For Claude/Anthropic tools
- **[.github/copilot-instructions.md](.github/copilot-instructions.md)** - For GitHub Copilot
- **[GEMINI.md](GEMINI.md)** - For Google Gemini tools
- **[GPT.md](GPT.md)** - For OpenAI/ChatGPT tools
- **[.cursor/rules/dev-standard.mdc](.cursor/rules/dev-standard.mdc)** - For Cursor editor
---
**LLM Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

View File

@@ -1,63 +0,0 @@
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"?>
<rdf:RDF xml:lang="en"
xmlns="http://usefulinc.com/ns/doap#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:asfext="http://projects.apache.org/ns/asfext#"
xmlns:foaf="http://xmlns.com/foaf/0.1/">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<Project rdf:about="https://superset.apache.org">
<created>2024-04-10</created>
<license rdf:resource="https://spdx.org/licenses/Apache-2.0" />
<name>Apache Superset</name>
<homepage rdf:resource="https://superset.apache.org" />
<asfext:pmc rdf:resource="https://superset.apache.org" />
<shortdesc>Apache Superset™ is an open-source modern data exploration and visualization platform.</shortdesc>
<description>Superset is a fast, lightweight, intuitive, business intelligence platform. Loaded with options, Superset makes it easy for users of all skill sets to explore and visualize their data, from simple line charts to highly detailed geospatial charts.
* Powerful yet easy to use:
Superset makes it easy to explore your data, using either our simple no-code viz builder or state-of-the-art SQL IDE.
* Integrates with modern databases
Superset can connect to any SQL-based databases including modern cloud-native databases and engines at petabyte scale.
* Modern architecture
Superset is lightweight and highly scalable, leveraging the power of your existing data infrastructure without requiring yet another ingestion layer.
* Rich visualizations and dashboards
Superset ships with 40+ pre-installed visualization types. Our plug-in architecture makes it easy to build custom visualizations.
</description>
<bug-database rdf:resource="https://github.com/apache/superset/issues" />
<mailing-list rdf:resource="https://lists.apache.org/list.html?dev@superset.apache.org" />
<download-page rdf:resource="https://dist.apache.org/repos/dist/release/superset/" />
<programming-language>JavaScript</programming-language>
<programming-language>Python</programming-language>
<programming-language>Typescript</programming-language>
<category rdf:resource="https://projects.apache.org/category/big-data" />
<category rdf:resource="https://projects.apache.org/category/database" />
<category rdf:resource="https://projects.apache.org/category/data-engineering" />
<category rdf:resource="https://projects.apache.org/category/geospatial" />
<repository>
<GitRepository>
<location rdf:resource="https://github.com/apache/superset.git"/>
<browse rdf:resource="https://github.com/apache/superset/"/>
</GitRepository>
</repository>
</Project>
</rdf:RDF>

View File

@@ -37,16 +37,4 @@ under the License.
- [3.0.3](./CHANGELOG/3.0.3.md)
- [3.0.4](./CHANGELOG/3.0.4.md)
- [3.1.0](./CHANGELOG/3.1.0.md)
- [3.1.1](./CHANGELOG/3.1.1.md)
- [3.1.2](./CHANGELOG/3.1.2.md)
- [3.1.3](./CHANGELOG/3.1.3.md)
- [4.0.0](./CHANGELOG/4.0.0.md)
- [4.0.1](./CHANGELOG/4.0.1.md)
- [4.0.2](./CHANGELOG/4.0.2.md)
- [4.1.0](./CHANGELOG/4.1.0.md)
- [4.1.1](./CHANGELOG/4.1.1.md)
- [4.1.2](./CHANGELOG/4.1.2.md)
- [4.1.3](./CHANGELOG/4.1.3.md)
- [4.1.4](./CHANGELOG/4.1.4.md)
- [5.0.0](./CHANGELOG/5.0.0.md)
- [6.0.0](./CHANGELOG/6.0.0.md)

View File

@@ -1,75 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Change Log
### 3.1.1 (Fri Feb 9 21:49:33 2024 +0100)
**Fixes**
- [#27066](https://github.com/apache/superset/pull/27066) fix: Drill by with GLOBAL_ASYNC_QUERIES (@kgabryje)
- [#27039](https://github.com/apache/superset/pull/27039) fix: bump FAB to 4.3.11 (@dpgaspar)
- [#26993](https://github.com/apache/superset/pull/26993) fix: chart import validation (@dpgaspar)
- [#27096](https://github.com/apache/superset/pull/27096) fix(big_number): white-space: nowrap to prevent wrapping (@mistercrunch)
- [#27073](https://github.com/apache/superset/pull/27073) fix(drill): no rows returned (@betodealmeida)
- [#27069](https://github.com/apache/superset/pull/27069) fix: Filters sidebar stretching dashboard height (@kgabryje)
- [#27068](https://github.com/apache/superset/pull/27068) fix: Exclude header controls from dashboard PDF export (@kgabryje)
- [#27041](https://github.com/apache/superset/pull/27041) fix(plugins): missing currency on small number format in table chart (@justinpark)
- [#27023](https://github.com/apache/superset/pull/27023) fix(explore): allow free-form d3 format on custom column formatting (@justinpark)
- [#27019](https://github.com/apache/superset/pull/27019) fix: safer error message in alerts (@betodealmeida)
- [#27015](https://github.com/apache/superset/pull/27015) fix(security manager): Users should not have access to all draft dashboards (@Vitor-Avila)
- [#26965](https://github.com/apache/superset/pull/26965) fix(tags): Improve support for tags with colons (@Vitor-Avila)
- [#26946](https://github.com/apache/superset/pull/26946) fix: column values with NaN (@betodealmeida)
- [#26964](https://github.com/apache/superset/pull/26964) fix(plugin-chart-table): Prevent misalignment of totals and headers when scrollbar is visible (@kgabryje)
- [#26332](https://github.com/apache/superset/pull/26332) fix(embedded+async queries): support async queries to work with embedded guest user (@zephyring)
- [#22849](https://github.com/apache/superset/pull/22849) fix(cache): remove unused webserver config & handle trailing slashes (@Usiel)
- [#26889](https://github.com/apache/superset/pull/26889) fix: Allow exporting saved queries without schema information (@sbernauer)
- [#26887](https://github.com/apache/superset/pull/26887) fix: dashboard import validation (@dpgaspar)
- [#26911](https://github.com/apache/superset/pull/26911) fix: handle CRLF endings causing sqlglot failure (@mapledan)
- [#26906](https://github.com/apache/superset/pull/26906) fix(pinot): typo in the name for epoch_ms_to_dttm (@ege-st)
- [#26817](https://github.com/apache/superset/pull/26817) fix: Bar charts horizontal margin adjustment error (@michael-s-molina)
- [#26749](https://github.com/apache/superset/pull/26749) fix: prevent guest user from modifying metrics (@betodealmeida)
- [#25923](https://github.com/apache/superset/pull/25923) fix(deck.gl Multiple Layer Chart): Add Contour and Heatmap Layer as options (@Mattc1221)
- [#26476](https://github.com/apache/superset/pull/26476) fix(sqlparse): improve table parsing (@betodealmeida)
- [#26922](https://github.com/apache/superset/pull/26922) fix(sqllab): autosync fail on migrated queryEditor (@justinpark)
- [#26814](https://github.com/apache/superset/pull/26814) fix(time-series table): Can't compare from the beginning of the time range (@michael-s-molina)
- [#26701](https://github.com/apache/superset/pull/26701) fix(tags): Filter system tags from the tags list (@Vitor-Avila)
- [#26807](https://github.com/apache/superset/pull/26807) fix: Row limit hardcoded (@michael-s-molina)
- [#26674](https://github.com/apache/superset/pull/26674) fix: helm chart comment on SECRET_KEY (@dpgaspar)
- [#26652](https://github.com/apache/superset/pull/26652) fix(import): only import FORMULA annotations (@mistercrunch)
- [#26314](https://github.com/apache/superset/pull/26314) fix(logging): Filter out undefined columns (@john-bodley)
- [#26461](https://github.com/apache/superset/pull/26461) fix(BigQuery): Support special characters in column/metric names used in ORDER BY (@Vitor-Avila)
- [#26744](https://github.com/apache/superset/pull/26744) fix(db2): Improving support for ibm db2 connections (@Vitor-Avila)
- [#26705](https://github.com/apache/superset/pull/26705) fix(legacy-charts): Show Time Grain control for legacy charts (@Vitor-Avila)
- [#26709](https://github.com/apache/superset/pull/26709) fix: do not use lodash/memoize (@rusackas)
- [#25550](https://github.com/apache/superset/pull/25550) fix: Catch ImportErrors for Google SDKs (@skion)
- [#26645](https://github.com/apache/superset/pull/26645) fix(translation): correct translation errors for Chinese(zh) (@Waterkin)
- [#26638](https://github.com/apache/superset/pull/26638) fix: Avoid 500 if end users write bad SQL (@Khrol)
- [#26644](https://github.com/apache/superset/pull/26644) fix: unnecessary logic on CI ephemeral (@dpgaspar)
- [#26625](https://github.com/apache/superset/pull/26625) fix: create virtual dataset validation (@dpgaspar)
- [#26634](https://github.com/apache/superset/pull/26634) fix: RLS modal styling (@geido)
- [#26469](https://github.com/apache/superset/pull/26469) fix(database): allow filtering by UUID (@betodealmeida)
- [#26412](https://github.com/apache/superset/pull/26412) fix(embedded): Hide dashboard fullscreen option for embedded context (@Vitor-Avila)
- [#26355](https://github.com/apache/superset/pull/26355) fix: Trino - handle table not found in SQLLab (@Khrol)
**Others**
- [#26716](https://github.com/apache/superset/pull/26716) build(deps): bump csstype from 2.6.9 to 3.1.3 in /superset-frontend (@dependabot[bot])
- [#26707](https://github.com/apache/superset/pull/26707) chore(helm): Upgrade default Superset version to 3.1.0 (@dnskr)
- [#26431](https://github.com/apache/superset/pull/26431) chore: bump prophet to 1.1.5 (@villebro)

View File

@@ -1,93 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Change Log
### 3.1.2 (Thu Mar 28 11:32:00 2024 -0300)
**Fixes**
- [#27706](https://github.com/apache/superset/pull/27706) fix: Select onChange is fired when the same item is selected in single mode (@michael-s-molina)
- [#27744](https://github.com/apache/superset/pull/27744) fix: reduce alert error to warning (@eschutho)
- [#27644](https://github.com/apache/superset/pull/27644) fix: Provide more inclusive error handling for saved queries (@john-bodley)
- [#27646](https://github.com/apache/superset/pull/27646) fix: Leverage actual database for rendering Jinjarized SQL (@john-bodley)
- [#27636](https://github.com/apache/superset/pull/27636) fix(sqllab): unable to remove table (@justinpark)
- [#27022](https://github.com/apache/superset/pull/27022) fix(Chart Annotation modal): Table and Superset annotation options will paginate, exceeding previous max limit 100 (@rtexelm)
- [#27552](https://github.com/apache/superset/pull/27552) fix(AlertReports): defaulting grace period to undefined (@fisjac)
- [#27551](https://github.com/apache/superset/pull/27551) fix(AlertReports): clearing custom_width when disabled (@fisjac)
- [#27601](https://github.com/apache/superset/pull/27601) fix: Persist query params appended to permalink (@kgabryje)
- [#27470](https://github.com/apache/superset/pull/27470) fix(sql_parse): Ensure table extraction handles Jinja templating (@john-bodley)
- [#19595](https://github.com/apache/superset/pull/19595) fix: Volatile datasource ordering in dashboard export (@pnadolny13)
- [#27613](https://github.com/apache/superset/pull/27613) fix(Dashboard): Add editMode conditional for translate3d fix on charts to allow intended Fullscreen (@rtexelm)
- [#27388](https://github.com/apache/superset/pull/27388) fix(utils): fix off-by-one error in how rolling window's min_periods truncates dataframe (@sfirke)
- [#27577](https://github.com/apache/superset/pull/27577) fix: sqlglot SQL Server (@betodealmeida)
- [#27576](https://github.com/apache/superset/pull/27576) fix: bump sqlglot to support materialized CTEs (@betodealmeida)
- [#27567](https://github.com/apache/superset/pull/27567) fix(db_engine_specs): Update convert_dttm to work correctly with CrateDB (@hlcianfagna)
- [#27605](https://github.com/apache/superset/pull/27605) fix: Skips Hive tests that are blocking PRs (@michael-s-molina)
- [#27566](https://github.com/apache/superset/pull/27566) fix: guest queries (@betodealmeida)
- [#27464](https://github.com/apache/superset/pull/27464) fix: pass valid SQL to SM (@betodealmeida)
- [#26748](https://github.com/apache/superset/pull/26748) fix: `improve _extract_tables_from_sql` (@betodealmeida)
- [#27260](https://github.com/apache/superset/pull/27260) fix(alerts/reports): implementing custom_width as an Antd number input (@fisjac)
- [#27487](https://github.com/apache/superset/pull/27487) fix(postprocessing): resample with holes (@villebro)
- [#27484](https://github.com/apache/superset/pull/27484) fix: check if guest user modified query (@betodealmeida)
- [#27471](https://github.com/apache/superset/pull/27471) fix(webpack): remove double-dotted file extensions in webpack config (@rusackas)
- [#27411](https://github.com/apache/superset/pull/27411) fix(dashboard): Only fetch CSS templates for dashboard header menu when in edit mode (@mskelton)
- [#27262](https://github.com/apache/superset/pull/27262) fix(Alerts & Reports): Fixing bug that resets cron value to default when empty (@fisjac)
- [#27315](https://github.com/apache/superset/pull/27315) fix(deps): resolving canvg and html2canvas module not found (@fisjac)
- [#27403](https://github.com/apache/superset/pull/27403) fix: missing shared color in mixed timeseries (@justinpark)
- [#27391](https://github.com/apache/superset/pull/27391) fix(sqllab): Close already removed tab (@justinpark)
- [#27364](https://github.com/apache/superset/pull/27364) fix(API): Updating assets via the API should preserve ownership configuration (@Vitor-Avila)
- [#27395](https://github.com/apache/superset/pull/27395) fix: improve explore REST api validations (@dpgaspar)
- [#26205](https://github.com/apache/superset/pull/26205) fix(docker): Remove race condition when building image (@alekseyolg)
- [#27366](https://github.com/apache/superset/pull/27366) fix: Results section in Explore shows an infinite spinner (@michael-s-molina)
- [#27187](https://github.com/apache/superset/pull/27187) fix: numexpr to fix CVE-2023-39631 (2.8.4 => 2.9.0) (@nigzak)
- [#27360](https://github.com/apache/superset/pull/27360) fix: Heatmap numeric sorting (@michael-s-molina)
- [#27308](https://github.com/apache/superset/pull/27308) fix(dashboard): table chart drag preview overflowing container (@rtexelm)
- [#27295](https://github.com/apache/superset/pull/27295) fix(sqllab): invalid dump sql shown after closing tab (@justinpark)
- [#27285](https://github.com/apache/superset/pull/27285) fix(plugin-chart-echarts): calculate Gauge Chart intervals correctly when min value is set (@goto-loop)
- [#27307](https://github.com/apache/superset/pull/27307) fix: Incorrect data type on import page (@michael-s-molina)
- [#27291](https://github.com/apache/superset/pull/27291) fix: Data zoom with horizontal orientation (@michael-s-molina)
- [#27273](https://github.com/apache/superset/pull/27273) fix: Navigating to an invalid page index in lists (@michael-s-molina)
- [#27271](https://github.com/apache/superset/pull/27271) fix: Inoperable dashboard filter slider when range is <= 1 (@michael-s-molina)
- [#27154](https://github.com/apache/superset/pull/27154) fix(import-datasources): Use "admin" user as default for importing datasources (@ddxv)
- [#27258](https://github.com/apache/superset/pull/27258) fix: Sorting charts/dashboards makes the applied filters ineffective (@michael-s-molina)
- [#27213](https://github.com/apache/superset/pull/27213) fix(trino): bumping trino to fix hudi schema fetching (@rusackas)
- [#27236](https://github.com/apache/superset/pull/27236) fix(reports): fixing unit test (@fisjac)
- [#27217](https://github.com/apache/superset/pull/27217) fix(sqlglot): Address regressions introduced in #26476 (@john-bodley)
- [#27233](https://github.com/apache/superset/pull/27233) fix: bump FAB to 4.4.1 (perf issue) (@dpgaspar)
- [#27167](https://github.com/apache/superset/pull/27167) fix: setting important lower bounds versions on requirements (@dpgaspar)
- [#27215](https://github.com/apache/superset/pull/27215) fix: no limit in SELECT \* for TOP dbs (@betodealmeida)
- [#27191](https://github.com/apache/superset/pull/27191) fix: Failed to execute importScripts on worker-css (@michael-s-molina)
- [#27181](https://github.com/apache/superset/pull/27181) fix(sqllab): typeahead search is broken in db selector (@justinpark)
- [#27161](https://github.com/apache/superset/pull/27161) fix(ci): mypy pre-commit issues (@dpgaspar)
- [#27135](https://github.com/apache/superset/pull/27135) fix: Duplicated toast messages (@michael-s-molina)
- [#27132](https://github.com/apache/superset/pull/27132) fix: Plain error message when visiting a dashboard via permalink without permissions (@michael-s-molina)
- [#22840](https://github.com/apache/superset/pull/22840) fix(pivot-table-v2): Added forgotten translation pivot table v2 (@Always-prog)
- [#27128](https://github.com/apache/superset/pull/27128) fix: RLS modal overflow (@michael-s-molina)
- [#27112](https://github.com/apache/superset/pull/27112) fix: gevent upgrade to 23.9.1 (@dpgaspar)
- [#27124](https://github.com/apache/superset/pull/27124) fix: bump grpcio, urllib3 and paramiko (@dpgaspar)
- [#27113](https://github.com/apache/superset/pull/27113) fix: upgrade cryptography to major 42 (@dpgaspar)
- [#27106](https://github.com/apache/superset/pull/27106) fix: Timeseries Y-axis format with contribution mode (@michael-s-molina)
**Others**
- [#27281](https://github.com/apache/superset/pull/27281) chore: bump cryptography minimum to 42.0.4 (@sadpandajoe)
- [#27232](https://github.com/apache/superset/pull/27232) chore: Removes Chromatic workflow and dependencies (@michael-s-molina)
- [#27159](https://github.com/apache/superset/pull/27159) chore: bump FAB to 4.4.0 (@dpgaspar)
- [#27129](https://github.com/apache/superset/pull/27129) chore: lower cryptography min version to 41.0.2 (@sadpandajoe)

View File

@@ -1,47 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Change Log
### 3.1.3 (Tue Apr 30 15:32:33 2024 -0700)
**Fixes**
- [#28277](https://github.com/apache/superset/pull/28277) fix(ci): adding codecov token (@rusackas)
- [#28156](https://github.com/apache/superset/pull/28156) fix(sqllab): invalid css scope for ace editor autocomplete (@justinpark)
- [#28222](https://github.com/apache/superset/pull/28222) fix: Dremio alias (@betodealmeida)
- [#28152](https://github.com/apache/superset/pull/28152) fix(sql_parse): Provide more lenient logic when extracting latest[_sub]\_partition (@john-bodley)
- [#28117](https://github.com/apache/superset/pull/28117) fix(sql_parse): Support Jinja format() filter when extracting latest[_sub]\_partition (@john-bodley)
- [#28036](https://github.com/apache/superset/pull/28036) fix: Dynamic filter does not show all values on blur/clear events (@michael-s-molina)
- [#28018](https://github.com/apache/superset/pull/28018) fix: bump client side chart timeouts to use the SUPERSET_WEBSERVER_TIMEOUT (@eschutho)
- [#28017](https://github.com/apache/superset/pull/28017) fix: Select is accepting unknown pasted values when `allowNewOptions` is false (@michael-s-molina)
- [#27996](https://github.com/apache/superset/pull/27996) fix: Incorrect onChange value when an unloaded value is pasted into AsyncSelect (@michael-s-molina)
- [#25407](https://github.com/apache/superset/pull/25407) fix(frontend): allow "constructor" property in response data (@SpencerTorres)
- [#27919](https://github.com/apache/superset/pull/27919) fix: add mariadb engine spec same as MySQL (@dpgaspar)
- [#27593](https://github.com/apache/superset/pull/27593) fix(Dashboard): Add border to row when hovering HoverMenu in edit mode (@rtexelm)
- [#27883](https://github.com/apache/superset/pull/27883) fix(bar-chart): change legend padding for horizontal orientation (@lilykuang)
- [#27845](https://github.com/apache/superset/pull/27845) fix(dashboard): missing null check in error extra (@justinpark)
**Others**
- [#28278](https://github.com/apache/superset/pull/28278) chore: allow codecov to detect SHA (@mistercrunch)
- [#27725](https://github.com/apache/superset/pull/27725) chore(sqllab): Do not strip comments when executing SQL statements (@john-bodley)
- [#27843](https://github.com/apache/superset/pull/27843) chore: Default to engine specification regarding using wildcard (@john-bodley)
- [#27858](https://github.com/apache/superset/pull/27858) chore(sql_parse): Provide more meaningful SQLGlot errors (@john-bodley)
- [#27842](https://github.com/apache/superset/pull/27842) chore(sql_parse): Strip leading/trailing whitespace in Jinja macro extraction (@john-bodley)

View File

@@ -19,7 +19,7 @@ under the License.
## Change Log
### 4.0 (Mon Apr 1 10:04:00 2024 -0500)
### 4.0 (Tue Feb 20 10:04:00 2024 -0500)
**Database Migrations**
@@ -32,6 +32,7 @@ under the License.
- [#26369](https://github.com/apache/superset/pull/26369) refactor: Removes the filters set feature (@michael-s-molina)
- [#26416](https://github.com/apache/superset/pull/26416) fix: improve performance on reports log queries (@dpgaspar)
- [#26290](https://github.com/apache/superset/pull/26290) feat(echarts-funnel): Implement % calculation type (@kgabryje)
- [#26288](https://github.com/apache/superset/pull/26288) chore: Ensure Mixins are ordered according to the MRO (@john-bodley)
**Features**
@@ -64,69 +65,6 @@ under the License.
**Fixes**
- [#27706](https://github.com/apache/superset/pull/27706) fix: Select onChange is fired when the same item is selected in single mode (@michael-s-molina)
- [#27763](https://github.com/apache/superset/pull/27763) fix: Removes filter plugins from viz gallery (@michael-s-molina)
- [#27744](https://github.com/apache/superset/pull/27744) fix: reduce alert error to warning (@eschutho)
- [#27558](https://github.com/apache/superset/pull/27558) fix(explore): drag and drop indicator UX (@justinpark)
- [#27644](https://github.com/apache/superset/pull/27644) fix: Provide more inclusive error handling for saved queries (@john-bodley)
- [#27646](https://github.com/apache/superset/pull/27646) fix: Leverage actual database for rendering Jinjarized SQL (@john-bodley)
- [#27550](https://github.com/apache/superset/pull/27550) fix(AlertReports): disabling value when not null option is active (@fisjac)
- [#27636](https://github.com/apache/superset/pull/27636) fix(sqllab): unable to remove table (@justinpark)
- [#27022](https://github.com/apache/superset/pull/27022) fix(Chart Annotation modal): Table and Superset annotation options will paginate, exceeding previous max limit 100 (@rtexelm)
- [#27552](https://github.com/apache/superset/pull/27552) fix(AlertReports): defaulting grace period to undefined (@fisjac)
- [#27551](https://github.com/apache/superset/pull/27551) fix(AlertReports): clearing custom_width when disabled (@fisjac)
- [#27470](https://github.com/apache/superset/pull/27470) fix(sql_parse): Ensure table extraction handles Jinja templating (@john-bodley)
- [#27601](https://github.com/apache/superset/pull/27601) fix: Persist query params appended to permalink (@kgabryje)
- [#27613](https://github.com/apache/superset/pull/27613) fix(Dashboard): Add editMode conditional for translate3d fix on charts to allow intended Fullscreen (@rtexelm)
- [#27388](https://github.com/apache/superset/pull/27388) fix(utils): fix off-by-one error in how rolling window's min_periods truncates dataframe (@sfirke)
- [#19595](https://github.com/apache/superset/pull/19595) fix: Volatile datasource ordering in dashboard export (@pnadolny13)
- [#27577](https://github.com/apache/superset/pull/27577) fix: sqlglot SQL Server (@betodealmeida)
- [#27576](https://github.com/apache/superset/pull/27576) fix: bump sqlglot to support materialized CTEs (@betodealmeida)
- [#27567](https://github.com/apache/superset/pull/27567) fix(db_engine_specs): Update convert_dttm to work correctly with CrateDB (@hlcianfagna)
- [#27605](https://github.com/apache/superset/pull/27605) fix: Skips Hive tests that are blocking PRs (@michael-s-molina)
- [#27566](https://github.com/apache/superset/pull/27566) fix: guest queries (@betodealmeida)
- [#27464](https://github.com/apache/superset/pull/27464) fix: pass valid SQL to SM (@betodealmeida)
- [#26748](https://github.com/apache/superset/pull/26748) fix: `improve _extract_tables_from_sql` (@betodealmeida)
- [#27539](https://github.com/apache/superset/pull/27539) fix(explore): Allow only saved metrics and columns (@justinpark)
- [#27260](https://github.com/apache/superset/pull/27260) fix(alerts/reports): implementing custom_width as an Antd number input (@fisjac)
- [#27487](https://github.com/apache/superset/pull/27487) fix(postprocessing): resample with holes (@villebro)
- [#27484](https://github.com/apache/superset/pull/27484) fix: check if guest user modified query (@betodealmeida)
- [#27471](https://github.com/apache/superset/pull/27471) fix(webpack): remove double-dotted file extensions in webpack config (@rusackas)
- [#27186](https://github.com/apache/superset/pull/27186) fix: SSH Tunnel configuration settings (@geido)
- [#27411](https://github.com/apache/superset/pull/27411) fix(dashboard): Only fetch CSS templates for dashboard header menu when in edit mode (@mskelton)
- [#27315](https://github.com/apache/superset/pull/27315) fix(deps): resolving canvg and html2canvas module not found (@fisjac)
- [#27403](https://github.com/apache/superset/pull/27403) fix: missing shared color in mixed timeseries (@justinpark)
- [#27402](https://github.com/apache/superset/pull/27402) fix: typescript errors in 4.0 (@justinpark)
- [#27390](https://github.com/apache/superset/pull/27390) fix: Re-enable CI checks on release branches (@michael-s-molina)
- [#27391](https://github.com/apache/superset/pull/27391) fix(sqllab): Close already removed tab (@justinpark)
- [#27364](https://github.com/apache/superset/pull/27364) fix(API): Updating assets via the API should preserve ownership configuration (@Vitor-Avila)
- [#27395](https://github.com/apache/superset/pull/27395) fix: improve explore REST api validations (@dpgaspar)
- [#27262](https://github.com/apache/superset/pull/27262) fix(Alerts & Reports): Fixing bug that resets cron value to default when empty (@fisjac)
- [#27366](https://github.com/apache/superset/pull/27366) fix: Results section in Explore shows an infinite spinner (@michael-s-molina)
- [#27187](https://github.com/apache/superset/pull/27187) fix: numexpr to fix CVE-2023-39631 (2.8.4 => 2.9.0) (@nigzak)
- [#27361](https://github.com/apache/superset/pull/27361) fix: Missing SQL Lab permission (@michael-s-molina)
- [#27360](https://github.com/apache/superset/pull/27360) fix: Heatmap numeric sorting (@michael-s-molina)
- [#27313](https://github.com/apache/superset/pull/27313) fix(sqllab): Missing empty query result state (@justinpark)
- [#27308](https://github.com/apache/superset/pull/27308) fix(dashboard): table chart drag preview overflowing container (@rtexelm)
- [#27295](https://github.com/apache/superset/pull/27295) fix(sqllab): invalid dump sql shown after closing tab (@justinpark)
- [#27285](https://github.com/apache/superset/pull/27285) fix(plugin-chart-echarts): calculate Gauge Chart intervals correctly when min value is set (@goto-loop)
- [#27307](https://github.com/apache/superset/pull/27307) fix: Incorrect data type on import page (@michael-s-molina)
- [#27291](https://github.com/apache/superset/pull/27291) fix: Data zoom with horizontal orientation (@michael-s-molina)
- [#27273](https://github.com/apache/superset/pull/27273) fix: Navigating to an invalid page index in lists (@michael-s-molina)
- [#27271](https://github.com/apache/superset/pull/27271) fix: Inoperable dashboard filter slider when range is <= 1 (@michael-s-molina)
- [#27154](https://github.com/apache/superset/pull/27154) fix(import-datasources): Use "admin" user as default for importing datasources (@ddxv)
- [#27258](https://github.com/apache/superset/pull/27258) fix: Sorting charts/dashboards makes the applied filters ineffective (@michael-s-molina)
- [#27213](https://github.com/apache/superset/pull/27213) fix(trino): bumping trino to fix hudi schema fetching (@rusackas)
- [#27236](https://github.com/apache/superset/pull/27236) fix(reports): fixing unit test (@fisjac)
- [#27217](https://github.com/apache/superset/pull/27217) fix(sqlglot): Address regressions introduced in #26476 (@john-bodley)
- [#27233](https://github.com/apache/superset/pull/27233) fix: bump FAB to 4.4.1 (perf issue) (@dpgaspar)
- [#27167](https://github.com/apache/superset/pull/27167) fix: setting important lower bounds versions on requirements (@dpgaspar)
- [#27215](https://github.com/apache/superset/pull/27215) fix: no limit in SELECT \* for TOP dbs (@betodealmeida)
- [#27214](https://github.com/apache/superset/pull/27214) fix(releasing): fixes npm script for release validation (@rusackas)
- [#26074](https://github.com/apache/superset/pull/26074) fix: Translations related to the date range filter (@Ralkion)
- [#26699](https://github.com/apache/superset/pull/26699) fix(dashboard): drag and drop indicator UX (@justinpark)
- [#27191](https://github.com/apache/superset/pull/27191) fix: Failed to execute importScripts on worker-css (@michael-s-molina)
- [#27181](https://github.com/apache/superset/pull/27181) fix(sqllab): typeahead search is broken in db selector (@justinpark)
- [#27164](https://github.com/apache/superset/pull/27164) fix: unlock and bump werkzeug (@dpgaspar)
- [#27161](https://github.com/apache/superset/pull/27161) fix(ci): mypy pre-commit issues (@dpgaspar)
- [#27138](https://github.com/apache/superset/pull/27138) fix(plugins): Apply dashboard filters to comparison query in BigNumber with Time Comparison chart (@Antonio-RiveroMartnez)
@@ -193,12 +131,6 @@ under the License.
**Others**
- [#27441](https://github.com/apache/superset/pull/27441) chore: Adds the 4.0 release notes (@michael-s-molina)
- [#27768](https://github.com/apache/superset/pull/27768) chore(docs): Cleanup UPDATING.md (@john-bodley)
- [#27625](https://github.com/apache/superset/pull/27625) perf(explore): virtualized datasource field sections (@justinpark)
- [#27488](https://github.com/apache/superset/pull/27488) perf(sqllab): reduce bootstrap data delay by queries (@justinpark)
- [#27281](https://github.com/apache/superset/pull/27281) chore: bump cryptography minimum to 42.0.4 (@sadpandajoe)
- [#27232](https://github.com/apache/superset/pull/27232) chore: Removes Chromatic workflow and dependencies (@michael-s-molina)
- [#27169](https://github.com/apache/superset/pull/27169) chore: Updates CHANGELOG.md with 3.0.4 data (@michael-s-molina)
- [#27166](https://github.com/apache/superset/pull/27166) docs: add Dropit Shopping to users list (@IlyaDropit)
- [#27143](https://github.com/apache/superset/pull/27143) refactor: Migrate ErrorBoundary to typescript (@EnxDev)
@@ -469,4 +401,3 @@ under the License.
- [#26100](https://github.com/apache/superset/pull/26100) build(deps-dev): bump @types/node from 20.9.4 to 20.10.0 in /superset-websocket (@dependabot[bot])
- [#26099](https://github.com/apache/superset/pull/26099) build(deps-dev): bump @types/cookie from 0.5.4 to 0.6.0 in /superset-websocket (@dependabot[bot])
- [#26104](https://github.com/apache/superset/pull/26104) docs: update CVEs fixed on 2.1.2 (@dpgaspar)
- [#26288](https://github.com/apache/superset/pull/26288) chore: Ensure Mixins are ordered according to the MRO (@john-bodley)

View File

@@ -1,61 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Change Log
### 4.0.1 (Tue Apr 30 15:32:33 2024 -0700)
**Fixes**
- [#28269](https://github.com/apache/superset/pull/28269) fix(explore): cannot reorder dnd of Metrics (@justinpark)
- [#28271](https://github.com/apache/superset/pull/28271) fix: % replace in `values_for_column` (@betodealmeida)
- [#28277](https://github.com/apache/superset/pull/28277) fix(ci): adding codecov token (@rusackas)
- [#28242](https://github.com/apache/superset/pull/28242) fix(dashboard): unable to drop tabs in columns (@justinpark)
- [#28241](https://github.com/apache/superset/pull/28241) fix(explore): temporal column mixin (@justinpark)
- [#28226](https://github.com/apache/superset/pull/28226) fix(maps): adds Crimea back to Ukraine 🇺🇦 (@rusackas)
- [#28156](https://github.com/apache/superset/pull/28156) fix(sqllab): invalid css scope for ace editor autocomplete (@justinpark)
- [#28222](https://github.com/apache/superset/pull/28222) fix: Dremio alias (@betodealmeida)
- [#28152](https://github.com/apache/superset/pull/28152) fix(sql_parse): Provide more lenient logic when extracting latest[_sub]\_partition (@john-bodley)
- [#27554](https://github.com/apache/superset/pull/27554) fix(AlertsReports): making log retention "None" option valid (@fisjac)
- [#28117](https://github.com/apache/superset/pull/28117) fix(sql_parse): Support Jinja format() filter when extracting latest[_sub]\_partition (@john-bodley)
- [#28036](https://github.com/apache/superset/pull/28036) fix: Dynamic filter does not show all values on blur/clear events (@michael-s-molina)
- [#28018](https://github.com/apache/superset/pull/28018) fix: bump client side chart timeouts to use the SUPERSET_WEBSERVER_TIMEOUT (@eschutho)
- [#28017](https://github.com/apache/superset/pull/28017) fix: Select is accepting unknown pasted values when `allowNewOptions` is false (@michael-s-molina)
- [#27996](https://github.com/apache/superset/pull/27996) fix: Incorrect onChange value when an unloaded value is pasted into AsyncSelect (@michael-s-molina)
- [#27934](https://github.com/apache/superset/pull/27934) fix(time_offset): improved LIMIT-handling in advanced analytics (@Antonio-RiveroMartnez)
- [#27941](https://github.com/apache/superset/pull/27941) fix(drillby): Enable DrillBy in charts w/o filters (dimensions) (@sowo)
- [#27239](https://github.com/apache/superset/pull/27239) fix(alerts/reports): removing duplicate notification method options (@fisjac)
- [#27968](https://github.com/apache/superset/pull/27968) fix(Dashboard): Add aria-label to filters and search forms (@geido)
- [#27701](https://github.com/apache/superset/pull/27701) fix: useTruncation infinite loop, reenable dashboard cross links on ChartList (@kgabryje)
- [#27926](https://github.com/apache/superset/pull/27926) fix: Locale sent to frontend (@michael-s-molina)
- [#25407](https://github.com/apache/superset/pull/25407) fix(frontend): allow "constructor" property in response data (@SpencerTorres)
- [#27919](https://github.com/apache/superset/pull/27919) fix: add mariadb engine spec same as MySQL (@dpgaspar)
- [#27593](https://github.com/apache/superset/pull/27593) fix(Dashboard): Add border to row when hovering HoverMenu in edit mode (@rtexelm)
- [#27883](https://github.com/apache/superset/pull/27883) fix(bar-chart): change legend padding for horizontal orientation (@lilykuang)
- [#27700](https://github.com/apache/superset/pull/27700) fix: row limits & row count labels are confusing (@mistercrunch)
- [#27845](https://github.com/apache/superset/pull/27845) fix(dashboard): missing null check in error extra (@justinpark)
**Others**
- [#28278](https://github.com/apache/superset/pull/28278) chore: allow codecov to detect SHA (@mistercrunch)
- [#27717](https://github.com/apache/superset/pull/27717) chore(explore): Hide non-droppable metric and column list (@justinpark)
- [#27725](https://github.com/apache/superset/pull/27725) chore(sqllab): Do not strip comments when executing SQL statements (@john-bodley)
- [#27843](https://github.com/apache/superset/pull/27843) chore: Default to engine specification regarding using wildcard (@john-bodley)
- [#27858](https://github.com/apache/superset/pull/27858) chore(sql_parse): Provide more meaningful SQLGlot errors (@john-bodley)
- [#27842](https://github.com/apache/superset/pull/27842) chore(sql_parse): Strip leading/trailing whitespace in Jinja macro extraction (@john-bodley)

View File

@@ -1,78 +0,0 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Change Log
### 4.0.2 (Wed Jun 26 14:26:58 2024 -0300)
**Fixes**
- [#28639](https://github.com/apache/superset/pull/28639) fix: adds the ability to disallow SQL functions per engine (@dpgaspar)
- [#29367](https://github.com/apache/superset/pull/29367) fix(explore): don't respect y-axis formatting (@justinpark)
- [#29345](https://github.com/apache/superset/pull/29345) fix(revert 27883): Excess padding in horizontal Bar charts (@michael-s-molina)
- [#29349](https://github.com/apache/superset/pull/29349) fix(explore): restored hidden field values has discarded (@justinpark)
- [#29346](https://github.com/apache/superset/pull/29346) fix: Cannot delete empty column inside a tab using the dashboard editor (@michael-s-molina)
- [#29314](https://github.com/apache/superset/pull/29314) fix: Remove recursive repr call (@jessie-ross)
- [#29301](https://github.com/apache/superset/pull/29301) fix(metastore-cache): prune before add (@villebro)
- [#29278](https://github.com/apache/superset/pull/29278) fix(sqllab): invalid empty state on switch tab (@justinpark)
- [#29291](https://github.com/apache/superset/pull/29291) fix: filters not updating with force update when caching is enabled (@ka-weihe)
- [#28744](https://github.com/apache/superset/pull/28744) fix(permalink): adding anchor to dashboard permalink generation (@fisjac)
- [#29260](https://github.com/apache/superset/pull/29260) fix: Custom SQL filter control (@michael-s-molina)
- [#29248](https://github.com/apache/superset/pull/29248) fix(sqllab): Do not strip comments when executing SQL statements (@john-bodley)
- [#28755](https://github.com/apache/superset/pull/28755) fix: Workaround for Pandas.DataFrame.to_csv bug (@john-bodley)
- [#29234](https://github.com/apache/superset/pull/29234) fix(Explore): Keep necessary form data to allow query mode switching (@rtexelm)
- [#29230](https://github.com/apache/superset/pull/29230) fix(sqllab): run previous state query (@justinpark)
- [#29119](https://github.com/apache/superset/pull/29119) fix(mixed-timeseries-plugin): Second query stacks stacked on top of first query series (@kgabryje)
- [#28932](https://github.com/apache/superset/pull/28932) fix(embedded): add missing GUEST_TOKEN_HEADER_NAME to bootstrap data (@hexcafe)
- [#29084](https://github.com/apache/superset/pull/29084) fix: Remove BASE_AXIS from pre-query (@john-bodley)
- [#29081](https://github.com/apache/superset/pull/29081) fix(explore): Drill to detail truncates int64 IDs (@justinpark)
- [#28771](https://github.com/apache/superset/pull/28771) fix(Mixed Chart Filter Control): Allow delete condition for `adhoc_filters_b` (@rtexelm)
- [#28772](https://github.com/apache/superset/pull/28772) fix(dashboard): unable to resize due to the overlapped droptarget (@justinpark)
- [#28750](https://github.com/apache/superset/pull/28750) fix: do not close database modal on mask click (@eschutho)
- [#28745](https://github.com/apache/superset/pull/28745) fix(reports): Update the element class to wait for when taking a screenshot (@Vitor-Avila)
- [#28749](https://github.com/apache/superset/pull/28749) fix(sqllab): Sort db selector options by the API order (@justinpark)
- [#28653](https://github.com/apache/superset/pull/28653) fix: Handling of column types for Presto, Trino, et al. (@john-bodley)
- [#28422](https://github.com/apache/superset/pull/28422) fix: Update migration logic in #27119 (@john-bodley)
- [#28349](https://github.com/apache/superset/pull/28349) fix: Add back description column to saved queries #12431 (@imancrsrk)
- [#28512](https://github.com/apache/superset/pull/28512) fix: improve df to records performance (@dpgaspar)
- [#28613](https://github.com/apache/superset/pull/28613) fix: revert fix(presto preview): re-enable schema previsualization for Trino/Presto table/schemas" (@john-bodley)
- [#28567](https://github.com/apache/superset/pull/28567) fix: Revert "fix: don't strip SQL comments in Explore (#28363)" (@michael-s-molina)
- [#28555](https://github.com/apache/superset/pull/28555) fix(explore): hide a control wrapped with StashFormDataContainer correctly (@justinpark)
- [#28507](https://github.com/apache/superset/pull/28507) fix(dashboard): invalid drop item on a tab (@justinpark)
- [#28432](https://github.com/apache/superset/pull/28432) fix: Time shifts calculation for ECharts plugins (@michael-s-molina)
- [#26782](https://github.com/apache/superset/pull/26782) fix(presto preview): re-enable schema previsualization for Trino/Presto table/schemas (@brouberol)
- [#28409](https://github.com/apache/superset/pull/28409) fix(ar-modal): updateNotificationSettings not updating state (@fisjac)
- [#28395](https://github.com/apache/superset/pull/28395) fix(dashboard): Change class name on last Droppable in a column (@rtexelm)
- [#28396](https://github.com/apache/superset/pull/28396) fix: type annotation breaking on py3.9 (@dpgaspar)
- [#28368](https://github.com/apache/superset/pull/28368) fix: Contribution percentages for ECharts plugins (@michael-s-molina)
- [#28386](https://github.com/apache/superset/pull/28386) fix: Scroll to top when selecting a global dashboard tab (@michael-s-molina)
- [#28312](https://github.com/apache/superset/pull/28312) fix(explore): hide advanced analytics for non temporal xaxis (@justinpark)
- [#28363](https://github.com/apache/superset/pull/28363) fix: don't strip SQL comments in Explore (@mistercrunch)
- [#28341](https://github.com/apache/superset/pull/28341) fix: Remedy logic for UpdateDatasetCommand uniqueness check (@john-bodley)
- [#28334](https://github.com/apache/superset/pull/28334) fix: Small tweaks for Line and Area chart migrations (ECharts) (@michael-s-molina)
- [#28266](https://github.com/apache/superset/pull/28266) fix: use pessimistic json encoder in SQL Lab (@mistercrunch)
- [#28113](https://github.com/apache/superset/pull/28113) fix: Rename legacy line and area charts (@john-bodley)
- [#28279](https://github.com/apache/superset/pull/28279) fix(sql_parse): Ignore USE SQL keyword when determining SELECT statement (@john-bodley)
- [#28322](https://github.com/apache/superset/pull/28322) fix(sql_parse): Add Apache Spark to SQLGlot dialect mapping (@john-bodley)
**Others**
- [#29360](https://github.com/apache/superset/pull/29360) chore: Rename Totals to Summary in table chart (@michael-s-molina)
- [#29249](https://github.com/apache/superset/pull/29249) test(Explorer): Fix minor errors in ExploreViewContainer syntax, add tests (@rtexelm)
- [#28876](https://github.com/apache/superset/pull/28876) chore(sqllab): Add logging for actions (@justinpark)

Some files were not shown because too many files have changed in this diff Show More