diff --git a/UPDATING.md b/UPDATING.md index 65598d0025a..9d05091fd60 100644 --- a/UPDATING.md +++ b/UPDATING.md @@ -304,6 +304,36 @@ Schedule the cutover in a quiet window. Runtime reads use only the single config The migration is transactional (all-or-nothing) and idempotent — it can be safely re-run or resumed. Note that AES-GCM, unlike AES-CBC, does not support querying directly over encrypted columns; audit any code that filters on an encrypted column before switching. See the SIP at `docs/sip/authenticated-encryption-at-rest.md` for details. +### Soft delete and restore for datasets + +**The soft-delete behavior in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/dataset/` continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below. + +**Flag-toggle caveat:** the soft-delete visibility filter is evaluated per query while the flag is on. If datasets are soft-deleted during a flag-on window and the flag is later turned **off**, those rows reappear as live datasets in all lists, lookups, and relationship loads (including charts that reference them). The `POST //restore` endpoint and the `dataset_deleted_state` list filter remain functional regardless of the flag, deliberately, so rows soft-deleted during a flag-on window stay discoverable and restorable after a rollback of the flag. + +**Flag-independent parts of this work** (active even with `SOFT_DELETE` off): the restore endpoint and deleted-state filter (above); the database-deletion guard counting soft-deleted datasets; the `get_or_create_dataset` soft-deleted-twin pre-check; the combined datasource listing (`GET /api/v1/datasource/...`) always excluding soft-deleted datasets; and the two uniqueness-validation changes documented at the end of this section. Everything else — the soft DELETE itself and the visibility filtering — is flag-gated. + +With the flag enabled: `DELETE /api/v1/dataset/` no longer hard-deletes the dataset (the bulk-delete endpoint behaves the same way). The row is marked with a `deleted_at` timestamp and hidden from all list, detail, and lookup endpoints. Datasets in this state are excluded from default queries and from relationship loads (e.g. `database.tables`). + +**No cascade in v1.** Soft-delete does not propagate to dependent charts or dashboards: they remain visible. Loading a chart whose dataset is soft-deleted surfaces a "datasource not found" error at chart-load time. Restore the dataset to recover. + +**Database deletion is blocked by soft-deleted datasets.** Superset already refuses to delete a database that still has datasets (`DatabaseDeleteDatasetsExistFailedError`); that check now explicitly counts soft-deleted datasets too (it bypasses the visibility filter), since the soft-deleted `tables` rows still reference the database via `database_id` and must not be orphaned. Consequence: because dataset `DELETE` is soft and v1 ships no hard-delete/purge, **a database that has ever had datasets cannot be deleted through the API once those datasets are soft-deleted** — the rows remain and keep blocking the delete. Until a purge capability lands, operators who must remove such a database have to hard-delete the underlying `tables` rows out-of-band first. This is a deliberate trade-off (no orphaned rows / restorable datasets) and is expected to be resolved by the planned purge work. + +**Side-effect change for operators.** Because the row is no longer physically deleted, FAB `ab_view_menu` / permission-view rows tied to the dataset are also preserved. Downstream automation that relied on `DELETE /api/v1/dataset/` cleaning up those rows must now react to the new `POST /api/v1/dataset//restore` lifecycle, or call the eventual hard-delete endpoint. + +**New endpoint** — `POST /api/v1/dataset//restore` clears `deleted_at` and returns the dataset to active state. Requires `can_write on Dataset` and ownership of the row (or admin). Soft-deleted datasets can also be surfaced in the list endpoint via the new `dataset_deleted_state` rison filter: `include` returns both live and soft-deleted rows, `only` returns just the soft-deleted ones. Any other value is ignored. For non-admin users, soft-deleted rows are limited to datasets they own — the same audience that can restore them. + +**Permissions migration:** existing role grants of `can_write on Dataset` cover the new restore endpoint automatically; no role migration is required. + +**Schema migration:** the migration adds a nullable `deleted_at` column and an index on it (`ix_tables_deleted_at`) to the `tables` table. The column add is instant; the index build runs inline (no `CONCURRENTLY`) and may briefly block writes on the `tables` table (INSERT/UPDATE/DELETE are queued while the index builds; reads are unaffected) on large Postgres deployments. MySQL InnoDB builds the index online (no blocking). Production deployments with many thousands of datasets should run this migration during a maintenance window. + +**Rollback note:** if the application code is rolled back after datasets have been soft-deleted, the older code path's visibility filter no longer applies and previously hidden rows become visible to the older code. Pair the rollback with a data decision (restore the rows, hard-delete them, or also downgrade the migration) rather than assuming the old hard-delete semantics still hold. **Downgrading the migration destroys the deletion markers**: `downgrade()` drops the `deleted_at` column, so any not-yet-restored soft-deleted datasets silently become live, active datasets with no record they were ever deleted. Reconcile the trash (restore or hard-delete each row) *before* downgrading, and disable the `SOFT_DELETE` flag first so no new soft deletes land mid-rollback. + +**SQL Lab / dataset-creation flows:** creating a dataset over a table whose dataset sits in the trash is refused. The SQL Lab "save as dataset" flow (`get_or_create_dataset`) and file uploads return a **422 naming the hidden twin and the restore endpoint**; the plain create, update, and duplicate paths currently fail with the generic "already exists" 422. In all cases the remediation is the same: restore the hidden dataset (or use a different table name). Perm-string maintenance also covers hidden rows: renaming a database rewrites `perm`/`schema_perm`/`catalog_perm` on soft-deleted datasets and their charts, so a later restore does not resurrect stale permission strings. + +**Importer behavior:** importing a dataset YAML whose UUID matches an existing **soft-deleted** dataset is treated as an implicit restore-with-update — **and this happens even when `overwrite` is not set**. This is a deliberate asymmetry with active rows: an active dataset imported without `overwrite=true` is returned unchanged, but a soft-deleted UUID match is restored *and* has the upload's contents applied regardless of the `overwrite` argument, on the reasoning that re-importing a deleted dataset's exact UUID is an explicit request to bring it back. The restore preserves the original PK, the chart back-reference, `table_columns`, and `sql_metrics`. Non-owners get `ImportFailedError`. Callers without `can_write` get `ImportFailedError` instead of silently receiving the soft-deleted row. + +**Uniqueness-validation changes that apply regardless of the feature flag:** two dataset uniqueness checks were tightened alongside this work and are active even with `SOFT_DELETE` off. (1) Create/update uniqueness treats a dataset whose `catalog` is `NULL` as belonging to the database's default catalog, so a legacy twin pair (`catalog=NULL` vs. `catalog=`, same database/schema/name) that older versions allowed now fails validation with "already exists" when either row is edited — resolve by renaming or removing one of the twins. (2) Duplicating a dataset now checks name collisions scoped to the target (database, catalog, schema) instead of globally by name alone: duplicates into other databases that were previously blocked are now allowed. + ### Soft delete and restore for charts **Everything in this section applies only when the `SOFT_DELETE` feature flag is enabled. The flag defaults to `False`** (`@lifecycle: development`), so on a default deployment `DELETE /api/v1/chart/` continues to **hard-delete permanently** — nothing is recoverable. Enable `SOFT_DELETE` to get the behavior described below. diff --git a/superset/commands/database/delete.py b/superset/commands/database/delete.py index bf499dac4ff..b0c0b41ab85 100644 --- a/superset/commands/database/delete.py +++ b/superset/commands/database/delete.py @@ -25,6 +25,7 @@ from superset.commands.database.exceptions import ( DatabaseDeleteDatasetsExistFailedError, DatabaseDeleteFailedError, DatabaseDeleteFailedReportsExistError, + DatabaseDeleteSoftDeletedDatasetsExistFailedError, DatabaseNotFoundError, ) from superset.daos.database import DatabaseDAO @@ -61,6 +62,41 @@ class DeleteDatabaseCommand(BaseCommand): report_names=",".join(report_names), ) ) - # Check if there are datasets for this database - if self._model.tables: + # Check if there are datasets for this database. ``self._model.tables`` + # would now hide soft-deleted datasets (``SqlaTable`` inherits + # ``SoftDeleteMixin``, so the relationship lazy-load applies the + # visibility filter), letting a database whose datasets are all + # soft-deleted look empty and be hard-deleted while ``tables.database_id`` + # rows still reference it. Count with the visibility filter bypassed so + # soft-deleted datasets still block the delete. + from superset.connectors.sqla.models import ( # pylint: disable=import-outside-toplevel + SqlaTable, + ) + from superset.extensions import ( # pylint: disable=import-outside-toplevel + db, + ) + from superset.models.helpers import ( # pylint: disable=import-outside-toplevel + skip_visibility_filter, + ) + + with skip_visibility_filter(db.session, SqlaTable): + has_live = db.session.query( + db.session.query(SqlaTable.id) + .filter( + SqlaTable.database_id == self._model_id, + SqlaTable.deleted_at.is_(None), + ) + .exists() + ).scalar() + has_any = db.session.query( + db.session.query(SqlaTable.id) + .filter(SqlaTable.database_id == self._model_id) + .exists() + ).scalar() + # Both cases block the delete (a soft-deleted dataset still FK-references + # the database), but the message differs: with only hidden rows left the + # operator's dataset list looks empty, so say so explicitly. + if has_live: raise DatabaseDeleteDatasetsExistFailedError() + if has_any: + raise DatabaseDeleteSoftDeletedDatasetsExistFailedError() diff --git a/superset/commands/database/exceptions.py b/superset/commands/database/exceptions.py index ed9cbfb8a57..654da0693bd 100644 --- a/superset/commands/database/exceptions.py +++ b/superset/commands/database/exceptions.py @@ -123,6 +123,28 @@ class DatabaseUploadFileTooLarge(CommandException): message = _("Database upload file exceeds the maximum allowed size.") +class DatabaseUploadSoftDeletedDatasetExistsError(DatabaseUploadFailed): + """The upload targets a table whose dataset sits soft-deleted in the trash. + + Creating a new dataset over the same physical table would make an active + twin of the hidden row and permanently block its restore, so the upload + is refused before any file data is written. The message names only + executable recoveries (restore, or a different table name) — there is no + hard-delete/purge API surface until the purge work lands. + """ + + def __init__(self, dataset_uuid: str) -> None: + super().__init__( + _( + "A soft-deleted dataset (uuid %(uuid)s) already references " + "this table. Restore it via POST " + "/api/v1/dataset/%(uuid)s/restore before uploading, or upload " + "to a different table name.", + uuid=dataset_uuid, + ) + ) + + class DatabaseUploadSaveMetadataFailed(CommandException): status = 500 message = _("Database upload file failed, while saving metadata") @@ -156,6 +178,21 @@ class DatabaseDeleteDatasetsExistFailedError(DeleteFailedError): message = _("Cannot delete a database that has datasets attached") +class DatabaseDeleteSoftDeletedDatasetsExistFailedError( + DatabaseDeleteDatasetsExistFailedError +): + # Subclasses the live-datasets error so the existing API handler catches + # both; only the message differs, telling the operator that the blockers + # are hidden (soft-deleted) rows even though their dataset list looks empty. + message = _( + "Cannot delete a database whose only remaining datasets are " + "soft-deleted. Restore them (POST /api/v1/dataset//restore) " + "and delete them permanently once a purge capability ships, or " + "remove the underlying rows out-of-band, before deleting the " + "database." + ) + + class DatabaseDeleteFailedError(DeleteFailedError): message = _("Database could not be deleted.") diff --git a/superset/commands/database/sync_permissions.py b/superset/commands/database/sync_permissions.py index 03e0386ba0e..af5064ad3ea 100644 --- a/superset/commands/database/sync_permissions.py +++ b/superset/commands/database/sync_permissions.py @@ -36,12 +36,15 @@ from superset.commands.database.utils import ( add_vm, ping, ) +from superset.connectors.sqla.models import SqlaTable from superset.daos.database import DatabaseDAO from superset.daos.dataset import DatasetDAO from superset.db_engine_specs.base import GenericDBException from superset.exceptions import OAuth2RedirectError from superset.extensions import celery_app, db from superset.models.core import Database +from superset.models.helpers import skip_visibility_filter +from superset.models.slice import Slice from superset.utils.decorators import on_error, transaction logger = logging.getLogger(__name__) @@ -283,17 +286,29 @@ class SyncPermissionsCommand(BaseCommand): if existing_pvm: existing_pvm.view_menu.name = new_schema_perm_name - # rename permissions on datasets and charts - for dataset in DatabaseDAO.get_datasets( - self.db_connection_id, - catalog=catalog, - schema=schema, - ): - dataset.catalog_perm = new_catalog_perm_name - dataset.schema_perm = new_schema_perm_name - for chart in DatasetDAO.get_related_objects(dataset.id)["charts"]: - chart.catalog_perm = new_catalog_perm_name - chart.schema_perm = new_schema_perm_name + # Rename permissions on datasets and charts. Bypass the + # soft-delete visibility filter: a soft-deleted dataset's + # ``schema_perm``/``catalog_perm`` (and its charts') must still be + # rewritten on a database rename — otherwise restoring the dataset + # later brings back stale perm strings referencing the old + # database name (fail-closed for legitimate users, and matchable + # by a schema_access grant on a new database reusing the old + # name). Mirrors the same bypass in + # ``security_manager._update_vm_datasources_access``, which + # maintains the dataset-level ``perm`` string. ``Slice`` is + # included so chart perm rewrites keep working once charts gain + # soft delete (a no-op until then). + with skip_visibility_filter(db.session, SqlaTable, Slice): + for dataset in DatabaseDAO.get_datasets( + self.db_connection_id, + catalog=catalog, + schema=schema, + ): + dataset.catalog_perm = new_catalog_perm_name + dataset.schema_perm = new_schema_perm_name + for chart in DatasetDAO.get_related_objects(dataset.id)["charts"]: + chart.catalog_perm = new_catalog_perm_name + chart.schema_perm = new_schema_perm_name @celery_app.task(name="sync_database_permissions", soft_time_limit=600) diff --git a/superset/commands/database/uploaders/base.py b/superset/commands/database/uploaders/base.py index dbe892c1bb5..6a2475f8f11 100644 --- a/superset/commands/database/uploaders/base.py +++ b/superset/commands/database/uploaders/base.py @@ -33,6 +33,7 @@ from superset.commands.database.exceptions import ( DatabaseUploadFileTooLarge, DatabaseUploadNotSupported, DatabaseUploadSaveMetadataFailed, + DatabaseUploadSoftDeletedDatasetExistsError, ) from superset.connectors.sqla.models import SqlaTable from superset.daos.database import DatabaseDAO @@ -167,8 +168,6 @@ class UploadCommand(BaseCommand): ) ) - self._reader.read(self._file, self._model, self._table_name, self._schema) - sqla_table = ( db.session.query(SqlaTable) .filter_by( @@ -178,6 +177,32 @@ class UploadCommand(BaseCommand): ) .one_or_none() ) + if not sqla_table: + # The lookup above runs through the soft-delete visibility filter, + # so a soft-deleted dataset over this table is invisible here. + # Without this guard the upload would create an active twin of the + # hidden row — permanently blocking its restore — or die on the + # legacy unique constraint. Check BEFORE ``reader.read`` writes + # the file's contents into the analytics database: that write is + # outside this command's metadata transaction and would not roll + # back. With SOFT_DELETE off, leftover soft-deleted rows are + # visible to the lookup above, so this branch is never reached + # for them (degraded-mode semantics, consistent with the create + # paths). + # Deferred import: daos.dataset pulls in views.base, which + # circularly imports back into the commands package at app init + # (same constraint documented in daos/dataset.py re: PR #40573). + from superset.daos.dataset import ( # noqa: PLC0415 + DatasetDAO, + ) + + if soft_twin := DatasetDAO.find_soft_deleted_logical_duplicate( + self._model, Table(self._table_name, self._schema) + ): + raise DatabaseUploadSoftDeletedDatasetExistsError(str(soft_twin.uuid)) + + self._reader.read(self._file, self._model, self._table_name, self._schema) + if not sqla_table: sqla_table = SqlaTable( table_name=self._table_name, diff --git a/superset/commands/dataset/create.py b/superset/commands/dataset/create.py index bc19a4f04f1..33335ebbfcb 100644 --- a/superset/commands/dataset/create.py +++ b/superset/commands/dataset/create.py @@ -28,6 +28,7 @@ from superset.commands.dataset.exceptions import ( DatasetDataAccessIsNotAllowed, DatasetExistsValidationError, DatasetInvalidError, + DatasetSoftDeletedTwinExistsError, TableNotFoundValidationError, ) from superset.daos.dataset import DatasetDAO @@ -74,6 +75,14 @@ class CreateDatasetCommand(CreateMixin, BaseCommand): table = Table(table_name, schema, catalog) if not DatasetDAO.validate_uniqueness(database, table): + # Distinguish the hidden-twin case: uniqueness fails while + # the caller's dataset list looks empty. Raise the targeted + # 422 (naming the twin's uuid and the restore endpoint) + # instead of the opaque "already exists". + if soft_twin := DatasetDAO.find_soft_deleted_logical_duplicate( + database, table + ): + raise DatasetSoftDeletedTwinExistsError(str(soft_twin.uuid)) exceptions.append(DatasetExistsValidationError(table)) # Validate table exists on dataset if sql is not provided diff --git a/superset/commands/dataset/duplicate.py b/superset/commands/dataset/duplicate.py index 2be7be5690b..b0da1d44a59 100644 --- a/superset/commands/dataset/duplicate.py +++ b/superset/commands/dataset/duplicate.py @@ -68,6 +68,7 @@ class DuplicateDatasetCommand(CreateMixin, BaseCommand): table = SqlaTable(table_name=table_name, owners=owners) table.database = database table.schema = self._base_model.schema + table.catalog = self._base_model.catalog table.template_params = self._base_model.template_params table.normalize_columns = self._base_model.normalize_columns table.always_filter_main_dttm = self._base_model.always_filter_main_dttm @@ -121,7 +122,19 @@ class DuplicateDatasetCommand(CreateMixin, BaseCommand): if self._base_model and self._base_model.kind != "virtual": exceptions.append(DatasourceTypeInvalidError()) - if DatasetDAO.find_one_or_none(table_name=duplicate_name): + # Use the shared uniqueness check (same as create/update) rather than a + # name-only filtered lookup: it scopes to the base model's + # database/schema, is catalog-NULL-aware, and bypasses the soft-delete + # visibility filter. A filtered lookup misses a soft-deleted twin, so + # the duplicate would proceed and either hit a DB constraint as an + # opaque IntegrityError or — where no constraint applies (the + # model-level UniqueConstraint is metadata-only and the legacy + # _customer_location_uc is NULL-leaky) — create an active twin that + # permanently blocks restore of the soft-deleted dataset. + if base_model and not DatasetDAO.validate_uniqueness( + base_model.database, + Table(duplicate_name, base_model.schema, base_model.catalog), + ): exceptions.append(DatasetExistsValidationError(table=Table(duplicate_name))) try: diff --git a/superset/commands/dataset/exceptions.py b/superset/commands/dataset/exceptions.py index f8d0bd5a28d..9624bac2ba4 100644 --- a/superset/commands/dataset/exceptions.py +++ b/superset/commands/dataset/exceptions.py @@ -170,6 +170,49 @@ class DatasetUpdateFailedError(UpdateFailedError): message = _("Dataset could not be updated.") +class DatasetRestoreFailedError(UpdateFailedError): + # Restore semantically clears ``deleted_at``; it is an UPDATE, not a new + # row. ``UpdateFailedError`` is the nearest typed middle-tier base in the + # codebase. A dedicated ``RestoreFailedError`` in + # ``superset/commands/exceptions.py`` would be more precise across the + # entity rollouts but lives in already-merged infrastructure (#39977); + # introducing it can be a cross-entity follow-up. + message = _("Dataset could not be restored.") + + +class DatasetSoftDeletedTwinExistsError(CommandException): + """A soft-deleted dataset holds the physical table the caller targets. + + Single source of the blocked-by-hidden-twin message: the create command, + ``get_or_create_dataset``, and any future caller raise this so the + wording (restore pointer, executable recoveries only) cannot drift + between surfaces. + """ + + status = 422 + + def __init__(self, dataset_uuid: str) -> None: + super().__init__( + _( + "A soft-deleted dataset (uuid %(uuid)s) already references " + "this table. Restore it via POST " + "/api/v1/dataset/%(uuid)s/restore before creating a new " + "dataset over this table, or use a different table name.", + uuid=dataset_uuid, + ) + ) + + +class DatasetLogicalDuplicateError(CommandException): + status = 422 + message = _( + "Dataset cannot be restored because another active dataset already " + "references the same physical table (same database, catalog, schema, " + "and table name). Delete the duplicate or rename the table before " + "restoring." + ) + + class DatasetDeleteFailedError(DeleteFailedError): message = _("Datasets could not be deleted.") diff --git a/superset/commands/dataset/importers/v1/utils.py b/superset/commands/dataset/importers/v1/utils.py index f3d751193f6..8cb9c8c556a 100644 --- a/superset/commands/dataset/importers/v1/utils.py +++ b/superset/commands/dataset/importers/v1/utils.py @@ -37,7 +37,10 @@ from superset.commands.dataset.exceptions import ( MultiCatalogDisabledValidationError, ) from superset.commands.exceptions import ImportFailedError +from superset.commands.importers.v1.utils import find_existing_for_import from superset.connectors.sqla.models import SqlaTable +from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES +from superset.daos.dataset import DatasetDAO from superset.exceptions import SupersetSecurityException from superset.models.core import Database from superset.sql.parse import Table @@ -200,27 +203,204 @@ def import_dataset( # noqa: C901 force_data: bool = False, ignore_permissions: bool = False, ) -> SqlaTable: + """Import a dataset from a config dict, handling existing matches. + + Permission model for an existing UUID match: + + +--------------+---------------+---------------------+-----------------+ + | Existing row | overwrite arg | Caller has perms? | Outcome | + +==============+===============+=====================+=================+ + | alive | False | (n/a) | return existing | + +--------------+---------------+---------------------+-----------------+ + | alive | True | can_write + owner | UPDATE in place | + +--------------+---------------+---------------------+-----------------+ + | alive | True | can_write, | raise | + | | | not owner/admin | | + +--------------+---------------+---------------------+-----------------+ + | soft-deleted | False or True | can_write + owner | restore + UPDATE| + +--------------+---------------+---------------------+-----------------+ + | soft-deleted | False or True | can_write, | raise | + | | | not owner/admin | | + +--------------+---------------+---------------------+-----------------+ + | soft-deleted | False or True | not can_write | raise (Case B) | + +--------------+---------------+---------------------+-----------------+ + + Re-importing a soft-deleted UUID is implicitly a restore-with-update: + the user is bringing the dataset back by uploading it again. We apply + the same ownership check as the explicit overwrite path so non-owners + cannot resurrect via re-import, and we raise rather than silently + returning a soft-deleted row to callers without write permission + (which would let them reattach charts/dashboards to a deleted dataset). + """ can_write = ignore_permissions or security_manager.can_access( "can_write", "Dataset", ) - existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).first() + # `user` is None for background / example-loader paths (no Flask request + # user). Combined with ``can_write=True`` (typically from + # ``ignore_permissions=True``), the ownership checks in the restore / + # overwrite branches below are intentionally skipped because the caller has + # already established trust at the command level. user = get_user() - if existing: - if overwrite and can_write and user: - if user not in existing.owners and not security_manager.is_admin(): + # Tracks whether we entered the soft-deleted mutation path so the + # downstream `sync` decision (below) can reflect that an + # implicit-restore re-import is a clean replacement, not a merge. + is_soft_deleted_match = False + + if existing := find_existing_for_import(SqlaTable, config["uuid"]): + if existing.deleted_at is not None: + # RESTORE path — re-importing a soft-deleted UUID is an implicit + # restore-with-update, a distinct operation from overwriting an + # alive row, so it is handled in its own branch. + if not can_write: + # Case B: don't silently return a soft-deleted row to a caller + # without write permission — that would let the importer + # reattach charts/dashboards to a deleted dataset and produce + # broken charts. raise ImportFailedError( - "A dataset already exists and user doesn't " - "have permissions to overwrite it" + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "was deleted and re-import requires can_write permission " + "to restore it" ) - if not overwrite or not can_write: - return existing - config["id"] = existing.id + # ``user`` is None on background / example-loader paths; combined + # with ``can_write`` (typically from ``ignore_permissions=True``) + # the ownership check is intentionally skipped because the caller + # already established trust. + if user and ( + user not in existing.owners and not security_manager.is_admin() + ): + raise ImportFailedError( + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "already exists and user doesn't have permissions to " + "restore it" + ) + # Before clearing ``deleted_at``, refuse if another active dataset + # already references the same physical table. Otherwise the + # restore would produce two live ``SqlaTable`` rows for one + # physical table, breaking the logical-uniqueness contract. The + # shared DAO helper relies on the SoftDeleteMixin listener to + # consider only active rows, excludes ``existing`` itself via + # ``id !=``, and normalizes the catalog the same way create/update + # uniqueness does. + # Probe the POST-update identity: the uploaded config may rename + # table_name/schema/catalog, and the collision that matters is + # against the identity the restored row will end up with — not + # the stale stored one. Absent keys fall back to the stored + # values; explicit nulls are respected. + incoming_identity = Table( + config.get("table_name") or existing.table_name, + config.get("schema", existing.schema), + config.get("catalog", existing.catalog), + ) + if DatasetDAO.has_active_logical_duplicate(existing, incoming_identity): + raise ImportFailedError( + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "cannot be restored via re-import because another active " + "dataset already references the same physical table " + f"({incoming_identity.table!r}). Rename one of them and " + "retry." + ) + # Restore in place (clear ``deleted_at``) rather than + # hard-delete-and-replace: a hard delete would cascade through the + # chart back-reference and the delete-orphan child relationships + # (table_columns, sql_metrics), which the import would then need to + # reconstruct. + # + # How the restore lands as an UPDATE: clearing + # ``existing.deleted_at`` marks the in-session row dirty and the + # explicit flush emits the ``deleted_at = NULL`` UPDATE before + # ``SqlaTable.import_from_dict`` (below) does its own query-by-uuid + # lookup. Without the flush we would be relying on autoflush ahead + # of that internal query — correct under default session config but + # a hidden contract; the explicit flush makes it robust. The lookup + # then finds the now-live row (the listener filters ``deleted_at IS + # NULL``) and ``import_from_dict`` applies the config as field + # updates on the existing object, preserving the PK. Note: + # ``config["id"]`` is set defensively, but + # ``ImportExportMixin.import_from_dict`` strips it because + # ``SqlaTable.export_fields`` does not contain "id"; what actually + # binds to the existing row is the uuid uniqueness constraint used + # inside ``import_from_dict``. + # + # Snapshot ``deleted_at`` first so we can roll back the clear if the + # downstream ``import_from_dict`` hits the legacy + # ``MultipleResultsFound`` fallback. Without the rollback, an + # ambiguous import would leave the dataset half-restored + # (``deleted_at = None`` but upload contents unapplied). + original_deleted_at = existing.deleted_at + existing.restore() + db.session.flush() + is_soft_deleted_match = True + config["id"] = existing.id + else: + # OVERWRITE path — existing alive row. Without ``overwrite`` or + # write permission, return it unchanged (the pre-soft-delete + # overwrite-without-permission behaviour). + if not overwrite or not can_write: + return existing + if user and ( + user not in existing.owners and not security_manager.is_admin() + ): + raise ImportFailedError( + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "already exists and user doesn't have permissions to " + "overwrite it" + ) + # Mirror the REST update path's uniqueness contract: the uploaded + # config may rename this dataset onto the physical identity of a + # soft-deleted twin. ``import_from_dict``'s lookup cannot see the + # hidden row (visibility filter), so without this check the + # update would land cleanly and the live row would silently squat + # the trash row's identity — permanently 422-blocking its + # restore. ``validate_update_uniqueness`` bypasses the filter by + # design, so hidden twins block here exactly as they block + # ``UpdateDatasetCommand``. + overwrite_identity = Table( + config.get("table_name") or existing.table_name, + config.get("schema", existing.schema), + config.get("catalog", existing.catalog), + ) + if not DatasetDAO.validate_update_uniqueness( + existing.database, overwrite_identity, dataset_id=existing.id + ): + raise ImportFailedError( + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "cannot be overwritten: another dataset (possibly " + "soft-deleted) already references the target table " + f"({overwrite_identity.table!r}). Restore or rename the " + "other dataset, or change this upload's table name." + ) + config["id"] = existing.id elif not can_write: raise ImportFailedError( "Dataset doesn't exist and user doesn't have permission to create datasets" ) + else: + # Creating a brand-new dataset (no UUID match). A soft-deleted dataset + # may still claim this physical table; ``import_from_dict`` cannot see it + # (the visibility filter hides soft-deleted rows), so without this guard + # the import would create an active twin of a hidden dataset. The REST + # create path blocks the same collision via ``validate_uniqueness`` — + # mirror it here and direct the user to restore the existing dataset + # instead of leaving two rows for one physical table. + database = ( + db.session.query(Database).filter_by(id=config["database_id"]).first() + ) + if database is not None and ( + soft_twin := DatasetDAO.find_soft_deleted_logical_duplicate( + database, + Table( + config["table_name"], config.get("schema"), config.get("catalog") + ), + ) + ): + raise ImportFailedError( + f"Dataset {config['table_name']!r} cannot be imported because " + f"a soft-deleted dataset (uuid {soft_twin.uuid}) already " + "references the same physical table; restore that dataset " + "instead of importing a duplicate" + ) # Trusted imports (e.g. example loading) carry curated configs; only # untrusted user imports validate the catalog, like the access checks below. @@ -247,7 +427,12 @@ def import_dataset( # noqa: C901 attributes["extra"] = None # should we delete columns and metrics not present in the current import? - sync = ["columns", "metrics"] if overwrite else [] + # Restore-via-import of a soft-deleted dataset is implicitly a clean + # replacement (Option C): the user is bringing the dataset back by + # uploading it again, so children present in the live DB but absent + # from the upload should be removed, not silently merged. This matches + # what an explicit overwrite would do. + sync = ["columns", "metrics"] if (overwrite or is_soft_deleted_match) else [] # should we also load data into the dataset? data_uri = config.get("data") @@ -255,7 +440,7 @@ def import_dataset( # noqa: C901 # import recursively to include columns and metrics try: dataset = SqlaTable.import_from_dict(config, recursive=True, sync=sync) - except MultipleResultsFound: + except MultipleResultsFound as ex: # Finding multiple results when importing a dataset only happens because initially # noqa: E501 # datasets were imported without schemas (eg, `examples.NULL.users`), and later # they were fixed to have the default schema (eg, `examples.public.users`). If a @@ -263,8 +448,39 @@ def import_dataset( # noqa: C901 # fail because the UUID match will try to update `examples.NULL.users` to # `examples.public.users`, resulting in a conflict. # - # When that happens, we return the original dataset, unmodified. - dataset = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + # In the soft-deleted-restore case we cannot silently return + # the unmodified row: ``existing.deleted_at`` was already + # cleared above and the operator expects a restore-with-update. + # Returning the row without applying the upload would produce a + # half-restored state. Roll back the ``deleted_at`` clear and + # raise so the operator can resolve the legacy-NULL-schema + # ambiguity before re-uploading. + if is_soft_deleted_match: + # ``is_soft_deleted_match`` is only ever set inside the + # ``if existing := ...`` walrus block, so ``existing`` is + # guaranteed non-None here. The assert pins the invariant + # for mypy. + assert existing is not None + existing.deleted_at = original_deleted_at + db.session.flush() + raise ImportFailedError( + f"Dataset {existing.table_name!r} (uuid {config['uuid']}) " + "matches more than one existing row, so the restore-and-" + "update cannot pick a target. Resolve the duplicate rows " + "manually before retrying." + ) from ex + # On the non-soft-deleted overwrite path the legacy contract + # holds: return the existing row unmodified. Bypasses the + # visibility filter so a soft-deleted duplicate can be located + # too — without the bypass the listener would hide the row and + # the ``.one()`` would raise NoResultFound, masking the + # original MultipleResultsFound. + dataset = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter_by(uuid=config["uuid"]) + .one() + ) if dataset.id is None: db.session.flush() @@ -289,7 +505,7 @@ def import_dataset( # noqa: C901 if data_uri and (not table_exists or force_data): load_data(data_uri, dataset, dataset.database) - if (user := get_user()) and user not in dataset.owners: + if user and user not in dataset.owners: dataset.owners.append(user) return dataset diff --git a/superset/commands/dataset/restore.py b/superset/commands/dataset/restore.py new file mode 100644 index 00000000000..baf2ade34c3 --- /dev/null +++ b/superset/commands/dataset/restore.py @@ -0,0 +1,60 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Command to restore a soft-deleted dataset.""" + +from superset.commands.dataset.exceptions import ( + DatasetForbiddenError, + DatasetLogicalDuplicateError, + DatasetNotFoundError, + DatasetRestoreFailedError, +) +from superset.commands.restore import BaseRestoreCommand +from superset.connectors.sqla.models import SqlaTable +from superset.daos.dataset import DatasetDAO + + +class RestoreDatasetCommand(BaseRestoreCommand[SqlaTable]): + """Restore a soft-deleted dataset by clearing its ``deleted_at`` field. + + Most behaviour is inherited from ``BaseRestoreCommand``. The override + here adds a logical-duplicate check: DB-level enforcement of + ``SqlaTable`` logical uniqueness is inconsistent (the model-level + 4-column ``UniqueConstraint`` is metadata-only — no migration creates + it — and the legacy 3-column ``_customer_location_uc`` has no catalog + leg and is NULL-leaky), so another active dataset may have claimed the + same physical identity while this one was soft-deleted. The + application-level check refuses the restore with a clean 422 rather + than relying on the DB, which would either reject with an opaque + IntegrityError or — where no constraint applies — silently allow two + active datasets pointing at the same physical table. + """ + + dao = DatasetDAO + not_found_exc = DatasetNotFoundError + forbidden_exc = DatasetForbiddenError + restore_failed_exc = DatasetRestoreFailedError + + def validate(self) -> SqlaTable: # type: ignore[override] + model = super().validate() + # DB-level uniqueness enforcement is inconsistent across schema + # builds (see class docstring), so restoring must refuse here if + # another active dataset claimed the same physical table while this + # one was soft-deleted — a clean 422 instead of an IntegrityError + # or a silent twin. + if DatasetDAO.has_active_logical_duplicate(model): + raise DatasetLogicalDuplicateError() + return model diff --git a/superset/commands/restore.py b/superset/commands/restore.py index e90a569959b..6b3207b3324 100644 --- a/superset/commands/restore.py +++ b/superset/commands/restore.py @@ -74,14 +74,19 @@ class BaseRestoreCommand(BaseCommand, Generic[T]): _perform() def validate(self) -> T: # type: ignore[override] - # ``skip_visibility_filter=True`` is the *only* bypass — the - # entity's RBAC ``base_filter`` stays in effect, matching the - # behavior of ``find_by_ids`` on the existing delete paths. - # Restore should not see rows the user cannot see in the live - # UI; ownership is then verified by ``raise_for_ownership``. + # Both bypasses are deliberate. ``skip_visibility_filter`` lets the + # lookup see the soft-deleted row at all. ``skip_base_filter`` keeps + # an owner's own trash reachable even when the entity's RBAC + # ``base_filter`` has no ownership leg (charts/datasets filter by + # datasource access): the security model's ``raise_for_access`` + # counts ownership as datasource access, so a lost grant must not + # hide a row from the one audience that can restore it. The restore + # audience is enforced below by ``raise_for_ownership`` — owners or + # admin; anyone else holding a valid uuid gets 403. model = self.dao.find_by_id( self._model_uuid, id_column="uuid", + skip_base_filter=True, skip_visibility_filter=True, ) if model is None: diff --git a/superset/connectors/sqla/models.py b/superset/connectors/sqla/models.py index 735adb0817b..c02dfbfb1f6 100644 --- a/superset/connectors/sqla/models.py +++ b/superset/connectors/sqla/models.py @@ -104,6 +104,7 @@ from superset.models.helpers import ( ExploreMixin, ImportExportMixin, QueryResult, + SoftDeleteMixin, SQLA_QUERY_KEYS, validate_adhoc_subquery, ) @@ -1303,6 +1304,7 @@ sqlatable_user = DBTable( class SqlaTable( CoreDataset, + SoftDeleteMixin, BaseDatasource, ExploreMixin, ): # pylint: disable=too-many-public-methods diff --git a/superset/daos/dataset.py b/superset/daos/dataset.py index 4d819deb5bd..b69e4b760d5 100644 --- a/superset/daos/dataset.py +++ b/superset/daos/dataset.py @@ -155,6 +155,31 @@ class DatasetDAO(BaseDAO[SqlaTable]): logger.warning("Got an error %s validating table: %s", str(ex), table) return False + @staticmethod + def _catalog_identity_filter( + catalog: str | None, + default_catalog: str | None, + ) -> Any: + """Null-aware catalog predicate for physical-identity matching. + + A row stored with ``catalog = NULL`` is semantically the database's + default catalog (older datasets were created before the catalog column + existed, and multi-catalog-disabled databases never set it). So when the + normalized probe catalog resolves to the default, ``catalog = NULL`` rows + must also match — otherwise create/update/restore/import disagree on what + counts as the same physical table and let a default-catalog twin slip + through. + + - probe is the default catalog → match ``= default`` OR ``IS NULL`` + - probe is ``None`` (no catalog support) → match ``IS NULL`` + - probe is a non-default catalog → match ``= probe`` exactly + """ + if catalog is not None and catalog == default_catalog: + return or_(SqlaTable.catalog == catalog, SqlaTable.catalog.is_(None)) + if catalog is None: + return SqlaTable.catalog.is_(None) + return SqlaTable.catalog == catalog + @staticmethod def validate_uniqueness( database: Database, @@ -163,20 +188,45 @@ class DatasetDAO(BaseDAO[SqlaTable]): ) -> bool: # The catalog might not be set even if the database supports catalogs, in case # multi-catalog is disabled. - catalog = table.catalog or database.get_default_catalog() + default_catalog = database.get_default_catalog() + catalog = table.catalog or default_catalog - dataset_query = db.session.query(SqlaTable).filter( - SqlaTable.table_name == table.table, - SqlaTable.schema == table.schema, - SqlaTable.catalog == catalog, - SqlaTable.database_id == database.id, + # Bypass the soft-delete visibility filter so a soft-deleted row with + # the same physical identity still blocks the create. Otherwise a + # delete-then-create-then-restore sequence could produce two live + # datasets pointing at the same physical table. + # + # The bypass MUST be session-scoped, not per-query. The + # ``db.session.query(dataset_query.exists()).scalar()`` pattern below + # builds an OUTER query whose ``execution_options`` are independent + # of the inner ``dataset_query`` — the listener fires on the outer + # execute and would attach ``deleted_at IS NULL`` to all SqlaTable + # references in the statement (including inside the EXISTS + # subquery) via ``with_loader_criteria(include_aliases=True)``. + # A per-query option on the inner query never reaches that listener. + # + # avoid app-init regression: a module-top import from + # ``superset.models.helpers`` eagerly loads ``core.py``, whose model + # classes evaluate ``encrypted_field_factory.create(...)`` at + # class-definition time and raise "App not initialized yet" when no + # Flask app context exists (see PR #40573). + from superset.models.helpers import ( # pylint: disable=import-outside-toplevel + skip_visibility_filter, ) - if dataset_id: - # make sure the dataset found is different from the target (if any) - dataset_query = dataset_query.filter(SqlaTable.id != dataset_id) + with skip_visibility_filter(db.session, SqlaTable): + dataset_query = db.session.query(SqlaTable).filter( + SqlaTable.table_name == table.table, + SqlaTable.schema == table.schema, + DatasetDAO._catalog_identity_filter(catalog, default_catalog), + SqlaTable.database_id == database.id, + ) - return not db.session.query(dataset_query.exists()).scalar() + if dataset_id: + # make sure the dataset found is different from the target (if any) + dataset_query = dataset_query.filter(SqlaTable.id != dataset_id) + + return not db.session.query(dataset_query.exists()).scalar() @staticmethod def validate_update_uniqueness( @@ -184,18 +234,108 @@ class DatasetDAO(BaseDAO[SqlaTable]): table: Table, dataset_id: int, ) -> bool: - # The catalog might not be set even if the database supports catalogs, in case - # multi-catalog is disabled. - catalog = table.catalog or database.get_default_catalog() + """Update-time twin of ``validate_uniqueness``. - dataset_query = db.session.query(SqlaTable).filter( - SqlaTable.table_name == table.table, - SqlaTable.database_id == database.id, - SqlaTable.schema == table.schema, - SqlaTable.catalog == catalog, - SqlaTable.id != dataset_id, + Delegates outright — the identity rule, catalog normalization, and + the session-scoped visibility bypass (and their rationale) live in + ``validate_uniqueness`` so they cannot drift between the create and + update paths. + """ + return DatasetDAO.validate_uniqueness(database, table, dataset_id) + + @staticmethod + def has_active_logical_duplicate( + model: SqlaTable, + table: Table | None = None, + ) -> bool: + """Return True iff another *active* dataset shares model's physical table. + + ``table`` overrides the identity to probe (used by restore-via-import, + where the uploaded config may rename the dataset — the collision that + matters is against the *post-update* identity, not the stored one). + Physical identity is ``(database_id, catalog, schema, table_name)``. + ``model``'s catalog is normalized to the database default when unset — + the same rule ``validate_uniqueness``/``validate_update_uniqueness`` + apply — so create, update, restore, and re-import agree on what counts + as the same physical table. Catalog matching is null-aware via + ``_catalog_identity_filter``: when the normalized catalog is the database + default, both ``catalog = default`` and ``catalog IS NULL`` rows match, + so a default-catalog twin is caught whichever way either row stored its + catalog. + + Unlike ``validate_uniqueness``, this does NOT use + ``skip_visibility_filter``: it relies on the ``SoftDeleteMixin`` listener + to auto-append ``deleted_at IS NULL`` so only *active* rows match. Do not + add the bypass here — it would broaden the check to soft-deleted rows and + silently refuse legitimate restores. ``id != model.id`` excludes the row + itself. + + Assumes an active app context and a session-attached ``model`` so + ``db.session`` and the lazy ``model.database`` relationship resolve. + """ + # The catalog might not be set even if the database supports catalogs, + # in case multi-catalog is disabled. + default_catalog = model.database.get_default_catalog() + probe_table_name = table.table if table else model.table_name + probe_schema = table.schema if table else model.schema + probe_catalog = (table.catalog if table else model.catalog) or default_catalog + # This is a best-effort read for a friendly early error; it does not + # lock, so two concurrent creates could both pass it. That race is + # benign: the DB-level UniqueConstraint on + # (database_id, catalog, schema, table_name) (see SqlaTable.__table_args__) + # is the real guard — the losing commit fails with IntegrityError rather + # than landing a second active dataset over the same physical table. + return ( + db.session.query(SqlaTable.id) + .filter( + SqlaTable.database_id == model.database_id, + DatasetDAO._catalog_identity_filter(probe_catalog, default_catalog), + SqlaTable.schema == probe_schema, + SqlaTable.table_name == probe_table_name, + SqlaTable.id != model.id, + ) + .first() + is not None ) - return not db.session.query(dataset_query.exists()).scalar() + + @staticmethod + def find_soft_deleted_logical_duplicate( + database: Database, + table: Table, + ) -> SqlaTable | None: + """Return a *soft-deleted* dataset sharing table's physical identity. + + Used by the YAML importer's create path: ``import_from_dict`` can't see + soft-deleted rows (the visibility filter hides them), so importing a + dataset with a fresh UUID but the same physical table as a soft-deleted + dataset would create an active twin of a hidden row. This bypasses the + visibility filter (``skip_visibility_filter``) and restricts to + ``deleted_at IS NOT NULL`` so the caller can block the import and direct + the user to restore the existing dataset instead. Catalog matching is + null-aware via ``_catalog_identity_filter`` so a default-catalog twin is + caught however either row stored its catalog. + """ + # avoid app-init regression: see ``validate_uniqueness`` — a module-top + # import of ``skip_visibility_filter`` eagerly loads model classes that + # raise "App not initialized yet" outside an app context (PR #40573). + from superset.models.helpers import ( # pylint: disable=import-outside-toplevel + skip_visibility_filter, + ) + + default_catalog = database.get_default_catalog() + catalog = table.catalog or default_catalog + with skip_visibility_filter(db.session, SqlaTable): + return ( + db.session.query(SqlaTable) + .filter( + SqlaTable.database_id == database.id, + DatasetDAO._catalog_identity_filter(catalog, default_catalog), + SqlaTable.schema == table.schema, + SqlaTable.table_name == table.table, + SqlaTable.deleted_at.is_not(None), + ) + .first() + ) @staticmethod def validate_columns_exist(dataset_id: int, columns_ids: list[int]) -> bool: diff --git a/superset/daos/datasource.py b/superset/daos/datasource.py index 1a0a9531501..61f12a34538 100644 --- a/superset/daos/datasource.py +++ b/superset/daos/datasource.py @@ -114,6 +114,17 @@ class DatasourceDAO(BaseDAO[Datasource]): db_table.c.id == ds_table.c.database_id, ) + # ``build_dataset_query`` uses a Core ``select`` rather than an ORM + # query, so the ``SoftDeleteMixin`` listener (which runs on + # ``do_orm_execute``) does not append the visibility filter. Add it + # explicitly here so soft-deleted datasets don't count toward the + # combined datasource list, pagination totals, or count totals. + # Deliberately NOT gated on the SOFT_DELETE flag (unlike the ORM + # listener): always-hiding is the safer failure mode for this + # aggregate listing, and the flag-independence is documented in + # UPDATING.md's "flag-independent parts" list. + ds_q = ds_q.where(ds_table.c.deleted_at.is_(None)) + if not security_manager.can_access_all_datasources(): ds_q = ds_q.where(get_dataset_access_filters(SqlaTable)) diff --git a/superset/dashboards/filters.py b/superset/dashboards/filters.py index f43d8dfb27b..faf4827d313 100644 --- a/superset/dashboards/filters.py +++ b/superset/dashboards/filters.py @@ -273,31 +273,12 @@ class DashboardDeletedStateFilter( # pylint: disable=too-few-public-methods ): """Rison filter for the GET list that exposes soft-deleted dashboards. - Soft-deleted rows are additionally scoped to the **restore audience**: - only the dashboard's owners (or admins) may enumerate them. This mirrors - ``RestoreDashboardCommand``'s ``raise_for_ownership`` check, so a - read-access non-owner (who can see the dashboard via published-datasource - access or dashboard RBAC) cannot list soft-deleted dashboards they could - never restore. Live rows are unaffected — they keep their normal - ``DashboardAccessFilter`` visibility. + Soft-deleted rows are scoped to the **restore audience** (owners or + admins) by ``BaseDeletedStateFilter._scope_to_restore_audience`` — the + cross-entity contract lives on the base, so this class is a pure + declaration. Live rows keep their normal ``DashboardAccessFilter`` + visibility. """ arg_name = "dashboard_deleted_state" model = Dashboard - - def apply(self, query: Query, value: Any) -> Query: - query = super().apply(query, value) - normalized = self._normalize(value) - if normalized not in {"include", "only"} or security_manager.is_admin(): - return query - - # Non-admins may only see soft-deleted dashboards they own. ``any()`` - # emits an EXISTS subquery so it composes with the base access filter - # without producing duplicate rows from a join. - owned = Dashboard.owners.any(security_manager.user_model.id == get_user_id()) - if normalized == "only": - # ``super().apply`` already restricted to ``deleted_at IS NOT NULL``. - return query.filter(owned) - # ``include``: keep all live rows (normal access) and add only the - # soft-deleted rows this user owns. - return query.filter(or_(Dashboard.deleted_at.is_(None), owned)) diff --git a/superset/datasets/api.py b/superset/datasets/api.py index 4801e3afcc5..1167dd60d37 100644 --- a/superset/datasets/api.py +++ b/superset/datasets/api.py @@ -42,13 +42,17 @@ from superset.commands.dataset.exceptions import ( DatasetDeleteFailedError, DatasetForbiddenError, DatasetInvalidError, + DatasetLogicalDuplicateError, DatasetNotFoundError, DatasetRefreshFailedError, + DatasetRestoreFailedError, + DatasetSoftDeletedTwinExistsError, DatasetUpdateFailedError, ) from superset.commands.dataset.export import ExportDatasetsCommand from superset.commands.dataset.importers.dispatcher import ImportDatasetsCommand from superset.commands.dataset.refresh import RefreshDatasetCommand +from superset.commands.dataset.restore import RestoreDatasetCommand from superset.commands.dataset.update import UpdateDatasetCommand from superset.commands.dataset.warm_up_cache import DatasetWarmUpCacheCommand from superset.commands.exceptions import CommandException @@ -59,7 +63,11 @@ from superset.constants import MODEL_API_RW_METHOD_PERMISSION_MAP, RouteMethod from superset.daos.dashboard import DashboardDAO from superset.daos.dataset import DatasetDAO from superset.databases.filters import DatabaseFilter -from superset.datasets.filters import DatasetCertifiedFilter, DatasetIsNullOrEmptyFilter +from superset.datasets.filters import ( + DatasetCertifiedFilter, + DatasetDeletedStateFilter, + DatasetIsNullOrEmptyFilter, +) from superset.datasets.schemas import ( DatasetCacheWarmUpRequestSchema, DatasetCacheWarmUpResponseSchema, @@ -87,25 +95,40 @@ from superset.views.base_api import ( statsd_metrics, ) from superset.views.error_handling import handle_api_exception -from superset.views.filters import BaseFilterRelatedUsers, FilterRelatedOwners +from superset.views.filters import ( + BaseFilterRelatedUsers, + FilterRelatedOwners, + SoftDeleteApiMixin, +) logger = logging.getLogger(__name__) -class DatasetRestApi(BaseSupersetModelRestApi): +class DatasetRestApi(SoftDeleteApiMixin, BaseSupersetModelRestApi): datamodel = SQLAInterface(SqlaTable) base_filters = [["id", DatasourceFilter, lambda: []]] resource_name = "dataset" allow_browser_login = True class_permission_name = "Dataset" - method_permission_name = MODEL_API_RW_METHOD_PERMISSION_MAP + # Custom methods (``restore``) need an explicit entry; FAB's @protect() + # decorator falls back to ``can__`` (i.e. + # ``can_restore_Dataset``) when the mapping is missing, which standard + # roles don't carry. Mirrors the permission model documented for + # ``DELETE`` / ``bulk_delete``: endpoint-level ``can_write`` plus + # resource-level ``raise_for_ownership``. See themes/api.py for the + # established pattern. + method_permission_name = { + **MODEL_API_RW_METHOD_PERMISSION_MAP, + "restore": "write", + } include_route_methods = RouteMethod.REST_MODEL_VIEW_CRUD_SET | { RouteMethod.EXPORT, RouteMethod.IMPORT, RouteMethod.RELATED, RouteMethod.DISTINCT, "bulk_delete", + "restore", "refresh", "related_objects", "duplicate", @@ -138,7 +161,6 @@ class DatasetRestApi(BaseSupersetModelRestApi): "schema", "sql", "table_name", - "uuid", ] list_select_columns = list_columns + ["changed_on", "changed_by_fk"] order_columns = [ @@ -273,7 +295,7 @@ class DatasetRestApi(BaseSupersetModelRestApi): } search_filters = { "sql": [DatasetIsNullOrEmptyFilter], - "id": [DatasetCertifiedFilter], + "id": [DatasetCertifiedFilter, DatasetDeletedStateFilter], } search_columns = [ "id", @@ -365,6 +387,8 @@ class DatasetRestApi(BaseSupersetModelRestApi): data=new_model.data, uuid=new_model.uuid, ) + except DatasetSoftDeletedTwinExistsError as ex: + return self.response_422(message=str(ex)) except DatasetInvalidError as ex: return self.response_422(message=ex.normalized_messages()) except DatasetCreateFailedError as ex: @@ -470,10 +494,18 @@ class DatasetRestApi(BaseSupersetModelRestApi): log_to_statsd=False, ) def delete(self, pk: int) -> Response: - """Delete a Dataset. + """Delete a dataset. + + When the ``SOFT_DELETE`` feature flag is enabled, marks the dataset + as deleted (sets ``deleted_at``) and hides it from list/detail + endpoints and relationship loads; the row is preserved and + recoverable via ``POST /api/v1/dataset//restore`` by an owner + or admin. With the flag disabled (the default), the dataset is + permanently hard-deleted and is not recoverable. --- delete: - summary: Delete a dataset + summary: Delete a dataset (soft delete, recoverable via restore, + when the SOFT_DELETE feature flag is enabled; permanent otherwise) parameters: - in: path schema: @@ -866,9 +898,17 @@ class DatasetRestApi(BaseSupersetModelRestApi): ) def bulk_delete(self, **kwargs: Any) -> Response: """Bulk delete datasets. + + When the ``SOFT_DELETE`` feature flag is enabled, marks each dataset + as deleted (sets ``deleted_at``) and hides it from list/detail + endpoints and relationship loads; rows are preserved and recoverable + via ``POST /api/v1/dataset//restore`` by an owner or admin. + With the flag disabled (the default), the datasets are permanently + hard-deleted and are not recoverable. --- delete: - summary: Bulk delete datasets + summary: Bulk delete datasets (soft delete, recoverable via restore, + when the SOFT_DELETE feature flag is enabled; permanent otherwise) parameters: - in: query name: q @@ -917,6 +957,64 @@ class DatasetRestApi(BaseSupersetModelRestApi): except DatasetDeleteFailedError as ex: return self.response_422(message=str(ex)) + @expose("//restore", methods=("POST",)) + @protect() + @safe + @statsd_metrics + @event_logger.log_this_with_context( + action=lambda self, *args, **kwargs: f"{self.__class__.__name__}.restore", + log_to_statsd=False, + ) + def restore(self, uuid: str) -> Response: + """Restore a soft-deleted dataset. + --- + post: + summary: Restore a soft-deleted dataset + parameters: + - in: path + schema: + type: string + format: uuid + name: uuid + responses: + 200: + description: Dataset restored + content: + application/json: + schema: + type: object + properties: + message: + type: string + 401: + $ref: '#/components/responses/401' + 403: + $ref: '#/components/responses/403' + 404: + $ref: '#/components/responses/404' + 422: + $ref: '#/components/responses/422' + 500: + $ref: '#/components/responses/500' + """ + try: + RestoreDatasetCommand(uuid).run() + return self.response(200, message="OK") + except DatasetNotFoundError: + return self.response_404() + except DatasetForbiddenError: + return self.response_403() + except DatasetLogicalDuplicateError as ex: + return self.response_422(message=str(ex)) + except DatasetRestoreFailedError as ex: + logger.error( + "Error restoring model %s: %s", + self.__class__.__name__, + str(ex), + exc_info=True, + ) + return self.response_422(message=str(ex)) + @expose("/import/", methods=("POST",)) @protect() @statsd_metrics @@ -927,6 +1025,15 @@ class DatasetRestApi(BaseSupersetModelRestApi): @requires_form_data def import_(self) -> Response: """Import dataset(s) with associated databases. + + When the ``SOFT_DELETE`` feature flag is enabled and an imported + dataset's UUID matches an existing **soft-deleted** dataset, the + import restores that dataset and applies the upload's contents — + **even when ``overwrite`` is not set**. Active datasets keep the + usual contract (never mutated without ``overwrite=true``); a + soft-deleted UUID match is treated as an explicit request to bring + the dataset back. Requires ``can_write`` and ownership of the + deleted row (or admin). See UPDATING.md for details. --- post: summary: Import dataset(s) with associated databases @@ -1139,6 +1246,10 @@ class DatasetRestApi(BaseSupersetModelRestApi): try: tbl = CreateDatasetCommand(body).run() return self.response(200, result={"table_id": tbl.id}) + except DatasetSoftDeletedTwinExistsError as ex: + # Targeted hidden-twin 422 (single-sourced in the exception): + # names the twin's uuid and the restore endpoint. + return self.response_422(message=str(ex)) except DatasetInvalidError as ex: return self.response_422(message=ex.normalized_messages()) except DatasetCreateFailedError as ex: diff --git a/superset/datasets/filters.py b/superset/datasets/filters.py index e72eb281818..a2704545080 100644 --- a/superset/datasets/filters.py +++ b/superset/datasets/filters.py @@ -14,12 +14,15 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. + + from flask_babel import lazy_gettext as _ from sqlalchemy import not_, or_ from sqlalchemy.orm.query import Query from superset.connectors.sqla.models import SqlaTable from superset.views.base import BaseFilter +from superset.views.filters import BaseDeletedStateFilter class DatasetIsNullOrEmptyFilter(BaseFilter): # pylint: disable=too-few-public-methods @@ -51,3 +54,19 @@ class DatasetCertifiedFilter(BaseFilter): # pylint: disable=too-few-public-meth ) ) return query + + +class DatasetDeletedStateFilter( # pylint: disable=too-few-public-methods + BaseDeletedStateFilter +): + """Rison filter for the GET list that exposes soft-deleted datasets. + + Soft-deleted rows are scoped to the **restore audience** (owners or + admins) by ``BaseDeletedStateFilter._scope_to_restore_audience`` — the + cross-entity contract lives on the base, so this class is a pure + declaration. Live rows keep their normal ``DatasourceFilter`` + visibility. + """ + + arg_name = "dataset_deleted_state" + model = SqlaTable diff --git a/superset/mcp_service/dataset/tool/create_dataset.py b/superset/mcp_service/dataset/tool/create_dataset.py index 5767a450551..599b81b5fcd 100644 --- a/superset/mcp_service/dataset/tool/create_dataset.py +++ b/superset/mcp_service/dataset/tool/create_dataset.py @@ -33,6 +33,7 @@ from superset.commands.dataset.create import CreateDatasetCommand from superset.commands.dataset.exceptions import ( DatasetCreateFailedError, DatasetInvalidError, + DatasetSoftDeletedTwinExistsError, ) from superset.extensions import event_logger from superset.mcp_service.dataset.schemas import ( @@ -149,6 +150,13 @@ async def create_dataset( ) return result + except DatasetSoftDeletedTwinExistsError as exc: + # Raised directly by validate() (not wrapped in DatasetInvalidError): + # a soft-deleted dataset still occupies this physical table. Surface + # the actionable restore-or-rename message rather than letting it fall + # through to the generic "unexpected error" branch. + await ctx.warning("Dataset creation blocked by soft-deleted twin: %s" % (exc,)) + return DatasetError.create(error=str(exc), error_type="DatasetExistsError") except DatasetInvalidError as exc: # CreateDatasetCommand.validate() collects validation errors into # DatasetInvalidError.exceptions, never raising them directly. diff --git a/superset/mcp_service/dataset/tool/create_virtual_dataset.py b/superset/mcp_service/dataset/tool/create_virtual_dataset.py index a8cea008cbe..c134f97ddf5 100644 --- a/superset/mcp_service/dataset/tool/create_virtual_dataset.py +++ b/superset/mcp_service/dataset/tool/create_virtual_dataset.py @@ -114,6 +114,7 @@ async def create_virtual_dataset( from superset.commands.dataset.exceptions import ( DatasetCreateFailedError, DatasetInvalidError, + DatasetSoftDeletedTwinExistsError, DatasetUpdateFailedError, ) from superset.mcp_service.utils.url_utils import get_superset_base_url @@ -164,6 +165,20 @@ async def create_virtual_dataset( url=dataset_url, ) + except DatasetSoftDeletedTwinExistsError as exc: + # Raised directly by validate() (not wrapped in DatasetInvalidError): + # a soft-deleted dataset still occupies this physical table. Return the + # actionable restore-or-rename message instead of an unexpected error. + await ctx.warning(f"Virtual dataset blocked by soft-deleted twin: {exc}") + return CreateVirtualDatasetResponse( + id=None, + dataset_name=request.dataset_name, + sql=request.sql, + database_id=request.database_id, + columns=[], + url=None, + error=str(exc), + ) except DatasetInvalidError as exc: messages = exc.normalized_messages() await ctx.warning(f"Virtual dataset validation failed: {messages}") diff --git a/superset/migrations/versions/2026-05-08_12-10_3a8e6f2c1b95_add_deleted_at_to_tables.py b/superset/migrations/versions/2026-05-08_12-10_3a8e6f2c1b95_add_deleted_at_to_tables.py new file mode 100644 index 00000000000..59b286cf331 --- /dev/null +++ b/superset/migrations/versions/2026-05-08_12-10_3a8e6f2c1b95_add_deleted_at_to_tables.py @@ -0,0 +1,58 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Add deleted_at column and index to tables for soft-delete. + +Adds a nullable ``deleted_at`` column and an index on it to the +``tables`` table to support soft deletion of datasets. Companion to +the ``SoftDeleteMixin`` infrastructure shipped in PR #39977. + +Revision ID: 3a8e6f2c1b95 +Revises: 7c4a8d09ca37 +Create Date: 2026-05-08 12:10:00.000000 +""" + +from sqlalchemy import Column, DateTime + +from superset.migrations.shared.utils import ( + add_columns, + create_index, + drop_columns, + drop_index, +) + +# revision identifiers, used by Alembic. +revision = "3a8e6f2c1b95" +# Re-pointed onto the charts soft-delete migration (add_deleted_at_to_slices), +# the current master head, so this stacks linearly rather than forking a +# second head. The two deleted_at additions are independent; ordering is +# arbitrary, only linearity matters. +down_revision = "7c4a8d09ca37" + +TABLE_NAME = "tables" +INDEX_NAME = f"ix_{TABLE_NAME}_deleted_at" + + +def upgrade() -> None: + """Add the soft-delete column and its supporting index to ``tables``.""" + add_columns(TABLE_NAME, Column("deleted_at", DateTime(), nullable=True)) + create_index(TABLE_NAME, INDEX_NAME, ["deleted_at"]) + + +def downgrade() -> None: + """Reverse ``upgrade``: drop the index, then the column.""" + drop_index(TABLE_NAME, INDEX_NAME) + drop_columns(TABLE_NAME, "deleted_at") diff --git a/superset/security/manager.py b/superset/security/manager.py index 352edf7e9c3..d065668fb97 100644 --- a/superset/security/manager.py +++ b/superset/security/manager.py @@ -2552,6 +2552,9 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods from superset.connectors.sqla.models import ( # pylint: disable=import-outside-toplevel SqlaTable, ) + from superset.models.helpers import ( # pylint: disable=import-outside-toplevel + skip_visibility_filter, + ) from superset.models.slice import ( # pylint: disable=import-outside-toplevel Slice, ) @@ -2560,11 +2563,16 @@ class SupersetSecurityManager( # pylint: disable=too-many-public-methods sqlatable_table = SqlaTable.__table__ # pylint: disable=no-member chart_table = Slice.__table__ # pylint: disable=no-member new_database_name = target.database_name - datasets = ( - self.session.query(SqlaTable) - .filter(SqlaTable.database_id == target.id) - .all() - ) + # Bypass the soft-delete visibility filter: a soft-deleted dataset's perm + # strings (and its charts') must still be rewritten on a database rename, + # otherwise restoring that dataset later brings back stale dataset/schema/ + # catalog permission strings referencing the old database name. + with skip_visibility_filter(self.session, SqlaTable): + datasets = ( + self.session.query(SqlaTable) + .filter(SqlaTable.database_id == target.id) + .all() + ) updated_view_menus: list[ViewMenu] = [] for dataset in datasets: old_dataset_vm_name = self.get_dataset_perm( diff --git a/superset/translations/ar/LC_MESSAGES/messages.po b/superset/translations/ar/LC_MESSAGES/messages.po index be919c1e838..d2d450d1c4e 100644 --- a/superset/translations/ar/LC_MESSAGES/messages.po +++ b/superset/translations/ar/LC_MESSAGES/messages.po @@ -36,6 +36,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -5091,6 +5118,9 @@ msgstr "تعذر إنشاء مجموعة بيانات." msgid "Dataset could not be duplicated." msgstr "تعذر تكرار مجموعة البيانات." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "تعذر تحديث مجموعة البيانات." diff --git a/superset/translations/ca/LC_MESSAGES/messages.po b/superset/translations/ca/LC_MESSAGES/messages.po index 1f3ce639b43..87d40017619 100644 --- a/superset/translations/ca/LC_MESSAGES/messages.po +++ b/superset/translations/ca/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4962,6 +4989,9 @@ msgstr "El conjunt de dades no s'ha pogut crear." msgid "Dataset could not be duplicated." msgstr "El conjunt de dades no s'ha pogut duplicar." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "El conjunt de dades no s'ha pogut actualitzar." diff --git a/superset/translations/cs/LC_MESSAGES/messages.po b/superset/translations/cs/LC_MESSAGES/messages.po index 739675d3a09..8a13e526e1f 100644 --- a/superset/translations/cs/LC_MESSAGES/messages.po +++ b/superset/translations/cs/LC_MESSAGES/messages.po @@ -30,6 +30,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4747,6 +4774,9 @@ msgstr "Sadu dat se nepodařilo vytvořit." msgid "Dataset could not be duplicated." msgstr "Sadu dat se nepodařilo duplikovat." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Sadu dat se nepodařilo aktualizovat." diff --git a/superset/translations/de/LC_MESSAGES/messages.po b/superset/translations/de/LC_MESSAGES/messages.po index f8b789bd100..e3719dc01e9 100644 --- a/superset/translations/de/LC_MESSAGES/messages.po +++ b/superset/translations/de/LC_MESSAGES/messages.po @@ -29,6 +29,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4672,6 +4699,9 @@ msgstr "Datensatz konnte nicht erstellt werden." msgid "Dataset could not be duplicated." msgstr "Der Datensatz konnte nicht dupliziert werden." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Der Datensatz konnte nicht aktualisiert werden." diff --git a/superset/translations/en/LC_MESSAGES/messages.po b/superset/translations/en/LC_MESSAGES/messages.po index a732cccdcc7..fa18a95b4e6 100644 --- a/superset/translations/en/LC_MESSAGES/messages.po +++ b/superset/translations/en/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4241,6 +4268,9 @@ msgstr "" msgid "Dataset could not be duplicated." msgstr "" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "" diff --git a/superset/translations/es/LC_MESSAGES/messages.po b/superset/translations/es/LC_MESSAGES/messages.po index 3dc171c34ab..cac6cdb06a4 100644 --- a/superset/translations/es/LC_MESSAGES/messages.po +++ b/superset/translations/es/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4913,6 +4940,9 @@ msgstr "No se ha podido crear el conjunto de datos." msgid "Dataset could not be duplicated." msgstr "No se ha podido duplicar el conjunto de datos." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "No se ha podido actualizar el conjunto de datos." diff --git a/superset/translations/fa/LC_MESSAGES/messages.po b/superset/translations/fa/LC_MESSAGES/messages.po index dcefbf6f033..21f5cdf47b4 100644 --- a/superset/translations/fa/LC_MESSAGES/messages.po +++ b/superset/translations/fa/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4963,6 +4990,9 @@ msgstr "گزارش نمی‌تواند ایجاد شود." msgid "Dataset could not be duplicated." msgstr "مجموعه داده نمی‌تواند تکرار شود." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "مجموعه داده نتوانست به‌روزرسانی شود." diff --git a/superset/translations/fi/LC_MESSAGES/messages.po b/superset/translations/fi/LC_MESSAGES/messages.po index c9ef1c62571..435a6567353 100644 --- a/superset/translations/fi/LC_MESSAGES/messages.po +++ b/superset/translations/fi/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ru, sk, sl, uk] #, fuzzy @@ -8298,6 +8325,9 @@ msgstr "Tietojoukkoa ei voitu monistaa." # de, es, fa, fr, it, ja, lv, mi, nl, pl, pt, pt_BR, ru, sk, sl, tr, uk, zh, # zh_TW] #, fuzzy +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Tietojoukkoa ei voitu päivittää." diff --git a/superset/translations/fr/LC_MESSAGES/messages.po b/superset/translations/fr/LC_MESSAGES/messages.po index 59e97333980..380e4b8065a 100644 --- a/superset/translations/fr/LC_MESSAGES/messages.po +++ b/superset/translations/fr/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Plural-Forms: nplurals=2; plural=(n > 1);\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data accumulates over different\n" @@ -4824,6 +4851,9 @@ msgstr "Le jeu de données n'a pas pu être créé." msgid "Dataset could not be duplicated." msgstr "Le jeu de données n'a pas pu être dupliqué." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Le jeu de données n'a pas pu être mis à jour." diff --git a/superset/translations/it/LC_MESSAGES/messages.po b/superset/translations/it/LC_MESSAGES/messages.po index fe702ba68dc..8db6d8ef9c2 100644 --- a/superset/translations/it/LC_MESSAGES/messages.po +++ b/superset/translations/it/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -6379,6 +6406,9 @@ msgstr "La tua query non può essere salvata" msgid "Dataset could not be duplicated." msgstr "Il dataset non può essere duplicato." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "La tua query non può essere salvata" diff --git a/superset/translations/ja/LC_MESSAGES/messages.po b/superset/translations/ja/LC_MESSAGES/messages.po index 849ebfc3c1c..3f883435e1e 100644 --- a/superset/translations/ja/LC_MESSAGES/messages.po +++ b/superset/translations/ja/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Plural-Forms: nplurals=1; plural=0;\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data accumulates over different\n" @@ -4380,6 +4407,9 @@ msgstr "データセットを作成できませんでした。" msgid "Dataset could not be duplicated." msgstr "データセットを複製できませんでした。" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "データセットを更新できませんでした。" diff --git a/superset/translations/ko/LC_MESSAGES/messages.po b/superset/translations/ko/LC_MESSAGES/messages.po index abf4a0836f0..90612f3134a 100644 --- a/superset/translations/ko/LC_MESSAGES/messages.po +++ b/superset/translations/ko/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -7217,6 +7244,9 @@ msgstr "데이터셋을 생성할 수 없습니다." msgid "Dataset could not be duplicated." msgstr "데이터베이스를 업데이트할 수 없습니다." +msgid "Dataset could not be restored." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ar, ca, cs, # de, es, fa, fr, it, ja, lv, mi, nl, pl, pt, pt_BR, ro, ru, sk, sl, sr, # sr_Latn, tr, uk, zh, zh_TW] diff --git a/superset/translations/lv/LC_MESSAGES/messages.po b/superset/translations/lv/LC_MESSAGES/messages.po index fef70f25dfb..d5503a951c6 100644 --- a/superset/translations/lv/LC_MESSAGES/messages.po +++ b/superset/translations/lv/LC_MESSAGES/messages.po @@ -33,6 +33,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4684,6 +4711,9 @@ msgstr "Datu kopu nevarēja izveidot." msgid "Dataset could not be duplicated." msgstr "Datu kopu nevarēja dublēt." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Datu kopu nevarēja atjaunināt." diff --git a/superset/translations/messages.pot b/superset/translations/messages.pot index f4b7f922d3e..442976fd95b 100644 --- a/superset/translations/messages.pot +++ b/superset/translations/messages.pot @@ -34,6 +34,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4248,6 +4275,9 @@ msgstr "" msgid "Dataset could not be duplicated." msgstr "" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "" diff --git a/superset/translations/mi/LC_MESSAGES/messages.po b/superset/translations/mi/LC_MESSAGES/messages.po index 2ef1b597691..4c035ca6bb3 100644 --- a/superset/translations/mi/LC_MESSAGES/messages.po +++ b/superset/translations/mi/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4900,6 +4927,9 @@ msgstr "Kāore i taea te hanga i te rārangi raraunga." msgid "Dataset could not be duplicated." msgstr "Kāore i taea te tārua i te rārangi raraunga." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Kāore i taea te whakahou i te rārangi raraunga." diff --git a/superset/translations/nl/LC_MESSAGES/messages.po b/superset/translations/nl/LC_MESSAGES/messages.po index a5befeee807..117f35e6e63 100644 --- a/superset/translations/nl/LC_MESSAGES/messages.po +++ b/superset/translations/nl/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -5119,6 +5146,9 @@ msgstr "Dataset kon niet worden aangemaakt." msgid "Dataset could not be duplicated." msgstr "Dataset kon niet worden gedupliceerd." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Dataset kon niet worden bijgewerkt." diff --git a/superset/translations/pl/LC_MESSAGES/messages.po b/superset/translations/pl/LC_MESSAGES/messages.po index e46fe1c7809..b4d5713433c 100644 --- a/superset/translations/pl/LC_MESSAGES/messages.po +++ b/superset/translations/pl/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -5189,6 +5216,9 @@ msgstr "Nie udało się utworzyć zestawu danych." msgid "Dataset could not be duplicated." msgstr "Nie udało się zduplikować zestawu danych." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Nie udało się zaktualizować zestawu danych." diff --git a/superset/translations/pt/LC_MESSAGES/messages.po b/superset/translations/pt/LC_MESSAGES/messages.po index 36fe4bd6a4b..7e1329419f5 100644 --- a/superset/translations/pt/LC_MESSAGES/messages.po +++ b/superset/translations/pt/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -7207,6 +7234,9 @@ msgstr "Não foi possível gravar a sua query" msgid "Dataset could not be duplicated." msgstr "Não foi possível gravar a sua query" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Não foi possível gravar a sua query" diff --git a/superset/translations/pt_BR/LC_MESSAGES/messages.po b/superset/translations/pt_BR/LC_MESSAGES/messages.po index ae3db8d64e0..4dcef8b9ce2 100644 --- a/superset/translations/pt_BR/LC_MESSAGES/messages.po +++ b/superset/translations/pt_BR/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -5102,6 +5129,9 @@ msgstr "Não foi possível criar o conjunto de dados." msgid "Dataset could not be duplicated." msgstr "Não foi possível duplicar o conjunto de dados." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Não foi possível atualizar o conjunto de dados." diff --git a/superset/translations/ro/LC_MESSAGES/messages.po b/superset/translations/ro/LC_MESSAGES/messages.po index bb6b77b1ff3..d50c1ea83ad 100644 --- a/superset/translations/ro/LC_MESSAGES/messages.po +++ b/superset/translations/ro/LC_MESSAGES/messages.po @@ -37,6 +37,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.9.1\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " diff --git a/superset/translations/ru/LC_MESSAGES/messages.po b/superset/translations/ru/LC_MESSAGES/messages.po index 02372849216..39113cc611a 100644 --- a/superset/translations/ru/LC_MESSAGES/messages.po +++ b/superset/translations/ru/LC_MESSAGES/messages.po @@ -29,6 +29,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4664,6 +4691,9 @@ msgstr "Не удалось создать датасет" msgid "Dataset could not be duplicated." msgstr "Датасет не может быть дублирован" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Не удалось обновить датасет" diff --git a/superset/translations/sk/LC_MESSAGES/messages.po b/superset/translations/sk/LC_MESSAGES/messages.po index 9b4b458f254..13908c4197c 100644 --- a/superset/translations/sk/LC_MESSAGES/messages.po +++ b/superset/translations/sk/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4749,6 +4776,9 @@ msgstr "Sadu dat sa nepodarilo vytvoriť." msgid "Dataset could not be duplicated." msgstr "Sadu dat sa nepodarilo duplikovať." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Sadu dat sa nepodarilo aktualizovať." diff --git a/superset/translations/sl/LC_MESSAGES/messages.po b/superset/translations/sl/LC_MESSAGES/messages.po index 322cfd151ad..4bfaf1ceaf6 100644 --- a/superset/translations/sl/LC_MESSAGES/messages.po +++ b/superset/translations/sl/LC_MESSAGES/messages.po @@ -29,6 +29,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4984,6 +5011,9 @@ msgstr "Podatkovnega niza ni mogoče ustvariti." msgid "Dataset could not be duplicated." msgstr "Podatkovnega niza ni mogoče duplicirati." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Podatkovnega niza ni mogoče posodobiti." diff --git a/superset/translations/sr/LC_MESSAGES/messages.po b/superset/translations/sr/LC_MESSAGES/messages.po index 0c92bafcbc3..413f97b3002 100644 --- a/superset/translations/sr/LC_MESSAGES/messages.po +++ b/superset/translations/sr/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Plural-Forms: nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # es, fa, fr, ja, lv, mi, pt_BR, ru, sk, sl, uk] msgid "" "\n" diff --git a/superset/translations/sr_Latn/LC_MESSAGES/messages.po b/superset/translations/sr_Latn/LC_MESSAGES/messages.po index c9e17ca290a..3a129f6fb2c 100644 --- a/superset/translations/sr_Latn/LC_MESSAGES/messages.po +++ b/superset/translations/sr_Latn/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Plural-Forms: nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # es, fa, fr, ja, lv, mi, pt_BR, ru, sk, sl, uk] msgid "" "\n" diff --git a/superset/translations/th/LC_MESSAGES/messages.po b/superset/translations/th/LC_MESSAGES/messages.po index ca33fa4f51f..e3be5bb62c7 100644 --- a/superset/translations/th/LC_MESSAGES/messages.po +++ b/superset/translations/th/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ru, sk, sl, uk] #, fuzzy @@ -8153,6 +8180,9 @@ msgstr "ไม่สามารถทำสำเนาชุดข้อมู # de, es, fa, fr, it, ja, lv, mi, nl, pl, pt, pt_BR, ru, sk, sl, tr, uk, zh, # zh_TW] #, fuzzy +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "ไม่สามารถอัปเดตชุดข้อมูลได้" diff --git a/superset/translations/tr/LC_MESSAGES/messages.po b/superset/translations/tr/LC_MESSAGES/messages.po index 2fc8b0e4262..d786fb6e106 100644 --- a/superset/translations/tr/LC_MESSAGES/messages.po +++ b/superset/translations/tr/LC_MESSAGES/messages.po @@ -32,6 +32,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4927,6 +4954,9 @@ msgstr "Veriseti oluşturulamadı." msgid "Dataset could not be duplicated." msgstr "Veriseti çoğaltılamadı." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Veriseti güncellenemedi." diff --git a/superset/translations/uk/LC_MESSAGES/messages.po b/superset/translations/uk/LC_MESSAGES/messages.po index 26cd8c2bc9c..d20372b034d 100644 --- a/superset/translations/uk/LC_MESSAGES/messages.po +++ b/superset/translations/uk/LC_MESSAGES/messages.po @@ -30,6 +30,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + msgid "" "\n" " The cumulative option allows you to see how your data " @@ -4663,6 +4690,9 @@ msgstr "Не вдалося створити набір даних." msgid "Dataset could not be duplicated." msgstr "Не вдалося дублювати набір даних." +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "Не вдалося оновити набір даних." diff --git a/superset/translations/zh/LC_MESSAGES/messages.po b/superset/translations/zh/LC_MESSAGES/messages.po index 50cc9fd5bba..9c6ab55a393 100644 --- a/superset/translations/zh/LC_MESSAGES/messages.po +++ b/superset/translations/zh/LC_MESSAGES/messages.po @@ -29,6 +29,33 @@ msgstr "" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -5165,6 +5192,9 @@ msgstr "无法创建数据集。" msgid "Dataset could not be duplicated." msgstr "数据集无法复制。" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "无法更新数据集。" diff --git a/superset/translations/zh_TW/LC_MESSAGES/messages.po b/superset/translations/zh_TW/LC_MESSAGES/messages.po index bf406403b80..90fedd04882 100644 --- a/superset/translations/zh_TW/LC_MESSAGES/messages.po +++ b/superset/translations/zh_TW/LC_MESSAGES/messages.po @@ -28,6 +28,33 @@ msgstr "" "Plural-Forms: nplurals=1; plural=0;\n" "Generated-By: Babel 2.17.0\n" +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before creating a new " +"dataset over this table, or use a different table name." +msgstr "" + +#, python-format +msgid "" +"A soft-deleted dataset (uuid %(uuid)s) already references this table. " +"Restore it via POST /api/v1/dataset/%(uuid)s/restore before uploading, or " +"upload to a different table name." +msgstr "" + +msgid "" +"Cannot delete a database whose only remaining datasets are soft-deleted. " +"Restore them (POST /api/v1/dataset//restore) and delete them " +"permanently once a purge capability ships, or remove the underlying rows " +"out-of-band, before deleting the database." +msgstr "" + +msgid "" +"Dataset cannot be restored because another active dataset already references " +"the same physical table (same database, catalog, schema, and table name). " +"Delete the duplicate or rename the table before restoring." +msgstr "" + # Machine-translated via backfill_po.py (claude-sonnet-4-6) [refs: ca, cs, de, # es, fa, fr, ja, lv, mi, pt_BR, ro, ru, sk, sl, sr, sr_Latn, tr, uk] #, fuzzy @@ -5144,6 +5171,9 @@ msgstr "無法創建數據集。" msgid "Dataset could not be duplicated." msgstr "數據集無法複製。" +msgid "Dataset could not be restored." +msgstr "" + msgid "Dataset could not be updated." msgstr "無法更新數據集。" diff --git a/superset/views/base.py b/superset/views/base.py index 7bd76ef5824..41abd29a653 100644 --- a/superset/views/base.py +++ b/superset/views/base.py @@ -42,6 +42,7 @@ from flask_appbuilder.security.sqla.models import User from flask_babel import get_locale, gettext as __ from flask_jwt_extended.exceptions import NoAuthorizationError from flask_wtf.form import FlaskForm +from sqlalchemy import and_, or_ from sqlalchemy.orm import Query from wtforms.fields.core import Field, UnboundField @@ -769,7 +770,23 @@ class DatasourceFilter(BaseFilter): # pylint: disable=too-few-public-methods models.Database, models.Database.id == self.model.database_id, ) - return query.filter(get_dataset_access_filters(self.model)) + access_filter = get_dataset_access_filters(self.model) + # Owners keep sight of their own soft-deleted rows regardless of + # datasource grants: ``raise_for_access`` counts ownership as + # datasource access, and the restore audience is owners/admins. + # The leg is inert for live rows (``deleted_at IS NULL`` fails it) + # and only ever matters when a deleted-state rison filter has opted + # the request in to seeing soft-deleted rows — without that session + # bypass, no soft-deleted row reaches this query at all. + deleted_at = getattr(self.model, "deleted_at", None) + owners = getattr(self.model, "owners", None) + if deleted_at is not None and owners is not None: + owned_trash = and_( + deleted_at.is_not(None), + owners.any(security_manager.user_model.id == utils.get_user_id()), + ) + return query.filter(or_(access_filter, owned_trash)) + return query.filter(access_filter) class CsvResponse(Response): diff --git a/superset/views/filters.py b/superset/views/filters.py index c55265e7fde..258fa9c356b 100644 --- a/superset/views/filters.py +++ b/superset/views/filters.py @@ -27,6 +27,7 @@ from sqlalchemy.orm import Query from superset import security_manager from superset.extensions import db from superset.models.helpers import SKIP_VISIBILITY_FILTER_CLASSES, SoftDeleteMixin +from superset.utils.core import get_user_id logger = logging.getLogger(__name__) @@ -196,8 +197,31 @@ class BaseDeletedStateFilter(BaseFilter): # pylint: disable=too-few-public-meth return query self._opt_into_deleted_state(query) if normalized == "only": - return query.filter(self.model.deleted_at.is_not(None)) - return query + query = query.filter(self.model.deleted_at.is_not(None)) + return self._scope_to_restore_audience(query, normalized) + + def _scope_to_restore_audience(self, query: Query, normalized: str) -> Query: + """Cross-entity contract: non-admins may only enumerate soft-deleted + rows they own — the same audience that can restore them (mirrors + ``BaseRestoreCommand``'s ``raise_for_ownership``). Live rows are + unaffected: they keep the entity's normal access filtering. + + Lives on the base so the per-entity filters stay pure declarations + (``arg_name`` + ``model``) instead of carrying verbatim copies of + this body. ``any()`` emits an EXISTS subquery so it composes with + the entity's base access filter without duplicate rows from a join. + Entities without an ``owners`` relationship opt out automatically. + """ + owners = getattr(self.model, "owners", None) + if owners is None or security_manager.is_admin(): + return query + owned = owners.any(security_manager.user_model.id == get_user_id()) + if normalized == "only": + # ``apply`` already restricted to ``deleted_at IS NOT NULL``. + return query.filter(owned) + # ``include``: keep all live rows (normal access) and add only the + # soft-deleted rows this user owns. + return query.filter(or_(self.model.deleted_at.is_(None), owned)) def _opt_into_deleted_state(self, query: Query) -> None: """The two-step opt-in shared by ``include`` and ``only``: install diff --git a/tests/integration_tests/dashboard_utils.py b/tests/integration_tests/dashboard_utils.py index 1b900ecbcc8..3a50486c04f 100644 --- a/tests/integration_tests/dashboard_utils.py +++ b/tests/integration_tests/dashboard_utils.py @@ -19,9 +19,11 @@ from typing import Optional from pandas import DataFrame # noqa: F401 +from sqlalchemy import or_ from superset import db from superset.connectors.sqla.models import SqlaTable +from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES from superset.models.core import Database from superset.models.dashboard import Dashboard from superset.models.slice import Slice @@ -35,10 +37,29 @@ def get_table( schema: Optional[str] = None, ): schema = schema or get_example_default_schema() + # Bypass the soft-delete listener so the helper finds rows previously + # soft-deleted by other tests in the same session. Without the + # bypass, the listener hides them and a subsequent INSERT collides + # with the underlying ``(database_id, catalog, schema, table_name)`` + # unique constraint that survives soft-delete. + # + # Match rows whose catalog is either unset (NULL) or the database + # default — these are "the same physical table" the way + # ``DatasetDAO.validate_uniqueness`` treats them (``table.catalog or + # default``). Example rows are seeded with ``catalog = NULL`` while a + # connection that reports a default catalog (e.g. Postgres) would + # normalize to that default, so both must qualify. This still excludes + # an unrelated third catalog variant of the same table_name. Within the + # matched set, prefer live rows over soft-deleted ones via + # ``ORDER BY deleted_at IS NULL DESC``. + catalog = database.get_default_catalog() return ( db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) .filter_by(database_id=database.id, schema=schema, table_name=table_name) - .one_or_none() + .filter(or_(SqlaTable.catalog.is_(None), SqlaTable.catalog == catalog)) + .order_by(SqlaTable.deleted_at.is_(None).desc(), SqlaTable.id) + .first() ) @@ -60,6 +81,12 @@ def create_table_metadata( always_filter_main_dttm=False, ) db.session.add(table) + elif table.deleted_at is not None: + # Restore a soft-deleted leftover from a prior test so the row is + # usable for this setup. Cleaning up via re-create-then-collide + # would fail on the underlying unique constraint that survives + # soft-delete. + table.deleted_at = None if fetch_values_predicate: table.fetch_values_predicate = fetch_values_predicate table.database = database diff --git a/tests/integration_tests/datasets/api_tests.py b/tests/integration_tests/datasets/api_tests.py index c5eae19943c..b159af76fc7 100644 --- a/tests/integration_tests/datasets/api_tests.py +++ b/tests/integration_tests/datasets/api_tests.py @@ -27,13 +27,14 @@ import pytest import rison import yaml from freezegun import freeze_time -from sqlalchemy import inspect, text +from sqlalchemy import text from sqlalchemy.exc import SQLAlchemyError from sqlalchemy.orm import joinedload from sqlalchemy.sql import func from superset.commands.dataset.exceptions import DatasetCreateFailedError from superset.connectors.sqla.models import SqlaTable, SqlMetric, TableColumn +from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES from superset.extensions import db, security_manager from superset.models.core import Database from superset.models.slice import Slice @@ -154,6 +155,7 @@ class TestDatasetApi(SupersetTestCase): def get_fixture_datasets(self) -> list[SqlaTable]: return ( db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) .options(joinedload(SqlaTable.database)) .filter(SqlaTable.table_name.in_(self.fixture_tables_names)) .all() @@ -191,19 +193,40 @@ class TestDatasetApi(SupersetTestCase): @pytest.fixture def create_datasets(self): with self.create_app().app_context(): + # Purge any soft-deleted rows that occupy the unique constraint. + # Restrict to ``deleted_at IS NOT NULL``: ``get_fixture_datasets`` + # bypasses the visibility filter and matches by table name only, + # so an unrestricted purge would also hard-delete *live* datasets + # other suites created over the same AB tables. + stale = [ + ds for ds in self.get_fixture_datasets() if ds.deleted_at is not None + ] + for ds in stale: + db.session.delete(ds) + if stale: + db.session.commit() + datasets = [] admin = self.get_user("admin") main_db = get_main_database() for tables_name in self.fixture_tables_names: datasets.append(self.insert_dataset(tables_name, [admin.id], main_db)) + # Capture IDs eagerly — dataset objects may be detached after yield + dataset_ids = [ds.id for ds in datasets] + yield datasets - # rollback changes - for dataset in datasets: - state = inspect(dataset) - if not state.was_deleted: - db.session.delete(dataset) + # rollback changes (including soft-deleted rows) + for dataset_id in dataset_ids: + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + if row: + db.session.delete(row) db.session.commit() @staticmethod @@ -1951,23 +1974,40 @@ class TestDatasetApi(SupersetTestCase): db_connection, ] + @with_feature_flags(SOFT_DELETE=True) def test_delete_dataset_item(self): """ Dataset API: Test delete dataset item """ dataset = self.insert_default_dataset() + dataset_id = dataset.id view_menu = security_manager.find_view_menu(dataset.get_perm()) assert view_menu is not None view_menu_id = view_menu.id self.login(ADMIN_USERNAME) - uri = f"api/v1/dataset/{dataset.id}" - rv = self.client.delete(uri) - assert rv.status_code == 200 - non_view_menu = db.session.query(security_manager.viewmenu_model).get( - view_menu_id - ) - assert non_view_menu is None + try: + uri = f"api/v1/dataset/{dataset.id}" + rv = self.client.delete(uri) + assert rv.status_code == 200 + # With soft delete, the row still exists (with deleted_at set) so + # FAB permissions are preserved for potential restore. + non_view_menu = db.session.query(security_manager.viewmenu_model).get( + view_menu_id + ) + assert non_view_menu is not None + finally: + # Hard-delete the (possibly soft-deleted) row even on assertion + # failure, to avoid unique constraint collisions in later tests. + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + if row: + db.session.delete(row) + db.session.commit() def test_delete_item_dataset_not_owned(self): """ @@ -2149,6 +2189,7 @@ class TestDatasetApi(SupersetTestCase): assert rv.status_code == 422 assert data == {"message": "Dataset metric delete failed."} + @with_feature_flags(SOFT_DELETE=True) @pytest.mark.usefixtures("create_datasets") def test_bulk_delete_dataset_items(self): """ @@ -2175,9 +2216,9 @@ class TestDatasetApi(SupersetTestCase): .all() ) assert datasets == [] - # Assert permissions get cleaned + # With soft delete, FAB permissions are preserved for potential restore for view_menu_name in view_menu_names: - assert security_manager.find_view_menu(view_menu_name) is None + assert security_manager.find_view_menu(view_menu_name) is not None @pytest.mark.usefixtures("create_datasets") def test_bulk_delete_item_dataset_not_owned(self): diff --git a/tests/integration_tests/datasets/soft_delete_tests.py b/tests/integration_tests/datasets/soft_delete_tests.py new file mode 100644 index 00000000000..dfb4dd95f8c --- /dev/null +++ b/tests/integration_tests/datasets/soft_delete_tests.py @@ -0,0 +1,584 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Integration tests for dataset soft-delete and restore.""" + +from datetime import datetime + +from superset import security_manager +from superset.connectors.sqla.models import SqlaTable +from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES +from superset.extensions import db +from superset.models.core import Database +from superset.models.slice import Slice +from superset.utils import json +from tests.integration_tests.base_tests import SupersetTestCase +from tests.integration_tests.conftest import with_feature_flags +from tests.integration_tests.constants import ( + ADMIN_USERNAME, + ALPHA_USERNAME, + GAMMA_USERNAME, +) + + +def _restore_dataset(dataset_id: int) -> None: + """Restore a soft-deleted dataset (cleanup helper). + + Module-level so every test class in this file can use it. Used in + ``finally`` blocks so a failed assertion can't strand a soft-deleted + row and leak it into later tests; re-queries with the + visibility-filter bypass and only restores if still soft-deleted. + """ + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + if row is not None and row.deleted_at is not None: + row.restore() + db.session.commit() + + +class TestDatasetSoftDelete(SupersetTestCase): + """Tests for dataset soft-delete behaviour (T015, T018).""" + + def _get_example_dataset_id(self) -> int: + """Get an existing example dataset ID for testing.""" + dataset = db.session.query(SqlaTable).first() + assert dataset is not None, "No datasets found — load examples first" + return dataset.id + + def _restore_dataset(self, dataset_id: int) -> None: + """Class-method shim over the module-level helper (existing call sites).""" + _restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_delete_dataset_soft_deletes(self) -> None: + """DELETE should set deleted_at instead of removing the row.""" + dataset_id = self._get_example_dataset_id() + self.login(ADMIN_USERNAME) + + try: + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + assert row is not None + assert row.deleted_at is not None + finally: + self._restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_soft_deleted_dataset_excluded_from_list(self) -> None: + """GET /api/v1/dataset/ should not include soft-deleted datasets.""" + dataset_id = self._get_example_dataset_id() + self.login(ADMIN_USERNAME) + + try: + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + + rv = self.client.get("/api/v1/dataset/") + data = json.loads(rv.data) + ids = [d["id"] for d in data["result"]] + assert dataset_id not in ids + finally: + self._restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_soft_deleted_dataset_included_in_list_when_requested(self) -> None: + """GET /api/v1/dataset/ with dataset_deleted_state=include returns deleted datasets.""" # noqa: E501 + dataset_id = self._get_example_dataset_id() + self.login(ADMIN_USERNAME) + + try: + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + + rison_query = ( + "(filters:!((col:id,opr:dataset_deleted_state,value:include)))" + ) + rv = self.client.get(f"/api/v1/dataset/?q={rison_query}") + assert rv.status_code == 200 + + data = json.loads(rv.data) + deleted_row = next( + (row for row in data["result"] if row["id"] == dataset_id), + None, + ) + assert deleted_row is not None + assert deleted_row["deleted_at"] is not None + finally: + self._restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_only_filter_returns_only_soft_deleted_datasets(self) -> None: + """dataset_deleted_state=only excludes live rows and returns only deleted ones.""" # noqa: E501 + ids = [row.id for row in db.session.query(SqlaTable).limit(2).all()] + assert len(ids) >= 2, "Need at least two example datasets for this test" + live_id, deleted_id = ids[0], ids[1] + self.login(ADMIN_USERNAME) + + try: + rv = self.client.delete(f"/api/v1/dataset/{deleted_id}") + assert rv.status_code == 200 + + rison_query = "(filters:!((col:id,opr:dataset_deleted_state,value:only)))" + rv = self.client.get(f"/api/v1/dataset/?q={rison_query}") + assert rv.status_code == 200 + + data = json.loads(rv.data) + returned_ids = {row["id"] for row in data["result"]} + assert deleted_id in returned_ids + assert live_id not in returned_ids + finally: + self._restore_dataset(deleted_id) + + def _hard_delete_created(self, dataset_id: int, database: Database) -> None: + """Remove a test-created dataset + its database (visibility bypassed).""" + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + if row: + db.session.delete(row) + db.session.delete(database) + db.session.commit() + + @with_feature_flags(SOFT_DELETE=True) + def test_deleted_state_list_shows_owner_their_own_deleted(self) -> None: + """A non-admin owner can still enumerate their own soft-deleted datasets. + Deleted-state scoping mirrors the restore audience, so it must not lock + owners out of their own trash.""" + alpha = self.get_user(ALPHA_USERNAME) + database = Database(database_name="sd_owner_db", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + dataset = SqlaTable( + table_name="sd_owner_tbl", database=database, owners=[alpha] + ) + db.session.add(dataset) + db.session.commit() + dataset_id = dataset.id + + dataset.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.commit() + + try: + self.login(ALPHA_USERNAME) + rison_query = ( + "(filters:!((col:id,opr:dataset_deleted_state,value:only))," + "page_size:200)" + ) + rv = self.client.get(f"/api/v1/dataset/?q={rison_query}") + assert rv.status_code == 200 + ids = [r["id"] for r in json.loads(rv.data)["result"]] + assert dataset_id in ids + finally: + # Matches the sibling tests: a failed assertion must not strand + # the soft-deleted dataset/database in the shared test DB. + self._hard_delete_created(dataset_id, database) + + @with_feature_flags(SOFT_DELETE=True) + def test_deleted_state_list_hides_non_owned_from_read_access_user(self) -> None: + """A read-access non-owner must not enumerate a dataset once it is + soft-deleted. + + Gamma is granted ``datasource_access`` to the dataset, so + ``DatasourceFilter`` makes it visible to gamma while live. After + soft-delete, the deleted-state list is scoped to the restore audience + (owners/admins), so gamma — who could never restore it — must not see it + via ``include`` or ``only``. + """ + admin = self.get_user(ADMIN_USERNAME) + database = Database(database_name="sd_acl_db", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + dataset = SqlaTable(table_name="sd_acl_tbl", database=database, owners=[admin]) + db.session.add(dataset) + db.session.commit() + dataset_id = dataset.id + + gamma_role = security_manager.find_role("Gamma") + pvm = security_manager.add_permission_view_menu( + "datasource_access", dataset.perm + ) + gamma_role.permissions.append(pvm) + db.session.commit() + + try: + # Precondition: gamma can see the dataset while it is live. + self.login(GAMMA_USERNAME) + rv = self.client.get("/api/v1/dataset/?q=(page_size:200)") + assert dataset_id in [r["id"] for r in json.loads(rv.data)["result"]], ( + "precondition: gamma should see the live dataset via datasource access" + ) + + reloaded = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one() + ) + reloaded.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.commit() + + for value in ("include", "only"): + rison_query = ( + f"(filters:!((col:id,opr:dataset_deleted_state,value:{value}))," + "page_size:200)" + ) + rv = self.client.get(f"/api/v1/dataset/?q={rison_query}") + assert rv.status_code == 200 + ids = [r["id"] for r in json.loads(rv.data)["result"]] + assert dataset_id not in ids, ( + "read-access non-owner must not enumerate a soft-deleted " + f"dataset via dataset_deleted_state={value}" + ) + finally: + pvm = security_manager.find_permission_view_menu( + "datasource_access", dataset.perm + ) + if pvm: + security_manager.del_permission_role(gamma_role, pvm) + db.session.commit() + self._hard_delete_created(dataset_id, database) + + @with_feature_flags(SOFT_DELETE=True) + def test_deleted_state_list_shows_gamma_owner_without_datasource_grant( + self, + ) -> None: + """An owner keeps sight of their own trash even without a datasource + grant. + + Gamma owns the dataset but holds NO ``datasource_access`` on it, so + every access leg of ``DatasourceFilter`` fails — the owned-soft-deleted + leg is what makes the row reachable, mirroring ``raise_for_access`` + counting ownership as datasource access. The live-row precondition + also pins the leg's inertness: while the dataset is live, gamma + (no grant) must NOT see it, because the leg requires + ``deleted_at IS NOT NULL``. + """ + gamma = self.get_user(GAMMA_USERNAME) + database = Database(database_name="sd_own_db", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + dataset = SqlaTable(table_name="sd_own_tbl", database=database, owners=[gamma]) + db.session.add(dataset) + db.session.commit() + dataset_id = dataset.id + + try: + self.login(GAMMA_USERNAME) + # Precondition: no grant -> gamma cannot see the LIVE row (the + # owned-trash leg must not leak live rows). + rv = self.client.get("/api/v1/dataset/?q=(page_size:200)") + assert rv.status_code == 200 + assert dataset_id not in [r["id"] for r in json.loads(rv.data)["result"]], ( + "precondition: ungranted gamma must not see the live dataset" + ) + + reloaded = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one() + ) + reloaded.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.commit() + + for value in ("include", "only"): + rison_query = ( + f"(filters:!((col:id,opr:dataset_deleted_state,value:{value}))," + "page_size:200)" + ) + rv = self.client.get(f"/api/v1/dataset/?q={rison_query}") + assert rv.status_code == 200 + ids = [r["id"] for r in json.loads(rv.data)["result"]] + assert dataset_id in ids, ( + "owner without a datasource grant must still enumerate " + f"their own trash via dataset_deleted_state={value}" + ) + finally: + self._hard_delete_created(dataset_id, database) + + @with_feature_flags(SOFT_DELETE=False) + def test_delete_dataset_flag_off_hard_deletes(self) -> None: + """Default-deployment contract: with SOFT_DELETE off, DELETE + physically removes the row (even a bypass query finds nothing) and + the restore endpoint 404s — there is nothing to restore. + + Every other delete-path test in this module runs flag-ON; without + this test the path every default deployment actually runs would be + the untested one. + """ + admin = self.get_user(ADMIN_USERNAME) + database = Database(database_name="sd_off_db", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + dataset = SqlaTable(table_name="sd_off_tbl", database=database, owners=[admin]) + db.session.add(dataset) + db.session.commit() + dataset_id = dataset.id + dataset_uuid = str(dataset.uuid) + + try: + self.login(ADMIN_USERNAME) + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one_or_none() + ) + assert row is None, "flag-off DELETE must hard-delete the row" + + rv = self.client.post(f"/api/v1/dataset/{dataset_uuid}/restore") + assert rv.status_code == 404, "hard-deleted rows are not restorable" + finally: + self._hard_delete_created(dataset_id, database) + + @with_feature_flags(SOFT_DELETE=True) + def test_no_cascade_to_dependent_charts(self) -> None: + """Soft-deleting a dataset should NOT cascade to its charts (FR-009, T018).""" + dataset_id = self._get_example_dataset_id() + self.login(ADMIN_USERNAME) + + # Find charts that depend on this dataset + dependent_charts = ( + db.session.query(Slice) + .filter(Slice.datasource_id == dataset_id, Slice.datasource_type == "table") + .all() + ) + dependent_chart_ids = [c.id for c in dependent_charts] + + try: + # Soft-delete the dataset + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + + # Dependent charts should still be active (no cascade). On this + # branch ``Slice`` does not yet carry ``deleted_at`` (added by the + # charts soft-delete PR), so we only verify the row is still + # loadable through the default visibility-filtered query — which + # would return None if the chart had been soft-deleted. + for chart_id in dependent_chart_ids: + chart = ( + db.session.query(Slice).filter(Slice.id == chart_id).one_or_none() + ) + assert chart is not None, f"Chart {chart_id} should still be active" + finally: + self._restore_dataset(dataset_id) + + +class TestDatasetRestore(SupersetTestCase): + """Tests for dataset restore behaviour (T027).""" + + def _get_example_dataset(self) -> SqlaTable: + dataset = db.session.query(SqlaTable).first() + assert dataset is not None + return dataset + + @with_feature_flags(SOFT_DELETE=True) + def test_restore_soft_deleted_dataset(self) -> None: + """POST /api/v1/dataset//restore should make it visible again.""" + dataset = self._get_example_dataset() + dataset_id = dataset.id + dataset_uuid = str(dataset.uuid) + self.login(ADMIN_USERNAME) + + try: + self.client.delete(f"/api/v1/dataset/{dataset_id}") + + rv = self.client.post(f"/api/v1/dataset/{dataset_uuid}/restore") + assert rv.status_code == 200 + + rv = self.client.get(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200 + finally: + # This test soft-deletes the SHARED example dataset; a failed + # assertion must not strand it and cascade failures through + # every later suite that queries it. + # TestDatasetRestore has no _restore_dataset method (that lives on + # TestDatasetSoftDelete); call the module-level helper directly. + _restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_restore_failure_returns_422(self) -> None: + """A failure during restore surfaces as a clean 422 via the + ``DatasetRestoreFailedError`` handler rather than an unhandled 500. + + ``RestoreDatasetCommand.run`` wraps the restore in ``@transaction`` + and rethrows ``DatasetRestoreFailedError`` on any underlying + SQLAlchemy error; this pins that the endpoint maps it to 422. + """ + from unittest.mock import patch + + from superset.commands.dataset.exceptions import ( + DatasetRestoreFailedError, + ) + + dataset = self._get_example_dataset() + dataset_id = dataset.id + dataset_uuid = str(dataset.uuid) + self.login(ADMIN_USERNAME) + try: + self.client.delete(f"/api/v1/dataset/{dataset_id}") + + with patch( + "superset.commands.dataset.restore.RestoreDatasetCommand.run", + side_effect=DatasetRestoreFailedError(), + ): + rv = self.client.post(f"/api/v1/dataset/{dataset_uuid}/restore") + assert rv.status_code == 422 + finally: + # The mocked restore leaves the example dataset soft-deleted; a + # failed assertion must not strand it for later tests. + _restore_dataset(dataset_id) + + @with_feature_flags(SOFT_DELETE=True) + def test_restore_uses_can_write_permission(self) -> None: + """Non-admin owner with ``can_write_Dataset`` can hit the restore + endpoint. + + Pins the permission contract: ``method_permission_name`` must map + ``restore`` to ``write`` so FAB's ``@protect`` resolves the gate to + ``can_write_Dataset`` (which Alpha already carries), not the + implicit fallback ``can_restore_Dataset`` (which no standard role + carries). + + Without the mapping FAB defaults to ``can__`` and + every non-admin would get 403 here — admins bypass FAB permission + checks entirely, so the admin-authed restore test above doesn't + exercise the mapping. + """ + dataset = self._get_example_dataset() + dataset_id = dataset.id + dataset_uuid = str(dataset.uuid) + alpha = self.get_user(ALPHA_USERNAME) + + # Make Alpha an owner so raise_for_ownership passes. + dataset.owners = list(dataset.owners) + [alpha] + db.session.commit() + + try: + self.login(ALPHA_USERNAME) + rv = self.client.delete(f"/api/v1/dataset/{dataset_id}") + assert rv.status_code == 200, ( + f"Alpha owner soft-delete failed: {rv.status_code} {rv.data!r}" + ) + + rv = self.client.post(f"/api/v1/dataset/{dataset_uuid}/restore") + assert rv.status_code == 200, ( + f"Expected 200 from Alpha owner restore (can_write_Dataset), " + f"got {rv.status_code}: {rv.data!r}. If 403, " + "method_permission_name is missing 'restore': 'write'." + ) + finally: + # Restore example dataset state: remove Alpha from owners and + # ensure deleted_at is cleared in case the restore attempt failed. + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == dataset_id) + .one() + ) + row.owners = [o for o in row.owners if o.id != alpha.id] + if row.deleted_at is not None: + row.restore() + db.session.commit() + + # Note: a ``test_restore_blocked_by_active_logical_duplicate`` integration + # test is deliberately absent: the "delete -> seed twin -> restore" setup + # is blocked at step 2 by a DB-level constraint, though *which* constraint + # depends on how the schema was built. ``metadata.create_all`` (unit-test + # schemas) materializes the model's otherwise metadata-only 4-column + # ``UniqueConstraint``; migration-built databases instead still carry the + # legacy 3-column ``_customer_location_uc`` from the 2016 ``b4456560d4f3`` + # migration — the 2024 ``df3d7e2eb9a4`` migration that intends to drop it + # is a silent no-op (it passes a list to + # ``generic_find_uq_constraint_name``, which compares it to a set; the + # comparison never matches). Seeding a NULL-schema twin would dodge the + # constraint, but a NULL-schema row would not match the application + # check's identity predicate either. The restore-side check + # ``DatasetDAO.has_active_logical_duplicate`` (called from + # ``RestoreDatasetCommand.validate`` and the v1 importer) yields a clean + # 422 instead of an opaque IntegrityError and guards any schema where the + # legacy constraint is eventually dropped for real; it is covered by + # ``tests/unit_tests/commands/dataset/restore_test.py:: + # test_restore_dataset_logical_duplicate_raises`` plus the catalog- + # normalization tests in ``tests/unit_tests/dao/dataset_test.py``. The + # create-side defense is covered end-to-end by + # ``test_create_blocked_by_soft_deleted_logical_duplicate`` below. + + @with_feature_flags(SOFT_DELETE=True) + def test_create_blocked_by_soft_deleted_logical_duplicate(self) -> None: + """Create returns 422 when a soft-deleted dataset references the same + physical table. + + Pins the contract that ``DatasetDAO.validate_uniqueness`` bypasses + the soft-delete visibility filter, so a soft-deleted row blocks + creation of a new dataset at the same logical identity at the + application layer — a clean 422 instead of an opaque + IntegrityError from whichever DB-level constraint applies (or a + silent active twin where none does; DB-level enforcement is + inconsistent across schema builds). + """ + dataset = self._get_example_dataset() + original_id = dataset.id + database_id = dataset.database_id + catalog = dataset.catalog + schema = dataset.schema + table_name = dataset.table_name + + self.login(ADMIN_USERNAME) + + rv = self.client.delete(f"/api/v1/dataset/{original_id}") + assert rv.status_code == 200 + + try: + rv = self.client.post( + "/api/v1/dataset/", + json={ + "database": database_id, + "catalog": catalog, + "schema": schema, + "table_name": table_name, + }, + ) + assert rv.status_code == 422, ( + f"Expected 422 for create-blocked-by-soft-deleted, " + f"got {rv.status_code}: {rv.data!r}" + ) + finally: + row = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == original_id) + .one() + ) + row.restore() + db.session.commit() diff --git a/tests/integration_tests/security_tests.py b/tests/integration_tests/security_tests.py index c089bf4cf25..9378e6d1eca 100644 --- a/tests/integration_tests/security_tests.py +++ b/tests/integration_tests/security_tests.py @@ -560,6 +560,79 @@ class TestRolePermission(SupersetTestCase): db.session.delete(tmp_db1) db.session.commit() + @with_feature_flags(SOFT_DELETE=True) + def test_after_update_database__perm_datasource_access_soft_deleted(self): + """A soft-deleted dataset's perm strings must still be rewritten on a + database rename. The maintenance query bypasses the soft-delete + visibility filter so hidden datasets are updated too; otherwise + restoring the dataset later would resurrect stale dataset/schema/catalog + permission strings pointing at the old database name. + + The flag must be ON here: with SOFT_DELETE off the visibility filter + never engages, so the query this test guards (the + ``skip_visibility_filter`` bypass in + ``_update_vm_datasources_access``) would be exercised vacuously — the + test would pass even with the bypass removed. + """ + from datetime import datetime # noqa: PLC0415 + + from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES # noqa: PLC0415 + + security_manager.on_view_menu_after_update = Mock() + + tmp_db1 = Database(database_name="tmp_db1", sqlalchemy_uri="sqlite://") + db.session.add(tmp_db1) + db.session.commit() + + table1 = SqlaTable( + schema="tmp_schema", + table_name="tmp_table1", + database=tmp_db1, + ) + db.session.add(table1) + db.session.commit() + table1 = db.session.query(SqlaTable).filter_by(table_name="tmp_table1").one() + table1_id = table1.id + + assert table1.perm == f"[tmp_db1].[tmp_table1](id:{table1_id})" + + # Soft-delete the dataset, then rename its database. + table1.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.commit() + + tmp_db1 = db.session.query(Database).filter_by(database_name="tmp_db1").one() + tmp_db1.database_name = "tmp_db2" + db.session.commit() + + # The old perm is gone and the new one exists, even though the dataset + # is soft-deleted (the maintenance query bypassed the visibility filter). + assert ( + security_manager.find_permission_view_menu( + "datasource_access", f"[tmp_db1].[tmp_table1](id:{table1_id})" + ) + is None + ) + assert ( + security_manager.find_permission_view_menu( + "datasource_access", f"[tmp_db2].[tmp_table1](id:{table1_id})" + ) + is not None + ) + + # The hidden row's own ``perm`` column was rewritten too. + reloaded = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter(SqlaTable.id == table1_id) + .one() + ) + assert reloaded.perm == f"[tmp_db2].[tmp_table1](id:{table1_id})" + + # Cleanup + db.session.delete(reloaded) + db.session.delete(tmp_db1) + db.session.commit() + def test_after_delete_database(self): tmp_db1 = Database(database_name="tmp_db1", sqlalchemy_uri="sqlite://") db.session.add(tmp_db1) diff --git a/tests/unit_tests/commands/databases/sync_permissions_test.py b/tests/unit_tests/commands/databases/sync_permissions_test.py index 55a6e4501db..2400c726d03 100644 --- a/tests/unit_tests/commands/databases/sync_permissions_test.py +++ b/tests/unit_tests/commands/databases/sync_permissions_test.py @@ -409,3 +409,49 @@ def test_sync_permissions_command_rename_db_in_perms( assert ( mock_chart.schema_perm == f"[{database_with_catalog.name}].[catalog1].[schema1]" ) + + +def test_sync_permissions_command_rename_bypasses_visibility_filter( + mocker: MockerFixture, database_with_catalog: MagicMock +): + """The dataset/chart perm rewrite must run with the soft-delete + visibility filter bypassed for ``SqlaTable`` and ``Slice``. + + A soft-deleted dataset's ``schema_perm``/``catalog_perm`` (and its + charts') must still be rewritten on a database rename; without the + bypass, the listener hides those rows from ``get_datasets`` and a + later restore resurrects stale perm strings referencing the old + database name — matchable by a ``schema_access`` grant on a new + database reusing that name. + """ + from superset.connectors.sqla.models import SqlaTable + from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES + from superset.models.slice import Slice + + mocker.patch( + "superset.commands.database.sync_permissions." + "security_manager.find_permission_view_menu", + return_value=None, + ) + observed_bypass: list[set[type]] = [] + + def capture_bypass(*args, **kwargs): + observed_bypass.append( + set(db.session.info.get(SKIP_VISIBILITY_FILTER_CLASSES, set())) + ) + return [] + + mock_database_dao = mocker.patch( + "superset.commands.database.sync_permissions.DatabaseDAO" + ) + mock_database_dao.get_datasets.side_effect = capture_bypass + + cmmd = SyncPermissionsCommand( + 1, None, old_db_connection_name="old_name", db_connection=database_with_catalog + ) + cmmd._rename_database_in_permissions("catalog1", ["schema1"]) + + assert observed_bypass, "get_datasets was never called" + assert all({SqlaTable, Slice} <= bypass for bypass in observed_bypass), ( + "dataset perm rewrite ran without the soft-delete visibility bypass" + ) diff --git a/tests/unit_tests/commands/databases/upload_command_test.py b/tests/unit_tests/commands/databases/upload_command_test.py index 2268dc67db5..5d4854835c3 100644 --- a/tests/unit_tests/commands/databases/upload_command_test.py +++ b/tests/unit_tests/commands/databases/upload_command_test.py @@ -145,3 +145,90 @@ def test_validate_file_size_skips_when_not_seekable( ) # size can't be determined cheaply -> skip rather than raising a 500 UploadCommand.validate_file_size(_non_seekable_file()) + + +def _stub_run_environment(mocker: MockerFixture) -> MagicMock: + """Stub everything ``run()`` needs up to the dataset lookup.""" + model = mocker.MagicMock() + model.db_engine_spec.supports_file_upload = True + model.db_engine_spec.normalize_table_name_for_upload.return_value = ("t", None) + mocker.patch( + "superset.commands.database.uploaders.base.DatabaseDAO.find_by_id", + return_value=model, + ) + mocker.patch( + "superset.commands.database.uploaders.base.schema_allows_file_upload", + return_value=True, + ) + mocker.patch.dict( + "superset.commands.database.uploaders.base.current_app.config", + {"UPLOAD_MAX_FILE_SIZE_BYTES": 1024}, + ) + db_mock = mocker.patch("superset.commands.database.uploaders.base.db") + # No visible dataset over the target table. + db_mock.session.query.return_value.filter_by.return_value.one_or_none.return_value = None # noqa: E501 + return model + + +def test_run_blocks_upload_over_soft_deleted_twin( + app_context: None, mocker: MockerFixture +) -> None: + """An upload targeting a table held by a soft-deleted dataset must be + refused BEFORE any file data is written. + + The dataset lookup in ``run()`` goes through the soft-delete visibility + filter, so the hidden twin is invisible and, unguarded, the upload would + create an active twin (permanently blocking restore) or hit the legacy + unique constraint — after ``reader.read`` had already loaded the file's + contents into the analytics database, outside the metadata transaction. + """ + from superset.commands.database.exceptions import ( + DatabaseUploadSoftDeletedDatasetExistsError, + ) + + _stub_run_environment(mocker) + soft_twin = MagicMock() + soft_twin.uuid = "11111111-2222-3333-4444-555555555555" + mocker.patch( + # the guard imports DatasetDAO inside run() (deferred to avoid a + # circular import), so patch it at the source module + "superset.daos.dataset.DatasetDAO.find_soft_deleted_logical_duplicate", + return_value=soft_twin, + ) + + reader = MagicMock() + command = UploadCommand( + model_id=1, table_name="t", file=_file(b"x"), schema=None, reader=reader + ) + with pytest.raises(DatabaseUploadSoftDeletedDatasetExistsError) as excinfo: + command.run() + + assert "11111111-2222-3333-4444-555555555555" in str(excinfo.value) + # The guard must fire before the file contents are written. + reader.read.assert_not_called() + + +def test_run_proceeds_when_no_soft_deleted_twin( + app_context: None, mocker: MockerFixture +) -> None: + """Control: with no hidden twin, the upload reads the file and creates + the dataset as before.""" + _stub_run_environment(mocker) + mocker.patch( + # the guard imports DatasetDAO inside run() (deferred to avoid a + # circular import), so patch it at the source module + "superset.daos.dataset.DatasetDAO.find_soft_deleted_logical_duplicate", + return_value=None, + ) + mocker.patch( + "superset.commands.database.uploaders.base.SqlaTable", + return_value=MagicMock(), + ) + mocker.patch("superset.commands.database.uploaders.base.get_user") + + reader = MagicMock() + command = UploadCommand( + model_id=1, table_name="t", file=_file(b"x"), schema=None, reader=reader + ) + command.run() + reader.read.assert_called_once() diff --git a/tests/unit_tests/commands/dataset/restore_test.py b/tests/unit_tests/commands/dataset/restore_test.py new file mode 100644 index 00000000000..4807de38e56 --- /dev/null +++ b/tests/unit_tests/commands/dataset/restore_test.py @@ -0,0 +1,181 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Unit tests for RestoreDatasetCommand.""" + +from __future__ import annotations + +import uuid +from datetime import datetime, timezone +from unittest.mock import MagicMock, patch + +import pytest + +# Stable UUIDs make the test boundary realistic: the production command +# loads by UUID, not integer ID. The test only mocks the DAO lookup, but +# the argument shape should still match. +_DATASET_UUID = str(uuid.uuid4()) +_MISSING_UUID = str(uuid.uuid4()) + + +def test_restore_dataset_clears_deleted_at(app_context: None) -> None: + """RestoreDatasetCommand.run() restores a soft-deleted dataset.""" + from superset.commands.dataset.restore import RestoreDatasetCommand + + dataset = MagicMock() + dataset.deleted_at = datetime(2026, 1, 1, tzinfo=timezone.utc) + dataset.id = 1 + + with ( + patch( + "superset.daos.dataset.DatasetDAO.find_by_id", return_value=dataset + ) as mock_find, + patch("superset.commands.restore.security_manager") as mock_sec, + patch( + "superset.daos.dataset.DatasetDAO.has_active_logical_duplicate", + return_value=False, + ), + ): + mock_sec.raise_for_ownership.return_value = None + + cmd = RestoreDatasetCommand(_DATASET_UUID) + cmd.run() + + mock_find.assert_called_once_with( + _DATASET_UUID, + id_column="uuid", + # Both bypasses are the lookup's contract: skip_base_filter keeps an + # owner's trash reachable when the RBAC base_filter has no ownership + # leg (audience is enforced by raise_for_ownership instead); + # skip_visibility_filter lets the lookup see soft-deleted rows. + skip_base_filter=True, + skip_visibility_filter=True, + ) + dataset.restore.assert_called_once() + + +def test_restore_dataset_not_found_raises(app_context: None) -> None: + """RestoreDatasetCommand raises DatasetNotFoundError for missing dataset.""" + from superset.commands.dataset.exceptions import DatasetNotFoundError + from superset.commands.dataset.restore import RestoreDatasetCommand + + with patch("superset.daos.dataset.DatasetDAO.find_by_id", return_value=None): + cmd = RestoreDatasetCommand(_MISSING_UUID) + with pytest.raises(DatasetNotFoundError): + cmd.run() + + +def test_restore_active_dataset_raises_not_found(app_context: None) -> None: + """RestoreDatasetCommand raises DatasetNotFoundError for non-deleted dataset.""" + from superset.commands.dataset.exceptions import DatasetNotFoundError + from superset.commands.dataset.restore import RestoreDatasetCommand + + dataset = MagicMock() + dataset.deleted_at = None # not soft-deleted + + with patch("superset.daos.dataset.DatasetDAO.find_by_id", return_value=dataset): + cmd = RestoreDatasetCommand(_DATASET_UUID) + with pytest.raises(DatasetNotFoundError): + cmd.run() + + +def test_restore_dataset_forbidden_raises(app_context: None) -> None: + """RestoreDatasetCommand raises DatasetForbiddenError on permission check.""" + from superset.commands.dataset.exceptions import DatasetForbiddenError + from superset.commands.dataset.restore import RestoreDatasetCommand + from superset.exceptions import SupersetSecurityException + + dataset = MagicMock() + dataset.deleted_at = datetime(2026, 1, 1, tzinfo=timezone.utc) + + def raise_security(*args: object, **kwargs: object) -> None: + raise SupersetSecurityException(MagicMock()) + + with ( + patch("superset.daos.dataset.DatasetDAO.find_by_id", return_value=dataset), + patch("superset.commands.restore.security_manager") as mock_sec, + ): + mock_sec.raise_for_ownership = raise_security + + cmd = RestoreDatasetCommand(_DATASET_UUID) + with pytest.raises(DatasetForbiddenError): + cmd.run() + + +def test_restore_dataset_logical_duplicate_raises(app_context: None) -> None: + """Restore raises DatasetLogicalDuplicateError when another active dataset + already references the same physical table. + + DB-level enforcement of SqlaTable logical uniqueness is inconsistent + across schema builds (the model-level UniqueConstraint is metadata-only; + the legacy _customer_location_uc has no catalog leg and is NULL-leaky), + so a soft-deleted dataset can have its logical slot claimed by a new + active row before restore. The command refuses the restore so the + operator sees a clean error instead of an IntegrityError or a silent + twin. + """ + from superset.commands.dataset.exceptions import DatasetLogicalDuplicateError + from superset.commands.dataset.restore import RestoreDatasetCommand + + dataset = MagicMock() + dataset.deleted_at = datetime(2026, 1, 1, tzinfo=timezone.utc) + dataset.id = 42 + + with ( + patch("superset.daos.dataset.DatasetDAO.find_by_id", return_value=dataset), + patch("superset.commands.restore.security_manager") as mock_sec, + patch( + "superset.daos.dataset.DatasetDAO.has_active_logical_duplicate", + return_value=True, + ) as mock_dup_check, + ): + mock_sec.raise_for_ownership.return_value = None + + cmd = RestoreDatasetCommand(_DATASET_UUID) + with pytest.raises(DatasetLogicalDuplicateError): + cmd.run() + + mock_dup_check.assert_called_once_with(dataset) + + +def test_restore_dataset_no_logical_duplicate_when_none_exists( + app_context: None, +) -> None: + """The duplicate check is consulted on every restore, even when the + answer is no — guards against a regression where the override silently + short-circuits and never queries. + """ + from superset.commands.dataset.restore import RestoreDatasetCommand + + dataset = MagicMock() + dataset.deleted_at = datetime(2026, 1, 1, tzinfo=timezone.utc) + dataset.id = 42 + + with ( + patch("superset.daos.dataset.DatasetDAO.find_by_id", return_value=dataset), + patch("superset.commands.restore.security_manager") as mock_sec, + patch( + "superset.daos.dataset.DatasetDAO.has_active_logical_duplicate", + return_value=False, + ) as mock_dup_check, + ): + mock_sec.raise_for_ownership.return_value = None + + cmd = RestoreDatasetCommand(_DATASET_UUID) + cmd.run() + + mock_dup_check.assert_called_once_with(dataset) + dataset.restore.assert_called_once() diff --git a/tests/unit_tests/commands/dataset/test_create.py b/tests/unit_tests/commands/dataset/test_create.py index c84b1664c77..5313f28a169 100644 --- a/tests/unit_tests/commands/dataset/test_create.py +++ b/tests/unit_tests/commands/dataset/test_create.py @@ -171,3 +171,84 @@ def test_create_dataset_physical_table_no_parse_error( # Should not raise any parsing errors command.validate() + + +def test_create_dataset_blocked_by_soft_deleted_twin() -> None: + """Uniqueness failure caused by a hidden twin raises the targeted, + single-sourced 422 (naming the twin's uuid and the restore endpoint) + instead of the opaque "already exists" validation error.""" + from superset.commands.dataset.exceptions import ( + DatasetSoftDeletedTwinExistsError, + ) + + mock_database = Mock(spec=Database) + mock_database.id = 1 + mock_database.get_default_catalog.return_value = None + twin = Mock() + twin.uuid = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + + with patch( + "superset.commands.dataset.create.DatasetDAO.get_database_by_id", + return_value=mock_database, + ): + with patch( + "superset.commands.dataset.create.DatasetDAO.validate_uniqueness", + return_value=False, + ): + with patch( + "superset.commands.dataset.create." + "DatasetDAO.find_soft_deleted_logical_duplicate", + return_value=twin, + ): + command = CreateDatasetCommand( + {"database": 1, "table_name": "blocked_tbl"} + ) + with pytest.raises(DatasetSoftDeletedTwinExistsError) as exc_info: + command.validate() + + message = str(exc_info.value) + assert "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" in message + assert "restore" in message.lower() + # Executable recoveries only — no hard-delete/purge claims. + assert "hard-delete" not in message.lower() + assert "purge" not in message.lower() + + +def test_create_dataset_generic_exists_error_when_no_twin() -> None: + """Control: uniqueness failure with no hidden twin keeps the existing + generic validation-error path.""" + mock_database = Mock(spec=Database) + mock_database.id = 1 + mock_database.get_default_catalog.return_value = None + + with patch( + "superset.commands.dataset.create.DatasetDAO.get_database_by_id", + return_value=mock_database, + ): + with patch( + "superset.commands.dataset.create.DatasetDAO.validate_uniqueness", + return_value=False, + ): + with patch( + "superset.commands.dataset.create." + "DatasetDAO.find_soft_deleted_logical_duplicate", + return_value=None, + ): + with patch( + "superset.commands.dataset.create.DatasetDAO.validate_table_exists", + return_value=True, + ): + with patch( + "superset.commands.dataset.create." + "security_manager.raise_for_access", + ): + with patch( + "superset.commands.dataset.create." + "CreateDatasetCommand.populate_owners", + return_value=[], + ): + command = CreateDatasetCommand( + {"database": 1, "table_name": "existing_tbl"} + ) + with pytest.raises(DatasetInvalidError): + command.validate() diff --git a/tests/unit_tests/commands/dataset/test_duplicate.py b/tests/unit_tests/commands/dataset/test_duplicate.py index 28b047e3e68..742a3e8abc8 100644 --- a/tests/unit_tests/commands/dataset/test_duplicate.py +++ b/tests/unit_tests/commands/dataset/test_duplicate.py @@ -19,6 +19,7 @@ from unittest.mock import MagicMock, patch import pytest +from sqlalchemy.orm.session import Session from superset.commands.dataset.exceptions import DatasetAccessDeniedError from superset.errors import ErrorLevel, SupersetError, SupersetErrorType @@ -84,8 +85,8 @@ def test_duplicate_dataset_access_check_passes_through() -> None: "superset.commands.dataset.duplicate.security_manager.raise_for_access" ) as mock_access: with patch( - "superset.commands.dataset.duplicate.DatasetDAO.find_one_or_none", - return_value=None, + "superset.commands.dataset.duplicate.DatasetDAO.validate_uniqueness", + return_value=True, ): with patch( "superset.commands.dataset.duplicate.DuplicateDatasetCommand.populate_owners", @@ -101,3 +102,66 @@ def test_duplicate_dataset_access_check_passes_through() -> None: command.validate() # should not raise # Confirm access check was called with the base dataset mock_access.assert_called_once_with(datasource=mock_dataset) + + +def test_duplicate_dataset_blocked_by_soft_deleted_twin(session: Session) -> None: + """validate() rejects a duplicate whose name matches a soft-deleted + dataset at the same (database, schema). + + The previous name-only ``find_one_or_none`` lookup ran through the + soft-delete visibility filter, so a soft-deleted twin was invisible and + the duplicate proceeded — hitting whichever DB constraint applies as an + opaque IntegrityError, or creating an active twin that permanently + blocks restore where none does. The shared ``validate_uniqueness`` + check bypasses the filter and refuses with a clean validation error. + """ + from datetime import datetime, timezone + + from superset import db + from superset.commands.dataset.duplicate import DuplicateDatasetCommand + from superset.commands.dataset.exceptions import DatasetInvalidError + from superset.connectors.sqla.models import SqlaTable + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="dup_db", sqlalchemy_uri="sqlite://") + base = SqlaTable( + table_name="base_view", + schema="main", + database=database, + sql="select 1", + ) + hidden_twin = SqlaTable( + table_name="dup_name", + schema="main", + database=database, + deleted_at=datetime(2026, 1, 1, tzinfo=timezone.utc), + ) + db.session.add_all([database, base, hidden_twin]) + db.session.flush() + + with ( + # find_by_id applies the security base filter, which needs a request + # user; return the seeded row directly. validate_uniqueness — the + # behaviour under test — runs for real against the session. + patch( + "superset.commands.dataset.duplicate.DatasetDAO.find_by_id", + return_value=base, + ), + patch("superset.commands.dataset.duplicate.security_manager.raise_for_access"), + patch( + "superset.commands.dataset.duplicate." + "DuplicateDatasetCommand.populate_owners", + return_value=[], + ), + ): + command = DuplicateDatasetCommand( + { + "base_model_id": base.id, + "table_name": "dup_name", + "is_managed_externally": False, + } + ) + with pytest.raises(DatasetInvalidError): + command.validate() diff --git a/tests/unit_tests/commands/test_base_restore_command.py b/tests/unit_tests/commands/test_base_restore_command.py index 927cdda5a02..656858e1f99 100644 --- a/tests/unit_tests/commands/test_base_restore_command.py +++ b/tests/unit_tests/commands/test_base_restore_command.py @@ -124,13 +124,16 @@ def test_validate_raises_forbidden_when_ownership_check_fails( cmd.validate() -def test_validate_calls_dao_with_visibility_bypass_only(app_context: None) -> None: +def test_validate_calls_dao_with_both_bypasses(app_context: None) -> None: """The DAO load uses ``skip_visibility_filter=True`` (so the - soft-deleted row is visible) and ``id_column='uuid'`` — but does - NOT bypass the entity's ``base_filter``. Restore should honor RBAC - the same way ``delete`` does (which loads through ``find_by_ids`` - without ``skip_base_filter=True``); the visibility bypass is the - only escape hatch needed for restore.""" + soft-deleted row is visible), ``skip_base_filter=True``, and + ``id_column='uuid'``. The base-filter bypass is deliberate: an + entity's RBAC ``base_filter`` may have no ownership leg (charts and + datasets filter by datasource access), and ``raise_for_access`` + counts ownership as datasource access — so a lost grant must not + hide a row from the one audience that can restore it. The restore + audience (owners or admin) is enforced by ``raise_for_ownership`` + instead.""" soft_deleted = MagicMock() soft_deleted.deleted_at = datetime(2026, 1, 1) cmd = _make_command(dao_find_result=soft_deleted) @@ -142,6 +145,7 @@ def test_validate_calls_dao_with_visibility_bypass_only(app_context: None) -> No cmd.dao.find_by_id.assert_called_once_with( "uuid-1", id_column="uuid", + skip_base_filter=True, skip_visibility_filter=True, ) diff --git a/tests/unit_tests/dao/dataset_test.py b/tests/unit_tests/dao/dataset_test.py index cf66062b8a4..f33254b9669 100644 --- a/tests/unit_tests/dao/dataset_test.py +++ b/tests/unit_tests/dao/dataset_test.py @@ -87,6 +87,99 @@ def test_validate_update_uniqueness(session: Session) -> None: ) +def test_logical_duplicate_catalog_predicate_is_null_aware(session: Session) -> None: + """A row stored with ``catalog = NULL`` is the default catalog, so a probe + normalized to the database default must still match it. + + Before the null-aware predicate the stored ``catalog`` was compared as-is, + so a ``catalog = NULL`` row and a ``catalog = `` probe were treated + as different physical tables and a default-catalog twin slipped through. + """ + from superset import db + from superset.connectors.sqla.models import SqlaTable + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="cat_db", sqlalchemy_uri="sqlite://") + stored_null = SqlaTable( + table_name="t", schema="main", catalog=None, database=database + ) + db.session.add_all([database, stored_null]) + db.session.flush() + + # Pretend the database exposes a non-NULL default catalog so the probe + # normalizes to it; the NULL-stored row must still be recognized as a twin. + with patch.object(Database, "get_default_catalog", return_value="default_cat"): + probe = SqlaTable( + table_name="t", schema="main", catalog="default_cat", database=database + ) + db.session.add(probe) + db.session.flush() + assert DatasetDAO.has_active_logical_duplicate(probe) is True + + # A genuinely different (non-default) catalog is not a twin. + other = SqlaTable( + table_name="t", schema="main", catalog="other_cat", database=database + ) + db.session.add(other) + db.session.flush() + assert DatasetDAO.has_active_logical_duplicate(other) is False + + +def test_find_soft_deleted_logical_duplicate(session: Session) -> None: + """The importer create-path helper returns a soft-deleted twin (despite the + visibility filter) and ignores active rows.""" + from superset import db + from superset.connectors.sqla.models import SqlaTable + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="sd_db", sqlalchemy_uri="sqlite://") + twin = SqlaTable(table_name="t", schema="main", database=database) + db.session.add_all([database, twin]) + db.session.flush() + + table = Table("t", "main") + + # Active row: not a soft-deleted twin. + assert DatasetDAO.find_soft_deleted_logical_duplicate(database, table) is None + + # Soft-delete it: now found despite the visibility filter. + twin.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + found = DatasetDAO.find_soft_deleted_logical_duplicate(database, table) + assert found is not None + assert found.id == twin.id + + # Catalog mismatch: a soft-deleted row in a *different* (non-default) + # catalog is not the same physical table, so the null-aware predicate must + # exclude it. (sqlite has no default catalog, so an explicit "other_cat" + # row must only match an "other_cat" probe.) + other_catalog_twin = SqlaTable( + table_name="t2", + schema="main", + catalog="other_cat", + database=database, + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + db.session.add(other_catalog_twin) + db.session.flush() + assert ( + DatasetDAO.find_soft_deleted_logical_duplicate( + database, Table("t2", "main", "my_cat") + ) + is None + ) + # Same explicit catalog: found. + same_catalog = DatasetDAO.find_soft_deleted_logical_duplicate( + database, Table("t2", "main", "other_cat") + ) + assert same_catalog is not None + assert same_catalog.id == other_catalog_twin.id + + @freeze_time("2025-01-01 00:00:00") @patch.object(DatasetDAO, "update_columns") @patch.object(DatasetDAO, "update_metrics") @@ -176,3 +269,92 @@ def test_update_dataset_related_metadata_updates_changed_on( update_metrics_mock.assert_not_called() base_update_mock.assert_called_once_with(item, expected_attributes) + + +def _mock_dataset(catalog: str | None, default_catalog: str) -> MagicMock: + """A SqlaTable-shaped mock with controllable catalog values.""" + model = MagicMock() + model.id = 1 + model.database_id = 7 + model.schema = "public" + model.table_name = "users" + model.catalog = catalog + model.database.get_default_catalog.return_value = default_catalog + return model + + +def test_has_active_logical_duplicate_normalizes_unset_catalog( + app_context: None, +) -> None: + """A row stored with ``catalog = None`` is matched against the database + default catalog, the same rule ``validate_uniqueness`` applies. + + This is the gap the soft-delete reviews flagged: restore/re-import + compared the raw stored ``catalog`` while create/update normalized it, so + a soft-deleted ``catalog = NULL`` row and an active default-catalog twin + could be treated as different physical tables. + """ + model = _mock_dataset(catalog=None, default_catalog="default_cat") + + with patch("superset.daos.dataset.db") as mock_db: + mock_db.session.query.return_value.filter.return_value.first.return_value = None + + assert DatasetDAO.has_active_logical_duplicate(model) is False + + # Normalization must consult the database default when catalog is unset. + model.database.get_default_catalog.assert_called_once() + + +def test_has_active_logical_duplicate_keeps_explicit_catalog( + app_context: None, +) -> None: + """An explicit catalog is matched exactly, but the database default is still + consulted: the null-aware predicate needs it to decide whether the explicit + catalog *is* the default (and should therefore also match ``catalog IS + NULL`` rows).""" + model = _mock_dataset(catalog="explicit_cat", default_catalog="default_cat") + + with patch("superset.daos.dataset.db") as mock_db: + mock_db.session.query.return_value.filter.return_value.first.return_value = None + + assert DatasetDAO.has_active_logical_duplicate(model) is False + + model.database.get_default_catalog.assert_called_once() + + +def test_has_active_logical_duplicate_true_when_twin_found( + app_context: None, +) -> None: + """A matching active row makes the helper report a duplicate.""" + model = _mock_dataset(catalog="explicit_cat", default_catalog="default_cat") + + with patch("superset.daos.dataset.db") as mock_db: + mock_db.session.query.return_value.filter.return_value.first.return_value = ( + MagicMock() + ) + + assert DatasetDAO.has_active_logical_duplicate(model) is True + + +def test_has_active_logical_duplicate_never_bypasses_visibility( + app_context: None, +) -> None: + """Contract tripwire for the docstring's do-not-add-the-bypass clause. + + The helper's active-rows-only semantics come from the SoftDeleteMixin + listener; wrapping the query in ``skip_visibility_filter`` — the file's + dominant pattern, so a plausible future "consistency" edit — would + broaden the check to soft-deleted rows and silently refuse legitimate + restores of legacy twin pairs. The suite stays green under that + mutation without this pin. + """ + model = _mock_dataset(catalog=None, default_catalog="default_cat") + + with ( + patch("superset.daos.dataset.db") as mock_db, + patch("superset.models.helpers.skip_visibility_filter") as bypass_spy, + ): + mock_db.session.query.return_value.filter.return_value.first.return_value = None + DatasetDAO.has_active_logical_duplicate(model) + + bypass_spy.assert_not_called() diff --git a/tests/unit_tests/dao/datasource_test.py b/tests/unit_tests/dao/datasource_test.py new file mode 100644 index 00000000000..113201464f9 --- /dev/null +++ b/tests/unit_tests/dao/datasource_test.py @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from datetime import datetime, timezone +from unittest.mock import patch + +from sqlalchemy.orm.session import Session + + +def test_build_dataset_query_excludes_soft_deleted(session: Session) -> None: + """The Core ``select`` in ``build_dataset_query`` must filter + ``deleted_at IS NULL`` explicitly. + + The ``SoftDeleteMixin`` listener runs on ``do_orm_execute`` only, so it + never fires for this Core statement — the explicit ``where`` clause is + the sole thing keeping soft-deleted datasets out of the combined + datasource list and its pagination counts. This test fails if that + clause is removed. + """ + from superset import db, security_manager + from superset.connectors.sqla.models import SqlaTable + from superset.daos.datasource import DatasourceDAO + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="ds_q_db", sqlalchemy_uri="sqlite://") + live = SqlaTable(table_name="live_t", schema="main", database=database) + hidden = SqlaTable( + table_name="hidden_t", + schema="main", + database=database, + deleted_at=datetime(2026, 1, 1, tzinfo=timezone.utc), + ) + db.session.add_all([database, live, hidden]) + db.session.flush() + + with patch.object( + security_manager, "can_access_all_datasources", return_value=True + ): + query = DatasourceDAO.build_dataset_query(name_filter=None, sql_filter=None) + # Core execution — exactly the path the ORM listener cannot see. + names = {row.table_name for row in session.execute(query)} + + assert "live_t" in names + assert "hidden_t" not in names diff --git a/tests/unit_tests/databases/commands/delete_test.py b/tests/unit_tests/databases/commands/delete_test.py new file mode 100644 index 00000000000..4636c726b26 --- /dev/null +++ b/tests/unit_tests/databases/commands/delete_test.py @@ -0,0 +1,129 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from datetime import datetime + +import pytest +from pytest_mock import MockerFixture +from sqlalchemy.orm.session import Session + + +def test_delete_database_blocked_by_soft_deleted_dataset( + mocker: MockerFixture, + session: Session, +) -> None: + """A database whose only datasets are soft-deleted must NOT be deletable. + + ``Database.tables`` now hides soft-deleted datasets (``SqlaTable`` inherits + ``SoftDeleteMixin``), so the validation counts datasets with the visibility + filter bypassed — otherwise the database looks empty and gets hard-deleted + while ``tables.database_id`` rows still reference it. + """ + from superset import db + from superset.commands.database.delete import DeleteDatabaseCommand + from superset.commands.database.exceptions import ( + DatabaseDeleteSoftDeletedDatasetsExistFailedError, + ) + from superset.connectors.sqla.models import SqlaTable + from superset.daos.database import DatabaseDAO + from superset.daos.report import ReportScheduleDAO + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="del_db", sqlalchemy_uri="sqlite://") + soft_deleted = SqlaTable( + table_name="gone", + database=database, + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + db.session.add_all([database, soft_deleted]) + db.session.flush() + + # Isolate the dataset check: the report/model lookups are unrelated to the + # soft-delete behaviour under test. + mocker.patch.object(DatabaseDAO, "find_by_id", return_value=database) + mocker.patch.object(ReportScheduleDAO, "find_by_database_id", return_value=[]) + + command = DeleteDatabaseCommand(database.id) + # The soft-deleted-only branch raises the *specific* subclass so the + # message tells the operator the blockers are hidden rows. + with pytest.raises(DatabaseDeleteSoftDeletedDatasetsExistFailedError): + command.validate() + + +def test_delete_database_blocked_by_live_dataset( + mocker: MockerFixture, + session: Session, +) -> None: + """A database with a live (non-deleted) dataset raises the base error, not + the soft-deleted subclass — pinning that the two ``validate`` branches map + to distinct messages.""" + from superset import db + from superset.commands.database.delete import DeleteDatabaseCommand + from superset.commands.database.exceptions import ( + DatabaseDeleteDatasetsExistFailedError, + DatabaseDeleteSoftDeletedDatasetsExistFailedError, + ) + from superset.connectors.sqla.models import SqlaTable + from superset.daos.database import DatabaseDAO + from superset.daos.report import ReportScheduleDAO + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="live_db", sqlalchemy_uri="sqlite://") + live = SqlaTable(table_name="present", database=database) + db.session.add_all([database, live]) + db.session.flush() + + mocker.patch.object(DatabaseDAO, "find_by_id", return_value=database) + mocker.patch.object(ReportScheduleDAO, "find_by_database_id", return_value=[]) + + command = DeleteDatabaseCommand(database.id) + with pytest.raises(DatabaseDeleteDatasetsExistFailedError) as exc_info: + command.validate() + # The live branch must NOT surface the hidden-rows message. + assert not isinstance( + exc_info.value, DatabaseDeleteSoftDeletedDatasetsExistFailedError + ) + + +def test_delete_database_allowed_when_no_datasets( + mocker: MockerFixture, + session: Session, +) -> None: + """With no datasets at all (soft-deleted or active), validation passes.""" + from superset import db + from superset.commands.database.delete import DeleteDatabaseCommand + from superset.connectors.sqla.models import SqlaTable + from superset.daos.database import DatabaseDAO + from superset.daos.report import ReportScheduleDAO + from superset.models.core import Database + + SqlaTable.metadata.create_all(session.get_bind()) + + database = Database(database_name="empty_db", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + mocker.patch.object(DatabaseDAO, "find_by_id", return_value=database) + mocker.patch.object(ReportScheduleDAO, "find_by_database_id", return_value=[]) + + command = DeleteDatabaseCommand(database.id) + # Should not raise. + command.validate() diff --git a/tests/unit_tests/datasets/commands/importers/v1/import_test.py b/tests/unit_tests/datasets/commands/importers/v1/import_test.py index e4d6559bc5e..4b9b8af175b 100644 --- a/tests/unit_tests/datasets/commands/importers/v1/import_test.py +++ b/tests/unit_tests/datasets/commands/importers/v1/import_test.py @@ -20,6 +20,7 @@ import copy import io import re import uuid +from datetime import datetime from typing import Any from unittest.mock import Mock, patch from urllib import request @@ -1032,7 +1033,8 @@ def test_import_dataset_without_owner_permission( assert ( str(excinfo.value) - == "A dataset already exists and user doesn't have permissions to overwrite it" # noqa: E501 + == "Dataset 'imported_dataset' (uuid 10808100-158b-42c4-842e-f32b99d88dfb) " + "already exists and user doesn't have permissions to overwrite it" # noqa: E501 ) # Assert that the can write to chart was checked @@ -1077,6 +1079,438 @@ def test_import_dataset_access_check( import_dataset(config) +def test_import_soft_deleted_dataset_overwrite_restores_in_place( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Overwrite-importing a soft-deleted dataset must restore the row in + place rather than hard-delete-and-replace. A hard delete would + cascade through the chart back-reference and table_columns / + sql_metrics rows; in-place restore preserves them. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + initial = import_dataset(config) + original_id = initial.id + + existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + existing.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + + admin = User( + first_name="Alice", + last_name="Doe", + email="adoe@example.org", + username="admin", + roles=[Role(name="Admin")], + ) + + with override_user(admin): + restored = import_dataset(config, overwrite=True) + + assert restored.id == original_id + assert restored.deleted_at is None + + +def test_import_soft_deleted_dataset_non_overwrite_restores_for_owner( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Non-overwrite re-import of a soft-deleted UUID is implicitly a + restore-and-update: the user is bringing the dataset back by + uploading it again. The same ownership rule as the overwrite path + applies, so an owner (or admin) succeeds without setting + overwrite=True. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + initial = import_dataset(config) + original_id = initial.id + + existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + existing.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + + admin = User( + first_name="Alice", + last_name="Doe", + email="adoe@example.org", + username="admin", + roles=[Role(name="Admin")], + ) + + with override_user(admin): + restored = import_dataset(config, overwrite=False) + + assert restored.id == original_id + assert restored.deleted_at is None + + +def test_import_soft_deleted_dataset_non_overwrite_raises_for_non_owner( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Non-overwrite re-import that would resurrect a soft-deleted dataset + must respect ownership: a non-owner without admin role cannot + restore-via-import. Mirrors the explicit /restore endpoint's check. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + import_dataset(config) + existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + existing.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + + non_owner = User( + first_name="Bob", + last_name="Roe", + email="bob@example.org", + username="bob", + roles=[Role(name="Gamma")], + ) + + with override_user(non_owner): + with pytest.raises(ImportFailedError) as excinfo: + import_dataset(config, overwrite=False) + assert "permissions to restore" in str(excinfo.value) + # Verify the permission check fired before any mutation: if a regression + # cleared ``deleted_at`` before raising, this would silently produce a + # half-restored row and the test would still pass on the message check + # alone. + db.session.refresh(existing) + assert existing.deleted_at is not None, ( + "deleted_at was cleared before the exception — restore mutation " + "happened before the ownership check" + ) + + +def test_import_soft_deleted_dataset_raises_when_caller_lacks_can_write( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Case B: re-import of a soft-deleted UUID by a caller without + can_write must raise, not silently return the soft-deleted row. + + Real-world scenario: a user has can_write Dashboard but not + can_write Dataset, and they import a dashboard zip that references + a soft-deleted dataset. Silently returning the row would let the + dashboard importer wire the dashboard's charts to a deleted dataset + and produce broken chart loads. + """ + mocker.patch.object(security_manager, "can_access", return_value=False) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + # Seed a soft-deleted dataset with the matching UUID directly, so the + # test doesn't need to flip permissions mid-test. + existing = SqlaTable( + table_name="soft_deleted_dataset", + database_id=database.id, + uuid=config["uuid"], + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + db.session.add(existing) + db.session.flush() + + with pytest.raises(ImportFailedError) as excinfo: + import_dataset(config, overwrite=False) + assert "can_write" in str(excinfo.value) + # Case B contract: deleted_at must remain set after the exception. A + # regression that clears deleted_at before the can_write check would + # leave the row in a half-restored state and silently pass the message + # assertion above. + db.session.refresh(existing) + assert existing.deleted_at is not None, ( + "Case B: deleted_at was cleared before raising — mutation happened " + "before the can_write check" + ) + + +def test_import_existing_active_dataset_overwrite_without_can_write_returns_existing( + mocker: MockerFixture, + session: Session, +) -> None: + """ + An *active* (not soft-deleted) dataset re-imported with overwrite=True by a + caller without can_write must fall through to returning the existing row, + not raise the restore error. Case B is keyed on ``is_soft_deleted``, so the + fused ``needs_mutation`` condition must not pull active rows into the + restore-without-permission branch (pre-soft-delete overwrite behaviour). + """ + mocker.patch.object(security_manager, "can_access", return_value=False) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + existing = SqlaTable( + table_name=config["table_name"], + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid=config["uuid"], + ) + db.session.add(existing) + db.session.flush() + assert existing.deleted_at is None + + result = import_dataset(config, overwrite=True) + + assert result.id == existing.id + assert result.deleted_at is None + + +def test_import_blocked_by_soft_deleted_logical_duplicate_with_new_uuid( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Importing a dataset with a fresh UUID but the same physical table as a + soft-deleted dataset must raise. ``import_from_dict`` can't see the hidden + row (the visibility filter hides soft-deleted rows), so creating would + produce an active twin of a soft-deleted dataset. This mirrors the REST + create path's ``validate_uniqueness`` block. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + # A soft-deleted dataset with a DIFFERENT UUID but the same physical table. + twin = SqlaTable( + table_name=config["table_name"], + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid="ffffffff-ffff-ffff-ffff-ffffffffffff", + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + db.session.add(twin) + db.session.flush() + + with pytest.raises(ImportFailedError) as excinfo: + import_dataset(config) + assert "same physical table" in str(excinfo.value) + + +def test_import_soft_deleted_dataset_restore_removes_orphan_children( + mocker: MockerFixture, + session: Session, +) -> None: + """ + Restoring a soft-deleted dataset via re-import (non-overwrite, + Option C) syncs columns and metrics — children present in the live + row but absent from the uploaded config are removed, not silently + merged. + + Without forcing sync on the implicit-restore path, ``sync=[]`` + would mean "upsert by UUID, leave non-matching children alone", + so the restored dataset would carry stale columns from before the + soft-delete. That's a surprising merge of two states; treating + re-import as a clean replacement is what an explicit ``overwrite`` + would do anyway. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + initial = import_dataset(config) + original_id = initial.id + + existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + # Add an orphan column that the upload doesn't know about. + orphan = TableColumn( + column_name="orphan_col", + type="STRING", + table=existing, + ) + db.session.add(orphan) + existing.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + orphan_uuid = orphan.uuid + + admin = User( + first_name="Alice", + last_name="Doe", + email="adoe@example.org", + username="admin", + roles=[Role(name="Admin")], + ) + + with override_user(admin): + restored = import_dataset(config, overwrite=False) + + assert restored.id == original_id + assert restored.deleted_at is None + assert orphan_uuid not in {c.uuid for c in restored.columns}, ( + "orphan column survived restore-via-import; the implicit-restore " + "path must force sync so re-import is a clean replacement" + ) + + +def test_import_dataset_multiple_results_on_soft_delete_match_raises_and_rolls_back( + mocker: MockerFixture, + session: Session, +) -> None: + """ + When ``find_existing_for_import`` resolves a soft-deleted row by UUID + and the subsequent ``import_from_dict`` hits the legacy NULL-schema + ambiguity (``MultipleResultsFound``), the importer must: + + 1. Roll back the ``deleted_at`` clear it just applied — without + the rollback the dataset would be left half-restored + (``deleted_at = None`` but no upload content applied). + 2. Raise ``ImportFailedError`` with the legacy-duplicate message + so the operator resolves the duplicate manually before retrying. + + Reproduce: seed a soft-deleted row with the target UUID and monkey- + patch ``import_from_dict`` to raise ``MultipleResultsFound``. The + importer must surface the guard exception, and the row's + ``deleted_at`` must still be set after the call returns. + """ + from sqlalchemy.exc import MultipleResultsFound + + from superset.commands.exceptions import ImportFailedError + from superset.constants import SKIP_VISIBILITY_FILTER_CLASSES + + mocker.patch.object(security_manager, "can_access", return_value=True) + mocker.patch.object( + SqlaTable, + "import_from_dict", + side_effect=MultipleResultsFound("simulated duplicate"), + ) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + original_deleted_at = datetime(2026, 1, 1, 12, 0, 0) + soft_deleted = SqlaTable( + table_name="ambiguous_dataset", + database_id=database.id, + uuid=config["uuid"], + deleted_at=original_deleted_at, + ) + db.session.add(soft_deleted) + db.session.flush() + + with pytest.raises(ImportFailedError, match="matches more than one existing row"): + import_dataset(config) + + reloaded = ( + db.session.query(SqlaTable) + .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}}) + .filter_by(uuid=config["uuid"]) + .one() + ) + assert reloaded.deleted_at == original_deleted_at, ( + "deleted_at was not rolled back after MultipleResultsFound on " + "the soft-delete-match path; the row is left half-restored" + ) + + +def test_import_soft_deleted_dataset_ignore_permissions_restores_in_place( + mocker: MockerFixture, + session: Session, +) -> None: + """ + The example loader path: ignore_permissions=True with no logged-in + user. Previously the rewrite gated id-preservation on `user`, so this + path skipped both branches and INSERT collided on the UUID unique + index. The fix restores master's behavior: id is preserved on the + fallthrough overwrite path regardless of whether `user` is set. + """ + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + + initial = import_dataset(config, ignore_permissions=True) + original_id = initial.id + + existing = db.session.query(SqlaTable).filter_by(uuid=config["uuid"]).one() + existing.deleted_at = datetime(2026, 1, 1, 12, 0, 0) + db.session.flush() + + restored = import_dataset(config, overwrite=True, ignore_permissions=True) + assert restored.id == original_id + assert restored.deleted_at is None + + @pytest.mark.parametrize( "allowed_urls, data_uri, expected, exception_class", [ @@ -1240,3 +1674,108 @@ def test_redirect_handler_blocks_disallowed_redirect_target() -> None: {}, "http://169.254.169.254/latest/meta-data/", ) + + +def test_import_overwrite_rename_onto_soft_deleted_twin_blocked( + mocker: MockerFixture, + session: Session, +) -> None: + """Overwriting an alive dataset must not rename it onto a hidden twin's + physical identity. + + ``import_from_dict``'s lookup cannot see the soft-deleted row (visibility + filter), so without the identity re-validation the update would land + cleanly and the live row would silently squat the trash row's identity — + permanently blocking its restore. Mirrors ``UpdateDatasetCommand``'s + ``validate_update_uniqueness`` contract. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + config["table_name"] = "squatted_tbl" # rename onto the twin's identity + + # The alive dataset being overwritten (matches the config's UUID). + existing = SqlaTable( + table_name="original_tbl", + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid=config["uuid"], + ) + # The hidden twin holding the target identity. + twin = SqlaTable( + table_name="squatted_tbl", + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid="ffffffff-ffff-ffff-ffff-fffffffffff0", + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + db.session.add_all([existing, twin]) + db.session.flush() + + with pytest.raises(ImportFailedError) as excinfo: + import_dataset(config, overwrite=True) + assert "cannot be overwritten" in str(excinfo.value) + assert config["uuid"] in str(excinfo.value) + # The alive row keeps its original identity. + assert existing.table_name == "original_tbl" + + +def test_import_restore_blocked_by_active_twin_at_incoming_identity( + mocker: MockerFixture, + session: Session, +) -> None: + """The restore-via-import duplicate check probes the POST-update identity. + + An uploaded config that renames the soft-deleted dataset onto an ACTIVE + dataset's identity must be refused up front (check-before-mutate: the row + stays soft-deleted), not fall through to a downstream + ``MultipleResultsFound`` with a misdiagnosing message. + """ + mocker.patch.object(security_manager, "can_access", return_value=True) + + engine = db.session.get_bind() + SqlaTable.metadata.create_all(engine) # pylint: disable=no-member + + database = Database(database_name="my_database", sqlalchemy_uri="sqlite://") + db.session.add(database) + db.session.flush() + + config = copy.deepcopy(dataset_fixture) + config["database_id"] = database.id + config["table_name"] = "claimed_tbl" # rename onto the active row's identity + + # The soft-deleted dataset being restored (matches the config's UUID). + existing = SqlaTable( + table_name="old_tbl", + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid=config["uuid"], + deleted_at=datetime(2026, 1, 1, 12, 0, 0), + ) + # An unrelated ACTIVE dataset already holding the target identity. + active = SqlaTable( + table_name="claimed_tbl", + schema=config.get("schema"), + catalog=config.get("catalog"), + database_id=database.id, + uuid="ffffffff-ffff-ffff-ffff-fffffffffff1", + ) + db.session.add_all([existing, active]) + db.session.flush() + + with pytest.raises(ImportFailedError) as excinfo: + import_dataset(config) + assert "another active dataset" in str(excinfo.value) + # Check-before-mutate: the failed import leaves the row soft-deleted. + assert existing.deleted_at is not None diff --git a/tests/unit_tests/migrations/test_add_deleted_at_to_tables.py b/tests/unit_tests/migrations/test_add_deleted_at_to_tables.py new file mode 100644 index 00000000000..3ddeba215f7 --- /dev/null +++ b/tests/unit_tests/migrations/test_add_deleted_at_to_tables.py @@ -0,0 +1,189 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Tests for migration ``3a8e6f2c1b95_add_deleted_at_to_tables``. + +Runs the migration's ``upgrade()`` and ``downgrade()`` against an +in-memory SQLite engine with a real Alembic ``Operations`` context. +The behaviour being pinned is the operator-facing contract documented +in ``UPDATING.md``: ``downgrade()`` reverses the schema but does not +hard-delete or otherwise mutate rows that were soft-deleted before +the migration was reversed — those rows survive the downgrade and +become visible to any code path that no longer applies the +soft-delete visibility filter. +""" + +from __future__ import annotations + +from datetime import datetime +from importlib import import_module + +import pytest +from alembic.migration import MigrationContext +from alembic.operations import Operations +from sqlalchemy import ( + Column, + create_engine, + insert, + inspect, + Integer, + MetaData, + select, + String, + Table, +) +from sqlalchemy.engine import Engine + +migration = import_module( + "superset.migrations.versions." + "2026-05-08_12-10_3a8e6f2c1b95_add_deleted_at_to_tables" +) + +TABLE_NAME = migration.TABLE_NAME # "tables" +INDEX_NAME = migration.INDEX_NAME # "ix_tables_deleted_at" + + +@pytest.fixture +def engine() -> Engine: + """In-memory SQLite seeded with a minimal pre-migration ``tables`` table. + + The real ``tables`` table has many columns; the migration only touches + ``deleted_at`` and its index, so only the columns that participate in + the test are seeded. + """ + engine = create_engine("sqlite:///:memory:") + md = MetaData() + Table( + TABLE_NAME, + md, + Column("id", Integer, primary_key=True), + Column("table_name", String(250), nullable=False), + ) + md.create_all(engine) + return engine + + +def _columns(engine: Engine) -> set[str]: + return {col["name"] for col in inspect(engine).get_columns(TABLE_NAME)} + + +def _indexes(engine: Engine) -> set[str]: + return {ix["name"] for ix in inspect(engine).get_indexes(TABLE_NAME)} + + +def test_upgrade_adds_deleted_at_column_and_index(engine: Engine) -> None: + with engine.connect() as conn: + ctx = MigrationContext.configure(conn) + with Operations.context(ctx): + migration.upgrade() + + assert "deleted_at" in _columns(engine), "upgrade() must add the deleted_at column" + assert INDEX_NAME in _indexes(engine), ( + "upgrade() must create the supporting index on deleted_at" + ) + + +def test_downgrade_drops_deleted_at_column_and_index(engine: Engine) -> None: + with engine.connect() as conn: + ctx = MigrationContext.configure(conn) + with Operations.context(ctx): + migration.upgrade() + migration.downgrade() + + assert "deleted_at" not in _columns(engine), ( + "downgrade() must drop the deleted_at column" + ) + assert INDEX_NAME not in _indexes(engine), ( + "downgrade() must drop the supporting index" + ) + + +def test_downgrade_preserves_soft_deleted_row_data(engine: Engine) -> None: + """Pin the operator-facing contract from ``UPDATING.md``: rows that + were soft-deleted before the migration is reversed survive the + downgrade. The ``deleted_at`` column is gone, so those rows are + indistinguishable from live rows to any code path that no longer + applies the visibility filter — operators must decide on + hard-delete, restore, or rename BEFORE downgrading. See the + "Rollback note" in ``UPDATING.md``. + """ + with engine.connect() as conn: + ctx = MigrationContext.configure(conn) + with Operations.context(ctx): + migration.upgrade() + + # Seed one live row and one soft-deleted row. + tables = Table(TABLE_NAME, MetaData(), autoload_with=conn) + conn.execute( + insert(tables).values( + [ + { + "id": 1, + "table_name": "live_table", + "deleted_at": None, + }, + { + "id": 2, + "table_name": "archived_table", + "deleted_at": datetime(2026, 1, 1, 12, 0, 0), + }, + ] + ) + ) + + migration.downgrade() + + # Re-reflect after schema change. Both rows must survive; no + # column to distinguish them remains. + tables_after = Table(TABLE_NAME, MetaData(), autoload_with=conn) + rows = conn.execute( + select(tables_after).order_by(tables_after.c.id) + ).fetchall() + + assert [(r.id, r.table_name) for r in rows] == [ + (1, "live_table"), + (2, "archived_table"), + ], ( + "downgrade() must not delete or mutate row data — soft-deleted " + "rows become indistinguishable from live rows but they remain" + ) + assert "deleted_at" not in { + c["name"] for c in inspect(engine).get_columns(TABLE_NAME) + } + + +def test_upgrade_is_idempotent(engine: Engine) -> None: + """The migration helpers (``add_columns``, ``create_index``) are + idempotent skip-if-exists; running ``upgrade()`` twice must not + raise. + """ + with engine.connect() as conn: + ctx = MigrationContext.configure(conn) + with Operations.context(ctx): + migration.upgrade() + migration.upgrade() # second call must be a no-op + + +def test_downgrade_is_idempotent(engine: Engine) -> None: + """``drop_columns`` / ``drop_index`` are skip-if-not-exists; running + ``downgrade()`` twice must not raise. + """ + with engine.connect() as conn: + ctx = MigrationContext.configure(conn) + with Operations.context(ctx): + migration.upgrade() + migration.downgrade() + migration.downgrade() # second call must be a no-op