feat(explore): adhoc column expressions [ID-3] (#17379)

* add support for adhoc columns to api and sqla model

* fix some types

* fix duplicates in column names

* fix more lint

* fix schema and dedup

* clean up some logic

* first pass at fixing viz.py

* Add frontend support for adhoc columns

* Add title edit

* Fix showing custom title

* Use column name as default value in sql editor

* fix: Adds a loading message when needed in the Select component (#16531)

* fix(tests): make parquet select deterministic with order by (#16570)

* bump emotion to help with cache clobbering (#16559)

* fix: Support Jinja template functions in global async queries (#16412)

* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check

* fix: impersonate user label/tooltip (#16573)

* docs: update for small typos (#16568)

* feat: Add Aurora Data API engine spec (#16535)

* feat: Add Aurora Data API engine spec

* Fix lint

* refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)

* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues

* refactor sql_json view endpoint: separate concern into ad hod method (#16595)

* feat: Experimental cross-filter plugins (#16594)

* fix:fix get permission function

* feat: add cross filter chart in charts gallery under FF

* chore(deps): bump superset-ui to 0.18.2 (#16601)

* update type guard references

* fix imports

* update series_columns schema

* Add changes that got lost in rebase

* Use current columns name or expression as sql editor init value

* add integration test and do minor fixes

* Bump superset-ui

* fix linting issue

* bump superset-ui to 0.18.22

* resolve merge conflict

* lint

* fix select filter infinite loop

* bump superset-ui to 0.18.23

* Fix auto setting column popover title

* Enable adhoc columns only if UX_BETA enabled

* put back removed test

* Move popover height and width to constants

* Refactor big ternary expression

Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com>
Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: joeADSP <75027008+joeADSP@users.noreply.github.com>
Co-authored-by: ofekisr <35701650+ofekisr@users.noreply.github.com>
Co-authored-by: simcha90 <56388545+simcha90@users.noreply.github.com>

tests/integration_tests/charts/data/api_tests.py

@@ -43,6 +43,7 @@ from superset.errors import SupersetErrorType
 from superset.extensions import async_query_manager, db
 from superset.models.annotations import AnnotationLayer
 from superset.models.slice import Slice
+from superset.typing import AdhocColumn
 from superset.utils.core import (
     AnnotationType,
     get_example_database,
@@ -61,6 +62,12 @@ from tests.integration_tests.fixtures.query_context import (
 
 CHART_DATA_URI = "api/v1/chart/data"
 CHARTS_FIXTURE_COUNT = 10
+ADHOC_COLUMN_FIXTURE: AdhocColumn = {
+    "hasCustomLabel": True,
+    "label": "male_or_female",
+    "sqlExpression": "case when gender = 'boy' then 'male' "
+    "when gender = 'girl' then 'female' else 'other' end",
+}
 
 
 class BaseTestChartDataApi(SupersetTestCase):
@@ -820,3 +827,23 @@ class TestGetChartDataApi(BaseTestChartDataApi):
         )
 
         self.assertEqual(rv.status_code, 404)
+
+    @pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
+    def test_chart_data_with_adhoc_column(self):
+        """
+        Chart data API: Test query with adhoc column in both select and where clause
+        """
+        self.login(username="admin")
+        request_payload = get_query_context("birth_names")
+        request_payload["queries"][0]["columns"] = [ADHOC_COLUMN_FIXTURE]
+        request_payload["queries"][0]["filters"] = [
+            {"col": ADHOC_COLUMN_FIXTURE, "op": "IN", "val": ["male", "female"]}
+        ]
+        rv = self.post_assert_metric(CHART_DATA_URI, request_payload, "data")
+        response_payload = json.loads(rv.data.decode("utf-8"))
+        result = response_payload["result"][0]
+        data = result["data"]
+        assert {column for column in data[0].keys()} == {"male_or_female", "sum__num"}
+        unique_genders = {row["male_or_female"] for row in data}
+        assert unique_genders == {"male", "female"}
+        assert result["applied_filters"] == [{"column": "male_or_female"}]