Cache persistence: persist bq_memory_limited flag in QueryCacheManager
so truncation warnings survive cache hits. The flag is now saved into
the cache value dict in set_query_result (reading from g and resetting
it there), and restored from cache in get(). get_df_payload now reads
cache.bq_memory_limited instead of g, removing the need for g entirely
in query_context_processor.py.
Frontend test: add two tests to chartActions.test.ts verifying that
addWarningToast is dispatched (with noDuplicate: true) when a query
response carries a warning field, and not dispatched when it doesn't.
Updated existing test_query_context_processor tests to set the flag on
mock_cache directly rather than patching g.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use has_app_context()/has_request_context() guards so fetch_data
is safe to call outside a Flask request (fixes RuntimeError on g
writes and current_app access in non-request paths)
- Replace sys.getsizeof(str(batch)) with per-row getsizeof sum for
more accurate memory estimation without the str() allocation
- Fix false-positive truncation: fetch remaining+1 rows and check
len > remaining to confirm more data exists beyond the cap
- Reset g.bq_memory_limited/g.bq_memory_limited_row_count after
reading in get_df_payload to prevent flag leaking across multiple
queries in the same request
- Wrap warning string in _() for i18n
- Add warning field to ChartDataResponseResult Marshmallow schema
- Pass noDuplicate: true to addWarningToast to suppress duplicate
toasts when a multi-query chart has multiple truncated responses
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement memory-aware progressive fetching in BigQuery's fetch_data
method. Large result sets (950+ MB) previously crashed Chrome by loading
everything into memory at once. The fix samples an initial batch to
estimate row size, then fetches only as many rows as fit within the
BQ_FETCH_MAX_MB config limit (default 200 MB). A warning toast is shown
to users when results are truncated.
This is always-on with no feature flag -- operators control the budget
via the BQ_FETCH_MAX_MB config constant.
Originally by @ethan-l-geotab in apache#36387.
Co-authored-by: ethan-l-geotab <ethanliong@geotab.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
