QT/superset2
1
0
Fork 0
mirror of https://github.com/apache/superset.git synced 2026-05-29 20:29:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
19,215 Commits 490 Branches 545 Tags
1894b77b447df8f08decad2deeaffef595e239aa
Commit Graph

97 Commits

Author SHA1 Message Date
Claude Code
1894b77b44 fix(bigquery): address review comments on memory-limited fetch 
- Use has_app_context()/has_request_context() guards so fetch_data
  is safe to call outside a Flask request (fixes RuntimeError on g
  writes and current_app access in non-request paths)
- Replace sys.getsizeof(str(batch)) with per-row getsizeof sum for
  more accurate memory estimation without the str() allocation
- Fix false-positive truncation: fetch remaining+1 rows and check
  len > remaining to confirm more data exists beyond the cap
- Reset g.bq_memory_limited/g.bq_memory_limited_row_count after
  reading in get_df_payload to prevent flag leaking across multiple
  queries in the same request
- Wrap warning string in _() for i18n
- Add warning field to ChartDataResponseResult Marshmallow schema
- Pass noDuplicate: true to addWarningToast to suppress duplicate
  toasts when a multi-query chart has multiple truncated responses

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-07 13:16:58 -07:00
Claude Code
1773531807 fix(bigquery): limit result set size to prevent browser memory crashes 
Implement memory-aware progressive fetching in BigQuery's fetch_data
method. Large result sets (950+ MB) previously crashed Chrome by loading
everything into memory at once. The fix samples an initial batch to
estimate row size, then fetches only as many rows as fit within the
BQ_FETCH_MAX_MB config limit (default 200 MB). A warning toast is shown
to users when results are truncated.

This is always-on with no feature flag -- operators control the budget
via the BQ_FETCH_MAX_MB config constant.

Originally by @ethan-l-geotab in apache#36387.

Co-authored-by: ethan-l-geotab <ethanliong@geotab.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-11 13:20:26 -07:00
Amin Ghadersohi
7f061a3764 fix(bigquery): pass dialect instead of engine to select_star in get_extra_table_metadata (#38281) 2026-02-27 02:11:40 +01:00
Vitor Avila
228b598409 feat: Labels for encrypted fields (#38075) 2026-02-23 13:23:33 -03:00
Elizabeth Thompson
e9b494163b refactor(db): use Dialect instead of Engine in select_star to avoid SSH tunnels (#35540) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-02-02 10:26:35 -08:00
Evan Rusackas
b460ca94c6 feat(docs): auto-generate database documentation from lib.py (#36805) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-21 10:54:01 -08:00
Daniel Vaz Gaspar
bb22eb1ca8 feat: add option for hash algorithms (#35621) 
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
 2025-12-09 16:59:07 +00:00
Beto Dealmeida
cb24737825 feat(bigquery): show materialized views (#34766) 2025-08-26 11:21:43 -04:00
Beto Dealmeida
a26e1d822a chore: remove sqlparse (#33564) 2025-06-04 19:31:41 -04:00
Beto Dealmeida
d88cba92c0 feat: optimize catalog permission sync (#33000) 2025-04-10 17:38:34 -07:00
Maxime Beauchemin
c83eda9551 feat: add latest partition support for BigQuery (#30760) 2025-04-01 17:13:09 -07:00
Ville Brofeldt
c90e45a373 feat: make user agent customizable (#32506) 2025-03-05 16:33:24 -08:00
Beto Dealmeida
f235787703 fix(bigquery): return no catalogs when creds not set (#31837) 2025-01-15 11:58:44 -05:00
Beto Dealmeida
09802acf0d refactor: remove more sqlparse (#31032) 2024-11-26 17:01:07 -05:00
Jack
849d426e06 feat(oauth): adding necessary changes to support bigquery oauth (#30674) 2024-10-30 14:56:22 -05:00
Jack
95325c4673 fix(imports): Error when importing charts / dashboards with missing DB credentials (#30503) 2024-10-04 15:43:29 -05:00
Beto Dealmeida
4b59e42d3f chore: improve mask/unmask encrypted_extra (#29943) 2024-08-22 16:45:32 -04:00
Eyal Ezer
07b2449bd7 refactor: Unify all json.(loads|dumps) usage to utils.json (#28702) 
Co-authored-by: Eyal Ezer <eyal.ezer@ge.com>
 2024-05-28 14:17:41 -07:00
Beto Dealmeida
5da6d2bd88 feat: add support for catalogs (#28416) 2024-05-16 12:49:31 -04:00
Beto Dealmeida
e90246fd1f feat(SIP-95): permissions for catalogs (#28317) 2024-05-06 11:41:58 -04:00
Beto Dealmeida
6cf681df68 feat(SIP-95): new endpoint for table metadata (#28122) 2024-04-25 12:23:49 -04:00
Maxime Beauchemin
2d63722150 chore: set up ruff as a new linter/formatter (#28158) 2024-04-24 17:19:53 -07:00
Beto Dealmeida
68a982dfe6 feat(sip-95): new endpoint for extra table metadata (#28063) 2024-04-18 10:42:53 -04:00
Beto Dealmeida
99a1601aea refactor: rename get_sqla_engine_with_context (#28012) 2024-04-12 13:31:05 -04:00
Beto Dealmeida
c0b57bd1c3 feat(sqlparse): improve table parsing (#26476) 2024-01-22 11:16:50 -05:00
Pieter Ennes
effd73f2cc fix: Catch ImportErrors for Google SDKs (#25550) 2024-01-19 15:00:43 -08:00
Sebastian Liebscher
067bfbf950 chore: move TypedDict from typing_extensions to typing (#25121) 2023-08-30 09:49:45 -07:00
Sebastian Liebscher
e585db85b6 chore: use contextlib.surpress instead of passing on error (#24896) 
Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com>
 2023-08-29 10:09:01 -06:00
Hugh A. Miles II
93e1db4bd9 fix: save columns reference from sqllab save datasets flow (#24248) 2023-06-20 13:54:19 -04:00
Michael S. Molina
e5b7f7c9b5 fix: Time shifts with different granularity for ECharts (#24176) 2023-06-08 16:03:37 -03:00
John Bodley
a4d5d7c6b9 chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
John Bodley
d583ca9ef5 chore: Embrace the walrus operator (#24127) 2023-05-19 00:37:13 -07:00
Sebastian Liebscher
b31efbae7c chore: remove RemovedInMarshmallow4 warnings (#23704) 2023-04-21 09:11:46 +03:00
John Bodley
b35b5a6e05 chore(db_engine_specs): Refactor get_index (#23656) 2023-04-13 09:23:16 +12:00
Beto Dealmeida
8d14420d14 feat(snowflake): get_catalog_names (#23602) 2023-04-06 12:17:30 -07:00
Beto Dealmeida
7a1aa6387c feat(bigquery): get_catalog_names (#23461) 2023-04-05 13:32:28 -07:00
Ville Brofeldt
cd6fc35f60 chore(db_engine_specs): clean up column spec logic and add tests (#22871) 2023-01-31 15:54:07 +02:00
Antonio Rivero Martinez
ad758c0802 feat(ssh_tunnel): Rename allow_ssh_tunneling and change the default value to False (#22723) 2023-01-17 19:11:41 -08:00
Zamar
001100ddf0 feat(db_engine_specs): big query cost estimation (#21325) 
Co-authored-by: zamar roura <zamar.roura@cabify.es>
Co-authored-by: Zamar Roura <zamarfazal@gmail.com>
 2023-01-09 08:56:19 +02:00
Antonio Rivero Martinez
60a617eaba fix(bigquery): Properly display errors for BigQuery DBs (#22349) 2022-12-07 13:34:30 -05:00
EugeneTorap
7bc5f04368 chore: Bump bigquery libs (#21466) 2022-12-02 12:48:36 +02:00
Hugh A. Miles II
e23efefc46 feat: refactor all get_sqla_engine to use contextmanager in codebase (#21943) 2022-11-15 13:45:14 -05:00
Antonio Rivero Martinez
95b4c7b7fe chore(bigquery): Add extra logging for BigQuery exceptions so we can have better insight on exceptions (#22024) 2022-11-09 22:56:08 -05:00
Ville Brofeldt
dc7399540b chore: bubble up more db error messages (#21982) 2022-11-01 07:54:27 +02:00
Antonio Rivero Martinez
203b289021 feat(bigquery): Custom message when Service Account doesnt have the correct Roles and Permissions (#21838) 2022-10-26 19:44:09 -04:00
Elizabeth Thompson
e98943e580 refactor: pass all properties to validate_parameters (#21487) 2022-10-03 17:48:54 -07:00
Elizabeth Thompson
ef78ec6b30 fix: catch error when masking encrypted extra is none (#21570) 2022-10-02 19:49:01 -07:00
Beto Dealmeida
34a79add04 feat: filter parameters from DB API (#21248) 2022-09-02 11:50:04 -07:00
Beto Dealmeida
7e501cd816 fix: BigQuery get_parameters_from_uri (#20966) 2022-08-03 17:27:40 -07:00
Diego Medina
5afeba34bd fix(viz): BigQuery time grain 'minute'/'second' throws an error (#20350) 2022-06-21 10:06:52 -03:00