Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_gbq: add mechanism to ensure BQ storage API usage #886

Open
calbach opened this issue Feb 28, 2025 · 0 comments
Open

read_gbq: add mechanism to ensure BQ storage API usage #886

calbach opened this issue Feb 28, 2025 · 0 comments
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API.

Comments

@calbach
Copy link

calbach commented Feb 28, 2025

Assumption: there is currently no supported way to force read_gbq to use the BQ storage API. I'd be happy to be corrected if I missed something!

Is your feature request related to a problem? Please describe.

I have cases where read_gbq's heuristic chooses the JSON API, when I want the storage API. This is most noticeable for me on medium sized tables, which might take 5-20 seconds to load via the JSON API (comparatively these were much faster via the storage API). For many of my use cases: to make interactive use cases more bearable, I am very willing to pay the additional storage API cost.

Describe the solution you'd like

A parameter to read_gbq which forces the usage of the BQ storage API (including raising an error if the necessary deps are not available to do so). I won't try to be prescriptive about the details, though I'll note that the desired behavior I've described is what I expected from use_bqstorage_api, based on the name. From my understanding of the current behavior, allow_bqstorage_api is maybe more accurate.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Feb 28, 2025
@GarrettWu GarrettWu assigned TrevorBergeron and unassigned GarrettWu Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API.
Projects
None yet
Development

No branches or pull requests

3 participants