Troubleshooting BigQuery Rate Limit Issues with Dust Agent

·

[Hitting rate limit/quotas on Bigquery] Hey, we're playing with Dust & Bigquery. We've setup a first Agent into Slack. It works fine most of the time but sometimes we hit a limit (see thread for detailed error) I checked the logs of the service account and it seems to launch many queries at a time, is it a bug or is our setup wrong ?

✅1

7 comments

Hugo Hahn

·

Error running TablesQuery app: Error getting table metadata: Response error (error: ResponseError { error: NestedResponseError { code: 403, errors: [{"domain": "usageLimits", "reason": "rateLimitExceeded", "message": "Exceeded rate limits: too many api requests per user per method for this user_method (TableService.getTable). For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas"}], message: "Exceeded rate limits: too many api requests per user per method for this user_method (TableService.getTable). For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas", status: "PERMISSION_DENIED" } })

Hugo Hahn
·
·
346 requests/s seems high 😬
Daphné
·
·
Hi! You can try to untick the "Use descriptions" option to decrease the number of queries.
Daphné
·
·
Hugo Hahn
·
·
Daphné thank you for the quick turnaround.
Hugo Hahn
·
·
If we do not send the column descriptions, how do you advise sharing with the Agent the descriptions ? Provide them directly in the prompt ?
Hugo Hahn
·
·
I guess the best way is to make our tables smaller and have dedicated tables for our agent.
👍🏻1