Question on Dust's Use of Snowflake Metadata Descriptions

Hi everyone, I have a question about Snowflake metadata that I couldn’t find an answer to in the documentation: https://docs.dust.tt/docs/snowflake-connection. Does Dust utilize the column and table descriptions we add to the objects in Snowflake? In short, I’d like to know if Dust leverages the documentation we include directly in Snowflake (for example: “to identify customers in the trial period, you must filter on status=‘trial’”), or do we need to input this information directly into Dust? Thanks in advance for your help!

3 comments

henry
·
·
Hi Thibaut Courtain ! Not currently — the reason for this is that these descriptions are often incomplete / outdated / abandoned for many Snowflake DBs and wrong documentation can be worse than no documentation at all. For now, we recommend documenting tables (what they contain, when to use them, how to use them) in the instructions of your agents. That being said, your feedback is useful and we’re always looking for ways to improve. Pulling tables documentation from Snowflake, BigQuery or DBT might be one of them. Out of curiosity, do you know if it is possible to pull this documentation directly from a SQL query?
✨1
Thibaut Courtain
·
·
Hello henry and thank you for your feedback. Indeed, primarily for AI purposes, we have begun documenting our main tables and columns in dbt. This documentation is directly pushed to Snowflake and can be retrieved using the following commands:
DESC TABLE XXXX.XXXXX; to get column descriptions. I noticed in the query history that Dust performs these kinds of queries, which is why I was curious if it was used to provide more context to the assistants.
SELECT table_name, comment FROM information_schema.tables; to obtain the table description.
It would be incredibly beneficial for us to handle documentation in dbt alone, without needing to insert instructions into the agent. This ensures that everyone using the table is working with the correct guidance.
henry
·
·
We'll keep you posted on how / when we implement this ! In the meantime, it's not super ergonomic but you can emulate this by asking your agent to fetch the table comments once before running SQL queries. This might be a bit slow because the assistant will need to generate this query but it will work. Alternatively, you can make a dust app where you hard-code a SQL query that retrieves the table comments, and ask your agent to always call this dust app before executing SQL queries (this will be faster) Happy to help with either of those if needed !
🙌2