Hello, Does anyone have experience with Data Streams into Dust? We have a lot of operational data we would like our bot to have access to. Other applications get sent this data through Segment, is there any option to bring data from various sources into Dust?
Remi, could you help answer this question?
Kevin Straszburger, do you have any ideas?
Let me have a quick look 👀
Jasper Lanz Here is the answer from @help on Dust - which I think is relevant in your case! I can help explain the different ways you can bring operational data into Dust. There are several options available: 1. Native Connections Dust offers built-in connections with many platforms including: - Confluence - Google Drive - GitHub - Notion - Slack - Snowflake - Zendesk - Intercom - Microsoft SharePoint 2. Zapier Integration If you're using Segment, you can leverage Zapier to automatically sync data into Dust. Dust has pre-built Zapier templates for: - Gmail - Hubspot - Zendesk And you can create custom Zaps for other data sources. 3. API Integration You can programmatically add data using Dust's API. The process includes: - Creating a data source - Adding documents via API - Documents are automatically chunked, embedded and indexed for semantic search 4. Custom Scripts Dust provides open-source scripts for specific integrations like: - Hubspot data importer - Salesforce accounts importer - Jira issues importer When data is added to Dust, it's automatically processed to enable semantic search capabilities and can be used by your agents for retrieval-augmented generation (RAG). For your specific case with Segment data, I would recommend either: 1. Using the Zapier integration to automatically sync data from your Segment sources to Dust 2. Using the Dust API to build a custom integration that receives data from Segment and pushes it to Dust
Relevant docs:
Dust integration Cookbook (concrete examples): https://github.com/dust-tt/dust-labs/tree/main
Zapier integration doc: https://docs.dust.tt/docs/trigger-your-dust-agent-from-anywhere-using-zapier
Let me know if you have any questions!
Remi Kevin Straszburger, found an interim solution by connecting our BigQuery project to Dust, however, the bot seems to struggle pulling data. Is there something that you see we are doing wrong here?
Apologies for the late reply here Jasper Lanz ! Were you able to make it work? Can it retrieve some data or it doesn’t work at all?