The Shift from Full Ingestion to 3rd Party APIs: Implications for Dust
Echoing to this conversation https://dustcommunity.slack.com/archives/C07FXL42MT2/p1756908534985679?thread_ts=1756729176.764359&cid=C07FXL42MT2 It seems that a lof of players, including Dust, are moving more and more from full ingestion to reliance on 3rd party MCP/Search APIs. (Dust native connections to Hubspot or Linear moved to search tools, Slack blocking its ingestion, etc.) Happy to be convinced of the opposite but: MCP/Search =/= Ingestion with retrieval on top. And it is really visible in the quality of the outputs. Dust's retrieval is very good, there is a lot of added value in it, when the data is not ingested in Dust, but agents rely on the search api exposed by the third party tool, the agent relies on the response of the API, which is often subpar and therefore the agent's output is also subpar In the end, a custom GPT with the MCP/API of a 3rd party tool as an "Action" is an already available feature in ChatGPT. I believe the RAG engine of Dust is a key differentiator and if more and more data slips out of its scope, the value of the platform as a whole would decrease Is this something the Dust team is taking into account? Am I reading the dynamics wrong?
