Dust Community Icon
Join the Community
Help

Questions on RAG Performance: Reranking and Markdown Benefits

Avatar of Romain DechampsRomain Dechamps
·
·

Quick questions on RAG performances :

  • Is Dust using some reranking before providing an answer?

  • Is there any (proven) improvement on performances if you transform you content in markdown before you upload it to dust ?

There were a lot interesting articles about these 2 topics lately :)

1 comment

  • Avatar of Alban
    Alban
    ·
    ·

    • We're not currently using reranking but vector similarity to retrieve at most 32 chunks of data

    • Nothing proven on that side - we do process the documents to chunk them correctly, no matter their format. The main thing is that they need to be structured so that chunks end up with the right amount of header information (markdown or not shouldn't matter, as we have website datasources, etc.. not only documents)

    👍1