Big performance issues on all models

Hello there, I’ve seen there are some degraded performance on Claude but using GPT feels also the same, both models are stupid on Dust.tt compared to original chats.

Example right now: asking GPT to analyze a .csv file and do a ranking of the occurrences of a word he sees in the file. Not only the number are wrong but the final ranking mix different numbers.

I tried on the free version of ChatGPT, same file, same prompt, and he nailed it on first try.

What’s the reason for this? Thank you

It’s hard to be sure what to answer without looking at the CSV.

There are a few possibilities:

  • ChatGPT has a code interpreter enabled by default that might manipulate your csv before answering - we are currently working on this feature!
  • The model you talked to might not be exactly the same, you can create an assistant that calls a specific GPT4 version - we’ve had better results on GPT-4 turbo than on GPT4-o in some cases

Screenshot 2024-09-05 at 10.54.32

But it’s really likely about the code interpretation: your csv might be “understood” by a program before it gets sent to the LLM, that will help perform operations on it.

We’re planning to release something similar in the future, I unfortunately don’t have an ETA to share for now!

If I am not mistaking, until the new features arrive to enable you to do it directly from the chat interface, you can upload your csv in a Dust folder as “table”. Then you can create a custom assistant with a “query table” tool and link it to your table.

The assistant will then manipulate the csv like chatGPT would. :blush:

2 Likes

Thank you for your answers, I used @remicoffenbach solution and it worked. A bit overkill for quick tasks but good to know ^^

1 Like