Measuring Agent Impact on Team Productivity: Best Practices and Insights

·Jan 02, 2025 09:46 AM·

Happy New Year, Dust Community! 🎉 Quick question: How are you measuring the impact of your agents on your teams' productivity? At Doctolib, we've deployed agents to automate repetitive tasks and support data-driven decisions. While early feedback are very promising, we want to set a quantitative targets for these productivity gains Has anyone built a similar productivity measurement system? We're specifically interested in:

What metrics are you tracking?
How do you collect and validate this data?
Have you found ways to isolate AI's impact from other productivity factors?

If you're interested in building something similar (my scope is T&P), I'd be happy to share our findings and learnings so far. Thanks

5 comments

· Sorted by Oldest

Remi
·
·
Hey Bastien Mouquet ! Happy new year to you too 🌟 Unfortunately measuring impact is never easy. When I was at Alan we decided to have a double approach:
Marco:
Dust daily active members
Avg daily message per active members
A qualitative survey: "how much time do you save per week"
Micro:
For specific, high leverage use cases, we'd have a look time taken for achieving a task before & after
Example: reducing meeting prep time for sales without Dust vs with Dust
Then we look at adoption of the given assistant
With the 2, we could get closer to a overall impact for Dust. But it wasn't perfect 😕
Remi
·
·
I am personally very excited about some of the tools coming soon for builders to get feedback on their assistants. You'll see usage per assistant & 👍/👎 convo from the team. Those will also help measure impact in a more decentralized way.
Remi
·
·
Let us know how you end up cracking this ! I am sure it will be helpful to others too !
Bastien Mouquet
·
·
Brilliant, thank you very much! I'll share significant findings!
✨1
yutcam (Dust)
·
·
Alex Laloo can share regarding this as well 😇