User Evaluation
View the product ↗AI research, with less cost per job.
User Evaluation turns interviews into findings backed by source material.
We optimised the AI pipeline behind its transcription, chat and reports, reducing the cost of each job while keeping its internal quality benchmarks in place.
The challenge
Keep research useful as usage grows.
Audio, video and text processing adds inference cost to every session. As volume grew, they needed lower cost and less waiting for synthesis.
The existing pipeline and benchmarks were the starting point. Every change had to earn its place against their internal eval suite.
The solution
Change the pipeline. Check the output.
We measured the cost and latency of every call across transcription, synthesis and reporting, then matched models to tasks: lighter models where they meet the bar, larger ones where the task needs them.
We batched transcription where timing allows, shortened synthesis prompts and cached repeated retrieval. Every change ran through the regression harness before release, and per-call cost is monitored in production.
[Demo] — User Evaluation
- 01 / 06
Ask Eva about all your research
Eva searches every study, agent run and upload. Each answer cites the clips it uses.
- 02 / 06
Plan a study in minutes
Eva writes the interview guide, and the summary shows the interviews and the total price before launch.
- 03 / 06
Watch interviews live
Eva runs the interview by voice. Follow the live transcript and send one line to steer the next question.
- 04 / 06
Track every session
Status, length and sentiment for each interview. No-shows are rebooked and replaced free.
- 05 / 06
Sort clips into themes
The board codes every clip by theme on a first pass. Rename, merge or move cards, then export to CSV.
- 06 / 06
Test your site with AI agents
Swarm sends agents with different personas through your site and records the friction they find.
The results
Want results like this?
Lower per-job cost
Faster synthesis
Eval regression
[Keep scrolling to see more]Next caseFreeLLMAPI