Voice Product Search from 57% to 93%: a Voice Agent Taking Orders for a Bakery
A regional bakery calls its retail outlets every day and collects orders by hand. We built analytics, a CRM and a voice agent that calls the outlets itself, takes the order and creates a draft invoice.
- 93%items recognised in test dialogues (was 57%)
- 2 119phrasings in the search test
In hours and people: The honest result of the first weeks: the agent became the manager’s assistant, not a replacement. It collects the order and the manager checks the invoice. The next step is already visible in the numbers: a stronger model reviewing the recording after the call reached 96% accuracy versus 85% live.
Industry context
In bread distribution orders are collected daily within a short window: an outlet has to order before routes are built. An operator knows what each outlet usually takes and in a minute of conversation gets 5 to 15 items named any way the customer likes. That is the real difficulty for a voice agent: not voice synthesis or telephony, but recognising the item from live speech, mixed languages and recognition errors, and not losing the ones said in the middle of its own reply.
The challenge
Orders are collected by a team of operators who call outlets along delivery routes every day. Accounting lives in an old desktop program with no API, so nobody sees which customers are dropping off or which items they stopped ordering. The calling takes many people and does not scale.
Our approach
- 01
Data without replacing the old system
A daily copy of the accounting data on a separate server: analytics, customer tiers and churn signals without changing how people work.
- 02
Code decides, the model speaks
The tool decides the item, availability and price and returns a ready line. The prompt went from 19.7k to 7.9k characters.
- 03
Similarity search
We tested 2,119 real phrasings across 267 price-list items. Similarity search with clarifying questions replaced hand-written synonyms.
How the solution works
- 01VapiThe agent calls the outlet
- 02Soniox + ElevenLabsRecognises speech and answers by voice
- 03n8nFinds items and the customer’s usual orders
- 04CRMCreates a draft invoice
- 05ManagerChecks the invoice and escalations
A Vapi agent with n8n calls the outlet, takes the order in natural language, matches items against the price list and this customer’s usual orders, and creates a draft invoice in the CRM. Problem calls reach a manager as escalations. Alongside it, a web CRM with customer cards, invoices, stock, label and waybill printing, and a call log.
Why this stack
Vapi provides telephony and the recognition, model and synthesis pipeline without our own infrastructure. Soniox handled mixed Ukrainian and Russian product names better than the alternatives, ElevenLabs gives a natural voice. n8n holds the tools: item search, the customer’s usual orders, invoice creation. All decision logic lives in tool code rather than the prompt, so behaviour does not depend on the model and can be verified by tests.
Where projects like this usually break
Silence mid-dictation
The database answered in a fraction of a second, yet the customer waited long enough to say "hello?" several times: each item was a separate round trip to a tool. A short holding phrase, plus a second one for slow lookups, fixed it.
Test data in the live prompt
A forgotten "demo card" in the prompt made the agent offer what the customer had just ordered and quote stock nobody had entered. Now quantities come only from a tool, and the prompt is reread in full before every release.
People did not know who was calling
The first robot calls got sharp reactions. We advised that a dispatcher warns each outlet once in person, the agent’s first line explains the purpose of the call, and the rollout starts with a small group of loyal customers.
Results
| Before | After |
|---|---|
| Operators called every outlet by hand daily | The agent calls and creates a draft invoice itself |
| Item search recognised 57% of phrasings | 93% of items recognised in test dialogues |
| No view of which customers drop off | Customer tiers and churn signals every day |
Analytics from mid-July, the voice agent from late August, real calls to outlets from October. Work continues.
Who this fits
A fit for producers and distributors who collect orders by phone from regular customers every day: bread, dairy, drinks, water, pet food. The biggest gain is where orders repeat and the customer names 5 to 20 familiar items. If every order is unique and needs negotiation, start with analytics and a CRM and add the voice agent later.
Frequently asked
Does the agent replace the operators?
Not at the start: it collects the order and a manager checks the invoice. We honestly count how many invoices needed no edits and move towards replacement where accuracy is already high enough.
How does the agent understand mixed speech and shorthand?
Similarity search with clarifying questions, tested on 2,119 real phrasings, plus priority for items this customer ordered in the last 90 days. When two managers map a heard word to the same item, it becomes a synonym.
Do we need to replace the old accounting program?
No. We make a daily copy of the data on a separate server and build analytics, the CRM and the agent on top. The team keeps working as usual.
What does a call cost?
On real calls about €0.12 per conversation on the Vapi platform plus telephony. That is well below the cost of an operator’s minute.