AI voice agent on the phone: setup and tools

A voice agent that holds the conversation instead of an operator: connecting ElevenLabs, the SIP trunk, 20 CRM tools, outbound AI dialer campaigns, agent permissions, the call length limit and the post-call summary in the lead card.

10 min readUpdated September 22, 2026

The AI voice agent holds a conversation instead of an operator: speech recognition, reasoning and speech synthesis run on ElevenLabs Agents, while the data comes from the CRM. This is neither an answering machine nor an IVR: it understands free speech, asks clarifying questions and changes data in the CRM during the call.

#How a call goes

01
The inbound call arrives at your telephony
Routing decides where it goes: to operators during business hours, to the agent at night, on a dedicated number, or when the queue is full.
02
The agent answers
The first message and the character of the conversation come from the prompt. You can load an existing call script from the CRM so the agent says what the operators say.
03
It identifies the caller
A lookup_customer call by phone number: a returning client, a student, or an unknown number. The scenario follows from that.
04
It performs actions in the CRM
Searches the catalogue, checks free slots, creates a lead, books an appointment — through the same mechanisms people use.
05
It sends the summary afterwards
post_call_webhook puts the summary, the transcript and the recording into the lead card.

#Tools available to the agent

Customer and lead

Tool	Action
`lookup_customer`	who is calling and whether they have an active lead
`get_order`	order contents, total, waybill number
`create_lead`	create a lead for an unknown number
`update_customer`	record the name, address or pickup point

Catalogue and cart

Tool	Action
`search_products`	search the products and services catalogue
`get_product`	the item card
`get_cart`	what is already in the lead's cart
`add_to_cart`	add an item
`set_cart_quantity`	change the quantity

Calendar

Tool	Action
`list_free_slots`	free time for a particular service
`book_appointment`	book a specific slot
`my_next_lesson`	answers “when is my next class?”
`cancel_appointment`	cancel a booking

Ending the call

Tool	Action
`set_call_outcome`	the call result: a status code plus a note for the manager
`confirm_order`	confirm the order
`cancel_order`	cancel the order
`schedule_meeting`	a meeting: time, channel, messenger, email, topic
`schedule_callback`	call back later
`transfer_to_operator`	hand over to a human

#Boundaries the CRM enforces

The agent has no separate route to the data: it uses the same engine people use and runs into the same checks.

Rule	Why
Double booking is impossible	a booking passes the same availability check as one made by hand
Cancellation deadline	“no later than N hours before the start” applies to the agent too
Agent permissions	separate flags for “may create leads” and “may change the cart”
Duration limit	a hard cut-off: billing is per minute, and a looping dialogue will not end by itself
Refusals in plain words	the agent reads out the reason, not an error code
The agent never marks “no answer”	an unanswered call is a conversation that never happened, so no tool was called in it; those numbers show up in the campaign queue instead
In a product project	calendar tools answer that time-slot booking is not offered here

#Outbound calls

The same agent also works outbound. There are two routes, and they are not equivalent:

Route	Who owns the queue	When to pick it
The CRM AI dialer	the CRM: queue, retries, calling hours, concurrent call limit, lead locking	almost always — the result lands in the same pipeline as every other lead
ElevenLabs batch calling	the ElevenLabs dashboard	when the campaign is already built there; the CRM then finds the lead by phone number

Both are covered step by step in a separate article: outbound AI agent campaigns. It also covers the call-outcome statuses, the post-call webhook (without it the dialer never closes the queue) and conversation analytics.

#Agent settings

Section Integrations → AI agent.

Agent parameters

Prompt and first messagetextrequired
character, topic boundaries and the conversation scenario
Call scriptfrom CRM
load an existing script so the agent says what operators say
Voice, language, speedchoicerequired
any ElevenLabs voice; in a live conversation the speed ceiling is 1.2×
Projectchoicerequired
which project the created leads land in
Status for new leadschoice
which status to create them with; a dedicated one makes the agent's work visible in the pipeline
Manager for meetingsuser
who schedule_meeting books time with
Transfer groupqueue
where transfer_to_operator sends the call
Call length limitsecondsrequired
protection against an endless dialogue — but with headroom above the target length: a timeout cut-off kills the set_call_outcome call and the summary with it

#Integration settings

Section Integrations → ElevenLabs — access to the service itself, shared by the agent and by menu voice-overs.

Integration parameters

API keysecretrequired
your account and your per-minute bill
Default voicechoice
also used for message voice-overs
Modelchoice
the quality-versus-latency trade-off
Stability and similarity0–1
how consistent the voice sounds between utterances
Daily character limitnumber
a fuse against an unexpected bill
Webhook secretsecretrequired
signs inbound tool calls and the post-call webhook
Pull call history automaticallyflag
every half hour the CRM fetches conversations it does not have yet — a safety net for when the webhook was never set up or a delivery was lost

#Voice-overs for the phone menu

The same service voices IVR prompts: you write the text, the system generates the file and the telephony plays it. Regeneration is free as long as the text has not changed, so the menu can be edited freely.

Voice-over speed goes up to 1.5×, but in a live conversation with the agent the ceiling is 1.2×: faster speech breaks recognition of the other party's reply.

#Before the first real call

The call length limit is set — before launch, not after the bill.
The daily character limit is set in the integration.
Agent permissions are narrowed to what it actually needs.
There is a dedicated status for leads created by the agent.
Hand-over has been tested: the transfer group exists and has operators in it.
The “no free slots” and “client wants to cancel past the deadline” scenarios have been tested.
The privacy policy states that audio is passed to an external service.
The summary language is set to the language of your leads — by default ElevenLabs summarises the call in English.

The summary, transcript and recording of an agent call land in the lead card alongside ordinary calls, so the same tools apply: call recordings and transcription.