LiveKit SIP Echo Cancellation: Reference Audio, Delay, and Troubleshooting

The short answer

LiveKit SIP echo cancellation needs a far-end reference: the audio your agent sends that may return through the caller's handset, speakerphone, or carrier path. In livekit-plugins-denoise, EchoReferenceTap captures that outgoing chain and supplies it to the TelephonyDenoiser. Attach the tap after session.start(), keep one denoiser per call, and tune stream_delay_ms against a controlled phone route. Noise suppression can reduce background sound, but it cannot replace a correctly aligned echo reference.

What creates echo in a phone call?

Echo is a copy of the agent's voice that leaks from the caller's earpiece or speaker into the caller's microphone and travels back to the agent. The return path can include handset acoustics, room reflections, Bluetooth processing, codec buffering, a SIP trunk, and carrier jitter. The returned signal is delayed and changed, so an echo canceller compares it with a reference instead of simply deleting anything that sounds like speech.

Noise is different. A fan, car, keyboard, or street is not known to the agent's output, so it has no matching reference. A noise suppressor estimates which parts of the inbound signal are unwanted and reduces them. In a production inbound voice agent, both controls may be useful, but they solve different signal paths. LiveKit's official noise and echo cancellation documentation is a useful overview of those deployment concepts.

How EchoReferenceTap fits the chain

The package exposes a TelephonyDenoiser for inbound processing and an EchoReferenceTap for outbound reference capture. The tap is not a second output; it forwards audio to the existing next processor while also making a copy available to the echo canceller.

The ordering is important:

await session.start(
    agent=agent,
    room=room,
    room_options=room_options,
)

session.output.audio = telephony_denoise.EchoReferenceTap(
    denoiser,
    next_in_chain=session.output.audio,
)

RoomIO may install its output chain during session startup. If you attach the tap before startup, your assignment can be replaced. If you attach it afterward to the current output, the reference observes the audio that the session is actually sending.

Read the combined integration guide for the full input and output example, including noise suppression options.

Tune stream_delay_ms methodically

The stream_delay_ms option tells the processor approximately how far the returning echo is from the outgoing reference. The default is 120 milliseconds, which is only a starting point. A local softphone, mobile call, PSTN call, and SIP trunk can each produce a different delay.

Use a repeatable test instead of changing several controls at once:

1. Choose one route and one endpoint, ideally a handset or speakerphone that reliably produces echo.
2. Speak short phrases with pauses so the returning copy is easy to hear.
3. Confirm that EchoReferenceTap is active and that the agent output is audible at the far end.
4. Change only stream_delay_ms and repeat the same phrases.
5. Note residual echo, double-talk behavior, speech coloration, and clipping.
6. Repeat on the next meaningful carrier or device route.

The best setting is the one that reduces returned agent speech without damaging the caller's own speech. Do not treat one successful laptop test as proof that every phone route is aligned.

A practical troubleshooting tree

Echo is unchanged

First verify the reference path, not the suppression level. Is the tap attached after session.start()? Is it wrapping the current session.output.audio? Does the output chain actually carry the agent's audio? Is the denoiser instance the same one passed to the tap? A missing reference makes delay tuning ineffective.

Echo is reduced but speech sounds hollow

The reference may be misaligned or another echo canceller may be processing the stream. Confirm whether the endpoint, SIP trunk, browser, and agent each have cancellation enabled. Keep a single owner for the stage whenever possible. Try a nearby delay value and compare with a clean handset route.

Echo happens only during double-talk

Double-talk is harder because caller speech and returned agent speech overlap. Test interruptions, not only silent pauses. Verify that the agent is not clipping its own output and that the carrier is not applying an additional voice-activity gate that changes timing.

Echo happens only on speakerphone

Speakerphone distance and room reflections can overwhelm an otherwise good reference. Test a handset and headset separately. If only one endpoint fails, document that endpoint behavior rather than weakening every call's processing.

Some routes work and others fail

Compare sample rate, codec, carrier, endpoint, and measured delay. Route-specific behavior is common in telephony. Keep route-level settings in configuration, and make the chosen delay visible in startup logs without recording private audio.

WebRTC versus DeepFilterNet3 for the noise stage

WebRTC suppression is a sensible lower-latency choice when the worker needs a small processing path. DeepFilterNet3 through deepfilter-stream is the default enhancer when you want neural speech enhancement and can accommodate model initialization and cache management. Neither choice fixes a missing echo reference. Select the enhancer for background noise, then validate echo cancellation independently.

Controlled validation calls

Create a test matrix before production rollout:

For each call, record configuration, not caller content: enhancer, stream_delay_ms, sample rate, endpoint type, and whether a second cancellation stage was active. Keep a short operator rating for residual echo and speech quality. If you use recordings, obtain consent and apply your retention policy.

Deployment checklist

• Create one TelephonyDenoiser per call.
• Attach EchoReferenceTap after session.start().
• Start with stream_delay_ms=120 and tune by route.
• Avoid stacking AGC or multiple echo cancellers.
• Prewarm DeepFilterNet3 before the first caller.
• Test handset, speakerphone, mobile, PSTN, and softphone paths.
• Keep a human handoff path for calls the system cannot resolve.

If you are migrating a larger phone workflow, compare the managed versus self-hosted denoising guide, then see the voice-agent product overview and the traditional IVR migration guide.