XiaoZhi AI and DeepSeek: How the Device and the Large Model Split the Work

January 15, 2025

Many people read “XiaoZhi + DeepSeek” and assume the model runs on the ESP32. It does not: a XiaoZhi device does not connect to a large model directly. It is a voice entry point, and the model is wired in on the server side. The upstream README describes the project this way: the XiaoZhi AI chatbot is a voice interaction entry point that uses the AI capabilities of large models such as Qwen and DeepSeek and controls many endpoints through the MCP protocol.

This article spells out that division of labor and shows, using a community server, how to configure DeepSeek on the server.

Revised on 2026-10-23: we removed the performance comparison numbers, the “hybrid routing algorithm”, the roadmap and the application examples that had no upstream basis. Only content that can be checked against the firmware source, official documents and the server repository remains. This site has not benchmarked any model and provides no comparison data.

1. What the device does and what the server does

Stage	Where	Notes
Wake word detection	Device	Offline wake word based on ESP-SR, custom wake words supported
Audio capture and playback	Device	The microphone is captured and encoded as an Opus stream going up; audio sent down by the server is decoded and played
Display and emotion	Device	OLED / LCD text and emotion display
Speech recognition (ASR)	Server	Turns the uploaded speech into text
LLM conversation	Server	DeepSeek and other models are plugged in at this stage
Speech synthesis (TTS)	Server	Turns the reply text into speech
Device control	Device-side MCP	Controls volume, lights, motors, GPIO and more

The upstream README states that the firmware uses Opus audio streams and supports both the classic streaming ASR + LLM + TTS pipeline and Realtime end-to-end voice models. The protocols between device and server are described in WebSocket and MQTT + UDP.

2. Two ways to connect

Path	Where the model comes from	Best for
Official server (default)	Whatever the official console offers; personal users can register and use the Qwen realtime model for free	You just want it to work and don’t want to run a server
Self-hosted server	Whatever your server supports, including DeepSeek	You want to choose your own LLM, speech recognition and speech synthesis

The firmware does not have and does not need an LLM API key. Keep keys only in your server configuration. For the overall approach to self-hosting, the security warning, and how to point a device at your server, see Self-Hosted Server.

3. Configuring DeepSeek on the server (xiaozhi-esp32-server)

xinnan-tech/xiaozhi-esp32-server is a community open-source server (MIT license). The configuration below comes from main/xiaozhi-server/config.yaml at its v0.9.7 tag:

# Excerpt of main/xiaozhi-server/config.yaml (xiaozhi-esp32-server v0.9.7)
selected_module:
  LLM: DeepSeekLLM

LLM:
  DeepSeekLLM:
    type: openai
    model_name: deepseek-v4-flash
    url: https://api.deepseek.com
    api_key: <your DeepSeek API key>

Notes:

selected_module.LLM picks which LLM entry to use; here it points at the DeepSeekLLM entry below;
type: openai means the model is called through an OpenAI-compatible interface; the server currently supports the LLM types openai, dify and ollama;
model_name changes as DeepSeek updates its models, so follow the latest information on the DeepSeek platform;
the server documentation suggests putting your own changes in data/.config.yaml, which overrides config.yaml, so upgrades do not wipe them;
an API key is sensitive. Do not commit it to a repository or paste it into a public issue.

4. Five steps to a first conversation

Flash the firmware: pick your board’s variant on the Firmware Download page and write it with a flashing tool.
Provision and activate: follow First Boot to set up Wi-Fi.
Deploy the server: follow the server project’s documentation and note its OTA URL and WebSocket URL.
Point the device at your server: enter the OTA URL on the “Advanced” tab of the provisioning page, or set Default OTA URL at build time.
Configure and verify: enter the DeepSeek endpoint and key on the server, start a conversation with the wake word or a button, and confirm in the server log that the request arrived.

5. What to think about before choosing DeepSeek

These are general engineering considerations, not benchmark results:

Latency: a voice turn passes through recognition, the model and synthesis in sequence, and the time to the model’s first token directly affects how long the user waits. Measure on your own network and server.
Cost and rate limits: models are billed per call and may be rate-limited. Read the provider’s pricing and terms.
Privacy: conversation text is sent to a third-party API. If privacy matters, read the provider’s data policy first and consider a model you can host locally.
Replaceability: the server treats the LLM as a configuration entry, so switching models usually only means changing configuration; do not hard-code model details in business logic.
Security: a self-hosted server has not undergone a security assessment. Protect it properly before exposing it to the internet; see the warning in Self-Hosted Server.

Sources

Upstream 78/xiaozhi-esp32 v2.5.2: README.md.
Community project xiaozhi-esp32-server, main/xiaozhi-server/config.yaml at the v0.9.7 tag.
Checked on 2026-10-23. Send corrections to [email protected].

Which Board Should Run XiaoZhi? A Three-Step Guide by Chip, Network and Ready-Made Board