Development Guide
XiaoZhi AI Development Documentation
This documentation is intended for developers who want to compile or modify the XiaoZhi AI program themselves. It includes the complete development environment setup process, core communication protocols, and implementation methods for extended functionality.
Development Environment
- ESP-IDF Development Environment Setup and XiaoZhi Compilation - requires ESP-IDF v6.0.1 or later
- Custom Board Guide - add your own board,
config.jsonandscripts/build.py - Custom Assets Guide - build
assets.bin: wake word, fonts, emoji and chat background - Ecosystem Projects - community servers, clients and ports
- BluFi Provisioning Guide - BLE Wi-Fi provisioning
- Code Style Guide - clang-format and C++ runtime rules
Communication Protocol Documentation
WebSocket Communication Protocol
Detailed explanation of WebSocket communication implementation between devices and servers, including connection establishment, message formats, audio transmission, and other core functions.
MQTT + UDP Hybrid Protocol
MQTT control channel combined with an encrypted UDP audio channel.
MCP Protocol
JSON-RPC 2.0 based MCP flow between the device and the backend: tool discovery, tool calls and user-only tools.
MCP IoT Control Usage
Device-side tool registration, built-in tools and common call examples.
Dynamic Text Glyph Push Extension
The server pushes bitmap glyphs missing from the device font; WebSocket and MQTT/UDP share one format.
Asynchronous Voice Notifications
The cloud plays a one-way voice notification while the device is idle, streaming Ogg Opus audio with timed subtitles.
Emoji Emotion Display
Introduction to how large language models use Emoji to express emotional states, as well as related data formats and usage instructions.
Project Source Code Structure
The XiaoZhi AI project consists of the following main components:
- Main Control Program: Runs on ESP32 series chips
- Server Side: Handles device connections and voice forwarding
- Frontend Interface: Network configuration and settings page
Detailed source code structure and function descriptions can be found in the GitHub repository.
Common Development Issues
- Compilation Error Resolution: Usually related to ESP-IDF version mismatch; please use the specific ESP-IDF version recommended in the documentation
- Connection Timeout: Check WebSocket server configuration and network connection
- Insufficient Memory: ESP32-S3 development boards offer higher performance and are recommended
Documentation Index
| Document Name | Description | Update Time |
|---|---|---|
| ESP-IDF Development Environment Setup | ESP-IDF v6.0.1+ environment setup and XiaoZhi build guide | 2026-10-23 |
| Custom Board Guide | Directory layout, configuration and build flow for a new board | 2026-10-23 |
| Custom Assets Guide | Pack and write wake word, fonts, emoji and chat background | 2026-10-23 |
| Ecosystem Projects | Community servers, clients and ports | 2026-10-23 |
| WebSocket Communication Protocol | WebSocket client and server communication protocol documentation | 2026-10-23 |
| MQTT + UDP Hybrid Protocol | MQTT control channel and UDP audio channel | 2026-10-23 |
| MCP Protocol | MCP flow, tool discovery and calls | 2026-10-23 |
| MCP IoT Control Usage | Tool registration, built-in tools and examples | 2026-10-23 |
| Dynamic Text Glyph Push Extension | glyph_push protocol extension v1 | 2026-10-23 |
| Asynchronous Voice Notifications | notify message and Ogg Opus streaming playback | 2026-10-23 |
| Emoji Emotion Display | Communication protocol for large language models to express emotional states via Emoji | 2025-03-06 |