Firmware Code Tour: Directory Layout, State Machine and Development Rules

The XiaoZhi firmware is a C/C++ ESP-IDF project that supports many chips, boards, screens, audio devices and network channels. One build selects exactly one board implementation. This page follows AGENTS.md, main/audio/README.md and the source of upstream v2.5.2, so you can find the right place before you touch the code.

1. Source Map

2. Runtime Flow

After initializing NVS, main.cc calls Application::GetInstance().Initialize() and Run(); Run() is a main event loop that never returns. The runtime state is kept by DeviceStateMachine, and DeviceState has these values:

Application exposes entry points such as ToggleChatState(), StartListening(), StopListening(), WakeWordInvoke(), Alert(), Schedule() and SendMcpMessage(). Board code (for example button callbacks) triggers behavior through them.

3. Board Abstraction

Board (main/boards/common/board.h) is the interface the core code depends on. Its main pure virtual methods are GetBoardType(), GetAudioCodec(), GetNetwork(), StartNetwork(), GetNetworkStateIcon(), SetPowerSaveLevel(), GetBoardJson() and GetDeviceStatusJson(); backlight, LED, display, camera and battery are optional capabilities.

boards/common/ provides base classes for different network types, such as WifiBoard, Ml307Board, DualNetworkBoard, EthernetBoard, RndisBoard and Nt26Board, plus helpers for buttons, power management and sleep timers. Every concrete board inherits from one of these base classes and exports itself with DECLARE_BOARD(ClassName); a build must contain exactly one.

For the complete procedure to add a board, see the Custom Board Guide.

4. Audio Path

Audio is managed by AudioService: codec I/O, Opus encoding/decoding and network queues are separated from chip-specific speech processing.

• Uplink: microphone → AudioCodec → AudioInputTask → one AudioEngine (AfeAudioEngine on S3 / P4 / S31, LiteAudioEngine on the other chips) → 16 kHz mono PCM → OpusCodecTask encoding → send queue → network.
• Downlink: network → decode queue → OpusCodecTask decoding → playback queue → AudioOutputTask → codec → speaker.
• The Opus frame length is 60 ms; queues use storage with a capacity fixed at compile time.

For the engine to chip mapping, see Chip Support and Storage Specs.

5. Protocol Layer

Protocol abstracts actions such as opening/closing the audio channel, sending audio, sending the wake word, starting/stopping listening, aborting playback and MCP messages. The concrete transports are WebsocketProtocol and MqttProtocol. For protocol details, see the WebSocket Communication Protocol, the MQTT + UDP Hybrid Protocol and the MCP Protocol.

6. Development Rules

These rules are taken from upstream AGENTS.md. Check them before you change code:

• Keep changes focused and do not mix in unrelated formatting. Format only the C/C++ files you changed, with the repository .clang-format; see the Code Style Guide.
• A build must export exactly one DECLARE_BOARD(...).
• Do not change the pins of an existing board to fit other hardware. Add a new board or release variant with a unique name instead, because the board type identifier affects OTA compatibility.
• Core code depends only on the Board interface, not on a concrete board class or the config.h of a particular board.
• Treat camera, backlight, display, LED and battery as optional capabilities.
• Change the runtime state only through Application::SetDeviceState() and the state machine.
• Callbacks may not run in the main task: use Application::Schedule() or event bits to switch back to the main task before changing application state.
• Do not block the main event loop or the audio tasks; avoid unbounded queues and repeated large allocations in the audio path.
• Put shared message semantics in Protocol; when you change the protocol contract, verify both WebSocket and MQTT/UDP.
• Validate network input and keep the cJSON ownership conventions; NVS keys are a persistent API, so changing them needs a migration.
• Constrain target-specific features with Kconfig or component rules; do not assume every chip has PSRAM or the resources of S3/P4.
• Do not hand-edit generated or third-party output: build/, releases/, managed_components/, components/, sdkconfig*, main/assets/lang_config.h and so on.

7. Build and Verification

Load the ESP-IDF environment first (source /path/to/esp-idf/export.sh, confirm with idf.py --version), then use the commands recommended upstream:

# Discover exact board and variant names
python3 scripts/build.py --list-boards

# Canonical variant build
python3 scripts/build.py <board-directory> --name <variant-name>

# Host-side build tests
python3 -m unittest discover -s scripts/tests -v

scripts/build.py rewrites the local sdkconfig and build state. After switching boards, do not assume the old build directory still matches the previous target.

Upstream expects the following verification:

• Board-only changes: build the affected variants and smoke-test on real hardware.
• Changes to core, common board code, audio, protocol, display, dependencies, Kconfig or CMake: run the host-side tests and build representative chip and network paths.
• Audio changes: verify capture, playback, wake/VAD, barge-in, reconnect and the applicable AEC modes.
• UI or assets changes: verify the no-screen, OLED and LVGL paths and the partition sizes.
• A successful build is not hardware verification. When you report, state what you tested and what still needs real hardware.

Related Documents

• ESP-IDF Development Environment Setup: IDF v6.0.1 or later
• Custom Board Guide: complete steps to add a board
• Advanced Extension Points: network types, wake words, MCP, notifications and asset customization
• Troubleshooting: build, flashing and runtime problems

Sources: AGENTS.md, main/audio/README.md, main/application.h, main/device_state.h, main/boards/common/board.h and main/protocols/protocol.h of 78/xiaozhi-esp32 v2.5.2.