Debug ESP32 crashes in production

See when your ESP32 goes offline, track heap and per-task stack usage, and use crash reports to find the exact line of firmware that failed. In this demo, I connect an ESP32 to mDash, crash it on purpose, and use the matching firmware ELF to trace the crash to main.c:59.

Build, flash and connect the ESP32-C6

Start with the minimal ESP32 example. Set your Wi-Fi credentials in main/main.c. Create a device at mdash.net, set MG_ENABLE_MDASH to 1 in main/mongoose_config.h, and put the device password in MG_MDASH_KEY.

The example enables MG_ENABLE_HEALTH and configures ESP-IDF to save core dumps to flash. For an existing project, follow the Health Monitor setup guide, including the core dump partition.

cd tutorials/esp32/minimal
make build ARCH=esp32c6
make flash ARCH=esp32c6 PORT=/dev/cu.usbmodem2101

Use your board's serial port. The Makefile builds with ESP-IDF in a container. Check the serial logs for Wi-Fi, then check that the device is online in mDash. Keep build/mongoose-esp32-minimal.elf for this build.

Check connection drops, heap and FreeRTOS stacks

The device card shows the firmware version and last reset reason. Red marks in the 24-hour connectivity history show connection drops. In the video, I force a reconnect to show what a drop looks like.

The heap row shows total, current and peak usage. For each FreeRTOS task, subtract peak stack usage from its allocated stack size to see the smallest stack headroom measured since boot.

Trigger a crash and find the source line

The demo's /api/crash handler deliberately dereferences a null pointer. Keep that endpoint on the test board only.

curl http://DEVICE_IP/api/crash

After the board reboots and reconnects, copy the suggested command from mDash. Replace firmware.elf with the ELF from the build that crashed and run the command where the ESP-IDF tools are installed. In the video, that is inside the build container.

riscv32-esp-elf-addr2line -e build/mongoose-esp32-minimal.elf -pfCs <PC> <RA>

Substitute the hexadecimal program counter and return address from the report for <PC> and <RA>. The PC maps to the fault location. For a full call stack, download the crash core from mDash and analyse it with the matching ELF using ESP-IDF's core dump tools.

Why are there no usable crash addresses?

A reset reason can arrive without a valid crash summary. A watchdog reset, for example, may leave no addresses to decode. Check the serial boot log, verify that core dumps to flash are enabled, and confirm that the firmware has a core dump partition.