[Bug]: Task watchdog got triggered FromRadio=STATE_SEND_PACKETS on ai-c3

[Styne13]

Category

Hardware Compatibility

Hardware

DIY

Is this bug report about any UI component firmware like InkHUD or Meshtatic UI (MUI)?

• Meshtastic UI aka MUI colorTFT
• InkHUD ePaper
• OLED slide UI on any display

Firmware Version

v2.6.8.ef9d0d7

Description

On a ai-c3 board variant with RFM95 module connected as defined in variant.h, but #define LORA_RESET RADIOLIB_NC as only difference, the watchdog is triggered if a package is send out.

This might be a bug or please tell me if I've configured something wrong or forgot some essential settings.

If you tell me how to debug further, I can also add more logs.

BR

Marcel

Relevant log output

[...]
DEBUG | 13:11:57 30 [NodeInfo] enqueue for send (id=0xcd0eb9b6 fr=0x27adfb74 to=0xffffffff, WantAck=0, HopLim=5 Ch=0x8 encrypted len=102 rxtime=1748171516 hopStart=5 relay=0x74 priority=10)
DEBUG | 13:11:57 30 [NodeInfo] txGood=0,txRelay=0,rxGood=0,rxBad=0
INFO  | 13:11:57 30 Tell client we have new packets 4
DEBUG | 13:11:57 30 BLE notify fromNum
DEBUG | 13:11:57 30 FromRadio=STATE_SEND_PACKETS
E (121223) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (121223) task_wdt:  - loopTask (CPU 0)
E (121223) task_wdt: Tasks currently running:
E (121223) task_wdt: CPU 0: loopTask
E (121223) task_wdt: Aborting.

abort() was called at PC 0x4208137d on core 0

=> 0x4208137d: task_wdt_isr at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/task_wdt.c:176 (discriminator 3)
Core  0 register dump:
MEPC    : 0x40382ae6  RA      : 0x4038bad0  SP      : 0x3fca08a0  GP      : 0x3fc93c00  

=> 0x40382ae6: panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:408

=> 0x4038bad0: __ubsan_include at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/ubsan.c:294
TP      : 0x3fc66acc  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130  
S0/FP   : 0x3fca08cc  S1      : 0x3fca08cc  A0      : 0x3fca08d8  A1      : 0x3fca08ba  
A2      : 0x00000000  A3      : 0x3fca0905  A4      : 0x00000001  A5      : 0x3fca6000  
A6      : 0x7a797877  A7      : 0x76757473  S2      : 0x3fca6404  S3      : 0x7fffffff  
S4      : 0x3fc978fc  S5      : 0x3fc978ec  S6      : 0x3fc978f4  S7      : 0x3fc978ec  
S8      : 0x0000782e  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938  
MSTATUS : 0x00001801  MTVEC   : 0x40380001  MCAUSE  : 0x00000007  MTVAL   : 0x00000000  

=> 0x40380001: _vector_table at ??:?
MHARTID : 0x00000000  

Stack memory:
3fca08a0: 0x00000000 0x00000000 0x3fca08b8 0x40391c6e 0x3fc978f4 0x3fc978ec 0x3fc90030 0x3fc98d38

=> 0x40391c6e: __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:41
3fca08c0: 0x3fca08cc 0x3fc98d54 0x3fca08b8 0x38303234 0x64373331 0x00000000 0x726f6261 0x20292874
3fca08e0: 0x20736177 0x6c6c6163 0x61206465 0x43502074 0x34783020 0x31383032 0x20643733 0x63206e6f

=> 0x43502074: ?? ??:0
3fca0900: 0x2065726f 0x00000030 0x3c180000 0x800062a0 0x600c2000 0x3c18f000 0x3fca6000 0x42081380

=> 0x42081380: task_wdt_isr at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/task_wdt.c:157
3fca0920: 0x3fce0000 0x403849ba 0x3fce0000 0x0001d987 0x3fc99a44 0x00000013 0x3fc99a8c 0x3fc99ac4

=> 0x403849ba: semphr_give_from_isr_wrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/bt/controller/esp32c3/bt.c:574
3fca0940: 0x3fc99a44 0x00000013 0x3fc99a8c 0x00000001 0x00001881 0x80000009 0x3fcb6678 0x403801ee

=> 0x403801ee: _interrupt_handler at ??:?
3fca0960: 0x00000001 0x3fca0968 0xffffffff 0x3fcabed4 0x3fcabed4 0x00000001 0x3fca097c 0xffffffff
3fca0980: 0x3fcaee68 0x3fcaee68 0x00000000 0x3fca0990 0xffffffff 0x3fca0990 0x3fca0990 0x00000000
3fca09a0: 0x3fca09a4 0xffffffff 0x3fca09a4 0x3fca09a4 0x00000000 0x3fca09b8 0xffffffff 0x3fca09b8
3fca09c0: 0x3fca09b8 0x00000000 0x3fca09cc 0xffffffff 0x3fca09cc 0x3fca09cc 0x00000000 0x3fca09e0
3fca09e0: 0xffffffff 0x3fca09e0 0x3fca09e0 0x00000000 0x3fca09f4 0xffffffff 0x3fca09f4 0x3fca09f4
3fca0a00: 0x00000000 0x3fca0a08 0xffffffff 0x3fca0a08 0x3fca0a08 0x00000000 0x3fca0a1c 0xffffffff
3fca0a20: 0x3fca0a1c 0x3fca0a1c 0x00000000 0x3fca0a30 0xffffffff 0x3fca0a30 0x3fca0a30 0x00000000
3fca0a40: 0x3fca0a44 0xffffffff 0x3fca0a44 0x3fca0a44 0x00000000 0x3fca0a58 0xffffffff 0x3fca0a58
3fca0a60: 0x3fca0a58 0x00000000 0x3fca0a6c 0xffffffff 0x3fca0a6c 0x3fca0a6c 0x00000000 0x3fca0a80
3fca0a80: 0xffffffff 0x3fca0a80 0x3fca0a80 0x00000000 0x3fca0a94 0xffffffff 0x3fca0a94 0x3fca0a94
3fca0aa0: 0x00000000 0x3fca0aa8 0xffffffff 0x3fca0aa8 0x3fca0aa8 0x00000000 0x3fca0abc 0xffffffff
3fca0ac0: 0x3fca0abc 0x3fca0abc 0x00000000 0x3fca0ad0 0xffffffff 0x3fca0ad0 0x3fca0ad0 0x00000000
3fca0ae0: 0x3fca0ae4 0xffffffff 0x3fca0ae4 0x3fca0ae4 0x00000000 0x3fca0af8 0xffffffff 0x3fca0af8
3fca0b00: 0x3fca0af8 0x00000000 0x3fca0b0c 0xffffffff 0x3fca0b0c 0x3fca0b0c 0x00000000 0x3fca0b20
3fca0b20: 0xffffffff 0x3fca0b20 0x3fca0b20 0x00000000 0x3fca0b34 0xffffffff 0x3fca0b34 0x3fca0b34
3fca0b40: 0x00000000 0x3fca0b48 0xffffffff 0x3fca0b48 0x3fca0b48 0x00000000 0x3fca0b5c 0xffffffff
3fca0b60: 0x3fca0b5c 0x3fca0b5c 0x00000000 0x3fca0b70 0xffffffff 0x3fca0b70 0x3fca0b70 0x00000000
3fca0b80: 0x3fca0b84 0xffffffff 0x3fca0b84 0x3fca0b84 0x00000004 0x3fca0b98 0xffffffff 0x3fca9e64
3fca0ba0: 0x3fcbe1b0 0x00000000 0x3fca0bac 0xffffffff 0x3fca0bac 0x3fca0bac 0x00000002 0x000038f2
3fca0bc0: 0x3fcc2c10 0x4038cc06 0x00000003 0x00000000 0x3fcc2c10 0x4038cc06 0x00000004 0x000003e8

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724
3fca0be0: 0x3fcc2c10 0x4038cc06 0x00000002 0x000076d4 0x3fcc2c10 0x4038cc06 0x00000003 0x00000000

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724
3fca0c00: 0x3fcc2c10 0x4200052a 0x00000003 0x00000000 0x3fcc2c10 0x4038cc06 0x00000004 0x000003e8

=> 0x4200052a: npl_freertos_mutex_pend.constprop.1137 at ??:?

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724
3fca0c20: 0x3fcc2c10 0x4038cc06 0x00000002 0x00003119 0x3fcc2c10 0x4038cc06 0x00000003 0x00000000

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724
3fca0c40: 0x3fcc2c10 0x4038cc06 0x00000004 0x000003e8 0x3fcc2c10 0x4038cc06 0x00000000 0x3fca0c60

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724

=> 0x4038cc06: xQueueTakeMutexRecursive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:724
3fca0c60: 0xffffffff 0x3fca0c60 0x3fca0c60 0x00000000 0x3fca0c74 0xffffffff 0x3fca0c74 0x3fca0c74
3fca0c80: 0x3fca0bb8 0x3fca0bf8 0x3fca0c58 0x3fca0be8 0x00000000 0x3fca0c98 0xffffffff 0x3fca0c98



ELF file SHA256: 379e2f7a7be5ab0a

E (11121) esp_core_dump_flash: Core dump flash config is corrupted! CRC=0x7bd5c66f instead of 0x0
Rebooting...
ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0xf (SPI_FAST_FLASH_BOOT)
Saved PC:0x403826e8

=> 0x403826e8: esp_restart_noos_dig at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/esp_system.c:53 (discriminator 1)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5810,len:0x438
load:0x403cc710,len:0x90c

=> 0x403cc710: ?? ??:0
load:0x403ce710,len:0x2624

=> 0x403ce710: ?? ??:0
entry 0x403cc710

=> 0x403cc710: ?? ??:0
E (322) esp_core_dump_flash: No core dump partition found!
E (322) esp_core_dump_flash: No core dump partition found!
��@INFO  | ??:??:?? 0 

//\ E S H T /\ S T / C

INFO  | ??:??:?? 0 Booted, wake cause 0 (boot count 1), reset_reason=reset
DEBUG | ??:??:?? 0 Filesystem files (16384/1048576 Bytes):
DEBUG | ??:??:?? 0  /prefs/channels.proto (57 Bytes)
DEBUG | ??:??:?? 0  /prefs/config.proto (215 Bytes)
DEBUG | ??:??:?? 0  /prefs/device.proto (116 Bytes)
DEBUG | ??:??:?? 0  /prefs/module.proto (96 Bytes)
DEBUG | ??:??:?? 0  /prefs/nodes.proto (277 Bytes)
[   381][I][esp32-hal-i2c.c:75] i2cInit(): Initialising I2C Master: sda=8 scl=9 freq=100000
DEBUG | ??:??:?? 0 Power::lipoInit lipo sensor is not ready yet
INFO  | ??:??:?? 0 Scan for i2c devices
DEBUG | ??:??:?? 0 Scan for I2C devices on port 1
INFO  | ??:??:?? 0 No I2C devices found
DEBUG | ??:??:?? 0 acc_info = 0
INFO  | ??:??:?? 0 S:B:53,2.6.8.6bba17d46
DEBUG | ??:??:?? 0 Total heap: 232260
DEBUG | ??:??:?? 0 Free heap: 200932
DEBUG | ??:??:?? 0 Total PSRAM: 0
DEBUG | ??:??:?? 0 Free PSRAM: 0
DEBUG | ??:??:?? 0 NVS: UsedEntries 98, FreeEntries 532, AllEntries 630, NameSpaces 5
DEBUG | ??:??:?? 0 Setup Preferences in Flash Storage
DEBUG | ??:??:?? 0 Number of Device Reboots: 25
ESP_ERROR_CHECK_WITHOUT_ABORT failed: esp_err_t 0x105 (ESP_ERR_NOT_FOUND) at 0x420414c7

=> 0x420414c7: BleOta::findEspOtaAppPartition() at ??:?
file: "src/platform/esp32/BleOta.cpp" line 16
func: static const esp_partition_t* BleOta::findEspOtaAppPartition()
expression: esp_ota_get_partition_description(part, &app_desc)
ESP_ERROR_CHECK_WITHOUT_ABORT failed: esp_err_t 0x102 (ESP_ERR_INVALID_ARG) at 0x4204425b

=> 0x4204425b: BleOta::getOtaAppVersion() at ??:?
file: "src/platform/esp32/BleOta.cpp" line 30
func: static String BleOta::getOtaAppVersion()
expression: esp_ota_get_partition_description(part, &app_desc)
INFO  | ??:??:?? 0 No BLE OTA firmware available
INFO  | ??:??:?? 0 No WiFi OTA firmware available
INFO  | ??:??:?? 0 Init NodeDB
[   456][E][vfs_api.cpp:105] open(): /littlefs/static/static does not exist, no permits for creation
[   475][E][vfs_api.cpp:105] open(): /littlefs/prefs/db.proto does not exist, no permits for creation
INFO  | ??:??:?? 0 Load /prefs/nodes.proto
INFO  | ??:??:?? 0 Loaded /prefs/nodes.proto successfully
INFO  | ??:??:?? 0 Loaded saved nodedatabase version 24, with nodes count: 1
INFO  | ??:??:?? 0 Load /prefs/device.proto
INFO  | ??:??:?? 0 Loaded /prefs/device.proto successfully
INFO  | ??:??:?? 0 Loaded saved devicestate version 24
INFO  | ??:??:?? 0 Load /prefs/config.proto
INFO  | ??:??:?? 0 Loaded /prefs/config.proto successfully
INFO  | ??:??:?? 0 Loaded saved config version 24
INFO  | ??:??:?? 0 Load /prefs/module.proto
INFO  | ??:??:?? 0 Loaded /prefs/module.proto successfully
INFO  | ??:??:?? 0 Loaded saved moduleConfig version 24
INFO  | ??:??:?? 0 Load /prefs/channels.proto
INFO  | ??:??:?? 0 Loaded /prefs/channels.proto successfully
INFO  | ??:??:?? 0 Loaded saved channelFile version 24
[   565][E][vfs_api.cpp:105] open(): /littlefs/prefs/uiconfig.proto does not exist, no permits for creation
ERROR | ??:??:?? 0 Could not open / read /prefs/uiconfig.proto
DEBUG | ??:??:?? 0 cleanupMeshDB purged 0 entries
DEBUG | ??:??:?? 0 Use nodenum 0x27adfb74 
DEBUG | ??:??:?? 0 Number of Device Reboots: 25
DEBUG | ??:??:?? 0 Expand short PSK #1
INFO  | ??:??:?? 0 Wanted region 3, using EU_868
DEBUG | ??:??:?? 0 Coerce telemetry to min of 30 minutes on defaults
DEBUG | ??:??:?? 0 Save to disk 4
DEBUG | ??:??:?? 0 Opening /prefs/device.proto, fullAtomic=1
INFO  | ??:??:?? 0 Save /prefs/device.proto
DEBUG | ??:??:?? 0 Use GPIO09 for button
DEBUG | ??:??:?? 0 SPI.begin(SCK=4, MISO=5, MOSI=6, NSS=7)
DEBUG | ??:??:?? 0 Use compiled/slipstreamed tzplaceholder                                         
DEBUG | ??:??:?? 0 Saved TZ: MEZ-1MESZ,M3.5.0/2:00:00,M10.4.0/3:00:00 
DEBUG | ??:??:?? 0 Set Timezone to MEZ-1MESZ,M3.5.0/2:00:00,M10.4.0/3:00:00
DEBUG | ??:??:?? 0 Read RTC time as 1748171608
DEBUG | ??:??:?? 0 Start multicast thread
DEBUG | ??:??:?? 0 NeighborInfoModule is disabled
DEBUG | ??:??:?? 0 Rescan for I2C keyboard
DEBUG | ??:??:?? 0 Scan for I2C devices on port 1
DEBUG | ??:??:?? 0 Scan address 0x1f
DEBUG | ??:??:?? 1 Scan address 0x34
DEBUG | ??:??:?? 2 Scan address 0x55
DEBUG | ??:??:?? 3 Scan address 0x5a
DEBUG | ??:??:?? 4 Scan address 0x5f
INFO  | ??:??:?? 5 External Notification Module Disabled
DEBUG | ??:??:?? 5 RF95Interface(cs=7, irq=10, rst=-1, busy=3)
INFO  | ??:??:?? 5 Start meshradio init
INFO  | ??:??:?? 5 Radio freq=869.525, config.lora.frequency_offset=0.000
INFO  | ??:??:?? 5 Set radio: region=EU_868, name=LongFast, config=0, ch=0, power=27
INFO  | ??:??:?? 5 myRegion->freqStart -> myRegion->freqEnd: 869.400024 -> 869.650024 (0.250000 MHz)
INFO  | ??:??:?? 5 numChannels: 1 x 250.000kHz
INFO  | ??:??:?? 5 channel_num: 1
INFO  | ??:??:?? 5 frequency: 869.525024
INFO  | ??:??:?? 5 Slot time: 28 msec
INFO  | ??:??:?? 5 Set radio: final power level=20
DEBUG | ??:??:?? 5 Current limit set to 100.000000
DEBUG | ??:??:?? 5 Current limit set result 0
INFO  | ??:??:?? 5 RF95 init result 0
INFO  | ??:??:?? 5 Frequency set to 869.525024
INFO  | ??:??:?? 5 Bandwidth set to 250.000000
INFO  | ??:??:?? 5 Power output set to 20
INFO  | ??:??:?? 5 RF95 init success
[...]

[Styne13]

Seems like this is related to RF95 hardware. Used the same setup with a SX1262 and had no problems with ESP32 C3.
So it looks like the ESP32 is waiting for something from the LoRa radio in a loop, which leads to a watchdog trigger.