ESP32-S3 Stucked in 'Starting app cpu', 'call_start_cpu1 at xxx' when Running TVM Project Generated by ESP-DL Tools (IDFGH-12803) (AIV-732)

[Sinkratee]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.1.2-release

Espressif SoC revision.

ESP32-S3 (revision v0.1)

Operating System used.

Linux

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32-S3-WROOM-1 development board

Power Supply used.

USB

What is the expected behavior?

I am developing a project based on esp-dl lib, using tvm to auto generate an nn model and deploy on my esp32s3 board, to avoid IRAM overflow, I use PSRAM to restore model's global workspace and it's output.
in /project/components/tvm_model/model/codegen/host/src/default_lib0.c
__attribute__((section(".bss.noinit.tvm"), aligned(16)))
static uint8_t global_workspace[294912];
I modified it as:
//__attribute__((section(".bss.noinit.tvm"), aligned(16)))
static EXT_RAM_BSS_ATTR uint8_t global_workspace[294912];

this progress is from a csdn blog and is proved to be run successfuly by the writer. I followed all the steps in this blog(including add esp_attr.h head file and modify the output_data.h and allow .bss segment placed in external memory )

but after flash the device, it seems to stuck at starting app cpu.

What is the actual behavior?

Steps to reproduce.

1. use tvm to generate a project follow the authority doc by esp-dl
2. add code add_compile_options(-w) in CMakeLists.txt to avoid warning, as the compile would stopped by some warnings.
3. modify the model c code as in some src will error as it has two same input parameter, like:
   
   I deleted the second parameter in these functions, all the functions I modified in this step has nothing in it, just return 0.
4. move the model and output to psram as the blog did
5. modify the partitions.csv:
   
6. modify the Offset of partition table to 0x10000.
7. build, flash, and monitor.

Debug Logs.

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3818,len:0x19a4
load:0x403c9700,len:0x4
load:0x403c9704,len:0xd48
load:0x403cc700,len:0x3288
entry 0x403c9994
I (27) boot: ESP-IDF v5.1.2-dirty 2nd stage bootloader
I (27) boot: compile time May 11 2024 19:57:52
I (27) boot: Multicore bootloader
I (30) boot: chip revision: v0.1
I (34) qio_mode: Enabling default flash chip QIO
I (39) boot.esp32s3: Boot SPI Speed : 80MHz
I (44) boot.esp32s3: SPI Mode       : QIO
I (49) boot.esp32s3: SPI Flash Size : 16MB
I (54) boot: Enabling RNG early entropy source...
I (59) boot: Partition Table:
I (63) boot: ## Label            Usage          Type ST Offset   Length
I (70) boot:  0 factory          factory app      00 00 00020000 00659000
I (78) boot:  1 nvs              WiFi data        01 02 00679000 00004000
I (85) boot:  2 fr               Unknown data     01 06 0067d000 00020000
I (93) boot: End of partition table
I (97) esp_image: segment 0: paddr=00020020 vaddr=3c070020 size=33e798h (3401624) map
I (578) esp_image: segment 1: paddr=0035e7c0 vaddr=3fc92a00 size=01858h (  6232) load
I (579) esp_image: segment 2: paddr=00360020 vaddr=42000020 size=6f2c8h (455368) map
I (647) esp_image: segment 3: paddr=003cf2f0 vaddr=3fc94258 size=01484h (  5252) load
I (648) esp_image: segment 4: paddr=003d077c vaddr=40374000 size=0e8f4h ( 59636) load
I (669) boot: Loaded app from partition at offset 0x20000
I (669) boot: Disabling RNG early entropy source...
I (669) cpu_start: Multicore app
I (673) octal_psram: vendor id    : 0x0d (AP)
I (678) octal_psram: dev id       : 0x02 (generation 3)
I (684) octal_psram: density      : 0x03 (64 Mbit)
I (689) octal_psram: good-die     : 0x01 (Pass)
I (694) octal_psram: Latency      : 0x01 (Fixed)
I (700) octal_psram: VCC          : 0x01 (3V)
I (705) octal_psram: SRF          : 0x01 (Fast Refresh)
I (711) octal_psram: BurstType    : 0x01 (Hybrid Wrap)
I (717) octal_psram: BurstLen     : 0x01 (32 Byte)
I (722) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)
I (728) octal_psram: DriveStrength: 0x00 (1/1)
I (734) MSPI Timing: PSRAM timing tuning index: 5
I (739) esp_psram: Found 8MB PSRAM device
I (743) esp_psram: Speed: 80MHz
I (747) cpu_start: Pro cpu up.
I (751) cpu_start: Starting app cpu, entry point is 0x40375430
0x40375430: call_start_cpu1 at /home/zyt/esp/esp-idf/components/esp_system/port/cpu_start.c:157

More Information.

Not only one development board will occur this problem, I tested two boards from different vendor, which perform the same.
So this may not cause by a hardware fail.

I found a similar issue on esp32 forum but it seems nobody answer.

[BlueSkyB]

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware.
CONFIG_ESPTOOLPY_NO_STUB=y
CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

[Sinkratee]

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

Lots of Thanks for your advice, I will try it later. However I found a more efficient way to deploy the model by using ncnn, and it runs pretty great on esp32!

[zorosmith]

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

hello, I meet the same issue.I use your advice, however, I meet the same error.
Could you offer more idea to solve the problem?

[zorosmith]

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

Lots of Thanks for your advice, I will try it later. However I found a more eff

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

Lots of Thanks for your advice, I will try it later. However I found a more efficient way to deploy the model by using ncnn, and it runs pretty great on esp32!

hello. Do you solve this issue?
Could you offer more information about your solution?

[BlueSkyB]

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

Lots of Thanks for your advice, I will try it later. However I found a more eff

You can first try deleting the following configurations in the sdkconfig.defaults.esp32s3 file, then delete the previous sdkconfig, generate a new one, and recompile and flash the firmware. CONFIG_ESPTOOLPY_NO_STUB=y CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y

Lots of Thanks for your advice, I will try it later. However I found a more efficient way to deploy the model by using ncnn, and it runs pretty great on esp32!

hello. Do you solve this issue? Could you offer more information about your solution?

Hello, due to the previous TVM method being somewhat immature, it has been suspended from maintenance and iteration.
We have recently restructured esp-dl, making significant improvements, enhancing the convenience of deployment, and resolving some of the issues that existed before. You can try the latest deployment method of esp-dl from the master branch: https://docs.espressif.com/projects/esp-dl/en/latest/how_to_load_model.html