Greater module not supported even though it is listed in the repo (AIV-742)

[morenzoe]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

Quantized model deployed and all layer infos printed.

Actual behavior (suspected bug)

Greater ops not supported.

Error logs or terminal output

\moren\SNN\06_01_2025\esp-dl-master\examples\snn_simple> $env:IDF_PATH = 'C:\Users\Asus\esp\esp-idf';
PS D:\moren\SNN\06_01_2025\esp-dl-master\examples\snn_simple>  & 'C:\Users\Asus\.espressif\python_env\idf5.3_py3.11_env\Scripts\python.exe' 'C:\Users\Asus\esp\esp-idf\tools\idf_monitor.py' -p COM10 -b 115200 --toolchain-prefix xtensa-esp32s3-elf- --target esp32s3 'd:\moren\SNN\06_01_2025\esp-dl-master\examples\snn_simple\build\mobilenet_v2_example.elf'
--- Warning: GDB cannot open serial ports accessed as COMx
--- Using \\.\COM10 instead...
--- esp-idf-monitor 1.5.0 on \\.\COM10 115200
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce2820,len:0x568
load:0x403c8700,len:0x4
load:0x403c8704,len:0xc14
load:0x403cb700,len:0x2bf4
entry 0x403c8904
I (142) octal_psram: vendor id    : 0x0d (AP)
I (142) octal_psram: dev id       : 0x02 (generation 3)
I (143) octal_psram: density      : 0x03 (64 Mbit)
I (148) octal_psram: good-die     : 0x01 (Pass)
I (153) octal_psram: Latency      : 0x01 (Fixed)
I (158) octal_psram: VCC          : 0x01 (3V)
I (163) octal_psram: SRF          : 0x01 (Fast Refresh)
I (169) octal_psram: BurstType    : 0x01 (Hybrid Wrap)
I (175) octal_psram: BurstLen     : 0x01 (32 Byte)
I (180) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)
I (186) octal_psram: DriveStrength: 0x00 (1/1)
I (192) MSPI Timing: PSRAM timing tuning index: 4
I (197) esp_psram: Found 8MB PSRAM device
I (202) esp_psram: Speed: 80MHz
I (206) cpu_start: Multicore app
I (221) cpu_start: Pro cpu start user code
I (221) cpu_start: cpu freq: 240000000 Hz
I (221) app_init: Application information:
I (224) app_init: Project name:     mobilenet_v2_example
I (230) app_init: App version:      1
I (234) app_init: Compile time:     Jan  9 2025 10:40:06
I (240) app_init: ELF file SHA256:  64f7f6775...
I (246) app_init: ESP-IDF:          v5.3.1
I (250) efuse_init: Min chip rev:     v0.0
I (255) efuse_init: Max chip rev:     v0.99 
I (260) efuse_init: Chip rev:         v0.2
I (265) heap_init: Initializing. RAM available for dynamic allocation:
I (272) heap_init: At 3FC9CC60 len 0004CAB0 (306 KiB): RAM
I (278) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (285) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator
I (293) spi_flash: detected chip: generic
I (297) spi_flash: flash io: qio
I (301) sleep: Configure to isolate all GPIO pins in sleep state
I (307) sleep: Enable automatic switching of GPIO sleep configuration
I (315) main_task: Started on CPU0
I (335) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (335) main_task: Calling app_main()
I (335) SNN_SIMPLE: get into app_main
I (345) FbsLoader: The storage free size is 23872 KB
I (345) FbsLoader: The partition size is 7900 KB
I (355) dl::Model: model:main_graph, version:0

I (355) dl::Model: /1/Gemm: Gemm
I (395) dl::Model: /2/Sub_1: Sub
I (395) dl::Model: PPQ_Operation_1: DequantizeLinear
I (395) dl::Model: /2/Greater: Greater
E (395) dl::Model: Do not support Greater, please implement and register it first.
I (405) SNN_SIMPLE: exit app_main
I (405) main_task: Returned from app_main()

Steps to reproduce the behavior

Model Quantization

1. Clone esp-dl github
2. Make venv in VS Code using Python 3.10.6
3. Install esp-ppq, onnxruntime, pandas, and torchvision
4. Open quantize_onnx_model.py
5. Change INPUT_SHAPE to [784]
6. Change ONNX_PATH and ESPDL_MODLE_PATH
7. Comment out line 92-102 about downloading model and dataset
8. Change quant_setting_mobilenet_v2 input 1 to None
9. Comment out line 177-183 about evaluating quantized model
10. Run python file

Model Deployment

1. Open movilenet_v2 folder in examples
2. Replace the model files (.onnx, .espdl, .json, .info)
3. Change path in sdkconfig and sdkconfig.defaults.esp32s3
4. Comment out line 104-115 in app_main.cpp
5. Select port COM10
6. Select device target esp32s3 via built-in USB-JTAG
7. Select flash method UART
8. Run Build, Flash, and Monitor

Project release version

latest

System architecture

Intel/AMD 64-bit (modern PC, older Mac)

Operating system

Windows

Operating system version

Windows 10

Shell

PowerShell

Additional context

Hi, I am using VS Code with ESP-IDF extension v1.6.5 and ESP-IDF v5.3.1. I am deploying simple quantized snnTorch model from an .onnx file. The Greater ops is not supported even though it is in the ".\esp-dl\dl\module\include" folder. I have uploaded the ESP-IDF Project folder with the model files in GitHub repo https://github.com/morenzoe/esp-dl-snn-simple. The folder should be placed in the esp-dl GitHub repo folder because it is using esp-dl folder as component in idf_component.yml file.

Is there a bug in the Greater ops module? Is the implementation of Greater ops in the model wrong? Did I do anything wrong in the quantization? Thank you in advance!

[sun-xiangyu]

We have not yet tested the greater operator, so this operator has not been registered in dl_module_creator.hpp.

[morenzoe]

Hi @sun-xiangyu thank you for your fast response. Is there any plan on testing and registering the Greater ops soon? If not, do you have any pointer on how to try and register it myself? Thanks!

[sun-xiangyu]

Please try to register Greater by adding the following code in dl_module_creator.hpp

#include "dl_module_greater.hpp"

this->register_module("Greater", Greater::deserialize);

[morenzoe]

In ".\esp-dl\dl\module\include\dl_module_creator.hpp", I have added #include "dl_module_greater.hpp" to line 13 and this->register_module("Greater", Greater::deserialize); to line 116 (in void register_dl_modules()). However, another error came out as listed bellow.

Rebooting...
���ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40379b0c
--- 0x40379b0c: esp_restart_noos at C:/Users/Asus/esp/esp-idf/components/esp_system/port/soc/esp32s3/system_internal.c:158

SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce2820,len:0x568
load:0x403c8700,len:0x4
load:0x403c8704,len:0xc14
load:0x403cb700,len:0x2bf4
entry 0x403c8904
I (147) octal_psram: vendor id    : 0x0d (AP)
I (147) octal_psram: dev id       : 0x02 (generation 3)
I (148) octal_psram: density      : 0x03 (64 Mbit)
I (152) octal_psram: good-die     : 0x01 (Pass)
I (158) octal_psram: Latency      : 0x01 (Fixed)
I (163) octal_psram: VCC          : 0x01 (3V)
I (168) octal_psram: SRF          : 0x01 (Fast Refresh)
I (174) octal_psram: BurstType    : 0x01 (Hybrid Wrap)
I (180) octal_psram: BurstLen     : 0x01 (32 Byte)
I (185) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)
I (191) octal_psram: DriveStrength: 0x00 (1/1)
I (197) MSPI Timing: PSRAM timing tuning index: 4
I (202) esp_psram: Found 8MB PSRAM device
I (206) esp_psram: Speed: 80MHz
I (210) cpu_start: Multicore app
I (226) cpu_start: Pro cpu start user code
I (226) cpu_start: cpu freq: 240000000 Hz
I (226) app_init: Application information:
I (229) app_init: Project name:     mobilenet_v2_example
I (235) app_init: App version:      1
I (239) app_init: Compile time:     Jan 10 2025 08:30:17
I (245) app_init: ELF file SHA256:  ac966cc3d...
I (251) app_init: ESP-IDF:          v5.3.1
I (255) efuse_init: Min chip rev:     v0.0
I (260) efuse_init: Max chip rev:     v0.99 
I (265) efuse_init: Chip rev:         v0.2
I (270) heap_init: Initializing. RAM available for dynamic allocation:
I (277) heap_init: At 3FC9CC60 len 0004CAB0 (306 KiB): RAM
I (283) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (289) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator
I (297) spi_flash: detected chip: generic
I (301) spi_flash: flash io: qio
I (306) sleep: Configure to isolate all GPIO pins in sleep state
I (312) sleep: Enable automatic switching of GPIO sleep configuration
I (320) main_task: Started on CPU0
I (340) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations       
I (340) main_task: Calling app_main()
I (340) SNN_SIMPLE_09: get into app_main
I (350) FbsLoader: The storage free size is 23808 KB
I (350) FbsLoader: The partition size is 3900 KB
I (360) dl::Model: model:main_graph, version:0

I (360) dl::Model: /1/Gemm: Gemm
W (370) calloc_aligned: heap_caps_aligned_calloc failed, retry with MALLOC_CAP_8BIT
E (380) calloc_aligned: Fail to malloc -35287936 bytes from DRAM(313715 bytyes) and PSRAM(8385720 bytes), PSRAM is on.

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x4037c506  PS      : 0x00060f30  A0      : 0x8202e2f8  A1      : 0x3fc9f1f0
--- 0x4037c506: dl_tie728_memcpy at ??:?

A2      : 0x00000000  A3      : 0x3c8ca510  A4      : 0xfde58c80  A5      : 0x3fc9f210
A6      : 0x3fc9f1f0  A7      : 0x0000000c  A8      : 0x82090844  A9      : 0x3fc9f180
A10     : 0x00000083  A11     : 0x00000010  A12     : 0x3fc9f1f0  A13     : 0x00000000
A14     : 0x00000001  A15     : 0x00000010  SAR     : 0x00000004  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000000  LBEG    : 0x400556d5  LEND    : 0x400556e5  LCOUNT  : 0xffffffff
--- 0x400556d5: strlen in ROM
0x400556e5: strlen in ROM



Backtrace: 0x4037c503:0x3fc9f1f0 0x4202e2f5:0x3fc9f210 0x4200b05d:0x3fc9f230 0x420620c5:0x3fc9f280 0x4202112f:0x3fc9f370 0x4201c35c:0x3fc9f3d0 0x4202ca7a:0x3fc9f410 0x4202cce6:0x3fc9f500 0x42009857:0x3fc9f530 0x4208fb03:0x3fc9f560 0x40380dd9:0x3fc9f580
--- 0x4037c503: dl_tie728_memcpy at ??:?
0x4202e2f5: dl::tool::copy_memory(void*, void*, unsigned int) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/tool/src/dl_tool.cpp:152
0x4200b05d: dl::TensorBase::TensorBase(std::vector<int, std::allocator<int> >, void const*, int, dl::dtype_t, bool, unsigned long) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/tensor/src/dl_tensor_base.cpp:166 (discriminator 1)
0x420620c5: fbs::FbsModel::get_operation_parameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, unsigned long) at /home/fanshenwei/Downloads/tmp/1206/esp-dl/esp-dl/fbs_loader/src/fbs_model.cpp:510 (discriminator 3)
0x4202112f: dl::module::Gemm::deserialize(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/module/include/dl_module_gemm.hpp:158 (discriminator 1)
0x4201c35c: dl::module::Module* std::__invoke_impl<dl::module::Module*, dl::module::Module* (*&)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__invoke_other, dl::module::Module* (*&)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), fbs::FbsModel*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) at C:/Users/Asus/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/xtensa-esp-elf/include/c++/13.2.0/bits/invoke.h:61 (discriminator 1)
 (inlined by) std::enable_if<is_invocable_r_v<dl::module::Module*, dl::module::Module* (*&)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), 
fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, dl::module::Module*>::type std::__invoke_r<dl::module::Module*, dl::module::Module* (*&)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> 
> >(dl::module::Module* (*&)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), fbs::FbsModel*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) at C:/Users/Asus/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/xtensa-esp-elf/include/c++/13.2.0/bits/invoke.h:114 (discriminator 1)
 (inlined by) std::_Function_handler<dl::module::Module* (fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), dl::module::Module* (*)(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::_M_invoke(std::_Any_data const&, fbs::FbsModel*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) at C:/Users/Asus/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/xtensa-esp-elf/include/c++/13.2.0/bits/std_function.h:290 (discriminator 
1)
0x4202ca7a: std::function<dl::module::Module* (fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::operator()(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) const at C:/Users/Asus/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/xtensa-esp-elf/include/c++/13.2.0/bits/std_function.h:591
 (inlined by) dl::module::ModuleCreator::create(fbs::FbsModel*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/module/include/dl_module_creator.hpp:87
 (inlined by) dl::Model::load(fbs::FbsModel*) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/model/src/dl_model_base.cpp:135
0x4202cce6: dl::Model::load(char const*, fbs::model_location_type_t, unsigned char*, bool) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/model/src/dl_model_base.cpp:81 (discriminator 1) (inlined by) dl::Model::Model(char const*, fbs::model_location_type_t, int, dl::memory_manager_t, unsigned char*, bool) at D:/moren/SNN/09_01_2025/esp-dl-master/esp-dl/dl/model/src/dl_model_base.cpp:19 (discriminator 1)
0x42009857: app_main at D:/moren/SNN/09_01_2025/esp-dl-master/examples/snn_simple/main/app_main.cpp:103 (discriminator 1)
0x4208fb03: main_task at C:/Users/Asus/esp/esp-idf/components/freertos/app_startup.c:208
0x40380dd9: vPortTaskWrapper at C:/Users/Asus/esp/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:134





ELF file SHA256: ac966cc3d

Rebooting...

I have tried changing the heap_caps_aligned_calloc input to MALLOC_CAP_8BIT in ".\main\app_main.cpp", ".\managed_components\espressif__esp_jpeg\examples\get_started\main\lcd_tjpgd_example_main.c" and ".\managed_components\espressif__esp_jpeg\jpeg_decoder.c", yet the same error persists and the board keeps rebooting. The memory allocation in the error log is weird, was it caused by the Greater ops?

Would be grateful to get another help on this!

[sun-xiangyu]

I looked at your open-source model files, the Greater operator is not quantized. Currently, esp-dl does not support floating-point operators, which is likely the cause of the error.
We also need to add Greater op in esp-ppq.

[sun-xiangyu]

We will support Greater op as soon as possible.

[morenzoe]

I see, thank you very much for reviewing my code! Looking forward to the update 😁