Can't follow readme please help

[leenowell]

Hi,

I have my esp IDF stored in /opt/esp/esp-idf. So for step 1 from within /opt/esp I clone the repository and make it so end up with 2 new directories
/opt/esp/qemu_esp32
/opt/esp/qemu-xtensa-esp32

step 2 then puts the bin files in qemu_esp32

Step 3 - I am unclear what directory it assumes I am in to run cd esp. In my example does it assume I should be in /opt and therefore cd esp takes me to /opt/esp? If so, doesn't the clone conflict with step 1?

I have tried doing step 3 in ~ instead so it doesn't conflict but that doesn;t seem to work either.

I guess my challenge is I don;t quite understand what the steps are doing and therefore follow what I need to do - sorry.

Any help greatly appreciated.

thanks

Lee.

[Ebiroll]

Hello.
Sorry for being busy last week and not been able to be so helpful.
The example assumes that you are using your $HOME directory.
~=$HOME
Did you solve the Issue or do you need more help?

[leenowell]

Thanks for getting back to me. No unfortunately I couldn't get it up and running. Does it have to be installed in $HOME?

I think my issue is that I don't understand conceptually what each step is doing and how they interrelate. Especially the use of qemu_esp32 directory twice.

The last step I then don't understand is what I need to do to my existing project to get it to run and debug using this. E..g I was looking at $IDF_PATH/examples/protocols/coap_client and voap_server.

When I run QEMU it core dumps so suspect I have mixed up the bin files or something.

Sorry if I am being dozy

[leenowell]

Hi,

I have just retried the steps in the readme and ended up with
~/qemu_esp
~/qemu-xtensa-esp32
~/esp/qemu_esp (now noticed the esp step which solves my question about overwriting.).
My esp-idf is in /opt/esp/esp-idf

Having completed steps 1 to 6 above, when I run step 6 I get the following core dump

lee@leelaptop:~/qemu_esp32$ xtensa-softmmu/qemu-system-xtensa -d guest_errors,unimp  -cpu esp32 -M esp32 -m 4M  -s  > io.txt
esp32flash.bin @ 01800000
VNC server running on 127.0.0.1:5900
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:9036
load:0x40078000,len:12300
ho 0 tail 12 room 4
load:0x40080400,len:7120
entry 0x400807a0
D (12) bootloader_flash: mmu set block paddr=0x00000000 (was 0xffffffff)
I (4) boot: ESP-IDF v3.3-beta2-39-g539d85e-dirty 2nd stage bootloader
I (4) boot: compile time 07:18:08
D (4) boot: Enabling RTCWDT(9000 ms)
I (4) boot: Enabling RNG early entropy source...
D (4) boot: magic e9
D (4) boot: segments 04
D (4) boot: spi_mode 02
D (4) boot: spi_speed 00
D (4) boot: spi_size 01
I (4) boot: SPI Speed      : 40MHz
I (4) boot: SPI Mode       : DIO
I (4) boot: SPI Flash Size : 2MB
D (4) bootloader_flash: mmu set paddr=00000000 count=1 size=c00 src_addr=8000 src_addr_aligned=0
D (4) boot: mapped partition table 0x8000 at 0x3f408000
D (5) flash_parts: partition table verified, 4 entries
I (5) boot: Partition Table:
I (5) boot: ## Label            Usage          Type ST Offset   Length
D (5) boot: load partition table entry 0x3f408000
D (5) boot: type=1 subtype=2
I (5) boot:  0 nvs              WiFi data        01 02 00009000 00006000
D (5) boot: load partition table entry 0x3f408020
D (5) boot: type=1 subtype=1
I (5) boot:  1 phy_init         RF data          01 01 0000f000 00001000
D (5) boot: load partition table entry 0x3f408040
D (5) boot: type=0 subtype=0
I (5) boot:  2 factory          factory app      00 00 00010000 00100000
I (5) boot: End of partition table
D (5) boot: Trying partition index -1 offs 0x10000 size 0x100000
D (5) esp_image: reading image header @ 0x10000
D (5) bootloader_flash: mmu set block paddr=0x00010000 (was 0xffffffff)
D (5) esp_image: image header: 0xe9 0x06 0x02 0x01 40080ecc
V (5) esp_image: loading segment header 0 at offset 0x10018
V (5) esp_image: segment data length 0x153d0 data starts 0x10020
V (5) esp_image: segment 0 map_segment 1 segment_data_offs 0x10020 load_addr 0x3f400020
I (6) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x153d0 ( 86992) map
D (6) bootloader_flash: mmu set paddr=00010000 count=2 size=153d0 src_addr=10020 src_addr_aligned=10000
V (17) esp_image: loading segment header 1 at offset 0x253f0
D (17) bootloader_flash: mmu set block paddr=0x00020000 (was 0xffffffff)
V (17) esp_image: segment data length 0x2538 data starts 0x253f8
V (17) esp_image: segment 1 map_segment 0 segment_data_offs 0x253f8 load_addr 0x3ffb0000
I (17) esp_image: segment 1: paddr=0x000253f8 vaddr=0x3ffb0000 size=0x02538 (  9528) load
D (17) bootloader_flash: mmu set paddr=00020000 count=1 size=2538 src_addr=253f8 src_addr_aligned=20000
V (19) esp_image: loading segment header 2 at offset 0x27930
D (19) bootloader_flash: mmu set block paddr=0x00020000 (was 0xffffffff)
V (19) esp_image: segment data length 0x400 data starts 0x27938
V (19) esp_image: segment 2 map_segment 0 segment_data_offs 0x27938 load_addr 0x40080000
I (19) esp_image: segment 2: paddr=0x00027938 vaddr=0x40080000 size=0x00400 (  1024) load
D (19) bootloader_flash: mmu set paddr=00020000 count=1 size=400 src_addr=27938 src_addr_aligned=20000
V (19) esp_image: loading segment header 3 at offset 0x27d38
D (19) bootloader_flash: mmu set block paddr=0x00020000 (was 0xffffffff)
V (19) esp_image: segment data length 0x82d0 data starts 0x27d40
V (19) esp_image: segment 3 map_segment 0 segment_data_offs 0x27d40 load_addr 0x40080400
I (19) esp_image: segment 3: paddr=0x00027d40 vaddr=0x40080400 size=0x082d0 ( 33488) load
D (20) bootloader_flash: mmu set paddr=00020000 count=2 size=82d0 src_addr=27d40 src_addr_aligned=20000
V (25) esp_image: loading segment header 4 at offset 0x30010
D (25) bootloader_flash: mmu set block paddr=0x00030000 (was 0xffffffff)
V (25) esp_image: segment data length 0x68ef0 data starts 0x30018
V (25) esp_image: segment 4 map_segment 1 segment_data_offs 0x30018 load_addr 0x400d0018
I (25) esp_image: segment 4: paddr=0x00030018 vaddr=0x400d0018 size=0x68ef0 (429808) map
D (25) bootloader_flash: mmu set paddr=00030000 count=7 size=68ef0 src_addr=30018 src_addr_aligned=30000
V (79) esp_image: loading segment header 5 at offset 0x98f08
D (79) bootloader_flash: mmu set block paddr=0x00090000 (was 0xffffffff)
V (79) esp_image: segment data length 0x68c0 data starts 0x98f10
V (79) esp_image: segment 5 map_segment 0 segment_data_offs 0x98f10 load_addr 0x400886d0
I (79) esp_image: segment 5: paddr=0x00098f10 vaddr=0x400886d0 size=0x068c0 ( 26816) load
D (79) bootloader_flash: mmu set paddr=00090000 count=1 size=68c0 src_addr=98f10 src_addr_aligned=90000
V (83) esp_image: image start 0x00010000 end of last section 0x0009f7d0
D (83) bootloader_flash: mmu set block paddr=0x00090000 (was 0xffffffff)
D (83) esp_image: Calculated hash: a373ddb746ca7b863d1bd9ee3d9d544a526c8e217164fdc327b19ee434353087
D (83) bootloader_flash: mmu set paddr=00090000 count=1 size=20 src_addr=9f7e0 src_addr_aligned=90000
D (83) bootloader_flash: mmu set paddr=00090000 count=1 size=20 src_addr=9f7e0 src_addr_aligned=90000
I (87) boot: Loaded app from partition at offset 0x10000
I (87) boot: Disabling RNG early entropy source...
D (87) boot: Mapping segment 0 as DROM
D (87) boot: Mapping segment 4 as IROM
D (87) boot: calling set_cache_and_start_app
D (87) boot: configure drom and irom and start
V (87) boot: d mmu set paddr=00010000 vaddr=3f400000 size=86992 n=2
V (87) boot: rc=0
V (87) boot: rc=0
V (87) boot: i mmu set paddr=00030000 vaddr=400d0000 size=429808 n=7
V (87) boot: rc=0
V (87) boot: rc=0
D (87) boot: start: 0x40080ecc
I (87) cpu_start: Pro cpu up.
I (87) cpu_start: Application information:
I (87) cpu_start: Project name:     app-template
I (87) cpu_start: App version:      e5f0168-dirty
I (87) cpu_start: Compile time:     Mar 18 2019 07:18:13
I (87) cpu_start: ELF file SHA256:  f3700f6d0a6cc5a6...
I (87) cpu_start: ESP-IDF:          v3.3-beta2-39-g539d85e-dirty
I (87) cpu_start: Single core mode
I (88) heap_init: Initializing. RAM available for dynamic allocation:
I (88) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (88) heap_init: At 3FFB9380 len 00026C80 (155 KiB): DRAM
I (88) heap_init: At 3FFE0440 len 0001FBC0 (126 KiB): D/IRAM
I (88) heap_init: At 40078000 len 00008000 (32 KiB): IRAM
I (88) heap_init: At 4008EF90 len 00011070 (68 KiB): IRAM
I (88) cpu_start: Pro cpu start user code
esp32_i2c_interruptSet: new IRQ val 0x526d2cc0
TBD(pc = 4008e6af): /home/lee/qemu-xtensa-esp32/target-xtensa/translate.c:1409
I (240) cpu_start: Starting scheduler on PRO CPU.
NV_INIT_CALLED

EFUSE MAC ,3FF5A004=A449ACA8
EFUSE MAC ,3FF5A008=FFAA30AE
Running in qemu
ESP_ERROR_CHECK failed: esp_err_t 0x102 (ESP_ERR_INVALID_ARG) at 0x40086b38
file: "/opt/esp/esp-idf/components/lwip/port/esp32/vfs_lwip.c" line 73
func: esp_vfs_lwip_sockets_register
expression: esp_vfs_register_fd_range(&vfs, NULL, LWIP_SOCKET_OFFSET, MAX_FDS)

ELF file SHA256: f3700f6d0a6cc5a630b787b341321da27fc1b8b2714f7db42f2fba24c53086f3

Backtrace: 0x4008671c:0x3ffba750 0x40086b3b:0x3ffba770 0x40132051:0x3ffba790 0x4012d871:0x3ffba850 0x40121057:0x3ffba870 0x4011e63f:0x3ffba890 0x400e43a8:0x3ffba8b0 0x400d2d7f:0x3ffba8d0 0x400d1699:0x3ffba900 0x4008c539:0x3ffba920

Entering gdb stub now.

Any ideas?

[leenowell]

Sorry forgot to mention. In the readme it mentions to watch out or hard coded paths in the to flash.c file. I can see some commented out ones at the top but everything else looks like they are fine as relative to the directory in which it is installed. Have I missed something.

[Ebiroll]

It is working. Only the emulated network part seems to have an issue.
See also the readme file.
What version of esp-idf are you using?

Networking support

There exists networking support of an emulated opencore network device,
components/emul_ip/lwip_ethoc.c is the driver.
All files in the net/ direcory is just for reference, they are currently not used.

To get emulated network to work with 3.1 version of esp-idf you must patch esp-idf components/lwip/port/freertos/sys_arch.c
sys_init(void) ...
//esp_vfs_lwip_sockets_register();

[leenowell]

Thanks for getting back to me so quickly Looks like our messages crossed :).

I am using the latest version of IDF 3.3. Do I still need to apply the patch?

Assuming I get this working , how do I go about getting another project working ? I guess I am missing the link between what I run in step 6 and the compiled eso32 program (eg the elf file)

[leenowell]

Hi,

I have applied the patch and have made some progress. So far I have this....

1. In window 1 having executed step 6 I get a log with no apparent errors. The process waits with the last log lines saying

I (87) cpu_start: Pro cpu start user code
esp32_i2c_interruptSet: new IRQ val 0x62b9ac60
TBD(pc = 4008e6af): /home/lee/qemu-xtensa-esp32/target-xtensa/translate.c:1409
I (239) cpu_start: Starting scheduler on PRO CPU.
NV_INIT_CALLED

EFUSE MAC ,3FF5A004=A449ACA8
EFUSE MAC ,3FF5A008=FFAA30AE
Running in qemu
ethoc: num_tx: 8 num_rx: 8
TCP/IP initializing...
TCP/IP initialized.
Applications started.

GPIO STRAP REG=00000013
RTC_CNTL_RESET_STATE_REG,3FF48034=00003390
RTC_CNTL_INT_ST_REG; 3FF48044=00000000
0x6001c018 ; 6001C018=11E15054
0x60033c00; 60033C00=0002BBED
0x6001c018 ; 6001C018=11E15054
0x60033c00; 60033C00=0002BBED
0x6001c018 ; 6001C018=11E15054
0x60033c00; 60033C00=0002BBED
0x6001c018 ; 6001C018=11E15054
0x60033c00; 60033C00=0002BBED

Is this correct?

2. As far as I can make out, steps 3 to 5 builds a sample app - is this correct? Looking at the code for toflash.c seems to merge some bin files in the build sub-directories with the app-template.bin file (is the application bin file?) and ultimately copying the esp32flash.bin to the window1 directory where the rom.bin and rom1.bin files are stored. Have I got this correct?

3. From the app directory (i.e. window2 ~/esp/qemu_esp32/bin) I copied xtensa-esp32-elf-gdb to xtensa-esp32-elf-gdb.qemu in the same directory. I run `xtensa-esp32-elf-gdb.qemu ../build/app-template.elf -ex 'target remote:1234' and get the following log

lee@leelaptop:~/esp/qemu_esp32/bin$ ./xtensa-esp32-elf-gdb.qemu  ../build/app-template.elf -ex 'target remote:1234'
Python Exception <type 'exceptions.ImportError'> No module named gdb: 
./xtensa-esp32-elf-gdb.qemu: warning: 
Could not load the Python gdb module from `/home/olas/esp/crosstool-NG/builds/xtensa-esp32-elf/share/gdb/python'.
Limited Python support is available from the _gdb module.
Suggest passing --data-directory=/path/to/gdb/data-directory.

GNU gdb (crosstool-NG crosstool-ng-1.22.0-59-g8d95cad) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../build/app-template.elf...done.
:1234: Connection timed out.
(gdb) quit

There is no change to window 1.

Any thoughts on how I might get one of the ESP32 examples up and running with this?

Thanks in advance for your help

Lee.

[Ebiroll]

Yes. You are doing it correctly.
You only need to add -s -S options when starting qemu. Start qemu first then the debugger.

In the other window after the debugger is connected do
(gdb) b app_main
(gdb) c
When breakpoint is hit press, Ctrll-X and then o
Then
(gdb) s
then use return to repeat the step instruction.
I have not tested version 3.3 yet. Will test that today
GL

[leenowell]

Thanks. I do start QEMU first but from the last line of the debug log, it seems to say it fails to connect to 1234 so assumed that meant couldn't connect to the QEMU?

[Ebiroll]

yes. you probably forgot -s

[Ebiroll]

xtensa-softmmu/qemu-system-xtensa -d guest_errors,unimp -cpu esp32 -M esp32 -m 4M -s -S > io.txt

[Ebiroll]

OK. I just checked out v3.3 I noticed one difference that also could be important
ESPTOOL_ELF2IMAGE_OPTIONS += --elf-sha256-offset 0xb0
I will test this later.

[leenowell]

Ah yes I didn't add the -S so assume it just run to completion before I fired up the debugger? I had assumed the QEMU process would terminate. I will try again

Couple of follow up questions

1. Can I run more than one ESP32 at the same time but in different terminals?
2. With the network emulation, are they able to communicate with each other via an emulated wireless network? Also will they get an IP address and can another process on the laptop connect to it as a client ?

Thanks for all your help

Lee.

[leenowell]

Looks like our posts crossed. Assuming I can get this up and running later, if you would like me to run any tests my side please shout.

[Ebiroll]

Hello.
I guess you should be able to run more than one if you use different directories, else they will be using the same flash file. esp32flash.bin. You must also add -p PORT with different ports for the gdb option.
However the serial emulation will not work properly as they use the same ports 8881-8884.

[Ebiroll]

I tested wit v3.3, It seems to work OK.
xtensa-esp32-elf-gdb.qemu build/app-template.elf -ex 'target remote:1234'

Reading symbols from build/app-template.elf...done.
(gdb) add-symbol-file rom.elf 0x40000000
add symbol table from file "rom.elf" at
.text_addr = 0x40000000
Remote debugging using :1234
0x40000400 in _ResetVector ()
(gdb) b app_main
Breakpoint 1 at 0x400d2d94: file /home/olof/esp/qemu_esp32/main/main.c, line 612.
(gdb) c

[leenowell]

Thanks for getting back to me. In the new directory, do I need anything other than the esp32flash.bin file? So it sounds like I can debug them on different ports but wasn't sure of the implication of your statement about serial emulation. Does this mean that they should talk to each other via the wireless drivers but not if anything is going via the serial port (eg logging)? As an example, I am trying to get an app I have written using espnow and espmesh to work. If this could work I could simulate a much bigger network than I have physical devices.

[leenowell]

Thanks for doing the test. Assume it still needs the patch to work? What is the implication of commenting out the line? Worried it might cause me issues on the other projects I am working on

[Ebiroll]

No. I would not expect any of the wifi stuff to work. I have provided some networking support but it uses a virtual driver.
I tested the rom dump app (top dir),
I think you should start with a simpler example from the example dir.
The qemu_flash app creates the esp32flasj.bin file and the copies it to /home/name/qemu_esp32
(1) xtensa-softmmu/qemu-system-xtensa -d guest_errors,unimp -cpu esp32 -M esp32 -m 4M -s -S > io.txt
(2)xtensa-esp32-elf-gdb.qemu build/app-template.elf -ex 'target remote:1234'
You can start 2 qemu sessions but will have to figure out the netwoking and -p -gdb or whatever option is used to set the port for the gdb session.
However, I was able to run emulation on two cores. I think it is because nv_init() was never called.

[Ebiroll]

What is the implication of commenting out the line?
You should probably not keep it that way.
Add something like this in your program,
if (!is_running_qemu()) {
wifi_init();
}
Then you could test with both actual hardware and in qemu()

[leenowell]

Ok thanks. Will try a simple example just to see if I can get it to work. For my project, I will need network support to enable the esp32's to talk to each other using mesh / espnow. I believe both use UDP under the covers so it would need this to be emulated. Lack of IP connectivity I could work around.

Is there a plan to emulate the network layers in the future as given the IoT nature of the chips would have thought about this would be a popular option.

I think what you have has a lot of benefits and a great opportunity.