Segmentation fault in iSER discovery #277
Comments
|
@clicx could you include a backtrace and confirm that you're on the latest git version of libiscsi? |
|
I use git clone into a new machine to test this, therefor it is newest version of master branch. diff --git a/utils/iscsi-ls.c b/utils/iscsi-ls.c
|
|
@clicx OK so latest git. In terms of backtrace I meant like the |
|
I don't have access to any RDMA hw so I can well imagine that there might
be gaps.
This code was donated by mellanox and I think they primarily cared about
getting qemu + libiscsi to work with RDMA.
If you get a stack backtrace I can try to fix the crash but I can not test
:-(
…On Tue, Oct 16, 2018 at 1:41 PM Sitsofe Wheeler ***@***.***> wrote:
@clicx <https://github.com/clicx> OK so latest git. In terms of backtrace
I meant like the thread apply all bt output you get from GDB (e.g. see
https://wiki.debian.org/HowToGetABacktrace ).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#277 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAeNkHygtE-F7lpjBmx4Q-Yu8JE_Ycqgks5ulVVMgaJpZM4XbqDC>
.
|
|
Hi Ronnie, have you considered to use the rdma_rxe driver and LIO to set up an iSER target stack? |
|
|
My environment: I had a similar problem locally, the backtrace is as follows, it looks like memory corruption, dangling pointer or access after free. (gdb) bt
#0 0x00007f5569f7d53b in iscsi_iser_queue_pdu (iscsi=0x5654b07e3480, pdu=0x5654b114b980) at iser.c:704
#1 0x00007f5569f69452 in iscsi_nop_out_async (iscsi=0x5654b07e3480, cb=cb@entry=0x0, data=data@entry=0x0, len=len@entry=0, private_data=private_data@entry=0x0) at nop.c:80
#2 0x00007f556a174fe1 in iscsi_nop_timed_event (opaque=opaque@entry=0x5654b07e31f0) at ../../block/iscsi.c:1417
#3 0x0000565494eaf710 in timerlist_run_timers (timer_list=0x5654b057f460) at ../../util/qemu-timer.c:576
#4 0x0000565494eaf7ee in timerlist_run_timers (timer_list=<optimized out>) at ../../util/qemu-timer.c:509
#5 timerlistgroup_run_timers (tlg=0x5654b056b490) at ../../util/qemu-timer.c:615
#6 0x0000565494e96037 in aio_dispatch (ctx=<optimized out>) at ../../util/aio-posix.c:426
#7 0x0000565494ea9dde in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at ../../util/async.c:352
#8 0x00007f556c4dd7a9 in g_main_dispatch (context=0x5654b0578290) at ../../../glib/gmain.c:3454
#9 g_main_context_dispatch (context=context@entry=0x5654b0578290) at ../../../glib/gmain.c:4172
#10 0x0000565494eac3d8 in glib_pollfds_poll () at ../../util/main-loop.c:297
#11 os_host_main_loop_wait (timeout=429840) at ../../util/main-loop.c:320
#12 main_loop_wait (nonblocking=nonblocking@entry=0) at ../../util/main-loop.c:606
#13 0x0000565494b08957 in qemu_main_loop () at ../../softmmu/runstate.c:739
#14 0x0000565494d2c956 in qemu_default_main () at ../../softmmu/main.c:37
#15 0x00007f556bf2f24a in __libc_start_call_main (main=main@entry=0x56549493b570 <main>, argc=argc@entry=126, argv=argv@entry=0x7ffd72b6cd08) at ../sysdeps/nptl/libc_start_call_main.h:58
#16 0x00007f556bf2f305 in __libc_start_main_impl (main=0x56549493b570 <main>, argc=126, argv=0x7ffd72b6cd08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd72b6ccf8) at ../csu/libc-start.c:360
#17 0x000056549493ce51 in _start ()
(gdb) p iscsi->waitpdu
$11 = (struct iscsi_pdu *) 0x5654b0597c00
(gdb) p iscsi->waitpdu->next
$12 = (struct iscsi_pdu *) 0x5651d41ead87
(gdb) p *(iscsi->waitpdu->next)
Cannot access memory at address 0x5651d41ead87
(gdb) p *(iscsi->waitpdu)
$10 = {next = 0x5651d41ead87, flags = 0, lun = 0, itt = 2975180832, cmdsn = 22100, datasn = 49, response_opcode = ISCSI_PDU_NOP_OUT, callback = 0x5651d4112d37, private_data = 0x0,
outdata = {size = 1, data = 0x1 <error: Cannot access memory at address 0x1>}, outdata_written = 0, payload_offset = 33, payload_len = 0, payload_written = 3571875879, indata = {
size = 0, data = 0x0}, scsi_cbdata = {callback = 0x31, private_data = 0x5651d5127987, task = 0x0}, scsi_timeout = 1, expxferlen = 1}(gdb) bt
#0 0x00007f9c855a853b in iscsi_iser_queue_pdu (iscsi=0x5590aa00a480, pdu=0x5590ab305090) at iser.c:704
#1 0x00007f9c855963f9 in iscsi_scsi_command_async (iscsi=iscsi@entry=0x5590aa00a480, lun=0, task=0x5590aa0a1410, cb=cb@entry=0x7f9c85a2e010 <iscsi_aio_ioctl_cb>, d=<optimized out>, private_data=private_data@entry=0x7f9c5800b080) at iscsi-command.c:282
#2 0x00007f9c85a2c7ed in iscsi_aio_ioctl (bs=<optimized out>, req=<optimized out>, buf=<optimized out>, cb=<optimized out>, opaque=<optimized out>) at ../../block/iscsi.c:1099
#3 0x0000559074619a16 in bdrv_co_ioctl (bs=0x5590aa002890, req=8837, buf=0x5590ab1b7180) at ../../block/io.c:3210
#4 0x00005590746199cd in bdrv_co_ioctl (bs=0x5590aa02d570, req=8837, buf=buf@entry=0x5590ab1b7180) at ../../block/io.c:3208
#5 0x0000559074608c74 in blk_co_do_ioctl (buf=0x5590ab1b7180, req=8837, blk=0x5590abe881c0) at ../../block/block-backend.c:1655
#6 blk_aio_ioctl_entry (opaque=0x5590aa0a49f0) at ../../block/block-backend.c:1676
#7 0x0000559074728ceb in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at ../../util/coroutine-ucontext.c:177
#8 0x00007f9c8757e9c0 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#9 0x00007f9c76ffba70 in ?? ()
#10 0x0000000000000000 in ?? ()
(gdb) p iscsi->waitpdu->next->next
$1 = (struct iscsi_pdu *) 0x3e6e6f69
(gdb) p *(iscsi->waitpdu->next->next)
Cannot access memory at address 0x3e6e6f69
(gdb) p *(iscsi->waitpdu->next)
$16 = {next = 0x3e6e6f69, flags = 161, lun = 0, itt = 2870404640, cmdsn = 21904, datasn = 2865212400, response_opcode = 21904, callback = 0x20, private_data = 0x30, outdata = {
size = 94102499316027, data = 0x5590ab01fb10 "?\034\315\363\225U"}, outdata_written = 0, payload_offset = 0, payload_len = 0, payload_written = 0, indata = {size = 81,
data = 0x5590ab2ff5b0 "ion>"}, scsi_cbdata = {callback = 0x5590aac32c30, private_data = 0x20, task = 0x30}, scsi_timeout = 94102528558683, expxferlen = 0} |
|
It looks like the memory of the pdu struct is being stomped. |
|
I tried compiling version 1.20.0 of libiscsi, replacing version 1.19.0 in debian 12, and it looks like the problem is fixed.
|
iser discovery seems not fully implemented, I can't find an example for it. Therefor I simply modify from iscsi-ls, change iscsi:// to iser://
Then got segmentation fault after iscsi context free.
The text was updated successfully, but these errors were encountered: