OpenSIPS 3.3.4 crashes intermittently

[baevga]

Hello OpenSIPS team!

We facing with crashes OpenSIPS v3.3.4 sometimes.

Please, look into this coredump file - https://pastebin.com/9f9i6BSN

Steps to reproduce the behavior:

1. Start OpenSIPS
2. Start traffic
3. Check OpenSIPS crashed

OS/environment information

• Operating System: Ubuntu 22
• OpenSIPS installation: source

[github-actions]

Any updates here? No progress has been made in the last 15 days, marking as stale. Will close this issue if no further updates are made in the next 30 days.

[bogdan-iancu]

Hi @baevga. The backtrace indicates some memory corruption, but there are not hints about the actual point/module where the corruption happens. To help troubleshooting this, please see this tutorial on how to enable the memory debugging (no logging needed). Once the mem debugging is on, OpenSIPS will report whenever coming across a memory corruption.

[baevga]

Hi @bogdan-iancu,

I’ve enabled QMALLOC_DBG and obtained the following trace: https://pastebin.com/LVK7AdVn.

Could you please take a look at it? If there’s anything specific you’d like me to check in the core dump, please let me know, and I’ll be happy to assist.

Thank you in advance for your help!

[bogdan-iancu]

You had some "CRITICAL"log before the shutdown.... Even if the backtrace does not show a "stop" in the memory debugger (due to the detection of a mem corruption)....
So any special log?

[baevga]

Yes, i have a critical record but nothing more:

Jan 15 11:07:14 balancer-node /usr/sbin/opensips[974853]: [MAIN] From customer to worker (rm=REGISTER, src=udp:88.8.8.8:5060, dst=9.9.9.9:5060, ru=sip:client.vpbx.ru, du=<null>, [email protected])
Jan 15 11:07:14 balancer-node /usr/sbin/opensips[974853]: [RELAY] Request leaving server (rm=REGISTER, src=udp:88.8.8.8:5060, dst=9.9.9.9:5060, ru=sip:client.vpbx.ru, du=sip:192.168.1.2:5060, [email protected])
Jan 15 11:07:14 balancer-node /usr/sbin/opensips[974856]: [MAIN] Incoming request (rm=REGISTER, src=udp:88.8.8.8:5060, dst=9.9.9.9:5060, ru=sip:client.vpbx.ru, du=<null>, [email protected])
Call-ID: [email protected]
Jan 15 11:07:14 balancer-node /usr/sbin/opensips[974856]: [MAIN] From customer to worker (rm=REGISTER, src=udp:88.8.8.8:5060, dst=9.9.9.9:5060, ru=sip:client.vpbx.ru, du=<null>, [email protected])
Jan 15 11:07:14 balancer-node /usr/sbin/opensips[974856]: [RELAY] Request leaving server (rm=REGISTER, src=udp:88.8.8.8:5060, dst=9.9.9.9:5060, ru=sip:client.vpbx.ru, du=sip:192.168.1.2:5060, [email protected])
Jan 15 11:07:24 balancer-node /usr/sbin/opensips[974848]: CRITICAL:core:sig_usr: segfault in process pid: 974848, id: 2

And after that, it rebooted itself and continued working.

One more thing REGISTER request that was passed to t_relay() wasn't sent out.

[bogdan-iancu]

Unfortunately this new point of crash is not related to the memory manager (and its consistency checks). So no clue. Rather keeping the memory debugger ON and hoping to catch something at a point.
BTW, the report is on 3.3, which is no longer maintain, so my advice here will be to try to migrate to a newer version.

[baevga]

Understood. Previously, we encountered this issue on version 3.5, so we kindly request that this issue remains open. I will update our instances to version 3.5, attempt to reproduce the crash, and provide updates here.

[bogdan-iancu]

If it's a different version, I would strongly recommend opening a new report, with the new version and new info. What was already discussed here will be irrelevant while moving to 3.5

[baevga]

Ok, then i will open a brand new issue when gather info relevant to OpenSIPS v3.5.

Thank you for assist!

[baevga]

Hi @bogdan-iancu,

I hope you're doing well!

It seems I’m unable to reproduce the issue on version 3.5 because, after updating OpenSIPS, I encountered the exact same problem described in this ticket: #3542.

This issue is currently blocking our update. Could you please take a look at it?

Thank you in advance!