– Kamailio SIP Server –

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
troubleshooting:corefiles [2009/03/19 14:24] – created 212.227.35.68troubleshooting:corefiles [2011/02/17 14:42] (current) – 212.227.35.74
Line 1: Line 1:
 +====== Troubleshooting crashes of your server ======
  
 +
 +[color=red]How to get the reason for a crash of your Kamailio (OpenSER) server?[/color]
 +
 +After a crash of the server it will generate a core file in its working directory, if the functionality is not disabled. This core file can be used to get the cause (or at least a hint) of the crash. The core file contains the memory of the process at the moment it crashes, so it could contain sensitive data.
 +
 +===== Potential causes =====
 +
 +  * Invalid input data, e.g. some messages or from the database
 +  * Errors in the program code, e.g. null pointer access
 +===== What do do with a core file =====
 +
 +Core files can be examined with **gdb**, the gnu debugger. It can read the crash informations, and display (among other things) the backtrace that leads to the crash. Just run:
 +
 +<code>
 +gdb $kamailio_binary $core_file 
 +</code>
 +
 +(Substitute the variables with the information in your installation.) After gdb finished to read its input type **backtrace**, gdb will then output the backtrace. This information should added to your bug report of the crash. Please don't delete the core files after the backtrace generation, as its really helpful for eventual further investigations.
 +
 +In order to get a meaningful backtrace gdb needs some debugging informations in its binary. If you compiled from the source, no additional steps should be necessary from you. If you install a binary package, e.g. from debian, you need to install the **-dbg** package too.
 +
 +If you don't understand the information that gdb prints, some developer on the mailling list should be able to help you.
 +
 +===== How do get a core file =====
 +
 +Sometimes the server don't write the core file after a crash.
 +INFO:core:handle_sigs: core was not generated
 +
 +First check if the "disable_core_dump" variable is set in your configuration. If you run the server from a debian package, check the DUMP_CORE setting in the "/etc/default/kamailio" file. If this are all not set, then there is probably some ulimit configured that prohibits the core file generation. Just run:
 +
 +<code>
 +ulimit -c unlimited
 +</code>
 +
 +in the shell where you started the server. If you run from a init script it should normally already take care of this. If its still not create a core file then perhaps the server has no permissions to write into its working directory, check its permissions and the startup option "-w"..
 +
 +To get per-process core file then you can use on Linux:
 +
 +<code>
 +echo "1" > /proc/sys/kernel/core_uses_pid
 +</code>

Trace: