Development/Tutorials/Debugging/How to create useful crash reports: Difference between revisions

    From KDE TechBase
    (Replaced content with "{{Moved To Community | Guidelines_and_HOWTOs/Debugging }}")
     
    (4 intermediate revisions by 4 users not shown)
    Line 1: Line 1:
    This document describes how to reproduce a useful backtrace of crashing KDE applications. First, some general information is given. Then, we will describe for several distributions how to prepare your KDE packages and gaining the backtrace. This should be enough for most people.
    {{Moved To Community | Guidelines_and_HOWTOs/Debugging }}
    There are additional sections on how to create backtraces with the GNU Debugger and with Valgrind, which are in some cases useful.
     
    =How to create useful crash reports=
     
    A good crash report at [http://bugs.kde.org Bugzilla] consists of two parts: a '''description''' of how to reproduce the crash and a '''backtrace''' of the crash. With one of those elements missing, it is much harder (if not impossible) for developers to tackle the problem.
     
    A description should consist of more than only "it crashed". Try to describe everything you did prior to the crash. Did you click on a button, opened a particular website or file which caused problems? That little detail which may look useless to you may be useful for the developer, so just write it down.
     
    A more insightful article on how to write good bug descriptions is available [http://www.chiark.greenend.org.uk/~sgtatham/bugs.html at this link], please read that before reporting bugs.
     
    Don't attach the backtrace to the bug report. Instead, simply paste it. This way it is much easier for developers to search for duplicate reports, because attachments will not be searched.
     
    If you paste a backtrace to a report, make sure you strip all but one or two of the
    (no debugging symbols found)
    lines from the backtrace as they make it harder to read.
     
    Even though pasting backtraces directly is preferred over adding an attachment, please do not paste other things like logs (valgrind, strace or terminal output) or example data (mails, HTML files and so on). Use attachments for these items.
     
    ==Backtraces==
     
    Backtraces are essential. They may look meaningless to you, but they might actually contain a wealth of useful information. A backtrace describes which functions were called prior to the crash, so that developers may track down in which function the mess started. Having good backtraces has a downside: [[Development/Tutorials/Debugging/Debugging_symbols|libraries and executables occupy much more disk space than their optimized counter parts]]. That's the reason why many distros choose to install stripped files, '''which results in useless backtraces''':
     
    (no debugging symbols found)
    Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    [Thread debugging using libthread_db enabled]
    [New Thread -1233848624 (LWP 12212)]
    [New Thread -1255081072 (LWP 12820)]
    [New Thread -1240921200 (LWP 12819)]
    [New Thread -1266680944 (LWP 12818)]
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    (no debugging symbols found)
    0xffffe410 in __kernel_vsyscall ()
    #0  0xffffe410 in __kernel_vsyscall ()
    #1  0xb6a1210b in ?? () from /lib/tls/i686/cmov/libpthread.so.0
    #2  0xb6a85afe in ?? () from /usr/lib/libX11.so.6
    #3  0x00000003 in ?? ()
    #4  0x082149c0 in ?? ()
    #5  0x00003ffc in ?? ()
    #6  0x00000000 in ?? ()
     
    But no worries, with some modifications you can create full blown backtraces for KDE applications.
     
    ==Preparing your KDE packages==
     
    If your distribution has debugging-enabled packages, install them.
     
    It is easy to see which debug packages you are missing from looking at the backtrace. For example, take the following line from a backtrace:
     
    #6  0xb7975bdc in ?? () from /usr/lib/libkmailprivate.so.4
     
    The <tt>??</tt> indicates that the library <tt>libkmailprivate.so.4</tt> does not have debug information, which might be available in separate debug packages. In this case, it is pretty easy to guess that you need to install debug packages for KMail to get a better backtrace.
     
    Sometimes, you need to install more than one debug package to get a good backtrace. This depends on how the distribution splits up the packages. For example, for some distributions it is enough to install the debug package for <tt>kdepim</tt> to get enough debugging information for a crash in KMail, for other distributions there is an additional debug package just for KMail.
     
    Here's a list of how to obtain debug packages for some distributions:
     
    ===Arch===
    Arch currently doesn't offer debug packages with symbols ([https://bugs.archlinux.org/task/10975 Feature Request]). You therefore need to recompile all concerned packages with debugging symbols. See their informative document [https://wiki.archlinux.org/index.php/Debug_-_Getting_Traces Getting Traces].
     
    ===Chakra===
    Chakra packages are already build with a minimal set of symbols.
     
    To get more, you need the full debug packages. To install the most basic one, <tt>kdelibs-debug</tt>, just enter <tt>pacman -Sy kdelibs-debug</tt> in your console. You might find a separate debug package for the concerned KDE application in the [http://chakra-project.org/packages/index.php?subdir=desktop%2Fx86_64&sortby=name&order=ascending&act=search&searchpattern=debug list of all debug packages], that you can additionally install.
     
    To install the whole group of debug packages, do a <tt>pacman -Sy kde-debug</tt> <em>(Careful: enormous size)</em>.
     
    ===Debian===
    Debian offers <tt>-dbg</tt> packages to easy create useful backtraces. Just install the corresponding <tt>-dbg</tt> package. e.g. <tt>kdepim-dbg</tt> for KMail crashes. The dependencies of -dbg makes sure to pull in the other right packages (kdelibs-dbg, gdb, and so on). See also the Debian document on [http://wiki.debian.org/HowToGetABacktrace How to get a Backtrace].
     
    ===Fedora===
    In Fedora you just need to do:
    # <tt>yum provides "/usr/lib/libkmailprivate.so.4"</tt> to find out which package provides that file;
    # <tt>debuginfo-install kdepim</tt> as root to install debug packages for that package (dependencies included).
    A complete and detailed guide for Fedora is available in [http://fedoraproject.org/wiki/StackTraces this document] describing how to proceed. Fedora uses a separate debuginfo repository that has to be enabled. Also consider <tt>auto-update-debug-info</tt> yum plugin to keep debuginfo packages up to date.
     
    ===FreeBSD ports===
    You need to recompile the concerned packages with the <tt>WITH_DEBUG</tt>-flag. Please refer to the [http://freebsd.kde.org/faq.php#AKDEapplicationcrashedandIwanttofileabugreportathttpbugskdeorgbutthebacktraceintheKDEcrashmanagerisuselessWhatcanIdo KDE on FreeBSD FAQ].
     
    ===Gentoo===
    Gentoo has its [http://www.gentoo.org/proj/en/qa/backtraces.xml own document] describing how to proceed.
     
    ===Kubuntu/Ubuntu===
    The Ubuntu family makes things quite easy. Every official KDE module has an additional package in the repository, suffixed with <tt>-dbg</tt>. Always install <tt>kdelibs5-dbg</tt>, because all KDE applications use kdelibs (kdelibs-dbg for KDE 3 applications). Then you should install a -dbg package for the application which crashed. For example if KOrganizer crashed you should install <tt>kdepim-dbg</tt> as well.  If the program is not from an official KDE module and has no -dbg package, you can install the -dbgsym package from the repository listed on this [https://wiki.kubuntu.org/DebuggingProgramCrash Debugging Program Crashes] page. 
    *During the Ubuntu development cycle the Apport crash handler is turned on which will report crashes to launchpad.net and do the backtrace for you, if you would rather use the KDE crash handler turn Apport off in /etc/defaults/apport
    *Starting with Lucid Lynx (10.04) Kubuntu will be forwarding all non kubuntu specific bugs upstream and had disabled Apport so that DrKonqui will be the default crash handler.
     
    ===Linux Mint===
    Linux Mint mostly uses Ubuntu packages, so all KDE packages are accompanied by a <tt>-dbg</tt> package containing the corresponding symbols.
     
    ===Mageia===
    Mageia offers <tt>-debug</tt> packages in a separate repository. See [https://wiki.mageia.org/en/Debugging_software_crashes their wiki page].
     
    ===Mandriva===
    Mandriva 2007.0 and up has additional debugging packages for every single KDE package. Just install the corresponding <tt>-debug</tt> package, like <tt>kdebase-debug</tt> and <tt>kdemultimedia-debug</tt>. You probably want to install <tt>kdelibs-debug</tt> anyways.
    Note: the <tt>-debug</tt> packages are in separate repositories. For instance, for all packages in <tt>main</tt>, you'll find the debugging package in repository <tt>debug_main</tt>.
    See also [http://wiki.mandriva.com/en/Development/Howto/Software_Crash this document].
     
    ===Project Neon===
    Project Neon is a nightly build of the latest KDE trunk for Ubuntu.
    The debugging symbols for every package are in its corresponding -dbg package, so to install the debugging symbols for kdelibs you can use: <tt>sudo apt-get install project-neon-kdelibs-dbg</tt>
    Find more information in [[Getting_Started/Using_Project_Neon_to_contribute_to_KDE#Debugging_symbols|Using Project Neon to contribute to KDE]].
     
    ===openSUSE===
    You should only install the <tt>-debuginfo</tt> packages, for example: <tt>kdepimlibs4-debuginfo</tt>. You can find these packages in [http://en.opensuse.org/KDE/Repositories KDE repositories]. There is also a dedicated [http://en.opensuse.org/openSUSE:Bugreport_application_crashed openSUSE debugging page].
     
    ===Sabayon===
    Packages are, by default, installed without debugging symbols in Sabayon. Installing them is however very easy: just change the value of the splitdebug variable to <tt>enable</tt> in <tt>/etc/entropy/client.conf</tt>. After this, next time you install a package, you will also get the corresponding debug symbols. For details, see the document [https://wiki.sabayon.org/?title=Debugging_Symbols_-_splitdebug Debugging Symbols - Splitdebug].
     
    ===Compile from sources===
     
    If your distribution doesn't have debugging-enabled packages for KDE, you'll have to compile KDE from sources:
     
    * If you're using KDE 3, then at the configure stage, you should supply the parameter <tt>--enable-debug=full</tt> in order to build debug symbols in the resulting files.
    * If you're using KDE 4, then at the cmake stage, you should supply the parameter <tt>-DCMAKE_BUILD_TYPE=Debug</tt>.  If you want to specify your own CXXFLAGS, then use <tt>-DCMAKE_BUILD_TYPE=None -DCMAKE_CXX_FLAGS=$foo</tt> (where $foo is whatever flags you want to set). Note that the Debug build type is highly recommended for most testers.
     
    Then it's just <tt>make</tt> and <tt>make install</tt> as you're used to.
     
    ==Crash!==
     
    Now it's time to crash your application. The KDE Crash Dialog should appear right after the crash, which shows the "Developer Information" tab.
     
    [[Image:Kde-crash-handler.png|center|300px|KDE Crash Dialog]]
     
    Click that tab and wait for a minute. This process may take quite some memory, so things may go sluggish all of a sudden. But the result should look much better. For example:
     
    Using host libthread_db library "/lib/libthread_db.so.1".
    [Thread debugging using libthread_db enabled]
    [New Thread -1232783168 (LWP 7604)]
    [KCrash handler]
    #6  0x0806be76 in TreeMapItem::parent (this=0x0)
        at /home/bram/KDE/kde3/kdeaddons/konq-plugins/fsview/treemap.h:285
    #7  0x08065fea in TreeMapItemList::compareItems (this=0xbfec04a8, item1=0x0,
        item2=0x0)
        at /home/bram/KDE/kde3/kdeaddons/konq-plugins/fsview/treemap.cpp:720
    #8  0xb7281619 in QGList::operator== () from /usr/qt/3/lib/libqt-mt.so.3
    #9  0x0806d498 in QPtrList<TreeMapItem>::operator== (this=0xbfec04a8,
        list=@0xbfec0468) at /usr/qt/3/include/qptrlist.h:74
    #10 0x08062e18 in TreeMapWidget::mousePressEvent (this=0xbfec03ac,
        e=0xbfebff1c)
        at /home/bram/KDE/kde3/kdeaddons/konq-plugins/fsview/treemap.cpp:1840
    #11 0xb7004a63 in QWidget::event () from /usr/qt/3/lib/libqt-mt.so.3
    #12 0xb6f6bca7 in QApplication::internalNotify ()
        from /usr/qt/3/lib/libqt-mt.so.3
    #13 0xb6f6ca88 in QApplication::notify () from /usr/qt/3/lib/libqt-mt.so.3
    #14 0xb7725a84 in KApplication::notify (this=0xbfec055c, receiver=0xbfec03ac,
        event=0xbfebff1c)
        at /home/bram/KDE/kde3/kdelibs/kdecore/kapplication.cpp:550
    #15 0xb6f0bfd2 in QETWidget::translateMouseEvent ()
        from /usr/qt/3/lib/libqt-mt.so.3
    #16 0xb6f0b8b0 in QApplication::x11ProcessEvent ()
        from /usr/qt/3/lib/libqt-mt.so.3
    #17 0xb6f1b761 in QEventLoop::processEvents () from /usr/qt/3/lib/libqt-mt.so.3
    #18 0xb6f82831 in QEventLoop::enterLoop () from /usr/qt/3/lib/libqt-mt.so.3
    #19 0xb6f826b6 in QEventLoop::exec () from /usr/qt/3/lib/libqt-mt.so.3
    #20 0xb6f6b72f in QApplication::exec () from /usr/qt/3/lib/libqt-mt.so.3
    #21 0x0805181e in main (argc=134673960, argv=0xffffffff)
        at /home/bram/KDE/kde3/kdeaddons/konq-plugins/fsview/main.cpp:55
     
    This looks better, right? It shows memory addresses, the source files and line numbers and the parameters passed to functions. Which make it more helpful to the developer where to look for the problem.
     
    {{note|You '''need''' GDB installed to get the backtrace of a crash. Please read the next section to know what GDB is, and how to install it.}}
     
    ==Retrieving a backtrace with GDB==
     
    In some cases, it is not possible to create a backtrace with the KDE Crash Dialog. This may be caused by an application which entered an infinite loop, or the crash dialog did not appear at all for some reason. You can try to grab a backtrace with <tt>gdb</tt>, the [http://sourceware.org/gdb/ GNU Debugger]. GDB is widely available through distribution packages.
     
    Invoking GDB differs from the situation. You can run an application from inside gdb, or attach gdb to an already running process. The latter may be useful when an application already has entered an infinite loop. But we will first start with running an application inside gdb. From the shell, run:
     
    $ gdb someKDEapp
     
    The GDB prompt will appear. Note that this does not start the application itself, you should run it by invoking the <tt>run</tt> command:
     
    (gdb) run
     
    This will run the application like you are used to, and you can work with it like normal (it only consumes far more memory and may feel sluggish). Now it's time to reproduce your crash. When you succeed, the application just closes and you should return to your GDB prompt. Now it's time to run the 'backtrace' command:
     
    {{note|Some KDE applications (such as JuK and KTorrent) have special code to ensure that there is only one running instance of the application at a time.  For these applications you should type in "run --nofork" at the (gdb) prompt instead of "run" because otherwise gdb will try to debug the wrong process.  If you are unsure as to whether to use --nofork just try it.  If the application says it's an unknown option you can remove --nofork.}}
     
    (gdb) thread apply all backtrace
     
    This should give a good backtrace which can be posted at the KDE Bugzilla.
     
    In case you want to attach to an existing process, run the following command in the shell:
     
    $ gdb someKDEapp pid
     
    where ''pid'' is the process ID of the process you want to attach to. Once attached, and the process is in an infinite loop, after using the 'backtrace' command again a useful backtrace will appear. You can use 'continue' command to let the application run again and press Ctrl+C in gdb to be able to again enter commands.
     
    == Retrieving a backtrace when an uncaught exception is causing a crash ==
     
    If your backtrace looks like this:
     
      Thread 1 (Thread 0xb77be730 (LWP 5794)):
      [KCrash Handler]
      #7  0x00e9b416 in __kernel_vsyscall ()
      #8  0x002b1c8f in raise () from /lib/i386-linux-gnu/libc.so.6
      #9  0x002b52b5 in abort () from /lib/i386-linux-gnu/libc.so.6
      #10 0x001c14ed in __gnu_cxx::__verbose_terminate_handler() () from
      /usr/lib/i386-linux-gnu/libstdc++.so.6
      #11 0x001bf283 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
      #12 0x001bf2bf in std::terminate() () from
      /usr/lib/i386-linux-gnu/libstdc++.so.6
      #13 0x001bf464 in __cxa_rethrow () from /usr/lib/i386-linux-gnu/libstdc++.so.6
      [...]
     
    then only GDB can help getting a useful backtrace. A few more commands are needed. From the shell, run:
     
      $ gdb someKDEapp
     
    Then start loading the application:
     
      (gdb) start
     
    {{note|Some KDE applications (such as KMail or Kontact) have special code to ensure that there is only one running instance of the application at a time.  For these applications you should type in "start --nofork" at the (gdb) prompt instead of "start" because otherwise gdb will try to debug the wrong process.  If you are unsure as to whether to use --nofork just try it.  If the application says it's an unknown option you can remove --nofork.}}
     
    The GDB prompt will reappear. Turn logging on:
     
      (gdb) set logging file ~/gdblog.txt
      (gdb) set logging on
     
    We must now tell gdb to stop just before the exception is thrown. enter:
     
      (gdb) break __cxa_throw
      (gdb) break __cxa_rethrow
     
    and finally:
     
      (gdb) cont
     
    This will run the application. Note that the application will consume more memory and be slower.
    Now reproduce the crash. When you succeed, the application window will look frozen.
     
    Don't worry and switch back to the shell window.
    If you see the gdb prompt, type:
     
      (gdb) thread apply all bt
     
    press enter until the (gdb) prompt reappears.
     
    All done! you can now quit gdb:
     
      (gdb) kill
     
    The kill command will stop the program and now enter:
     
      (gdb) quit
     
    Copy/paste the content saved in the ~/gdblog.txt file to your bug report.
     
    ==Retrieving a backtrace with Valgrind==
     
    When it comes to crashes, [http://www.valgrind.org Valgrind] is also a useful tool to create a backtrace. It's not a substitution for GDB, but rather a supplement.
     
    When you run an application in valgrind, every piece of memory read or written by the application is being checked. Valgrind will report erroneous memory operations in the standard output or in a log file. Since most crashes are due to an invalid memory read, valgrind can be useful to track down where the problem occurs.
     
    {{note|Valgrind consists of several tools in order to check or profile an application. For this article, we only use memcheck, the default tool when valgrind is being invoked.}}
     
    Like GDB, Valgrind makes running an application much slower, while consuming a lot more resources.
     
    Start the application within valgrind:
     
    $ valgrind --log-file=someKDEapp someKDEapp
     
    Now reproduce the crash. As soon as this happens, the application and valgrind will terminate. What's left is a file named <tt>someKDEapp.pid</tt> where ''pid'' is replaced by the process ID of the valgrind process. The file may list more errors than the one causing the crash. Here's the bit causing the crash which corresponds to the GDB backtrace above:
     
    ==23292== Invalid read of size 4
    ==23292==    at 0x806BD9E: TreeMapItem::parent() const (treemap.h:285)
    ==23292==    by 0x8065FB9: TreeMapItemList::compareItems(void*, void*) (treemap.cpp:720)
    ==23292==    by 0x50AC618: QGList::operator==(QGList const&) const (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x806D3BF: QPtrList<TreeMapItem>::operator==(QPtrList<TreeMapItem> const&) const (qptrlist.h:74)
    ==23292==    by 0x8062DE7: TreeMapWidget::mousePressEvent(QMouseEvent*) (treemap.cpp:1840)
    ==23292==    by 0x4E2FA62: QWidget::event(QEvent*) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x4D96CA6: QApplication::internalNotify(QObject*, QEvent*) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x4D97A87: QApplication::notify(QObject*, QEvent*) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x4809AC3: KApplication::notify(QObject*, QEvent*) (kapplication.cpp:550)
    ==23292==    by 0x4D36FD1: QETWidget::translateMouseEvent(_XEvent const*) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x4D368AF: QApplication::x11ProcessEvent(_XEvent*) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==    by 0x4D46760: QEventLoop::processEvents(unsigned) (in /usr/qt/3/lib/libqt-mt.so.3.3.8)
    ==23292==  Address 0x2C is not stack'd, malloc'd or (recently) free'd
     
    But to be sure, just attach the whole log file to the crash report.

    Latest revision as of 09:09, 5 August 2016

    This page is now on the community wiki.