Currently we have a problem: big GNOME deployments + Distributions = People using binary packages => no debug information => useless backtraces from bug-buddy
Asking the user to install 100MB of debug info packages is not a solution
So we are going to try a different approach:
- If the user has debug symbols + gdb get the usual backtrace with gdb
- If no debugs symbols (most common case), send a minidump file to a debug server.
Arguments for using minidump format instead a core file:
- Is smaller than core file
- We can use airbag for creating them easily and for processing on the debug server
Using GoogleAirbag project will bring us code for:
- Installing the segmentation handler
- Creating the minidump file
- Processing the minidump file on the server side
Having a SymbolStore with all the debug symbols
The code that we need to write is:
- libgnomeui/whatever code for getting the segmentation and dumping the minidump
- bug-buddy code for sending the minidump to the debug server
SymbolProvider infrastructure for getting packages+debug pakacges from most common distros and storing them in the SymbolStore
The idea is to implement the SymbolProvider in python, because we have nice high-level bingins for managing rpm/deb based distros. Some code has started to support Fedora distribution.
here is a diagram of the process:
http://www.gnome.org/~fherrera/img/debug-diagram.svg
We could implement some kind of filter between debug server and bugzilla to avoid duplicates.
For crashes on python/mono-managed code we already can send the exception trace using bug-buddy
We need to ask the foundation for hardware for the debug server (it would need good bandwidth and huge amount of disc space [aprox 7.2 GB per distro supported]
For supporting testing distros like rawhide we cuold ask distros to host their own debug server to enrich the backtraces.
Comments
MarkHoward: Did you consider distribution specific patches/extension points here? Ubuntu has done a lot of work creating a system-wide crash monitor and automatic debug symbol packages for all of their packages. I think the server-side component for filling in back traces is still under design. It would be a real shame if bug-buddy and ubuntu implement similar things in different ways.
MarkHoward: Speaking of distributions, have you considered support for distribution-specific settings? Many prefer to have all bugs sent to their own bug trackers (since they have many patches) and then only forward interesting reports to gnome. Is there any way that bug-buddy could check for existing stack traces on bugzilla and if none exist then forward to the distro?
DanielHolbach: Ubuntu uses automatic debug symbols and apport from Edgy on.
FernandoHerrera: If the distros bug systems implement a XML-RPC it would be easy to patch bug-buddy in every distro to submit bugs against its own bug tool. About implementing the server-side component, of course, sharing efforts would be ideal. We are going to implement a SymbolStore on top of airbag, a StackWalker to enrich stacks with thouse symbols (also using aribag library) and then we need a SymbolProvieder per distribution. Instead of doing an ubuntu-specific debug-server we could share efforts to create a common one.
AndrewSobala: If a distro wants bug information to go to their tracker, they should be patching bug-buddy / installing their own tool instead of bug-buddy. Since if you're running Stock GNOME, the distros won't be wanting bug reports from you even if you're running Stock GNOME On Redhat.