Linux Kernel Debuggers, KDB or KGDB?
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
Signature · 2 lines
Paul J Albrecht [email protected]
Quoted 3 lines · [email protected] said:
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
You ought to add UML to the list, since it is useful for debugging any part of the kernel that's not arch code or a hardware device driver (except that there's now USB support for UML).
Jeff
Quoted 3 lines · Paul J Albrecht
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
kdb is a really low level debugger which understands the kernel structures. It does its utmost to stop all kernel activity while it is running and to use as few kernel services as possible so it can run even when the kernel is dead. It (currently) has no source level debugging.
kgdb relies on gdb so you loose the knowledge of kernel internals (no, I am *not* going to teach gdb about kernel stacks, out of line lock code etc.). kgdb has more of a dependency on a working kernel. It provides source level debugging, although stack backtrace tends not to work unless you compile the kernel with frame pointers.
UML is great for debugging generic kernel code such as filesystems, but cannot be used for most arch code or hardware drivers.
My ideal debugger is one that combines the internal knowledge of kdb with the source level debugging of gdb. I know how to do this over a serial line, finding time to write the code is the problem.
Hi,
I mostly agree with Keith.
kdb and kgdb have similar capabilities. kgdb can be used to do kdb like assembly level debugging too, though it needs some knowledge of gdb and kgdb internals.
You can analyze any problems with either of the debuggers. You'll need to decide which debugger to go for depending on your requirements.
I've got many queries asking kgdb capabilities. I guess it's time for a webpage comparing the two debuggers. People who are new to linux kernel and have not used any of the debuggers can hardly arrive at a decision as to which debugger is better suited for their work.
I always suggest for kgdb over kdb because, IMO source level debugging makes a programmers life much easier.
UML too is a good kernel debugging tool. It offers the advantage of source level debugging on a single machine. IMHO it's more useful for beginners.
Quoted 31 lines · Keith Owens
Quoted 3 lines · Paul J Albrecht
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
kdb is a really low level debugger which understands the kernel structures. It does its utmost to stop all kernel activity while it is running and to use as few kernel services as possible so it can run even when the kernel is dead. It (currently) has no source level debugging.
kgdb relies on gdb so you loose the knowledge of kernel internals (no, I am *not* going to teach gdb about kernel stacks, out of line lock code etc.). kgdb has more of a dependency on a working kernel. It provides source level debugging, although stack backtrace tends not to work unless you compile the kernel with frame pointers.
UML is great for debugging generic kernel code such as filesystems, but cannot be used for most arch code or hardware drivers.
My ideal debugger is one that combines the internal knowledge of kdb with the source level debugging of gdb. I know how to do this over a serial line, finding time to write the code is the problem.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 2 lines
Amit Kale Veritas Software ( http://www.veritas.com )
Quoted 6 lines · Keith Owens
Quoted 3 lines · Paul J Albrecht
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
Quoted 6 lines
kdb is a really low level debugger which understands the kernel structures. It does its utmost to stop all kernel activity while it is running and to use as few kernel services as possible so it can run even when the kernel is dead. It (currently) has no source level debugging.
I'd like to know more about your plans to enhance KDB with source level debug capability. What sort of compiler debug output would be supported? STABs or DWARF? Both? What hardware platforms would be supported? What about gui support?
KDB is console debugger, i.e., it doesn't support a remote serial protocol so I don't understand how source level debug would work. Would you have to boot an unstripped kernel executable whenever you wanted to debug?
KGDB is better because it's a source level debugger and because you get a graphical interface, Data Display Debugger (DDD), for free when you use gdb.
KGDB is better because it's more platform agnostic than KDB. The only architecture dependent part in the kernel is the gdb stub which is small and well defined.
Quoted 1 line
kgdb relies on gdb so you loose the knowledge of kernel
internals (no, > I am *not* going to teach gdb about kernel stacks, out of line lock > code etc.). kgdb has more of a dependency on a working kernel. It
Quoted 3 lines
provides source level debugging, although stack backtrace tends not to work unless you compile the kernel with frame pointers.
I don't know what you mean when you say "loose knowledge of kernel internals" with KGDB ... I'd rather develop a set of standard gdb user defined commands to expose kernel internals than hard code them in the linux kernel.
As for stack frame analysis, both debuggers have problems with backtrace if the kernel is compiled without CONFIG_FRAME_POINTER option. Whether the problem is fixed in gdb for KGDB users or the kernel for KDB users is immaterial in choosing one debugger over the other.
Signature · 2 lines
Paul J Albrecht [email protected]
Quoted 2 lines · Paul J Albrecht
I'd like to know more about your plans to enhance KDB with source level debug capability.
Use a combination of gdb and kdb. kdb to support kernel internals, gdb to take the kdb output and add source level data. It needs two machines, one that is running to support gdb, the second machine is being debugged, with a serial console between them.
The problem will be stopping gdb from making assumptions about the machine being debugged. Instead of changing gdb code, use a gdb wrapper program to intercept user commands and gdb serial protocol and convert them to kdb commands.
Quoted 2 lines
Would you have to boot an unstripped kernel executable whenever you wanted to debug?
Boot, no. But the machine running gdb will need an copy of the unstripped vmlinux and module objects to get the debug information.
Quoted 10 lines · Keith Owens
Quoted 2 lines · Paul J Albrecht
I'd like to know more about your plans to enhance KDB with source level debug capability.
Use a combination of gdb and kdb. kdb to support kernel internals, gdb to take the kdb output and add source level data. It needs two machines, one that is running to support gdb, the second machine is being debugged, with a serial console between them.
This is how solaris ksld (kernel source level debugger works). solaris dbx connects to a kernel stub which serves as kadb (solaris assembly level debugger) as well as a debugger stub.
Quoted 4 lines
The problem will be stopping gdb from making assumptions about the machine being debugged. Instead of changing gdb code, use a gdb wrapper program to intercept user commands and gdb serial protocol and convert them to kdb commands.
I am not sure if kdb provides access to kernel threads in a form that can be conveniently by a wrapper program.
Interested people can check whether all features of kgdb
(http://kgdb.sourceforge.net/) are available in convenient form in kdb.
The logic of holding slave cpus (the cpu in the debugger is the master while others are slaves) is different in kdb and kgdb. Handling of nmi-watchdog too is different.
Quoted 6 lines
Quoted 3 lines
Would you have to boot an unstripped kernel executable whenever you wanted to debug?
Boot, no. But the machine running gdb will need an copy of the unstripped vmlinux and module objects to get the debug information.
Plus all the sources.
All this is required by gdb. In theory gdb could do well with just source code and symbol information only, though for lack of a symbol information requesting packet in the gdb remote protocol, gdb can't get symbol information directly from the stub.
There is some effort in gdb world to allow a stub to query gdb for symbol information. Probably the reverse can also be added. Need someone good understanding of gdb to do that.
It's because of these reasons that kgdb module debugging setup is combersome.
Quoted 6 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 2 lines
Amit Kale Veritas Software ( http://www.veritas.com )
Quoted 9 lines · Jeff Dike
Quoted 3 lines · [email protected] said:
Where can I find an analysis of the relative strengths and weaknesses of KDB and KGDB for kernel debug? Has the linux community come to any consensus regarding the utility one or the other?
You ought to add UML to the list, since it is useful for debugging any part of the kernel that's not arch code or a hardware device driver (except that there's now USB support for UML).
Basically you could add support for ALL generic subsystems, that support dummy hardware, like SCSI and ISDN for example.
Is that planned or do I suggest sth. stupid here? ;-)
Regards
Ingo Oeser
Signature · 2 lines
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
<<<<<<<<<<<< been there and had much fun >>>>>>>>>>>>Quoted 4 lines · [email protected] said:
Basically you could add support for ALL generic subsystems, that support dummy hardware, like SCSI and ISDN for example.
Is that planned or do I suggest sth. stupid here? ;-)
Neither. I know squat about hardware, so I had no idea that SCSI and ISDN would be easy to do from UML.
If the SCSI and ISDN people want to produce appropriate UML drivers, I take patches :-)
Jeff
Quoted 10 lines · Jeff Dike
Quoted 3 lines · [email protected] said:
Basically you could add support for ALL generic subsystems, that support dummy hardware, like SCSI and ISDN for example. Is that planned or do I suggest sth. stupid here? ;-)
Neither. I know squat about hardware, so I had no idea that SCSI and ISDN would be easy to do from UML.
If the SCSI and ISDN people want to produce appropriate UML drivers, I take patches :-)
Everything is there. SCSI and ISDN have the equivalent devices of the "lo" driver for the networking layer. Or the equivalent of tun/tap devices for the ethernet layer.
It just have to be an config.in option in UML and every other adapters switched off.
The problem is: I still do not really get how UML really works. Many of the mapping rules (Kernel machanism on normal arch -> UML) are not quite clear to me.
Is there a paper or sth. like that describing the design a bit more in detail? I only found usage papers on the user-mode-linux home page.
Regards
Ingo Oeser
Signature · 2 lines
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
<<<<<<<<<<<< been there and had much fun >>>>>>>>>>>>Quoted 3 lines · [email protected] said:
Everything is there. SCSI and ISDN have the equivalent devices of the "lo" driver for the networking layer. Or the equivalent of tun/tap devices for the ethernet layer.
Is this sufficient to do driver development? TUN/TAP doesn't let me write ethernet drivers inside UML.
Quoted 2 lines
It just have to be an config.in option in UML and every other adapters switched off.
This is a matter of fiddling the config process. It might not be easy, but there isn't much doubt that it's possible :-)
Quoted 5 lines
The problem is: I still do not really get how UML really works. Many of the mapping rules (Kernel machanism on normal arch -> UML) are not quite clear to me.
Is there a paper or sth. like that describing the design a bit more in detail? I only found usage papers on the user-mode-linux home page.
First, have a look at the USB patch at http://user-mode-linux.sourceforge.net/patches/uml-hcd-2.4.3.patch
My ALS paper has a description of how UML basically works inside and it's not too much out of date - http://user-mode-linux.sourceforge.net/als2000/index.htm
l
Also, my ALS and LCA slides have this sort of information -
http://user-mode-linux.sourceforge.net/slides/als2000/slides.html
http://user-mode-linux.sourceforge.net/slides/lca2001/lca.html
The LCA slides are probably better. They are from a few months later, and I had a longer slot, so I needed more slides.
If you have questions which aren't answered on my site, send me mail.
Jeff
Quoted 6 lines
Quoted 3 lines
Everything is there. SCSI and ISDN have the equivalent devices of the "lo" driver for the networking layer. Or the equivalent of tun/tap devices for the ethernet layer.
Is this sufficient to do driver development? TUN/TAP doesn't let me write ethernet drivers inside UML.
For ISDN not really. For SCSI yes - scsi generic would let you write a virtual scsi adapter 'owning' some physical devices
Quoted 5 lines · [email protected] said:
Quoted 3 lines
Is this sufficient to do driver development? TUN/TAP doesn't let me write
ethernet drivers inside UML.For ISDN not really. For SCSI yes - scsi generic would let you write a virtual scsi adapter 'owning' some physical devices
Fine, so go ahead and write a UML SCSI adapter...
I would love to see this happen. If you need UML help that's not on the site, let me know, and I'll be happy to do what I can.
Jeff
Quoted 10 lines · Jeff Dike
Quoted 5 lines
Quoted 3 lines
Is this sufficient to do driver development? TUN/TAP doesn't let me write
ethernet drivers inside UML.For ISDN not really. For SCSI yes - scsi generic would let you write a virtual scsi adapter 'owning' some physical devices
Fine, so go ahead and write a UML SCSI adapter...
I would love to see this happen. If you need UML help that's not on the site, let me know, and I'll be happy to do what I can.
There is the simulator SCSI, net, and serial drivers in the ia64 tree. Dunno how similar that would be to what you need.
Bill
Quoted 3 lines · Keith Owens
My ideal debugger is one that combines the internal knowledge of kdb with the source level debugging of gdb. I know how to do this over a serial line, finding time to write the code is the problem.
http://pice.sourceforge.net is one approach to it; but it's currently a bit limited: e.g. ATM UP-i386-2.2 only and somewhat windowish; also the approach only really works for modules not complete kernel sources unless you have a *lot* of memory. Still in its limits it works.
-Andi