Skip to contentSkip to conversation

kvm guest debug using gdb on x86

DISCUSSION Aneesh Kumar K.V 4 participants 10 messages May 19, 2009 – May 21, 2009 KVM

lore
Share & tools
lore
#0 lore raw

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
(gdb)

any patches that i can try ?

-aneesh

#1 lore raw
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f03000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 3 lines

(gdb)

any patches that i can try ?

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

-aneesh

#2 lore raw
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 10 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#3 lore raw
Quoted 16 lines · Jan Kiszka
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f03000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 7 lines

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 6 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Works better with the four patches found at

Quoted 8 lines
Quoted 4 lines

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Both host and guest are x86-32. For kvm user space I did a git clone from git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git. I tested with commit 22d239bcee126742df46938ee8ddc7c6b9209e23. That version fails with the "remote g packet reply is too long" error. Then i applied the four patches from your git repo. That gets the guest to stop execution on breakpoint, i can check the stack. but single stepping doesn't work.

Guest kernel I tried was linux-2.6-tip

-aneesh

#4 lore raw
Quoted 17 lines · Aneesh Kumar K.V
Quoted 16 lines · Jan Kiszka
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f03000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00

Quoted 14 lines
Quoted 4 lines

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 3 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Quoted 9 lines
Quoted 6 lines

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Quoted 8 lines

Both host and guest are x86-32. For kvm user space I did a git clone from git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git. I tested with commit 22d239bcee126742df46938ee8ddc7c6b9209e23. That version fails with the "remote g packet reply is too long" error. Then i applied the four patches from your git repo. That gets the guest to stop execution on breakpoint, i can check the stack. but single stepping doesn't work.

Guest kernel I tried was linux-2.6-tip

Does debugging work with -no-kvm, ie. in TCG mode?

Jan

PS: For qemu-kvm.git you can now use my qemu gdb queue directly, ie. you don't need to rebase manually.

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#5 lore raw
Quoted 18 lines · Jan Kiszka
Quoted 17 lines · Aneesh Kumar K.V
Quoted 16 lines · Jan Kiszka
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 22 lines

00

Quoted 21 lines
Quoted 4 lines

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 3 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Quoted 9 lines
Quoted 6 lines

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Both host and guest are x86-32. For kvm user space I did a git clone from git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git. I tested with commit 22d239bcee126742df46938ee8ddc7c6b9209e23. That version fails with the "remote g packet reply is too long" error. Then i applied the four patches from your git repo. That gets the guest to stop execution on breakpoint, i can check the stack. but single stepping doesn't work.

Quoted 3 lines
Quoted 1 line

Guest kernel I tried was linux-2.6-tip

Does debugging work with -no-kvm, ie. in TCG mode?

Yes. debugging works with -no-kvm option. I can do single stepping.

-aneesh

#6 lore raw
Quoted 50 lines · Aneesh Kumar K.V
Quoted 18 lines · Jan Kiszka
Quoted 17 lines · Aneesh Kumar K.V
Quoted 16 lines · Jan Kiszka
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 21 lines

00

Quoted 20 lines
Quoted 4 lines

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 3 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Quoted 9 lines
Quoted 6 lines

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Both host and guest are x86-32. For kvm user space I did a git clone from git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git. I tested with commit 22d239bcee126742df46938ee8ddc7c6b9209e23. That version fails with the "remote g packet reply is too long" error. Then i applied the four patches from your git repo. That gets the guest to stop execution on breakpoint, i can check the stack. but single stepping doesn't work.

Quoted 6 lines
Quoted 2 lines
Quoted 1 line

Guest kernel I tried was linux-2.6-tip

Does debugging work with -no-kvm, ie. in TCG mode?

Yes. debugging works with -no-kvm option. I can do single stepping.

Sigh, confirmed, there is a regression in qemu-kvm. Will dig into this. Older kvm-userspace should be fine (e.g. the last version you can pull from my tree).

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#7 lore raw
Quoted 20 lines · Jan Kiszka
Quoted 19 lines · Aneesh Kumar K.V
Quoted 18 lines · Jan Kiszka
Quoted 17 lines · Aneesh Kumar K.V
Quoted 16 lines · Jan Kiszka
Quoted 15 lines · Aneesh Kumar K.V
Quoted 14 lines · Aneesh Kumar K.V

Hi,

With the latest qemu-kvm and 2.6.30-rc6 kernel i am not able to get the guest debugging with gdb. I get the following error.

$gdb ./vmlinux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

This GDB was configured as "i486-linux-gnu"...
(gdb) b do_fork
Breakpoint 1 at 0xc106cfc8: file kernel/fork.c, line 1347.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
[New Thread 1]
Remote 'g' packet reply is too long:
7fa5000000000000000000000000000057e209c10000000004000000000000000000000000000000c8b3d0c100000000c03fd1c100000000a83fd1c10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000912d03c1000000000202000060000000680000007b0000007b000000d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f60b801540000000000000000000007f03000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

0

Quoted 22 lines
Quoted 22 lines

000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 21 lines

00

Quoted 20 lines
Quoted 4 lines

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Quoted 3 lines
Quoted 3 lines

(gdb)

any patches that i can try ?

Quoted 9 lines
Quoted 6 lines

Works better with the four patches found at

http://git.kiszka.org/?p=kvm-userspace.git;a=shortlog;h=refs/heads/queues/gdb

But a next and continue doesn't get the prompt back on gdb. The guest does stops the execution.

What arch is host and guest (x86-32 or -64)? What KVM versions are you using? Are you sure that guest and vmlinux matches?

Both host and guest are x86-32. For kvm user space I did a git clone from git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git. I tested with commit 22d239bcee126742df46938ee8ddc7c6b9209e23. That version fails with the "remote g packet reply is too long" error. Then i applied the four patches from your git repo. That gets the guest to stop execution on breakpoint, i can check the stack. but single stepping doesn't work.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line

Guest kernel I tried was linux-2.6-tip

Does debugging work with -no-kvm, ie. in TCG mode?

Yes. debugging works with -no-kvm option. I can do single stepping.

Sigh, confirmed, there is a regression in qemu-kvm. Will dig into this.

Hmpf, the regression is also present in qemu upstream in kvm mode.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#8 lore raw
Quoted 5 lines · Jan Kiszka
Quoted 2 lines

Sigh, confirmed, there is a regression in qemu-kvm. Will dig into this.

Hmpf, the regression is also present in qemu upstream in kvm mode.

We need gdb tests in kvm-autotest.

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka
Quoted 2 lines

Sigh, confirmed, there is a regression in qemu-kvm. Will dig into this.

Hmpf, the regression is also present in qemu upstream in kvm mode.

That was only local confusion, the regression turned out to be qemu-kvm-only.

Quoted 3 lines

We need gdb tests in kvm-autotest.

Yes, I know. It just takes a bit free resources to set this up...

Jan