gdbstub: packet reply is too long
Hi,
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but
gdb always refuses it with:
Remote 'g' packet reply is too long:
0000000000000000000000000000000000000000000000002306000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f0ff0000000000000230020000f00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I tried kvm-79 and kvm-git from today. gdb is v6.8 (ubuntu intrepid)
Any hints?
Regards
Andreas
Quoted 11 lines · Andreas Schultz
Hi,
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but
gdb always refuses it with:
Remote 'g' packet reply is too long:
0000000000000000000000000000000000000000000000002306000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f0ff0000000000000230020000f00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000I tried kvm-79 and kvm-git from today. gdb is v6.8 (ubuntu intrepid)
Any hints?
QEMU and also KVM's x86_64 version are currently broken /wrt debugging targets < 64 bits. I've posted a fix a while ago, but there were concerns that gdb should better be enhanced (which is basically true, but unrealistic to achieve in the near future).
In the meantime, you may want to have a look at my kvm [1] or qemu patch queue [2]. Note that unless you provide a binary to gdb that is 32 bit already, you may have to switch the architecture manually ("set arch i386" or "i8086") before attaching.
Jan
[1] git://git.kiszka.org/kvm-userspace.git gdb-queue
[2] git://git.kiszka.org/qemu.git gdb-queue
Quoted 7 lines
Quoted 2 lines
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but gdb always refuses it with:
QEMU and also KVM's
x86_64version are currently broken /wrt debugging targets < 64 bits. I've posted a fix a while ago, but there were concerns that gdb should better be enhanced (which is basically true, but unrealistic to achieve in the near future).
I still maintain that making the g packet format depend on the current CPU more is absolutely the wrong way to fix this.
Paul
Quoted 10 lines · Paul Brook
Quoted 7 lines
Quoted 2 lines
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but gdb always refuses it with:
QEMU and also KVM's
x86_64version are currently broken /wrt debugging targets < 64 bits. I've posted a fix a while ago, but there were concerns that gdb should better be enhanced (which is basically true, but unrealistic to achieve in the near future).I still maintain that making the g packet format depend on the current CPU more is absolutely the wrong way to fix this.
From a higher perspective, it is surely not the cleanest approach. But it still appears to be the only one which helps us working around this gdb shortcoming.
There are internal issues in gdb (hard coupling of current and target arch) that will not allow this to be fixed in the near future, check the code (or my posting I sent out last month). Moreover, this only restores qemu to a state that it used to have for quite a while. I really wonder why we should voluntarily break this support for the users.
Jan
Quoted 9 lines · Jan Kiszka
Quoted 8 lines · Paul Brook
Quoted 7 lines
Quoted 2 lines
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but gdb always refuses it with:
QEMU and also KVM's
x86_64version are currently broken /wrt debugging targets < 64 bits. I've posted a fix a while ago, but there were concerns that gdb should better be enhanced (which is basically true, but unrealistic to achieve in the near future).
Quoted 6 lines
Quoted 2 lines
I still maintain that making the g packet format depend on the current CPU more is absolutely the wrong way to fix this.
From a higher perspective, it is surely not the cleanest approach. But it still appears to be the only one which helps us working around this gdb shortcoming.
Actually it isn't. You could add an explicit switch.
Paul
Quoted 8 lines · Paul Brook
Quoted 7 lines · Jan Kiszka
Quoted 6 lines · Paul Brook
Quoted 6 lines
Quoted 2 lines
I'm trying to debug a 32bit (i386) kernel on a 64bit host under kvm, but gdb always refuses it with:
QEMU and also KVM's
x86_64version are currently broken /wrt debugging targets < 64 bits. I've posted a fix a while ago, but there were concerns that gdb should better be enhanced (which is basically true, but unrealistic to achieve in the near future).
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I still maintain that making the g packet format depend on the current CPU more is absolutely the wrong way to fix this.
From a higher perspective, it is surely not the cleanest approach. But it still appears to be the only one which helps us working around this gdb shortcoming.
Actually it isn't. You could add an explicit switch.
And what would this buy us? I would have to go from your gdb terminal to qemu, probably the monitor, just to switch manually what now happens automatically. I don't see the case where you wouldn't want to switch when you try to debug 16 or 32 bit code, so what would be the gain? Or do you want some switch to disable this automatic register format switching?
Jan
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
From a higher perspective, it is surely not the cleanest approach. But it still appears to be the only one which helps us working around this gdb shortcoming.
Quoted 8 lines
Quoted 1 line
Actually it isn't. You could add an explicit switch.
And what would this buy us? I would have to go from your gdb terminal to qemu, probably the monitor, just to switch manually what now happens automatically. I don't see the case where you wouldn't want to switch when you try to debug 16 or 32 bit code, so what would be the gain? Or do you want some switch to disable this automatic register format switching?
Because, as I've said several times before, the "automatic switching" is wrong. It may happen to work in your very limited circumstances, but there are many fairly common circumstances (e.g. stepping between a 64-bit kernel and a 32-bit userspace) where it's just plain broken. I object strongly to a "fix" that prevents a proper gdb from working.
Quoted 2 lines
There are internal issues in gdb (hard coupling of current and target arch) that will not allow this to be fixed in the near future
Really? I'm pretty sure other architectures already manage it.
Paul
Quoted 11 lines · Paul Brook
Quoted 10 lines
Quoted 4 lines
Quoted 3 lines
From a higher perspective, it is surely not the cleanest approach. But it still appears to be the only one which helps us working around this gdb shortcoming.
Actually it isn't. You could add an explicit switch.
And what would this buy us? I would have to go from your gdb terminal to qemu, probably the monitor, just to switch manually what now happens automatically. I don't see the case where you wouldn't want to switch when you try to debug 16 or 32 bit code, so what would be the gain? Or do you want some switch to disable this automatic register format switching?
Quoted 5 lines
Because, as I've said several times before, the "automatic switching" is wrong. It may happen to work in your very limited circumstances, but there are many fairly common circumstances (e.g. stepping between a 64-bit kernel and a 32-bit userspace) where it's just plain broken. I object strongly to a "fix" that prevents a proper gdb from working.
Well, I'm using gdb over qemu and kvm in precisely that hybrid scenarios, but also in normal ones. Heavily. And I'm only able to do this because of the workaround. But I'm willing to learn about scenarios where it causes /regressions/.
Quoted 5 lines
Quoted 3 lines
There are internal issues in gdb (hard coupling of current and target arch) that will not allow this to be fixed in the near future
Really? I'm pretty sure other architectures already manage it.
What other archs are comparably weird like x86?
Please have a look at the gdb code that I pointed out and explain to me how it can be fixed so that the next gdb version will be fine again with current qemu. I will happily do the patching then. Though, it would still leave us with broken setups until that release is spread sufficiently.
Jan
Quoted 4 lines
Well, I'm using gdb over qemu and kvm in precisely that hybrid scenarios, but also in normal ones. Heavily. And I'm only able to do this because of the workaround. But I'm willing to learn about scenarios where it causes /regressions/.
I find that hard to believe. Doesn't it break horribly as soon as the CPU switches modes?
It's not just regressions that are important. It's the fact that once qemu has your automatic switching hack it's impossible to make it work properly.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
There are internal issues in gdb (hard coupling of current and target arch) that will not allow this to be fixed in the near future
Really? I'm pretty sure other architectures already manage it.
What other archs are comparably weird like x86?
ARM has multiple instruction sets/cpu modes (and can mix the two within the same process). PPC and MIPS also have something similar, though I'm not sure how well they're supported by gdb.
I suspect you may be approaching this the wrong way. Instead of trying to switch architectures, you probably need to teach the 64-bit debugger how to debug 32-bit code.
Paul
Quoted 7 lines · Paul Brook
Quoted 4 lines
Well, I'm using gdb over qemu and kvm in precisely that hybrid scenarios, but also in normal ones. Heavily. And I'm only able to do this because of the workaround. But I'm willing to learn about scenarios where it causes /regressions/.
I find that hard to believe. Doesn't it break horribly as soon as the CPU switches modes?
It still breaks in very few corner cases, but that's (most probably) due to the fact the gdb is not yet well prepared to switch sub-architectures properly during runtime. However, it only breaks in cases that wouldn't work without the switching anyway. And you can heal them by restarting gdb.
Quoted 3 lines
It's not just regressions that are important. It's the fact that once qemu has your automatic switching hack it's impossible to make it work properly.
Interestingly, qemu used to work precisely like that before.
Quoted 9 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
There are internal issues in gdb (hard coupling of current and target arch) that will not allow this to be fixed in the near future
Really? I'm pretty sure other architectures already manage it.
What other archs are comparably weird like x86?
ARM has multiple instruction sets/cpu modes (and can mix the two within the same process). PPC and MIPS also have something similar, though I'm not sure how well they're supported by gdb.
Do those archs also have multiple register layouts that are coupled to those different instruction sets? Do they switch the instruction sets via 'set arch'? I think x86 is (historically) special here.
Quoted 4 lines
I suspect you may be approaching this the wrong way. Instead of trying to switch architectures, you probably need to teach the 64-bit debugger how to debug 32-bit code.
As I said, the problem in gdb is the hard coupling of the target architecture (which could perfectly stay at x86_64 for a session) and the current debugger architecture (which should be switched according to the current CPU mode). Unfortunately, this is not yet feasible with gdb, see gdbarch_update_p().
Fixing this (once understood what are all the problems preventing a fix for several years now) is one thing, keeping the workaround for current gdb in qemu is, IMHO, another. Right now we don't have a gdb fix in sight, so I'm simply voting for reintroducing the workaround. That's all. We can kill it or make it optional once the issue is solved. But we should _not_ do this _before_ it is solved, causing only pain to people who just want to use the gdbstub.
Jan
Quoted 3 lines
Do those archs also have multiple register layouts that are coupled to those different instruction sets? Do they switch the instruction sets via 'set arch'?
No, but neither does x86. You just need to teach gdb how to debug 32-bit code on a 64-bit target.
Quoted 1 line
I think x86 is (historically) special here.
I think it's just that noone's made the effort to fix this problem.
Quoted 7 lines
Fixing this (once understood what are all the problems preventing a fix for several years now) is one thing, keeping the workaround for current gdb in qemu is, IMHO, another. Right now we don't have a gdb fix in sight, so I'm simply voting for reintroducing the workaround. That's all. We can kill it or make it optional once the issue is solved. But we should _not_ do this _before_ it is solved, causing only pain to people who just want to use the gdbstub.
I don't buy this argument. You already have to tell gdb which mode to work in, so I don't see any reason why you shouldn't also tell qemu.
Recent GDB interrogate the target when they connect, and the target describes the registers it has. This isn't currently implemented on x86 in qemu, but I'm pretty sure it'd be incompatible with dynamic switching.
Paul
Quoted 6 lines · Paul Brook
Quoted 3 lines
Do those archs also have multiple register layouts that are coupled to those different instruction sets? Do they switch the instruction sets via 'set arch'?
No, but neither does x86. You just need to teach gdb how to debug 32-bit code on a 64-bit target.
And that means setting current_gdbarch while keeping target_gdbarch - that's where reality (existing gdb code) bites us. Again, I'm not arguing against fixing this, I'm arguing in keeping qemu's workaround until this is done. I will look into the gdb part, but one after the other.
Quoted 13 lines
Quoted 2 lines
I think x86 is (historically) special here.
I think it's just that noone's made the effort to fix this problem.
Quoted 7 lines
Fixing this (once understood what are all the problems preventing a fix for several years now) is one thing, keeping the workaround for current gdb in qemu is, IMHO, another. Right now we don't have a gdb fix in sight, so I'm simply voting for reintroducing the workaround. That's all. We can kill it or make it optional once the issue is solved. But we should _not_ do this _before_ it is solved, causing only pain to people who just want to use the gdbstub.
Quoted 2 lines
I don't buy this argument. You already have to tell gdb which mode to work in, so I don't see any reason why you shouldn't also tell qemu.
Sorry, this doesn't parse for me. What mechanism for qemu do you have in mind precisely?
However, do we agree now that we have to adopt qemu to make it work with current gdb?
Quoted 4 lines
Recent GDB interrogate the target when they connect, and the target describes the registers it has. This isn't currently implemented on x86 in qemu, but I'm pretty sure it'd be incompatible with dynamic switching.
I haven't checked gdb /wrt this protocol and x86 in details yet. But as far as I understand it, it does negotiation during startup. I wouldn't be surprised that all what the backend told gdb becomes pointless once you type "set arch i386". But this seems to be an orthogonal issue. Of course, we would have to communicate x86-64 capabilities this way, not the mode-dependent set.
Jan
Quoted 4 lines · Jan Kiszka
And that means setting
current_gdbarchwhile keepingtarget_gdbarch- that's where reality (existing gdb code) bites us. Again, I'm not arguing against fixing this, I'm arguing in keeping qemu's workaround until this is done. I will look into the gdb part, but one after the other.
No, it does not mean setting current_gdbarch different from target_gdbarch. With the current gdbarch set to a 64-bit one that accurately describes the target, GDB should be able to debug code running in 32-bit mode. If it can't, there are simply bugs in GDB to fix.
If you'd like to reach some solution to this problem, which I've seen come up on the QEMU list a half-dozen times now, please describe how you're using GDB on the [email protected] mailing list and let's see if we can't fix the GDB bugs. I'm pretty sure that any solution is going to involve always transferring the x86-64 register set, though.
Signature · 2 lines
Daniel Jacobowitz CodeSourcery
Quoted 11 lines · Daniel Jacobowitz
Quoted 4 lines · Jan Kiszka
And that means setting
current_gdbarchwhile keepingtarget_gdbarch- that's where reality (existing gdb code) bites us. Again, I'm not arguing against fixing this, I'm arguing in keeping qemu's workaround until this is done. I will look into the gdb part, but one after the other.No, it does not mean setting
current_gdbarchdifferent fromtarget_gdbarch. With the current gdbarch set to a 64-bit one that accurately describes the target, GDB should be able to debug code running in 32-bit mode. If it can't, there are simply bugs in GDB to fix.
Well, in the current gdb design, current_gdbarch is consulted when disassembling the code while target_gdbarch defines the register set that is exchanged with the remote stub.
Quoted 6 lines
If you'd like to reach some solution to this problem, which I've seen come up on the QEMU list a half-dozen times now, please describe how you're using GDB on the [email protected] mailing list and let's see if we can't fix the GDB bugs. I'm pretty sure that any solution is going to involve always transferring the x86-64 register set, though.
I'm pretty sure that the final solution will involve extended x86 register sets in order to inform the frontend about the full target CPU state so that it can set the right current_gdbarch automatically. That's one reason (the other is current/target_gdbarch decoupling) why I see no quick "bug fix" on the gdb side to actually solve the issue and suggest the reintroduction of the qemu workaround until gdb is enhanced appropriately.
But you I right, it's time to start a discussion on the gdb list, hopefully laying the ground for a better x86 low-level support. And Maybe I actually miss some smart intermediate step towards this.
Jan
Quoted 3 lines · Jan Kiszka
Well, in the current gdb design,
current_gdbarchis consulted when disassembling the code whiletarget_gdbarchdefines the register set that is exchanged with the remote stub.
This is a transitional state. Really, there isn't supposed to be a 'current' gdbarch; we're already moving away from it.
Thinking about it some more you may be right about the overall solution though, sorry. The target_gdbarch idea is likely to stick around for a while. But some work will have to be done if current and target architectures have different register sets :-(
Quoted 3 lines
I'm pretty sure that the final solution will involve extended x86 register sets in order to inform the frontend about the full target CPU state so that it can set the right
current_gdbarchautomatically.
Isn't everything we need for this in eflags already?
Signature · 2 lines
Daniel Jacobowitz CodeSourcery
Quoted 5 lines · Daniel Jacobowitz
Quoted 3 lines · Jan Kiszka
Well, in the current gdb design,
current_gdbarchis consulted when disassembling the code whiletarget_gdbarchdefines the register set that is exchanged with the remote stub.
Quoted 7 lines
This is a transitional state. Really, there isn't supposed to be a 'current' gdbarch; we're already moving away from it.
Thinking about it some more you may be right about the overall solution though, sorry. The
target_gdbarchidea is likely to stick around for a while. But some work will have to be done if current and target architectures have different register sets :-(
I'll start a thread on the gdb list today, CC'ing you. Would be nice if you could then add more details on what you think would be required to achieve this.
Quoted 6 lines
Quoted 4 lines
I'm pretty sure that the final solution will involve extended x86 register sets in order to inform the frontend about the full target CPU state so that it can set the right
current_gdbarchautomatically.Isn't everything we need for this in eflags already?
You need CR0.PE to detect if you are in real or protected mode. And then you need GDTR/LDTR to find the descriptor CS is pointing at, parsing it to detect if you are running 16, 32 or 64 bit code (by default). Those extensions would also be useful in order to decode memory addresses in case descriptor.base != 0 (or if it's CS >> 4, ie. you are in real mode). We have some usable patches for this @work, at least for 16 vs. 32 bit. But it's clear that more work is needed to get things upstream and we should first agree on how things should be done there, e.g. how to extend the register set and how to communicate that extension between backend and frontend.
Jan
Quoted 6 lines · Jan Kiszka
You need CR0.PE to detect if you are in real or protected mode. And then you need GDTR/LDTR to find the descriptor CS is pointing at, parsing it to detect if you are running 16, 32 or 64 bit code (by default). Those extensions would also be useful in order to decode memory addresses in case descriptor.base != 0 (or if it's CS >> 4, ie. you are in real mode).
If you're going to decode segment descriptors (great idea, btw, and helpful for threaded code), it might be better to supply the CPU's internal segment state, if that's possible, instead of looking at the LDT/GDT in memory, since the CPU's state can differ from the memory version when the latter is written to.
-- Jamie
Quoted 8 lines · Jamie Lokier
Quoted 6 lines · Jan Kiszka
You need CR0.PE to detect if you are in real or protected mode. And then you need GDTR/LDTR to find the descriptor CS is pointing at, parsing it to detect if you are running 16, 32 or 64 bit code (by default). Those extensions would also be useful in order to decode memory addresses in case descriptor.base != 0 (or if it's CS >> 4, ie. you are in real mode).
Quoted 5 lines
If you're going to decode segment descriptors (great idea, btw, and helpful for threaded code), it might be better to supply the CPU's internal segment state, if that's possible, instead of looking at the LDT/GDT in memory, since the CPU's state can differ from the memory version when the latter is written to.
Good point. I included this in an initial suggestion of an extended register set, see [1]. Providing this information will likely remain VM-business, but that doesn't mean we shouldn't use it when available.
Jan