Skip to contentSkip to conversation

virtio breakage with 2.6.25 guest kernel

Qemu-devel Alexander Graf 5 participants 8 messages Jan 13, 2009 – Jan 15, 2009 KVM

lore
Share & tools
lore
#0 lore raw

Hi,

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

Alex

#1 lore raw

Hi Alex,

(Dropping [email protected] - it's [email protected] now anyway)

Quoted 18 lines · Alexander Graf

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

Quoted 6 lines

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

Yeah, I noticed this one last week.

The issue here is that 2.6.25 virtio erroneously says it supports all features supported by the host - i.e. it's a guest kernel bug.

With recent qemu, that causes us to enable mergeable receive buffers and the guest barfs. See:

        mask = ioread32(vp_dev->ioaddr + VIRTIO_PCI_HOST_FEATURES);
        if (mask & (1 << bit)) {
                mask |= (1 << bit);
                iowrite32(mask, vp_dev->ioaddr + VIRTIO_PCI_GUEST_FEATURES);

This got fixed in 2.6.26 by:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=c45a6816

I did consider forwarding this to [email protected], but apparently the 2.6.25.x series is finished now anyway.

Cheers,
Mark.

#2 lore raw
Quoted 4 lines · Mark McLoughlin

Hi Alex,

(Dropping [email protected] - it's [email protected] now anyway)

Thunderbird keeps selecting that one and I'm sometimes too confused to see it's actually the wrong one.

Quoted 42 lines · Alexander Graf
Quoted 20 lines

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

Quoted 7 lines

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

Yeah, I noticed this one last week.

The issue here is that 2.6.25 virtio erroneously says it supports all features supported by the host - i.e. it's a guest kernel bug.

With recent qemu, that causes us to enable mergeable receive buffers and the guest barfs. See:

        mask = ioread32(vp_dev->ioaddr + VIRTIO_PCI_HOST_FEATURES);
        if (mask & (1 << bit)) {
                mask |= (1 << bit);
                iowrite32(mask, vp_dev->ioaddr + VIRTIO_PCI_GUEST_FEATURES);
Quoted 7 lines

This got fixed in 2.6.26 by:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=c45a6816

I did consider forwarding this to [email protected], but apparently the 2.6.25.x series is finished now anyway.

Ok, that's good to know. I don't think anyone officially supported virtio in 2.6.25 days, but I'll check internally if we need to backport anything.

Thanks a lot

Alex

#3 lore raw
In reply to Alexander Graf
Quoted 20 lines · Alexander Graf

Hi,

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

Quoted 7 lines

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

I think this was reported and fixed, possibly by c45a681.

Signature · 1 line
error compiling committee.c: too many arguments to function
#4 lore raw
In reply to Alexander Graf
Quoted 9 lines · Alexander Graf

Hi,

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

There were a couple of old-guest-breaking regressions. I think we've fixed all of them but there could be more. Are you using the latest kvm-userspace?

Regards,

Anthony Liguori

Quoted 20 lines

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

Alex

#5 lore raw
Quoted 5 lines · Anthony Liguori
Quoted 4 lines · Alexander Graf

Hi,

while I don't fully understand the problem, here's what I experience so far:

Quoted 7 lines
Quoted 5 lines

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's rewrite). It shows the following message (comes from qemu):

There were a couple of old-guest-breaking regressions. I think we've fixed all of them but there could be more. Are you using the latest kvm-userspace?

This one is definitely due to the broken guest kernel. I applied the patch mark mentioned to ours and things started working.

So the only way I can think of to 'fix' it is by detecting broken guests. We could supply a host mask of 0xffffffff and see if tge guest feature mask is tge same. If so, feature masking is probably broken.

Alex

Quoted 27 lines

Regards,

Anthony Liguori

Quoted 20 lines

virtio-net header not in first element

This is because qemu expects a message with mergeable rx bufs (12 bytes), but if I see things correctly the old kernel doesn't support that feature (sends 10 bytes). I put in some debug messages on IO reads/writes in the qemu virtio handler and got these:

virtio IO read: 0 = 0x100bba3
virtio IO write: 0x4 = 0x100bba3
virtio: setting features 0x100bba3

So I believe the feature masking is somewhat non-functional? Or did I read the masking code in qemu's virtio-net wrong?

Basically for me, current KVM git broke old guest support, which is a clear regression and should not happen. Does anybody with more expertise in virtio want to take on this?

Alex

#6 lore raw
Quoted 16 lines · Alexander Graf
Quoted 14 lines · Anthony Liguori
Quoted 12 lines · Alexander Graf

Hi,

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap
breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's
rewrite). It shows the following message (comes from qemu):

Quoted 3 lines
Quoted 3 lines

There were a couple of old-guest-breaking regressions. I think we've fixed all of them but there could be more. Are you using the latest kvm-userspace?

See:

http://lists.gnu.org/archive/html/qemu-devel/2009-01/msg00574.html

Quoted 6 lines

This one is definitely due to the broken guest kernel. I applied the patch mark mentioned to ours and things started working.

So the only way I can think of to 'fix' it is by detecting broken guests. We could supply a host mask of 0xffffffff and see if tge guest feature mask is tge same. If so, feature masking is probably broken.

Nice idea, but no way of making the guest through feature detection negotiation, I don't think.

Cheers,
Mark.

#7 lore raw
Quoted 29 lines · On Wednesday 14 January 2009 19:52:58 Mark McLoughlin wrote:
Quoted 16 lines · Alexander Graf
Quoted 14 lines · Anthony Liguori
Quoted 12 lines · Alexander Graf

Hi,

while I don't fully understand the problem, here's what I experience so far:

When using an openSUSE 11.0 kernel (2.6.25) in the guest, virtio on tap
breaks with current KVM git, while it used to work before (haven't bisected, definitely worked in kvm-78, but is probably due to Anthony's
rewrite). It shows the following message (comes from qemu):

Quoted 3 lines
Quoted 3 lines

There were a couple of old-guest-breaking regressions. I think we've fixed all of them but there could be more. Are you using the latest kvm-userspace?

See:

http://lists.gnu.org/archive/html/qemu-devel/2009-01/msg00574.html

Quoted 3 lines

This one is definitely due to the broken guest kernel. I applied the patch mark mentioned to ours and things started working.

Quoted 6 lines
Quoted 3 lines

So the only way I can think of to 'fix' it is by detecting broken guests. We could supply a host mask of 0xffffffff and see if tge guest feature mask is tge same. If so, feature masking is probably broken.

Nice idea, but no way of making the guest through feature detection negotiation, I don't think.

Just add a feature "VIRTIO_F_LIES_ABOUT_FEATURES"? No guest should ever set this.

But I'm not sure it's worth the pain...
Rusty.