Allow TAP to unnegotiate virtio-net header
cover Allow TAP to unnegotiate virtio-net header
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel, but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement in tap_receive_iov()). However, the tap_using_vnet_hdr() has an assert() that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net header
net/tap.c | 1 - 1 file changed, 1 deletion(-)
Signature · 1 line
2.20.1
PATCH 1/1 net: tap: allow net frontends to un-negotiate virtio-net header
The implementation of the TAP net backend already supports the case (s->host_vnet_hdr_len && !s->using_vnet_hdr), which means that the TAP device is expecting the header, while the net frontend (emulated device) is not aware of it. This case is handled by stripping or prepending the (zeroed) header on the fly. However, the function tap_using_vnet_hdr() has an assert() that explicitly prevents this situation to happen.
This patch removes the assert(), so that net frontends are free to un-negotiate the header.
--- net/tap.c | 1 - 1 file changed, 1 deletion(-)
net/tap.c +0 -1
@@ -264,7 +264,6 @@ static void tap_using_vnet_hdr(NetClientState *nc, bool using_vnet_hdr)TAPState *s =DO_UPCAST (TAPState, nc, nc);assert (nc->info->type ==NET_CLIENT_DRIVER_TAP ); -assert (!!s->host_vnet_hdr_len == using_vnet_hdr); s->using_vnet_hdr = using_vnet_hdr; }
Signature · 1 line
2.20.1
Quoted 8 lines · Vincenzo Maffione
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel,
Quoted 8 lines
but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement intap_receive_iov()). However, thetap_using_vnet_hdr()has anassert()that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.
Hi:
If I understand this correctly, your PV device is API compatible with TAP? Then you may just adding code to call TUNSETIFF without IFF_VNET_HDR?
Thanks
Quoted 7 lines
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net headernet/tap.c | 1 - 1 file changed, 1 deletion(-)
Hi Jason,
Thanks for the quick reply.
My PV device (to be open sourced soon) uses the QEMU net backend interface, in a way similar to virtio-net.
For example it uses qemu_set_offload(), qemu_has_vnet_hdr_len(), qemu_using_vnet_hdr(), qemu_send_packet(), etc.
This means that the device itself does not know which net backend is using, of course.
In addition to TAP, also the netmap backend supports the virtio-net header, and so the PV device will work with both.
Regarding the virtio-net header, the netmap backend is more flexible because you can change the virtio-net header length as you wish (choosing between 0, 10 and 12 bytes).
My problem cannot be solved by making sure that TAPs are opened without IFF_VNET_HDR. What I want instead
is to dynamically change the length of the virtio-net header that the backend accepts, switching between 12 bytes (virtio 1.0 header) and 0 (no header). By "dynamically" I mean while the guest (and the device) is running. This is possible and safe if we get rid of that assert(). The virtio-net device does not need that flexibility (once you enable the vtnet header, you never disable that again), so I guess that's why you have that assert().
My PV device can be reconfigured on the fly in such a way it may or not may be aware of the virtio-net header, and as a consequence it may need to call qemu_using_vnet_hdr() with true or false (multiple times).
Does it make sense?
Cheers,
Vincenzo
Il giorno lun 18 feb 2019 alle ore 05:19 Jason Wang <[email protected]> ha scritto:
Quoted 36 lines · Vincenzo Maffione
Quoted 8 lines
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel,Quoted 8 lines
but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement intap_receive_iov()). However, thetap_using_vnet_hdr()has anassert()that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.Hi:
If I understand this correctly, your PV device is API compatible with TAP? Then you may just adding code to call TUNSETIFF without
IFF_VNET_HDR?Thanks
Quoted 7 lines
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net headernet/tap.c | 1 - 1 file changed, 1 deletion(-)
Signature · 1 line
Vincenzo
Quoted 14 lines · Vincenzo Maffione
Hi Jason,
Thanks for the quick reply.My PV device (to be open sourced soon) uses the QEMU net backend interface, in a way similar to virtio-net.
For example it usesqemu_set_offload(),qemu_has_vnet_hdr_len(),qemu_using_vnet_hdr(),qemu_send_packet(), etc.
This means that the device itself does not know which net backend is using, of course.
In addition to TAP, also the netmap backend supports the virtio-net header, and so the PV device will work with both.
Regarding the virtio-net header, the netmap backend is more flexible because you can change the virtio-net header length as you wish (choosing between 0, 10 and 12 bytes).
Interesting, actually, there's case that vnet header is not used even for virtio-net, e.g for the case of XDP withouth csum support. If we don't pass vnet header in this case, we may get even higher PPS.
Quoted 14 lines
My problem cannot be solved by making sure that TAPs are opened without
IFF_VNET_HDR. What I want instead
is to dynamically change the length of the virtio-net header that the backend accepts, switching between 12 bytes (virtio 1.0 header) and 0 (no header). By "dynamically" I mean while the guest (and the device) is running. This is possible and safe if we get rid of thatassert(). The virtio-net device does not need that flexibility (once you enable the vtnet header, you never disable that again), so I guess that's why you have thatassert().
My PV device can be reconfigured on the fly in such a way it may or not may be aware of the virtio-net header, and as a consequence it may need to callqemu_using_vnet_hdr()with true or false (multiple times).
Quoted 1 line
Does it make sense?
Yes, it is. But I think maybe it's better to send this patch with your new PV device implementation.
Thanks
Quoted 41 lines
Cheers,
VincenzoIl giorno lun 18 feb 2019 alle ore 05:19 Jason Wang <[email protected]> ha scritto:
Quoted 32 lines · Vincenzo Maffione
Quoted 8 lines
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel,Quoted 8 lines
but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement intap_receive_iov()). However, thetap_using_vnet_hdr()has anassert()that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.Hi:
If I understand this correctly, your PV device is API compatible with TAP? Then you may just adding code to call TUNSETIFF without
IFF_VNET_HDR?Thanks
Quoted 6 lines
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net headernet/tap.c | 1 - 1 file changed, 1 deletion(-)
Il giorno mar 19 feb 2019 alle ore 07:48 Jason Wang <[email protected]> ha scritto:
Quoted 21 lines · Vincenzo Maffione
Quoted 4 lines
Hi Jason,
Thanks for the quick reply.My PV device (to be open sourced soon) uses the QEMU net backend
interface,
Quoted 4 lines
in a way similar to virtio-net.
For example it usesqemu_set_offload(),qemu_has_vnet_hdr_len(),qemu_using_vnet_hdr(),qemu_send_packet(), etc.
This means that the device itself does not know which net backend isusing,
Quoted 2 lines
of course.
In addition to TAP, also the netmap backend supports the virtio-netheader,
Quoted 4 lines
and so the PV device will work with both.
Regarding the virtio-net header, the netmap backend is more flexible because you can change the virtio-net header length as you wish (choosing between 0, 10 and 12 bytes).
Quoted 4 lines
Interesting, actually, there's case that vnet header is not used even for virtio-net, e.g for the case of XDP withouth csum support. If we don't pass vnet header in this case, we may get even higher PPS.
I guess in this case we simply pass a zeroed header. According to my experience I think it's unlikely that PPS are affected by having 12 additional bytes to copy around, as long as no processing is done on the header. I may be wrong of course.
Quoted 19 lines
Quoted 11 lines
My problem cannot be solved by making sure that TAPs are opened without
IFF_VNET_HDR. What I want instead
is to dynamically change the length of the virtio-net header that the backend accepts, switching between 12 bytes (virtio 1.0 header) and 0 (no header). By "dynamically" I mean while the guest (and the device) is running. This is possible and safe if we get rid of thatassert(). The virtio-net device does not need that flexibility (once you enable the vtnet header, you never disable that again), so I guess that's why youhave
Quoted 2 lines
that
assert().
My PV device can be reconfigured on the fly in such a way it may or notmay
Quoted 1 line
be aware of the virtio-net header, and as a consequence it may need to
call
Quoted 2 lines
qemu_using_vnet_hdr()with true or false (multiple times).
Quoted 7 lines
Quoted 1 line
Does it make sense?
Yes, it is. But I think maybe it's better to send this patch with your new PV device implementation.
Sounds good, thanks.
Cheers,
Vincenzo
Quoted 47 lines
Thanks
Quoted 31 lines
Cheers,
VincenzoIl giorno lun 18 feb 2019 alle ore 05:19 Jason Wang <[email protected]
ha scritto:
Quoted 22 lines · Vincenzo Maffione
Quoted 8 lines
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel,Quoted 8 lines
but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement intap_receive_iov()). However, thetap_using_vnet_hdr()has anassert()that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.Hi:
If I understand this correctly, your PV device is API compatible with TAP? Then you may just adding code to call TUNSETIFF without
IFF_VNET_HDR?Quoted 11 lines
Quoted 10 lines
Thanks
Quoted 6 lines
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net headernet/tap.c | 1 - 1 file changed, 1 deletion(-)
Signature · 1 line
Vincenzo
Quoted 22 lines · Vincenzo Maffione
Il giorno mar 19 feb 2019 alle ore 07:48 Jason Wang <[email protected]> ha scritto:
Quoted 19 lines · Vincenzo Maffione
Quoted 4 lines
Hi Jason,
Thanks for the quick reply.My PV device (to be open sourced soon) uses the QEMU net backend
interface,
Quoted 4 lines
in a way similar to virtio-net.
For example it usesqemu_set_offload(),qemu_has_vnet_hdr_len(),qemu_using_vnet_hdr(),qemu_send_packet(), etc.
This means that the device itself does not know which net backend isusing,
Quoted 2 lines
of course.
In addition to TAP, also the netmap backend supports the virtio-netheader,
Quoted 4 lines
and so the PV device will work with both.
Regarding the virtio-net header, the netmap backend is more flexible because you can change the virtio-net header length as you wish (choosing between 0, 10 and 12 bytes).
Quoted 8 lines
Quoted 4 lines
Interesting, actually, there's case that vnet header is not used even for virtio-net, e.g for the case of XDP withouth csum support. If we don't pass vnet header in this case, we may get even higher PPS.
I guess in this case we simply pass a zeroed header. According to my experience I think it's unlikely that PPS are affected by having 12 additional bytes to copy around, as long as no processing is done on the header. I may be wrong of course.
The problem is device doesn't know whether it was zeroed, so it need to examine some fields.
Thanks
Quoted 71 lines
Quoted 16 lines
Quoted 10 lines
My problem cannot be solved by making sure that TAPs are opened without
IFF_VNET_HDR. What I want instead
is to dynamically change the length of the virtio-net header that the backend accepts, switching between 12 bytes (virtio 1.0 header) and 0 (no header). By "dynamically" I mean while the guest (and the device) is running. This is possible and safe if we get rid of thatassert(). The virtio-net device does not need that flexibility (once you enable the vtnet header, you never disable that again), so I guess that's why youhave
Quoted 2 lines
that
assert().
My PV device can be reconfigured on the fly in such a way it may or notmay
Quoted 1 line
be aware of the virtio-net header, and as a consequence it may need to
call
Quoted 8 lines
Quoted 3 lines
qemu_using_vnet_hdr()with true or false (multiple times).Does it make sense?
Yes, it is. But I think maybe it's better to send this patch with your new PV device implementation.
Sounds good, thanks.
Cheers,
VincenzoQuoted 42 lines
Thanks
Quoted 4 lines
Cheers,
VincenzoQuoted 25 lines
Il giorno lun 18 feb 2019 alle ore 05:19 Jason Wang <[email protected]
ha scritto:
Quoted 21 lines · Vincenzo Maffione
Quoted 8 lines
Hi,
I was doing experiments with a custom paravirtualized net device,
and I ran into a limitation of the TAP backend. I see from the kernel code that it is not possible to set the TAP virtio-net header length to something different from 10 or 12, which means that it is not possible to set it to 0. That's fine.
The QEMU implementation of the TAP backend already supports the case where virtio-net header is set to 10 or 12 in kernel,Quoted 8 lines
but the emulated/paravirtualized net device is not using that:
the TAP backend will just strip/prepend a zeroed header in this case (e.g., see the if statement intap_receive_iov()). However, thetap_using_vnet_hdr()has anassert()that prevents this situation to happen, and I don't understand why. Maybe it is a leftover? I tried to remove the assert and by doing that my paravirtualized device was able to stop using the virtio-net header.Hi:
If I understand this correctly, your PV device is API compatible with TAP? Then you may just adding code to call TUNSETIFF without
IFF_VNET_HDR?Quoted 9 lines
Quoted 9 lines
Thanks
Quoted 6 lines
Vincenzo Maffione (1):
net: tap: allow net frontends to un-negotiate virtio-net header
net/tap.c| 1 -
1 file changed, 1 deletion(-)