virtio net requirements for 1.4
v4 cover virtio net requirements for 1.4
Hi All,
This document captures the virtio net device requirements for the upcoming release 1.4 that some of us are currently working on.
This is live document to be updated in coming time and work towards it for its design which can result in a draft specification.
The objectives are:
1. to consider these requirements in introducing new features listed in the document and otherwise and work towards the interface design followed by drafting the specification changes.
2. Define practical list of requirements that can be achieved in 1.4 timeframe incrementally and also have the ability to implement them.
Please review mainly patch 5 at the priority.
Receive flow filters is the first item apart from counters to complete in this iteration to start drafting the design spec.
Rest of the requirements are largly untouched other than Stefan's comment.
TODO:
1. Some more refinement needed for rx low latency and header data split requirements.
---
note:
This week update is late as I was unwell for several days, but managed to cover all comments from Heng, David, Satananda.
We must close flow filters requirements this week to take to spec drafting design phase.
---
changelog:v3->v4:
- receive flow filters requirements undergo major updates to take to spec draft level.
- Addressed comments from Xuan, Heng, David, Satananda.
- Refined wordings in rest of the requirementsv2->v3:
- addressed comments from Stefan for tx low latency and notification
- redrafted the requirements to use rearm term and avoid queue enable confusion for notification
- addressed all comments and refined receive flow filters requirements to take to design levelv1->v2:
- major update of receive flow filter requirements updated based on last two design discussions in community and offline research
- examples added
- link to use case and design goal added
- control and operation side requirements split
- more verbosev0->v1:
- addressed comments from Heng Li
- addressed few (not all) comments from Michael
- per patch changelog
Parav Pandit (7):
net-features: Add requirements document for release 1.4
net-features: Add low latency transmit queue requirements
net-features: Add low latency receive queue requirements
net-features: Add notification coalescing requirements
net-features: Add n-tuple receive flow filters requirements
net-features: Add packet timestamp requirements
net-features: Add header data split requirements
net-workstream/features-1.4.md | 375 +++++++++++++++++++++++++++++++++ 1 file changed, 375 insertions(+) create mode 100644 net-workstream/features-1.4.md
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v4 1/7 net-features: Add requirements document for release 1.4
Add requirements document template for the virtio net features.
Add virtio net device counters visible to driver.
---
changelog:v3->v4:
- Addressed comment from David
- Added link to more counters that we are already discussing v0->v1:
- removed tx dropped counter
- updated requirements to mention about virtqueue interface for counters query
--- net-workstream/features-1.4.md | 43 ++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 net-workstream/features-1.4.md
net-workstream/features-1.4.md new file +43 -0
@@ -0,0 +1,43 @@ +# 1. Introduction + +This document describes the overall requirements for virtio net device +improvements for upcoming release 1.4. Some of these requirements are +interrelated and influence the interface design, hence reviewing them +together is desired while updating the virtio net interface. + +# 2. Summary +1. Device counters visible to the driver + +# 3. Requirements +## 3.1 Device counters +1. The driver should be able to query the device and/or per vq counters for + debugging purpose using a virtqueue directly from driver to device for + example using a control vq. +2. The driver should be able to query which counters are supported using a + virtqueue command, for example using an existing control vq. +3. If this device is migrated between two hosts, the driver should be able + get the counter values in the destination host from where it was left + off in the source host. +4. If a virtio device is group member device, a group owner should be able + to query all the counter attributes using the administration command which + a virtio member device will expose via a virtqueue to the driver. +4. If a virtio device is a group member device, a group owner should be able + to query all of the member device counter attributes and counters via the + group owner device. + +### 3.1.1 Per receive queue counters +1. le64 rx_oversize_pkt_errors: Packet dropped due to receive packet being + oversize than the buffer size +2. le64 rx_no_buffer_pkt_errors: Packet dropped due to unavailability of the + buffer in the receive queue +3. le64 rx_gso_pkts: Packets treated as receive GSO sequence by the device +4. le64 rx_pkts: Total packets received by the device + +### 3.1.2 Per transmit queue counters +1. le64 tx_gso_pkts: Packets send as transmit GSO sequence +2. le64 tx_pkts: Total packets send by the device + +### 3.1.3 More counters +More counters discussed in [1]. + +[1] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00176.html
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH requirements v4 1/7] net-features: Add requirements document for release 1.4
Quoted 38 lines · Parav Pandit
Add requirements document template for the virtio net features.
Add virtio net device counters visible to driver.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comment from David
- Added link to more counters that we are already discussingv0->v1:
- removed tx dropped counter
- updated requirements to mention about virtqueue interface for counters query--- net-workstream/features-1.4.md | 43 ++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 net-workstream/features-1.4.mdnet-workstream/features-1.4.md new file +13
-0@@ -0,0 +1,43 @@ +# 1. Introduction + +This document describes the overall requirements for virtio net device +improvements for upcoming release 1.4. Some of these requirements are +interrelated and influence the interface design, hence reviewing them +together is desired while updating the virtio net interface. + +# 2. Summary +1. Device counters visible to the driver + +# 3. Requirements +## 3.1 Device counters +1. The driver should be able to query the device and/or per vq counters for
net-workstream/features-1.4.md line 23
Quoted 10 lines
net-workstream/features-1.4.md new file +10
-0+ debugging purpose using a virtqueue directly from driver to device for + example using a control vq. +2. The driver should be able to query which counters are supported using a + virtqueue command, for example using an existing control vq. +3. If this device is migrated between two hosts, the driver should be able + get the counter values in the destination host from where it was left + off in the source host. +4. If a virtio device is group member device, a group owner should be able + to query all the counter attributes using the administration command which + a virtio member device will expose via a virtqueue to the driver.
The language seems confusing here. How about:
If a virtio device is a group member device, it must be possible to query all of the group member counter attributes via the group owner device.
net-workstream/features-1.4.md line 26
Quoted 3 lines
net-workstream/features-1.4.md new file +3
-0+4. If a virtio device is a group member device, a group owner should be able + to query all of the member device counter attributes and counters via the + group owner device.
Similarly:
If a virtio device is a group member device, it must be possible to query all of the group member counters via the group owner device.
Quoted 17 lines
net-workstream/features-1.4.md new file +17
-0+ +### 3.1.1 Per receive queue counters +1. le64 rx_oversize_pkt_errors: Packet dropped due to receive packet being + oversize than the buffer size +2. le64 rx_no_buffer_pkt_errors: Packet dropped due to unavailability of the + buffer in the receive queue +3. le64 rx_gso_pkts: Packets treated as receive GSO sequence by the device +4. le64 rx_pkts: Total packets received by the device + +### 3.1.2 Per transmit queue counters +1. le64 tx_gso_pkts: Packets send as transmit GSO sequence +2. le64 tx_pkts: Total packets send by the device + +### 3.1.3 More counters +More counters discussed in [1]. + +[1] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00176.html
Signature · 17 lines
It's gettin', it's gettin', it's gettin' kinda hectic. This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
RE: [virtio-comment] [PATCH requirements v4 1/7] net-features: Add requirements document for release 1.4
Quoted 15 lines
From: David Edmondson <[email protected]>
Sent: Tuesday, August 15, 2023 2:26 PMQuoted 2 lines
+4. If a virtio device is group member device, a group owner should be able
+ to query all the counter attributes using the administration commandwhich
Quoted 1 line
+ a virtio member device will expose via a virtqueue to the driver.
The language seems confusing here. How about:
If a virtio device is a group member device, it must be possible to query all of the group member counter attributes via the group owner device.
Quoted 9 lines
net-workstream/features-1.4.md line 26
Quoted 3 lines
+4. If a virtio device is a group member device, a group owner should be able
net-workstream/features-1.4.md new file +2
-0+ to query all of the member device counter attributes and counters via the + group owner device.Similarly:
If a virtio device is a group member device, it must be possible to query all of the group member counters via the group owner device.
I messed up this point 4 while addressing your comments from v3.
How about,
If a virtio device is a group member device, it must be possible to query supported counters bitmap through the group owner device. The group owner driver should be able to query such bitmap using administration command.
Notice that group owner only cares for the supported bitmap, no need to query the counters through this interface.
With this we can reuse single structure across AQ and CVQ to learn about supported bitmap in v14 patch from Xuan.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
Re: [virtio-comment] [PATCH requirements v4 1/7] net-features: Add requirements document for release 1.4
Quoted 23 lines · Parav Pandit
Quoted 14 lines
From: David Edmondson <[email protected]>
Sent: Tuesday, August 15, 2023 2:26 PMQuoted 2 lines
+4. If a virtio device is group member device, a group owner should be able
+ to query all the counter attributes using the administration commandwhich
Quoted 1 line
+ a virtio member device will expose via a virtqueue to the driver.
The language seems confusing here. How about:
If a virtio device is a group member device, it must be possible to query all of the group member counter attributes via the group owner device.
Quoted 9 lines
net-workstream/features-1.4.md line 26
Quoted 3 lines
+4. If a virtio device is a group member device, a group owner should be able
net-workstream/features-1.4.md new file +2
-0+ to query all of the member device counter attributes and counters via the + group owner device.Similarly:
If a virtio device is a group member device, it must be possible to query all of the group member counters via the group owner device.
Quoted 8 lines
I messed up this point 4 while addressing your comments from v3.
How about,
If a virtio device is a group member device, it must be possible to query supported counters bitmap through the group owner device. The group owner driver should be able to query such bitmap using administration command.
Notice that group owner only cares for the supported bitmap, no need to query the counters through this interface.
Why do we think this is the case? It would be useful for a group owner to be able to see the statistics as they are being reported to the member consumer.
Quoted 1 line
With this we can reuse single structure across AQ and CVQ to learn about supported bitmap in v14 patch from Xuan.
Makes sense.
Signature · 17 lines
I'm not living in the real world, no more, no more. This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
RE: [virtio-comment] [PATCH requirements v4 1/7] net-features: Add requirements document for release 1.4
Quoted 7 lines
From: David Edmondson <[email protected]>
Sent: Tuesday, August 15, 2023 2:44 PMQuoted 2 lines
If a virtio device is a group member device, it must be possible to query
supported counters bitmap through the group owner device.
Quoted 9 lines
Quoted 1 line
The group owner driver should be able to query such bitmap using
administration command.
Quoted 2 lines
Notice that group owner only cares for the supported bitmap, no need to
query the counters through this interface.
Why do we think this is the case? It would be useful for a group owner to be able to see the statistics as they are being reported to the member consumer.
I agree, it is useful.
For LM scenario, ability to query supported bit is minimum needed, actual counters is nice to have. Usually the actual counters are located where the switch is located.
Quoted 6 lines
Quoted 1 line
With this we can reuse single structure across AQ and CVQ to learn about
supported bitmap in v14 patch from Xuan.
Makes sense.
--
I'm not living in the real world, no more, no more.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
PATCH v4 2/7 net-features: Add low latency transmit queue requirements
In reply to Parav PanditAdd requirements for the low latency transmit queue.
---
chagelog:v3->v4:
- Addressed comments from David
- rewrote timestamp and completions pcie transcation requirement v1->v2:
- added generic requirement to inline the request content along with the descriptor for non virtio-net devices
- added requirement to inline the header content along with the descriptor for virtio flow filter queue as two features are similarv0->v1:
- added design goals for which requirements are added
--- net-workstream/features-1.4.md | 88 ++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+)
net-workstream/features-1.4.md +88 -0
@@ -7,6 +7,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver +2. Low latency tx virtqueue for PCI transport # 3. Requirements ## 3.1 Device counters
@@ -41,3 +42,90 @@ together is desired while updating the virtio net interface. More counters discussed in [1]. [1] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00176.html + +## 3.2 Low PCI latency virtqueues +### 3.2.1 Low PCI latency tx virtqueue +0. Design goal + a. Reduce PCI access latency in packet transmit flow + b. Avoid O(N) descriptor parser to detect a packet stream to simplify device + logic + c. Reduce number of PCI transmit completion transactions and have unified + completion flow with/without transmit timestamping + d. Avoid partial cache line writes on transmit completions + +1. Packet transmit descriptor should contain data descriptors count without any + indirection and without any O(N) search to find the end of a packet stream. + For example, a packet transmit descriptor (called vnet_tx_hdr_desc + subsequently) to contain a field num_next_desc for the packet stream + indicating that a packet is located in N data descriptors. + +2. Packet transmit descriptor should contain segmentation offload-related fields + without any indirection. For example, packet transmit descriptor to contain + gso_type, gso_size/mss, header length, csum placement byte offset, and + csum start. + +3. Packet transmit descriptor should be able to place a small size packet that + does not have any L4 data after the vnet_tx_hdr_desc in the virtqueue memory. + For example a TCP ack only packet can fit in a descriptor memory which + otherwise consume more than 25% of metadata to describe the packet. + +4. Packet transmit descriptor should be able to place a full GSO header (L2 to + L4) after header descriptor and before data descriptors. For example, the + GSO header is placed after struct vnet_tx_hdr_desc in the virtqueue memory. + When such a GSO header is positioned adjacent to the packet transmit + descriptor, and when the GSO header is not aligned to 16B, the following + data descriptor to start on the 8B aligned boundary. + +5. An example of the above requirements at high level is: + +``` +struct virtio_packed_q_desc { + /* current desc for reference */ + u64 address; + u32 len; + u16 id; + u16 flags; +}; + +/* Constant size header descriptor for tx packets */ +struct vnet_tx_hdr_desc { + u16 flags; /* indicate how to parse next fields */ + u16 id; /* desc id to come back in completion */ + u8 num_next_desc; /* indicates the number of the next 16B data desc for this + * buffer. + */ + u8 gso_type; + le16 gso_hdr_len; + le16 gso_size; + le16 csum_start; + le16 csum_offset; + u8 inline_pkt_len; /* indicates the length of the inline packet after this + * desc + */ + u8 reserved; + u8 padding[]; +}; + +/* Example of a short packet or GSO header placed in the desc section of the vq + */ +struct vnet_tx_small_pkt_desc { + u8 raw_pkt[128]; +}; + +/* Example of header followed by data descriptor */ +struct vnet_tx_hdr_desc hdr_desc; +struct vnet_data_desc desc[2]; + +``` + +6. Ability to zero pad the transmit completion when the transmit completion is + shorter than the CPU cache line size. + +7. Ability to write per packet timestamp and also write multiple + transmit completions using single PCIe transcation. + +8. A generic feature of the virtqueue, to contain such header data inline for virtio + devices other than virtio-net. + +9. A flow filter virtqueue also similarly need the ability to inline the short flow + command header.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH requirements v4 2/7] net-features: Add low latency transmit queue requirements
Quoted 3 lines · Parav Pandit
Add requirements for the low latency transmit queue.
Signed-off-by Parav Pandit <[email protected]>
Quoted 49 lines
---
chagelog:v3->v4:
- Addressed comments from David
- rewrote timestamp and completions pcie transcation requirementv1->v2:
- added generic requirement to inline the request content along with the descriptor for non virtio-net devices
- added requirement to inline the header content along with the descriptor for virtio flow filter queue as two features are similarv0->v1:
- added design goals for which requirements are added--- net-workstream/features-1.4.md | 88 ++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+)net-workstream/features-1.4.md +17
-0@@ -7,6 +7,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver +2. Low latency tx virtqueue for PCI transport # 3. Requirements ## 3.1 Device counters@@ -41,3 +42,90 @@ together is desired while updating the virtio net interface. More counters discussed in [1]. [1] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00176.html + +## 3.2 Low PCI latency virtqueues +### 3.2.1 Low PCI latency tx virtqueue +0. Design goal + a. Reduce PCI access latency in packet transmit flow + b. Avoid O(N) descriptor parser to detect a packet stream to simplify device + logic + c. Reduce number of PCI transmit completion transactions and have unified + completion flow with/without transmit timestamping + d. Avoid partial cache line writes on transmit completions + +1. Packet transmit descriptor should contain data descriptors count without any + indirection and without any O(N) search to find the end of a packet stream. + For example, a packet transmit descriptor (called vnet_tx_hdr_desc + subsequently) to contain a field num_next_desc for the packet stream + indicating that a packet is located in N data descriptors.
net-workstream/features-1.4.md together is desired while updating the virtio net interface. · line 70
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ +2. Packet transmit descriptor should contain segmentation offload-related fields + without any indirection. For example, packet transmit descriptor to contain + gso_type, gso_size/mss, header length, csum placement byte offset, and + csum start. + +3. Packet transmit descriptor should be able to place a small size packet that + does not have any L4 data after the vnet_tx_hdr_desc in the virtqueue memory. + For example a TCP ack only packet can fit in a descriptor memory which + otherwise consume more than 25% of metadata to describe the packet.
The presence of "L4 data" seems like a distraction - just give an indication of size ranges being considered.
Quoted 61 lines
net-workstream/features-1.4.md +61
-0+ +4. Packet transmit descriptor should be able to place a full GSO header (L2 to + L4) after header descriptor and before data descriptors. For example, the + GSO header is placed after struct vnet_tx_hdr_desc in the virtqueue memory. + When such a GSO header is positioned adjacent to the packet transmit + descriptor, and when the GSO header is not aligned to 16B, the following + data descriptor to start on the 8B aligned boundary. + +5. An example of the above requirements at high level is: + +``` +struct virtio_packed_q_desc { + /* current desc for reference */ + u64 address; + u32 len; + u16 id; + u16 flags; +}; + +/* Constant size header descriptor for tx packets */ +struct vnet_tx_hdr_desc { + u16 flags; /* indicate how to parse next fields */ + u16 id; /* desc id to come back in completion */ + u8 num_next_desc; /* indicates the number of the next 16B data desc for this + * buffer. + */ + u8 gso_type; + le16 gso_hdr_len; + le16 gso_size; + le16 csum_start; + le16 csum_offset; + u8 inline_pkt_len; /* indicates the length of the inline packet after this + * desc + */ + u8 reserved; + u8 padding[]; +}; + +/* Example of a short packet or GSO header placed in the desc section of the vq + */ +struct vnet_tx_small_pkt_desc { + u8 raw_pkt[128]; +}; + +/* Example of header followed by data descriptor */ +struct vnet_tx_hdr_desc hdr_desc; +struct vnet_data_desc desc[2]; + +``` + +6. Ability to zero pad the transmit completion when the transmit completion is + shorter than the CPU cache line size. + +7. Ability to write per packet timestamp and also write multiple + transmit completions using single PCIe transcation. + +8. A generic feature of the virtqueue, to contain such header data inline for virtio + devices other than virtio-net. + +9. A flow filter virtqueue also similarly need the ability to inline the short flow + command header.
Signature · 17 lines
And you're standing here beside me, I love the passing of time. This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 2/7] net-features: Add low latency transmit queue requirements
Quoted 4 lines
From: David Edmondson <[email protected]>
Sent: Tuesday, August 15, 2023 2:15 PM
Quoted 9 lines
Quoted 2 lines
+3. Packet transmit descriptor should be able to place a small size packet that
+ does not have any L4 data after thevnet_tx_hdr_descin the virtqueuememory.
net-workstream/features-1.4.md together is desired while updating the virtio net interface. · line 70
Quoted 2 lines
net-workstream/features-1.4.md +2
-0+ For example a TCP ack only packet can fit in a descriptor memory which + otherwise consume more than 25% of metadata to describe the packet.The presence of "L4 data" seems like a distraction - just give an indication of size ranges being considered.
Yes, its mainly the size consideration.
Will update.
Thanks.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
PATCH v4 3/7 net-features: Add low latency receive queue requirements
In reply to Parav PanditAdd requirements for the low latency receive queue.
---
changelog:v0->v1:
- clarified the requirements further
- added line for the gro case
- added design goals as the motivation for the requirements
--- net-workstream/features-1.4.md | 45 +++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-)
net-workstream/features-1.4.md +44 -1
@@ -7,7 +7,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver -2. Low latency tx virtqueue for PCI transport +2. Low latency tx and rx virtqueues for PCI transport # 3. Requirements ## 3.1 Device counters
@@ -129,3 +129,46 @@ struct vnet_data_desc desc[2]; 9. A flow filter virtqueue also similarly need the ability to inline the short flow command header. + +### 3.2.2 Low latency rx virtqueue +0. Design goal: + a. Keep packet metadata and buffer data together which is consumed by driver + layer and make it available in a single cache line of cpu + b. Instead of having per packet descriptors which is complex to scale for + the device, supply the page directly to the device to consume it based + on packet size +1. The device should be able to write a packet receive completion that consists + of struct virtio_net_hdr (or similar) and a buffer id using a single DMA write + PCIe TLP. +2. The device should be able to perform DMA writes of multiple packets + completions in a single DMA transaction up to the PCIe maximum write limit + in a transaction. +3. The device should be able to zero pad packet write completion to align it to + 64B or CPU cache line size whenever possible. +4. An example of the above DMA completion structure: + +``` +/* Constant size receive packet completion */ +struct vnet_rx_completion { + u16 flags; + u16 id; /* buffer id */ + u8 gso_type; + u8 reserved[3]; + le16 gso_hdr_len; + le16 gso_size; + le16 csum_start; + le16 csum_offset; + u16 reserved2; + u64 timestamp; /* explained later */ + u8 padding[]; +}; +``` +5. The driver should be able to post constant-size buffer pages on a receive + queue which can be consumed by the device for an incoming packet of any size + from 64B to 9K bytes. +6. The device should be able to know the constant buffer size at receive + virtqueue level instead of per buffer level. +7. The device should be able to indicate when a full page buffer is consumed, + which can be recycled by the driver when the packets from the completed + page is fully consumed. +8. The device should be able to consume multiple pages for a receive GSO stream.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH requirements v4 3/7] net-features: Add low latency receive queue requirements
Quoted 27 lines · Parav Pandit
Add requirements for the low latency receive queue.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v0->v1:
- clarified the requirements further
- added line for the gro case
- added design goals as the motivation for the requirements--- net-workstream/features-1.4.md | 45 +++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-)net-workstream/features-1.4.md +1
-1@@ -7,7 +7,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver -2. Low latency tx virtqueue for PCI transport +2. Low latency tx and rx virtqueues for PCI transport # 3. Requirements ## 3.1 Device counters
net-workstream/features-1.4.md struct vnet_data_desc desc[2]; · line 137
Quoted 9 lines
net-workstream/features-1.4.md +6
-0@@ -129,3 +129,46 @@ struct vnet_data_desc desc[2]; 9. A flow filter virtqueue also similarly need the ability to inline the short flow command header. + +### 3.2.2 Low latency rx virtqueue +0. Design goal: + a. Keep packet metadata and buffer data together which is consumed by driver + layer and make it available in a single cache line of cpu + b. Instead of having per packet descriptors which is complex to scale for
This really is "per buffer" rather than "per packet".
Quoted 37 lines
net-workstream/features-1.4.md +37
-0+ the device, supply the page directly to the device to consume it based + on packet size +1. The device should be able to write a packet receive completion that consists + of struct virtio_net_hdr (or similar) and a buffer id using a single DMA write + PCIe TLP. +2. The device should be able to perform DMA writes of multiple packets + completions in a single DMA transaction up to the PCIe maximum write limit + in a transaction. +3. The device should be able to zero pad packet write completion to align it to + 64B or CPU cache line size whenever possible. +4. An example of the above DMA completion structure: + +``` +/* Constant size receive packet completion */ +struct vnet_rx_completion { + u16 flags; + u16 id; /* buffer id */ + u8 gso_type; + u8 reserved[3]; + le16 gso_hdr_len; + le16 gso_size; + le16 csum_start; + le16 csum_offset; + u16 reserved2; + u64 timestamp; /* explained later */ + u8 padding[]; +}; +``` +5. The driver should be able to post constant-size buffer pages on a receive + queue which can be consumed by the device for an incoming packet of any size + from 64B to 9K bytes. +6. The device should be able to know the constant buffer size at receive + virtqueue level instead of per buffer level. +7. The device should be able to indicate when a full page buffer is consumed, + which can be recycled by the driver when the packets from the completed + page is fully consumed. +8. The device should be able to consume multiple pages for a receive GSO stream.
Signature · 17 lines
Swimming around in a plastic bag. This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v4 4/7 net-features: Add notification coalescing requirements
In reply to Parav PanditAdd virtio net device notification coalescing improvements requirements.
---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description
--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)
net-workstream/features-1.4.md +11 -0
@@ -8,6 +8,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device counters
@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from the completed page is fully consumed. 8. The device should be able to consume multiple pages for a receive GSO stream. + +## 3.3 Virtqueue notification coalescing re-arming support +0. Design goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and when such + a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 36 lines · Parav Pandit
Add virtio net device notification coalescing improvements requirements.
Signed-off-by Parav Pandit <[email protected]>Acked-by David Edmondson <[email protected]>---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)net-workstream/features-1.4.md +1
-0@@ -8,6 +8,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device counters@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from the completed page is fully consumed. 8. The device should be able to consume multiple pages for a receive GSO stream.
net-workstream/features-1.4.md struct vnet_rx_completion { · line 185
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ +## 3.3 Virtqueue notification coalescing re-arming support +0. Design goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and when such + a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.
I'm wondering how this relates to the existing notification coalesing[1] and notification suppression[2]:
[1]
The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}.
[2]
If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:
\begin{itemize}
\item The driver MUST ignore the \field{avail_event} value.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If \field{flags} is 1, the driver SHOULD NOT send a notification. \item If \field{flags} is 0, the driver MUST send a notification. \end{itemize}
\end{itemize}
Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:
\begin{itemize}
\item The driver MUST ignore the lower bit of \field{flags}.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If the \field{idx} field in the available ring (which determined where that descriptor index was placed) was equal to \field{avail_event}, the driver MUST send a notification. \item Otherwise the driver SHOULD NOT send a notification. \end{itemize}
\end{itemize}
Regarding notification suppression:
1.When there is VIRTIO_NET_F_EVENT_IDX, even if the notification coalesing condition is met, we need to wait for
the used_event notification condition to be met(the driver does not rearms the notification of the virtqueue now and the avail ring is set VRING_AVAIL_F_NO_INTERRUPT in flag). 2.When there is no VIRTIO_NET_F_EVENT_IDX, if the driver turns off the notification, even if the notidication condition is met, the device cannot send the notification.
Therefore, if I'm not wrong, a device can issue a notification only if the device is not suppressed from notifying the driver. [1][2] seems to have met this condition.
Thanks!
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 96 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 2:01 PMQuoted 33 lines · Parav Pandit
Add virtio net device notification coalescing improvements requirements.
Signed-off-by Parav Pandit <[email protected]>Acked-by David Edmondson <[email protected]>---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex 72d04bd..cb72442 100644net-workstream/features-1.4.md +1
-0@@ -8,6 +8,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device countersnet-workstream/features-1.4.md struct vnet_rx_completion { · line 175
Quoted 3 lines
net-workstream/features-1.4.md +0
-0@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from the completed page is fully consumed. 8. The device should be able to consume multiple pages for a receive GSOstream.
net-workstream/features-1.4.md struct vnet_rx_completion {
Quoted 6 lines
net-workstream/features-1.4.md +6
-0+ +## 3.3 Virtqueue notification coalescing re-arming support 0. Design +goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and whensuch
net-workstream/features-1.4.md struct vnet_rx_completion { · line 185
Quoted 4 lines
net-workstream/features-1.4.md +4
-0+ a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.I'm wondering how this relates to the existing notification coalesing[1] and notification suppression[2]:
[1]
The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}.[2]
If theVIRTIO_F_EVENT_IDXfeature bit is not negotiated:
\begin{itemize}
\item The driver MUST ignore the \field{avail_event} value.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If \field{flags} is 1, the driver SHOULD NOT send a notification. \item If \field{flags} is 0, the driver MUST send a notification. \end{itemize}
\end{itemize}Otherwise, if the
VIRTIO_F_EVENT_IDXfeature bit is negotiated:
\begin{itemize}
\item The driver MUST ignore the lower bit of \field{flags}.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If the \field{idx} field in the available ring (which determined where that descriptor index was placed) was equal to \field{avail_event}, the driver MUST send a notification. \item Otherwise the driver SHOULD NOT send a notification. \end{itemize}
\end{itemize}Regarding notification suppression:
1.When there isVIRTIO_NET_F_EVENT_IDX, even if the notification coalesing condition is met, we need to wait for theused_eventnotification condition to be met(the driver does not rearms the notification of the virtqueue now and the avail ring is setVRING_AVAIL_F_NO_INTERRUPTin flag).
2.When there is noVIRTIO_NET_F_EVENT_IDX, if the driver turns off the notification, even if the notidication condition is met, the device cannot send the notification.
Quoted 3 lines
Therefore, if I'm not wrong, a device can issue a notification only if the device is not suppressed from notifying the driver.
[1][2] seems to have met this condition.
Notification suppression using _EVENT_IDX for non-memory transport is just sub-optimal for two reasons.
1. It requires device to poll on the used event to learn about when to un-suppress. (arm)
2. this bit also controls driver notifications yet again demand device to arbitrarily poll on new descriptors posting
Hence, an efficient scheme is needed and device notifications to be detached from driver notification.
And now that VQ level notification coalescing is in place, which suppresses the device notifications, it is logical to combine it with VQ device notifications.
[virtio-comment] Re: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 97 lines · Parav Pandit
Quoted 94 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 2:01 PMQuoted 33 lines · Parav Pandit
Add virtio net device notification coalescing improvements requirements.
Signed-off-by Parav Pandit <[email protected]>Acked-by David Edmondson <[email protected]>---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex 72d04bd..cb72442 100644net-workstream/features-1.4.md +1
-0@@ -8,6 +8,7 @@ together is desired while updating the virtio net interface. # 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device countersnet-workstream/features-1.4.md struct vnet_rx_completion { · line 175
Quoted 3 lines
net-workstream/features-1.4.md +0
-0@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from the completed page is fully consumed. 8. The device should be able to consume multiple pages for a receive GSOstream.
net-workstream/features-1.4.md struct vnet_rx_completion {
Quoted 6 lines
net-workstream/features-1.4.md +6
-0+ +## 3.3 Virtqueue notification coalescing re-arming support 0. Design +goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and whensuch
net-workstream/features-1.4.md struct vnet_rx_completion { · line 185
Quoted 4 lines
net-workstream/features-1.4.md +4
-0+ a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.I'm wondering how this relates to the existing notification coalesing[1] and notification suppression[2]:
[1]
The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}.[2]
If theVIRTIO_F_EVENT_IDXfeature bit is not negotiated:
\begin{itemize}
\item The driver MUST ignore the \field{avail_event} value.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If \field{flags} is 1, the driver SHOULD NOT send a notification. \item If \field{flags} is 0, the driver MUST send a notification. \end{itemize}
\end{itemize}Otherwise, if the
VIRTIO_F_EVENT_IDXfeature bit is negotiated:
\begin{itemize}
\item The driver MUST ignore the lower bit of \field{flags}.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If the \field{idx} field in the available ring (which determined where that descriptor index was placed) was equal to \field{avail_event}, the driver MUST send a notification. \item Otherwise the driver SHOULD NOT send a notification. \end{itemize}
\end{itemize}Regarding notification suppression:
1.When there isVIRTIO_NET_F_EVENT_IDX, even if the notification coalesing condition is met, we need to wait for theused_eventnotification condition to be met(the driver does not rearms the notification of the virtqueue now and the avail ring is setVRING_AVAIL_F_NO_INTERRUPTin flag).
2.When there is noVIRTIO_NET_F_EVENT_IDX, if the driver turns off the notification, even if the notidication condition is met, the device cannot send the notification.Quoted 3 lines
Therefore, if I'm not wrong, a device can issue a notification only if the device is not suppressed from notifying the driver.
[1][2] seems to have met this condition.
Quoted 8 lines
Notification suppression using
_EVENT_IDXfor non-memory transport is just sub-optimal for two reasons.1. It requires device to poll on the used event to learn about when to un-suppress. (arm)
2. this bit also controls driver notifications yet again demand device to arbitrarily poll on new descriptors postingHence, an efficient scheme is needed and device notifications to be detached from driver notification.
And now that VQ level notification coalescing is in place, which suppresses the device notifications, it is logical to combine it with VQ device notifications.
Let me summarize:
1. When used idx notification is satisfied, but coalescing is not satisfied, the driver continues to suppress device notifications.
2. When used idx notification is not satisfied, even if coalescing is satisfied, the device still cannot notify the driver. I think that's what coalescing does, and the description below has satisfied this behavior:
"The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}."
Or we want to say that it has nothing to do with the used idx notification. When the coalescing is satisfied and the driver rearms the notification of the virtqueue, the device now send a notification.
Thanks!
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 118 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 6:07 PMQuoted 25 lines · Parav Pandit
Quoted 25 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 2:01 PMQuoted 20 lines · Parav Pandit
Add virtio net device notification coalescing improvements requirements.
Signed-off-by Parav Pandit <[email protected]>Acked-by David Edmondson <[email protected]>---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)Quoted 5 lines
Quoted 5 lines
net-workstream/features-1.4.md together is desired while updating the virtio net
Quoted 5 lines
(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex 72d04bd..cb72442 100644net-workstream/features-1.4.md +0
-0@@ -8,6 +8,7 @@ together is desired while updating the virtio netinterface.
Quoted 9 lines
Quoted 9 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 173
Quoted 9 lines
net-workstream/features-1.4.md +1
-0# 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device counters@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from thecompleted
Quoted 49 lines
Quoted 49 lines
Quoted 3 lines
net-workstream/features-1.4.md +0
-0page is fully consumed. 8. The device should be able to consume multiple pages for areceive GSO
stream.
net-workstream/features-1.4.md
Quoted 8 lines
net-workstream/features-1.4.md +8
-0+ +## 3.3 Virtqueue notification coalescing re-arming support 0. +Design +goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and +whensuch
net-workstream/features-1.4.md struct vnet_rx_completion { · line 185
Quoted 4 lines
net-workstream/features-1.4.md +4
-0+ a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.I'm wondering how this relates to the existing notification coalesing[1] and notification suppression[2]:
[1]
The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}.[2]
If theVIRTIO_F_EVENT_IDXfeature bit is not negotiated:
\begin{itemize}
\item The driver MUST ignore the \field{avail_event} value.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If \field{flags} is 1, the driver SHOULD NOT send a notification. \item If \field{flags} is 0, the driver MUST send a notification. \end{itemize}
\end{itemize}Otherwise, if the
VIRTIO_F_EVENT_IDXfeature bit is negotiated:
\begin{itemize}
\item The driver MUST ignore the lower bit of \field{flags}.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If the \field{idx} field in the available ring (which determined where that descriptor index was placed) was equal to \field{avail_event}, the driver MUST send a notification. \item Otherwise the driver SHOULD NOT send a notification. \end{itemize}
\end{itemize}Quoted 5 lines
Quoted 5 lines
Regarding notification suppression:
1.When there isVIRTIO_NET_F_EVENT_IDX, even if the notification coalesing condition is met, we need to wait for theused_eventnotification condition to be met(the driver does not rearms the notification of the virtqueue now and the avail ring is set
VRING_AVAIL_F_NO_INTERRUPTin flag).Quoted 8 lines
Quoted 7 lines
2.When there is no
VIRTIO_NET_F_EVENT_IDX, if the driver turns off the notification, even if the notidication condition is met, the device cannot send the notification.Therefore, if I'm not wrong, a device can issue a notification only if the device is not suppressed from notifying the driver. [1][2] seems to have met this condition.
Notification suppression using
_EVENT_IDXfor non-memory transport is justsub-optimal for two reasons.
Quoted 6 lines
1. It requires device to poll on the used event to learn about when to un-suppress. (arm) 2. this bit also controls driver notifications yet again demand device to arbitrarily poll on new descriptors posting
Hence, an efficient scheme is needed and device notifications to be detached
Quoted 8 lines
from driver notification.
Quoted 1 line
And now that VQ level notification coalescing is in place, which suppresses
the device notifications, it is logical to combine it with VQ device notifications.
Quoted 1 line
Let me summarize:
1. When used idx notification is satisfied, but coalescing is not satisfied, the driver continues to suppress device notifications.
Ack.
Quoted 2 lines
2. When used idx notification is not satisfied, even if coalescing is satisfied, the device still cannot notify the driver.
Ack.
Quoted 7 lines
I think that's what coalescing does, and the description below has satisfied this behavior:
"The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}."
Ack.
the proposal here is to not use EVENT_IDX scheme, instead driver to enable/disable notification coalescing in different way, even when notification coalescing parameters are configured. And this to be done in fairly fast way (not like a cvq) command. For example like driver notifications.
Quoted 1 line
Or we want to say that it has nothing to do with the used idx notification.
When
Quoted 3 lines
the coalescing is satisfied and the driver rearms the notification of the virtqueue, the device now send a notification.
Right.F_NOTIFION_ARM is mutually exclusive with F_EVENT_IDX. (Like packed vq is mutually exclusive with split q.)
[virtio-comment] Re: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 138 lines · Parav Pandit
Quoted 114 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 6:07 PMQuoted 24 lines · Parav Pandit
Quoted 24 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 2:01 PMQuoted 20 lines · Parav Pandit
Add virtio net device notification coalescing improvements requirements.
Signed-off-by Parav Pandit <[email protected]>Acked-by David Edmondson <[email protected]>---
changelog:v3->v4:
- no change
v1->v2:
- addressed comments from Stefan
- redrafted the requirements to use rearm term and avoid queue enable confusionv0->v1:
- updated the description--- net-workstream/features-1.4.md | 11 +++++++++++ 1 file changed, 11 insertions(+)Quoted 5 lines
Quoted 5 lines
net-workstream/features-1.4.md together is desired while updating the virtio net
Quoted 5 lines
(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex 72d04bd..cb72442 100644net-workstream/features-1.4.md +0
-0@@ -8,6 +8,7 @@ together is desired while updating the virtio netinterface.
Quoted 9 lines
Quoted 9 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 173
Quoted 9 lines
net-workstream/features-1.4.md +1
-0# 2. Summary 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport +3. Virtqueue notification coalescing re-arming support # 3. Requirements ## 3.1 Device counters@@ -172,3 +173,13 @@ struct vnet_rx_completion { which can be recycled by the driver when the packets from thecompleted
Quoted 49 lines
Quoted 49 lines
Quoted 3 lines
net-workstream/features-1.4.md +0
-0page is fully consumed. 8. The device should be able to consume multiple pages for areceive GSO
stream.
net-workstream/features-1.4.md
Quoted 8 lines
net-workstream/features-1.4.md +8
-0+ +## 3.3 Virtqueue notification coalescing re-arming support 0. +Design +goal: + a. Avoid constant notifications from the device even in conditions when + the driver may not have acted on the previous pending notification. +1. When Tx and Rx virtqueue notification coalescing is enabled, and +whensuch
net-workstream/features-1.4.md struct vnet_rx_completion { · line 185
Quoted 4 lines
net-workstream/features-1.4.md +4
-0+ a notification is reported by the device, the device stops sending further + notifications until the driver rearms the notifications of the virtqueue. +2. When the driver rearms the notification of the virtqueue, the device + to notify again if notification coalescing conditions are met.I'm wondering how this relates to the existing notification coalesing[1] and notification suppression[2]:
[1]
The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}.[2]
If theVIRTIO_F_EVENT_IDXfeature bit is not negotiated:
\begin{itemize}
\item The driver MUST ignore the \field{avail_event} value.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If \field{flags} is 1, the driver SHOULD NOT send a notification. \item If \field{flags} is 0, the driver MUST send a notification. \end{itemize}
\end{itemize}Otherwise, if the
VIRTIO_F_EVENT_IDXfeature bit is negotiated:
\begin{itemize}
\item The driver MUST ignore the lower bit of \field{flags}.
\item After the driver writes a descriptor index into the available ring:
\begin{itemize}
\item If the \field{idx} field in the available ring (which determined where that descriptor index was placed) was equal to \field{avail_event}, the driver MUST send a notification. \item Otherwise the driver SHOULD NOT send a notification. \end{itemize}
\end{itemize}Quoted 5 lines
Quoted 5 lines
Regarding notification suppression:
1.When there isVIRTIO_NET_F_EVENT_IDX, even if the notification coalesing condition is met, we need to wait for theused_eventnotification condition to be met(the driver does not rearms the notification of the virtqueue now and the avail ring is set
VRING_AVAIL_F_NO_INTERRUPTin flag).Quoted 8 lines
Quoted 7 lines
2.When there is no
VIRTIO_NET_F_EVENT_IDX, if the driver turns off the notification, even if the notidication condition is met, the device cannot send the notification.Therefore, if I'm not wrong, a device can issue a notification only if the device is not suppressed from notifying the driver. [1][2] seems to have met this condition.
Notification suppression using
_EVENT_IDXfor non-memory transport is justsub-optimal for two reasons.
Quoted 4 lines
1. It requires device to poll on the used event to learn about when to un-suppress. (arm) 2. this bit also controls driver notifications yet again demand device to arbitrarily poll on new descriptors posting
Quoted 7 lines
Quoted 1 line
Hence, an efficient scheme is needed and device notifications to be detached
from driver notification.
Quoted 1 line
And now that VQ level notification coalescing is in place, which suppresses
the device notifications, it is logical to combine it with VQ device notifications. Let me summarize:
1. When used idx notification is satisfied, but coalescing is not satisfied, the driver continues to suppress device notifications.Ack.
Quoted 2 lines
2. When used idx notification is not satisfied, even if coalescing is satisfied, the device still cannot notify the driver.
Ack.
Quoted 7 lines
I think that's what coalescing does, and the description below has satisfied this behavior:
"The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Supppression}."Ack.
the proposal here is to not useEVENT_IDXscheme, instead driver to enable/disable notification coalescing in different way, even when notification coalescing parameters are configured. And this to be done in fairly fast way (not like a cvq) command. For example like driver notifications.
Quoted 7 lines
Quoted 1 line
Or we want to say that it has nothing to do with the used idx notification.
When
Quoted 3 lines
the coalescing is satisfied and the driver rearms the notification of the virtqueue, the device now send a notification.
Right.
F_NOTIFION_ARMis mutually exclusive withF_EVENT_IDX.
OK, I think I get your point, F_NOTIFION_ARM is mutually exclusive
with VIRTQ_AVAIL_F_NO_INTERRUPT\used idx\VIRTIO_F_NOTIFY_ON_EMPT,
and it seems that F_NOTIFION_ARM has the highest priority, and it needs a new feature bit. Am I right :)?
Thanks!
Quoted 1 line
(Like packed vq is mutually exclusive with split q.)
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 4/7] net-features: Add notification coalescing requirements
Quoted 6 lines
From: Heng Qi <[email protected]>
Sent: Thursday, August 17, 2023 10:44 AM
Quoted 7 lines
Quoted 1 line
F_NOTIFION_ARMis mutually exclusive withF_EVENT_IDX.OK, I think I get your point,
F_NOTIFION_ARMis mutually exclusive with VIRTQ_AVAIL_F_NO_INTERRUPT\used idx\VIRTIO_F_NOTIFY_ON_EMPT, and it seems thatF_NOTIFION_ARMhas the highest priority, and it needs a new feature bit. Am I right :)?
Yes for the new feature bit.
Since its mutually exclusive, there is no notion of priority.
Quoted 3 lines
Thanks!
Quoted 1 line
(Like packed vq is mutually exclusive with split q.)
PATCH v4 5/7 net-features: Add n-tuple receive flow filters requirements
In reply to Parav PanditAdd virtio net device requirements for receive flow filters.
---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- made flow_id flat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria
--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)
net-workstream/features-1.4.md +151 -0
@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport 3. Virtqueue notification coalescing re-arming support +4 Virtqueue receive flow filters (RFF) # 3. Requirements ## 3.1 Device counters
@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) +0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack driven receive + processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for ETHTOOL_SRXCLSRLINS + and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec to 1M/sec + or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow filter queue + count and its start vq index to the driver. +3. As each device may be operating for different performance characteristic, + start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled. +5. Flow filter operations are often accelerated by device in a hardware. Ability + to handle them on a queue other than control vq is desired. This achieves near + zero modifications to existing implementations to add new operations on new + purpose built queues (similar to transmit and receive queue). + Therefore, when flow filter queues are supported, it is strongly recommended + to use it, when flow filter queues are not supported, if the device support + it using cvq, driver should be able to use over cvq. +6. The filter masks are optional; the device should be able to expose if it + support filter masks. +7. The driver may want to have priority among group of flow entries; to facilitate + the device support grouping flow filter entries by a notion of a flow group. + Each flow group defines priority in processing flow. +8. The driver and group owner driver should be able to query supported device + limits for the receive flow filters. + +### 3.4.2 flow operations path +1. The driver should be able to define a receive packet match criteria, an + action and a destination for a packet. For example, an ipv4 packet with a + multicast address to be steered to the receive vq 0. The second example is + ipv4, tcp packet matching a specified IP address and tcp port tuple to + be steered to receive vq 10. +2. The match criteria should include exact tuple fields well-defined such as mac + address, IP addresses, tcp/udp ports, etc. +3. The match criteria should also optionally include the field mask. +5. Action includes (a) dropping or (b) forwarding the packet. +6. Destination is a receive virtqueue index. +7. Receive packet processing chain is: + a. filters programmed using cvq commands VIRTIO_NET_CTRL_RX, + VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN. + b. filters programmed using RFF functiionality. + c. filters programmed using RSS VIRTIO_NET_CTRL_MQ_RSS_CONFIG command. + Whichever filtering and steering functionality is enabled, they are applied + in the above order. +9. If multiple entries are programmed which has overlapping filtering attributes + for a received packet, the driver to define the location/priority of the entry. +10. The filter entries are usually short in size of few tens of bytes, + for example IPv6 + TCP tuple would be 36 bytes, and ops/sec rate is + high, hence supplying fields inside the queue descriptor is preferred for + up to a certain fixed size, say 96 bytes. +11. A flow filter entry consists of (a) match criteria, (b) action, + (c) destination and (d) a unique 32 bit flow id, all supplied by the + driver. +12. The driver should be able to query and delete flow filter entry by the + the device by the flow id. + +### 3.4.3 interface example + +1. Flow filter capabilities to query using a DMA interface such as cvq +using two different commands. + +``` +/* command 1 */ +struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group; + le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group; +}; + +/* command 2 */ +struct flow_filter_fields_support_mask { + le64 supported_packet_field_mask_bmap[1]; +}; + +``` + +2. Group add/delete cvq commands: +``` + +struct virtio_net_rff_group_add { + le16 priority; + le16 group_id; +}; + + +struct virtio_net_rff_group_delete { + le16 group_id; + +``` + +3. Flow filter entry add/modify, delete over flow vq: + +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding; + u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; + +``` + +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct virtio_net_rff_add_modify or + * struct virtio_net_rff_delete + */ +}; + +``` + +### 3.4.4 For incremental future +a. Driver should be able to specify a specific packet byte offset, number + of bytes and mask as math criteria. +b. Support RSS context, in addition to a specific RQ. +c. If/when virtio switch object is implemented, support ingress/egress flow + filters at the switch port level.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Comments below from today's bi-weekly meeting to address in v5.
Quoted 81 lines
From: Parav Pandit <[email protected]>
Sent: Tuesday, August 15, 2023 1:16 PMAdd virtio net device requirements for receive flow filters.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- madeflow_idflat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)net-workstream/features-1.4.md +0
-0@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues forPCI transport 3. Virtqueue notification coalescing re-arming support +4 Virtqueue receive flow filters (RFF)
# 3. Requirements
## 3.1 Device countersnet-workstream/features-1.4.md +6
-0@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) 0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack driven receive + processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for
ETHTOOL_SRXCLSRLINSnet-workstream/features-1.4.md +4
-0+ and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec to1M/sec
net-workstream/features-1.4.md +3
-0+ or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow filterqueue
net-workstream/features-1.4.md +10
-0+ count and its start vq index to the driver. +3. As each device may be operating for different performance characteristic, + start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled. +5. Flow filter operations are often accelerated by device in a hardware. Ability + to handle them on a queue other than control vq is desired. This achievesnear
+ zero modifications to existing implementations to add new operations on
newnet-workstream/features-1.4.md +2
-0+ purpose built queues (similar to transmit and receive queue). + Therefore, when flow filter queues are supported, it is stronglyrecommended
net-workstream/features-1.4.md struct vnet_rx_completion { · line 215
Quoted 2 lines
net-workstream/features-1.4.md +2
-0+ to use it, when flow filter queues are not supported, if the device support + it using cvq, driver should be able to use over cvq.
Rephase is like below.
0. Flow filter queues and flow filter commands on cvq are mutually exclusive.
1. When flow queues are supported, driver should create flow filter queues and use it. (Since cvq is not enabled for flow filters, any flow filter command coming on cvq must fail).
2. If driver wants to use flow filters over cvq, driver must explicitly enable flow filters on cvq via a command, when it is enabled on the cvq driver cannot use flow filter queues. This eliminates any synchronization needed by the device among different types of queues.
Quoted 55 lines
+6. The filter masks are optional; the device should be able to expose if it
+ support filter masks.
+7. The driver may want to have priority among group of flow entries; to facilitatenet-workstream/features-1.4.md +3
-0+ the device support grouping flow filter entries by a notion of a flow group. + Each flow group defines priority in processing flow. +8. The driver and group owner driver should be able to query supporteddevice
net-workstream/features-1.4.md +9
-0+ limits for the receive flow filters. + +### 3.4.2 flow operations path +1. The driver should be able to define a receive packet match criteria, an + action and a destination for a packet. For example, an ipv4 packet with a + multicast address to be steered to the receive vq 0. The second example is + ipv4, tcp packet matching a specified IP address and tcp port tuple to + be steered to receive vq 10. +2. The match criteria should include exact tuple fields well-defined such asmac
net-workstream/features-1.4.md +9
-0+ address, IP addresses, tcp/udp ports, etc. +3. The match criteria should also optionally include the field mask. +5. Action includes (a) dropping or (b) forwarding the packet. +6. Destination is a receive virtqueue index. +7. Receive packet processing chain is: + a. filters programmed using cvq commands VIRTIO_NET_CTRL_RX, + VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN. + b. filters programmed using RFF functiionality. + c. filters programmed using RSS VIRTIO_NET_CTRL_MQ_RSS_CONFIGcommand.
net-workstream/features-1.4.md +27
-0+ Whichever filtering and steering functionality is enabled, they are applied + in the above order. +9. If multiple entries are programmed which has overlapping filtering attributes + for a received packet, the driver to define the location/priority of the entry. +10. The filter entries are usually short in size of few tens of bytes, + for example IPv6 + TCP tuple would be 36 bytes, and ops/sec rate is + high, hence supplying fields inside the queue descriptor is preferred for + up to a certain fixed size, say 96 bytes. +11. A flow filter entry consists of (a) match criteria, (b) action, + (c) destination and (d) a unique 32 bit flow id, all supplied by the + driver. +12. The driver should be able to query and delete flow filter entry by the + the device by the flow id. + +### 3.4.3 interface example + +1. Flow filter capabilities to query using a DMA interface such as cvq +using two different commands. + +``` +/* command 1 */ +struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group;
net-workstream/features-1.4.md struct vnet_rx_completion { · line 276
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group; +}; + +/* command 2 */ +struct flow_filter_fields_support_mask { + le64 supported_packet_field_mask_bmap[1]; +};
Explain this bitmap that it indicates well known packet field such as src mac, dest ip, etc.
Also expose it on AQ command so that live migration flow/provision flow can decide which device to use.
Quoted 61 lines
net-workstream/features-1.4.md +48
-0+ +``` + +2. Group add/delete cvq commands: +``` + +struct virtio_net_rff_group_add { + le16 priority; + le16 group_id; +}; + + +struct virtio_net_rff_group_delete { + le16 group_id; + +``` + +3. Flow filter entry add/modify, delete over flow vq: + +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding; + u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; + +``` + +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct
virtio_net_rff_add_modifyornet-workstream/features-1.4.md +11
-1+ * struct virtio_net_rff_delete + */ }; + +``` + +### 3.4.4 For incremental future +a. Driver should be able to specify a specific packet byte offset, number + of bytes and mask as math criteria. +b. Support RSS context, in addition to a specific RQ. +c. If/when virtio switch object is implemented, support ingress/egress flow + filters at the switch port level. --
Quoted 1 line
2.26.2
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 1 line · Parav Pandit
Comments below from today's bi-weekly meeting to address in v5.
Thanks Parav!
Quoted 67 lines
Quoted 67 lines
From: Parav Pandit <[email protected]>
Sent: Tuesday, August 15, 2023 1:16 PMAdd virtio net device requirements for receive flow filters.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- madeflow_idflat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)net-workstream/features-1.4.md +0
-0@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues forPCI transport 3. Virtqueue notification coalescing re-arming support +4 Virtqueue receive flow filters (RFF)
# 3. Requirements
## 3.1 Device countersnet-workstream/features-1.4.md +6
-0@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) 0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack driven receive + processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for
ETHTOOL_SRXCLSRLINSnet-workstream/features-1.4.md +4
-0+ and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec to1M/sec
net-workstream/features-1.4.md +3
-0+ or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow filterqueue
Quoted 8 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 208
Quoted 8 lines
net-workstream/features-1.4.md +8
-0+ count and its start vq index to the driver. +3. As each device may be operating for different performance characteristic, + start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled.
Regarding this description, I want to say that ARFS will be enabled at runtime. But ethtool RFF will be used at any time as long as the device is ready.
Combining what was discussed in today's meeting, flow vqs and ctrlq are mutually exclusive, so if flow vqs are supported, then ethtool RFF can use flow vq:
"
0. Flow filter queues and flow filter commands on cvq are mutually exclusive.
1. When flow queues are supported, the driver should create flow filter queues and use it. (Since cvq is not enabled for flow filters, any flow filter command coming on cvq must fail).
2. If driver wants to use flow filters over cvq, driver must explicitly enable flow filters on cvq via a command, when it is enabled on the cvq driver cannot use flow filter queues. This eliminates any synchronization needed by the device among different types of queues.
"
Well the "likely create these queues when flow filters are enabled" described here is confusing. Because if ethtool RFF is used, we need to create a flow vq in the probe stage, right?
There are several other reasons:
1. The behavior of dynamically creating flow vq will break the current virtio spec. Please see the "Device Initialization" chapter. ctrlq, as a configuration queue similar to flow vq, is also created in the probe phase. So if we support the "dynamically creating", we need to update the spec.
2. Flow vq is similar to transmit q, and does not need to fill descriptors in advance, so the consumption of resources is relatively small.
3. Dynamic creation of virtqueue seems to be a new thread of virtio spec, and it should also be applicable to rxqs and txqs. We can temporarily support creating flow vq in the probe stage, and subsequent dynamic creation can be an extension.
So, should we create the flow vqs at the initial stage of the driver probe?
Thanks!
Quoted 143 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 215
Quoted 10 lines
+5. Flow filter operations are often accelerated by device in a hardware. Ability
+ to handle them on a queue other than control vq is desired. This achieves
near
+ zero modifications to existing implementations to add new operations on
newnet-workstream/features-1.4.md +2
-0+ purpose built queues (similar to transmit and receive queue). + Therefore, when flow filter queues are supported, it is stronglyrecommended
net-workstream/features-1.4.md +2
-0+ to use it, when flow filter queues are not supported, if the device support + it using cvq, driver should be able to use over cvq.Rephase is like below.
0. Flow filter queues and flow filter commands on cvq are mutually exclusive.1. When flow queues are supported, driver should create flow filter queues and use it. (Since cvq is not enabled for flow filters, any flow filter command coming on cvq must fail).
2. If driver wants to use flow filters over cvq, driver must explicitly enable flow filters on cvq via a command, when it is enabled on the cvq driver cannot use flow filter queues. This eliminates any synchronization needed by the device among different types of queues.
Quoted 55 lines
+6. The filter masks are optional; the device should be able to expose if it
+ support filter masks.
+7. The driver may want to have priority among group of flow entries; to facilitatenet-workstream/features-1.4.md +3
-0+ the device support grouping flow filter entries by a notion of a flow group. + Each flow group defines priority in processing flow. +8. The driver and group owner driver should be able to query supporteddevice
net-workstream/features-1.4.md +9
-0+ limits for the receive flow filters. + +### 3.4.2 flow operations path +1. The driver should be able to define a receive packet match criteria, an + action and a destination for a packet. For example, an ipv4 packet with a + multicast address to be steered to the receive vq 0. The second example is + ipv4, tcp packet matching a specified IP address and tcp port tuple to + be steered to receive vq 10. +2. The match criteria should include exact tuple fields well-defined such asmac
net-workstream/features-1.4.md +9
-0+ address, IP addresses, tcp/udp ports, etc. +3. The match criteria should also optionally include the field mask. +5. Action includes (a) dropping or (b) forwarding the packet. +6. Destination is a receive virtqueue index. +7. Receive packet processing chain is: + a. filters programmed using cvq commands VIRTIO_NET_CTRL_RX, + VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN. + b. filters programmed using RFF functiionality. + c. filters programmed using RSS VIRTIO_NET_CTRL_MQ_RSS_CONFIGcommand.
net-workstream/features-1.4.md +27
-0+ Whichever filtering and steering functionality is enabled, they are applied + in the above order. +9. If multiple entries are programmed which has overlapping filtering attributes + for a received packet, the driver to define the location/priority of the entry. +10. The filter entries are usually short in size of few tens of bytes, + for example IPv6 + TCP tuple would be 36 bytes, and ops/sec rate is + high, hence supplying fields inside the queue descriptor is preferred for + up to a certain fixed size, say 96 bytes. +11. A flow filter entry consists of (a) match criteria, (b) action, + (c) destination and (d) a unique 32 bit flow id, all supplied by the + driver. +12. The driver should be able to query and delete flow filter entry by the + the device by the flow id. + +### 3.4.3 interface example + +1. Flow filter capabilities to query using a DMA interface such as cvq +using two different commands. + +``` +/* command 1 */ +struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group;net-workstream/features-1.4.md struct vnet_rx_completion { · line 276
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group; +}; + +/* command 2 */ +struct flow_filter_fields_support_mask { + le64 supported_packet_field_mask_bmap[1]; +};Explain this bitmap that it indicates well known packet field such as src mac, dest ip, etc.
Also expose it on AQ command so that live migration flow/provision flow can decide which device to use.
Quoted 54 lines
net-workstream/features-1.4.md +48
-0+ +``` + +2. Group add/delete cvq commands: +``` + +struct virtio_net_rff_group_add { + le16 priority; + le16 group_id; +}; + + +struct virtio_net_rff_group_delete { + le16 group_id; + +``` + +3. Flow filter entry add/modify, delete over flow vq: + +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding; + u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; + +``` + +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct
virtio_net_rff_add_modifyornet-workstream/features-1.4.md +5
-0+ * struct virtio_net_rff_delete + */ }; + +``` +
Quoted 8 lines
Quoted 8 lines
net-workstream/features-1.4.md +6
-1+### 3.4.4 For incremental future +a. Driver should be able to specify a specific packet byte offset, number + of bytes and mask as math criteria. +b. Support RSS context, in addition to a specific RQ. +c. If/when virtio switch object is implemented, support ingress/egress flow + filters at the switch port level. --2.26.2
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 85 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 1:08 PMQuoted 45 lines
Quoted 45 lines
From: Parav Pandit <[email protected]>
Sent: Tuesday, August 15, 2023 1:16 PMAdd virtio net device requirements for receive flow filters.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- madeflow_idflat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex cb72442..78bb3d2 100644net-workstream/features-1.4.md +0
-0@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rxvirtqueues for PCI transport 3. Virtqueue notification coalescing re-arming support
+4 Virtqueue receive flow filters (RFF)Quoted 10 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 190
Quoted 10 lines
# 3. Requirements
## 3.1 Device countersnet-workstream/features-1.4.md +4
-0@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) 0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack drivenreceive
Quoted 8 lines
Quoted 8 lines
net-workstream/features-1.4.md +2
-0+ processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for
ETHTOOL_SRXCLSRLINSnet-workstream/features-1.4.md +5
-0+ and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec +toQuoted 8 lines
Quoted 8 lines
1M/sec
net-workstream/features-1.4.md +4
-0+ or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow +filterqueue
+ count and its start vq index to the driver.
+3. As each device may be operating for different performancecharacteristic,
Quoted 6 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 208
Quoted 6 lines
net-workstream/features-1.4.md +6
-0+ start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled.
Quoted 3 lines
Regarding this description, I want to say that ARFS will be enabled at runtime. But ethtool RFF will be used at any time as long as the device is ready.
Yes, but ethool RFS is blocking callback in which slow task such as q creation can be done, only when one wants to add flows. ARFS is anyway controlled using set_features() callback.
Quoted 12 lines
Combining what was discussed in today's meeting, flow vqs and ctrlq are mutually exclusive, so if flow vqs are supported, then ethtool RFF can use flow vq:
"
0. Flow filter queues and flow filter commands on cvq are mutually exclusive.1. When flow queues are supported, the driver should create flow filter queues and use it.
(Since cvq is not enabled for flow filters, any flow filter command coming on cvq must fail).
Quoted 7 lines
2. If driver wants to use flow filters over cvq, driver must explicitly enable flow filters on cvq via a command, when it is enabled on the cvq driver cannot use flow filter queues.
This eliminates any synchronization needed by the device among different types of queues.
"
Ack.
Quoted 5 lines
Well the "likely create these queues when flow filters are enabled" described here is confusing.
Because if ethtool RFF is used, we need to create a flow vq in the probe stage, right?
Current spec wording limits one to create queues before DRIVER_OK.
But with introduction of _RESET bit one can create an empty queue and disable it (reset it! What a grand name).
And re-enable it during ethtool callbacks.
This would be workaround to dynamically create the queue.
Quoted 8 lines
There are several other reasons:
1. The behavior of dynamically creating flow vq will break the current virtio spec.
Please see the "Device Initialization" chapter. ctrlq, as a configuration queue similar to flow vq, is also created in the probe phase. So if we support the
"dynamically creating",
we need to update the spec.
Quoted 4 lines
2. Flow vq is similar to transmit q, and does not need to fill descriptors in advance,
so the consumption of resources is relatively small.
Only the queue descriptors memory is consumed, which is not a lot.
But concept of creating resource without consuming is just bad.
We learnt the lesson from mlx5 driver that dynamic creation is efficient.
Many part of Linux kernel also moving in this direction, all the way upto dynamically individual msix vector.
So we should strive to enable them dynamically and improve the virtio spec.
It should an orthogonal feature, sadly how the RING_RESET feature is done. :(
Quoted 7 lines
3. Dynamic creation of virtqueue seems to be a new thread of virtio spec, and it should also be
applicable to rxqs and txqs. We can temporarily support creating flow vq in
the probe stage,
and subsequent dynamic creation can be an extension.So, should we create the flow vqs at the initial stage of the driver probe?
One option is to follow the above workaround.
Second option is to add feature bit to feature bit to indicate dynamic Q creation.
[virtio-comment] Re: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 114 lines · Parav Pandit
Quoted 76 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 1:08 PMQuoted 45 lines
Quoted 45 lines
From: Parav Pandit <[email protected]>
Sent: Tuesday, August 15, 2023 1:16 PMAdd virtio net device requirements for receive flow filters.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- madeflow_idflat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)(quoted diff) +0
-0
b/net-workstream/features-1.4.mdindex cb72442..78bb3d2 100644net-workstream/features-1.4.md +0
-0@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rxvirtqueues for PCI transport 3. Virtqueue notification coalescing re-arming support
+4 Virtqueue receive flow filters (RFF)Quoted 10 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 190
Quoted 10 lines
# 3. Requirements
## 3.1 Device countersnet-workstream/features-1.4.md +4
-0@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) 0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack drivenreceive
Quoted 8 lines
Quoted 8 lines
net-workstream/features-1.4.md +2
-0+ processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for
ETHTOOL_SRXCLSRLINSnet-workstream/features-1.4.md +5
-0+ and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec +toQuoted 8 lines
Quoted 8 lines
1M/sec
net-workstream/features-1.4.md +4
-0+ or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow +filterqueue
+ count and its start vq index to the driver.
+3. As each device may be operating for different performancecharacteristic,
Quoted 9 lines
Quoted 6 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 208
Quoted 6 lines
net-workstream/features-1.4.md +6
-0+ start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled.Regarding this description, I want to say that ARFS will be enabled at runtime. But ethtool RFF will be used at any time as long as the device is ready.
Yes, but ethool RFS is blocking callback in which slow task such as q creation can be done, only when one wants to add flows. ARFS is anyway controlled using
set_features()callback.Quoted 12 lines
Combining what was discussed in today's meeting, flow vqs and ctrlq are mutually exclusive, so if flow vqs are supported, then ethtool RFF can use flow vq:
"
0. Flow filter queues and flow filter commands on cvq are mutually exclusive.1. When flow queues are supported, the driver should create flow filter queues and use it.
(Since cvq is not enabled for flow filters, any flow filter command coming on cvq must fail).Quoted 7 lines
2. If driver wants to use flow filters over cvq, driver must explicitly enable flow filters on cvq via a command, when it is enabled on the cvq driver cannot use flow filter queues.
This eliminates any synchronization needed by the device among different types of queues.
"Ack.
Quoted 5 lines
Well the "likely create these queues when flow filters are enabled" described here is confusing.
Because if ethtool RFF is used, we need to create a flow vq in the probe stage, right?
Quoted 5 lines
Current spec wording limits one to create queues before
DRIVER_OK.
But with introduction of_RESETbit one can create an empty queue and disable it (reset it! What a grand name).And re-enable it during ethtool callbacks.
This would be workaround to dynamically create the queue.
Yes, this is workaround, we can just set the number of flow vqs for the device, but not allocate resources nor enable. But this is not exhaustive, because xdp may also require dynamic q creation/destruction.
Quoted 9 lines
Quoted 9 lines
There are several other reasons:
1. The behavior of dynamically creating flow vq will break the current virtio spec.
Please see the "Device Initialization" chapter. ctrlq, as a configuration queue similar to flow vq, is also created in the probe phase. So if we support the
"dynamically creating",
we need to update the spec.
Quoted 8 lines
Quoted 4 lines
2. Flow vq is similar to transmit q, and does not need to fill descriptors in advance,
so the consumption of resources is relatively small.Only the queue descriptors memory is consumed, which is not a lot.
But concept of creating resource without consuming is just bad.
We learnt the lesson from mlx5 driver that dynamic creation is efficient.
Many part of Linux kernel also moving in this direction, all the way upto dynamically individual msix vector.
Ok. I got it.
Quoted 4 lines
So we should strive to enable them dynamically and improve the virtio spec.
It should an orthogonal feature, sadly how the
RING_RESETfeature is done. :(
RING_RESET is performed without changing the number of queues. But what you said above is a workaround.
Quoted 10 lines
Quoted 8 lines
3. Dynamic creation of virtqueue seems to be a new thread of virtio spec, and it should also be
applicable to rxqs and txqs. We can temporarily support creating flow vq in
the probe stage,
and subsequent dynamic creation can be an extension.So, should we create the flow vqs at the initial stage of the driver probe?
One option is to follow the above workaround.
Second option is to add feature bit to feature bit to indicate dynamic Q creation.
I'm leaning towards the second option, which makes the work orthogonal and also works in the case of XDP.
Thanks!
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 11 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 4:41 PMQuoted 2 lines
One option is to follow the above workaround.
Second option is to add feature bit to feature bit to indicate dynamic Qcreation.
I'm leaning towards the second option, which makes the work orthogonal and also works in the case of XDP.
Yes. lets add the bit. I will send patch as part of this work.
[virtio-comment] Re: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
In reply to Parav PanditHi, Parav.
There are some minor updates!
Quoted 120 lines · Parav Pandit
Add virtio net device requirements for receive flow filters.
Signed-off-by Parav Pandit <[email protected]>---
changelog:v3->v4:
- Addressed comments from Satananda, Heng, David
- removed context specific wording, replaced with destination
- added group create/delete examples and updated requirements
- added optional support to use cvq for flor filter commands
- added example of transporting flow filter commands over cvq
- made group size to be 16-bit
- added concept of 0->n max flow filter entries based on max count
- added concept of 0->n max flow group based on max count
- split field bitmask to separate command from other filter capabilities
- rewrote rx filter processing chain order with respect to existing filter commands and rss
- madeflow_idflat across all groupsv1->v2:
- split setup and operations requirements
- added design goal
- worded requirements more preciselyv0->v1:
- fixed comments from Heng Li
- renamed receive flow steering to receive flow filters
- clarified byte offset in match criteria--- net-workstream/features-1.4.md | 151 +++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+)net-workstream/features-1.4.md +75
-0@@ -9,6 +9,7 @@ together is desired while updating the virtio net interface. 1. Device counters visible to the driver 2. Low latency tx and rx virtqueues for PCI transport 3. Virtqueue notification coalescing re-arming support +4 Virtqueue receive flow filters (RFF) # 3. Requirements ## 3.1 Device counters@@ -183,3 +184,153 @@ struct vnet_rx_completion { notifications until the driver rearms the notifications of the virtqueue. 2. When the driver rearms the notification of the virtqueue, the device to notify again if notification coalescing conditions are met. + +## 3.4 Virtqueue receive flow filters (RFF) +0. Design goal: + To filter and/or to steer packet based on specific pattern match to a + specific destination to support application/networking stack driven receive + processing. +1. Two use cases are: to support Linux netdev set_rxnfc() for ETHTOOL_SRXCLSRLINS + and to support netdev feature NETIF_F_NTUPLE aka ARFS. + +### 3.4.1 control path +1. The number of flow filter operations/sec can range from 100k/sec to 1M/sec + or even more. Hence flow filter operations must be done over a queueing + interface using one or more queues. +2. The device should be able to expose one or more supported flow filter queue + count and its start vq index to the driver. +3. As each device may be operating for different performance characteristic, + start vq index and count may be different for each device. Secondly, it is + inefficient for device to provide flow filters capabilities via a config space + region. Hence, the device should be able to share these attributes using + dma interface, instead of transport registers. +4. Since flow filters are enabled much later in the driver life cycle, driver + will likely create these queues when flow filters are enabled. +5. Flow filter operations are often accelerated by device in a hardware. Ability + to handle them on a queue other than control vq is desired. This achieves near + zero modifications to existing implementations to add new operations on new + purpose built queues (similar to transmit and receive queue). + Therefore, when flow filter queues are supported, it is strongly recommended + to use it, when flow filter queues are not supported, if the device support + it using cvq, driver should be able to use over cvq. +6. The filter masks are optional; the device should be able to expose if it + support filter masks. +7. The driver may want to have priority among group of flow entries; to facilitate + the device support grouping flow filter entries by a notion of a flow group. + Each flow group defines priority in processing flow. +8. The driver and group owner driver should be able to query supported device + limits for the receive flow filters. + +### 3.4.2 flow operations path +1. The driver should be able to define a receive packet match criteria, an + action and a destination for a packet. For example, an ipv4 packet with a + multicast address to be steered to the receive vq 0. The second example is + ipv4, tcp packet matching a specified IP address and tcp port tuple to + be steered to receive vq 10. +2. The match criteria should include exact tuple fields well-defined such as mac + address, IP addresses, tcp/udp ports, etc. +3. The match criteria should also optionally include the field mask. +5. Action includes (a) dropping or (b) forwarding the packet. +6. Destination is a receive virtqueue index. +7. Receive packet processing chain is: + a. filters programmed using cvq commands VIRTIO_NET_CTRL_RX, + VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN. + b. filters programmed using RFF functiionality. + c. filters programmed using RSS VIRTIO_NET_CTRL_MQ_RSS_CONFIG command. + Whichever filtering and steering functionality is enabled, they are applied + in the above order. +9. If multiple entries are programmed which has overlapping filtering attributes + for a received packet, the driver to define the location/priority of the entry. +10. The filter entries are usually short in size of few tens of bytes, + for example IPv6 + TCP tuple would be 36 bytes, and ops/sec rate is + high, hence supplying fields inside the queue descriptor is preferred for + up to a certain fixed size, say 96 bytes. +11. A flow filter entry consists of (a) match criteria, (b) action, + (c) destination and (d) a unique 32 bit flow id, all supplied by the + driver. +12. The driver should be able to query and delete flow filter entry by the + the device by the flow id. + +### 3.4.3 interface example + +1. Flow filter capabilities to query using a DMA interface such as cvq +using two different commands. + +``` +/* command 1 */
net-workstream/features-1.4.md struct vnet_rx_completion { · line 270
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group; + le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group;
+ u8 padding[3];
Quoted 4 lines
net-workstream/features-1.4.md +4
-0+}; + +/* command 2 */ +struct flow_filter_fields_support_mask {
net-workstream/features-1.4.md struct vnet_rx_completion { · line 284
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ le64 supported_packet_field_mask_bmap[1]; +}; + +``` + +2. Group add/delete cvq commands: +``` + +struct virtio_net_rff_group_add { + le16 priority;
Please explicitly explain the relationship between the number and the priority, for example, the smaller the number, the higher the priority :)
Quoted 5 lines
net-workstream/features-1.4.md +5
-0+ le16 group_id; +}; + + +struct virtio_net_rff_group_delete {
net-workstream/features-1.4.md struct vnet_rx_completion { · line 299
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ le16 group_id; + +``` + +3. Flow filter entry add/modify, delete over flow vq: + +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding;
s/padding/priority
Each rule needs a priority.
Quoted 16 lines
net-workstream/features-1.4.md +16
-0+ u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; + +```
net-workstream/features-1.4.md struct vnet_rx_completion { · line 325
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+ +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct virtio_net_rff_add_modify or + * struct virtio_net_rff_delete
For flow vq, we no longer distinguish operations by command, but by flow_op.
But for ctrlq, this field will be carried. We should make it clear that when ctrlq is delivered based on cmd, the flow_op field is ignored.
Thanks!
net-workstream/features-1.4.md struct vnet_rx_completion { · line 336
Quoted 11 lines
net-workstream/features-1.4.md +11
-0+ */ +}; + +``` + +### 3.4.4 For incremental future +a. Driver should be able to specify a specific packet byte offset, number + of bytes and mask as math criteria. +b. Support RSS context, in addition to a specific RQ. +c. If/when virtio switch object is implemented, support ingress/egress flow + filters at the switch port level.
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] RE: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 4 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 5:12 PM
Quoted 14 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 270
Quoted 11 lines
net-workstream/features-1.4.md +11
-0+/* command 1 */ +struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group; + le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group;+ u8 padding[3];
Ack.
Quoted 6 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 284
Quoted 2 lines
net-workstream/features-1.4.md +2
-0+struct virtio_net_rff_group_add { + le16 priority;Please explicitly explain the relationship between the number and the priority, for example, the smaller the number, the higher the priority :)
Right. Will do.
I was thinking of higher the value higher the priority, so that one doesnt need to invert this in brain every time seeing the priority field. :)
Quoted 9 lines
Quoted 9 lines
net-workstream/features-1.4.md +9
-0+ le16 group_id; +}; + + +struct virtio_net_rff_group_delete { + le16 group_id; + +``` +
Quoted 9 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 299
Quoted 6 lines
+3. Flow filter entry add/modify, delete over flow vq:
net-workstream/features-1.4.md +5
-0+ +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding;s/padding/priority
Ack.
Quoted 28 lines
Each rule needs a priority.
Quoted 15 lines
net-workstream/features-1.4.md +15
-0+ u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; +net-workstream/features-1.4.md struct vnet_rx_completion {
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+``` + +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct
virtio_net_rff_add_modifyor
Quoted 6 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 325
Quoted 1 line
net-workstream/features-1.4.md +1
-0+ * struct virtio_net_rff_deleteFor flow vq, we no longer distinguish operations by command, but by
flow_op. But for ctrlq, this field will be carried. We should make it clear that when ctrlq is delivered based on cmd, theflow_opfield is ignored.
Since cvq is only the communication medium for delivering of command, it is better use the flow_op as is, and cvq commands field to be ignored. This way, software layers are more organized cvq or flow vq.
[virtio-comment] Re: [PATCH requirements v4 5/7] net-features: Add n-tuple receive flow filters requirements
Quoted 18 lines · Parav Pandit
Quoted 16 lines
From: Heng Qi <[email protected]>
Sent: Wednesday, August 16, 2023 5:12 PMnet-workstream/features-1.4.md struct vnet_rx_completion { · line 270
Quoted 11 lines
net-workstream/features-1.4.md +11
-0+/* command 1 */ +struct flow_filter_capabilities { + le16 start_vq_index; + le16 num_flow_filter_vqs; + le16 max_flow_groups; + le16 max_group_priorities; /* max priorities of the group */ + le32 max_flow_filters_per_group; + le32 max_flow_filters; /* max flow_id in add/del + * is equal = max_flow_filters - 1. + */ + u8 max_priorities_per_group;+ u8 padding[3];
Ack.
Quoted 8 lines
Quoted 5 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 284
Quoted 2 lines
net-workstream/features-1.4.md +2
-0+struct virtio_net_rff_group_add { + le16 priority;Please explicitly explain the relationship between the number and the priority, for example, the smaller the number, the higher the priority :)
Right. Will do.
I was thinking of higher the value higher the priority, so that one doesnt need to invert this in brain every time seeing the priority field. :)
It's ok :)
Quoted 47 lines
Quoted 8 lines
Quoted 8 lines
net-workstream/features-1.4.md +8
-0+ le16 group_id; +}; + + +struct virtio_net_rff_group_delete { + le16 group_id; + +```Quoted 9 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 299
Quoted 7 lines
net-workstream/features-1.4.md +7
-0+ +3. Flow filter entry add/modify, delete over flow vq: + +``` +struct virtio_net_rff_add_modify { + u8 flow_op; + u8 padding;s/padding/priority
Ack.
Quoted 28 lines
Each rule needs a priority.
Quoted 15 lines
net-workstream/features-1.4.md +15
-0+ u16 group_id; + le32 flow_id; + struct match_criteria mc; + struct destination dest; + struct action action; + + struct match_criteria mask; /* optional */ +}; + +struct virtio_net_rff_delete { + u8 flow_op; + u8 padding[3]; + le32 flow_id; +}; +net-workstream/features-1.4.md struct vnet_rx_completion {
Quoted 10 lines
net-workstream/features-1.4.md +10
-0+``` + +4. Flow filter commands over cvq: + +``` + +struct virtio_net_rff_cmd { + u8 class; /* RFF class */ + u8 commands; /* RFF cmd = A */ + u8 command-specific-data[]; /* contains struct
virtio_net_rff_add_modifyor
Quoted 7 lines
Quoted 5 lines
net-workstream/features-1.4.md struct vnet_rx_completion { · line 325
Quoted 1 line
net-workstream/features-1.4.md +1
-0+ * struct virtio_net_rff_deleteFor flow vq, we no longer distinguish operations by command, but by
flow_op. But for ctrlq, this field will be carried. We should make it clear that when ctrlq is delivered based on cmd, theflow_opfield is ignored.Since cvq is only the communication medium for delivering of command, it is better use the
flow_opas is, and cvq commands field to be ignored. This way, software layers are more organized cvq or flow vq.
Agree.
Thanks!
Quoted 1 line
This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting.
Subscribe: [email protected]
Unsubscribe: [email protected]
List help: [email protected]
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
PATCH v4 6/7 net-features: Add packet timestamp requirements
In reply to Parav PanditAdd tx and rx packet timestamp requirements.
---
changelog:v3->v4:
- no change
--- net-workstream/features-1.4.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
net-workstream/features-1.4.md +26 -0
@@ -10,6 +10,7 @@ together is desired while updating the virtio net interface. 2. Low latency tx and rx virtqueues for PCI transport 3. Virtqueue notification coalescing re-arming support 4 Virtqueue receive flow filters (RFF) +5. Device timestamp for tx and rx packets # 3. Requirements ## 3.1 Device counters
@@ -334,3 +335,28 @@ a. Driver should be able to specify a specific packet byte offset, number b. Support RSS context, in addition to a specific RQ. c. If/when virtio switch object is implemented, support ingress/egress flow filters at the switch port level. + +## 3.5 Packet timestamp +1. Device should provide transmit timestamp and receive timestamp of the packets + at per packet level when the device is enabled. +2. Device should provide the current free running clock in the least latency + possible using an MMIO register read of 64-bit to have the least jitter. +3. Device should provide the current frequency and the frequency unit for the + software to synchronize the reference point of software and the device using + a control vq command. + +### 3.5.1 Transmit timestamp +1. Transmit completion must contain a packet transmission timestamp when the + device is enabled for it. +2. The device should record the packet transmit timestamp in the completion at + the farthest egress point towards the network. +3. The device must provide a transmit packet timestamp in a single DMA + transaction along with the rest of the transmit completion fields. + +### 3.5.2 Receive timestamp +1. Receive completion must contain a packet reception timestamp when the device + is enabled for it. +2. The device should record the received packet timestamp at the closet ingress + point of reception from the network. +3. The device should provide a receive packet timestamp in a single DMA + transaction along with the rest of the receive completion fields.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
PATCH v4 7/7 net-features: Add header data split requirements
In reply to Parav PanditAdd header data split requirements for the receive packets.
--- net-workstream/features-1.4.md | 13 +++++++++++++ 1 file changed, 13 insertions(+)
net-workstream/features-1.4.md +13 -0
@@ -11,6 +11,7 @@ together is desired while updating the virtio net interface. 3. Virtqueue notification coalescing re-arming support 4 Virtqueue receive flow filters (RFF) 5. Device timestamp for tx and rx packets +6. Header data split for the receive virtqueue # 3. Requirements ## 3.1 Device counters
@@ -360,3 +361,15 @@ c. If/when virtio switch object is implemented, support ingress/egress flow point of reception from the network. 3. The device should provide a receive packet timestamp in a single DMA transaction along with the rest of the receive completion fields. + +## 3.6 Header data split for the receive virtqueue +1. The device should be able to DMA the packet header and data to two different + memory locations, this enables driver and networking stack to perform zero + copy to application buffer(s). +2. The driver should be able to configure maximum header buffer size per + virtqueue. +3. The header buffer to be in a physically contiguous memory per virtqueue +4. The device should be able to indicate header data split in the receive + completion. +5. The device should be able to zero pad the header buffer when the received + header is shorter than cpu cache line size.
Signature · 18 lines
2.26.2 This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/
[virtio-comment] Re: [PATCH requirements v4 7/7] net-features: Add header data split requirements
Quoted 3 lines · Parav Pandit
Add header data split requirements for the receive packets.
Signed-off-by Parav Pandit <[email protected]>
Quoted 32 lines
--- net-workstream/features-1.4.md | 13 +++++++++++++ 1 file changed, 13 insertions(+)net-workstream/features-1.4.md +13
-0@@ -11,6 +11,7 @@ together is desired while updating the virtio net interface. 3. Virtqueue notification coalescing re-arming support 4 Virtqueue receive flow filters (RFF) 5. Device timestamp for tx and rx packets +6. Header data split for the receive virtqueue # 3. Requirements ## 3.1 Device counters@@ -360,3 +361,15 @@ c. If/when virtio switch object is implemented, support ingress/egress flow point of reception from the network. 3. The device should provide a receive packet timestamp in a single DMA transaction along with the rest of the receive completion fields. + +## 3.6 Header data split for the receive virtqueue +1. The device should be able to DMA the packet header and data to two different + memory locations, this enables driver and networking stack to perform zero + copy to application buffer(s). +2. The driver should be able to configure maximum header buffer size per + virtqueue. +3. The header buffer to be in a physically contiguous memory per virtqueue +4. The device should be able to indicate header data split in the receive + completion. +5. The device should be able to zero pad the header buffer when the received + header is shorter than cpu cache line size.
Signature · 17 lines
I used to worry, thought I was goin' mad in a hurry. This publicly archived list offers a means to provide input to the OASIS Virtual I/O Device (VIRTIO) TC. In order to verify user consent to the Feedback License terms and to minimize spam in the list archive, subscription is required before posting. Subscribe: [email protected] Unsubscribe: [email protected] List help: [email protected] List archive: https://lists.oasis-open.org/archives/virtio-comment/ Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists Committee: https://www.oasis-open.org/committees/virtio/ Join OASIS: https://www.oasis-open.org/join/