ethdev: add min/max MTU to device info
RFC 1/1 ethdev: add min/max MTU to device info
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
The initial version sets max mtu to normal Ethernet, it is up to PMD to set larger value if it supports Jumbo frames.
Fixing the drivers to use this is trivial and can be done by 18.11. Already have some of the patches done.
--- lib/librte_ethdev/rte_ethdev.c | 7 +++++++ lib/librte_ethdev/rte_ethdev.h | 2 ++ 2 files changed, 9 insertions(+)
lib/librte_ethdev/rte_ethdev.c +7 -0
@@ -2408,6 +2408,8 @@ rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info) dev_info->rx_desc_lim = lim; dev_info->tx_desc_lim = lim; dev_info->device = dev->device; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETHER_MTU ;RTE_FUNC_PTR_OR_RET (*dev->dev_ops->dev_infos_get); (*dev->dev_ops->dev_infos_get)(dev, dev_info);
@@ -2471,12 +2473,17 @@ int rte_eth_dev_set_mtu(uint16_t port_id, uint16_t mtu) { int ret; +struct rte_eth_dev_info dev_info;struct rte_eth_dev *dev;RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); dev = &rte_eth_devices[port_id];RTE_FUNC_PTR_OR_ERR_RET (*dev->dev_ops->mtu_set, -ENOTSUP ); +rte_eth_dev_info_get (port_id, &dev_info); +if (mtu < dev_info.min_mtu || mtu > dev_info.max_mtu) +return -EINVAL ; + ret = (*dev->dev_ops->mtu_set)(dev, mtu);if (!ret) dev->data->mtu = mtu;
lib/librte_ethdev/rte_ethdev.h +2 -0
@@ -1015,6 +1015,8 @@ struct rte_eth_dev_info {const char *driver_name;/**< Device Driver name. */ unsigned int if_index;/**< Index to bound host interface, or 0 if none. Use if_indextoname() to translate into an interface name. */ +uint16_t min_mtu;/**< Minimum MTU allowed */ +uint16_t max_mtu;/**< Maximum MTU allowed */ const uint32_t *dev_flags;/**< Device flags */ uint32_t min_rx_bufsize;/**< Minimum size of RX buffer. */ uint32_t max_rx_pktlen;/**< Maximum configurable length of RX pkt. */
Signature · 1 line
2.17.1
Wednesday, September 5, 2018 7:42 PM, Stephen Hemminger:
Quoted 6 lines
Subject: [dpdk-dev] [RFC] ethdev: add min/max MTU to device info
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
Quoted 7 lines
The initial version sets max mtu to normal Ethernet, it is up to PMD to set larger value if it supports Jumbo frames.
Fixing the drivers to use this is trivial and can be done by 18.11. Already have some of the patches done.
Signed-off-by Stephen Hemminger <[email protected]>
Makes sense.
Quoted 53 lines
---
lib/librte_ethdev/rte_ethdev.c| 7 +++++++lib/librte_ethdev/rte_ethdev.h
| 2 ++2 files changed, 9 insertions(+)lib/librte_ethdev/rte_ethdev.c +2
-0@@ -2408,6 +2408,8 @@ rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info) dev_info->rx_desc_lim = lim; dev_info->tx_desc_lim = lim; dev_info->device = dev->device; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETHER_MTU ;RTE_FUNC_PTR_OR_RET (*dev->dev_ops->dev_infos_get);lib/librte_ethdev/rte_ethdev.c +2
-0(*dev->dev_ops->dev_infos_get)(dev, dev_info); @@ -2471,12 +2473,17 @@ int rte_eth_dev_set_mtu(uint16_t port_id, uint16_t mtu) {int ret; +struct rte_eth_dev_info dev_info;struct rte_eth_dev *dev;RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); dev = &rte_eth_devices[port_id]; RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->mtu_set, -ENOTSUP);
lib/librte_ethdev/rte_ethdev.c +4
-0+rte_eth_dev_info_get (port_id, &dev_info); +if (mtu < dev_info.min_mtu || mtu > dev_info.max_mtu) +return -EINVAL ; + ret = (*dev->dev_ops->mtu_set)(dev, mtu);if (!ret) dev->data->mtu = mtu;lib/librte_ethdev/rte_ethdev.h +0
-0@@ -1015,6 +1015,8 @@ struct rte_eth_dev_info {const char *driver_name;/**< Device Driver name. */ unsigned int if_index;/**< Index to bound host interface, or 0 if none.
lib/librte_ethdev/rte_ethdev.h +2
-0Use if_indextoname() to translate into an interface name. */ +uint16_t min_mtu;/**< Minimum MTU allowed */ +uint16_t max_mtu;/**< Maximum MTU allowed */ const uint32_t *dev_flags;/**< Device flags */ uint32_t min_rx_bufsize;/**< Minimum size of RX buffer. */ uint32_t max_rx_pktlen;/**< Maximum configurable length of RX pkt. */
--
2.17.1
Quoted 3 lines · Stephen Hemminger
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
It is true for amd64, but it looks like it is false on 32-bit. So, ABI breakage.
Quoted 30 lines
The initial version sets max mtu to normal Ethernet, it is up to PMD to set larger value if it supports Jumbo frames.
Fixing the drivers to use this is trivial and can be done by 18.11. Already have some of the patches done.
Signed-off-by Stephen Hemminger <[email protected]>--- lib/librte_ethdev/rte_ethdev.c | 7 +++++++ lib/librte_ethdev/rte_ethdev.h | 2 ++ 2 files changed, 9 insertions(+)lib/librte_ethdev/rte_ethdev.c +3
-0@@ -2408,6 +2408,8 @@ rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info) dev_info->rx_desc_lim = lim; dev_info->tx_desc_lim = lim; dev_info->device = dev->device; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETHER_MTU ;RTE_FUNC_PTR_OR_RET (*dev->dev_ops->dev_infos_get); (*dev->dev_ops->dev_infos_get)(dev, dev_info);@@ -2471,12 +2473,17 @@ int rte_eth_dev_set_mtu(uint16_t port_id, uint16_t mtu) {int ret; +struct rte_eth_dev_info dev_info;
lib/librte_ethdev/rte_ethdev.c int · line 2486
Quoted 10 lines
lib/librte_ethdev/rte_ethdev.c +4
-0struct rte_eth_dev *dev;RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -ENODEV ); dev = &rte_eth_devices[port_id];RTE_FUNC_PTR_OR_ERR_RET (*dev->dev_ops->mtu_set, -ENOTSUP ); +rte_eth_dev_info_get (port_id, &dev_info); +if (mtu < dev_info.min_mtu || mtu > dev_info.max_mtu) +return -EINVAL ; +
The check breaks set MTU to value larger than ETHER_MTU for not updated drivers. So, IMHO, it should be pushed only with appropriate updates in all drivers which support bigger MTU.
Quoted 16 lines
lib/librte_ethdev/rte_ethdev.c +0
-0ret = (*dev->dev_ops->mtu_set)(dev, mtu);if (!ret) dev->data->mtu = mtu;lib/librte_ethdev/rte_ethdev.h +2
-0@@ -1015,6 +1015,8 @@ struct rte_eth_dev_info {const char *driver_name;/**< Device Driver name. */ unsigned int if_index;/**< Index to bound host interface, or 0 if none. Use if_indextoname() to translate into an interface name. */ +uint16_t min_mtu;/**< Minimum MTU allowed */ +uint16_t max_mtu;/**< Maximum MTU allowed */ const uint32_t *dev_flags;/**< Device flags */ uint32_t min_rx_bufsize;/**< Minimum size of RX buffer. */ uint32_t max_rx_pktlen;/**< Maximum configurable length of RX pkt. */
Quoted 7 lines · On Thu, 6 Sep 2018 09:29:32 +0300 Andrew Rybchenko <[email protected]>…
Quoted 3 lines · Stephen Hemminger
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
It is true for amd64, but it looks like it is false on 32-bit. So, ABI breakage.
Yes it is ABI change on 32 bit, but 18.11 is a major release where this is allowed/expected.
Quoted 7 lines
Quoted 4 lines · On Thu, 6 Sep 2018 09:29:32 +0300 Andrew Rybchenko <[email protected]>…
Quoted 3 lines · Stephen Hemminger
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing
Quoted 7 lines
structure.
Quoted 3 lines
It is true for amd64, but it looks like it is false on 32-bit. So, ABI breakage.
Yes it is ABI change on 32 bit, but 18.11 is a major release where this is allowed/expected.
Thanks for this work Stephen, I've tested it with OVS DPDK and it resolves the issues as described, if it's to be part of DPDK 19.02 I guess there should be an ABI breakage notification in 18.11?
Hi Stephen,
Can we expect a v2 of this patch? At least for removing deprecation and bump ethdev version, etc?
05/09/2018 18:41, Stephen Hemminger:
Quoted 11 lines
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
The initial version sets max mtu to normal Ethernet, it is up to PMD to set larger value if it supports Jumbo frames.
Fixing the drivers to use this is trivial and can be done by 18.11. Already have some of the patches done.
Signed-off-by Stephen Hemminger <[email protected]>
Quoted 21 lines · On Wed, 19 Dec 2018 03:37:16 +0100 Thomas Monjalon <[email protected]> wrote:
Hi Stephen,
Can we expect a v2 of this patch? At least for removing deprecation and bump ethdev version, etc?
05/09/2018 18:41, Stephen Hemminger:
Quoted 11 lines
This addresses the usability issue raised by OVS at DPDK Userspace summit. It adds general min/max mtu into device info. For compatiablity, and to save space, it fits in a hole in existing structure.
The initial version sets max mtu to normal Ethernet, it is up to PMD to set larger value if it supports Jumbo frames.
Fixing the drivers to use this is trivial and can be done by 18.11. Already have some of the patches done.
Signed-off-by Stephen Hemminger <[email protected]>
It needs more than just this patch to be worthwhile.
Need device drivers to fill in the max_mtu field, otherwise it would just break jumbo frame support.
I suppose an initial patch to bump ethdev version would be possible for 19.05, are there other API changes doing that already for that relaease.
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522 bytes incl. header and CRC, and should be allowed with an Ethernet MTU of 1500 bytes. There's even a #define ETHER_MAX_VLAN_FRAME_LEN for this, but that's as far as it goes...
But how about frames with even larger headers, e.g. 4 MPLS labels makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534 bytes... is such a frame acceptable with an MTU of 1500 bytes?
According to Wikipedia (https://en.wikipedia.org/wiki/Maximum_transmission_unit), MTU describes the maximum payload size, and is not tied to the maximum frame size. However, the Linux kernel (at least the older versions I have been working with) incorrectly ties the MTU directly to the maximum frame size, so the MTU has to increased to support larger headers (e.g. QinQ or an MPLS stack). Do none, all or some DPDK PMDs suffer from the same error?
Med venlig hilsen / kind regards
- Morten Brørup
Quoted 7 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522 bytes incl. header and CRC, and should be allowed with an Ethernet MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...But how about frames with even larger headers, e.g. 4 MPLS labels makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534 bytes... is such a frame acceptable with an MTU of 1500 bytes?
No. According to standard practice in Linux and FreeBSD, only the first VLAN tag is free. After that any other headers count against MTU.
Quoted 3 lines
According to Wikipedia (https://en.wikipedia.org/wiki/Maximum_transmission_unit), MTU describes the maximum payload size, and is not tied to the maximum frame size. However, the Linux kernel (at least the older versions I have been working with) incorrectly ties the MTU directly to the maximum frame size, so the MTU has to increased to support larger headers (e.g. QinQ or an MPLS stack). Do none, all or some DPDK PMDs suffer from the same error?
Maximum Transmission Unit and Maximum Receive Unit (MRU) are separate. On most hardware they are the same but there is variation. Some hardware can receive larger sizes than MTU and some have max receive length register.
Quoted 6 lines
From: dev on behalf of Stephen Hemminger
Quoted 2 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
Quoted 8 lines
Quoted 5 lines
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522 bytes incl. header and CRC, and should be allowed with an Ethernet MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...But how about frames with even larger headers, e.g. 4 MPLS labels makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534 bytes... is such a frame acceptable with an MTU of 1500 bytes?
No. According to standard practice in Linux and FreeBSD, only the first VLAN tag is free. After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased by the size of one VLAN tag, because the first VLAN tag is free? Or must the MTU be increased by the size of two VLAN tags, because only the special case of exactly one VLAN tag is free?
I wish details like this were documented... for the benefit of both PMD developers and DPDK users. Which is why I CC'ed the Documentatation and Ethernet API maintainers.
It reminds me of the previous discussion about some Ethernet PMD's not padding to minimum Ethernet frame size... DPDK users expecting a certain behavior based on Linux/BSD behavior, but not documented, so the PMD developers were unaware (or possibly disagreed).
We all know that the devil is in the details! Which is why documenting such details is important.
Outside the DPDK core libraries, the Ethernet PMDs are probably the closest to the DPDK core you are going to get. The core libraries have a bunch of test cases. Perhaps there should be more PMD test cases.
Quoted 7 lines
Quoted 4 lines
According to Wikipedia (https://en.wikipedia.org/wiki/Maximum_transmission_unit), MTU describes the maximum payload size, and is not tied to the maximum frame size. However, the Linux kernel (at least the older versions I have been working with) incorrectly ties the MTU directly to the maximum frame size, so the MTU has to increased to support larger headers (e.g. QinQ or an MPLS stack). Do none, all or some DPDK PMDs suffer from the same error?
Maximum Transmission Unit and Maximum Receive Unit (MRU) are separate. On most hardware they are the same but there is variation. Some hardware can receive larger sizes than MTU and some have max receive length register.
Yes. This is also not obvious, although the name Maximum Transmission Unit strongly indicates that it does not apply to ingress. It took me by surprise a few years ago while working with the Linux kernel.
Quoted 17 lines
Quoted 12 lines
From: dev on behalf of Stephen Hemminger
Quoted 5 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522 bytes incl. header and CRC, and should be allowed with an Ethernet
MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...Quoted 2 lines
Quoted 2 lines
But how about frames with even larger headers, e.g. 4 MPLS labels makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534
bytes... is such a frame acceptable with an MTU of 1500 bytes?
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
No. According to standard practice in Linux and FreeBSD, only the first VLAN tag is free. After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased by the size of one VLAN tag, because the first VLAN tag is free? Or must the MTU be increased by the size of two VLAN tags, because only the special case of exactly one VLAN tag is free?
Can we introduce new function at ehtdev API to query PMD frame size based on MTU? Something like: rte_ethdev_mtu_to_frame_size(uint32_t mtu);
Provide default behavior and allow PMD to overwrite it?
Konstantin
Quoted 29 lines
I wish details like this were documented... for the benefit of both PMD developers and DPDK users. Which is why I CC'ed the Documentatation and Ethernet API maintainers.
It reminds me of the previous discussion about some Ethernet PMD's not padding to minimum Ethernet frame size... DPDK users expecting a certain behavior based on Linux/BSD behavior, but not documented, so the PMD developers were unaware (or possibly disagreed).
We all know that the devil is in the details! Which is why documenting such details is important.
Outside the DPDK core libraries, the Ethernet PMDs are probably the closest to the DPDK core you are going to get. The core libraries have a bunch of test cases. Perhaps there should be more PMD test cases.
Quoted 3 lines
Quoted 3 lines
According to Wikipedia (https://en.wikipedia.org/wiki/Maximum_transmission_unit), MTU describes the maximum payload size, and is
not tied to the maximum frame size. However, the Linux kernel (at least the older versions I have been working with) incorrectly ties the MTU directly to the maximum frame size, so the MTU has to increased to support larger headers (e.g. QinQ or an MPLS stack). Do none, all or some DPDK PMDs suffer from the same error?
Quoted 4 lines
Quoted 1 line
Maximum Transmission Unit and Maximum Receive Unit (MRU) are separate. On most hardware they are the same but there is variation. Some hardware can receive larger sizes than MTU and some have max receive length register.
Yes. This is also not obvious, although the name Maximum Transmission Unit strongly indicates that it does not apply to ingress. It took me by surprise a few years ago while working with the Linux kernel.
Quoted 28 lines
From: dev [mailto:[email protected]] On Behalf Of Ananyev, Konstantin
Quoted 10 lines
Quoted 10 lines
From: dev on behalf of Stephen Hemminger
Quoted 5 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522
bytes incl. header and CRC, and should be allowed with an Ethernet
Quoted 1 line
MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
But how about frames with even larger headers, e.g. 4 MPLS labels
makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534
Quoted 3 lines
bytes... is such a frame acceptable with an MTU of 1500 bytes?
Quoted 2 lines
No. According to standard practice in Linux and FreeBSD, only the
first VLAN tag is free.
Quoted 5 lines
Quoted 1 line
After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased bythe size of one VLAN tag, because the first VLAN tag is free?
Quoted 8 lines
Quoted 1 line
Or must the MTU be increased by the size of two VLAN tags, because
only the special case of exactly one VLAN tag is free?
Can we introduce new function at ehtdev API to query PMD frame size based on MTU?
Something like:rte_ethdev_mtu_to_frame_size(uint32_t mtu); Provide default behavior and allow PMD to overwrite it? Konstantin
This assumes that the Layer 2 headers are fixed size. If you add e.g. an MPLS stack to the packet, the number of MPLS labels can vary, and thus the size of the Layer 2 headers varies with each packet.
It is a problem if Layer 3/4 offload features make assumptions about the Layer 3/4 MTUs based on the Layer 2 MTU without considering the size of the actual Ethernet headers of each packet, but simply assume that the Ethernet header size is fixed.
It might currently be calculated correctly for untagged or single VLAN tagged packets (assuming the VLAN tag is not already part of the packet data, but is set in the mbuf for the NIC to add).
-Morten
Quoted 32 lines
Quoted 32 lines
From: dev [mailto:[email protected]] On Behalf Of Ananyev, Konstantin
Quoted 10 lines
Quoted 10 lines
From: dev on behalf of Stephen Hemminger
Quoted 5 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522
bytes incl. header and CRC, and should be allowed with an Ethernet
Quoted 1 line
MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
But how about frames with even larger headers, e.g. 4 MPLS labels
makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534
Quoted 3 lines
bytes... is such a frame acceptable with an MTU of 1500 bytes?
Quoted 2 lines
No. According to standard practice in Linux and FreeBSD, only the
first VLAN tag is free.
Quoted 5 lines
Quoted 1 line
After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased bythe size of one VLAN tag, because the first VLAN tag is free?
Quoted 1 line
Or must the MTU be increased by the size of two VLAN tags, because
only the special case of exactly one VLAN tag is free?
Quoted 8 lines
Quoted 5 lines
Can we introduce new function at ehtdev API to query PMD frame size based on MTU?
Something like:rte_ethdev_mtu_to_frame_size(uint32_t mtu); Provide default behavior and allow PMD to overwrite it? KonstantinThis assumes that the Layer 2 headers are fixed size. If you add e.g. an MPLS stack to the packet, the number of MPLS labels can vary, and thus the size of the Layer 2 headers varies with each packet.
If all this extra stuff (MPLS labels, etc.) is added by SW - then yes, it will be SW (upper layer) to take account about such overhead and it would be accounted by PMD frame-size caluclation.
If these fields would be added by HW/PMD - then it would be PMD responsibility to take these extras into account when calculating frame size.
Quoted 6 lines
It is a problem if Layer 3/4 offload features make assumptions about the Layer 3/4 MTUs based on the Layer 2 MTU without considering the size of the actual Ethernet headers of each packet, but simply assume that the Ethernet header size is fixed.
It might currently be calculated correctly for untagged or single VLAN tagged packets (assuming the VLAN tag is not already part of the packet data, but is set in the mbuf for the NIC to add).
That could be a default behavior for that function. Konstantin
Quoted 47 lines · Konstantin
Quoted 32 lines
Quoted 32 lines
From: dev [mailto:[email protected]] On Behalf Of Ananyev, Konstantin
Quoted 10 lines
Quoted 10 lines
From: dev on behalf of Stephen Hemminger
Quoted 5 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522
bytes incl. header and CRC, and should be allowed with an Ethernet
Quoted 1 line
MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
But how about frames with even larger headers, e.g. 4 MPLS labels
makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534
Quoted 3 lines
bytes... is such a frame acceptable with an MTU of 1500 bytes?
Quoted 2 lines
No. According to standard practice in Linux and FreeBSD, only the
first VLAN tag is free.
Quoted 5 lines
Quoted 1 line
After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased bythe size of one VLAN tag, because the first VLAN tag is free?
Quoted 1 line
Or must the MTU be increased by the size of two VLAN tags, because
only the special case of exactly one VLAN tag is free?
Quoted 8 lines
Quoted 5 lines
Can we introduce new function at ehtdev API to query PMD frame size based on MTU?
Something like:rte_ethdev_mtu_to_frame_size(uint32_t mtu); Provide default behavior and allow PMD to overwrite it? KonstantinThis assumes that the Layer 2 headers are fixed size. If you add e.g. an MPLS stack to the packet, the number of MPLS labels can vary, and thus the size of the Layer 2 headers varies with each packet.
If all this extra stuff (MPLS labels, etc.) is added by SW - then yes, it will be SW (upper layer) to take account about such overhead and it would be accounted by PMD frame-size caluclation.
If these fields would be added by HW/PMD - then it would be PMD responsibility to take these extras into account when calculating frame size.Quoted 1 line
Quoted 9 lines
Quoted 5 lines
It is a problem if Layer 3/4 offload features make assumptions about the Layer 3/4 MTUs based on the Layer 2 MTU without considering the size of the actual Ethernet headers of each packet, but simply assume that the Ethernet header size is fixed.
It might currently be calculated correctly for untagged or single VLAN tagged packets (assuming the VLAN tag is not already part of the packet data, but is set in the mbuf for the NIC to add).
That could be a default behavior for that function. Konstantin
Hi All,
with a view to help progress this I've posted an RFC series based on Stephens work and some of the PMD drivers
http://mails.dpdk.org/archives/dev/2019-February/124938.html
Any feedback welcome
Thanks
Ian
Hi All,
Since Ian's change didn't include MTU interpretation issue, we want to see if we can send out a patch for this discussion to conclude as we have a similar problem.
Currently two ways are as below.
#1 Konstantin's solution of adding new api rte_ethdev_mtu_to_frame_size(devid, mtu), that would dynamically provide the new max mtu.
#2 Can we have the value of max_mtu field in rte_eth_dev_info be dynamic i.e based on device configure to indicate that new Tx offloads would need more space for L2.
For this we can update max_mtu field documentation without abi change.
I also see a problem with above two solutions that is not present with Linux like approach. It is that currently MTU configuration both effects Rx frame size and Tx frame size. If we change the MTU interpretation based on Tx offloads, then for Rx, there is a confusion of what will be its max frame size accepted.
Thanks
Nithin
-----Original Message-----
From: dev <[email protected]> On Behalf Of Ian Stokes
Sent: Wednesday, February 20, 2019 9:32 PM
To: Ananyev, Konstantin <[email protected]>; Morten Brørup <[email protected]>; Stephen Hemminger <[email protected]>
Cc: [email protected]; Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>; Thomas Monjalon <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>
Subject: Re: [dpdk-dev] [RFC] ethdev: add min/max MTU to device info
Quoted 47 lines · Konstantin
Quoted 32 lines
Quoted 32 lines
From: dev [mailto:[email protected]] On Behalf Of Ananyev, Konstantin
Quoted 10 lines
Quoted 10 lines
From: dev on behalf of Stephen Hemminger
Quoted 5 lines · On Wed, 6 Feb 2019 14:05:34 +0100 Morten Brørup <[email protected]>…
Good work, Stephen.
It should also be documented how PMDs should interpret this MTU.
Obviously, a VLAN tagged Ethernet frame grows from 1518 to 1522
bytes incl. header and CRC, and should be allowed with an Ethernet
Quoted 1 line
MTU of 1500 bytes. There's even a #define
ETHER_MAX_VLAN_FRAME_LENfor this, but that's as far as it goes...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
But how about frames with even larger headers, e.g. 4 MPLS labels
makes a frame 16 bytes longer, i.e. it grows from 1518 to 1534
Quoted 3 lines
bytes... is such a frame acceptable with an MTU of 1500 bytes?
Quoted 2 lines
No. According to standard practice in Linux and FreeBSD, only the
first VLAN tag is free.
Quoted 5 lines
Quoted 1 line
After that any other headers count against MTU.
Thank you for the insights. Just to clarify:
1 VLAN tag is allowed for free.
But on order to support two VLAN tags, the MTU must be increased bythe size of one VLAN tag, because the first VLAN tag is free?
Quoted 1 line
Or must the MTU be increased by the size of two VLAN tags, because
only the special case of exactly one VLAN tag is free?
Quoted 8 lines
Quoted 5 lines
Can we introduce new function at ehtdev API to query PMD frame size based on MTU?
Something like:rte_ethdev_mtu_to_frame_size(uint32_t mtu); Provide default behavior and allow PMD to overwrite it? KonstantinThis assumes that the Layer 2 headers are fixed size. If you add e.g. an MPLS stack to the packet, the number of MPLS labels can vary, and thus the size of the Layer 2 headers varies with each packet.
If all this extra stuff (MPLS labels, etc.) is added by SW - then yes, it will be SW (upper layer) to take account about such overhead and it would be accounted by PMD frame-size caluclation.
If these fields would be added by HW/PMD - then it would be PMD responsibility to take these extras into account when calculating frame size.Quoted 1 line
Quoted 9 lines
Quoted 5 lines
It is a problem if Layer 3/4 offload features make assumptions about the Layer 3/4 MTUs based on the Layer 2 MTU without considering the size of the actual Ethernet headers of each packet, but simply assume that the Ethernet header size is fixed.
It might currently be calculated correctly for untagged or single VLAN tagged packets (assuming the VLAN tag is not already part of the packet data, but is set in the mbuf for the NIC to add).
That could be a default behavior for that function. Konstantin
Hi All,
with a view to help progress this I've posted an RFC series based on Stephens work and some of the PMD drivers
http://mails.dpdk.org/archives/dev/2019-February/124938.html
Any feedback welcome
Thanks
Ian